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Preface 


Welcome  to  Denver,  Colorado,  USA  and  to  ED-MEDIA  2002.  We  hope  you  will  discover  many 
exciting  ideas,  be  challenged,  meet  new  people,  and  find  yourself  energized  by  the  conference! 

This  year  marks  the  fourteenth  appearance  of  the  ED-MEDIA  conference,  making  ED-MEDIA 
one  of  the  oldest  and  well-established  conferences  in  the  area  of  educational  technology.  In 
celebration  of  our  longevity,  we  have  complimented  our  strong  paper  program  with  some 
particularly  notable  keynotes  and  invited  speakers.  We  a-e  particularly  proud  of  this  year’s 
keynote  speakers,  each  of  whom  has  made  strong  contributions  to  the  founding  of  educational 
computing  as  a field  of  study.  Alan  Kay,  one  of  the  leading  visionaries  in  personal  computing, 
has  also  taken  an  early  leadership  role  in  thinking  about  ways  to  use  computers  to  teach  children 
(and  about  ways  to  teach  children  about  computers).  Andy  Van  Dam  built  one  of  the  first 
hypertext  systems  (in  the  late  1960s,  for  those  who  think  hypertext  started  with  the  World  Wide 
Web)  and  is  a leader  in  computer  graphics.  Anne  Wright  has  been  a leader  in  educational 
technology  in  the  UK  and  beyond  and  recently  served  as  Chief  Executive  of  Ufi  Ltd  (which 
provided  an  online  “University  for  Industry”).  She  is  currently  an  E-Leaming  Strategic  Adviser  to 
the  Department  for  Education  and  Skills  in  the  UK.  Finally,  Hermann  Maurer  helped  found  the 
ED-MEDIA  series  of  conferences  and  developed  two  leading  hypertext  systems.  Dr.  Maurer  will 
reflect  on  the  history  of  multimedia  in  education. 

ED-MEDIA  prides  itself  on  being  an  international  conference,  and  you  can  see  many  countries 
and  perspectives  reflected  in  our  162  Full  Papers,  255  Brief  Papers,  and  220  posters.  This  year, 
we  have  papers  and  posters  from  researchers  in  over  60  countries.  This  program  was  assembled 
by  the  members  of  our  valued  international  Program  Committee,  who  carefully  and  rigorously 
reviewed  the  many  submissions  received.  The  Full  Papers  received  particular  attention,  with  each 
being  reviewed  by  at  least  two  Program  Committee  members  and  only  the  most  meritorious  being 
accepted  to  the  conference.  The  Program  Committee  members  are  asked  to  consider  a number  of 
aspects  of  each  paper,  particularly  the  quality  of  the  work  and  the  expected  benefits  to  the 
educational  community.  After  the  review  process,  the  Chairs  and  Steering  Committee  assembled 
to  make  final  decisions.  This  year,  we  accepted  slightly  less  than  half  of  the  Full  Papers  submitted 
to  the  conference. 

Behind  the  scenes,  much  was  different  in  the  review  process.  We  have  installed  a new  online 
reviewing  system  that  makes  it  easier  for  Program  Committee  members  to  make  and  add 
comments  on  papers  and  for  the  Steering  Committee  to  review  the  work  of  the  Program 
Committee.  We  hope  that  those  of  you  who  submitted  papers  found  the  online  submission  system 
also  worked  more  smoothly.  The  wonderful  folks  at  AACE  are  continually  working  to  make  the 
system  better  and  we  expect  many  more  improvements  in  the  years  to  come.  In  the  near  future, 
ED-MEDIA  attendees  can  expect  to  see  an  online  “plan  your  conference”  system  and  access  to  a 
new  AACE  Digital  Library.  If  you’d  like  to  learn  more  about  the  reviewing  process  or  are 
interested  in  joining  the  Program  Committee,  please  attend  the  special  session  on  “Reviewing  for 
ED-MEDIA” 

This  year,  as  in  recent  years,  we  identified  a set  of  papers  as  “outstanding”.  We  began  with  the 
Full  and  Brief  Papers  that  received  a score  of  5 (on  a 5 point  scale)  from  the  reviewers  and  the 
Full  and  Brief  student  papers  that  received  a score  of  4.5  or  above  from  the  reviewers.  The 
Program  Chairs  re-reviewed  those  papers  and  accepted  as  outstanding  only  those  that  clearly 
stood  out  from  the  rest.  Papers  were  judged  on  a number  of  important  criteria  such  as 
originality/novelty,  writing  style,  technical  content,  fiiture  impact,  evaluation,  and  so  on.  We 
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found  that  the  outstanding  papers  reflected  the  topical  diversity  of  ED-MEDIA.  In  the  designated 
outstanding  papers,  you  will  find  careful  studies  of  the  effects  of  certain  technological 
interventions,  case  studies  that  provide  more  narrative  descriptions  of  the  use  of  technology  in 
teaching  (and  which  we  often  found  to  be  “real  page  turners”  to  use  an  American  colloquialism), 
descriptions  of  new  projects  in  educational  technology,  and  even  instruments  to  use  in  future 
studies. 

As  you  scan  through  the  papers,  you  will  find  many  different  kinds  of  paper  tied  together  by  some 
common  threads.  While  the  focus  of  ED-MEDIA  is  technology  in  education,  there  are  many 
different  approaches  to  using  the  available  technology  for  the  realisation  of  educational  aims. 

These  include: 

• Preliminary  evaluations  of  new  teaching  techniques  and  tools; 

• Case  studies  on  the  use  of  technology  in  physical  or  virtual  classrooms; 

• Discussion  of  new  technologies  and  applications; 

• Applications  of  educational  technology  in  a variety  of  disciplines;  and 

• Theoretical  considerations  of  the  motivations  and  impact  of  technology. 

We  expect  you’ll  see  this  variety  reflected  in  the  program. 

This  year,  we  have  added  a special  strand  on  accessibility  issues  for  the  disabled.  As  the  Web  has 
grown,  so  has  the  importance  of  making  Web  materials  accessible  to  all  kinds  of  readers. 
Accessibility  encompasses  a wide  variety  of  issues,  including  access  to  the  Internet,  support  for 
multiple  languages  and  multiple  cultures,  and  support  for  users  with  disabilities.  As  the  premiere 
international  conference  in  educational  multimedia  and  hypermedia,  ED-MEDIA  has  long  been  a 
leader  in  publishing  papers  which  consider  issues  raised  by  different  languages  and  cultures.  This 
year’s  special  strand  both  broadens  and  solidifies  our  emphasis  on  accessibility.  We  both  hope 
and  expect  that  it  will  be  continued  at  future  sessions  of  ED-MEDIA. 

The  best  educational  technology  projects  look  to  accommodate  different  learning  styles.  While  it 
is  not  possible  to  accommodate  every  learning  style  in  a conference,  we  have  tried  to  provide  a 
variety  of  different  types  of  session.  Hence,  in  addition  to  the  keynote  speakers,  invited  speakers, 
and  paper  presentations,  all  of  which  are  generally  held  in  a lecture  plus  question-and-answer 
format,  we  have  included  a number  of  other  venues  for  learning  and  talking  about  educational 
multimedia,  hypermedia,  and  telecommunications.  These  other  venues  include  two  poster 
sessions  (with  225  posters),  which  provide  more  time  for  careful  consideration  and  discussion 
with  researchers;  10  panels,  which  present  different  perspectives  on  topics  and  give  attendees  a 
chance  to  discuss  issues  with  panelists;  tutorials,  which  provide  small- group  learning  situations 
on  a variety  of  topics;  workshops,  which  provide  small-group,  hands-on  learning  sessions  on  a 
variety  of  topics;  an  evening  of  special  interest  group  (SIG)  sessions,  in  which  attendees  with 
similar  interests  can  discuss  particular  topics;  a varied  social  program,  which  gives  attendees  the 
chance  to  interact  informally;  a group  diimer  program,  which  gives  attendees  better  chances  to 
meet  new  people  and  discuss  topics  informally;  a variety  of  meet-the-speakers  sessions  to  give 
you  a chance  to  interact  with  keynote  speakers,  invited  speakers,  and  outstanding  paper 
presenters;  and  a newcomers  session  to  help  new  attendees  plan  how  to  make  the  best  use  of  the 
many  compelling  opportunities  at  the  conference. 

As  Conference  Program  Co-chairs,  we  have  been  aided  considerably  by  a number  of  volunteers 
and  helpers  who  have  given  tirelessly  of  their  time.  In  particular,  we  must  thank  the  Conference 
Steering  Committee  of  Erik  Duval  (Chair),  Shelly  Heller,  Gary  Marks,  Craig  Montgomerie,  and 
Ron  Oliver,  who  led  and  coordinated  the  conference  this  year.  We  would  like  to  thank  the  Chairs 
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of  the  various  Program  Sub-Committees:  Karen  Swan,  Panels  Chair;  and  Catherine  McLoughlin, 
TutorialsAVorkshops  Chair.  We  would  also  like  to  thank  the  85  members  of  the  international 
Program  Committee  who  provided  timely  and  insightful  reviews  without  complaint  and  little 
credit.  Finally,  we  would  like  to  thank  Gary  Marks  and  the  staff  at  the  Association  for  the 
Advancement  of  Computing  in  Education  (AACE),  who  organize  and  sponsor  this  conference, 
for  their  support  in  this  massive  endeavor.  These  folks  work  incredibly  hard  behind  the  scenes  to 
manage  all  the  aspects  of  the  conference.  They  deal  with  many  complicated  situations  and  handle 
a variety  of  requests  from  the  Steering  Committee  and  others. 

We  look  forward  to  meeting  with  you  during  the  conference.  Remember  to  spend  time  talking  to 
colleagues  from  previous  ED-MEDIA  conferences  and  make  new  friends.  As  long-time  attendees 
will  tell  you,  what  you  learn  during  informal  conversations  can  have  as  much  or  even  greater 
impact  than  what  you  learn  during  more  formal  parts  of  the  program.  Think  about  new 
collaborations,  new  projects,  and  new  uses  of  the  work  you  hear  about  at  this  conference.  We 
hope  that  you  will  develop  new  work  that  you  can  present  next  year  at  ED-MEDlA  2003  in 
Honolulu,  Hawaii,  June  23-28,  2003  or  at  the  following  year’s  ED-MEDIA,  which  is  still  in  its 
planning  stages.  However,  we  do  know  that  it  will  be  held  at  a European  venue. 

Program  Chairs: 

Philip  Barker 

University  of  Teesside,  UK 
<Philip.Barker@tees.ac.uk> 

Samuel  A.  Rebelsky 
Grinnell  College,  USA 
<rebelsky@grinnell  .edu> 
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• Information  Technology  in  Chiidhood  Education  Annual  (ITCE) 

ELECTRONIC  JOURNALS 

• Educational  Technology  Review  (ETR) 

• Contemporary  Issues  in  Technology  & Teacher  Education  (CITE) 

Article  abstracts  tor  each  journal  are  available  at 
http://www.aace.org/pubs 


CONFERENCES 

ED-MEDIA  - World  Conference  on  Educational  Multimedia  and 
Hypermedia  & Telecommunications 

June  23-28,  2003;  Honolulu,  Hawaii  USA 

E-Leam  - World  Conference  on  E-Leaming  In  Corporate, 

Healthcare,  Government,  and  Higher  Education 

October  15-19,  2002;  Montreal,  Canada 

SITE  - Society  for  Information  Technology  & Teacher  Education 
International  Conference 

March  24-29. 2003;  Albuquerque,  New  Mexico  USA 

ICCE  - International  Conference  on  Computers  in  Education 
(Asia-Pacific  Chapter) 

December  3-6, 2002;  Auckland,  New  Zealand 
International  Headquarters: 

AACE.  PO.  Box  3728,  Norfolk,  VA  23514-3728  USA 

Phone:  757-623-7588  E-mail:  info@aace.org 

Fax:  703-997-8760  www.aace.org 


Submission  information 


FULL  PAPERS 

Presentation  time:  25  minutes 

Submission  length:  A-6  pages  (2.500-3,750  words) 

Proceedings  length:  S pages  maximum 

AV  equipment  provided:  PC,  Projector,  internet,  VCR 

Papers  present  reports  of  significant  work  or  integrative  reviews  in  research, 

development,  appiications,  and  societai  issues  reiated  to  aii  aspects  of  the 

conference  topics. 

Systems  & Resources 

Papers  related  to  projects,  technical  developments,  systems,  and  resources. 
The  paper  shouid  inciude: 

• Whether  this  work  is  just  beginning,  on-going,  or  compieted; 

• The  partners  invoived; 

• The  major  goals  and  the  basic  approach  - this  inciudes  the  educational 
problem  addressed  or  the  new  educational  opportunity  created; 

• A brief  review  of  previous  work  as  a background: 

• A clear  description  of  what  has  been  deveioped,  inciuding  schematic 
overviews  and  screen  dumps  (if  appiicable); 

• Any  vaiidation  (usabiiity  testing,  piiot  testing,  expert  evaiuation,  etc.)  that 
has  taken  piace;  and 

• Future  work  and  impiications  for  others. 


BRIEF  PAPERS 

Presentation  time:^5  minutes 
Submission  length:  2-3  pages.  1 .250-1 ,875  words 
Proceedings  length:  2 pages  maximum 
AV  equipment  provided:  PC,  Projector,  internet,  VCR 

These  papers  are  brief,  more  condensed  presentations  or  work-in-progress  projects. 

New  Developments 

Descriptions  of  new  extensions  to  existing  projects  or  newiy  Initiated  projects. 

Project  Opportunities 

Descriptions  of  projects  underway  that  include  opportunities  for  additional  pro- 
ject partners. 

Demonstrations 

Other 

For  authors  who  do  not  feel  they  fit  in  any  of  the  above.  The  Program 
Committee  may  request  that  the  author  revise  the  paper  to  reiate  to  one  of  the 
above  categories,  or  may  accept  it  as  submitted. 

The  submission  shouid  inciude: 

• What  is  going  to  be  shown  or  demonstrated  or  offered; 

• The  major  aspects; 

• The  context  or  motivation;  and 

• Reievant  URLs  or  literature  references. 


The  reference  list  for  this  kind  of  paper  should  inciude  URLs  for  the  work,  and 
at  least  a few  literature  references. 

Conceptual  & Empirical  Studies 

Reviews,  conceptual  overviews,  evaluations,  and  empirical  studies. 

The  paper  shouid  inciude: 

• The  topic; 

• The  motivation  for  the  work; 

• The  major  questions  addressed; 

• The  generai  process  and  conceptuai  framework,  with  references  to  iiterature; 

• The  concrete  method,  with  sufficient  detaii  on  instruments  and  procedures: 

• Major  points  or  resuits;  and 

• Implications. 

This  paper  category  requires  a strong  and  up-to-date  weii  synthesized  litera- 
ture review,  with  work  from  a variety  of  sources  (not  just  the  research  team  of 
the  author),  and  aiso  an  appropriate  writing  and  presentation  styie  for  a schoi- 
arly  review  or  overview. 

Case  Studies 

Papers  reiated  to  local  experiences  (e.g.,  a course  or  a prototype  tested  in  a 
local  situation). 

The  paper  shouid  include: 

• The  most  important  features  of  the  context; 

• A description  of  the  probiem  in  both  iocai  and  conceptuai  terms; 


PANEI^ 

Presentation  time:  ^ hour 

Submission  length:  2-3  pages,  1 ,250-1 ,875  words 

Proceedings  length:  A maximum 

AV  equipment  provided:  PC,  Projector,  internet,  VCR 

A panei  offers  an  opportunity  for  3-5  peopie  (inciuding  the  chair)  to  present 
their  views  or  resuits  on  a common  theme,  issue,  or  question  and  discuss 
them  with  the  audience.  Paneis  shouid  cover  timeiy  topics  reiated  to  the  con- 
ference areas  of  interest.  Panei  seiection  wiil  be  based  on  the  importance, 
originaiity,  focus,  and  timeiiness  of  the  topic;  expertise  of  proposed  paneiists; 
as  weii  as  the  potentiai  for  informative  (and  even  controversiai)  discussion. 

Panels  must  allot  at  least  50%  of  the  time  for  interaction  and  discussion 
with  the  audience. 

The  panei  proposais  shouid  inciude: 

• Description  of  the  panel  topic,  inciuding  why  this  topic  is  important  to 
Conference  attendees;  and 

• Brief  position  statement  and  quaiifications  of  each  paneiist. 

ROUNDTABLES 

Presentation  time:  ^ hour 

Submission  length:  2-3  pages,  1 ,250-1 .875  words 

Proceedings  length:  1 50  words  (to  be  pubiished  oniy  in  Abstract  Book) 

AV  equipment  provided:  eiectricity 


• A description  of  who,  what,  why,  and  how.  inciuding  screen  dumps  if  reievant: 

• Implications  for  the  iocai  setting  or  the  local  prototype;  and 

• Implications  for  others  outside  the  iocai  setting. 

This  paper  category  shouid  include  a reference  list  with  reievant  URLs,  and  a 
few  citations  to  papers  reiated  to  the  iocai  probiem,  but  from  outside  the  iocai 
setting.  The  latter  is  to  indicate  that  the  author  can  see  the  iocai  situation  in  a 
broader  and  more  generai  context. 

Other 

For  authors  who  do  not  feei  they  fit  in  any  of  the  above.  The  Program 
Committee  may  request  that  the  author  revise  the  paper  to  reiate  to  one  of  the 
above  categories,  or  may  accept  it  as  submitted. 
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These  sessions  aiiow  maximum  interaction  in  informal,  small-group  discus- 
sions on  a singie  topic.  The  format  is  appropriate  for  papers,  projects,  or 
works-in-progress  that  encourage  discussion.  Roundtabies  share  a room  with 
2-3  other  concurrentiy  heid  Roundtabies. 

VIDEO  FESTIVAL 

Presentation  time:  1 hour 

Submission  length:  2-3  pages.  1 ,250-1 ,875  words 

Proceedings  length:  2 pages 

AV  equipment  provided:\iCR,  PC,  Projector,  internet 

The  fieid  has  matured  to  the  point  that  we  have  a growing  number  of  video 
resources  - cases,  ciassroom/training  video  clips,  and  many  other  forms. 
Presenters  shouid  have  a short  handout  on  the  video  and  use  most  of  the  time 
showing  the  video. 

BIST  COPY  AYAlIABUe 


Submission  information 


INTERACTIVE  SESSIONS 

Presentation  time:  1 hour 

Submission  length:  2-3  pages,  1 ,250-1 ,875  words 

Proceedings  length:  3 pages 

AV  equipment  provided:  PC,  Monitor,  Internet 

If  you  have  mastered  a new  piece  of  software  (or  a new  version)  that  others 
would  be  interested  in  learning  about,  submit  a proposal  for  an  Interactive 
Session.  In  this  informal  session,  you  will  demonstrate  the  software,  illustrate 
the  process  of  using  it,  show  participants  some  of  the  complexities  and  tricks 
about  it,  and  give  them  an  opportunity  to  try  it  themselves. 

These  sessions  are  also  appropriate  for  instructional  strategies,  procedures, 
and  evaluation  procedures.  The  idea  is  not  to  give  a definitive  workshop  on 
the  topic,  but  to  provide  participants  with  enough  information  to  help  them 
decide  whether  their  needs  can  be  met  with  the  program  or  procedure. 

Include  a description  of  the  software  to  be  used  by  participants,  the  objectives 
of  the  session,  and  the  intended  audience  (experience  level  and  prerequisites). 
These  presentations  share  a room  with  2-3  other  concurrently  held  Interactive 
Sessions. 

TUTORIALS  / WORKSHOPS 

Presentation  time:  3.5  or  7 hours 
Submission  length:  see  information  below 
Proceedings  length:  no  pages 
AV  equipment  provided:  PC,  Projector,  Internet  (Tutorial); 

PC,  Projector,  PC  Lab,  Internet  (Workshop) 

Tutorials  and  Workshops  are  intended  to  enhance  the  skills  and  broaden  the 
perspective  of  their  attendees.  They  should  be  designed  to  introduce  a rigor- 
ous framework  for  learning  a new  area  or  to  provide  advanced  technical  train- 
ing in  an  area.  Submissions  will  be  selected  on  the  basis  of  the  instructors' 
qualifications  for  teaching  the  proposed  Tutorial  or  Workshop  and  their  contri- 
bution to  the  overall  conference  program.  Workshops  differ  from  Tutorials  by 
involving  hands-on  experience  with  hardware/software  provided. 

Note:  Few  Workshops  are  selected  because  a lab  of  equipment  Is  required  for 
each.  If  you  submit  a Workshop  proposal,  please  indicate  if  your  proposal  is 
also  appropriate  for  presentation  as  a non-hands  on  Tutorial. 

Tutorial/Workshop  proposals  should  include: 

• Clear  description  of  the  objectives: 

• Intended  audience  (experience  level  and  prerequisites); 

• Proposed  length  (3.5  hours  or  7 hours): 

• 200-word  abstract: 

• 1 -page  topical  outline  of  the  content;  and 

• Summary  of  the  instructor's  qualifications. 

POSTER  / DEMONSTRATIONS 

Presentation  time:  2 hours 

Submission  length:  2-3  pages,  1 ,250-1 ,875  words 

Proceedings  length:  2 pages 

AV  equipment  provided:  A'yB'  poster  board,  6'  table,  2 chairs,  electricity, 
Internet  if  wireless  card  & PC  brought 

Poster/Demonstration  sessions  enable  researchers  and  non-commercial  devel- 
opers to  demonstrate  and  discuss  their  latest  results  and  developments  in 
progress  in  order  to  gain  feedback  and  to  establish  contact  with  similar  projects. 

Poster/Demonstration  proposals  should  include: 

• Description  of  the  planned  Poster/Demonstration:  should  emphasize  the 
problem,  what  was  done,  and  why  the  work  is  important. 

Poster/Demonstration  presenters  will  be  required  to  arrange  for  their  own  sys- 
tems software  and  hardware. 

O 
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CORPORATE  DEMONSTRATIONS/ 
LITERATURE 

Presentation  time:  2 hours 
Submission  length:  1 -2  paragraphs 
Proceedings  length:  no  pages 

AV  equipment  provided:  A xQ'  poster  board,  6'  table,  2 chairs,  electricity. 

Internet  if  wireless  card  & PC  brought 

Demonstrate  and  discuss  your  company's  products,  services,  developments, 
applications  and  research,  inform  the  audience  of  your  future  directions,  gain 
feedback,  and  establish  contacts. 

Scheduled  with  Poster/Demonstrations  grouped  together  in  open  exhibition- 
style,  usually  all  in  one  hall.  This  is  an  informal  event  with  a circulating  con- 
ference-wide audience.  Sales  are  permitted.  You  may  stock  and  sell  your 
product  at  your  table. 

CORPORATE  SHOWCASES 

Presentation  time:  30  minutes 
Submission  length:  1 -2  paragraphs 

Proceedings  length:  1 50  words  (to  be  published  only  in  Abstract  Book) 

AV  equipment  provided:  PC,  Projector,  Internet,  VCR 

Demonstrate  and  discuss  your  company's  products,  services,  developments, 
applications  and  research,  inform  the  audience  of  your  future  directions,  gain 
feedback,  and  establish  contacts. 

Scheduled  concurrently  only  with  other  Showcases.  Presentation  rooms  gen- 
erally accommodate  50-150  people,  theatre-style.  This  is  more  of  a formal 
presentation  than  the  Corporate  Demonstration. 

SIG  (SPECIAL  INTEREST  GROUP) 
DISCUSSIONS 

Presentation  time:  1 hour 

Submission  length:  1 -3  pages  (625-1 ,875  words) 

Proceedings  length:  no  pages 

AV  equipment  provided:  PC,  Projector,  Internet,  VCR 

To  encourage  informal  interaction  among  individuals  with  common  interests, 
SIG  discussion  groups  will  be  formed  based  upon  proposals  accepted  under 
the  Call  for  Participation.  Also,  new  SIG  discussion  groups  may  be  formed 
either  formally  or  on  an  Impromptu  basis  when  at  the  conference. 

SIG  Discussion  proposals  should  include: 

• Description  of  the  discussion  topic  emphasizing  the  problem  or  issue 
and  why  the  work  is  important:  and 

• Indication  of  whether  you  are  willing  to  chair  the  discussion. 


SUBMISSION  REQUIREMENTS 

Submit  all  proposals  by  completing  the  Web  form 
at:  http://www.aace.org/conf/edmedia 

All  proposals  must  be  submitted  by  uploading  PDF,  Word,  RTF,  or 
Postscript  file  using  the  Web  form.  No  hard  copy  paper,  faxed,  or  e-mail 
submissions  will  be  accepted. 

Please  send  your  proposal  only  ONE  time. 

Questions?  Contact  AACE  at:  Phone:  757-623-7588 

Fax:  703-997-8760 

E-mail:  conf@aace.org  www.aace.org/conf/edmedla 
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he  exciting  island  of  O'ahu, 
Hawaii,  the  home  of  Honolulu 
and  Waikiki  is  the  site  of 
ED-MEDIA  2003. 

Whether  your  idea  of  fun  is  soaking  up 
the  sun  on  a pristine,  white  sand  beach  or 
nightclubbing  in  Waikiki,  hiking  the  trails  or 
sampling  some  of  the  fantastic  Hawaiian 
Regional  Cuisine,  we  know  that  you're 
going  to  enjoy  yourself  on  O'ahu. 

Over  125  miles  of  beaches  surround  two 
magnificent  mountain  ranges  punctuated 
with  a vibrant  city,  colorful  little  communi- 
ties and  highlighted  by  dozens  of  the  most 
scenic  spots  on  earth.  Far  and  away  the  most 
diverse  of  the  Hawaiian  Islands,  there's 
something  here  for  everyone.  No  other 
American  city  could  offer  you  the  opportuni- 
ty to  surf  the  world's  biggest  waves,  snorkel 
a lagoon,  hike  into  a dormant  volcano,  golf 
at  a dozen  championship  courses,  and  catch 
the  sunset  from  a five-star  restaurant.  All 
within  an  hour's  drive  of  your  hotel  room. 

The  island  is  a study  in  contrasts:  green 
forests  and  glorious  beaches;  fresh,  clean 
bays,  rolling  hills,  dramatic  cliffs  and  sweep- 
ing vistas  counterpoised  with  the  life  of  a 
vibrant  city.  From  the  hustle  and  bustle  of 
Chinatown  to  the  rolling  surf  of  O'ahu's 
spectacular  North  Shore,  get  ready  for  an 
adventure. 

For  sheer  variety  of  things  to  do,  the 
Island  of  O'ahu  is  unparalleled.  Outdoor 
activities  range  from  the  calm  (a  cool  moon- 
light walk  along  Waikiki  Beach),  to  the 
heart-stopping  (hang-gliding  off  a thousand 
foot  cliff  in  Waimanalo).  All  103  sandy 
beach  sites  on  O'ahu  are  open  to  the  pub- 
lic, and  nearly  600  of  the  state's  top  surfing 


spots  are  here,  including  Waimea  Bay  and 
the  Banzai  Pipeline,  considered  among  the 
most  challenging  surf  breaks  in  the  world. 

Nowhere  does  East  meet  West  more 
obviously  or  successfully  than  in  the  arts 
(and  the  cuisine)  of  O'ahu.  Collections  built 
over  the  last  two  centuries  reflect  influences 
from  Europe  and  Asia.  Each  wave  of  immi- 
gration brought  its  own  cultural  traditions 
and  the  result  is  that  O'ahu,  like  no  other 
place  on  earth,  has  become  a cultural  repos- 
itory for  the  artistic  traditions  of  Polynesia, 
China,  japan.  Southeast  Asia  and  the  West. 

Over  fifty  ethnic  groups  are  represented 
on  the  island  of  O'ahu,  making  this  one  of 
the  most  culturally  diverse  and  racially  inte- 
grated places  on  the  globe.  For  the  million 
people  who  live  in  the  State  of  Hawaii, 
Honolulu  is  'the  big  city,'  so  we're  fortunate 
to  have  more  than  our  share  of  nightlife  and 
cultural  activity.  This  includes  theater,  opera, 
an  active  symphony  program,  stage  shows 
and  comedy,  as  well  as  traditional  island 
entertainment  such  as  the  hula. 

O'ahu  is  where  the  cultures  of  the  Orient 
have  blended  with  the  culinary  traditions  of 
Europe  to  create  "Hawaiian  Regional 
Cuisine,"  combining  the  spices  from  all  over 
the  world  in  a remarkable  blending  of 
Eastern  presentation  and  Western  sub- 
stance. You  can  also  sample  ethnic  dishes 
ranging  from  the  exotic  spices  of  Southeast 
Asia,  to  the  drama  of  an  original  Hawaiian 
luau  to  the  more  familiar  western  cuisine  of 
Italy  and  France. 

Sound  like  a great  place  for  a conference? 
You  bet!  Plan  to  join  us  for  ED-MEDIA  2003, 
at  the  Sheraton  Waikiki,  June  23-28,  2003. 

Copyright  O 2002  O'ahu  Visitors  Bureau;  adapted  for  use  in  this  Call 


me 


Explore  Hawaii  online  at: 

www.gohawaii.com  • www.visit-oahu.com 
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Association  for  the 
Advancement  of 
Computing  in 
Education 


Membership  Information  • Conferences  • Publications 


Invitation  to  Join 

The  Association  for  the  Advancement  of  Computing  in  Education  (AACE)  is  an 
internationai,  non-profit  educational  organization.  The  Association’s  purpose  is  to 
advance  the  knowledge,  theory,  and  quality  of  teaching  and  learning  at  all  levels  with 
information  technology.  This  purpose  is  accomplished  through  the  encouragement  of  ' ^ . 
scholarly  inquiry  related  to  technology  in  education  and  the  dissemination  of  research  results 
and  their  applications  through  AACE  sponsored  publications,  conferences,  and  other  opportunities 
for  professional  growth. 

AACE  members  have  the  opportunity  to  participate  in  topical  and  regional  divisions/societies/chapters, 
high  quality  peer-reviewed  publications,  and  conferences. 


\ , 
•'A 


Join  with  fellow  professionals  from  around  the  world  to  share  knowledge  and  ideas  on  research,  develop- 
ment, and  applications  in  information  technology  and  education.  AACE’s  membership  includes 
. researchers,  developers,  and  practitioners  in  schools,  colleges,  and  universities;  administrators, 
policy  decision-makers,  professional  trainers,  adult  educators,  and  other  specialists  in  edu- 
> cation,  industry,  and  government  with  an  interest  in  advancing  knowledge  and  learning 

with  information  technology  in  education. 

Membership  Benefit  Highlights 

H Gain  professional  recognition  by  participating  in  AACE  sponsored 
internationai  conferences 

H Enhance  your  knowledge  and  professional  skills  through  interaction 
with  colleagues  from  around  the  world 

H Learn  from  colleagues'  research  and  studies  by  receiving  AACE's 
weihrespected  journals  and  books 

H Receive  a subscription  to  the  Professional  Member  periodical 
Educational  Technology  Review  [electronic] 

H Receive  discounts  on  multiple  Journal  subscriptions,  conference 
registration  fees,  proceedings  books  & CD-ROMs 


Association  for  the  Advancement  of  Computing  in  Education  | 


www.aace.org 
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Advancing  Knowledge  and  Learning  with  Information  Technology 
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What  Have  we  Learnt  in  15  Years  About  Educational  Multimedia? 


Hermann  MAURER,  Graz  University  of  Technology,  Graz/ Austria 
Web:  www.iicm.edu/maurer 
Email:  hmaurer@iicm.edu 


Abstract:  This  talk  consists  of  three  parts:  First,  I start  with  a brief  review  of  the  history  of 
multimedia  in  education  as  we  have  experienced  it  since  the  first  ED -MEDIA:  it  was  then  still 
called  ICCAL  and  held  in  Calgary  in  1987. 

I will  argue  that  the  progress  in  handling  media  has  helped,  but  that  the  major  breakthroughs 
have  been  on  other  fronts,  such  as  the  better  understanding  of  different  learning  models 
(behaviorism,  cognitivism  and  constructivism  or  incidental  vs.  explicit  learning).  We  have  a 
more  realistic  appreciation  of  what  intelligent  tutoring  systems  can  do  and  cannot  do. 

We  have  learnt  that  the  administration  of  students,  authors,  tutors,  courses  etc.  is  much  more 
important  than  previously  thought.  We  seem  to  agree  that  powerful  authoring  tools  are 
essential,  but  substantial  courseware  libraries  cannot  be  built  unless  systematic  re-use  of 
modules  based  on  metardata  and  presentation  standards  is  supported.  We  also  have  learnt  that 
standardization  as  a whole  is  a very  central  issue. 

In  the  second  part  of  the  talk  I concentrate  on  the  current  state  of  affairs:  After  all  we  now 
realize  that  computer  networks  have  opened  the  way  for  entirely  new  approaches  involving 
much  more  communication,  collaboration  and  team  work:  some  ideas  of  intelligent  tutoring 
systems  have  e.g.  been  successfully  replaced  by  techniques  involving  networks;  I will  mention 
the  importance  of  feedback,  active  documents  and  knowledge  management  in  this  context.  In 
general,  the  most  dramatic  step  forward  has  been  the  recognition  that  individual  multimedia 
packages  cannot  be  as  effective  as  they  should  be  h the  absence  of  powerful,  networked 
learning  environments.  Such  environments  are  starting  to  be  visible  on  the  horizon.  I also 
mention  how  the  role  and  success  of  educational  multimedia  is  rather  different  in  schools  and 
universities  when  compared  to  industrial  applications. 

In  the  third  part  of  the  talk  I take  a peek  into  the  future,  by  presenting  one  scenario  for  mobile 
computing  and  the  serious  implications  this  has  not  just  for  how  we  will  learn  in  the  future,  but 
also  for  what  we  will  have  to  learn. 


1,  A Short  Overview  of  Some  Developments  Between  the  First  ED-MEDIA  (ICCAL) 
Conference  in  1987  and  Today, 

It  is  well  known  that  the  first  serious  attempt  to  use  computers  for  educational  purposes  was  the  PLATO  system 
developed  in  the  early  sixties  at  the  University  of  Illinois  and  later  commercialized  by  the  then  powerful  Control 
Data  Corporation.  Of  its  many  competitors,  PLATO  did  fairly  well  for  quite  a long  time.  PLATO  went  through 
many  versions,  but  the  basic  idea  was  to  offer  a central  computer  with  courseware  and  use  the  material  from  a 
(then  very  rudimentary)  terminal.  However,  the  fact  that  all  students  accessed  the  same  computer  allowed  for 
some  communication  between  participants,  for  keeping  track  of  scores  and  other  statistical  information, 
including  feedback  on  the  quality  of  the  courseware.  Despite  all  these  features,  PLATO  never  achieved  a real 
breakthrough. 

Because  of  this,  alternatives  started  to  be  considered.  It  is  typical  that  at  the  first  of  our  meetings  (Calgary,  1987) 
topics  such  as  hypertext  (allowing  for  more  freedom  in  the  navigation  of  material),  pedagogical  and  didactic 
rules  for  preparing  courseware,  using  the  emerging  color-graphic  features  on  PC's  and  using  (analog)  videodisks 
that  then  represented  the  ultimate  multimedia  technology  were  main  topics.  The  networking  aspect,  one  of  the 
main  assets  of  PLATO,  was  almost  forgotten.  I happened  to  be  one  of  the  program  chairs  of  that  first  conference 
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and  was  probably  one  of  the  few  who  had  emphasized  the  need  to  not  forget  networking  already  even  before  this 
first  meeting  in  Calgary  at  the  major  IFIP  conference  in  Ireland  [1]. 

It  is  intriguing  to  see  that  the  topics  mentioned,  combined  with  attempts  to  use  elements  of  artificial  intelligence 
for  intelligent  tutoring  systems  (student  modeling)  and  the  advent  of  all  kinds  of  authoring  systems  including 
HyperCard  on  Apple  computers  kept  dominating  our  conference  series  for  the  next  three  meetings  in  Dallas 
1989,  Hagen  1990  and  Wolfville  1992.  Indeed,  no  researcher  in  the  field  dared  not  to  produce  such  a system:  I 
was  no  exception  [7].  Mind  you,  the  underlying  technology  started  to  change:  instead  of  videodisks,  CD  ROM’s 
and  CD-I’s  (the  latter  now  almost  forgotten)  and  the  examination  of  strategies  for  learning  gained  importance, 
and  networks  started  to  play  a more  prominent  role  again,  mainly  due  to  the  impact  Hypermedia  systems  (such 
as  Intermedia  at  Brown  University);  also,  the  first  signs  of  Internet  based  systems  from  Gopher  to  Hyper-G  (now 
Hyperwave)  and  WWW  started  to  appear. 

By  the  time  ICCAL  had  changed  its  name  and  had  become  ED-MEDIA  (Orlando  1 993)  Hypermedia  based  ideas 
(indicating  of  what  would  happen  with  WWW)  and  advanced  authoring  facilities  competed  with  the  continuous 
discussion  of  learning  models.  It  is  now  recognized  that  the  behavioristic  approach  is  justified  only  for  ’’rote 
learning  of  facts”,  and  that  cognitivistic  approaches  (requiring  real  understanding)  or  constructivistic  techniques 
(where  learners  have  to  actually  discover,  not  just  to  understand  situations  and  solutions)  are  more  desirable. 
However,  the  battle  between  cognitivism  and  constructivism  continued  well  beyond  ED -MEDIA  94  (in 
Vancouver):  we  believe  that  most  persons  are  now  happy  to  accept  that  both  approaches  have  their  role  to  play, 
yet  some  extremists  will  still  claim  that  one  of  the  two  will  be  ideal  for  all  situations.  Also,  other  ideas  that  are 
kind-of  orthogonal  like  explicit  vs.  implicit  learning  are  still  being  discussed  till  today  [9].  ED-MEDIA  95  in 
Graz,  Austria,  was  the  first  ED -MEDIA  where  the  Internet  and  WWW  started  to  become  central  topics...  and 
this  has  not  changed  since.  The  importance  of  communication  and  computer  supported  collaborative  work  as 
component  of  Web-based  teaching  support  systems  started  to  grow  increasingly,  as  is  witnessed  by  a slate  of 
papers  in  this  area  presented  at  ED -MEDIA  96  in  Boston.  Despite  the  fact  that  a new  conference  series  WebNet 
(now  e-Leam  with  WebNet  symposium,  this  year  in  Montreal,  see  www.aace.org/conf/webnet)  was  split  off 
from  ED-MEDIA  in  96,  ED-MEDIA  continued  to  explode:  the  1997  conference  that  returned  to  its  roots  in 
Calgary  was  the  first  one  where  the  printed  proceedings  consisted  of  two  volumes  (to  expand  to  three  volumes 
by  2001!).  The  fact  that  the  97  conference  had  as  subtitle  ED -TELECOM  97  says  it  all:  educational  applications 
without  communicational  networks  started  to  look  very  obsolete.  This  trend  continued  in  98  at  ED-MEDIA  in 
Freiburg  and  99  in  Seattle:  however,  the  continuing  spread  of  large  educational  environments  made  it  more  and 
more  important  to  talk  about  standardizing  formats  and  using  meta-data  and  meta-data  standards.  To  use 
electronic  educational  material  on  a large  scale  it  is  necessary  to  be  able  to  locate  and  re-use  material  created 
elsewhere.  It  is  with  satisfaction  that  we  can  observe  a continued  harmonization  between  North-American  and 
European  efforts  in  this  important  area.  It  is  also  gratilying  to  see  that  the  necessity  for  communication, 
collaboration  and  large  digital  libraries  accessible  via  the  Web  is  now  universally  accepted  as  basis  for  viable 
learning  systems  as  is  clearly  documented  in  the  large  number  of  papers  in  this  area  both  at  ED-MEDIA  2000  in 
Montreal  and  2001  in  Tampere,  Finland.  It  has  also  become  clear  that  a substantial  learning  system  needs  a huge 
array  of  administrative  features  and  that  the  authoring  of  material  is  just  a small  part  of  what  is  necessary  to 
make  e- Learning  work:  this  is  quite  in  contrast  to  the  days  when  e -Learning  and  authoring  of  material  was 
almost  used  synonymously. 


2.  Current  State  and  Lessons  Learnt 

Typical  for  today's  situation  is  the  fact  that  we  have  ceased  to  talk  about  learning  applications,  are  using  the  term 
learning-platform  less  and  less,  and  seem  to  be  moving  to  the  terminology  "Learning  Environment".  Such  a 
learning  environment  consists  of  (a  network  of)  servers  that  offer  a variety  of  components: 
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Courseware  modules  (re-usable  since  they  follow  one  of  the  accepted  standards  and  are  searchable  due 
to  meta-data  that  is  attached  to  the  modules  and  is  also  hopefully  standardized) 

Tools  to  create,  combine  and  modify  such  modules  into  new  ones 

Tools  that  allow  the  administration  of  the  modules  mentioned,  of  authors,  teachers,  tutors  and  students, 
each  with  a block  of  statistical  data  that  is  gathered  as  the  system  is  used 

A set  of  features  for  communication  and  collaboration  including  e.g.  chats  and  discussion  forums 
Facilities  that  allow  to  use  the  system  for  various  learning  paradigms,  different  level  of  learners, 
provides  a high  degree  of  interactivity  and  powerful  tools  for  (self)  testing  and  feedback 

Above  list  looks  innocently  simple  and  complete.  It  is  certainly  neither. 

First,  throughout  the  discussion  we  have  been  using  the  term  "courseware  module"  without  further  explanation, 
nor  have  we  ever  stated  what  it  means  to  have  a "high  level  of  interactivity".  In  fact,  both  things  are  closely 
related.  An  interactive  courseware  module  is  not  a sequence  of  Web  pages  that  one  can  click  through,  and  it  does 
not  become  much  more  interactive  if  a movie,  even  a simulation  allowing  various  parameters,  or  some  fill- in 
questions  are  allowed.  In  a cognitivistic  approach  (and  this  is  still  more  true  in  the  case  of  constructivism)  a 
courseware  module  will  be  a sophisticated  piece  of  software,  hardly  the  kind  of  thing  one  can  create  with  a 
WWW  editor,  or  the  standard  authoring  system.  A high  level  of  interactivity  also  implies  facilities  for 
customization,  personalization  and  active  documents,  as  will  be  discussed  below! 

Second,  it  would  take  a lengthy  paper  on  its  own  to  describe  tools  for  handling  and  modifying  courseware 
modules  to  the  extent  desirable;  and  this  is  even  more  true  of  administrative  tools  that  can  range  from  trivial  (and 
hence  useless)  to  very,  very  sophisticated.  Some  more  details  can  be  found  in  [8]. 

Third,  innocent  looking  words  like  "discussion  forum"  are  really  like  the  mythical  box  of  Pandora:  once  you  start 
to  seriously  use  a discussion  forum  with  a sizeable  group  of  users  the  number  of  issues  arising  is  mind  boggling: 
from  questions  of  the  levels  of  anonymity,  to  tools  for  structuring  a discussion,  to  the  issue  of  meta-data  for 
contributions  (I),  to  the  issue  of  active  notification  of  users,  to  the  fact  that  any  large  discussion  forum  has  to 
provide  a substantial  number  of  different  views  of  the  contributions,  to  tools  for  "positive"  and  "negative" 
filtering,  just  to  name  a few  examples.  If  more  complex  collaboration  tools  are  to  be  involved  (and  they  should 
be)  all  issues  surrounding  computer  supported  collaborative  work  (CSCW),  now  discussed  in  entirely  separate 
conferences,  start  to  be  important. 

Fourth,  systems  that  really  support  all  kinds  of  learning  paradigms  (from  e.g.  behavioristic  fact  learning  to 
constructivistic  knowledge  discovery)  are  not  available  "from  the  rack"  but  can  at  best  be  customized  on  the 
basis  of  generic  platforms.  To  build  Learning  Environments  catering  to  various  learning  styles  and  levels  of 
knowledge  requires  sophisticated  pre-tests  both  on  a cognitive  and  a knowledge  level:  again,  both  tasks  have  not 
even  been  solved  in  full  generality,  so  far. 

Fifth,  systems  should  at  least  allow  to  work  with  them  by  adding  notes  (for  oneself  or  a group).  They  should 
allow  to  insert  stuff  from  other  sources  (marked  as  such),  including  of  course  links,  and  must  provide  access  to 
rich  background  libraries  [2]:  it  must  be  possible  to  incorporate  parts  of  material  found  in  such  libraries  into  the 
courseware  modules  for  the  benefit  of  an  individual  or  a whole  group  of  persons.  They  must  allow  asking 
questions  at  any  point,  with  the  system  answering  automatically  if  similar  questions  have  been  asked  before  (as 
is  done  in  the  "active  document  concept"  [3])  or  else  they  pass  on  the  question  to  tutors  or  experts  who  answer 
them  immediately  (if  online)  or  "asynchronously",  otherwise.  Customization  should  go  beyond  the  courseware 
module  level:  it  should  allow  persons  to  view  the  catalogue  of  material  offered  from  their  perspective. 

Above  points  are  not  supposed  to  be  exhaustive  (e.g.  the  self-testing  and  evaluation  aspect  have  not  even  been 
touched  upon),  but  just  serve  to  show  how  complex  good  e-Leaming  environments  are.  In  this  sense  we  have 
learnt  that  e-Leaming  goes  much  beyond  a sequence  of  beautiful  Web-pages  with  some  question/answer  material 
and  a few  simulations  plus  Email  for  communication  thrown  in.  Rather,  much  effort  is  required  to  build  a 
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credible  ©-Learning  platform.  My  team  and  I have  worked  on  a system  Hyperwave  that  is  the  basis  of  an 
advanced  e-Leaming  system  for  many  years  [4]:  it  is  available  for  non-commercial  use  free  of  charge  for  schools 
and  accredited  universities.  It  is  the  outgrowth  of  hundreds(!)  of  person  years.  It  keeps  amazing  me  that  such 
huge  efforts  keep  being  ignored  by  some  groups  who  start  to  re-invent  the  wheel  all  by  themselves,  not  realizing 
how  complex  a system  has  to  be  to  be  really  successful. 

In  passing,  let  me  also  mention  that  using  e-Leaming  environments  in  schools  and  universities  is  rather  different 
from  using  them  in  companies.  In  the  latter  case,  they  are  seen  as  cost  effective  tools  for  in -time  learning  with 
high  self-motivation  of  the  learners,  in  the  former  they  are  seen  as  improving  the  teaching  and  easing  the 
learning  process  for  persons  often  requiring  external  motivation  (through  grading  or  such).  It  is  surprising  that 
many  people  do  not  seem  to  grasp  that  the  two  situations  described  are  sufficiently  different  to  require  rather 
different  e-Leaming  environments! 


3.  The  Future 


While  we  are  still  groping  for  the  ultimate  approach  to  e-Leaming  environments,  technology  is  moving  on  with 
tremendous  speed.  It  will  not  only  change  how  we  learn  but  much  more  crucial  what  we  learn.  I believe  that  this 
should  be  taken  into  account  as  of  yesterday  © , and  much  research  should  go  into  this  direction.  To  show 
dramatically  what  will  happen  I will  present  one  scenario  what  the  computer  of  the  future  (say  ten  years  from 
now)  will  look  like.  The  visible  part  of  this  computer  (basically  its  I/O  device)  is  shown  in  Figure  1 . The  actual 
computer  consists  of  an  innocent  looking  little  box  (a  bit  larger  than  a box  of  matches)  and  contains: 
a high  speed  processor  (working  at  10  Giga -Hertz) 
a large  internal  memory  (50  Gigabytes) 

an  I/O  device  for  memory  cards  (like  the  ones  now  used  in  digital  cameras  but  with  a capacity  of  some 
100  Terrabytes) 

a mobile  phone  unit  using  fast  packet-switched  technology  like  UTMS  or  its  successor 
a global  positioning  system  (GPS) 

all  this  connected  wireless  (with  next  generation  Blue-Tooth)  to  a device  like  the  one  shown  in  Figure 
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Figure  1 


The  glasses  shown  in  Figure  1 have  a number  of  special  features  built-in:  they  have  two  loudspeakers  near  the 
back  end  (providing  stereo  sound  to  the  wearer),  they  have  two  small  mirror-like  gadgets  that  allow  to  project  a 
picture  generated  by  the  computer  into  each  eye  (thus  e.g.  providing  full  three  dimensbnal  scenes),  it  has  a 
direction  measurement  (electronic  compass)  built  in  so  that  together  with  the  GPS  the  computer  knows  not  only 
where  the  person  is,  but  what  direction  (at  what)  the  user  is  looking.  It  also  has  a camera  and  a microphone  (seen 
in  the  niddle  of  the  glasses)  that  records  all  that  is  going  on,  with  image  processing  capabilities,  including 
gesture  recognition  of  the  wearer  for  data  input.  The  two  spikes  going  upward  at  the  end  of  the  glasses  are 
optional:  in  the  further  future  they  will  be  able  to  detect  a variety  of  states  of  the  brain,  providing  a direct  brain- 
computer  interface  that  will  further  enhance  facilities  for  data  input.  Much  of  the  power  of  such  a device  is  not 
necessary  for  the  discussion  in  this  paper:  rather,  it  is  the  subject  of  a separate  paper  [5].  The  main  point  is  that 
through  novel  input  devices  (such  as  voice,  gesture  recognition  as  mentioned,  or  even  input  directly  from  the 
brain)  users  can  access  (through  the  computer)  at  any  time  both  huge  local  data-bases  and  through  the  mobile 
phone  future  versions  of  the  WWW,  i.e.  also  remote  ones;  thus  users  can  retrieve  just  about  any  information 
desired,  or  solve  most  problems  imaginable,  with  the  results  communicated  through  3D  scenes  and  stereo  sound 
through  the  glasses  shown  in  Figure  1.  Of  course  this  will  require  advanced  techniques  of  searching  to  find  the 
relevant  information,  and  only  relevant  and  reliable  items:  this  is  exactly  one  of  the  main  areas  addressed  in 
modem  Knowledge  Management  systems  such  as  Hyperwave  [4].  Deeper  aspects  are  explained  in  the  paper  [6]. 

Let  us  stop  and  ponder  for  a moment  only  the  trivial  aspects  of  such  a situation,  where  information  and  problem 
solving  is  not  just  available  “at  your  fingertips”  but  even  more,  at  the  “wink  of  your  eye  or  hand”:  in  this  case, 
external  databases  are  very  much  a more  or  less  direct  extension  of  the  human  brain.  How  much  factual 
knowledge  about  history,  geography,  nature,  medicine  (you  name  it)  will  we  still  have  to  learn  when  it  is 
available  any  time  we  need  it?  If  we  want  to  know  what  we  see  and  what  the  history  of  it  is,  GPS  and  the 
direction  measurement  will  allow  to  give  us  all  the  information  we  care  for;  if  we  want  to  know  what  kind  of 
flower  we  are  looking  at,  the  flower  held  against  the  camera  and  image  processing  software  will  give  us  the 
answer;  the  composition  of  a certain  medication  and  all  its  benefits  and  side  effects  will  flash  on  our  screen  when 
we  need  it.  Specific  first  aid  help  will  be  made  available  to  us  after  we  have  performed  a computer-supported 
diagnosis.  And,  of  course,  the  voice  of  someone  talking  in  any  language  to  us  will  have  been  picked  up, 
translated  into  English  and  spoken  into  our  ears,  not  noticeable  to  others.  Similarly,  if  a problem  in  mathematics, 
or  physics,  or  chemistry  comes  up,  it  will  be  easy  to  solve  it,  using  the  power  of  the  computer  and  the  databases 
of  tools  and  problem  solvers  it  has  available. 

Thus,  what  (beyond  skills,  like  skiing,  riding  a bike,  juggling  balls,  or  doing  a surgical  operation)  do  we  still 
have  to  learn  if  we  have  all  what  we  usually  learn  available  to  us  any  time,  any  place?  This  is  one  of  the  most 
serious  problems  that  will  confront  us  faster  than  we  will  be  ready  to  answer  it:  what  I have  outlined  will  be 
reality  soon.  Much  of  the  material  we  learn  today  will  not  be  on  the  curricula  of  future  students.  Yet,  since  it  is 
not  easy  to  imagine  an  intelligent  discourse  without  having  certain  basic  facts  in  the  brain  (rather  than  in  an 
“external  brain”)  what  will  those  basics  look  like? 

I do  hope  to  hear  attempts  of  partial  answers  at  future  ED  -MEDIA  conferences! 


4.  References 

To  write  about  the  development  of  ED-MEDIA  from  1987  to  2002  is  to  write  about  the  history  and  the 
developments  of  all  of  e-Leaming.  After  all,  the  proceedings  of  the  conferences  contain  a total  of  some  3000 
contributions  and  many  thousands  of  references  to  important  material.  Thus,  discussing  all  this  at  any  level  of 
detail  would  require  a major  book,  and  a very  long  bibliography.  I have  thus  decided  to  omit  names  and 
references  and  to  only  include  a very  few  references  to  some  of  my  own  contributions.  Anyone  interested  in 
seeing  a bit  more  of  what  I have  done,  please  visit  my  website.  Anyone  interested  in  more  than  my  brief  historic 
outline  is  urged  to  consult  the  proceedings  of  ED-MEDIA,  many  of  them  available  at  AACE’ s website:  they 
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very  much  reflect  what  has  happened  in  the  last  15  years!  1 apologize  to  all  my  colleagues  since  1 have  not 
mentioned  any  by  name,  but  once  1 would  have  started  to  include  some,  others  would  have  complained  with 
justification  that  1 have  omitted  them. 
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Macromedia-Washington  State  Digital  Design  Curriculum  Partnership 

This  partnership  highlights  a successful  public/ private  partnership  designed  to  develop  the  IT 
workforce  — specifically  in  Web  design.  By  working  together,  Macromedia  and  Washington 
State  Office  of  Superintendent  of  Public  Instruction  were  able  to  maximize  the  quality  of  the 
curriculum  by  focusing  on  teacher  and  student  needs. 

Program  developers  at  the  Washington  Office  of  the  Superintendent  of  Public  Instruction 
(OSPI)  knew  that  they  wanted  to  develop  their  students'  professional  technology  skills.  They 
wanted  a project-based,  state-of-the-art  program  that  helped  teachers  teach  technology  in  ways 
that  engaged  students  in  real-world  learning.  But  they  didn't  have  the  resources  or  broad 
expertise  to  build  it  all  on  their  own.  Many  states  face  this  challenge.  They  understand  the 
problem  but  can't  afford  the  solution— without  taking  an  innovative  approach. 

Knowing  that  corporations  want  to  grow  technically  skilled  workers,  OSPI  sought  a corporate 
partner  with  whom  they  could  collaborate  on  the  curriculum.  They  selected  Macromedia 
because  it  produced  industry-standard  professional  Web  design  tools  that  their  students  were 
clamoring  to  learn.  Macromedia  was,  in  turn,  looking  for  a school  system  that  could  provide 
valid  — and  vital— educator  feedback  from  concept  through  to  the  classroom  on  its  new 
curriculum. 

What  OSPI  brought  to  the  partnership: 

• Teachers  excited  and  willing  to  try  something  new  with  their  students 

• Cross-demographic  group  of  schools  to  pilot  test  curriculum 

• Insight  into  the  NWCET  Pathway  standards 

• Opportunity  for  change  in  Ed  system 

What  Macromedia  brought  to  the  partnership: 

• Expertise  in  technology  curriculum  development 

• Corporate  approach,  and  funding,  to  get  a curriculum  project  done 

• Opportunity  for  teachers  to  work  with  industry  professionals 

• Business  people  taking  teachers  seriously 

In  the  last  year  we  have  brainstormed  the  course  content,  beta-tested  ideas,  reshaped  the 
syllabus  based  on  a wide  base  of  faculty  input  and  pilot-tested  with  teachers  across 
Washington.  We've  developed  a curriculum  that  addresses  classic  design  principles  and 
professional  design  processes,  but  we've  made  it  approachable  to  teachers  relatively  new  to 
technology.  We've  found  that  students  really  want  to  learn  technology.  But  they  need  projects 
that  are  relevant  to  them  — and  they  want  to  start  right  away!  In  this  session  we  will  talk  about 
how  to  make  a successful  partnership,  describe  the  benefits  from  business/ education 
collaboration,  and  share  what  we've  learned  along  the  way. 
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Abstract:  This  paper  describes  some  of  the  features  of  Information  Communication  Technology  (ICT)  and 
its  uses  in  the  teaching- learning  process  in  elementary  schools,  A number  of  articles  have  addressed  various 
issues  concerning  the  many  difficulties  of  how  to  integrate  ICT  into  the  school  curriculum.  In  most  schools, 
it  is  difficult  for  ICT  to  be  used  effectively  in  the  teaching-learning  process.  The  author  participated  and 
observed  many  classes  using  ICT  in  elementary  schools.  Qualitative  data  such  as  interviews  with  teachers 
and  observation  of  student  conversations  and  behaviors  were  analyzed.  The  research  findings  were  based  on 
ethnographic  case  studies  and  are  summarized  as  statements.  The  findings  were  summarized  and  are 
presented  as  a causal  relation  graph.  The  main  results  were:  (1)  Basic  computer  operating  skills  such  as  type 
writing  depends  largely  on  the  classroom  teacher’s  attitude  and  computer  literacy.  (2)  Most  computer 
operation  skills  such  as  file  saving,  and  design  skills  such  as  designing  and  developing  a homepage  were 
mastered  through  reciprocal  teaching  and  modeling  among  children.  (3)  Teacher’s  advice  helps  children  to 
search  and  access  information  related  to  their  topics  and  it  was  further  found  that  to  access  topic  relevant 
information  was  the  result  of  a comprehensive  understanding  of  the  information  domain  rather  than 
operational  skills.  (4)  Learning  motivation  was  highly  promoted  by  collaborative  and  competitive  group 
activities.  (5)  Integratbn  of  synchronized  system  such  as  videoconference  into  classes’ needs  professional 
technical  support. 

Introduction 

There  are  many  articles  addressing  the  difficulties  facing  those  who  wish  to  integrate  Information 
Communication  Technology  (ICT)  into  schod  curriculums.  Tebbutt,  M.  (1999)  highlights  that  there  are  various 
influences  on  the  process  of  implementing  ICT  in  science:  the  infrastructure  for  ICT  in  science,  the  philosophy  for 
ICT  in  the  school/department,  and  the  external  influences  on  philosophy4»ractice  (refs.  [1]).  Lawson,  T.  and 
Comber,  C.  (1999)  have  also  identified  four  factors:  teacher^  attitudes  prior  to  the  innovation,  the  role  of  the  ICT 
coordinator,  the  attitude  of  senior  management  and  the  existence  of  adequate  support  and  training  (refs.  [2]). 
Ertmer,  P.A.  has  discussed  two  kinds  of  barriers  on  technology  integration  (refs.  [3]),  and  some  articles  have 
identified  that  it  is  not  easy  to  integrate  ICT  into  school  curriculums  (refs  [4], [5]),  as  Selwyn,  N.  describes  ICT 
can  be  a headache  for  teachers  (refs. [6]).  Collis,  B.,  Peters,  O.  and  Pals,  N.  further  detail  (refs. [7]),  many  factors 
that  can  influence  an  individual’s  decision  to  make  or  not  make  use  of  some  form  of  ICT  in  his  or  her 
learning- related  activities.  They  highlight  the  four  most  important  factors,  the  4-E  model,  as  being  environmental 
factors,  effectiveness,  ease  of  use  and  personal  engagement.  This  model  was  tested  using  statistical  analysis  using 
the  questionnaire  method.  The  author  has  also  been  engaged  in  research  concerning  the  use  of  ICT  in  schools  such 
as  project  study,  in-service  teacher  training  and  so  on  (refs, [8], [9]). 

In  order  to  further  investigate  the  factors  that  affect  the  integration  of  ICT  in  schools  an  ethnographic  study  was 
undertaken  where  the  author  participated  in  actual  classes  in  elementary  schools  for  a one  year  period  (refs  [10]). 
The  chosen  research  approach  w as  based  on  a partnership  between  the  university  and  schools,  which  has  been 
reported  as  an  effective  approach  (refs.[10],[l  1]).  Almost  all  data  was  gathered  through  interview^  informal 
conversations,  observation  of  classes,  meetings  and  so  on 

Research  objectives  and  methodology 

The  purpose  of  this  paper  was  to  find  some  features  of  ICT  use  in  practical  classes  and  to  derive  findings  useful  to 
teachers  and  educators  in  aiding  ICT  integration  into  classes.  In  order  to  conduct  the  research,  the  author  visited 
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two  elementary  schools  and  observed  1 5 classes  for  a period  of  one  year.  The  author  recorded  the  scenes,  teachers  ’ 
student  conversations  and  the  behavior  of  classes  in  field  notes,  pictures  and  videotapes,  which  were  selected 
based  upon  the  author’s  impressions. 

In  order  to  find  the  definitive  variables  that  affect  the  use  of  ICT  in  classes,  the  author  adopted  the  following 


methodology. 


Figure  1.  Configuration  of  large-scale  variables  as  assumption 

1.  To  list  large-scale  variables  (as  assumptions! 

• Large-scale  variables  (shown  as  ovals)  are  assumed  variables  and  were  elicited  based  from  researchers 
own  experiences,  their  pedagogical  view  and  their  view  of  the  learning  environment  (See  Figure  1). 

2.  Qualitative  data  collection 

• Data  collection  was  conducted  based  upon  qualitative  research  methodology.  Data  was  collected  by 
observational  participation  of  actual  classes;  observation  of  teacher’s  talk,  students’  talk  and  behaviors, 
and  interviews  with  teachers  and  students. 

• Most  of  the  data  was  recorded  using  video,  pictures  and  notes.  lesson  plans,  teaching  materials,  the 
configuration  of  the  computers  in  the  classroom,  the  atmosphere  of  the  classroom  and  social  relations 
between  students  and  teachers  were  recorded.  These  factors  were  often  found  toplay  an  important  role  in 
the  success  of  lessons.  However,  it  is  difficult  to  record  inner  factors  such  as  human  relations  or  teacher’s 
classroom  management  competency  through  observations  and  interviews,  even  though  these  factors  are 
important. 

1 To  pick  up  definitive  keywords  through  interpretation  of  qualitative  data 

• Keywords  were  added  to  these  data.  By  observing  pictures  and  videotapes,  reading  notes  and  interviews’ 
records,  researchers  were  able  to  think  of  many  keywords  corresponding  to  the  classroom  scenes  that  had 
been  recorded.  These  are  ranged  from  simple  facts  to  more  meaningful  findings.  These  keywords  leading 
to  meaningful  findings  a*e  defined  as  ‘definitive  keywords’  in  this  paper  and  were  added  to  the  casual 
relation  graph  as  sub- variables  (rectangle  in  shape). 

• Researchers  picked  up  definitive  keywords,  which  were  selected  based  upon  their  impression  and 
interpretation  of  data. 

4.  To  find  a relation  between  large-scale  variables  and  keywords 

• Researchers  linkedthese  definitive  keywords  to  the  large-scale  variables  based  upon  their  judgment. 
Because  (fefinitive  keywords  are  crucial,  important  keywords,  these  can  be  related  to  large-scale 
variables  in  most  cases. 

• Keywords  were  added  to  the  large-scale  variables.  All  ^ariables  could  be  listed  by  adding  definitive 
keywords  tothe  large-scale  variables. 

5.  To  express  some  findings  as  statements  and  to  produce  a causal  relation  graph 

• By  analyzing  the  data  generated  by  the  observational  study,  researchers  were  not  only  able  to  identify 
keywords  but  also  identify  key  features  of  the  class.  Researchers  attempted  to  investigate  the  reason  why 
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teachers*  and  students*  talk  occurred,  tried  to  extract  background  theory  and  models  connected  to  these 
activities  and  tried  to  find  new  features  of  the  class  by  comparing  it  to  other  classes. 

• The  extracted  findings  were  expressed  as  statements  these  were  added  to  our  graph  with  causal 
connections  being  drawn  to  signify  the  relationship  between  one  issue  and  another  and  specific  examples 
of  the  observations  listed  below  being  denoted  as  dotted  rectangles  (See  Figure  2). 

Findings  based  on  the  observation  of  teachers  and  student  activities. 

Twelve  findings  are  summarized  and  presented  as  the  following  statements. 

(1)  Keeping  and  memorizing  a password  is  difficult  for  children.  Example:  A teacher  comments  “Some 
children  often  forget  their  own  passwords,  therefore  requesting  children  to  change  their  passwords  for  protecting 
security  is  impossible.  So,  we  introduced  a pas  sword- card  and  asked  the  children  to  keep  the  card.” 

(2)  Children’s  keyboard  typewriting  skills  were  affected  by  the  classroom  teachers  ’ aptitude. 

A variety  of  skills  were  found  amongst  the  children  from  high-level  skills  such  as  blind  touch-typingto  the  low 
level  skills  such  as  one  fingered-typing.  The  development  of  these  skills  seemed  mainly  dependent  on  the 
frequency  of  access  to  computers  for  these  children.  Example:  A teachers  comments  “There  is  some  evidence 
showing  the  reason  why  children*s  typewriting  skills  are  so  different.  Children  in  the  class,  of  a teacher  that  often 
uses  the  computer  room  are  generally  more  skillful  in  typewriting.  On  the  other  hand,  children  of  teachers  that  do 
not  use  the  computer  room  often  are  not  as  good  at  typewriting.  And  there  is  also  appears  to  be  a difference 
between  boys  and  girls. 

(3)  A teacher’s  advice  appears  to  work  most  effectively  when  talking  to  children  individually  (directly  by 
face-to-face  communication). 

It  is  often  difficult  for  a teacher  to  communicate  individually  with  a child  during  a class.  However,  in  the 
computer  room  the  teacher  is  able  to  talk  directly  to  each  child  face-to-face.  It  provides  satisfaction  to  both  the 
teacher  and  the  children;  it  therefore  works  effectively  from  the  point  of  view  of  an  instructional  methodology. 
Example:  When  children  encountered  unfamiliar  material/procedures  such  as  operations,  messages  or 
instructions  from  the  computer,  he/she  asks  questions  to  teacher,  who  is  walking  around  the  computer  room.  A 
teacher  comments,  “1  was  satisfied  to  communicate  with  children  directly  face-to  face 

(4)  Children  learn  mainly  by  observing  contents  displayed  on  the  other  student^  screen;  it  is  interpreted  as 
them  odelingeffect. 

Children  show  their  interests  when  they  encounter  content  or  information  different  from  their  own  thinking.  When 
they  have  an  interest  in  the  information  on  the  screen,  they  observe  it  for  a much  longer  period  of  time.  Through 
peer  observations,  they  appear  to  learn  a lot  from  each  other;atype  ofmodeling  learning  environment  is  produced. 
Example:  When  children  find  the  content  or  information  different  from  their  own,  they  observe  the  screen  for  a 
much  longer  period  of  time.  Children  say,  “ This  mail  is  much  longer  than  the  one  1 sent,  so  1 am  reading  the  mail, 
and  thinking  how  to  write  more  sentences.” 

(5)  Reciprocal  teaching  will  naturally  happen,  when  they  encounter  the  unknown. 

When  children  encounter  the  unknown  or  find  some  questions,  they  naturally  begin  to  talk  together  in  order  to 
solve  the  problem.  And  they  exchange  their  ideas  and  products;  this  is  considered  to  be  a type  of  reciprocal 
teaching.  Example:  When  children  encountered  the  unknown,  they  asked  the  teacher  or  their  classroom  peers.  A 
girl  asked  another  child  how  to  make  a report  and  learnt  by  observing  another  child ’s  product  and  through  the 
exchange  of  ideas. 

(6)  Children  learn  more  by  accessing  the  homepages  made  by  older  children. 

Children  learn  more  by  accessing  the  products  made  by  children  in  the  upper  grade  than  in  the  same  grade.  The 
lesson  plans  that  were  designed  to  connect  children  to  different  grade  children  worked  effectively.  It  appears  that 
in  these  situations  learning  is  promoted  through  an  effect  known  as  the  zone  of  proximal  development 
(refs.[13],[14]).  Example:  Children  accessed  the  homepages  made  by  children  in  the  upper  grade,  and  they  said 
“How  wonderful  this  report  is,  1 would  like  to  know  how  to  make  it.” 

(7)  On  (Dllaborative  working  in  the  classroom,  the  group  leader’s  role  and  leadership  skills  including 
intelligence  are  important  and  crucial  for  leading  the  group  through  to  success. 

It  is  known  that  the  leader’s  role  is  very  important  not  only  for  group  work  in  schools  but  also  in  general  office 
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work.  Expert  teachers,  who  are  experienced  users  and  instructors  of  ICT  education,  talked  of  the  importance  of 
the  above-described  findings.  Example:  Children  producing  a report  through  a group  working  activity.  A child, 
the  leader  of  the  group,  says  “Be  careful  when  saving,  because  we  must  make  the  file  name  easy  to  access  and 
easily  recognized  by  our  members.”  A teacher  remarks,  “ The  outcome  of  group  work  often  depends  on  the 
leader’s  power  and  character.  ” 

(8)  Comparing  and  observing  other  group’s  work  through  a computer  network  brings  competitive  feelings 
to  group  members,  and  a competitive  willingness  promotes  children’s incentivestowards  producing  a better 
performance. 

The  computer  network  enables  children  to  access  and  observe  other  groups*  work  and  performance;  it  leads  to 
group  members  ’ exhibiting  competitive  feeling  and  willingness.  Sometimes  it  is  more  effective  to  use  group  work 
to  produce  better  products.  Children  learn  a lot  of  techniques  and  superior  design  methods  through  group 
competition.  Example:  Through  accessing  and  observing  other  group’s  work,  children  can  compare  and  evaluate 
their  own  performance.  Group  members  say,  “Let’s  introduce  more  sophisticated  techniques  so  that  we  can 
compete  with  the  other  groups.” 

(9)  When  group  members  combine  the  individual  work  made  separately  into  one  product  at  final  stage, 
children  show  a lot  of  satisfaction. 

Collaborative  learning  makes  children  produce  a more  superior  product  than  individual  learning.  It  also  brings 
deep  satisfaction  to  children  especially  when  combining  their  individual  pieces  of  work  into  one  product  at  the 
final  stage.  It  is  also  important  from  a pedagogical  point  of  view.  The  computer  network  system  enables  to  realize 
these  learning  activities.  Example:  Group  members  are  combining  individual  work  made  separately  into  one 
product.  They  watch  enthusiastically  the  screen  displaying  their  product.  They  say  “Oh,  great.  Let’s  revise  here 
and  add  another  picture” 

(10)  Technical  support  is  needed  especially  when  using  synchronized  telecommunication  systems  such  as  a 
videoconference  system. 

Most  classroom  teachers  are  not  computer  network  experts;  therefore  technical  problems  that  may  occur  may  not 
be  able  to  be  dealt  with  by  the  teacher.  Team -teaching  such  as  the  classroom  teacher  coupled  with  computer 
experts  such  as  computer  teacher  or  technical  assistant  is  much  more  effective  when  conducting  a class  using 
complicated  systems.  To  remove  the  classroom  teacher’s  fear  is  essential.  Example:  A class  using  a 
videoconference  system  when  a technical  trouble  occurred.  Classroom  teacher  comments,  “I  cannot  do  anything, 
because  I ‘m  not  computer  network  expert.  We  don’t  have  enough  time  to  check  as  to  how  this  problem  occurred.” 
When  later  asked  about  their  feelings  about  working  with  technical  equipment  in  the  classroom  one  teacher 
remarked,  “My  fear  disappears  when  team  teaching  together  with  the  technical  assistants 

(11)  Accessing  desirable  sites  using  search  engines  with  keyword  inputs  is  not  easy  for  children  it  takes  time 
and  requires  domain  knowledge. 

Most  children  would  like  to  retrieve  desirable  sites  using  search  engine  with  keyword  inputs,  but  in  many  cases 
they  are  unsuccessful  with  their  attempts.  In  order  to  get  desirable  information,  children  must  have  domain 
knowledge  related  to  the  topics  they  are  searching  for,  not  only  operational  skills  such  as  how  to  use  a search 
engine.  Without  domain  knowledge,  they  often  failed  to  access  desirable  information.  In  addition,  it  is  often  not 
easy  for  children  to  remember  suitable  keywords  for  retrieving  information,  because  these  keywords  are  often 
closely  connected  to  the  domain  knowledge.  Scaffoldings  by  teacher  or  other  students’  having  more  domain 
knowledge  are  required.  Example:  From  the  observation  of  children’s  activities  in  retrieving  a site  using  search 
engines  the  following  was  found.  Children  that  had  been  attempting  to  access  topic  specific  sites  for  a long  period 
of  time  abandoned  their  attempts  to  try  and  retrieve  information  using  a search  engine. 

(12)  Using  a variety  of  resources  enables  children  to  search  out  information  more  effectively. 

There  are  many  learning  resources  such  as  the  Internet,  CD-ROM,  newspapers,  books,  videotapes  and  so  on. 
When  children  search  for  topic  related  or  make  reports  a variety  of  resources  that  holds  relevant  material  should 
be  made  available  to  them.  Information  on  the  Internet  is  not  always  reliable  as  is  well  known.  At  present  a 
combination  approach  is  required  which  would  allow  children  to  use  the  Internet  as  well  as  other  resources  in 
order  to  ensure  an  effective  learning  environment.  Example:  When  children  are  told  to  produce  reports  as  their 
assignments,  they  often  go  to  search  for  additional  information.  As  resources  for  investigation  they  can  use  the 
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Internet,  CD-ROM,  newspapers,  books  in  the  library,  and  construct  a report  according  to  their  teachers* 


recommendations. 


Figure  2.  Causal  relation  graph  based  on  qualitative  data  analysis 

Oval-  assumed  laree-scale  variables,  rectanele-  sub  variables 

Conclusion 

The  twelve  findings  described  above  are  represented  as  a causal  relation  graph  shown  in  Figure. 2 The  important 
findings  from  pedagogical  view  point  can  be  summarized  as  the  following  statements. 

(1)  Basic  computer  operating  skills  such  as  type  writing  often  depend  on  the  classroom  teacher*s  attitude  and 
computer  literacy  level. 

(2)  Most  computer  operation  skills  such  as  file  saving,  and  design  skills  such  as  homepage  making  are  mastered 
through  reciprocal  teaching  andmodeling  among  children. 

(3)  Teacher’s  advice  helps  children  to  search  and  access  information  related  to  their  topics,  moreover  it  was 
found  that  the  access  of  topic  related  effective  information  is  closely  tied  to  domain  knowledge  a child  has 
rather  than  operational  skills. 

(4)  Learning  motivation  is  highly  promoted  by  collaborative  and  competitive  group  activities. 

(5)  Integration  of  ynchronized  systems  such  as  videoconferencing  in  classes  needs  professional  technical 
support. 

From  the  pedagogical  view,  reciprocal  teaching  and  modeling  among  children  (2),  and  teachers  ’ advice  (3)  seems 
to  be  more  important  than  other  findings,  as  it  effectively  highlights  the  different  roles  of  novice  and  expert  users 
of  ICT  in  classes.  In  order  to  integrate  ICT  into  classes,  ICT  literacy  such  as  type  writing  skills,  file  operation 
skills,  design  skills,  and  a competent  understanding  of  the  learning  domain  are  required.  Integration  of  ICT 
literacy  and  domain  understanding  is  essential  in  promoting  ICT  education  in  schools.  Traditional  teaching 
methodology  is  mainly  based  on  the  teacher’s  knowledge  and  their  teaching  strategy  regarding  how  to  transmit 
their  expert  knowledge  using  authorized  teaching  materials.  However,  ICT  literacy  can  be  acquired  by  reciprocal 
teaching  and  modeling  among  children  without  the  teachers’  assistance.  In  the  classroom,  there  are  often  some 
children  that  are  competent  computer  users  because  they  have  access  to  and  often  use  a computer  in  their  own 
home.  These  children  play  an  important  role  in  the  transmission  of  skills  to  other  children  through  cooperative 
activities.  Collaborative  learning  among  children  was  found  to  be  most  effective  in  the  domain  of  computer  skills  ’ 
acquisition.  This  finding  can  be  interpreted  by  usingthe  concept  of  Vygotsky ’s  zone  of  proximal  development,  a 
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concept  that  has  previously  been  noted  as  playing  an  important  role  inof  ICT  education  (refs. [14], [15]). 

Most  children  do  however  fail  to  access  suitable  Web  sites.  This  suggests  that  information  literacy  or  the  ability  to 
search  for  information,  utiliz^ion,  representation  and  so  on  are  closely  related  to  the  child’s  knowledge  of  the 
each  specific  information  domain.  The  teacher  as  an  expert  of  each  domain  can  play  the  important  role  in 
cultivating  the  child’s  ability  to  strengthen  their  information  gathering  skills.  It  can  be  concluded  that  information 
literacy  or  capability  cannot  be  separated  from  domain  knowledge,  and  that  information  capability  needs  to  be 
acquired  through  subject  learning  such  as  cross-curriculum  through  teacher  guided  assistance,  while  computer 
operation  skills  can  be  acquired  through  peer  teaching  and  modeling  from  other  children  through  collaborative 
work  sessions . 
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Abstract 

The  paper  describes  the  issues  and  prospects  of  e-learning  in  Oman  educational  system. 
Technology  has  transformed  the  practice  of  education  in  many  countries  and  Oman  is  set  to 
modernize  her  education  to  align  with  the  rest  of  the  world.  Several  issues  are  receiving 
attention  as  policies  are  formulated  and  the  requisite  foundations  are  being  laid  in  preparation 
for  the  national  distance  education  program  in  Oman. 

Introduction 

The  Ministry  of  Education  in  the  Sultanate  of  Oman  has  continued  to  put  the  education  of  both  girls 
and  boys  high  on  her  priority  list.  During  the  Five-Year  Plan  1996-2000,  a total  of  107  new  schools 
were  built  as  the  Ministry  has  a comprehensive  plan  to  modernize  the  education  system  to  meet  with 
meet  the  needs  of  the  Century.  By  the  year  2000  about  565,856  boys  and  girls,  aged  6-18  years 
were  in  schools  (Ministry  of  Information,  2000).  In  1994,  there  were  about  3,600  students,  in  the 
various  specializations  at  the  only  university,  Sultan  Qaboos  University  (SQU)  with  56%  females  and 
44%  males.  Only  a small  fraction  of  high  school  graduates  get  admitted  into  SQU. 

E-leaming  is  of  interest  in  Oman  because  of  the  apparent  inability  of  the  only  public  university  to  cope 
with  the  large  number  of  students  who  qualify  for  higher  education  but  cannot  be  offered  places. 
Research  and  experience  around  the  world  have  shown  that  e-leaming  can  bridge  this  gap.  Canning- 
Wilson  (2000)  reports  experts’  prediction  that  in  the  next  few  decades,  over  50%  of  student  population 
will  be  educated  using  on-line  learning  and/or  technology.  What  then  is  e-leaming? 

The  most  important  concerns  at  this  point  in  Omani  educational  development  are  whether  or  not  e- 
leaming  is  the  best  alternative  in  order  to  increase  University  admissions  of  Omani  high  school 
graduates?  To  what  extent  can  policy-makers  and  educators  tmst  e-leaming  in  producing  high  quality 
university  graduates?  What  are  the  criteria  for  assessing  the  quality  of  e-leaming  to  accord  it  the 
reasonable  credibility  for  acceptance?  What  are  the  issues  involved  in  e-leaming  security?  These 
concerns  are,  in  a sense  universal.  A survey  entitled  'The  States  of  E-Learning  in  the  States"  found 
that  the  current  challenges  most  frequently  cited  by  the  States  are  the  costs  of  developing  content  and 
training  instmctors;  the  necessary  enlargement  of  infrastmcture  capacity;  the  quality  of  courses  and 
content;  agreements  on  articulation  and  residency;  the  responsiveness  of  traditional  institutions;  and 
issues  of  privacy  and  intellectual  property  rights.  The  report  shows  that  States  rank  quality  issues  as 
their  highest  concern  (NGA,  2001). 

IT  Status  and  Prospects  of  {^Learning  in  Oman 

The  Sultanate  of  Oman  recognized  IT  as  an  enormous  untapped  wealth  and  began  to  focus  her  efforts 
in  setting  up  IT  business  parks  and  Information  Technology  institutions.  Apart  from  IT  adoption  in 
Governmental  and  the  private  sectors,  educational  institutions  have  embraced  IT  on  a large  scale.  In 
the  last  few  years,  there  had  been  a proliferation  of  the  Internet  Cafes  in  major  cities  of  Oman  and  IT 
literacy  continues  to  improve  with  many  citizens  using  the  email  facilities  and  surfing  the  Web  for 
needed  information  on  a daily  basis. 

It  is  important  that  Oman  is  able  to  keep  pace  with  the  rest  of  the  world.  Al  Balooshi  (2001)  asserts 
that  e-leaming  is  the  “now  big  thing”  and  not  the  “next  big  thing”  and  that  e-leaming  must  be  viewed 
with  some  seriousness  in  the  Gulf  region.  On-line  learning  is  the  future  of  education  and  it  is  certain 
that  those  teachers  who  are  still  trapped  in  the  in  the  chalk  and  talk  tradition  will  be  left  behind  as 
education  advances  into  the  21*^  century.  Canning- Wilson  (2000)  Further  states  an  assumption  that 
“Educators  in  the  Arabian  Gulf  and  worldwide  will  need  to  be  more  proficient  in  Educational 
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Technology,  more  aware  of  the  theoretical  and  practical  aspects  of  foreign  and  second  language 
teaching,  as  well  as  increase  recognizing  the  need  to  build  further  awareness  of  how  teaching 
methodologies,  learning  strategies,  and  learning  may  be  altered  based  on  this  new  medium  of  on-line 
education”. 

In  Oman,  the  population  is  spread  thinly  over  a wide  geographic  area.  To  ensure  that  the  populace  has 
access  to  the  resources  they  need,  two  solutions  exist  which  concern  telecommunications.  First, 
electronic  storage  and  retrieval  of  information  via  the  Internet  will  continue  to  erode  the  role, 
traditionally  filled  by  books  and  printed  media.  Secondly,  e-leaming  technologies  will  bring  education 
to  the  Omanis  by  providing  access  to  learning  resources  at  a wide  variety  of  locations,  rather  than 
making  people  travel  to  education.  The  World  Wide  Web  offers  a truly  global  library  of  a scale 
unimaginable  and  it  is  available  equally  to  students  studying  at  home  or  anywhere  in  Oman.  Oman 
Telecommunication  Company’s  Internet  service  has  struggled  from  time  to  time  as  demands 
constantly  outstripped  capacity.  There  are  about  40,000  registered  Internet  customers  and  over  90,000 
users.  It  is  understood  that  today  SQU  alone  has  over  15,000  users.  Omani  students  get  linked  with 
their  counterparts  in  any  part  of  the  world  through  the  Internet.  (A1  Rawahy,  2001). 

Internet  instructional  uses  by  SQU  faculty  members  are  however,  mostly  limited  to  obtaining 
information  and  rich  resources  available  at  all  times.  This  suggests  that  they  should  be  trained  and 
encouraged  to  broaden  their  use  beyond  the  present  status  (Abdel  Rahim  & A1  Musawi,  2002). 
Currently,  there  are  attempts  at  e-leaming,  using  the  WebCT  by  faculty  at  SQU  and  a study  on  the 
implementation  and  perspectives  of  the  early  adopters  has  been  conducted  (Akinyemi  et  al,  2002). 
Omani  educational  system  needs  to  learn  from  other  countries’  experiences  in  her  development. 

Conclusions 

While  Oman  is  not  oblivious  of  the  advantages  of  e-leaming  many  Omani  educators  will  be  better 
convinced  with  more  research  evidence  on  the  quality,  security  and  credibility  issues  of  e-leaming. 
The  Omani  concern  for  comparability  of  standards  with  the  traditional  system  is  a genuine  one  which 
must  be  attended  to  before  e-leaming  can  be  developed,  disseminated  and  diffused  into  higher 
education  on  a large  scale.  According  to  the  call  by  Al  Balooshi  (2001)  and  Al  Majdoub  (2001),  there 
is  a need  for  ^learning  strategic  plan  for  the  Arab  world  and  Oman  will  be  a fore-runner  in  the 
formulation  of  such  e-leaming  strategies. 
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Abstract;  This  paper  describes  the  development  of  a web-enabled  information  service  for 
constituents  of  the  Information  Resources  Management  College.  The  constituents  of  the 
College,  who  include  graduates,  current  students,  and  prospective  students,  typically  work  in 
the  Chief  Information  Officer  (CIO)  office  of  United  States  federal  agencies.  The  web-enabled 
information  service,  known  as  “Knowledge  Net,”  is  intended  to  tie  the  College  constituents 
located  throughout  the  world  into  a virtual  community  - sharing  technical  information, 
emeiging  problems,  and  potential  solutions.  Knowledge  Net  has  evolved  over  a three-year 
span  from  a skunk- works  project  of  several  faculty  members  to  an  institutionalized  system 
supported  by  the  University.  The  most  significant  lessons  learned  to  date  include  the 
requirement  for  a web-enabled  content  management  system  to  ease  posting  of  information  to 
the  website  and  the  need  to  adjust  administrative  policy  to  encourage  faculty  to  take  on  and 
integrate  Knowledge  Net  related  activities  into  their  ongoing  academic  responsibilities. 


Introduction 

The  Information  Resources  Management  (IRM)  College  of  the  National  Defense  University  is,  in  a 
sense,  a Corporate  University  for  information  technology  executives  in  the  Department  of  Defense  (DOD)  and 
other  federal  agencies.  The  IRM  College,  located  in  Washington  D.C.,  offers  several  graduate-level  programs 
dealing  with  policy,  strategy,  development,  and  management  of  information  technology  systems. 

One  of  the  major  programs  offered  by  the  College  is  the  Chief  Information  Officer  (CIO)  Certificate 
program.  This  program,  attended  by  mid-to-senior  level  managers  in  the  Department  of  Defense  as  well  as  all 
other  federal  agencies,  leads  to  a CIO  Certificate  acknowledged  by  the  Federal  CIO  Council.  The  certificate 
attests  that  graduates  of  the  program  have  received  education  in  the  skill  and  knowledge  required  of  federal 
Chief  Information  Officers.  Also,  students  completing  the  program  earn  15  graduate-level  credit  hours  that  can 
be  applied  towards  Master’s  Degrees  at  several  civilian  institutions.  The  students  earn  the  CIO  Certificate  by 
successfully  completing  eight  of  approximately  20  graduate-level  courses  from  ten  defined  competency  areas: 
policy,  strategic  planning,  leadership,  process  improvement,  capital  planning,  performance  measurement, 
technology  assessment,  systems  architecture,  security^  emd system  acquisition. 

Like  many  other  corporate  universities,  the  IRM  College  conducts  its  courses  face-to-face  and  via  an 
asynchronous  distributed  education.  However,  in  its  continuing  education  role  for  our  primary  constituents,  the 
College  recognized  that  formal  courses  (residential  and  distributed)  are  only  one  way  of  meeting  their  ongoing 
information  needs.  There  are  perhaps  twenty  thousand  people  worldwide  serving  in  the  CIO  organizations  of 
DOD  and  other  federal  agencies.  The  IRM  College  has  the  capacity  for  handling  a maximum  of  2,000  students 
per  year.  How  should  the  College  support  the  other  18,000  individuals?  How  can  it  better  serve  the  graduates  of 
its  program? 

These  twenty  thousand  people  are  also  in  need  of  continuing  development  and  support.  They  work  in 
the  field  of  information  technology  that  changes  radically,  not  just  in  the  superficial  “bells  and  whistles”  of  the 
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systems,  but  even  more  so  in  the  requirements  for  management  oversight.  How  will  conversion  to  a wireless 
strategy  change  the  fundamental  way  business  is  accomplished?  How  can  the  greater  demand  for  information 
security  be  balanced  with  increasing  concerns  of  information  privacy?  The  people  who  work  in  federal  CIO 
organizations  worldwide  are  continually  bombarded  with  new  demands  and  challenges  for  which  they  were 
unprepared  by  educational  programs  taken  just  six  months  earlier.  Curriculum  development  activities  at  the  IRM 
College  indicate  that  about  half  of  the  program  content  must  be  significantly  revised  each  year.  In  addition,  even 
though  there  is  a commonality  of  issues  through  which  all  members  of  federal  CIO  staff  must  work  - whether  at 
an  outpost  in  Korea  or  at  a government  research  facility  in  North  Carolina  - each  unit  tends  to  do  so 
individually,  on  its  own.  Historically,  there  has  been  no  community  by  which  federal  CIO  staff  can  share 
common  problems  and  solutions.  With  the  exception  of  the  infrequent  periods  of  time  when  some  of  them  are 
jointly  enrolled  in  courses  at  the  IRM  College,  they  are  on  their  own  - keeping  up  with  the  technology  as  best 
they  can,  testing  out  their  own  solutions  until  their  next  opportunity  to  occupy  one  of  the  limited  slots  for 
attendance  at  the  IRM  College. 


The  Information  Service 

The  emergence  of  the  World  Wide  Web,  its  growing  accessibility,  and  rapidly  developing  functionality 
offer  a supplemental  approach  for  achieving  the  mission  of  continuing  education.  The  model  conceived  to  take 
advantage  of  the  evolving  capabilities  of  the  Web  - beyond  conducting  formal  courses  online  - has  been 
designated  as  the  NDU  Knowledge  Net  (http://www.nduknowledge.net/).  The  NDU  Knowledge  Net  is  a web- 
enabled  service  seen  as  an  alternative  Information  Age  strategy  for  the  University  to  deliver  just-in-time 
continuing  education  to  its  constituency.  As  originally  conceived,  it  provided  the  following  web-enabled 
services  for  each  of  the  ten  competencies  identified  by  the  Federal  CIO  Council: 

• FAOs:  Frequently  Asked  Questions  and  answers  concerning  fundamental  concepts 

• NEWS:  Summary  and  links  to  recent  developments  concerning  the  competency 

• GUIDES:  Software  tools,  decision  aids,  and  “How  to”  guidelines  to  do  things. 

• EVENTS:  Links  to  training/educational  programs  and  conferences  related  to  the  competency 

• EXAMPLES : Success  stories  and  sample  products  of  real-world  applications 

• RESOURCES:  Annotated  links  to  related  references  on  the  Web 

Two  additional  services  were  envisioned  for  the  information  service: 

• FORUM:  Threaded  discussions  on  topical  issues  facilitated  by  faculty  and  users 

• LIVE:  Entry  to  scheduled  real-time  discussions  and  briefings  on  topical  issues 

The  home  page  for  the  original  Knowledge  Net  provided  direct  access  to  each  of  the  ten  competency 
areas  (e.g.,  policy,  strategic  planning,  leadership,...).  Access  was  also  provided  to  a search  engine  for 
searching  all  the  Knowledge  Net  content,  a statement  of  the  web  policy  governing  the  Knowledge  Net  web  site, 
a glossary  of  terms  associated  with  the  CIO  position,  and  links  to  related  sites. 

When  the  user  clicked  on  the  name  of  a competency  on  the  home  page,  the  NEWS  page  for  the  selected 
competency  area  opened.  From  here,  the  users  had  a number  of  options.  They  could... 

• scroll  down  the  summaries  of  articles  until  they  find  one  in  which  they  are  interested,  and  then 
click  on  the  arrow  icon  to  open  that  article  in  a separate  window. 

• open  a different  kind  of  page  for  the  same  competency  (e.g.,  EVENTS,  GUIDES,...)  by  clicking 
on  one  of  the  tabs  near  the  top  of  the  display. 

• access  the  same  kind  of  page  in  another  competency  area  (e.g..  Policy,  Strategic  Planning, 
Leadership,..)  by  clicking  on  one  of  the  icons  at  the  top  of  the  page  . 

• access  any  of  the  general  purpose  pages  (e.g.,  search,  glossary,  contact  us,...)  by  clicking  on 
an  appropriate  section  of  the  image  map  in  the  upper  left  section  of  the  display. 
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Each  of  the  other  kinds  of  pages  has  the  same  functionality  as  the  NEWS  page. 

One  Content  Manager  was  responsible  for  each  of  the  ten  CIO  competency  areas.  Volunteer  faculty 
members  with  technical  specialties  in  the  associated  competency  accomplished  these  roles.  They  searched  for 
relevant  content,  in  part  to  support  their  normal  course  development  and  teaching  duties,  and  in  part  to  maintain 
the  currency  of  the  Knowledge  Net  competency  web  pages  for  which  they  are  responsible.  To  assist  them  in  this 
role,  a pool  of  students  doing  independent  study  courses  in  related  topics  and  research  assistants  was  made 
available  to  them.  They  were  also  encouraged  to  establish  special  interest  groups  composed  of  other  faculty, 
alumni,  and  practitioners  in  the  field  to  feed  them  relevant  sources  of  information.  Content  Manager  are  also 
provided  with  an  account  for  a software  agent  under  their  control  that  continually  crawls  the  web  and  reports 
back  on  a periodic  basis  with  updates  to  its  discovery  of  relevant  content. 

As  the  Content  Managers  identified  items  for  their  Knowledge  Net  area,  they  entered  the  required 
information  into  an  electronic  form  designed  for  this  purpose.  Their  submissions  were  entered  into  a database. 
At  this  time,  items  were  uploaded  from  the  database  to  the  Knowledge  Net  server  manually  by  a Webmaster,  to 
allow  for  a quality  control  check. 

The  faculty  member  who  conceived  the  idea  of  Knowledge  Net  served  as  its  coordinator.  His  primary 
roles  were  to  coordinate  the  activities  of  the  Content  Managers,  provide  quality  control  over  the  website,  and 
serve  as  a catalyst  for  continual  improvement  and  redesign  of  the  site.  He  was  supported  by  a Graduate 
Research  Assistant  who  acted  as  the  Knowledge  Net  Webmaster.  Site  statistics  were  made  available  from  the 
external  website  hosting  service,  and  mechanical  checks  on  the  site  were  conducted  periodically  by  a 
subscription  service  looking  for  broken  links,  misspellings,  and  sources  of  excessive  downloading  times. 
Regular  meetings  were  conducted  with  the  Content  Managers  and  typical  users  to  help  identify  opportunities  for 
improvement. 


Lessons  Learned 

The  initial  phase  of  Knowledge  Net  development  covered  in  this  paper  is  restricted  to  the  veb 
publishing  function  - the  posting  of  relevant  information  for  users.  The  discussion  and  streaming  capabilities 
will  be  introduced  into  Knowledge  Net  later  this  year. 

In  the  earliest  stage.  Knowledge  Net  was  hosted  on  a local  University  server.  The  Content  Managers 
created  and  updated  pages  {News,  Events,  ...)  associated  with  their  assigned  competency  area  using  a web 
editor  software  package  (i.e..  Frontpage).  Updated  pages  were  sent  to  the  University  Webmaster  who  in  turn 
would  upload  the  pages  to  the  server.  This  approach  proved  disastrous.  Firstly,  although  IRM  College  faculty 
members  are  quite  proficient  in  management  concepts  associated  with  information  technology,  their  expertise 
with  web  editors  varied  considerably.  The  updating  process  required  that  the  Content  Managers  first  download 
the  current  page  from  the  web  server  (to  assure  that  the  page  being  revised  included  all  current  changes),  update 
it  with  the  web  editor,  and  then  send  it  to  the  Webmaster  for  uploading  back  to  the  server.  This  did  not  always 
happen  this  way.  Occasionally,  a Content  Manager  would  update  the  web  page  on  his  or  her  local  drive  and 
send  it  to  the  Webmaster  without  first  downloading  the  most  current  page  from  the  server.  If  the  page  on  the 
server  was  a more  recent  version  than  the  page  on  the  Content  Manager’s  local  drive,  this  action  would 
overwrite  previous  changes  and  corrections.  In  other  words,  configuration  management  of  the  website  was  a 
nightmare.  Plus,  the  lack  of  capability  with  web  editors  by  some  Content  Managers  led  to  some  very  strange 
looking  web  pages  uploaded  to  the  server. 

Another  problem  associated  with  this  early  stage  in  Knowledge  Net  development  was  its  dependence 
on  University  assets  - the  University  Webmaster  and  the  hosting  of  the  website  on  the  University  web  server. 
As  an  experimental  project  of  the  IRM  College,  Knowledge  Net  was  not  high  on  the  list  of  University  priorities. 
The  University  Webmaster  was  managing  multiple  web  projects,  some  of  great  concern  to  the  University  senior 
administration.  It  was  not  unusual  for  pages  updated  by  the  Content  Managers  to  remain  on  the  Webmaster’s 
system  for  a week  or  more  waiting  to  be  uploaded  to  the  web  server  while  the  Webmaster  attended  to  more 
pressing  University  duties.  Also,  periodic  weekend  maintenance  of  the  University’s  systems  made  Knowledge 
Net  frequently  inaccessible  to  users. 
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Both  these  problems  were  solved  fairly  easily  by  hiring  a part-time  Graduate  Research  Assistant  as  the 
Knowledge  Net  Webmaster  and  outsourcing  the  hosting  of  Knowledge  Net  to  an  external  organization  at  a cost 
of  about  $200  per  year.  The  website  is  now  available  24/7  with  rare  periods  of  inaccessibility.  The  new 
Knowledge  Net  Webmaster  quickly  created  simple  web-based  transaction  forms  and  scripts  to  elicit  new  content 
items  from  the  Content  Managers.  In  this  situation,  a Content  Manager  wishing  to  update  a page  with  new 
information  accesses  a web-based  form  customized  to  the  type  of  page  being  updated.  After  entering  the 
requested  data  (e.g.,  title,  date,  link,  summary),  the  Content  Manager  can  preview  the  appearance  of  the  new 
item  and  then  click  on  the  Submit  button  to  send  it  to  the  Webmaster.  Content  management  no  longer  required 
the  use  of  web  editors  and  configuration  management  was  completely  in  the  hands  of  the  Webmaster.  The 
Webmaster  was  asked  to  upload  new  items  to  the  server  within  three  days  of  receipt,  a requirement  that  was 
generally  met  except  during  periods  of  illness,  vacation,  and  competing  academic  pressures  on  the  Research 
Assistant.  More  recently,  the  Webmaster  created  a new  system  in  which  new  items  submitted  by  the  Content 
Managers  are  entered  directly  into  a database  that,  in  turn,  generates  the  updated  web  pages  on  the  fly.  In  other 
words,  as  soon  as  a Content  Manager  submits  a new  item,  it  is  immediately  displayed  on  the  Knowledge  Net 
website.  Content  Managers  now  also  have  the  ability  to  edit  and  delete  items  from  the  database  themselves. 
They  are  now,  indeed,  content  managers. 

Another  problem  led  to  a major  overhaul  of  the  Knowledge  Net  website  this  year.  Up  until  recently,  the 
Knowledge  Net  areas  (e.g..  Strategic  Planning,  Capital  Planning,,,,)  were  based  on  the  ten  competencies 
originally  specified  for  Department  of  Defense  CIO’s.  As  such,  ten  IRM  faculty  members  were  recruited  to  serve 
as  Knowledge  Net  Content  Managers,  one  for  each  competency.  The  selected  faculty  members  undertook  this 
new  incremental  task  with  varying  degrees  of  enthusiasm.  Some  were  highly  excited  about  the  prospect  of 
managing  a portion  of  a website  devoted  to  a content  area  associated  with  their  own  intellectual  pursuits.  These 
faculty  members  saw  their  content  management  responsibility  as  an  opportunity,  spending  several  hours  a week 
in  the  search  for  new  relevant  content,  publishing  new  items  almost  every  day,  and  integrating  the  website  into 
their  academic  courses  as  a major  information  resource.  However,  other  faculty  members  were  more  reluctant  to 
embrace  their  content  management  responsibility.  They  seemed  to  see  their  role  in  Knowledge  Net  as  a burden, 
almost  a distraction  to  their  ongoing  academic  duties.  As  a result,  the  content  on  web  pages  associated  with 
these  competency  areas  might  not  be  updated  for  weeks  on  end.  This  is  a serious  problem  for  a web-based 
information  service  such  as  Knowledge  Net.  Websites  of  this  sort  must  have  fresh  content  in  order  to  attract 
and  bring  visitors  back.  At  the  request  of  the  Knowledge  Net  Coordinator,  the  College  administration  granted 
teaching  load  credit  for  Content  Managers.  It  was  hoped  that  the  ability  to  reduce  teaching  hours  in  return  for 
work  done  on  Knowledge  Net  would  both  free  up  the  necessary  time  and  serve  as  an  incentive.  Moreover,  the 
granting  of  contact  hours  gave  recognition  that  Knowledge  Net  was  a legitimate  and  important  service  of  the 
College.  Unfortunately,  this  management  action  had  virtually  no  affect  on  the  frequency  with  which  web  pages 
of  the  various  competency  areas  were  updated.  The  pace  of  activity  on  Knowledge  Net  by  both  groups  of 
Content  Managers  - those  who  saw  it  as  an  opportunity  and  those  who  saw  it  as  a problem  - went  on  as  it  was. 

The  proposed  solution  to  this  issue  led  to  a redesign  of  Knowledge  Net.  The  number  of  competency 
areas  has  been  collapsed  from  ten  to  six  and  these  six  areas  are  now  aligned  with  the  organizational  structure  of 
the  College.  The  six  areas  are  now  e-Government,  Performance  Management,  Enterprise  Architecture,  Systems 
Acquisition,  Technology  Capabilities,  and  Information  Assurance,  Each  of  the  three  academic  departments  in 
the  College  now  has  responsibility  for  two  Knowledge  Net  areas,  and  two  of  the  areas  - e-Government  and 
Information  Assurance  - are  each  aligned  directly  with  a center  of  excellence  at  the  College.  Department  Chairs 
at  the  IRM  College  have  supervisory  responsibility  over  faculty  (i.e.,  they  play  a prominent  role  in  hiring, 
contract  renewal,  work  assignment,  promotions,  and  bonuses).  The  redesign  was  based  on  the  belief  that  a more 
consistent  level  of  performance  will  be  attained  by  reducing  the  number  of  required  Knowledge  Net  Content 
Managers  (i.e.,  it  should  be  easier  to  find  six  personally  motivated  faculty  members  than  ten  such  individuals) 
and  by  delegating  their  selection  and  management  to  the  Department  Chairs.  This  change  was  initiated  about 
three  months  ago  and,  although  performance  has  been  more  consistent,  it  is  too  early  to  declare  a success. 
Everyone  is  still  exploring  the  requirements  and  practices  of  his/her  new  responsibilities. 

The  redesign  of  Knowledge  Net  also  involves  the  following  changes  that  are  expected  to  be  completed 
by  the  end  of 2002: 

• Redesign  of  the  pages  for  a common  look  & feel  consistent  with  a new  IRM  College  website. 

• Reduction  in  the  number  of  kinds  of  pages  for  each  competency  (Basics,  Articles,  Events, 
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Documents,  Websites),  which  are  easier  to  distinguish  among  than  the  previous  page  types. 

• Introduction  of  a more  powerful  search  engine. 

• Addition  of  a threaded  discussion  capability  to  be  facilitated  by  the  Knowledge  Net 
Coordinator,  with  the  voluntary  assistance  of  other  Content  Managers,  to  enable  users  to 
share  field  problems  and  solutions  with  each  other. 

• Ability  of  users  to  suggest  sources  and  items  for  inclusion  into  Knowledge  Net. 

• Presentation  of  scheduled  lectures  by  faculty  and  guest  speakers  of  the  IRM  College  using 
streaming  video. 


Conclusions 

The  existence  of  Knowledge  Net  offers  a number  of  benefits  to  the  College  and  its  constituents. 
Benefits  to  the  Constituents  of  the  College 

Knowledge  Net  enables  the  20,000  thousand  people  working  in  federal  CIO  officers  to  continue  their 
professional  development,  even  when  they  are  unable  to  enroll  in  formal  College  courses.  They  have  access  to 
current  information  about  the  field  anytime  and  from  anywhere  - information  that  has  been  selected  and  filtered 
by  faculty  experts  in  the  CIO  competency.  When  the  full  functionality  of  Knowledge  Net  comes  online  in  2002 
with  streaming  video  and  discussion  boards,  users  will  also  be  able  to  hear  and  see  guest  speakers  at  the  College 
talk  about  advanced  topics  related  to  the  field,  and  discuss  issues  with  faculty,  peers,  and  experts  around  the 
world. 

Moreover,  Knowledge  Net  benefits  the  College’s  constituents  by  helping  improve  their  job 
performance  - it  serves  as  a kind  of  electronic  performance  support  system.  Users  can  stay  abreast  of  late 
breaking  news  concerning  CIO  responsibilities.  They  can  more  easily  find  resources  to  help  do  their  jobs,  just 
when  they  need  them.  Using  the  discussion  boards,  they  can  note  how  things  should  be  done  and  how  others 
are  actually  doing  them  - noting  what  works  and  what  does  not.  They  can  even  collaborate  with  others  doing 
the  same  kinds  of  things  no  matter  where  they  are  doing  them. 

Benefits  to  the  Institution 

The  mission  of  the  IRM  College  is  to  “prepare  military  and  civilian  leaders  to  direct  the  information 
component  of  national  power  by  leveraging  information  and  information  technology  for  strategic  advantage.” 
Note  that  the  mission  says  nothing  about  preparing  these  leaders  by  means  of  educational  programs. 

Knowledge  Net  is  an  alternative  means  to  accomplish  the  College’s  mission,  one  that  is  likely  to  play  an  even 
more  prominent  role  in  the  future. 

Knowledge  Net  also  benefits  the  College  by  strengthening  its  relationship  with  its  constituents. 
Certainly,  the  information  available  on  Knowledge  Net  is  a resource  available  to  students  currently  enrolled  in 
courses,  both  residential  and  online.  It  provides  a searchable  database  of  relevant  information  useful  for  papers 
required  in  the  academic  programs.  As  mentioned  previously.  Knowledge  Net  is  also  available  to  graduates  of 
our  programs  for  continuing  education  and  development.  Alumni  thereby  remain  connected  to  the  College  long 
after  graduation.  Knowledge  Net  can  even  be  used  to  promote  the  College  programs.  Many  of  the  20,000  people 
working  in  federal  CIO  organizations  are  unaware  of  the  IRM  College’s  existence  and  the  availability  of  the  CIO 
Certificate  program.  If  Knowledge  Net  is  perceived  as  a useful  tool  on  the  job,  its  association  with  the  IRM 
College  can  continually  attract  new  students. 

Implications  For  Other  Organizations 

The  Knowledge  Net  strategy  being  developed  by  the  IRM  College  seems  to  have  value  for  other 
corporate  universities,  provided  they  see  their  missions  as  developing  and  supporting  the  job  performance  of 
constituents,  rather  than  just  conducting  training  and  education  courses.  In  a similar  vein,  professional 


associations  whose  members  share  a common  set  of  competencies  can  also  apply  the  strategy.  Even  subject- 
specific  centers  of  excellence  resident  in  institutions  of  higher  learning  can  implement  a Knowledge  Net-type 
website  to  benefit  both  their  constituents  and  the  center  itself  In  fact  any  organization  seeking  to  support 
constituents  with  a common  set  of  competencies  and  that  possess  a group  of  specialists  who  are  willing  to  adapt 
web  technology  to  serve  their  role  can  implement  a Knowledge  Net  strategy. 
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Abstract:  We  describe  the  integration  of  handheld  computer  technology  into  an  existing 
web- based  educational  platform,  the  Web-based  Inquiry  Science  Environment  (WISE)  and 
the  synergy  it  produces.  This  solution  facilitated  a research  program  that  explores  how 
handheld  computers  (PDAs,  palmtops,  etc.)  can  expand  the  scope  and  functionality  of  inquiry 
activities  in  K-12  science  and  mathematics  curriculum.  We  present  the  WISE  software  and 
curriculum  and  explain  how  combining  it  with  handheld  technology  creates  unique 
educational  opportunities.  We  then  describe  the  system  we  have  developed  and  its  future. 


Introduction 

This  paper  will  present  an  innovative  application  of  handheld  technology  for  science  education.  For 
the  past  seven  years,  the  Web-based  Inquiry  Science  Environment  (WISE)  project,  funded  by  the  National 
Science  Foundation,  has  explored  the  most  effective  designs  for  inquiry  activities  that  draw  upon  the  wealth  of 
Web  resources.  We  have  designed  an  effective  browser-based  learning  environment  that  scaffolds  students  as 
they  work  collaboratively  on  inquiry  projects. 

The  designs  of  the  WISE  learning  environment,  inquiry  curriculum  and  assessments  are  based  on  a 
pedagogical  framework  called  Scaffolded  Knowledge  Integration.  This  framework,  developed  by  Dr.  Marcia 
Linn  and  her  colleagues  (Linn  and  Hsi,  2000),  has  been  developed  through  twenty  years  of  classroom  research 
with  technology  and  inquiry  (Linn  and  Songer,  1982,  Bell,  Davis  and  Linn,  1995;  Slotta  and  Linn,  2000). 

Continuous  improvement  of  the  WISE  technology  has  resulted  in  easy-to-use  software  that  scaffolds 
students  as  they  perform  critique,  design  or  debate  projects.  A growing  library  of  such  projects  has  been 
developed  by  "WISE  Design  Partnerships"  that  include  scientists  from  agencies  like  NASA,  NOAA,  and  The 
National  Geographic  Society.  Over  the  past  several  years,  both  our  curriculum  library  and  user  base  have 
grown  dramatically.  Thousands  of  teachers  and  many  thousands  of  their  students  now  use  WISE  inquiry 
projects  in  their  science  courses. 

We  have  begun  to  research  the  challenges  faced  by  science  teachers  as  they  adopt  the  innovative 
technology  and  inquiry  methods  provided  by  WISE.  To  that  end,  we  have  formed  partnerships  with  two  large 
school  districts  — Denver  Public  Schools  (Colorado)  and  Desert  Sands  Unified  School  District  (California). 
Working  closely  with  administrators  within  these  districts,  we  have  helped  science  teachers  integrate  WISE 
activities  into  their  courses,  and  developed  networks  of  WISE  mentors  in  the  districts  to  support  them. 

Together  with  these  school  district  partnerships,  we  were  recently  awarded  a grant  of  500  Palm  IIIc 
handhelds  from  the  SRI  Palm  Education  Partnership  to  design  effective  uses  for  handheld  technology  in 
education.  Our  goal  was  to  develop  Palm  activities  that  would  complement  our  existing  library  of  Web-based 
projects,  benefiting  from  the  scaffolding  of  the  WISE  technology  and  curriculum.  We  sought  to  integrate  the 
use  of  handheld  computers  with  WISE  curriculum,  enabling  both  data  collection  activities  like  surveys  and 
field  observations  and  further,  reflection  activities  in  analyzing  the  collected  data. 

In  this  paper  we  present  an  overview  of  the  WISE  environment,  detail  the  goals  of  our  handheld 
technology  initiative,  and  describe  it  functionally.  We  then  present  our  experiences  with  its  use  to  date  and 
look  toward  the  future. 
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WISE:  The  Web-based  Inquiry  Science  Environment 


WISE  offers  a powerful  browser-based  learning  environment  for  middle  and  high  school  inquiry 
science  projects.  Students  work  collaboratively  in  these  projects,  actively  using  materials  and  software  from 
the  World  Wide  Web.  In  one  project,  students  evaluate  the  health  of  a local  creek,  modeling  the  factors  that 
contribute  to  pollution.  In  another,  they  compare  two  competing  theories  about  why  deformed  frogs  are 
appearing  in  American  waterways.  Figure  1 shows  a screenshot  of  the  WISE  learning  environment  and  Figure 
2 shows  the  growth  of  our  user  base.  An  inquiry  map  on  the  left-hand  side  of  the  browser  window  coordinates 
all  student  activities,  resulting  in  Web  materials,  pop-up  notes  or  hints  (shown  in  the  Figure),  or  any  of  a 
variety  of  other  tools  and  features,  such  as  online  discussions,  journals,  causal  maps,  data  visualizations,  and  an 
argument  editor.  Throughout  the  project,  the  inquiry  map  scaffolds  students  as  they  work  collaboratively  to 
perform  carefully  designed  inquiry  projects. 
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Figure  1 WISE  user  interface,  notes,  hints 


Figure  2 WISE  user  signups  in  past  2 years 


WISE  projects  are  designed  to  offer  teachers  a means  of  adding  inquiry  and  technology  to  their 
existing  curriculum.  Teachers  choose  from  a library  of  projects,  each  accompanied  by  a lesson  plan, 
assessments,  scoring  rubrics,  connections  to  standards,  and  opportunities  to  customize  the  project  to  local 
issues  and  curriculum  topics.  Teachers  can  monitor  and  grade  student  work,  provide  formative  feedback  during 
a project  run,  and  manage  their  student  accounts.  The  reader  is  invited  to  visit  http://wise.berkelev.edu/  where 
she  will  find  more  than  30  different  projects,  in  topics  of  physical,  earth  and  life  sciences  at  grade  levels  from  4 
through  12. 

We  have  researched  the  effectiveness  of  WISE  activities  for  student  understanding  in  a wide  range  of 
classroom  studies,  summarized  in  a recent  book  by  Linn  and  Hsi  (2000).  All  WISE  activities  are  assessed  by 
pre-post  test  items,  as  well  as  embedded  assessments,  which  show  that  students  develop  a deep  understanding 
of  the  science  content,  and  gain  important  lifelong  learning  skills  related  to  critique  of  evidence,  debate  of 
arguments,  and  design  of  personally  relevant  solutions.  WISE  research  has  demonstrated  an  impact  on  student 
understanding  of  science,  fluency  with  technology,  and  literacy  in  the  use  of  language  and  argumentation  (Linn 
and  Slotta,  2000). 


Goal  of  project 

We  seek  to  leverage  the  strength  of  the  WISE  platform  and  curriculum  to  provide  effective  new 
applications  for  handheld  technology.  Our  innovation  focuses  on  two  main  affordances  of  handheld 
technology:  1)  convenient  and  portable  data  creation,  2)  portable  dynamic  content.  Within  the  scaffolding  and 
instructional  context  of  the  WISE  learning  environment,  these  become  powerful  pedagogical  instruments. 

For  example,  working  in  the  Genetically  Modified  Foods  project,  students  download  a carefully 
designed  survey  into  their  handheld,  and  then  interview  their  friends  and  family  between  classes  or  after  school 
(e.g.,  collecting  age,  gender,  dietary  habits,  and  beliefs  about  GM  Foods).  Later,  they  put  them  in  the  syncing 
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cradle  and  all  their  data  is  transferred  to  the  WISE  databases.  This  provides  collaborative  data  for  the  students 
to  draw  upon  as  they  continue  to  debate  whether  GM  foods  are  dangerous. 

By  facilitating  data  collection,  we  enable  useful  analysis.  For  example,  in  the  Healthy  project, 

students  take  measurements  of  the  chemistry  of  the  creek  and  input  it  on-site  automatically  with  probeware  or 
manually  with  the  stylus.  In  the  classroom,  they  analyze  the  variance  among  their  class  and  how  it  relates  to 
other  classes.  Using  the  data  analysis  tools,  the  students  can  explore  the  data.  For  example,  comparing  the 
average  pH  of  the  creek  each  month  for  the  past  2 years.  (The  Creeks  project  will  be  presented  in  detail  below.) 
Such  activities  empower  the  handheld  with  the  context  of  a broader  curriculum. 


System  description 

In  the  course  of  bringing  our  vision  to  handhelds,  we  have  developed  a versatile  collaborative 
database  that  can  be  used  to  collect  and  analyze  any  sort  of  data  in  any  context.  To  explain,  we  will  describe  a 
mock  application  from  start  to  finish. 

Remember  that  a WISE  project  consists  of  steps  that  the  student  navigates  through  using  the  inquiry 
map  (far  left  of  Fig  1).  To  bring  collaborative  data  into  a WISE  project,  the  author  adds  a step  from  the  Data 
Collection  / Analysis  family.  We  begin  by  selecting  the  “Form  blank”  step  (Figure  3).  A form  blank  is  the 
empty  form  that  is  to  be  filled  out. 
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Figure  3:  Creating  a new  step  in  the  project 
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Figure  4:  Authoring  an  analysis  step 
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Within  the  form  blank  authoring  window,  we  author  the  form  blanks  for  this  project.  A form  blank  is 
comprised  of  a sequence  of  form  fields.  A form  field  is  comprised  of  a prompt  and  a data  input  definition.  Data 
input  types  include  text,  numbers,  radio  buttons,  checkboxes,  pop-up  menus,  lists,  etc.  To  define  a form  field, 
the  author  chooses  the  type  and  then  sets  the  parameters  of  that  type.  For  instance,  if  it  is  a checkbox  field  they 
will  define  the  choices  and  optionally  include  an  image  to  represent  each  choice. 

For  the  sake  of  explanation,  let  us  author  a form  called  “Fishies”.  It  consists  of  three  form  fields.  The 
first  has  a prompt  of  “Which  tank  did  you  watch?”  and  a pop-up  menu  to  choose  between  Main,  Tropical,  and 
Arctic.  The  second,  “Which  fish  did  you  watch?”  lets  the  user  choose  images  of  a guppy,  a tuna,  and  a shark. 
The  third,  “How  many  did  you  count?”  lets  the  user  input  a number. 

We  also  add  a “Form  analysis”  step.  Here  we  choose  which  fields  to  analyze.  We  also  choose  the 
scopes  of  data  to  include  in  the  analysis:  just  the  group’s  data,  the  whole  class’s  data,  data  for  all  users  of  the 
project,  data  for  all  projects.  (A  form  field,  once  defined,  can  be  reused  in  any  form  blank  even  across  different 
projects.)  We  also  choose  what  data  filters  to  offer  to  the  student  in  order  to  guide  their  reflection.  Filters  can 
omit  or  include  responses  that  meet  certain  criteria,  such  as  temporal  range  or  response  to  a field. 

For  our  project,  let  us  choose  to  analyze  the  fish  count  based  on  the  tank  and  fish  that  the  student 
watched  (Figure  4).  Now  we  have  our  collaborative  data  collection  and  reflection  working  in  the  web  browser 
(Figure  5).  This  is  all  well  and  good,  but  we  want  the  students  to  have  Figure  5 on  their  handhelds.  To  do  this. 
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we  go  to  the  handheld  setup  page  of  the  authoring  environment  and  define  channels  to  send  to  the  handhelds.  A 
channel  is  a selection  of  form  blanks  to  fill  out  preceded  by  introductory  text,  much  like  a WISE  project  step.  It 
is  separate  so  that  the  author  can  differentiate  the  tasks  to  perform  on  the  handheld  from  those  to  perform  in  the 
browser.  (An  in  -class  form  may  have  the  students  make  predictions  and  on-site  collect  the  true  data.) 


■IWISEH' 


Fill  out  form  Fighies 


lltaua  (few  rttcpcfttt;  | „ . 


I 


M8AQ 


1.  Whtb  bEJs,  4d  yea  watoW  jd 


X HfiW  ciicgf  <fc3  yew 


Figure  5:  Filling  out  the  form  in  the  browser 
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Figure  6:  Choosing  a form  on  the  handheld 


Once  we  have  defined  the  channels,  we  click  to  subscribe  the  handhelds  to  them.  We  currently  make 
use  of  a service  called  AvantGo  to  transfer  the  data.  (AvantGo  is  a commercial  service  and  may  not  suit  us  in 
the  long  term,  but  because  we  have  built  our  system  on  open  standards  we  can  easily  shift  to  other  services  or 
create  our  own.) 

Once  this  is  done,  the  students  put  their  handhelds  in  the  cradle  and  let  them  sync  with  the  WISE 
servers.  This  downloads  the  channels  and  form  blanks  for  them  to  fill  out.  Each  set  of  handhelds  shares  one 
AvantGo  account.  The  account  lists  the  channels  that  have  been  subscribed  (left  Figure  6).  When  the  student 
picks  up  their  handheld  for  a WISE  project,  they  select  its  channel.  In  our  example,  “EdMedia  Demo”,  They 
are  then  presented  with  the  instructions  and  list  of  form  blanks  that  the  project  author  specified  (right  Figure  6). 

In  our  example,  they  select  the  "Fishies”  form  blank.  (Figure  7 shows  how  this  form  blank  appears  on 
an  actual  Palm  IIIc,  in  contrast  to  the  web  browser  in  Figure  5.)  The  students  then  fill  out  the  form  wherever 
they  may  be.  Since  the  handhelds  are  shared  by  the  whole  class,  each  student  signs  their  form  with  their  name 
in  order  to  receive  credit.  (Any  illegible  or  no-name  work  is  sent  to  the  teacher’s  management  window  to 
reattribute.) 

When  they  return  to  the  classroom,  the  students  drop  their  handhelds  in  the  cradle  to  submit  their 
responses  to  the  WISE  server.  Then  they  return  to  their  web  browsers  to  explore  the  data  and  reflect.  The 
inquiry  map  guides  them  through  this  exploration.  Within  a Form  analysis  step  they  experiment  with  the 
parameters  of  the  filters.  The  system  then  renders  the  data  for  the  subset  they  have  defined.  (Figure  8)  This  is 
how  the  students  mine  the  meaning  out  of  their  data. 
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First  Trip:  Aquarium 


But  how  does  the  technology  work  in  the  real  world?  Our  first  trial  with  students  confirms  that  it  does 
work  in  real  classrooms.  In  partnership  with  the  Monterey  Bay  Aquarium,  we  developed  an  inquiry  project  to 
help  ’’extend  the  visit.”  (Figure  9)  Before  the  trip,  the  project  led  students  to  ask  themselves  questions  that 
helped  their  investigations  during  the  aquarium  visit.  During  the  trip  the  handheld  forms  prompted  the  student 
for  data  to  collect.  After  the  trip,  the  students  worked  through  the  analysis  steps  of  their  WISE  project  to  help 
make  sense  of  their  observations  and  integrate  their  field  experience  with  the  classroom  curriculum. 


Figure  9:  Student  recording  observations  at  aquarium 

Educators  from  the  aquarium  wanted  to  help  students  focus  on  marine  science  concepts,  including  the 
factors  relating  to  fisheries  decline.  We  co-designed  an  activity  where  students  explore  the  fish  in  Monterey 
Bay,  reflecting  on  why  some  fish  are  placed  on  a "Seafood  Watch  List  ” while  others  are  not.  Students  chose 
one  fish  for  specialization,  which  they  investigated  at  the  aquarium.  The  forms  prompted  them  to  record 
observations  about  the  fish  and  other  features  of  the  aquarium.  We  designed  an  observation  form  that  would 
help  students  reflect  on  the  habitat  and  adaptations  of  "their  fish,"  as  well  as  a checklist  of  aquarium  activities 
that  was  beamed  to  students  by  aquarium  docents.  These  observations  provided  a focus  for  student  activities  at 
the  aquarium  while  enabling  the  collection  of  student  data  for  use  in  later  stages  of  the  project. 

When  we  ran  the  project  with  students  from  an  oceanography  club  at  North  Salinas  High  School,  they 
embraced  the  handhelds  even  more  quickly  and  easily  than  we  had  expected.  After  a few  minutes,  they  had  no 
difficulties  navigating  the  forms.  They  became  adept  quickly  at  inputting  text  using  Palm’s  special  system. 
Graffiti.  But  most  importantly,  we  came  away  with  new  ideas  to  improve  the  system.  In  their  next  trip  to  the 
aquarium,  students  could  refer  to  on-line  guidance  while  using  the  Palm.  If  they  did  not  understand  a term,  they 
could  click  next  to  it  to  view  a glossary.  If  they  did  not  know  where  to  find  what  they  were  looking  for,  they 
could  click  another  button  to  pull  up  a “field  guide.”  In  this  way,  handheld  technology  can  bring  the  aquarium 
to  the  classroom  and  the  classroom  to  the  aquarium. 


Second  Trip:  Creeks 

Our  second  major  run  was  using  the  Healthy  Creeks  project,  mentioned  earlier.  In  this  project, 
students  visit  their  local  creek  and  use  chemical  tests  to  take  real  measurements.  They  collect  data  from 
several  sites  and  then  upload  them  all  back  at  the  classroom. 

These  students  were  from  Foothill  Middle  School.  Even  these  & grade  students  had  little  or  no 
difficulty  navigating  the  forms  and  entering  data.  Furthermore,  they  were  visibly  excited  to  learn  that  they 
would  be  using  the  handheld  computers.  “Do  we  get  to  use  those?!”  We  discovered  that  the  cool  factor  of  the 
technology  motivates  many  students  like  nothing  else  could. 


Future 


Our  future  research  is  to  further  explore  the  pedagogical  value  of  the  technology.  One  experiment  in 
development  is  to  study  several  classrooms  of  students  at  the  Monterey  Bay  Aquarium,  only  some  of  which 
will  use  the  handheld  computers.  To  evaluate  the  impact  of  the  technology,  we  will  observe  how  the  activities 
at  the  museum  and  the  take-home  learning  differ  between  the  groups. 

We  also  plan  to  bring  a handheld  component  to  our  longstanding  Genetically  Modified  Foods  Debate 
project.  Currently,  eighth-grade  students  engage  in  a two -week  inquiry  based  curriculum  on  genetically 
modified  foods.  Students  learn  to  weigh  the  tradeoffs  involved  in  using  one  method  of  agriculture  or  another, 
as  well  as  to  support  their  position  with  appropriate  evidence.  Students  are  given  many  opportunities  to  express 
their  ideas,  and  learn  from  each  other  while  debating  the  controversy  over  whether  GM  foods  are  safe  for 
human  consumption.  With  handheld  forms,  students  will  be  able  to  extend  their  own  investigations  beyond  the 
classrooms  and  into  their  social  networks.  Back  in  the  classroom,  they  can  visualize  the  data  and  compare 
based  on  criteria  they  choose.  Additionally,  they  will  be  able  to  incorporate  and  compare  responses  from  their 
teacher’s  other  classes,  or  from  classes  in  another  district,  state,  or  even  country. 

We  also  seek  to  understand  the  effects  of  bringing  learning  into  a social  context.  In  the  process  of 
surveying,  we  expect  students  to  learn  to  evaluate  information  sources,  to  synthesize  from  observations,  to 
interview  effectively,  and  to  reflect  on  their  role.  We  believe  that  they  can  develop  a coherent  understanding  by 
first  asking  people  what  they  know,  discussing  the  issues  with  them,  and  then  researching  the  relevant  facts  and 
concepts  independently  using  WISE.  We  want  to  know  how  the  use  of  handheld  technology  can  improve 
students’  general  inquiry  skills. 

From  the  technical  side,  we  plan  to  research  the  data  representation  component  itself.  We  are 
developing  many  different  visualizations  and  comparison  schemata  for  the  data  and  in  the  process  of 
evaluating  their  relative  effectiveness.  And  more  specifically,  what  representations  are  most  effective  for  what 
pedagogical  goals.  The  challenge  is  to  develop  software  that  is  1)  flexible,  2)  intuitive,  and  3)  works  well  on 
the  computers  found  in  most  classrooms.  Even  further  down  the  road,  we  plan  to  streamline  the  technology  and 
make  it  even  easier  to  administer.  We  envision  a turn-key  solution  that  a school  can  place  in  the  computer  lab. 
This  box  would  track  and  mange  each  handheld  device  and  help  the  teachers  focus  on  teaching. 


Conclusion 

The  WISE  learning  environment  and  handheld  computing  are  synergistic.  The  handheld  provides  the 
mobility  necessary  to  collect  real-world  data  and  observations  and  to  bring  dynamic  content  outside  the 
classroom.  WISE  provides  the  scaffolding  to  give  these  data  pedagogical  value.  By  integrating  them,  we  have 
created  the  exciting  education  opportunities  described  and  certainly  even  more  opportunities  that  we  have  not 
yet  imagined. 
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MusicImprovAnEdT  echT  ool 


Douglas  B.  Allen 
Dwight  W.  Allen 
Carl  Hoagland 
Gwendolyn  Watson 

Musical  Improvisation:  A Model  For  Infusing  New  Technologies  Into  The  Design  Of  Instruction 
Panel  Position  (shared  by  the  four  panelists'): 

In  an  age  of  rapid  change  and  almost  unlimited  opportunity  for  continuous  technological 
innovation  in  schools,  traditional  communication  and  decision-making  systems  characterized  by 
structure  and  linear  process  must  be  made  more  responsive.  Lessons  from  jazz  and  other  forms 
of  musical  improvisation  can  help  schools  develop  new  approaches  to  discussion,  decision- 
making and  experimentation  which  will  allow  schools  to  more  fully  engage  technology  and  new 
educational  practices,  thus  better  meeting  the  often  complex  expectations  of  a diverse  range  of 
stakeholders. 

Each  panelist  brings  a unique  perspective  to  this  topic: 

Dwight  Allen,  is  Eminent  Professor  of  Education  Reform  at  Old  Dominion  University  and 
former  Dean  of  the  University  of  Massachusetts  School  of  Education.  A recognized  thought 
leader  in  the  field  of  education  innovation  for  more  than  four  decades,  his  latest  book  on 
education  reform,  co-authored  with  Bill  Cosby,  was  published  as  one  of  Time-Wamer’s  first  e- 
books.  The  book  was  a finalist  in  the  Frankfurt  Global  E-book  Awards,  sponsored  by  the 
International  E-Book  Award  Foundation  this  past  Fall. 

Carl  Hoagland,  holds  the  Emerson  Electric  Endowed  Professor  of  Technology  and  Learning  at 
the  University  of  Missouri,  St  Louis.  He  is  also  the  director  of  the  E.  Desmond  Lee  Technology 
and  Learning  Center.  With  over  35  years  of  experience  in  all  levels  of  education  as  teacher, 
consultant  and  professor,  he  specializes  in  the  application  of  new  technologies  as  learning 
support  tools. 

Douglas  Allen  is  Associate  Professor  of  Management  at  the  University  of  Denver.  Former 
MBA  Course  Coordinator  for  the  Daniels  College  of  Business  High  Performance  Management 
course,  he  has  participated  in  a wide  range  of  educational  innovations  at  the  University  of 
Denver.  A frequent  consultant  to  Fortune  Global  500  companies  in  the  US  and  overseas,  he  has 
written  and  conducted  research  on  a variety  of  topics  related  to  change  in  domestic  and  global 
companies. 

Gwendolyn  Watson  is  concert  cellist,  composer,  improviser  and  teacher.  Described  in  a New 
York  Times  review  as  a “zany  but  serious  musician,”  she  has  performed  in  some  of  the  great 
concert  venues  of  the  world  and  continues  a life-long  ambition  to  extend  musical  appreciation  to 
the  general  public. 
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More  complete  bios  are  attached  at  back  of  proposal. 


Description  of  the  Panel  Topic 

In  the  current  age  of  technological  frenzy  with  its  promises  of  new  potential,  enhanced  options, 
vastly  increased  choices  (and  occasiona  1 cacophony),  educational  practitioners  are  bombarded 
with  so  many  opportunities  for  innovation,  that  it  is  very  difficult  for  them  to  know  how  to 
respond.  Adding  to  the  confusion,  equally  impressive  developments  in  both  cognitive  and 
neurological  learning  sciences  suggest  that  the  very  processes  of  education  can  be  restructured  to 
improve  effectiveness.  Further  complicating  this  instructional  melee  is  increased  public  and 
policy-maker  demand  for  accountability. 

In  this  turbulent  environment,  traditional  organizing  and  decision-making  systems  are  being 
taxed  to  their  limit.  Perhaps  we  are  in  need  of  some  new  organizing  principles  - a new  approach 
to  sorting  out  the  complicated  opportunities  and  demands  imposed  upon  educators  by  a diverse 
range  of  stakeholders. 

What  we  will  demonstrate 


Dwight  Allen,  Eminent  Professor  of  Education  Reformat  Old  Dominion  University,  and  Carl 
Hoagland,  Emerson  Electric  Endowed  Professor  of  Technology  and  Learning  will  join 
participants  in  an  exploration  of  the  power  of  musical  improvisation  as  a metaphor  for  decision- 
making and  experimentation  with  the  infusion  of  technology  into  educational  processes. 
Specifically  the  use  of  improvisation  will  be  examined  as  a tool  from  the  perspectives  and  for  the 
use  of:  educators,  students,  parents,  technologists,  as  well  as  software  and  materials  developers. 

The  panel  will  identify  four  roles  in  improvisation:  to  lead,  to  follow,  to  reflect  (wait  out),  and  to 
redirect.  In  this  session,  we  argue  that  members  of  an  education  system  (whether  elementary, 
high  school  or  university)  can  become  individually  and  collectively  more  effective  by  building 
capacity  in  each  of  these  four  skill  areas.  Through  impromptu  musical  participation  (utilizing  a 
variety  of  hand-held  musical  instruments  from  around  the  world)  by  the  audience  - led  by  world- 
famous  cellist  Gwendolyn  Watson  - the  session  will  explore  how  technology  can  shape  and  be 
shaped  by  each  of  these  roles. 

Referencing  the  work  of  Kao  (1996)  and  others,  Douglas  Allen,  associate  professor  of 
management  at  the  University  of  Denver,  will  show  how  traditionally  emphasized  economies  of 
scale  in  organizations  including  schools  may  need  to  be  supplemented  with  an  emphasis  on 
economies  of  discovery.  The  theory  of  scale  economies  has  emphasized  efficiency  through 
standardization-  often  achieved  through  high  degrees  of  linear  structure.  While  these  are 
certainly  still  important,  they  are  no  longer  sufficient.  To  fully  prepare  students  to  be  effective 
participants  in  the  new,  information  and  technology  rich  environment,  an  emphasis  on  creativity, 
innovation  and  spontaneity  is  required  as  well.  By  adopting  improvisational  behaviors,  schools 
may  become  more  open  to  the  approaches  that  can  effectively  prepare  students  for  life  in  an 
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information-rich  society  and  model  the  very  capacity  for  change  and  opermess  that  our  students 
must  learn. 

Throughout  the  session,  we  will  demonstrate  how  the  principles  of  jazz  and  other  forms  of 
musical  improvisation  help  us  find  “the  sweet  spot”  between  precise  scripting  of  learning  and  the 
chaos  of  random  response,  and  what  this  means  for  the  adoption,  development  and  use  of 
technology  in  schools. 

Context 


We  will  faciliate  our  session  as  an  emergent  mix  of  content-oriented  discussion  and  experiential 
exercise  involving  session  participants  in  musical  improvisation—  taking  time  as  we  go,  to 
discuss  the  takeaways  of  each  improvisational  experience.  The  exercises  will  provide  a 
metaphor  for  the  discourse  that  could  effectively  take  place  in  schools,  helping  them  to  become 
more  open  to  technological  change.  Through  the  experience  offered  in  this  panel,  session 
participants  will  discuss  and  explore  first-hand,  how  having  a multitude  of  resources  a\uilable  - 
sometimes  without  coordination,  but  with  the  awareness  of  the  availability  and  utility  of  each  - 
can  enrich  the  instructional  environment  by  facilitating  and  encouraging  impromptu 
experimentation,  trial  and  adoption  of  new  technologies  and  the  new  pedagogical  approaches 
they  may  allow. 
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Abstract 

WQ  is  a Web-based  system  which  reflects  some  of  the  ideas  found  in  WebQuests.  We  intend 
to  analyze  the  characteristics  that  make  the  WebQuests  so  popular  and  determine  which  of  their 
components  give  them  the  greatest  educational  value.  The  WQ  system  which  we  have  implemented 
presents  sites  to  be  browsed  and  searched,  it  allows  students  to  make  notes  on  those  sites,  and  it  lets 
the  students  manage  those  notes  to  respond  to  the  Quest.  Ultimately,  we  hope  that  the  WQ  system 
will  incorporate  collaboration,  integrate  digital  libraries,  be  scalable,  and  support  a wide  variety  of 
content  areas. 


1 WebQuests 

WebQuests  are  Web-based  learning  activities  that  have  become  very  popular  with  both  teachers  and 
students  [6].  Students  are  given  questions  to  investigate  and  Web-based  resources  to  use  in  answering 
them.  The  students  produce  answers  to  the  questions  that  include  references  to  the  resources.  WebQuests 
are  closely  related  to  other  systems  for  inquiry-based  learning  such  as  ThinkQuests  [18]  and  Internet 
Quests  [5j.  In  a larger  view,  they  are  related  to  e-books  [9]  and  digital  libraries. 

1.1  Variety  of  WebQuests 

WebQuests  are  not  a single,  simple  educational  approach  but  a collection  of  related  approaches.  The 
essence  is  that  students  synthesize  answers  to  questions  based  on  Web  resources.  These  resources  may 
be  either  public  Web  sites  or  sites  custom-designed  for  the  Quests.  Types  of  resources  may  vary  greatly 
depending  on  the  educational  goals  and  the  resource  familiarity  of  both  teachers  and  students.  Some 
WebQuests  also  require  cooperation  among  teams  of  students. 

1.2  Underlying  Skills 

WebQuests  are  a type  of  inquiry-based  learning  [2,  7,  10,  19] . WebQuests  combine  many  layers  of  skills. 
At  a low  level,  the  student  must  pick  out  material  that  is  relevant  to  the  question.  This  is  similar  to 
the  challenge  of  reading  comprehension  [13,  14] . There  is  also  the  challenge  of  navigating  the  text  on 
the  page  via  cognitive  organizers  [8|.  At  a still-higher  level,  the  student  must  engage  in  problem  solving 
to  determine  where  to  find  the  best  material.  There  are  aspects  of  “active  reading”  in  note  taking  (e.g., 
[12])  and  in  the  management  of  the  notes. 

1.3  Digitcd  Libraries  and  WebQuests 

A WebQuest  may  be  thought  of  as  a very  simple  digital  library  which  the  student  has  to  search.  We 
take  that  analogy  literally  and  attempt  to  combine  digital  library  technologies  (e.g.,  [3])  and  educational 
.strategies.  Digital  libraries  can  support  teachers  in  creating  a collection  of  works  for  the  students  quest. 
The  student  then  navigates  within  the  colection.  Various  different  strategies  can  be  applied  in  scoping 
the  collection,  such  as  appropriate  levels  of  scaffolding. 
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2 The  WQ  System 

While  current  WebQuests  generally  require  the  student  to  write  essays  either  by  hand  or  with  a word 
processor i it  is  useful  to  develop  a completely  integrated  framework.  In  this  case,  rather  than  writing 
essays,  we  simply  ask  the  students  to  make  notes  on  a selection  of  articles.  These  notes  can  then 
be  arranged  to  present  a response  to  the  question.  WQ  provides  a single  interface  for  the  associated 
activities  of  searching,  reading,  and  writing. 

The  main  control  for  the  WQ  system  is  via  a tree  widget  which  is  presented  in  the  left  frame.  This 
widget  lists  several  Web  Quests  and  for  each  Quest  it  presents  background  information  and  separate 
questions.  In  the  present  version,  the  basic  paradigm  for  student  interaction  is  creation  of  notes  on  the 
texts  provided.  The  collection  of  notes  taken  together  represents  the  student’s  response  to  the  questions. 
Later,  we  propose  models  for  extending  this  interactivity. 


0!  rba 
Q Help 

01  Dinosaur  Quest 
Information 

"0  What  was  distinctive  aboi 
r{^  Recommend  Web  Site 
Notes 

"C3  What  was  an  allosaurus? 
Health  quest  l\ 


What  was  distinctive  about  dinosaur  behavior? 


jAbout  the  Alosaunis 


[Dinosaur  efigs 


iDinosaur  brains 


Dinosaur  walking 


[where  to  find  dinosaurs 


Search 


Figure  1:  The  Quest  Manager  (left)  allows  selection  of  Quests  and  questions  within  those  Quests.  Recom- 
mended Web  pages  (right)  can  be  accessed  either  by  browsing  or  via  search. 


2.1  Selecting  Web  Pages 

The  right  side  of  Figure  1 shows  the  screen  for  presenting  the  list  of  web  pages  that  have  been  recom- 
mended by  the  teacher  for  a particular  WebQuest.  While  the  current  five  articles  are  small  enough  to 
be  browsed,  we  have  also  implemented  a search  feature  that  will  be  useful  for  larger  collections  which 
may  be  more  properly  thought  of  as  digital  library  collections.  The  search  utility  allows  teachers  to 
provide  a controlled  domain  of  material,  supporting  the  student  by  narrowing  the  field  of  resources  but 
still  providing  an  environment  of  exploration. 

2.2  Making  Notes 

The  students  interact  with  the  Web  sites  by  selecting  relevant  sentences.  Specifially,  the  students  click 
on  sentences  to  annotate  them  and  to  add  those  annotations  to  the  list  of  notes.  This  is  illustrated 
in  Figure  2 which  shows  the  highlighted  text  and  a box  for  adding  the  note  title  and  body.  The  main 
paradigm  we  wish  to  explore  with  this  first  version  is  linking  into  the  text  of  documents.  However,  the 
current  implementation  allows  access  to  any  Web  page  in  addition  to  formatted  documents.  For  these 
Web  pages,  the  notes  can  be  posted  only  at  the  top  of  the  page.  In  the  future  it  may  be  possible  to 
attach  notes  to  particular  passages  as  with  the  formatted  docs. 

2.3  Notes  Manager 

All  of  the  student  notes  are  collected  into  a single  list  as  shown  in  Figure  3.  The  student  can  arrange 
the  order  of  the  notes  to  respond  to  the  questions.  In  particular,  notes  are  managed  by  selection  of 
the  note  with  a radio  button  and  the  selection  can  then  be  moved  up/down  or  right/left  in  an  indented 
hierarchic  list.  Integration  of  the  notes  manager  with  the  browsing  utility  allows  the  student  to  respond 
to  questions  in  the  quest  with  out  leaving  the  browsing  environment. 


73 


JEST  COPT  AVAILABLE 
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teeth  ' a formidable  killing  machine  and  a ferocious  hunt 


North  America.  ADosaurs  were  made  for  r 


rdatives  such  as  birds  and  crocodilians  to  find  similar  behaviours.  When  we  observe  most  birds,  both  bdiaviours  are  observed.  We  sc 
but  others  have  a minimum  amount  of  contact  with  their  own  species,  Crocodilians  are  generally  solitary  by  nature.  The  main  interacli 
mating  and  defending  territorial  boundaries.  Mother  aDigalors  are  caring  and  protective  of  their  young  only  while  they  are  small  Once 
alligators  win  happily  make  a meal  of  them.  Can  we  apply  the  observations  of  birds  and  crocodilians  to  dmosaur  behaviour?  Uke  their 
social  or  antisocial  depending  on  circumstances  of  age,  population  dynamics  or  availability  of  food. 


Figure  2:  The  student  can  create  a note  by  clicking  on  a sentence  from  the  text.  This  screen  shows  a frame 
with  control  buttons,  a text  with  a highlighted  (selected)  sentence,  and  a note-entry  box. 

2.4  Implementation 

This  implementation  uses  a frameset  which  includes  HTML,  Javascript,  and  Java.  In  order  to  be  able  to 
add  notes  by  clicking  the  text,  we  built  a Java  applet  for  the  presentation  of  the  text.  This  also  meant 
that  specialized  Web  pages  had  to  be  constructed.  This  is  in  the  spirit  of  WebQuests  that  have  tailored 
Web  pages.  The  text  of  five  Web  pages  from  [4]  was  selected  and  reformatted  for  presentation  via  the 
Java  applet.  A future  implementation  may  be  adapted  to  automate  reformatting  either  in  batches  or  on 
the  fly.  The  login  control,  recommended  site  listings,  and  notes  are  all  managed  with  Perl  scripts  which 
connect  to  a Postgres  database. 

3 Digital  Library  Resources  for  WebQuests 

WebQuests  depend  on  high-quality  content  for  the  students  to  access.  This  information  resource  may 
range  from  a single  Web  page  to  a large  collection  of  Web  resources.  We  treat  the  Web  resources 
as  a scalable  digital  library.  We  have  recently  completed  work  on  a digital  library  for  The  Maryland 
Electronic  Learning  Community  (MELC),  a part  of  the  Baltimore  Learning  Community  (BLC),  has 
focused  on  the  creation,  maintenance,  and  cultivation  of  a digital  library  for  teachers  in  five  middle 
schools  in  the  Baltimore  area  [16]. 

OAI,  the  Open  Archives  Initiative,  has  developed  an  open  protocol  supporting  free  sharing  of  meta- 
data among  diverse  digital  libraries.  Using  the  OAI  open  protocol  and  an  XML  interface  provides 
navigable  access  of  the  metadata  from  the  MELC  digital  library  as  well  as  other  digital  collections. 
The  OAI  protocol  [11]  was  initially  developed  within  the  e-print  community  with  very  task  specific  fea- 
ture sets.  Early  directions  evolved  to  open  the  standard  and  lower  the  barrier  to  entry  for  non-e-print 
collections.  Thise  opens  the  door  to  a great  many  potential  resources  for  Web  Quests.  With  further 
development,  WQ  could  be  expanded  to  take  advantage  of  any  OAI  compliant  resource.  Here  we  see  a 
different  balance  being  struck  between  specificity  and  generality.  Building  on  that  foundation,  informa- 
tion providers  have  a great  deal  of  freedom  to  develop  specific  enhancements.  The  approach  has  great 
strength  in  that  it  offers  one  of  the  first  structured  protocols  for  metadata  sharing.  Our  system  uses 
XML  to  provide  a minimal  foundation  of  interoperability. 
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Figure  3:  The  entire  set  of  notes  for  a Quest  can  be  viewed.  In  addition,  the  order  and  nesting  of  notes  on 
the  list  can  be  be  controlled. 


4 Future  Directions 

We  will  extend  the  initial  work  to  incorporate  additional  features  including  (1)  access  to  video  and  other 
multimedia  resources,  (2)  support  for  collaboration,  and  (3)  the  ability  for  the  teacher  to  harvest  resources 
and  evaluate  student  work  in  progress.  We  will  also  systematically  examine  the  use  of  Web- based 
online  resources  in  inquiry  based-learning.  Ultimately,  students  could  have  flexible  resource  management 
desktops  (e.g.,  extending  the  ideas  in  [1])  and  the  underlying  model  could  allow  flexible  interaction  among 
the  components. 


4.1  Multimedia  and  Collaboration 

We  will  incorporate  videos  as  information  resources,  add  the  ability  to  jump  to  specific  frames  of  the 
video,  and  we  will  develop  simple  summaries  and  indexes  for  those  videos.  The  WebQuests  of  students 
with  different  multimedia  indexing  and  access  tools  will  be  contrasted.  Some  existing  WebQuests  include 
multimedia,  but  that  multimedia  is  not  seamlessly  integrated  with  the  Quest.  Because  XML  is  rapidly 
being  extended  to  encompass  multimedia,  our  XML-based  framework  should  be  able  to  leverage  that 
work.  Similarly,  simulation  may  be  added. 

Collaboration  often  makes  learning  more  engaging  [17].  Our  existing  annotation  facility  can  be 
extended  for  use  by  several  students  at  one  time.  We  will  extend  the  current  model  by  adding  features 
to  give  each  student  the  ability  to  have  distinct  annotations  and  resources  while  also  being  able  to  read 
and  comment  on  the  work  of  the  other  student(s). 


4.2  Teacher’s  Interface 

While  we  have  focused  on  the  student’s  environment  for  completing  WebQuests,  it  is  also  possible  to 
support  the  teacher  while  they  build  the  WebQuests  and  while  they  interact  with  the  students  who  are 
completing  the  WebQuests.  The  teacher’s  needs  in  many  ways  mirror  those  of  the  student  interfaces. 

The  teachers  in  the  Baltimore  Learning  Community  have  found  WebQuests  to  be  highly  effective. 
Building  a WebQuest,  as  well  as  many  lesson  plans,  requires  identification  of  useful  and  appropriate 
Web  materials.  Selection  criteria  for  inclusion  in  a lesson  plan  generally  hinge  on  readability.  Many 
teachers  report  that  the  challenge  is  to  find  the  right  digital  collections.  There  has  to  be  enough  text, 
at  the  appropriate  reading  level,  but  not  too  much  to  be  digested.  When  teachers  find  these  resources, 
it  would  be  helpful  if  they  could  manage  them  in  personalized  collections  with  annotation  capabilities. 

Moreover,  teachers  are  looking  not  just  for  evidence  in  support  of  lesson  objectives,  they  must 
simultaneously  screen  for  appropriate  presentation.  In  building  a WebQuest,  the  teacher  plays  the 
role  of  collection  builder  for  the  student.  A collection  that  covers  the  topic  being  studied  but  presents 
it  at  an  inappropriate  level  is  of  limited  use. 


The  materials  in  an  OAI  compliant  collection  can  be  a resource  of  great  value  to  teachers.  The  Open 
Archives  Initiative  also  provides  open  access  for  "harvesting”  collections,  but  the  protocol  alone  is  not 
enough  to  make  the  data  useful.  We  have  explored  metadata  sharing  via  the  OAI  in  light  of  teachers’ 
needs  in  the  BLC.  We  find  a need  for  a system  that  allows  users  to  integrate  collections  of  annotations 
of,  and  associations  between,  Web  resources.  For  instance,  tools  built  on  top  of  the  OAI  architecture 
could  allow  teachers  to  add  value  to  the  metadata  or  resources  they  find. 

We  find  in  usage  analysis  that  many  of  the  same  materials  within  a very  diverse  collection  are  reused, 
not  necessarily  because  of  their  content  but  because  of  "word  of  mouth”  [15]  and  the  barriers  to  easily 
finding  alternative  materials.  One  of  the  greatest  obstacles  teachers  face  is  the  ability  to  obtain  and 
interpret  organized  collection  data.  Teachers  find  an  overwhelming  amount  of  resources  on  the  Web. 
But  the  transition  from  collection  to  collection  can  be  a somewhat  nomadic  process.  Teachers  use  words 
like  ’’hunting”  and  ’’searching”  when  describing  the  process  of  obtaining  task-appropriate  materials  from 
multi- media  sources  and  digital  libraries. 

Finally,  teachers  could  also  monitor  the  progress  of  the  students  and,  eventually,  could  even  provide 
interactive  suggestions  to  the  students  while  they  worked,  using  the  WQ  interface.  We  envision  the 
computer  as  a support  for  the  teachers  to  track  the  progress  of  individual  students.  WQ  could  be 
extended  to  help  teachers  identify  which  students  need  the  most  help  with  a particular  Quest. 


5 Conclusion 

Although  Web  Quests  are  very  popular,  their  educational  value  is  not  well  understood.  We  have  devel- 
oped the  first  stage  in  a unified  environment  for  managing  WebQuests.  The  current  system  allows  the 
students  to  attach  notes  directly  to  texts.  The  collection  of  the  notes  allows  the  students  to  respond  to 
the  questions.  This  is  not  the  final  version  of  this  system;  we  are  close  to  completing  access  to  multimedia 
and  support  for  collaboration  in  note  taking.  We  anticipate  that  WQ  will  become  a successful  model  for 
an  integrated  WebQuest  interface. 
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Abstract:  In  order  to  reuse  and  exchange  learning  objects  we  need  information  about  these 
learning  objects.  The  LOM  draft  standard  defines  a set  of  more  than  70  attributes,  which 
specify  learning  object  properties  like  author,  title,  subject,  and  many  others.  Even  though  the 
LOM  draft  includes  a category  educational,  no  information  is  included  in  the  standard  to 
specify,  which  instructional  roles  are  played  by  a learning  object  in  a course.  We  show  how  to 
include  this  important  didactic  information  using  the  concept  of  instructional  roles  and 
relations  in  a way,  which  is  extensible  and  flexible  enough  to  specify  not  only  general  didactic 
criteria,  but  rather  specific  criteria,  as  prescribed  by  different  instructional  theories. 


To  allow  the  reuse  of  learning  objects  (LO),  various  standards  have  been  developed  to  describe  learning  objects, 
their  relationships,  etc.  The  IEEE  LOM  Draft  Standard  for  Learning  Object  Metadata  (LOM2001)  specifies  a 
variety  of  bibliographic  and  technical  properties  of  LOs,  as  well  as  different  relationships  between  LOs,  and 
make  exchange,  reuse,  and  search  of  LOs  based  on  these  metadata  possible.  Even  though  the  LOM  draft  includes 
a category  educational,  no  information  is  included  in  the  standard  to  specify,  which  instructional  roles  are  or  can 
be  played  by  a LO  within  a course.  As  curriculum  programs  do,  LOM  concentrates  on  what  should  be  taught  and 
when,  rather  than  how  to  teach.  Obviously,  a standard  for  learning  objects  metadata  should  not  tell  how  to  teach, 
but  it  should  definitively  be  able  to  provide  information  on  how  to  specify  pedagogical  aspects  of  LOs.  This 
difficulty  is  caused  by  the  fact,  that  LOM  attributes  specify  properties  only  at  a very  basic  abstraction  level. 
LOM  specifies  annotations  for  content  and  activities  but  does  not  support  metadata  about  instructional  models 
and  theory,  even  though  authors  are  implicitly  or  explicitly  using  specific  instructional  theories.  LOM  does  not 
support  information  about  the  use  of  LOs  in  learning  processes,  which  are  a central  concern  in  instructional 
design.  The  current  paper  introduces  additional  abstraction  layers  to  the  LOM  specification  which  explicitly 
takes  different  instructional  theories  into  account. 

The  Current  LOM  Model:  The  Learning  Object  Metadata  Standard  (LOM  for  short)  defines  a structure  for 
interoperable  descriptions  of  LOs.  It  aims  at  facilitating  search,  management  and  (re)use  of  LOs  by  authors  of 
online-courses,  teachers  and  learners.  A LO  is  defined  in  the  LOM  specification  as  “a/iy  entity,  digital  or  non- 
digital, that  may  be  used  for  learning,  education  or  training''  (LOM  2001).  The  LOM  basic  schema  consists  of 
nine  categories  and  simply  of  two  types/classes:  the  LOM  resource  and  the  LOM  type,  linked  by  LOM  attributes 
(LOM  data  elements).  In  the  model  layer  we  have  only  the  LO  itself,  the  attributes  describing  the  LO  and  the 
datatypes  for  those  attributes.  The  descriptions  of  LOM  are  context -independent  and  static  classifications.  This  is 
not  appropriate  for  many  instructional  aspects:  To  characterize  “collaborative  learning”  the  type  resp.  the 
vocabulary  “collaborative”  can  hardly  be  added  to  a single  category,  as  “collaborative  learning”  is  an 
instructional  principle  which  affects  and  shifts  the  entire  environment:  the  role  of  teacher  and  learner  (Intended 
End  User  Role  (LOM  5.5)),  activities,  interactivity  type  (LOM  5.1),  Typical  Learning  Time  (LOM5.9),  purpose, 
organizational  framework,  and  many  more.  The  current  LOM  model  does  neither  provide  concepts  for  modelling 
instructional  models,  instructional  principles,  nor  to  specify  epistemological  approaches. 

Basic  Instructional  Framework 

Pedagogical  Dimensions  — Abstraction  Layers'.  We  start  by  defining  what  we  consider  as  pedagogical 
dimensions  in  this  context.  We  choose  a top- down-model  in  which  pedagogical  dimensions  are  embedded  in 
different  layers  of  abstraction  - according  to  educational  and  cognitive  sciences.  In  this  model,  LOM  only 
addresses  the  bottom  layer  which  is  the  most  basic.  4^  (highest)  layer  of  abstraction:  epistemology,  paradigm: 
The  highest  level  of  abstraction  addresses  (whether  implicitly  or  explicitly)  broad  orientation  concerning 
epistemology  resp.  theory  of  cognition,  layer  of  abstraction:  (instructional)  principles:  Merrill  refers  to  this 
layer  as  “set  of  underlying  principles”.  In  literature  as  well  as  in  practice  we  often  find  fixed  terms  comprising 
some  well- agreed  principles:  Problem  Based  Learning  (PBL);  Case  Based  Learning  e.g.  2”^  layer  of  abstraction: 
instructional  models,  learning  theories  as  well  as  communication  theories  and  sets  of  strategies.  Models  often 
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structure  and  organize  learning  processes  in  several  phases,  (basic)  layer  of  abstraction:  content,  practices, 
activities,  sets  of  activities  and  scenarios.  Curriculum  programs  assembling  content  are  located  at  this  layer.  This 
layer  tells  which  resources  are  actually  used.  The  current  LOM  Metadata  Schema  only  addresses  this  layer. 

The  top -down-model  presented  in  this  paper  is  derived  from  German  tradition  of  education.  But  there  is 
obviously  an  urgent  need  for  implementing  a model  in  LOM  that  reflects  both,  American  as  well  as  European 
thought,  and  hopefully  also  other  tradition  in  the  field  of  instructional  design.  Analogies  between  these  different 
traditions  can  be  stated:  Merrills  approach  of  abstraction  layers  in  instructional  design  (Merrill  2001)  seems  to  be 
comparable  to  our  perspective  which  is  presented  by  German  authors  (Kosel  1993;  Klafki  1993).  A decision 
made  on  a higher  layer  of  abstraction  shifts  the  entire  setting  at  the  level  of  the  basic  layer:  the  ‘Typical  Learning 
Time’  spend  with  one  and  the  same  LO  may  be  different  in  a PBL  scenario  from  an  expository  scenario  e.g..  In 
order  to  solve  the  need  for  dynamic  classification,  we  suggest  to  include  the  concept  of  roles  in  the  meta-model. 
Instructional  Roles  - Instructional  Aptitude:  Using  LOM  metadata  we  cannot  specify  the  instructional  aptitude 
of  a LO.  Is  a LO  suitable  to  be  used  in  a scenario  of  collaborative  learning  or  in  a scenario  enabling  problem 
solving?  Learning  Sequences  - Learning  Processes:  In  separating  content  from  structure,  LOs  are 
decontextualized.  In  our  meta-model,  LOs  are  integrated  into  different  learning  cycles  supporting  processes 
which  are  derived  from  different  instructional  models.  In  the  role-concept  of  our  meta-model  learning  theories 
and  instructional  models  represent  context.  Merrill  stresses  the  importance  of  phases  in  learning  cycles  (Merrill 
2001).  One  and  the  same  LO  (types)  may  fill  different  instructional  roles  within  learning  processes  set  up  by 
different  learning  theories.  Our  concept  of  roles  stringently  and  clearly  distinguishes  the  natural  types  of  LOs 
(media  type,  tools  e.g.)  from  their  instructional  role/purpose  puarino  1992;  Steimann  2000).  The  top-down- 
model  can  be  mapped  to  the  concept  of  types  (class)  and  roles  (figure  1). 


Roles  for  Modelling 


There  is  a wide  choice  and  diversity  of  definitions  of  the  role  concept  in  literature  (in  semantics  e.g.).  The 
concept  of  Roles  in  Modelling  has  been  elaborated  in  (Steimann  2000a).  Role  expectations  include  both  actions 
and  qualities.  Steimann  distinguishes  definition  of  characeri sties  from  the  player  itself:  „(...)  „a  role  is  a kind  of 
protocol  specificification  specifying  behaviour  and  characteristics,  but  not  the  role  player  itself*  (Steimann 
2000b).  As  opposed  to  natural  types  or  classes,  roles  have  strong  dynamic  aspects.  Types/classes,  which  are  the 
fundamental  concepts  in  the  object  oriented  modelling  are  inherently  static:  an  instance  of  a class  once  and 
forever  belongs  to  that  class  - it  cannot  change  it  without  loosing  its  identity.  A LO  may  fill  different 
insructional  roles.  Learning  Models  are  used  as  context  of  roles  and  determine  relationships  between  entities. 


Meta- Model 
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Figure  1:  The  extended  LOM  Meta-Model  (The  model  distinguishes  roles  from  types/classes) 


The  instructional  role  is  compatible  with  the  idea  of  the  role  concept:  a type  must  have  certain  characterising 
predicates  (Guarino  1994),  qualities,  attribute,  or  requirements  in  order  to  be  able  to  fill  a certain  role. 
Characterising  predicates,  attributes  and  requirements  are  matchable  with  the  concept  of  instructional  aptitude.. 
We  suggest  to  name  instructional  aptitutes  ‘instructional  qualities’.  The  important  advantage  of  the  approach  of 
instructional  roles  is  its  ability  to  deal  with  dynamic  modelling  and  instantiation,  in  contrast  to  a standard  class- 
oriented  approach  which  is  suitable  for  the  static  attributes  currently  included  in  LOM. 
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Abstract:  Dual  Coding  Theory  has  quite  specific  predictions  about  how  information  in  different  media  is 
stored,  manipulated  and  recalled.  Different  combinations  of  media  are  expected  to  have  significant  effects 
upon  the  recall  and  retention  of  information.  This  obviously  may  have  important  consequences  in  the  design 
of  computer-based  programmes.  The  paper  describes  an  experimental  approach  which  has  been  developed 
using  the  Statistical  domain  in  which  the  presentation  media  have  been  varied  (Text  only,  Text  and  Diagrams 
and  Diagrams  with  Voice-over).  The  results  are  compared  with  Dual  Coding  theory  predictions  and  the 
effects  of  Student  Learning  Style  explored. 


1.  An  Early  Attempt  at  Using  the  Web  to  Communicate  Statistics  Knowledge 

Our  interest  in  this  work  arose  out  of  a desire  to  measure  the  effectiveness  of  Distance  Learning  compared  with 
Face-to-face  presentations.  An  opportunity  arose  in  a second  year  HCl  module  which  included  two  lectures  on 
the  use  of  the  Null  Hypothesis  and  Statistical  Inference  in  HCl  evaluation.  We  decided  to  divide  the  class  into 
two  groups,  one  of  which  would  be  taught  face-to-face  and  the  others  over  the  Web.  Presentation  was  mainly 
textual  but  some  animations  created  using  Visual  Basic  were  also  included.  Students  were  first  given  a test  to 
ascertain  their  knowledge  of  the  material  to  be  presented.  There  were  some  students  with  detailed  knowledge, 
but  most  knew  little  about  the  domain.  Throughout  both  presentations,  questions  were  asked  to  enable  students 
to  chedc  if  they  had  understood  the  material.  The  Face-to-face  class  was  taught  the  material  on  the  last  day  of 
term,  and  the  Distance  Learners  were  given  access  to  the  material  once  they  arrived  home  in  the  vacation. 

Access  was  protected  by  password  and,  because  the  student  homes  were  widely  dispersed  it  is  unlikely  that  any 
of  the  “lecture”  students  could  gain  access  to  the  material.  On  their  return  to  the  University  in  the  next  semester, 
the  students  were  tested  for  knowledge  of  the  material  through  a series  of  questions.  We  marked  the  test  with  a 
simple  scoring  system  - one  point  for  a correct  answer. 

The  results,  as  one  might  expect,  were  inconclusive.  Overall,  the  performance  of  subjects  on  the  post-test  was 
similar  for  both  groups.  It  was  interesting  that  the  Distance  Learners  did  no  worse  overall  that  the  Face-to-face 
Learners,  though  a number  of  them  did  report  that  they  found  working  at  home,  alone,  not  very  motivating. 

Scores  varied  widely  (from  13  to  60).  It  was  clear  that  some  students  in  both  groups  had  not  taken  the  exercise 
seriously.  It  was  also  clear  that  previous  knowledge  had  an  effect  (though  students  who  professed  previous 
knowledge  did  not  always  do  well).  Previous  Mathematics  expertise  also  had  a positive  effect  on  the  score  as 
might  be  expected.  We  had  asked  the  Distance  Learners  to  keep  a log  of  their  access  to  the  system.  Most  did  this 
and  total  access  times  reported  varied  from  1 hour  to  6 hours.  Some  students  also  made  comments  on  the  way  in 
which  the  material  was  presented.  The  material  was  heavily  text  -based  and  most  comments  suggested  that  more 
“interesting”  media  should  have  been  used  (whatever  that  meant). 

Although  this  was  only  an  exercise  partly  to  illustrate  to  the  students  the  problems  involved  in  such 
experimentation,  it  did  raise  some  major  design  issues  for  any  future  experiment: 

1.  By  how  far  do  different  media  affect  the  learning  process? 

2.  At  what  level  should  the  learning  material  for  the  experiment  be  pitched  and  how  extensive  should  it 
be? 

3.  How  can  the  students  be  m otivated  so  that  they  make  a serious  attempt  at  learning? 

4.  How  is  the  amount  of  material  learned  measured?  Clearly  correct  answers  to  questions  do  give  an 
indication,  but  how  should  the  questions  be  phrased? 

5.  How  is  previous  knowledge  allowed  for?  Aremarks  only  given  for  improvements  in  knowledge 
transfer?  Does  someone  who  knows  it  all  before  get  zero! 
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6.  Should  we  be  considering  different  learning  styles? 

2,  Effect  of  Different  Media  on  Learning  and  Performance 

2.1  Some  Process  Control  Results 

We  have  previously  studied  the  effects  of  using  different  media  on  the  performance  of  operators  carrying  out  a 
laboratory  task.  The  task  chosen  was  that  of  Crossman’s  Water  Bath  [Crossman  & Cooke,  1974].  The  task  is 
closely  related  to  the  process  domain  [Sanderson  et  Al.,  1989]. 

There  was  a marked  difference  in  performance  across  media.  For  example,  the  performance  results  differed 
when  using  Text  and  Graphics.  At  low  task  complexity  the  performance  was  very  similar,  but  as  complexity 
increased  the  differences  start  to  show.  In  the  more  complex  tasks  the  results  are  significant  (p<0.03)  with  , 
Graphical  interfaces  giving  the  best  performance.  After  each  sequence  of  tests,  the  understanding  of  subjects 
was  tested  with  a questionnaire.  The  comprehension  results  showed  significant  differences  in  the  performance 
between  the  use  of  Verbal  and  Spatial  Coding  in  the  interface. 

Interestingly,  spatial  coding  improved  comprehension  of  the  two  least  understood  variables  and  this  was  a 
significant  result  (p<0.01).  There  is  not  space  to  report  all  the  findings  but  the  other  variables  showed  similar 
trends  (Alty,  1999). 

Thus  the  choice  of  medium  used  does  seem  to  have  an  effect  on  comprehension  and  this  could  be  important  in 
learning  situations. 

2.2  Dual  Coding  Theory 

Dual  Coding  Theory  (Paivio,  1986)  assumes  an  orthogonal  relationship  between  symbolic  systems  and  sensory 
motor  systems.  The  theory  suggests  that  there  are  clear  distinctions  between  the  internal  representations  of 
Symbolic  and  Sensory- motor  events.  The  stored  versions  of  visual,  verbal,  and  haptic  events  retain  the 
modalities  of  these  events.  For  example,  in  the  visual  modality  there  are  printed  words  and  images.  In  the 
auditory  modality  there  are  spoken  words  and  sound  events.  can  be  stored  as  an  imagp  of  a lion,  the  word 
“lion”,  or  both  but  within  distinct  systems.  Of  major  importance  is  the  verbal/nonverbal  distinction.  The  verbal 
and  non-verbal  processing  systems  are  assumed  to  be  functionally  independent  though  there  are  cross  linkages 
between  the  two.  If  the  Dual  Coding  theory  is  relevant  then  the  recall  of  material  will  be  affected  by  the  way  it  is 
presented. 

3.  Choice  of  Learning  Domain  and  Subject  Motivation 

Many  experiments  that  have  examined  the  effects  of  different  media  on  learning  have  been  constructed  over 
relatively  simple  subject  domains.  The  material  being  communicated  to  learners  is  often  limited  in  scope  and 
usually  not  complex  in  nature.  There  are,  of  course,  good  reasons  for  this.  A complex  domain  necessarily 
requires  a specialised  user  base,  and  an  extensive  set  of  learning  material  will  impose  serious  time  requirements 
on  the  subjects  taking  part.  Yet  it  is  important  to  really  challenge  subjects  both  with  domain  complexity  and  the 
extent  of  the  material  in  order  to  obtain  results  which  will  scale-up  for  real  situations. 

One  possible  solution  to  this  problem  is  to  choose  a domain  that  is  acknowledged  to  be  inherently  difficult,  is 
difficult  to  teach,  and  yet  for  many  people,  constitutes  a most  desirable  skill  to  attain.  Within  a University 
context  (the  population  from  we  from  which  we  are  most  likely  to  draw  subjects)  there  was  an  obvious 
candidate  - Statistics. 

Most  Masters  and  PhD.  students  require  statistical  knowledge  for  analysing  their  experiments,  and  yet  it  is  a 
subject  most  feared  by  many  of  them.  Thus  the  Statisitical  Domain  fulfils  our  requirements  of  being  complex,  of 
being  not  easy  to  teach,  and  yet  would  be  regarded  by  subjects  as  a required  and  desirable  skill.  There  should 
therefore  be  a strong  motivation  to  take  such  a course  and  to  take  it  seriously.  Furthermore,  at  Loughborough 
University,  we  have  a Masters  course  on  Multimedia  Interface  Design,  and  an  important  aspect  of  the  course  is 
the  design  and  evaluation  of  HCI  experiments  using  Statistics.  The  material  is  typically  taught  in  four  one-hour 
lectures  on  the  course  and  covers  basic  information  about  the  Null  Hypothesis,  the  Binomial  Distribution,  the 
Sign  Test,  the  Wilcoxon  Ranking  Non-parametric  Test  and  Normal  distributions  and  their  use.  We  therefore 
decided  to  develop  Web-based  material  to  teach  the  subject  matter  on  the  course  and  to  additionally  make  the 
exercise  an  example  of  HCI  experimentation  and  its  evaluation. 
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We  constructed  four  separate  computer-based  modules  in  FLASH  to  teach  this  basic  stat  istical  knowledge: 

1 . The  Null  Hypothesis  and  the  relevance  of  Statistics 

2.  The  Binomial  Distribution 

3.  Non^arametric  distributions  - i.e.  Ranking  (Wilcoxon) 

4.  The  Normal  Distribution  and  the  Central  Limit  Theorem 

The  presentations  were  constructed  using  three  different  media  combinations  of  voice,  text  and 
diagrams/pictures.  The  material  and  timing  was  identical  in  each  presentation.  In  other  words,  the  text  and  the 
voice-over  content  were  identical.  Each  of  these  media  could  be  disabled  so  that  a number  of  different 
presentation  formats  was  possible.  We  actually  used: 

1.  Text  only 

2.  Text  + Diagrams 

3.  Voice  + Diagrams 

These  formats  were  preserved  over  a complete  module  presentation  session. 

4.  Proposed  Experimental  Approach 

Each  developed  presentation  had  an  elapse  time  of  about  10-15  minutes  and  each  was  presented  in  a serial  non- 
interactive  fashion.  We  planned  to  place  subjects  in  a room  and  present  the  material  on  a screen  using  a 
computer  projector.  At  the  commencement  of  the  first  session  all  subjects  were  to  be  given  a short  questionnaire 
to  determine  their  previous  knowledge  of  the  subject  and  would  be  told  to  write  nothing  down  during  the 
experiment,  but  that  at  the  conclusion  of  the  experiment  they  would  be  required  to  answer  a series  of  questions 
on  the  material.  If  students  were  subjects,  they  would  be  assured  that  the  results  did  not  form  any  part  of  the 
evaluation  process  for  their  module 

The  subjects  would  be  tested  for  immediate  recall  of  the  material  presented  at  the  conclusbn  of  the  presentation. 
The  post  experimental  test  would  consist  of  a series  of  questions  about  the  presented  material.  In  order  to  allow 
for  the  effect  of  previous  knowledge,  subjects  would  be  able  to  indicate  alongside  their  answer  whether  they 
already  knew  the  answer  before  the  session,  whether  the  session  had  reminded  them  of  the  answer,  or  whether 
the  material  in  the  question  was  completely  new  to  them. 

5.  The  Construction  of  the  Material 

Because  all  the  three  different  media  presentations  had  to  be  based  upon  identical  material,  the  construction  of 
the  different  sessions  was  an  interesting  exercise  in  itself.  When  constructing  the  Voice-over  + Diagrams 
presentation  we  had  to  be  mindful  of  the  fact  that  the  Voice-over  (as  text)  had  also  to  work  when  presented  as  a 
Text -only  session.  This  exercise  in  itself  provides  useful  insights  into  the  nature  of  multimedia  presentations, 
how  media  are  implicitly  allocated  for  communicating  particular  types  of  material  and  the  limitations  some 
media  have  for  communicating  particular  types  of  material.  For  example,  communicating  the  idea  that  the 
results  of  an  experiment  might  be  caused  by  a set  of  independent  random  events  is  quite  straightforward  in  text, 
but  is  not  easy  to  do  in  a diagram.  On  the  other  hand  some  material  (describing  the  shape  of  a normal 
distribution)  is  easy  with  a diagram  but  laborious  using  text.  The  Null  Hypothesis  concept  is  relatively 
straightforward  to  communicate  using  text  but  more  difficult  using  a diagram.  All  the  material  was  constructed 
in  FLASH  and  each  module  was  divided  into  a number  of  FLASH  scenes.  The  scenes  were  transparent  to  the 
subjects,  but  we  used  this  approach  so  that  later  the  material  could  be  adapted  so  that  it  could  be  presented  in  a 
more  parallel,  interactive  manner  at  a future  date. 

6.  An  Initial  Experiment 

The  first  module  was  actually  constructed  in  Australia  whilst  the  author  was  a Visiting  Research  Fellow  at 
Melbourne  University  Department  of  Information  Systems.  Towards  the  end  of  the  Fellowship  the  author  gave  a 
commercial  course  on  Multimedia  Interface  Design  to  a class  of  about  50  commercial,  industrial  and  academic 
participants,  which  lasted  two  days.  It  was  decided  that  this  would  be  a good  opportunity  to  run  a pilot  study  of 
the  first  module  during  the  course  to  give  the  participants  an  insight  into  evaluation  techniques  and  to 
demonstrate  the  effects  of  different  media  on  learning.  From  the  point  of  view  of  the  experiment,  it  would 
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provide  an  exercise  in  creating  the  material,  a test  of  the  suitability  of  t he  material  and  provide  experience  in 
setting  the  post-experiment  test.  The  participants  (most  of  whom  in  the  pre-test  claimed  not  to  have  significant 
knowledge  in  the  Statistics  area)  were  divided  into  three  groups  and  presented  with  the  three  different 
presentation  styles  - Text -only,  Text  + Diagrams  and  Voice-over  + Diagrams. 

All  presentations  were  given  in  separate  room  areas  using  a computer  projector.  At  the  end  of  the  test  all 
subjects  answered  the  same  test  to  check  recall  of  material.  No  learning  style  classification  of  users  was 
attempted  in  this  pilot  run.  The  test  administered  at  the  conclusion  of  the  experiment  consisted  of  a series  of 
questions,  some  of  which  were  multiple- choice  questions. 


A: Knew  it  B:  Previously  C:  Never  met  it 


Figure  1:  Raw  results  from  the  Australian  Experiment  on  Recall. 


The  results  are  shown  in  (Fig  1).  The  three  vertical  columns  refer  to  whether  the  subject  had  met  the  domain  or 
concepts  before  (A:  Knew  it,  B;  Previously  learned  and  forgotten,  but  reminded^  CiDid  not  know  it).  One  can 
clearly  see  the  effect  of  previous  knowledge.  The  marks  decrease  as  subject  knowledge  decreases.  This 
experiment  was  only  carried  out  as  a pilot,  to  guide  future  experimental  design  (for  example,  we  did  not  expect 
to  carry  out  any  statistical  analysis  on  the  result)  the  results  seemed  initially  to  contradict  Dual  Coding  Theory 
predictions.  For  example,  the  Text  Only  group  (A)  had  a tendency  to  do  better  than  the  Voice-over-Diagrams 
group  (B),  whereas  we  might  have  expected  the  results  to  be  in  the  opposite  direction.  However,  a replay  of  the 
experimental  material  discovered  an  error.  We  had  met  with  some  trouble  in  starting  the  three  presentations  and, 
in  the  confusion,  had  actually  presented  Voice -Over+Text  (not  Voice  Over+Diagrams)  to  Group  (B).  Thus  in 
retrospect  it  was  not  surprising  that  the  Group  (B)  performed  badly.  Previous  experiments  have  shown  that  the 
redundancy  of  Text  and  Voice-over  can  worsen  performance.  Other  interesting  features  were  noted.  Subjects 
who  professed  to  know  the  material  often  still  made  recall  errors.  Finally  we  had  great  difficulty  in  analyzing  the 
questionnaires  in  giving  marks  for  recall  and  realized  that  the  questions  needed  to  be  more  carefully  designed 
and  related  more  carefully  to  the  material. 

7.  The  First  Student  Class  Experiment 

We  reorganized  the  material  so  that  each  scene  had  clear  learning  objectives  and  the  resulting  questions  were 
derived  from  these  objectives.  Some  questions  were  carefully  chosen  such  that  even  previous  knowledge  in 
itself  would  not  enable  a student  to  answerit.  We  also  decided  to  test  the  students  Learning  Style  using  the 
Felder  and  Soloman  Test  (Felder  1993).  The  test  identifies  preferences  on  the  axes  Active/Reflective, 
Sensing/Intuitive,  VisuaWerbal  and  Sequential/Global. 

We  asked  students  on  a first  year  HCI  course  to  take  the  learning  style  test.  The  spread  of  learning  styles 
indicated  that  there  were  similar  sized  groups  of  Sensing  Learners  and  Intuitive  Learners  in  the  class,  and  the 
rest  of  the  students  had  Learning  Styles  balanced  between  the  two.  The  class  was  then  divided  into  three  groups 
each  group  having  an  equal  number  of  Sensing  Learners,  Intuitive  Learners  and  Balanced  subjects.  A further 
regrouping  improved  the  gender  balance  without  disturbing  the  Learning  Style  balance. 

The  first  module  (Null  Hypothesis)  was  then  presented  to  the  three  groups  of  students  -Text  -only.  Text  and 
Diagrams,  and  Diagrams  with  Voice  over.  The  results  of  this  exercise  are  reported  elsewhere  in  this  conference 
(Beacham,  Elliott,  Alty  and  AlSharrah,  2002).  The  media  combination  used  was  significant  with  respect  to 
recalled  material  and  it  was  shown  that  Learning  style  played  an  important  part  in  the  accuracy  of  recall. 
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8.  The  Extended  Student  Class  Experiment 


As  a result  of  the  success  of  the  first  experiment  we  completed  all  four  modules  and  tested  them  on  the  students 
attending  a postgraduate  Multimedia  Interface  Design  course  (59  students,  though  all  did  not  take  part  in  all 
sessions).  The  students  answered  a questionnaire  to  determine  their  learning  style.  The  test  administered  was 
again  that  of  Felder  and  Soloman.  They  were  then  again  divided  into  three  groups  (A,  B and  C)  according  to 
their  learning  style  SENSING  / INTUITIVE  / BALANCED. 

As  far  as  possible,  each  group  was  balanced  for  gender  and  learning  styles.  Groups  were  given  the  four  modules 
(in  the  different  presentation  formats)  as  detailed  below  in  (Tab.  1). 


Text 

Text  + Diagrams 

Voice  + Diagrams 

Null  Hypothesis 

Group  A 

Group  B 

Group  C 

Binomial  Distribution 

Group  B 

Group  C 

Group  D 

Ranking 

Group  C 

Group  A 

Group  B 

Normal  Distribution 

Group  A 

Group  B 

Group  C 

Table  1:  Groups  and  Presentation  Formats 


Since  learning  about  Statistics  was  part  of  the  course  as  well  as  the  experiment,  we  did  not  want  to  disadvantage 
any  students  who  might  do  less  well  on  a particular  presentation  technique.  We  also  did  not  want  succeeding 
modules  to  be  affected  by  any  lack  of  knowledge  gained  in  previous  modules  (though  generally  this  was  not  a 
problem  because  most  of  the  material  was  quite  different).  All  students  were  therefore  given  a normal  lecture 
on  the  material  of  a module  after  the  presentation.  They  were  able  to  interact  and  ask  questions  to  ensure  they 
fully  understood  the  material.  Any  students  who  did  not  complete  all  sessions  were  removed  from  the  sampb. 
This  left  37  students. 


Mean  learning  scores 


□ sound  and 
diagrams 

^text  and  diagrams 
■ text  only 


Figure  2:  Preliminary  Experimental  Results 

The  results  for  the  different  media  can  be  seen  in  (Fig.  2).  In  all  cases  the  predictions  of  Dual  Coding  theory  are 
borne  out 

Our  sample  included  some  dyslexic  learners.  Although  the  sample  was  too  small  to  be  significant  (6  subjects), 
the  results  did  suggest  that  such  learners  might  react  differently  see  (Fig.  3),  and  this  looks  like  an  interesting 
area  for  more  experimentation.  The  dyslexic  subjects  do  not  seem  to  follow  the  predictions  of  Dual  Coding 
Theory  (or  at  least  there  seems  to  be  more  complex  interactions  taking  place).  The  results  also  indicated  that  the 
material  based  upon  real  world  objects  were  recalled  more  accurately  than  more  abstract  visual  material. 

Certain  learning  styles  were  influenced  more  (or  less)  by  different  media  (Fig.  4).  For  example.  Intuitive 
Learners  tended  to  out -perform  Sensing  Learners  in  all  presentations,  although  in  certain  experiments,  Sensing 
Learners  using  Voice/Diagrams  outperformed  Intuitive  Learners  using  Text -only. 


Expt  1 Expt  2 Expt  3 Expt  4 Overall 
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Mean  learning  scores  for  dyslexics  and  non-dyslexics 
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Figure  3:  The  Preliminary  results  for  Dyslexia 


Mean  learning  scores  for  learning  styles 
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Figure  4:  Performance  for  Different  Learning  Styles 
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Using  Critical  Components  of  Communicative  to  Measure  the  Distance  Learning  Experience 

Distance  delivery  of  educational  material  and  adult-learning  are  not  new  concepts.  From  Paul’s  letters  to  the 
Corinthians,  to  books  from  printing  presses,  to  the  mass  media  approach  of  film,  radio,  TV,  and  today  the  Internet, 
distance  education  has  a long  tradition  of  making  use  of  available  communication  technology  to  store,  transport,  and 
disseminate  information.  But  the  storage,  transportation,  and  dissemination  of  information  do  not  constitute  higher-level 
learning  according  to  Bloom’s  Taxonomy.  Whether  engaged  in  formal  or  informal  higher-level  learning,  humans 
incorporate  new  information  into  their  prior  experience  base  (Mezirow,  1991).  Humans  ascribe  value  and  meaning  to 
their  experiences  and  operationalize  their  knowledge  according  to  their  interactions  with  others  (Mead,  1917). 

Limited  interaction,  student  to  faculty  and  student  to  student,  has  always  been  one  of  the  rationale  for  distance 
education’s  struggled  to  gain  equal  prestige  with  traditional  classroom  settings.  Major  universities  administer  distance 
education  programs  as  part  of  their  extension  services.  Experienced  educators’  perceive  face-to-face  communication  as 
an  essential  part  of  the  learning  process.  However,  the  advent  of  multimedia,  hypermedia,  and  telecommunications  for 
educational  purposes  has  created  a new  opportunity  to  overcome  not  only  the  geographic  barriers  which  have  separated 
DE  students  in  the  past,  but  to  provide  all  students  the  learning  advantages  of  interactive  communication.  The  ever- 
growing ability  of  computing  and  telecommunications  to  support  interactive  learning  allows  educators  new 
opportunities  to  construct  communities  of  learners  to  assist  students  in  higher-level  learning  (Palloff  & Pratt,  1999). 

As  the  twenty-first  century  begins  traditional  educational  institutions  face  several  challenges: 

• Mounting  economic  pressure  (Increasing  costs  & reduced  funding) 

• The  need  to  produce  college  graduates  with  the  ability  to  function  well  in  a knowledge  society  (Palloff  & Pratt, 
1999). 

• Enrollment  in  post-secondary  institutions  reached  15.3  million  students  by  the  Fall  of  2002, 
(http://chronicle.eom/dailv/2001/08/2001 08 1 701  n.htm). 

• Enrollment  in  post-secondary  institutions  is  expected  to  reach  17.5  million  by  2010,  (U.  S.  Department  of 
Education,  1996). 

• As  few  as  1 in  4 college  students  fit  the  description  of  the  traditional  full  time  student,  (Twigg,  1994). 

Statement  of  the  Problem 

These  non -traditional  and  part-time  students  are  turning  with  increasing  frequency  to  distance  delivery  to 
address  their  educational  needs  (Palloff  & Pratt,  1999).  Educational  institutions  are  turning  more  frequently  to  the  use  of 
the  Internet  to  deliver  courses  to  nontraditional  students,  (Palloff  & Pratt,  1999).  The  advent  of  the  personal  computer 
and  access  to  the  Internet  has  increased  the  opportunity  for  universities  to  offer  on-line  courses  to  DE  students.  The 
University  of  South  Dakota  began  using  a PC  delivery  platform  in  1997.  Within  five  years  the  number  of  courses 
archived  in  the  system  had  grown  to  338  courses.  While  researchers  have  examined  the  design  and  implementation 
aspects  of  on-line  instruction,  few  studies  have  documented  the  experience  from  an  online  student's  perspective, 

(Turkle,  1995). 

The  purpose  of  this  study  is  to  examine  the  relationship  between  graduate  student  satisfaction  with  the  on-line 
learning  experience  and  four  criteria.  The  four  criteria  were  established  by  Junger  Habermas  in  his  work,  “The  Theory 
of  Communicative  Action”,  1984.  This  study  will  attempt  to  ascertain  by  what  degree  graduate  student  satisfaction  with 
the  on-line  learning  experience  is  related  to  the  interactive  communication  facilitated  by  multi-media  and  computer 
mediated  learning  situations. 

Habermas  states  that  all  human  communication  is  judged  by  the  receiver  using  four  critical  components  of 
interactive  communication;  coherence,  relevance,  accuracy,  and  honesty.  Each  message  received  is  critiqued  according 
to  the  receiver’s  ability  to  understand  the  message,  the  perceived  usefulness  of  the  information  transferred,  the 
receiver’s  perception  of  the  correctness  of  the  information,  and  the  receiver’s  perceptions  of  the  motives  of  the  sender. 
The  student’s  perceptions  are  key  to  developing  understanding  and  trust  for  the  other  members  of  the  group.  Through 
interactive  communication  the  group  modified  and  re-enforced  understanding  and  incorporation  of  information. 

Significance  of  the  Study 

Many  are  the  conditions,  which  must  be  fulfilled  if  the  Great  Society  is  to  become  a Great 
Community....  The  highest  and  most  difficult  kind  of  inquiry  and  a subtle,  delicate,  vivid  and  responsive 
art  of  communication  must  take  possession  of  the  physical  machinery  of  transmission  and  circulation 
and  breathe  life  into  it.  When  the  machine  age  has  thus  perfected  its  machinery,  it  will  be  a means  of  life 
and  not  its  despotic  master. 

John  Dewey  (1938,  p.  38) 
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Dewey’s  choice  of  words  reflects  the  pessimistic  view  of  the  human/machine  relationship  of  many  during  the 
Industrial  Age.  And  his  words  are  nearly  prophetic  as  to  the  focus  and  accomplishments  of  the  Information  Age.  The 
Industrial  Age  was  defined  by  rules  and  calculations,  while  the  Information  Age  is  defined  by  communication  and 
collaboration.  'Today,  the  lessons  of  computing  have  little  to  do  with  calculation  and  rules;  instead  they  concern 
simulation,  navigation,  and  interaction",  (Turkic,  1995,  p.  19). 

This  study  explores  the  communication  phenomenon  of  the  internet  learning  experience  from  the  graduate 
student’s  perspective  and  seeks  to  increase  educator  awareness  of  the  theories  guiding  interactive  communication. 

While  there  is  a need  for  quantitative  measurements  to  insure  system  functionality  and  course  content,  there  is  an  equal 
need  to  make  qualitative  measurements  of  the  DE  communication  experience.  Addressing  the  communication 
experience  of  the  graduate  and  professional  level  students  is  essential  for  DE’s  evolving  mission  and  desire  to  maximize 
student  learning  opportunities. 

The  general  purpose  of  this  study  is  to  analyze  the  relationships  between  the  critical  components  of  interactive 
communication  and  graduate  student  perceptions  of  the  on-line  learning  experience.  Specifically  this  study  will  ask 
graduate  students  taking  DE  courses  through  South  Dakota’s  DDN  and  computer-mediated  communication  network. 
Students  will  be  asked  questioned  concerning  administrative,  technical,  and  interactive  communication  issues. 
Questions  dealing  with  interactive  communication  issues  will  be  constructed  and  critiqued  according  to  the  critical 
components  of  interactive  communication  theorized  in  “Communicative  Action  Theory”,  Hubermas  (1984). 

Population  and  Sample  Selection 

The  population  for  this  study  was  limited  to  graduate  students  studying  at  three  public  universities  located  in 
South  Dakota.  Participants  in  the  study  have  all  recently  completed  a graduate  level  course  delivered  primarily  by 
remote  means  or  containing  remote  components  through  South  Dakota’s  DDN  and  computer-mediated  communication 
network 

Instrumentation 

A narrative  approach  will  be  utilized  to  achieve  the  purposes  of  this  study.  A select  group  of  graduate  students 
participating  in  CMC  distance  delivery  courses  will  be  ask  to  share  their  experiences  with  and  observation  of  the  CMC 
classroom.  A narrative  approach  to  research  allows  people  to  voice  their  values  and  preferences,  increasing  our 
collective  knowledge  of  the  human  experience  (Bakan,  1996).  The  scale  for  this  study  will  have  six  components: 
frequency,  honesty,  relevance,  openness,  affiliation,  and  student  satisfaction.  The  student  satisfaction  component  will 
draw  from  questions  (p.  158)  and  issues  concerning  online  learning  communities  posed  by  Pratt  and  Palloff  (1999). 
Issues  to  be  explored  will  include:  frequency  and  mode  of  communication,  student  perceptions  of  the  honesty, 
relevance,  and  openness  of  the  online  classroom  environment,  and  student  perception  of  the  effectiveness  of  CMC 
communication. 

This  study  will  primarily  use  interviews  to  obtain  data.  Interviews  will  be  conducted  in  person,  by  phone,  or 
over  the  Internet.  Participants  will  be  asked  to  disclose  basic  demographic  information. 
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Abstract:  In  close  collaboration  with  the  staff  of  the  Pew  Internet  & American  Life  Project 
(www.pewintemet.org),  the  American  Institutes  for  Research  (AIR)  is  conducting  a qualitative  study  on  the 
impact  of  the  Internet  on  schooling  and  education  from  the  perspective  of  middle  and  high  school  students. 
Primarily  through  the  conduct  of  multiple  in-person  focus  groups  that  are  being  held  across  the  country,  we  focus 
on  the  behaviors,  beliefs,  and  attitudes  of  middle  and  high  school  students  about  access  to  and  use  of  the  Internet 
for  schooling,  including  (intended  and  unintended)  outcomes  of  those  uses.  Among  the  questions  we  are  most 
interested  in  pursuing:  Is  the  Internet  helping  students  learn  and  why?  What  activities  do  students  consider  and 
pursue  as  “educational”  on  the  Internet?  What  are  the  other  outcomes  (intended  and  unintended)  of  educational 
Internet  use  (e.g.,  social  interaction,  interest  and  motivation,  writing  skills,  post-secondary  goals,  etc.)?  Based  on 
this  knowledge,  how  could  the  Internet  be  better  used  by  schools?  Other  data  collection  strategies  include  the 
surveying  of  focus  group  participants  (to  better  describe  our  sample),  as  well  as  the  online  solicitation  of  stories 
about  the  use  of  the  Internet  for  education  directly  from  students. 


Introduction 

“Overall,  I really  think  that  the  Internet  has  a great  impact  on  how  much  I learn  at  school  and 
without  it  1 don’t  think  that  I would  have  the  opportunities  to  learn  as  much  as  I do.” 

“ Midwestern  female  HS  student,  online  storyteller 

Given  the  relatively  high  levels  of  Internet  access  in  schools  and  homes,  it  stands  to  reason  that  youth  are 
increasingly  online.  In  fact,  this  is  the  case.  By  the  age  of  1 0,  youth  are  more  likely  to  use  the  Internet  than  anyone 
at  any  age  beyond  25.  As  of  September  2001,  about  two-thirds  of  10  to  13  year  olds — and  more  than  three-quarters 
of  14  to  17  year  olds — reported  using  the  Internet  at  any  location.  We  know,  then,  that  youth  are  online  more  and 
more.  However,  we  know  less  about  what  they  do  online  - especially  for  school. 

To  date,  most  research  on  the  use  of  the  Internet  in  schools  and  for  school — as  distinguished  from 
research  on  the  use  of  computers  and  other  technologies — has  focused  on  access.  This  has  consisted  of 
measuring  the  extent  of  connectedness  to  the  Internet  and  supports  for  Internet  use  in  schools  (i.e.,  the 
amount  and  adequacy  of  teacher  professional  development  and  technical  support).  A few  large-scale 
surveys,  most  now  several  years  old,  provide  some  modest  insights  into  the  extent  and  types  of  Internet 
use  in  schools,  though  typically  from  teachers’  or  school  technology  coordinators’  perspectives. 

Unfortunately,  the  current  knowledge  base  is  exceedingly  limited  in  its  ability  to  describe  the  rich 
details  of  educational  student  Internet  use  and  outcomes,  particularly  when  this  use  is  occurring  outside 
of  schools  and  classrooms. 

The  Study 

To  provide  insight  into  the  rapidly  evolving  influence  of  the  Internet  on  schooling,  the  American  Institutes  for 
Research  was  commissioned  by  the  Pew  Internet  & American  Life  Project  to  conduct  a study  to  describe  the  rich 
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and  varied  ways  that  middle  and  high  school  students  use  the  Internet  for  school  and  learning,  their  attitudes  toward 
the  Internet,  and  what  they  would  like  from  their  education.  Data  for  this  study  were  collected  primarily  through 
three  mechanisms: 

• The  conduct  of  12  focus  groups  of  a total  of  [136J  diverse  middle  and  high  school  students  drawn 
from  across  the  country  who  characterized  themselves  as  heavy  Internet  users,  coupled  with  2 
focus  groups  of  light  Internet  users  (comprised  of  one  middle  and  one  high  school  group); 

• The  administration  of  questionnaires  to  each  focus  group  participant  in  order  to  help  characterize 
their  school-related  Internet  use  both  in-  and  out-of-school;  and, 

• The  online  solicitation  of  student-written  stories  detailing  how  they  use  the  Internet  for  school. 
Nearly  200  middle  and  high  school  students  from  across  the  country  wrote  and  submitted  their 
stories  to  AIR  through  the  study’s  Website. 


Findings 

Preliminary  findings  show  that  students  who  use  the  Internet  a great  deal  use  it  for  communication,  entertainment, 
and  learning  purposes  - including  gathering  and  synthesizing  information  for  school.  There  seems  to  be  a 
disconnect,  however,  with  the  high  Internet  skill  and  knowledge  that  these  students  have,  and  the  access  and  uses 
that  that  schools  and  teachers  provide  for  them.  A final  report  with  these  and  numerous  other  findings  and 
implications  will  be  available  at  http  ://www.Dewi  internet,  ora/re  ports  in  the  summer  of  2002. 
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Abstract:  This  presentation  demonstrates  the  SculptureQuest  Microworld  Prototype,  designed  to 
allow  sculpture  students  to  implement  rapid  prototyping  in  their  production  process.  The 
Microworld’s  purpose  is  to  promote  metacognitive  learning  outcomes  and  enhance  domain-specific 
skills  and  knowledge  related  to  the  art  of  sculpture.  The  Microworld’s  programmed  environment 
delivers  customizable,  constructivist  strat^ies  that  drive  the  effective  use  of  technology  within  a 
classroom  context.  Teachers  and  students  are  encouraged  to  add,  delete,  or  reprogram  features  as 
necessary  to  adapt  the  tool  to  their  particular  needs. 


Introduction 

The  SculptureQuest  Microworld  (SQMW)  is  a learning  environment  in  which  the  student  uses  interface  and 
display  capabilities  of  a contemporary  3D  computer  modeling  GUI  to  perform  functions  which  rapidly  prototype 
sculptural  pieces.  In  addition,  analytical  features  output  the  physical  parameters  of  prototyped  sculptures  and 
installations  as  if  actually  built.  This  allows  recognition  of  the  strengths  and  weaknesses,  or  logical  outcomes,  of 
decision  points  encountered  throughout  the  creative  process  of  sculpture.  SQMW  is  meant  for  use  in  a classroom 
setting  with  emphasis  on  examining  relationships  between  the  artist’s  mental  models  and  artistic  processes. 

In  a non-traditional  environment,  where  knowledge  is  assumed  to  be  constructed  and  experiential,  learning 
objectives  must  reside  within  the  learner  and  her  cognitive/metacognitive  abilities  (Lin,  2001).  One  major  outcome  of 
employing  SQMW  in  a pedagogical  setting  is  the  integration  of  metacognitive  and  domain -specific  activities, 
scaffolding  student  self-knowledge  and  domain  knowledge  simultaneously.  The  output  data  directly  relates  actual 
material  outcomes  with  specific  design  decisions  and  promotes  transfer  of  knowledge  to  the  larger  physical  world. 

The  Microworld  Model 

The  use  of  the  microworld  model  to  represent  the  technology  piece  of  this  course  is  a good  fit.  Strohecker 
(2000)  identifies  microworlds  as  inquiry -based  learning  environments,  inherently  constructivist  in  nature,  simple  in 
principle  but  rich  in  possibilities  for  playful,  thoughtful  activity.  Land  and  Hannafin  (1996)  add  that  students  rely  on 
the  open-ended  nature  of  microworlds  to  support  experiences  necessary  to  identify,  question,  and  test  the  limits  of 
their  intuitive  beliefs. 

Battista  (1998)  provides  the  most  concise  criteria  for  what  constitutes  a microworld  by  stating  that  this 
environment  should:  1.  Support  problem-centered  inquiry  2.  Be  based  on  research  of  students’  domain -specific 

learning  3.  Cultivate  mental  models  of  abstract  ideas  and  4.  Induce  reflection  and  abstraction.  SQMW  supports  each 
of  these  criteria  using  programmable  features  including  the  Diary  tool,  the  Self  Assess  tool,  the  Playback  tool  and  the 
Video  Assistant. 

The  Course 

College  students  who  possess  familiarity  with  3D  computer  drawing  programs  comprise  the  target  audience 
for  this  course.  It  is  likely  that  these  students  will  initially  possess  only  a beginner’s  awareness  of  metacognitive 
techniques.  Despite  the  fact  that  this  Microworld  exists  as  a computer  environment,  its  real  context  is  the  social  milieu 
of  the  classroom.  Students  and  teacher  must  together  navigate  techniques  for  developing  metacognition,  using 
sculpture  as  a focus.  It  is  necessary  for  all  to  meet  in  the  spirit  of  collaboration  to  develop  a mutually  agreed  upon 
approach  for  the  semester.  It  is  at  this  time  that  SculptureQuest  is  programmed  for  specific  needs  that  the  class  has 
developed.  Next,  all  parties  agree  to  follow  through  with  the  agenda,  using  the  Microworld  features  to  record  and 
sequence  events,  observations  and  insights  that  arise. 
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How  the  Microworld  Works 


There  are  two  worlds  found  in  SQMW:  Explore  and  Prototype.  Explore  provides  2D  and  3D  objects  as 
building  blocks  that  can  be  assembled  into  complicated  three-dimensional  arrangements  that  may  be  assigned 
material  characteristics.  This  is  the  environment  most  attractive  to  the  novice  learner  who  is  examining  basic  shape 
relationships,  physical/  structural  properties  and  sculptural  concepts.  Constructions  may  be  exported  to  “Prototype” 
for  further  study.  The  Video  Assistant  (V/A),  Playback  and  Diary  tools  are  also  available  in  Explore. 

Prototype  provides  a more  demanding-for-mastery  set  of  tools  and  an  environment  in  which  construction 
and  analysis  take  place.  This  area  is  the  primary  strength  of  the  Microworld.  All  material  and  structural  analysis  takes 
place  here  and  a sculpture  may  be  continuously  modified  through  design  iterations.  It  is  within  the  Prototype 
environment  that  the  metacognitive  activities  incorporated  in  the  Video  Assistant  (V/A),  the  Diary,  the  Playback,  and 
the  Self-Assess  tools  (combined  with  the  social  environment  of  the  classroom)  come  into  full  play. 

The  Microworld  Tools 

The  Tool  designs  are  based  on  findings  from  cognitive  research.  The  Diary  Tool  has  its  roots  in  Lesh’s 
Construct  Documentation  Principle  (2001 ) that  requires  activities  to  contribute  to  both  learning  and  documentation  of 
learning,  thereby  fostering  self-reflection.  Lesh  claims  that  thinking  about  thinking  does  not  come  easily  without 
scaffolding.  His  work  recommends  thought-revealing  activities  that  encourage  students  to  externalize  their  thought 
processes  in  order  to  facilitate  reflection  (Lesh,  et  al.  2001). 

The  goal  of  the  Self  Assess  Tool  is  to  stimulate  self-explanation  through  prompts  that  serve  to  guide 
student  attention  to  conflicting  thoughts  and  build  coherent  understanding  of  the  domain  tasks.  Findings  by  Chi,  et. 
al  (1994)  show  prompts  lead  to  extensive  inference. 

Lin  and  Lehman  (1999)  also  use  prompts  in  a computer-simulation  developed  to  aid  students  in  designing 
biology  experiments.  Monitoring  how  and  why  certain  decisions  were  made  is  effective  because  students  are  able  to 
pin  down  specifically  where  and  what  they  did  not  understand.  Instead  of  self-assessing  at  a general  level  as 
expressed  in  the  statements  “I  don’t  understand”  or  “I  am  confused”,  the  students  are  able  to  explain  specifically 
what  they  did  not  understand  and  where  the  difficulty  occurred  (Lin  and  Lehman,  1999). 

Conclusions 

The  SculptureQuest  Microworld  has  been  integrated  into  a course  on  sculpture  studio  practice  to 
demonstrate  that  technology  may  effectively  support  Lin’s  findings  that  the  best  approach  for  optimal  learning 
outcomes  is  to  practice  metacognition  within  a domain -specific  context  (Lin,  2001).  Hence,  while  the  design  of  the 
computer  piece  is  significant,  it  is  the  design  of  the  course  in  which  it  is  used  that  will  determine  the  effectiveness  of 
SQMW  for  cognitive  and  metacognitive  learning  outcomes. 
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Abstract : An  increasing  number  of  nations  connect  their  schools  on  the  Internet  as  an 
acknowledgment  to  its  extreme  importance  in  the  educatk>n  area.  Our  study  specifies  the 
perils  that  arise  from  its  use  when  the  users  are  minors  and  evaluates  the  technologies  that  are 
currently  available  to  address  the  filtering  issues.  The  thoughts  presented  in  this  paper  outline 
our  proposed  solution  for  the  Greek  School  Network.  By  no  means  the  thoughts  that  are 
presented  here,  apply  for  adults  or  people  who  pay  to  establish  access  to  the  Internet. 


INTRODUCTION 

As  the  number  of  computers  in  schools  and  the  number  of  children  accessing  the  Internet  from  the  classroom 
have  grown  exponentially  over  the  past  few  years,  so  too  have  the  challenges  facing  educators  trying  to  ensure 
that  children  have  a positive  experience  when  they  go  online.  The  educational  community  cannot  ignore  the 
problem,  as  it  will  be  like  it  accepts  the  use  of  the  school  network  resources  for  purposes  irrelevant  to  every 
possible  educational  goal. 

A related  debate  rages  over  what  percentage  of  Web  sites  would  truly  be  considered  objectionable.  Some 
advocates  argue  that  sites  that  would  be  considered  harmful  to  minors  represent  only  a very  small  proportion  of 
the  Web.  What  is  of  greater  concern,  they  say,  is  that  perfectly  benign  and  possibly  very  useful  information 
could  be  blocked  when  software  is  used  to  screen  inappropriate  material.  The  actual  extent  to  which  adult- 
oriented  materials  are  available  on  the  Internet  is  irrelevant  according  to  those  who  support  content  controls. 
They  believe  that  any  amount  of  inappropriate  content  is  too  much,  when  children  are  concerned.  Some  experts 
argue  that  the  Web  is  expanding  so  fast  that  it  is  virtually  impossible  to  track  every  site  that  could  be 
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objectionable.  The  flip  side  of  that  argument  is  that  it  is  better  to  minimize  access  to  objectionable  content  as 
best  we  can,  even  if  the  occasional  site  slips  through  the  cracks. 

Although  pornography  on  the  Internet  has  captured  the  greatest  attention  on  the  part  of  policy-makers,  it  is  not 
the  only  area  of  potential  concern  for  parents  and  educators.  Many  adults  are  concerned  about  Web  sites  that  are 
created  by  hate  groups  or  devoted  to  topics  such  as  bomb  making  and  weaponry,  gambling  or  alcohol  and 
smoking.  The  World  Wide  Web,  however,  is  not  the  only  source  of  concern.  Children  can  receive  email 
messages  with  pornographic  file  attachments.  Of  special  concern,  too,  are  Internet  chat  rooms  and  so-called 
Instant  Messaging,  where  children  can  communicate  online  in  real-time  with  adult  strangers  who  may  not  have 
their  best  interests  at  heart. 

A wealth  of  information  is  available  on  the  Internet.  White  papers  [4,5]  are  opposed  to  the  use  of  censor  ware 
programs  (they  call  it)  in  the  American  libraries  and  schools,  and  there  are  some  interesting  links  that  are 
indicative  of  the  ongoing  debate  [2,6,9,10]. 

The  Internet  truly  is  like  a vast  library  including  millions  of  readily  available  and  indexed  publications, 
containing  content  as  diverse  as  human  thought.  Throughout  the  past  decade,  policy  makers,  industry  advocates, 
parents  and  teachers  have  tried  to  address  these  concerns  especially  in  the  more  technologically  advanced  areas 
of  the  earth.  This  decade  may  well  be  the  decade  of  decisions  in  a much  broader  spectrum  than  ever  before. 

From  our  point  of  view  it  is  an  unarguable  fact  that  Internet  offers  a vast  mass  of  information,  some  of  which 
is  suitable  for  schools  whilst  some  is  not.  Without  meaning  that  we  should  implement  extreme  measures  of 
censorship  or  suffocative  limitations  to  the  content  we  allow  our  users  to  access,  we  should  be  alert  and  monitor 
the  things  that  happen  while  students  are  online.  The  problem  with  Internet  content  seems  to  have  social, 
cultural,  pedagogic  and  scientific  viewpoints:  Schools*  trustworthiness  is  endangered  if  the  school  network  is 
used  (systematically  or  occasionally)  for  other  purposes  than  those  for  which  it  was  designed  and  implemented. 
The  parents  and  the  public  believe  that  their  children  and  students  proportionately,  should  be  protected  from 
illegal,  offensive  and  inappropriate  content.  Whilst  some  may  argue  that  it  is  a means  for  censorship  and  nothing 
more,  others  believe  it  is  the  only  way  societies  have  to  inform  and  protect  the  children  from  the  potential  danger 
the  Internet  poses. 

Every  country  should  establish  its  own  policies  to  deal  with  it  without  raising  public  concern  regarding  human 
rights  and  individual’s  freedoms.  This  document  will  try  to  group  all  potential  dangers  the  Internet  can  pose  and 
some  possible  solutions  to  those  dangers.  For  us  the  best  solution  is  the  one  that  combines  the  right  guidance  of 
the  students  from  the  educational  authority,  the  infcrming  of  parents  and  the  training  of  educators  together  with 
the  technical  solution.  No  solitary  technical  working-out  can  resolve  such  a problem  and  whichever  policies  put 
forth  should  take  in  mind  all  the  above  parameters. 


Status  in  the  other  Countrie  s 

Each  of  the  countries  presented  here,  establishes  it’s  own  policies  and  methods.  However,  their  perception  of 
the  problem  presents  some  surprisingly  common  characteristics.  In  our  survey  we  browsed  some  interesting  and 
concise  pages  regarding  the  pronounced  decisions  on  the  Internet  content  issue  in  Australia  [12],  Canada  [7], 
United  States  of  America  [3]  and  the  European  Union  [11]. 

Internet  is  being  used  in  Australian  schools  with  highly  increasing  rates,  as  a learning  driving  force  in 
education.  43%  of  the  children  asked  in  Australia  said  that  Internet  improved  their  perception  about  school.  The 
federal  government  of  Australia  responded  to  the  issues  posed  by  the  entrance  of  Internet  in  the  everyday  and 
school  life,  with  the  Broadcasting  Services  Amendment  Act  (1999).  A hot  line  was  created  to  allow  people 
report  what  they  think  is  illegal  and  express  their  worries  about  the  content  that  can  be  accessed  on  the  Internet. 
The  industry  plays  important  role  by  establishing  policies  for  the  improvement  of  the  information  exchange 
regarding  the  content  of  the  web  pages.  In  some  cases  schools  store  pages  certified  as  ‘safe  quality  pages’  in 
their  cache,  ensuring  by  that  way  that  their  students  have  quick  access  to  them.  While  these  pages  are  available 
for  the  benefit  of  students  reducing  the  bandwidth  consumption,  the  costs  for  schools  remain  low  because  of  the 
quicker  access  time  achieved  for  stored  sites.  In  Australia  the  decisions  are  made  on  the  school  level  (in  some 
states  there  is  a trend  for  more  obligatory  policies  decided  on  a higher  level  of  authority).  Education  authorities 
stress  the  importance  of  accessing  pages  of  high  quality.  Educators  in  national  level  do  the  evaluation  of  pages 
selected  to  meet  certain  specifications.  Recent  studies  point  out  that  almost  98%  of  public  schools  connected  to 
the  Internet  have  established  Acceptable  Use  Policies.  74%  of  schools  with  AUP  use  software  to  block  or  filter 
pages  of  certain  content,  64%  uses  rules  of  conduct  and  28%  uses  an  intranet  in  their  effort  to  control  access. 
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To  date,  Canadians  have  established  a wide  range  of  partnerships  to  deal  with  the  issues  of  the  Internet 
content.  The  strategy  of  Canada’s  Government  is  to  make  Canada  the  most  connected  country  in  the  world. 
Illegal  content  — content  that  violates  Canada’s  laws — is  of  key  concern  to  Canadians.  Child  pornography  and 
hate  propaganda  are  particularly  troubling,  because  they  pose  the  greatest  and  most  immediate  risk  to  the  safety 
and  well  being  of  Canadians.  Enforcing  the  law  in  cyberspace,  however,  presents  significant  challenges, 
particularly  in  view  of  rapid  technological  change.  For  Canadians  it  is  important  to  distinguish  between  Internet 
content  that  is  illegal,  and  content  that  is  offensive  to  some,  but  is  not  illegal.  The  control  and  management  of 
offensive  content,  however,  calls  for  different  approaches,  such  as  empowering  users,  educating  consumers  to 
make  informed  choices,  and  establishing  responsible  industry  practices.  The  Canadian  Charter  of  Rights  and 
Freedoms  guarantees  all  persons  in  Canada  "freedom  of  thought,  belief,  opinion  and  expression,  including 
freedom  of  the  press  and  other  media  of  communication".  The  federal  government’s  approach  is  to  involve  a 
broad  spectrum  of  Canadians  in  addressing  the  issues.  Its  priorities  include  supporting  initiatives  that  educate 
and  empower  users,  promoting  effective  industry  self-regulation,  strengthening  the  enforcement  of  laws  in 
cyberspace,  implementing  hotlines  and  complaint  reporting  systems  and  fostering  consultation  between  the 
public  and  private  sectors,  and  their  counterparts  in  other  countries.  A considerable  number  of  important 
initiatives,  studies  and  discussion  have  taken  place  or  are  in  progress  in  Canada.  In  1994,  the  Government  of 
Canada  established  the  private  sector  Information  Highway  Advisory  Council  (IHAC)  to  provide  advice  on  the 
best  way  to  develop  Canada’s  Information  Highway.  According  to  its  recommendations  the  federal  government 
should  a.  fine-tune  existing  laws  to  make  them  more  applicable  and  enforceable  in  the  changing  world  of  global 
networks  and  b.  encourage  research  and  the  development  of  technical  tools  that  can  protect  users  against 
offensive  content  on  the  Internet,  and  assist  in  law  enforcement. 

The  United  States  of  America  Congress  passed  legislation  requiring  Internet  blocking  technology  to  block 
pornographic  materials  in  all  public  schools  and  libraries  funded  through  certain  federal  programs.  The 
Children's  Internet  Protection  Act  -CIPA  passed  the  senate  and  the  congress  on  December  2000  as  part  of  a big 
government  budget  for  the  year  2001.  To  date  the  issue  of  content  control  in  the  United  States  of  America  is  in  a 
critical  point:  while  the  congress  decided  the  mandatory  implementation  of  filtering  in  schools  and  libraries, 
parents,  organizations  and  politicians  are  steadily  opposed  to  the  legislation  passed.  Influential  organizations  are 
hostile  to  it  and  government  bills  mandating  filtering  in  schools  and  libraries  face  legal  challenges  on 
constitutional  issues.  Some  libraries  have  already  been  sued  for  nstalling  filtering  software  onto  their 
computers.  Parents,  schools  and  libraries  face  difficulties  to  decide  whether,  and  how,  to  filter  Internet  content 
[8].  Still  nobody  denies  the  fact  that  the  World  Wide  Web  is  not  content  zoned,  which  means  that  kids  can 
access  anything  on  it  very  fast  and  very  easily.  As  for  the  libraries,  while  school  libraries  have  special 
characteristics,  public  libraries  are  intended  for  free  willing  inquiry  and  access  control  is  a more  complicated 
issue. 

The  European  Union  has  piblished  an  Action  Plan  on  promoting  safer  use  of  the  Internet  [4].  While  it 
recognises  the  positive  benefits  of  Internet  (particularly  in  education)  it  states  that  the  amount  of  harmful  and 
illegal  content  carried  over  the  Internet,  while  limited,  could  adversely  affect  the  establishment  of  the  necessary 
favourable  environment  for  initiatives  and  undertakings  to  flourish.  A safer  environment  should  be  provided  by 
combating  illegal  use  of  the  technical  possibilities  of  the  Internet,  in  particular  for  offences  against  children  and 
trafficking  in  human  beings  or  for  the  dissemination  of  racist  and  xenophobic  ideas,  ensuring  that  consumers 
make  full  use  of  the  Internet.  Europe  should  promote  industry  self-regulation  and  content-monitoring  schemes, 
the  development  of  filtering  tools  and  rating  systems  by  the  industry  and  of  course  international  cooperation. 
While  any  hot-line  reporting  mechanisms  should  support  and  promote  measures  taken  by  the  Member  States, 
duplication  of  work  should  be  avoided.  The  responsibility  for  prosecuting  and  punishing  those  responsible  for 
illegal  content  should  remain  with  the  national  law-enforcement  authorities. 


Techniques  and  Solutions 

The  techniques  that  can  be  used  to  achieve  content  filtering  are  keyword  blocking,  negative  and  positive  lists 
and  content  labeling  and  rating  systems.  Keyword  blocking  prohibits  access  to  pages  that  contain  the  specified 
words  (words  can  be  tobacco,  wine,  drug,  sex  etc).  Unless  this  is  a ‘clever’  software  (rarely  this  is  the  case)  this 
technique  cannot  successfully  address  the  modem  issues.  It  can  be  easily  bypassed  as  it  has  been  reported  that 
web  developers  add  additional  characters  to  the  words  they  suspect  that  can  be  searched  for.  The  use  of  negative 
and  positive  lists  is  easy  from  filters  or  browsers.  The  positive  lists  contain  URLs  and  domains  to  which  access 
is  allowed  to  and  negative  lists  contain  the  URLs  that  access  is  prohibited  to.  Use  of  white  lists  will  be  very 
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restricting  regarding  the  amount  of  available  pages  on  the  net.  Labeling  and  rating  systems  provide  a way  to 
categorize  pages  according  to  their  content  and  provide  that  categorization  to  the  user.  That  technique  can 
proved  to  be  inefficient  if  nobody  offers  to  undertake  the  task  of  labeling.  The  main  problem  is  the  size  of  the 
Internet  and  the  rate  of  its  daily  expansion  (there  is  some  indication  that  Internet  growth  today  has  gone  from 
exponential  to  linear). 

Access  control  can  be  achieved  using  Commercial  Software,  Freeware  Programs,  Rating  Systems  or  Hardware 
Solutions  (all  of  them  are  mainly  structured  on  the  use  of  lists). 

Many  commercial  filtering  programs  advertise  that  can  block  access  to  the  harmful  or  improper  pages  on  the 
Internet  (in  fact  they  claim  they  can  do  much  more).  There  are  programs  that  can  be  installed  at  the  server  side, 
some  are  for  clients  only,  while  other  offer  versions  for  home  and  for  education.  The  problem  with  commercial 
software  is  that  the  companies  usually  do  not  reveal  the  pages  they  block  access  to  [4].  This  solution  is  not 
favorable  for  a school  environment  for  one  more  reason:  The  filtering  is  done  by  a third  and  not  by  the 
government  or  an  educational  authority.  This  gives  to  the  company  that  implements  the  filtering,  access  to 
sensitive  data.  The  use  of  proxies  has  a policy  problem:  proxy  servers  keep  log  files.  These  files  contain  all  the 
requests  a proxy  receives,  causing  complaints  and  worries  regarding  the  people  that  gain  access  to  that  data  and 
the  processes  that  can  be  put  in  place  to  monitor  individual  users’  behavior.  Although  it  seems  that  today  there  is 
a some  development  under  way  on  rating  systems  everyone  should  remember  that  this  method  might  proved  to 
be  of  little  efficiency  if  there  is  not  someone  who  will  undertake  the  task  of  labeling  and  rating  of  sites  in 
Internet  in  a continual  basis.  Hardware  solutions  appear  to  be  more  complete,  with  specialized  systems  of  black 
box  type.  They  are  intended  as  big  scale  solutions  for  huge  users’  databases  and  lines  of  high  capacity.  They 
require  minimal  efforts  to  be  installed,  configured  and  operate  while  full  support  is  offered  from  their  company. 
Along  with  that  comes  the  usually  high  cost  to  obtain  such  a system. 


Policies 

Currently  in  Greece  there  is  no  government  decision  regarding  the  policies  mentioned  here.  The  policies’  part 
of  our  suggestions  -it  could  also  serve  to  initiate  a public  discussion  regarding  content  filtering  and  content 
blocking  in  Greece-  is: 

1.  Creation  of  Acceptable  Use  Policies  for  the  Greek  School  Network  to  inform  users  for  the  purposes  of  the 
creation  of  the  technological  infrastructure  of  the  school  network  and  the  actions  or  behaviors  that  are 
considered  acceptable.  Most  such  policies  include:  Description  of  the  underlying  philosophy  and  strategy 
implemented  into  the  school  network  for  the  access  to  Internet,  Report  of  the  educational  uses  and  advantages  of 
Internet,  List  with  the  catalogue  of  the  duties  teachers  and  students  have  on  the  issues  involved  by  the  use  of 
Internet,  Description  of  what  is  considered  acceptable  and  what  not  acceptable  use  of  the  school  network  and  of 
the  Internet,  Reminder  that  the  access  to  the  school  network  and  to  the  Internet  is  a privilege.  Specification  of  the 
pages  considered  as  ‘improper’,  ‘harmful’,  ‘illegal’,  or  ‘void  of  educational  content’.  2.  A critical  point  is  the 
responsible,  with  no  exaggerations  informing  of  the  teachers.  Without  their  sincere  participation  success  will 
prove  hard.  3.  Control  of  the  time  and  opportunities  students  will  have  access  to  the  computers  (when  in  lesson 
hour,  they  will  be  supervised  from  their  teachers).  4.  Control  of  installation  of  software  on  school  computers. 
The  common  problems  are  license  issues,  installation  of  ‘Trojan  horse’  programs  (programs  with  malicious  or 
harmful  code),  worms,  etc.  5.  Availability  of  educational  material  to  constitute  the  positive  counteroffer.  The 
network  itself  is  not  the  main  objective:  The  main  intension  of  schools  is  the  use  of  all  possible  ways  and  new 
technologies  for  educational  purposes,  and  therefore  the  use  of  the  school  network  and  equipment  for  specific 
cognitive  purpose.  6.  The  age  of  students  should  be  taken  in  consideration(  ages  from  6 to  1 1 are  the  most 
vulnerable).  7.  Provision  of  the  cachemaster’s  mail  address  for  communication  between  him  and  the  school 
community.  8.  Extra  security  considerations  and  policies  if  the  school  provides  email  accounts  for  students  and 
teachers  (not  true  currently  in  Greece  but  visible  in  the  very  near  future). 


Architecture 

The  Greek  School  Network's  topology  is  illustrated  in  the  next  figure. 
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Figl  : The  Greek  School  Network  topology 

The  backbone  network  is  the  Greek  Research  and  Technology  Network  (GRNET),  which  provides  Internet 
Services  to  the  Greek  Academic  and  Research  community.  The  distribution  network  is  the  part  of  the  network 
that  interconnects  points  of  presence  (nodes)  with  the  backbone  network.  The  topology  has  such  design  in  order 
to  preserve  the  operational  cost  in  low  levels,  which  is  particularly  critical  in  large  geographical  region 
networks.  The  proposed  architecture  is  in  favor  of  the  use  of  a proxy -cache  server  for  the  school  network 
because  the  proxy  server  on  a network  -especially  when  it  is  configured  as  transparent-  is  the  ideal  point  for  the 
implementation  of  control  on  the  requested  objects. 

The  extent  of  Squid’s  usage  in  educational  networks  all  over  the  world  and  the  level  of  acceptance  it  enjoys 
from  the  global  networking  community  (especially  researchers,  scholars  and  educators),  is  obvious  from  the 
number  of  networks  implemented  on:  the  JANET  network  (a  private,  government  funded  network  for  education 
and  research  in  England  with  90  to  120  million  hits  per  day),  the  italian  GARR  network,  the  german  DFN 
network,  the  Swiss  network  SWITCH,  the  Dutch  network  SURFnet  and  the  USA  academic  network  the 
NLANR.  We  are  going  to  use  Squidguard  as  the  redirector  of  our  choice  because  it  is  faster  and  opens  less 
redirectors  than  it’s  competitor  programs. 


Implementation  Issues 

Squid  is  free  software,  licensed  under  the  terms  of  the  GNU  (General  Public  License).  The  most  important 
resource  for  squid’s  performance  is  physical  memory,  so  fast  disks  are  important  for  high-volume  caches.  Some 
new  options  were  added  to  squid’s  configuration  file  after  its  last  release.  All  new  parameters  intend  to  improve 
the  performance,  the  security  or  the  flexibility  of  the  program.  Available  options  are  selective  logging  and  the 
use  of  time  spaces,  ip  ranges  or  users  ids  in  order  to  grant  or  deny  access  at  certain  hours  of  day,  at  certain  ips  or 
at  certain  users. 


Conclusions 

The  rapidly  evolving  nature  of  the  Internet  virtually  ensures  that  no  filtering  technology  can  be  a hundred 
percent  perfect.  Certain  kinds  of  network  management  products  may  provide  basic  information  on  how  students 
and  staff  are  using  the  network  but  the  parental  and  school  guidance  wfll  remain  the  basic  factor  to  the  solution 
of  the  problem. 

If  a school  district  employs  monitoring  or  filtering,  its  Acceptable  Use  Policy  should  explain  what  it  would  be 
doing,  and  the  procedures  a student  should  follow  if  he  or  she  encounters  a site  that  would  be  considered 
inappropriate. 

The  practices  are  quite  the  same  across  the  different  nations.  This  happens  because  the  filtering  problem  is 
relatively  new  and  those  who  started  first  to  deal  with  it  lead  the  way.  The  content  filtering  issue  we  discuss 
herein  has  never  caused  a central  debate  either  from  parents  for  their  kids  or  from  school  teachers  or  even  from 
the  government  in  Greece.  We  work  on  that  issue  because  we  believe  that  every  country  should  be  aware  of  the 
possible  dangers  and  be  as  prepared  as  possible  to  deal  with  them.  If  we  know  and  understand  what  happens  in 
the  rest  of  the  world  and  in  countries  that  have  a lead  over  Greece  in  technological  issues  we  will  save  ourselfs 
time  and  trouble. 

The  biggest  percentage  of  the  pages  on  the  net  is  written  in  English  which  is  nowadays  an  international 
language.  Therefore  the  most  interesting  sites  and  definitely  those  which  get  read  from  the  biggest  audience  are 
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the  same  even  for  the  countries  whose  language  is  other  than  english.  That  means  that  the  problems  are  the  same 
since  their  source  is  the  same.  This  is  why  many  countries  seem  to  adopt  the  same  strategies  despite  their  natural 
differences. 

A convenient  solution  for  the  most  educational  institutions  has  been  the  installation  of  filtering  software  on  the 
proxy  server.  This  is  the  ideal  place  to  do  filtering  for  a network  because  it  is  the  one  point  through  which  all 
network  communications  pass.  Filtering  on  a proxy  server  can  have  an  impact  on  network  performance  because 
of  the  need  to  match  a URL  against  what  may  be  a long  list  of  blocked  sites.  The  use  of  a caching  server  can 
help  speed  access  and  reduce  the  bandwidth  that  would  otherwise  be  needed. 

Web  server  surveys  have  shown  there  were  more  than  27  million  web  servers  in  operation  as  of  January  2001 
[15].  One  study  estimated  the  Web  to  have  approximated  800  million  pages  in  February  1999  [13].  While  the 
above  results  are  not  accurate  for  2001,  they  are  indicative  of  the  order  of  magnitude  of  the  sites  on  the  Internet. 
If  this  is  to  change,  obviously  it  won’t  shrink.  The  lists  every  software  uses  to  block  access  to  sites  will  probably 
include  controversial  entries.  As  these  lists  cannot  be  human  reviewed  the  common  way  to  cope  with  problems 
that  might  arise  is  the  creation  of  an  environment  which  allow  the  modification  of  the  lists  and  provide  the 
required  information  to  the  users. 

Even  if  we  find  the  ideal  solution  and  implement  it  in  our  school  networks,  no  one  can  guarantee  that  this  will 
be  the  ideal  solution  or  even  an  appropriate  one,  in  one  or  two  years  time.  People  with  wide  knowledge  of  the 
Internet  and  the  emerging  technologies  should  monitor  the  developments  that  take  place  and  adjust  their  policies 
to  the  new  era. 

Implementation  of  content/access  control  doesn’t  solve  radically  the  problem  of  students  using  the  school 
networks  to  gain  access  to  objectionable  or  illegal  material.  There  is  a number  of  ways  that  can  be  used  to 
exchange  that  material  and  email  and  ICQ  are  only  the  two  most  common  programs  to  do  that.  While  solutions 
for  that  issues  do  exist  (there  are  programs  that  allow  only  the  execution  of  authorized  programs,  while  the  rest 
are  locked  out),  the  governments  and  the  school  communities  should  decide  if  they  wish  to  implement  so  hard 
constraints. 


Future  Work 

Our  future  plans  include  the  improvement  of  the  current  design  and  architecture  to  best  fit  the  continually 
emerging  technologies  and  to  therefore  address  successfully  the  new  problems  that  almost  certainly  will  come 
up.  The  listing  of  the  blocked  sites  (everybody  could  see  it)  and  the  ability  to  add  or  remove  sites  to  the 
squidquard  database  (administrators  only)  through  a web  interface  is  a big  priority  already  on  discussion. 
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An  investigation  of  collaborative  problem  solving  using  different 
communications  modes  in  Distance  Education 

Abstract:  This  paper  reports  results  of  a mixed  method,  multiple  case  study  of 
collaborative  problem  solving  by  distance  education  students.  Four  groups  of  students 
were  assigned  to  solve  a multifaceted  task  and  write  a comprehensive  solution  report 
using  the  following  net-based  technologies:  synchronous  audiographics,  synchronous  text 
chat,  asynchronous  voice  conferencing  and  asynchronous  text  conferencing.  Time  on  task 
logs  and  reflections  on  group  process  as  well  as  final  assignment  marks  are  compared 
among  the  four  groups. 

Researchers:  Terry  Anderson,  Ph.D.  Athabasca  Institute  for  Research  on  Open  and 
Distance  Learning 

Liam  Rourke,  Ph.D.  student.  University  of  Alberta 

Research  Rationale:  An  increasingly  large  number  of  technologies  can  be  used  to 
support  collaborative  learning  and  teaching  at  a distance.  Generally  these  technologies 
can  be  divided  into  those  that  support  synchronous  vs.  asynchronous  collaboration  and 
those  that  support  voice  vs.  text  interaction.  Instructors  and  course  developers  are  forced 
to  make  decisions  about  which  medium  or  which  combination  of  media  to  use  when 
assigning  collaborative  activities.  There  is  little  empirical  work  with  actual  students,  in 
natural  conditions,  that  informs  this  decision  making  process.  This  study  will  provide 
insights  based  upon  analysis  of  a variety  of  triangulated  data  as  a first  step  in  developing 
guidelines  and  theory  relevant  to  this  decision  making  process. 

A long  history  of  media  comparison  studies  has  provided  little  evidence  that  any 
particular  media  has  inherent  education  advantage  over  any  other  (Clark,  2000;  Clark, 

1 994),  however  it  is  likely  that  particular  media  attributes  are  more  or  less  congruent  with 
particular  learning  activity  and  instmctional  designs  (Kozma,  1 994)  and  perhaps  with 
different  types  of  learners  (Lara,  Howell,  Dominguez,  & Navarro,  2001).  Many  of  these 
media  comparison  studies  have  been  conducted  in  laboratory  settings  and  lack  ecological 
validity  and  authenticity.  This  study  uses  groups  of  students  who  have  some  prior 
knowledge  and  relationship  with  each  other  (they  are  program  students  in  a large  Masters 
of  Distance  Education  Program),  the  collaborative  project  is  meaningful  in  that  it  relates 
directly  to  the  course  requirements,  and  is  relevant  to  problems  they  face  in  their  own 
workplaces  and  finally  the  study  takes  place  in  the  context  of  a real  distance  education 
program.  Finally,  the  type  of  collaborative  problem  solving  and  report  generation  is  an 
activity  very  common  to  actual  workplace  tasks  assigned  to  distributed  teams  in  business 
and  public  sector  employment. 

Each  of  the  technologies  used  is  web  based  and  commercially  available  at  relatively  low 
cost  - at  least  as  compared  to  the  costs  of  face-to-face  interaction. 

Research  Question:  What  is  the  efficacy  and  the  participants’  perceptions  of 
synchronous/asynchronous  and  text/oral  modes  of  interaction  in  distributed  teams 
engaged  in  collaborative  problem  solving  and  report  generation? 


Study  Design: 

The  study  is  based  on  mixed  method,  multiple  case  study  design  using  each  of  the  four 
media  groups  as  separate  cases  (Yin,  1994).  Four  to  five  students  will  be  randomly 
assigned  to  each  case.  The  technologies  used  will  be: 

1 . Asynchronous  text  based  conferencing  - Bazar 

2.  Asynchronous  voice  based  conferencing  using  - Wimba 

3.  Synchronous  audio  graphics  - Tutor’s  Edge 

4.  Synchronous  chats  -Chat  using  WebCT 

The  groups  will  work  over  a two-week  period  to  research  and  produce  a final  report 
addressing  the  following  problem. 

The  Board  of  SaveMore  College  has  just  developed  a strategic  plan  calling  for  an 
increase  of  50%  in  enrollment  using  distance  education  technologies.  The  Faculty 
Association  has  threatened  to  strike  if  this  new  policy  has  any  adverse  effect  on 
Faculty  work  loads.  The  Vice  President  has  asked  your  team  to  prepare  a 8-10 
page  report  for  both  the  Board  and  the  Association  that  alleviates  the  Faculty 
concerns  and  provides  a set  of  Action  Recommendations  that  allows 
implementation  of  the  new  strategic  plan.  The  report  should  recommend  a 
technology  platform  and  discuss  the  effect  on  the  college  budget  of  your 
recommendations . 

Subjects  will  be  asked  to  refrain  from  using  other  technologies  to  support  collaboration 
except  private  email  (the  extent  of  email  used  will  be  logged). 


Data  Collection  and  Analysis 

The  following  data  will  be  collected,  analyzed  and  results  triangulated: 

• Time  on  task  spent  by  each  individual  (aggregated  to  obtain  group  totals)  group 
gathered  from  time  logs  of  individual  and  group  work. 

• Data  on  perceptions  of  ease,  understandability  and  confidence  in  the  media 
(Steinfield  et  al.,  1997)  obtained  from  3 Likert  like  scales  included  with  the  time 
logs. 

• Perceptions  of  efficacy  of  process  and  output  gathered  from  summary  reflection 
memo’s  from  each  participant  and  analyzed  qualitatively  for  themes  using 
constant  comparative  methodologies  (Strauss  and  Corbin,  1990).  A guide 
identifying  issues  to  be  addressed  will  be  given  to  each  participant  and  serve 
roughly  the  same  function  as  a semi-structured  interview  guide. 

• Final  team  mark  awarded  for  each  group 

• Audio  conference  focus  groups  with  each  group  conducted  at  the  end  of  the 
experiment  to  develop  collaborative  understanding  of  process  and  gain 
understanding  of  divergent  views  of  media  effect  on  that  process  (McNamara, 
1999). 

• Thematic  qualitative  analysis  of  transcripts  and  logs  derived  from  group 
interactions  captured  by  the  system  software(s).  Particular  attention  will  be  paid  to 


appearance  and  frequency  of  group  process  questions,  communications 
breakdowns  and  other  indicators  of  the  quality  of  the  mediated  interaction. 

These  data  sources  will  be  analyzed  to  calculate  the  efficacy  of  each  system  by 
determining  objectively  the  mean  time  spent  on  the  project  by  each  team  and  dividing 
this  by  the  final  mark  to  get  a measure  of  efficiency  of  each  media  in  supporting  the  team 
and  its  work  task. 

Results:  As  the  study  will  not  be  run  until  March  2002,  results  are  not  available  at  this 
time.  It  is  likely  that  there  will  be  little  significant  differences  between  the  groups  based 
on  teacher  assessment  of  final  product,  however  the  time  on  task  and  participants 
perceptions  of  learning  curve,  ease  with  which  organizational  tasks  were  supported, 
reports  of  communications  breakdowns  and  related  organizational  issues  and  perceptions 
of  efficacy  of  the  four  media  types  will  be  rich  in  insights  for  both  researchers  and 
practitioners. 
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Abstract 

Ninety-eight  technology-enhanced  lessons  are  now  available  on-line. 
Teams  of  professors,  K-6  teachers,  and  pre-service  teachers  in  South 
Florida  have  worked  together  to  write,  refine,  and  field  test  lessons  based 
on  the  ISTE  and  Sunshine  State  Standards.  These  tried  and  true 
technology  lessons  are  complete  with  downloadable  resources  and 
assessment  rubrics  for  each  standard  taught.  Math,  Science,  Social 
Studies,  General  Methods,  Curriculum  and  Reading/Language  Arts 
lessons  round  out  the  subject  areas  addressed  in  the  lessons.  There  are 
also  specific  lessons  that  address  special  needs  populations  including 
English  as  a Second  Language  and  Exceptional  Education  students.  The 
lessons  have  been  field  tested  in  classrooms  with  urban  high- need 
students,  limited  English  proficient  students,  and  suburban  students.  An 
overview  of  the  resources  available  will  be  presented  with  opportunities 
for  interaction.  Elementary  Education  Teacher  Education  Professors, 
Curriculum  Developers,  and  K-6  Teachers  can  make  use  of  these  well 
developed,  peer  reviewed  resources. 


Contribution  to  the  Field  of  Educational  Technology 

1 . The  research  involved  in  the  creation  of  these  technology  enhanced  lesson  plans 
reflects  the  cooperative  effort  of  pre-service  teachers,  professors,  and  in-service 
teachers.  Thus,  the  product  is  well  balanced  with  a combination  of  sound 
research  and  field-based  experiences. 
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2.  The  recent  development  of  the  ISTE  standards  has  opened  up  a new  area  of 
curriculum  development  based  on  those  standards.  This  research  project  provides 
materials  that  address  the  need  for  high  quality  field-tested  and  peer  reviewed 
lessons  based  on  the  new  standards. 

3.  The  lessons  developed  reflect  the  up-to-date  availability  of  resources  in  the 
college  and  K-6  classrooms. 

4.  The  lessons  are  based  on  curriculum  standards  as  well  as  ISTE  standards. 

5.  Professors  from  several  institutions  of  higher  education  have  worked  together  to 
create  the  lessons. 

Objectives  and  Outcomes: 

1 . This  session  will  address  the  need  for  high  quality  technology  enhanced  lessons 
based  on  both  State  and  National  Standards  by  providing  on-line  resources  to 
attendees. 

2.  This  session  will  supply  assessment  instruments  to  evaluate  the  use  of  technology 
for  all  lessons  that  address  the  State  and  National  Standards. 

3.  Technology  enhanced  lessons  at  the  K-6  level  will  be  shown  for  Math,  Science, 
Social  Studies,  Reading/Language  Arts,  Curriculum,  General  Methods,  ESE  and 
ESOL. 

Importance: 

1 . As  professors  in  schools  of  education  seek  to  integrate  technology  into  their 
teacher  education  classes,  they  can  use  these  resources  to  assist  them. 

2.  As  K-6  teachers  seek  to  utilize  technology  in  their  classes,  they  will  be  able  to 
find  resources  on  the  curriculum  area  of  choice  by  using  the  search  engine 
presented  based  on  the  Sunshine  State  Curriculum  Standards. 


Associated  Web  Resources: 

The  lesson  plans  produced  by  this  group  can  be  found  at  www.beaconlc.ora 
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Abstract  : In  this  paper  we  propose  adaptive  hypermedia  educational  system  using  XML 
technologies  such  as  XML,  XSL,  XSLT,  and  XLink.  Adaptive  systems  are  capable  of 
altering  the  presentation  of  the  content  of  the  hypermedia  on  the  basis  of  a dynamic 
understanding  of  the  individual  user.  The  user  profile  can  be  collected  in  a user  model, 
while  the  knowledge  about  the  domain  can  be  represented  in  the  form  of  a concept  based 
domain  model.  So  we  have  defined  two  different  markup  languages  using  XML.  And  for 
adaptivity  of  system,  adaptive  presentation  of  the  data  comes  using  XSL  and  adaptive 
navigation  of  link  comes  using  XLink. 

Introduction 

With  the  rapid  advances  in  WWW  interactive  technologies,  the  use  of  Internet-based  distance  learning  tools  is 
rapidly  growing.  Most  of  these  products  are  nothing  more  than  a network  of  static  hypermedia  pages.  In  fact  the 
domain  knowledge  implicit  in  traditional  educational  hypermedia  is  well  defined  and  carefully  structured  and 
provides  an  only  learning  path  optimal  for  a generic  average  student  [Ekll997].  Otherwise,  a Web  application, 
which  is  designed  with  a particular  class  of  users  in  mind  may  not  suit  other  users.  Moreover,  "static" 
hypermedia  assume  that  the  users  can  make  sensible  decisions  about  when  to  use  navigation  tools,  about  when 
to  proceed  in  the  learning  process,  about  when  they  need  an  explanation,  etc.  [Brul998].  This  could  be  a 
problem  for  those  users  who  access  the  hypermedia  through  the  Internet  and  that  can’t  have  a teacher  at  their 
disposal. 

Adaptivity  is  the  feature  of  hypertext  and  hypermedia  that  allows  one  to  adapt  the  contents  to  the  user 
needs[Ada2001].  Adaptive  hypermedia  systems  modify  the  presentation  of  the  domain  knowledge  according  to 
the  user  profile.  This  mechanism  permits  to  personalize  the  Hypermedia  in  terms  of  contents  and  of  navigation 
tools  for  each  user. 

The  focus  of  this  paper  is  cn  the  general  architecture  and  the  implementation  issues  of  a adaptive  educational 
hypermedia  system.  This  system  has  been  implemented  by  using  the  functionalities  provided  by 
XML(XSL,XLink)  in  order  to  stress  the  separation  of  the  information  content  from  presentation.  The  basic  idea 
is  to  define  a general  hypertext  structure  in  order  to  create  pages  dynamically  using  a structured  description  of 
the  domain  knowledge  and  a model  of  the  current  user. 

Adaptive  hypermedia  technoiogies  and  XML  techniques 

Adaptive  hypermedia  systems  build  a model  of  the  goals,  preference,  and  knowledge  of  each 
individual  user,  and  use  this  model  throughout  the  interaction  with  the  user,  in  order  to  adapt  to  the 
needs  of  that  user[Bru  2001].  In  adaptive  hypermedia  literature  they  are  referred  respectively  as 
adaptive  presentation  and  adaptive  navigation  support 

XML  is  designed  to  be  the  data  format  for  the  web,  and  at  the  time  of  writing  is  being  used  by  many 
thousands  of  web-based  applications.  XML  is  a bit  like  HTML,  the  format  used  for  web  documents, 
but  it’s  focused  on  representing  data,  rather  than  describing  the  presentation  of  data.  Presentation  of 
the  data  comes  using  XSL  and  navigation  of  hyperlink  comes  later,  using  XLink.  Also  XML  is 
independent  from  a specific  platform  and  operating  system  and  provides  the  DOM(Document  Object 
Model)  as  a platform-  and  programming  language-independent  API  for  the  development  of  the 


necessary  tools.  XML  grammars  can  be  constructured  in  a way  that  they  easily  can  be  extended  by 
new  or  alternate  elements.  There  already  exists  a metadata  scheme  for  teaching  resource  - IMS 
projects[IMS2001]. 

Design  of  adaptive  educational  hypermedia  system 

The  adaptive  hypermedia  system  architecture  (see  Figure  1)  consists  of  three  main  components.  Such  as  a 
student  model  that  represents  the  student  knowledge,  a domain  model  that  contains  the  domain  knowledge 
information  structured  in  topics  and  relationships,  and  a adaptation  model  that  is  responsible  for  creating  pages 
dynamically  by  using  the  information  contained  both  in  the  domain  and  in  the  user  modules. 

XML  is  a meta-language  that  allows  us  to  create  specialized  markup  languages  for  specific  purposes 
[XML  1999].  For  example,  it  is  possible  to  create  a tag  <TOPIC>  (and  the  closure  tag  </TOPIC>)  that  identifies 
a particular  knowledge  element. 


Figure  1:  System  architecture 


For  implementing  our  adaptive  hypermedia  system  we  have  implemented  two  languages: 

the  Domain  Structure  Markup  Language(DSML)  in  order  to  describe  the  knowledge  general  structure, 
the  Student  Model  Markup  Language(SMML)  used  by  the  system  in  order  to  describe  the  student 
model  status. 

In  particular  we  have  implemented  two  different  Document  Type  Definitions  (DTDs)  that  define  the  fixed 
structure  of  the  two  above  mentioned  markup  languages.  In  the  following  two  subsections  we  will  give  some 
examples  of  the  new  tags  that  we  have  introduced.  Finally  in  the  last  subsection  we  describe  the  Adaptation 
Model. 

Domain  Model 

To  be  dynamically  processed,  the  domain  knowledge  has  been  structured  into  topics  and  each  topic  is 
associated  to  a page.  Each  topic  is  characterized  by:  1)  a title,  2)  a set  of  main  keywords,  called  concepts,  3)  a 

set  of  conditioned  relationships,  4)  a set  of  explanations,  5)  a set  of  possible  examples,  exercises,  etc. 

Therefore  a link  from  a page  to  another  one  represents  a relationship  between  the  two  topics.  We  have 
considered  six  different  types  of  relationships  between  topics . 

super/sub  that  is  a bi-directional  relationship  between  a topic  and  all  its  sub-topics 
next  that  is  a relationship  between  a topic  and  the  next  topic  to  learn 

contrary  that  is  a bi-directional  relationship  between  a topic  and  the  "opposite"  one.  example  that  is  a 

relationship  between  a topic  and  an  example  that  explains  it 

widening  that  is  a relationship  between  a topic  and  a study  in  depth  of  it 

exception  that  is  a relationship  between  a topic  and  an  exception  or  a particular  case 
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Figure  2:  example  of  domain  knowledge  structure 

The  first  relationship  defines  the  hierarchical  structure  of  the  knowledge  domain.  Knowledge  is  represented  as  a 
tree  in  which  each  node  identifies  a topic  and  a root  topic  is  defined.  The  next  relationship  represents  the 
sequence  of  topics  suggested  by  the  teacher.  All  the  other  relationships  create  irregular  connections  between 
topics  that  transform  the  original  tree-scheme  in  a graph.  The  student  can  navigate  the  graph  following  all  the 
different  types  of  relationships  and  defining  a personal  learning  path. 

Each  relationship  is  associated  to  a cluster  of  conditions  that  defines  which  kind  of  requirements  the  student 
needs  to  follow  the  link.  If  the  student  model  satisfies  all  the  conditions  the  link  is  displayed.  The  link 
associated  to  the  super/sub  relation  always  appears  without  requirements.  All  the  other  links  can  be  conditioned, 
and  in  particular  the  teacher  links  can  have  different  arriving  pages  in  function  of  the  student  model  status.  In 

this  way  each  node  is  associated  to  a set  of  next  links,  each  one  valid  aider  a different  set  of  conditions.  The 

condition  scheme  is  organized  as  follows. 

Each  condition  is  divided  in  different  condition  terms  composed  with  and  and/or  operators.  We  have  considered 
that  students  are  classified  in  three  stereotypes  and  we  have  associated  to  each  argument  that  needs  to  be  known 
a stereotype  and  a certain  minimum  knowledge  level.  So  that  a condition  term  can  be  read  as:  "if  the  student  of 
a level  X have  got  a knowledge  level  Y on  the  argument  A,  then  the  condition  term  is  true".  By  composing 
condition  terms  with  and/or  operators  we  can  obtain  complex  didactical  requirements. 

The  DTD  that  describes  DSML  contains  the  definition  the  following  tags: 

<ADAT></ADAT>  is  the  root  of  the  adaptive  hypertext.  The  <ADAT>  tag  can  contain  different  <TOPIC>  tags. 
By  using  <ADAT>  attributes  the  author  can  also  define  the  TITLE,  the  AUTHOR  name  and  other  proprieties  of 
the  hypermedia.  <TOPIC></TOPIC>  represents  a topic  and  is  identified  by  using  the  attributes  TITLE  and  ID; 
each  <TOPIC>  can  contain  different  relational  tags  (<PARENT>  and  <SUBTOPIC>  described  below), 
different  conditioned  elements  (introduced  by  using  the  <CONDITION>  tag)  and  different  <CONCEPT>  tags. 
<C0ND1T10N></C0NDITI0N>  defines  the  requirement  needed  to  follow  a certain  link  or  to  give  a certain 
explanation.  Nested  <CONDITION>  tags  can  be  used  to  realize  and  between  different  conditions;  an  attribute 
OR  associated  to  a <CONDITION>  tag  is  used  to  realize  an  or  between  this  condition  and  the  following  one. 
Each  set  of  nested  conditbns  contains  an  <EXPLANATION>  or  a relational  tag.  <EXPLANATION> 
</EXPLANATION>  contains  the  real  explanation  of  the  topic.  The  <EXPLANATION>  tag  can  contain  any 
HTML  tags.  This  choice  lets  one  use  different  media  to  give  the  explanation  to  the  student. 
<CONCEPT></CONCEPT>  defines  a keyword  associated  to  the  <TOPIC>  tag  that  contains  the  <CONCEPT>. 
<EXAMPLE></EXAMPLE>  defines  a particular  kind  of  topic  that  can  be  used  as  an  example  of  a certain 
<TOPIC>;  a <EX-LINK>  relationship  links  this  <TOPIC>  to  the  <EXAMPLE>.  <EXERCISE></EXERCISE> 
defines  an  exercise.  The  <EXERCISE>  tag  contains  a <TEXT>  a <SOLUTION>  and  an  <HELP>tag. 

The  following  code  fragment  represents  a simple  example  of  the  DSML  tags. 

<TOPIC>  Topic  Title 

<CONCEPT>  Keyword</CONCEPT> 
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<C0ND1T10N  USER_LEVEL="3"> 

<EXPLANAT10N> 

HTML  Explanation  description 

</EXPLANAT10N> 

</C0NDlT10N> 

</TOPlC> 

Three  different  type  of  tags  are  used: 

<CONCEPT>:  that  includes  the  keyword  related  to  the  topic. 

<C0ND1T10N>:  this  tag  indicates  a didactical  requirement.  In  this  case  there  is  a very  simple  condition  based 
on  the  student’s  knowledge  level.  Here  an  expert  level  (USER_LEVEL=3)  is  required.  The  student  model, 
illustrated  in  the  next  section,  manages  values  of  the  USER_LEVEL  variable. 

<EXPLANAT10N>:  this  tag  includes  the  real  explanation  of  the  topic.  Text  between  this  tag  and  its  closure  can 
be  written  in  HTML. 

Other  tags  are  defined  in  order  to  describe  relationships  between  the  topics  and  in  particular: 

<SUPERT0P1C>  and  <SUBT0P1C>  are  used  torepresent  the  super/sub  relationship.  We  have  used  two  tags  to 
describe  the  relation,  one  for  each  direction.  <NEXT>  is  used  to  define  the  next  relationship  and  represents  the 
path  suggested  by  the  teacher;  the  previous  topic  is  dynamically  computed.  <W1DEN1NG>  defines  a widening 
relationship.  <EX  -L1NK>  represents  the  relationship  between  a <T0P1C>  and  an  <EXAMPLE>  related  to  it. 
<CONTRARY>  defines  a contrary  relationship.  <EXCEPT10N>  represents  an  exception  relationship. 

Student  Model 

The  student  model  is  a structure  maintained  by  the  system  with  the  aim  of  being  able  to  respond  to  the 
individual  student's  learning  style  and  deliver  customized  instruction.  There  are  different  ways  to  approach  the 
student  model  problem.  The  most  commonly  used  student  modeling  technique  in  adaptive  hypermedia  systems 
is  the  overlay  model[Ekll997].  In  this  model,  the  student  knowledge  is  considered  an  overlay  of  the  domain 
knowledge  and  is  frequently  represented  as  a set  of  concept/value  pairs.  For  each  domain  model  concept  the 
student  model  stores  some  values  that  estimate  the  student  knowledge  level  of  this  concept.  This  model  assumes 
that  the  student  knowledge  is  a subset  of  the  knowledge  domain  described  by  the  teacher  and  cannot  capture 
different  knowledge  conception.  In  spite  of  this,  the  overlay  model  is  commonly  used  in  the  adaptive  hypertext 
field  because  the  content  of  the  adaptive  hypertext  itself  can  be  considered  as  a description  of  the  entire  domain 
knowledge.  This  involves  that  the  student  can  be  represented  simp^  overlaying  the  hypertext  structure. 

Other  adaptive  hypermedia  systems  use  simple  stereotype  model  [Ekll997].  Student  knowledge  is  represented 
as  a set  of  concept/value  pairs  (as  in  overlay  model),  but  the  values  are  not  completely  independent.  The  student 
can  be  assigned  to  one  or  more  class  (called  stereotypes)  and  each  class  is  identified  by  a fixed  set  of 
concept/value  pairs. 

On  the  other  hand  the  most  part  of  the  Intelligent  Tutoring  System  [And  1995]  are  developed  using  a 
representation  of  the  student's  knowledge  based  on  student  bugs  and  misconception.  The  buggy  model  represent 
the  student's  knowledge  in  terms  of  deviations  from  an  expert's  knowledge.  The  system  classifies  each  student 
error  and  then  uses  this  information  to  predict  the  student’s  behavior  in  future  situations.  The  buggy  model  is 
really  useful  in  leaming-by-doing  systems  in  which  the  system  needs  to  monitor  a student  activity  (as  for 
example,  exercises  solving  procedure). 

To  implement  the  student  model  we  have  used  a mix  of  the  overlay  model  and  of  the  stereotype  model. 

The  overlay  model  has  been  implemented  associating  to  each  topic  a value  that  weights  the  level  of  the  student 
knowledge  of  that  topic.  This  evaluation  is  given  measuring  the  time  spent  reading  the  topic  and  the  number  of 
visits.  Each  time  the  student  requests  a page,  then  the  associated  value  is  updated. 

We  have  also  classified  possible  students  into  three  different  stereotypes:  expert  user,  medium-level  user  and 
beginner  user.  This  evaluation  refers  to  all  the  arguments  of  the  domain  model  and  is  a general  assessment  of 
the  student  ability.  We  have  used  a scheme  based  on  three  stereotypes  referring  to  different  system  described  in 
well-known  literature  [Wenl987,  Ekll997].  Anyway  our  system  can  recognize  each  set  of  integer  values  as  a 
stereotype  scheme. 

The  student  model  is  realized  using  two  DTDs:  the  former  defines  a structure  that  contains  the  user's  logins  and 
the  related  passwords,  the  latter  defines  the  structure  for  the  SMML  in  order  to  describe  the  personal  student 
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model  file.  The  DTD  that  describes  SMML  contains  the  definition  of  the  following  tags: 
<MODEL></MODEL>  defines  all  the  visited  topics  described  by  using  different  <VISIT>  tags  and  a 
<LEVEL>  tag.  <VISIT>  is  used  to  represent  a visited  topic;  the  argument  is  specified  using  the  TOPIC 
attribute.  <LEVEL>  </LEVEL>  and  <LEVELT>  </LEVELT>  define  respectively  the  general  level  of  the 
student  knowledge  using  the  DEGREE  attribute  and  the  level  of  knowledge  specific  of  the  topic.  <LAST> 
defines  the  last  visited  topic.  <NUMVISIT>  </NUMVISIT>  defines  the  number  of  times  that  the  student  has 
visited  this  topic  page 

A simple  example  of  student  model  is  the  following: 

<MODEL> 

<LAST  topic=’T’> 

<LEVEL  degree="0"> 

<VISIT  topic=’T"> 

<LEVELT>0</LEVELT> 

<NUMVISIT>1  </NUMVISIT> 

</VISIT> 

</MODEL> 

The  example  shows  the  student  model  of  a beginner  level  user  f^EVEL  DEGREE=0).  In  the  <MODEL> 
structure  there  is  only  one  VISIT  tag  meaning  that  the  student  has  just  run  the  hypermedia  and  has  visited  only 
the  first  page  identified  by  the  topic='T".The  LAST  visited  topic  is  obviously  the  number  "1".  The  example 
shows  the  student  model  of  a beginner  level  user  O^EVEL  degree="0").  The  nested  tags  <LEVELT>  and 
<NUMVISIT>  identify  respectively  the  level  of  knowledge  of  the  topic  md  that  the  student  visits  this  topic 
page  for  the  first  time. 

It  is  worth  to  note  that  the  student  models  are  automatically  created  and  managed  by  the  system  so  that  the 
SMML  is  used  only  by  the  system  to  store  relevant  information. 

Adaptation  Model 

The  Adaptation  Model  (AM)  creates  the  pages  of  adaptive  hypermedia  on  the  basis  of  the  Knowledge  Domain 
Model  and  the  Student  Model.  The  general  architecture  of  AM  consists  of  four  main  components:  the  User 
Action  Management  Module(UAMM),  the  Adaptivity  Management  Module  (AMM),  the  Domain  Model 
Management  Module(DMMM)  and  the  User  Model  Management  Module  (UMMM). 

The  UAMM  interprets  the  student's  behaviors  sending  information  to  the  other  modules  (e.g.  possible  changes 
to  Student  model  status  to  the  UMMM;  student's  request  of  a page  to  the  AMM). 

The  DMMM  retrieves  topics  from  the  knowledge  domain  model  on  the  basis  of  the  student's  requests  and 
interactions.  The  UMMM  updates  the  student  model  sending  the  related  information  to  the  other  modules.  The 
AMM  adapts  the  presentation  of  the  domain  knowledge  to  the  user;  creating  hypermedia  pages  dynamically.  In 
particular  each  time  a condition  is  found,  the  AMM  asks  the  UMMM  the  Student  Module  status  in  order  to 
solve  the  condition  and  create  the  new  page  on  the  basis  cf  the  topics  already  known  by  the  student  (provided 
by  the  DMMM)  and  of  his  knowledge  level. 

Implementation  and  Results 

We  have  described  the  developing  of  two  new  markup  languages  that  allow  us  to  describe  complex  knowledge 
structures.  It's  worth  to  note  that  the  system  is  designed  in  order  to  realize  adaptive  educational  hypermedia,  but 
it  can  be  used  for  every  kind  of  adaptive  hypermedia  thanks  to  the  flexibility  of  the  used  structures. 

Our  adaptive  hypermedia  system  has  been  implemented  using  a client-server  scheme  based  on  HTTP.  The 
server  application  that  automatically  produces  personalized  XML  pages  has  been  described  in  the  previous 
sections.  The  client  is  a common  used  browser,  Netscape  and  Internet  Explorer  that  can  evaluate  and  display 
XML  pages.  Before  starting  the  navigation  in  the  adaptive  hypermedia  system,  the  user  has  to  identify  himself 
with  a username  and  a password.  The  server  uses  this  information  to  store  the  appropriate  student  model  (or  to 
create  a new  student  model  for  a new  user).  Then,  each  time  the  student  asks  for  a new  page,  the  browser  asks 
for  it  to  the  server  side  application  that  compute  the  content  of  the  page  dynamically  on  the  basis  of  the  student 
model.  A Java  applet,  which  communicates  to  the  server  application  all  the  events  generated  by  the  browser, 
updates  the  student  model  status. 

We  are  currently  testing  an  adaptive  hypermedia  system  on  programming,  created  with  XML  dynamic  support, 
that  is  developed  for  a university  undergraduate  program  course.  This  test  has  underlined  different  aspects  of 
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the  system  that  need  to  be  improved 
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Figure  3:  A screen  of  the  adaptive  educational  hypermedia  on  programming 


In  particular  we  are  working  in  order  to  improve  the  student  model  and  the  user  interface. 

We  are  working  on  the  student  model  to  insert:  an  entry  test  in  order  to  evaluate  the  initial  level  of  knowledge 
of  the  student.  This  feature  can  improve  the  value  of  the  DEGREE  attribute  to  the  LEVEL  tag  that  implements 
the  stereotype  student  model.  A set  of  tests  related  to  each  main  topic  in  order  to  modify  the  LEVEL  tag 
associated  to  each  topic.  This  feature  can  improve  the  overlay  student  model. 

Evaluation  tests  can  be  closed  answer  tests,  exercises  that  need  to  be  solved,  every  different  type  of  evaluation 
procedure  that  can  give  back  an  integer  evaluation.  The  solving  procedure  associated  to  the  exercises  of  the 
evaluation  test  can  be  monitored  using  every  type  of  user  model.  Moreover  we  are  working  on  new  user 
interface  features  to  improve  the  navigation  tools  in  the  adaptive  hypermedia  system[Pasl992]. 

Further  works  will  consider  the  implementation  of  an  authoring  tool  that  will  allow  teachers  to  structure  the 
domain  knowledge  without  introducing  the  tags  directly.  This  tool  should  be  a graphic  editor  for  developing 
adaptive  hypermedia  courseware. 
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Abstract:  A learning  environment  is  described  in  which  students  collaborate  in  small  groups  to 
develop  screen  movies  in  which  they  use  a statistical  cognitive  tool  to  interpret  published  research 
and  to  demonstrate  their  understanding  of  least  squares  statistical  concepts.  Evaluation  data  are 
reported  which  indicate  that,  although  some  groups  thrive  in  this  environment,  other  struggle  to 
cope.  Enhancements  are  proposed  based  on  the  outcome  of  the  evaluation. 


The  teaching  of  applied  statistics  to  social  science  undergraduates  is  a much  discussed  topic  (Becker,  1996),  no  doubt 
because  it  presents  a disproportionate  challenge  to  students,  many  of  whom  do  not  have  extensive  mathematical 
preparation.  Although  various  teaching  methods  have  been  adopted,  many  use  prepared  data  sets  in  which  relationships 
between  variables  are  to  be  revealed  (and  tested  for  significance)  through  analyses  undertaken  by  the  student  (****).  In 
recent  years  the  data  sets  have  been  related  to  research-based  scenarios  (e.g.,  Derry  et  al.,  1995;  Fischer,  1996; 
Thompson,  1994),  and  the  analyses  are  performed  with  one  of  a growing  number  of  computer  packages  which  have 
user-friendly  interfaces  and  powerful  graphing  capabilities  (e.g.,  DataDesk,  SAS/JMP,  SPSS,  Statistica,  Models  *n  'Data, 
Statview). 

The  approach  upon  which  the  present  work  is  centred  shares  several  of  these  features  (exercises  anchored  to  published 
research,  use  of  a computer  program  in  the  learning  process,  heavy  emphasis  on  graphical  representation),  but  it  differs 
in  several  important  respects: 

• the  data  are  created  and  altered  by  the  student  using  a graphical  rather  than  spreadsheet  interface 

• the  emphasis  is  upon  modelling — and  playing  ‘what  if  with — the  relationships  between  the  variables  represented  in 
the  program 

• the  students"  task  is  to  demonstrate  an  understanding  of  statistical  and  research  concepts  by  recording  a screen 
movie,  with  voice-over,  in  which  they  ‘teachback’  what  the  statistical  and  research  concepts  mean  and  why  they  are 
relevant  to  the  research  scenario 

• the  students  work  in  small  groups  (2-4)  to  develop  and  record  their  teachback  movies. 

This  paper  provides  details  of  the  teaching/  learning  arrangements  for  this  approach  to  statistics,  and  reports  a formative 
evaluation  of  the  learning  process  and  outcome. 


The  Computer  Program 

The  program  upon  which  the  learning  process  is  centred  is  Bivar Describe,  a Macintosh  program  designed  by  the  first 
author.  This  program  enables  users  to  explore  the  least  squares  properties  of  correlation,  regression  and  one-way 
analysis  of  variance  (anova).  Consistent  with  calls  for  regression  and  anova  to  be  taught  as  variants  of  the  general  linear 
model  (Thompson,  1993),  anova  is  treated  as  a special  case  of  multiple  regression,  and  the  focus  is  upon  how  the 
relational  structure  of  the  data  is  modelled  by  the  least  squares  approach,  not  on  sampling  distributions  and  statistical 
significance.  Thus  the  program  is  intended  to  redress  some  pervasive  misconceptions,  namely:  that  anova  and 
correlation  are  different  procedures  (Keppel  & Zedeck,  1989;  Thompson,  1993);  that  the  concept  of  ‘explained 
variance’  (or  ‘predictable  variance’)  only  applies  to  correlational  data  (Hays,  1981;  Huberty,  1987);  that  statistical 
significance  is  more  important  to  research  interpretation  than  effect  size  (Rosenthal  & Rosnow,  1985;  Thompson,  1993; 
Wilkinson  et  al.,  1999);  and  that  omnibus  anova  adequately  models  relational  structure  in  a one-way  design  (Rosnow  & 
Rosenthal  1989;  Thompson,  1993). 
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As  already  noted,  this  program  differs  from  statistical  analysis  programs  because  it  facilitates  the  creation  of  data  (by 
clicking  the  ‘dot’  cursor  on  the  data  plot)  and  the  alteration  of  data  (by  dragging  data  points  to  new  locations,  or  altering 
variances  with  a ‘variance  change’  tool).  An  example  of  the  main  plot  window  in  which  a data  point  has  been  dragged 
to  an  outlying  position  is  provided  in  Figure  1. 

Figure  1.  The  main  plot  window  of  BivarDescrihe  depicting  a positive  linear  relationship  between  two  variables, 
Severity  of  Eating  Problem  and  Neuroticism,  but  with  one  data  point  dragged  to  an  outlying  position. 


Some  of  the  additional  features  of  this  program  are: 

• the  strength  of  the  relationship  or  effect  size  (the  Proportion  of  Predictable  Variance)  is  represented  by  a vertical 
meter  that  changes  dynamically  as  points  are  added  or  altered 

• users  may  access  additional  information  about  applicable  statistics  by  opening  mini-windows  in  which  the  statistics 
are  defined  algebraically  and  illustrated  numerically 

• there  is  a dynamic  coupling  between  the  graphical  and  numerical  states  of  the  program  (points  selected  or  changed 
in  one  are  selected  or  changed  in  the  other) 

• users  may  use  the  Enquiry  Tool  to  see  how  scores  are  predicted  or  to  view  the  deviations  with  which  predicted  and 
residual  variances  are  calculated 

• the  user  can  specify  a linear  prediction  and  compare  it  with  the  least  squares  regression  line 

• a correlational  design  can  be  converted  to  an  anova  design 

• contrasts  between  the  conditions  of  an  anova  design  can  be  specified  graphically  and  turned  on  and  off  to  see  how 
they  contribute  to  the  predictable  variance 

• anova  as  a special  cases  of  regression  can  be  seen  in  additional  window  which  shows  the  prediction  of  scores  from 
the  weights  defining  each  contrast. 


The  Course  Context 

The  course  in  which  the  present  project  is  embedded  is  a one  semester  second  year  course  in  research  methods  for 
psychology  students  at  The  University  of  Southern  Queensland.  The  content  includes  a wide  range  of  research  design 
and  methods  concepts,  for  which  the  supporting  text  is  Shaughnessy,  Zechmeister,  & Zechmeister,  (2000),  as  well  as 
the  statistical  concepts  hvolved  in  bivariate  regression,  multiple  regression  and  analysis  of  variance,  for  which  the 
supporting  text  is  Keppel  & Zedeck  (1989).  In  addition,  there  are  extensive  course  notes  (the  course  is  also  offered  in 
external  mode)  and  lectures  and  tutorials.  The  exercises  with  which  BivarDescrihe  is  used  comprise  25%  of  the 
assessment  for  the  course.  Students  are  prepared  for  the  exercises  during  several  tutorials  in  one  of  which  they  develop 
and  rehearse  a simple  teachback  movie.  Students  also  have  access  to  demonstration  movies  made  with  BivarDescrihe 
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by  the  authors  (about  8 hours  in  total  duration).  These  movies  are  based  on  a research  study  not  included  in  the 
exercises,  and  their  topic  structures  differ  from  those  required  by  the  exercises  (to  avoid  direct  modelling). 


The  Learning  Process 

The  intended  learning  process  consists  of  several  interdependent  activities: 

• The  ‘priming’  of  relevant  statistical  and  research  concepts  by  viewing  the  demonstration  movies  and  using  them  as 
analogues,  not  direct  models,  for  the  exercise  movies. 

• The  construction  of  statistical  understanding  using  BivarDescribe  as  a cognitive  tool  with  which  “[IJeamers 
themselves  function  as  designers  using  technologies  as  tools  for  analysing  the  world,  accessing  information, 
interpreting  and  organizing  their  personal  knowledge,  and  representing  what  they  know  to  others.”  (Jonassen  & 
Reeves,  1996,  p.  694). 

• The  mapping  back  and  forth  between  real  research  variables  and  relationships  and  their  representations  in  the 
statistical  domain  (Derry,  Levin  & Schauble,  1995;  Laurillard,  1993) 

• Collaboration  in  small  groups  to  encourage  the  development  and  refinement  of  understanding  (Roschelle,  1992) 


The  Intended  Learning  Outcomes 

The  learning  outcomes  of  most  relevance  here  are  focused  on  the  ability  of  students  to  understand  and  use  least  squares 
statistical  methods  to  interpret  data  within  a realistic  research  setting,  as  manifested  in  the  teachback  movies  created  by 
each  group.  ‘Understanding’  in  this  context  has  several  aspects:  (a)  the  ability  to  map  a research  question  into  a 
statistical  framework  and  vice  versa;  (b)  the  ability  to  translate  flexibly  between  the  graphical,  algebraic  and 
arithmetical  representations  of  statistical  relationships  with  a clear  sense  of  why  they  correspond  and  what  they  mean 
(i.e.,  the  understanding  should  be  ‘relational’  rather  than  ‘instrumental’  — Skemp,  1976);  (c)  the  ability  to  explain  how 
basic  least  squares  concepts  apply  to  correlation  and  anova  (Thompson,  1993);  and  (d)  the  ability  to  ‘perform’ 
understanding  rather  than  to  reproduce  it  (Perkins  & Blythe,  1994)  as  a consequence  of  the  ‘teachback’  format  (Pask, 
1976). 

Evaluation 

The  aim  of  the  evaluation  reported  here  was  not  to  determine  whether  the  learning  model  outlined  above  is  better  than 
others  Kported  in  the  literature,  but  rather  to  determine  whether  it  is  functioning  much  as  intended  and  to  suggest 
improvements  (Bain,  1999).  Accordingly,  data  were  collected  about  the  learning  process  (by  video-recording  the 
discussions  of  participating  groups  and  coding  for  key  features  of  the  process)  and  about  the  learning  outcome  (by 
coding  the  teachback  movies  submitted  by  each  group). 


Participants 

Eighteen  groups  (54  students)  completed  the  assessment  for  the  course,  of  which  10  groups  agreed  to  take  part  in  the 
evaluation  (i.e.,  agreed  to  have  their  discussions  video-taped).  Four  of  the  ten  participating  groups  dropped  out  of  the 
evaluation  early,  leaving  6 groups  from  which  the  present  data  were  obtained.  Most  groups  comprised  3 students,  the 
range  being  from  2 to  4. 


Teachback  Exercise 

Two  substantial  exercises  comprised  the  assessment  for  the  teachback  component  of  the  course,  for  each  of  which 
several  teachback  movies  were  to  be  produced.  The  data  reported  here  are  for  the  first  of  four  movies  in  the  first 
exercise  for  which  the  research  scenario  was  based  on  the  article  by  McFarland  (1989).  The  scenario  asked  students  to 
contemplate  a study  examining  the  relationship  between  religious  orientation  (as  measured  by  the  Quest  self-report 
scale)  and  a scale  measuring  a general  tendency  to  discriminate  (piscrimination)  consisting  of  attitudes  toward  several 
minority  groups.  The  expected  relationship  was  inverse,  given  the  findings  reported  by  McFarland — i.e.,  the  higher  the 


111 


score  on  Quests  the  lower  the  Discrimination  score — but  this  was  not  stated  in  the  scenario  outline.  The  first  movie  was 
specified  as  follows: 

Use  BivarDescribe  to  model  the  data  pattern  (relationship)  predicted  in  the  scenario,  showing  the  appropriate  variable 
labels  and  scales.  Initially  assume  a fairly  weak  relationship,  and  show  what  that  might  look  like.  Then: 

• refer  to  the  Proportion  of  Predictable  Variance  (PPV)  indicator  to  describe  and  explain  the  strength  of  the 
relationship  that  you  have  created 

• open  the  PPV  window  and  explain  what  the  predicted  scores  are  and  what  the  proportion  of  predictable  variance 
means 

• apply  the  Variance  Change  tool  to  the  grand  mean  (using  increasing  as  well  as  decreasing  modes)  and  describe  and 
explain  (a)  the  changes  in  the  data  and  (b)  what  happens  to  the  Least  Squares  Coefficient  (LSC)  and  PPV  indicators 

• make  sure  you  tie  all  your  explanations  back  to  the  scenario  study  and  discuss  the  meaning  of  the  data  patterns  in 
terms  of  the  scenario  variables. 


Procedure 

Our  intention  was  to  record  two  learning  sessions,  one  early  and  one  late  in  each  group’s  preparations  for  an  exercise  so 
that  improvements  during  learning  could  be  documented.  However,  for  logistical  reasons  this  proved  impossible  to 
arrange,  so  the  data  reported  below  are  for  one  session  conducted  part  way  through  the  preparation  for  the  exercise.  The 
session  was  recorded  with  a split  screen  format  that  enabled  the  group  members  to  be  seen  as  well  as  the  details  of 
activity  on  the  computer  screen.  The  teachback  movies  comprising  an  exercise  were  recorded  when  the  group  indicated 
that  it  was  ready  to  do  so,  typically  2-3  weeks  after  preparations  began.  The  recording  was  managed  by  a tutor  who 
operated  the  recording  software  {SnapzPro  2),  leaving  the  group  free  to  concentrate  on  producing  its  teachback  movies. 
The  video  recordings  and  teachback  movies  were  coded  by  a project  assistant  using  a 5 point  scale  ranging  from  ‘not  at 
all’  through  ‘poor’,  ‘limited’  and  ‘adequate’  to  ‘excellent’,  where  the  descriptors  for  each  point  on  the  scale  were 
contextualised  to  the  characteristic  being  rated.  The  coding  process  involved  repeated  checks  with  the  senior  author 
about  the  codes  assigned  to  each  case. 


Findings 

The  means  reported  in  the  last  column  of  Tabic  1 indicate  that  the  groups  were  better  able  to  effect  the  intended 
social  aspects  of  the  learning  process  (discussion  and  collaboration)  than  they  were  able  to  use  the  resources  to 
construct  their  understanding  of  statistical  concepts  (building  understanding  with  BivarDescribe  and  correcting 
misunderstandings  being  rather  limited).  However,  these  general  trends  are  qualified  by  substantial  differences  between 
the  groups,  two  (F,  G)  being  conspicuously  sound,  two  adequate  (E,  J)  and  two  others  obviously  struggling  to  cope  (D, 
K).  Similar  patterns  were  evident  when  general  and  specific  aspects  of  understanding  were  coded  (Table  2).  In  this 
case  it  was  possible  to  use  the  same  scales  to  code  understanding  during  the  learning  processes  as  well  as  in  the 
teachback  movies,  and  the  mean  data  and  ranges  are  reported  in  Table  2.  Although  the  mean  values  on  both  sets  of 
scales  were  mostly  in  the  ‘limited’  to  ‘adequate’  range,  this  pattern  masks  the  fact  that  there  were  marked  differences 
between  the  groups  much  as  was  evident  in  the  process  scales:  the  ranges  provide  some  idea  of  the  performance 
differences  involved,  although  not  the  consistent  patterns  of  group  performance  (space  limitations  prevent  reporting  of 
the  group  data). 

Another  aspect  of  the  consistency  of  group  performance  is  the  degree  of  correspondence  between  the  understanding 
‘profiles’  (pattern  of  scores  on  the  understanding  scales)  obtained  during  the  learning  process  and  evident  in  the  learning 
outcome.  An  index  of  profile  similarity  is  provided  by  the  Euclidean  Distance  measure  (Table  3)  which  varies  from 
zero  when  the  profiles  are  identical  to  an  empirical  maximum  when  the  profiles  are  most  dissimilar.  As  is  evident  from 
Table  3,  the  understanding  profiles  of  the  groups  were  relatively  similar  between  the  preparation  sessions  and  the 
teachback  movies.  In  other  words,  the  level  of  understanding  reached  about  mid -way  through  group  discussion  was 
similar  to  the  understanding  evident  in  the  final  product.  This  suggests  that  further  assistance  is  needed  to  ensure  that 
knowledge  construction  continues  to  grow  as  groups  discuss  their  approaches  to  the  teachback  movies. 


Future  development 
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The  evaluation  data  reported  here  (and  supported  by  the  other  data  collected)  indicate  that  some  groups  were  able  to 
construct  their  understanding  in  the  intended  manner,  two  groups  in  particular  being  Conspicuously  capable,  but  at  least 
two  groups  (and  maybe  also  those  groups  which  withdrew  early  in  the  semester)  were  not  able  to  perform  to  a 
satisfactory  standard.  Three  enhancements  may  assist  such  groups  in  the  next  offering  of  the  course: 

• there  will  be  more  extensive  lead-up  work  in  the  lab  to  encourage  greater  fluency  with  the  software  and  statistical 
concepts  before  the  teachback  exercises  begin  in  earnest; 

• a more  useful  retrieval  and  playback  interface  for  the  demonstration  movies  will  be  available:  statistical  and  research 
concepts  and  associated  movies  will  be  accessible  with  a zooming  concept  map  built  with  TheBrain  technology 
(http://www.thebrain.com),  and  the  movies  will  be  indexed  with  subheads  in  a synchronised  text  window  to  allow 
ready  access  to  relevant  subtopics; 

• students  will  be  able  to  seek  the  advice  of  a tutor  when  they  reach  an  impasse  in  their  exercise  discussions,  an  option 
not  provided  in  the  present  implementation. 


Social  and  cognitive  processes 

D 

E 

Group 

F G 

J 

K 

M 

SD 

Nature  of  discussion  and  ownership  of 

4 

3 

3 

4 

4 

1 

3.17 

1.07 

ideas 

Amount  of  collaboration  evident 

4 

3 

3 

4 

4 

1 

3.17 

1.07 

during  session 

Organisation  of  time  during  session 

3 

1 

4 

4 

1 

1 

2.33 

1.37 

Building  of  understanding  with 

1 

2 

3 

3 

2 

1 

2.00 

0.82 

BivarDescribe 

Correction  of  statistical 

0 

1 

4 

2 

2 

0 

1.50 

1.38 

misunderstanding 

Reference  to  demonstration  movies 

1 

0 

3 

1 

0 

1 

1.00 

1.00 

Mean 

2.17 

1.67 

3.33 

3.00 

2.17 

0.83 

Standard  deviation 

1.57 

1.11 

0.47 

1.15 

1.46 

0.37 

Table  1:  Ratings  on  six  general  social  and  cognitive  process  variables  for  groups  participating  in  the  video  recording  of 

the  preparation  session 


Process 

Mean  Range 

Outcome 

Mean 

Range 

General  understanding  scales 

Level  of  understanding  of  statistical  concepts 

3.17 

1-4 

2.67 

1-4 

Integration  of  statistical  concepts 

2.33 

0-4 

2.33 

1-4 

Integration  of  levels  of  representation  (graphical,  conceptual, 
algebraic,  computational) 

2.50 

0-4 

2.33 

1-4 

Extent  to  which  statistical  concepts  were  related  back  to 

2.00 

0-4 

2.00 

0-4 

scenario 

Sense  of  intended  audience 

2.17 

0-4 

2.17 

0-4 

Specific  understanding  scales 

Label  and  explain  the  variables  correctly 

2.33 

0-4 

2.83 

1-4 

Comment  on  the  measurement  scale  of  the  variables 

2.17 

0-4 

2.17 

0-4 

Note  and  explain  the  negative  relationship  between  the  Quest 
and  Discrimination  variables 

1.83 

0-4 

1.83 

0-4 

Explain  the  strength  of  relationship  using  the  PPV  indicator  and 
discuss  why  this  relationship  is  strong  or  weak 

2.67 

1-4 

2.83 

2-4 

Apply  the  variance  change  tool  to  the  grand  mean 

1.83 

1-4 

2.67 

0-4 

Explain  the  changes  to  the  data  when  applying  the  VCT  the 
grand  mean 

1.67 

0-  3 

2.33 

0-4 

Note  and  explain  why  LSC  and  PPV  do  not  move  when  points 
move  around  grand  mean  with  VC  tool 

2.17 

0-4 

2.17 

0-4 

Explain  how  variance  change  relative  to  grand  mean  influences 
interpretation  of  the  scales 

1.50 

0-4 

1.83 

0-4 
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Table  2:  Mean  ratings  (and  obtained  ranges)  on  six  general  and  eight  specific  understanding  scales  for  the  preparation 

sessions  (process)  and  teachback  movies  (outcome) 


Social  and  cognitive  processes 

D 

E 

Group 

F G 

J 

K 

M 

Range 

General  understanding  profiles 

2.65 

2.24 

1.00 

1.41 

1.73 

1.41 

1.74 

0-  8.94 

Specific  understanding  profiles 

3.87 

1.73 

2.00 

5.29 

2.65 

1.41 

2.83 

0-  11.31 

Table  3:  Euclidean  distances  (dissimilarities)  between  the  process  and  outcome  profiles  defined  on  the  six  general  and 

eight  specific  understanding  scales 
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The  Catalyst  Initiative:  Integrating  Support  and  Development  through  Collaboration 

Background 

In  1994,  the  Provost  charged  three  top-level  administrators  at  the  University  of  Washington  (UW)  to  "do  something  about 
technology."  They  collaborated  with  faculty,  librarians,  technologists,  and  students,  and  gathered  resources  from  wherever  they 
could  to  bring  technology  into  the  service  of  teaching  and  learning  in  a more  systematic  way,  first  with  the  Center  for  Teaching, 
Learning  and  Technology  (CTLT),  established  in  1996.  By  1998,  drop-in  visits  to  the  CTLT  had  increased  dramatically,  seriously 
taxing  staff  resources.  At  this  point,  staff  conducted  thorough  assessments  with  faculty,  librarians,  instructors,  and  teaching  assistants 
who  expressed  three  overriding  needs:  technology  support  that  is  smooth  and  easy-to-use,  flexible,  and  scalable.  The  Catalyst 
Initiative  was  created  to  meet  these  objectives.’ 

The  Catalyst  Initiative 

The  Catalyst  Initiative,  a result  of  collaborations  between  many  partners,  provides  UW  instructors  with  four  levels  of  support  for 
integrating  technology  into  their  teaching:  1)  the  Catalyst  Web  site,  2)  Catalyst  Web  Tools,  3)  face-to-face  consultations  at  the 
CTLT,  and  4)  Catalyst  workshops.  The  Catalyst  model  provides  the  grounds  for  a symbiotic  and  iterative  relationship  where  the 
support  activities  feed  and  lead  to  partnerships,  which  in  turn  result  in  the  development  of  more  and  better  support  resources  and 
activities. 

The  Catalyst  Initiative  is  carried  out  by  a permanent  staff  that  understands  the  needs  of  both  instructors  and  the  units  on  campus  with 
a stake  in  teaching  and  technology.  This  staff,  called  the  Educational  Technology  Development  Group  (ETDG),  is  perfectly  situated 
to  arrange  collaborations  and  bridgp  gaps  between  educators  and  other  units  on  campus.  This  position  as  an  intermediary  and  a 
change  agent  has  allowed  ETDG  staff  to  not  only  develop  a scalable  support  strategy  but  to  participate  in  high-level  experiments  that 
push  the  boundaries  of  innovation. 

Partnerships 

Excellent  support  resources  and  exciting  technology  infused  teaching  practices  are  found  all  across  campus,  but  few  people  know 
about  these  resources  and  practices  beyond  their  own  departments.  The  Catalyst  Initiative  was  created  to  identify  and  the  make  these 
resources  and  practices  visible,  scaling  innovation  across  campus  through  collaborative  partnerships.  Resources  and  materials 
generated  from  these  partnerships  are  placed  on  Catalyst  and  co-branded,  giving  credit  to  the  partners  who  helped  create  them. 

The  co-branding  strategy  allows  staff  to  maintain  current  and  innovative  resources  on  Catalyst  and  to  use  them  in  the  workshops  and 
one-to-one  consultations  in  the  CTLT  while  alleviating  the  burden  of  generating  new  materials  whole  cloth.  Informal  partnerships 
also  yield  tremendous  benefits.  Instructors  who  use  the  Catalyst  Web  site,  attend  workshops,  seek  help  at  the  CTLT,  or  use  the 
Catalyst  Tools  offer  feedback  which  is  incorporated  into  all  parts  of  the  Catalyst  Initiative.  Not  only  do  these  strategies  leverage 
resources,  but  they  also  forge  a sense  of  ownership  and  a measure  of  comfort  with  educational  technology  among  faculty  and  support 
staff  who  come  to  see  themselves  in  the  offerings  on  Catalyst. 

Catalyst  Web  site  (http://catalyst.washington.edu) 

The  Catalyst  Web  site  provides  examples,  promotes  good  teaching  practices,  builds  technology  skills,  and  makes  technology  easy  for 
instructors  to  use.^  Catalyst  places  good  teaching  and  student  learning  at  the  forefront,  treating  technology  as  a means  to  these  ends. 
The  resources  on  the  Catalyst  Web  site  shape  and  inform  the  other  levels  of  the  Catalyst  Initiative:  Catalyst  workshops,  one-on-one 
consulting  delivered  in  the  CTLT,  and  Catalyst  Web  Tools.  This  interlocking  support  strategy  ensures  that  campus  educators  receive 
clear,  consistent  help  for  integrating  technology  into  their  teaching. 

A great  deal  has  been  written  about  the  creation  of  the  Catalyst  Web  site,^  but  it  is  worth  reemphasizing  that  its  homegrown  content  is 
built  solely  through  collaborative  partnerships  with  campus  teaching  practitioners.  A look  at  the  six  major  content  categories  on 
Catalyst  makes  clear  the  close  connections  to  campus  instructors:  \)  Profiles  tell  stories  of  educators  who  are  using  technology  in 
teaching;  2)  Teaching  lets  instructors  explore  the  ways  that  technology  can  help  achieve  specific  teaching  and  learning  goals;  3) 

Action  Plans  are  "road  maps"  for  particular  tasks,  such  as  creatinga  class  Web  site;  4)  How-to  pages  take  users  step  -l^-step  through 
specific  tasks  needed  to  make  technology  work;  5)  Learning  offers  information  on  CTLT  workshops  and  other  campus  activities 
related  to  teaching  with  technology;  and  6)  Web  Tools  is  a gateway  to  Web-based  software  and  also  links  to  innovative  uses  of 
Catalyst  Web  tools  by  UW  educators,  again  helping  to  spread  good  practice  beyond  disciplinary  boundaries. 

Leveraging  the  expertise  available,  the  ETDG  staff  has  collaborated  with  specialists  on  campus  to  create  the  following  pages  and 
tools  with  partners: 

• Distance  learning  courses  : UW  Educational  Outreach  co-developed  Catalyst  content  that  helps  faculty  design  distance- 
learning  courses. 

• Evaluations,  Web-Ed forums,  how-to  documentation:  The  Program  for  Educational  Transformation  Through  Technology 
works  with  ETDG  staff  to  evaluate  the  Catalyst  Web  Tools,  to  post  streaming  video  from  the  Web-Ed  forums  on  the 
Catalyst  Web  site,  and  to  contribute  how-to  documentation. 

• Online  journals:  UW  Libraries’  co-branded  content  explains  how  to  locate  and  link  to  online  journal  articles. 

• Accessible  Web  sites  : The  UW  DO-IT  office  is  working  with  the  ETDG  staff  to  integrate  accessibility  issues  into  the 
content  on  the  Catalyst  Web  site. 
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• MyUWClass:  Computing  and  Communications  (C&C)  is  currently  working  with  the  ETDG  to  create  MyUWClass,  a 
personalized  teaching  portal  for  UW  instructors.  This  portal  integrates  Catalyst  Web  Tools  with  course  and  student 
information  systems  and  applications  that  let  instructors  post  content  directly  to  the  Web.  C&C  will  also  continue  to 
collaborate  on  technical  documentation. 

The  Catalyst  Web  site  fulfills  the  three  needs  expressed  by  faculty  and  staff  during  the  foundational  assessment  period.  First,  the 
varied  stories  of  teaching  with  technology,  the  step-ty-step  instructions,  and  the  best  practices  make  an  educator's  initial  foray  into 
educational  technology  smooth  and  easy.  Second,  Catalyst  is  extremely  flexible;  faculty  needs  and  wants  vary  markedly  across 
departments,  discip  lines,  and  course  types,  but  the  Catalyst  Initiative  allows  them  to  pick  and  choose  support  options  that  suit  their 
environment  and  needs.  Finally,  the  Catalyst  Web  site  and  Web  Tools  are  a campus-wide  resource,  available  24-7,  which  scales  local 
solutions  throughout  the  campus  teaching  community. 

The  Catalyst  Web  site  is  widely  used  by  UW  faculty  — instructors  and  staff  view  upwards  of  1 00,000  pages  per  month.  Clearly,  the 
Catalyst  Initiative  has  reached  instructors  from  a wide  range  of  departments. 

Catalyst  Web  Tools 

The  ETDG  created  the  Catalyst  Web  Tools  to  allow  educators  to  add  advanced  content  to  their  Web  pages  without  learning  complex 
programming  languages.  The  Web  Tools  themselves  are  all  founded  in  the  wants  and  needs  expressed  by  UW  instructors  and  are 
constantly  updated  according  to  their  feedback  and  suggestions.  For  example,  the  Political  Science  Writing  Center  asked  the  ETDG 
staff  to  develop  a tool  that  would  allow  students  to  review  each  other’s  work.  After  determining  that  the  tool  could  be  used  campus 
wide,  ETDG  developers  created  PeerReview.  As  more  instructors  use  the  tools,  they  ask  for  more  features  and  additional 
functionality  and,  indeed,  new  tools.  The  current  set  of  nine  tools  allows  instructors  to  easily  create  threaded  discussions;  receive 
student  assignments  online;  manage  ListProcs;  facilitate  case-based  learning  on  the  Web;  receive  anonymous  feedback;  administer 
one-question  surveys,  online  quizzes,  and  peer  reviews;  and  even  create  course  Web  sites  entirely  through  a Web  browser  from  any 
computing  platform.  Instructors  have  requested  over  a dozen  more  tools,  which  will  be  assessed,  prioritized,  and  developed.  The 
next  in  line  is  MyUW  Portfolio,  an  online  portfolio  system  that  students  will  use  on  their  own  and  with  guidance  from  faculty, 
advisors,  and  career  counselors. 

Because  the  tools  are  based  on  feedback  from  instructors  and  may  be  used  by  people  in  every  department,  they  are  popular  with  UW 
educators.  More  than  4000  instructors  have  used  Catalyst  Web  Tools,  and  learners  log  in  to  use  the  tools  more  than  100,000  times 
every  month. 

Drop-in  visits  to  the  CTLT 

Every  year,  1700  instructors  on  average  visit  the  CTLT  for  assistance  with  integrating  technology  into  their  courses.  The  Catalyst 
Web  site  supports  these  activities,  and  their  face-to-face  feedback  often  leads  to  the  development  of  new  content,  tool  features,  and 
workshop  materials. 

Catalyst  Workshops 

Over  900  instructors  attended  the  popular  Catalyst  workshops  last  year.  The  workshop  materials  and  topics  are  continually  modified 
as  a result  of  feedback  received  from  workshop  attendees. 

To  take  advantage  of  the  expertise  available  beyond  the  small  ETDG  staff,  several  of  the  workshops  are  co-taught  by  the  staff  and 
specialists  from  other  campus  organizations.  For  example,  an  instructor  from  Educational  Outreach  team-teaches  a workshop  on 
developing  a distance  learning  course,  and  the  DO-IT  office  works  with  ETDG  staff  to  integrate  accessible  design  issues  into  the 
workshops  and  co-teaches  an  accessibility  workshop. 

Summary 

The  Catalyst  Initiative’s  wild  success  is  the  result  of  the  innovative  model  of  working  with  partners  -both  formally  and  informally  — 
to  create  support  materials  and  resources  that  scale  campus-wide.  Using  four  levels  of  support  ensures  that  instructors  have  access  to 
support  that  suites  them,  support  that  is  created  from  their  own  suggestions,  wants,  and  feedback. 
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Abstract:  Many  new  technologies  are  now  available  for  delivering  and/or  providing 
access  to  computer-based  learning  (CBL)  materials.  These  technologies  vary  in 
sophistication  in  many  important  ways  - depending  upon  the  bandwidth  that  they  provide, 
the  interactivity  that  they  offer  and  the  types  of  end-user  connectivity  that  they  support. 
Invariably,  appropriate  combinations  of  the  available  technologies  are  needed  in  order  to 
produce  the  most  effective  and  efficient  learning  environment  for  any  given  application. 
Bearing  this  in  mind,  it  is  important  to  consider  how  multimedia  resources,  interactivity 
and  global  connectivity  can  best  be  used  in  order  to  produce  a software  product  that  best 
fulfils  the  requirements  identified  in  any  given  courseware  requirements  specification.  In 
this  paper  we  discuss  the  types  of  model  that  are  needed  to  create  effective  interactive, 
multimedia  courseware.  We  also  indicate  the  nature  of  the  interactions  that  exist  between 
these  models  and  the  ways  in  which  these  can  be  used  to  optimise  the  trade-offs  that  are 
inherent  in  the  creation  of  multimedia  CBL  materials. 


Introduction 

Multimedia  computing  technology  has  opened  up  many  interesting  opportunities  for  the  creation  of  new 
types  of  learning  and  training  products  covering  a wide  range  of  subject  areas  for  many  different  audiences 
(Barker,  1996;  1999).  Examples  of  the  types  of  product  include:  electronic  books;  educational  games; 
interactive  products  based  on  the  use  of  compact  disk  (CD)  technology;  intranet  and  World  Wide  Web 
pages;  and  tools  to  facilitate  ‘electronic’  knowledge  sharing.  Together,  products  of  this  sort  can  be  used  to 
enrich  the  scope  and  quality  of  computer-based  learning  (CBL)  experiences. 

Within  the  remainder  of  this  paper  we  shall  use  the  term  interactive  multimedia  courseware'  to  refer  to 
computer-based  learning  products  in  which  some  skill  or  knowledge  is  intentionally  transferred  to  a user  of 
that  product  as  a consequence  of  its  use.  Such  software  can  be  defined  in  terms  of  ‘learning  products’  in 
which  optimal  combinations  of  text,  sound  and  images  are  used  to  achieve  particular  learning  outcomes 
using  interactivity  in  various  ways  to  achieve  these  objectives.  Typically,  interactivity  is  used:  to  facilitate 
navigation  through  the  corpus  of  materials  that  make  up  the  product;  to  provide  assessment  and  feedback 
mechanisms;  and  to  facilitate  communication  between  the  group  of  users  that  constitute  the  ‘learning 
community’  at  which  the  product  is  focused. 

When  designing  courseware,  it  is  important  to  consider  how  multimedia  resources,  interactivity  and 
global  connectivity  can  best  be  used  in  order  to  produce  a software  product  that  successfully  fulfils  the 
conditions  identified  in  any  given  courseware  requirements  specification.  Because  of  the  complex  nature  of 
both  software  development  and  the  cognitive  processes  that  take  place  during  learning,  this  is  no  easy  task. 
The  design  and  development  processes  therefore  need  to  be  guided  by  appropriate  models  that  encapsulate 
‘best  practice’  with  respect  to  both  system  design  and  implementation  issues.  Relevant  pedagogic  models 
also  need  to  be  employed  in  order  to  ensure  that  the  subsequent  learning  processes  are  relevant  to  the 
knowledge  and  skills  that  are  to  be  acquired.  Therefore,  in  this  paper  we  shall  discuss  the  types  of  model 
that  are  needed  in  order  to  create  effective  interactive,  multimedia  courseware.  We  also  indicate  the  nature 
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of  the  interactions  that  exist  between  these  models  and  the  ways  in  which  these  can  be  used  to  optimise  the 
trade-offs  that  are  inherent  in  the  creation  of  multimedia  CBL  materials. 

Underlying  Models  and  Methodologies 

There  are  two  basic  approaches  to  developing  courseware  products;  we  shall  refer  to  these  as  the 
^empirical  approach'  and  the  theoretical  approach'.  The  first  of  these  uses  a strategy  that  is  based 
essentially  on  a ‘trial  and  error’  procedure;  that  is,  a learning  product  is  produced  and  its  effectiveness  is 
determined;  if  necessary,  an  iterative  ‘fine-tuning’  mechanism  is  then  used  in  order  to  improve  it.  In  the 
second  approach,  appropriate  theories  and  models  are  used,  in  so  far  as  they  exist,  in  order  to  create  a 
learning  product  that  falls  directly  within  a given  ‘region  of  acceptability*.  This  latter  approach  is  attractive 
because  it  reduces  the  amount  of  uncertainty  involved  in  product  creation.  It  is  our  opinion  that  design  and 
development  models  (and  methodologies  derived  from  them)  should  play  a fundamental  underlying  role  in 
guiding  the  production  of  interactive  multimedia  courseware.  The  underlying  importance  of  these  basic 
tools  is  reflected  schematically  in  Figure  1. 
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simple  models,  such  as  the  ‘waterfall’  model  to  more  advanced  models  and  methodologies  including:  the 
CASE  approach,  SSADM,  the  spiral  model,  Siegal’s  approach,  December’s  methodology,  Dinucci’s 
method,  and  so  on.  However,  these  models  and  methodologies  are  mostly  used  in  the  development  and 
implementation  of  business-oriented  information  systems  and  are  not  normally  suitable  for  the  development 
of  multimedia  learning  products. 

Designing  and  developing  multimedia  learning  products  necessitates  close  co-operation  between  people 
with  specific  skills  and  expertise.  Furthermore,  the  existence  of  diverse  modes  of  storage  and  delivery 
necessitate  the  adoption  of  appropriate  methodologies  and  suitable  design  models  in  order  to  create  high 
quality  learning  products.  In  the  remainder  of  this  paper  we  shall  discuss  some  of  the  more  important  issues 
relating  to  the  formulation  of  the  various  models  and  methodologies  that  we  feel  are  needed  in  order  to 
underpin  the  successful  development  of  interactive  multimedia  courseware  products. 
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Problems  Arising  in  Multimedia  Development 


The  development  of  interactive  learning  products  usually  requires  a team  of  skilled  specialists.  Whilst 
many  of  these  teams  adopt  their  own  approaches  to  the  analysis,  design  and  development  of  their  products, 
there  are  no  generally  accepted  models  and  methodologies  specifically  designed  to  assist  in  the  creation  of 
multimedia  learning  products.  As  the  complexity  of  the  products  and  the  size  of  the  development  teams 
increase,  it  becomes  more  difficult  to  manage  the  production  process  efficiently.  Most  existing  models  and 
methodologies  stress  the  importance  of  operation,  providing  information,  handling  events,  and  so  on. 
However,  these  elements  are  not  so  important  in  courseware  development,  where  the  process  involves  the 
integration  of  all  the  available  resources  into  a single  piece  of  software.  In  this  situation,  the  emphasis  is  on 
how  to  present  a topic  to  users  in  a way  that  increases  their  knowledge  and/or  skills. 

A multimedia  project  is  normally  composed  of  four  equally  important  components:  planning  and 
design;  resource  and  program  development;  testing  and  evaluation;  and  dissemination.  Ideally,  if  we  had 
adequate  models,  it  would  be  possible  to  predict  exactly  how  much  time  should  be  spent  on  each  of  the  four 
phases.  Invariably,  the  time  spent  on  each  individual  phase  is  not  equal  and  is  often  not  predictable.  In 
reality,  many  multimedia  projects  spend  a much  greater  proportion  of  time  on  resource  and  program 
development  and  relatively  little  time  on  pfenning  and  design,  testing  and  quality  control,  and  dissemination 
and  evaluation.  Neglecting  these  major  elements  of  the  project  life  cycle  could  ultimately  lead  to  a 
deficient  product. 

Of  course,  there  are  many  other  reasons  why  multimedia  projects  fail.  For  example,  in  a survey  of  the 
members  of  one  multimedia  development  team,  the  three  main  factors  contributing  to  unsuccessful  projects 
were  identified  as:  poor  design;  ineffective  project  co-ordination;  and  inefficient  communication  channels. 
Undoubtedly,  most  of  these  arise  due  to  lack  of  a structured  approach.  In  the  following  section  of  the  paper 
we  introduce  a comprehensive  methodology  that  is  designed  to  improve  the  efficiency  and  effectiveness  of 
multimedia  development  teams. 


A Methodology  for  Creating  Multimedia  Learning  Products 

There  are  several  good  arguments  in  favour  of  adopting  a sound  methodology  for  developing 
multimedia  learning  projects.  These  include  the  necessity  for  good  communication;  consistency;  effective 
quality  control  strategies;  and  accurate  cost  estimation.  Some  of  the  important  issues  that  need  to  be 
addressed  are  outlined  below. 

Project  Planning:  the  scope  of  a project,  its  budget  and  its  intended  completion  date  are  usually  negotiated 
prior  to  the  commencement  of  any  development  activity.  Once  a project  is  formally  agreed  upon, 
management  strategies  are  identified,  milestones  are  established  and  tasks  and  resources  (including 
individual  team  members)  are  identified. 

Team  Management;  effective  team  management  helps  to  establish  a productive  working  atmosphere,  deals 
with  problems  more  readily  and  shares  the  workload  efficiently.  Figure  2 shows  the  structure  of  a typical 
multimedia  development  team  in  which  the  courseware  designer  and  project  manager  co-ordinate  the 
planning,  design,  development  and  testing.  A clear  definition  of  the  roles  and  responsibilities  of  each  team 
member  ensures  that  all  project  tasks  are  allocated  and  allows  contingency  plans  to  be  put  in  place  in  the 
event  of  any  member  of  the  team  becoming  unavailable. 

Project  Life  Cycle:  Figure  3 shows  the  four  main  phases  involved  in  a typical  multimedia  project.  The 
first  three  of  these  (preparation,  design  and  development)  are  iterative  processes.  The  fourth  phase 
(dissemination)  can  be  repeated  if  the  product  is  revised  or  modified  between  releases. 

Analysis  of  Requirements:  a comprehensive  project  analysis  determines  the  specific  requirements  of  the 
end^roduct  and  should  include  an  in-depth  study  of  the  users,  the  intended  delivery  systems  and  the  scope 
of  the  project.  These  should  be  documented  so  that  developers  and  clients  can  refer  to  them  in  order  to 
ascertain  whether  the  product  is  meeting  the  required  standards. 
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Design  Specifications:  the  design  specifications  provide  a detailed  description  of  the  program  requirements 
to  all  members  of  the  development  team.  It  incorporates  essential  information  such  as  the  font  sizes  to  be 
used,  colour  palettes,  file  formats  and  flowcharts  that  outline  the  proposed  content. 


Figure  2:  Structure  of  a Typical  Multimedia  Development  Team 


Human-Computer  Interface  (HCI)  Issues:  the  end-user  interface  encompasses  a number  of  factors 
including  the  proposed  screen  layouts,  navigation  controls,  user  interaction,  ‘look  and  feel’  and  the  nature 
of  the  various  metaphors  that  are  embedded  within  the  courseware. 

Delivery  Medium:  multimedia  learning  products  can  be  delivered  in  a number  of  ways.  For  example,  CD- 
ROM  publication,  the  Internet,  an  intranet  or  local  area  network,  and  various  types  of  turnkey  solution. 
Each  delivery  method  has  constraints  and  limitations  that  should  be  addressed  at  the  planning  stag^  of  the 
project. 

Resource  Production:  multimedia  learning  products  can  contain  a combination  of  text,  graphics,  audio , 
video  and  animation.  The  establishment  of  appropriate  parameters  for  each  of  these  at  the  outset  of  the 
project  avoids  inconsistencies  within  the  final  product. 

Program  Integration:  the  ease  with  which  multimedia  elements  are  integrated  depends  critically  upon  the 
availability  of  a comprehensive  script  and  detailed  flowcharts  to  assist  the  programmers  and  resource 
producers.  A good  file  naming  strategy  is  also  essential  to  the  integration  of  resource  files. 

Quality  Control  and  Evaluation:  both  formative  and  summative  evaluation  strategies  need  to  be  identified 
during  the  planning  stage  and  quality  control  should  be  regarded  as  a continuous  process.  A thorough 
testing  strategy  should  be  established  that  allows  all  errors  to  be  noted  and  amended  effectively.  Ongoing 
testing  and  evaluation  should  be  employed  throughout  Phases  1,  2 and  3 in  order  to  ensure  that  the  final 
product  falls  within  a pre-specified  ‘region  of  acceptability’.  As  we  discuss  later,  'ongoing  evaluation'  is 
an  important  aspect  of  the  dissemination  phase  of  the  methodology. 

Documentation:  good  documentation  is  essential  to  ensure  that  any  project  life  cycle  evolves  efficiently 
and  within  the  allotted  time.  The  basic  documentation  required  for  most  multimedia  projects  comprises:  a 
project  proposal;  design  specifications;  scripts  and  storyboards;  quality  control  and  testing  strategies;  and 
progress  reports 
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Scripts  and  Storyboards:  scripts  are  paper-based  documents  that  provide  a detailed  description  of  the 
courseware  content  and  howthis  is  to  be  implemented  within  the  final  product.  Scripts  contain  details  of  all 
on-screen  text,  audio  transcripts,  written  descriptions  of  graphics,  video  and  animation  along  with  their  file 
names.  These  form  the  basis  of  audio  scripts,  graphics  listings  and  video  and  animation  storyboards. 
Scripts  also  provide  special  instructions  and  detailed  flowcharts  for  the  programming  team. 


Figure  3:  Development  Methodology  for  Interactive  Multimedia  Courseware 

Dissemination;  the  dissemination  phase  of  product  development  involves  all  the  processes  necessary  to 
package  the  product  and  make  it  available  to  customers.  Of  course,  even  after  a learning  product  has  been 
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released  for  distribution  (see  Phase  4 in  Figure  3)  it  is  important  to  undertake  a programme  of  'ongoing 
evaluation'.  The  results  of  this  type  of  evaluative  study  can  subsequently  be  used  to  fine-tune  future 
releases  of  the  learning  product  and  also  inform  the  design  and  development  processes  involved  in  the 
creation  of  new  courseware  products. 


Discussion 

One  of  the  most  convincing  arguments  for  the  use  of  models  stems  from  the  fact  that  they  enable  us  to 
make  accurate  predictions  about  the  properties  and  behaviour  of  the  systems  that  they  describe.  In  this 
paper  we  have  described  a methodology  to  facilitate  the  development  of  interactive  multimedia  learning 
products.  The  methodology  that  was  outlined  in  the  previous  section  has  been  derived  primarily  from  our 
experiences  with  conventional  multimedia  learning  products  involving  the  creation  of  interactive  CDs.  It  is 
therefore  necessary  to  discuss  whether  or  not  the  methodology  (as  presented  in  Figure  3)  could  be  applied 
to  the  development  of  other  types  of  interactive  learning  product  - particularly  those  involving  the  use  of  an 
intranet  or  the  World  Wide  Web.  Our  experiences  to  date  suggest  that  the  basic  methodology  (with  some 
minor  amendments)  would  easily  handle  this  latter  type  of  product.  Essentially,  Phases  1 through  3 could 
be  used  as  they  stand.  So  far,  we  have  found  that  the  only  area  where  minor  amendments  may  be  needed  is 
in  Phase  4.  In  this  phase,  the  ‘replication’  and  ‘packaging’  steps  may  be  less  important  for  an 
intranet/Intemet  project  than  they  would  be  for  a CD  project.  We  are  therefore  optimistic  that  the  proposed 
methodology  can  be  used  to  cater  for  a wide  range  of  interactive  multimedia  learning  product  development 
projects 


Conclusion 

Developing  interactive  multimedia  courseware  products  is  a complex  and  costly  process  that  requires  a 
wide  range  of  powerful  models  and  a workable,  practical  design  and  development  methodology.  In  this 
context,  a structured  design  and  development  approach  has  a number  of  advantages.  For  example, 
templates  and  scripts  provide  essential  information  in  easy -to -read  formats  that  can  be  tailored  to  meet  the 
requirements  of  individual  team  members.  In  addition,  the  production  process  can  be  made  simpler  and 
more  efficient  - as  fewer  program  and  resource  changes  are  required  at  the  development  stage. 
Furthermore,  quality  control  is  made  more  effective  by  providing  testers  and  evaluators  with  a program 
‘blueprint’. 

A learning  product  development  methodology,  such  as  the  one  that  has  been  described  in  this  paper, 
should  enable  multimedia  courseware  products  to  be  developed  in  a more  predictable  way  - both  in  terms  of 
learning  outcomes  and  from  the  perspective  of  resource  utilisation  (time  and  money).  Implicit  in  the 
various  project  phases  that  we  have  described  in  our  methodology  are  a number  of  local  and  global 
interactions  - both  between  the  underlying  models  and  the  system  variables  that  are  involved  in  managing 
the  preparation,  design,  development  and  dissemination  processes.  It  is  through  a greater  understanding  of 
these  interactions  that  we  will  be  able  to  gain  further,  much  needed,  insight  into  the  various  factors  that  are 
responsible  for  causing  multimedia  learning  projects  to  fail. 
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Abstract:  the  electronic  dissemination  of  information  via  computer  networks  is  having  a considerable 
impact  on  educational  institutions  in  terms  of  the  mechanisms  that  they  are  able  to  employ  for  course 
delivery.  As  a result  of  this,  the  number  of  courses  that  are  taught  electronically  through  the  Internet 
and/or  the  World  Wide  Web  has  increased  considerably  over  the  last  few  years.  Such  courses  require  two 
fundamental  types  of  resource:  appropriately  designed  electronic  course  materials  (for  students  to  study) 
and  a network  of  online  tutors  who  can  provide  active  support,  advice  and  encouragement  for  students. 
This  paper  discusses  the  nature  of  the  skill  sets  needed  by  online  tutors  in  order  to  fulfil  their  roles 
effectively  and  efficiently  within  an  online  learning  community.  Some  mechanisms  for  acquiring  these 
skills  are  also  briefly  discussed. 


Introduction 

Electronic  teaching  through  the  Internet  and  the  World  Wide  Web  is  rapidly  becoming  an  established  practice  in 
many  educational  institutions.  This  is  particularly  the  case  in  situations  where  there  is  a need  for  flexible  course 
provision  and/or  a demand  for  ‘time  and  place’  independence  of  course  delivery  (Beckstrand  et  al,  2001;  2002).  Non- 
academic  institutions  are  also  using  ‘e-leaming’  techniques  as  a mechanism  for  facilitating  staff  training,  reducing  the 
costs  of  taining,  improving  the  effectiveness  of  educational  delivery  and  providing  continuing  professional 
development  opportunities. 

Naturally,  designing  electronic  courses  for  online  delivery  is,  in  many  ways,  different  from  the  ways  in  which 
conventional  courses  are  developed  and  used.  Indeed,  according  to  Galpin  (2001),  '‘simply  taking  a traditional 
classroom  or  paper-based  experience  and  putting  it  online  is  not  enough'.  Techniques  for  designing  and 
supporting  online  courses  are  well-documented  in  the  literature  - see,  for  example,  Thomas  et  al  (1998),  Lockwood  and 
Gooley  (2001)  and  Jolliffe,  Ritter  and  Stevens  (2001).  Our  own  approach  to  electronic  course  design  and  delivery  has 
been  described  in  detail  elsewhere  (Barker,  1999;  2000;  2001;  2002).  Essentially,  the  methodology  involves  the 
creation  of  four  basic  types  of  resource.  First,  an  online  corpus  of  study  material  (a  teaching  web)  that  students  can 
access  through  an  intranet  or  via  the  World  Wide  Web.  Second,  the  use  of  computer-mediated  communication  to 
facilitate  both  synchronous  and  asynchronous  computer  conferencing  activities.  Third,  the  creation  of  shared 
workspaces  to  facilitate  collaborative  group  working.  Finally,  the  provision  of  appropriate  assessment  infrastructures 
to  enable  students  (and  staff)  to  monitor  and  gauge  the  progress  they  are  making  with  the  course. 

Obviously,  online  teaching  is  different  from  face-to-face  instruction.  The  main  differences  between  the  two 
approaches  arise  because  teachers  and  students  are  not  physically  present  together  at  the  same  geographical 
location.  This  absence  of  ^proximal presence'  has  two  important  effects,  which  are  briefly  discussed  below. 

First,  the  shared  environmental  artefacts  that  may  be  present  in  a face-to-face  teaching  situation  (such  as 
walls  and  windows,  posters,  desks,  books,  students,  and  so  on)  are  usually  ‘absent’  from  a distributed  online 
teaching  environment.  Of  course,  within  an  online  system,  virtual  analogues  of  these  ‘missing’  objects  may  be 
generated  in  order  to  substitute  for  them.  However,  many  of  the  more  common  approaches  to  online  teaching  do  not 
do  this  due  to  the  complexity  of  the  software  that  is  needed. 

The  second  ‘missing  component’  from  online  teaching  environments  is  the  various  types  of  body  language 
that  are  normally  associated  with  small-group,  face-to-face  teaching  situations.  Within  online  teaching,  various 
textual  conventions  are  sometimes  used  to  substitute  for  the  absence  of  non-verbal  communication  - for  example,  the 
use  of  emoticons,  various  font  styles  for  representing  different  modes  of  expression  (such  as  the  use  of  coloured  text. 
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italicisation,  and  so  on)  and  iconic  forms.  However,  these  conventions  have  only  met  with  various  degrees  of 
success  (Yazdani  and  Barker,  2000). 

Because  online  teaching  is  different  from  face-to-face  instruction,  it  therefore  requires  different  skill  sets  - 
both  for  the  students  who  use  the  electronic  resources  (Montieth  and  Smith,  2001)  and  for  the  staff  involved  in 
supporting  the  various  activities  associated  with  a particular  course  (Duggleby,  2000;  Salmon,  2000;  Bennett  and 
Marsh,  2002).  Because  of  the  growing  importance  of  electronic  course  delivery  and  online  tutoring,  the  remainder  of 
this  paper  attempts  to  identify  and  discuss  some  of  the  staff  roles  and  responsibilities  associated  with  online 
teaching  and  learning  - and  the  skill  sets  needed  to  undertake  these  activities. 


Possible  Roles  for  Online  Tutors 

Within  environments  that  utilise  electronic  course  delivery,  there  are  a number  of  different  course  ‘development 
and  delivery'  scenarios.  These  are  likely  to  require  instructors  to  play  several  different  roles.  Bearing  this  in  mind. 
Table  1 depicts  some  of  the  major  roles  that  any  particular  individual  may  have  to  fulfil  in  relation  to  producing  and/or 
using  an  online  course. 


Table  1:  Roles  for  Online  Tutors 


- a member  of  a course  design  team 

-the  sole  designer  of  a course  and  one  of  its  online  tutors 

- an  online  tutor  for  courses  produced  by  others 

- a moderator  and/or  examiner  for  various  courses 


It  is  easy  to  deduce  from  Table  1 that  there  are  three  basic  types  of  involvement  that  an  online  tutor  might  have 
with  any  given  online  course.  First,  in  the  development  stage,  a tutor  might  become  involved  in  various  aspects  of 
designing  (and  possibly,  producing)  an  online  course.  Second,  in  the  delivery  stage  of  a given  course,  particular 
individuals  (who  might  not  have  been  involved  in  the  development  phase)  might  act  as  an  online  tutor  for  that 
course.  Third,  in  the  assessment  phases  of  a course  the  tutor  might  be  involved  in  various  quality  assessment  or 
quality  assurance  activities. 

Each  of  the  above  roles  will  have  an  appropriate  skill  set  associated  with  it.  Each  skill  set  will  require  an 
appropriate  mix  of  pedagogic,  technical  and  organisational  skills.  The  nature  of  these  three  skill  sets  is  discussed 
in  more  detail  in  the  following  section  of  the  paper. 


What  Skills  are  Needed? 

In  order  to  undertake  their  responsibilities  in  an  effective  way,  online  tutors  will  need  to  be  equipped  with  the 
skills  and  knowledge  needed  to  perform  the  tasks  that  they  are  required  to  undertake.  The  detailed  nature  of  these 
tasks  is  discussed  elsewhere  (Barker,  2002;  Duggleby,  2000;  Salmon,  2000).  This  section  of  the  paper  therefore  briefly 
describes  and  discusses  the  three  major  skill  sets  (pedagogic,  technical  and  organisational)  that  we  believe  are  a 
necessary  pre-requisite  for  effective  online  tutoring. 

(a)  Pedagogic  Skills 

Skills  in  this  category  relate  to  the  ability  of  an  online  tutor  to  design  and  create  teaching  and  learning  materials 
for  online  use.  There  will  normally  be  two  contexts  in  which  these  resources  might  be  used.  In  the  first  instance,  they 
could  form  the  primary  resource  set  that  students  use  in  order  to  study  and  learn.  Alternatively,  they  could 
constitute  an  ancillary  resource  set  that  is  intended  to  complement  a primary  resource  set  that  has  been  produced  by 
someone  else.  Some  of  the  important  pedagogic  skills  needed  to  facilitate  the  design  and  development  of  these 
learning  resource  sets  are  listed  in  Table  2. 

Most  of  the  necessary  skills  in  this  category  will  be  similar  to  those  needed  to  prepare  and  use  conventional 
teaching  and  learning  resources.  For  example,  topic  research,  knowledge  structuring  and  the  design  of  self-study 
tasks  and  assessment  strategies  are  fairly  generic  activities  that  are  applicable  to  virtually  all  modes  of  teaching. 
Naturally,  it  will  be  necessary  to  consider  what  special  requirements  need  to  be  accommodated  as  a consequence  of 
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using  web-based  delivery.  This  is  especially  so  in  the  case  of  knowledge  structuring  - where  electronic  resources 
can  be  interlinked  in  many  different  ways  that  are  not  feasible  with  conventional  media.  Of  course,  the  use  of 
electronic  resources  also  offers  many  pedagogic  facilities  (such  as  animation  and  interactivity)  that  may  not  be 
possible  using  conventional  approaches  to  instruction. 

Table  2 Major  Pedagogic  Skills  for  Online  Tutoring 

- Researching  a Topic 

- Knowledge  Structuring 

- Designing  Individualised  Self-Study  Tasks 

- Designing  Group  Work  Activities 

- Formulating  Assessment  Strategies 

- Mentoring,  Counselling  and  Advising 

- Marking,  Monitoring  and  Giving  Feedback 


(b)  Technical  Skills 

The  skills  in  this  category  relate  to  the  ability  of  an  online  tutor  to  use  a range  of  different  software  tools  in  order 
to  conduct  his/her  tutoring  tasks  in  an  effective  and  efficient  way.  Some  of  the  more  important  of  these  tools  are 
listed  in  Table  3. 


Table  3 Technical  Skill  Requirements  for  Online  Tutoring 

- Using  Electronic  Mail 

- Creating,  Managing  and  Participating  in  Asynchronous  Conferences 

- Designing,  Creating  and  Controlling  Real-time  Chat  Rooms 

- Using  a Word  Processor,  Spreadsheet  and  Database 

- Using  Web  Page  Authoring  Tools 

- Using  Special  Purpose  Software 


Although  online  tutors  often  use  telephone,  facsimile  transmission  and  postal  services  in  order  to  communicate 
with  their  students,  by  far  the  most  important  (and  heavily  used)  mechanisms  of  communication  are  computer- 
mediated.  Undoubtedly,  electronic  mail  is  probably  the  most  widely  used  tool.  This  is  used  both  to  send  ‘simple’ 
messages  and  ‘compound’  messages  containing  various  types  of  attachment  - such  as  text  files,  spreadsheets, 
sound  and  image  files,  multimedia  documents,  and  so  on.  A variety  of  different  computer  conferencing  tools  (both 
synchronous  and  asynchronous)  are  also  usually  employed  to  facilitate  teaching  and  learning  tasks.  These 
electronic  tools,  and  the  others  identified  in  Table  3 are  discussed  in  more  detail  elsewhere  (Barker,  2002). 

(c)  Organisational  Skills 

Skills  in  this  category  refer  to  the  ability  of  online  tutors  to  organise  their  own  activities  as  well  as  those  of  the 
students  who  are  following  the  particular  courses  for  which  they  are  responsible.  Some  of  the  main  skill  requirements 
in  this  category  are  listed  in  Table  4. 

Table  4 Organisational  Skills  for  Online  Tutors 

- select  and  organise  cohorts  of  students  for  group  activities 

- maintain  online  conference  structures  in  a logical  fashion 

- structure  and  maintain  students’  demographic  data 

- collect  and  archive  students’  work  for  assessment 

- archive  and  maintain  personal  teaching  resources 

- maintain  administrative  proformas  and  document  templates 
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As  is  the  case  in  conventional  teaching  situations,  one  of  the  major  organisational  responsibilities  that  online 
tutors  have  to  undertake  is  the  facilitation  of  students’  learning  activities  - especially  for  group  work  and  projects.  A 
useful  way  of  achieving  this  is  through  the  use  of  an  electronic  ‘noticeboard’.  Such  a tool  can  be  used  to  publish 
lists  of  students  and  details  of  group  activities  - for  example,  project  specifications,  milestones,  deliverables  and  time 
schedules.  Tools  of  this  sort  can  be  regarded  as  being  a part  of  a broader  organisational  toolset  that  is  provided 
within  the  framework  of  an  ‘electronic  office’  environment.  Many  organisations  involved  in  electronic  teaching  and 
learning  especially  within  a virtual  campus  context)  now  provide  their  network  of  online  tutors  with  this  type  of 
facility  - see,  for  example,  the  facilities  at  the  Robert  Gordon  virtual  campus  at  h ttp : //campus,  r^u.  com. 


Skill  Acquisition  Mechanisms 

Any  discussion  of  the  skills  that  an  online  tutor  needs  to  possess  must  also  consider  how  these  skills  might  be 
acquired.  Naturally,  there  are  two  major  issues  that  need  to  be  discussed.  First,  we  must  consider  how  ‘new’  online 
tutors  gain  the  skills  that  they  need  in  order  to  participate  in  an  effective  way  within  an  online  learning  community. 
Second,  we  should  consider  how  established  online  tutors  participate  in  ‘professional  updating’  activities  - that  is, 
the  mechanisms  that  they  use  in  order  to  keep  up  to  date  with  the  new  software  systems,  technologies  and  course 
modules  that  they  may  be  required  to  use. 

Ideally,  the  mechanisms  for  each  of  the  above  skill  development  processes  should  be  found  embedded 
within  the  online  learning  communities  themselves.  That  is,  the  organisations  responsible  for  providing  courses  for 
students  should  also  provide  online  training  opportunities  for  the  network  of  tutors  that  is  needed  to  support  these 
courses.  Many  distance  learning  universities  do  in  fact  provide  online  courses  and  online  training  for  their  staff. 
The  UK’s  Open  University  (OU),  for  example,  uses  its  computer  network  infrastructure  in  order  to  provide  online, 
interactive  training  for  many  of  its  tutors.  As  an  example  of  this,  the  OU  makes  available  online  training  for  the 
FirstClass  conferencing  system  (which  it  uses  for  many  of  its  distance  learning  courses). 

Many  other  universities  also  provide  various  online  courses  both  for  their  own  staff  and  for  other  people 
who  want  to  find  out  more  about  electronic  distance  learning  and  online  tutoring  activities.  The  'virtual  campus^  at 
the  Robert  Gordon  University  in  Scotland  ih ttp : //campus. r^u.com\,  for  example,  has  a number  of  online  courses  that 
visitors  to  the  campus  can  study  - see,  for  example,  the  'Campus  Induction  Module\  This  module  shows  users  how 
to:  build  an  online  office;  access  and  use  course  materials;  enter  online  discussions;  join  an  online  community;  and 
use  the  virtual  library.  It  also  provides  a series  of  ‘top  tips’  for  online  learning. 

The  need  for  professional  development  opportunities  for  staff  in  the  area  of  online  learning,  tutoring  and 
distance  education  has  been  recognised  by  a number  of  authors  - see,  for  instance,  Salmon  (2000),  Montieth  and 
Smith  (2001),  MacKenzie  and  Staley  (2001),  Bemath  and  Ruben  (2001)  and  Bennett  and  Marsh  (2002).  Based  on  the 
results  of  their  study  into  using  a virtual  campus,  Montieth  and  Smith  (2001 : p.  1 26)  report  the  finding  that  'It  is  in  the 
area  of  computer  mediation  we  consider  staff  need  more  training  ...  staff  skills  with  respect  to  mediation  through 
CMC  do  require  updating'.  Bennett  and  Marsh  (2002)  have  reported  similar  findings.  They  suggest  that  ‘...  the 
majority  of  tutors  new  to  online  teaching  do  not  have  the  background  of  online  learning  experience  upon  which  to 
draw ...  ’. 

Various  mechanisms  for  the  provision  of  online  training  opportunities  for  academic  staff  have  been  explored. 
For  example,  in  their  research,  Bemath  and  Ruben  (2001)  have  described  the  use  of  a 'Virtual  Seminar  in  Distance 
Education'  for  facilitating  training  opportunities.  This  is  an  online,  asynchronous  discussion  forum  (based  on  the 
World  Wide  Web)  that  was  designed  to  provide  university  faculty  and  administration  with  professional  development 
in  the  field  of  distance  education.  The  success  of  their  research  leads  one  to  suggest  that  the  provision  of  online 
training  opportunities  for  online  tutors  is  of  paramount  importance.  This  view  is  supported  by  the  findings  of 
Bennett  and  Marsh  (2002)  who  have  described  their  experiences  in  preparing  ‘new’  online  tutors  for  the  roles  they 
will  play  within  an  online  electronic  learning  community.  Their  study  outlines  the  nature  of  the  training  given  to 
potential  online  tutors  and  how  the  effectiveness  of  this  was  evaluated.  The  results  of  their  evaluation  suggest  that 
the  'online  teaching  practice  may  be  the  single  most  important  element  of  the  training  and  development  process'. 

Naturally,  as  the  roles  of  academic  staff  in  higher  education  institutions  change,  there  is  obviously  considerable 
scope  for  the  introduction  of  online  training  courses  that  can  provide  adequate  and  appropriate  skill  development  for 
the  new  tasks  that  tutors  have  to  undertake. 
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Using  Online  Learning  for  Skill  Acquisition 

One  important  approach  that  we  have  used  for  developing  the  skills  needed  to  become  an  online  tutor  is  through 
on-the-job  training  using  an  ‘apprenticeship’  model  (Barker,  1994).  In  this  situation,  potential  online  tutors  become 
involved  in  running  tutorial  and  teaching  groups  in  conjunction  with  an  experienced  and/or  ‘qualified’  e-tutor. 
Another  approach,  that  we  are  exploring,  involves  the  creation  of  an  online  environment  for  facilitating  skill 
development.  The  architecture  for  our  proposed  training  facility  is  illustrated  schematically  in  Figure  1 . 
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Figure  1:  Skill  acquisition  environment  for  an  online  community  of  trainee  e-tutors. 

There  are  four  important  components  within  the  skill  development  system:  the  knowledge  corpus  that  documents 
the  skill  sets  and  procedures  to  facilitate  their  acquisition;  an  electronic  performance  support  system  (EPSS)  that  is 
able  to  assess  the  ability  of  a potential  tutor  and  his/her  current  knowledge/skill  levels  and  then  recommend  a ‘course 
of  study’  for  that  tutor;  various  online  presentations  that  are  designed  to  facilitate  knowledge/skill  acquisition;  a 
range  of  practice  conferences;  and  a discussion  forum.  The  latter  two  resources  are  particularly  important  because  it 
is  through  these  that  new  tutors  can  practice  their  skills,  discuss  any  problems  that  they  are  having  and  share  their 
experiences. 


Conclusion 

For  a variety  of  reasons,  many  academic  (and  non-academic)  institutions  are  now  becoming  involved  in  electronic 
learning  initiatives.  There  are  two  main  ‘driving  forces’  underlying  the  developments  that  are  taking  place.  The  first 
of  these  arises  as  a result  of  the  need  for  many  organisations  to  become  involved  in  computer-based  distance 
education  involving  the  Internet  and  the  World  Wide  Web.  The  second  arises  as  a result  of  organisations  wanting 
to  take  advantage  of  the  many  pedagogic  benefits  that  online  learning  can  bring.  A computer-based  network 
infrastructure  (such  as  the  Internet  or  an  intranet)  is  able  to  facilitate  two  major  educational  objectives.  First,  the 
delivery  of  electronic  learning  resources  in  a flexible  way  that  is  independent  of ‘time  and  place’;  and  second,  the 
realisation  of  ‘educational  conversations’  between  students  themselves  and  with  their  tutors.  Naturally,  with  the 
growing  demand  for  online  learning,  it  is  important  that  students  following  particular  courses  are  supported  by  an 
adequately  trained  network  of  online  tutors.  Such  a requirement  necessitates  that  we  should  identify  the  skill  sets 
that  online  tutors  need  and  then  provide  appropriate  (online)  infrastructures  and  mechanisms  to  enable  the  requisite 
skills  to  be  developed.  This  paper  has  attempted  both  to  identify  some  of  the  required  skills  and  to  suggest  how  the 
training  mechanisms  might  be  realised. 


127 


References 


Barker,  P.G.,  (1994).  Designing  Interactive  Learning,  1-30  in  Design  and  Production  of  Multimedia  and  Simulation- 
Based  Learning  Material,  edited  by  T.  de  Jong  and  L.  Sarti,  Kluwer  Academic  Publishers,  Dordrecht,  The 
Netherlands. 

Barker,  P.G.,  (1999).  Using  Intranets  to  Support  Teaching  and  Learning,  Innovations  in  Education  and  Training 
International,  36(1),  3-11. 

Barker,  P.G.,  (2000).  Developing  Teaching  Webs  - Advantages,  Problems  and  Pitfalls,  54-59  in  Proceedings  of  the 
EDMEDIA  2000  International  Conference  on  Educational  Multimedia,  Hypermedia  and  Telecommunications, 
Montreal,  Canada,  edited  by  J.  Bourdeau  and  R.  Heller,  Association  for  the  Advancement  of  Computing  in 
Education,  Virginia,  USA.  (This  paper  is  also  available  in  electronic  form  at  http://www,philip- 

barker.  demon,  co,uk/edmd2000,  htm,) 

Barker,  P.G.,  (2001).  Creating  and  Supporting  Online  Communities  of  Learners,  92-97  in  Proceedings  of  the  EDMEDIA 
2001  International  Conference  on  Educational  Multimedia,  Hypermedia  and  Telecommunications,  University 
of  Tampere,  Finland,  edited  by  C.  Montgomerie  and  J.  Vitali,  Association  for  the  Advancement  of  Computing  in 
Education,  Virginia,  USA.  (This  paper  is  also  available  in  electronic  form  at  http://www,philip- 

barker,demon.co,uk/edmd200l,doc.) 

Barker,  P.G.,  (2002).  On  Being  an  Online  Tutor,  Innovations  in  Education  and  Teaching  International,  39(1),  3-13. 

Beckstrand,  S.,  (2002).  A Comparison  of  Distance  Education  Delivery  Methods  in  Southern  Nevada,  PhD  Thesis, 
University  of  Teesside,  Middlesbrough,  UK. 

Beckstrand,  S.,  Barker,  P.G.  and  van  Schaik,  P.,  (2001).  Towards  More  Independent  Learning:  a Southern  Nevada 
Perspective,  106-110  in  Proceedings  of  the  EDMEDIA  2001  International  Conference  on  Educational 
Multimedia,  Hypermedia  and  Telecommunications,  University  of  Tampere,  Finland,  edited  by  C.  Montgomerie 
and  J.  Vitali,  Association  for  the  Advancement  of  Computing  in  Education,  Virginia,  USA. 

Beckstrand,  S.,  Barker,  P.G.  and  van  Schaik,  P.,  (2002).  Computer-Based  Distance  Education  for  Vocational  Use  in 
Southern  Nevada,  paper  submitted  to  the  EDMEDIA  2002  International  Conference  on  Educational 
Multimedia,  Hypermedia  and  Telecommunications,  Denver,  Colorado,  USA. 

Bennett,  S.  and  Marsh,  D.,  (2002).  Are  we  Expecting  Online  Tutors  to  Run  Before  they  Can  Walk?,  Innovations  in 
Education  and  Teaching  International,  39{\),  14-20. 

Bemath,  U.  and  Rubin,  E.,  (2001).  Professional  Development  in  Distance  Education  - a Successful  Experiment  and 
Future  Directions,  213-223  in  Innovations  in  Open  and  Distance  Learning  - Successful  Development  of  Online 
and  Web-Based  Learning,  edited  by  F.  Lockwood  and  A.Gooley,  Kogan  Page,  London. 

Duggleby,  J.,  (2000).  How  to  be  an  Online  Tutor,  Gower,  Aldershot,  Hampshire,  UK. 

Galpin,  G.,  (2001)  Exodus  to  E-leaming,  Computer  Bulletin,  Series  V,  5(5)  28-29. 

Jolliffe,  A.,  Ritter,  J.  and  Stevens,  D.,  (2001).  The  Online  Learning  Handbook,  Kogan  Page,  London. 

Lockwood,  F.  and  Gooley,  A.,  (2001).  Innovations  in  Open  and  Distance  Learning  - Successful  Development  of 
Online  and  Web-Based  Learning,  Kogan  Page,  London. 

MacKenzie,  N.  and  Staley,  A.,  (2001).  Online  Professional  Development  for  Academic  Staff:  Putting  the  Curriculum 
First,  Innovations  in  Education  and  Teaching  International,  38(1),  42-53. 

Montieth,  M.  and  Smith,  J.,  (2001).  Learning  in  a Virtual  Campus:  the  Pedagogical  Implications  of  Students’ 
Experiences, « 5 in  Education  and  Teaching  International,  38(2),  119-127. 

Salmon,  G.,  (2000).  E-Moderating  - The  Key  to  Teaching  and  Learning  Online,  Kogan  Page,  London. 

Thomas,  P.,  Carswell,  L.,  Price,  B.  and  Petre,  M.,  (1998).  A Holistic  Approach  to  Supporting  Distance  Learning  using 
the  Internet:  Transformation,  not  Translation,  British  Journal  of  Educational  Technology,  29(2),  149-161. 

Yazdani,  M.  and  Barker,  P.G.,  (2000).  Iconic  Communication,  Intellect  Books,  Bristol,  UK. 


128 


Distributed  Internet2-Based  Mission  Training  and  Collaborative  Research 

Using  an  Agent-Based  Platform 

Christopher  Barnes 
Warfighter  Training  Research  Division 
Air  Force  Research  Laboratory 
2504  Gillingham  Drive 
Brooks  AFB,  TX  78235-5100 
(210)  536-2177 

christopher.bames@brooks.afmil 

Dr.  Linda  R.  Elliott 
Veridian  Engineering 
2504  Gillingham  Drive 
Brooks  AFB,  TX  78235-5100 
210-534-1134  X 19 
linda.Elliott@brooks.af.mil 


Dr.  Plamen  V.  Petrov 
21®*  Century  Systems  Inc. 
12612  Decatur  Street 
Omaha,  NE  68154 
402-384-9093 
Plamen@2 1 csi.com 

Dr.  Alex  Stoyen 
21®*  Century  Systems  Inc. 
12612  Decatur  Street 
Omaha,  NE  68154 
402-384-9893 
alex@21csi.com 

Stuart  L.  Aldridge 
21®*  Century  Systems  Inc. 

9830  S.  Flower  Court 
Littleton,  CO  80127-8547 
720-981-8731 
stuart@21csi.com 


Abstract  This  paper  describes  approach,  methodology,  and  potential  application  areas  for  agent  technologies  in 
Command  and  Control  (C2)  performance  and  primarily  focuses  on  the  development  of  agent-based  constructed  forces. 
The  platform  involved  is  the  AWACS-AEDGE'^^.  This  is  a distributed,  real-time  team  decision  support  environment 
comprised  of  simulators,  entity  framework,  intelligent  agents  and  user  interfaces.  The  AEDGE  is  constructed  as  a 
federation  of  intelligent  agent-based  functions  that  enable  user-friendly  scenario  construction,  emulation  of  friendly  and 
hostile  entities,  and  dynamic  scenario  control.  Its  architecture  and  decision  making  algorithms  are  examined,  as  well  as 
agent  technology  and  utilization  in  the  realms  of  constructive  forces,  synthetic  team  members,  and  decision  support. 

Introduction 

This  paper  describes  the  approach,  methodology,  and  potential  application  areas  for  agent-based  synthetic  task 
platforms  (STE)  to  enhance  C2  performance,  as  exemplified  in  the  AWACS-AEDGE™  . We  focus  particularly  on  the 
development  of  agent-based  constructed  forces  and  decision  support  to  enhance  C2  training  and  performance  (Barnes  et 
al,  in  review). 


The  Airborne  Warning  and  Control  System  (AWACS)  Weapons  Director  (WD)  team  serves  as  a vital  airborne 
Command  and  Control  (C2)  node,  providing  airborne  surveillance,  control,  and  communications  functions  for  tactical 
and  air  defense  forces.  It  is  clear  that  AWACS  WD  duties  exemplify  core  characteristics  of  a C2  team.  They  perform 
in  highly  interdependent  roles,  tracking  and  coordinating  some  type  of  tactical  action,  in  a manner  consistent  with 
overall  strategic  goals  and  procedures,  for  a defined  sector  of  air  and/or  land  space  over  a sustained  period  of  time 
(Elliott  et  al.,  2001).  To  accomplish  this,  they  exchange,  interpret  and  effectively  weight  information  and  optimize 
resource  allocation  decisions  across  team  members.  As  a prime  example  of  a command,  control,  and  communications 
paradigm,  the  AWACS  team  is  an  area  where  C2  research  has  often  centered. 

The  Air  Force  has  recently  focused  on  investigations  and  enhancement  of  operational  expert  training  through 
an  internet-based  research  infrastructure  to  enhance  Distributed  Mission  Training  (DMT).  The  USAF  DMT  program  is 
national  in  scope,  with  the  goal  of  enhancing  operational  training  through  the  use  of  high-fidelity  military  simulation 
systems  that  are  networked  using  secure,  classified  systems.  The  DMT-Research  Net  (DMT-RNet)  project,  the  subject 
of  this  paper,  is  a local  project  that  will  support  DMT  through  basic  research  accomplished  using  PC-based  systems 
networked  through  the  Internet.  The  project  will  establish  an  T2  based  infrastructure  for  collaborative  research  and 
training,  along  with  identification  of  specific  research  issues  related  to  enhanced  skill  acquisition  and  operational 
performance.  This  research  will  guide  improvements  in  the  operational  USAF  DMT  training  environment. 

The  USAF  DMT  project  relies  on  a network  of  highly  realistic  battle  simulators  that  allow  expert  operators  to 
train  in  a virtual  battlespace  across  a highly  secure  and  classified  communication  network.  In  contrast,  the  DMT-Rnet 
project  is  developing  less  expensive  PC-based  systems  that  can  run  in  unclassified  mode  on  1-2.  This  allows  the  PCs  to 
be  distributed  and  deployed  as  training  systems  in  almost  any  setting.  It  also  allows  research  to  be  conducted  in 
nonclassified  environments,  using  these  simulation  systems  that  reflect  essential  components  of  operator  expertise.  For 
example,  multiple  universities  and  research  companies  can  and  have  been  networked  to  enable  real-time 
multidisciplinary  collaborative  research. 

As  a pioneering  technology,  DMT-Rnet  shows  how  other  collaborations  in  research  will  be  possible.  Other 
universities,  agencies,  and  companies  will  be  able  to  link  up  to  similar  sorts  of  networks  and  pool  their  talents  and 
resources  to  produce  high  level  research.  The  applications  of  these  concepts  and  technologies  to  other  realms  are  nearly 
limitless,  as  future  research  will  show  to  be  the  case  (Barnes,  Elliott,  & Entin,  2001). 

This  research  network  has  thus  far  developed  two  platforms.  The  initial  phase  of  the  DMT-Rnet  project 
utilized  the  dynamic  distributed  decision  making  (DDD)  team-in-the-loop  simulation  environment  (Hess,  MacMillan, 
Elliott,  & Schiflett,  1999;  Kleinman  and  Serfaty  1989).  An  internet-based  version  of  the  DDD  was  developed,  the  DDD 
Network  (DDDnet),  which  allows  players  in  distributed  locations  to  connect  and  perform  a distributed  mission  in  real 
time.  The  DDDnet  is  an  internet-ready  version  of  a Linux-based  collaborative  gaming  space  that  connects  players  to 
each  other  and  to  others,  such  as  observers,  confederates,  trainers,  or  researchers.  In  the  DDDnet  observers  at  any 
location  in  the  network  are  able  to  observe  the  scenario  play  in  real  time.  They  can  view  the  screen  display  and 
electronic  communications  of  any  player,  and  communicate  to  one  another  via  email  or  voice.  In  addition,  the  DDDnet 
can  connect  players  to  one  another  for  interactive  mission  planning,  debriefings  and  after-action  reviews. 

The  second  platform  developed  is  the  AWACS-AEDGE^*^  (Agent  Enabled  Decision  Guide  Environment). 
The  AEDGE  is  constructed  as  a federation  of  intelligent  agent-based  functions  that  enable  user-friendly  scenario 
construction,  emulation  of  friendly  and  hostile  entities,  and  dynamic  scenario  control.  The  remainder  of  this  paper  goes 
into  the  detail  of  this  platform  and  the  agent  technology  that  it  employs. 

It  has  been  stated  that  advanced  technology  in  itself  is  not  a training  system  and  should  not  be  seen  as  a 
training  solution  (Salas,  Cannon-Bowers,  & Kozlowski,  1997).  We  are  in  complete  agreement.  The  purpose  of  this 
effort  was  to  build  a platform  with  advanced-technology  features  that  will  augment,  enable,  and/or  investigate  new 
approaches,  methods  and  measures  in  training  and  performance  research. 

AEDGE^^^*  ‘ 

The  Airborne  Warning  and  Control  System  (AWACS)  Weapons  Director  (WD)  team  serves  as  a vital  airborne 
Command  and  Control  (C2)  node,  providing  airborne  surveillance,  control,  and  communications  functions  for  tactical 
and  air  defense  forces.  In  this  paper,  we  describe  development  of  an  agent-based  C2  team  decisionmaking  platform  for 
research  and  training,  the  AWACS-AEDGE^^  (Agent  Enabled  Decision  Guide  Environment). 


‘ AEDGE,  AWACS -AEDGE,  and  21CSI  are  registered  trademarks  of  2f‘  Century  Systems  Inc. 
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The  AWACS-AEDGE,  built  using  21®^  Century  Systems  Inc/s  AEDGE’*'*^  infrastructure,  is  a distributed,  real- 
time team  decision  support  environment  comprised  of  simulators,  entity  framework,  intelligent  agents  and  user 
interfaces  (Barnes  et  al,  in  review).  The  environment  supports  a wide  variety  of  air,  sea  (surface  and  sub -surface),  and 
ground  assets  in  a combat  environment  (Chiara  & Stoyen,  1997),  primarily  based  on  the  roles  and  responsibilities  of 
AWACS  WD  team  members.  The  environment  has  been  tested  with  an  excess  of  two  hundred  physical  entities  (planes, 
ships,  SAM  sites,  etc.)  operating  with  realistic  yet  non-classified  performance  characteristics  in  an  interactive 
environment  in  which  real-time  decision  support  is  available  to  each  WD. 

The  behavior  and  decisionmaking  of  all  hostile  and  friendly  entities  not  controlled  by  humans  is  directed  by 
agent-based  technology.  If  a human  decides  to  “log  in“  as  a particular  entity,  he/she  may  choose  to  view 
recommendations  generated  by  the  agent  for  that  entity.  Even  if  the  human  operator  chooses  not  to  view 
recommendations,  the  agent  recommendations  are  still  logged  by  the  computer.  This  enables  direct  comparison  of 
human  to  agent  decisionmaking.  We  expect  these  capabilities  will  facilitate  skill  acquisition,  decision  making,  skills 
assessment,  and  human/team  performance  modeling. 

AEDGE  agent  capabilities  enable  more  detailed  and  innovative  approaches  to  measurement  and  modeling  of 
individual  and  team  workload,  communication  and  decisionmaking.  Tracking  the  number  and  type  of 
recommendations  generated  by  the  agent  at  any  given  time  contributes  toward  new  ways  of  conceptualizing  and 
representing  cognitive  workload  of  individuals  and  teams.  Agent-based  recommendations  may  also  serve  as  a 
standardized  benchmark  by  which  human  tactics  and  decisions  can  be  compared.  In  addition,  the  AED  GE  platform  can 
operate  through  speech  - operators  can  speak  to  the  system  using  predefined  jargon,  request  tasks  be  performed  or 
information  provided/transferred,  and  the  agents  will  respond  verbally  to  the  speech -driven  requests,  using  voice 
generation  technology.  All  agent  communications  to  each  other,  as  well  as  to  humans,  are  transcribed,  logged  to  data 
output  files,  and  are  available  online. 

The  AWACS-AEDGE  was  conceived  through  cognitive  and  functional  analysis  of  team  member  roles, 
responsibilities,  and  decisionmaking  (Dalrymple,  1991),  to  optimize  generalizability  of  results  to  operational  settings. 
Systematic  descriptions  of  AWACS  roles,  responsibilities,  requirements,  interdependencies,  tactics,  strategies,  and  task 
demands  were  collected  from  subject  matter  experts,  cognitive  task  analyses  (Fahey  et  al.,  1998;  MacMillan  et  al., 
1998)  and  focal-group  interviews  (Elliott  et  al.,  1999;  Elliott  et  al.,  2001).  These  data  were  examined  to  identify 
decision  events,  which  were  generic  to  performance,  regardless  of  mission  scenario,  and  likely  to  bottleneck  under  high 
tempo  situations. 

AEDGE  architecture 

The  AEDGE  product  is  based  on  an  extensible  distributed  component-based  architecture,  which  defines 
entities,  agents,  players/users,  and  their  interactions.  The  interaction  and  communications  among  AEDGE  elements  are 
based  on  the  Service  Provider/Service  Requester  protocol  (SPSRP),  using  flexible  services  and  messages  to  exchange 
information  among  any  two  components. 

In  SPSRP,  Service  Providers  implement  a number  of  services  and  register  service-templates  with  a Component 
Registry,  which  maintains  the  location  of  all  components  and  the  services  provided  or  required  by  each.  The  registry  is 
used  by  Service  Requesters  to  locate  components  that  provide  the  services  required  by  the  requester.  After  the  requester 
is  matched  with  one  or  more  Service  Providers,  a direct  connection  is  established  between  Service  Providers  and 
Service  Requesters.  This  prevents  the  Registry  from  being  a service  dispatcher  and  a potential  bottleneck.  Let  us 
consider  the  interaction  between  users  (via  user  interface  components)  and  agent  components.  In  most  cases  users  will 
interface  with  specialized  Agents,  called  Agent  Managers.  The  managers  are  designed  to  coordinate,  synchronize  and 
manage  the  work  of  multiple  “worker”  agents.  Without  an  Agent  Manager,  the  user  will  need  to  interfaces  with  each 
“worker”  agent  individually,  while  using  the  Manager,  the  user  is  able  to  issue  higher-level  requests  (e.g.  “Send  me 
your  current  recommendations”)  by  letting  the  Agent  Manager  (who  knows  the  capabilities  of  its  workers)  to  distribute 
and  correlate  individual  agent  tasks. 

The  Service  Requester  then  sends  Service  objects  directly  to  one  or  more  Service  Providers,  who  respond  with 
ServiceResult  objects.  A Requester  may  wish  to  subscribe  for  service  updates,  in  which  case,  the  Service  Provider  will 
send  a Message  object  to  the  requester  every  time  it  needs  to  advertise  an  update;  it  is  up  to  the  Requester  to  respond  to 
that  message  by  requesting  the  actual  update  (i.e.  data  is  advertised,  not  pushed,  to  avoid  client-side  congestion). 

The  user  may  trigger  a request  for  recommendations,  which  is  sent  to  the  Agent  Manager  via  a Service  request 
object.  After  receiving  the  request,  the  Agent  Manager  finds  the  best-suited  collection  of  agents  to  perform  the  job  (it 
may  take  one  or  more  worker  agents)  and  forwards  specialized  Service  requests  to  each  of  them.  The  worker  agents 
formulate  their  responses  (usually,  sets  of  recommendations  and  rationale)  and  send  them  back  to  the  Agent  Manager 
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via  Service  Result  objects.  The  Agent  Manager  may  correlate  (and  even  remove  redundant  or  inconsistent 
recommendations)  all  service  results  and  then  sends  the  combined  set  of  recommendations  to  the  user,  again  via  a 
ServiceResult  object.  The  user  interface  component  then  knows  how  to  extract  the  recommendations  and  present  them 
to  the  human  user  for  evaluation. 

AEDGE  decision  making  algorithms 

The  AEDGE  architecture  provides  multiple  levels  of  agent-based  algorithms.  Generic  resource  allocation, 
search  and  optimization  algorithms  are  a core  part  of  the  AEDGE  product.  Each  AEDGE  application  can  use  and  further 
extend  these  fundamental  agent  algorithms  by  either  providing  parameters  and  applications-specific  values,  functions 
and  rules,  or  by  combining,  modifying  or  supplying  new  algorithms.  All  new  and  modified  algorithms  must  comply 
with  a well-defined  agent  interaction  interfaces,  similarly  to  the  generic  algorithms. 

The  AWACS-AEDGE  extends  resource  allocation,  optimization  and  other  algorithms  with  AWACS/WD- 
specific  objective  functions  and  constraints.  For  example,  the  AWACS  weapon-target  allocation  algorithm,  based  on  a 
generic  resource-allocation  with  heuristic  function  evaluation,  defines  extended  constraints  such  as  Table  1. 

Similarly,  the  AWACS  weapon-target  allocation  algorithms  define  objective  and  cost  functions  for  any 
potential  allocation  and  let  the  generic  allocator  agent  arrive  at  a (near-)optimal  set  of  weapon-target  pairings.  The 
objective  functions  are  based  on  the  individual  target  values  (as  well  as  other  factors,  such  as  target  priorities, 
probabilities  for  success  and  so  forth)  and  cost  functions  are  based  on  the  risk  for  the  team  if  the  allocation  is  to  be 
committed. 

Further,  the  AWACS-AEDGE  agents  use  the  extended  algorithms  as  a model  of  the  desired.  WD  performance. 
Thus,  the  agents  are  able  to  generate  a set  of  recommendations  pertinent  to  a particular  tactical  position  and  the  events 
that  lead  to  it  - agents  do  keep  even  history.  Such  recommendations  can  either  be  presented  to  the  user  (who  may 
choose  to  accept  or  ignore  them)  or  be  used  for  internal  evaluation  of  the  user’s  performance  as  a function  of  the 
similarity  of  recommended-action  versus  actually-executed-action.  A new  application  of  the  agent  recommendation 

analysis  involves  the  measurement  of  AWACS  WD 
cognitive  workload  based  on  the  volume  and  complexity  of 
agent  recommendations  at  any  given  time  (Chaiken  et  al, 
2001). 

To  enhance  the  utility  of  AWACS-AEDGE  as  a 
decision  support  tool,  recommendations  must  not  only  be 
presented  to  the  user  in  a unambiguous  and  intuitive 
manner,  but  in  some  cases  they  may  need  to  be  pre- 
processed  to  ensure  that  the  human  user  can  maintain 
strong  situational  awareness  and  be  alert.  For  example,  in 
periods  of  exceptionally  high-activity,  unsupervised  agents 
may  tend  to  generate  large  number  of  recommendations 
that  will  be  confusing  and  even  detrimental  to  the  human 
performance.  The  Agent  Manager  must  thus  not  only 
coordinate  and  synchronize  recommendations,  but  also 
prioritize  and  reduce  the  number  of  presented 
recommendations  to  only  the  top-most  critical  ones 
(Chaiken  et  al,  2001).  Conversely,  in  periods  of  lull,  artificially  increasing  the  number  of  recommendations  may  help 
keep  the  human  alert  and  situationally  aware. 

Utilization 

In  the  AEDGE,  the  experimenter  can  control  the  autonomy  and  configuration  of  the  decision  aide  agent.  If  the 
agent  is  allowed  to  make  all  decisions,  the  scenario  is  effectively  being  run  independently  of  any  human  intervention. 
This  allows  (a)  assessment  of  reliability  of  recommendations,  (b)  assessment  of  effects  of  uncertainty  in  a dynamic 
environment,  and  (c)  investigation  of  “what-if’  scenarios,  where  algorithms  underlying  recommendations  are 
manipulated. 

The  aide  is,  and  should  be,  configurable.  First,  it  needs  to  be  configurable  in  order  to  maintain  effectiveness. 
The  manner  in  which  decisions  are  made  can  and  will  change  according  to  the  particular  mission  scenario  and  rules  of 
engagement.  The  software  allows  variations  in  its  rule  structure.  It  also  allows  changes  (in  real-time)  to  the  perceived 
value  and  priority  of  various  assets  and  targets.  This  enables  fine-tuned  research  in  decision  process,  as  the  agent  can  be 


IN_RANGE(TargetCandidate, 
WeaponCandidate ) OR 

INTERCEPT_TIME(  Weapon  Platform, 
TargetCandidate ) < MAX_TIME 
AND 

Pk(  TargetCandidate,  WeaponCandidate ) 
> DesiredPk(  TargetCandidate ) 

AND 

FUEL_TO_INTERCEPT( 
WeaponPlatform,  TargetCandidate ) + 

FUEL_TO_BASE(  InterceptPoint, 
WeaponPlatform ) < WeaponPlatform.currentFuel 


Table  1:  Decision  Making  Algorithms 
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tailored  to  be  more  or  less  risk-taking  (when  information  is  uncertain),  have  directional  bias  (more  or  less  “aggressive”, 
“passive”  in  threat  assessment  or  rules  of  engagement),  or  bias  in  central  tendency  (decisions  are  always  “moderate”). 
In  addition  the  probabilistic  nature  of  the  environment  can  be  manipulated  by  specifying  the  probability  that  the 
decision  made  will  be  successfully  executed.  When  that  probability  is  very  high,  the  environment  is  deterministic  and 
very  reliable.  When  probabilities  are  lowered,  scenario  events  will  unfold  in  different  ways,  each  time  the  scenario  is 
run. 


Configurable  decision  algorithms  enable  in-depth  descriptive  and  prescriptive  investigations  of  decision 
process.  Particular  heuristics,  biases,  and  models  of  decision  choice  can  be  predicted  and  compared  to  algorithm 
function.  Results  from  descriptive  investigations  can  inform  refinement  of  the  decision  tool.  In  turn  the  algorithm  can 
be  modified  to  reflect  a particular  decision  model,  and  compared  to  other  models  with  regard  to  the  degree  to  which 
either  model  accounts  for  performance  data.  For  example,  threat  assessment  decision  events  have  been  shown  to  be 
sensitive  to  order  effects,  in  that  information  presented  first  or  last  (depending  on  tempo)  is  given  more  weight,  even 
when  other  information  is  more  important.  The  algorithm  can  be  adjusted  to  reflect  this  tendency,  and  results  compared 
to  actual  data.  Other  facets  relevant  to  the  decision  process  can  also  be  investigated  with  this  approach,  such  as  risk- 
taking, aggressiveness,  and  information  uncertainty. 

Discussion 

Despite  agent  limitations,  we  expect  the  AWACS  AEDGE™  to  enhance  research,  training,  and  performance  in 
complex  high-tempo  scenarios.  The  aide  as  decision  support  is  particularly  useful  where  multiple  decisions  must  be 
made  within  a short  time  frame  or  where  an  “out -of-the -ordinary”  event  must  be  spotted  among  numerous  seemingly 
normal  processes.  The  aide  has  the  advantage  of  complete  reliability  regardless  of  stress,  sustained  operations,  or 
consequences  of  failure.  It  will  never  forget  to  refuel  a plane  because  of  a tense  situation  occurring  at  a different  part  of 
the  scope. 

While  the  usefulness  of  the  DSS  function  is  apparent,  the  potential  utilization  of  this  platform  for  training  and 
performance  research  is  its  greatest  asset.  The  benefits  of  this  general  approach  to  STE»based  research  is  detailed 
elsewhere  (Schiflett  & Elliott,  2000).  To  summarize  briefly,  the  AWACS  AEDGE™  was  developed  to  primarily  to 
support  trainers  and  researchers.  In  fact,  every  characteristic  and  feature  within  this  platform  was  developed  in  order  to 
empower  trainers  and  researchers  with  regard  to  methods,  measures,  manipulations,  and  transfer  of  training.  First, 
internal  validity  is  enhanced  by  providing  researchers  with  more  detailed  performance  measures,  increased  scenario 
realism,  ease  in  generating  scenario  events,  agent-based  performance  models,  and  comprehensive  data  output  files.  In 
addition,  further  control  is  provided  to  team  performance  researchers  through  the  provision  of  synthetic  team 
members — thus  allowing  investigations  of  performance  within  more  highly  controlled  team  contexts.  It  provides 
trainers  with  online  scenario  revision  capabilities  and  visual  online  performance  feedback  for  operators.  Finally,  this 
system  was  developed  to  enhance  external  validity — the  degree  to  which  research  transitions  to  the  operational 
performance  environment.  This  was  accomplished  through  comprehensive  cognitive  task  analyses  of  the  operational 
performance  domain.  While  use  of  this  system  is  no  guarantee  of  good  training  or  research  per  se,  we  hope  it  will 
accomplish  its  purpose — to  provide  tools  that  empower  experts  to  more  easily  accomplish  research  and/or  training 
goals. 
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Technology  and  Collaboration:  A Qualitative  Study  in  one  Secondary  School  in  Botswana 
Introduction 

Technology  use  in  Botswana  schools  is  relatively  new  compared  to  many  other  parts  of  the  world,  and  because  of  its 
financial  implications  the  decision  to  implement  it  had  to  be  weighed  against  many  other  pressing  needs  such  as  access 
to  basic  education,  food  and  shelter.  However,  improving  the  quality  of  education  is  one  of  the  top  priorities  for  the 
country  mainly  because  education  has  been  heavily  criticized  for  failing  to  better  prepare  students  to  work  and 
participate  in  their  society.  In  1994,  the  Revised  National  Policy  in  Education  (RNPE)  was  released  and  the  policy 
stated  that  computers  be  introduced  in  schools  so  as  to  help  enhance  learning  and  also  help  the  country  it  its  transition 
from  an  agro -based  economy  to  an  industrial  one.  Computers  in  Ledumang  were  introduced  in  1997.  These  computers 
are  used  for  different  purposes.  There  is  a course  called  Computer  Studies,  taken  by  only  a few  students  because  of  the 
limited  number  of  computers.  The  course  is  aimed  at  equipping  students  with  different  computer  skills  that  would  be 
useful  for  them  once  they  leave  school.  Computers  are  also  used  in  different  subjects  to  assist  learning. 

Purpose  of  study 

The  purpose  of  this  study  was  to  investigate  the  kind  of  learning  that  takes  place  when  comp  uters  are  used,  and  also  find 
out  the  roles  that  both  the  teacher  and  students  play  in  the  classroom. 

Significance  of  study 

Computers  have  been  introduced  in  schools  as  an  effort  to  improve  learning  and  one  of  the  ways  through  which  we  can 
find  out  if  this  is  happening  is  by  examining  how  students  and  teachers  interact  with  these  computers.  Such  knowledge 
is  very  useful  for  educators  in  determining  whether  technology  is  playing  any  beneficial  role  in  education. 

Description  of  site. 

The  school  under  this  study  is  called  Ledumang  Senior  Secondary  School,  The  school  is  located  in  Gaborone,  which  is 
the  capital  city  of  Botswana.  Students  going  to  this  school  are  doing  four  and  five,  which  are  the  last  two  grades  of  high 
school.  Students  in  this  school  come  from  diverse  educational  background  in  terms  of  computer  use.  Some  come  from 
junior  high  schools  which  used  computers,  some  had  used  computers  during  their  primary  or  pre  -school  education,  and 
some  have  had  no  experience  with  computers  at  all. 

Method 

This  study  engaged  qualitative  methodology  of  in-depth  interviewing  and  participant  observation  to  collect  data. 
Interviews  were  conducted  among  students,  teachers,  headmaster  and  officials  from  the  Ministry  of  education.  Each 
interview  lasted  for  about  SOminutes.  The  total  number  of  participants  in  this  study  was  25,  There  were  15  students,  6 
teachers,  1 headmaster  and  3 officials  from  the  Ministry,  Observations  were  also  made  in  computer-assisted  classrooms 
while  in  session  to  get  a better  feel  of  what  is  happening  in  the  classes.  The  study  was  done  over  a period  of  two  months 
June  2001  to  August  2001 . 


Findings  of  the  study 

Participants  in  this  study  reported  a more  changed  environment  in  the  classroom  when  technology  is  part  of  learning. 
Teachers  reported  that  students  are  more  excited  and  enthusiastic  to  learn.  Collaboration  came  up  as  the  most  reported 
effect  of  computers  in  the  classroom.  The  teachers  mentioned  that  even  though  each  student  has  their  own  computers  to 
work  at,  they  still  spend  a lot  of  time  working  with  each  other  and  learning  from  each  other.  Students  enjoy  watching 
over  each  other’s  work  and  finding  out  how  they  do  things.  Most  of  the  time  students  prefer  to  ask  their  classmates  if 
they  do  not  understand  something  rather  than  ask  the  teacher.  The  teacher  just  acts  as  a facilitator  in  the  classroom.  One 
teacher  said  “I  think  it  is  good  when  students  work  together  and  like  this  and  learn  from  each  other,  but  I also  have  to 
make  sure  that  things  do  not  go  out  of  hand.  I have  to  make  sure  that  they  stay  on  the  job  and  do  not  just  play  around 
and  I also  have  to  make  sure  that  they  do  not  mislead  each  other.  I encourage  them  to  check  with  me  if  they  are  not  very 
sure  of  what  their  classmates  are  telling  them.”  The  students  on  the  other  hand  expressed  that  they  enjoy  working  with 
computers  because  they  are  kept  so  busy  and  the  teacher  allows  them  to  freely  talk  with  each  other,  do  things  together, 
which  is  not  always  something  they  do  in  other  classes. 

Collaboration  is  also  enhanced  outside  the  classroom  and  school  premises.  The  teachers  report  that  they  have  put  up  an 
e-mail  system  for  students  to  communicate  with  students  in  other  senior  secondary  schools  in  the  country  to  discuss 
their  work.  Students  also  communicate  with  other  students  in  the  United  Kingdom,  which  they  find  very  exciting.  This 
gives  them  an  opportunity  to  bond  with  students  in  other  parts  of  the  world  and  learn  from  them. 
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Another  significant  way  in  which  collaboration  is  fostered  is  through  the  final  project  that  is  done  by  form  five  students. 
At  the  end  of  their  senior  secondary  education,  students  write  final  examinations,  which  mainly  determine  their 
acceptance  into  institutions  of  higher  education  or  work  places.  As  part  of  these  examinations,  students  doing  computer 
studies  have  to  do  a project  in  which  they  consult  with  local  companies  or  government  agencies  and  together  with  the 
help  of  the  teacher,  identify  a problem  that  they  would  like  to  tackle  and  use  computer  simulations  to  work  out  the 
solution.  Students  work  on  this  project  pretty  much  for  the  whole  year.  During  the  whole  process,  students  work  closely 
with  the  teacher  and  the  companies  they  are  working  for.  Students  get  grades  for  this  work  and  they  also  have  to  present 
their  solutions  to  the  companies  and  discuss  their  findings.  This  gives  students  a great  opportunity  to  interact  and 
collaborate  with  other  people  outside  their  school  environment.  It  also  helps  to  narrow  down  the  gap  between  school 
knowledge  and  the  outside  world.  In  this  way  students  can  make  a connection  between  what  they  learn  at  school  and 
what  is  happening  outside.  The  companies  even  hire  some  of  these  students. 

Conclusion 

Collaboration  is  enhanced  when  students  are  encouraged  to  work  with  each  other.  They  learn  to  respect  each  other’s 
opinion  and  take  criticism.  These  are  some  of  the  qualities  that  they  would  need  once  they  enter  the  world  of  work. 
Taking  learning  outside  the  classrooms  or  even  school  premises  is  very  important  in  helping  students  see  the  connection 
between  what  they  are  doing  and  real  life.  The  teacher  play  a crucial  role  in  directing  this  kind  of  learning  and  students 
on  the  other  hand  take  a big  responsibility  for  their  learning. 
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Abstract:  This  paper  describes  the  design  and  development  of  TruVision,  an  online 
learning  environment  designed  to  enable  blind  and  vision  impaired  students  to 
develop  skills  and  expertise  in  elementary  and  advanced  information  processing 
strategies  to  enable  them  tom  seek  full-time  employment  within  industry  in  such 
positions  as  administrative  assistants,  Help  Desk  personnel  and  data  entry  operators. 
Truvision  is  a product  within  The  Flexible  Learning  Toolboxes  Project,  a component 
of  the  Australian  Flexible  Learning  Framework  for  the  National  Vocational 
Education  and  Training  System  2000-2004  (AFL  Framework).  The  AFL 
Framework  is  designed  to  support  the  accelerated  take-up  of  flexible  learning  modes 
and  position  Australian  VET  as  a world  leader  in  applying  new  technologies  to 
vocational  education  products  and  services.  This  paper  describes  the  TruVision 
product  and  showcases  its  innovative  design  based  on  very  stringent  accessibility 
needs  and  guidelines. 


Introduction 

Online  learning  environments  in  higher  education  are  frequently  criticised  for  their  failure  to  make  the  most  of 
the  opportunities  afforded  by  the  new  technologies  (eg.  Dehoney  & Reeves,  1999).  Too  often  the  design  of  the 
settings  is  constrained  by  such  practical  factors  as  inflexible  delivery  systems,  the  need  for  converged  on  and  off- 
campus  settings,  and  the  quest  to  generate  on-line  courses  from  existing  print-based  materials.  Rarely  do 
developers  have  the  opportunity,  the  direction  and  the  support  to  plan  and  build  high  quality  online  learning 
materials.  One  project  in  Australia,  however,  that  differs  from  others  in  this  regard,  is  the  Flexible  Learning 
Toolboxes  Project  (ANTA,  2001). 

The  Toolbox  initiative  is  part  of  the  Australian  Flexible  Learning  Framework  for  the  National  Vocational 
Education  and  Training  System  2000-2004  (AFL  Framework).  The  AFL  Framework  is  designed  to  support  the 
accelerated  take-up  of  flexible  learning  modes  and  position  Australian  VET  as  a world  leader  in  applying  new 
technologies  to  vocational  education  products  and  services  (EdNA  VET  Advisory  Group,  2000).  A Toolbox  is 
a collection  of  resources,  suggested  learning  strategies  and  supporting  material  for  the  online  delivery  of 
vocational  education  and  training.  The  learning  resources  are  Web-based  and  designed  in  a manner  which 
facilitates  customisation  and  reuse  in  the  National  Training  Framework,  the  basis  of  qualifications  and 
accreditation  in  the  Australian  VET  sector. 

Flexible  Learning  in  the  VET  sector 

National  and  state  authorities  responsible  for  VET  within  Australia  have  recognized  for  some  years  that 
developing  online  materials  is  expensive  and  best  achieved  through  a collaborative  approach.  It  was  this 
challenge  that  prompted  the  Australian  National  Training  Authority  (ANTA)  to  consider  strategies  that  could 
motivate  and  support  registered  training  organisations  (RTOs)  to  embrace  modes  of  flexible  delivery  for  their 
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students.  A resource  strategy  was  formulated  by  which  sets  of  generic  and  customisable  materials  could  be 
developed  and  applied  widely  throughout  the  training  sector.  The  intention  was  to  create  efficiency  and 
economies  of  scale  though  large  scale  production  of  learning  materials  capable  of  wide  acceptance  and  use.  This 
plan  became  the  ANTA  Flexible  Toolbox  Project  which  has  evolved  since  1999  into  the  Flexible  Learning 
Toolbox  Project  of  today.  The  Toolbox  initiative  was  a strategy  to  encourage  development  and  delivery  of  more 
flexible  learning  materials  for  the  training  market  - particularly  for  online  learning. 


Flexible  Learning  Toolboxes 

The  phrase  “Training  Package”  has  a specialised  meaning  in  the  Australian  VET  sector,  referring  to  a nationally 
endorsed  statement  incorporating  the  competency  standards,  assessment  guidelines  and  qualifications  relating  to 
training  in  a particular  industry  sector.  These  components  are  uniform  at  a national  level  but  allow  for  a range  of 
flexible  training  pathways  for  achievement  and  assessment. 

The  introduction  of  Training  Packages  within  the  National  Framework  in  1998  provided  a perfect  opportunity 
for  the  Australian  National  Training  Authority  to  explore  the  concept  of  Flexible  Toolboxes  to  support  learning. 
In  the  first  instance,  any  products  developed  in  this  setting  would  have  widespread  application  providing 
significant  economies  of  scale.  Secondly  the  use  of  on-line  technologies  appeared  to  provide  and  promote  the 
many  forms  of  flexibility  associated  with  the  delivery  of  the  Training  Packages  in  the  national  setting. 

The  Flexible  Learning  Toolbox  Project  has  carried  three  main  development  stages  to  date,  from  Series  One  h 
1999  to  the  recently  completed  Series  Three.  Developers  bid  for  funding  to  build  Toolboxes  from  funding 
managed  by  the  Flexible  Learning  Advisory  Group  (FLAG),  a committee  with  strong  interest  and  expertise  in 
flexible  delivery  in  the  VET  sector.  Teams  and  consortia  work  together  to  develop  bids  to  build  online  learning 
resources  for  qualifications  within  the  national  Training  Packages.  Teams  are  required  to  demonstrate  strong 
links  with  the  industry  groups  and  to  propose  online  materials  that  demonstrate  evidence  of  sound  contemporary 
learning  designs  and  development  processes  that  create  materials  with  high  levels  of  product  utility.  The 
following  descriptions  from  the  developers’  guidelines  highlight  these  aspects  of  Toolbox  design: 

a.  Toolbox  learning  designs 

A fundamental  requirement  of  Toolbox  resources  is  the  need  to  exhibit  effective  teaching  and  learning 
approaches.  To  support  this  requirement,  proponents  need  to  demonstrate  their  capacity  to  develop 
resources  with  the  following  features: 

• a firm  basis  in  an  educational  model  which  recognises  an  active,  constructive  role  for  learners; 

• learning  activities  which  engage  the  learner  in  active  processing  of  the  subject  matter  rather  than 
mere  knowledge  acquisition; 

• learning  settings  and  tasks  that  encourage  meaningful  online  communication  and  interaction 
(between  learners  as  well  as  between  teachers  and  learners); 

• content  resources  which  are  visually  attractive,  motivating  to  use  and  organised  logically  for  ease 
of  navigation;  and 

• representations  of  authentic  and  real  life  settings  in  preference  to  textual  descriptions. 

b.  Toolbox  product  utility 

A second  important  consideration  concerns  product  utility  characteristics.  Toolboxes  are  expected  to 
allow  for  wide  applicability  across  the  target  audience  for  whom  they  are  intended.  For  this  reason  they 
are  expected  to  be  flexible,  portable  to  a range  of  delivery  platforms,  and  readily  customised.  Desirable 
characteristics  include: 

• the  use  of  readily  available  non-proprietary  development  software; 

• the  use  of  HTML  code  where  customisation  can  be  anticipated,  with  more  sophisticated 
development  software  (eg  Flash)  reserved  for  components  that  are  unlikely  to  be  changed; 

• the  development  of platform  independent  resources  which  allow  for  maximum  portability  to  users ' 
delivery  platforms; 

• avoidance  of  rigid  structuring  devices  locking  the  learner  into  one  pathway  through  the  material 
(although  a suggested  learning  sequence  may  be  desirable);  and 

• a file  and  directory  structure  that  facilitates  the  location  of  a particular  learning  segment,  or  the 
selection  of  a single  unit  as  well  as  the  use  of  the  whole  Toolbox. 
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Toolboxes  are  also  expected  to  be  widely  accessible.  They  are  expected  to  operate  on  client 
computers  running  at  300  MHz  and  guidelines  are  provided  to  developers  for  aligning  the  Toolboxes 
with: 

• the  EdNA  metadata  standards; 

• the  W3C  content  accessibility  guidelines  (Priority  1);  and 

• the  Preferred  Standards  Project. 

TruVision 

TruVision  (http://www.clcam.wa.cdu.au/truvision)  is  one  of  six  ANTA  Equity  and  Access  Online  Product 
Development  projects  commissioned  in  2001.  It  is  an  online  learning  environment  that  has  been  designed  to 
offer  on-  and  off-line  options  in  the  vocational  education  sector  for  learners  who  are  blind  or  vision  impaired. 
Through  the  establishment  of  a simulated  Help  Desk  entity,  TruVision  provides  a variety  of  authentic,  work- 
based  online  learning  experiences  that  prepare  learners  for  vocational  outcomes  in  the  IT  industry.  IT  Help  Desk 
occupations  are  particularly  attractive  to  people  with  a visual  disability. 

The  framework  that  was  used  to  inform  and  guide  the  instructional  design  process  is  a component  model, 
described  in  Table  1,  which  comprises  three  elements  which  represoit  critical  components  of  any  learning 
settings.  In  particular,  the  framework  highlights  connections  and  distinctions  between  the  elements  which  can  be 
made  in  the  design  of  online  learning  settings  (Oliver,  1999). 


Table  1:  Framework  describing  critical  elements  of  online  learning  settings 


learning  design 
elements 

description 

examples 

learning 

activities 

The  tasks,  problems,  interactions 
used  to  engage  the  learners  and  upon 
which  learning  is  based 

Reading  activities,  computer-based  interactions, 
simulations,  inquiry  tasks,  projects,  open-ended 
problems,  inquiry  tasks,  collaborative  tasks 

learning 

resources 

The  content,  information  and 
resources  with  which  the  learners 
interact  in  completing  the  tasks 

Web  pages,  readers,  textbooks,  computer-based 
tools,  Web  links,  notes,  documents,  workplace 
manuals,  case  studies,  databases 

learning 

supports 

The  scaffolds,  structures, 
motivations,  assistances  and 
connections  used  to  support  learning 

Learning  guides,  discussions,  chats,  suggested 
learning  pathways,  mentors,  buddies,  workplace 
trainers 

The  framework  provides  a means  to  isolate  and  study  the  various  elements  within  learning  settings  and  suggests 
emphases  which  can  be  made  in  the  instructional  design  process.  Contemporary  learning  theories  posit  that  the 
forms  of  learning  design  most  appropriate  to  higher  education  are  those  based  on  constructivist  learning 
principles.  The  above  framework  takes  on  particular  and  discrete  forms  when  applied  this  way. 

In  the  context  of  the  instructional  design  for  the  TruVsion  setting,  authentic  tasks  were  used  as  the  organising 
framework  for  the  learning  process.  Students  are  provided  with  realistic  and  relevant  tasks  which  they  are 
required  to  complete.  In  order  to  assist  them  in  completing  the  tasks,  a range  of  learning  resources  and  scaffolds 
for  learning  are  provided.  The  learning  setting  is  comprised  of  these  carefully  chosen  learning  tasks  and  learning 
supports  to  scaffold  learners  as  they  undertake  them.  The  content  in  the  setting  takes  the  form  of  a variety  of 
learning  resources  which  are  organised  within  the  virtual  office  setting  in  much  the  same  way  as  the  learners  will 
find  resources  in  the  workplace. 

Design  Considerations 

With  most  online  settings,  the  instructional  design  precedes  any  decisions  about  interface  design.  In  this 
product,  it  was  essential  to  marry  the  two  so  that  the  learning  setting  would  provide  the  accessibility  required. 
The  design  of  TruVision  needed  to  go  beyond  compliance  to  W3C  standards  alone.  In  its  final  form,  it  provides 
an  audio  and  text -based  interface  for  people  that  are  totally  blind  so  that  they  can  hear  the  learning  context  and 
interactions  through  a combination  of  screen  reader  and  streamed  audio.  In  addition,  TruVision  interprets  this 
audio  and  text -based  interface  with  a graphical  version  for  people  with  vision.  In  this  way,  two  versions  of  the 


139 


learning  environment  are  provided  on  each  and  every  web  page,  both  of  which  are  invisible  to  the  other.  The 
concept  is  represented  as  Figure  1. 


Hidden  interpretive  text  for  screen  reader  software 
(eg  Jaws). 

Audio  link  extracted  from  Flash 

Navigation  links  extracted  from  Flash 


Figure  1:  The  TruVision  Design  Model 


The  TruVision  Setting 

The  task  of  designing  for  the  range  of  vision  impairment  and  adaptive  technologies  was  complex.  The  following 
sections  of  the  paper  explain  the  design  considerations  underpinning  TruVision,  and  demonstrate  how 
educational  design  decisions  were  taken. 


Figure  2:  Authentic  learning  tasks 
TruVision  is  a learning  environment  based 
around  authentic  tasks.  The  learners  find 
themselves  immersed  in  a virtual  workplace 
and  undertake  tasks  and  activities  which 
accurately  reflect  how  they  will  need  to  apply 
their  learning  in  the  workplace.  Assessment, 
activities  and  resources  are  closely  aligned  with 
the  learning  outcomes  that  are  sought. 
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Figure  3:  Rich  media  usage 
In  order  to  provide  accessible  materials  for  blind 
and  vision  impaired  learners,  the  setting  makes 
extensive  use  of  audio  and  visual  materials  to 
cater  for  the  range  of  people  that  are  vision 
impaired.  The  learners  use  the  site  with  the  same 
tools  and  processes  that  vision  impaired  people 
currently  use  in  workplace  settings. 


Figure  4:  Customisation  and  Reuse 
The  learning  setting  has  been  designed  and  built 
for  customisation  and  reuse.  There  are  many 
elements  within  to  support  this  application 
including  a comprehensive  Teachers*  Guide  to 
assist  others  to  implement  TruVision.  The 
Teachers’  Guide  provides  a range  of  suggestions 
and  strategies  to  help  facilitators  balance  content 
and  collaborative  dimensions  of  online  learning. 


Figure  5:  Assessment 

The  learning  setting  includes  a range  of  assessment 
activities  to  support  independent  and  classroom 
learning  usage.  The  assessment  tasks  include  a range 
of  formative  and  summative  strategies  as  both 
supports  and  scaffolds  for  learning.  The  assessment 
tasks  also  provide  strong  measures  of  competence 
and  achievement  of  course  aims  through  the  authentic 
tasks  on  which  they  are  based. 


Figure  6:  Flexible  use 

TruVision  has  been  designed  in  ways  which  will 
support  a variety  of  instructional  applications  and 
able  to  support  learners  with  a range  of  learning 
styles.  The  flexible  nature  of  the  setting  with  a 
separation  of  learning  tasks,  learning  resources  and 
learning  supports  makes  it  a tool  that  can  be  adapted 
to  suit  the  needs  of  a broad  range  of  learners  and 
their  preferred  learning  styles. 
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User-Testing 

The  development  of  TruVision  was  accompanied  by  a rigorous  and  comprehensive  program  of  usability  testing 
to  ensure  the  product  was  able  to  meet  the  needs  of  the  diverse  target  audience  (eg.  Nielsen,  1 993).  A 
comprehensive  user-testing  program  was  undertaken  as  part  of  the  design  process  and  feedback  was  gathered 
from  learners  in  a variety  of  contexts.  These  include  learners  using  screen  readers,  screen  magnification 
software  and  with  learners  who  were  using  no  adaptive  technologies  at  all.  This  program  was  undertaken  in 
collaboration  with: 

-The  Association  for  the  Blind  of  WA  (Inc) 

-The  Royal  Blind  Society  (NSW) 

-The  Royal  Victorian  Institute  of  the  Blind 

Coupled  with  the  usability  testing  was  a series  of  formative  assessments  to  explore  the  success  of  the  learning 
environment  among  learners  who  were  unaccustomed  to  online  and  computer-based  environments.  More 
comprehensive  testing  and  evaluation  of  the  learning  setting  is  being  planned  to  enable  its  capability  to  serve  the 
diverse  audience  to  be  more  fully  explored.  This  testing  will  also  explore  strategies  which  will  enable  the 
learners  to  undertake  collaborative  and  communicative  tasks  and  to  enjoy  the  enhanced  learning  opportunities 
that  these  strategies  afford  sighted  learners. 

Conclusion 

The  TruVision  project  is  going  a long  way  to  be  meeting  the  expectations  of  the  stakeholders  in  terms  of 
innovative  and  accessible  design.  Initial  feedback  from  learners  and  teachers  suggests  that  it  is  a successful 
learning  setting  for  the  target  audience  and  that  it  provides  a valuable  template  for  sound  educational  practice 
coupled  with  accessibility  for  other  online  product  development  to  follow. 
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Abstract 

As  computing  and  embedded  systems  become  ubiquitous  in  our  world,  the  importance  of  user  interface 
design  knowledge  increases  in  our  curriculum.  Students  of  undergraduate  information  systems  or  computer 
science  programs  should  possess  some  competence  in  this  computing  sub-discipline.  However,  many 
programs  do  not  have  the  curricular  space  to  host  a separate  course  in  usability  or  user  interface  design.  To 
address  this  concern,  results  and  observations  of  incorporating  user  interface  design  pedagogy  in  the 
context  of  a software  engineering  project  course  are  presented.  The  project  centers  around  a data  collection 
application  to  be  hosted  on  a PDA  (Personal  Digital  Assistant).  The  application  has  significant  constraints 
concerning  usability  and  human  factors  that  provide  a rich  context  for  teaching  and  demonstrating  user 
interface  design  concepts. 

Keywords:  user  interface  design,  usability,  ergonomics,  software  engineering,  laboratory  course,  PDA. 


1.  INTRODUCTION  packing  more  topics  into  a finite  curricular  space. 

Therefore,  an  approach  that  has  gained  momentum 
While  change  in  computing  seems  inevitable,  is  to  integrate  some  topics 
educators  continue  to  wrestle  with  the  problem  of 

into  the  fabric  of  courses  rather  than  creating  individual  courses  for  each  topic  [Johnson  1997].  In  this  work, 
we  will  investigate  the  goals  and  outcomes  of  integrating  user  interface  design  pedagogy  into  a traditional 
software  engineering  course.  Specifically,  we  will  look  at  the  benefits  and  detriments  of  utilizing  the  Palm 
PDA  platfomi  as  the  focus  of  a semester-long  student  project. 

The  software  engineering  course  for  this  project  is  a four-credit  course  taken  during  the  senior  year  of  a 
traditional  undergraduate  program  in  Computer  Science.  A significant  component  of  this  course  is  the 
semester  long  project.  The  project  allows  students  to  work  in  teams  of  no  more  than  three  students.  The 
four  phases  of  the  project  are:  requirements  specification,  design  specification,  implementation,  and  test 
plan  design  and  execution.  Thus,  the  teams  have  the  benefit  of  seeing  a project  through  many  typical 
phases  of  a software  lifecycle. 

Of  particular  interest  is  the  utilization  of  the  Palm  OS®  platform  as  a vehicle  to  reinforce  fundamental  user 
interface  design  and  software  engineering  principles.  This  project  concept  has  been  employed  for  two 
consecutive  academic  years.  The  students  have  a background  in  Java/C++  programming,  but  no  prior 
experience  with  the  Palm  OS®  platform.  The  implementation  and  testing  phases  require  the  use  of  the 
Metrowerks  Code  Warrior®  IDE,  which  supports  C++  for  the  Palm  OS®  [Rhodes  1999]. 
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2.  USER  INTERFACE  DESIGN  GOALS 


The  software  project  in  this  instance  is  a basketball  statistics  data  collection  program  for  the  Palm  IIIc  and 
Palm  Illxe  PDAs.  The  program  must  allow  a novice  Palm  user  to  track  basic  statistics  (e.g.,  field  goals 
attempted/made,  personal  fouls)  on  an  individual  player  and  team  basis  during  the  fast-paced  action  of  a live 
NCAA  basketball  game.  Therefore,  the  program’s  interface  must  be  simple,  minimize  the  user’s  memory 
load,  provide  intuitive  short  cuts,  allow  for  easy  reversal  of  actions  (e.g.,  undo),  and  prove  to  be  reliable  in 
the  real-time  setting  of  a basketball  game.  Participants  are  given  the  opportunity  to  test  their  product  at  an 
actual  NCAA  Division  II  basketball  game  during  the  final  week  of  classes  of  the  fall  semester. 

During  lecture,  students  are  given  the  opportunity  to  dialog  on  a number  of  outside  resources  related  to 
user  interface  design.  Students  are  exposed  to  a variety  of  literature,  some  from  the  popular  press  [Carlton 
1994]  that  illustrate,  often  in  a very  humorous  fashion,  the  broad  range  of  expertise  that  is  common  among 
users.  Students  are  also  given  supplemental  material  from  the  traditional  user  interface  design  discipline 
[Shneiderman  1998,  Nielsen  1993].  The  following  concepts  are  presented  in  this  segment  of  the  course: 

• Prototyping 

• Parallel  Design 

• Know  the  User 

• Heuristics  for  Interface  Design 

• Usability  Testing. 

The  students  are  required  to  develop  interface  prototypes  as  part  of  their  requirements  and  design 
specification  documents.  The  value  of  this  has  many  facets,  not  the  least  of  which  is  the  reinforcement  of 
demonstrating  an  iterative  design  process.  Thus,  we  see  an  effective  interface  developing  overtime  rather 
than  developed  in  its  entirety  the  first  time.  Furthermore,  since  each  team  is  developing  a unique  interface, 
students  may  then  compare  designs  prior  to  implementation  in  order  to  see  the  benefits  of  participating  in  a 
parallel  design.  Related  to  this  point  is  the  need  for  the  instructor  to  communicate  a relaxing  of  the 
traditional  standards  of  academic  dishonesty  (i.e.,  we  encourage  the  participants  of  this  course  to  share 
ideas  and  to  incorporate  the  “best  of  class”  into  their  own  implementation). 

Classical  user  interface  design  pedagogy  invariably  includes  the  novice/expert  user  continuum.  To  apply 
this  knowledge,  students  are  encouraged  to  consider  the  attributes  of  the  typical  user  of  their  system. 
Basketball  scouts  are  typically  assistant  coaches  who  are  novice  Palm  users  that  will  use  the  system  on  an 
intermittent  basis.  Thus,  many  of  the  heuristics  discussed  in  the  course  are  applicable. 

Simplicity  of  the  interface  is  essential.  For  starters,  the  typical  PDA  does  not  provide  an  abundance  of 
display  real  estate.  Secondly,  all  game  time  features  need  to  be  placed  on  a single  screen  (navigating 
multiple  screens  during  the  fast-paced  action  is  too  burdensome).  Student-developers  are  also  encouraged 
to  provide  informational  feedback,  minimize  the  user’s  memory  load,  and  provide  consistent  short  cuts 
where  possible.  Given  that  users  will  invariably  make  input  mistakes,  the  system  must  allow  for  the  easy 
reversal  of  actions  as  well.  The  system  also  provides  the  opportunity  to  emulate  direct  manipulation  versus 
indirect  manipulation  as  developers  are  encouraged  to  minimize  user  input  via  Graffiti^  in  favor  of  stylus 
input  (potentially  integrating  an  error  prevention  strategy  into  the  interface  design). 

The  course  introduces  students  to  the  differences  between  utilizing  heuristics  for  user  interface  design  (as 
discussed  above)  and  actual  usability  testing.  Although  students  don’t  typically  have  the  time  typically 
(nor  the  inclination)  to  recruit  human  subjects  for  usability  studies,  they  are  encouraged  to  give  their  system 
to  a non-CS  student  for  review.  The  process  of  looking  at  a software  system’s  interface  through  someone 
else’s  eyes  is  an  enlightening  experience  for  many. 


3.  SOFTWARE  ENGINEERING  GOALS 
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The  software  engineering  course  that  is  the  focus  of  this  research  is  in  many  ways  traditional.  We  utilize  a 
well-known  text  [Pressman  2001].  We  teach  a variety  of  process  models,  paradigms,  and  techniques  with  an 
eye  toward  balancing  theory  with  practice.  However,  after 

ten  years  of  teaching  this  course,  we  have  found  difficult  to  reinforce  beyond  telling  (i.e.,  our 

that  some  unique  aspects  of  our  Palm  OS®  project  assumption  is  that  telling  is  not  teaching  and  that 

illuminate  certain  lessons  that  are  historically  students  will 

embrace  that  which  they  self-discover  through  practice).  Those  lessons  are: 

• Difficulties  of  software  maintenance 

• Learning  new  tools/platforms 

• Differentiating  essential  from  accidental 

• Applicability  of  non-technical  skills 

• Working  in  a team  environment 

• Dealing  with  non-contrived  constraints. 

The  Palm  project  in  this  course  gives  many  students  their  first  hands-on  experience  with  software 
maintenance.  For  most  of  them,  the  project  represents  their  first  attempt  at  modifying  a non-trivial  program 
written  by  another  developer.  This  is  accomplished  with  the  students  being  given  a base  Palm  C++  project 
as  a jump-start.  The  base  program  includes  over  1400  lines  of  C++  spread  over  four  classes.  The  original 
motivation  was  that  a typical  Palm  application  was  deemed  too  large  for  undergraduate  students  to  develop 
in  the  context  of  a one  semester  course,  given  that  they  must  develop  requirements  and  design 
specifications  before  commencing  with  implementation.  Students  have  typically  added  another  500  to  1,500 
lines  of  C++  to  the  base  project.  Post  semester  student  assessments  have  shown  that  at  the  conclusion  of 
the  project,  students  say  they  would  prefer  to  write  their  own  application  as  opposed  to  maintaining  code 
written  by  someone  else.  Thus,  we  have  achieved  a significant  learning  outcome  by  allowing  students  to 
self-discover  the  hardships  of  software  maintenance. 

The  Palm  project  also  provides  students  with  the  opportunity  to  apply  their  knowledge  of  object-oriented 
software  development  to  a novel  taiget  platform.  For  all  of  the  students  to  date,  this  represents  their  first 
exposure  to  the  Palm  OS®  Metrowerks  CodeWarrior®  development  environment.  Although  the  learning 
curve  for  a new  tool  causes  a certain  level  of  anxiety  in  most  of  us,  the  experience  creates  a nice  backdrop  to 
discuss  the  differences  between  the  essential  and  accidental  activities  for  a software  project  [Brooks  1986]. 
Thus,  we  are  able  to  highlight  the  essential  nature  of  many  non-technical  skills  such  as  writing  and 
speaking.  Since  the  project  includes  written  documents  (requirements,  design,  test  plan)  in  addition  to 
source  code,  students  are  able  to  see  first  hand  the  essential  nature  of  effective  communication.  The 
potential  pitfalls  of  communication  are  further  highlighted,  some  might  say  exacerbated,  by  virtue  of  the  fact 
that  the  students  work  in  teams. 

Perhaps  the  most  beneficial  attribute  of  a learning  experience  is  creating  an  enthusiastic  atmosphere.  We 
have  found  that  the  combination  of  a novel  platform  and  a believable  application  provides  students  with 
enthusiasm  in  that  they  see  the  project  as  more  “believable”  than  others  that  may  have  been  contrived  in 
the  past.  At  the  start  of  the  term,  students  are  shown  a commercial  Palm  OS®  application  for  tracking 
football  statistics.  The  real-time  differences  between  football  and  basketball  game  situations  provide  a nice 
scenario  for  discussing  usability  issues.  Center  stage  in  this  discussion  is  the  fact  that  football  provides 
frequent,  short  breaks  in  the  action  that  basketball  does  not  permit.  Fortunately,  college  students  are  almost 
universally  aware  of  the  differences  between  the  games,  although  the  presence  of  international  students 
provides  a ready  reminder  of  cultural  issues  in  software  design.  Thus,  the  elaborate  multi-screen  interface  in 
the  football  scouting  application  is  insufficient  in  this  case. 


4.  DIFFICULTIES  & DISTRACTIONS 

Without  question  this  project  provides  some  hurdles  that  every  team  mentioned  in  their  post-mortem 
survey: 
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Limitations  of  the  Palm  screen 
Lack  of  implementation  tools 
Frustrations  with  maintenance. 


• 

Due  to  a combination  of  the  usability  demands  of  the  application  and  the  limited  screen  size  on  the  Palm  111, 
all  development  teams  concluded  that  the  challenges  in  user  interface  design  were  more  significant  than 
they  had  faced  previously.  Specifically,  participants  noted  that  buttons  needed  to  be  large  enough  to  tap 
accurately,  yet  not  so  large  as  to  chew  up  too  much  screen  real  estate.  On  a related  note,  teams  also 
mentioned  that  they  were  challenged  to  come  up  with  meaningful  abbreviations  for  each  button  (e.g., 
“FreeTAtm’'  or  “FTA”  as  an  abbreviation  for  “free  throw  attempt”).  For  these  reasons  and  others,  the  use 
of  the  PDA  in  this  course  provides  a pedagogical  platform  that  allows  students  to  deal  with  interface  design 
issues  that  are  commonplace  among  practitioners. 

Every  team  lamented  the  fact  that  they  were  limited  to  the  C++  programming  language  and  the  Code  Warrior® 
IDE.  Participants  are  accustomed  to  a richer  set  of  interface  development  tools  such  as  JBuilder®  or  Visual 
Basic®.  Previous  work  has  considered  user  interface  design  in  the  context  of  the  PC  [Battig  2000].  However, 
the  PDA  target  environment  is  significantly  different  from  the  PC.  For  starters,  the  resources  of  memory 
(both  primary  and  secondary)  and  processor  are  significantly  less  on  the  PDA.  Secondly,  the  PDA 
development  platform  has  not  been  around  for  over  two  decades  like  the  PC  and  thus  has  not  been  the 
recipient  of  the  sophisticated  tools  that  accompany  such  tenure.  The  result  is  that  students  are  given  the 
opportunity  to  learn  a new  development  platform  during  development  in  much  the  same  way  as  practitioners 
do  routinely. 

On  the  post-mortem  surveys,  a majority  of  teams  indicated  that  they  would  have  preferred  to  develop  their 
own  application  from  “scratch”  instead  of  maintaining  an  existing  one.  For  most,  this  is  the  first  exposure  to 
the  tyranny  of  software  maintenance.  Professional  developers  experience  this  frustration  more  than  some 
care  to  admit.  However,  the  constraints  of  the  marketplace  do  not  usually  permit  developers  to  take  the  time 
and  money  to  build  new  systems,  even  when  it  seems  justified  from  their  perspective.  Therefore,  the  project 
has  the  benefit  of  demonstrating  to  participants  the  difficulties  inherent  in  software  maintenance. 


5.  CONCLUSION 

The  results  and  observations  of  using  the  PDA  platform  to  teach  user  interface  design  and  implementation 
to  undergraduate  computing  students  include  many  benefits.  Students  are  confronted  with  usability 
obstacles  that  are  more  challenging  than  most  they  have  faced  to  date.  For  example,  several  project  teams 
labeled  their  player  buttons  in  sequence  (from  1 to  5)  instead  of  using  the  player  jersey  number.  This 
example  provides  a poignant  application  of  reducing  the  user’s  memory  load.  Although  usability  heuristics 
like  this  are  commonly  taught,  one  final  attribute  of  this  project  merits  discussion.  Because  the  PDA 
platform  is  fairly  novel  (i.e.,  most  students  own  a PC,  but  none  of  the  participants  own  a PDA),  it  provides  a 
“gee-whiz”  factor  that  increases  student  motivation.  In  other  words,  students  are  more  enthusiastic 
participants  and  learners  in  a project  that  they  perceive  is  using  cutting-edge  technology. 

The  project  also  addresses  many  software  engineering  issues  that  present  problems  for  pedagogues. 
Because  this  project  is  larger  than  the  typical  student  project  (several  thousand  lines  of  code),  it  presents 
issues  of  software  engineering  that  are  hard  to  expose  in  the  classroom  and  yet  are  common  in  practice 
[Dawson  1997].  Chief  among  the  software  engineering  issues  is  the  difficulty  associated  with  software 
maintenance. 

Most  computing  curricula  [ACM  1991,  AITP  1997]  recognize  the  importance  of  user  interface  design. 
However,  many  undergraduate  programs  do  not  have  the  freedom  in  the  curriculum  to  include  a separate 
course  in  user  interface  design.  Therefore,  the  benefits  of  teaching  usability  issues  in  a related  course  (such 
as  software  engineering)  prove  compelling.  As  an  added  benefit,  project  participants  begin  to  see  the 
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important  ways  in  which  many  usability  heuristics  are  integrated  into  the  fabric  of  user  interface  design, 
software  implementation,  and  human  relationships. 
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APPENDDC 

To  give  the  reader  a better  understanding  of  the  data  collection  application  and  user  interface  employed  in  this 
project,  this  section  will  highlight  the  evolution  of  the  actual  interface  developed  by  one  of  the  development  teams. 
The  user  interface  design  changes  were  the  result  of  feedback  that  students  received  from  four  sources:  course 
materials  on  usability,  direct  instmctor  feedback,  fellow  classmate  feedback,  and  outside-the-course  student 
feedback. 


Figure  1 . First  Prototype  User  Interface 
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Figure  1 shows  the  first  interface  implemented.  This  interface  has  numerous  deficiencies.  The  most  glaring  is  the 
memory  load  placed  on  the  user  to  associate  players  on  the  floor  with  the  numbers  1 through  5.  Figure  2 shows  the 
improved  interface  with  jersey  numbers  providing  a more  natural  approach.  Note  also  that  the  improved  version 
provides  the  ability  to  reverse  previous  actions  via  the  “Undo”  button.  The  latter  interface  includes  features  for  two 
teams  instead  of  Just  one.  The  developers  of  this  interface  also  incorporated  some  feedback  on  usability  to  change 
the  abbreviations  used  on  the  buttons.  Notice  that  the  abbreviation  for  a three-point  shot  attempt  changed  from 
“3pa”  to  “3ptAtm.” 


Figure  2.  Revised  User  Interface 


The  revised  interface  in  Figure  2 also  provides  buttons  for  substitutes  (“Subs”)  and  summary  of  statistics  (“Report”). 
Screen  shots  for  these  features  are  shown  in  Figures  3 and  4 respectively.  To  make  a substitution,  the  user  clicks  a 
player  currently  on  the  floor  and  a player  currently  on  the  bench  and  then  clicks  “Sub.”  Figure  3 shows  that  Johnson 
will  be  coming  off  the  bench  as  a substitution  for  Strickland.  Lastly,  Figure  4 shows  the  summary  statistics  collected 
for  Team  2.  These  statistics  may  also  be  viewed  on  an  individual  player  basis. 


Figure  3.  Substitution  Screen  Shot 


Interested  readers  my  access  the  syllabus  for  this 
http://personalweb.smcvt.edu/mbattig/CS407%20Svllabus.htm 
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Figure  4.  Summary  Statistics  Screen  Shot 
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Abstract;  This  paper  reports  initial  results  from  a study  which  investigated  if  different  media 
combinations  could  be  shown  to  improve  students’  understanding  of  computer-based  learning 
materials  and  to  determine  whether  student  learning  style  affected  student  understanding  for 
different  media  combinations.  Three  groups  of  participants  were  given  a presentation,  each 
using  different  media  combinations  to  present  a topic.  Dual  coding  theory  was  used  as  the 
basis  for  designing  the  presentation.  Results  indicate  that  participants’  understanding  was 
enhanced  when  the  computer-based  learning  materials  were  presented  using  sound  and 
diagrams.  Understanding  was  worse  when  materials  were  presented  using  text  and  diagrams. 
The  result  supports  the  predictions  of  dual  coding  theory.  Furthermore,  the  results  indicate  that 
the  sound  and  diagram  combination  can  improve  participants’  understanding  regardless  of 
their  preferred  learning  style,  and  that  intuitive  learners  seem  to  be  exceptionally  volatile  to 
different  media  combinations. 


Introduction 

Many  researchers  of  educational  technology  agree  that  learning  materials  should  be  designed  for  all 
types  of  learners,  and  learning  styles,  not  just  reflect  the  tutor’s  style  of  teaching  (Felder,  1996).  An  effective 
way  of  achieving  this  is  by  using  multiple  media  to  target  combination  of  media  on  the  different  styles  of 
learner.  There  are  many  studies  about  the  effectiveness  of  multimedia  and  learning  styles  in  educational 
systems,  but  very  few  give  an  insight  into  why  some  combinations  of  media  are  more  effective  than  others 
(Najjar,  1996).  One  theory  that  does  provide  such  an  insight  is  Dual  Coding  theory  (Paivio,  1971,  1986). 

According  to  Paivio ’s  (1971,  1991;  Clark  and  Paivio,  1991)  dual  coding  theory,  information  is 
processed  through  one  of  two  generally  independent  channels.  One  channel  processes  verbal  information  such 
as  text  and  audio  and  the  other  channel  processes  visual  information  such  as  diagrams,  animations  and 
photographs.  Paivio  also  advocates  two  different  types  of  internal  representational  unit;  ‘imagens’  for  mental 
images  and  ‘logogens’  for  verbal  entities.  Logogens  are  organized  in  terms  of  associations  and  hierarchies  while 
imagens  are  organized  in  terms  of  part- whole  relationships.  Three  types  of  internal  processing  are  identified: 
representational  - the  direct  activation  of  verbal  or  non-verbal  representations;  referential  - the  activation  of  the 
verbal  system  by  the  nonverbal  system  or  vice-versa;  and  associated  processing  - the  activation  of  related 
presentations  within  the  same  verbal  or  nonverbal  system.  Whilst  it  is  agreed  that  a given  task  may  require  any 
or  all  of  the  three  kinds  of  processing,  it  is  not  clear  what  and  in  which  order  people  with  different  learning 
styles  activate  these  types  of  processing.  Furthermore,  the  effect  on  the  learning  outcomes  of  students  with 
different  learning  styles  is  not  clear  when  different  media  combinations  are  used. 

Studies  by  Paivio  and  others  (Baggett,  1989;  Kozma,  1991)  suggest  that  by  choosing  an  appropriate 
combination  of  media,  learning  outcomes  can  be  improved.  For  example,  information  that  uses  verbal  and 
relevant  visual  illustrations  will  likely  be  learned  better  than  information  that  uses  text  alone,  audio  alone,  a 
combination  of  text  and  audio,  or  illustrations  alone.  Since  1971,  dual  coding  theory  has  been  used  as  an 
explanation  for  the  effects  of  multimedia  information  on  learning  in  a number  of  studies  (Baggett,  1989; 
Kozma,  1991;  Moreno  and  Mayer,  2000;  Paivio,  1971).  Many  researchers  agree  that  dual  coding  theory  gives  a 
reasonable  explanation  for  the  results  of  a large  number  of  studies  of  multimedia  learning  (Najjar,  1996).  The 
theory  is  particular  useful  because  it  gives  an  explanation  for  the  way  information  is  processed  by  human  beings 
and  relates  this  to  visual  and  verbal  styles  of  learning.  A number  of  principles  and  guidelines  for  designing 
multimedia  have  since  been  derived  from  dual  coding  theory  and  applied  to  the  development  of  computer-based 
learning  materials.  What  is  more,  this  theory  seems  to  be  as  relevant  today  as  it  was  some  30  years  ago,  in  spite 
the  advancements  in  new  technology  and  changes  in  education  (Paivio,  1991;  Sadoski  and  Pavio,  2001). 

Even  after  30  years,  there  exist  alternative  views  of  memory  storage  that  go  against  dual  coding  theory, 
such  as  propositional  theory  (Rieber,  1994).  Propositional  theorists  argue  that  a transformation  takes  place  of 
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visual  information  into  a semantic  form  of  storage  in  long-term  memory.  The  propositional  theory  disputes  the 
superiority  of  pictures  over  words  because  people  process  and  rehearse  pictures  more  fully  than  words.  Paivio 
(1991),  however,  still  believes  that  the  evidence  shows  that  dual  coding  theory  has  more  than  held  its  own 
relative  to  the  prepositional  alternative  and  that  other  new  theories  such  as  the  computational  theory 
“connect! onism”  remain  to  be  demonstrated. 

Although  much  work  has  been  done  to  date,  more  studies  are  needed  to  ascertain  the  effects  of 
multimedia  information  on  learning  by  students  with  different  learning  styles.  It  has  long  been  acknowledged 
that  dual  coding  theory  can  be  used  to  study  the  effects  of  visual  and  verbal  learners.  However,  little  is  known 
about  the  effects  of  multimedia  information  on  learning  by  students  with  other  learning  styles,  such  as  sensing, 
intuitive,  sequential,  global,  active,  and  reflective  styles. 

One  of  the  aims  of  the  study  described  in  this  paper  was  to  ascertain  the  effects  of  using  multimedia 
information  on  learning  by  sensing  and  intuitive  learners,  as  well  as  others,  as  compared  to  visual  and  verbal 
learners.  The  purpose  of  the  study  was  to  answer  the  following  questions: 

• Can  different  media  combinations  improve  students’  understanding? 

• Do  scores  differ  between  learning  styles  for  different  media  combinations? 

This  area  of  research  was  identified  as  being  of  particular  importance  to  researchers  and  authors  of 
multimedia  in  providing  them  the  necessary  background  to  help  select  an  appropriate  combination  of  media  for 
designing  computer-based  learning  materials  that  are  appropriate  for  all  learning  styles. 


Experimental  Study 

To  ascertain  the  effects  of  using  multimedia  information  on  learning  by  sensing  and  intuitive  learners, 
three  different  versions  of  learning  material  were  presented  to  three  groups  of  students.  One  group  was  shown  a 
version  of  the  presentation  containing  text  and  diagrams,  a second  group  was  shown  a version  containing  text 
only  and  a third  group  was  given  a version  containing  sound  and  diagrams.  Data  collected  from  each  group 
were  then  compared.  The  remainder  of  this  section  provides  further  details  of  the  presentation,  the  group 
characteristics,  and  the  instruments  and  the  methodology  used  in  the  experiment. 


Presentation 

A learning  module,  12  minutes  in  duration,  was  developed  in  order  to  help  assess  the  effectiveness  of 
different  combinations  of  media  (see  Figure  1). 


Tltc  Null  Hypothesis  and  Testing  its  Validity 
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Figure  1:  Screen  shot  of  the  module  on  statistics. 


The  module  was  developed  using  Macromedia  Flash  5 and  was  about  the  topic  of  the  use  of  statistics 
in  experimental  evaluation.  The  subject  area  of  statistics  was  selected  because  it  was  felt  that  most  students 
would  have  little  previous  knowledge  about  this  topic  and  those  students  that  did  have  previous  knowledge 
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about  it  often  found  the  topic  difficult.  The  aims  and  objectives  of  the  module  were  defined  and  used  to  produce 
a series  of  ‘messages’.  These  messages  represented  key  points  that  a student  needed  to  learn  from  the  module. 
The  messages  were  then  represented  and  replicated  in  a textual,  visual  and  audio  form.  Three  copies  of  the 
module  were  then  produced  as  presentations.  Presentation  1 presented  the  topic  using  text  and  diagrams, 
Presentation  2 using  text  only  and  Presentation  3 using  sound  and  diagrams.  To  ensure  that  only  the  media 
combinations  changed,  the  three  presentations  ran  for  the  same  length  of  time.  Each  presentation  was  then 
published  in  an  executable  form  and  each  was  placed  on  a different  but  comparable  laptop. 


Group  Characteristics 

Forty- four  students  participated  in  the  study.  The  participants  were  from  the  courses  Computing  and 
Management  and  Computer  Science  at  Loughborough  University.  The  study  was  carried  out  as  part  of  a module 
on  Human  Computer  Interaction,  which  the  participants  were  taking.  In  the  three  groups,  there  were  13,  14  and 
17  participants,  respectively.  The  age  of  the  participants  ranged  from  20  to  24.  The  three  groups  of  participants 
were  selected  according  to  there  learning  style.  This  criterion  was  implemented  to  assure  a similar  number  of 
participants  with  the  same  type  of  learning  style  were  present  in  each  group. 


Instruments 

The  test  instruments  employed  in  this  study  were  a pre-test  questionnaire,  a revised  version  of  the 
Felder-Silverman  Learning  Style  Model  inventory  on  learning  styles  (by  Richard  M.  Felder  and  Barbara  A. 
Soloman),  and  a post-test  questionnaire.  The  pre-test  questionnaire  obtained  the  participants’  familiarity  with 
the  statistical  terms  ‘the  null  hypothesis’  and  ‘significance’  using  ‘yes/no’  questions. 

The  learning  style  inventory  consisted  of  44  questions  about  a learners  preferred  way  of  learning  and  a 
score  sheet  for  calculating  a learners  preferred  style.  The  inventory  chosen  was  selected  after  reviewing 
numerous  different  inventories  that  are  available  in  electronic  and/or  paper  form.  We  discounted  inventories 
that:  (a)  would  have  taken  too  long  to  complete  in  a classroom  situation;  (b)  were  aimed  at  children  (i.e.  pre- 
sixteen);  (c)  were  specifically  related  to  visual,  auditory,  tactile,  since  we  decided  these  will  reveal  little  new 
information  or  interest  to  us  in  general;  and  (d)  were  difficult  to  fill-in  and  could  easily  be  skewed  by  erroneous 
answers.  The  Felder-Silverman  Learning  Model  was  used  for  a number  of  reasons.  It  did  not  contravene  the 
criteria  given  above  and  the  inventory  has  been  tested  and  validated,  and  shown  to  produce  reliable  results 
(Felder,  1996). 

The  post-test  questionnaire  consisted  of  10  questions  on  a single  doubled-sided  piece  of  A4  paper.  The 
questions  were  about  the  learning  material  contained  in  the  presentation.  For  each  question  there  was  also  a 
field  asking  participants  to  identify  whether  they  knew  the  answer  before  seeing  the  presentation,  whether  the 
presentation  helped  them  to  recall  the  answer,  or  whether  they  had  not  known  the  answer  before  seeing  the 
presentation. 


Methodology 

All  three  groups  were  tested  simultaneously  in  different  locations.  The  aims  of  the  study  were 
explained  to  the  students,  taking  care  not  to  give  away  details  about  the  content  of  the  presentations  and  the 
expected  result.  A handout  was  then  given  to  each  student  containing  the  learning  style  inventory  and  score 
sheet.  Students  were  asked  to  complete  the  inventory.  Instructions  were  then  given  to  the  students  about  how  to 
complete  the  score  sheet.  Afterwards,  the  students  were  asked  to  focus  on  the  score  for  the  learning  style 
category  sensing  and  intuitive  learners.  This  category  of  learning  styles  was  chosen  because  the  class  contained 
a broad  spread  of  styles.  In  contrast,  the  category  of  visual  and  verbal  learning  styles  was  not  selected  because 
the  population  was  predominantly  visual.  Those  subjects  who  were  predominantly  sensing  learners  were 
equally  divided  into  three  groups.  Subjects  who  were  predominantly  intuitive  learners  were  also  equally  divided 
into  three  groups  and  each  assigned  to  one  of  the  three  sensing  groups.  Finally,  subjects  who  had  a balanced 
style  of  learning  (neither  being  predominantly  sensing  nor  intuitive)  were  equally  divided  into  three  groups  and 
each  assigned  to  one  of  the  three  ‘sensing/intuitive’  groups.  This  resulted  in  three  groups  consisting  of  a random 
selection  of  subjects  containing  a similar  number  of  learning  styles.  Each  group  was  then  placed  in  a separate 
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room  where  they  were  given  one  of  the  presentations.  The  first  group  was  given  the  presentation  containing  text 
and  diagrams,  the  second  group  was  given  the  presentation  containing  text  only,  and  a final  group  was  given  the 
presentation  containing  audio  and  diagrams.  The  audio  in  the  third  group  consisted  of  a voiceover  that  spoke 
the  text  given  in  the  first  two  groups. 

Each  presentation  was  presented  using  a laptop  and  a data  projector.  After  each  group  had  watched 
their  particular  presentation,  they  were  given  the  post-test  questionnaire  to  complete.  Each  participant’s 
completed  post-test  questionnaire  was  then  stapled  to  his  or  her  learning  style  inventory  and  score  sheet.  This 
assured  that  we  could  compare  the  scores  of  those  participants  with  different  learning  styles.  All  the  learning 
inventories,  score  sheets  and  post-test  questionnaires  were  then  collected  within  the  groups. 

Having  collected  the  data,  the  results  were  then  analyzed  using  a 2 X 3 ANOVA  test  to  identify  whether  there 
was  a difference  in  the  participants’  scores  of  each  group.  Further  analysis  was  also  carried  out  to  ascertain  the 
effects  of  multimedia  information  on  learning  regarding  each  of  the  different  learning  styles. 


Results 

As  part  of  the  pre-test,  participants  in  the  study  confirmed  whether  they  knew  what  the  statistical  terms 
‘a  null  hypothesis’  and  ‘significance’  meant.  53.8%  of  the  text  and  diagrams  group,  14.3%  of  the  text  only 
group  and  29.4%  of  the  audio  and  diagrams  group  indicated  that  they  knew  what  a ‘null  hypothesis*  was. 
Similarly,  61.5%  of  the  text  and  diagrams  group,  35.7%  of  the  text  only  group  and  35.3%  of  the  audio  and 
diagrams  group  indicated  that  they  knew  what  ‘significance’  was.  These  scores  are  shown  in  the  bar  charts  in 
Figure  2.  In  the  overall  sample,  the  majority  of  participants  had  no  prior  knowledge  of  these  statistical  terms. 
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Figure  2:  Comparison  of  prior  knowledge  with  final  scores. 


Figure  2 also  shows  each  group’s  post-test  mean  score  after  seeing  a presentation.  Participants  who 
were  shown  the  presentation  containing  sound  and  diagrams  tend  to  score  the  highest  marks  followed  by 
participants  who  were  shown  the  presentation  containing  text  only.  Participants  who  were  shown  the 
presentation  containing  text  and  diagrams  tend  to  score  the  lowest  marks  in  the  study.  Interestingly,  the  text  and 
diagrams  group  had  reported  a higher  level  of  familiarity  with  the  subject  area  than  the  other  two  groups  (see 
above),  so  we  would  have  expected  them  to  do  better  in  the  test. 

Statistically,  overall  there  was  no  significant  difference  between  the  scores  of  sensing  learners, 
intuitive  learners  and  balanced  learners.  However,  although  not  statistically  significant,  ‘sensors’  did  seem  to 
score  higher  marks  than  ‘intuitors’.  Moreover,  as  also  suggested  in  the  literature  on  learning  theory  ‘balanced’ 
learners  did  seem  to  score  the  highest  marks.  The  findings  seem  to  support  the  literature  on  learning  theory  that 
no  single  learning  style  is  better  than  another,  but  balanced  learners  do  seem  to  perform  better  overall. 

Scores  differ  between  learning  styles  for  different  media  combinations.  For  some  learning  styles,  the 
difference  in  scores  is  more  evident.  In  fact,  when  learning  style  pairs  are  compared,  one  of  the  two  is  always 
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more  affected  by  the  change  in  media,  than  the  other.  With  the  sensing-intuitive  pair,  there  is  a statistically 
significant  difference  in  ‘intuitor’  scores  between  each  media  combination  group,  but  not  for  the  ‘sensor’ 
scores.  ‘Intuitors’  noticeably  scored  the  highest  marks  when  shown  learning  material  containing  sound  and 
diagrams  and  performed  worst  when  shown  learning  material  containing  text  and  diagrams.  This  trend  was  also 
true  of  other  learning  style  pairs  and  can  be  seen  in  Figure  3. 


Figure  3:  A comparison  of  learning  styles. 


‘Global’  learners  tended  to  be  more  affected  by  media  combinations  than  did  ‘sequential’  learners 
(sequential  -global  pair).  In  the  reflective-active  pair,  ‘reflective’  learners  performed  much  worse  with  text  and 
diagrams,  than  with  audio  and  diagrams  and  with  text  only.  However,  the  ‘active’  learners  had  closer  scores  for 
text  only  and  text  and  diagrams,  but  performed  much  better  with  audio  and  diagrams. 


Discussion 

The  results  from  this  study  offer  two  main  findings.  Firstly,  different  combinations  of  media,  used  to 
present  learning  materials  to  students,  lead  to  significant  differences  in  their  understanding.  Secondly, 
participants  with  different  learning  styles  perform  differently  for  different  combinations  of  media.  These 
findings  support  some  of  the  ideas  associated  with  dual  coding  theory.  Our  findings  suggest  that  when 
information  is  represented  by  a verbal  and  visual  combination  people  recall  more  items  than  when  using  text 
alone.  However,  the  findings  in  our  study  also  show  that  there  are  cases  when  a verbal  and  visual  combination 
can  cause  people  to  recall  fewer  items  than  when  using  text  alone  (in  our  case  text  and  diagrams).  A reason  for 
this  might  be  the  way  that  the  text  and  diagrams  were  integrated.  As  a result,  students  may  have  been 
experiencing  split-attention  effect,  an  effect  covered  in  cognitive  load  theory  (Sweller  et  al,  1998). 

Whilst  our  findings  suggest  that  participants  with  some  learning  styles  score  higher  than  others 
between  each  group,  this  may  be  because  people  with  particular  learning  styles  are  more  suited  to  a type  of 
media.  For  example,  the  presentations  may  have  appealed  more  to  sensing  learners  than  intuitive  learners 
because,  as  dual  coding  theory  suggests,  concrete  nouns  are  better  remembered  than  adjectives  or  abstract 
nouns.  Sensing  learner  tend  to  prefer  this  type  of  information,  whereas,  intuitive  learners  tend  to  prefer  to 
discover  possible  connections  and  relationships.  As  a result,  intuitive  learners  may  not  have  had  the  time  to 
activate  their  preferred  types  of  processing.  Of  course,  another  reason  may  have  been  because,  more  sensing 
learners  were  using  a surface  learning  strategy  and  intuitive  learners  a deep  learning  strategy  (Entwistle,  1997). 

A difference  in  what  and  which  order  the  students  activated  types  of  processing  may  also  account  for 
why  participants’  scores  of  some  learning  styles  increased  more  than  others  between  each  group.  Since  dual 
coding  supports  the  idea  that  people  learn  by  connecting  new  knowledge  to  prior  knowledge,  intuitive  learners 
may  have  been  able  to  exploit  this  when  sound  and  diagrams  were  used  because  time  was  freed  up  using  this 
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combination  of  media.  Media  synchronization  may  also  have  been  a factor  when  comparing  the  difference  in 
participants’  scores  between  text  and  diagrams  and  sound  and  diagrams  because  dual  coding  helps  learners  to 
increase  interconnections  to  information  already  in  memory  (Baggett,  1989). 

A possible  explanation  as  to  why  the  scores  of  most  learners  were  significantly  different  between  each 
group  is  probably  the  result  of  the  redundancy  effect,  the  split-  attention  effect  and  the  modality  effect 
associated  with  cognitive  load  theory  (Sweller  et  al,  1 998). 

It  is  clear  that  more  research  is  needed  to  investigate  whether  there  is  a relationship  between  the  effects 
on  learning  and  on  different  types  of  learning  style  and  media  combinations.  At  this  stage,  it  is  unclear  whether 
similar  results  would  have  been  obtained  given  that  participants  had  been  divided  into  groups  based  on  a 
different  learning  style  category.  Further  investigation  is  required  to  understand  why  those  subjects  with 
previous  knowledge  about  the  topic  in  text  and  diagrams  group  were  unable  to  exploit  this.  More  research  is 
also  needed  to  investigate  whether  special  groups  of  learners  with  a particular  style  of  learning  (such  as 
dyslexics)  conform  not  only  to  dual  coding  theory  but  also  to  our  overall  findings.  These  investigations  will 
help  to  understand  the  complex  relationship  between  learning  tasks  and  media  combinations.  From  our 
research,  we  intend  producing  guidelines  to  help  developers  of  computer-based  learning  materials  and  develop 
an  approach  for  developing  Adaptive  User  Interfaces  that  change  according  to  style  and  media  combination. 


Conclusions 

This  study  of  using  media  combinations  to  deliver  learning  materials  has  produced  findings  that  not 
only  support  dual  coding  theory,  but  also  suggest  that  students’  understanding  seems  to  improve  the  most  when 
using  sound  and  diagrams  to  present  the  learning  materials.  Furthermore,  students’  understanding  seems  to 
improve  for  some  learning  styles  more  than  others.  We  have  learned  that  the  affects  of  dual  coding  theory  are 
more  influential  towards  learning  styles  and  in  delivery  of  learning  materials  than  previously  thought.  These 
results  seem  to  provide  tentative  evidence  that  the  media  combination  used  to  present  learning  materials  is  just 
as  important  as  the  style  of  learning  adopted  by  the  student.  If  the  combination  of  media  is  inappropriate,  it  does 
not  matter  which  learning  style  a student  adopts.  Students  will  still  not  perform  to  their  full  potential.  This 
finding  is  particularly  pertinent  given  that  more  attention  is  often  given  by  tutors  to  the  way  learning  materials 
are  taught  and  not  to  the  types  of  media  used.  The  use  of  dual  coding  theory  as  an  approach  to  accessing  the 
effectiveness  of  learning  materials  is  still  as  relevant  today  with  web -based  and  mobile  learning  environments. 
Further  research  is  needed  to  determine  how  some  learning  styles  come  to  be  affected  more  than  others  when 
using  different  combinations  of  media. 
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Non-traditional  Students  Learning  in  a Non-traditional  Environment 

Students  24  years  of  age  or  older  learning  in  a college  setting  with  new  technologies 

The  learning  styles  of  undergraduate  students  vary  a great  deal.  Technology  is  often  used  to  address  a myriad  of 
learning  needs.  However,  as  non-traditional  college  students  (24  years  of  age  or  older)  enter  the  university  setting, 
technology  is  often  a hindrance  to  their  success.  The  instructor  and  the  non-traditional  student  must  address  not  only 
the  learning  style  issue,  but  also  the  challenge  of  learning  with  technology  developed  since  their  last  formal  educational 
experiences.  The  combination  of  unique  learning  styles  and  lack  of  technology  skills  presents  an  additional  challenge 
for  these  students.  As  instructors  design  their  courses  to  include  a variety  of  media  tools,  specific  measures  can  be 
taken  to  keep  the  non-traditional  students  abreast  of  the  class.  The  session  will  focus  on  the  unique  learning  of  non- 
traditional  undergraduates  and  a process  for  applying  specific  media  tools  to  the  instructional  components  of  the 
university  classroom. 


Background 

What  started  as  a Cherokee  Female  Seminary  in  1887  is  known  today  as  Northeastern  State  University  (NSU),  the 
second  oldest  institution  of  higher  learning  west  of  the  Mississippi  River.  Since  its  inception,  NSU  has  evolved  into  one 
of  the  leading  regional  institutions  of  higher  education  in  Oklahoma,  rich  in  the  American  Indian  heritage  of  years  past 
and  strong  in  the  changing  world  of  technology  today.  The  mission  of  the  University  is  to  provide  undergraduate  and 
graduate  education  leading  to  64  bachelor’s  degrees,  14  masters’  degrees  in  selected  areas  and  a doctoral  degree  in 
Optometry. 

Northeastern  State  University  currently  has  an  undergraduate  student  population  of  8,378.  While  the  average  age  of  a 
traditional  college  undergraduate  is  between  eighteen  and  twenty-three  years  old,  the  average  age  of  a Northeastern 
State  University  undergraduate  student  is  twenty-seven.  The  majority  of  students  attending  Northeastern  State 
University  bring  new  challenges  to  the  campus  and  in  turn  to  the  classroom.  One  of  the  most  ominous  challenges  these 
students  and  their  instructors  face  is  the  application  of  technology  to  the  learning  process. 

Problem 

The  vast  majority  of  traditional  students  in  the  Northeastern  State  University  classrooms  have  been  exposed  to  the  use 
of  current  technology  throughout  their  educational  experience.  These  students  are  comfortable  with  the  computer, 
Internet  and  multimedia  software  used  by  the  instructor  in  the  classroom.  Furthermore,  traditional  students,  by  in  large, 
use  technology  in  their  learning  process.  This  natural  extension  of  the  learning  environment  assists  students  in 
successfully  attaining  the  learning  goals  of  the  college  classroom. 

Each  semester.  Northeastern’s  College  of  Education  offers  a course  entitled.  Technology  in  Education.  Through  this 
course,  teacher  candidates  learn  appropriate  application  of  technology  to  the  teaching  and  learning  process.  Successful 
completion  of  this  course  is  required  for  all  education  majors.  Nontraditional  students  enroll  in  the  course  as  well  as 
traditional  undergraduates.  The  instructor  of  this  course  conducted  action  research  to  investigate  two  questions.  The 
first  question  to  be  addressed:  What  is  the  nontraditional  students’  perspective  of  the  challenges  technology  presents  in 
their  learning  process?  The  second  question:  What  resources,  digital,  analog  and  human,  could  assist  nontraditional 
students  in  accommodating  digital  resources  for  learning? 

Method 

During  the  Fall  2001  and  Spring  2002  semester,  the  instructor  invited  students  from  six  sections  of  the  Technology  in 
Education  course  to  participate  in  the  action  research.  Information  concerning  student  age,  gender  and  computer 
anxiety  level  was  collected  using  the  Computer  Anxiety  Index  (CAIN).  The  CAIN  was  administered  to  the  students  at 
the  beginning  of  the  course  and  again  at  the  end  of  the  course  in  order  to  measure  the  reduction  or  increase  in  computer 
anxiety.  Student  s also  maintained  a journal  of  reflections  completed  at  the  conclusion  of  each  major  technology 
assignment.  The  assignments  introduced  various  digital  and/or  multimedia  applications  for  learning.  These  tools 
included  PowerPoint  presentations,  web  page  construction,  Internet  scavenger  hunts  and  focused  searches  and 
WebQuest  construction.  The  CAIN  and  the  reflective  journals  contributed  information  to  determine  the  answer  to  each 
research  question. 
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Application 


Although  the  study  is  action  research,  the  findings  could  be  applied  to  the  wider  population  of  nontraditional  students 
attending  regional  universities.  The  digital  and  multimedia  applications  can  be  adjusted  to  accommodate  other  content 
areas  outside  teacher  education.  Instructors  in  other  university  settings  can  easily  duplicate  the  introduction  of  these 
tools  and  the  support  model  developed  through  the  process. 

Results 

The  month  of  December  marks  the  midpoint  of  the  action  research.  A combined  total  (traditional  and  non-traditional 
students)  of  forty-seven  students  participated  in  the  study.  The  journal  entries  and  CAIN  scores  are  collected.  The 
remaining  data  will  be  collected  during  the  Spring  2002  semester.  At  this  time,  all  data  collected  thorough  the  CAIN 
will  be  analyzed  using  a statistical  software  package.  The  reflected  journals  will  be  compiled  and  presented  in  a concise 
manner. 
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Abstract:  This  paper  discusses  the  results  of  a study  that  compared  two  different  course  delivery  methods. 
One  of  these  used  the  Internet  exclusively  for  module  delivery  while  the  other  used  a traditional  campus 
setting.  The  two  delivery  methods  were  compared  in  order  to  determine  if  the  Internet  method  was  as  good  as 
the  traditional  approach.  Results  and  suggestions  for  further  study  are  described  and  some  of  the  concerns 
identified  are  discussed.  The  module,  which  prepares  students  to  take  and  pass  an  industry-recognised 
certification  test,  is  currently  being  offered  at  the  Community  College  of  Southern  Nevada. 


Introduction 

The  Community  College  of  Southern  Nevada  (CCSN)  is  responsible  for  delivering  educational  opportunities  to 
a 45,000  square  mile  area  in  Southern  Nevada.  Enrolment  at  CCSN  was  approximately  40,000  students  for  the 
Autumn  semester  2001  (September  through  December  2001).  In  order  to  meet  the  needs  of  so  many  students, 
CCSN  has  built  six  major  and  over  a dozen  outreach  campuses.  Despite  this,  many  students,  are  still  not  able  to 
take  classes  due  to  working  schedules  or  because  they  reside  at  a significant  distance  from  one  of  the  campuses. 

A continuing  option  for  students  who  are  unable  to  attend  normal  classroom  instruction  has  always  been 
distance  education  (DE).  The  increased  availability  and  power  of  computers  has  created  distance  education 
opportunities  that  were  not  available  years  ago.  However,  the  use  of  these  is  often  sporadic,  if  they  are  used  at 
all  due  to  many  factors  including  the  cost  of  entry  for  both  the  institution  (Matthews,  1999;  Chambers,  1 999; 
Barley,  1999)  and  for  students  (Matthews,  1999;  Barley,  1999;  Blumenstyk  and  McCollum,  1999).  Many  of  the 
distance  education  modules  at  CCSN  are  delivered  in  a manner  that  requires  students  and  instructors  to  meet  at 
specific  times  for  discussion,  interaction  and  presentation.  These  modules  offer  students  the  opportunity  to 
study  in  ways  that  are  independent  of  place  and  time;  indeed  time  and  place  flexibility  are  the  two  greatest 
advantages  of  distance  education  (Shave,  1998;  Fender,  1999;  Alexander,  1999).  This  is  an  attractive 
proposition  for  students  with  unusual  or  irregular  work  schedules  or  other  time  conflicts.  This  is  especially  the 
case  for  many  people  in  the  Las  Vegas  area  due  to  the  twenty-four  hour  nature  of  the  entertainment  industry. 
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An  additional  constraint  is  the  lack  of  experience  and  knowledge  that  instructors  may  have  (Shave,  1998; 
Mudge,  1999;  Twigg,  2001).  Technology  improvements  alone  do  not  offer  most  instructors  the  ability  to  offer 
distance  education  modules  unless  they  are  familiar  and  comfortable  with  the  technology  (Shave,  1999; 
Alexander,  1999;  Matthews,  1999;  Camevale,  1999;  Trinkle,  1999;  Grill,  1999;  Mudge,  1999).  In  addition, 
instructors  must  have  the  time  and  patience  to  learn  how  to  use  the  technology.  This  can  include  expanding  the 
frame  of  reference  an  instructor  has  from  classroom-based  instruction  to  instruction  that  is  independent  of  place 
and  time.  Many  instructors  find  this  difficult.  The  steep  learning  curve  experienced  by  instructors  can  be 
partially  alleviated  by  a good  computer  support  department  at  the  college  or  university.  However,  the  use  of 
outside  developers  or  content  providers  can  affect  a faculty  member’s  control  over  his  or  her  intellectual 
property. 

Delivery  that  is  independent  of  time  requires  the  creation  of  online  courses  that  can  be  accessed  at  any  time  and 
place,  as  long  as  Internet  access  is  available.  The  module  selected  for  delivery  using  the  two  methods  described 
above  was  Computer  and  Information  Technology  (CIT)  106B.  This  module  is  a preparatory  course  for 
students  who  are  preparing  to  take  the  Computing  Technology  Industry  Association’s  (CompTIA)  A+  Core 
certification  test.  This  test  is  one  of  two  tests  required  to  become  an  A+  Certified  PC  Technician.  Both  delivery 
methods  used  the  same  textbook  and  tests. 

In  order  for  the  Internet  module  to  be  as  close  as  possible  to  the  actual  on-campus  presentation,  streaming  audio 
and  video  were  used  to  transmit  the  classroom  lectures  over  the  Internet  (Beckstrand,  Barker  and  van  Schaik, 
2001).  Streaming  does  not  require  files  to  be  downloaded  before  use.  The  use  of  streaming  technology  does  not 
require  specific  class  meeting  times.  Support,  independent  of  time  and  place,  was  offered  using  email,  voice 
mail,  conventional  mail,  and  office  visits.  All  tests  and  questionnaires  were  delivered  through  computers  using 
WebCT. 

System  overview 

Any  module  that  is  delivered  through  distance  education  has  three  pre-requisites.  First,  the  learning  material 
that  is  covered  needs  to  be  disseminated.  Second,  communication  between  the  instructor  and  student  for 
questions  and  other  feedback  needs  to  be  set  up.  Finally,  learning  must  be  evaluated  and  assessed.  CCSN 
adopted  a Web  development  tool  set  called  WebCT.  WebCT  provides  Internet-based  tools  for  testing.  While 
there  is  a learning  curve  associated  with  using  WebCT,  the  curve  is  similar  to  that  experienced  when  learning  to 
use  a new  word  processing  program  instead  of  a programming  language.  Once  the  tests  are  in  place,  students 
can  receive  immediate  feedback  after  a test  has  been  submitted  for  grading. 

A Web  site  was  developed  to  provide  communication  between  the  instructor  and  students.  The  Web  site  could 
be  used  to  post  messages  of  concern  for  all  students  as  well  as  contain  an  email  link  so  that  students  could  send 
the  instructor  an  email  during  the  delivery  of  the  module.  Additional  information  could  be  added  to  the  Web 
site  if  needed.  Students’  learning  was  evaluated  and  assessed  through  four  tests  administered  on  demand 
through  WebCT.  The  first  three  tests  were  not  comprehensive.  The  fourth  test  was  comprehensive  with 
questions  covering  all  of  the  learning  material  presented.  The  Web  site  (http;//itcert.ccsn.net)  contained  module 
syllabi,  course  information,  textbook  listings,  and  certification  testing  information. 

Evaluation 

An  evaluation  of  the  Internet  delivery  of  the  module,  compared  to  the  on-campus  delivery,  was  conducted.  This 
section  describes  the  analysis  plan  and  the  results  obtained. 

The  Computer  Literacy  Questionnaire  (CLQ)  (Murphy,  Coover,  and  Owen,  1989)  was  administered  to 
investigate  any  differences  between  the  groups  and  as  a possible  predictor  of  students'  performance.  The  first 
four  questions  on  the  CLQ  collected  demographic  information.  The  thirty-seven  literacy  questions  of  the  CLQ 
assessed  self-perceived  computer  literacy.  The  scoring  of  the  questionnaire  was  based  on  a five-point  scale. 
Lower  scores  indicate  less  familiarity  with  computers  while  higher  scores  indicate  more  experience. 
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Analysis  plan 


The  analysis  of  the  study  results  consisted  of  four  parts.  The  first  part  sought  to  discover  if  there  were  any 
differences  between  the  Internet  modules  offered  during  the  Summer  time-frame  and  the  module  offered  during 
the  Autumn  time-frame.  Any  major  differences  in  the  modules  would  prevent  the  combination  of  the  groups  into 
a single  unit  for  comparison  with  students  enrolled  in  the  on-campus  module.  The  second  part  of  the  analysis 
checked  for  any  initial  differences  between  students  on  the  Internet  module  and  students  on  the  on-campus 
module  that  might  have  affected  the  outcomes  and  conclusions.  The  third  part  of  the  analysis  involved  the 
actual  comparison  of  the  on-campus  module  and  the  Internet  modules.  The  results  were  expected  to  support  the 
hypothesis  of  no  (statistically  significant)  differences  between  the  module  outcomes.  This  would  indicate  that  if 
an  online  module  was  designed  correctly,  students  should  be  able  to  achieve  the  same  results  taking  the  online 
module  as  those  taking  the  on-campus  module.  Finally,  regression  analysis  was  conducted  in  order  to  uncover 
factors  that  explain  students'  performance. 

Data  analysis  was  performed  using  SPSS  for  Windows  (version  8).  Students’  names  were  not  recorded  in  the 
data  files.  Students  who  registered  and  paid  for  the  module  were  monitored  to  measure  attrition.  A total  of  91 
students  were  registered  for  the  three  Internet  deliveries.  Comparisons  between  the  different  presentation 
methods  included  age,  pre-test,  other  tests,  questionnaires,  attrition,  and  final  grade. 

Results 

All  students  responding  on  the  CLQ  indicated  that  they  owned  and  used  a personal  computer. 

Students  enrolled  on  the  Summer  Internet  delivery  and  Autumn  Internet  delivery  did  not  have  any  statistically 
significant  differences  in  the  areas  evaluated  that  precluded  them  from  being  combined  as  one  unit.  More 
Summer  students  registered  for  the  Internet  module  because  it  was  the  only  offering  of  the  module  and  they  did 
not  want  to  wait  until  Autumn.  Overall  motivation  concerning  the  modules  was  very  similar  between  the  two 
student  groups.  Since  the  results  indicated  that  the  two  groups  were  similar,  they  were  treated  as  one  group  in 
comparisons  with  the  traditional  module  presented  in  the  Autumn. 

The  analysis  of  the  pre-test  scores  indicated  that  both  groups  were  starting  at  an  equivalent  knowledge  level.  It 
was  observed  that  the  Internet  module  was  the  first  non-traditional  module  for  many  of  the  enrolled  students. 
However,  as  might  be  expected,  most  students  following  the  on-campus  delivery  had  previously  taken  other  on- 
campus  modules.  Each  of  the  tests  administered  during  the  module  delivery  was  compared  for  differences. 
Questionnaire  responses  were  also  compared  for  changes  during  the  delivery  time  frame.  It  was  found  that 
students  registered  for  the  delivery  method  they  desired. 

A number  of  different  measurements  were  used  to  determine  if  the  research  h}q)Othesis  identified  above  was 
supported.  Questionnaire  One  measured  students’  attitudes  concerning  motivation,  delivery  method  motivation, 
and  collected  some  demographic  information.  Questionnaires  Two  and  Three  measured  students’  attitudes 
concerning  motivation,  delivery  method  motivation  and  module  delivery  evaluation.  Students’  attitudes 
remained  consistent  during  the  module  delivery  with  little  difference  between  the  two  groups.  A 2 x 4 mixed 
measures  ANOVA  showed  there  were  no  differences  in  test  performance  between  the  two  delivery  methods 
over  time;  the  pattern  of  differences  between  the  scores  on  the  four  tests  was  the  same  for  both  delivery 
methods.  The  test  results  thus  supported  the  hypothesis  stated  above. 

Regression  analysis  showed  that  students'  test  performance  on  test  1 was  not  related  to  their  performance  on  the 
pre-test.  Test  performance  on  test  2 was  significantly  related  to  performance  on  the  pre-test  and  test  1 
combined,  where  performance  on  test  1 was  a significant  predictor.  On  test  3,  performance  was  significantly 
related  to  performance  on  all  previous  tests  combined,  and  performance  on  test  2 was  a significant  predictor. 

Test  performance  on  the  final  test  was  significantly  related  to  performance  on  all  previous  tests  combined,  with 
performance  on  test  3 and  the  pre-test  being  significant  predictors. 

Overall,  the  results  indicated  that  students  in  the  two  delivery  types  were  more  similar  than  dissimilar.  This 
conclusion  was  supported  by  the  results  of  the  first  three  steps  of  the  data  analysis.  Even  the  attrition  rate 


159 


between  the  two  delivery  methods  was  not  significantly  different.  Detailed  results  will  be  presented  at  the 
conference. 


Lessons  Learned  and  Emerging  Guidelines 

The  study  results  support  the  stated  hypothesis.  The  Internet  module  was  comparable  to  the  on-campus  module 
in  helping  students  attain  results.  The  students  in  the  groups  were  very  similar  in  all  areas  of  comparison,  in 
spite  of  the  lack  of  random  assignment  to  the  different  delivery  methods.  Any  differences  identified  were  not 
major  and  did  not  influence  the  outcome.  This  study  supports  the  developmental  path  used  to  facilitate  the 
Internet  delivery  of  the  module.  By  using  a known  and  established  module,  as  well  as  the  same  support 
material,  a successful  non-traditional  version  of  the  module  was  created. 

The  study  does  identify  some  issues  that  need  to  be  addressed.  The  attrition  rate  of  both  delivery  methods  is  a 
concern.  Additional  study  should  be  considered  to  identify  the  reason  why  many  students  began  the  module  but 
then  left  before  completing  the  work.  Another  issue  is  the  large  number  of  students  that  completed  the  modules 
but  received  fail  grades.  This  is  especially  important  for  two  reasons.  First,  CIT  106B  is  an  introductory 
module  for  new  and  beginning  students.  Second,  CIT  106B  is  an  elective  for  many  other  study  programmes  in 
the  CIT  department.  Failure  in  this  module  could  have  a negative  effect  on  new  students  at  the  college  and  on 
students  completing  programmes  of  study. 

The  high  attrition  and  large  number  of  fail  grades  could  be  attributed  to  students'  self-assessed  levels  of 
knowledge  of  computers  and  technology  indicated  by  the  CLQ  results.  Two-thirds  of  the  students  who 
responded  on  the  CLQ  scored  in  the  novice  range.  The  CIT  106B  module  is  totally  about  computer  technology. 
Identifying  ways  to  help  students  develop  basic  computing  skills  prior  to  taking  the  module  might  help  lower  the 
high  attrition  and  reduce  the  number  of  fail  grades. 

The  CIT  106B  module,  developed  using  the  above  methodology  can  be  viewed  at 
http://webcampus.ccsn.nevada.edu.  Select  the  “Logon  to  MyWebCT”  option.  The  sample  ID  is  tstudent  and 
the  password  is  temp.  All  lectures  will  be  available,  but  due  to  academic  constraints,  the  module  tests  will  not 
be  available.  The  account  above  will  be  active  during  the  ED  MED  I A 2002  conference. 


Conclusions 

The  use  of  Internet  technology,  along  with  various  tools  can  make  truly  independent  learning  a reality.  The 
offering  of  a module  with  time  and  place  flexibility  offers  opportunities  for  students  who  otherwise  would  not 
be  able  to  take  advantage  of  higher  education  and  provides  as  good  a learning  opportunity  in  this  instance  as  the 
on-campus  module.  Because  the  module  is  delivered  via  the  Internet,  learning  can  take  place  at  home,  while 
travelling,  or  at  an  institutional  computer  lab.  Such  modules  can  also  be  offered  by  an  institution  to  students  not 
included  in  its  normal  audience  - a global  audience.  This  type  of  module  is  also  attractive  because  it  offers  self- 
paced  study  to  learners.  Lectures  can  be  repeated  multiple  times  if  a student  is  having  difficulty  grasping  the 
subject  - something  that  is  not  possible  in  a normal  classroom  environment.  The  advantages  associated  with  this 
type  of  distance  education  module  were  sufficient  motivation  to  continue  the  development  of  two  similar 
modules.  The  first  of  the  follow-on  modules  was  made  available  in  Spring  2001  and  the  second  of  the  follow- 
on  modules  will  be  made  available  in  Spring  2002. 

Not  all  disciplines  will  be  able  to  use  this  type  of  distance  education  module.  However,  the  ability  to  offer  such 
modules  provides  benefits  both  for  the  college  (CCSN)  and  the  students.  These  benefits  far  outweigh  the 
negative  aspects.  This  first  module,  along  with  the  additional  modules,  will  provide  educational  opportunities 
for  students  that  would  otherwise  not  be  able  to  enjoy  the  fruits  of  higher  education. 
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Abstract:  This  paper  examines  the  benefits  and  challenges  of  for  an  institution  to  support 
cooperative  learning  through  assessment.  The  paper  will  focus  on  the  key  issues  behind  the  design 
and  management  of  teamwork  assessment,  and  the  development  of  guidelines  to  undertake  the 
implementation  of  an  institution-wide  approach  to  cooperative  learning. 

1.0  Introduction 

The  main  objective  of  cooperative  learning  is  for  learners  to  collaborate  and  use  others  as  a resource,  for 
sharing  knowledge,  “challenge  other  and  their  own  views,  hence  to  serve  as  a source  of  puzzlement  that  stimulates 
new  learning”  (VonGlaserfeld,  1989);  yet  stand  separately  for  individual  assessment. 

In  many  cases  the  complexity  or  purpose  of  the  task  extend  to  include  assessment,  as  part  of  the 
cooperative  work.  Therefore  the  need  to  develop  different  methods  that  explain  ‘how’  to  calculate  individual  marks 
from  group  marks  (Lejk  and  Wyvill  1996;  Brown,  1995). 

But  the  true  effectiveness  of  the  assessment  of  cooperative  work  doesn’t  lay  in  the  method,  but  in  the  goals, 
behind  its  design  and  implementation. 

2.0  The  Need  for  Collaborative  Learning 

Cooperative  learning  is  among  most  effective  learning  method  in  higher  education  (Astin,  1993;  Johnson, 
et  al.,  1991;  CELT,  1995).  It  became  the  central  strategy  to  learning  at  the  Technical  University  of  British  Columbia 
(TechBC). 

Critically,  TechBC  embedded  the  goals  of:  learning  effectiveness,  learner  centeredness,  responsiveness  to 
workplace  needs,  and  effective  use  of  educational  technology,  at  an  institutional  and  cross-program  level.  The 
adoption  of  cooperative  learning  supported  and  interwove  these  goals  very  effectively. 

How  do  you  translate  all  the  benefits  of  cooperative  learning  into  an  active  every  day  working  and  learning 
environment  that  thrives  across  programs,  schools,  levels,  instructors,  diverse  learners,  cultures,  and  subject  areas? 

To  date,  research  has  dealt  with  common  problems  of  cooperative  learning  at  the  course  level,  but  many  of 
the  issues  that  were  encountered  at  TechBC,  are  institution-wide  problems,  such  as  managing  team  activities  across 
courses,  team  dynamics,  and  extending  the  cooperative  work  all  the  way  to  team  assessment. 

5.0  Results 

At  TechBC,  we  developed  an  approach  that  takes  into  account  the  values  of  cooperative  learning  and  a 
framework  that  supports  good  instructional  design,  providing  developers  and  instructors  with  guidelines  that  enforce 
sound  team  assessment  practices. 

In  the  following  segments  we  describe  why  we  considered  instructional  design,  the  principles  of 
cooperative  learning,  peer  and  self-assessment  and  the  assessment  of  learning  best  practices  to  be  the  core  elements 
to  be  addressed  in  the  development  of  our  team  assessment  guidelines. 

5.1  Instructional  Design 

Instructional  design  is  a systematic  approach  to  curriculum  and  course  development  that  focuses  on 
learning  objectives  as  a means  for  the  development  of  learning  experiences. 

The  learning  objectives  communicate  the  knowledge  and  skills  learners  can  expect  as  outcomes  of  the 
completed  unit  of  study.  They  should  be  adopted  as  the  basis  for  the  design  of  assessment  and  feedback  techniques 
and  seen  as  an  inherent  part  of  the  educational  structure. 
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5.2  Principles  of  Cooperative  Learning 

The  method  of  collaboration  in  the  learning  environment  is  for  learners  to  work  together  in  groups  and 
individually  assessed  (Slavin,  1995). 

The  purpose  behind  cooperative  learning  is  to  design  activities  that  motivate  learners  to  work  with  each 
other,  because  they  see  the  value  of  others  views  and  sharing  of  knowledge,  but  also  because  the  learning  of  their 
peers  is  important  to  them  (Slavin,  1995a).  It  is  in  the  course  of  this  interaction  that  all  group  members'  benefit,  and 
it  is  through  individual  assessment  that  the  value  is  measured. 

However  it  is  also  often  to  see  some  examples  in  which  the  group  activity  is  designed  in  a way  that  group 
cooperation  and  the  assessment  are  linked  together.  It's  when  learning  activities  are  translated  into  assessment 
activities  (in  whch  marks  or  grades  are  assigned);  that  learners  start  to  worry  about  the  effect  that  the  actions  of 
others  will  have  on  their  marks. 

There  are  five  essential  elements  that  make  cooperative  learning  activities  work:  a)  Positive 
Interdependence,  b)  Promoting  Interaction,  c)  Individual  Accountability,  d)  Team  skills  and  e)  Groups  Processing. 
But  it's  Positive  Interdependence  and  Individual  Accountability  that  have  a mayor  role  in  the  assessment  of 
teamwork. 

These  two  principles  of  cooperative  learning  seem  so  far  apart,  and  difficult  to  accomplish  at  the  same  time. 
The  success  of  team-based  learning  relies  in  heavily  on  the  ability  to  design  and  activity  balances  both  elements. 
There  is  a very  fine  line  between  collaboration  and  collusion  (Isaacs,  2002). 

Slavin  (1995),  describes  the  success  of  cooperative  learning  as  the  ability  to  have  group  goals  but  be 
capable  of  assessing  individual  accountability. 

5.3  Peer  and  Self-assessment 

Both  peer  and  self-assessment  methods  have  been  widely  used  as  variables  to  ensure  that  either  or  both 
individual  accountability  and  positive  interdependence  have  been  achieved  in  team-based  assessments. 

Some  research  seem  to  suggest  that  peer  and  self  assessment  are  more  useful  in  the  assessment  of  group 
processing,  team  skills  and  positive  interdependence,  while  others  suggest  that  it  is  successfully  applied  to 
assessment  of  individual  accountability  (skills  and  knowledge). 

5.4  Assessment  of  Learning 

“Effective  learning  occurs  when  learners  come  to  internalize  the  project  of  learning,  benefit  from  having 
meaningful  assignments  that  develop  their  knowledge  and  skills,  and  receive  useful  feedback  in  order  to  improve 
performance"  (Battersby,  1999).  For  that  to  happen,  assessment  must  follow  these  basic  principles: 

• It  should  be  clearly  linked  to  the  learning  objectives. 

• The  benchmarks  of  satisfactory  performance  are  clearly  established  before  learner  outcomes  are  measured. 

• The  assessment  supports  the  learning  process  by  providing  clear  and  valuable  feedback.  Assessment 
provides  data  on  performance  for  grading  purposes,  as  well  as  information  on  the  effectiveness  of  the 
instruction.  (Diamond,  1998) 

These  principles  can  be  attained  following  good  instructional  design  practices.  Assessment  and  learning  are 
intrinsically  related, "...  the  assessment  methods  and  requirements  probably  have  a greater  influence  on  how  and 
what  learners  learn  than  any  other  single  factor"  (Boud,  1995). 

6.0  Conclusion 

The  design,  development  and  implementation  of  cooperative  learning  can  become  a very  complicated  task, 
especially  if  the  learning  activities  are  extended  to  the  assessment  of  teamwork.  Specifically,  critical  issues  of  team 
assessment  need  to  be  addressed  to  achieve  the  effective  learning,  especially  for  the  learners.  Institutionally,  if 
assessment  supports  the  overall  goals  for  learning,  across  programs,  learning  staff,  and  supporting  staff,  the  goals  are 
more  attainable. 

Hopefully,  we  have  demonstrated  the  need,  challenges,  and  the  benefits  of  developing  assessment 
guidelines  for  cooperative  learning  support  the  goals  of  a new  and  innovative  university. 
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Abstract  This  paper  is  a discussion  of  the  importance  of  teaching  cyberethics  to 
K-12  students.  It  includes  the  results  of  two  informal  surveys:  one  of  teachers 
and  librarians  regarding  their  experiences  with  cyberethics  instruction,  and  one 
of  students  regarding  their  knowledge  of  cyberethics.  A description  of  a school- 
wide project  to  emphasize  cyberethics  through  a web  collection  of  student 
created  clip  art  is  also  included. 


Introduction 


Computer  ethics,  or  cyberethics,  is  an  area  of  technology  education  that  is  frequently 
neglected  by  teachers,  librarians,  and  administrators.  Students  are  using  computers  in  ever-growing 
numbers,  but  in  many  schools  are  receiving  little  or  no  guidance  about  the  issues  relating  to  ethical  use  of 
technology.  Defining  cyberethics,  whether  for  adults  or  for  students,  is  a bit  daunting  because  the  term 
covers  a multi-faceted  array  of  issues,  including  at  least  the  following:  respect  for  intellectual  property, 
respect  for  privacy,  and  hacking.  Indeed  one  expert.  Dr.  Marvin  Berkowitz,  has  included  seven  specific 
issues:  computer  hacking,  copyright  issues,  hate  speech,  privacy,  computer  addiction,  plagiarism,  and 
personal  identity. 

Discussion 


Part  of  the  reason  why  students  are  not  being  offered  information  about  cyberethics  is 
because  the  adults  who  should  be  leading  the  way  are  themselves  uninformed.  Many  publications, 
workshops,  and  technology  courses  do  present  information  to  educators  about  cyberethics.  Unfortunately, 
as  with  other  areas  of  professional  development,  too  often  people  do  not  receive  training  due  to  lack  of 
time,  lack  of  access  to  workshops  or  seminars,  or  lack  of  awareness  of  these  issues.  Teachers  and  librarians 
must  become  informed  leaders  in  this  arena  before  they  can  help  students  to  make  changes. 

Recently,  I explored  the  issue  of  cyberethics  and  its  emphasis  in  schools  through  an  informal 
survey  of  teachers  and  librarians.  The  survey  was  directed  to  members  of  LM_Net  and  EDTECH 
discussion  groups,  and  also  to  library  science  students.  I posed  questions  about  how  ethical  issues  were 
handled  with  teachers  and  staff,  and  also  about  how  these  issues  were  addressed  with  students  at  their 
schools.  I received  thirty-four  responses,  with  equal  representation  from  librarians  and  classroom  teachers. 

I asked  eight  questions  about  the  ways  in  which  cyberethics  was  addressed  on  campuses  with 
faculty  and  staff: 

• Who  is  the  voice  for  copyright  and/or  respect  for  intellectual  property  on  your  campus?  A 
three  to  one  ratio  of  responses  pointed  to  the  librarian  or  media  specialist  as  the  responsible 
spokesman.  One  notable  exception  was  an  individual  who  reported  that  her  district  had  a 
specific  position.  Copyright  Compliance  Officer,  who  monitored  copyright  issues  and 
situations.  That  administrator  then  directed  training  to  librarians,  who  served  as  campus 
liaisons.  Other  comments  about  the  role  of  librarian  as  champion  of  copyright  included,  “If 
anyone  is  going  to  do  this,  it  will  be  me,  the  librarian.” 

• Are  copyright  notices  posted  on  your  campuses?  Less  than  half  of  the  respondents  said  yes, 
with  the  majority  indicating  notices  were  posted  on  or  near  photocopy  machines. 
Unfortunately,  this  simple  step  does  seem  to  be  neglected  in  a number  of  schools. 

• Who  is  the  voice  on  your  campus  for  protection  of  computer  software?  Responses  to  this 
question  were  evenly  split  between  the  librarian  and  the  campus  technology  teacher  or 
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specialist.  Only  two  individuals  indicated  a direct  knowledge  of  a district  wide  emphasis  on 
compliance  in  this  area.  Many  people  said  the  issue  was  not  addressed  at  all  at  their  schools. 

• How  widespread  is  the  violation  of  software  copyright  by  teachers  and  staff  on  your  campus? 
Almost  all  respondents  reported  that  it  was  their  impression  that  copyright  was  respected  in 
most  cases.  Several  people  reported  campus  guidelines  and  policies.  As  one  computer  teacher 
commented,  “The  fines  for  piracy  are  well  known  here.”  Another  technology  teacher  offered  a 
somewhat  jaded  comment,  “I  would  not  think  it  is  very  much  as  our  school  has  provided  the 
necessary  software  for  its  teachers.  If  the  software  is  not  provided,  teachers  will  violate 
copyright  laws.”  Many  respondents  were  unaware  of  what  measures  were  taken  to  protect 
software  copyright  in  their  buildings  or  districts. 

• Who  is  the  voice  for  privacy  regarding  technology,  and  is  this  issue  taken  seriously?  As  with 
the  responsibility  for  software  copyright,  the  vote  was  split  between  librarians  and  technology 
teachers.  This  is  an  issue  that  seems  to  elicit  growing  awareness  and  concern.  Some 
respondents  said  building  and  district  administrators  supervised  privacy  issues.  Several 
librarians  pointed  out  that  their  automated  circulation  systems  did  not  save  student  checkout 
records,  thus  making  it  easier  to  protect  students’  reading  histories. 

• Who  speaks  out  against  computer  hacking  and  in  what  context?  This  issue  was,  in  most  cases, 
said  to  be  under  the  domain  of  technology  specialists  and  teachers.  In  many  cases, 
respondents  also  mentioned  the  district  acceptable  use  policy  as  a measure  against  hacking. 

The  second  part  of  the  survey  was  about  how  cyberethics  are  being  taught  to  students,  and 
comprised  of  five  questions: 

• Who  tell  students  about  copyright?  As  with  informing  faculty,  the  responsibility  for  telling 
students  about  copyright  fell  almost  entirely  to  librarians.  Most  computer  teachers  who 
responded  also  reported  that  they  try  to  include  copyright  in  their  lessons,  but  also  named 
librarians  as  campus  leaders.  One  librarian  said  she  mentions  copyright  every  time  a class 
comes  in  for  research  during  her  introductory  remarks. 

• Who  talks  to  students  about  software  and  copyright?  Librarians  and  computer  teachers  were 
equally  cited.  However,  about  one  third  of  the  respondents  were  under  the  impression  that  it 
was  not  addressed  at  all  on  their  campuses. 

• Who  talks  to  students  about  computer  hacking  as  an  ethical  issue?  For  the  most  part, 
computer  teachers  were  named  as  the  voices  for  this  issue.  One  such  teacher  commented,  “We 
attempt  to  assuage  the  hackers  by  keeping  a half  step  ahead  of  them.  We  don’t  always 
succeed!  However,  I have  learned  a lot  from  the  kids  when  security  is  breached.”  Teachers 
and  librarians  from  schools  with  strong  acceptable  use  policies  felt  that  guidance  for  students 
was  enhanced  by  the  policies,  if  students  were  sufficiently  aware  of  them. 

• Are  there  specific  lessons  taught  about  cyberethics  on  your  campus?  About  half  of  the 
respondents  said  yes,  but  not  necessarily  to  all  students.  Usually  the  lessons  were  offered  in 
computer  classes.  The  remainder  said  such  lessons  were  not  offered  at  all.  Some  of  those  who 
said  yes  mentioned  state  computer  competency  tests,  which  target  these  issues  along  with 
other  computer  skills,  as  the  impetus  for  instruction  in  cyberethics. 

• The  final  question  was:  In  general,  is  your  campus  really  pushing  ethical  issues?  Responses 
were  evenly  split,  with  half  saying  the  issues  were  truly  emphasized  and  the  other  half  saying 
not  at  all.  Some  were  emphatic  in  stating  that  the  issues  were  neglected.  One  respondent  said, 
“Not  enough  time.  Just  trying  to  teach  the  basic  subjects  is  the  problem.”  Another  said, 
“Mostly  no.  It  is  not  a subject  that  fits  into  class  lessons  or  requirements.”  On  the  other  hand, 
a computer  teacher  reported,  “At  this  point  we  are  stressing  the  responsibility  of  the  staff 
member.  In  a faculty  meeting  the  medial  specialist  goes  over  both  our  acceptable  use  policy 
and  copyright  policy  every  school  year.”  There  were  indications  from  several  people  of  a 
growing  sense  of  need  for  instruction  about  cyberethics.  As  one  reported,  “This  year  one  of 
our  goals  is  to  raise  awareness  through  improved  signage,  reminders  to  staff,  and  workshops 
with  students.”  Another  said,  “It  is  difficult  to  find  the  staff  and  time  to  cover  everything  that 
needs  to  be  discussed,  but  we  are  making  an  effort.” 

What  conclusions  can  be  drawn  from  this  informal  survey?  Several  points  came  to  mind  as  I 
reviewed  the  responses: 
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• First,  I did  not  receive  a single  submission  where  the  respondent  thought  that  enough  was 
being  done  at  his  or  her  school  to  address  cyberethics.  Everyone  reported  room  for 
improvement.  As  computer  use  becomes  more  universal  and  pervasive,  the  need  for  adequate 
training  about  cyberethics  will  surely  increase. 

• Librarians  are  presently  front  and  center  in  efforts  to  inform  faculty  and  students  about 
cyberethics,  and  this  will  continue  to  be  the  case.  The  general  consensus  from  respondents 
was  that  if  anyone  on  their  campus  were  going  to  speak  out  on  such  issues,  it  would  be  the 
librarian. 

• One  thing  that  could  be  done  easily  and  quickly  would  be  to  post  signs  with  copyright 
guidelines  near  copy  machines,  computers,  software  shelving,  and  audiovisual  equipment 
storage  areas.  This  beginning  step  can  provide  initial  awareness  of  the  issue  with  other 
initiatives  to  follow. 

• An  AUP  (Acceptable  Use  Policy)  is  an  important  first  step  in  promoting  ethical  computer  use, 
but  it  is  not  a solution  in  and  of  itself  Many  respondents  indicated  that,  while  their  schools 
had  such  documents,  awareness  on  the  part  of  students,  parents,  and  faculty  was  not 
widespread. 

• While  instruction  about  cyberethics  may  be  lacking,  there  is  a growing  awareness  and 
.concern.  In  the  near  future,  librarians  and  teachers  will  need  to  address  these  issues  with  both 
faculty  and  students  if  they  are  not  already  doing  so. 

Librarians  are  in  an  excellent  position  to  be  leaders  in  promoting  cyberethics  because  they  work 
with  staff  and  students,  because  they  assist  with  research,  and  because  they  are  often  leaders  in  technology 
on  their  campuses.  Training  and  informing  teachers  and  staff  should  be  offered  through  a variety  of  means, 
such  as:  in-service,  web  presence,  handouts,  one-on-one  contacts,  and  any  other  creative  means  which  can 
be  developed.  Often  it  is  best  to  concentrate  on  one  issue  at  a time  when  launching  efforts  to  inform  people 
about  cyberethics,  to  avoid  overwhelming  them  with  information  and  guidelines.  The  emphasis  should  be 
on  positive  reasons  to  respect  the  issues  rather  than  negativism  or  scare  tactics.  Once  teachers  on  a campus 
begin  to  gain  awareness,  they  can  plan  lessons  to  share  the  information  with  their  students. 

What  are  some  effective  ways  to  teach  students  about  ethical  computer  use?  As  with  adults, 
students  will  do  best  if  they  are  not  presented  with  an  overwhelming  amount  of  information  all  at  once. 
After  a general  discussion,  it  is  probably  wise  to  present  one  issue  at  a time  and  explain  each  one  simply 
and  clearly.  Students  are  even  less  apt  than  adults  to  respond  to  stem  admonishing  rhetoric.  While  it  may  be 
tme,  for  instance,  that  stealing  ideas  or  words  is  as  bad  as  stealing  merchandise  from  a store,  lecturing 
students  along  these  lines  is  likely  to  be  poorly  received  and  counter-productive.  Teachers  should  explain 
each  concept  thoroughly,  making  it  clear  exactly  what  rights  students  do  have  and  offering  positive  reasons 
for  behaving  ethically.  This  approach  is  much  more  likely  to  succeed  in  gaining  students’  attention  and 
compliance.  Students  may  be  informed  of  the  negative  consequences  of  violating  ethical  guidelines,  of 
course,  but  the  information  should  be  accurate  and  not  couched  in  overly  threatening  tones. 

Usually  a school  or  district  will  have  an  Acceptable  Use  Policy  (AUP)  that  addresses  ethical  issues 
for  its  students  and  staff.  Unfortunately,  in  many  cases,  students  and  faculty  are  not  thoroughly  informed 
about  their  policy  and  may  even  be  unaware  of  its  existence.  Making  sure  they  get  this  basic  information  is 
one  way  to  begin  discussing  cyberethics.  Then  the  various  sub  topics  can  be  addressed  one  at  a time.  One 
Texas  school  district,  Conroe  Independent  School  District,  developed  a test  for  students  over  the 
Acceptable  Use  Policy.  The  test  was  put  into  place  as  a requirement  before  students  could  gain  permission 
to  use  the  Internet.  Teachers  and  staff  were  also  required  to  pass  the  same  test  and  answer  additional 
questions  about  their  responsibilities  in  order  to  get  computer  accounts.  This  tactic  required  everyone  to 
have  at  least  a basic  awareness  of  district  policies  and  gave  teachers  a starting  point  for  instruction.  The 
tests  were  made  available  online  via  the  district  web  pages,  to  be  taken  online  with  the  resulting  passing 
scores  stored  in  a database  so  that  teachers  could  check,  if  necessary,  to  verily  whether  a student  has 
passed.  If  such  a requirement  is  not  in  place,  a district  Acceptable  Use  Policy  is  still  an  excellent 
springboard  for  discussions  about  cyberethics. 

Another  major  resource  to  reference  when  discussing  computer  ethics  with  both  teachers  and 
students  is  the  Ten  Commandments  of  Computer  Use.  This  widely  quoted  document  was  originated  by  The 
Computer  Ethics  Institute,  and  appears  at  the  end  of  this  paper.  Doug  Johnson,  Director  of  Technology, 
Mankato  Public  Schools,  Mankato,  MN,  has  further  synthesized  ethical  concepts  into  his  “3  P’s  of 
Technology  Ethics: 

1.  Privacy — I will  protect  my  privacy  and  respect  the  privacy  of  others. 
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2.  Property — I will  protect  my  property  and  respect  the  property  of  others. 

3.  a(P)priate  Use — I will  use  technology  in  constructive  ways  and  in  ways  which  do  not  break 
the  rules  of  my  family,  church,  school  or  government. 

Exposing  students  to  clear  and  understandable  guidelines  such  as  these  is  instrumental  in  gaining  their 
comprehension  and  compliance. 

Respect  for  intellectual  property  is  an  obvious  starting  point  for  discussing  cyberethics  with 
students,  since  it  is  an  ongoing  and  increasing  problem.  Students  have  been  copying  information  for  school 
assignments  forever,  of  course,  but  it  has  never  been  eas  ier  than  it  is  today.  Sites  that  sell  or  give  away 
complete  research  papers  abound  on  the  net.  A quick  search  engine  query  brought  up  such  sites  as 
BuyPapers.com,  ResearchPapers.com,  CustomWriting.com  and  SchoolSucks.com,  offering  some  instantly 
downloadable  papers  free  and  others  for  nominal  charges.  Students  can  easily  cut  and  paste  any  amount  of 
text  from  a digital  reference  and  add  it  to  their  own  work.  In  a recent  New  York  Times  article.  Dr.  Donald 
McCabe  described  his  surveys  regarding  student  plagiarism  in  high  school.  He  reported  that  more  than  half 
of  students  surveyed  in  one  project  reported  downloading  an  entire  paper  or  a substantial  amount  of  text 
and  pasting  into  a piece  of  work  without  citation.  He  went  on  to  describe  how  some  enterprising  high 
school  and  college  students  have  mastered  the  art  of  pasting  copied  text  into  a word  processing  application, 
using  the  program’s  auto-summarize  feature,  and  then  pasting  the  summarized  results  into  their  own  papers 
in  an  attempt  to  foil  the  teacher’s  efforts  to  detect  plagiarism.  Dr.  Jamie  McKenzie,  editor  of  From  Now 
On.  The  Educational  Technology  Journal,  suggests  that  such  widespread  student  copying  is  indicative  of 
“mental  softness,”  which  results  in  the  “cut  and  paste  mentality.”  Teachers  need  to  combat  such  plagiarism 
in  order  to  encourage  students  to  really  learn  from  their  research  and  the  experience  of  creating  final 
products. 

One  of  the  best  ways  to  combat  these  sorts  of  creative  plagiarism  is  to  give  assignments  that  do 
not  lend  themselves  to  copying.  The  old  standby  assignment  to  write  a three-page  report  on  a given  topic 
almost  cries  out  to  students  to  plagiarize.  Having  students  present  their  final  product  in  the  form  of  a 
multimedia  presentation,  newsletter,  annotated  bibliography,  or  some  other  format  precludes  cutting  and 
pasting  if  the  assignment  is  designed  carefully  with  that  as  one  goal.  Also,  teachers  and  schools  can  use 
services  such  as  Tumltln.com,  plagiarism  detection  software  that  scans  student  work  and  checks  for 
plagiarized  text.  In  order  to  really  combat  plagiarism,  though,  teachers  and  librarians  must  teach  students 
that  they  should  not  plagiarize  regardless  of  the  nature  of  the  task  assigned,  as  well  as  find  ways  to  detect  or 
discourage  their  doing  so. 

Respect  for  intellectual  property  does  not  apply  only  to  words.  One  area  of  concern  regarding 
plagiarism  is  the  use  of  graphics  and  images.  As  with  words,  pictures  can  easily  be  copied  and  pasted  into 
reports.  They  are  even  more  likely  to  be  appropriated  for  multimedia  productions,  where  the  student  may 
be  less  apt  to  copy  text.  Often  students  and  teachers  do  not  even  think  about  the  fact  that  copying  artwork, 
photographs,  and  other  images  can  also  be  defined  as  plagiarism. 

This  particular  type  of  plagiarism  raised  concern  in  one  Conroe  ISD  school,  York  Junior  High, 
where  the  librarian  noticed  how  frequently  students  were  copying  and  using  images  without  any  regard  for 
citing  their  sources.  The  librarian  and  two  art  teachers  teamed  up  to  increase  student  awareness  of  the 
principle  of  respect  for  intellectual  property  and  graphics.  They  developed  a lesson  and  a project  to  provide 
a positive  alternative  to  copying  images.  The  lesson  covered  copyright  basics,  and  the  project  was  to  build  a 
collection  of  student  generated  clip  art  to  be  given  away  royalty  free  as  a service  to  other  students  and 
teachers.  On  the  first  day  of  instruction,  the  librarian  led  a discussion  about  copyright  and  gave  short  quiz 
to  test  students’  awareness  of  the  issues.  Before  the  lesson,  students  were  unaware  of  fair  use  and  actually 
thought  they  had  fewer  rights  to  use  copied  words  and  images  than  they  do.  They  were  also  uninformed 
about  when  and  how  to  cite  sources.  The  tests  results  and  other  class  discussion  focused  on  positive  reasons 
to  honor  other  people’s  work.  The  teacher  and  librarian  asked  students  to  put  themselves  in  the  place  of  an 
artist  and  imagine  how  they  would  feel  to  find  their  work  taken,  and  even  marketed  by  someone  else. 
Another  reason  to  honor  copyright  that  students  seemed  to  accept,  was  that  it  encourages  original  work.  If 
there  were  not  copyright,  the  librarian  suggested,  we  would  never  hear  a new  song  or  see  a new  work  of  art. 
Everyone  would  use  the  same  old  things  over  and  over,  with  only  slight  variations  or  adaptations. 

Before  they  began  work,  students  were  given  a brief  waiver  to  sign  that  waived  their  royalty  rights 
to  drawings  contributed  to  the  site.  Students  were  told  not  to  submit  anything  that  they  might  someday  want 
to  market,  such  as  an  original  cartoon  character  that  they  might  want  to  develop  in  the  future.  The  class 
brainstormed  various  topics  for  drawings  such  as  academics,  sports,  animals,  holidays,  nature,  fiin,  sports, 
and  others.  By  the  end  of  class  discussion,  they  were  thinking  about  ideas  for  images  to  create. 
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During  the  second  class  session,  the  students  worked  on  clip  art  to  contribute.  The  art  teacher  gave 
each  student  a four-inch  square  of  white  paper,  and  told  students  to  fold  the  squares  into  fourths.  Then  she 
directed  them  to  create  drawings  to  fill  each  of  the  four  one-inch  squares,  using  colored  markers,  map 
colors,  or  crayons.  The  librarian  brought  in  picture  books  to  help  generate  ideas,  but  reminded  students  that 
work  should  be  entirely  original.  Two  areas  of  concern  were  gang  related  drawings  and  reproducing  of 
cartoon  characters  not  well  known  to  the  teachers.  Students  were  reminded  to  avoid  commonly  used  images 
in  gang  graffiti,  and  to  refrain  from  reproducing  look-alikes  of  popular  cartoon  and  animated  characters. 
Before  posting  any  pictures,  the  school’s  police  officer  reviewed  images  for  gang  connotations.  Students 
had  two  class  periods  to  work  on  the  pictures  and  could  work  on  them  at  home  as  well.  The  resulting 
products  were  clever  and  colorful.  Student  assistants  and  the  librarian  worked  together  to  scan  the  images 
and  post  them  to  web  pages.  The  only  editing  done  to  images  was  to  remove  smudges  or  stray  marks. 
Otherwise,  any  picture  accepted  for  use  was  posted  without  digital  altering.  After  completing  the  web 
pages,  the  librarian  and  teachers  unveiled  them  to  the  school  with  much  fanfare  and  praise  for  the  unselfish 
sharing  and  original  work  on  the  part  of  all  participating  students.  Quickly  images  started  to  appear  on 
school  web  pages,  teacher  made  handouts,  in  the  school  newspaper,  and  in  student  projects.  Students 
continued  throughout  that  year  and  subsequent  years  to  add  images.  During  fall  2001,  due  to  the  world 
situation,  students  decided  patriotic  images  would  be  an  especially  important  category.  This  additional 
collection  grew  rapidly  with  other  topics  receiving  additions  as  well,  and  new  topics  being  added  as 
needed.  Students  expressed  pride  in  their  clip  art  collection  and  enjoyed  seeing  their  images  used  by 
teachers  and  fellow  students.  They  also  enjoyed  sharing  the  work  with  their  families  and  friends  via  the 
school  web  site. 

During  the  2001-2002  school  year  the  librarian  and  computer  teacher  decided  to  expand  the  clip 
art  collection  and  involve  computer  students  as  well  as  art  students,  and  to  encourage  submissions  of 
computer  generated  artwork.  They  presented  much  of  the  same  information  to  the  students  as  the  art 
teacher,  but  because  the  computer  teacher  wanted  to  take  the  students  beyond  the  issue  of  plagiarism,  the 
quiz  also  covered  the  issues  of  hacking  and  respect  for  privacy.  Again  results  showed  that  students  were 
unclear  on  many  of  the  issues,  and  the  discussion  that  followed  focused  on  clarifying  misconceptions. 
Students  reviewed  the  existing  clip  art  collection  and  agreed  to  increase  and  add  dimension  to  the  collection 
by  contributing  computer  created  creations.  They  signed  release  forms,  as  had  their  classmates  in  art  class, 
allowing  their  work  to  be  shared  freely.  The  resulting  images  have  increased  the  size  and  variety  of  the 
collection  and  have  encouraged  participation  by  additional  students. 

Conclusion 

Today’s  children  are  growing  up  with  computers  in  their  schools  and  homes.  Dr.  Marin  Berkowitz 
asserts  that  educators  should  not  wait  until  high  school  or  even  junior  high  to  talk  to  students  about 
cyberethics.  He  suggests  that  9-12  year  old  children  can  grasp  basic  issues  and  benefit  from  lessons  about 
the  ethical,  legal,  and  safety  issues  related  to  computer  and  Internet  use.  Realizing  this  goal  involves 
informing  educators,  incorporating  the  issues  into  instruction,  and  making  ongoing  awareness  an  integral 
part  of  how  we  instruct  students  regarding  the  responsible  use  of  technology. 

Helpful  Websites:  An  Annotated  Bibliography 

Copyright  Tips  and  Issues,  West  Loogootee  ISD,  West  Loogootee,  Indiana 
http://www.siec.kl  2.in.us/-^west/online/copv.htm 

This  page  offers  many  helpful  links  and  considerations  for  planning  an  Internet  web  page  and 
paying  particular  attention  to  the  ethical  issues  involved. 

Crash  Course  in  Copyright: 

http://www.utsvstem.edU/OGC/lntellectualPropertv/cprtindx.htm#top 

This  University  of  Texas  site  lives  up  to  its  title  in  presenting  basic  issues  related  to  intellectual 
property  and  copyright  for  the  layman. 

The  Cyber  Citizen  Partnership  Page 

http://www.cvbercitizenship.org/ 

This  site,  created  with  a grant  from  the  U.S.  Department  of  Justice,  is  dedicated  to  teaching 
children  safe  and  responsible  use  of  the  Internet. 

Ethics  in  Computing: 
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http://courses.ncsu.edu/classes-a/computer  ethics/ 

The  first  page  of  this  project  is  a site  map  resembling  an  actual  map,  with  regions  marked  off  for 
various  ethical  issues  such  as  computer  abuse,  intellectual  property,  and  privacy.  Clicking  on  the 
locations  leads  to  articles  and  activities  relating  to  the  respective  issues. 

From  Now  On:  The  Educational  Technology  Journal 
http://www.fno.org 

Jamie  McKenzie,  author  and  consultant,  edits  and  publishes  this  web  Journal  which  has  a number 
of  excellent  articles  relating  to  technology  and  ethics. 

Homepage  for  Doug  Johnson,  Author,  Speaker,  and  Consultant  on  School  Library  Media  Center 
Issues 

http://www.dougHohnson.com/index.html 

Doug  Johnson,  Director  of  Media  and  Technology  at  Mankato  Schools,  offers  articles  and 
activities  related  to  ethics  and  technology. 

Internet  Do’s  and  Don’ts 

http://www.usdoj.gov/kidspage/do-dont/kidintemet.htm 

The  U.S.  Department  of  Justice  sponsors  this  site  which  stresses  safe  and  responsible  Internet  use. 
Original  K-12  School  Student  Clip  Art 

http://www.shsu.edu/~lis  mah/documents/clipartportal.html 

This  is  the  royalty  free  clipart  collection  described  in  the  article,  created  by  students  and  teachers 
at  York  Junior  High  School,  Conroe  Independent  School  District,  Texas. 

Technology  and  Learning’s  Copyright  Primer:  Fair  Use  Copyright  Quiz 
http://www.hal  Ida  vidson.net/Quiz%2Q. pdf 

This  pdf  document  is  an  interesting  quiz  for  teachers  and  librarians,  with  answers  provided. 

The  Ten  Commandments  of  Computer  Ethics: 

http://www.cpsr.org/program/ethics/cei.html 

This  is  the  official  web  site  for  this  document,  offered  by  the  Computer  Ethics  Institute. 

Web  Awareness:  Knowing  the  Issues: 

http://www.media-awareness.ca/eng/webaware/home.htm 
Canada’s  Media  Awareness  Network  offers  this  site.  It  has  information  and  activities  for  students,  parents, 
and  educators  regarding  the  Internet  and  ethical  issues. 
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Abstract:  Placing  powerful  technology  tools  in  students’  hands  along  with  activities 
designed  to  encourage  collaboration,  exploration,  and  critical  thinking,  causes  a role  shift 
to  occur.  Students  become  researchers  and  presenters,  constructing  knowledge  and 
adding  it  to  a global  knowledge  base.  Faculty  who  incorporate  technology  into  their 
course  work  then  can  become  guides  and  facilitators.  However,  the  challenges  created  by 
this  role  shift  require  a reframing  of  our  faculty  development  model.  The  Innovative 
Technology  Center  at  the  University  of  Tennessee  designed  and  implemented  a pilot 
project,  putting  powerful,  wireless  laptops  into  student  and  faculty  hands.  The 
Humanities  area  of  Arts  and  Sciences  was  chosen  for  this  project  because  they  are 
typically  underserved  in  terms  of  technology  resources,  yet  rich  in  opportunities  for 
collaborative  work.  With  laptop  computers,  a versatile  suite  of  software  tools,  and  a 
customized  WISH  institute  for  training  before  project  launch,  our  implementation  team 
crafted  a model  faculty  development  project  that  addressed  key  aspects  of  transitioning 
traditional  faculty  to  a more  collaborative  and  dynamic  learning  environment. 


Context  of  the  WISH  Project 

“To  ensure  that  America's  students  are  the  best-educated  in  the  world,  we  have  to  have  top -quality, 
innovative  higher  education;  replicate  the  educational  approaches  that  work;  and  take  advantage  of  all  the 
new  technologies  that  can  strengthen  student  learning”  (Office  of  Postsecondary  Education,  2001).  Many 
voices  agree  that  students  need  to  emerge  from  their  undergraduate  experience  equipped  for  the  modem 
work  environment.  Employers  state  unequivocally  that  they  want  information-savvy,  team-comfortable, 
and  problem-solving  employees.  In  a government  report  surveying  tfie  impact  of  IT  on  the  American 
economy,  the  authors  note  that  “Even  for  workers  in  non-IT  jobs,  basic  IT  skills  are  becoming  a 
requirement.... Educators,  too,  recognize  the  growing  need  for  skilled  workers  and  are  designing  curricula 
to  include  basic  IT  skills  training...”  (Office  of  Policy  Development,  2001,  p.  41).  We  must  move  rapidly 
toward  creating  learning  environments  in  which  students  receive  the  skills,  both  cognitive  and  technical, 
which  will  enable  them  to  succeed  in  the  contemporary  world.  While  curriculum  content  is  the  same  as  in  a 
traditional  classroom,  the  ways  in  which  we  engage  students  with  this  content  need  to  be  dramatically 
altered.  So  our  team  at  ITC  sought  a way  to  bring  technology  out  of  the  lab  into  the  classroom  and  then 
examined  one  model  for  how  to  prepare  faculty  for  new  opportunities  and  challenges. 

A pilot  project  involving  the  use  of  technology  and  collaborative  methods,  appeared  certain  to 
pose  challenges  for  traditional  faculty.  The  shift  from  teacher-centric  to  student -centric  classroom  does  not 
follow  a smooth  path.  From  the  inception  of  the  project,  the  implementation  team  understood  that 
resistance  to  change  would  appear,  even  among  self-described  innovators.  This  paper  will  describe  the 
WISH  institute  as  a creative  vehicle  for  assisting  faculty  in  transforming  the  classroom.  The  workshops  of 
our  WISH  institute  were  designed  to  explore  and  create  new  strategies  for  delivering  material  and 
structuring  classroom  interactions.  Our  training  focused  on  designing  collaborative  activities  that  would 
increase  students’  engagement  with  the  material.  We  looked  at  ways  to  utilize  common  communication 
tools  such  as  email  and  discussion  forums  to  leverage  the  power  of  networked  laptops  to  enhance  writing 
and  discussion  skills.  At  every  stage  of  the  project,  we  gathered  faculty  feedback  and  addressed 
pedagogical  and  technical  issues  as  they  arose. 
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Transformation  of  teaching  practice  in  higher  education  requires  a convergence  of  initiatives  and 
the  vision  to  identify  opportunities  for  pedagogical  research.  In  the  WISH  project,  three  university  groups, 
the  Network  Services,  ITC,  and  the  College  of  Arts  and  Sciences,  converged  to  allow  exploration  of 
numerous  issues  of  critical  importance  today  as  higher  education  responds  to  the  call  for  change  in  the  way 
it  educates  students.  The  campus-wide  wireless  initiative  was  launched  by  the  administration  in  the  summer 
of  2001  and  is  being  implemented  by  Network  Services,  The  Innovative  Technology  Center  brings  together 
faculty  development  specialists,  instructional  designers,  network  and  programming  specialists,  and  graphic 
designers,  to  serve  the  pedagogical  needs  of  the  entire  faculty.  The  College  of  Arts  and  Sciences, 
traditionally  underserved  in  terms  of  access  to  technology,  offered  us  access  to  traditional  faculty  who  had 
expressed  a desire  to  explore  new  pedagogical  possibilities. 

University  Wireless  Initiative 

An  administrative  push  for  a wireless  campus  network  had  created  the  context  within  which  the 
project  unfolded.  In  the  summer  of  2001,  Ihe  University  of  Tennessee  was  launching  one  of  the  largest 
802.1  lb  wireless  initiatives  in  the  United  States  based  on  number  of  access  points,  and  the  WISH  project 
was  developed  concurrently,  (For  more  information,  please  see  http://wireless.utk.edu).  This  simultaneous 
development  of  initiatives,  one  technical  and  one  pedagogical,  added  to  the  challenges  we  faced,  but  also 
offered  extraordinary  opportunities  to  observe  how  faculty  and  students  function  and  adapt  in  an 
experimental  environment.  One  of  the  first  casualties  of  the  project  was  the  collaborative  networking 
software,  whose  primary  purpose  was  to  permit  the  instructor  to  call  up  a student’s  screen  onto  the  laptop 
connected  to  the  projector.  As  Network  Services  added  security  layers  to  the  wireless  rollout  and  as  they 
expanded  the  subnets,  the  collaborative  software  proved  unable  to  negotiate  clients  on  a network  that 
dynamically  assigned  IP  addresses  to  individual  machines.  The  software  had  functioned  well  in  a static  IP 
environment,  but  simply  collapsed  when  dynamic  host  addressing  was  introduced.  However,  it  had 
functioned  long  enough  to  allow  us  to  see  the  enormous  potential  for  true  exploratory  sharing.  In  their  Final 
Reports,  several  WISH  participants  referenced  the  teaching  potential  of  this  software,  so  our  search  for  that 
particular  tool  continues.  A positive  convergence  emerged  from  the  classroom  renovation  project  that  was 
occurring  concurrently.  Smart  classrooms  were  being  created  across  campus,  and  our  participants  saw  a 
concerted  University  effort  to  provide  technology-enhanced  learning  environments  for  instructors  and 
students. 

In  addition,  the  wireless  cards  often  experienced  difficulty  when  they  tried  to  connect  to  the 
network  through  a malfunctioning  access  point.  Unfortunately,  the  task  bar  icon  which  indicated  a network 
connect  was  not  discriminating  enough  to  know  that  the  access  point  was  not  offering  “access”  to  the 
Internet.  The  network  monitoring  software  also  gave  false  positive  readings  for  these  “live”  but 
malfunctioning  access  points.  Much  fhistration  and  confusion  ensued.  Trouble-shooting  these  early 
obstacles  proved  time-consuming  for  our  network  specialists  and  frustrating  for  the  instructors  who  simply 
wanted  the  technology  tools  to  work. 

Innovative  Technology  Center 

The  WISH  Pilot  Project  was  framed  from  the  very  start  as  an  exploration  of  how  to  redefine  the 
traditional  classroom  in  a networked  environment  as  well  as  the  diverse  support  issues  surrounding  the 
University’s  wireless  initiative.  We  invited  faculty  who  wanted  to  establish  innovative  teaching  practices 
and  explore  research  possibilities  related  to  implementing  wireless  computing  in  a classroom  setting  to 
apply  for  participation.  We  asked  what  the  classroom  might  look  and  feel  like  when  powerful  computing 
potential  was  wedded  to  collaborative  activities,  specifically  designed  to  challenge  students  to  critical 
thinking  and  problem-solving.  Unlike  traditional  grants,  this  call  for  proposals  was  open-ended  in  unusual 
ways.  We  asked  for  faculty  willing  to  re-examine  their  traditional  teaching  strategies  in  the  light  of  new 
possibilities  offered  by  mobile,  networked  classrooms. 

Supporting  the  project  successfully  involved  a group  of  professionals  with  a wide  range  of 
expertise,  a shared  desire  to  expand  definitions  of  teaching  and  learning,  and  a commitment  to  customer 
service.  The  implementation  team  consisted  of  the  Director  of  ITC  with  her  humanities  background  behind 
a doctorate  in  instructional  technology,  our  technologies  integration  specialist,  a web  instructional 
technologist  with  a strong  humanities  background,  and  a program  coordinator  who  handled  technical 
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issues,  group  communications,  and  the  complexities  of  scheduling  the  two  carts  filled  with  laptops.  In 
addition,  the  program  coordinator  did  in-class  training  on  the  use  of  the  laptops  and  the  network,  thus 
freeing  faculty  to  concentrate  on  reframing  the  learning  environment.  Brainstorming  sessions  had  launched 
the  project  and  the  team  met  regularly  to  deal  with  the  issues  as  they  arose. 

College  of  Arts  and  Sciences 

“Deep,  hands-on  learning  through  research  is  commonplace  in  the  sciences,  but  it  is  more 
challenging  to  design  these  experiences  for  students  in  the  humanities.”  (Brown,  2000,  p.  222).  Our  project 
was  deliberately  crafted  to  examine  the  pedagogy  as  well  as  the  mental  maps  of  traditional  instructors 
whose  previous  access  to  technology -rich  environments  had  been  limited.  Thirteen  faculty  from  Modem 
Foreign  Language  and  Literature,  Political  Science,  English,  History,  Philosophy,  Religious  Studies, 
African-American  Studies,  and  Human  Services  were  chosen  to  participate  in  the  project.  The  courses 
impacted  ranged  from  “Images  of  Jesus”  in  the  Department  of  Religious  Studies  to  “Introductory 
Conversational  Portuguese”  in  the  Modem  Foreign  Language  and  Literature  department.  Several  WISH 
team  members  possessed  strong  humanities  backgrounds  and  were  intrigued  by  the  idea  of  developing 
workshops  to  transition  traditional  faculty  to  new  ways  of  constructing  learning  experiences  in  a networked 
environment.  After  putting  out  the  call  for  proposals  to  this  college  and  establishing  project  parameters,  we 
then  developed  the  WISH  Institute  to  frame  the  key  issues  and  open  the  dialogue.  We  now  faced  the 
conceptual  challenge  of  moving  instmctors  from  lecture  and  face-to-face  discussion  paradigm  to  new  ways 
to  engage  their  students  with  course  material.  A shift  in  the  stmcture  and  interaction  in  the  classroom 
challenges  faculty  in  two  ways:  their  comfort  level  with  the  technology  itself  and  their  comfort  level  with 
new  ways  of  designing  and  implementing  learning  activities. 

Project  Requirements 

The  call  for  proposals  asked  about  faculty  willingness  to  innovate  and  to  explore  current  pedagogy 
with  an  eye  to  transforming  it.  The  call  asked  about  current  uses  of  technology  in  teaching  and  thoughts  on 
potential  for  change.  From  the  pool  of  proposals  we  selected  only  14,  based  upon  our  ability  to  offer  ideal 
support  as  well  as  upon  the  energy  and  commitment  we  sensed  in  their  writing.  In  addition,  we  deliberately 
chose  faculty  with  diverse  skill  levels.  The  selected  faculty  received  a fully  equipped  wireless  laptop,  a 
suite  of  software  tools,  and  a stipend.  In  return  they  were  required  to  commit  to  the  project  requirements, 

• Integrate  wireless  laptops  in  your  course  for  a minimum  of  1/3  of  the  semester. 

• Participate  in  the  WISH  Summer  Institute.  The  WISH  Summer  Institute  will  require  six  hours  of  core 
classes  (to  be  held  at  ITC)  and  ten  hours  of  individualized  training  (that  can  be  defined  through  flexible 
learning  opportunities).  The  core  classes  will  address  technical,  instructor-centered,  and  student- 
centered  strategies.  The  individualized  training  will  focus  on  reshaping  the  curriculum  and  content  in 
order  to  make  maximum  use  of  networked  resources. 

• Attend  bimonthly  meetings  with  the  ITC  WISH  Implementation  Team  (WISH  IT)  to  be  held  for  the 
duration  of  the  project. 

• Conduct  student  assessment  at  the  end  of  the  Fall  2001  term.  ITC  will  provide  the  basic  assessment 
tools. 

• Participate  in  a WISH  Faculty  pre  - and  post-project  assessment. 

• Submit  a final  report. 

WISH  Institute 

The  WISH  Institute  held  in  the  summer  of  2001  directly  addressed  the  challenges  of  weaving 
collaboration  and  connectivity  into  the  traditional  classroom.  After  surveying  participants  to  identify  their 
comfort  level  with  software,  hardware,  and  networks,  the  implementation  team  developed  three  courses  to 
address  pedagogical  and  technical  issues  and  prepare  the  foundation  for  our  research  interests.  The  first 
course,  “Technical  Strategies  for  Wireless  Laptops,”  focused  on  basics  of  handling,  maintaining,  and  using 
a laptop  computer.  Issues  included  battery  charging,  hooking  up  projectors,  and  mobility  across  campus 
subnets.  Participants  discussed  multiple  log-ons,  the  realities  of  shared  bandwidth,  and  explored  classroom 
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management  issues.  Laptops  were  stored  on  a “mobile”  cart,  but  the  term  “mobile”  was  a stretch  and 
negotiating  crowded  hallways  and  multiple  floors  challenged  the  instructors  as  well  as  the  support  staff. 
Each  cart  contained  16  PC  laptops  and  one  projector. 

The  second  course  focused  on  “Instructional  Strategies  for  the  Wireless  Classroom.”  The 
University  uses  Blackboard  as  its  course  management  system  and  our  Institute  addressed  some  of  the  built- 
in  tools.  Participants  explored  strategies  for  maximizing  use  of  networked  resources,  from  managing  email 
to  receiving  assignments  via  the  digital  dropbox  in  Blackboard,  We  discussed  how  to  manage  increasing 
numbers  of  email  messages  to  designing  a discussion  forum  topic  that  would  engage  students  in  dynamic, 
professional  discourse.  We  examined  strategies  for  integrating  the  Web  into  the  classroom  environment. 
We  went  on  virtual  field  trips  and  used  a web-page  archiving  software  to  capture  a website  to  the  hard 
drive,  in  case  of  connectivity  failure.  Noted  one  WISH  participant,  “I  have  also  learned  and  have  re- 
committed myself  to  the  notion  of  planning  back-up  lessons.  When  working  with  the  wireless,  my  thought 
is  that  three  backup  plans  are  necessary....”  (Woodside,  2001). 

The  final  course,  “Collaborative  Models  in  the  Wireless  Laptop  Classroom,”  explored  ways  to 
group  students  for  effective  collaborative  work  through  issue  groups,  triads,  and  collaborative  writing 
teams.  We  shared  possible  methods  for  assigning  students  individual  roles  based  on  divisions  of  labor  to 
enhance  learning.  Participants  ventured  on  a “webquests  about  webquests,”  in  order  to  experience  both  the 
structure  of  a webquests  and  the  way  it  felt  to  work  collaboratively  toward  an  end  product.  One  of  the  key 
obstacles  to  faculty  adopting  collaborative,  problem-based  learning  is  the  challenge  of  assessing  both  the 
process  and  the  product,  so  we  assembled  readings  and  samples  of  assessment  tools  to  address  this  potential 
barrier. 

In  addition  to  the  six-hour  WISH  Institute,  faculty  were  asked  to  schedule  an  additional  ten  hours 
of  customized  support,  based  on  their  curricular  plans  and  their  levels  of  expertise,  Paul  Hagner  has  noted 
that  “many  faculty,  especially  the  Risk  Aversives,  need  flesh  and  blood  support  to  make  their 
transformation.”  (Hagner,  2001),  The  WISH  implementation  team  provided  that  “flesh  and  blood  support” 
to  assist  these  instructors  in  transforming  their  teaching  as  well  as  the  learning  environment. 

Conclusions 

We  had  initiated  an  ambitious  project  and  garnered  many  insights  from  our  experiences.  Our 
greatest  insights  and  understandings  came  from  carefully  watching  these  faculty  members  as  they  moved 
through  the  experience.  Our  recommendations  emerge  from  a careful  examination  of  the  project  as  an 
experiential  whole.  Two  major  categories  deserve  addressing,  technical  and  teaching/leaming. 

In  the  technical  arena,  expect  heavy  support  demands  and  meet  each  demand  with  patient 
attention.  Always  expect  the  unexpected.  The  recognition  of  the  false  positives  on  connectivity  did  not 
emerge  until  we  began  to  deconstruct  the  entire  project,  in  search  of  areas  to  address  in  the  next  iteration.  In 
addition,  we  would  caution  that  you  be  very  selective  about  the  software  that  is  included  in  the  package  for 
faculty  and  students.  Two  critical  pieces  of  software,  the  collaborative  networking  software  as  well  as  the 
networked  writing  environment  software,  did  not  function  as  advertised,  causing  mutual  frustration  all 
around.  In  addition,  the  critical  website  capture  tool  did  not  function  as  efficiently  as  we  had  hoped  as  a 
backup  tool.  Increasing  faculty  comfort  level  with  these  new  tools  is  essential.  Student  comments  at  the 
close  of  the  project  indicated  that  they  were  keenly  aware  of  the  impact  on  the  learning  environment  of  a 
faculty’s  readiness  to  use  technology  tools  effectively. 

In  the  area  of  teaching  and  learning,  we  recommend  staying  focused  on  pedagogy,  instructional 
design,  and  multiple  strategies  for  using  the  tools,  even  as  the  p*oject  seems  enmeshed  in  technical 
challenges.  Deliberately  create  community  around  the  project  and  plan  for  its  continuation.  Only  through 
rich  dialogue  will  we  move  together  into  the  future.  Communicate  constantly  with  all  stakeholders,  using 
all  the  tools  available,  from  the  ubiquitous  email  to  discussion  forums,  to  bi-monthly  debriefings,  and 
micro -teachings  where  successes  are  shared. 

Success  can  be  measured  in  many  ways.  Thanks  in  part  to  the  range  of  faculty  skill  levels,  we 
were  able  to  note  a wide  range  of  successful  learning  strategies.  One  technically  literate  faculty  member 
restructured  an  entire  course  around  the  wireless  laptops.  She  comments  in  her  final  report,  “I  thought 
about  why  I was  doing  what  I was  doing  in  the  classroom,  right  down  to  the  last  detail”  (Kelly,  2001,  p,  1). 
Another  instructor  used  the  brainstorming  tool.  Inspiration,  for  a collaborative  exercise  within  one  class. 
His  initial,  small  success  allowed  him  to  be  more  aware  of  the  possibilities  for  enhancing  language  learning 
with  such  tools. 


As  James  Ellsworth  notes,  “When  change  is  being  driven  by  changes  in  the  society  around  us  (a 
suprasystem),  we  must  be  willing  to  rethink  everything  about  the  way  you  teach  and  learn  (Ellsworth,  1997, 
p.  6).  The  WISH  project,  with  an  ambitious  team  of  four  ITC  members,  began  the  journey  to  identify  the 
training  that  would  empower  the  traditional  faculty  who  have  long  been  the  mainstay  of  our  universities  to 
become  the  innovative  teachers  of  the  next  generation.  The  task  is  urgent;  the  challenge  formidable,  and  the 
journey  exciting  in  every  way. 
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Faculty  do  not  often  engage  the  problem  of  designing  research  tools  that  evaluate  the  effectiveness  of  specific 
technologies  in  teaching/leaming.  When  we  do,  we  most  often  defer  to  student  comments  (e.g.,  "I  liked  the  PowerPoint 
slides  a lot!")  which  tend  not  to  be  very  analytic  or  to  our  own  informal  observations.  While  data  from  a well-designed 
research  study  would  help  us  to  make  better  choices  regarding  pedagogical  strategies  using  technology,  it  seems  more 
common  that  we  make  those  choices  based  on  "teacher's  instinct"  and  "what  I like  to  do."  It  is  hard  to  justify  using  the 
very  latest  technology  simply  because  it  is  exciting  and  new.  It  is  equally  difficult  to  argue  that  we  should  teach 
students  presentation  skills  when  our  content  area  is  math. 

Clearly  the  complexities  of  pedagogy  stand  in  the  way  of  developing  assessment  tools  that  evaluate  the  real  impact  of 
technology  in  teaching/leaming  in  complex  subject  domains.  Not  only  do  teachers  need  to  consider  student  learning 
styles  in  designing  instructional  materials,  but  they  must  also  take  into  account  the  needs  of  their  discipline  as  well  as 
their  course  content,  development,  texts  and  objectives.  Much  of  the  literature  on  developing  technology  for  learning 
seems  to  focus  on  the  potential  of  the  hardware/software  toolbox  rather  than  the  potential  of  the  learner. 

In  this  short  paper,  I propose  to  develop  and  explore  a taxonomy  of  characteristics  specific  to  different  educational 
disciplines.  I do  not  expect  this  list  to  be  exhaustive.  The  goal  is  to  develop  a research  question  that  identifies 
discipline-specific  needs  that  can  be  linked  with  specific  technology-using  pedagogical  strategies  (Benney,  1999).  I 
suggest  that  the  following  list  of  "Primary  Course  Characteristics"  illustrates  my  beginning  point.  To  each  is  attached 
descriptive  attributes  and  what  seems  to  be  an  obvious  connection  with  a discipline,  a field  of  study  or  a specific 
learning  problem.  This  is  an  exploratory  and  experimental  approach. 

Primary  Course  Characteristics: 

1.  Content  intensive  - the  major  emphasis  here  is  factual  information  presented  in  formulas,  laws  or  contained  in  a 
relational  database.  There  are  many  applications  from  engineering  data  to  the  phone  book.  Historical  information 
certainly  needs  a factual  base,  but  most  historians  would  claim  it  is  more  than  that. 

Comment  - It  seems  obvious  that  presentation  software,  web  pages  and  video,  like  text,  can  present  even  relatively 
complex  factual  information.  But  can  a documentary  film  present  the  information  needed  by  a history  class  in  a 
way  that  simple  text  could  not?  Would  a 30  minute  “talking  head”  interview  with  an  eye  witness  to  the  Vietnam 
war  provide  a more  interesting  learning  experience  than  six  5 minute  clips  on  various  aspects  of  the  same  topic? 
What  is  appropriate  for  the  content  needs  of  the  course? 

2.  Analytical  — reducing  an  idea  to  its  logical  components,  but  also  in  a laboratory  setting  (broadly  defined),  the 
strategy  of  experimentation. 

Comment  - The  noted  scholar  of  Theology,  Bernard  Cooke  has  suggested  that  the  biggest  problem  in 
undergraduate  education  today  is  that  students  cannot  read!  He  explains  that  students  cannot  extract  information  — 
analyze  the  information  - from  text  (Benney,  A.  (2001).  How  can  technology  be  used  to  exemplify  analytic  skills 
and/or  to  demonstrate  the  process?  When  the  process  is  the  objective,  it  seems  experience  becomes  more  important 
than  data. 

3.  Poetic  “ the  use  of  words  or  images  in  a metaphorical  and  imaginative  way  to  stimulate  the  creative  interpretation 
of  the  learner.  This  could  be  practical  in  developing  inventions  or  simply  entertaining. 

Comment  - Is  it  possible  to  teach  creativity?  Or  can  it  only  be  exemplified?  Does  using  a computer  to  demonstrate 
puzzles  that  have  multiple  solutions  or  requiring  students  to  develop  their  own  problenvsolving  web  sites 
encourage  creativity?  This  is  another  example  of  a process  being  central  to  the  learning  objective. 
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4.  Organizational  - More  the  development  of  managing  strategies,  but  also  implies  the  need  to  understand  social  and 
psychological  structures  in  a practical  way. 

Comment  - The  use  of  role  playing  games  as  well  as  video  presentations  of  cases  for  discussion  and  analysis  have 
the  potential  of  teaching  (again  by  example)  the  skills  needed  for  management  tasks.  Simulations  have  an  effective 
application  in  many  educational  settings.  For  example,  a flight  simulator  lets  you  learn  from  your  mistakes  and 
walk  away  unscathed. 

5.  Theoretical  - Similar  to  “Poetic,”  tries  to  get  at  the  underlying  reasons  for  why  things  are  the  way  they  are; 
grounded  in  fact  and  logically  constructed. 

Comment  - Are  students  so  programmed  that  they  think  every  question  has  an  answer  or  is  it  possible  to  teach  the 
hypothesis  behind  the  question?  Perhaps  concept  mapping  that  layers  the  question  could  exemplify  the  process  of 
developing  theories. 

6.  Sensory  — certainly  oriented  towards  entertainment  and  enjoyment.  Art,  music  and  theater  dramatizes  the  human 
environment.  But  the  skill  of  a surgeon  as  well  as  a pilot  have  a sensory  element. 

Comment  - One  of  the  best  examples  of  an  application  of  media  to  support  this  characteristic  is  M-TV.  What  is 
missing  from  a flight  simulator  is  the  "feel"  of  the  controls  - something  essential  to  actual  flying. 

7.  Manipulative  — forming  artifacts  from  raw  materials  as  in  any  manufacturing  or  construction  process.  The  process 
itself  is  described  in  a complex  series  of  interrelated  steps. 

Comment  - This  is  a clear  case  of  providing  instructions  using  video  examples  or  animated  step-by-step  cases.  But 
either  the  students  must  commit  the  process  to  memory  or  have  the  example  available  at  the  time  of  need. 

8.  Linguistic  — the  basic  art  of  communicating  through  words,  symbols  and  images  requires  both  the  memorization  of 
vocabulary  as  well  as  the  experience  of  syntax  and  grammar  none  of  which  may  have  any  rational  foundation. 
Comment  - The  benefits  of  audio  tapes,  video  and  animated  clips  are  well  documented  in  language  learning. 
Repetitive  hearing,  seeing  and  doing  clearly  facilitate  the  development  of  communication  skills.  But  motive  still 
remains  a problem  and  it  would  seem  that  the  task  of  memorizing  vocabulary  would  make  text  more  useful  because 
it  is  more  portable.  Can  technology  motivate  or  would  the  possibility  of  contacting  foreign  language  speakers  via 
the  web  discourage  learners  who  progress  slowly? 

This  list  is  not  final,  but  is  intended  as  a place  to  begin  a conversation.  T hope  that  discussions  with  technology 
users/teachers  from  different  content  areas  will  lead  to  a refinement  and  clarification  so  that  it  might  be  posstole  to  use 
these  discriminators  to  design  multimedia  applications  that  are  appropriate  for  discipline  specific  objectives. 


Partial  Bibliography 

Benney,  A..  (1999).  T Made  You  Look  You  Dirty  Crook,  You  Stole  Your  Mother's  Pocketbook!’  How  To  Surprise 
Your  Students  With  the  Opportunity  to  Learn.  Proceedings  of  Ed-Media  - Ed-Telecom  99,  Charlottesville,  VA: 
Association  for  the  Advancement  of  Computing  in  Education. 

Clark,  R.  E.  and  Craig,  T.  G.  (1992).  Research  and  Theory  on  Multi- media  Learning  Effects.  In  M.  Giardina  (Ed.), 
Interactive  Multimedia  Learning  Environments.  Human  Factors  and  Technical  Considerations  on  Design  Issues  (pp. 
19-30).  Heidelberg:  Springer. 

Kozma,  R.  B.  (1991).  Learning  With  Media.  Review  of  Educational  Research,  61(2),  179-211. 

Liu,  M.  and  Reed,  W.  M.  (1994). The  Relationship  Between  the  Learning  Strategies  and  Learning  Styles  in  a 
Hypermedia  Environment.  Computers  in  Human  Behavior,  1 0, 4 1 9-434. 


177 


Building  a Molecular  Workbench:  Underpinning  Core  Science  Concepts 

Dr.  Boris  Berenfeld,  Dan  Damelin  and  Dr.  Qian  Xie 

Concord  Consortium 
http://www.concord.org 


To  develop  a scientific  way  of  reasoning  and  to  understand  macroscopic  phenomena  discovered  by  modem  science 
and  their  applications  to  practical  fields  such  as  nanotechnology,  molecular  genomics  or  proteomics,  students  need 
robust  mental  models  of  atoms,  molecules,  and  their  interactions.  Such  understanding  allows  students  to  grasp  what 
lies  at  the  heart  of  modem  biology,  chemistry,  and  physics.  There  is  considerable  evidence  that  students  have  major 
misconceptions  about  atoms  and  molecules,  and  few  ideas  about  how  the  forces  and  motions  at  this  scale  relate  to 
macroscopic  properties. 

The  Molecular  Workbench  project  rhttp://www. concord.org/workbcnch/index.htmn.  which  is  funded  by  the  US 
National  Science  Foundation,  is  studying  the  effectiveness  of  using  highly  manipulable  molecular  dynamics  models 
to  enhance  student  learning.  It  facilitates  learning  through  the  use  of  guided  inquiry  about  the  connections  between 
microscopic  and  macroscopic  properties  of  materials.  To  this  purpose  we  are  creating  atomic-scale  models,  several 
macro-scale  simulations  with  interactive  Flash  applications,  and  are  using  an  integrating  language  called 
Pedagogica.  In  collaboration  with  teachers,  instructional  materials  using  these  tools  are  now  being  tested  in  a 
variety  of  middle  and  high  school  classrooms. 

OviX  Molecular  Workbench  model  can  already  demonstrate  states  of  matter,  gas  laws,  phase  change,  latent  heat, 
diffusion,  gas  absorption,  osmosis,  thermal  diffusion,  conformational  changes,  and  some  of  the  properties  of  liquid 
crystals.  Using  scaffolded  models  and  supporting  curricula,  students  are  able  to  predict  new  micro-macro 
connections. 

The  Molecular  WorkbenchlD  and  Molecular  WorkbenchSD  (formerly  called  Oslet)  are  molecular  simulation 
engines  developed  for  the  project.  MW2D  is  written  in  pure  Java.  MW3D  is  written  in  Java  and  Java3D.  The  models 
are  based  on  molecular  mechanics,  an  important  part  of  contemporary  computational  chemistry  and  biology.  MW2D 
develops  an  object-oriented  framework  (OOF)  for  interactive  mo  lecular  simulations.  The  core  of  OOF  provides 
software  abstractions  of  basic  objects  for  atoms,  molecules,  fields,  boundaries,  data  sets,  and  the  interactions  among 
them.  MW2D  consists  of  four  major  functional  modules:  a builder,  a viewer,  a simulator,  and  a data  analysis 
environment.  The  software  abstraction  model  in  the  OOF  provides  the  interfaces  for  these  modules  to  interact  and 
integrate. 

Pedagogica,  which  was  developed  by  Paul  Horwitz  originally  to  control  a genetics  simulation  cdAXcdBioLogica,  has 
now  been  interfaced  With  MW2D.  Pedagogica  simplifies  the  conversion  of  open-ended  models  and  tools  into 
learning  activities  that  support  guided  inquiry  and  generate  specific,  useful  assessment  data.  We  call  the  combination 
of  a model  with  Pedagogica  a "hypermodel". 

Pedagogica  is  a client  application  that  controls  models  and  their  appearance,  displays  what  options  are  available, 
receives  input  concerning  the  state  of  the  model,  controls  the  interactions  with  the  user,  and  coordinates  other 
resources  that  might  be  used  with  the  model.  Pedagogica  not  only  facilitates  student  choices,  but  also  queries  the 
users  and  monitors  their  progress. 

The  curriculum  activities  are  designed  to  encourage  students  to  uncover  key  attributes  of  molecular  systems, 
develop  their  own  models,  and  test  these  against  a professional  model.  Proprioceptive  activities  with  visual 
feedback,  a “virtual  construction  kit”  to  arrange  molecules  in  polymer  chains,  explorations  with  interactive  models 
followed  by  discussions  revealing  the  explanatory  and  predictive  power  of  the  models  --  all  work  to  build  and  enrich 
the  student's  mental  models  of  the  connection  between  macroscopic  phenomena  and  molecular  worlds. 


Eight  curriculum  activities  on  Atoms  in  Motion,  and  a week-long  “mini-unit”  on  States  of  Matter,  both  based  on 


atomic -scale  models  have  undergone  formative  testing  and  are  now  undergoing  summative  testing  in  public  middle 
school  8*^,  and  1 1th  grade  classrooms . The  activities  include  integrated  hands-on  labs,  computer  modeling,  and 
kinesthetic  modeling  activities.  New  research  modules  on  Chemical  Bonds  and  Solutions  will  be  tested  this  Spring 
and  Fall. 

Our  primary  research  questions  are: 

- Does  experience  with  such  a tool  help  students  in  8-9^^  grades  internalize  accurate  mental  models  of  atomic -scale 
phenomena,  and 

- Does  this  in  turn  help  them  become  more  expert  in  reasoning  about  phenomena  at  different  levels,  from  micro  to 
macro. 


We  are  conducting  our  research  now.  Students  to  date  have  been  highly  engaged,  responsive  in  discussions,  and 
have  made  connections  between  the  various  types  of  activities  conducted.  For  example,  they  performed  a computer 
simulation  and  a kinesthetic  simulation  of  kinetic  energy  exchange  between  gaseous  atoms  during  collisions — in 
other  words,  heat  transfer  through  direct  mixing  of  atoms.  Their  written  responses  regarding  these  activities 
indicated  that  most  found  both  the  computer  and  kinesthetic  experiences  helpful  in  understanding  the  concept  and 
that  doing  both  of  them  was  better  than  doing  one  or  the  other. 

For  more  details  about  the  project,  including  the  software,  images  and  descriptions  of  activities,  scripted 
hypermodels,  and  curricula,  see:  http://www.concord.org/workbench/download.html 

Boris  Berenfeld,  co-PI:  Dr.  Berenfeld  oversees  the  curriculum  design  and  development.  With  a Ph.D.  in  radiation 
biophysics,  he  combines  his  interest  in  frontier  science,  curriculum  design  and  networking  technologies  to  advance 
the  quality  and  accessibility  of  science  education.  His  current  research  involves  innovative  approaches  to  teaching 
biochemistry  through  visualization.  He  has  led  innovative  educational  initiatives  in  Russia,  Europe,  Latin  America, 
United  States  and  in  the  Middle  East,  organized  several  East-West  Conferences  on  New  Technologies  in  Education, 
and  served  on  the  faculty  of  the  Salzburg  Seminar. 

Qian  Xie  at  the  Concord  Consortium  is  the  primary  scientist  on  the  project,  responsible  for  developing  the 
algorithms  and  designing  the  Molecular  Workbench  code.  Dr.  Xie  is  a computational  scientist  with  a PhD  in 
Materials  Physics  from  University  of  Science  and  Technology,  Beijing,  and  post-doctoral  experience  at  the  Max 
Plank  Institute  in  Dresden  and  at  the  University  of  Cyprus.  He  has  developed  the  Molecular  Workbench  core 
software. 

Dan  Damelin  teaches  Chemistry  and  coordinates  the  computer  lab  at  Lincoln- Sudbury  High  School.  With  a MAT 
from  Tufts  in  Chemistry,  Biology  and  General  Science,  and  a triple  major  BA  from  Tufts  that  included  computer 
science,  he  has  been  able  both  to  develop  and  to  test  Molecular  Workbench  activitie.s. 
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Abstract:  This  paper  presents  a case  study  in  adapting  an  existing  educational  pen-based 
groupware  system  for  use  in  a distance-learning  environment.  After  reviewing  the  key 
characteristics  of  the  original  educational  groupware  system,  we  describe  our  motivation  for 
adapting  the  system  for  use  at  a distance.  Next  we  describe  the  process  that  was  used  to  cany 
out  this  adaptation,  the  problems  that  were  encountered,  and  the  resulting  solutions  that  were 
developed.  The  paper  concludes  with  the  results  of  a user  study  demonstrating  that  the 
modified  system  can  be  successfully  used  at  a distance. 


Background  and  Introduction 


Software  that  supports  cooperative  work,  often  called  groupware,  is  commonly  classified  as  belonging  to  one  of 
the  four  quadrants  of  the  time -place  matrix  shown  in  the  following  table  (Ellis  et  al.  1991). 


Same  Time  Different  Ti  mes 

Same  Place 
Different  Places 


Table  1.*  Groupware  Classification 

For  example,  on-line  chat  systems  belong  to  quadrant  III  of  the  matrix,  while  electronic  mail  systems  belong  to 
quadrant  IV.  The  interested  reader  is  referred  to  (Baecker,  1993)  for  an  excellent  overview  of  groupware 
systems  including  some  examples  of  educational  groupware  that  fall  into  each  of  the  time -place  categories 
noted  above.  The  goal  of  the  current  paper  is  to  describe  a case  study  in  “changing  places”;  specifically  we 
describe  our  work  in  adapting  an  existing  pen-based  educational  groupware  system  from  the  “same  time,  same 
place”  paradigm  to  the  “same  time,  different  places”  paradigm. 

In  the  remainder  of  this  introduction  we  describe  the  DePauw  Electronic  Black  Board  for  Interactive  Education 
(DEBBIE),  a groupware  system  originally  developed  by  the  first  author  for  enhancing  the  way  students  and 
teachers  share  written  information  during  class  (the  DEBBIE  system  is  patent  pending).  In  subsequent  sections 
we  describe  the  process  of  adapting  DEBBIE  for  use  in  a distance  environment,  the  obstacles  we  encountered 
and  our  progress  in  overcoming  them,  and  an  evaluation  of  the  resulting  system. 

DEBBIE  was  originally  designed  as  a same  time,  same  place  groupware  system  and  has  been  in  regular  use 
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since  the  spring  semester  of  2000.  Courses  that  use  the  DEBBIE  system  meet  in  DePauw’s  electronic 
classroom,  which  houses  one  instructor  station  and  sixteen  student  stations  (Fig.  1).  Each  of  these  seventeen 
stations  is  equipped  with  a Pentium  PC  that  uses  a high-resolution  flat-panel  pen-based  video  tablet  as  its 
display  device  (see  WACOM  2001).  Our  electronic  classroom  is  distinguished  from  others  that  are  described  in 
the  literature  (see  for  example  Shneiderman  et  al.  1995)  by  the  fact  that  users  can  control  the  computers  using  a 
special  electronic  stylus  in  addition  to  using  a keyboard  and  mouse.  For  example,  when  running  a standard 
“paint  program,”  a user  can  use  the  electronic  stylus  to  make  a sketch  by  drawing  freehand  directly  on  the 
surface  of  the  video  tablet.  Allowing  this  type  of  freehand  input  is  extremely  useful  when  teaching  topics  that 
are  hard  to  draw  using  only  a keyboard  and  mouse.  Examples  include  diagrams  of  molecules  in  a chemistry 
class,  graphs  in  an  economics  class,  complex  formulae  in  a mathematics  class,  musical  notation  in  a music 
theory  class,  and  finite  state  machines  (Fig  1.)  in  a computer  science  class. 

The  teacher’s  station  is  attached  to  a large  rear-projection  electronic  whiteboard  (see  SMART  2001)  located  at 
the  front  of  the  room.  The  image  from  the  teacher-station’s  video  tablet  is  also  displayed  on  the  electronic 
whiteboard.  Since  the  electronic  whiteboard  is  touch-sensitive,  the  teacher  can  use  a finger  (or  just  about 
anything  else)  to  draw  directly  on  the  surface  of  the  board  as  an  alternative  to  writing  directly  on  the  video 
tablet.  The  display  on  the  electronic  whiteboard  is  approximately  five  feet  wide  by  four  feet  high  and  requires 
approximately  four  and  one-half  feet  of  space  behind  it  for  the  projection  system.  In  our  classroom  the 
projection  system  actually  sits  behind  the  classroom’s  front  wall  so  that  the  display  surface  of  the  electronic 
whiteboard  is  flush  with  the  standard  (non-electronic)  whiteboards  that  flank  it  on  each  side. 


Figure  1 ; Electronic  Classroom 


DEBBIE  allows  the  students  and  teacher  in  a pen-based  electronic  classroom  to  share  written  information 
during  class.  For  example,  when  using  the  system,  the  teacher  can  extemporaneously  draw  freehand  sketches 
directly  on  the  surface  of  the  teacher-station’s  video-tablet  or  electronic  whiteboard.  The  teacher  can  also  use  a 
keyboard  to  type  material,  and  can  import  material  that  was  prepared  ahead  of  time  for  use  during  class.  All 
information  sketched,  typed,  or  imported  by  the  teacher  is  transmitted  over  a network  so  that  it  appears  on  each 
student’s  video  tablet. 

Each  student  can  write  freehand  on  his  or  her  display  to  make  private  annotations  to  the  teacher’s  material. 
During  class  the  teacher  can  import  portions  of  a student's  workspace  for  viewing  and  discussion  by  the  entire 
class.  Because  of  this,  class  sessions  tend  to  unfold  as  highly  interactive  activities.  In  a typical  scenario  the 
teacher  first  presents  some  new  material  to  the  class  and  then  asks  the  students  to  sketch  answers  to  problems 
that  are  related  to  this  material.  The  teacher  then  uses  the  system  to  share  some  or  all  of  the  student's  answers 
with  the  class,  responds  to  questions  about  these  answers,  offer  alternative  solutions,  and  determines  if  the  class 
is  ready  for  new  material  in  which  case  the  cycle  repeats. 

In  order  to  accommodate  an  entire  class  session,  each  user's  workspace  is  indefinitely  scrollable  in  the  vertical 
direction  like  a word  processor,  and  each  user  can  scroll  through  the  workspace  independently  of  the  other 
users.  At  the  end  of  the  class  period  each  user's  workspace  can  be  printed  or  saved  for  later  retrieval.  Portions 
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of  the  workspace  can  optionally  be  “replayed”  in  a stroke -by-stroke  fashion  that,  for  example,  allows  a student 
to  review  how  a complex  diagram  evolved.  The  reader  who  is  interested  in  more  information  about  the  software 
itself  is  referred  to  (Berque  et  al.  2000)  for  a more  complete  description  ofthe  system’s  user  interface. 

The  first  author  has  taught  four  upper-level  college  classes  that  were  delivered  entirely  using  DEBBIE,  and 
other  faculty  members  have  used  the  system  to  conduct  individual  class  sessions  in  several  other  disciplines 
including  mathematics  and  history.  Attitude  surveys  demonstrate  that  students  believe  they  are  more  attentive 
when  using  the  DEBBIE  system  and  that  they  prefer  this  environment  to  a traditional  classroom.  While 
admittedly  anecdotal,  comments  from  anonymous  student  course  evaluation  forms  also  indicate  student 
satisfaction  with  the  system  (Fig.  2).  The  interested  reader  is  referred  to  (Berque  et  al.  2001)  for  a more  detailed 
and  quantitative  description  of  students'  reactions  to  the  system. 


This  level  of  interaction  in  this  class  is  more  than  I have  experienced  in  any  other  class.  There  is  a lot 
of  interaction  between  the  teacher  (you)  and  us  that  has  been  very  valuable.  I don  7 think  this  could 
have  been  achieved  outside  ofthe  DEBBIE  classroom. 


Figure  2 

The  Adaptation  Process:  Motivation,  Goais,  and  Methods 

Our  motivation  for  adapting  the  DEBBIE  system  for  use  at  a distance  stems  from  several  sources.  First, 
instructors  at  other  schools  have  expressed  interest  in  using  the  DEBBIE  system  to  support  traditional 
synchronous  distance  education  classes  where  some  or  all  of  the  students  may  be  taking  classes  from  a large 
number  of  remote  locations.  We  also  envision  more  limited  scenarios  in  which  the  students  and  faculty  may  be 
located  at  only  a small  number  of  locations.  For  example,  the  Computer  Science  Department  at  the  authors' 
home  institution  does  not  currently  offer  a course  in  “Analysis  of  Algorithms”  although  such  a course  is  offered 
by  the  Computer  Science  Department  at  a neighboring  college  located  approximately  thirty  miles  away. 
Conversely,  the  neighboring  college  does  not  offer  a course  in  “Human  Computer  Interaction”  although  such  a 
course  is  offered  at  the  authors'  home  institution.  It  is  easy  to  see  that  an  agreement  enabling  students  at  each 
campus  to  take  classes  from  the  other  school  remotely  would  be  mutually  beneficial.  Each  school  could  provide 
a distance-learning  classroom  with  network  and  audio  connections  to  the  corresponding  room  at  the  other 
school.  In  this  case  all  of  the  students  would  attend  classes  in  one  ofthe  two  classrooms. 

As  we  began  to  consider  what  would  be  required  to  adapt  DEBBIE  for  use  in  a distance  education  environment 
we  decided  to  operate  within  a framework  formed  from  several  underlying  assumptions.  First,  we  assume  that 
DEBBIE  classes  will  be  delivered  synchronously  since  DEBBIE's  feature  set  is  built  around  the  notion  of  same  - 
time interaction.  In  addition  we  assume  that  DEBBIE  will  be  used  to  transmit  data  between  the  teacher  and 
students,  while  an  external  system  such  as  a telephone-based  or  internet-based  conference  call  will  be  used  to 
provide  a bi-directional  audio  link.  Since  reliable  and  inexpensive  audio  links  are  readily  available,  we  see  no 
reason  to  try  to  replicate  this  functionality  within  the  DEBBIE  system.  Further,  since  fast,  reliable,  inexpensive 
video  links  are  less  common,  we  assume  that  none  will  be  available  for  use  during  a DEBBIE  class.  Thus  we 
need  to  ensure  that  users  can  interact  using  DEBBIE  even  if  they  cannot  see  each  other.  Finally,  and  most 
importantly,  we  strive  to  provide  the  same  feature  set  for  remote  users  that  we  have  been  providing  to  local 
users  of  the  DEBBIE  system  for  the  past  several  years.  For  example,  since  students  who  are  in  the  same  room 
as  the  teacher  can  use  DEBBIE  to  share  their  work  with  the  rest  of  the  class,  we  insist  that  remote  users  have 
access  to  the  same  functionality. 


Major  Problems  Identified  and  Resolved 

Working  within  the  framework  outlined  in  the  previous  section,  we  developed  a list  of  problems  that  would 
interfere  with  the  use  of  the  original  version  of  DEBBIE  in  a distance  environment.  These  problems  were  then 
categorized  as  “critical”,  “high  priority”,  or  “low  priority.”  Problems  were  assigned  a critical  rating  if  DEBBIE 
simply  could  not  be  used  in  a distance  environment  until  this  problem  was  resolved.  An  assignment  of  “high 
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priority”  indicated  that  a solution  to  the  problem  would  significantly  enhance  the  experience  of  remote  users, 
while  the  “low  priority”  category  was  used  for  problems  whose  solution  would  have  less  of  an  impact.  This 
problem  categorization  was  subsequently  used  as  a guide  during  the  development  phase  of  the  project  as  we 
designed,  implemented,  and  tested  solutions  to  the  problems  starting  with  the  critical  ones.  Approximately  ten 
problems  were  addressed  during  the  process  of  augmenting  the  DEBBIE  system  for  use  in  a distance 
environment.  Although  a detailed  description  of  every  problem  is  beyond  the  scope  of  this  paper,  we 
summarize  several  of  the  problems  and  their  associated  solutions  below  (Tab.  2).  The  problems  were  selected 
to  give  the  reader  a feeling  for  the  range  of  issues  that  were  addressed. 


Descriptions  of  Example  Problems Associated  Solutions 


Problem  1.  Students  at  a remote  location  cannot  see 
the  electronic  whiteboard  by  glancing  at  the  front  of  the 
room  the  way  local  students  can.  Therefore,  if  the 
teacher  points  to  an  item  on  the  board  and  says  “Look 
how  this  works...”  the  remote  students  will  have  no 
way  to  know  what  the  teacher  is  pointing  to. 

Solution  1.  The  augmented  version  of  DEBBIE 
includes  a new  telepointer  tool.  Selecting  this  tool 
causes  a pointer  icon  to  track  the  teacher's  hand 
movements  on  the  electronic  whiteboard  or  video 
tablet.  Since  this  icon  is  also  displayed  at  the 
corresponding  location  on  each  student's  display,  the 
students  can  easily  determine  what  the  teacher  is 
pointing  to. 

Problem  2.  Since  each  user’s  scroll  can  be  positioned 
independently  of  the  other  users,  students  at  a remote 
location  will  experience  difficulty  each  time  the 
teacher’s  scroll  is  moved  to  a new  position.  Local 
students  can  glance  at  the  electronic  whiteboard  at  the 
front  of  the  room,  notice  that  the  teacher's  scroll  has 
been  repositioned,  and  then  reposition  their  scrolls 
appropriately.  However,  remote  students  have  no  way 
to  get  this  visual  feedback. 

Solution  2.  We  added  a toolbar  button  to  the  student 
version  of  DEBBIE  that  displays  one  of  two  images  to 
signify  whether  or  not  the  student’s  scroll  is  currently 
synchronized  with  the  teacher's  scroll.  If  the  student’s 
scroll  is  not  synchronized,  the  student  can  click  on  this 
button,  which  causes  the  scroll  to  be  adjusted 
appropriately.  We  also  added  an  auto  scroll  feature 
which,  when  enabled,  causes  a student’s  scroll  to  adjust 
automatically  as  the  teacher’s  scroll  position  changes. 

Problem  3.  When  working  with  local  students,  the 
teacher  can  simply  look  into  the  classroom  to  see  who 
is  present.  This  can  help  the  teacher  to  determine  when 
enough  students  have  arrived  to  start  class,  who  to 
mark  absent  for  a given  day,  and  who  to  call  on  to 
answer  a question.  However,  when  dealing  with 
remote  students  the  teacher  has  no  easy  way  to  sense 
who  is  participating  in  the  current  class  session. 

Solution  3.  The  augmented  version  of  DEBBIE 
requires  each  student  to  log  into  a DEBBIE  session 
with  a username.  The  teacher  can  then  display  a list  of 
the  currently  connected  users,  and  can  use  this  list  to 
import  portions  of  selected  student’s  notebooks. 
Students  who  join  the  class  late  will  be  added  to  the  list 
as  soon  as  they  are  connected  and  they  will 
automatically  receive  the  content  transmitted  by  the 
teacher  earlier  in  the  class. 

Problem  4.  The  teacher  can  transmit  hypertext  links 
(to  spreadsheets  files,  movie  clip  files,  etc.)  to  the 
students  by  inserting  the  links  into  the  electronic 
notebook.  If  the  linked  files  are  located  in  a shared 
directory  on  a local  area  network,  each  local  student 
will  then  be  able  to  open  the  file.  Since  remote 
students  will  generally  not  have  access  to  the  local  area 
network,  they  will  not  be  able  to  open  these  files. 

Solution  4.  In  the  augmented  version  of  DEBBIE  the 
teacher  can  transmit  a hypertext  link  that  refers  to  a 
URL  for  the  external  file.  The  first  time  the  student 
clicks  on  this  link  an  FTP  client  is  automatically 
launched  to  download  the  specified  file  to  the  student’s 
PC.  To  handle  subsequent  references  to  this  file  by  the 
student,  DEBBIE  simply  opens  this  local  copy  of  the 
file. 

Table  2:  Example  Problems  and  their  Solutions 


Evaluation  and  Results 

After  receiving  approval  from  the  Institutional  Review  Board  (Human  Subject  Review  Board),  we  invited  seven 
Computer  Science  majors  to  participate  in  a usability  test  to  determine  whether  or  not  the  augmented  version  of 
DEBBIE  could  be  used  in  a distance  education  environment.  All  seven  students  agreed  to  participate  in  return 
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for  a gift  certificate  to  a local  restaurant,  and  they  met  as  a group  with  the  authors  in  our  electronic  classroom  in 
order  to  begin.  After  signing  an  informed  consent  form,  each  student  was  given  a pre-test  consisting  of  six 
questions  on  the  topic  of  “Regular  Expressions.”  This  topic  was  selected  because:  (a)  we  believed  most 
students  would  have  little  prior  experience  with  this  area  of  Computer  Science,  (b)  it  is  a topic  that  the  first 
author  regularly  teaches  in  one  of  his  advanced  classes  and  therefore  he  had  a good  sense  of  how  to  present  the 
material,  and  (c)  the  topic  involves  complex  notation  that  is  best  explained  using  free-hand  sketches. 

After  the  pre-tests  were  collected,  the  students  were  given  a five-minute  orientation  to  the  DEBBIE  system. 
During  this  orientation  the  students  learned  how  to  create  a new  electronic  notebook,  connect  to  a network 
session,  draw  and  erase  using  the  electronic  stylus,  and  navigate  through  the  notebook.  At  the  conclusion  of  the 
orientation  session  the  first  author  went  to  a nearby  classroom  that  had  been  temporarily  equipped  with  a PC 
running  the  teacher's  version  of  DEBBIE.  A bi-directional  audio  link,  external  to  the  DEBBIE  system,  had 
previously  been  set  up  between  the  teacher's  classroom  and  the  student's  classroom.  When  the  teacher  arrived 
at  his  classroom,  he  used  the  audio  link  to  tell  the  students  to  connect  to  the  DEBBIE  session.  The  teacher  then 
used  the  DEBBIE  system  to  monitor  the  students  as  they  connected  to  the  session. 

When  the  teacher  saw  that  all  of  the  students  had  connected,  he  began  to  teach  a thirty-minute  class  on  the  topic 
of  Regular  Expressions.  The  class  started  as  a lecture  with  the  teacher  presenting  new  material  by  transmitting 
extemporaneously  drawn  free-hand  sketches  using  DEBBIE  while  simultaneously  offering  an  oral  explanation. 
During  this  time  the  teacher  frequently  used  the  telepointer  to  focus  the  student's  attention  on  specific  content 
matter,  and  the  students  used  the  auto  synchronization  feature  to  keep  their  scrolls  synchronized  with  the 
moderator.  The  students  were  also  free  to  use  their  electronic  pens  to  make  personal  annotations  alongside  the 
instructor's  material.  Next  the  teacher  had  the  students  work  through  several  interactive  exercises  that  required 
the  students  to  sketch  answers  to  problems  using  their  electronic  pens.  At  the  end  of  each  exercise  the  teacher 
collected  the  student's  answers  electronically  and  quickly  read  them  over  in  order  to  gauge  how  the  class  was 
doing.  At  several  points  the  teacher  chose  to  share  one  or  more  student  solutions  with  the  entire  class  so  that 
alternative  approaches  could  be  compared.  While  explaining  the  correct  answer  to  one  of  these  exercises  the 
teacher  intentionally  introduced  a mistake  into  his  explanation.  In  response  the  student  participants  quickly 
interrupted  the  teacher  by  calling  out  a correction  thus  providing  additional  evidence  that  the  students  were 
actively  following  the  session. 

At  the  conclusion  of  the  class  the  students  were  given  a post-test  consisting  of  exactly  the  same  questions  that 
comprised  the  pre-test.  A faculty  colleague  in  the  Computer  Science  Department  who  was  not  associated  with 
this  study  graded  the  pre-tests  and  post-tests.  The  tests  were  provided  to  the  faculty  member  in  coded  format  so 
that  he  could  not  tell  which  were  pre  -tests  and  which  were  post-tests.  In  addition,  the  student's  names  were  not 
revealed  to  the  faculty  grader.  The  results  (Tab.  3)  indicate  a dramatic  improvement  on  the  post-test  scores  as 
compared  to  the  pre-test  scores. 


Pre-Test  Post-Test 

Low  Score 
Mean  Score 
High  Score 

Table  3 


0% 

67% 

27% 

91% 

73% 

100% 

After  completing  the  post-test,  the  students  answered  some  questions  that  were  designed  to  measure  how  well 
they  perceived  they  had  learned  the  material.  These  questions  were  answered  using  a semantic  differential  scale 
with  endpoints  defined  with  1 meaning  very  comfortable  and  5 meaning  not  at  all  acquainted.  When  the 
students  were  asked  to  use  this  scale  to  rate  the  statement  "Prior  to  participating  in  the  session  today  how 
comfortable  were  you  with  the  concept  of  a Regular  Expression?  ” the  mean  response  was  4.4.  When  the 
students  were  asked  to  rate  the  statement  ''After  participating  in  the  session  today  how  comfortable  were  you 
with  the  concept  of  a Regular  Expression?  ” the  mean  response  was  1 .6. 
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The  user  test  described  in  the  previous  section  was  carried  out  using  computers  located  in  close  proximity  to 
each  other  and  connected  to  the  Internet  via  a relatively  fast  10 -megabit  local  area  network.  In  order  to 
determine  the  responsiveness  of  the  system  when  used  in  a variety  of  different  configurations,  we  carried  out 
several  additional  tests.  Specifically,  we  measured  the  time  required  to  transfer  a section  of  an  electronic 
notebook  consisting  of  a question  posed  by  the  teacher  along  with  the  student's  associated  answer  between  two 
computers  under  a variety  of  conditions.  We  repeated  the  test  several  times,  each  time  using  a section  of  an 
electronic  notebook  that  had  been  saved  during  a real  class  session.  When  DEBBIE  was  used  to  transfer  these 
sections  between  two  computers  that  were  both  located  at  the  author's  own  University,  the  transfer  times  ranged 
from  229  milliseconds  to  1,463  milliseconds.  Next  we  established  a DEBBIE  session  between  a computer  at 
the  authors’  home  institution  (located  in  the  USA)  and  a computer  located  at  a University  in  England  that  was 
also  connected  to  the  Internet  via  a 10-megabit  local  area  network.  The  transfer  times  remained  similar  in  this 
scenario,  this  time  ranging  from  492  milliseconds  to  1,483  milliseconds.  Finally,  we  established  a DEBBIE 
session  between  a computer  at  the  authors'  home  University  and  a computer  using  a 56K  modem  to  connect  to 
the  Internet  from  a local  home  via  a community-based  Internet  Service  Provider.  The  transfer  times  were 
somewhat  slower  in  this  scenario,  ranging  from  1,379  milliseconds  to  6,386  milliseconds;  however,  these  times 
demonstrate  that  the  system  can  perform  one  of  its  most  communication  intensive  tasks  interactively  even  under 
sub-optimal  network  conditions. 


Conclusions 

The  user  test  described  in  the  previous  section  has  demonstrated  that  the  augmented  version  of  DEBBIE  can  be 
used  to  support  classes  delivered  at  a distance,  even  when  those  classes  use  a mixture  of  pedagogical 
approaches  ranging  from  a lecture -based  approach  to  an  approach  that  incorporates  more  student  involvement. 
The  responses  to  the  survey  questions  presented  in  the  previous  section  show  that  the  students  perceive  that  they 
are  learning  while  using  DEBBIE  in  a distance  environment,  and  more  importantly  the  results  of  the  pre-test 
and  post-test  provide  evidence  that  the  students  really  did  learn  the  material.  Finally,  the  networks  tests 
demonstrate  that  DEBBIE  continues  to  perform  interactively  even  under  sub-optimal  network  conditions. 
Collectively  these  results  encourage  us  to  try  to  use  DEBBIE  to  deliver  an  entire  distance  education  course. 
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Prototyping  the  Development  of  Groupware  to  Help  Low-Vision  Students 
View  an  Instructor's  Blackboard  Notes 


Ben  Betz,  Computer  Science  Department,  DePauw  University,  USA,  bbetz@depauw.edu 


Abstract:  The  current  lack  of  satisfactory  technology  to  aid  low  vision  students  drives  the 
development  of  a handheld  Windows  CE-based  viewer  for  Visually  Impaired  Students  (v- 
VIS)  designed  to  assist  in  the  realtime  visualization  of  professor-written  blackboard  material. 
Instead  of  writing  on  a normal  blackboard,  the  instructor  writes  on  an  electronic  whiteboard. 
Students  with  normal  vision  view  this  material  in  the  usual  way,  namely  by  looking  to  the 
front  of  the  room.  Simultaneously,  the  v-VIS  system  transmits  the  whiteboard  contents,  in 
realtime,  to  the  low-vision  student’s  handheld  computer  where  it  can  be  magnified  and  viewed 
at  close  range.  After  describing  the  v-VIS  system,  we  discuss  some  of  the  problems  we 
encountered  during  its  development  and  our  methods  of  overcoming  these  problems.  Results 
of  preliminary  user  testing  are  also  presented. 


Introduction 

Classrooms  present  inherent  problems  for  low-vision  students,  including  difficulties  reading  written 
information  on  blackboards.  In  traditional  classrooms,  a significant  portion  of  the  lecture  or  discussion  depends 
on  the  student’s  ability  to  refer  to  this  visual  information.  Thus,  students  with  reduced  vision  often  employ 
human  note-taking  services  to  transcribe  blackboard  writing  for  later  review.  While  utilizing  such  a service 
provides  the  student  with  a copy  of  the  professor’s  writing,  it  is  not  immediate  and  it  is  subject  to  human 
misinterpretation  of  the  blackboard  information.  Our  current  research  has  centered  on  designing  a networked 
computer  system  called  v-VIS  (viewer  for  Visually  impaired  Students).  The  prototype  is  designed  to  alleviate 
note-taking  difficulties  by  providing  immediate  access  to  blackboard  writing  for  low-vision  students.  Thus  v- 
VIS  is  an  example  of  assistive  technology,  the  use  of  technology  to  aid  disabled  persons,  which  as  (Perkins, 
2001)  states,  may  be  the  best  method  of  providing  immediate  access  to  class  information. 

Interestingly,  while  significant  prior  research  has  developed  assistive  technology  for  completely  blind  users, 
little  effort  has  been  made  toward  helping  the  much  larger  population  of  low-vision  users.  Low-vision  persons, 
who  can  make  functional  use  of  some  portion  of  their  vision  with  appropriate  aids,  likely  outnumber  fully  blind 
persons  three  to  one  (Jacko,  1998).  Our  work  on  v-VIS  serves  this  audience. 

In  the  simplest  sense,  v-VIS  collects  the  information  written  on  an  electronic  whiteboard  (instead  of  the 
traditional  blackboard)  and  transmits  it  to  a handheld  viewer  near  the  student  (see  Fig.  1 and  2).  This  viewer 
not  only  displays  a copy  of  the  professor’s  writing  in  real-time,  but  also  enhances  it  through  magnification  and 
color  pallette  manipulation,  which  according  to  (Kline,  1995)  are  essential  features  for  low-vision  GUI 
accessability.  The  v-VIS  prototype  uses  a recent  advent  in  mobile  computing  technology,  the  Handheld  PC,  for 
the  viewer  hardware,  which  classifies  v-VIS  as  mobile  assistive  technology.  This  allows  for  increased 
portability  of  the  system  as  well  as  several  advantages  that  will  be  described  later. 


Prototype  Development 

The  current  v-VIS  prototype  has  been  developed  for  a Handheld  PC  running  Windows  CE  2000.  While  a 
Handheld  PC  has  a relatively  small  screen  (approx.  2"  x 5"),  we  feel  that  a handheld  will  provide  several 
advantages  over  a desktop  system.  One  benefit  is  that  the  low  weight  of  a handheld  allows  it  to  be  lifted  to  the 
user’s  eyes,  greatly  reducing  the  neck  strain  that  many  low-vision  computer  users  experience  when  leaning  to  as 
close  as  one  inch  from  the  screen  (Vener,  1988).  Handheld  PCs  are  also  portable  devices,  which  allows  them  to 
be  used  in  multiple  locations,  especially  when  configured  for  wireless  networking.  Additionally,  the  hardware 
chosen  for  v-VlS  is  one-fourth  the  cost  of  the  desktop  system  that  would  be  required  to  achieve  similar 
functionality. 
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Figure  1:  The  v-VIS  system 
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Figure  2:  The  v-VIS  viewer  (handheld)  main  screen 


The  v-VIS  Interface 

The  foundation  of  the  v-VIS  interface  is  the  draw  space;  the  main  screen  of  v-VIS  is  dominated  by  a large, 
blank  panel  (see  Fig.  2).  When  the  professor  wishes  to  write  a particular  item  for  the  class  to  see,  he  moves  his 
finger  (or  stylus)  on  the  large  touch-sensitive  rear-projection  whiteboard  screen  (approx.  4'  x 5')  and  writes  just 
as  he  would  on  a marker  board  or  chalkboard.  A desktop  computer  attached  to  the  whiteboard  runs  the 
professor  or  “moderator”  version  of  the  v-VIS  software  and  interprets  each  motion,  laying  digital  “ink”  onto  the 
draw  space.  Students  with  normal  vision  view  the  professor’s  drawing  by  looking  at  the  large  image  on  the 
whiteboard.  v-VIS  also  transmits  the  coordinates  of  each  ink  “droplet”  in  real-time  over  a high  speed  network 
to  the  viewer,  which  then  recreates  the  image  on  its  screen  for  the  low-vision  student,  after  adjusting  for  his  or 
her  magnification  and  color  preferences. 

As  each  screenful  of  space  is  used  up,  a new  panel  or  “page”  of  drawing  space  is  automatically  made  available. 
In  fact,  the  v-VIS  panels  are  infinitely  scrollable  in  the  vertical  direction,  as  if  each  new  page  was  taped  to  the 
bottom  of  its  predecessor.  Due  to  the  smaller  handheld  screen  of  v-VIS,  each  panel  is  also  scrollable  in  the 
horizontal  direction,  to  allow  the  full  width  of  the  whiteboard  draw  space  to  be  viewable  on  the  handheld.  In 
this  way,  the  handheld  becomes  a viewing  “window”  into  the  selected  page  of  the  whiteboard  draw  space. 

Beyond  the  basic  interface  specifications,  v-VIS  is  designed  to  operate  in  a flexible  manner.  The  goal  has  been 
to  make  the  system  as  customizable  as  possible,  so  that  it  may  best  serve  the  needs  of  students  with  many  vision 
impairments.  According  to  (Jacko,  1998),  assistive  devices  must  be  fully  configurable  in  order  to  serve  the 
large  range  of  vision  profiles  that  exist.  Thus,  the  v-VIS  viewer  prototype  has  multiple  control  methods  aid 
two  independently  configurable  image  enhancements  - magnification  and  color  customization. 


Interface  Control 

Control  of  the  handheld  viewer  is  accomplished  in  many  ways.  As  with  most  current  software  packages,  a 
menu-toolbar  set  is  accessible  at  the  top  of  the  screen.  This  may  not  be  functionally  visible  to  low-vision  users, 
however,  so  every  command  has  a keyboard  shortcut.  Scrolling,  for  example,  is  accomplished  in  three  different 
ways:  By  using  the  stylus  to  control  the  scrollbar,  by  using  the  arrow  keys,  and  by  selecting  a “grabber  tool” 
that  allows  the  user  to  place  a stylus  on  the  draw  space  and  drag  it  to  its  desired  position,  much  like  moving  a 
piece  of  paper  to  a different  location  on  a desk. 


Magnification 

Magnification  is  one  of  the  most  important  visual  enhancements  for  a low-vision  person  (Fraser,  2000). 
Making  the  drawn  objects  larger  allows  students  with  low  visual  acuity  to  see  the  objects  more  clearly, 
effectively  capitalizing  on  their  available  vision  (Jacko,  1998).  v-VIS  takes  the  mathematical  description  of 
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each  drawn  object  and  scales  it  by  a factor  ranging  from  100%  to  1000%,  which  is  set  by  the  user  according  to 
his  or  her  individual  needs.  Not  only  do  the  dimensions  of  the  objects  increase,  but  so  does  the  thickness  of 
their  defining  lines,  making  the  digital  ink  appear  as  if  it  were  being  examined  under  a powerful,  distortionless 
magnifying  glass  (see  Fig.  3). 


Figure  3:  Effects  of  magnification  in  v-VIS 


Color  Customization 

As  important  as  magnification  is  to  the  use  of  v-VIS,  many  persons  with  reduced  vision  cannot  see  even 
magnified  images  unless  they  are  presented  in  specific  colors  or  contrast  schemes  (Kline,  1995).  For  this 
reason,  the  draw  space  of  v-VIS  can  be  set  to  use  any  combination  of  colors.  Both  the  background  and  four 
distinct  drawing  colors  can  be  chosen  from  any  RGB  value  based  on  the  user’s  specific  needs. 


User  Response 

Two  low-vision  persons  have  informally  evaluated  and  commented  on  the  v-VIS  prototype  during  its 
development.  One  of  these  evaluators  was  pleased  with  the  magnification  features  of  an  early  version,  but 
made  several  suggestions  for  improving  the  color  customization  option.  The  second  evaluator  also  found  the 
magnification  settings  helpful.  This  person  did,  however,  mention  a preference  for  a larger  monitor,  since  it  can 
display  the  same  amount  of  draw  space  as  the  whiteboard.  Since  she  cannot  see  to  the  front  of  the  room,  she 
has  no  indication  of  where  on  the  large  screen  the  professor  is  writing  when  looking  at  the  image  on  the  smaller 
handheld  screen.  She  felt  a larger  monitor  would  help  her  locate  the  newest  writing  on  the  screen  with 
contextual  relation  to  previously  drawn  objects.  Interestingly,  this  evaluator  also  suggested  the  addition  of 
auditory  feedback  to  signal  the  arrival  of  new  information  to  the  viewer,  since  writing  on  the  whiteboard  does 
not  make  an  audible  sound  like  chalk  on  a blackboard  does.  Thus  the  evaluator  was  concerned  that  she  would 
have  trouble  knowing  that  new  drawings  had  arrived  at  the  viewer  and  discerning  where  to  look  for  them. 


The  Object-In-View  Problem 


As  mentioned  by  one  of  the  evaluators,  the  handheld’s  limited  screen  size  makes  it  difficult  to  know  where  the 
professor  is  writing.  The  large  whiteboard  and  the  smaller  handheld  screen  display  corresponding  images,  but 
the  complete  whiteboard  image  cannot  appear  on  the  handheld  at  any  given  time,  since  this  requires  reducing  its 
size.  v-VIS  aims  to  make  the  professor's  writing  more  visible  for  a low-vision  student,  not  to  shrink  it.  In  fact, 
it  is  likely  that  reduced-vision  students  will  have  their  view  magnified,  centering  on  an  even  smaller  section  of 
the  professor’s  writing.  In  our  prototype,  the  handheld  maintains  the  same  amount  of  draw  space  as  the  large 
forward  display,  but  only  a portion  of  it  is  visible  at  any  given  time.  The  student’s  view  thus  acts  as  a scrollable 
“window”  into  the  whiteboard  draw  space,  like  a camera  zoomed  in  on  one  particular  section  of  the  whiteboard. 
The  major  problem  for  low-vision  students  is  that  as  the  professor  draws  new  objects  on  the  whiteboard,  it  is 
not  likely  that  they  will  be  within  the  current  window’s  focus  on  the  handheld.  In  this  situation,  the  new  writing 
will  not  be  displayed  on  the  handheld  until  the  view  is  scrolled  to  its  location  on  the  draw  space.  We  call  this 
the  object-in -view  problem. 

This  problem  is  only  a minor  inconvenience  for  a user  with  normal  vision,  who  simply  has  to  look  at  the 
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whiteboard  and  reposition  the  viewing  window  on  the  handheld  to  the  area  where  the  professor  is  writing.  The 
users  of  v-VIS,  however,  will  likely  have  no  ability  to  see  the  whiteboard  or  the  professor.  Therefore,  not  only 
will  it  be  impossible  for  them  to  adjust  their  scroll,  but  they  will  have  no  indication  as  to  how  the  newest 
drawings  are  oriented  among  previous  writing.  Since  the  professor  will  likely  write  several  v-VIS  “pages”  of 
information  during  a class  session,  the  draw  space  can  be  expected  to  change  numerous  times.  A low-vision 
user  cannot  be  expected  to  follow  along  with  the  rest  of  the  class  without  an  effective,  automatic  method  of 
displaying  new  writing  on  the  handheld  viewer  as  the  draw  space  contents  change.  We  have  attempted  to 
develop  a solution  for  this  problem,  a feature  called  AutoScroll. 


AutoScroll  - A Possible  Solution 


The  solution  to  the  object-in-view  problem  that  has  been  developed  is  called  AutoScroll.  As  the  viewer  receives 
new  objects  drawn  by  the  professor,  it  repositions  its  scrollable  draw  space  to  place  the  newest  object  on  the 
screen.  This  is  similar  in  functionality  to  a visually  enhanced  text  editor,  MAGNEX,  which  maintains  the 
cursor,  and  thus  new  text  input,  near  the  center  of  the  screen  (Vener,  1988).  Initial  efforts  have  also  been  made 
in  v-VlS  to  ensure  that  previously  existing  objects  nearby  are  retained  on  the  screen. 

The  basic  principle  behind  AutoScroll  is  rather  simple:  If  a newly  received  object  is  not  completely  visible  on 
the  screen,  the  system  automatically  repositions  the  draw  space  so  that  it  is.  At  the  same  time,  AutoScroll 
attempts  to  retain  the  context  of  the  newest  object  by  keeping  other  recently  drawn  objects  on  the  screen.  This 
is  especially  important  to  low  vision  students  who  have  no  other  means  of  knowing  where  the  newest 
information  is  being  written.  By  keeping  nearby  and  often  related  objects  on  the  screen,  AutoScroll  provides 
contextual  information  to  the  v-VIS  user. 


AutoScroll  is  based  on  the  fact  that  each  object  drawn  on  the  v-VIS  screen  has  a bounding  box,  a rectangular  set 
of  boundaries  that  define  the  object’s  position  (see  Fig.  4).  This  box  allows  v-VIS  to  easily  determine  if  an 
object  is  visible  at  all  and  if  it  is,  the  portion  of  the  object  that  is  visible  on  screen.  The  algorithm  uses  a 
Tolerance  Frame  to  determine  if  a newly  received  object  is  completely  on  screen  (see  Fig.  5).  This  frame  is  a 
small  area  around  each  edge  of  the  viewing  area  outside  of  which  an  object  is  considered  to  be  out  of  view.  The 
Interior  Tolerance  Border  marks  the  beginning  of  the  frame  on  the  inside  of  the  viewing  area,  and  the  Exterior 


Tolerance  Border  marks  the  outside  edge  of  this  area. 


Figure  4:  An  object  with  its  bounding  box  crossing 
the  Interior  Tolerance  Border 
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Figure  5:  AutoScroll  borders 


When  new  objects  are  received,  their  bounding  box  location  is  compared  to  the  location  of  the  Interior  and 
Exterior  Tolerance  Borders  to  determine  what  portion  of  the  object  is  visible.  If  an  object’s  bounding  box  is 
completely  outside  of  both  borders,  the  draw  space  is  automatically  scrolled  so  that  the  object  is  just  inside  the 
left  and/or  upper  Interior  Tolerance  Borders;  this  action  is  called  a “jump”  (see  Fig.  6).  The  draw  space  is  only 
scrolled  in  the  direction  this  border  is  crossed,  so  the  Interior  Tolerance  Border  is  not  really  a rectangle,  but  a set 
of  four  boundaries  that  extend  across  the  entire  view.  AutoScroll  has  been  designed  in  this  manner  to  move 
new  letters  (drawn  objects)  that  are  far  from  the  current  location  of  the  viewing  window  to  the  left  of  the  screen, 
providing  space  for  the  fetters  the  professor  may  soon  write.  Preference  is  also  given  to  the  upper  half  of  an 
object  in  the  event  it  is  too  tall  to  fit  completely  on  screen,  since  the  upper  half  of  handwritten  letters  is  more 
recognizable  than  the  lower  half  (Lynch,  1999). 


189 


best  copy  available 


If  an  object  has  crossed  the  Interior  Tolerance  Border  but  some  part  of  its  bounding  box  still  resides  inside  the 
Exterior  Tolerance  Border,  AutoScroll  performs  a “slide”.  Here,  the  scroll  places  the  object  just  inside  the 
Interior  Tolerance  Border  that  was  crossed  (see  Fig.  7).  Slide  actions  are  designed  to  place  in  view  objects  that 
are  close  to  the  current  location  of  the  viewing  window  while  keeping  other  nearby  objects  on  screen  as  well. 
This  allows  the  completion  of  words  that  are  being  continued  to  the  right,  making  them  visible  without  pushing 
the  first  portion  of  the  word  off  the  left  edge  of  the  screen,  which  would  happen  if  a jump  action  occurred. 
Slides  then  function  to  group  letters  of  words  together  as  the  professor  writes  them  and  also  to  group  phrases  to 
relate  the  context  of  new  words. 


Figure  6:  A “jump”  repositioning  of  an  object 


Figure?:  A “slide”  repositioning  of  an  object 


AutoScroll  Performance 

The  performance  of  AutoScroll  was  evaluated  objectively  by  counting  handwritten  word  completion  as 
described  below.  Ten  pages  of  data  recorded  from  a class  taught  on  an  electronic  whiteboard  were  fed  to  the  v- 
VIS  viewer  stroke  -by-stroke.  Two  statistics  were  collected  at  multiple  zoom  factors: 

• How  many  times  the  most  recent  word  was  completely  on  the  screen  when  it  was  fully  received 

• How  many  complete  words  were  concurrently  on  the  screen  at  that  time 


Word  ComptGliofi  as  a Funrticn  of  Zoom  Factor 
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Figure  8:  Completeness  of  one  word  Figure  9:  Completeness  of  n or  more  words 

(Fig.  8)  shows  that  AutoScroll  is  effective  at  grouping  individual  letters  into  words.  Even  at  400% 
magnification,  80.86%  of  possible  words  were  completely  displayed  and  close  to  100%  were  complete  at  the 
lower  zoom  factors  tested.  Performance  decreases  at  higher  magnification  levels,  since  the  effective  draw  space 
is  reduced  and  a large  number  of  letters  cannot  be  physically  displayed  on  the  screen.  Similarly,  the  grouping 
of  multiple  words  on  screen  at  the  same  time  to  provide  context  is  rather  effective  at  lower  magnification  (see 
Fig.  9).  For  example,  at  100%  magnification,  4 or  more  words  were  completely  on  the  screen  a majority 
(67.1%)  of  the  time.  Again,  performance  decreased  at  higher  magnification  due  to  the  limited  screen  size. 
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Conclusions  and  Future  Work 


The  initial  reactions  from  the  two  evaluators  of  v-VIS  indicate  that  the  system  will  likely  be  helpful  in  a 
classroom  environment,  after  several  improvements  are  made.  The  vision  enhancements  seem  to  be  quite 
helpful,  although  the  screen  size  of  the  Handheld  PC  may  be  an  issue. 

Based  upon  the  performance  test,  AutoScroll  appears  to  be  a good  method  of  placing  the  most  recent  objects  on 
screen,  helping  to  overcome  the  object-in -view  problem.  The  results  of  the  test,  however,  suggest  that  while  the 
AutoScroll  algorithm  works  well,  its  performance  can  still  be  improved,  a fact  mentioned  by  one  of  the  low- 
vision  evaluators.  The  current  Tolerance  Frame,  for  example,  would  most  likely  result  in  better  performance  if 
it  were  scaled  in  size  based  on  the  magnification  level. 

For  future  work,  we  plan  to  continue  development  of  AutoScroll  algorithms  in  order  to  improve  the 
performance  of  this  feature.  The  v-VIS  interface  can  also  be  improved  by  including  the  addition  of  auditory 
feedback  to  announce  the  arrival  of  new  information,  indicating  the  location  of  the  new  objects  through  tonal 
differences.  We  also  plan  to  develop  v-VIS  for  a desktop  system,  so  its  usability  can  be  compared  with  the 
handheld.  After  continued  development,  we  plan  to  test  the  resulting  v-VIS  system  in  a classroom  with  low- 
vision  students  to  determine  its  usability  in  a more  formal  manner. 
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Introduction  of  Technology  Integration  through  Case  Based  Learning 

Abstract 

In  response  to  school-needs  and  state-requirements,  the  University  of 
Northern  Colorado  is  redesigning  the  two  requisite  educational  technology 
courses  for  the  undergraduate  teacher  preparation  programs.  The  first 
course  redesign  has  been  implemented;  the  program’s  one-credit  200-level 
course  is  currently  equipping  UNC  teacher  candidates  with  basic  computer 
and  software-tool  skills.  The  second  requisite  course — the  300-level — has 
been  redesigned  and  is  now  in  stages  of  development.  The  first  part  of  the 
course  will  instruct  teacher  candidates  in  the  concepts  of  technology 
integration.  Then,  to  make  those  concepts  more  meaningful  and 
transferable,  the  course  will  immerse  students  in  a case-based  learning 
unit.  This  proposal  describes  the  unit’s  problem  scenarios.  It  also  details 
the  electronic  environment  and  graphical  user  interface  in  which  the 
scenarios  and  the  information  necessary  for  problem  solving  are  presented. 

A study  by  Persichitte,  Tharp,  and  Caffarella  (1999)  found  that  while  technology  is  readily 
available  to  teacher  education  programs,  only  45%  of  the  programs’  faculty  regularly  use 
technology  during  class  and  only  40%  of  those  programs  require  preservice  teacher  candidates  to 
use  technology  to  design  and  deliver  instruction.  Those  statistics  exist  despite  current  standards 
for  technology-use  in  education. 

Recently  the  Colorado  legislature  passed  Senate  Bill  99-154,  which  included  eight 
standards  for  teacher  education  candidates  to  meet.  The  seventh  standard  states,  “The  teacher  is 
skilled  in  technology  and  is  knowledgeable  about  using  technology  to  support  instruction  and 
enhance  student  learning.”  However,  a recent  survey  of  the  University  of  Northern  Colorado’s 
(UNC’s)  partner  schools  (i.e.,  local  K-12  schools)  found  a high  need  for  successful 
implementation  of  that  standard.  In  fact,  many  schools  rated  teacher  technology  skills  as  their 
highest  need.  Clearly,  preservice  teacher  candidates  have  not  been  equipped  to  meet  the  demand 
for  trained  teachers  who  can  integrate  technology  throughout  their  instruction. 

Currently,  UNC’s  undergraduate  teacher  education  programs  require  preservice  teachers 
to  take  two  educational  technology  courses.  However,  each  course  is  worth  only  one  credit.  The 
preservice  teachers  generally  take  the  first  course  at  the  beginning  of  their  teacher  preparation 
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and  the  second  course  just  before  their  final  student  teaching  experiences.  The  courses’  curricula 
vary  for  elementary,  middle  grades,  and  secondary  preservice  teachers. 

In  2000,  UNC  was  awarded  a Preparing  Tomorrow’s  Teachers  To  use  Technology  (PT3). 
A major  initiative  of  the  grant  was  to  redesign  the  aforementioned  courses  so  that  UNC’s  School 
for  the  Study  of  Teaching  and  Teacher  Education  will  better  prepare  students  to  meet  new 
standards  for  using  technology  for  instruction.  The  first  year  of  the  grant,  the  university’s  PT3 
team  redesigned  the  200-level  courses  (i.e.,  the  first  educational  technology  course  students 
take).  The  PT3  team  crafted  those  courses  to  give  students  the  technology  tools  they  will  need 
for  superior  performance  throughout  their  university  careers,  for  successful  synthesis  of 
technology  integration  theories  with  other  learning  theory,  and,  ultimately,  for  effective 
classroom  teaching.  The  curricula  include  introductions  to  both  MAC  and  PC  operating  systems 
and  introduce  students  to  a variety  of  software,  including  Word,  Excel,  PowerPoint,  HyperStudio, 
Inspiration,  and  Dreamweaver. 

Because  the  200-level  courses  emphasize  tools,  the  300-level  courses  will  concentrate  on 
preparing  preservice  teachers  to  integrate  those  tools  and  other  technology  into  instruction.  The 
PT3  team  wants  the  300-level  courses  to  give  preservice  teachers  a picture  of  how  integration 
looks  in  K-12  classrooms  and  how  they  can  use  technology  integration  to  meet  state  standards 
and  solve  instructional  problems.  After  determining  the  course  objectives,  the  PT3  team  split 
into  sub-teams  for  designing  and  developing  the  different  parts  of  the  courses.  The  first  part  of 
the  courses  will  provide  more  theoretical  information  about  such  topics  as  instructional  design, 
technology  management,  and  teaching  strategies.  Although  assignments  throughout  the  course 
will  require  preservice  teachers  to  apply  the  concepts  they  learn,  the  PT3  felt  that  situating 
knowledge-application  in  a more  authentic,  problem-based  environment  would  help  the  teachers 
better  understand  all  that  technology  integration  involves.  Therefore,  the  team  portioned  the 
second  half  of  the  courses  to  be  case-based  learning  scenarios  that  immerse  teachers  in  rich 
environments  for  active  learning  (REALs). 

Constructivist  environments,  like  those  of  case-based  learning,  work  well  when 
“conceptual  complexity  and  case-to-case  irregularity”  characterize  the  information  to  be  learned. 
Learners  of  such  information  need  to  develop  cognitive  flexibility  in  order  to  transfer  their 
knowledge  to  other  situations  and  varying  problems  (Spiro  et  al.,  1992).  Constructivist,  and 
especially  case-based  learning  environments  permit  such  development.  A case-based  learning 
environment  will  immerse  students  in  multitudinous  information  and  let  them  to  make  the 
connections  necessary  for  understanding  the  importance  of  technology  integration  and  how  such 
integration  might  look  in  a classroom  setting. 

As  is  characteristic  of  all  REALs,  the  300-level  courses  environments  will  rely  on 
authenticity  as  a student-motivator.  The  environments’  authenticity  will  also  give  students  the 
opportunity  to  organize  concepts  in  such  a way  that  they  will  transfer  to  the  situations  they  may 
find  in  practice  (Grabinger,  1996).  The  PT3  team  talked  with  the  university’s  teacher  education 
professors  and  K-12  teachers  to  devise  weekly  scenarios  enriched  by  reality-based  electronic 
artifacts  (e.g.,  emails  from  other  teachers,  phone  messages  in  the  form  of  audio-files).  The  team 
wanted  the  case-based  learning  unit’s  scenarios  and  materials  to  imitate  typical  experiences  of 
first-year  teachers. 

The  REALs  for  the  300-level  courses  specifically  meet  the  criteria  for  case-based 
learning  and  instruction.  Ertmer  and  Russel  (1995)  write,  “Case  based  instruction  is  a teaching 
method  which  requires  students  to  actively  participate  in  real  or  hypothetical  problem  situations, 
reflecting  the  kind  of  experiences  naturally  encountered  in  the  discipline  under  study.”  Students 
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actively  apply  learned  knowledge  and  skills  to  a problem  situation.  Ertmer  describes  these  cases 
to  be  “ambiguous,  messy,  and  recalcitrant.” 

Such  a problem  lies  at  the  heart  of  the  300-level  REAL.  In  fact,  students  must  first 
analyze  a fictitious  school’s  Colorado  Student  Assessment  Program  (CSAP)  scores  in  order  to 
define  the  case’s  problem.  The  unit’s  Web  site  will  then  offer  the  resources  and  information 
necessary  for  solving  the  problem.  However,  the  300-level  course  instructors  will  model  ways  in 
which  the  problems  can  be  solved  and  will  also  act  as  technical  and  organizational  supports. 

Throughout  the  unit’s  design  and  development,  the  PT3  team  made  a priority  of  building 
instruction  that  teachers  toward  the  technology  standards  for  teacher  education  as  presented  by 
the  National  Education  Technology  Standards  and  by  the  state  of  Colorado.  Each  piece  of  the 
case-based  learning  unit  will  enhance  students’  understandings  of  technology  integration  as  well 
as  linked  directly  to  those  standards. 

The  Scenarios 

The  case-based  learning  experiences  will  evolve  from  initial  pretense  that  a Colorado 
school  has  hired  the  participating  preservice  teachers  for  their  intended  subject  matter  and  grade 
levels;  the  unit  will  ask  them  to  imagine  that  they  have  worked  all  summer  to  plan  for  their 
classes.  However,  much  of  that  work  will  be  undermined  when,  in  Week  One,  the  teachers 
receive  a “voice  mail”  (in  the  form  of  an  audio-file  embedded  in  the  unit’s  interface)  informing 
them  that  they  have  been  reassigned  to  other  schools  . . . with  below  average  standardized-test 
scores.  Week  One  of  the  case  based  learning  unit  will  expose  the  teachers  to  data — in  the  form 
of  school  profiles  and  standardized-test  scores — that  can  acquaint  them  with  the  new  school’s 
environment.  The  Week  One  graphical  user  interface  (GUI)  will  provide  job-aids  and  a video- 
file  of  a district  assessment  specialist  explaining  how  to  read  the  test  scores.  The  GUI  will  also 
link  to  electronic  “cumulative  folders,”  all  of  which  will  contain  report  cards  and  some  of  which 
will  contain  lEPs.  Teachers  will  then  analyze  the  available  data  and  prepare  a chart  of  class  and 
individual  strengths  and  weaknesses,  as  well  as  proposed  strategies  for  meeting  instructional 
needs.  Also  useful  for  helping  the  teachers  visualize  their  simulation  environments  will  be  a 
letter  embedded  in  the  Week  One  GUI.  The  letter  will  announce  that  the  teacher  has  received  a 
grant  that  will  provide  classroom  funding  for  six  computers,  a printer,  a scanner,  a TV,  a VCR,  a 
digital  camera,  and  $200  in  discretionary  funds.  In  addition  to  familiarizing  teachers  with  the 
simulated  school  environment.  Week  One  must  also  familiarize  them  with  the  unit’s  online 
environment.  Although  each  week’s  interface  will  be  self-contained  and  intuitive,  in  Week  One, 
the  course  instructors  will  model  standard  procedures. 

Week  Two’s  GUI  will  first  point  teachers  toward  an  email  from  the  district  technology 
manager  informing  them  that  he  has  set  up  their  computers,  network  connections,  and 
peripherals  and  installed  various  software.  The  students  can  also  access  the  classrooms’  floor 
plans  from  the  Week  Two  interface.  The  two  “referrals”  the  teachers  will  find  within  the 
interface  will  pose  the  second  week’s  problems;  While  working  at  the  computers,  students  have 
been  off-task  and  disruptive;  they  have  not  completed  their  work;  and  they  have  been  crowding 
and  rough-housing  around  the  computers.  To  combat  those  problems  and  make  the  best  use  of 
the  equipment,  the  teachers  must  create  a list  of  classroom  rules  for  technology-use.  The  teachers 
will  also  have  the  option  of  rearranging  the  classroom  floor  plan.  To  support  the  performance  of 
those  tasks,  the  GUI  will  make  available  a video  that  shows  another  teacher  giving  a tour  of  her 
classroom  and  showing  a poster  of  the  technology  rules  she  has  implemented. 
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In  Week  Three,  after  teachers  are  well  acquainted  with  their  environments,  they  will  be 
asked  to  design  instruction  for  their  students.  Week  Three’s  task  will  be  to  revise  a preexisting 
lesson  plan  so  that  the  lesson  meets  the  instructional  needs  determined  by  Week  One’s  data 
analyses.  The  modified  lesson  plans  must  also  use  the  classrooms’  technology  resources  to 
enhance  instruction.  The  Week  Three  GUI  will  provide  access  to  a database  of  lesson  plans  (i.e., 
the  fictitious  lesson  plan  bank  that  the  teachers  spend  their  summers  creating)  that  do  not  use 
technology.  Week  Three’s  GUI  will  also  open  a memo  that  contains  a copy  of  the  National 
Education  Technology  Standards  for  Students  as  well  as  a memo  from  the  grant-giving 
foundation  reminding  the  teachers  to  spend  their  discretionary  funds.  In  addition,  teachers  will  be 
able  to  open  an  audio-file  of  a teachers  discussing  best-practices  for  integrating  technology  into 
the  classroom. 

Week  Four  will  be  framed  as  a teacher  workday.  From  that  week’s  GUI , teachers  will 
access  collaboration  worksheets  then  work  with  each  other  to  improve  their  lesson  plans.  In 
Week  Five’s  GUI,  teachers  will  open  a letter  from  the  grant-giving  foundation,  asking  that  the 
teachers  submit  a report  about  how  they’ve  used  the  resources  that  the  foundation  has  provided. 
In  this  report,  teachers  will  need  to  reflect  on  how  they  can  use  technology  as  a tool  to  support 
their  teaching  strategies. 

Week  Six’s  GUI  simply  offers  online  resources  for  solving  the  problems  that  course 
instructors  will  create.  During  that  week’s  class,  the  instructors  will  give  each  teacher  a “trouble 
card”  that  plagues  their  lessons  with  technical  difficulties  (troubles  will  be  individualized  so  as  to 
disable  the  delivery  of  each  teacher’s  lesson).  In  response,  teachers  must  create  contingency 
plans  (e.g.,  handouts,  library  research  of  print  materials)  for  their  lessons.  In  the  unit’s  final 
week.  Week  Seven,  teachers  will  discuss  their  experiences  and  present  their  lesson  plans  and 
reports  to  the  foundation. 

The  Graphical  User  Interface 

Upon  entering  the  Web-based  environment,  the  preservice  teachers  will  not  find  the 
menus,  navigation  bars  or  buttons  typical  of  traditional  Web  sites.  Instead,  they  will  be  stepping 
into  their  new  offices.  Materials — in  the  form  of  email  and  voice  mail  messages,  letters,  memos, 
and  face-to-face  meetings — that  support  the  learning  unit,  will  be  stored  in  and  accessed  via  five 
active  zones:  the  computer,  telephone,  in-box,  file  cabinet,  and  daily  planner. 

The  daily  planner  will  be  the  main  user-cueing  device;  the  planner  is  the  user’s  the  to-do 
list.  The  list  will  remind  users  of  the  week’s  important  tasks,  such  as  checking  e-mail,  and 
attending  events  (e.g.,  a lunch  meeting  with  another  teacher).  A mouse-click  on  the  appropriate 
active  zone  will  open  a small  window  and  let  the  preservice  teacher  check  their  email,  listen  to 
voicemail  messages,  or  investigate  the  contents  of  folders  and  files.  The  windows,  which  users 
can  drag  anywhere  on  the  screen  and  have  open  simultaneously,  will  give  users  the  ability  to 
customize  their  work  areas. 

The  complex  design  of  the  user  interface  is  balanced  by  the  authenticity  of  its  elements. 
The  office  metaphor  is  not  only  relevant,  but  also  familiar.  Anyone  who  has  ever  read  a letter, 
opened  an  email,  or  used  an  answering  machine  possesses  the  prerequisite  skills  for  successfully 
navigating  the  interface. 

From  the  first  week  of  the  unit  to  the  last,  the  pre-service  teacher  will  see  this  case-based 
learning  environment  evolve,  as  new  scenarios  and  pieces  of  information  present  themselves.  As 
a result,  the  interface  must  be  flexible  enough  to  accommodate  this  evolution,  without  forcing 
users  to  re-orient  themselves  each  week.  Just  as  one  might  save  an  old  email  or  voice  message 


for  further  review,  the  preservice  teacher  will  have  access  to  items  introduced  in  previous 
weeks — as  well  as  new  items — ^through  the  same  five  active  zones. 

The  unit’s  scenarios  are  designed  to  realistically  mimic  a teacher’s  work  environment, 
including  interactions  with  fellow  teachers,  administrators,  and  support  staff.  A mixture  of 
audio,  video  and  HTML  file  formats  will  make  these  interactions  seem  more  real.  Video  clips 
and  sound  will  let  the  user  experience  their  environment  with  multiple  senses  and  as  though  they 
were  witnessing  them  first  hand. 

Initial  Sketch  of  Graphical  User  Interface 


Timeline 

The  PT3  team  has  designed  the  case-based  learning  units,  and  development  will 
commence  January,  2002.  The  implementation  of  the  course  will  begin  May,  2002.  A survey, 
the  Stages  of  Concern  Questioimaire  (SOCQ)  will  assist  the  team  in  gathering  data  about  the 
courses’  effectiveness.  The  SOCQ  measures  how  concerns  about  different  iimovations — in  our 
case,  technology  integration — change  over  time  (Horde  & Hall,  2001).  Currently,  students  are 
administered  the  SOCQ  in  both  the  200-  and  300-level  courses,  and,  after  implementation,  the 
team  will  continue  to  administer  the  SOCQ  several  times  throughout  the  course.  The  team  will 
also  analyze  student  projects  as  qualitative  data  that  will  indicate  whether  the  case-based  learning 
unit  is  effective  in  meeting  instructional  objectives. 

With  the  importance  of  training  teachers  to  effectively  integrate  technology,  case  based 
methods  may  be  one  of  the  most  powerful  tools  one  can  utilize  to  meet  state  and  national 
standards  regarding  this  goal.  Because  teacher  education  students  often  do  not  see  effective 
technology  integration  modeled  in  either  their  student  teaching  experiences,  or  in  the  majority  of 
their  college  courses,  one  class  often  may  not  be  enough  to  present  the  ill  defined  domain  of 
technology  integration  to  preservice  teacher  candidates.  This  case  study  may  prove  to  provide 
enough  information  to  give  learners  a basic  understanding  of  what  technology  integration  not 
only  looks  like  but  how  it  can  be  used  to  solve  problems  and  help  to  meet  state  and  national 
education  standards.  Activities  such  as  these  will  also  add  to  the  body  of  knowledge  regarding 
the  effectiveness  of  constructivist  environments,  in  particular,  case  based  learning  environments. 
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Satellite  Based  Synchronous  Tutorials  vs.  Satellite  Based  Asynchronous  Videocassettes:  Factors  Affecting  Students' 
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Background 

The  Open  University  of  Israel  (OUI)  is  a distance  learning  university.  Learning  is  based  mainly  on  specially  written  textbooks 
and  tutorial  meetings  (once  a week  or  once  in  three  weeks)  in  learning  centers  throughout  the  country.  Indeed,  meetings  with 
tutors  at  learning  centers  bring  the  university  closer  to  the  students  and  allow  for  face  to  face  interaction  between  tutors  and 
learners  and  among  the  learners  themselves.  However,  these  meetings  sometimes  do  not  materialize,  as  it  is  difficult  to  find 
enough  expert  tutors  for  all  groups  of  students,  or  students  are  so  dispersed  that  though  many  are  registered  for  a course,  there  are 
not  enough  students  in  each  region  to  justify  hiring  a tutor. 

Synchronous  virtual  tutorials,  via  satellite  communication,  from  a studio  at  the  university  to  classrooms  throughout  the  country 
(by  the  best  tutor)  - have  been  conducted  during  the  last  five  years  as  a solution  to  these  problems.  The  communication  between 
the  tutor  and  the  students  is  visual,  audio  and  data-based.  The  visual  communication  is  uni-directional  from  the  studio  to  the 
classrooms  (where  the  tutor  is  seen  on  a big  TV  screen).  Audio  information  is  bi-directional  (from  the  studio  to  the  classrooms 
and  from  the  classrooms  to  other  classrooms  and  to  the  studio).  The  lecturer  and  the  other  students  can  hear  the  student  who  has 
been  given  the  floor  by  the  tutor.  With  respect  to  data  communication,  the  lecturer  can  present  multiple  choice  questions  to  the 
students,  ask  them  to  answer  the  questions  by  pressing  an  appropriate  button  on  the  satellite  phone,  and  immediately  present  the 
distribution  of  answers  to  the  students. 

Seven  such  two-hour  satellite  tutorials  were  conducted  in  a "Basic  Research  Methods"  course  (for  social  science  students)  during 
5 semesters.  A number  of  factors  gave  rise  to  second  thoughts  about  the  choice  of  synchronous  satellite  tutorials  as  an  ultimate 
solution:  (a)  only  25%  of  the  registered  students  joined  the  tutorial  sessions;  (b)  most  students  who  joined  the  tutorials  did  not 
participate  actively  in  the  interactions  initiated  by  the  tutor  or  did  not  initiate  any  interaction  themselves;  and  (c)  the  satellite 
technology  has  proved  to  be  very  costly. 

Each  satellite-based  synchronous  tutorial  is  saved  on  a videocassette.  These  cassettes  can  be  sent  to  students  as  asynchronous 
tutorials.  Table  1 compares  satellite-based  synchronous  tutorials  to  satellite-based  asynchronous  tutorials  delivered  on 
videocassettes. 

Table  1:  Factors  differentiating  between  satellite- based  synchronous  tutorials  and  satellite-based  asynchronous  videocassettes 


Dimension 

Satellite -based  synchronous 
tutorials 

Satellite -based  asynchronous 
videocassettes 

The  location  of  the  tutorial 

In  classroom 

At  home 

Time  of  tutorial 

Specific  - tutor  and  students  at  the 
same  time 

Flexible  at  students’  convenience 

Accessibility  of  materials 

Not  accessible  after  the  end  of  the 
lesson 

Accessible  at  any  time 

Interaction  with  the  tutor  during  the 
tutorial 

Possible 

, Impossible 

Interaction  with  other  students 
during  the  tutorial 

Possible 

Impossible 

Cost  of  technology 

High 

Low 

In  light  of  the  above  comparison  and  students'  behavior  (absences  and  lack  of  participation),  we  decided  to  study  students' 
preferences  regarding  the  mode  of  tutorials.  In  particular: 

1.  What  are  students’  attitudes  towards  the  tutorial  modes  and  their  dimensions  (i.e.,  flexibility,  interaction  and  accessibility).  As 
can  be  seen  from  Table  1 , the  two  types  of  tutorials  differ  in  the  amount  of  control  the  student  has  over  the  learning  process  (the 
first  three  dimensions)  and  in  the  amount  of  interaction  between  tutor  and  students  (in  the  last  two  dimensions).  Learner  control  is 
known  to  be  associated  with  active  learning  and  student -centered  learning  (Doherty,  1998).  Interactions  between  students  and 
tutor  and  among  the  students  themselves  is  one  way  to  increase  instructional  immediacy  and  social  presence  which  are  positively 
related  to  students'  satisfaction  and  motivation  in  learning  (e.g.,  Christophel,  1990;  Swan,  2001).  Will  students  prefer  control  over 
their  learning  process  at  the  expense  of  losing  social  immediacy,  or  vice  versa? 

2.  Are  students'  attitudes  and  preferences  related  to  their  learning-habits  preferences?  Some  research  has  been  done  on  the 
relation  between  cognitive  styles  and  instructional  preferences  (e.g.,  Sadler- Smith  & Riding,  1999).  The  present  study  aims  to 
examine  the  relation  between  learning-habits  preferences  and  instructional  preferences. 

3.  Are  students'  attitudes  and  preferences  related  to  demographic  and  academic  variables? 

Some  of  our  findings  concerning  these  questions  will  be  discussed  in  the  next  sections. 
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Method 

Participants:  Two  groups  of  students  (in  the  Research  Methods  Course)  and  one  tutor  took  part  in  the  research. 

Group  1 : 92  students  who  studied  during  the  fall  semester  of  2001 . They  had  seven  satellite-based  synchronous  tutorials  (two 
hours  each)  with  their  tutor. 

Group  2:  73  students  who  studied  during  the  spring  semester  of  2001.  They  had  4 satellite-based  asynchronous  videocassettes 
(based  on  those  recorded  during  the  previous  semester)  and  3 satellite-based  synchronous  tutorials  with  the  same  tutor. 

Questionnaires:  Three  questionnaires  were  constructed- 

1.  Questionnaire  1 : A feedback  questionnaire  on  the  satellite-based  synchronous  tutorials.  The  questionnaire  included  items 
relating  to  (a)  attendance  rate;  (b)  attitudes  towards  the  tutor  and  the  tutorials;  (c)  attitudes  regarding  the  advantages  and 
disadvantages  of  meeting  and  interacting  with  peers;  (d)  comparisons  between  face  to  face  tutorials  and  satellite-based 
synchronous  tutorials  and  (e)  attitudes  toward  a potential  asynchronous  mode  of  tutorial  via  satellite-based  videocassettes. 

2.  Questionnaire  2:  A feedback  questionnaire  on  asynchronous  tutorials  delivered  via  videocassettes.  It  contained  items  similar  to 
those  in  the  first  questionnaire  but  relating  to  videocassettes  and  a comparison  between  three  modes  of  tutorials:  satellite-based 
synchronous  and  asynchronous,  and  face  to  face  tutorials. 

3.  Questionnaire  3 - "How  do  I prefer  to  study?":  This  questionnaire  contained  items  on  four  dimensions:  (a)  the  importance  of 
autonomy  in  management  of  learning  time;  (b)  the  importance  of  ease  of  accessibility  to  all  learning  materials;  (c)  the  importance 
of  synchronous  interactions  with  the  tutor  (d)  the  importance  of  synchronous  interactions  with  other  student  s.  For  each  of  the  four 
dimensions,  a number  of  statements  were  formulated  differing  in  the  affective  influence  and  in  the  cognitive  influence  of  the 
dimension  and  the  direction  of  that  influence  (positive  or  negative).  For  example,  with  regard  to  time  management: 

1.  When  I am  responsible  for  my  pace  of  learning  I feel  I have  control  (positive,  affective) 

2.  When  I am  responsible  for  my  pace  of  learning  I feel  helpless  (negative,  affective) 

3.  My  learning  is  more  efficient  when  I am  responsible  for  my  pace  of  learning  (positive,  cognitive) 

4.  My  learning  is  more  efficient  when  the  timetable  of  the  course  is  determined  by  the  teaching  team  (negative,  cognitive). 

The  questionnaire  included  56  statements  (in  random  order)  on  a Likert  scale  from  1-  "doesn't  describe  me  at  all"  to  5 - "describes 
me  very  well". 

The  "How  do  I prefer  to  study?"  questionnaire  was  administered  to  288  “Research  Methods”  students  who  were  not  part  of  the 
experiment.  Factor  analysis  revealed  four  independent  factors  (scales):  time  managpment  (high  score  indicates  learning 
autonomy),  ease  of  accessibility  to  learning  material  (high  score  indicates  the  subject’s  need  to  "have"  all  material),  positive 
aspects  of  interaction  (high  score  indicates  that  the  subject  has  more  positive  views  regarding  interactions  - with  tutor  and  other 
students)  and  negative  aspects  of  interaction  (high  score  indicates  that  the  subject  has  few  views  regarding  the  negative  aspects  of 
interaction)*.  Thus,  four  scores  were  constructed  for  each  subject,  one  for  each  factor.  There  was  no  correlation  between  the 
scores  and  the  following  variables:  gender,  age  and  number  of  credits  accumulated  at  the  OUI. 

Procedure:  At  the  end  of  the  fall  semester.  Questionnaire  1 (experience  with  seven  satellite-based  synchronous  tutorials),  and 
Questionnaire  3 ("How  do  I prefer  to  study?")  were  mailed  to  students  in  Group  1,  who  were  asked  to  fill  them  out  in  that  order. 
At  the  end  of  the  spring  semester.  Questionnaire  1 , Questionnaire  2 (experience  with  the  four  satellite-based  asynchronous 
tutorials  on  videocassette)  and  Questionnaire  3 (in  that  order)  were  sent  to  students  in  Group  2.  Forty -three  (46.7%)  students  in 
Group  land  31  (42.7%)  students  in  Group  2 returned  the  questionnaires. 

Results 

Group  participation  in  learning  actiMties:  On  average.  Group  1 students  participated  in  2.9  of  the  seven  satellite-based 
synchronous  meetings  (41.4%).  Group  2 students  participated  on  average  only  in  1.1  of  3 (36.7%).  This  difference  is  not 
significant.  All  students  rated  as  high  all  audio  and  visual  characteristics  of  the  two  technologies  used.  From  reports  of  Group  2 
students,  it  seems  they  created  a learning  environment  at  home  in  which  others  did  not  disturb  them  while  they  were  watching  the 
tutorial. 

With  regard  to  the  activities  associated  with  the  satellite-based  asynchronous  videocassettes,  41.9%  students  reported  they 
watched  all  4 lessons,  91 .7%  stated  they  watched  them  only  once,  56%  used  the  back  and  forward  option,  66.7%  summarized 
while  watching  (most  by  stopping  the  video)  and  most  of  those  who  summarized  (81 .2%)  indicated  that  they  would  read  the 
summary  before  the  final  examination. 


* Contrary  to  intuitve  perception  which  sees  the  two  extreme  sides  of  a dimension  (e.g.,  high  and  low)  as  negatively 
correlated,  the  Psychology  literature  on  attitudes  demonstrates  that  often  the  two  end  points  of  a dimension  are  two 
independent  factors  (Eagly  and  Chaiken,  1998). 
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Group  comparisons:  Groups  1 and  2 did  not  differ  significantly  in  gender  (males:  58.5%  and  57.6%  in  the  two  groups, 
respectively).  Table  2 presents  data  comparing  Groups  1 and  2 on  background,  academic  and  learning-habit  preferences 
variables. 


Table  2:  Background,  academic  and  learning-habit  preferences  in  Groups  1 and  2 


Variable 

Group  1 

Group  2 

M 

SD 

M 

SD 

Age 

29.6 

5.7 

30.0 

5.8 

Final  grade  in  course 

71.2 

8.9 

72.8 

8.3 

GPA* 

78.8 

6.5 

79.8 

6.0 

Scale  1 — Time  management 

3.5 

0.8 

3.4 

0.6 

Scale  2 - Ease  of  accessibility  to  learning  material 

4.0 

0.6 

3.9 

0.5 

Scale  3 - Positive  aspects  of  interactions 

3.1 

0.7 

3.2 

0.5 

Scale  4 - Negative  aspects  of  interactions 

3.8 

0.6 

3.9 

0.5 

*Grade  point  average  at  the  stage  the  student  registered  for  the  Research  Method  course 


There  were  no  significant  differences  between  the  two  groups  in  none  of  the  background,  academic  and  learning-habits 
preferences  variables  presented  in  Table  2.  Both  groups  were  also  similar  in  the  learning  materials  they  received  (the  same  books 
and  tutorials)  and  the  tutor  teaching  them.  As  all  these  variables  were  controlled  they  cannot  be  considered  an  alternative 
explanation  for  any  reported  results  (in  the  next  sections)  concerning  the  differences  in  the  students'  attitudes  towards  the 
different  learning  technologies  used  and  their  preferences  (The  Institute  for  Higher  Education  Policy,  1999). 

Groups  1 and  2 evaluated  the  tutor  and  the  satellite-based  synchronous  tutorials.  Students  were  asked  about  the  tutor's  expertise, 
teaching  pace,  attentiveness  to  students,  ability  to  excite  interest  and  to  clarify  the  material  as  well  as  command  of  the 
technology.  They  were  also  asked  to  rate  the  synchronous  tutorials  on  a number  of  dimensions  (how  significant,  interesting  and 
organized  they  were,  as  well  as  the  extent  to  which  they  contributed).  On  average,  the  tutor  and  the  satellite-based  synchronous 
tutorials  were  rated  hi^  (3-5  on  a 1-5  scale)  with  no  significant  differences  between  Group  1 and  Group  2.  Group  2 students 
were  asked  to  evaluate  in  a similar  manner  the  tutor  and  the  satellite-based  asynchronous  videocassettes  (not  including  questions 
regarding  the  tutor's  attentiveness  to  students  and  command  of  technology  which  were  not  relevant  in  the  asynchronous  mode  of 
tutorials).  Students'  ratings  of  the  tutor  and  the  tutorials  of  the  synchronous  tutorials  were  compared  to  their  ratings  of 
asynchronous  videocassettes  (t  test  for  dependent  variables).  No  significant  differences  were  detected.  Thus,  when  the  same 
tutor  gives  the  same  tutorials  to  similar  students,  the  evaluations  of  the  tutor  and  the  lessons  are  independent  of  the 
technology  used. 

Attitudes  toward  interaction  components:  In  both  groups,  subjects  were  asked  whether  the  tutor  devoted  enough  time  to 
interaction  with  students.  With  regard  to  the  synchronous  tutorials,  73%  of  students  were  satisfied  with  the  amount  of  interaction 
(no  significant  difference  between  the  groups).  Students  in  group  2 were  asked  the  same  question  twice  - once  for  the 
synchronous  lessons  and  once  for  the  asynchronous  lessons.  Most  students  indicated  there  was  enough  time  allocated  to 
interactions  in  both  types  of  tutorials  and  the  difference  in  response  to  the  two  questions  was  not  significant.  Most  students  in 
both  groups  (82.7%)  indicated  they  participate  less  in  synchronous  satellite  tutorials  than  in  face-to-face  tutorials.  Table  3 
describes  the  results  concerning  interaction  components  in  the  satellite-based  synchronous  tutorials  in  Groups  1 and  2. 


Table  3:  Group  1 and  2 mean  ratings  for  interaction  components  in  the  satellite-based  synchronous  tutorials. 


Group  1- 
synchronous 

Group  2 - 
synchronous 

M 

SD 

M 

SD 

During  the  satellite  lesson,  to  what  extent: 

. when  you  wanted  to  ask  or  respond,  were  you  given  the 
floor? 

4.32 

0.86 

4.14 

0.57 

. are  you  disturbed  by  the  fact  that  the  tutor  can't  see  you? 

2.53 

1.34 

2.38 

1.36 

Other  students  in  my  class: 

. are  important  to  me  from  a social  perspective 

2.66 

1.10 

2.24 

1.09 

. are  important  to  me  from  a learning  perspective 

2.72 

1.20 

2.76 

1.09 

. disturb  me  during  the  lesson 

1.69 

0.97 

1.57 

0.87 

ents  in  other  classrooms: 

1.  are  important  to  me  from  a social  perspective 

1.38 

0.75 

1.38 

0.87 

. are  important  to  me  from  a learning  perspective 

1.69 

0.97 

1.86 

1.01 

!.  disturb  me  during  the  lesson 

1.31 

0.59 

1.33 

0.48 

Questions  by  other  students  and  the  tutor's  answers: 
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'.  contribute  to  my  learning 

3.34 

0.86 

3.35 

0.93 

0.  disturb  my  learning 

2.28 

0.99 

2.19 

1.08 

None  of  the  differences  is  significant,  indicating  that  the  having  less  satellite -based  synchronous  tutorials  (3  instead  of  7) 
doesn’t  affect  students'  attitudes  regarding  the  interaction  components  of  those  tutorials. 


Table  4 compares  students'  attitudes  towards  the  interaction  components  of  satellito-based  synchronous  tutorials  to  those 
components  in  the  satellito-based  asynchronous  videocassettes.  The  numbers  given  to  questions  in  Table  4 are  similar  to  those  in 
Table  3.  None  of  the  differences  (for  identical  or  for  similar  questions)  were  significant  (t  tests  for  two  dependent  variables). 

Table  4:  Group  2 students'  attitudes  towards  the  interaction  components  in  satellito-based  synchronous  tutorials  and 
asynchronous  videocassettes. 


Group  2- 
synchronous*" 

Group  2 - 
asynchronous 

To  what  extent: 

M 

SD 

M 

SD 

a.  are  other  students  in  my  classroom  important  for  me  from 
a social  perspective? 

2.06 

0.83 

b.  do  I miss  the  presence  of  other  students  from  a social 
perspective? 

1.94 

1.08 

a.  are  other  students  in  my  classroom  important  to  me  from  a 
learning  perspective? 

2.53 

1.01 

b.  do  I miss  the  presence  of  other  students  from  a learning 
perspective? 

2.00 

1.12 

a.  are  other  students  in  my  classroom  important  to  me  from  a 
learning  perspective? 

2.53 

1.01 

c.  are  other  students  in  the  recorded  lesson  important  to  me 
from  a learning  perspective? 

2.35 

1.06 

a.  do  other  students  in  the  classroom  disturb  me  during  the 
lesson? 

1.59 

0.94 

b.  do  other  students  in  the  recorded  lesson  disturb  me  during 
the  lesson? 

2.06 

1.00 

'.  do  questions  by  other  students  and  the  tutor's  answers 
contribute  to  my  learning? 

3.13 

0.81 

2.81 

0.75 

0.  do  questions  by  other  students  and  the  tutor's  answers 
disturb  my  learning? 

2.35 

1.11 

2.41 

1.18 

The  results  in  the  first  two  columns  are  not  similar  to  those  in  the  last  two  columns  of  Table  3 as  not  all  subjects  in  Group  2 
answered  both  kinds  of  questions,  as  necessary  for  a within  subjects  comparison. 

Individual  differences  in  the  attitudes  towards  interaction  components:  Do  learning-habits  preferences,  as  measured  in 
questionnaire  3 correlate  with  students’  answers  to  the  interaction  questions  concerning  the  different  technologies?  Table  5 
presents  correlation  coefficients  which  were  computed  for  each  one  of  the  questions  presented  in  Tables  3 and  4 with  two  of  the 
interaction  scores  (positive  and  negative)  derived  from  the  learning  habit  preferences  questionnaire.  The  bold  correlations  with 
the  Positive  Interaction  factor  are  expected  to  be  positive.  The  bold  correlations  with  the  Negative  Interaction  factor  are  expected 
to  be  negative. 

Table  5:  Correlation  between  answers  to  interaction  questions  (in  Questionnaires  1 and  2)  and  interaction  factors  (derived  from 
Questionnaire  3) 


Question 

Positive  interaction 

Negative  interaction 

3a 

0.43** 

0.02 

3b 

0.41* 

0.03 

4a 

0.27 

-0.12 

4b 

0.31 

-0.02 

4c 

0.51* 

-0.11 

5a 

-0.14 

-0.08 

5b 

-0.07 

-0.54** 
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9a 

0.52*** 

0.12 

9b 

0.48* 

0.03 

10a 

-0.38** 

-0.51*** 

10b 

-0.26 

-0.45* 

* 0.05  **  0.01  ***0.001 

The  results  in  Table  5 strengthen  the  validity  of  the  learning-habit  preferences  questionnaire.  Students  with  a high  score  on  the 
positive  interaction  factor  (the  subject  holds  more  positive  views  regarding  interactions  * with  tutor  and  students)  believe  more 
than  others  in  the  social  contribution  of  other  students  and  in  their  learning  contribution.  Students  with  a high  score  on  the 
negative  interaction  factor  (the  subject  holds  few  views  regarding  the  negative  aspects  of  interaction)  perceive  students’ 
interactions  in  class  as  less  disturbing. 

Preferences  of  different  learning  technologies:  Subjects  in  both  groups  were  asked  the  following  question:  ’’Below  are  two 
types  of  tutorials  with  which  you  are  acquainted:  face-to-face  tutorials  and  satellite-based  classroom  tutorials.  On  the  assumption 
that  the  same  tutor  teaches  the  tutorials,  mark  with  an  X the  method  that  you  believe  is  the  best  way  of  learning  with  respect  to 
the  dimensions  below”.  The  1 1 dimensions  listed  were:  I understand  the  tutor  better;  I concentrate  better;  It  is  easier  for  me  to 
summarize  the  tutorial;  I ask  more  questions;  I answer  more  questions;  I enjoy  the  lesson  more;  I understand  the  material  better; 
The  questions  I ask  get  more  responses;  I feel  that  I have  better  control  of  the  situation;  I feel  more  obligated  to  attend  the 
tutorial;  I prepare  myself  better  for  the  tutorial.  For  each  dimension,  the  proportion  of  face-to-face  choices  was  calculated  (for 
each  group)  and  a Chi-Square  test  for  two  independent  groups  was  performed  to  identify  differences.  None  of  the  results  were 
significant,  thus  reinforcing  the  conclusion  that  having  fewer  satellite-based  synchronous  tutorials  does  not  affect  students’ 
attitudes  towards  this  type  of  teaching/leaming.  Face-to-face  tutorials  are  considered  advantageous  with  respect  to  ten  of  the 
eleven  cognitive  and  affective  dimensions  (for  most  dimensions,  face-to-face  tutorials  were  chosen  by  more  than  70%  of  the 
students).  On  the  last  dimension  (I  prepare  myself  better  for  the  tutorial),  only  48%  of  Group  1 and  66.7%  of  Group  2 chose  the 
face-to-face  tutorial  as  better. 

Group  2 students  were  asked  a similar  question  with  the  same  dimensions  and  an  additional  five  (I  remember  more;  It  is  more 
convenient;  I prepare  a written  summary;  I control  my  learning  pace;  Overall)  concerning  the  choice  between  face-to-face 
tutorials,  satellite-based  synchronous  tutorials  and  satellite-based  asynchronous  videocassettes.  For  most  dimensions  (except  for 
”It  is  more  convenient”  and  ”I  control  my  learning  pace”),  most  students  chose  face-to-face  tutorials.  Satellite-based 
asynchronous  videocassettes  over  rode  the  other  two  types  of  tutorials  for  all  other  students.  5 1 .9%  and  77.8%  of  students  chose 
satellite-based  asynchronous  videocassettes  when  asked  about  convenience  and  control,  respectively.  When  asked  to  make  an 
overall  choice  between  satellite  based  synchronous  tutorials  and  satellite  based  asynchronous  videocassettes,  60.7%  chose  the 
asynchronous  mode  and  only  32.1%  chose  the  synchronous  mode. 

Group  1 students  were  also  asked  about  a hypothetical  choice  between  satellite  based  synchronous  tutorials  and  satellite  based 
asynchronous  videocassettes  (hypothetical,  as  they  didn’t  experience  the  second  option).  On  most  dimensions,  the  dominant 
choice  was  satellite  based  asynchronous  videocassettes  (except  for  ”I  am  more  involved  in  the  learning  process”  and  "I  am  more 
active”).  When  asked  to  make  an  overall  choice  between  the  two,  76.3%  chose  the  asynchronous  mode  and  only  23.7%  chose  the 
synchronous  mode. 

Individual  differences  in  the  choice  of  learning  technologies:  Do  learning-habit  preferences,  as  measured  in  our  questionnaire, 
correlate  with  students'  choices  of  different  types  of  tutorials?  Students  in  both  groups  were  divided  according  to  their  overall 
choice  between  the  satellite  based  synchronous  tutorials  and  the  satellite  based  asynchronous  videocassettes.  The  two  groups 
were  compared  on  the  4 learning-habit  preferences  scores.  Table  6 presents  those  comparisons. 

Table  6:  Differences  in  learning-habit  preferences  between  students  who  chose  satellite  based  synchronous  tutorial  and  students 
who  chose  satellite  based  asynchronous  videocassettes. 


Factors 

Prefer  synchronous 
tutorials 

Prefer  asynchronous 
tutorials 

Significance  level 

Scale  1 - Time  management 

3.1  (0.9) 

3.6  (0.6) 

T(60)=2.36* 

Scale  2 - Ease  of  accessibility  to 
learning  material 

3.7  (0.6) 

4.1  (0.5) 

T(59)=2.4* 

Scale  3 - Positive  aspects  of 
interactions 

3.5  (0.5) 

3.0  (0.6) 

T(59)=2.4* 

Scale  - Negative  aspects  of 
interactions 

4.0  (0.5) 

3.8  (0.5) 

... 
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The  two  groups  of  subjects  differ  in  3 of  the  4 learning  habit  preferences  scores.  Those  who  chose  the  satellite-based  synchronous 
tutorials  were  significantly  higher  on  Scale  3 (believing  in  the  positive  aspects  of  interactions)  and  significantly  lower  on  scales  1 
(learning  autonomy)  and  scale  2 (the  need  to  "have"  all  the  material). 

Discussion  and  Conclusions 

When  choosing  between  face-to-face  tutorials  and  satellite  based-synchronous  tutorials,  most  students  prefer  the  former, 
replicating  a previous  study  done  at  the  OUI  (Beyth-Marom  et  al.,  2000).  However,  when  choosing  between  satellite  based- 
synchronous  tutorials  and  satellite-based  asynchronous  videocassettes,  two  thirds  of  subjects  prefer  the  latter.  As  in  the  previous 
study  done  at  the  OUI,  most  students  prefer  flexibility  and  control  of  their  learning  processes  while  studying  at  home  with  less 
social  immediacy  (with  videocassettes)  over  the  social  immediacy  they  have  (but  less  personal  control)  in  synchronous  satellite 
based  tutorials. 

In  an  effort  to  study  the  factors  affecting  students'  choices  and  preferences,  students  were  asked  about  their  attitudes  towards 
different  interaction  components.  In  both  groups,  students  did  not  rate  the  interaction  components  (synchronous  or  asynchronous) 
as  very  important  or  contributing.  All  students  answered  a questionnaire  about  their  learning-habit  preferences.  Students  with  a 
high  score  on  the  positive  interaction  factor  (the  subject  holds  more  positive  views  regarding  interactions  with  tutor  and  students) 
believe  more  than  others  in  the  social  contribution  of  other  students  and  in  their  learning  contribution.  Students  with  a high  score 
on  the  negative  interaction  factor  (the  subject  holds  few  views  regarding  the  negative  aspects  of  interaction)  perceive  students' 
interactions  in  class  as  less  disturbing. 

Students  who  preferred  the  satellite-based  synchronous  tutorials  were  compared  to  those  who  preferred  the  satellite-based 
asynchronous  videocassettes  on  the  4 scales  of  the  learning-habit  preferences  questionnaire.  Those  who  preferred  the 
synchronous  tutorial  were  significantly  higher  in  their  belief  in  the  positive  aspects  of  interactions  (Scale  3)  and  significantly 
lower  on  learning  autonomy  (Scale  1 ) and  the  need  to  "have"  all  material  (Scale  2)  than  those  who  preferred  the  asynchronous 
mode. 

In  general,  students’  tutorial-mode  preferences  depend  on  their  learning-habits  preferences  as  measured  on  a Likert  type 
questionnaire:  their  attitudes  toward  the  control  of  learning  and  the  possible  contribution  of  interactions. 
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Abstract:  Twenty-four  Teaching,  Technology,  and  Teamwork  (TTT)  projects  sponsored  by  the  Charles  E.  Culpeper 
Foundation  at  California  Lutheran  University  were  developed  from  Fall  1999  through  Spring  2001.  This  project 
update  reviews  strategies  used  in  this  team-based  model.  Trends  in  project  development  and  evaluation  findings  will 
also  be  reported. 


This  paper  presents  findings  from  the  Teaching,  Technology,  and  Teamwork  (TTT)  program  sponsored  by  the 
Charles  E.  Culpeper  Foundation  at  California  Lutheran  University.  CLU’s  Information  Technology  Advisory 
Committee  (ITAC)  solicited  aid  reviewed  faculty  proposals.  Faculty  whose  proposals  were  accepted  met  with  their 
team  members  once  a week  for  one  semester  to  develop  their  projects.  Each  project  team  included  a faculty  member 
(the  content  expert),  a reference  librarian  (the  information  specialist  who  helped  with  research),  an  educational 
technologist,  and  a student  assistant  who  implemented  the  technology  elements  of  the  project. 

The  projects  used  combinations  of  five  major  technologies.  WebCT  projects  (14)  outnumbered  slide  digitization  and 
database  development  (2)  and  video  production  (7);  custom  web  sites  (6),  and  CD-ROM’s  (1).  The  total  student 
development  time  in  hours  was  greatest  for  video  (840  hours)  followed  by  slide  digitization  (750  hours).  The 
fourteen  WebCT  projects  required  560  student  development  hours  followed  by  the  custom  web  projects  with  240 
student  hours.  The  digital  video  and  slide  projects  are  not  scaleable;  WebCT,  with  its  difficult  user  interface,  appears 
to  be  scaleable. 

One  course  used  the  EduCue  system  to  incorporate  problem -based  learning  in  the  classroom.  Students  submitted 
answers  via  hand-held  devices  and  the  instructor’s  computer  tabulated  and  displayed  their  responses.  Students 
discussed  the  results  and  used  the  EduCue  device  to  answer  the  same  questbns  again.  Because  each  of  the  devices 
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has  an  ID  number,  the  instructor  knows  who  attended  class  and  how  each  student  performed.  (For  more  information 
on  EduCue,  see  http://www.educue.com.) 

Some  of  the  most  successful  projects  have  multiple  audiences.  Our  California  history  project  has  been  used  by  four 
audiences:  pre -service  teachers  who  will  teach  California  history;  fourth  grade  history  teachers;  students  in  CLU’s 
California  history  class;  and  Computer  Science  students  who  are  learning  programming.  A marine  biology 
professor’s  Beneath  the  Surface  Catalina  Island  video  is  used:  by  CLU  students  preparing  for  a Catalina  Island  field 
trip;  and  by  passengers  who  visit  Catalina  Island  on  The  RapturC’-a  marine  biology  expedition  and  research  boat. 
Faculty  and  students  use  a WebCT  “course”  on  Service  Learning.  (For  more  information  on  these  projects  and  the 
Culpeper  projects,  see:  http://www.clunet.edu/TLC .)  .Our  assessment  of  the  TTT  program  included  formative  and 
summative  evaluation.  Our  campus  computing  committee  reviewed  and  critiqued  projects  at  midcourse.  In  May 
2001,  we  collected  evaluation  data  on  the  Culpeper  project  via  an  on-line  survey  form  using  AAHE’s  Project 
Flashlight  program.  Fourteen  of  24  faculty  responded  to  the  survey.  We  collected  additional  data  from  focused 
interviews  with  program  participants  during  Fall  2001. 

CLU’s  proposal  to  the  Charles  E.  Culpeper  foundation  suggested  that  faculty  lacked  the  time  and  expertise  to  extend 
their  most  successful  pedagogies  to  include  technology.  Providing  summer  stipends  or  one  course  release  during  the 
year  helped  solve  this  problem.  Half  of  the  survey  respondents  commented  on  the  importance  of  the  teamwork 
component.  For  most,  the  experience  of  collaborating  with  a supportive  team  was  very  useful. 

Not  all  faculty,  however,  embraced  the  team  approach.  One  respondent  commented,  “Don’t  force  the  ’’team”  aspect 
on  everyone-provide  the  parts  of  the  team  that  are  helpful  (for  each  case),  and  then  get  out  of  the  way!”  Several 
respondents  mentioned  problems  with  the  students  on  the  teams.  The  quality  of  student  help  in  completing  the 
project  varied  considerably.  One  faculty  member  observed,  “Much  was  accomplished  when  I had  good  help  working 
on  the  project.  Things  languished  when  I didn’t.”  During  the  initial  development  cycle,  student  help  was  not 
screened  as  closely  as  it  needed  to  be,  and  this  was  corrected.  The  development  teams  with  technically  competent 
and  responsible  student  assistants  usually  resulted  in  a higher  quality  product.  When  the  faculty  member  seemed  to 
be  in  the  program  for  the  money  or  to  teach  one  less  class,  progress  was  slow  or  the  final  product  weak.  In  a couple 
of  cases,  these  faculty  resisted  any  organizational  and  planning  meetings,  something  we  felt  was  critical  to  getting 
the  projects  started  and  developed  in  a timely  manner. 

Some  faculty  reported  new  ways  of  envisioning  how  to  teach.  “Actually,  the  Culpepper  project  (Thespian  Survivor) 
...  did  force  me  to  think  differently,  perhaps  in  more  visual  terms,  about  how  to  present  the  same  material  I had  used 
in  class  for  the  last  few  years.”  Several  slide  and  video  projects  were  completed,  and  one  faculty  member 
commented  that  his  project  provided  the  “chance  to  better  grasp  the  possibilities  of  how  my  images  can  be  used 
more  easily  and  (hopefully)  more  creatively  in  my  teaching.” 

For  faculty  who  do  not  understand  technology,  the  problems  surfaced  during  implementation  in  the  classroom. 
When  a faculty  member  would  make  the  time  to  work  with  the  team  in  the  classroom  prior  to  implementation,  there 
were  fewer  problems.  Media  Services  also  provided  technical  assistance  for  some  of  the  initial  class  sessions  in  case 
there  were  problems. 

Post-project  evaluations  demonstrated  that  implementation  of  their  online  projects  prompted  faculty  to  envision  new 
and  more  effective  ways  to  teach  and  that  they  had  observed  improved  student  learning.  Some  also  reported, 
however,  that  in  order  to  enhance  and  quantify  these  improvements,  new  assessment  strategies  need  to  be  identified 
and  incorporated  into  subsequent  versions  of  the  projects.  Other  evidence  of  the  Culpeper  program’s  success  can  be 
found  in:  the  fact  that  the  University  is  funding  the  Teaching  and  Learning  Center  and  its  two  new  positions;  and  the 
dramatic  increase  in  requests  for  and  the  availability  of  technology  facilities. 
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Abstract:  With  limits  on  both  personnel  and  time  available  to  conduct 

effective  instruction,  the  decision  is  being  made  increasingly  to  enhance  instructor-led 
courses  with  Computer-Based  Training  (CBT).  The  effectiveness  of  this  conversion  is 
often  unknown  and  in  many  cases  empirical  evaluations  are  never  conducted.  This  paper 
describes  and  discusses  the  evaluation  and  effectiveness  of  adaptive  courseware 
authoring  and  utilization  in  the  context  of  the  Submarine  Officer  Basic  Course  (SOBC)  at 
the  Naval  Submarine  School,  Groton,  CT. 


Introduction 

The  Naval  Submarine  School  (NAVSUBSCOL),  the  “Center  of  Excellence”  for  Submarine 
Warfare  Training,  has  recently  dedicated  a new,  state-of-the-art  electronic  training  facility  that  hosts  ten 
Advanced  Electronic  Classrooms  (AEC),  a building- wide  enterprise  Classified  LAN  (CLAN),  and  external 
high  bandwidth  connectivity.  The  NAVSUBSCOL  Officer  Training  Department  provides  the  Fleet  with 
Prospective  Commanding  Officers  (PCOs),  Prospective  Executive  Officers  (PXOs),  Department  Heads, 
Division  Officers,  and  a host  of  officer  billet  and  specialty  training  courses.  NAVSUBSCOL  training 
responsibilities  range  from  engineering,  damage  control,  and  ship  control,  to  combat  training  simulation 
and  tactical  employment  of  the  submarine.  Currently,  the  curricula  are  based  primarily  on  instructor-led 
presentations  and  practical  exercises  in  lab/simulator  environments. 

Recently,  NAVSUBSCOL  determined  that  the  demand  for  training  was  outpacing  the  available 
resources  (instructor-led  presentations  and  practical  exercises).  To  address  the  situation,  NAVSUBSCOL 
searched  for  a supplement  to  the  instructor-led  training  by  adding  computer-based  training  (CBT).  The  goal 
was  to  leverage  the  available  resources  of  the  lead  instructors  while  incorporating  additional  benefits  of 
current  courseware,  maintaining  internal  control  of  content,  and  reduce  staff  loading. 

This  case  study  discusses  the  efforts  made  at  NAVSUBSCOL  to  integrate  CBT  into  the 
curriculum  and  the  effects  this  type  of  learning  will  have  on  learning  outcomes.  Through  a comparison 
study  of  both  CBT  and  instructor-led  courseware,  supportive  evidence  can  be  garnered  that  support  a 
savings  in  development  time  and  improvements  in  test  scores. 

Traditional  multimedia  instruction  creates  an  engaging  learning  environment  in  which  multiple 
sensory  pathways  convey  information  to  the  student.  NAVSUBSCOL  wanted  to  choose  an  authoring  tool 
that  would  allow  instructors  to  re-use  content  and  learning  objectives  throughout  multiple  courses,  while 
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delivering  individualized  instruction  to  a large  volume  of  students.  Likewise,  it  wanted  the  authoring  tool  to 
be  easy  to  use  and  require  low  developmental  times  due  to  constant  turnover  of  personnel. 

InTrain™  provides  such  a tool.  InTrain  instruction  builds  and  tailors  the  content  and  form  of 
instruction  to  the  moment-to-moment  needs  of  the  student.  This  results  in  faster  acquisition  and  longer 
retention  than  is  seen  with  conventional  learning  approaches. 

In  a marked  contrast  to  conventional  CBT,  InTrain  develops  an  individual  education  plan  (lEP) 
for  each  student  and  makes  its  instructional  decisions  at  run-time.  Students  proceed  when  they  demonstrate 
that  they  are  meeting  the  goals  of  instruction  mastery.  Students  who  need  more  time  or  different  methods  to 
reach  the  same  level  of  performance  receive  the  instruction  they  need. 

While  forming  an  lEP,  InTrain  considers  information  about  the  student  (e.g.,  the  student’s  mastery 
profile  and  instructional  history)  and  about  the  goals  of  the  curriculum  (/.e.,  which  objectives  must  the 
student  meet  at  what  level  of  proficiency).  The  lEP  describes  the  order  in  which  the  system  will  introduce 
new  topics  and  the  training  techniques  that  will  support  the  introduction.  When  assessment  is  needed  (both 
pre-  and  post-tests  may  be  used),  InTrain  randomly  selects  among  the  assessment  options  (e.g.,  test 
questions)  associated  with  the  learning  objectives  under  investigation.  Similarly,  when  InTrain  wants  to 
present  instruction,  it  selects  among  the  instructional  options  associated  with  the  target  learning  objectives. 
Instruction  and  assessment  continue  until  the  student  has  adequately  mastered  all  of  the  required  learning 
objectives. 

Shorter  development  times  are  also  addressed  by  the  InTrain  Author\  software  {InTrain  Author\  is 
the  interface  through  which  InTrain  courseware  is  authored.  InTrain  does  not  require  any  programming 
experience  and  offers  an  “as  you  go”  tutorial  on  content  development.  This  ease  of  use  allows  for  shorter 
development  time  and  less  “training”  time  for  new  courseware  development. 


Method 

To  assess  the  effectiveness  of  the  training  to  be  demonstrated,  Sonalysts,  Inc.  conducted  a study  in 
conjunction  with  NAVSUBSCOL.  Traditional  instructor-led  classroom  students  were  given  8.5  hours  of 
instruction  about  SONAR,  as  were  the  CBT  students.  The  learning  objectives  and  goals  of  the  content 
presented  to  each  group  was  the  same.  In  contrast,  another  group  of  students  received  InTrain  instruction. 
Both  groups  were  given  the  same  amount  of  time  to  interact  with  the  content. 

After  the  second  group  completed  the  curriculum,  both  groups  were  then  given  a final  written 
examination,  comprising  of  80  questions.  Instructors  scored  the  tests  that  included  multiple-choice  and 
true/false  questions.  The  students  needed  to  achieve  a minimum  of  70%  to  pass  the  course.  The  scores  of 
the  instructor-led  group,  were  in-line  with  historical,  satisfactory  averages  for  this  content  and  type  of 
instruction. 

Traditionally,  training  for  the  NAVSUBSCOL  has  taken  place  in  instructor-led  classrooms.  Each 
classroom  comprises  approximately  24  students,  who  spend  a portion  of  a 10-week  course  learning  passive 
sonar  theory  and  underwater  acoustics.  The  focus  of  the  comparison  is  on  a specific  8.5  hours  of  the  Sonar 
curriculum  taught  during  the  ten-week  course.  The  8.5  hours  of  instruction  cover  the  following  courses: 
Passive  Sonar  Equation,  Total  Background  Noise,  Beamforming,  Figure  of  Merit,  Source  Level,  and 
Propagation  Loss. 

While  each  course  has  a standardized  set  of  Learning  Objectives  and  a Curriculum  Outline,  each  is 
modified  accordingly.  Courseware  for  traditional  instruction  has  usually  been  developed  in  PowerPoint 
lessons  with  Instructor  Discussion  Points  and  Lessons  Learned  being  conveyed  by  each  instructor. 
PowerPoint  are  inherited  by  the  previous  lesson  instructor  and  amended  when  the  need  occurs. 

Students  are  scheduled  for  each  of  the  above  topics  based  on  numbers  of  students,  classroom 
availability,  and  instructor  availability.  Delivery  time  for  each  course  is  a pre-defined  amount  and 
scheduled  by  a central  person.  Students  are  assigned  to  the  next  course  as  they  move  through  the 
curriculum.  At  the  end  of  each  component  of  instruction  (sonar,  tactics,  plots,  etc.)  students  are  given  a test 
comprising  80  questions. 

In  contrast  to  traditional  methods,  CBT  was  developed.  To  address  NAVSUBSCOL’s  goal  of 
authoring  CBT  “in  house,”  Submarine  School  staff  instructors,  working  with  guidance  from  instructional 
designers  and  subject  matter  experts  from  Sonalysts,  utilized  InTrain  AUTHORl  to  develop  an  eight-hour, 
adaptive  CBT  course  on  the  passive  sonar  theory  equation,  theories,  and  underwater  acoustics.  This 
development  effort  was  a multi-stage  process.  Initially  instructors  were  given  a basic  INTRO  courses  in 
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courseware  development  and  creating  computer-based  training.  Instruction  focused  on  creating  and 
decomposing  learning  objectives,  creating  assessment  items,  and  developing  content  (at  a variety  of 
targeted  levels).  Ultimately,  one  instructor  became  responsible  for  development  of  8.5  hours  of  SONAR 
courses.  InTrain  allows  for  instructional  method  neutrality.  The  tool  does  not  require  one  to  build/create 
Sharable  Content  Objects/presentations  in  a particular  manner,  thereby  allowing  the  developer  complete 
freedom  in  content  presentation  and  delivery.  Sonalysts  served  as  “consultants’*  to  the  instructor  in  charge 
of  development.  Media  was  reused  from  courseware  previously  authored  for  other  topics,  instructor-led 
training,  and  Submarine  Officer  On-Board  Training  (SOBT)  courses. 

The  content  developed  using  InTrain^  was  delivered  to  a class  of  21  students  that  utilized  the 
courseware  during  two  four-hour  dedicated  periods,  plus  a one-hour  wrap-up  following  the  testing.  The 
courseware  was  parsed  into  six  topics  that  mimicked  the  training  methodology  of  the  material  in  instructor- 
led  instruction.  The  CBT  was  deployed  using  options  that  permitted  the  student  to  learn  the  material  and 
demonstrate  mastery  through  assessment.  Students  were  permitted  two  attempts  at  the  assessments.  If  the 
student  failed  a second  assessment,  an  instructor-led  remediation  was  conducted. 

Students  were  given  the  same  amount  of  time  to  learn  using  the  computer-based  course  as  they 
were  given  to  learn  through  the  traditional  methods.  The  reasons  for  this  were  two-fold:  1)  Delivery  of 
CBT  was  an  emerging  concept  within  NAVSUBSCOL.  While  computer-based  training  is  available,  the 
courses  are  self-paced  auxiliary  learning  on  Lessons  Learned  and  Practice  for  defined  skills.  There  was  no 
model  from  which  to  build  time/scheduling  differences  around  this  type  of  learning.  2)  The  SOAC/SOBC 
Associate  Assistant  Director  of  Officer  Training  maintained  that  it  was  important  that  an  appropriate 
amount  of  time  was  scheduled  for  students  to  receive  the  training.  The  most  efficient  manner  to  achieve 
this  was  to  “schedule”  it  in  the  same  blocks  of  time  as  the  traditional  methods  of  delivery. 

Upon  completion  of  the  material,  students  were  able  to  review/retum  to  the  material  at  their 
convenience  for  follow-on  study. 

There  were  four  measures  by  which  results  were  garnered  for  this  comparison  study:  Survey,  Test, 
Historical  Averages,  and  Development  Time. 

Survey  - At  the  conclusion  of  the  computer-based  instruction  {InTrain  Author!),  students  and 
instructors  were  given  a Level  1 evaluation.  This  survey  consisted  of  both  open-ended  questions  and 
Likert-scale  questions. 

Test  - The  paper-based  test  included  80  questions  that  were  both  multiple-choice  and  true/false. 
Both  groups  of  students  (CBT  and  instructor-led)  were  given  the  same  test  and  graded  using  the  same  set  of 
instructors. 

Historical  Averages  - The  historical  averages  used  in  the  study  are  the  averages  of  students  who 
had  gone  through  the  instructor-led  training  in  the  past  two  years.  The  scores  were  from  a slightly  revised 
version  of  the  test  given  to  these  students  (The  revision  included  the  learning  objectives  but  a few  questions 
were  changed). 

Development  Time  - Development  time  for  the  InTrain  course  was  measured. 

Results 

The  results  of  both  courseware  development  and  actual  “testing”  of  the  courseware  as  indicated 
below  show  improved  in  results  in  both  the  authoring  content  and  the  student  performance. 


Survey 


The  results  of  the  students’  survey  were  interesting.  A 5-point  Likert  scale  was  used  for  the  20 
questions. 

The  areas  that  scored  highest  (4-4.5)  were  the  presentation  of  the  material  and  the  way  in  which 
students  interacted  with  the  course.  A score  of  4.2  was  given  to  the  overall  usefulness  of  the  course  within 
the  curriculum  in  general. 

The  students’  response  to  another  set  of  questions  (scoring  2. 6-2. 9)  demonstrated  that  no 
additional  instruction  or  assessment  was  necessary,  lending  credibility  to  the  individualized  component  of 
the  training  which  provides  “just  enough”  instruction  to  achieve  mastery  of  the  learning  objective.  There 
were  enough  assessment  and  instruction  to  teach  the  topic  without  being  overwhelmed  by  obscure 
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materials.  This  method  targets  the  learner  to  specific  information  to  support  specific  concepts  and 
objectives. 


Test  Results 


Students’  scores  in  both  the  instructor-led  group  and  the  InTrain  group  were  measured  and 
compared  on  the  basis  of  results  from  the  80-question  test. 


' - j ' ' ' ' ‘-i.  - r: 

Instructor-Led  Group 
(Last  2 classes  of 
students) 

InTrain  Students 
(One  class) 

Historical^ 

(Instructor-Led 

Group) 

Population 

75 

21 

>300 

Student  Mean^ 

88.5% 

95% 

84.5% 

No.  of  Exam  Failures^ 

3 

0 

N/A 

Table  1.  Evaluation  Results 


Courseware  Development 

No  metrics  exist  to  gauge  the  amount  of  development  time  required  to  prepare  instructor- led 
courseware  because  the  instructor-led  materials  are  inherited  from  the  original  instructor  and  revised  over 
several  years. 

Although  CBT  development  ratios  can  vary  widely,  actual  development  time  was  compared  to  a 
benchmark  of  200  hours  per  hour  of  CBT.  For  this  effort,  that  would  yield  an  approximate  effort  of  1600 
hours  of  authoring  time  to  design,  storyboard,  implement,  and  test  the  course. 

In  contrast  to  this  bench  work,  the  Submarine  School  staff  spent  a total  of  500  hours  for  the 
development  effort.  Sonalysts’  instructional  designers  and  subject  matter  experts  spent  an  additional  150 
hours  providing  feedback  and  guidance  on  effective  CBT  design  and  content  presentation. 


Discussion 

The  survey  results  yielded  some  interesting  information.  The  open-ended  questions  asked 
throughout  the  survey  indicated  that  the  students  were  favorably  impressed  with  and  receptive  to  the  CBT 
that  was  presented.  Additionally,  they  liked  the  “individualized  instruction”  that  was  presented  through 
InTrain  allowing  some  of  the  students  to  finish  quicker  and  more  efficiently  than  others  who  required  more 
time  or  instructor  intervention.  Some  negative  responses  about  the  CBT  included  the  need  to  have  an 
instructor  review  session  at  the  conclusion  of  training  (before  the  test).  Another  negative  comment  was  the 
length  of  time  dedicated  to  each  training  session  (taught  via  CBT)  was  too  long.  In  both  cases  each 
comment  can  be  resolved  in  a simple  and  effective  manner. 

NAVEDTRA  131  (Task  Based  Instruction),  the  controlling  doctrine  for  the  course  in  quesiton, 
requires  that  all  instruction  that  results  in  a bottleneck  be  serially  scheduled.  That  means,  for  a class  of  12 
students  participating  in  a learning  experience  that  takes  4 hours  and  has  capacity  for  only  6 students,  an 
entire  training  day  must  be  scheduled  against  it.  By  its  nature,  CBT  provides  unparalleled  flexibility  to 
obviate  bottleneck  scheduling.  Students  learn  at  their  own  pace,  on  their  own  schedule.  In  an  instant 
example,  the  other  six  students  not  in  the  learning  experience  can  complete  assigned  CBT  required  for 
another  part  of  the  course.  The  time  saved  by  removing  bottleneck  scheduling  is  approximately  24  hours. 

If  given  proper  integration  into  the  NAVSUBSCOL  curriculum  scheduling  process,  students 
would  proceed  through  the  courses  on  an  individualized  basis.  In  fact,  they  would  be  assigned  courses  and 
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a completion  date  and  proceed  at  their  own  pace  toward  that  completion  date.  This  is  the  way  that 
NAVSUBSCOL  is  moving  with  the  integration  of  a Learning  Management  System  (LMS)  to  create  such 
assignments. 

More  CBT  is  undergoing  development  in  other  areas  of  the  10-week  curriculum  matrix.  When 
more  of  the  courses  are  converted  to  CBT  and  an  entire  curriculum  can  address  scheduling,  changes  will  be 
easier  to  implement.  When  timesaving  is  realized,  an  instructor- led  review  session  can  be  addressed  and 
implemented  to  quantify  knowledge  gained  through  the  CBT. 

Additionally,  the  study  yielded  anecdotal  evidence  that  four  hour  training  periods  for  the  students 
were  too  long  and  resulted  in  student  “burn-out,”  an  assertion  supported  by  student  feedback  following  the 
training.  A consensus  of  input  suggests  that  a period  of  two  hours  at  any  given  time/topic  is  the  longest  that 
should  be  scheduled.  The  artificiality  introduced  by  the  beta  test  (i.e.,  the  test  dictated  two  four-hour 
periods  of  study)  required  the  student  not  study  when  they  were  ready  to,  but  rather  when  the  test  required 
it.  As  such,  student  mastery  and  navigation  through  the  material  suffered  at  the  end  of  the  first  training 
period  due  to  student  fatigue. 

Finally,  we  address  test  scores.  We  can  see  that  the  scores  on  the  test  were  higher  for  the  InTrain 
classes  than  the  traditional  instructor-led  method.  Since  both  groups  were  achieving  at  a high-level,  the 
observed  results  might  actually  understate  the  size  or  effect  one  would  expect  in  other  settings.  Another 
factor  that  influenced  the  “ceiling  effect”  introduced  by  these  scores  is  the  high  average  among  the 
students.  If  the  tests  were  longer  and  students  had  been  scoring  in  the  70%,  we  would  have  expected  to  see 
an  even  greater  range  of  effectiveness  defined. 


CBT  Benefits 

The  CBT  in  question  does  provide  some  benefits  when  measuring  interaction  time  metrics:  In  the 
study,  the  students  required  an  average  of  8.2  hours  to  complete  the  6 topics  in  the  CBT.  In  comparison, 
instructors  require  8.5  hours  to  teach  the  same  material  in  an  instructor-led  lecture  format.  One  would 
assume  that  the  CBT  seat  time  would  be  reduced  about  50-60%.  However,  given  the  restrictions  of  the 
imposed  time  schedule  this  was  not  seen.  One  would  assume  that  the  deployment  time  would  be  less  once  a 
new  scheduling  system  is  implemented  and  appropriate  time  values  are  added  to  the  scheduling  matrix  to 
accommodate  Computer-Based  Courses. 

Additionally,  there  was  a time-savings  in  terms  of  instructor  remediation  time.  A total  of  7 
students  required  remediation  by  an  instructor,  yielding  a total  of  approximately  1.5  man-hours  of 
instructor  time  for  student  instruction  (vice  the  nominal  8.5  hours  required  to  teach  the  material  plus  any 
time  spent  in  individual  tutoring).  For  the  beta  test.  Submarine  School  instructors  attended  all  training 
sessions,  but  it  is  envisioned  that  once  CBT  is  well  entrenched  in  the  SOBC  curriculum  and  students 
understand  how  to  use  it,  direct  instructor  supervision  will  not  be  required.  Rather,  CBT  will  be  employed 
to  teach  basic  theory  and  concepts,  and  instructor  led  seminars  synthesizing  material  and  covering  more 
advanced  concepts  will  be  used  to  periodically  ensure  students  have  satisfactorily  advanced  through  the 
material. 

On  its  face,  the  integration  of  Sonalysts’  tool  represents  a 59.3%  savings  in  development  time  over 
a typical  CBT  effort.  However,  some  elements  in  the  study  influence  this  apparent  savings: 

• The  Topic  Learning  Objectives  derived  from  course  Personal  Performance  Profiles  (PPP)  tables 
already  existed;  hence  the  course  was  not  built  “from  scratch.” 

• The  Submarine  School  staff  author  had  no  training  in  CBT  development.  While  this  is  a de 
rigueur  requirement  for  any  authoring  tool  fielded  for  Submarine  School,  it  is  also  true  that  those 
same  personnel  will  become  more  proficient  in  the  use  of  the  tool  as  more  CBT  is  authored. 

• A limited  amount  of  media  (3  graphics)  was  produced  in  support  of  this  effort.  For  the  remaining 
approximately  700  graphics  and  animations  used,  existing  media  and  or  local  generation  tools 
were  leveraged  for  the  media  production.  It  is  anticipated  that  during  a typical  CBT  authoring 
effort,  the  existing  media  bank  will  not  be  quite  as  well  populated. 

• The  purpose  of  the  study  was  also  to  test  InTrain  AUTHOR!  Several  software  problems  were 
identified  and  corrected  during  the  test.  This  resulted  in  an  aggregate  reduction  in  authoring 
capability  for  a period  of  about  one  week.  InTrain  has  completed  Beta  testing  and  made  changes, 
so  this  type  of  delay  should  no  longer  occur. 
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Conclusions 


The  entire  authoring  effort,  deployment  of  the  course,  and  post-CBT  analysis  spanned  3.5  months. 
Sonalysts,  provided  the  IT,  instructional  design,  and  subject  matter  expertise  in  an  effort  to  investigate  the 
usefulness  of  InTrain  CBT  in  an  established,  structured  U.S.  Navy  curriculum.  Submarine  School  provided 
exceptional  dedication  to  the  effort,  including  the  combined  work  of  the  CBT  author  and  testing  team. 

The  InTrain  CBT  test  provided  valuable  input  to  the  product  developers  at  Sonalysts,  both  in 
terms  of  performance  and  study  recommendations.  As  part  of  the  implementation  of  distance  learning 
efforts  at  NAVSUBSCOL,  Sonalysts  will  be  involved  with  the  integration  of  the  submarine  officer  training 
curricula  into  the  THINQ'*'^  LMS  (Training  Server).  This  initial  testing  allowed  the  software  engineering 
staff  at  Sonalysts  to  evaluate  the  requirements  and  use  patterns  of  CBT  (and  specifically  InTrain)  in  an 
enterprise  LAN  environment. 

As  part  of  the  population  of  the  LMS  with  content.  Submarine  School  must  develop  and  deploy  a 
robust  web-hosted  CBT  course  structure  to  allow  remote  users  access  to  training  prerequisites  and 
remediation  to  support  both  professional  development  and  continuing  training  onboard  submarines.  This 
test  provided  the  Submarine  School  staff  some  experience  in  CBT  design  and  provided  some  metrics  and 
recommendations  about  implementing  CBT  to  maximize  its  benefits  to  the  student  population. 


Summary 

Based  on  student  performance  and  feedback  received,  the  following  conclusions  can  be  drawn 
from  the  test: 

• The  benefits  of  implementing  adaptive  CBT  in  the  Submarine  School  range  from  possible 
timesavings  in  specific  curricula  to  automated  remediation  and  instruction.  Manpower  savings  will 
permit  Submarine  School  to  pursue  implementation  of  a Distance  Learning  support  division 
comprised  of  experienced  subject  matter  experts  who  have  instructional  expertise  in  front  of  a 
class  and  as  CBT  authors. 

• CBT  cannot  replace  the  instructor  completely.  Students  asserted  uniformly  that  the  face-to-face 
communications  were  at  times  critical  for  the  understanding  of  particularly  complex  concepts. 
While  not  applicable  to  the  material  delivery  in  the  Beta  test,  NAVSUBSCOL  is  planning  on 
maintaining  some  instructor-led  seminars  to  provide  higher-level  synthesis  of  read  material. 

• Training  via  CBT  provides  exceptional  flexibility  to  both  the  schoolhouse  (in  terms  of  resource 
scheduling)  and  to  the  student  (to  learn  at  his  convenience). 

The  potential  to  improve  training  efficiencies,  effectiveness,  and  achieve  Fleet  readiness 
improvements  with  the  infusion  of  advanced  training  methodology  and  technology  is  enormous  and  clearly 
measurable.  This  effort  is  not  envisioned  to  replace  the  resident  training  experience  but  to  enhance  the  level 
of  professional  development  that  can  be  achieved  and  to  provide  continuous  training  and  maintenance  of 
readiness  to  students  and  groups  ashore  or  at  sea.  A substantial  commitment  to  investing  in  information 
technology  infrastructure  has  been  made,  but  the  return  on  investment  will  not  be  realized  without  a 
concomitant  fielding  of  advanced  learning  methodology  products.  It  is  recognized  that  this  vision  in  its 
many  forms  and  variations  cannot  be  achieved  immediately;  the  first  small  steps  will  reflect  changes  in 
technology  and  processes. 


Historical  averages  are  based  on  exam  results  from  the  last  year. 

2 

The  written  examination  included  two  topics  that  were  not  developed  into  CBT  but  were  presented  to  the  students  in 
traditional  lecture  format.  The  average  for  those  two  questions  was  79%.  A corrected  average  for  performance  on  just 
the  InTrain  instructed  material  was  96.2%.  Removal  of  these  two  topics  from  the  other  tests  did  not  change  the 
historical  or  last  4 class  means  in  any  significant  fashion  (+  0.2%). 

Exam  failures  are  counted  in  the  student  mean  as  70%.  Remediation  and  re-exam  are  conducted  upon  exam  failure. 
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Collaborative  Integrated  Applications  for  Mathematics  and  Science  Content  and 
Pedagogy  Among  University  Faculty  and  Public  Schools 

El  Paso,  Texas  is  a border  community  with  a population  of  over  two  million 
people.  The  city  is  the  fifth  poorest  city  in  the  United  States  with  a minority 
(Hispanic)  population  of  about  78%.  A collaborative  effort  among  the  Colleges  of 
Science  and  Education  and  Public  Schools  was  sponsored  by  the  National 
Science  Foundation  (NSF)  to  improve  mathematics  and  science  education.  This 
paper  concludes  the  study  of  a six-year  project.  The  goals  of  the  project  were: 

1.  To  develop  a working  collaboration  among  Colleges  of  Science  and 
Education  and  Public  Schools 

2.  To  improve  pedagogical  applications  with  content  faculty  and  to  increase 
content  knowledge  with  pedagogical  faculty 

3.  To  bridge  the  gap  between  university  theoretical  learning  and  real  world 
practice  in  classrooms 

4.  To  develop  a model  of  teaching  that  produces  teachers  capable  of  teaching 
mathematics  and  science  to  “high  risk”  students 

No  one  would  argue  the  need  for  teachers  to  know  their  content  but  content 
alone  does  not  produce  the  results  needed  for  all  students  to  learn.  This 
paper  outlines  a four  step  model  of  teacher  training  based  on  theories  of 
Vygotsky  and  Piaget  and  the  research  resulting  from  the  Partners  for 
Excellence  in  Teacher  Education  (PETE),  an  NSF  initiative  project.  The  basic 
model  is  as  follows. 


Faculty  from  the  Colleges  of  Science  and  Education  team-teach  integrated 
courses  on-site  in  public  schools.  The  courses  are  1 . Block  I,  The  Property  of 
Real  Numbers,  Curriculum,  and  Mathematics  Pedagogy:  2.  Block  II,  Physical 
Science  and  Science  Pedagogy.  The  university  classes  are  taught  in  schools 
and  the  Block  Classes  include  a teaching  segment  in  which  the  university 
students  go  directly  from  class  to  a classroom  to  practice  teaching  the 
university  content  using  the  pedagogy.  University  faculty  and  public  school 
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teachers  provide  immediate  and  written  feedback  concerning  the  teaching 
segment. 

PETE  was  funded  in  1996  to  investigate  ways  to  improve  mathematics 
and  science  teaching  in  the  United  States.  The  first  step  was  to  develop  trust 
and  respect  between  faculty  in  the  Colleges  of  Science  and  Education.  Many 
hours  were  spent  discussing  theoretical  and  conceptual  beliefs  concerning 
learning.  Major  differences  in  the  culture  of  these  colleges  were  discovered 
concerning  learning  and  teaching.  Faculty  worked  to  identify  common  ground, 
agree  on  theoretical  approaches  to  learning,  and  determine  plans  to  combine 
courses  and  implement  an  experimental  approach  to  teacher  training.  The 
science  pedagogy  course  moved  to  the  school  site  in  1997.  Three  semesters 
later  the  Physics  department  joined  the  pedagogy  course  with  the  integrated 
Block  course.  Based  on  the  success  of  this  project  the  mathematical  block 
united  and  moved  to  a school  the  next  semester. 

Multiple  evaluation  has  been  used  during  the  course  of  the  experiment. 
Content  pre-  and  post-  tests,  attitudal  surveys,  video  analysis  of  teaching 
tapes  from  actual  classrooms,  and  cognitive  interviews  have  been  compiled 
and  analyzed.  ExCET  test  ( the  Texas  qualifying  exam  for  teachers)  scores 
have  been  compared. 

Implications  could  include  a new  model  for  training  teachers,  identifying 
the  process  for  successful  collaborations,  and  a research  design  for  teacher 
assessment.  Further  studies  of  this  model  will  be  expanded  to  other  classes 
at  UTEP  and  need  to  be  replicated  at  other  sites. 
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Introduction 

The  federal  flagship  project  “Virtual  University  of  Applied  Sciences  for  Technology,  Computer 
Sciences  and  Business  Administration”  (VFH)  predominantly  employs  “virtual”  teaching  methods  which 
require  a high  level  of  interaction.  Spontaneous,  informal  communication  is  emphasized,  and  is  indeed 
necessary  for  Computer  Supported  Cooperative  Learning  (CSCL).  Also,  a “virtual  community”  is  to  be 
formed  in  which  students  can  improve  their  communication  skills.  Awareness  and  cohesion  are  essential 
for  both,  CSCL  and  the  virtual  community,  and  their  importance  in  communication  will  be  explored, 
conceptualised  and  specified.  We  show  the  status  quo  and  possibilities  for  optimisation,  by  means  of  the 
evaluation  results  of  the  pilot  phases  of  the  VFH. 


Didactic  requirements  of  the  VFH  are  fulfilled  through  new  and  innovative  learning  forms.  These 
forms,  especially  CSCL,  require  a sufficient  level  of  interaction  (i.e.  cooperation,  communication, 
coordination).  Also,  the  cohesion  of  the  group  and,  fundamentally  the  conscious  awareness  of  other  group 
members  arc  important. 

Group-cohesion  and  awareness  are  not  only  relevant  for  the  CSCL-process.  Cooperative  processes 
in  the  VFH  should  not  be  limited  to  the  syllabus.  Under  the  view  of  lifelong  learning,  it  is  essential  to 
support  self-organised  learning  methods  within  the  framework  of  the  VFH.  An  example  is  a learning  group 
in  which  students  prepare  for  their  exams.  In  such  groups  informal  and  spontaneous  interaction  is  needed. 

Communication  without  a mentor  where  students  ‘chat’  about  predetermined  subjects  is  of 
paramount  importance  to  the  development  of  the  VFH  to  a virtual  community.  Rheingold  defines  a virtual 
community  as  “social  aggregations  that  emerge  from  the  Internet  when  enough  people  carry  on  public 
discussions  long  enough  and  with  sufficient  human  feeling  to  form  webs  of  personal  relationships  in 
cyberspace”  (Rheingold  2000).  This  should  also  pertain  to  the  VFH.  Without  sufficient  interaction  the  VFH 
remains  a largely  anonymous  learning  environment. 


The  quality  and  quantity  of  cooperation  depends  on  the  amount  of  cohesion  occurring  between 
those  cooperating.  This  cohesion  or  commitment  does  not  guarantee  a better  performance.  Nevertheless, 
intense  social  cohesion  within  a group  is  a great  motivating  factor.  Group  members  positively  perceive  this 
cohesion,  and  they  behave  in  a “group-sense”.  The  social  cohesion  perspective  focuses  on  the  fact  that 
students  want  one  another  to  succeed,  and  that  is  why  they  help  one  another  in  the  learning  process.  The 
usual  sanctions  controls  of  a fixed-group  norm  can  be  partially  avoided.  The  intensity  of  group-cohesion  is 
dependent  on  a multiplicity  of  factors,  like  awareness,  posibilities  of  interaction,  task,  group  size, 
homogeneity,  individual  factors  and  time  of  cooperation. 


Requests 


Cohesion 


But  the  explanation  of  the  learning  matter  relates  only  slightly,  or  has  no  impact  at  all  on  the 
learning  success  of  the  recipient.  The  advantages  lie  rather  clearly  on  the  person  who  gives  the 
explanations. 

Thus,  a structure  needs  to  be  created  which  makes  explanation  possible  for  all  group  members  — 
independently  from  group  status  and  individual  competence.  Within  this  context  the  main  starting  point  is 
the  creation  of  the  task,  if  designed  as  a jigsaw,  one  can  assume  that  explanatory  processes,  independent 
from  the  status,  are  to  be  stimulated. 

Awareness 


In  order  to  develop  social  obligations  and  responsibilities  within  group  cohesion,  interaction  must 
be  promoted.  The  basis  for  interaction  is  perception  of  the  participants  as  well  as  their  actions.  This 
“awareness”  is  fundamentally  essential  for  effective  performance  in  virtual  groups. 

Briefly  viewed,  awareness  enables  “///e  individual  to  group  the  situation  of  his/  her  current 
surroundings,  and  to  accordingly  adjust  his/her  actions"  (Pankoke-Babatz  1998).  In  the  area  of  CSCL,  this 
means  that  one  must  be  aware,  even  in  an  “electronic”  surrounding,  of  various  alterations  in  the 
atmosphere,  its  causes  and  effects,  the  actions  of  others  and  their  effect,  and  other  events  occurring. 
Awareness  is  summarized  as  understanding  of  the  activities  of  the  others  as  a context  for  its  own 
activities"  (Dourish  & Belotti  1992),  or  more  succinctly  ''knowing  what  is  going  on"  (Endsley  1995). 

What  is  essential  for  the  spontaneous,  informal  interaction  and  for  cohesion?  Primary  is  personal 
awareness  that  refers  to  a person.  This  comprises  synchronous  and  asynchronous  awareness  information. 
Asynchronous  awareness  information  is  attainable  for  example  through  a homepage  of  the  participants. 
Synchronous  awareness  information  can  be  sub-divided.  One  essential  aspect  is  the  transmission  of 
attention  focus,  e.g.  usage  of  a telepointer  on  a shared  whiteboard  enables  participants  to  follow  others 
actions.  The  other  aspect  is  the  ability  to  facilitate  contact.  An  example  is  the  information  about  the 
availability  of  the  participants  in  the  learning  environment. 

An  evaluation  result  of  the  pilot  phase  of  the  VFH  was,  that  the  main  problem  for  groups  was  the 
lack  of  synchronous  awareness  information  especially  the  missing  ease  of  making  contacts.  The  following 
remarks  were  expressed:  "I  can  V get  into  the  work  if  / don  7 know  who ’s  sitting  on  the  other  end  of  the 
computer"  The  result  was  a preponderating  asynchronous  communication.  The  students  were  unhappy 
about  "being  reliant,  that  the  others  constantly  check  E-mails,  news  or  discussions.  There  is  no  'wake  up ' 
alarm  per  telephone".  The  lack  of  contact  and  the  altogether  defective  personal  awareness  spawned  the 
following  remark:  "if  the  technical  and  personal  working  conditions  aren  7 clarified,  there  is  scarcely  a 
chance  to  cooperate." 

Spontaneous  communication,  which  is  an  important  success  factor  of  CSCL  and  the  establishment 
of  a virtual  community  in  the  VFH  is  not  possible  in  the  framework  of  externally  scheduled  ‘Chats’  and 
with  a learning  space  that  provides  asynchronous  group  communication  but  insufficient  awareness  support. 
Spontaneous  contacts  and  communication  must  be  improved.  This  requires,  in  addition  to  general  didactical 
reflections,  advanced  support  for  awareness,  especially  personal  awareness. 
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Abstract:  The  distribution  of  content  and  applications  over  the  Internet  is  one  of  the  most 
demanding  research  topics.  One  challenge  is  to  present  and  manage  interactive  multimedia 
services  in  different  networks  hosted  on  various  servers  with  diverse  operation  systems  and 
miscellaneous  system  configurations  and  conditions.  In  this  paper,  we  present  a delivery  server 
infrastructure  for  sophisticated  multimedia  content.  Comparable  with  the  Application  Service 
Providing  concept,  we  introduce  a platform  independent  framework  to  provide  interactive 
distance  learning  services.  The  aspects  of  scalability  and  load  balancing  are  a main  part  of  this 
approach  and  will  be  treated  respectively.  High  availability  and  security  are  also  mentioned. 


Introduction  and  Motivation 

On  the  threshold  of  the  information  society  to  the  knowledge  community  the  learning  and  training  habits 
have  to  be  transformed  to  meet  future  challenges.  Not  only  the  implementation  of  new  learning  methods  in 
traditional  learning  environments  but  also  the  necessity  for  lifelong  learning  increases  the  relevance  of  new 
information  technologies  for  teaching  and  learning  issues.  In  this  context,  several  projects  at  the  Institute  of 
Communication  Networks  at  the  Munich  University  of  Technology  (Germany)  deal  with  the  development  of 
new  techniques.  One  emphasis  is  the  implementation  of  the  infrastructure  because  the  supply  of  services  via 
networks  is  becoming  more  and  more  important. 

There  is  a lot  of  work  going  on  in  the  area  of  Application  Service  Providing  (ASP)  and  Content 
Management  (CM).  Other  institutes  and  laboratories  are  doing  research,  and  diverse  companies  offer  their 
hardware  or  software  solutions.  A good  overview  of  the  roles  and  issues  of  ASP  is  described  by  [Durlacher 
Research  (1999)]  including  some  economical  aspects.  About  content  and  knowledge  management  there  are  a 
multitude  of  sources  in  the  Internet,  see  also  “Contentworld”  (www.contentmanagement.com). 

From  the  technical  point  of  view,  we  differentiate  between  hardware  and  network  techniques  (e.g.  servers, 
switches)  offered  by  enterprises  like  Cisco  Systems,  Nortel  Networks,  Hewlett-Packard,  IBM  and  SUN 
Microsystems,  and  software  solutions  and  application  management  systems,  such  as  Microsoft,  Citrix  Systems, 
Oracle,  SAP,  Lotus  Development  and  so  on. 

At  universities  and  other  learning  centers  different  sorts  of  Web  portals  have  been  developed,  but  we 
consider  that  none  of  them  is  really  suitable  to  form  a modular  and  flexible  framework  to  provide  interactive 
distance  learning  services,  like  remote  simulation  services.  In  this  context,  it  should  be  noted  that  we  do  not  try 
to  compete  against  the  solutions  mentioned  above  by  implementing  another  software  suite.  We  rather  suggest  a 
new  tool  architecture,  which  allows  the  integration  of  existing  components  in  a more  flexible  manner.  To  our 
knowledge  none  of  the  developments  pursue  this  distributed  and  service-oriented  approach.  Though  we  consider 
a diversity  of  multimedia  services,  we  suggest  using  an  integrated  solution  of  centralized  server  system. 


Application  Service  Providing  for  Distance  Learning 

Also  in  the  educational  environment  effective  provisioning  of  distance  learning  services  is  an  important 
challenge.  Therefore,  we  first  discuss  what  the  expression  “ASP”  means  and  we  point  out  the  benefits  of  the 
ASP-concept  in  TeleEducation  scenarios. 

Application  Service  Providing  (ASP)  stands  for  the  management  and  the  distribution  of  software -based 
services  and  solutions  to  customers  across  a wide  area  network  from  a central  data  center.  Thereby  an 
application  server  is  frequently  viewed  as  part  of  a three-tier  application,  consisting  of  a graphical  user  interface 
server,  an  application  server  and  a database.  The  access  is  done  via  a web  browser.  Thus,  the  user  only  needs  a 
thin  client  to  execute  the  software.  High  availability  and  performance  of  the  applications  and  the  security  of  the 
user  data  are  some  of  the  advantages  the  ASP-concept  has  for  business  management.  Expense,  security  and 
reliability  may  be  some  reasons  to  implement  this  concept  at  universities  and  other  learning  centers.  For  offering 
content  and  applications  for  TeleEducation  issues,  we  distinguish  between  three  different  types  of  learning 
environments  in  the  Internet:  learning  portals,  e.g.  the  National  Technological  University  ^ww.ntu.edu). 
learning  platforms  (www.virtual-leaming.at)  and  enhanced  learning  delivery  server  infrastructures. 
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3IST  COPY  AVAIIABUS 


The  Delivery  Server  Infrastructure  “KIARA” 


At  our  institute  we  established  a server  system  KIARA,  which  offers  the  features  of  a network  portal,  like 
search  and  linking  functionalities  (Fig.  1).  The  framework  allows  not  only  the  ability  to  deliver  some  multimedia 
content  but  also  to  get  access  to  different  applications,  which  are  hosted  at  the  internal  network  computers.  A 
main  aspect  of  this  architecture  is  the  management  system,  which  deals  with  file  access,  simulation  demands  and 
security  requirements.  Together  with  its  billing  and  access  right  abilities,  the  management  system  presents  some 
sort  of  ASP  capabilities.  At  our  institute  the  system  is  used  to  support  simulation  programs  remotely  for  research 
work  and  educational  purposes. 
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Figure  1:  Scenario  of  a delivery  server  infrastructure 

Further  details  on  the  hardware  and  basic  software  of  the  architecture  are  described  in  [Merz,  Bor,  2000]. 


Application  Management  System 

The  outstanding  benefit  of  the  management  system  is  its  scalability.  Also  availability  and  mobility  are 
obtained  by  storing  relevant  information  in  a database  on  the  server-side.  One  main  task  of  the  server 
management  is  to  allocate  the  application  demands  to  different  computers  of  the  cluster  unit  for  better  load 
sharing.  When  the  user  starts  to  communicate  with  the  system,  the  user-management  checks  the  authentication. 
Subsequently,  the  user  gets  the  possibility  to  configure  the  parameters  for  a requested  application  with  the  help 
of  a graphical  user  interface  (GUI)  on  the  client-side.  Afterwards,  the  application  request  is  transmitted  to  the 
server.  In  addition  to  user-management  the  server  provides  a licensing  system.  With  the  help  of  a scheduling 
table,  the  applications  can  be  executed  in  a time -controlled  manner.  The  executions  of  different  individual 
applications  are  distributed  with  the  help  of  a load  balancing  system  on  different  cluster  computers  for  a better 
enhancement  of  computing  performance.  Further  details  about  the  implementation  of  the  system  are  described  in 
[Bor,  Jost,  2001]. 


Conclusion 

In  this  paper,  we  present  a delivery  server  infrastructure,  which  supports  not  only  standard  ASP-software,  but 
also  other  types  of  interactive  multimedia  applications,  like  sophisticated  distance  learning  services.  As  a result, 
the  system  has  to  provide  some  sort  of  management  architecture,  which  allows  user  management  and  other 
administrative  tasks,  for  example  billing.  Furthermore,  the  server  framework  fulfills  the  following  features: 

• High  scalability,  availability,  reliability,  security  and  support  of  mobility  aspects 

• Good  performance  with  regard  to  transfer  rate,  number  of  clients,  bandwidth 

• Integration  of  various  applications  (audio/video,  text,  graphics,  interactive  application  programs,  games...) 
Although  a prototypical  implementation  of  the  system  has  been  developed,  much  needs  to  be  done  before  the 
framework  can  be  applied  in  a commercial  environment.  Further  research  should  be  done  on  the  integration  of 
other  time -sensitive  applications,  for  example  interactive  games,  and  performance  aspects  have  to  be  evaluated. 

References 

Durlacher  Research  (1999)  Application  Service  Providers,  A Report  by  Durlacher  Research. 

Merz,  T.  & Bor,  A.  (2000).  Server  Concept  for  a User  Driven  Management  of  Networked  Multimedia  Application. 
In  Proceedings  Eunice  2000,  6th  Open  European  Summer  School  on  Innovative  Internet  Applications,  Twente,  Netherlands. 

Bor,  A.  & Jost,  J.  (2001).  A new  Framework  for  Enhanced  Application  Service  Prov/c//ng.InPnoceedingsSoftCOM2001, 
International  Conference  on  Software,  Telecommunications  & Computer  Networks,  Split  (Croatia). 


217 


COW  AVAILABUE 


A Services-Based  Architectural  Model  for  E-learning 


Dimiter  Bogdanov 
Software  Engineering  Department 
Institute  of  Computer  & Communication  Systems 
Bulgaria 

bogdanov@iccs.bas.bg 


Abstract:  A multi-model  services- based  approach  for  modeling  the  reach  functionality  of  a 
learning  process  is  presented.  It  allows  the  fixnctionality  to  be  split  in  different  layers  that  are 
functionally  orthogonal.  The  handling  of  activities  over  each  layer  is  mapped  to  a suite  of 
protocols.  This  way  a not  purely  hierarchical  decomposition  of  functionality  is  achieved  and  it 
is  closer  bound  to  distributive  nature  of  learning  environments.  Explored  generic  Streaming 
Learning  Architectural  Model  is  appropriate  to  cope  with  dynamic  tasks  in  learning  as  are  the 
bi-directional  interactions  and  messaging,  collaborative  learning,  learning  forums  and 
presentations  in  real-time. 


Introduction 

The  major  o learning  initiatives  (ADL  2001,  IEEE  2001,  IMS  2001)  model  the  e-leaming  via  a set  of 
end-users  requirements  and  try  to  map  them  onto  existing  digital  technologies  that  are  represented  by  Web 
paradigm  mainly.  They  are  concentrating  on  learning  content  delivering  first  and  foremost.  In  fact,  the  education 
process  is  more  complex  and  needs  more  than  the  Web  paradigm  - and  HTTP  protocol  behind  it  - could  offer. 
The  complexity  of  oleaming  is  arising  from  requirements  on  dynamism  and  entity  semantics  handling.  The 
Web,  even  with  its  extensions  as  DHTML  and  scripting  models,  is  not  capable  to  map  all  functionalities  that 
may  occur  in  a collaborating  learners  class  - bi-directional  interactivity,  events  handling,  learner  behavior 
tracing  and  steering,  system  state  preserving,  etc.  The  Web  is  also  suffering  on  lack  of  mechanisms  for  entities 
semantic  handling.  In  our  understanding  the  Web  is  just  one  of  possible  handlers  that  acts  in  a client/server 
model.  The  investigated  architectural  model  is  assuming  that  other  handlers  in  a client/server  environment  could 
be  used  also  and  other  than  client/server  paradigms  could  be  applied  like  P2P  model  for  instance. 


The  service^based  architectural  model 

The  e-leaming  architectural  model  under  consideration  is  not  assumed  to  be  Web-centric  one,  although 
the  Web  is  one  of  the  major  handlers  used  for  content  delivering  and  rendering  of  information  still.  The 
presented  architectural  model  is  built  up  around  two  main  ideas.  The  first  one  is  that  learning  is  modeled  after  a 
set  of  services  representing  some  pieces  of  functionalities  from  a subject  perspective.  On  the  other  hand,  from  an 
IT  perspective,  the  services  are  small  entities  that  could  be  transferred  over  the  network  and  programmatically 
handled.  The  second  e-leaming  model  idea  focuses  on  what  framework  is  needed  for  services  processing  and 
handling.  A protocol  framework  is  proposed  for  services  handling  over  the  distributed  learning  environment.  The 
protocol  framework  is  a suite  of  protocols  each  of  which  is  responsible  for  a primary  class  of  services.  The 
protocols  are  acting  in  layers  as  is  shown  in  the  illustrative  layers  decomposition  scheme  (Fig.  1). 

The  learning  activities  are  time-framed  as  they  are  to  be  performed  in  a finite  interval  in  general. 
Performing  of  the  learning  activities  is  usually  related  with  transition  through  different  states  and  an  initial  and 
end  state  of  this  process  could  be  differentiated.  In  most  cases  the  time  for  activities  is  not  crucial  and  it  is 
important  to  follow  a pro-programmable  sequence  of  steps.  That  is  why  we  are  viewing  the  learning  activities  as 
sequence-framed.  The  execution  of  a learning  sequence  is  considered  as  a stream  of  interchanges  that  are  behind 
the  sequencing.  The  carrier  of  encoded  expertise  is  called  stream.  The  stream  could  be  a peer-to-peer 
communication,  a multicast  or  a broadcast.  A stream  as  a logical  production  consists  of:  learning  services, 
instructions  and  a set  of  supervising  agents  all  organized  as  a scenario,  which  is  to  be  performed  within  a session. 
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The  logical  structure  of  a stream  is  embedded  in  a stack  of  layers  represented  by  XML  technologies  (XML 
2000).  The  functionalities  of  these  layers  are  illustrated  in  (Fig.l).  The  investigated  eleaming  architectural 
model  is  consequently  named  Streaming  Learning  Architecture  Model  (SLAM). 
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Figure  1:  Streaming  Learning  Architecture  Model 

The  sequencing  could  be  triggered  and/or  redirected  on  the  base  of  an  event  model.  The  streaming  layer 
protocol  is  responsible  for  monitoring  of  the  sequencing  of  learning  content.  Further  it  is  supposed  that  the 
stream  is  carrying  not  directly  the  learning  objects  but  aggregated  entities  called  services.  This  way  the  learning 
is  presented  by  a set  of  services.  All  learning  activities  could  be  decomposed  into  sub-groups  like  services, 
sequencing,  events,  etc.  and  these  sub-groups  form  functionally  orthogonal  layers.  A protocol  is  responsible  for 
each  layer  realization  that  technically  could  be  implemented  as  part  of  other  layer  protocol.  As  basic  protocol 
could  be  considered  the  stream  protocol  but  it  is  also  embedded  in  two  other  layers  - session  and  scenario  layers. 


Conclusions 

A multi- model  services-based  approach  for  modeling  the  reach  functionality  of  a learning  process  is 
presented.  It  allows  the  functionality  to  be  split  in  different  layers  that  are  functionally  orthogonal.  The  handling 
of  activities  over  each  layer  is  mjpped  to  a suite  of  protocols.  This  way  a not  purely  hierarchical  decomposition 
of  functionality  is  achieved  and  it  is  closer  bound  to  distributive  nature  of  learning  environments. 

Explored  generic  SLAM  model  is  appropriate  to  cope  with  dynamic  tasks  in  learning  as  are  the  bi- 
directional interactions  and  messaging,  collaborative  learning,  learning  forums  etc.  The  anticipated  base  for 
model  implementations  is  widely  accepted  and  agreed  for  standardization  XML  platform. 

The  presented  reference  eleaming  model  has  been  used  for  development  of  a set  of  specifications 
(Bogdanov  2001)  defining  a unified  methodology  for  designing  of  distributed  learning  environments.  The 
specifications  are  described  in  an  IEEE  LTSC  format. 
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E-Learning  in  College  and  Corporate  Settings:  The  Present  State  and  Beyond 

Abstract: 

Who  is  making  the  decisions  about  ©learning?  What  tools,  resources,  and  courseware  do  college 
instructors  and  corporate  trainers  favor?  And  what  are  some  common  experiences  and  trends  in  © 
learning?  Based  on  two  extensive  studies  in  the  college  and  corporate  markets,  this  session  will  document 
the  incentives,  support  structures,  skills,  tool  preferences,  and  experiences  of  both  college  faculty  and 
corporate  trainers.  In  effect,  the  presenter  will  summarize  two  comprehensive  survey  reports,  one  on  e- 
leaming  in  corporate  training  and  one  on  e- learning  in  higher  education. 

Introduction: 

During  the  past  few  years,  it  seems  that  every  technology-related  newsletter,  magazine,  conference,  and 
institute  has  expressed  at  least  some  an  interest  in  e-leaming.  Despite  the  high  interest,  countless 
questions  surround  this  new  training  delivery  mechanism.  Are  course  completion  rates  as  well  as  costs 
higher  or  lower  than  in  conventional  classroom  training?  What  types  of  tools  and  instructional  techniques 
are  most  prominent  on  the  Web?  And  what  evaluation  methods  are  valued  and  increasingly  common 
here?  In  effect,  just  why  are  different  firms  and  organizations  interested  in  placing  their  training 
programs  on  the  Web  and  how  are  they  allocating  resources  to  support  it?  And  what  about  the  emergence 
of  the  cyber  university  and  online  courses?  What  are  college  faculty  experiences,  preferences,  and 
attitudes  toward  e-leaming? 

In  attempting  to  answering  some  of  these  questions,  the  presenter  has  just  completed  two  fairly 
comprehensive  reports,  “Online  Training  in  an  Online  World,”  and  “Online  Teaching  in  an  Online 
World,”  co-sponsored  by  CourseShare.com  and  Jones  Knowledge,  Inc.  (see: 

http://www.publicationshare.com/  or  http://www.ionesknowledge.com/).  These  free  reports  (as  well  as 
briefer  summary  reports)  detail  the  e-leaming  attitudes  and  preferences  of  college  faculty  as  well  as 
corporate  trainers,  training  managers,  and  other  human  resource  personnel. 

These  reports  have  many  goals,  including: 

• Assessing  the  e-leaming  resources  and  tools  that  college  insfructors  trainers  currently  use  as  well 
as  desire, 

• Documenting  reasons  for  sharing  course  materials  and  resources  on  the  Web, 

• Determining  the  types  of  online  learning  tools  currently  used  as  well  as  needed, 

• Discerning  some  of  the  pedagogical  practices  and  motivational  techniques  supported  by  e- 
leaming, 

• Revealing  the  support  sfructures  for  online  learners,  insfructors,  trainers,  and  course  designers,  and 

• Pointing  to  the  present  state  of  e-leaming  as  well  as  future  trends  and  directions. 

Importantly,  these  report  addresses  emerging  topics  such  as  online  learning  communities,  reusable 
learning  objects,  and  freelance  insfruction.  In  addition,  there  are  findings  and  recommendations 
concerning  e-leaming  tool  development,  time,  organizational  support,  content  development  and 
outsourcing,  and  evaluation  and  assessment. 

Across  the  findings,  it  is  apparent  that  the  Web  is  flourishing  as  a teaching  and  training  delivery 
mechanism.  As  one  might  expect,  the  most  common  forms  of  online  training  are  computer  applications. 
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technical  skills,  and  job- related  skills.  In  higher  education,  online  training  is  more  varied.  In  stride  with 
recent  reports  on  e-leaming,  respondent  organizations  tended  to  rely  on  blended  or  hybrid  approaches 
wherein  Web-based  training  supplemented  and,  hopefully,  enhanced  face-to-face  instruction. 

Despite  the  explosion  of  interest,  the  survey  respondents  noted  significant  organizational  and  cultural 
barriers  to  e-learning  including  perceptions  of  high  cost  and  extensive  instructor  preparation  time.  In 
addition,  there  were  technological  problems  related  to  bandwidth,  firewalls,  and  limited  technical  support. 
Respondents  also  alluded  to  a need  for  more  innovative  e-learning  tools  that  provide  interactive 
feedback,  multiple  forms  of  collaboration,  and  enhanced  learner  evaluation  and  assessment.  In  terms  of 
the  latter,  some  firms  are  experimenting  with  alternative  forms  of  evaluation  that  extend  beyond  the  first 
two  or  three  levels  of  the  Kirkpatrick  framework  (i.e.,  reaction,  learner  achievement,  and  job 
performance).  Nevertheless,  when  it  comes  to  formal  e-learning  assessment,  few  instructors  and 
institutions  have  made  much  progress. 

Across  the  findings,  there  also  appeared  to  be  a need  to  train  instructors  and  instructional  designers  in 
Web-based  instructional  approaches  and  opportunities.  As  this  new  learning  format  gains  in  reliability, 
acceptability,  and  interactivity,  support  structures  are  necessary  for  those  building  and  refining  their 
online  courses,  those  administering  and  delivering  them,  and  those  taking  them.  In  fact,  innovative 
portals  might  provide  expert  guidance  on  purchasing  decisions  and  vendor  selection,  content 
development,  and  the  implementation  of  e-learning  systems.  And  when  high  quality  courses  are 
developed,  leased,  or  purchased,  they  need  to  be  promoted  by  the  university,  company,  or  organization. 

In  the  end,  successful  online  training  requires  comprehensive  support  programs;  if  one  aspect  is 
nonfunctional  (e.g.,  lack  of  student  or  employee  access  to  e-learning),  the  new  system  will  not  succeed. 
Support  might  be  live  as  well  as  online  and  intrinsic  as  well  as  extrinsic.  The  two  reports  will  hopefully 
help  sort  out  the  choices  and  resolve  some  of  the  complexity.  Specific  information  on  these  reports  is 
provided  below. 


Report  1.  "ONLINE  TEACHING  IN  AN  ONLINE  WORLD  " 

CourseShare.com  and  JonesKnowledge  announce  the  first  of  a series  of  joint  survey  research  reports 
related  to  the  use  of  the  Internet  in  teaching  and  learning.  This  comprehensive  report,  "Online  Teaching  in 
an  Online  World,"  is  available  for  free  (see  http://publicationshare.com/  or 

http://www.jonesknowledge.com/higher/index.php).  The  75  page  report  addresses  the  use  of  the  Internet 
by  222  college  instructors,  from  both  public  and  private  institutions,  who  were  early  adopters  of  the  Web. 
Instead  of  randomly  surveying  college  instructors  about  their  Web-based  teaching  needs  and  experiences, 
this  research  targeted  those  with  some  experience  in  using  the  Web  as  a teaching  and  learning  resource, 
either  in  the  World  Lecture  Hall  or  in  MERLOT.org.  Interesting  findings  are  reported  related  to  freelance 
instruction,  student  course  attrition,  and  e-leaming  decision-making.  Several  key  implications  regarding 
instmctor  training,  support,  and  resource  exchange  are  also  provided.  In  addition,  there  are 
recommendations  concerning  online  learning  policies,  pedagogy,  research,  and  comseware  development. 

• Bonk,  C.  J.  (2001,  May).  Online  teaching  in  an  online  world.  Bloomington,  IN: 

ComseShare.com.  Retrieved  March  12,  2002  from:  PublicationShare.com  and 
http://www.jonesknowledge.com/higher/index.php. 

Note:  The  Executive  Summary  of  this  report  was  published  in  January  2002  in  USDLA 
Journal.  http://www.usdla.org/html/joumal/JAN02_Issue/article02.html. 

Report  2,  “ONLINE  TRAINING  IN  AN  ONLINE  WORLD” 

This  January  2002  report,  "Online  Training  in  an  Online  World"  is  available  for  free  at: 
http://publicationshare.com/  or  http : //www.  i onesknowledge . com/com  orate/index . php . The  143  page 
document  (as  well  as  a briefer  summary  report)  details  the  e-leaming  attitudes  and  preferences  of  201 
trainers,  training  managers,  and  other  human  resource  personnel.  The  final  report  explores  a number  of  e- 
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learning  issues  and  future  trends,  such  as  the  types  of  training  offered  online,  the  range  of  organizational 
commitment  to  e- learning,  and  the  cultural  and  organizational  factors  limiting  the  adoption  of  e- learning. 
The  report  also  addresses  e- learning  content  development  as  well  as  outsourcing,  instructional  practices, 
motivation  techniques  and  incentives,  learner  and  instructor  support  mechanisms,  and  tool  growth  areas. 
Several  key  implications  are  noted  related  to  the  development  of  online  trainer  communities,  reusable 
learning  objects,  e- learning  partnerships,  and  freelance  instruction.  In  addition,  there  are 
recommendations  concerning  tool  and  content  development,  pedagogical  practices,  organizational 
promotion  and  support  policies,  evaluation  and  assessment,  and  next  steps  in  research  related  to  e- 
leaming. 

• Bonk,  C.  J.  (2002,  January).  Online  training  in  an  online  world.  Bloomington,  IN: 
CourseShare.com.  Retrieved  March  12,  2002  from:  PublicationShare.com  and 
http  ://www.  i onesknowledge  .com/  corporate/index,  php  ■ 

Note:  The  Executive  Summary  of  this  report  was  published  in  March  2002  in  USDLA 
(United  States  Distance  Learning  Association)  Journal. 
http://www.usdla.org/html/joumal/MAR02_Issue/article02.html 
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An  object  based  system  for  developing  and  delivering  e-learning  content 
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Abstract:  The  Web  as  a medium  for  the  delivery  of  learning  content  is  totally  different  from 
printed  text.  Some  of  the  major  differences  are  in  the  way  content  is  created,  formatted  and 
presented  to  learners.  If  traditional  textbooks  and  study  guides  are  simply  converted  to  Web  format 
using  for  example  a word  processor,  we  are  misusing  this  new  medium.  In  a Web  based  learning 
environment  content  should  be  fragmented  into  smaller  chunks.  There  are  many  reasons  for  this 
fragmentation:  readability,  manageability,  relevancy,  etc.  The  fragmentation  must  be  part  of  the 
content  writing  process  and  cannot  be  fabricated  afterwards.  We  believe  the  fragmentation, 
structuring  and  linking  of  Web  based  content  are  very  closely  coupled  to  the  content  creation 
process  itself.  A LCMS  was  developed  to  address  certain  content  authoring  requirements. 


Educational  Issues 

We  are  used  to  a sequential  world:  television  and  radio  are  all  examples  of  pre-defmed  sequences  of  information. 
Although  not  constructed  in  a linear  fashion,  the  result  is  a linear  sequence  of  information  blocks  that  cannot  be  re-arranged.  The 
same  is  true  for  printed  text.  Physically  a book  starts  at  page  one  and  is  usually  read  in  a linear  fashion  to  the  end.  These 
sequences  are  fixed  for  all  the  readers.  It  is  therefore  “normal”  to  think  that  we  should  create  and  deliver  E- learning  content  in  the 
same  linear  fashion. 

Information  technology  is  making  authors  from  all  of  us:  it  is  quick  and  easy,  and  therefore  I am  an  author.  To  make 
things  even  worse,  authors  and  instructional  designers  are  given  traditional  authoring  tools  like  Word  processors  to  create  web- 
based  content.  Although  tools  like  Dreamweaver  and  Frontpage  are  available,  they  are  much  too  complex  for  most  lectures.  New 
software  is  required  to  enable  structured  and  controlled  content  creation  within  a virtual  learning  environment  by  all  team 
members  (graphic  designer,  instructional  designers,  lectures,  administrators).  We  strongly  believe  that  Web  based  content  should 
be  developed  in  the  same  environment  where  it  will  be  delivered.  It  should  also  be  done  by  the  lecturers  themselves.  If  not,  they 
should  at  least  be  able  to  update  content.  The  authoring  tools  in  the  Varsite  Learning  Content  Management  System  (LCMS)  can 
address  a lot  of  these  issues  through  the  implementation  of  learning  objects  in  an  object  based  system. 


Previous  work  and  background 

Until  recently,  faculty  (authors)  at  the  PU  for  CHE  made  use  of  a Word  processor  to  create  web-based  study  content. 
Authors  were  given  a template  consisting  of  the  required  headings  and  sub-headings.  This  template  contained  the  structured 
levels  of  the  study  guide.  The  structuring  is  important  and  must  be  done  before  any  writing  commences.  A conversion  is  then 
made  from  the  linear  document  to  a web  document  using  a commercial  product.  The  converter  generates  an  HTML  document 
for  each  heading  level  of  the  original  document.  It  also  generates  a tree-view  table  of  contents  for  easy  navigation  with 
expandable  sections. 

This  process  worked  quite  well,  except  for  one  problem,  the  authors  never  get  to  know  the  medium  that  they  are 
actually  writing  for.  Even  though  the  instructional  design  is  for  the  Web,  authors  still  tend  to  write  in  a style  that  is  for  printed 
text.  Further  more,  they  could  only  see  their  results  after  each  conversion  made  by  Academic  Support  Services. 


The  Varsite  LCMS 

The  Varsite  system  is  a so-called  Learning  Content  Management  System  (LCMS).  It  differs  from  the  traditional  LMS 
in  that  it  also  includes  an  integrated  content  authoring  system  directly  linked  to  a central  database.  All  learning  content  is  treated 
as  learning  objects  with  properties  or  attributes.  Custom  properties  can  be  added  to  the  standard  properties  according  to  specific 
standards  e.g.  IMS. 

The  system  was  not  only  developed  to  address  the  educational  needs  mentioned  in  this  paper.  After  a thorough 
evaluation  of  a few  learning  management  systems  in  1 998,  the  university  decided  to  build  its  own.  The  reasons  are  beyond  the 
scope  of  this  paper.  But  it  is  worth  mentioning  that  critical  requirements  like  enterprise  level  integration,  integrated  authoring 
and  offline  functionality  was  key  factors  in  the  decision. 

The  object-based  design  forms  the  basis  of  the  complete  system.  Therefore  all  data,  whether  learning  content,  user 
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information,  curricula  information  or  management  information,  are  stored  as  objects  in  the  central  data  store. 


Content  management  and  authoring 

Our  requirements  for  web  based  authoring  are  stated  below.  All  of  these  objectives  were  met  by  implementing  the  Varsite 
system.  It  must  be  kept  in  mind  that  we  believe  lectures  must  eventually  be  able  to  create  their  own  Web-based  content.  This 
will  mostly  be  used  to  enrich  contact  education  on  campus. 

Objective  one:  Authoring  in  the  medium 

The  main  objective  here  is  to  enable  authors  to  develop  web-based  content  in  the  same  medium  in  which  it  is  delivered,  but 
without  the  technical  knowledge  that  is  usually  required  to  do  this.  This  was  achieved  through  the  implementation  of  a full-text 
Web-based  word  processor.  This  enables  full  text  formatting  inside  the  Internet  Explorer  web  browser.  The  content  is  saved 
directly  to  the  object  store  on  the  Varsite  server.  Text  from  other  Windows  based  applications  can  be  copied  and  pasted  into  this 
editor.  When  a user  is  logged  on  as  an  author,  an  edit  icon  will  appear  next  to  all  content. 

Objective  two;  Fragmented  authoring 

The  system  enables  the  hierarchical  creation  of  topics  or  pages  to  produce  study  guides.  Each  page  is  treated  as  an  individual 
piece  of  content  in  the  sense  that  it  is  individually  seen  and  edited.  There  is  no  linear  editable  document.  In  a sense  this  enforces 
the  basic  design  rules  for  the  web  for  example  manageable  chunks,  standalone  content,  single  page  topics  etc.  It  also  enables 
authors  to  focus  on  the  specific  topic  without  the  distraction  of  other  surrounding  content,  as  would  be  the  case  with  a traditional 
word  processor.  Authors  see  their  content  exactly  as  the  learners  would  see  it. 

Objective  three:  Tagging  en  describing  content 

Since  the  object -based  (and  object  orientated)  model  was  implemented  as  a technical  requirement  for  the  whole  system,  the 
implementation  of  learning  objects  came  natural  and  easy.  Some  basic  object  properties  were  implemented  and  these  will  later  be 
extended  to  include  some  of  the  standards  for  example  IMS.  Each  object  is  uniquely  defined  by  a human  readable  name,  in  the 
same  way  as  folder  paths  in  the  Windows  environment.  For  example  the  object  called 
RINLl  H:\BOOKS\COMPUTER_SKILLS\UNIT3\INTRO  is  an  introduction  page  of  study  unit  3 in  the  computer  skills  study 
guide  in  the  RINLl  1 1 course. 

Objective  four:  Storing  all  information  in  a central  database. 

Institutions  cannot  afford  to  have  their  intellectual  property  distributed  as  files  on  hard  disks,  network  drives  and  web  servers.  A 
Central  database  is  a must  for  backup,  security,  copyright  issues,  property  rights,  re-use,  searching  etc.  This  repository  will 
eventually  enable  the  creation  of  brand  new  modules  from  existing  content. 


About  learning  objects 

Potentially,  the  implementation  of  learning  objects  has  a lot  of  advantages  for  authors,  learners  and  administrators.  Some  of  these 
are  already  visible  as  indicated  in  the  content  design  objectives.  As  time  goes  by  and  the  need  arises,  we  will  implement  it 
further.  Future  development  will  include  (a)  real  reusability  between  lecturers,  university  departments  and  between  universities, 
(b)  adaptive  learning  paths  for  study  guides  and  assessment  and  (c)  custom  study  guides  according  the  learner  profiles. 


Results  and  feedback  from  authors 

In  2001  a complete  Web-based  course  was  developed  over  a period  of  about  6 months.  Once  the  structure  of  the  course  content 
was  designed,  each  of  the  four  authors  was  able  to  create  their  own  content  simultaneously  in  the  system.  This  shortened  the 
development  time  drastically.  Since  the  developed  content  was  immediately  visible  on  the  Web,  the  project  leader  and  quality 
control  members  could  monitor  the  project  continuously.  The  overall  feedback  from  the  authors  was  positive. 
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Abstract:  This  paper  is  a report  on  the  findings  of  a two  years  study  conducted  on  the 
undergraduate  course  ‘Teaching  and  Music  Technology”.  First,  the  components  of  the  WebCT 
content  developed  for  the  first  year  of  use  are  described.  Then,  the  results  of  the  assessment 
by  the  students  of  the  usefulness  of  each  of  these  components  are  presented.  Findings  indicate 
that  many  where  not  assessed  as  useful.  The  redesign  of  the  whole  components  is  explained 
and  the  second  year  WebCT  components  are  described.  Again,  the  results  of  the  assessment 
by  the  students  of  the  usefulness  of  each  of  the  components  are  presented.  The  results  allow 
us  to  provide  more  meaningful  definitions  of  several  design  criteria  for  Web-based  courses 
proposed  in  the  literature. 


Introduction 

“WebCT  (short  for  Web  Course  Tools)  is  an  integrated  set  of  tools  for  developing  and  delivering 
interactive  course  or  course  components  over  the  Web”  (Friesen,  2000,  p.3).  The  lack  of  equipments  available  to 
the  students  forced  the  professor  to  use  WebCT  to  deliver  the  course  components  of  the  undergraduate  course 
MUS-19673  Teaching  and  Music  Technology,  instead  of  developing  a fully  interactive  course.  As  the  author 
was  aware  that  the  adding  of  a Web  tool  does  not  mean  any  enhancement  of  the  course  because,  as  mentioned 
by  Heines  (2000),  “...  the  determination  of  what  really  works  is  so  difficult”,  she  used  what  we  will  call  several 
“content  prescriptions”  found  in  the  literature  (such  as  Dwyer  and  Baker,  2000;  Friesen,  2000).  She  also 
integrated  what  was  said,  during  the  so  numerous  sessions  and  workshops  she  followed,  as  being  “obviously 
helpful  even  if  not  empirical  evidence  found”. 


The  First  Year  Study 

There  were  30  students  using  the  first  release  of  the  course  “MUS-19673  Teaching  and  Music 
Technology”.  The  following  are  criteria  found  in  the  literature.  As  can  be  seen,  they  refer  to  what  Gagne  (1977) 
calls  a defined  concept.  The  main  problem  is  that  their  definitions  are  not  given: 

The  pictorial  information  is  presented  clearly. 

The  courseware  provides  information  that  accurately  represents  the  topic. 

The  courseware  screen  layout  is  easy  to  understand. 

The  courseware  content  is  presented  clearly. 

The  text  information  is  presented  clearly. 

The  graphical  information  is  presented  clearly. 


Findings  From  the  First  Year  Study 

In  accordance  with  the  first  criteria  listed,  and  because  the  professor  wanted  to  have  icons  nearest  of 
the  students  interest,  being  the  music,  the  icons  used  in  the  main  window  were  “musical”  ones.  This  design 
choice  was  assessed  by  the  student  as  being  a very  important  one.  The  different  parts  of  the  course  notes,  the 
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course  plan,  the  description  of  the  exercises  to  perform,  etc.,  were  put  on  the  web.  Students  found  it  was  not 
convenient  because  as  they  are  often  on  the  road  to  present  concerts,  they  don’t  have  any  access  to  computer 
equipments.  Students  did  not  like  the  “book-like”  table  of  contents  presentation  of  what  will  be  cover  during 
each  meeting  with  the  group.  They  would  have  preferred  a more  “mnemonic”  style.  Students  did  not  like  at  all 
the  quizz  options  mostly  because  it  did  not  correspond  to  anything  they  will  have  to  use  later  in  their  career. 
They  also  found  the  email  option  integrated  within  WebCT  not  pertaining.  They  all  wanted  to  keep  their  current 
email  account  and  did  not  even  check  their  mail  inside  WebCT.  Students  suggested  to  put  several  banks  of 
examples  and  counterexamples  that  would  complement,  in  a dynamic  way,  several  aspects  of  the  course  content 
(For  example,  the  principles  of  interface  design  or  evaluation  of  a CAI  tool  in  music). 


The  Second  Year  Study 

There  were  26  students  using  the  second  release  of  the  course.  As  suggested,  we  added  a large  amount 
of  examples,  analogies,  simulations,  etc.  We  suppressed  all  the  features  students  found  not  useful.  We  replaced 
the  wordy  table  of  contents  by  mnemonic  one. 


Findings  From  the  Second  Year  Study 

The  adding  of  examples  was  assessed  as  being  very  useful.  For  example,  in  the  course  notebook,  there 
is  a section  dedicated  to  the  choice  of  colours  for  the  interface  of  any  computer  product.  Several  recommended 
combinations  are  listed,  i.e.,  the  ones  that  does  not  cause  damage  to  the  retinal  of  the  eyes.  Visual  examples  of 
each  combination  mentioned  in  the  notebook  can  be  seen  on  WebCT.  Another  example  is  as  follows:  within  the 
course  notebook,  the  students  learn  that  8%  of  men  and  4%  of  women  have  difficulties  with  viewing  colours 
(colour  blind)  so,  the  learners  is  at  risk  to  not  see  the  very  important  infonnation  if  it  is  coded  with  colours. 
Examples  of  screens  that  uses  colour  to  present  very  important  information  are  shown. 


Conclusions 

In  this  paper  we  have  explained  how  a two  years  experimentation  of  our  WebCT  allows  us  to  refine  the 
definition  of  words  such  as  “accurately”,  “clearly”,  and  “easy”  used  in  the  literature  when  stating  criteria  of  Web 
design.  At  first,  when  answering  to  the  question  “Does  the  courseware  provide  information  that  accurately 
represent  the  topic?”,  we  were  believing  that  linking  the  course  notebook  content  to  the  table  presenting  what 
will  be  done  during  each  unit  was  accurate.  It  was  not  find  accurate  by  the  students.  It  would  rather  be 
something  that  shows  examples  of  what  can  be  read  in  a printed  course  notebook,  i.e.,  something  dynamic.  In  the 
same  way,  we  modified  the  meaning  we  first  gave  at  the  criteria  “The  text  information  is  presented  clearly”.  We 
found  that  students  do  not  want  to  have  wordy  titles,  but  more  mnemonic  words  that  aim  at  helping  them  to 
make  the  link  with  the  printed  course  notebook. 
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Abstract:  The  aim  of  this  paper  is  to  present  a tool  designed  to  help  teachers  (or  other  users)  to 
describe  their  learning  objects  with  IEEE  Learning  Object  Metadata.  This  semi-automatic  tool 
proposes  default  values  based  on  the  user  profile  for  some  descriptors  and  uses  relations 
between  documents  to  infer  other  descriptors. 


Introduction 

Metadata  are  necessary  for  retrieving  learning  resources  and  RDF  is  a good  way  to  implement  them,  as  we  said 
in  [3]  and  [4]. 

At  the  moment  a learning  object  is  created,  it  must  be  described  by  metada  (in  the  same  way  as  a programmer 
comments  the  code  that  he  writes,  and  if  he  does  not  do  that  immediately,  he  will  never  do  it).  Existing  learning 
objects  (with  many  possible  file  formats  like  pdf,  word,  power-point,  java  programs,  latex,  html  and  so  on)  have  also  to 
be  described  if  we  want  to  retrieve  them.  But  an  important  problem  is  that,  if  we  consider  the  IEEE  LOM  set  of  meta- 
data, for  example,  we  find  about  80  descriptors.  Some  of  them  may  be  produced  automatically  for  new  learning 
objects  (like  "general. title")  or  for  old  ones  (like  "technical. format")  but  there  are  so  few  of  them  that  the  time  spent  to 
describe  a learning  object  may  be  discouraging. 

We  propose  a solution  (a  semi-automatic  tool)  based  on  both  the  profile  of  the  person  indexing  resources  and 
on  the  fact  that  resources  are  linked  to  each  others  by  relations  allowing  to  index  a set  of  resources  instead  of  a unique 
one. 


Using  the  user  profile 

We  consider  that,  when  a user  is  indexing  a document  (a  new  or  an  old  one),  many  descriptors,  for  that  user, 
do  not  change  from  one  document  to  another  (like  "lifecycle. contribute").  Furthermore,  for  a subset  of  that  descrip- 
tors, some  values  remain  more  often  identical  for  teachers  coming  from  the  same  institution  (like  "rights,  "educa- 
tional.context"  or  "general. language").  So,  for  each  user,  we  keep  trace,  in  a database,  of  the  values  he  or  she  prefers. 
These  values  can  be  edited  either  for  a particular  document  or  for  the  next  ones.  First,  the  user  needs  to  be  referenced 
and  to  obtain  a login  name  and  a password  to  access  the  tool.  The  values  kept  for  an  institution  are  the  values  preferred 
by  the  teachers  of  that  institution.  On  the  first  access,  the  values  of  the  institution  (or  default  values  for  the  first  user  of 
the  institution)  are  proposed  to  the  user,  an  initial  but  still  editable  choice. 


Using  relations  between  documents 

It  is  really  uncommon  to  create  a unique  new  learning  object  and  index  it  (or  index  a unique  old  one).  In  fact, 
an  educational  resource  is  made  up  of  several  connected  learning  objects.  The  idea  is  to  take  advantage  of  that  fact  and 
to  use  the  meaning  of  the  relations  between  resources  to  index  them.  When  a teacher  creates  an  educational  resource, 
the  learning  objective  is  quite  clear,  the  intended  public  is  known  and  some  descriptors  are  very  close  for  all  the  com- 
posing objects.  This  is  not  always  true  may  be,  but  this  is  a semi-automatic  tool  and  it  is  always  possible  to  change  the 
values.  Furthermore,  the  descriptor  "general. AggregationLevel"  can  be  deduced  from  the  relations  "IspartOf",  from 
example,  and  the  "LifeCycle. Version"  from  the  relations  "IsVersionOf". 
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For  the  Indexation  of  a new  learning  object,  the  tool  asks  the  user  to  enter  the  relation  descriptor.  If  it  is  miss- 
ing or  meaningless,  the  tool  retrieves  the  default  values  for  the  users  profile  changing  some  descriptors  like  "title"  or 
"date".  A resource  may  be  linked  to  more  than  one.  So  we  propose  the  kinds  of  possible  relations  and  for  each  kind  we 
perform  a specific  task.  We  offer  the  user  to  enter  the  relations  In  a specific  order  ("IsVerslonOf",  "Replaces",  "IsFor- 
matOf ",...)  and  the  tool  retrieves  the  descriptors  values  from  the  first  met.  If  we  meet  a "IsVErsionOf"  relation,  the 
tool  changes  the  version  descriptor.  If  we  meet  a "IsPartOf " relation,  the  tool  changes  the  granularity.  If  the  learning 
object  has  several  "References"  relations,  the  tool  performs  the  union  of  the  keywords. 

Therefore,  our  tool  tries  to  guess  the  maximum  of  descriptors  values.  After  that,  all  these  values  are  proposed  to 
the  user  allowing  him  to  destroy  or  to  update  them  or  to  entry  new  ones. 

Furthermore,  our  tool  enables  the  user  to  add  some  metadata  to  linked  learning  objects.  If  the  learning  object 
being  actually  indexed  has  a "IsPartOfRelatlon"  with  an  existing  indexed  one,  the  tool  adds  a "HasPart"  relation  to  It. 

For  existing  resources  connected  to  indexed  ones,  the  tool  performs  the  same  way. 

For  existing  resources.  If  these  are  HTML  files  with  meta  tags  implementing  Dublin  Core  metadata,  the  tool 
takes  them  and  add  the  default  values  from  the  user  profile  for  the  descriptors  not  Included.  But  this  case  Is  very 
uncommon  (in  our  experience,  we  have  never  found  that  kind  of  resources  In  our  institution). 

So,  we  propose  the  user  to  Index  a first  learning  object,  with  default  values  and  we  let  him  changing  their  con- 
tents. After  that,  we  ask  the  user  to  enter  the  descriptors  about  relations.  For  each  linked  resource,  the  tool  proposes  to 
the  user  to  index  it  using  all  the  possible  deductions  due  to  the  type  of  the  relation. 

A problem,  not  yet  solved,  Is  what  to  do  when  the  deductions  the  tool  made  are  not  coherent  (like  two  "Has- 
Part" relations  with  two  different  granularities). 


Our  Goal 

Actually,  we  are  developing  the  described  tool  to  help  teachers  to  index  their  documents  (with  IEEE  LOM 
metadata)  based  on  the  user  profile  and  the  use  of  relations. 

In  the  first  step,  default  values  are  suggested,  assumed  to  be  pertinent,  for  a large  number  of  descriptors  letting 
the  user  verifying  their  relevance  and  entry  some  of  them  (like  "educational. difficulty"  or  "educational.TyplcalLearn- 
ingTime"  with  less  meaning  default  values). 

In  the  second  step,  more  useful,  the  user  is  helped  to  index  a set  of  connected  documents  with  default  and 
common  values.  Then,  the  user  may  edit  and  modify  the  values  retated  to  the  whole  set  of  documents  or  to  a specific 
one. 

As  learning  objects  cost  a lot  of  time  to  produce  them,  their  re-usability  must  be  improved.  For  such  a goal, 
they  have  to  be  indexed  by  metadata.  Our  tool  reduces  indexing  time  and  permits  a better  re-use  and  consequently  a 
cut  of  the  developing  costs  of  learning  objects. 
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Abstract:  This  paper  discusses  the  notion  of  community  as  an  outcome  of  working  within  an 
online  environment.  In  particular  the  paper  explores  the  concept  of  users’  development  of  a 
sense  of  community  as  an  outcome  of  working  within  an  online  environment  designed  to 
support  the  professional  and  personal  development  of  its  users.  The  paper  discusses  previous 
research  which  has  explored  the  development  of  a sense  of  community  and  reports  a study 
that  sought  to  investigate  the  development  of  a sense  of  community  among  users  of  a 
community-oriented  site  supporting  teachers’  professional  development. 


Introduction 

The  social  phenomenon  of  community  can  be  put  to  good  use  in  the  support  of  learning  through  the  development 
of  learning  communities.  Although  difficult  to  define  it  is  evident  that  community  is  a central  component  of  the 
lives  of  all  individuals.  People  have  always  and  will  always  form  communities  at  one  level  or  another  as  part  of 
their  existence.  Indeed,  community  is  such  a central  part  of  our  lives  that  "if  the  sense  of  living  in,  belonging  to, 
and  having  some  commitment  to,  a particular  community  is  threatened  then  the  prospect  of  living  rewarding 
lives  is  diminished"  (Puddifoot,  1996,  p.  327).  The  power  of  community  to  support  learning  rests  with  its 
centrality  to  our  daily  lives  and  an  unexplained  phenomenon  where  the  sum  of  the  parts  of  a community  is  in 
some  way  greater  than  the  whole. 

Virtual  learning  communities  and  their  role  in  supporting  learning 


As  we  begin  to  explore  the  concept  of  community  and  its  potential  to  support  learning  it  is  important  to 
acknowledge  that  a positive  sense  of  community  can  have  many  referents  ranging  from  “family,  fellow  workers, 
friends,  neighbours,  religious  and  fraternal  bodies"  (Sarason,  1974,  p.  153).  It  is  equally  important  to 
acknowledge  that  community  exists  in  both  a geographic  and  relational  sense  and  that  the  two  are  not  mutually 
exclusive  (eg.  Worsley  1991).  The  online  environment  provides  for  the  development  of  the  relational 
community  where  members  may  never  meet  face  to  face.  These  communities  have  been  dubbed  virtual 
communities  and  have  been  identified  as  ‘real’  communities  in  a sociological  sense  (Surratt,  1998). 

In  the  online  learning  environment  the  desired  community  is  known  as  a virtual  learning  community.  Maxwell 
(1998)  recognizes  the  value  of  a learning  community  reporting  on  the  positive  influence  learning  communities 
have  on  both  socialization  and  learning  outcomes.  Similarly  Kellogg  (1999)  argues  that  learning  communities 
promote  more  active  and  increased  intellectual  interaction  and  a sense  of  common  purpose  and  Palloff  and  Pratt 
(1999)  posit  that  the  creation  of  a learning  community  supports  knowledge  acquisition.  These  communities  are 
considered  to  be  of  such  value  in  the  support  of  learning  that  researchers  believe  the  formation  of  virtual  learning 
communities  is  central  to  the  success  of  online  learning  (Hiltz,  1997;  Palloff  and  Pratt,  1999). 

Learning  communities  and  learning 

Learning  communities  support  learning  by  promoting  the  benefits  of  collaborative  learning  environments. 
Socio -constructivist  approaches  to  learning  clearly  provide  evidence  that  when  students  work  together  their 
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cognitive  development  can  be  enhanced  (eg.  Glasssman,  2001).  In  collaborative  learning  environments  students 
can  become  actively  involved  in  the  construction  of  new  ideas  and  concepts  and  in  this  way  student  learning 
moves  beyond  the  information  presented  to  them  (eg.  Bruner,  2001).  Johnson  (1991)  asserts  “that  collaborative 
learning  methods  are  more  effective  than  traditional  methods  in  promoting  learning  and  achievement”  and 
Benbunan-Fich  (1997)  concludes  that  “working  in  groups,  instead  of  alone,  significantly  increases  motivation, 
perception  of  skill  development  and  solution  satisfaction”.  The  benefits  are  not  limited  to  the  cognitive  domain. 
Panitz  (1997)  identified  67  benefits  spanning  the  academic,  social  and  psychological  domains.  And  Slavin 
(1990)  posits  that  collaborative  learning  environments  promote  higher  levels  of  motivation  as  well  as  social  and 
attitudinal  benefits  concluding  that  the  effect  of  collaborative  learning  on  achievement  is  clearly  positive. 

In  addition  to  the  promotion  of  collaborative  learning  principles,  learning  communities  promote  a positive 
environment  created  by  members  who  actively  seek  participation  from  others,  valuing  all  members  and  sharing 
the  results  of  their  efforts  (Moore  & Brooks,  2001).  This  positive  environment  and  members’  preparedness  to 
work  collaboratively  are  indicative  of  virtual  learning  communities  and  fundamental  in  the  support  of  learning. 

Strategies  that  support  community  development 

Unfortunately  little  scientific  research  exists  to  guide  the  development  of  a virtual  learning  community  although 
some  strategies  have  been  identified  through  anecdotal  records  (Palloff  & Pratt,  1999).  Combining  these 
strategies  with  knowledge  of  community  gleaned  from  the  social  sciences  provides  initial  support  in  the 
development  of  virtual  communities. 

Establishing  a welcoming  personal  environment  is  an  important  strategy  in  constructing  the  ‘human’  elements  of 
community  (Hiltz,  1998;  Palloff  & Pratt,  1999).  And  facilitating  regular  and  meaningful  communications  where 
members  can  interact  and  share  their  ideas  and  concerns  (Moore  & Brooks,  2001),  is  central  to  the  functioning  of 
a community.  An  additional  strategy  is  to  provide  an  initiating  reason  which  may  be  a disorientating  dilemma,  an 
issue,  a concern,  a contentious  discussion  or  a particular  community  problem  (Moore  & Brooks,  2001).  Further 
impetus  can  be  gained  by  emphasising  the  benefits  associated  with  becoming  a community  member.  These 
benefits  could  include  an  increase  in  intellectual  capital  (Stewart,  1997),  reciprocity  or  an  increase  in  social 
capital  (Putnam,  2000).  Further  support  is  found  in  the  seven  basic  steps  outlined  by  Palloff  and  Pratt  (1999) 
(Table  1). 

Table  1 : Seven  Basic  Steps  in  the  Development  of  an  Online  Community 

1.  Clearly  define  the  purpose  of  the  group 

2.  Create  a distinctive  gathering  place  for  the  group 

3.  Promote  effective  leadership  from  within 

4.  Define  norms  and  a clear  code  of  conduct 

5.  Allow  for  a range  of  member  roles 

6.  Allow  for  and  facilitate  sub-groups 

7.  Allow  members  to  resolve  their  own  conflict 


Determinations  of  the  development  of  a sense  of  community 

Prior  to  engaging  in  the  measurement  of  community  it  is  important  to  understand  that  community  is  a sense 
rather  than  a tangible  entity  and  it  is  members’  sense  cf  community  (SOC)  that  should  be  identified  and 
measured.  McMillan  and  Chavis  (1986)  define  SOC  as  a “feeling  that  members  have  a belonging,  members 
matter  to  one  another  and  to  the  group,  and  a shared  faith  that  members’  needs  will  be  met  through  their 
commitment  to  be  together”  (p.  9).  McMillan  and  Chavis  further  developed  a four-dimensional  model  of  SOC 
arguing  the  presence  of  four  discrete  entities  in  members’  sense  of  community,  membership;  influence; 
integration  and  fulfillments  of  needs;  and  a shared  emotional  connection. 

The  model  provides  a mechanism  for  understanding  SOC  but  not  for  measuring  the  experience.  This  can 
be  achieved  via  the  Sense  of  Community  Index  (Chavis,  Hogge,  McMillan  & Wandersman,  1986)  based 
on  the  McMillan  and  Chavis  (1986)  four  dimensional  model  of  SOC.  A measurement  tool  that  has  been 
shown  to  adequately  assess  SOC  having  validity  across  contexts  (Chipuer  & Perry,  1999). 


231 


Learnscope  Virtual  Learning  Community 


The  dearth  of  empirical  information  describing  the  essence  of  community  development  and  our  interest  in 
exploring  how  communities  can  be  developed  and  maintained  in  online  settings  prompted  the  inquiry  we  report 
here.  As  part  of  a Web  site  evaluation  project,  we  undertook  to  explore  the  degree  to  which  users  of  the  site 
sensed  themselves  within  a community  of  users. 

The  Learnscope  Virtual  Learning  Community  is  a Web  site  within  Australia  that  has  been  designed  to  assist  in 
meeting  the  professional  needs  of  people  within  the  Australian  Vocational  Education  and  Training  (VET)  sector 
(Fig.  1.).  The  VLC  is  a dynamic  and  interactive  Web-based  support  system  that  seeks  to  support  the 
development  of  a critical  mass  of  VET  staff  who  are  able  to  use  flexible  learning  approaches  to  accelerate 
Australia’s  transition  to  the  information  economy.  The  VLC  is  supported  by,  and  works  within  the  Australian 
Flexible  Learning  Framework  and  represents  one  of  a number  of  activities  undertaken  in  the  Framework  to 
achieve  the  goal  of  providing  Creative,  Capable  People  (Australian  Flexible  Framework,  2001). 


Fig  1:  The  Learnscope  Virtual  Learning  Community  Home  Page  (www.leamscope.anta.gov.au) 

The  LeamScope  Virtual  Learning  Community  consists  of  3 main  elements:  the  Community  Hub,  supporting 
interaction  and  communication  between  the  various  members.  Includes  a login  area,  a personal  space, 
discussion  forums;  Go  Learn,  an  area  providing  access  to  a variety  of  resources  to  support  members’ 
professional  development;  and  a Resource  Centre,  access  to  information  and  content  about  flexible  delivery. 

Since  an  important  aim  of  the  Web  site  was  to  facilitate  professional  development,  the  form  of  community  which 
the  site  sought  to  develop  was  based  on  learning  and  personal  development.  We  organized  an  inquiry  process 
that  would  provide  some  information  on  the  capacity  of  the  site  to  create  a sense  of  community  among  the  users 
and  we  also  sought  to  explore  factors  that  might  influence  the  scope  and  extent  of  the  community  development. 
The  purpose  of  this  inquiry  was  to  explore  the  development  of  users*  sense  of  community  obtained  from  use  of 
the  site.  Users  of  the  VLC  were  asked  to  complete  an  online  questionnaire  which  probed  aspects  of  their  use  of 
the  site.  One  component  of  the  questionnaire  asked  users  to  respond  to  a series  of  questions  that  probed  their 
perceptions  about: 

• Their  sense  of  membership  of  the  VCL  community; 

• The  sense  of  influence  they  had  within  the  community  as  a consequence  of  their  involvement; 

• Their  sense  of  the  extent  to  which  the  VLC  facilitated  integration  and  fulfilment  of  needs 

• The  users’  sense  of  a shared  emotional  connection  with  other  users. 

The  series  of  questions  was  based  on  the  questionnaire  developed  by  Chavis,  Hogge,  McMillan  and 
Wandersman,  (1986),  and  included  the  items  shown  in  Table  2 below.  The  online  questionnaire  was  completed 
by  121  participants  who  considered  themselves  as  more  than  casual  users  of  the  site. 
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Table  2;  Member's  Sense  of  Community 

statement  scale  true  % false  % 


integration  and  fulfillment  of  needs  76  24 


I think  VLC  is  very  helpful  in  meeting  my  needs  in 
flexible  delivery 

People  using  the  VLC  seem  to  share  the  same  values 
Other  members  and  I want  the  same  things  from  VLC 
I think  the  VLC  has  an  appropriate  scope  in  what  it 
tries  to  do 

I can  recognize  most  of  the  people  who  participate  in 
the  VLC 

I feel  at  home  in  VLC 

Many  of  the  other  people  who  use  the  VLC  know  me 
Existing  members  of  the  VLC  welcome  new  members 
documents  etc. 

I care  about  what  other  members  think  of  my  actions 
when  using  the  VLC 

1 feel  I have  influence  over  what  happens  in  the  VLC 
I feel  that  other  people  in  the  VLC  would  help  me  if  I 
requested  help 

1 feel  my  opinions  and  ideas  are  welcomed  by  others 
in  the  VLC 

It  is  very  important  to  me  to  participate  in  the  VLC 
People  in  the  VLC  seem  generally  to  get  along  with 
each  other 

I expect  to  continue  to  participate  in  VLC  into  the 
future 

People  in  the  VLC  seem  to  have  similar 
understandings  and  interests 


integration  and  fulfillment  of  needs 

78 

22 

integration  and  fulfillment  of  needs 

63 

37 

Integration  and  fulfillment  of  needs 

94 

6 

sense  of  membership 

29 

71 

sense  of  membership 

53 

47 

sense  of  membership 

28 

72 

sense  of  membership 

68 

32 

sense  of  influence 

75 

25 

sense  of  influence 

46 

54 

sense  of  influence 

99 

1 

sense  of  influence 

83 

17 

shared  emotional  connection 

42 

58 

shared  emotional  connection 

93 

7 

shared  emotional  connection 

89 

11 

shared  emotional  connection 

84 

16 

Sense  of  Community 

The  feedback  provided  strong  support  for  the  supposition  that  the  site  did  provide  users  with  the  support  and 
connection  required  to  give  individuals  a sense  of  participation  within  a community.  Users’  responses  for  the 
scale  exploring  their  sense  of  the  extent  to  which  the  VLC  facilitated  integration  and  fulfillment  of  needs 
indicated  a strongly  positive  response.  The  questions  exploring  users’  sense  of  membership  tended  to  return  the 
least  positive  responses.  It  was  evident  that  within  the  large  user  base  that  while  many  users  were  comfortable 
with  the  interactions  and  connection,  the  size  of  the  user  base  precluded  any  personal  relationships  and  the 
ensuing  sense  of  membership  which  is  usually  associated  with  such  connections. 

The  users  seemed  to  gain  a relatively  strong  sense  of  shared  emotional  connection  from  their  experiences.  The 
item  with  the  lowest  rating  was  that  which  sought  to  determine  the  value  of  the  VLC  to  users.  While  the  users 
appeared  happy  to  participate  in  the  community  and  felt  that  they  were  dealing  with  people  with  similar  interests, 
there  was  a sense  among  many  that  the  community  was  not  overly  important  to  them  in  the  big  picture. 

Users  reported  quite  positive  responses  to  items  that  sought  to  establish  their  sense  of  influence  within  the 
community.  The  responses  indicated  that  users  generally  felt  well  supported  by  others  but  it  appeared  that  many 
felt  that  their  contributions  were  not  perhaps  that  significant  in  the  overall  community.  Given  the  large  numbers 
of  users  and  the  diverse  nature  of  their  backgrounds  and  previous  experiences,  the  responses  provided  relatively 
strong  evidence  of  the  site  giving  users  a sense  of  community  among  those  participating. 

Factors  influencing  sense  of  community 

The  inquiry  sought  to  establish  factors  which  appeared  to  influence  the  SOC  experienced  by  users  of  the  VLC. 
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a.  Level  of  expertise  wiuth  ICT:  Users  provided  an  indication  of  their  perceived  level  of  expertise  with  learning 
technologies  using  a scale  ranging  from  1 low  through  to  4,  very  high.  An  inspection  of  the  means  scores 
achieved  by  students  reporting  the  different  levels  of  expertise  showed  very  consisted  scores  (Figure  1).  It 
appeared  that  background  knowledge  of  the  subject  matter  on  offer  was  not  a factor  overly  influencing  users’ 
sense  of  community  development.  A one  factor  analysis  of  variance  was  used  to  test  for  differences  among  users 
sense  of  community  based  on  perceived  levels  of  LT  expertise  (Figure  1).  The  results  did  not  indicate  any 
significant  differences  in  sense  of  community  against  this  factor  (F  (3,99)  = 0.388,  ns). 

b.  Level  of  experience  with  iCT:  A second  test  was  carried  out  to  explore  whether  users’  perceived  levels  of 
ICT  experience  was  a factor  influencing  the  development  of  a sense  of  community.  Users  were  asked  to  report 
their  levels  of  expertise  with  information  and  communications  technologies  (ICT).  Their  responses  varied  from 
score  of  1,  low  levels  to  scores  of  4,  very  high  levels.  Once  again,  the  means  scores  achieved  by  students 
reporting  the  different  levels  of  expertise  were  very  consistent.  A one  factor  analysis  of  variance  was  used  to  test 
for  differences  among  users’  sense  of  community  based  on  perceived  levels  of  LT  expertise.  The  results  did  not 
indicate  any  significant  differences  in  sense  of  community  against  this  factor  (F  (3,99)  = 1.008,  ns). 

c.  Level  of  access  to  ICT:  A third  test  was  carried  out  to  explore  whether  users’  levels  of  access  to  ICT  was  a 
factor  influencing  development  of  a sense  of  community.  Users  were  asked  to  report  their  levels  of  ICT  access 
using  responses  of  1 (limited  access)  or  2 (access  not  limited).  A one  factor  analysis  of  variance  was  used  to  test 
for  differences  among  users’  sense  of  community  based  on  access  to  ICT.  The  results  did  not  indicate  any 
significant  differences  in  sense  of  community  against  this  factor  (F  (1,101)  = 1.192,  ns). 

(L  Level  of  use  of  the  VLC:  Another  test  was  carried  out  to  explore  whether  users’  levels  of  use  of  the 
Community  Hub  was  a factor  influencing  development  of  a sense  of  community.  Users  were  asked  to  report 
their  levels  of  use  of  the  Community  Hub  using  responses  of  1 (limited  use)  through  to  4 (regular  use).  A one 
factor  analysis  of  variance  was  used  to  test  for  differences  among  users  sense  of  community  based  on  use  of  the 
Community  Hub  (Figure  4).  The  results  indicated  a significant  difference  at  the  .05  level  (F  (3,99)  = 2.94,  p 
<0.05).  This  finding  was  in  accord  with  expectations  and  supportive  of  the  notion  that  the  interaction  was  an 
important  factor  in  building  the  sense  of  community  within  users. 

e.  Level  of  use  of  the  Resource  Centre:  A further  test  was  carried  out  to  explore  whether  users’  levels  of  use  of 
the  Resource  Centre  was  a factor  influencing  development  of  a sense  of  community.  Users  were  asked  to  report 
their  levels  of  use  of  the  Resource  Centre  using  responses  of  1 (limited  use)  through  to  4 (regular  use).  A one 
factor  analysis  of  variance  was  used  to  test  for  differences  among  users  sense  of  community  based  on  usage  of 
the  Resource  Centre.  The  results  indicated  a significant  difference  at  the  .05  level  (F  (3,99)  = 5.46,  p <0.05). 
This  finding  could  be  interpreted  in  much  the  same  way  as  we  interpreted  the  finding  that  use  of  the  Hub  was  a 
contributing  factor  to  the  development  of  a sense  of  community.  It  might  well  be  however,  that  this  finding 
shows  that  users  who  made  a high  level  of  use  of  the  Hub  also  made  a high  level  of  use  of  the  Resource  Centre. 
When  the  levels  of  usage  of  the  Hub  and  resource  Centre  were  compared,  there  was  a significant  correlation 
(t^=33). 


These  findings  are  quite  interesting  in  light  of  the  nature  of  learning  communities  supported  by  online 
environments.  It  is  difficult  to  speculate  too  much  with  the  results.  They  suggest  that  the  development  of  a 
sense  of  community  by  users  of  the  Leamscope  VLC  was  not  influenced  by  such  features  as  their  levels  of 
experience  and  ICT  access.  The  findings  suggest  that  the  sense  of  community  is  developed  very  much  by  use  of 
the  site  and  that  all  users  appear  capable  of  developing  community  membership  through  use  of  the  site.  We  have 
taken  this  finding  as  evidence  that  the  Leamscope  sight  is  in  fact  returning  in  some  way,  the  forms  of  outcomes 
that  it  was  developed  for. 

Summary  and  Conclusions 

The  use  of  online  facilities  to  support  and  sustain  the  development  of  communities  of  learners  is  becoming  quite 
common  among  many  educational  groups.  In  this  paper  we  have  discussed  the  notion  of  a learning  community 
and  possible  ways  by  which  the  development  of  communities  might  be  determined.  An  exploration  among  users 
of  the  Leamscope  Virtual  Community,  suggested  that  this  site  did  in  fact  provide  learners  with  a sense  of 
community.  A further  exploration  of  factors  that  might  have  been  likely  to  influence  learners’  sense  of 
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community  revealed  that  aspects  such  as  ICT  expertise,  levels  of  access  and  previous  experience  had  no 
discernible  impact  on  the  development  of  SOC  among  learners.  What  was  found  to  be  important,  and  as  might 
be  expected,  was  the  level  of  use  of  the  site.  Interestingly  we  were  unable  to  demonstrate  which  sorts  of  activities 
had  the  greatest  impact  on  the  development  of  SOC  and  this  arises  as  a possible  line  of  inquiry  for  further  work. 

The  findings  from  this  paper  appear  to  support  the  need  for  more  research  and  inquiry  into  the  development  of 
community  sense  among  users  of  online  sites  that  aim  to  promote  users  personal  development  through  such 
processes  as  interaction  and  communication  with  others.  The  data  gathered  in  this  inquiry  provided  evidence  that 
the  users  of  the  Leamscope  site  did  in  fact  establish  senses  of  community  of  varying  degrees  and  suggest  the 
need  for  more  work  to  more  fully  investigate  what  sorts  of  online  activities  hold  the  best  prospects  for 
community  development. 
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Abstract:  Interactive  visualization  is  a powerful  educational  tool.  It  has  been  used  to 
enhance  the  teaching  of  various  subjects  from  computer  science  to  chemistry  to 
engineering.  Surprisingly  enough,  in  computer  science  education,  this  powerful  tool  is 
used  almost  exclusively  in  programming  and  data  structure  courses.  This  paper  suggests 
that  visualization  could  be  very  helpful  in  teaching  a larger  variety  of  computer  science 
courses  and  also  presents  several  visualization  tools  that  have  been  used  in  the  context 
of  an  information  retrieval  course. 


Introduction 

Interactive  visualization  is  a powerful  educational  tool.  Visualization  can  provide  a clear  visual 
metaphor  for  understanding  complicated  concepts  and  uncovering  the  dynamics  of  important  processes  that  are 
usually  hidden  from  the  student’s  eye.  Visualization  has  been  used  to  enhance  the  teaching  of  various  subjects 
ranging  from  chemistry  (Yaron  et  al.,  2001)  to  mechanics  (Hampel,  Keil-Slawik  & Ferber,  1999)  to  physics 
(McKenna  & Agogino,  1997).  Computer  science  is  one  of  the  most  active  application  areas  for  educational 
visualization  research.  In  computer  and  information  science  (CIS)  education,  visualization  is  used  almost 
exclusively  in  programming  and  data  structure  courses.  We  can  name  dozens  of  papers  devoted  to  visualization 
of  program  execution  on  several  levels  from  machine-level  languages  (Butler  & Brockman,  2001)  to  high-level 
languages  (Domingue  & Mulholland,  1998;  Haajanen  et  al.,  1997;  Tung,  1998)  to  algorithms  and  data 
structures  (Brown  & Najork,  1997).  Our  claim  is  that  a number  of  other  traditional  CIS  courses  could  benefit 
from  this  powerful  technology. 

This  paper  explores  the  opportunities  for  using  interactive  visualization  in  the  context  of  an 
information  retrieval  course.  Information  retrieval  has  been  in  the  program  of  many  computer,  information,  and 
library  science  departments  for  more  than  30  years.  With  the  maturity  of  the  World  Wide  Web,  information 
retrieval  became  an  important  practical  subject.  Elements  of  information  retrieval  are  now  taught  to  students  of 
many  different  specialties.  We  think,  that  information  retrieval  provides  an  interesting  and  important 
application  area  for  exploring  the  power  of  interactive  visualization.  In  the  following  sections  we  discuss  the 
use  of  visualization  in  teaching  information  retrieval  and  present  several  visualization  tools  developed  at  the 
University  of  Pittsburgh. 


Visualization  for  Information  Retrieval 

The  core  of  a traditional  information  retrieval  (IR)  course  is  a set  of  models,  algorithms  and 
technologies  for  processing,  storing  and  retrieving  textual  information.  This  core  has  been  already  explored  by 
now  and  placed  on  a solid  mathematical  foundation.  Traditional  presentation  of  this  core  usually  starts  with 
several  IR  models  (such  as  Boolean,  vector,  probabilistic  and  several  variations  of  them)  and  then  follows  by 
explaining  how  the  information  is  organized  and  retrieved  in  each  of  these  models  (Baeza-Yates  & Ribeiro- 
Neto,  1999;  Korfhage,  1997). 

The  process  of  retrieving  the  information  in  different  models  is  one  of  the  hardest  topics  in  an  IR 
course.  Despite  being  formalized  and  well  understood  by  the  IR  research  community,  it  is  still  very  hard  for 
students  to  grasp.  We  have  observed  that  even  the  Boolean  information  retrieval,  the  simplest  of  the  models,  is 
difficult  for  students.  At  the  same  time,  traditional  educational  tools  - research  or  commercial  IR  systems  - 
offer  little  educational  help.  The  process  of  retrieving  the  information  has  several  steps,  from  getting  the  query 
in  to  matching  the  query  to  the  documents  to  prioritizing  the  results.  In  an  IR  system  all  these  steps  are  hidden 


236 


from  a user  - the  only  thing  that  a user  can  observe  are  the  final  results  - a list  of  ordered  documents.  In  that 
sense  it  is  similar  to  a non-visualized  execution  of  a computer  program.  A user  can  see  input  data  and  observe 
final  results,  but  it  offers  no  help  in  understanding  how  these  results  were  computed. 

Naturally,  similar  contexts  encourage  the  use  of  similar  remedies.  So,  the  first  thing  we  have  decided 
to  visualize  is  the  process  of  retrieving  the  information  in  several  known  models.  For  the  moment,  we  have 
developed  and  explored  interactive  visualization  environments  for  several  models  - Boolean,  fuzzy,  vector,  and 
extended  Boolean  (see  (Baeza-Yates  & Ribeiro-Neto,  1999;  Korfhage,  1997)  for  the  description  of  these  classic 
models).  Since  space  is  limited  and  our  environments  are  reasonably  similar  we  present  here  in  more  details  the 
one  for  exploring  Boolean  IR. 


Interactive  Visualization  of  Boolean  Information  Retrieval  Model 

The  Boolean  IR  model  is  the  oldest  and  the  simplest  of  known  IR  models.  In  this  model,  a query  is 
written  as  a set  of  elementary  queries  (usually  keywords)  connected  by  Boolean  operators  such  as  OR,  AND, 
NOT.  The  mechanism  of  this  model  is  set  theoretical.  Every  query  is  associated  with  a set  of  matching 
documents.  For  an  elementary  query  such  as  a keyword  the  set  of  matching  documents  is  simply  all  documents 
indexed  by  this  keyword.  To  obtain  the  set  of  matching  documents  for  two  queries  connected  by  a Boolean 
operator  one  has  simply  to  perform  the  corresponding  set  operation  on  their  matching  sets  (i.e.,  set  intersection 
for  AND,  complement  for  NOT,  etc.).  Thus  in  several  steps,  a matching  set  for  any  complex  Boolean  query  can 
be  found. 

While  it  all  sounds  quite  simple  and  clear,  we  have  found  that  many  of  our  students  have  problems 
understanding  how  the  Boolean  matching  works.  Our  talks  with  students  have  indicated  that  one  of  the  sources 
of  their  troubles  is  the  failure  to  perceive  Boolean  operators  as  operations  on  sets  of  matching  documents. 
Naturally,  our  students  have  good  programming  background  and  have  been  routinely  using  Boolean  operators 
for  writing  conditional  expression  in  their  programs.  Still,  many  of  them  have  problems  transferring  their 
knowledge  of  these  operators  to  the  set  context. 

In  developing  an  interactive  visualization  environment  for  the  Boolean  IR  model  we  were  trying  to 
achieve  two  goals:  to  provide  a helpful  visual  metaphor  and  to  visualize  the  process  of  Boolean  IR  step  by 
step.  Figure  1 presents  an  interface  of  our  environment.  The  core  of  this  interface  is  a set  of  all  documents 
visualized  in  a table  (one  document  per  row).  Note  that  in  our  system,  documents  are  textbooks  since  it  is  one 
of  the  most  traditional  kinds  of  documents. 
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Figure  1:  Boolean  Model  Environment.  Visualization  of  matching  for  a simple  Boolean  OR  query.  Documents 

matching  the  first  elementary  query  are  highlighted. 
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Figure  2:  Visualization  of  matching  for  a simple  Boolean  OR  query.  Documents  matching  the  second 

elementary  query  are  highlighted. 
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Figure  3:  Visualization  of  matching  for  a simple  Boolean  OR  query.  Documents  matching  the  whole  Boolean 

OR  query  are  highlighted. 
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The  goal  of  this  representation  is  to  help  the  student  to  understand  the  core  principle  of  this  model  - 
every  query  is  associated  with  a particular  subset  of  all  documents.  Showing  the  set  of  all  documents  on  the 
screen  makes  it  easy  to  demonstrate  different  subsets  of  the  whole  set  as  sets  of  differently  colored  table  rows. 
The  goal  of  the  whole  environment  is  to  help  the  student  to  understand  (a)  the  process  of  matching  an 
elementary  query  to  the  set  of  the  documents  and  (b)  how  different  set  theoretic  operations  work  in  obtaining  a 
new  subset  from  contributing  sets. 

The  student  can  explore  Boolean  matching  by  writing  simple  Boolean  queries  (two  terms  connected 
by  one  or  two  operators)  and  observing  the  matching  process  in  these  steps  by  clicking  on  each  of  the  three 
buttons  on  the  right  panel. 

The  first  button  highlights  the  subset  of  documents  matching  the  first  elementary  query  (Figure  1),  the 
second  highlights  the  subset  matching  the  second  query  (Figure  2)  and  the  third,  the  results  of  the  chosen  set 
operation  on  the  contributing  sets  (Figure  3).  We  choose  to  have  three  buttons  to  enable  the  student  to  explore 
the  matching  process  several  times  forwards  and  backwards. 

Beyond  the  component  shown  on  Figures  1-3  the  Boolean  IR  environment  has  several  other 
components.  In  particular,  to  help  the  student  transfer  the  understanding  of  Boolean  IR  from  classic  IR  to  the 
database  context,  we  have  provided  a very  similar  exploration  interface  where  elementary  queries  are  constructed 
not  from  keywords  as  in  classic  IR  but  from  restrictions  on  various  fields  of  a database  record  (i.e.,  year  = 2000 
and  publisher  !=  kluwer).  There  is  also  a registration  screen  and  an  interface  for  a teacher  to  edit  the  collection 
of  documents.  The  environment  works  on  the  Web  and  is  implemented  as  a set  of  CGI  scripts. 


Other  Learning  Environments 

As  we  have  mentioned,  the  Boolean  matching  environment  is  just  one  of  several  interactive 
visualization  environments  that  we  have  developed  and  explored.  The  environments  for  other  models  are 
reasonably  similar.  In  all  of  them  we  have  tried  to  center  the  visualization  of  the  matching  process  on  a visual 
representation  of  the  whole  set  of  documents  in  some  form.  The  environments  for  other  models  are  a bit  more 
complicated  (since  the  models  themselves  are  more  complicated)  but  they  also  provide  the  students  with  more 
opportunities  for  interactive  exploration.  For  example,  in  the  fuzzy  matching  model,  an  elementary  query 
corresponds  to  a fuzzy  subset.  A fuzzy  subset  can  not  be  shown  by  simply  highlighting  its  documents  - we 
need  to  show  the  “membership”  of  each  document  in  a set.  Thus  an  environment  for  fuzzy  matching  (Figure  4) 
has  to  provide  extra  fields  for  every  document  to  show  their  memberships  in  all  contributing  queries  (three 
rightmost  columns  on  Figure  4)  and  provide  the  student  an  opportunity  to  re-sort  the  documents  in  the  set  by 
the  value  of  each  of  these  memberships. 

Altogether  our  environments  provide  a very  useful  suite  of  tools  for  teaching  information  retrieval.  We 
anticipate  the  use  of  these  environments  as  both  teaching  and  learning  tools.  First,  we  have  found  that  these 
environments  provide  an  excellent  tool  for  a teacher  to  explain  complex  topics  of  IR  models.  Yes,  it  assumes 
that  a teacher  uses  a computer  and  a projector  in  the  classroom,  but  this  is  currently  the  standard  context  in 
most  Computer  and  Information  Science  Departments.  Still,  computer  projectors  are  most  often  used  in  classes 
to  show  the  same  static  slides  as  in  the  age  of  overhead  projectors  and  blackboards.  By  using  the  power  of 
interactive  visualization,  our  environments  go  well  beyond  traditional  whiteboard  and  slides.  At  the  same  time, 
their  use  requires  almost  zero  preparation  time  (just  to  plan  which  examples  to  show  in  order  to  cover  the  main 
set  of  ideas).  The  teacher  can  easily  accommodate  very  different  audiences  by  adjusting  the  number  of  examples 
to  show,  the  speed,  and  the  granularity  of  presentation.  Even  in  a department  with  “computerless  classes”  the 
environments  can  serve  as  a powerful  teaching  tool  to  handle  the  questions  and  the  problems  of  a troubled 
student  in  a one-to-one  “office”  context. 

Using  these  environments  as  learning  tools  provides  even  better  value.  They  let  the  students  to  switch 
from  passive  leaming-by-reading  to  active  and  interactive  exploratory  learning.  By  exploring  a number  of 
different  examples  with  an  interactive  visualization  environment  they  should  be  able  to  achieve  a better 
understanding  of  complex  IR  topics. 

We  have  already  performed  an  informal  evaluation  of  several  environments  as  teaching  and  learning 
tools  in  Summer  2001  semester  and  have  got  a very  positive  feedback  from  our  students.  Now  we  are  planning 
an  extensive  formal  evaluation  during  the  Summer  and  Fall  2002  semesters. 


239 


j £fc>  £<fi  'iW  FdvoriM 

J Bxk  r -.<1 

I<X)b  Help 

^ 12  t2  i 

Stop  RehMti  Horn* 

a a 

$Nich  Favo^ 

0 

Hatory 

^ 4*  [2i 

Mil  Print  EdK 

. M ^ 

Dwcum  HMtengu 

mm 

■g  ^Go  jLrki» 

ICateqoiv  i^li  | Computer  Science  [y  jAND 

jVeer^Mem 

1)d  , 

yi  ji  Submit  1 Reeat  { 

j Cetegoiy 

1 Title  f 

Year  | 

nice 

i Qneiyl  | 

[ QueTy2 

[Total  Quety 

: : j.r>  • ■ 

j Computer  S deuce 

j Operating  Systems  j 

85  1 

1 1 0 

0.9  1 0.9 

lUUU  1 

1 IV 

1 Computer  Sdence 

1 Turbo  C { 

1999  ( 

55  i 

1.0 

OR 

1 OE 

] Computer  Sdence 

1 Visual  C 1 

1997  i 

56  1 

i 

1 0.6 

1 0.6 

Sortitl  1 

1 Database  Systems 

j Data  Modelling  | 

1995  1 

ioo 

1 6.0 

1 6.4 

1 0.0 

It  • 

[ Electrical 

j Adaptive  Filters  | 

19^  1 

77  ! 

1 6.6 

I 0.7 

[ 6.6 

1 Electrical 

j Digital  Communications  | 

1 00^  1 

AA  1 

1 0 0 

1 n A 

1 0.0 

44  ! 

j Computer  Sdence 

j Computer  Algorithms  | 

1991  1 

30  ! 

i 10 

1 ojd 

1 0.0 

.'-rtfLTj 

1 Infonnation  Sdence  j Weaving  The  Web  | 

1999  ( 

15 

0.0 

[ OR 

I 6.6 

1 

1 Electrical 

j Phase  Locked  Loop  | 

1994  I 

95 

0.0 

i OJ 

1 0.0 

1 Electrical 

{ Fattem  ClassiTication  | 

2001  j 

66  1 

[ 6j6 

! 1-0 

1 6.6 

1 Information  Science 

j Infonnation  Retrieval  | 

1996  1 

40 

1 oo 

I 0.5 

I 0.0 

Return  to  Home  > 

t 

ifoen*. 

_x:i 



llJ Stall -Mtawoft sQTrtiet-unh(i.ei».p>teAj 

1 

4 ] 

Figure  4;  Fuzzy  Model  Environment.  Visualization  of  matching  and  ordering  for  a fuzzy  AND  query. 
The  documents  are  sorted  according  to  the  value  of  the  membership  function  of  the  compound  query. 


Implementation  Issues 

It  was  a critical  design  decision  for  us  to  implement  all  environments  as  Web-based  tools.  Web 
interface  ensures  that  our  tools  can  be  accessed  anytime  anywhere.  It  lets  us  forget  about  platform  differences 
and  avoid  troublesome  process  of  installation.  This  is  very  important  in  the  context  of  college  education  where 
neither  teachers  nor  students  have  fiill  control  over  the  computers  used  in  the  classrooms  and  labs.  Besides,  it 
offers  an  extra  benefit  in  the  context  of  Web-based  and  distance  education  enriching  the  learning  experience  of  a 
remote  student. 

In  fact,  most  of  our  environments  were  implemented  with  server-side  CGI  scripts.  In  the  age  of  Java 
this  choice  is  harder  to  advocate.  Indeed,  the  first  “pure  Web”  learning  environments  for  computer  science 
subjects  were  developed  with  CGI  technology.  Examples  include  both  algorithm  animation  (Campbell  et  al., 
1995;  Ibrahim,  1994),  and  program  visualization  (Brusilovsky,  Schwarz  & Weber,  1996).  However  nowadays, 
Java  is  becoming  a dominant  technology  for  developing  Web-based  interactive  learning  environments. 

We  also  use  Java  when  it  is  necessary  due  to  the  highly  graphical  interface  demands  (for  example,  for 
set  diagrams),  however  we  were  trying  to  avoid  it  when  it  is  not  necessary  for  several  reasons.  First,  we  have 
found  that  Java  applets  are  not  always  working  on  the  browsers  installed  on  various  campus  computers. 
Second,  one  of  our  projects  involves  wireless  access  to  learning  tools  using  handheld  computers  (that  have 
browsers  with  no  Java  support).  Third,  a CGI-based  tool  provides  an  easier  platform  for  centralized  collection 
of  interaction  logs,  which  provide  the  most  useful  source  of  data  for  our  educational  research.  Fourth,  we  are 
working  on  adaptive  interactive  visualization  (Brusilovsky,  1994)  that  requires  the  presence  of  a centralized 
server-side  student  model.  Overall,  we  think  that  in  the  cases  when  advanced  graphics  or  smooth  visualization 
is  not  important  the  old  CGI-based  technology  is  simply  better. 

We  should  add  that  the  suite  of  our  tools  for  teaching  information  retrieval  is  available  for  teachers  and 
students  of  any  IR  courses.  The  home  page  of  this  project  is  http://www2.sis.pitt.edu/-ir/Proiects/.  Currently, 
all  tools  are  running  on  our  servers  and  could  be  used  by  anyone  who  is  interested  to  teach  or  learn  information 
retrieval  and  have  access  to  the  Internet.  We  are  also  working  on  packaging  these  tools  as  public  domain 
software  to  be  installed  wherever  someone  wants  to  use  it. 
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Currently  we  are  working  on  developing  several  other  environments  to  support  teaching  and  learning 
of  IR.  In  these  environments  we  were  exploring  different  visualization  metaphors  (such  as  set  diagrams)  to 
demonstrate  matching  process  as  well  as  the  use  of  interactive  visualization  to  support  other  traditionally  hard 
topics  of  an  information  retrieval  course. 
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Abstract:  Assuming  that  learning  within  project  groups  requires  both  cognitive  skills 
and  a cooperative  orientation  of  the  students  involved  it  is  hypothesized  that  groups 
describing  themselves  as  ‘functional’  consist  of  both  more  'higher  achieving'  students 
and  more  'cooperatively  oriented'  students  as  compared  to  groups  describing 
themselves  as  ‘dysfunctional’.  Based  on  questionnaire-responses,  14  groups  of  about 
10  students  each  could  be  categorized  as  either  ‘very  functional’,  ‘functional’  and 
‘dysfunctional’.  As  hypothesized,  moving  from  the  ‘dysfunctional’  over  the 
‘functional’  to  the  ‘very  functional’  category,  group  members’  academic  achievement 
increases.  Opposite  to  expectations  however,  moving  from  the  ‘dysfunctional’  over 
the  ‘functional’  to  the  ‘very  functional’  category,  group  member’s  cooperative 
orientation  decreases. 


Introduction 

Small-group  projects  are  often  promoted  as  potentially  powerful  educational  vehicles  (e.g.  Slavin,  1996). 
Interacting  with  peers  might  provide  ample  opportunity  for  the  learner  to  engage  in  activities  that  facilitate 
long-lasting,  firmly  rooted  understanding.  Examples  of  such  learner  activities  are  “putting  material  into 
one’s  own  words”,  “explaining  own  opinions  and  questioning  those  of  others”,  ’’revealing  misconceptions 
that  show  up  during  group-discussions”,  “clearing  up  confusion”  and  “summarizing  or  recapitulating 
ideas”. 

In  order  to  accomplish  these  activities  learners  should  possess  the  necessary  cognitive  skills  and  maturity  to 
do  so.  Next  to  cognitive  abilities  however,  one  might  hypothesize  that  peer  learning  draws  upon  the 
learners’  interpersonal  skills  to  sustain  and  maintain  a cooperative  working  relationship  within  the  group. 
Learners  should  possess  the  competency  to  deliver  feedback  in  a non-threatening  and  supportive  way, 
asking  for  and  actually  taking  into  account  suggestions  of  other  group  members  and  attaching  importance 
to  collective  group-successes  rather  than  being  bent  on  personal  victory. 


Hypothesis 

It  was  hypothesized  that  groups  describing  themselves  as  ‘functional’  consist  of  students  with  both  a better 
‘academic  achievement’  and  a more  distinct  ‘cooperative  orientation’  as  compared  to  students  that  belong 
to  ‘dysfunctional’  groups. 
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Method 


Subjects  were  139  sociology  freshmen.  One  (small)  part  of  the  study  program  required  them  to  work  on  a 
group  project  covering  the  ftill  academic  year.  There  were  14  project  groups  of  about  10  students  each. 


Questionnaire  Research 

During  the  final  collective  session  (at  the  end  of  the  academic  year)  students  were  asked  to  fill  in  a 
questionnaire.  The  questionnaire  consists  of  two  parts.  One  part  comprised  10  existing  scales  measuring 
different  aspects  of  the  quality  of  group  functioning  (Bond  and  Shiu,  1997;  Bradshaw  and  Stasson,  1998; 
Freeman, 1996;  Kramer  et  al.,  1997;  Watson  et  al.,  1991;  Gaertner  and  Schopler,  1998;  Wheeless  et  al. 
,1982).  A second  part  measures  students’  ‘cooperative  orientation’  on  the  basis  of  Van  Lange’s  (1997a) 
‘social  value  questionnaire’.  Both  parts  of  the  questionnaire  were  contra -balanced. 


Part  A:  Quality  of  Group  Functioning 

For  every  group  10  scores  (one  for  each  scale)  were  calculated  by  averaging  its  members  scores  on  that 
scale.  On  the  resulting  matrix  (14  groups  by  10  scale-scores)  a cluster  analysis  was  performed.  Three 
clusters  of  groups  appeared  and  were  labeled  ‘very  functional’  (4  groups)  ; ‘functional’  (9  groups)  and 
‘dysfunctional’  (one  group).  Members  of  ‘very  functional’  groups  (upper  line  in  figure  1)  perceived  their 
group  as  a coherent  and  solidary  entity,  said  they  performed  more  efficient  than  if  there  were  no  groups, 
and  believed  their  interactions  resulted  in  decisions  of  good  quality.  Group-work  was  not  perceived  as  a 
waste  of  time  and  one  \^as  satisfied  with  both  the  final  result  of  the  group  work  and  the  way  group 
members  interacted  with  each  other.  Although  all  members  contributed  evenly,  some  members  were  more 
dominant  than  others.  Group  scores  on  all  scales  were  about  one  standard  deviation  lower  for  the 
‘functional’  cluster  and  about  two  standard  deviations  lower  for  the  remaining  ‘dysfunctional’  group  (lower 
line  in  figure  1). 


Aspect  of  Group  Functioning 


Figure  1 Perceived  quality  of  group  functioning  for  ‘very  functional  groups’,  ‘functional  groups’  and  one 
‘dysfunctional  group’. 
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Part  B:  Social  Value  orientation 


Van  Lange’s  measurement  of  social  value  orientation  (Van  Lange  et  al.,  1997)  consists  of  nine 
‘decomposed  games’.  On  every  ‘game’  the  subject  can  opt  for  either  a ‘cooperative’  an  ‘individualistic’ 
and  a ‘competitive’  alternative. 

The  number  of  ‘cooperative’  choices  was  used  as  an  index  of  a subjects’  cooperative  orientation.  The 
mean  score  was  6.19  (with  a standard  deviation  of  3.4). 


Academic  achievement 

Students’  global  exam  result  (minimum=0;  maximum  = 100)  was  taken  as  an  index  of  ‘academic 
achievement’.  The  mean  score  was  57.1  (with  a standard  deviation  of  8.95). 


Quality  of  group  functioning  according  to  the  project-groups’  instructors 

Additionally,  the  two  instructors  who  coached  the  project  groups  during  the  academic  year  were  asked 
(together)  to  categorize  the  14  project  groups  according  to  their  ‘functionality’.  Although  instructors 
judged  it  a difficult  task,  their  (collectively  reached)  solution  matches  quite  well  with  the  categorization 
based  on  students’  self-reportings.  Instructors  classified  three  out  of  the  four  groups  in  the  ‘very  functional’ 
cluster  as  ‘relative  superior’.  The  one  group  classified  as  ‘dysfunctional’  was  also  the  only  group  identified 
by  instructors  as  posing  ‘severe  problems’. 


Results 

A MANOVA  with  both  subjects’  ‘cooperative  orientation’  and  ‘academic  achievement’  as  the  dependent 
variables,  and  ‘group  functioning’  as  the  independent  variable  (containing  three  levels:  ‘very  functional’, 
‘functional’  and  ‘dysfunctional’)  reveals  a multivariate  effect  of  ‘group  functioning’  ^=.90,  F(4,270)  = 
3.8,  p = .005).  The  between-subject  test  reveals  that  the  multivariate  effect  is  accounted  for  by  both  a 
univariate  effect  for  ‘academic  achievement’  (F(2,136)  = 416,  p = .02)  and  ‘cooperative  orientation’ 
(F(2,136)  = 3.15,  p = .05).  Students  global  exam  results  in  the  ‘dysfunctional’,  ‘functional’  and  ‘very 
functional’  groups  were  54.8,  55.7  and  60.1  respectively.  Students’  cooperative  orientation  decreased 
moving  from  the  ‘dysfunctional’  over  the  ‘functional’  to  the  ‘very  functional’  groups.  Means  were  8.7, 
6.25  and  5.6  respectively.  Planned  comparisons  indicate  that  ‘very  functional  groups’  consist  of  students 
obtaining  better  academic  results  as  compared  to  ‘functional’  and  ‘dysfunctional’  groups  combined 
[F(l,136)=5,96;  p<.02].  Additionally,  students  in  ‘very  functional  groups’  are  significantly  less 
cooperatively  oriented  as  compared  to  students  in  ‘functional’  and  ‘dysfunctional’  groups  combined 
F(l,136)=5,78;  p<.021].  The  dysfunctional  group  on  the  other  hand  distinguishes  itself  from  ‘functional’ 
and  ‘very  functional’  groups  combined  only  by  its  members  outspoken  cooperative  orientation 
[F(l,136)=5,56;p<.02]. 


Discussion 

A first  result  of  the  research  reported  refers  to  the  correspondence  between  students’  self-reportings  on  the 
one  hand  and  the  instructors’  opinion  about  the  quality  of  group  functioning  on  the  other  hand.  According 
to  the  instructors  it  is  but  until  the  end  of  the  academic  year  they  obtain  a rough  picture  about  the  way  the 
different  project-groups  are  functioning.  Instructors  said  they  nearly  always  remain  uninformed  about  how 
student-groups  are  functioning  during  the  initial  stages  of  the  project.  Even  when  explicitly  asked  for, 
students  often  remain  reluctant  to  report  problems  within  their  group,  thereby  preventing  instructors  to 
timely  intervene.  Having  demonstrated  however  that  dysfunctional  groups  can  be  detected  using 
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questionnaires  measuring  different  aspects  of  group  functioning,  a hopeful  perspective  is  offered.  Currently 
we  are  investigating  if  and  to  what  extent  group  (dys)functioning  can  be  mapped  at  the  very  early  stages  of 
group  work. 

A second  result  refers  to  the  generalization  (and  the  qualifying  antecedents)  of  the  benefits  of  project-based 
peer  learning  as  put  forward  in  educational  research  (e.g.  Slavin,  1996).  On  the  one  hand,  it  seems  that 
relatively  large  groups  of  learners  are  not  necessarily  dysfunctional,  at  least  on  condition  they  are 
composed  of ‘high  achieving’  and  (relatively)  ‘noncooperatively  oriented’  students.  On  the  other  hand -and 
in  strong  contrast  with  our  hypothesis  - large  groups  that  merely  consist  of  extreme  cooperatively  oriented 
students  perhaps  run  the  risk  to  become  dysfunctional.  Asking  students  in  the  dysfunctional  group  to  try 
and  give  a reason  for  the  severe  problems  encountered,  they  indeed  refer  to  explanations  that  can  be  linked 
up  with  a ‘cooperative’  or  ‘pro-social  oriented’  social  orientation.  For  example,  one  subject  described 
extreme  frustration  that  resulted  from  writing,  re-writing  and  writing  again  texts  in  order  to  take  into 

account  all  group  members ' opinions  and  how  everybody  seemed  to  hesitate  cutting  knots  for  the  same 

reason"'  Additionally,  knowledge  of  both  learners’  social  orientations  and  their  level  of  academic 
achievement,  might  direct  both  anticipatory  and  supportive  actions  of  instructors  coaching  learners  working 
together  in  (large)  project  groups. 

Finally,  from  the  point  of  view  of  social  psychologist,  the  reported  findings  might  question  the 
unconditional  generalization  of  the  benefits  of  a pro-social  orientation.  Social  psychologists  have  stressed 
the  benefits  of  a cooperative  social  orientation  in  a very  broad  range  of  social  settings  (e.g.  Van  Lange  et 
al.,  1997b;  Van  Vugt  et  al.  1997).  Without  putting  too  heavy  a weight  on  our  findings  (based  on  a limited 
sample  it  surely  needs  to  be  replicated)  we  however  will  suggest  to  further  investigate  the  boundary 
conditions  of  the  presumed  benefits  of  a cooperative  social  orientation.  Perhaps  in  the  context  of  project 
groups,  the  absence  of  at  least  one  non-cooperative  group  member  (or  someone  that  can  fulfill  the  role  of  a 
leader?)  is  detrimental  for  the  functioning  of  that  group. 
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Abstract:  This  study,  currently  in  progress,  examines  how  several  groups  of  users  have 
made  use  of  our  Interactive  Paper  Project  (IPP)  online  resource  (Buell  & Levin,  1999; 
McCollum,  1999)  in  varied  academic  settings.  Situated  evaluations  (Bruce  & Peyton, 
1993)  involve  analysis  of  three  essential  elements:  the  idealization  of  the  innovation,  the 
settings  in  which  it  appears,  and  the  realizations  within  each  setting.  The  study  explores 
IPP  use  in  five  mini-cases,  involving  cross-classroom,  conference  and  journal-editing 
implementations. 


Introduction 

The  Interactive  Paper  Project  resource  (Buell  & Levin,  1999;  McCollum,  1999)  is  a web-enabled 
database  that  permits  authors  of  papers  to  upload  them  to  the  web  and  allows  readers  to  submit  written 
comments  at  specific  points  in  the  body  of  the  paper.  These  comments  are  immediately  available  for 
viewing  as  part  of  the  paper  itself  A two-level  commentary  structure  allows  for  both  comments  linked  to 
various  segments  of  the  paper,  and  responses  to  those  comments.  Figure  1 shows  part  of  an  IPP  text,  with 
comments  and  responses  interspersed. 


For  example,,  students  can  reuise  the  original  procedtuos  to  prod  we  more  versa tilaOQSSIP  programs.  They  can  brsai  apart  the  predication 
into  txanciUve  verbs  with  objects,  or  erjpiind  the  range  of  poisibto  svibjccb,  can  add  c^ndibonalacfaonr  to  ihs  procodurts,  for  irutanco, 
Uiat  c>n}y  certain  people  can  do  p&rit^u^  acbosii.  As  Utey  construct  tLeir  GOSSIP  programs,  they  are  forced  to  cortfronl  fundamental 
<^iLisaUom  such  'What  ts  the  fekUomhlp  between  syntax  and  serriantics?,"  "Wkwit  Is  a word?/  or  *V/hat  makes  a sentence 

interesUr^?*  White  llte  program  has  no  mfrans  for  arawerln^  each  qusiti^nir  »t  provides  an  environment  In  whsch  students  can  caek  answers 
themseJvetf  it  oltews  them  to  see  the  consequences  of  thsu  own  hypotfteses  cibout  tes^age. 

Commeuta.rles  ^ 

Tcieca  Own!  I used  « progremthaA.  hod  features  that  were  sunder  to  this  with  thml  gre  dei* . They  put  in  tnessing  odpoavtOr  advtrbertreitvrnnd 

rveurr  Un  icrfteccES  to  mefee  them  uttercstJc^  and  cnafcetKemlmvenieaiiLng. They  leeirred  end  hadfunatUieearrietime^ 

Christine  Smifh  Quly:?,  > When  1 wai  we  colled  these  Uhi . 

^cnrsfrent  J 


Add  yourown  comment 


Figure  1:  Portion  of  an  Interactive  Paper,  with  a comment  and  a response  displayed  'inline.' 

The  IPP  is  one  of  a variety  of  types  of  online  resources  allowing  readers  to  add  to  or  comment  on 
an  internet-accessible  document.  Others  include  Usenet  newsgroups,  bulletin  boards  such  as  WebBoard 
(www.webboard.com),  and  weblogs,  or  "blogs"  (www.blogger.com).  The  IPP  structure  places  more 
emphasis  on  the  original  document  than  these  other  alternatives;  different  "views"  allow  the  reader  to  view 
only  the  original  text,  to  view  the  text  with  comments  rendered  "inline"  (interspersed  with  the  original  text), 
or  to  view  the  comments  as  "linked"  via  a pulldown  menu. 

Interactive  Papers  were  initially  developed  as  an  online  version  of  academic  "in -progress  papers," 
draft  documents  passed  from  one  colleague  to  another  for  comments  and  advice.  While  this  has  been  one 
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major  use  to  which  the  IPP  has  been  put,  a review  of  the  documents  created  by  IPP  users  over  the  past  two 
years  shows  many  more  uses  as  well. 


The  Study 

Our  study  examines  the  IPP  from  the  viewpoint  of  Situated  Evaluation  (Bruce  & Peyton,  1993),  an 
analytical  framework  devised  to  evaluate  "innovations  in  use"  across  in  different  settings.  Situated 
evaluations  involve  analysis  of  three  essential  elements:  the  idealization  of  the  innovation,  the  settings  in 
which  it  appears,  and  the  realizations  within  each  setting. 

In  the  case  of  the  IPP,  an  innovation  originally  idealized  as  an  online  white-paper  delivery  system 
has  been  used  not  only  for  that  purpose,  but  also  as  a formal  journal-editing  mechanism,  an  avenue  for 
informal  communications  between  individual  students  and  professors,  an  adjunct  to  a national  conference, 
and  a launching  platform  for  segmented  streaming  video  files.  It  has  served  audiences  ranging  from 
individuals  sharing  an  office  or  classroom,  to  distance  education  students  in  the  same  course,  to  academic 
professionals  on  different  campuses  around  the  country.  In  several  instances,  developers  have  modified  the 
basic  IPP  mechanism  to  better  serve  those  interests  and  audiences;  some  modifications  have  afterward  been 
applied  to  the  basic  IPP,  while  others  have  not. 

Several  mini-cases  make  up  our  situated  evaluation  if  the  IPP.  One  involves  a single  paper  that  has 
garnered  more  than  300  comments,  mostly  from  students  in  classes  taught  by  one  of  that  paper's  co-authors. 
A second  case  involves  thirteen  different  papers  presented  at  a national  conference  in  Fall,  2000;  these 
papers  were  made  available  in  IPP  format  to  conference  attendees  beforehand,  and  readers  entered  several 
hundred  comments  and  responses.  A third  case  looks  at  how  the  IPP  has  been  adapted  to  serve  as  the 
primary  means  by  which  an  online  journal  prepares  articles  for  publication;  article  drafts  are  made  available 
to  reviewers  for  their  comments  in  a two -phase  process  which  initially  keeps  comments  private,  then  makes 
them  available  to  article  authors  and  fellow  revievvers.  A fourth  case  examines  how  members  of  a cross- 
institutional  research  group  are  making  use  of  the  IPP  as  one  of  several  mechanisms  for  promoting  multi- 
site collaboration.  A fifth  case  describes  how  the  IPP  structure  has  been  modified  to  construct  "interactive 
multimedia  papers,"  by  adding  features  for  selecting  and  segmenting  streamed  video  files  so  that  these  can 
be  viewed  and  commented  upon. 


Implications 

Our  ongoing  situated  evaluation  of  the  IPP  is  helping  us  understand  better  how  the  tool  is  being  defined  in 
use,  and  guiding  us  toward  further  revisions  and  improvements. 
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Abstract: 

An  instructional  application  consists  of  a set  of  resources  and  activities  that  implement 
interacting,  interrelated  and  structured  experiences  oriented  towards  achieving  specific 
educational  objectives.  Computer-based  instructional  applications  have  to  be  faced  as  any  other 
development  activity  following  a well  defined  process.  With  this  purpose  some  design  methods 
for  computer-based  instructional  applications  have  been  proposed.  However  most  of  them  are 
focused  on  "on-line"  courseware  structures  which  are  quite  rigid  as  far  as  they  have  serious 
shortcomings  to  deal  with  courses  structured  in  multiple  ways.  Moreover,  these  methods  usually 
lack  a specific  mechanism  to  model  instructional  concepts  and  strategies.  This  work  proposes  a 
design  framework  to  develop  multi-structured  instructional  applications  combining  a didactic 
model  with  a software  engineering  approach  to  deal  with  educational  and  technical 
requirements.  The  underlying  model  extends  knowledge  structures,  such  as  those  involved  in 
the  Merrill’s  Transaction  Instruction  Theory,  adding  them  didactic  information.  It  also  considers 
the  functional  aspects  of  these  structures.  An  XML-based  notation  is  proposed  to  represent  such 
structures  and  their  management. 

1.  Introduction 

An  instructional  application  can  be  defined  as  a set  of  resources  and  activities  which  implement  interacting, 
interrelated,  structured  experiences  that  are  designed  to  achieve  specific  educational  objectives.  Instructional 
applications  are  usually  structured  by  means  of  static  patterns  which  are  based  on  a sequence  of  book-like 
electronic  pages.  They  are  designed  using  courseware  methodologies  that  focus  on  presentation  and  navigation 
issues  but  lack  a didactic  basis.  The  resultant  products  are  mostly  "pretty-printing"  on-line  courses  but  didactic 
aspects  are  hardly  considered.  The  current  work  proposes  a design  framework  which  is  independent  from 
computer-based  delivery  technologies  and  it  allows  the  representation  of  multiple  didactic  structures  from  an 
instructional  point  of  view.  Such  a framework  requires  a model  to  deal,  on  one  hand,  with  instructional  design 
topics  and,  on  the  other  hand,  with  the  design  of  computer-based  applications. 

Instructional  design  can  be  defined  as  the  discipline  that  connects  descriptive  theories  with  instructional  practice. 
Among  the  different  instructional  theories  proposed  in  the  literature,  we  will  assume  the  Instructional  Transaction 
Theory  (Merrill  1996)  since  it  makes  possible  to  figure  out  the  relationships  between  educational  and  technical 
components.  Merrill’s  mental  models  consist  of  two  major  components:  knowledge  structures  (schema)  and 
processes  for  using  this  knowledge  (mental  operations).  The  Merrill's  hypothesis  states  that  adequate  instruction 
would  require  multiple  types  of  knowledge  structures  to  be  identified  and  made  explicit  to  the  learner.  Thus,  the 
Instructional  Transaction  theory  provides  a powerful  tool  to  structure  the  knowledge  about  a given  topic  and  to 
define  the  procedures  for  accessing  it.  In  the  framework  proposed  in  this  paper,  Merril’s  model  is  extended  to  add 
didactic  information  to  knowledge  objects  and  structures.  A Didactic  model  is  built  using  the  previous  entities  as 
the  basis  and  assigning  them  attributes  whose  values  suggest  how  they  can  be  used  or  even  adapted  in  a specific 
learning  or  teaching  scenario. 

On  the  other  hand,  the  proposed  framework  deals  with  the  design  of  educational  computer-based  applications. 
Most  of  the  existing  methodologies  are  addressed  to  author  courseware  using  formats  such  as  HTML  documents. 
They  seem  adequate  for  presenting  and  accessing  information  in  a open  environment  like  the  Web  but  lack  a data 
structuring  capability.  Another  option  is  using  proprietary  multimedia  authoring  tools  like  Authorware'*”'^  or 


Tooolbook'^^  that  provide  a more  powerful  data  model  but  they  bound  the  reuse  and  exchange  of  content  module 
(Wiest,  2001).  In  this  sense,  an  important  effort  has  been  made  to  organize  and  to  manage  educational  data  using 
metadata  (Duval  2001).  However,  standard  metadata  proposals  mainly  aim  at  the  reuse  and  exchange  of  learning 
material  and  they  are  not  directly  involved  in  the  context  of  instructional  application  design.  In  the  current  work, 
notational  systems  based  on  metadata  are  used  to  represent  instructional  and  didactic  entities.  The  proposal 
consists  of  defining  an  XML-based  notation  which  specifies  the  Merrill's  Instructional  Transaction  Theory 
entities  and  their  extension  introduced  in  our  Didactic  model.  Such  notation  will  allow  an  instructor  to  define  his 
own  didactic  structures  which  can  be  built  from  a common  repository  of  instructional  objects. 

The  remainder  of  the  paper  is  organized  as  follows.  Second  section  revises  some  related  works.  Third  section 
introduces  the  design  framework  proposed  (n  the  current  work.  Such  framework  is  based  on  a didactic  model 
whose  structural  components  are  presented  in  fourth  section.  Fifth  section  complements  this  model  with  its 
functional  view.  An  application  example  is  described  in  section  6.  Finally,  section  7 presents  some  remarking 
conclusions. 

2.  Related  work 

There  are  several  technology-based  educational  initiatives  that  propose  a separation  between  didactic  aspects  and 
content  related  issues.  In  this  context,  the  Multibook  project  §teinacker  et  al.  1999)  considers  two  domain 
spaces:  the  Concept  Space  and  the  Media  Brick  Space.  The  first  one  contains  a network  of  knowledge  topics 
which  are  connected  via  semantic  relations.  The  second  one  contains  information  units  of  various  multimedia 
formats.  Media  Brick  elements  are  linked  to  the  Concept  entities  and  instructional  mechanisms  such  as 
“example”,  “deepen”  or  “explain”  are  setup  between  these  elements.  A similar  approach  is  proposed  in  LMML 
(SiiB  et  al  2000)  which  differentiates  Pedagogical  and  Instructional  properties  from  the  Module  and  Content 
objects.  However,  only  strategy  attributes  with  possible  constant  values  “beh”  (behaviouristic)  or  “con” 
(constructivistic)  can  be  specified.  Other  proposals  such  as  the  Targeteam  project  5'eege  2000)  or  the  Palo 
language  (Rodriguez  et  al  1999)  are  also  constrained  to  relations  or  mechanisms  such  as  motivation,  illustration, 
exercise  or  explanation  without  an  additional  didactic  value. 

In  metadata  contexts,  there  are  Educational  Modeling  Languages  (EML)  which  allow  the  specification  of  many 
kinds  of  educational  data.  These  data  can  be  implemented  using  notations  which  take  advantage  of  standard 
proposals  on  one  hand,  or  particular  and  specific  proposals  on  the  other  hand.  The  first  option  is  used  in  the 
Multibook  project  to  define  the  Media  Brick  elements  from  IEEE  Learning  Objects  or  the  Chameleon  project 
and  itsTeachML(Wehner  2001)  notation  that  is  based  on  IMS  standards.  These  standard  formats  are  useful  for 
exchanging  them  in  different  learning  contexts  but  their  didactic  attributes  are  very  restrictive.  In  the  second 
category,  there  are  formats  such  as  LMML  that  defines  instructional  ContentObjects  which  contain  media  units 
such  as  tables,  lists,  images  or  text.  The  problem  is  that  these  units  are  strongly  coupled  with  instructional  objects 
and  it  prevents  to  assign  them  with  different  media  objects.  Similar  problems  are  found  on  other  EML  proposals 
such  as  Targeteam  contents,  EML  learning  objects  (Koper  2001)  or  Palo  elements. 

3.  Design  of  Instructional  Applications 

Figure  I shows  the  global  architecture  in  the  proposed  instructional  application  design  framework.  The  upper 
level  deals  with  Instructional  Design  Theory  which  manages  entities  such  as  "knowledge  objects",  "knowledge 
structures"  and  "transaction  shells".  According  to  Merrill,  a knowledge  object  is  defined  as  “a  precise  way  to 
describe  the  content  to  be  taught”.  Knowledge  objects  can  be  combined  into  knowledge  structures.  Knowledge 
structures  are  external  representations  of  knowledge  that  are  parallel  with  mental  models  that  in  turn  are  internal 
(cognitive)  representations  of  models.  Transaction  shells  consist  of  rules  for  selecting  and  sequencing  knowledge 
objects.  The  entities  defined  in  the  Merrill's  model  are  the  basis  for  the  next  level  (Instructional  Design 
Modeling).  This  central  level  is  characterized  by  a Didactic  Model  whose  components  extend  the  previous 
entities  with  didactic  information.  They  are  divided  in  two  categories  that  represent  the  structural  and  functional 
model,  respectively.  Structural  model  is  composed  by  instructional  objects,  derived  from  knowledge  objects,  and 
didactic  structures  which  extend  knowledge  structures.  Functional  model  is  based  on  instructional  tasks  and 
learning  scenarios.  An  EML  notation  is  being  designed  to  specify  both  structural  and  functional  components. 
Next  section  will  describe  them  in  depth.  The  lower  two  levels  deal  with  the  computational  implementation  of 
instructional  applications.  The  first  one  is  based  on  a hypermedia  model  to  represent  the  instructional  entities  and 
their  relationships  in  an  formal  and  abstract  notation  puendia  et  al  2001).  The  second  level  is  related  to  the 
technology  involved  in  the  delivery  of  instructional  applications  using  an  e-leaming  environment. 


Figure  1.  Instructional  Application  Design  Architecture. 


4.  Structural  model 

This  section  focuses  on  the  structural  components  of  the  Didactic  Model  proposed  in  this  paper.  The  simplest 
element  of  the  current  model  is  the  Instructional  Object  (10)  which  is  used  to  manage  Merrilfs  knowledge 
objects  from  a learning  point  of  view.  Instructional  Transaction  Theory  describes  knowledge  in  terms  of  three 
types  of  knowledge  objects:  entities,  activities  and  processes.  When  an  entity,  activity  or  process  is  addressed  in  a 
learning  context,  there  are  multiple  aspects  that  differ  depending  on  the  learning  conditions  around  the  knowledge 
object.  The  main  goal  of  10  elements  is  to  represent  such  aspects.  If  the  learning  target  is  an  entity,  lOs  such  as 
definitions,  statements  or  examples  can  be  used  to  describe  it.  Activity  knowledge  items  can  be  characterized 
using  lOs  such  as  exercises,  questions,  analogies,  hints  an  so  on,  and  finally,  processes  are  also  qualified  with 
simulations,  feedback  elements,  or  animations. 

lOs  are  not  only  isolated  information  units  such  as  definitions  or  exercises  but  they  are  also  concerned  with  the 
way  these  units  are  used  in  a learning  context.  For  instance,  an  entity  definition  is  assigned  with  a difficulty  level, 
a statement  declares  a property  value  assigning  it  a relevance  index  and  an  example  displays  the  entity  with  a 
specific  portrayal  (text,  audio,  image,  and  so  on).  Therefore,  lOs  must  consider  the  multiple  possibilities  for  a 
knowledge  object  to  be  learned  in  order  to  deal  with  different  learning  styles  and  needs. 

The  model  presented  in  this  paper  provides  a flexible  framework  to  incorporate  multiple  kinds  of  didactic 
information  which  can  be  added  to  lOs.  The  instructor  has  to  decide  the  different  difficulty  or  abstraction  levels 
assigned  to  each  object,  its  relevance  in  a certain  learning  context  or  the  portrayal  that  requires  a given  user 
profile.  This  design  process  is  highly  related  to  the  features  of  a subject  domain  and  it  becomes  a manual  and 
laborious  process.  Tools  as  metadata  notations  have  been  used  to  support  this  process. 

In  this  work,  there  is  a simple  XML  notation  proposal  and  the  definition  of  lOs  is  adapted  according  to  the 
subject  domain.  This  means  that  they  can  be  defined  on  top  of  other  entities  such  as  Merrill’s  knowledge  objects 
or  IEEE  learning  objects,  extending  them  with  didactic  elements  as  those  referred  in  Figure  2.  In  this  example,  a 
Description  10  is  used  to  teach  the  "magnetic  disk  structure"  (knowledge  object).  The  “ObjectDescription”  shows 
that  an  image  representation  has  been  selected  in  the  current  teaching  context.  Didactic  parameters  as  the 
portrayal  configuration  or  the  abstraction  type  indicate  how  this  image  is  displayed  and  the  level  of  abstraction  it 
represents.  EML  notations  usually  also  include  information  about  the  organization  of  the  educational  contents  in 
different  structures.  In  some  cases  like  the  Multibook  project  this  information  is  mixed  with  the  own  instructional 
objects.  In  our  model,  there  is  a strong  separation  between  both  types  of  information. 

Instructional  objects  are  organized  \xs\ng  Didactic  Structures  (DS)  which  can  be  managed  as  independent  entities. 
These  entities  are  addressed  to  capture  the  didactic  relationships  between  those  objects.  There  are  two  kinds  of 
didactic  relationships  in  the  current  model:  explicit  and  implicit.  Explicit  relationships  link  lOs  using  a specific 
action,  for  instance,  “an  example  illustrates  a concept  definition”  or  “a  question  evaluates  an  explanation”. 
Implicit  relationships  are  derived  from  the  way  knowledge  items  are  organized. 
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<ComponentValue  DidacticRef-' Image  reference^‘> 

<ObjectRef>Magnetic  Disk  Structure  reference  (storage  area).</ObjectRef> 

<lnstructionalObject  ObjectlD="  10-404"  ObjectName-' Description"  ObjectDescription="Bi-dlmensional  image"> 
<Portrayal  Dimension="2"  Background-Talse"  Position="center"/> 

<Abstraction  Type="structurar  Depth="07> 

<l\ nstructionalObject> 

</ComponentValue>  

Figure  2.-  Instructional  object  example. 

In  this  case,  we  are  interested  in  knowledge  structures  coming  from  the  Instructional  Transaction  Theory  which 
are  used  in  the  current  work  to  model  DSs.  Merrill  mentions  different  types  of  knowledge  structures  such  as  lists, 
taxonomies,  dependencies,  algorithms  and  causal  nets.  A DS  can  be  built  on  top  of  one  or  more  knowledge 
structures.  For  instance,  the  description  of  a magnetic  disk  in  a computer  system  can  be  based  on  identifying  its 
components  and  assigning  them  a portrayal  configuration.  It  can  also  include  an  algorithm  representation  to  show 
the  access  to  a specific  component  (e.g.  a cylinder).  The  DS  connections  with  other  model  entities  are  represented 
on  Figure  3 It  shows  a simplified  UML  diagram  which  specifies  that  DSs  are  entities  aggregated  from 
instructional  objects  (10)  and  knowledge  structures  (KS).  Both  10  and  KS  entities  are  based  on  knowledge 
objects  (KO)  which  are  composed  by  elements  such  as  Identify  Portrayal  and  Properties.  DS  entities  also  extend 
the  basic  information  coming  from  10  and  KS,  using  didactic  attributes  such  as  the  portrayal  selection. 


Figure  3,-  Structural  Design  Model  entities. 


The  current  work  is  closer  to  the  IMSDL  proposal  l^ilberhom  & Gaede  1999)  which  defines  instructional 
strategies,  responsible  for  structuring  the  information  units  to  be  learned  but  independent  from  the  subject 
domain.  However,  these  structures  are  addressed  mainly  for  courseware.  Nevertheless,  we  do  not  know  any 
proposal  focused  on  generating  structure  templates  which  can  be  applied  in  specific  didactic  contexts  in  a 
reusable  and  modular  way.  We  are  developing  XML  Schemas  to  represent  a wide  range  of  didactic  structures 
from  basic  knowledge  structures. 

5.  Functional  model 

The  functional  model  describes  the  entities  that  allow  a user  (student  or  instructor)  to  interact  with  DSs.  DS 
entities  are  the  nexus  with  the  other  structural  elements.  Figure  4 shows  a simplified  UML  diagram  that 
represents  the  main  functional  entities  and  their  relationships  with  the  remainder  entities.  These  entities  are 
learning  scenarios  and  instructional  tasks,  respectively. 

Learning  scenarios  (LS)  are  defined  as  the  set  of  terms  and  conditions  that  characterize  the  user  learning.  Each 
individual  user  or  group  of  them  is  assigned  with  one  or  more  LSs.  Instructors  have  to  configure  the  LS  entities, 
assigning  features  such  as  learning  modes,  timing  schedules,  instructional  methods  and  learning  goals.  Each  LS 
aggregates  one  or  more  Instructional  Tasks  (IT).  They  are  defined  as  the  operations  a user  has  to  perform  to 
achieve  a specific  learning  goal.  IT  entities  model  the  functions  associated  to  the  didactic  structure  interface. 
They  can  access  to  one  or  more  didactic  structures  and  a certain  DS  can  be  attached  to  multiple  ITs. 
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ITs  are  related  to  the  Merrill’s  concept  of  "transaction  shell"  which  consists  of  rules  for  selecting  and  sequencing 
knowledge  objects.  In  the  current  model,  IT  entities  do  not  access  directly  to  lOs  which  are  encapsulated  into  the 
didactic  structures.  This  feature  eases  the  design  process  because  the  instructor  deals  with  lOs  in  a more  abstract 
way.  For  instance,  in  a dependency  DS,  the  10  components  are  all  considered  as  chain  elements  without 
differentiating  them.  An  IT  like  Explain  specifies  the  navigational  operations  through  the  branches  of  the 
Dependency  structure.  A navigational  operation  can  consist  in  the  selection  of  a given  branch  and  it  could  depend 
on  learning  modes  such  as  “learning  by  examples”,  “by  doing”  or  by  "exploration  and  experimentation".  In  the 
first  case,  the  Explain  task  is  based  on  revising  Example  lOs  while  the  other  cases  involve  the  working  with 
Activity  10s.  One  of  the  main  instructor  responsibilities  is  to  define  the  ITs  attached  to  each  DS.  This  definition  is 
also  a manual  and  laborious  task  and  an  EML  notation  is  being  developed  to  assist  the  instructor  in  this  process. 

6.  Application  example. 

In  this  section,  an  example  is  used  to  show  the  application  of  the  proposed  Didactic  model.  The  example  consists 
of  a Didactic  Guide  called  "Learning  XML  basics"  which  is  based  on  a Dependency  structure.  Figure  5 shows  a 
diagram  of  top-down  dependency.  The  root  node  represents  the  main  learning  goal  described  as  "Learning 
XML".  The  descendent  node  has  assigned  a Definition  10  which  intends  to  answer  the  "What  is  XML"  question. 
The  didactic  relationship  between  the  Goal  and  Definition  lOs  is  an  IsBasedOn  relationship  type  and  it  means 
that  the  knowledge  about  the  XML  notion  (current  node)  is  required  to  meet  the  goal  defined  in  the  previous 
node.  From  the  current  node,  there  are  several  branches  which  represent  extensions  {IsExtendedBy  relationships) 
of  the  current  definition  node.  These  branches  store  requirements  in  order  to  understand  the  previous  node  and 
each  one  has  assigned  a specific  competency  (difficulty)  level.  According  to  this  assignment,  a different  IT  can 
be  attached  to  each  branch.  These  competency  levels  are  based  on  the  learning  modes  cited  previously.  Learning 
by  examples  can  involve  identifying  instances  of  a concept,  e.g.  an  XML  document  (left  branch)  or  a DTD 
document  (central  branch).  Moreover,  each  XML  document  example  can  be  assigned  with  a certain  difficulty 
level  which  is  used  as  parameter  by  the  IT.  It  causes  that  the  structure  branches  can  be  navigated-accessed  in 
different  ways.  K Predict  IT  can  check  a specific  process,  e.g.  the  validation  of  an  input  XML  document  using  a 
DTD  syntax.  This  validation  process  is  based  on  din  Activity  10  which  is  related  to  a previous  Example  object  by 
means  of  an  IsCheckedBy  didactic  relationship.  In  a higher  level  learning  mode,  the  Predict  task  could  include 
additional  activities  such  as  the  DTD  configuration. 

7.  Conclusions 

A design  framework  has  been  proposed  for  developing  instructional  applications  beyond  the  rigid  courseware 
structures  that  underlie  the  typical  Web-based  courses.  The  proposed  framework  is  based  on  a didactic  model 
which  provides  a gateway  between  instructional  theories  and  concepts,  and  hypermedia  and  Web  application 
engineering.  This  model  considers  two  issues:  a structural  view  based  on  instructional  objects  and  didactic 
structures,  and  a functional  view  that  manages  the  previous  entities  using  instructional  tasks  and  learning 
scenarios.  The  didactic  model  is  supported  by  an  XML-based  notation  which  eases  its  translation  to  computing 
environments. 
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Figure  5.-  Didactic  Structure  example. 


This  notation  has  been  applied  to  represent  taxonomy-like  didactic  structures.  It  provides  the  possibility  to 
organize  available  educational  resources  in  a way  closer  to  the  instructor  teaching  requirements.  We  have  also 
planned  to  extend  the  XML-based  specification  to  other  didactic  structures  based  on  dependencies  and 
algorithms.  Further  works  include  their  usage  in  specific  learning  contexts  such  as  Electric  and  Information 
Engineering  areas.  We  are  also  developing  a tool  to  allow  the  access  to  these  structures  in  a Web-based 
environment. 
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Abstract:  In  this  paper  we  propose  a construction  kit  approach  that  could  be  used  as  an  abstract 
model  to  implement  modular  defined  e-leaming  systems.  The  metaphor  construction  kit  results  from 
the  math-kit  project,  that  provides  students  and  lecturers  with  multimedia  support  for  central  topics 
in  undergraduate  mathematics.  The  lecturers  can  combine  different  learning  units  into  individual 
courses  as  easily  as  bricks  of  a construction  kit  can  be  used  to  build  a model.  Thus,  a learning  unit 
which  is  developed  by  one  author  could  be  used  by  a number  of  other  authors  as  well.  Furthermore, 
learning  objects  must  be  defined  independent  of  style  and  format,  because  the  definition  of  design 
and  style  makes  it  impossible  to  integrate  them  into  different  courses.  Therefore,  we  are  using  XML 
as  the  description  language  to  define  learning  content  in  a structural  manner.  Our  modular  concept 
is  based  on  the  Sharable  Content  Object  Reference  Model  (SCORM). 


Introduction 

When  teaching  mathematics  to  undergraduates  and  non-mathematical  students,  most  lecturers  agree  that  the  use  of 
computers  in  education  is  very  helpful.  Despite  this,  most  of  them  don’t  use  computers  in  their  standard  lectures.  The 
reason  for  this  is  that  they  lack  the  necessary  technical  knowledge  to  develop  and  integrate  computer  based  learning 
units  to  their  lectures.  Hence,  we  are  developing  a multimedia  construction  kit,  called  “math-kit”,  that  supports 
lecturers  and  students  with  mathematical  and  technical  content  (Unger  et  al.  2002).  Math-Kit  is  a joint  project,  which 
is  being  developed  at  the  German  universities  of  Hamburg,  Bayreuth,  Hagen  and  Paderborn.  The  developed  system 
provides  a simple  mechanism  to  find  and  combine  different  learning  objects  to  compose  them  to  learning  units  and 
courses.  These  learning  units  are  able  to  be  used  by  lecturers  for  presentation  as  well  as  by  students  for  exploration. 

In  a joint  project  the  creation  of  uniform  learning  content  is  very  difficult  to  achieve.  Often  different  developers 
produce  learning  objects  which  are  based  on  varying  description  languages  such  as  HTML  or  PDF.  Integrating 
learning  objects  into  existing  courses  requires  a lot  of  revision  work  to  achieve  consistency  of  appearance  and 
content.  To  find  an  optimal  solution,  several  learning  platforms  and  authoring  tools  were  evaluated  but  none  of  these 
met  our  requirements. 

The  Construction  Kit 

Nowadays,  object  oriented  software  engineering  is  a popular  programming  paradigm  and  provides  objects  as  units  of 
modeling.  Regrettably,  this  level  of  modeling  is  insufficient  for  the  development  of  complex  software  systems  with 
human  interaction.  To  bridge  the  gap  between  object  and  technical  models,  we  use  the  metaphor  concept  in  our 
project.  A metaphor  is  a linguistical  term  which  uses  a figurative  meaning  in  a different  context.  This  helps  both  the 
development  team  and  the  user.  The  developer  team  can  gain  better  understanding  of  the  concrete  problem  which  is 
to  be  solved  by  a software  system.  When  the  system  is  finished,  the  user  benefits  from  a intuitive  usage  model.  For 
math-kit,  we  introduce  the  construction  kit  term  as  metaphor  for  our  e-leaming  system. 

To  get  a better  understanding  of  this  metaphor,  think  of  a LEGO  construction  kit  v^th  all  contained  bricks.  Mostly, 
the  construction  kit  contains  a construction  plan  for  several  models  which  can  be  built  with  the  same  bricks.  But  the 
special  characteristic  of  the  bricks  is  the  patented  stud-and-tube  coupling  system.  This  coupling  principle  allows 
stable  models  and  a multitude  of  combinations  — no  matter  how  the  bricks  are  colored,  shaped,  or  sized.  Hence,  it  is 
also  possible  to  combine  bricks  of  different  construction  kits  in  one  model.  Thus,  the  contained  construction  plan  is  a 
tiny  subset  of  unlimited  possibilities. 

To  make  a connection  between  metaphor  and  planned  software  system  we  describe  the  demands  and  external 
circumstances.  The  main  goal  is  the  creation  of  a system  that  helps  both  students  and  lecturers  to  improve  their  work. 
Lecturers  need  pedagogic  concepts  and  human  intelligence  to  create  reasonable  content.  The  software  market 
provides  a lot  of  solutions  which  support  content  creation  e.g.  Adobe  Framemaker,  Microsoft  PowerPoint,  and 
Macromedia  Flash.  Most  of  these  programs  provide  special  functionality  for  specific  problems  and  have  weaknesses 
in  other  tasks.  It  makes  sense  to  combine  the  strengths  of  all  text,  graphic,  sound  and  multimedia  tools.  Therefore,  we 
use  the  Sharable  Content  Object  Reference  Model  (SCORM)  (Dodds  2001)  which  defines  a Sharable  Content  Object 
(SCO)  as  the  atomic  entity  of  a Learning  Management  System  (LMS).  Information  about  the  contents  of  the  SCO  is 


defined  in  extra  data  called  Learning  Object  Meta-Data.  The  SCO  wraps  any  kind  of  files,  called  assets,  and  can 
communicate  with  the  LMS  via  a specified  API.  The  SCO  fits  into  our  metaphor  as  the  brick  in  our  construction  kit. 
The  combination  of  several  bricks  or  SCOs  is  realized  through  a separate  mechanism  called  Content  Packaging.  The 
backbone  of  a content  package  is  a manifest  which  defines  the  paths  to  the  SCOs,  additional  meta-data  and  one  or 
more  structures  of  the  built  course.  In  contrast  to  the  LEGO  bricks,  which  include  their  own  coupling  system,  the 
SCOs  need  the  manifest  as  additional  “glue”.  Hence,  in  view  of  our  metaphor,  a built  model  or  course  is  described  by 
a content  package.  Every  LMS  that  complies  to  the  SCORM  standard  should  be  able  to  process  the  produced 
packages.  The  metaphor  of  the  construction  kit  helps  us  to  understand  how  course  development,  implementation  and 
management  should  be  handled  by  a technical  system.  Two  cases  still  remain  that  are  not  covered.  One  problem  is 
the  design  and  implementation  of  well  structured  SCOs.  As  already  mentioned,  math -kit  is  a joint  project  and  needs  a 
strict  separation  of  content  and  presentation  to  produce  courses  with  a uniform  appearance.  The  use  of  the  Extensible 
Markup  Language  (XML)  and  the  Extensible  Stylesheet  Language  Transformations  (XSLT)  can  help  to  force  the 
wished  separation  (Farinetti  et  al.  2000).  XML  allows  the  definition  of  a specialized  markup  language  with  tags  for 
mathematical  content  like  formulas,  definitions,  and  proofs.  Such  a restricted  language  helps  the  authors  to 
concentrate  on  the  content  and  hides  unnecessary  style  details.  The  complete  XML  document  can  be  transformed  by 
XSLT  into  a wide  spread  output  format  like  HTML  or  PDF.  This  approach  has  the  benefit  of  a consistent  and 
individual  style  of  the  whole  course,  even  if  foreign  content  is  used. 

Another  problem  not  covered  by  the  metaphor  is  the  publishing  of  complete  courses.  A construction  kit  doesn’t 
support  the  replication  and  distribution  of  built  models.  Although,  the  LMS  couldn’t  fit  into  the  construction  kit 
metaphor,  math-kit  is  a complete  system  for  the  distribution  of  mathematical  content.  However,  our  main  focus  lies 
on  the  development  of  an  authoring  tool  for  e-leaming. 

The  Roles 

We  introduce  roles  to  model  the  different  views  of  our  e-leaming  system.  It  is  important  to  comprehend  that  one 
person  can  appear  in  several  roles.  The  roles  help  us  to  model  the  static  and  dynamic  behavior  of  our  system  with  use 
cases  and  sequential  diagrams  (Bergner  et  al.  1998).  This  allows  us  to  have  a stringent  and  controllable  development 
process.  Math-Kit  comprises  six  different  roles:  lecturer,  student,  composer,  learning  object  developer,  publisher,  and 
administrator. 

The  lecturer  uses  single  blocks  or  complete  courses  to  extend  his  own  slides  or  to  create  new  ones.  Individuals  in  the 
role  student  want  to  learn  and  can  explore  the  entire  database  to  use  learning  objects  and  courses  of  interest.  These 
two  roles  don’t  depend  directly  on  the  construction  kit  metaphor,  because  they  work  with  their  favorite  LMS. 
Nevertheless,  the  construction  kit  is  the  foundation  of  the  created  content.  The  composer  uses  math-kit  to  combine 
existing  learning  objects  to  complete  courses.  In  relation  to  the  construction  kit  metaphor  the  composer  defines  the 
construction  plan.  The  learning  object  developer  designs  and  implements  new  learning  objects.  In  view  of  the 
metaphor,  the  learning  object  developer  creates  the  several  bricks.  In  context  of  math-kit,  the  learning  content  is 
coded  in  XML  with  a structure  for  mathematics.  This  XML  code  has  to  be  translated  into  a readable  format  by  the 
role  publisher.  The  last  role  administrator  installs  basic  packages  and  software  for  the  construction-kit. 

Assessment  and  Outlook 

The  use  of  the  construction  kit  metaphor  and  roles  should  support  us  in  planning  and  developing  an  e-leaming 
system  for  the  creation  and  distribution  of  mathematical  and  technical  content.  The  benefit  of  this  approach  is  the 
natural  connection  between  well  known  terms  and  a technical  system.  This  makes  the  further  development  of  the 
math-kit  project  easier.  Despite  this,  one  metaphor  cannot  cover  all  parts  of  such  a complex  software  system.  Hence, 
the  metaphor  has  to  be  extended  to  meet  the  requirements.  The  next  step  will  be  a description  of  the  system  by  use 
cases,  sequential  diagrams  and  an  object  model  which  paves  the  way  for  a later  implementation. 
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Abstract:  In  response  to  feedback  obtained  from  recent  graduates  during  a 

departmental  program  review,  Dr.  Janet  M.  Kelly,  Assoc.  Professor  in  Political 
Science  at  the  University  of  Tennessee  redesigned  an  upper  division  Urban 
Policy  class  explicitly  on  a technology  enhanced  collaborative  education  model. 
Taking  advantage  of  a wireless  laptop  pilot  project  sponsored  and  supported  by 
the  Innovative  Technology  Center,  Dr.  Kelly  used  laptop  computers,  a versatile 
suite  of  software  tools,  and  a customized  course  web  site  to  create  a technology- 
enhanced,  collaborative  Urban  Policy  course  that  addressed  specific  alumni 
recommendations  for  increased  use  of  computing  and  communications 
technologies  and  team-based  projects. 


Introduction 

“Our  students  are  entering  a world  in  which  60%  of  the  jobs  will  require  technological  competency... [w]e 
must  take  advantage  of  the  capacity  of  technology  to... engage  our  students  in  active  learning.”  (Morrison, 
1997).  Transformations  in  business,  social  and  educational  institutions  generated  by  technological 
evolution  require  that  we  address  the  demands  for  increased  technological  competency,  collaborative 
working  skills,  critical  thinking  and  rapid  problem  solving.  This  will  require  changes  in  the  ways  we 
prepare  our  students.  “[TJoday's  world  demands  critical  thinking  and  on-the-fly  problem  solving.  To  meet 
these  demands,  our  graduates  require  significantly  different  knowledge,  skills,  and  abilities  than  our 
educational  systems  typically  provide.”  (Ellsworth,  1997)  We  need  to  provide  problem-based  learning 
opportunities  that  are  both  constructivist  and  collaborative.  “[W]hat  we  should  foster  are  academic 
environments  in  which  students  are  free  to  explore  and  express  their  own  ideas... to  make  explicit  and 
elaborate  on  their  own  tacit  knowledge.”  (Watts,  1997) 


The  Case 

Recent  graduates  in  the  Political  Science  program  indicated  that  their  initial  experiences  in  the  workplace 
revealed  two  significant  weaknesses  in  their  preparation,  a lack  of  experience  working  with  common 
computing  and  communications  technologies,  and  a lack  of  experience  working  in  project-based  teams.  Dr. 
Kelly  set  out  to  design  a project-based  course  that  would  be  collaborative  in  design,  and  would  rely  heavily 
on  the  use  of  computer-based  productivity,  communication  and  presentation  applications.  Dr.  Kelly 
redesigned  Urban  Policy  410  to  simulate  a work  environment  in  which  students  worked  in  teams  to  study 
actual  urban  problems,  formulate  policy  statements  and  share  their  recommendations  in  a formal 
presentation.  In  addition  to  the  expected  discipline  specific  content  learning  objectives,  she  developed 
explicit  objectives,  assignments  and  assessment  instruments  for  technology  skill  acquisition,  written  and 
oral  communication,  and  group  process  skill  development. 

To  meet  each  objective,  Kelly  used  a variety  of  collaborative  activities.  All  course  materials,  with  the 
exception  of  the  two  required  texts,  were  available  online  through  the  course  website.  These  included  the 
syllabus  and  general  course  information,  course  documents  (including  class  lecture  notes,  a sample  policy 
report  and  student  policy  reports,  sample  presentation  and  group  presentations,  evaluation  guidelines  and 
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documents,  supplemental  reading  materials,  and  case  study  files),  course  assignments,  group 
communication  tools  (chat  room,  discussion  board,  file  sharing  and  email),  and  links  to  guides,  tutorials, 
and  university  resources. 

Dr.  Kelly  grouped  students  into  teams  from  the  outset,  and  provided  a set  of  guidelines  and  definitions  for 
the  roles  to  be  performed  within  the  teams.  Each  person  in  the  group  was  required  to  assume  a specific  role 
within  the  group.  Individuals  had  to  ‘apply’  for  their  roles,  and  the  group  had  some  flexibility  in  role 
assignments  and  definitions  within  the  guidelines  provided.  Each  team  was  responsible  for  researching  a 
specific  urban  policy  area  (housing,  immigration,  poverty,  crime  and  transportation),  developing  a 
comprehensive  written  report,  and  presenting  their  findings  to  the  class  in  a live  PowerPoint  lecture  format. 
Research,  writing  and  presentation  skills  were  developed  using  guidelines  and  examples  provided  by  the 
instructor.  In  addition,  each  student  was  required  to  use  the  specified  evaluation  criteria  to  evaluate  the 
products  (paper  and  presentation)  of  each  group,  as  well  as  the  process  (group  performance)  of  their  own 
group. 

In  an  interview  at  the  conclusion  of  the  semester.  Dr.  Kelly  made  the  following  comments: 

“(During  the  program  review)  the  graduates  said  that  the  biggest  disparity  in  the  graduate  training  in  Public 
Administration  ...was  that  when  they  got  to  the  workplace,  they  worked  with  technology  and  they  worked 
in  groups.  And  that  was  the  one  way  that  we  didn’t  teach,  we  didn’t  teach  with  technology  and  we  taught 
everything  on  the  basis  of  individual  effort.  And  they  had  the  hardest  time  trying  to  manage  the 
assignments  that  they  got  in  groups,  without  ever  having  developed  any  skills  for  working  in  groups,  and 
especially  working  with  technology.  I built  [the]  course,  integrating  those  two  concepts,  use  of  technology 
and  group  process  to  teach  two  skills  that  were  very  critical,  but  were  not  content  related.” 

“Developing  everything  in  advance,  the  assignments,  the  examples,  the  evaluation  guidelines,  everything, 
then  putting  it  on  the  website  is  critical.  The  only  way  for  this  to  work  is  that  the  answer  to  everything  has 
got  to  be  ‘it’s  on  the  website.’” 

“I  would  never  go  back  to  the  old  way  of  teaching  this  course,  using  only  readings,  lecture  and 
individualized  assessment.  The  quality  of  the  products  these  students  produced,  the  papers  and  the 
presentations... was  spectacular... it  was  so  clearly  superior  to  anything  I had  ever  seen  from  previous 
students  in  this  course  that  I wouldn’t  even  consider  going  back  to  the  old  way  of  doing  things.” 

Conclusions 

The  experiences  with  PS  410,  including  the  design  and  implementation  of  the  course  website,  collaborative 
activities,  and  assessment  instruments  provide  inspiration  and  guidance  to  faculty  and  instructional 
designers  in  a variety  of  disciplines  and  educational  settings.  The  careful  application  of  collaborative 
learning  methods,  along  with  the  integration  of  contemporary  technologies,  provide  the  foundation  for 
more  meaningful,  relevant,  and  engaging  learning  opportunities  for  students  regardless  of  their  academic 
discipline  or  career  aspirations. 
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Assessing  the  Use  of  Technology  Resources  in  the  College 
Classroom:  Student  Responses 


In  order  to  investigate  the  impact  of  technology  resources  for  teaching  and  learning  in  an  undergraduate 
classroom,  a research  project  was  conducted  at  Springfield  College.  Springfield  College  is  a small  four- 
year  college  located  in  Springfield,  Massachusetts.  For  this  study,  two  Elementary  Spanish  undergraduate 
classes  were  selected.  One  used  technology  to  complete  activities,  homework,  and  expand  on  the  aspects 
of  culture  studied  in  the  class,  while  the  other  class  used  traditional  textbook  methods.  Data  collected 
demonstrates  that  technology  is  a suitable  tool  to  increase  motivation,  participation  and  learning. 

Implementation  Method 

This  research  study  was  conducted  during  Fall  2001  and  a second  study  is  being  repeated  this  semester  to 
strengthen  and  adjust  findings,  conclusions,  and  results.  The  classes  were  taught  by  the  same  instructor, 
using  the  same  syllabus  but  one  of  the  classes  each  semester  was  randomly  selected  to  use  technology  as  a 
means  of  academic  instructional  support.  Approximately  20  students  enroll  in  these  first  level  Spanish 
courses;  a foreign  language  course  is  a graduation  requirement.  Most  of  the  students  — 80  percent  have 
a fragile  commitment  to  the  class. 

Students  in  the  class  using  technology  interacted  with  the  worldwide  web,  email  and  chat  rooms  to 
complete  activities  and  homework  assignments.  For  each  project,  throughout  the  semester,  students  in  the 
traditional  class  complied  the  homework  activities  in  the  exercise  manual  that  accompanies  the  book. 

Each  chapter  in  the  book  has  a corresponding  set  of  activities  supplemented  by  the  manual.  The  selection 
of  web  activities  was  designed  to  match  the  activities  of  the  traditional  class.  The  homework  activities  of 
both  classes  ~ the  traditional  and  technology  class-  required  similar  amounts  of  time  to  be  completed  and 
they  corresponded  to  the  themes  studies  in  class. 

The  students  in  the  technology-enhanced  class  had  access  to  course  management  software,  the  Manhattan 
Pilot,  to  check  their  grades,  assignments  and  lectures,  while  the  instructor  kept  a manual  record  and 
reported  to  the  traditional  class. 

Assessment  Method 

Different  assessment  methods  were  used  to  understand  the  impact  of  technology  in  the  classrooms.  The 
percentage  of  completed  homework  was  utilized  as  an  indicator  of  student  commitment  to  the  class.  To 
assess  the  effect  of  technology  on  learning  outcomes,  students'  grades  were  compared.  A questionnaire  was 
developed  to  measure  students'  perceptions  of  homework  activities.  Open-ended  questions  were  also  added 
for  students  using  technology  about  their  impressions  and  reactions  towards  the  activities. 

Results 

Learning  Outcomes:  Data  reflect  higher  learning  outcomes  in  the  class  using  technology,  (class  average) 


Learning  Outcomes 

Homework 

Quizzes 

Exam 

Traditional 

72% 

80% 

84% 

Technology 

98% 

84% 

90% 

Completed  Homework  Assignments:  Data  results  show  students  from  the  technology  class  completed 
more  homework  assignments  than  students  from  the  traditional  classroom. 


Ex.  Book 

Web 

Book 

Average 

Traditional 

N/A 

98% 

96% 

97% 

Technology 

63% 

N/A 

74% 

68.5  % 

258 


BEST  COPY  AVAILABLE 


Overall  Impressions  on  Homework  Activities: 

• Question  1 : 1 believe  homework  assignments  enhanced  my  learning  of  the  material. 

• Question  2:  Homework  assignments  were  good  indicators  of  my  learning  throughout  the  semester. 

• Question  3:  Overall,  1 had  a positive  learning  experience  doing  homework. 


Question  1 

Question  2 

Question  3 

Spring  2002 

Agree 

Disagree 

Agree 

Disagree 

Agree 

Disagree 

Traditional 

16% 

79% 

19% 

76% 

16% 

79% 

Technology 

84% 

12% 

86% 

10% 

84% 

12% 

Students’  Impressions  and  Reactions  Towards  Technology 

The  narrative  responses  indicated  that  many  students  were  exited  by  the  fast  feedback  they  received  from 
the  web  activities  and  felt  that  it  was  an  important  supplement  to  the  course.  Technical  problems  such  as  a 
slow  internet  connection  caused  the  most  dissatisfaction  but  in  general  students  were  able  to  deal  with 
occasional  technical  glitches. 

Many  students  preferred  the  ACE  Tests  to  the  Web  Search  Activities.  They  mentioned  that  many  of  the 
Web  Search  activities  were  ambiguous  and  not  as  useful  as  the  ACE  Tests  Activities.  Two  students 
recommended  discontinuing  them.  Students  were  positive  about  the  chat  room  activities.  They  mentioned 
that  they  usually  turned  to  English  while  working  in  face-to-face  activities  but  in  the  chat  rooms  they  were 
forced  to  communicate  in  Spanish  constantly. 

Findings,  Conclusions  and  Recommendations 

The  focus  of  many  studies  has  been  to  analyze  the  impact  of  technology  in  terms  of  student  achievement. 
When  adopting  technology  to  enhance  the  learning  process  there  are  many  factors  to  consider.  For  this 
project,  the  courses  were  taught  simultaneously.  If  course  content,  instructor  and  student  population  were 
all  the  same  in  theory,  any  differences  should  be  attributable  to  the  different  teaching  method  utilized. 

Students  in  the  two  classes  using  technology  performed  better  than  the  students  in  the  traditional  classroom; 
for  them,  homework  assignments  were  a positive  experience.  Students  in  the  technology  class  pointed  out 
that  they  benefited  the  most  from  the  ACE  Self  Tests  homework  assignments  because  these  web  tests 
provided  them  with  immediate  feedback  information  on  their  performance. 

ACE  Tests  -short  for  "A  Cyber  Evaluation"  is  a self-testing  program  that  allows  students  to  test  their 
knowledge  of  topics  covered  in  each  chapter.  Student  answers  are  scored  immediately,  allowing  students 
to  correct  their  mistakes  while  working  on  the  task.  The  complete  activity  and  scores  are  submitted  to  the 
instructor.  The  strength  of  the  activity  is  in  its  capacity  for  making  self-assessment  an  ongoing  and  integral 
part  of  the  curriculum.  It  reinforces  instruction  and  guides  students  in  reflecting  upon  their  work.  Even 
though  the  instructor  kept  manual  records  of  the  assignments  for  the  traditional  classes,  there  was  a 
minimum  of  two  days  before  feedback  occurred.  Students  from  this  class  declared  that  the  feedback  was 
never  relevant  for  their  learning.  According  to  these  findings,  technology  is  a suitable  tool  to  increase 
motivational  learning. 

For  a practical  matter,  the  traditional  method  was  used  as  a base  for  comparison  in  this  study.  It  is  not  the 
intention  of  this  researcher  to  suggest  the  implementation  of  technology  should  replace  traditional  teaching 
methods.  Rather  it  is  my  intention  to  suggest  that  technology's  main  role  should  be  to  complement  and 
further  effective  learning  methods  in  ways  not  possible  using  only  traditional  methodologies. 

The  implementation  of  technology  in  the  classroom  requires  time  and  effort  from  faculty  who  need  to  learn 
the  new  technology  and  design  effective  ways  to  integrate  its  use.  With  the  incorporation  of  technology, 
traditional  courses  get  redesigned.  Under  these  new  circumstances,  teaching  and  learning  is  explored  and 
innovatively  re-created. 
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First  Year  of  Technology  Mentoring  for  Teachers  and  Faculty:  Lessons  Learned 
Shirley  Campbell  and  Songiil  dzgiil 


Introduction: 

Rapid  changes  in  the  available  computer  hardware  and  software  applications  along  with  continuous  new  releases  of 
software  applications  have  created  a constant  challenge  for  teachers  who  are  trying  to  keep  up  with  new  technological  improvements. 
In  other  words,  in  order  to  teach  and  facilitate  their  students’  learning  with  the  support  of  the  available  computer  software 
applications  as  well  as  tools,  teachers  have  to  update  their  technology  skills  constantly. 

Many  schools  acquired  technology  tools  and  software  application  to  catch  up  the  current  technology  applications  in 
educational  field,  however,  without  proper  in-service  training  of  teachers,  the  use  of  these  technology  cannot  be  optimized.  The 
current  related  to  teachers’  effective  technology  integration  may  be  divided  into  following  sections: 

a)  On-going  training  with  rehearsal  time:  The  teachers’  technology  training  should  include  learning  experiences  that  give 
them  opportunities  to  practice  the  currently  learned  skills.  Without  practice,  it  should  not  be  expected  that  teachers  will  go 
to  their  classrooms  and  will  apply  those  skills. 

b)  Curricular  approaches  to  increased  technology  use:  The  technology  training  offered  to  the  teachers  should  include  the 
situations  that  they  will  be  using  in  their  professional  lives.  Especially  for  novice  technology  learners,  it  is  hard  to  make 
connections  between  the  skills  they  are  learning  and  how  to  integrate  those  skills  into  their  teaching  practice.  Therefore, 
showing  examples  in  some  teaching  related  situations  will  inspire  teachers  and  will  lead  them  to  think  of  more  innovative 
ways  to  use  those  skills  in  their  teaching  practices. 

c)  Increasing  comfort  with  the  technology:  Having  an  opportunity  to  practice  the  skills  they  have  learned  will  increase 
teachers  comfort  level  significantly.  Once  they  have  gained  experience  with  new  skills,  they  will  be  more  comfortable 
sharing  those  skills  with  their  colleagues  and  students  while  integrating  them  into  their  teaching. 

d)  Continuous  technical  support  related  to  learning  and  using  hardware  and  software:  It  is  always  frustrating  for  users  to 
be  stuck  on  a computer  related  problem  and  not  be  able  to  resolve  it  promptly.  In  such  circumstances,  if  it  is  possible  to  get 
technical  support  in  reasonably  enough  time,  frustration  will  be  diminished  and  the  teacher  dealing  with  the  problem  can 
productively  continue  his/her  work.  Prompt  technical  support  is  especially  very  important  if  the  technical  troubles  arise  in 

a classroom  during  the  learning  process.  Unfortunately,  this  kind  of  prompt  technical  support  is  not  usually  available  in 
many  schools  and  settings.  Anticipation  of  problems  that  won’t  be  resolved  in  time  to  complete  instruction  reduces  the 
likelihood  that  teachers  will  attempt  to  use  it  in  their  teaching.  It  creates  frustration  from  the  both  the  students’  and 
teachers’  perspective.  Sometimes  teachers  even  have  to  prepare  the  instructional  materials  in  another  format  just  incase 
any  technical  problem  arises,  so  it  doubles  the  work  of  teacher  preparation.  Available  technical  support  offered  to  the 
teachers  is  a crucial  factor  in  their  decisions  to  use  technology  in  the  classroom. 

The  PT3  Project:  Preparing  Tomorrow’s  Teachers  to  Use  Technology 

The  Preparing  Tomorrow’s  Teachers  to  Use  Technology  Project  was  created  by  the  U.S.  Department  of  Education  to 
address  the  need  for  newly  certified  teachers  to  be  able  to  integrate  technology  in  their  first  years  of  teaching.  The  U.S.  Education 
Department’s  PT3  Program  has  provided  funding  for  projects  nationwide.  At  the  School  of  Education  at  the  University  of  Pittsburgh, 
a funded  project  is  underway  and  is  designed  to  address  the  issues  of  technology  use  and  instruction  for  pre-service  teachers. 

The  PT3  Project  at  the  University  of  Pittsburgh  focuses  in  part  on  t he  faculty  and  teachers  who  work  closely  with  students 
who  are  working  toward  teaching  certification.  It  involves  increasing  the  integration  of  technology  into  the  training  offered  to  these 
teachers  and  faculty.  The  University  of  Pittsburgh  received  two  separate  grants:  a Capacity  Building  Grant  in  1999  to  serve  as  a 
pilot,  and  an  Implementation  Grant  to  fund  3 years  of  implementation  of  the  successful  pilot  beginning  in  2000. 

The  PT3  Project  at  the  University  of  Pittsburgh  was  designed  to  help  teachers  become  users  of  each  of  four  types  of 
technology  applications  that  are  regularly  implemented  by  teachers  who  are  considered  successful  technology  integrators.  These  four 
types  are: 


• Personal  Use  refers  to  using  technology  to  increase  personal  productivity.  The  first  application  of  the  technology  that 
teachers  are  likely  to  use  usually  related  to  technology  training  for  their  “personal  use”  of  the  technology.  This  refers  to  the 
idea  that  teachers’  experience  with  technology  tools  and  applications  that  produce  results  useful  outside  of  the  professional 
arena  will  help  teachers  significantly  transfer  those  knowledge  and  skills  into  their  professional  needs. 

• Classroom  Management  refers  to  the  use  of  computer  tools  and  applications  to  organize  classroom  information,  record 
keeping,  and  to  create  professional  looking  products. 

• Teaching  Technology  Skills  to  Students  implies  that  before  students  can  become  involved  in  the  activities  that  technology 
provides  that  promote  higher  order  thinking  skills,  they  have  to  be  able  to  use  the  tools,  and  teachers  have  to  know  how  to 
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teach  the  necessary  skills.  Students*  computer  experiences  vary  from  novice  to  expert  in  the  same  classroom  because  of 
their  different  backgrounds  and  exposure  to  technology.  Teachers  sometimes  report  difficulties  providing  satisfying 
answers  to  the  technology  related  questions  their  students’  ask, 

• Integration  to  Curriculum  refers  to  the  application  of  the  use  of  technology  to  teaching  in  ways  that  increase  learning  and 
frequently  require  students  to  use  higher  order  thinking  skills.  “Integration  to  curriculum”  is  significantly  important  in 
terms  of  teachers’  teaching  experiences  in  the  classrooms.  Being  comfortable  using  the  available  educational  and 
commercial  software  programs  and  integrating  them  effectively  into  their  teaching,  teachers  will  be  able  to  create  more 
innovative  and  up  to  date  as  well  as  interesting  learning  experiences  for  their  students  that  will  facilitate  student  learning  in 
more  unique  and  resourceful  ways. 

The  professional  development  package  created  for  technology  training  for  teachers  attempted  to  expose  teachers  to  all  of 
these  areas  of  technology  use,  but  was  also  devised  to  prevent  the  difficulties  that  teachers  oftai  have  when  making  attempts  to  use 
technology  in  the  classroom.  More  traditional  professional  development  strategies  often  are  not  as  effective  as  expected  in  improving 
the  implementation  of  classroom  technology.  Some  components  of  the  support  developed  within  this  PT3  Project  include: 
collaboration  and  community  that  includes  all  school  personnel,  technology  learning  support  in  addition  to  technical  support; 
cognitive  strategies  for  technology  learning;  and  the  building  of  comfort  and  confidence.  Within  each  of  these  components,  several 
additional  issues  are  addressed. 


Features  of  the  PT3  Project  at  University  of  Pittsburgh 

This  PT3  Project  was  constructed  of  a number  of  features,  each  expected  to  promote  increased  technology  use  by  building 
skills  and  community.  These  features  include  On-Site  Support  Staff  support,  individually  chosen  projects,  and  whole  group 
community  activities  with  community  building  activities  imbedded  into  each.  Each  of  these  features  includes  many  activities  to 
provide  technology  learning  support. 

On-Site  Support  Staff 

Each  teacher  and  faculty  member  participating  in  the  PT3  Project  at  University  of  Pittsburgh  was  supported  as  they  learned 
to  use  technology  by  On-Site  Support  Staff  members.  Teachers  and  faculty  were  provided  very  individualized  support  as  they 
learned  new  skills  and  worked  toward  their  own  technology  use  goals.  On-site  Support  Staff  identified  the  types  of  skills  participants 
needed  to  learn  in  order  to  complete  the  projects  they  chose,  and  adapted  training  very  specifically  to  the  needs  of  the  learners.  Also, 
keeping  constant  contact  by  e-mail  and  phone  to  remind  the  scheduled  weekly  appointments  as  well  as  the  monthly  meetings,  had 
promoted  teachers  participation  in  these  appointments  and  meetings. 

The  Support  Staff  also  held  mini-workshops  that  participants  were  invited  to  attend.  These  workshops  were  non- 
traditional,  in  that  they  were  expected  to  serve  fewer  than  ten  attended  at  a time,  and  the  covered  only  specific  skills  during  the  one  to 
one  and  a half  hour  meetings.  Support  Staff  often  focused  the  workshops  specifically  to  the  needs  of  the  individuals  attending,  and 
invited  those  individuals  who  would  benefit  from  a specific  skill  areas.  The  Support  Staff  were  skillful  in  finding  different  ways  of 
presenting  the  same  instruction  including  supporting  handouts,  examples  and  hands-on  practice  until  the  teachers  became 
comfortable  doing  the  tasks  alone.  They  used  these  techniques  when  working  with  individuals  as  well  as  in  workshops.  Many  of  the 
workshops  were  scheduled  at  times  when  the  Support  Staff  felt  that  a number  of  individuals  would  benefit  from  learning  the  same 
skills. 

In  the  on  site  training  sessions,  teachers  from  two  schools  were  initially  provided  support  individually  with  plans  to  work  as 
small  groups  later  on.  In  the  third  school,  teachers’  on-site  support  was  provided  in  a small  group  format  from  the  start.  The 
teachers  who  were  being  helped  individually  were  resistant  to  joining  together  with  other  teachers  when  meeting  with  Support  Staff, 
and  tended  to  want  to  continue  to  work  individually.  Based  on  this  difficulty  in  community  building,  On-Site  small  group  support  is 
recommended  instead  of  solely  individual  assistance. 

Individually  Chosen  Project 

Each  participating  teacher  and  faculty  member  in  the  PT3  Project  was  required  to  choose  a realistic  project,  called  the 
individually  chosen  project,  that  could  be  applied  in  their  own  classroom.  The  projects  were  required  to  meet  two  guidelines:  they 
had  to  be  related  to  classroom  work,  and  they  had  to  be  projects  that  required  the  teacher  to  add  to  their  own  technology  skills  in 
order  to  complete  them.  Depending  on  the  skill  level  of  the  teachers  at  the  onset  of  PT3  Participation  (which  ranged  from  none  to 
quite  proficient),  the  individually  chosen  projects  could  be  well  integrated  curriculum  enhancement  projects  to  be  implemented  by 
students,  or  they  could  require  that  only  the  teacher  use  the  computer,  while  becoming  more  comfortable  with  their  own  technology 
skills.  Allowing  each  individual  to  choose  a project  rather  than  assigning  projects  gave  each  individual  ownership  of  their  own 
learning,  and  allowed  them  to  choose  a project  that  they  felt  would  be  useful. 

One  of  the  most  difficult  aspects  of  the  individually  chosen  projects  for  the  participants  was  committing  to  a project  that 
they  did  not  have  the  skills  to  complete.  Much  reassurance  was  required  on  the  part  of  the  On-Site  Support  Staff  that  projects  were 
expected  to  be  revised  over  time,  as  participants  became  more  familiar  with  the  technology  and  its  implementation.  As  a result,  many 
projects  changed  between  the  time  that  they  were  chosen  at  the  beginning  of  the  school  year,  and  at  time  that  they  were  implemented. 
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Another  result  was  the  reaction  of  pilot  year  participants  to  the  discomfort  of  new  participants  in  the  first  full  year  of  the  grant. 
Experienced  participants  reassured  the  newcomers  that  they  should  chose  a project  that  they  found  exciting,  and  not  concern 
themselves  with  how  they  would  eventually  produce  it:  there  was  enough  support  and  flexibility  to  make  it  happen.  Also,  their 
technical  support  person  would  be  helping  them  to  achieve  their  goals  and  providing  them  with  the  required  technical  skills  so  that 
they  would  be  able  to  complete  their  project. 

The  teachers  were  encouraged,  but  not  required,  to  work  collaboratively  with  other  teachers  on  these  projects.  The  On-Site 
Support  Staff  who  assisted  the  teachers  at  times  suggested  matches  between  teachers  with  similar  interests  or  complimentary  skills, 
with  the  expectation  that  these  teachers  would  eventually  begin  to  see  one  another  as  resources.  Although  no  teachers  chose  to  work 
collaboratively  on  a project  during  the  first  year,  a number  of  collaborative  projects  emerged  during  planning  for  the  second  year. 

The  On-Site  Support  Staff  encouraged  collaborative  projects  with  expectations  that  these  efforts  would  help  to  increase  problem 
solving  and  promote  an  increase  in  confidence  for  participants.  Long  term  effectiveness  of  the  PT3  Project  is  also  based  on  the 
building  of  strong  and  supportive  learning  communities  among  peers,  allowing  growth  to  continue  long  after  funding  is  finished. 

Whole  Group  Community  Activities 

All  participants  of  the  project  were  required  to  take  part  in  activities  that  were  intended  to  bring  the  group  together,  with  the 
expectation  that  sharing  would  support  the  likelihood  of  the  formation  of  a community  of  technology  users.  Several  different  types 
of  meetings  were  held.  Monthly  meetings  were  held  on  Saturday  mornings,  and  activities  promoted  interaction  between  participants 
related  to  technology  use  and  education  of  students.  Participants  were  given  opportunities  to  talk  about  successes  as  well  as 
difficulties,  and  were  involved  in  activities  that  promoted  discussion  among  individuals.  Many  expressed  appreciation  for 
opportunities  to  become  familiar  with  teachers  from  other  schools. 

Two  other  whole  group  activities  promoted  through  this  PT3  Project  included  Summer  Camp,  and  Celebration  of 
Successes.  During  Summer  Camp,  participants  were  involved  in  community  building  activities  alternated  with  technical  skill 
building  and  activities  to  develop  skills  in  technology  integration.  As  often  as  possible,  sharing  was  expected  and  encouraged. 
Summer  Camp  was  held  at  the  beginning  of  grant  activities,  and  Celebration  of  Successes  was  the  culminating  activity  of  the  PT3 
Project  for  the  school  year.  During  the  Celebration  of  Successes,  each  participant  was  asked  to  present  the  individually  chosen  project 
that  they  completed  during  the  school  year.  Presentations  were  to  include  not  only  descriptions  of  the  projects  and  impact  of 
implementation,  but  also  barriers  faced  and  solutions  found,  and  personal  successes  involved.  All  participants  expressed  appreciation 
for  the  opportunity  to  present  their  own  work,  and  see  the  work  of  others.  “And  the  project[s]  demonstrated  at  the  Celebration  of 
Successes  on  the  final  day  of  the  program  [were]  fashioned  from  hard  work,  cooperation,  innovative  ideas,  comfort  with 
manipulation  of  the  technology,  and  effective  implementation  the  technology  in  their  classrooms”  (Campbell  & Ozgiil,  2001). 

Our  findings  at  the  end  of  the  first  year  of  the  project  period  showed  that  teachers  and  faculty  members  preferred  to  plan 
and  work  on  their  projects  alone.  However,  on  celebration  day  when  they  were  presenting,  sharing  and  discussing  their  projects  with 
the  others  we  have  observed  that  teachers  from  different  schools  came  up  with  different  ways  of  using  their  colleagues’  projects  in 
their  schools  or  they  came  up  with  combining  their  experiences  together  in  order  to  create  new,  innovative,  and  collaborative  projects 
to  be  accomplished  in  the  second  year  of  the  project.  Because  of  their  improved  technology  skills  level  as  well  as  the  satisfaction  of 
what  they  had  already  accomplished,  they  were  more  confident,  enthusiastic,  and  creative  and  also  willing  to  share  and  work 
collaboratively  with  the  other  teachers  to  create  better  products. 

During  the  planning  of  the  PT3  Project,  district  level  administrators  were  involved  in  planning  project  structure  and 
activities.  They  were  responsible  for  informing  administrators  at  all  school  levels,  and  PT3  Project  staff  contacted  school-level 
administrators  only  individually  prior  to  an  administrator’s  meeting  arranged  several  months  later.  This  was  a golden  opportunity  for 
the  PT3  project  staff  to  present  and  discuss  the  goals  and  possibilities  of  the  project  with  these  administrators.  Until  this  meeting, 
administrators  were  aware  of  the  PT3  Project  and  were  encouraging  their  teachers  to  participate,  but  this  communication  between  the 
administrators  and  project  staff  as  well  as  among  the  administrators  gave  everyone  a chance  to  grasp  the  scope  of  the  project  in  detail 
as  well  as  gain  realistic  views  of  seizing  the  opportunities  given  by  the  project. 

An  administrative  meeting  was  not  only  beneficial  for  opening  communication  with  Project  Staff  as  well  as  among 
administrators  themselves,  but  also  beneficial  in  resolving  several  issues  which  their  teachers  were  encountering  in  their  attempts  at 
effective  use  of  technology.  First  of  all,  administrators  showed  sensitivity  to  allocate  time  to  project  participants  to  work  with  their 
support  person  on  technology  training  activities,  especially  weekly  meetings  at  either  their  school  or  at  the  PT3  Project  community 
gatherings.  Also,  administrators  addressed  some  hardware  and  software  limitations  and  problems  that  teachers  encountered  in  their 
schools.  Administrators  were  also  better  able  to  affect  teachers’  success  because  reports  made  available  to  them  by  the  PT3  Staff 
provided  more  information  about  difficulties  teachers  had  in  implementing  technology  uses.  Consequently,  having  better 
understanding  of  teachers’  problems  with  technology,  then  sharing  and  discussing  these  issues  with  the  other  schools  administrators, 
inspired  them  to  deal  with  their  teachers’  difficulties. 

In  addition,  once  the  school  level  administrators  were  fully  aware  of  the  possible  impact  of  the  PT3  Project  in  their  schools, 
they  encouraged  the  teachers  in  their  schools  who  were  not  already  participating  to  become  active  in  the  project.  Participating 
schools  then  welcomed  PT3  activities  in  their  buildings. 


Findings 

During  the  course  of  the  pilot  year  of  the  PT3  Project  at  University  of  Pittsburgh,  several  strategies  for  increasing 
technology  learning  and  community  building  were  tested,  and  results  were  noted.  Promising  strategies  and  components  identified  by 
teachers  as  most  effective  in  helping  them  to  reach  their  technology  goals  are  noted. 
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BEST  COPY  AVAILABLE 


Journaling  f Learning  Log) 

Participant  teachers  were  required  to  keep  a reflective  journal  whenever  they  used  technology  applications.  The  reasons 
behind  the  requirement  of  journaling  were  two  fold.  Journaling  was  beneficial  to  both  the  project  staff  and  the  participants. 

Journaling  was  seen  by  Project  Staff  as  an  opportunity  to  gain  feedback  about  the  provided  training  as  well  as  track  the  progress  of 
participants.  Feedback  gathered  from  participant  journals  was  used  revise  the  design  and  implement  the  training  according  to  entries. 

While  many  of  the  participants  did  not  enjoy  journaling,  those  who  did  described  it  as  a reflective  tool,  providing  them  an 
opportunity  to  reflect  on  their  experiences  in  using  technology.  It  was  also  served  as  a means  for  participants  to  document  their 
experiences  with  technology. 

Journaling  was  also  a requirement  for  the  Project  Staff  At  the  end  of  the  first  year  of  the  project.  Project  Staff  reported  that 
journaling  was  a very  effective  tool  for  both  keeping  track  of  the  progress  of  several  teachers  that  they  were  assigned  to  work  with, 
and  also  preparing  for  teachers’  different  training  needs.  They  were  also  able  to  use  their  journals  as  logs,  to  identify  practices  that 
they  use  that  were  particularly  effective,  and  to  document  their  own  growth  over  time. 

As  a result,  journaling  was  considered  as  an  important  tool  that  contributed  to  following  the  progress  of  the  training  and 
adapting  it  as  needed.  Because  of  its  benefits  to  the  overall  project,  it  was  required  of  participants  in  the  following  years  of  the  PT3 
project  as  well  with  some  modifications.  During  the  pilot  year,  journaling  was  done  on  a notebook  and  these  reflections  were  copied 
and  given  to  the  their  onsite  Support  Staff.  In  the  following  year,  journaling  was  planned  to  be  done  on-line. 

Note  Taking 

Note  taking  plays  a very  essential  role  in  teachers  learning  technology  skills,  especially  novice  teachers.  If  these  newly 
learned  skills  are  not  practiced  frequently,  they  could  easily  be  forgotten.  To  prevent  that,  the  teachers  and  the  faculty  members  were 
provided  with  handouts  about  the  instruction.  Many  times  the  teachers  and  faculty  personalize  the  steps  by  adding  notes  to  the  given 
handouts,  and  using  these  notes  increased  the  likelihood  that  they  would  be  able  to  practice  the  same  skills  successfully  on  their  own. 
When  they  had  difficulty  recalling  the  skills  they  had  learned,  the  personalized  notes  served  to  refresh  their  memories.  When 
applications  shared  similar  sets  of  keystrokes  or  menu  options,  creation  of  handouts  that  repeated  the  same  formatting  for  such  items 
served  to  jog  the  memory  and  helped  to  make  connections  that  transferred  application  of  the  skills. 

Concept  Mapping 

Sometimes  it  is  very  difficult  for  teachers  and  faculty  who  were  novice  computer  users  to  understand  how  the  skills  they 
have  been  learning  are  related  to  their  final  projects.  Using  concept  mapping  as  a cognitive  strategy  gave  them  a visual  tool  for 
conceptualizing  the  relationship  between  the  each  group  of  skills  needed  to  be  learned  and  the  relationship  of  those  skills  to  the  main 
goal.  Therefore,  the  opportunity  of  seeing  visually  which  skills  are  required  to  reach  the  goal,  what  they  have  already  learned,  and 
what  still  needs  to  be  learned  helped  both  the  Support  Staff  as  well  as  the  teachers  to  plan  the  best  sequence  of  learning  activities. 

Comfort  and  Confidence 

Teachers  were  supported  in  many  ways  to  reach  their  goals  for  the  end  of  the  first  year  of  the  PT3  project.  For  example, 
flexible  technical  support,  opportunities  to  gather  the  projects  participants  in  workshops  as  well  as  monthly  meetings,  and  having  the 
flexibility  to  alter  the  final  projects  were  important  key  elements  in  increasing  the  teachers’  comfort  level  with  working  for  their  goals 
and  with  project  staff  Additionally,  seeing  the  final  projects  of  the  previous  years’  participants,  as  well  as  listening  to  their 
experiences,  increased  the  confidence  level  of  the  teachers  participating  in  the  project  in  the  second  year.  Therefore,  it  was  assumed 
that  the  anxiety  and  the  disappointment  that  the  first  year  participants  encountered  would  be  somewhat  reduced  for  participants 
entering  in  the  second  year. 

Technology  Learning  Support 

Much  of  the  success  of  the  first  year  of  implementation  of  the  PT3  project  was  attributed  to  the  availability,  flexibility, 
technical  skills,  and  knowledge  of  good  tutoring  practices  of  the  technical  Support  Staff  A number  of  teachers  had  difficulty 
scheduling  meeting  times  during  their  busy  daily  work  routine.  The  flexibility  of  the  technology  Support  Staff  made  it  possible  to 
schedule  time  slots  that  meet  the  schedules  of  the  teachers.  As  state  earlier,  it  was  very  important  to  maintain  contact  with 
participants  between  meetings,  by  email  or  phone. 

Participating  teachers  mentioned  in  their  evaluation  forms  that  one  of  the  greatest  assets  of  the  project  was  the  On-Site 
Support  Staff  members’  enormous  patience.  They  were  willing  to  repeat  the  same  instruction  many  times  when  teachers  needed  them 
to  do  so.  As  mentioned  earlier,  the  On-Site  Support  Staff  were  very  creative  in  finding  ways  to  present  the  same  information  in 
multiple  ways. 

Individual  Classroom  Projects 

Participating  teachers  and  faculty  members  were  required  to  choose  an  individual  project  at  the  beginning  of  PT3  Project 
participation.  They  were  expected  plan  it,  including  modifications,  then  present  it  at  the  end  of  the  year.  These  chosen  projects  could 
reflect  any  subject  area(s).  Projects  required  that  participants  learn  new  skills  as  they  created  the  projects,  but  the  only  software  and 
hardware  restrictions  were  those  imposed  at  their  schools  sites,  usually  based  on  availability.  Their  anxiety  and  disappointment  were 
resolved  by  comforting  them  that  the  final  project  requirement  was  meant  to  have  them  to  experience  from  the  beginning  of  the 
project  how  to  plan,  develop  and  design,  evaluate  and  redesign  their  final  projects  and  support  them  along  the  way.  Also,  their 
technical  support  person  would  be  helping  them  to  achieve  their  goals  and  would  be  providing  them  with  support  to  learn  the 
necessary  technical  skills  to  complete  their  projects. 


Summary 


From  the  implementation  of  the  PT3  Project,  many  lessons  were  learned  that  can  be  used  to  increase  the  technology 
learning  of  teachers  and  faculty.  The  following  list  summarizes  the  discoveries  described  here.  In  this  section,  the  term  learners 
refers  to  the  educator  who  is  attempting  to  learn  new  technology  skills  and  increase  technology  integration. 

• On-going  training  should  be  provided,  both  technical  and  technology  learning  support,  and  should  be  flexible  and 
individualized. 

• Support  provided  for  long  term  learning  should  include  regular  follow  up  and  contact  with  learners  - via  email  or  phone  when 
not  in  person. 

• Manuals  and  reference  documents  should  be  provided  when  learners  are  working  independently 

• Note  taking  done  while  learning  to  use  hardware  and  software  is  often  useful  for  learners  when  working  independently. 

• Learners  should  be  provided  concept  mapping  tools,  and  taught  to  use  concept  maps  and  other  visual  tools  for  project  planning, 
time  management  and  for  setting  and  achieving  goals. 

• Learners  should  be  provided  opportunities  and  encouragement  to  work  in  groups  right  from  the  start,  and  be  discouraged  from 
remaining  separate,  even  during  informal  training. 

• Maintain  contact  with  building  and  district  level  administrators  to  get  them  involved  and  excited,  and  keep  them  informed. 

• Experienced  participants  should  be  encouraged  to  share  their  successes  with  new  group  members. 

• All  projects  that  learners  are  expected  to  undertake  should  be  relevant  to  them. 

• If  taught  to  write  reflectively,  learning  logs  can  be  very  useful. 

Any  professional  development  project  should  look  closely  at  the  needs  of  its  community  members,  and  consider  which 
items  can  be  applied  most  successfully.  In  the  creation  and  implementation  of  University  of  Pittsburgh’s  PT3  Project  professional 
development  program  for  teachers,  discoveries  were  made  that  could  be  useful  in  helping  to  avoid  difficulties  and  identify  areas  of 
potential  weaknesses.  They  can  also  be  used  to  identify  strengths  that  can  increase  probability  of  success. 
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Abstract.  A teaching  experience  merging  many  collaboration  strategies  and  e-leaming 
activities  is  presented.  It  has  taken  place  at  the  University  of  Lugano  (Switzerland),  and 
involved  33  undergraduate  students  attending  the  course  “Advanced  Enterprise 
Applications  of  Information  and  Communication  Technologies”  in  the  Faculty  of 
Communication  Sciences.  The  experience  - actually,  a role-play  - consisted  in  developing 
an  online  course  on  business  models  on  the  web  (a  course  freely  available  online),  and 
showed  how  an  hybrid  approach  can  help  integrate  online  activities  and  other  academic 
teaching  strategies.  The  results  of  this  activity  are  discussed,  as  well  as  some  ideas  for 
future  experiences. 


Setting  the  scene 

The  Lugano  University  (USI)  was  bom  in  1996  and  is  the  most  recent  Swiss  University  and  the 
only  one  in  the  Italian  speaking  part  of  Switzerland  (Koenig  2001). 

USI  hosts  a Faculty  of  Communication  Sciences  that  offers  four  different  curricula:  Mass  and  new  Media 
Communication,  Corporate  and  Institutional  Communication;  Technologies  for  Communication  and,  since 
2000,  Communication  in  Educational  Settings.  While,  at  first,  degrees  were  awarded  after  four  years,  the 
University  has  adopted  the  3+2+3  model  following  the  European  Union  standards  (since  the  academic  year 
2001-2002).  Students  come  from  Switzerland,  Italy,  and  many  other  countries;  their  mother  tongues  are 
mainly  Italian,  German,  French  and  English,  and  courses  are  taught  in  Italian  as  well  as  in  other  languages. 

This  paper  presents  an  experience  done  during  the  first  semester  of  the  academic  year  2000-2001, 
with  students  of  the  fourth  year  (from  the  curricula  Corporate  and  Institutional  Communication  and 
Technologies  for  Communication)  attending  a course  called  “Advanced  Enterprise  Applications  of 
Information  and  Communication  Technologies”^. 

Students  were  supposed  to  attend  lectures  and  to  work  in  small  groups  at  different  projects 
concerned  with  the  course  topic:  1 1 groups  were  involved  in  the  project  that  is  now  to  be  presented. 


The  roles  and  the  play 

The  exercise  consisted  in  a role  play:  students  pretended  to  be  members  of  an  international 
consulting  agency  (one  of  the  top  ten  in  its  field),  and  had  to  build  up  an  online  course  for  their  colleagues 
junior  consultants,  in  order  to  make  them  aware  of  the  many  possibilities  for  (as  well  as  constraints  and 
shortcomings  of)  online  business.  Junior  consultants  attending  the  course  had  not  to  become  experts  in  the 
field  - the  company  already  had  those  experts:  namely  all  the  people  involved  in  the  role-play  - but  only  to 
become  capable  of  understanding,  when  doing  their  consulting  activity,  if  there  was  any  room  for  online 
business. 

The  course  had  to  be  in  Italian,  for  self  study,  and  all  the  materials  should  be  covered  in  not  more 
than  24  hours  (three  working  days).  The  platform  for  course  delivery  had  to  be  WebCT^,  while  all  the 


* Institute  for  Communication  and  Education  and  Doctoral  School  NewMinE:  New  Media  in  education  - 
www.lu.unisi.ch/newmine. 

^ TEC  lab:  Technology  Enhanced  Communication  Laboratory  - www.lu.unisi.ch/tec-lab. 

^ The  course  was  held  by  professor  Paolo  Paolini  assisted  by  Lorenzo  Cantoni,  who  designed  and  led  the 
experience,  and  Davide  Bolchini. 

^ WebCT  Version  3.6  Standard  Edition  (www.webct.com,  last  visit:  October  2001). 
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involved  groups  had  to  collaborate  also  using  the  University’s  intranet  and  a tool  for  online  discussions 
(WebBoard^). 

Course  developers  were  divided  into  three  different  types  of  groups:  a.  management  and  reporting: 
one  group  of  three  students,  randomly  chosen;  b.  pedagogical  design  and  technical  implementation:  two 
groups  of  two  students  each,  selected  because  they  were  more  expert  in  the  use  of  technologies  and  had 
previously  attended  a course  on  how  to  use  the  WebCT  platform;  c.  content  development:  eight  groups  of 
three  students  (randomly  chosen)  were  assigned  a different  business  model  according  to  a taxonomy  of 
Internet  business  models  proposed  by  Mike  Rappa  (2000):  1.  Brokerage,  2.  Advertising,  3.  Infomediary,  4. 
Merchant,  5.  Manufacturer,  6.  Affiliate,  7.  Community,  8.  Subscription  and  Utility. 
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Figure  1:  Synoptic  of  the  role-play 


Students  were  briefed  during  a lecture,  they  were  also  given  a detailed  printed  timetable  of  the  exercise, 
with  all  the  group  assignments^: 


Group/week 

week  1 

week  2 

week  3 

week  4 

weeks 

weeks 

week  7 

week  8 

weeks 

Man.  & reporting 

start 

reporting 

reporting 

reporting 

reporting 

reporting 

reporting 

reporting 

reporting 

ped.  design  & tech, 
implem.  (1) 

start 

format 

guidelines 

implementing 

implementing 

implementing 

implementing 

prototype 

ped.  design  & tech, 
implem.  (2) 

start 

format 

guidelines 

implementing 

implementing 

implementing 

implementing 

prototype 

Content  dev.  (1) 

start 

output  1 

output  2 

output  3 

Content  dev.  (2) 

start 

output  1 

output  2 

output  3 

Content  dev.  (3) 

start 

output  1 

output  2 

output  3 

Content  dev.  (4) 

start 

output  1 

output  2 

output  3 

Content  dev.  (5) 

start 

output  1 

output  2 

output  3 

Content  dev.  (6) 

start 

output  1 

output  2 

output  3 

Content  dev.  (7) 

start 

output  1 

output  2 

output  3 

Content  dev.  (8) 

start 

output  1 

output  2 

output  3 

Group  a.  was  in  charge  of:  supervising  and  monitoring  all  the  other  groups’  activities,  promoting  them, 
reporting  to  the  role-play  master^  being  an  interface  between  him  and  all  the  other  groups,  doing  a weekly 
report  of  about  5 minutes  during  course’s  lectures,  defining  a suitable  conference  structure  for  online 
communication  via  WebBoard.  The  group  had  also  to  write  a short  introduction  to  the  online  module,  in 
which  the  course  and  Business  Models  were  defined  and  introduced.  Group  a.  had  periodical  (usually  once 
a week)  meetings  with  the  role-play  master. 


^ WebBoard  4.2,  Web  Forums  & Chat  Software  (www.chatspace.com/products/webboard/index.cfin,  last 
visit:  October  2001). 

^ The  exercise  started  the  30^*^  November  2000  and  ended  the  S**’  February  2001;  only  working  weeks  are 
considered  in  the  table. 
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Groups  b.  had  to  firstly  define  guidelines  for  all  the  content  development  groups  on  how  to 
package  the  materials  they  had  to  prepare  (e.g.  document  length,  format  and  style),  and  to  design, 
implement  and  test  the  prototype  course. 

Groups  c.  had  first  to  research  their  specific  topics  (starting  from  Rappa’s  website  and  looking  for 
relevant  literature,  both  online  and  offline).  Students  were  asked  to  perform  their  research  as  a kind  of 
webquest  (Clarke  1990;  Dodge  2001),  and  select  relevant  references  from  the  results.  Moreover,  groups  c. 
had  to  write  a suitable  text  explaining  the  features  of  the  assigned  business  model  (output  1).  Groups  c.  had 
also  to  deliver  a slide  show  to  briefly  present  the  subject,  and  to  write  two  case  studies  of  online 
applications  implementing  the  business  model  (output  2);  in  addition  to  that  they  had  to  create  a self- 
assessed  test  that  could  enable  the  learners  to  test  their  understanding  of  the  contents  covered  by  the  module 
(output  3).  All  the  three  outputs  were,  in  turn,  input  for  groups  b.  All  the  materials  had  to  be  written  taking 
into  account  that  an  average  learner  (a  junior  consultant,  with  an  experience  similar  to  that  of  the  students) 
should  cover  every  module  in  no  more  than  three  hours.  Each  content  development  group  was  introduced  to 
the  key  features  of  the  assigned  business  models  by  the  role-play  master  in  a kick-off  meeting.  The  meeting 
allowed  the  group  members  to  acquire  the  conceptual  elements  to  interpret  the  large  amount  of  literature 
available  on  the  theme. 


Rehearsing  and  performing 

All  the  groups  worked  hard,  and  produced  their  output  on  time.  Group  a.  organized  - after  some 
trials  ~ 13  different  online  conferences  on  WebBoard:  1 1 devoted  to  every  group,  and  used  by  its  members 
for  their  internal  communication  as  well  as  by  the  management  group  to  post  messages  to  the  single  groups. 
The  remaining  two  conferences  were  titled  “General”  and  “FAQ”  and  were  used  by  group  a.  to  give  general 
instructions  and  information,  and  to  sum  up  organisational  topics  of  common  interest.  Except  for  the  FAQ 
conference,  with  5 messages,  all  the  others  got  from  16  to  45  messages  (mean  value:  28).  The  number  of 
messages  posted  on  the  WebBoard  for  each  conference  is  shown  in  Figure  2.  Groups  b.  and  c.  had  to  post 
and  update  a “state  of  the  art”  message  on  their  own  conferences,  where  they  summarized  weekly  what  they 
did  (acta),  what  they  were  going  to  do  the  following  week  (agenda),  and  if  they  needed  anything  from 
group  a.  (quaerenda).  The  weekly  state  of  the  art  reported  the  essential  information  needed  by  the 
management  team  in  order  to  have  an  overview  on  the  state  of  the  project  and  to  coordinate  and  plan  the 
work  in  parallel  of  the  nine  groups.  As  any  of  the  groups  failed  to  publish  the  state  of  the  art  (or  it  was  too 
vague  and  generic),  the  managers  posted  a message  in  the  group  conference  to  eagerly  solicit  to  deliver  the 
expected  report.  People  of  groups  c.  interacted  very  well  in  each  group,  but  did  not  show  much  interest  in 
what  the  others  were  doing  (anyway,  they  were  infoniied  regularly  by  group  a.).  Rich  communication  flows 
were  established  between  group  a.  and  all  the  others,  and  between  groups  b.  and  groups  c.  Figure  2 shows 
the  number  of  messages  posted  within  each  conference  on  the  WebBoard. 

Many  communication  exchanges  took  place  in  the  real  world:  students  met  almost  every  day  in  the 
university,  but  at  the  same  time  a lot  of  communication  was  also  carried  out  via  the  WebBoard,  sometime 
just  summarising  what  was  already  said/discussed/decided  in  face  to  face  communication  exchanges  (hence 
making  group  members  sure  that  they  all  shared  the  same  knowledge  and  the  same  decisions).  Besides 
WebBoard,  also  email  was  used  for  coordination  and  communication  among  group  members.  For  file 
exchanging  the  university  intranet  was  used:  so  there  were  not  any  upload/download  problem  for  groups  b. 
that  implemented  and  tested  all  the  course. 

The  online  modules  showed  that  students  had  learned  quite  a lot,  and  were  able  to  communicate 
their  knowledge  in  a format  suitable  for  an  online  course.  Not  all  students  delivered  the  same  quality:  some 
modules  were  better  than  others  in  terms  of  content  accuracy,  wording  style  and  rhetoric  effectiveness. 
However,  the  overall  quality  of  the  course  was  good. 
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Figure  2:  Number  of  messages  posted  on  the  WebBoard  for  each  conference. 

Due  to  the  quality  of  the  product,  it  has  been  later  made  freely  available  online  at  the  following 
location:  www.lu.unisi.ch/tec-lab/courseware.htm.  The  facility  of  having  the  module  content  both  in  textual 
form  and  in  slide-style  format  turned  out  to  be  very  fruitful.  In  fact,  the  course  modules  have  been 
effectively  reused  later  for  other  courses  and  workshops  organized  within  the  Faculty  (about  online 
business  models  and  electronic  commerce),  which  needed  both  slide  presentations  and  text  handouts. 

The  flexible  structure  of  the  module  allows  to  learn  a business  model  from  different  perspectives 
and  according  to  different  levels  of  depth:  the  text  explains  in  details  the  roles  and  the  relations  defining  an 
online  business  model  (with  its  variants  and  sub-models)  pointing  out  benefits  and  drawbacks;  the  case 
studies  illustrated  practical  implementation  of  the  model  in  a current  web  application;  the  slide  show  serves 
mainly  for  revision. 
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Figure  3:  Each  course  module  designed  by  the  students  is  made  by  four  components:  the  explanation  of  the 
business  model,  a set  of  references,  two  case  studies  and  a slide  presentation. 


Pedagogical  considerations  and  ideas  for  future  experiences 

The  project  allowed  the  students  to  learn  about  online  business  models  by  designing  and  creating 
an  online  course  on  that  topic:  this  was  a sort  of  loop-input,  i.e.  the  actual  correspondence  between  the 
content  taught  (advanced  applications  of  ICT)  and  the  way  it  was  learned  by  the  students  (designing  an  e- 
leaming  activity).  The  main  benefit  of  the  described  loop-input  strategy  is  that  students  could  live  two  kinds 
of  experiences  at  the  same  time  (collaborating  in  creating  the  online  modules  and  learning  about  online 
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business  models),  both  strongly  relevant  to  the  core  topics  of  the  academic  course.  The  two  experiences  are 
strongly  correlated  and  support  each  other,  enhancing  the  effectiveness  of  the  overall  learning  experience. 

Moreover,  this  experience  helped  combine  a number  of  collaboration  strategies  and  technology 
assisted  learning  activities:  1.  computer  assisted  collaboration  (communication  exchanges  between  work 
groups  mainly  by  means  of  WebBoard  and  University  intranet);  2.  web-based  distributed  learning  (the 
online  modules  delivered);  3.  meetings  and  lectures  (between  the  role>play  master  and  each  work  group; 
between  work  groups  and  the  managers;  between  work  groups  and  the  design  groups);  4.  webquests  (for 
searching  the  relevant  material  to  be  put  in  the  references  of  the  module);  5.  classroom  discussions  and 
presentations  (at  the  end  of  the  project  each  group  presented  the  work  to  their  colleagues  and  the  academic 
staff);  6.  a constructivist  activity  has  been  merged  with  academic  teaching,  leaving  to  all  “stakeholders”  the 
possibility  to  contribute  as  much  as  they  could  to  the  common  knowledge;  7.  e-learning  activities  have  been 
integrated  inside  a more  comprehensive  framework,  and  not  isolated  as  sometimes  happens  when  adopting 
the  “Manichean”  point  of  view:  “face-to-face  vs.  online”  (Rowley,  Lujan  & Dolence  1998;  see  also  Cantoni 
& Paolini  2001);  in  particular,  8.  also  a lively  relation  with  the  professor  and  the  assistants  took  room  as  a 
milestone  of  the  learning  experience  (Dufeu  1994). 

It  could  be  argued  that  in  a similar  organization  of  the  role-play  - where  the  collaboration  between 
groups  of  students  is  a necessary  condition  for  the  success  of  the  project  - the  failure  of  one  student  group 
(e.g.  no  output  delivered  on  time,  no  communication  with  other  groups,  no  respect  to  the  management 
guidelines)  could  cause  the  crash  of  the  entire  project.  Actually,  this  was  not  the  case:  the  project 
organization  took  into  account  this  possibility.  Even  if  any  of  the  8 groups  of  students  responsible  for  the 
content  development  had  failed,  that  would  have  not  stopped  the  project,  which  would  have  still  delivered  a 
running  online  course  (but  with  a module  missing). 

The  problem  would  have  been  harder  if  the  pedagogical-design  and  implementation  groups  had 
failed  the  job  (other  students  with  similar  technological  skills  were  difficult  to  find).  In  case  of  failure  of  the 
management  group,  that  could  have  been  replaced  by  the  role-play  master. 

The  robustness  of  the  project  organization  was  due  to  the  fact  that  the  content  developers  groups  could 
work  in  parallel  because  the  content  modules  had  minimum  or  none  overlapping. 

One  year  after  the  end  of  the  course,  students  who  were  involved  in  the  project  were  asked  to  give  their 
feedback  through  a simple  questionnaire.  Among  the  positive  aspects  of  the  project  experience,  students 
mentioned  that  they  learned  concepts  underlying  online  business  models;  learned  how  to  use  a collaboration 
tool  like  WebBoard;  combined  in  practice  technology  with  education;  learned  how  to  define  precise 
guidelines  for  content  developers;  performed  a creative  activity;  lived  a new  class  experience^.  Moreover, 
students  appreciated  that  the  members  of  the  management  group  were  colleagues  (peers)  and  not  the 
professor/assistants.  Respondents  pointed  out  also  weak  points  of  the  collaboration  experience,  such  as  the 
fact  that  some  content  development  groups  were  not  interested  in  the  work  of  the  other  groups;  on  the  other 
hand,  intra-group  collaboration  was  evaluated  as  very  effective.  Some  students  said  that  the  time  frame  of 
the  project  was  too  long  with  respect  to  the  final  result  delivered;  others  claimed  that  there  was  not  enough 
time  to  complete  an  in-depth  reflection  and  research  for  the  content  of  the  modules. 

The  questionnaire  had  seven  main  questions.  Nine  students  filled  in  the  questionnaire.  Following 
charts  (Figure  4 and  Figure  5)  show  the  marks  expressed  by  each  of  the  nine  students  for  the  two  most 
relevant  questions,  the  first  about  the  perceived  effectiveness  of  the  collaboration  within  the  group,  and  the 
second  about  the  perceived  learning  outcome.  The  scale  used  was  a five  points  scale  measuring  the  level  of 
agreement  (1 : not  at  all;  5:  at  all). 

Question  1 : Was  the  collaboration  within  your  group  positive  and  useful? 

Question  2:  Do  you  think  you  got  a useful  insight  in  the  theme  of  Web  Business  Models? 


^ For  the  majority  of  the  students  involved,  it  was  the  first  time  that  they  experienced  such  an  innovative 
project  in  a class.  Only  some  of  the  students  had  experienced  something  similar  (at  a smaller  scale)  at  the 
WebAtelier  of  the  Faculty  of  Communication  Sciences  (a  web  design  lab  led  by  Lorenzo  Cantoni, 
www.webatelier.net).  Each  term,  at  the  WebAtelier  a half-dozen  web  applications  are  designed  and 
produced  by  groups  of  three  students.  Clients  are  public  and  private  companies  in  Switzerland  and  Italy. 
While  the  WebAtelier  offers  a lively  and  realistic  working  experience,  the  project  presented  in  this  paper 
has  two  important  additional  features:  it  allowed  the  students  to  learn  about  the  content  of  the  application 
(web  business  models)  and  to  collaborate  in  a larger  project  (31  students  instead  of  3). 
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In  the  term  starting  in  November  2002,  we  would  like  to  perform  a similar  project.  One  of  the 
lessons  learned  from  this  experience  is  to  select  properly  the  students  and  assign  them  to  group  according  to 
their  competence  and  management  ability  (instead  of  randomly).  The  learning  outcome  of  the  class  could 
also  be  improved  in  the  future  by  verifying  that  the  pedagogical-design  and  development  groups  have 
enough  time  to  learn  the  topic  of  the  course,  without  being  overwhelmed  by  the  time-consuming 
implementation  activities. 

The  project  was  exciting  both  for  the  students  - who  were  committed  to  the  responsibility  of 
coordinating  the  work  and  learning  the  materials  for  delivering  a complete  running  application  - and  for  the 
teachers  - who  could  see  week  after  week  the  success  of  an  innovative  educational  experience.  The 
counterpart  - on  the  teacher  side  - was  the  amount  of  effort  required  for  the  preparation  of  all  the  details  of 
the  activity  (project  schedule  and  possible  breakdowns)  and  the  monitoring  of  the  project  (managers  - who 
represent  the  potential  weak  ring  of  the  chain  - had  to  be  followed  step  by  step). 
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Abstract.  The  last  few  years,  Information  and  communication  technology  (ICT)  has  been 
introduced  in  education  at  a very  rapid  pace.  Large  amounts  of  resources  (time  and  money) 
are  invested  in  the  development  of  educational  resources,  to  be  used  in  the  new  learning 
environments. 

Reuse  of  existing  material  is  often  cited  as  an  approach  to  reduce  the  investments  involved. 
Several  research  projects  focus  on  the  management  of  educational  content,  often  by 
associating  metadata  to  the  resources.  Research  about  the  reusability  of  documents  is  much 
more  rare,  because  it  is  much  more  difficult. 

In  this  paper  we  report  on  our  research  on  reusability  of  educational  documents.  This  is  part  of 
a PhD  research  in  the  context  of  ARIADNE.  We  have  investigated  different  educational 
environments,  focussing  on  the  type  of  documents  that  are  typically  used.  We  have  developed 
a classification  of  document  use,  based  on  four  questions:  how,  when,  who  and  what. 


1.  Introduction 

One  of  the  main  goals  in  research  on  learning  objects  is  to  enable  reuse  technically.  The  focus  is  often  on  the 
development  of  a management  system  for  educational  documents  and  their  metadata.  Users  can  search  learning 
object  repositories  for  relevant  content  and  reuse  it  in  their  courses.  An  example  of  such  a system  is  the 
ARIADNE  Knowledge  Pool  System  [ARIADNE  2000,  Duval  et  al.  2001]. 

Enabling  reuse  technically,  however,  is  only  one  side  of  the  medal;  designing  for  reuse  (i.e.  producing  reusable 
documents)  is  the  other.  Depending  on  the  context,  a document  can  be  more  or  less  reusable  for  a certain 
course.  Much  less  attention  is  paid  to  this  aspect  in  research. 

Reusability  aspects  can  be  divided  into  two  different  groups. 

1.  In  [Barrit  2000]  the  CISCO  reusable  learning  object  strategy  is  presented.  This  strategy  focuses  on 
how  a typical  document  should  be  compiled  from  smaller  units  to  form  a new  reusable  object. 

Z The  other  issue  considers  the  type  of  document,  its  adaptability,  teaching  style,  and  so  on. 

In  this  paper  we  report  on  our  preliminary  findings  on  the  reusability  of  educational  documents.  We  consider 
different  didactical  contexts  and  look  for  properties  a document  should  have  to  be  reusable  in  that  context.  In 
the  first  section  we  define  the  different  contexts  we  consider  for  study.  Then,  we  look  at  how  documents  are 
reused,  by  asking  four  questions  about  the  reuse.  How  are  documents  reused,  who  selects  the  documents,  when 
are  they  selected  and  what  is  reused? 

Reusability  is  also  stronly  influenced  by  the  metadata  for  the  documents.  Clear  and  precise  metadata  helps  the 
instructors  to  retrieve  appropriate  documents.  In  [Cardinaels  et  al.  2002]  we  focus  on  automatic  metadata 
generation  for  learning  objects.  In  this  paper  we  focus  on  the  documents  themselves. 

2.  Pedagogical  Environments  and  Approaches 

In  [Janssens  et  al.  2000]  three  different  learning  environments  are  described  that  are  applicable  for  different 
learning  goals.  A learning  environment  defines  how  the  content  is  handed  to  the  learners  and  how  those  learners 
can  reach  the  goals  of  the  course,  based  on  their  knowledge  and  possibilities.  In  each  environment,  the  teacher 
can  choose  between  different  approaches,  depending  on  his  preferences  and  the  goals  of  the  course.  In  this 
section,  we  briefly  discuss  the  different  environments  and  approaches. 

1.  The  first  environment  is  an  information  environment.  In  this  environment,  the  focus  in  on  providing 
external  or  new  information  to  the  learner.  The  learner  should  process  this  information,  by  memorising 
it,  giving  it  a meaning  or  applying  it  in  other  situations 

The  pedagogical  approaches  for  this  environment  depend  on  how  the  information  is  provided  to  the 
learner,  which  medium  is  used  to  ‘store’  the  information.  The  authors  distinguish  between  five  work 
forms  for  this  environment: 
i.  lectures. 
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ii,  written  material, 

iii,  video  and  documentaries, 

iv,  hypertext  and  hypermedia, 

V,  world  wide  web. 

In  our  discussion  the  distinction  between  written  material,  video  and  computer-based  material  isn’t 
important.  This  kind  of  environment  mostly  relies  on  so-called  ‘expositive’  learning  objects  [LOM 
2001], 

2,  The  second  environment  is  an  interaction  environment.  Here  the  learners  interact  with  each  other, 
with  the  teacher  or  computer  programs.  The  result  not  only  depends  on  the  individual  learner  and  his 
activities,  but  mainly  on  the  cooperation  with  the  environment.  It  isn’t  important  whether  the 
interaction  is  issued  using  some  medium  or  not,  Janssens  distinguishes  between  the  following 
pedagogical  approaches: 

i,  discussion, 

ii,  computer-supported  education, 

iii,  collaborative  work, 

iv,  computer-supported  collaborative  work, 

V,  internet -supported  collaborative  work. 

Document  reuse  in  the  last  three  forms  is  minimal,  mainly  giving  an  exercise  to  the  learners.  From  our 
point  of  view,  we  can  group  these  forms  together  with  the  first  work  form,  discussion, 

3,  The  third  environment  is  the  action  environment.  Its  goal  is  to  train  cognitive  skills  of  the  learners.  As 
in  the  interaction  environment,  the  emphasis  is  on  active  participation,  the  initiative,  however,  must 
mainly  come  from  the  learner  himself.  This  means  that  the  learner  works  mainly  individually  and  he 
guides  the  learning  process  himself.  Typical  work  forms  for  this  environment  are: 

i,  assignments, 

ii,  simulations, 

iii,  computers  as  a tool. 

This  kind  of  environment  mainly  relies  on  so-called  ‘active’  learning  objects  [LOM], 


3.  Characteristics  of  Document  Reuse 

In  this  section,  we  look  at  the  different  environments  and  investigate  how  documents  are  reused.  This  will  help 
us  to  define  characteristics  for  the  reusability  of  documents  in  a specific  context.  We  consider  four  questions 
about  the  reuse  of  documents: 

1,  JVho  selects  the  documents  that  will  be  used? 

2,  fVhen  is  the  selection  performed? 

3,  fVhich  types  of  documents  are  reused? 

4,  Now  are  the  documents  reused? 

In  question  (1)  we  distinguish  between  automatic  reuse  and  manual  reuse.  An  automatic  reuse  system  is  a 
computer-based  environment  in  which  the  computer  selects  the  documents  automatically  from  a pool  or 
repository.  This  selection  can  be  based  on  the  learner’s  capabilities  (as  in  a learner  model)  or  on  prescriptions 
provided  by  the  instructor  (as  in  a pedagogical  model).  With  manual  reuse,  the  instructor  selects  the  documents 
manually  from  the  pool. 

The  main  difference  for  reuse  requirements  on  learning  objects  relates  to  the  descriptions,  i.e.  the  metadata.  An 
instructor  performing  manual  selection  can  both  examine  the  document  contents  and  interpret  the  metadata;  a 
computer  application  can  only  look  at  formally  provided  metadata.  Of  course  intelligent  document  analysers 
could  study  the  content  itself  but  it  is  very  difficult  to  interpret  the  difficulty  of  the  document,  the  time  it  takes  to 
study  it,  and  so  on. 

Note  that  manual  reuse  often  takes  place  without  explicit  use  of  a reuse  system.  We  call  this  silent  reuse.  The 
World  Wide  Web  is  a practical  resource  for  demonstrations,  simulations  and  so  on.  These  are  made  available 
for  internet  users  and  can  easily  be  reused  in  courses.  This  type  of  reuse  is  opposed  to  explicit  reuse  through 
reuse  systems.  Of  course,  storing  documents  for  explicit  reuse  facilitates  searching,  retrieving  and  using 
documents.  Explicit  reuse  enables  specific  queries  with  pedagogical  metadata  parameters  [Duval  et  al.  2001, 
Duval  et  al,  2000], 

In  question  (2),  we  focus  on  when  the  documents  are  selected  for  reuse,  or,  more  specifically,  when  the  decision 
to  (re-)use  a certain  document  is  taken.  Again,  we  recognize  two  extreme  situations.  The  first  is  an  a-priori 


273 


selection  of  documents.  This  means  that  all  documents  are  selected  before  the  start  of  the  course,  or  at  least 
before  the  system  knows  about  the  learners.  In  the  other  extreme  the  documents  are  selected  at  runtime,  just  in 
time  to  be  displayed  to  the  learner.  In  the  latter  situation,  the  system,  being  a computer  application  or  the 
teacher,  can  consider  knowledge  about  the  learners  to  select  the  appropriate  documents. 

The  third  question  asks  about  the  type  of  documents  that  are  reused.  Are  the  documents  always  rather  large,  like 
a slide  set,  or  a complete  textbook?  Or  are  the  documents  very  small,  handling  one  specific  topic.  An  example 
of  such  small  documents  can  be  found  in  a questionnaire  system  in  which  single  questions  are  selected  to 
compile  a test. 

The  last  question,  but  maybe  the  most  important  one  for  the  reusability  aspect,  is  about  how  documents  are 
reused.  From  the  point  of  view  of  the  re-user,  the  easiest  and  most  powerful  reuse  type  is  as-is.  In  this 
approach,  the  document  is  retrieved  and  integrated  in  the  course  without  any  modifications.  More  likely, 
however,  is  reuse  with  some  degree  of  modification.  Such  modifications  can  range  from  technical  adjustments, 
for  example  changing  the  background  colour,  to  semantical  changes  like  changing  the  vocabulary  (e.g. 
mathematical  symbols). 

Another  form  of  reuse  that  is  applied  often  is  based -on  or  indirect  reuse.  With  this  term,  we  indicate  the 
creation  of  new  documents  that  are  based  on  other  documents.  From  the  reuse  perspective  this  reuse  probably  is 
mostly  overlooked,  but  it  certainly  is  reuse.  To  create  a learning  object,  several  such  objects  are  consulted  (cfr. 
the  bibliography  in  texts)  to  compose  a new  document  fulfilling  the  needs  for  that  specific  course. 

Why  a certain  document  is  reused,  with  or  without  modifications,  or  only  as  a source  of  inspiration,  also 
depends  on  didactical  aspects  and  the  learners.  Considering  the  knowledge  of  a group  of  learners,  a document 
can  be  appropriate  to  use.  For  another  group  the  same  document,  in  the  same  didactical  context  might  not  be 
suited  for  reuse  directly.  Maybe  the  document  should  be  extended  with  more  examples,  or  can  be  summarized 
to  a denser  document. 


4.  Document  Reuse  in  Different  Environments 

In  this  section,  we  combine  the  knowledge  about  the  different  learning  environments  and  pedagogical 
approaches  with  the  different  types  of  reuse.  For  each  pedagogical  approach,  we  indicate  the  most  likely  types 
of  reuse  for  the  four  questions.  Table  1 depicts  the  result  of  this  exercise. 


Who? 

When? 

What? 

How? 

Information  envirornnents 

Lectures 

instructor 

a priori 

(fairly)  large 

based -on 

‘Written’  material 

instructor 

a priori 

Small  or  large 

based -on  or 
as  is 

Interaction  environments 

Discussion 

lnstructor/1  earner 

at  runtime 

small 

based  on 

Computer-supported 

education 

system 

at  runtime 

small 

as  is 

Action  environments 

Assignments 

instructor 

at  runtime 

small 

based  on 

Computer  as  a tool 

system 

at  runtime 

small 

as  is 

Table  1:  Document  use  in  different  educational  environments  and  work  forms 


We  now  discuss  some  of  these  pedagogical  approaches,  through  specific  examples.  The  different  pedagogical 
approaches  we  describe  are: 

• Ex  cathedra  lectures 

• Intelligent  Tutoring  Systems 

• Questionnaire  systems 

Of  course,  these  three  are  only  examples  to  clarify  the  questions  asked  above. 

1.  With  ex  cathedra  lectures  we  refer  to  the  more  traditional  teaching  environment,  in  which  the  teacher 
explains  theory.  The  learners  mainly  listen  to  the  teacher  and  follow  in  their  textbook  or  presentation 
slides,  making  notes  where  necessary.  This  pedagogical  approach  in  fact  is  a combination  of  lectures  and 
written  documents. 
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For  the  lectures,  the  instructor  typically  selects  a textbook  before  the  beginning  of  the  course  and  uses  this 
as  a basis.  Examples,  definitions,  etc.  are  taken  from  the  book  and  used  without  serious  modifications. 
Other  reference  material  is  being  used  as  a basis  for  new  material.  The  reuse  is  thus  both  “as  is”  and  “based 
on”. 

The  handouts  and  other  material  that  students  receive  are  mostly  directly  related  to  the  lectures.  So,  these 
documents  are  also  created  from  base  material  and  other  modified  documents. 

All  the  material  is  selected  before  the  start  of  a course,  or  at  least  without  knowing  much  about  the  learners. 

2 Research  on  reuse  in  intelligent  tutoring  system  (ITS)  mainly  focuses  on  reuse  of  active  modules  or 
program  modules  to  make  the  system  intelligent  and  adaptive  to  the  learners  [Sarti  et  al.  1995].  Our  interest 
goes  to  the  reuse  of  the  documents.  One  of  the  main  problems  in  adaptive  systems  is  the  amount  of 
material  that  is  needed  to  serve  all  the  different  learners.  Each  learner  at  each  moment  in  time  should 
receive  a document  that  is  suited  for  his  knowledge  about  the  topic  at  hand,  his  learning  style,  etc.  The 
document  pool  in  these  systems  must  contain  a large  set  of  learning  objects  to  be  effective. 

The  adaptation  to  the  learners  in  an  ITS  is  performed  by  the  system  itself  As  a consequence,  the  system 
must  choose  the  documents  presented.  The  system  selects  the  document  at  runtime,  based  on  information  it 
gathered  previously.  Of  course  a system  cannot  change  much  to  a document,  except  for  maybe  layout  and 
ordering  in  specific  applications;  this  implies  a literal  or  as  is  reuse. 

3.  The  third  system  we  briefly  describe  is  a questionnaire  system.  In  such  systems,  a template  for 
questionnaires  (exams,  multple  choice  questions...)  is  defined  by  the  instructor  and  the  system  instantiates 
it  with  questions.  These  questions  are  selected  from  a repository  and  used  without  modification  (except 
maybe  for  layout  changes). 


5.  Conclusions  and  Further  Work 

In  this  paper,  we  presented  the  preliminary  findings  of  our  research  on  reusable  learning  objects.  Our  point  of 
interest  is  not  so  much  how  documents  can  be  reused,  but  mainly  how  it  can  be  ensured  that  a certain  document 
is  reusable  in  a specific  course.  A developer  only  benefits  from  the  reuse  if  it  doesn’t  cost  (in  time,  money,  etc.) 
more  to  adapt  and  reuse  a document  than  to  create  it  from  scratch.  Unfortunately,  reusabilitydepends  on  a 
multitude  of  parameters,  such  as  the  pedagogical  approach,  the  system  being  used,  the  information  known  about 
the  learners,  and  so  on. 

We  started  with  an  analysis  of  different  didactical  environments  and  pedagogical  approaches.  For  each 
environment,  we  asked  who  selects  the  documents,  when  the  documents  are  selected,  how  they  are  reused  and 
what  type  of  documents  is  reused.  Most  courses  in  this  environment  are  guided  self-study  courses,  with  a 
mixture  of  lectures  and  on-line  resources. 

Clearly,  this  research  is  only  at  the  beginning  and  many  questions  remain  unanswered  at  this  moment.  Topics 
we  will  consider  in  further  research  are  guidelines  for  building  courseware,  extensions  of  the  LOM  metadata  for 
reusability  aspects,  style  checkers  for  on-line  courses  concerning  their  consistency  (audience  consistency, 
presentation  consistency...). 

Looking  at  the  interest  for  learning  object  ‘pools’,  we  can  conclude,  however,  that  there  clearly  is  a need  and 
demand  for  reusable  courseware.  Enabling  search  and  retrieval  through  metadata  is  only  a small  step  towards 
reuse. 
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Issues  in  Automatic  Learning  Object  Indexation 
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Abstract:  Acquiring  a critical  mass  of  pedagogical  elements  is  a condition  sine  qua  non  for  reuse. 

One  of  the  main  obstacles  to  attain  a large  amount  of  documents  is  the  difficulty  in  creating  associated 
metadata.  Accurate  metadata  contain  a considerable  number  of  attributes.  Assigning  correct  values 
toeach  <f  these  attributes  is  a tedious  task  and  few  users  arewilling  to  invest  considerable  time  to  do 
this.  Automatic  indexation  of  learning  objects  could  solve  this  problem.  Furthermore,  it  could  help  to 
create  more  consistent  metadata.  In  this  paper,  we  dis  cuss  several  issues  for  the  development  of  an 
automated  indexation  system  for  IEEE  LTSC  Learning  Object  Metadata. 

1.  Introduction 

The  success  of  educational  metadata  strongly  depends  on  the  willingness  of  teachers,  authors  or  other  users  to 
provide  their  pedagogical  documents  with  accurate  metadata.  Indexing  learning  objects,  however,  is  a tedious 
task  and  an  extra  burden.  Automatic  indexation  tools  could  overcome  this  problem  and  increase  consistency  and 
quality  of  metadata. 

In  this  paper  we  discuss  three  different  approaches  to  automatic  metadata  generation:  profile-based  indexation 
(section  2.1),  document  properties  analysis  (section  2.2),  and  similarity  search  (section  2.3). 

In  the  last  section,  we  briefly  discuss  a framework  for  automatic  indexationthat  combines  these  three  techniques. 
We  are  developing  the  framework  in  the  context  of  the  ARIADNE  Knowledge  Pool  System  [Duval  et  al.,  2001]. 

2.  Different  Approaches  to  Automatic  Indexation 

In  this  section  we  discuss  three  above  mentioned  approaches  for  generating  metadata  about  learning  objects. 

2.1  Profile-based  Indexation 

With  profile-based  indexation,  information  about  the  author/indexer  is  used  to  deduce  document  properties.  The 
general  idea  is  that  different  documents  of  one  specific  author  are  often  quite  similar.  An  obvious  example  is  the 
institution  that  the  author  works  for:  this  information  can  be  inserted  automatically,  as  it  does  not  vary  for  most 
documents. 

We  are  investigating  how  this  profile-based  approach  can  be  extended  to  information  about  the  courses  the 
documents  are  used  in.  Quite  often,  a batch  of  documents  is  indexed  at  once  within  a same  course  context. 
Several  properties  can  be  assigned  to  all  these  documents.  At  the  beginning  of  the  indexation,  the  appropriate 
courseprcflle  canbe  loaded.  As  an  example,  while  indexing  documents  about  sorting  algorithms  for  a 
programming  course,  we  can  preserve  many  of  the  semantic  and  pedagogical  attributes.  The  concept  of  the 
documents  can  be  ‘sorting  algorithm’  and  the  didactical  context  can  be  ‘university  first  cycle’. 

2.2  Document  Properties  Analysis 

Most  technical  metadata  attributes  can  be  derived  automatically  from  the  /«/n«5/c  document  properties.  Obvious 
examples  are  file  size  and  name,  MIME  types,  and  so  on.  In  most  operat  ing  systems,  all  documents  have 
associated  metadata,  such  as  the  file  name,  permissions  for  different  classes  of  users,  etc.  Specific  document 
types  have  document  specific  attributes,  which  provide  more  detailed  information  about  the  document.  We  can 
also  declare  user-defined  attributes  for  the  documents  (e.g.  typical  learning  time,...).  All  these  attributes  can 
easily  be  retrieved  and  recycled  when  the  document  is  indexed  with  Learning  Object  Metadata. 

Many  operation  systems  also  support  inheritance  of  document  properties  through  document  templates.  MS  Word 
and  MS  PowerPoint  documents,  for  instance,  use  a document  template  (.dot  or  .pot  file)  for  their  style 
definitions.  These  templates  carry  metadata  (called  “properties”)  also  and  all  the  documents  based  on  them 
inherit  those  properties.  In  this  manner,  we  can  define  a ‘course specific’  template  with  the  appropriate  metadata 
for  attributes  like  discipline,  main  concept,  didactical  context,  and  so  on.  All  the  course  documents  based  on  this 
template  inherit  these  metadata  values  automatically  (see  Figure  1 ).  As  an  example  we  could  have 
databasecourse.pot  and  programming.pot  file  for  the  slides  of  two  courses,  one  about  databases  and  another 
about  introduction  to  programming. 
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figure  1 : Document  Properties  for  a MS  PowerPoint  Presentation 

2.3  Similarity  Search 

A potentially  very  strong  mechanism  for  automatic  indexation  is  the  reuse  of  existing  descriptions  when 
generating  metadata  for  new  documents.  In  the  ARIADNE  Knowledge  Pool  System,  the  indexer  can  do  this 
manually  [Duval  et  al.,  2001],  by  editing  an  existing  description  to  create  a new  one. 

Automating  this  process  can  be  based  on  similarity  search.  Similarity  search  is  often  implemented  in  web  search 
engines  (see  for  example  GoogleScout  from  http://www.googIe.com).  These  search  engines  compare  web  sites 
and  classify  them  according  to  an  ontology  (see  http://www.dmoz.org  about  the  open  directory  project).  Each 
line  in  the  result  can  be  used  to  query  for  pages  related  or  similar  to  the  page  in  that  line  (Figure  2). 

Extending  this  mechanism  allows  us  to  find  similar  learning  objects  in  the  repository  and  automatically  reuse 
their  descriptions  to  create  a new  one.  The  indexation  tool  creates  a summary  of  the  document  and  searches  for 
matching  summaries  in  the  databases.  The  metadata  for  the  best  matching  document  is  reused  as  the  basis  for 
the  new  metadata. 

3.  An  Indexation  Framework 

The  framework  we  are  developing  fits  in  the  ARIADNE  Knowledge  Pool  System  (KPS)  as  an  automatic 
Learning  Object  Metadata  indexation  tool.  It  tries  to  combine  the  above-mentioned  indexing  techniques.  A first 
prototype  analyses  HTML  and  MS  Word  documents. For  other  document  types,  only  evident  attributes  - mostly 
technical  values  - are  extracted.  The  current  version  of  this  tool  implements  the  document  properties  analysis 
and  profile -based  indexation;  similarity  search  is  not  supported  yet. 

The  result  of  automatic  indexation  is  a LOM  XML  instance,  which  can  be  insened  into  the  KPS  as  a new 
element.  The  framework  allows  easy  extensions  with  new  document -specific  index  classes,  which  handle  one 
type  of  dociments,  for  example  portable  document  format  (PDF)  documents. 

4.  Conclusions 

In  this  paper,  we  proposed  a combination  of  three  techniques  for  automatic  pedagogical  metadata  generation. 
We  are  convinced  that  (semi-)automatic  indexation  can  help  knowledge  pool  systems  to  gather  the  critical 
amount  of  data  to  be  widely  useful  and  used. 

The  prototype  indexation  tool  and  framework  we  discussed  fits  in  the  ARIADNE  Knowledge  Pool  System.  The 
current  version  of  this  tool  indexes  HTML  pages  and  MS  Word  documents.  It  can  be  extended  by  new  indexer 
classes. 

The  third  methodology  of  metadata  generation,  similarity  search,  must  be  studied  further.  In  our  opinion,  this 
can  be  very  helpful  to  generate  accurate  metadata  (semi-)automatically. 
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Abstract:  Considering  the  take-up  of  e-learning  platforms  at  Higher  Education 
Institutions(HEI)  as  an  innovation  process,  it  is  arguable  that  HEI  have  to  be  themselves  re- 
designed and  that  innovations  promoted  at  a course  level  should  therefore  be  articulated 
within  a broader  institutional  level.  This  creating  conditions  for  successful  re-think  the  HEI 
intervention,  allowing  more  flexible  operational  modes.  Focusing  in  the  development  of 
knowledge  and  understanding  about  the  adoption  and  use  of  e-leaming  systems  in  HEI,  a case 
study  was  structured  based  in  an  iniciative  being  launched  at  a traditional  university 
department.  Elements  of  the  environmental,  organisational  and  information  system  context 
are  identified  as  key  influences  of  the  innovation  process. 


Introduction 

Although  it  is  generally  accepted  that  web-based  technologies  offer  valuable  opportunities  for  innovation  in 
education,  their  generalized  integration  in  higher  education  activities  is  considered  to  be  still  relatively  incipient 
(Collis  & Pals  2000).  There  is  a great  interest  in  the  use  of  these  technologies  in  education  but  the  uptake  has 
been  slow  and  with  very  few  visible  changes  (Bates  2001).  In  order  to  consider  the  adoption  of  distributed 
learning  environments  (DLE),  based  in  e-leaming  platforms  in  high  education  institutions  (HEI)  as  an  innovation 
process,  it  is  necessary  to  contemplate  in  the  first  place  the  need  of  HEI  re-structuring  from  an  organizational 
point  of  view,  so  as  to  fully  exploit  the  new  opportunities  offered  by  these  learning  technologies  (Liber  1 999). 
The  innovations  promoted  at  course  level  should  be  articulated  in  a wider  institutional  viewpoint,  leading  to  a 
reappraisal  of  the  traditional  HEI  intervention  (Taylor  1998),  enhancing  the  development  of  more  flexible 
models  of  operation.  The  implementation  of  these  technologies  implies,  as  a rule,  significant  changes  in  both  the 
innovation  and  organization  (Rogers  1 995).  Focusing  on  the  contextual  and  processual  elements  associated  with 
the  action  of  key  players,  the  technology-based  organizational  change  may  be  represented  through  the  model 
adapted  from  Orlikowski’s  proposal  which  points  out  the  interactions  between  the  institutional  context  and  the 
process  of  adoption  and  use  of  the  technology  (Orlikowski  1 993).  Creating  conditions  for  the  adoption  and  use 
of  DLE  in  HEI  can  be  seen  as  influenced  by  the  environmental  context  of  HEI  operation,  from  the  organizational 
context  and  from  the  DLE  specific  information  system  (IS)  context.  Whereas  those  conditions  themselves  will 
also  influence  the  wider  institutional  context. 

A case  study  was  structured  based  on  an  innovation  process  being  launch  at  the  Information  Systems 
Department  (DSI)  of  Minho  University  - Portugal.  Data  collection  covering  the  initiation  phase  of  the  adoption 
process  included  relevant  documents,  reports,  meetings  notes,  and  written  communications  picked  up  in  the 
departmental  mailing  list  and  other  email  exchanges.  Fourteen  DSI  leaders  and  teachers  were  interviewed 
following  a semi-structured  format  during  2000/2001.  Interviews  cover  organizational,  social,  cultural, 
technological,  pedagogical  and  methodological  issues  (Fullan  1991).  The  collected  data  is  being  processed  using 
content  analysis,  which  is  the  main  qualitative  analysis  form  applied  in  this  research  (Bardin  2000). 


Results 

From  content  analysis,  the  key-concepts  associated  to  institutional  change  categories  (such  as  environmental, 
organizational  and  IS  context)  which  influence  the  conditions  for  adopting  and  use  DLE  in  HEI,  were  being 
identified  and  are  presented  below  (Figure  1). 
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Shared  understanding  about  the  Role  of  DL£ 


Technology-based  innovation  in  a HEI  organization  can  be  understood  as  the  result  of  human  interaction,  as  a 
process  where  meaning  is  gradually  worked  out  through  discussion,  dealing  with  different  arguments  and  points 
of  view.  DLE  can  be  exploited  by  teachers  and  by  the  institution  even  though  with  very  different  goals, 
implementation  speeds  and  achievements.  Building  a shared  vision  and  a common  understanding  of  the  role 
technology  plays  in  HEI  is  certainly  a major  issue  to  face.  The  adoption  of  a research  attitude,  along  with  the 
design  and  preparation  of  demonstration  trials  proved  to  be  a way  to  achieve  the  change  desired.  Such  trials  can 
also  contribute  to  a deeper  awareness  and  understanding  of  the  possible  role  of  e-leaming  systems.  They  have 
allowed  DSI  to  mark  "the  development  of  a coherent  and  mature  vision  of  the  use  of  these  products  in  higher 
education"  and  have  also  facilitated  institutional  decisions. 


Institutional  Context 


Environmental  Context  Organizational  Context  Information  Systems  Context 

• HE  system  • Strategic  alignment  • Previous  experience 

• HE  policies  • Culture  and  structure  • System  selection 

• Market  demands  • Leadership  • Technology  management 
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Figure  1.  Institucional  context  influence  in  the  process  of  adopting  and  using  DLE  in  HEI 


Pedagogical  Leadership 

The  discussion  concerning  the  role  technologies  play  in  education  and  HEI  development  is  still  very  active.  This 
type  of  innovation  will,  along  with  the  adoption  and  use  of  a new  technology  in  the  organization,  require  a real 
change  in  professional  practices.  With  this  change  happening  in  the  main  activity  of  an  organization,  even  if 
itself  helped  to  identify  and  give  merit  to  the  decision  to  innovate,  it  still  is  an  ambitious  goal  and  a deep  change. 
This  is  a situation  of  facing  a process  of  pedagogical  re-engineering:  course  enrichment  or  redesign  (Collis 
1 997).  The  competence  in  defining  a policy  for  the  adoption  and  use  of  technologies  in  an  uncertain  context  is  a 
key  issue,  whether  it  refers  to  the  evolution  of  this  technology  in  the  market  or  to  more  global  adoption  policies 
(at  a school  level,  university,  national....).  In  this  context,  emphasing  the  pedagogic  change  which  might  be 
involved,  and  the  notion  that  the  department  must  possess  a distributed  learning  system  turn  out  to  be  decisive 
factors. 
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Abstract: 

At  the  Faculty  Technology  Center  of  Adelphi  University,  we  support  and  advise  faculty 
in  technology -related  questions  or  projects.  We  offer  ongoing  workshops  on  a large 
variety  of  topics  in  educational  and  instructional  technology.  The  workshop  focuses  on 
communicating  difficult  to  explain  concepts  through  a variety  of  media. 

Introduction 

Our  workshop  uses  both  inductive  and  deductive  pedagogies  to  build  an  interactive  course/syllabus 
template  for  visual  learners  (Nishikant  & Sonwalker,2001).  Induction  suggests  that  learners  first  be 
introduced  to  a concept  or  a target  principle  using  specific  examples  that  pertain  to  a broader  topic.  The 
examples  enable  learners  to  develop  their  own  connections  between  the  concept  and  the  examples  that 
confirm  the  principle.  Recommended  content  and  media  objects  for  induction  including  text,  graphics, 
audio  and  video.  The  other  learning  style  is  deductive,  which  is  based  on  stimulating  the  discernment  of 
trends  through  the  presentation  of  data,  simulation,  graphs,  charts  or  other  data. 

Faculty  today  are  widely  required  to  include  online  components  in  their  teaching.  Developing  learning 
material  is  time  consuming  and  challenging.  A generic  template,  for  interactive  course  content  elements, 
can  be  adjusted  for  the  course  goals.  The  template  can  be  used  for  both  the  online  setting  and  face  to  face 
settings.  We  will  discuss  a general  educational  template  that  we  designed  with  Flash.  Results  and  data  will 
be  provided  on  how  faculty  at  Adelphi  used  such  an  application. 

The  workshop  for  designing  visual  learning  material  is  our  first  attempt  in  taking  full  advantage  of  the 
online  medium.  We  first  start  with  building  small  multimedia  elements  to  be  implemented  in  a media  rich 
adaptive  environment. 

Brief  overview  of  the  workshop 

The  workshop  described  here  covers  four  days  (six  hours  a day).  We  had  three  instructors  and  two  student 
Educational  Technologists  providing  information  and  assistance.  We  used  a five-part  framework  of  vision, 
design,  production,  outcomes  and  analysis.  Participants  introduced  themselves  to  each  other  and  conveyed 
their  expectations  and  their  goals. 

The  first  day  we  covered  the  theory  of  visualizing  data  and  basic  theories  of  displaying  visually  difficult 
concepts  (E.  R.  Tufte  ‘Envisioning  information’  Herdeg,  ‘Herdegs  graphics/diagrams’). 

We  used  induction  to  expose  the  participants  to  Flash  and  showed  examples  and  explained  what  might  be 
possible  to  exploit  with  Flash. 

We  employed  hands-on  activity  with  the  program  to  demonstrate  the  basic  elements;  symbol,  timeline, 
layers.  At  the  end  of  the  day  participants  suggests  learning  elements  (concepts)  they  might  want  to  explain 
using  multimedia  elements.  Discussion  with  all  participants  was  initiated. 

The  second  day  a brief  paper  design  is  made.  The  theory  of  design,  flowcharts  and  concept  mapping  is 
covered.  In  the  afternoon  we  started  with  the  first  principles  of  Flash  and  worked  out  a small  assignment 
(via  a template  of  3 layers  1 images  and  2 objects). 

We  discussed  how  effective  templates  can  be.  How  precisely  templates  work  to  explain  the  abstraction  for 
all  type  of  learners? 
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The  third  day,  we  discussed  their  storyboards  (at  least  four  screens  showing  the  animation,  text  and 
images).  We  talked  about  some  theory  for  recording  sound  and  shooting  video.  In  addition  copyright  issues 
are  pointed  out. 

We  wrote  instructions  to  explain  the  equipment  and  software.  To  record  sound,  we  use  the  computer  Sound 
recorder.  Each  participant  records  at  least  1 minute  of  voice  and  music  via  the  internal  CD  player.  For 
shooting  video,  small  personal  cameras  are  handed  out. 

The  assignment  for  video  is  to  give  a brief  introduction  to  each  participant’s  own  course  they  design  their 
visual  material  for.  The  assignment  is  to  find  a setting  and  an  activity,  which  demonstrate  who  or  what  the 
person  is.  Talking  heads  was  discouraged.  This  gathering  of  video  footage  was  a collaborative  effort;  each 
participant  recorded  the  video  of  another  participant. 

At  the  end  of  the  day  in  the  discussion  we  talked  about  the  difficulties  of  recording  voice  and  mixing  sound 
with  music,  and  gathering  some  good  video  footage.  Producing  audio  and  visual  material  stimulated  ideas 
about  how  to  present  this  material  in  an  online  class. 

Day  four  started  out  with  participants  reflecting  on  each  of  the  sound  and  video  products.  After  this  general 
session  the  participants  worked  on  their  animation  and  brought  in  sound  where  needed  to  support  the  visual 
elements.  The  video  clips  where  treated  as  ‘stand  alone’  and  saved  as  QuickTime  movies. 

The  animation  was  saved  as  a shockwave  file  for  easy  delivery  via  the  Internet.  All  these  elements  where 
brought  on  to  a WWW-  page  and  into  Blackboard. 

These  hands  on-activities  helped  communicate  the  wealth  of  opportunities  for  multimedia  to  enhance  the 
learning  experience.  The  design  and  development  of  the  combined  media  components  — text,  graphic,  audio 
video  animation-  will  eventually  depend  on  the  learning  model  appropriate  for  the  delivery  of  given  course 
content.  This  should  be  covered  in  a follow  up  workshop. 

Participants  feedback 

The  participants  expressed  surprise  how  far  they  had  come  in  only  four  days.  There  was  also  concern  that 
too  much  material  was  covered  for  that  condensed  time  period. 

The  workshop  has  made  some  faculty  aware  of  breaking  up  teaching  material  into  smaller  units. 

The  faculty  has  gained  insight  into  how  ‘easy’  it  is  to  use  or  develop  audio  and  video  and  animation.  They 
wanted  to  continue  to  build  more  animation  as  part  of  the  courses. 

Participants  find  using  or  having  a video  camera  in  the  classroom  a good  idea.  They  saw  how  helpful 
footage  might  be  for  reflection  on  teaching  and  for  backing-up  lectures  and  for  lab  experiences. 
Implementation  into  Blackboard  or  WWW-pages  was  relatively  easy  but  not  always  without  problems. 


Our  reflection 

Templates  did  not  really  save  time  or  provide  better  understanding  for  programming  Flash.  A format  can  be 
designed  in  which  elements  can  be  placed.  But  this  is  more  limited  than  a template. 

We  suggest  incentives  for  faculty  to  design  their  own  teaching  material. 
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Attempts  to  create  and  sustain  on-line  professional  communities  for  teacher  education 
have  received  considerable  attention  from  the  press.  Frequently,  however,  on-  line  groups 
do  not  deliver  what  on-line  users  want.  We  can,  with  reasonable  assurance,  identify  and 
nurture  those  factors  that  increase  the  likelihood  of  sustainable  on-line  learning 
community  success.  These  factors  fall  into  four  general  categories.  First,  the  selection  of 
an  ease  of  learning  and  ease  of  use  application  permits  an  adequate  default  setting  to 
begin  operation  yet  permits  users  and  managers  to  customize  their  on-line  environment. 
Second,  creation  and  support  of  users,  moderators,  and  administrators  by  providing 
training  and  supporting  excellent  experiences  that  evolve  into  an  on-line  learning  culture. 
Third,  both  users  and  managers  must  believe  that  content  of  an  on-line  group  is  an  added 
value.  Fourth,  the  effort  required  to  participate  in  an  on-line  professional  group  are  worth 
the  added  effort. 

Most  educators  will  admit  that  the  current  state  of  online  groups  are  not  as  good  as  they 
will  become  (Vail,  2001)  and  perhaps  view  on-line  groups  as  another  e-mail  like  media  to 
be  managed  (Barmann,  personal  communication,  February,  1993).  This  perspective  is 
due  to  educators  attempting  to  fit  technology  into  the  traditional  world  without 
recognizing  how  on-line  interactions  can  change  and  add  value  to  lives.  In  the 
professional  literature  one  rarely  reads  that  deeper  and  different  learning  can  happen  in  an 
on-  line  learning  community,  but  it  can 

Educators  more  commonly  read  about  and  subscribe  to  the  popular  idea  that  face  to  face 
interactions  in  a traditional  room  are  the  ideal.  In  traditional  interactions  the  presenter 
speaks  to  the  audience.  At  a conference  there  is  limited  interaction  and  contact  over  a 
relatively  short  time.  The  use  of  AACE  and  SITE  professional  boards  permit  and 
encourage  more  extensive  and  longer  lasting  connections. 

Personal  interactions  involved  in  face  to  face  learning  can  be  powerful.  We  must  also 
recognize  that  an  on- line  community  of  collaborating  peers  where  every  person  sees, 
discusses,  and  reacts  to  thoughts  and  ideas  of  individuals  within  a group  can  be  extremely 
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accommodating  related  to  content,  as  well  as  time  and  distance.  Participation  in  on-  line 
professional  groups  and  a skillful  use  of  technology  creates  more  than  just  a difference  in 
the  way  we  work.  It  will  provide  an  ubiquitous  on-  line  learning  vehicle,  a place  to 
express  and  develop  ideas,  and  provides  us  with  access  to  a support  group  of  our  peers. 
Such  a community  is  exceedingly  powerful.  An  on-line  learning  community  can  add 
muscle  and  flexibility  to  the  learning  experience.  An  on-  line  learning  community  has  the 
potential  to  add  value  to  the  learning  enterprise. 

In  a recent  welcoming  address  to  MIMIC,  the  Dean  of  Cleveland  State  University,  Dr. 
James  McLaughlin  said  it  exactly  right  when  he  said: 

“Technology  has  become  such  an  integral  part  of  my  life  that  I wonder  how  I did 
without  it.  I knew  that  it  would  allow  me  to  do  my  work  differently.  What  I did 
not  realize  was  how  it  would  change  my  thought  processes,  professional  identity, 
and  communication  frequency  and  style. . . . 

I think  differently  too.  Having  information  at  my  fingertips  allows  me  to  rely  on 
my  memory  for  other  things.  I scan  for  the  immediately  relevant  and  otherwise 
store  information  away  for  quick  retrieval.  I am  more  prone  to  think  ahead  than 
behind.  I seem  more  adventuresome  than  before.” 

Delivered  MAY  20,2002 
Dr.  James  McLaughlin,  Dean 
College  of  Education 
Cleveland  State  University 

Most  educators  still  use  technology  tangentially  as  an  add-on  and  are  not  motivated  to  use 
professional  on-line  groups  in  their  professional  lives.  They  too  often  continue  to 
perceive  the  use  of  technology  negatively  and  an  effort  without  adequate  incentives  or 
tangible  rewards  (Schlager,  Fusco,  and  Schank,  1998).  The  majority  of  teacher  education 
faculty  still  fails  to  infuse  educational  technology  procedures  into  their  educational 
curriculum  at  substantial  levels. 

The  point  is  that,  other  than  using  the  web  for  cheap  access  to  free  materials,  teacher 
educators  seemingly  do  not  recognize  that  on-line  participation  in  a professional  group  is 
beneficial  In  many  instances,  web-based  groups  and  listservs  are  led  by  those  who  have 
failed  to  experience  sufficient  on-line  activities  to  enable  them  to  plan  or  lead  on-line 
groups.  Sometimes  there  is  a failure  to  continue  to  maintain  the  on-  line  activities  once 
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they  were  launched.  Think  about  this,  how  often  is  webpage  content  out  of  date?  We 
believe  that  when  faculty  have  had  few  or  no  positive  experiences  creating  and  leading  an 
on-line  learning  community,  they  seem  to  retreat  toward  teaching  the  way  they  were 
taught,  that  is,  without  technology  (Schlager,  Fusco,  Schank,1999). 

Whether  one  is  an  on-line  learning  community  user  or  leader,  success  requires  the  use  of 
an  application  that  is  robust  and  flexible.  At  AACE  and  SITE  we  use  software  that: 

• is  easy  for  all  users  to  learn 

• is  easy  for  moderators  to  learn  and  manage 

• is  dependable  and  requires  little  maintenance  and  is  secure 

• can  attach  files  created  with  most  popular  applications 

• offers  a variety  of  personalized  settings 

Examples  of  personal  configuration  options  include,  notification  of  the  creation  of  new 
discussion  topics  being  formed  in  one’s  favorite  areas,  subscription  to  topic  threads, 
hiding  topics  and  conversation  areas  where  one  has  little  interest,  and  much  more. 

Many  universities  use  campus-wide  software  such  as  WebCT®  or  WebBoard®  as  their 
solution  for  web-based  content.  Both  of  these  applications  are  excellent.  This 
presentation  will  reference  these  applications  so  that  users  who  are  familiar  with  them 
can  understand  functionality  of  AACE’s  professional  boards. 

Initially  participants  may  view  on-  line  groups  as  being  unsatisfying  and  as  an  isolated 
activity.  This  need  not  be  so.  The  promise  of  anywhere,  anytime  connectivity  is 
becoming  a reality.  When  educators  are  uninitiated  into  an  on-line  culture  or  they  have 
inadequate  training  and  support  the  tendency  is  to  retreat  from  the  on-line  leadership 
position.  Content  flows  into  AACE  and  SITE  forums  participants  begin  to  support  each 
other.  Everyone  is  able  to  see  how  others  think.  This  brings  up  an  important  point  to 
remember,  participation  in  an  on-line  professional  group  is  not  like  sending  e-mail. 
Thoughts  should  be  well  crafted  and  often  demand  thoughtful  reading  and  pithy 
(responses  requiring  deliberation,  not  just  yes  or  no)  replies. 

The  AACE  and  SITE  professional  boards  represent  an  agora,  what  we  would  call  a forum 
for  open  discussions.  Agora  is  an  ancient  Greek  term  for  an  open  space  where  citizens 
come  together  for  public  meetings.  An  agora  was  first  attributed  to  Homer  and  connoted 


285 


both  a)  the  assembly  of  the  people  or  congregation  and  2)  the  actual  physical  setting.  An 
agora,  much  the  same  as  what  we  would  think  of  as  a “commons”  was  regarded  as  an 
ordinary  element  of  their  life.  Every  citizen  had  a voice.  These  are  similar  to  the  learning 
communities  that  AACE  and  SITE  are  developing  on-line. 

On-line  learning  communities  will  evolve.  One  must  be  careful,  however,  because  there 
are  assassins  waiting  to  kill  the  on-line  learning  community.  Apathy  and  non- 
participation is  a dangerous  thing.  When  teachers  and  administrators  fail  to  gradually 
acquire  on-line  management  knowledge  and  skills  they  limit  their  ability  to  imderstand 
and  develop  on-line  learning  communities.  When  one  uses  technology  one  improves 
speed  and  range  of  collaboration  and  communication.  The  use  of  on-line  commimication 
applications  also  exposes  charlatans.  Cultural  readiness  may  trump  some  efforts  to  form 
on-line  learning  communities.  In  the  past  we  have  waited  20  years  for  a new  generation 
of  educational  leaders.  Today  we  can  not  wait.  We  must  work  to  build  the  educational 
culture  that  will  support  and  value  on-line  knowledge  sharing. 

When  Thomas  Carroll  (2000)  asked  the  question,  “If  we  didn’t  have  the  schools  we  have 
today,  would  we  create  the  schools  we  have  today”  he  was  really  asking  us  to  make 
something  different.  An  electronic  on-line  learning  community  is  not  business  as  usual. 
On-line  learning  communities  permit  us  to  share  half  thoughts  and  enlist  others  in  their 
development.  It  permits  others  in  on  the  secret  and  the  private  information,  and  makes  the 
whole  process  more  democratic,  in  part  because  I can  access  so  many  people  so  easily 
(McLaughlin,  2002). 
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Abstract:  Students  today  expect  learning  to  be  linked  to  the  real  world  and  by  using  active 
learning  strategies  we  can  help  students  develop  those  links.  Active  web-based  learning 
involves  the  user  being  engaged  in  a knowledge -building,  skill-building,  or  attitude-changing 
task,  and  being  required  to  meaningfully  and  overtly  interact  with  instructional  content.  This 
brief  paper  explores  the  idea  of  active  learning  on  the  web  and  examines  some  good  examples 
of  web  resources  that  incorporate  basic  principles  of  active  learning. 


Active  Learning 

The  test  of  a good  teacher  ...  is,  “Do  you  regard  ‘learning’  as  a noun  or  a verb?”  If  as  a noun,  as  a thing 
to  be  possessed  and  passed  along,  then  you  present  your  truths,  neatly  packaged  to  your  students.  But  if 
you  see  “learning”  as  a verb[,]  the  process  is  different.  The  good  teacher  has  learning,  but  tries  to  instill 
in  students  the  desire  to  learn,  and  demonstrates  the  ways  one  goes  about  “learning”  (Schorske,  cited  in 
McCleery  1986,  p.  106). 

Schorske  invokes  the  argument  that  “good”  teachers  use  active,  not  passive,  processes  to  encourage 
learning.  This  brief  paper  explores  the  idea  of  active  learning  on  the  web  and  examines  some  good  examples  of 
web  resources  that  incorporate  basic  principles  of  active  learning. 

Many,  if  not  most,  faculty  members  have  an  intuitive  understanding  of  the  term  “active  learning,”  but 
have  difficulty  defining  it.  This  leads  to  general  agreement  over  the  need  for  active  learning,  but  ambiguity  and 
confusion  over  how  to  make  this  part  of  our  instruction.  For  most  of  us,  active  learning  means  learner 
involvement  with  both  the  instructional  content  and  learning  processes.  Logically,  this  implies  that  any  type  of 
interaction  between  the  learner  and  the  instructional  content  indicates  active  learning.  Yet  traditionalists  choose 
“passive”  activities  such  as  listening  to  deliver  instructional  content,  asserting  that  test  grades  “prove”  that 
learning  has  taken  place.  Students  can  learn  from  teachers  who  use  passive  strategies  - they  have  been  doing  so 
for  years.  Listening  to  lectures  is  probably  the  way  most  of  us  spent  the  majority  of  our  time  as  students  in 
various  classrooms.  How  can  we  respond  to  this  argument? 

Learning  strategies  form  a continuum  from  passive  through  active.  While  each  endpoint  is  useful  and 
has  value  in  different  situations,  we  also  agree  with  Bonwell  and  Eison  (1991)  that  in  active  learning 

• Information  transfer  is  less  important  than  skill  building, 

• Higher-order  thinking  skills  (problem  solving,  cognitive  strategies)  are  emphasized, 

• Students  pursue  activities  that  involve  doing  something  more  than  listening,  and 

• Students’  exploration  of  their  own  attitudes  and  values  are  emphasized. 


Examples  Of  Web-Based  Active  Learning  Sites 

It  should  be  an  easy  task  to  find  numerous  examples  of  active  learning  sites  on  the  web.  However,  it  is 
difficult  to  find  truly  interactive,  learning  sites.  Our  operational  definition  for  active  web-based  learning  is  that 
the  user  has  to  be  engaged  in 

• a knowledge-building,  skill -building,  or  attitude-changing  task,  and 

• during  the  usual  course  of  the  task  the  learner  must  be  required  to  meaningfully  and  overtly  interact 
with  instructional  content. 
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We  also  consider  the  core  techniques  Silberman  suggests:  team-building,  on-the-spot  assessment  and  immediate 
learning  involvement.  (1996,  p.  xi) 

A look  at  the  web  revealed  many  excellent  learning  aids,  but  few  active  learning  sites.  Most  of  the  sites 
that  we  classified  as  useful  for  instruction  came  in  the  form  of  web-based  books.  These  sites  were  usually 
colorful,  content-rich,  interesting,  well  organized,  and  focused,  but  lacked  any  feature  that  would  require  to 
learner  to  interact  with  the  content  short  of  clicking  “next”  or  choosing  from  a pull-down  menu.  Another 
common  site  layout  was  a question  bank  or  test  on  a particular  subject  area.  The  third  type  of  site  that  we  found 
was  one  that  promised  to  be  useful  and.  Unfortunately,  many  of  the  truly  interactive  sites  were  fee  based.  Some 
had  part  of  their  content  available  (and  we  used  some  of  these  sites  as  examples)  so  we  could  see  the  added 
value  of  the  active  instruction.  What  follows  is  a compendium  of  sites  that  exemplify  active  learning  principles. 

Sprocketworks  fhttp://www. sprocketworks.com/)  showcases  games  and  other  instructional  resources  designed 
with  Shockwave.  Categories  include  space,  music,  U.S.  history,  flight,  horses,  chemistry,  birds,  ships  and 
oceans.  While  some  of  the  games  are  strictly  knowledge  level  (ocean  trivia,  for  example)  others  do  build 
problem-solving  skills  in  a creative  an  interactive  way. 

A.  Pintura:  Art  Detective  rhttp://www.eduweb.com/pintura/)  is  an  online  game  about  art  history  and  art 
composition  designed  for  fourth  grade  and  up.  In  the  game,  you  play  a 1 940's  noir  detective  with  a degree  in  art 
history.  In  it,  a distraught  woman  asks  you  to  identify  the  artist  who  made  a painting  she  found  in  her 
grandfather’s  attic.  To  do  so,  you  must  examine  paintings  by  famous  artists  from  Gauguin  to  Van  Gogh.  Each 
example  highlights  an  art  concept  such  as  composition,  style  or  subject. 

Learn  Spanish  rhttp://www. studvspanish.com/ ) is  a site  designed  to  complement  an  audio  CD  series.  This  free 
online  tutorial  currently  covers  over  50  separate  topics,  including  written  and  oral  exercises  and  an  automatic, 
free  grading  service.  Also  available  is  a vocabulary  builder  with  interactive  games.  Cultural  Notes  for  Spanish- 
speaking countries,  a text  and  web  translator,  and  other  interesting  and  useful  features. 

Visual  Calculus  (http://archives.math.utk.edU/visual.calculus/l  is  a collection  of  tutorial  modules  that  can  be 
used  to  review  or  teach  calculus.  Examples  abound  in  this  NAWEB  99  winner  in  the  single  course  category. 
NAWEB  annually  honors  outstanding  achievement  in  the  field  of  web-based  instruction.  One  of  the  problems 
associated  with  this  site  is  the  requirement  to  have  a specific  (and  quirky)  plug-in  loaded.  It  may  not  be  for 
everybody. 

Biology  Labs  On-Line  (http://vcourseware.sonoma.edu/)  is  a series  of  twelve  (12)  different  inquiry -based, 
interactive  labs  designed  for  college  and  AP  high  school  students.  This  is  a diverse  collection  of  labs  that  cover 
many  different  subjects  in  the  biology  field.  All  of  these  labs  are  subscription  based.  A free  Sday  trial 
subscription  is  available. 

Who  Killed  William  Robinson?  Race,  Justice  and  Settling  the  Land  (http://web.uvic.ca/historY- 
robinson/indexmsn.html)  is  an  invitation  to  students  to  solve  an  1868  murder  by  examining  the  archival  records 
relating  to  the  death  of  William  Robinson,  a Black  American  murdered  in  the  British  Colony  of  British 
Columbia.  In  seeking  the  real  murderer  site-users  will  come  to  terms  with  racism,  (in)justice  and  patterns  of 
land  settlement  in  the  colonial  world. 


Conclusion 

The  web  is  a great  resource.  It  is  simple  to  present  information  to  students.  But,  learning  involves  more 
than  just  the  presentation  of  information  to  students  - it  involves  the  student  interacting  with  the  information  in 
such  a way  that  the  information  becomes  a part  of  his  knowledge  structure.  As  teachers,  we  are  charged  with  the 
responsibility  with  designing  and  developing  these  knowledge  systems  - systems  that  encourage  active 
participation  of  the  student.  The  preceding  examples  provide  models  of  good,  web-based  active  learning  sites. 
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Introduction  to  Educational  Computer  Technology:  Evaluating  a first  phase  of  a Distance  Learning 
Program  in  Brazil 

The  Government  of  Rio  de  Janeiro,  through  the  Secretariat  of  Science  and  Technology  -SECT  - created  the 
Center  of  Distance  Learning  of  Rio  de  Janeiro  - CEDERJ,  coordinated  by  the  SECT.  The  CEDERJ  is  a 
consortium  of  the  six  public  universities  of  Rio  de  Janeiro  (UERJ,  UNIRIO,  UENF,  UFRJ,  UFF  e UFRRJ.The 
aims  of  CEDERJ  are:  (a)  to  contribute  to  the  spread  of  public  higher  education  among  municipalities  in  the 
interior  of  the  state  of  Rio  de  Janeiro;  (b)  to  foment  the  access  to  higher  education  of  those  that  cannot  study  in 
the  traditional  schedules;  (c)  to  act  on  continuing  education  programs,  with  special  attention  to  teachers  of  the 
public  education  schools  using  distance  learning  mode;  and  (d)  to  increase  the  offer  of  vacancies  in 
undergraduate  and  masters  courses  in  the  State.  There  is  a technical  and  pedagogical  staff  that  is  in  charge  of 
elaborating  different  support  material  for  the  web,  printing,  and  video.  This  staff  is  also  responsible  for  the 
evaluation  process. 

A homepage  ( www.educacaopublica.rj.gov.br ) elaborated  and  maintained  by  our  team  is  a place  for  teachers 
to  get  information  about  courses  and  workshops,  to  play  author  role  writing  articles,  relating  their  experience 
and  interact  with  other  teachers  to  discuss  teaching  and  learning  processes. 

Regarding  one  of  the  CEDERJ  aims,  to  offer  teacher  professional  development  courses  and  workshops,  we 
will  present  an  analysis  of  one  educational  computer  technology  course.  The  purpose  is  to  evaluate  aspects 
such  as  interactivity,  student  autonomy,  and  student  cooperative  work  in  a program  based  on  a constructivist 
approach.  Participants  were  23  teachers  from  different  subjects,  teaching  in  public  middle  and  high  schools  in 
Niteroi,  Rio  de  Janeiro.  The  teachers/professors  who  implemented  this  course  were  members  of  the  CEDERJ 
staff.  They  planned  the  course  and  all  resource  material  to  be  used. 

The  Course 

The  course  was  developed  by  a multidisciplinary  team:  educators,  designers,  programmers,  illustrators  and 
editors.  The  lessons  dealt  with  hands-on  activities  to  develop  basic  computer  skills.  For  many  of  the  participant 
teachers,  the  face-to-face  phase,  focus  of  this  paper,  was  a unique  opportunity  for  handling  a computer.  The 
topics  discussed  included  the  Windows  environment,  using  the  Internet,  registering  an  e-mail  account,  using  a 
text  editor,  and  participating  in  educational  chats  and  forum.  To  better  achieve  our  goals,  instead  of  just  telling 
the  teacher/students  what  to  do,  all  the  tasks  were  always  initiated  with  a challenging  question  and  not  with  an 
explanation.  It  was  a 20  hour-course,  divided  in  five  meetings. 

The  Didactic  Material 

The  didactic  material  included:  material  to  be  used  by  teachers  or  future  tutors,  printed  material  for  participants 
and  a CD-Rom  with  programs  and  a navigation  simulator  to  be  used  in  the  computers  if  any  problem  emerged 
with  the  machines  during  the  meetings. 

Teachers  in  (inter)  action 

The  participants  worked  in  pairs,  attending  Vygotsky's  theory  about  knowledge  production.  They  registered 
their  findings,  suggestions  and  critiques  in  a daily  log,  written  at  the  end  of  a meeting  using  the  WordPad. 

The  advantage  of  using  a collaborative/cooperative  learning  approach  is  to  promote  a reflection  not  only  upon 
an  individual  work  but  also  on  the  group  production.  Students  come  to  view  their  peers  not  as  competitors  but 
as  resources.  Mutual  tutoring,  a sense  of  shared  progress  and  shared  goals,  and  a feeling  of  teamwork  are  the 
natural  outcomes  of  cooperative  problem-solving,  and  these  processes  have  been  shown  to  produce 
substantial  advances  in  leaming.The  teacher  served  as  a guide,  rather  than  the  source  of  knowledge.  The 
teacher  was  there  to  organize  and  to  assist  the  participants  while  they  take  the  initiative  of  their  self-directed 
explorations,  instead  of  directing  their  learning  autocratically.  Flexibility  was  the  most  important  feature  of 
teacher's  role  in  such  environment.  Sharing  their  findings  allows  teachers  and  participants  to  reflect  upon  their 
actions  and  to  recognize  all  contributions  as  valid  ones.  It  was  important  to  reassure  their  selfconfidence  and 
moreover  to  develop  a new  attitude. 

Evaluating  this  course 

Our  goal  was  to  evaluate  this  course,  to  do  so  it  was  important  to  find  a model  that  helped  us  to  identify 
indicators  to  be  used  in  future  evaluations.  We  found  out  that  the  conflict  between  quantitative  and  qualitative 
analysis  relies  more  on  the  nature  of  the  goals  than  in  a natural  cause. 


290 


A particular  method,  illuminative  evaluation,  is  now  being  widely  used  in  the  U.K.  for  evaluating  human- 
computer  interaction  and  computer-assisted  learning.  Illuminative  evaluation  approach,  first  outlined  by  Parlett 
and  Hamilton,  addressed  the  question  of  using  scientific  quantitative  approaches  to  educational  evaluation 
without  regarding  specificities  of  each  research  field.  Our  investigation  matches  the  suggestions  of  Parlett  and 
Hamilton  (1972)  to  embrace  a more  qualitative  approach. 

"...  to  study  the  innovatory  programme:  how  it  operates;  how  it  is  influenced  by  the  various  school 
situations  in  which  it  is  applied;  what  those  most  directly  concerned  regard  as  its  advantages  and 
disadvantages;  and  how  students'  intellectual  tasks  and  academic  experiences  are  most  affected,  it  aims 
to  discover  and  document  what  it  is  like  to  be  participating  in  the  scheme,  whether  as  teacher  or  pupil; 
and,  in  addition,  to  discern  and  discuss  the  innovation's  most  significant  features,  recurring  concomitants 
and  critical  processes.  In  short  it  seeks  to  illuminate  a complex  array  of  questions: ..." 

Observation  was  one  of  the  main  methods  of  collecting  data  and  was  especially  useful  during  the  early 
'immersion'  period.  The  teachers/investigators  and  researches  built  up  a continuous  record  of  events  that  were 
considered  important  for  this  study  taking  care  to  not  underestimate  events  that  could  also  bring  lights  to  this 
study.  Discussions  between  participants  are  recorded,  and  great  attention  was  on  language  conventions, 
slang,  jargon,  and  metaphors  that  characterize  conversations,  as  well  as  in  written  materials  as  logs  or  e-mail 
messages.  It  revealed  tacit  assumptions,  interpersonal  relationships  and  status  differentials.  Also,  we  were 
very  careful  while  analyzing  data  in  order  to  triangulate  or  cross  check  data  analysis. 

Evaluation  and  analysis  were  done  in  a daily  basis  as  mentioned  before.  After  the  20  hour  course  we  had 
collected  data  from  participants  log  and  e-mail  messages,  and  we  needed  to  categorize  the  indicators  in  order 
to  organize  them  according  to  their  nature.  We  will  present  and  discuss  each  of  the  indicators:  (1)  cooperative 
work;  (2)  autonomy;  (3)  viability  of  the  course;  and  (4)  Interaction. 

In  summary,  teachers  acted  as  participant  observers  and  appraisers  immersed  in  the  context  with  two  specific 
purposes:  to  better  understand  different  aspects  of  course  impact;  and  then  to  extract  considered  tracks  for 
future  change  or  continuance  of  the  program  course.  The  procedure  steps  were:  (a)  All  the  impact  indicators 
were  registered  (b)  Impact  evidences  were  selected,  then  (c)  Final  analysis  was  undertaken  and  generated  a 
discussion  study  group  that  is  still  in  the  process  of  planning  future  courses. 

Analysis  and  Partial  Outcomes  - Regarding  the  course  content  and  pedagogy 
The  group  chosen  for  this  study  was  quite  heterogeneous  and  some  of  the  participants  had  attended  other 
courses.  Eighteen  of  them  had  computers  at  home,  three  of  those  used  computer  a few  times  to  type  a text; 
only  six  considered  themselves  as  not  so  regular  users;  and  the  others  had  never  used  a personal  computer 
before  beginning  this  course.  They  were  quite  unanimous  when  expressed  their  impressions  of  the  first 
meeting,  in  their  words  “it  is  a first  time  I have  the  experience  of  learning  the  basics". 

Participants  who  had  difficulties  handling  the  mouse  in  the  first  days  were  encouraged  to  play  solitaire  or  free 
cell  before  class  started.  And  by  the  end  of  the  week  they  were  so  skilled  in  using  a mouse,  they  almost  forgot 
their  initial  difficulties.  Finally  they  were  comfortable  with  that  new  technology  and  above  all  thinking  about 
using  it  pedagogically  in  their  daily  routine. 

Looking  forward 

It  is  fundamental  in  planning  educational  computer  technology  courses  to  think  about  partnership  instead  of 
the  traditional  mode.  When  participants  are  not  treated  as  empty  vases  but  as  authors  of  their  learning  and 
knowledge  production  they  feel  confident  in  establishing  goals  for  significant  changes  in  their  praxis. 

This  evaluation  revealed  that  regarding  our  country  peculiarities  it  is  necessary  to  have  a face-to-face  phase 
before  a Distance  Education  course.  To  refer  to  one  crucial  point,  most  of  the  in-service  teachers  were  not 
exposed  to  any  computer  technology  in  their  undergraduate  education  and  as  we  mentioned  before  even 
those  who  have,  now  a day,  a personal  computer  do  not  know  how  to  use  it.  Therefore  it  was  and  still  is 
important  to  demystify  the  use  of  those  machines  in  a course  that  helps  participants  to  develop  autonomy,  self- 
confidence  and  consequently  desire  to  continue  learning. 
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Instigating  Cultural  Change  Via  Targeted  Initiatives  in  a Research- 

led  University 


Abstract:  Although  the  potential  exists  to  transform  higher  education  through  the 
impact  of  technology  founded  on  pedagogy,  many  itempts  have  met  with  mixed 
success  because  of  the  fundamental  premise  that  academics  have  to  be  intimately 
involved  with  the  selection  and  control  of  their  course.  The  “control  issue” 
argument  that  is  being  presented  here  is  that  the  key  to  successful  integration  of 
digital  technologies  into  the  curriculum  is  by  empowering  the  academics  themselves 
rather  than  provide  some  turnkey  development  service.  This  paper  describes  the 
challenges  faced  in  Trinity  College  Dublin,  a research- led  institution  and  highli^ts 
the  experience  over  three  years  of  target  initiatives  in  a research-led  university. 


Introduction 

Many  universities  worldwide  have  a reputation  based  on  research  excellence,  publications  and 
research  achievements.  In  addition  to  research,  teaching  and  learning  are  seen  to  be  vitally  important. 
Now  there  is  a powerful  new  force  entering  the  picture,  namely  digital  technology.  Technology  is 
impacting  higher  education,  stirring  up  debate,  and  is  beginning  to  restructure  the  learning-scape. 
Today  higher  education  finds  itself  in  the  midst  of  an  increasingly  globally  market-driven  new 
economy,  along  with  rapid  social  transformations  while  dealing  with  the  impact  of  technologies . Given 
changing  student  demographics,  rising  costs,  overcrowding,  faculty  shortages,  increased  accountability, 
and  other  external  factors,  universities  are  re-examining  their  long-standing  values  and  practices  in 
order  to  find  ways  in  which  to  meet  these  challenges  successfully  (Van  Dusen,  1997).  The  debate  over 
digital  technologies  and  meeting  these  challenges,  is  causing  a paradigm  shift  by  moving  higher 
education  away  from  an  educator-centric  environment  toward  a more  learner-centric  educational 
environment  presenting  even  further  challenges  to  educators  in  acquiring  the  necessary  skills  and 
competencies  related  to  technology  and  pedagogy.  In  the  not  too  distant  future,  institutions  that  are 
integrating  technology  successfully  to  improve  teaching  and  learning,  as  well  as  research  will  be  seen 
as  more  dynamic  and  effective  than  their  less-engaged  competitors  (Newman  & Scurry,  2001). 


Strategies  for  E-learning  Integration 

Cultural  change  is  required  for  colleges  and  universities  to  be  successful  in  harnessing  the 
power  of  digital  technologies  in  ways  that  capitalize  on  what  we  know  about  how  students  learn,  that 
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they  leam  more  when  they  are  actively  engaged  in  self-driven  rather  than  passive  learning  activities 
(Newman  & Scurry  2001  & Lim,  2001).  One  can  consider  cultural  change  at  three  levels,  academic, 
organisational  structure,  and  external  influences.  When  developing  policies  and  solutions  these  levels 
need  to  have  representation  across  college. 

We  believe  that  one  of  the  first  places  to  start  is  at  the  academic  level.  Some  of  the  key  factors 
for  instigating  change  at  this  level  include:  identifying  early  adopters,  increasing  awareness;  providing 
professional  development  and  incentives  (e.g.  monetary,  time  off,  recognition);  emphasizing  pedagogy 
over  technology;  ensuring  technical  support  and  sustaining  this  cultural  change.  Secondly,  cultural 
change  is  dependent  upon  the  organizational  structure.  Goals  of  the  university  must  include  harnessing 
the  power  of  technology  to  improve  teaching  and  learning.  The  organization  must  support  this  by 
restructuring  to  some  extent  if  necessary.  Integrating  with  existing  college  services  is  critical,  A 
combined  top-down  and  bottom-up  approach  is  essential.  Thirdly  external  influences  must  be 
considered  such  as  the  shift  from  local  markets  toward  a global  economy;  an  increase  in  the  demand  for 
quality  assessment;  and  the  growing  and  changing  role  the  government  in  higher  education. 


Case  Study 

At  Trinity  College  Dublin  (TCD),  the  University  Council  and  Board  established  the  Centre  for 
Learning  Technology  as  a joint  initiative  involving  College  Academics,  Staff  Development  Office  and 
Information  Systems  Services  which  is  responsible  for  supporting  academic  staff  instigating  & 
developing  innovations  in  teaching  and  learning  with  Information  and  Communication  Technology. 
The  Centre’s  objectives  are  to  support  e-learning  across  College  by:  supporting  numerous  (six  monthly) 
e-learning  projects  in  all  departments;  providing  consultancy  services  for  e-leaming  initiatives  in  TCD 
courses;  and  hosting  seminars,  workshops  and  courses  in  e-learning  pedagogy  & technical  skills.  One 
major  lesson  learned  from  the  Centre’s  successful  initiatives  is  that  a holistic,  pedagogically  led 
approach  to  enhancing  learning  with  digital  technologies  is  required  and  that  it  is  essential  to  integrate 
it  within  the  university’s  academic  community  and  organisational  infrastructure. 

Some  successful  strategies  for  providing  support  identified  here  at  Trinity  College  Dublin  over 
the  past  three  years  include:  integrating  services;  emphasizing  pedagogy  over  technology;  identifying 
early  adopters;  providing  high  quality  professional  development  and  incentives  (e.g.  monetary  or  time 
off);  increasing  awareness  in  incorporating  research  for  publication;  facilitating  knowledge  building 
and  sharing;  ensuring  recognition  and  support  by  college  body;  and  providing  technical/pedagogical 
support. 
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I.  Introduction 

NASA  Langley  Research  Center  and  Christopher  Newport  University  (CNU)  provide,  free  to  the  public,  distance  learning 
programs  that  focus  on  math,  science,  and/or  technology  over  a spectrum  of  education  levels  from  K-adult.  The  effort  started 
in  1997,  and  we  currently  have  a suite  of  five  distance-learning  programs.  We  have  around  450,000  registered  educators  and 
12.5  million  registered  students  in  60  countries.  Partners  and  affiliates  include  the  American  Institute  of  Aeronautics  and 
Astronautics  (AlAA),  the  Aerospace  Education  Coordinating  Committee  (AECC),  the  Alliance  for  Community  Media,  the 
National  Educational  Telecommunications  Association,  Public  Broadcasting  System  (PBS)  affiliates,  the  NASA  Learning 
Technologies  Channel,  the  National  Council  of  Teachers  of  Mathematics  (NCTM),  the  Council  of  the  Great  City  Schools, 
Hampton  City  Public  Schools,  Sea  World  Adventure  Parks,  Busch  Gardens,  ePALS.com,  and  Riverdeep. 

II.  Major  Goals  of  Distance  Learning  Programs 

Our  mission  is  based  on  the  “Horizon  of  Learning,”  a vision  for  inspiring  learning  across  a continuum  of  educational 
experiences.  The  programs  form  a continuum  of  educational  experiences  for  elementary  youth  through  adult  learners.  The 
strategic  plan  for  the  programs  will  evolve  to  reflect  evolving  national  educational  needs,  changes  within  NASA,  and 
emerging  system  initiatives.  Plans  for  each  program  component  include  goals,  objectives,  learning  outcomes,  and  rely  on 
sound  business  models. 

It  is  well  documented  that  if  technology  is  used  properly  it  can  be  a powerful  partner  in  education  [1].  Our  programs  employ 
both  advances  in  information  technology  and  in  effective  pedagogy  to  produce  a broad  range  of  materials  to  complement  and 
enhance  other  educational  efforts.  Collectively,  the  goals  of  the  five  programs  are  to  increase  educational  excellence;  enhance 
and  enrich  the  teaching  of  mathematics,  science,  and  technology;  increase  scientific  and  technological  literacy;  and 
communicate  the  results  of  NASA  discovery,  exploration,  innovation  and  research. 

All  pre-college  distance  learning  programs  support  the  national  mathematics,  science,  and  technology  standards;  support  K- 
12  systemic  change;  involve  educators  in  their  development,  implementation,  and  evaluation;  and  are  based  on  alliances  and 
partnerships. 

In  addition  the  programs  seek  to  invoke  a sense  of  geographic,  ethnic  and  cultural  diversity  by  featuring  schools  from  all  over 
the  U.S.;  schools  from  urban,  suburban,  and  rural  areas;  public,  private,  and  religious  schools;  and  schools  with  large 
populations  of  African-American,  Asian  and  Hispanic  students. 

III.  History  of  Distance  Learning  Programs 

The  Office  of  Education  at  NASA  Langley  began  its  first  distance  first  learning  program,  NASA  CONNECT,  in  1997.  The 
second  and  third  programs,  NASA  Science  Files  (formerly  NASA  Why?  Files)  and  NASA's  Destination  Tomorrow  premiered 
in  2000.  The  fourth  and  fifth  programs,  NASA's  Kids  Science  News  Network  (KSNN)  and  NASA  LIVE,  premiered  in  2001. 

CNU  joined  the  effort  in  1999.  The  programs  have  won  numerous  awards  for  distance  learning,  videography,  and  web  design. 

rV.  Description  and  Methodology  for  Distance  Learning  Programs 

We  define  distance  learning  as  “getting  people —and  often  video  images  of  people— into  the  same  electronic  space  so  they  can 
help  one  another  learn,”  and  we  provide  “a  system  and  process  that  connects  learners  with  distributed  sources”[2].  All  the 
programs  are  appropriate  for  one  or  more  of  the  following  national  audiences:  public,  private  and  parochial  schools;  home 
schoolers;  girls  and  boys  clubs,  colleges  and  universities,  parents,  teachers,  caregivers,  and  adult  learners. 

In  this  section  we  discuss  the  methodologies  used  to  create  the  various  distance  learning  programs.  The  broadcast 
components  air  nationally  on  PBS  stations  and  on  many  Cable  Access  Channels,  but  are  also  available  on  tape,  DVD,  and  the 
web. 
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A.  NASAKSNN 


Goal:  Support  national  efforts  to  prepare  all  children  to  succeed  in  mathematics,  science,  and  technology. 

Description:  NASA's  KSNN  is  a series  of  1 -minute  news  breaks  and  associated  web  support  and  activities  that  uses  children 
to  explain  to  people  of  all  ages  specific  mathematics,  science,  technology,  and  computer  science  concepts  that  often  address 
misconceptions.  Confronting  misconceptions  at  an  early  age  with  the  combination  of  lively,  kid-oriented  newsbreaks  and 
activities  will  fit  well  into  the  grade  3-5  classrooms.  The  newsbreaks  will  get  wide  visibility  as  interstitials  on  Public  Television 
and  commercial  networks.  See  the  web  site  at  ksnn.larc.nasa.gov  for  more  information. 

Methodology:  We  have  a pilot  program  in  place  at  CNU  to  develop  scripts,  activities,  and  web  page  content  for  NASA  KSNN. 
Teams  of  3-4  students  develop  scripts,  activities,  background  material  and  web  page  content  for  one  of  NASA's  KSNN 
newsbreaks  as  part  of  their  course  work  and  the  teams  find  a content  expert  for  their  project  at  CNU,  NASA,  Jefferson  Lab,  or 
another  appropriate  institution.  Effective  communication  with  3-5  graders  through  the  video  medium  is  presented  in  class; 
students  present  their  newsbreak  for  video  taping  and  report  on  the  other  components.  Finally,  a committee  of  CNU  and 
NASA  judges  selects  the  best  projects  each  semester  for  production.  Involving  pre -service  teachers  is  valuable  for  their 
education  and  introduces  them  to  NASA  distance  learning  programs  that  they  can  use  in  their  classrooms. 

Student  Outcomes:  1)  Stimulate  interest  in  and  increase  understanding  of  mathematics,  science,  and  technology;  2)  correct 
commonly  held  mathematics,  science,  and  technology  misconceptions;  3)  gain  awareness  of  NASA:  programs,  projects,  and 
facilities;  4)  increase  scientific  and  information  technology  literacy;  5)  increase  interest  of  minority  and  female  students  in 
mathematics,  science,  technology,  and  NASA;  and  6)  improve  literacy  of  students  who  do  not  use  English  as  their  primary 
language. 

B.  NASA  Science  Files  (formerly  NASA  Why?  Files) 

Goal:  The  NASA  Science  Files  television  broadcast,  web  site,  and  educators'  guide  use  Problem-Based  Learning  (PBL)  to  help 
students  in  grades  3-5  explore  topics  through  scientific  inquiry. 

Description:  This  program  consists  of  a series  of  four,  standards-based  instructional  programs  for  grades  3-5  that  introduce 
students  to  science  as  inquiry,  the  scientific  method,  problem-based  learning  and  NASA.  Each  60-minute  program  is  divided 
into  four  15-minute  “teachable  segments”  and  is  supported  by  a web  site  and  resource-rich  teacher  guide.  See  the  web  site  at 
scifiles.larc.nasa.gov  for  more  information. 

Methodology:  In  the  television  broadcast,  the  tree  house  detectives  are  confronted  with  real-world  problems  and  are  provided 
with  the  knowledge  and  tools  to  attack  the  problem.  The  tree  house  detectives  walk  through  the  problem  solving  process 
during  the  four  15-minute  instructional  segments,  reaching  a solution  in  the  final  segment.  The  television  broadcast  features 
content  experts  from  NASA  and  the  community  with  careers  requiring  problem-solving  skills,  museum  resources,  experiments, 
and  classroom  activities  to  help  the  tree  house  detectives  solve  the  problem.  The  web  component  of  the  NASA  Science  Files 
is  the  nerve  center  of  the  learning  experience  and  contains  separate  sections  for  students,  educators,  and  parents.  The  kids’ 
tree  house  contains  content  information,  experiments  and  activities,  experts  to  contact,  and  a variety  of  other  resources  to  aid 
student  teams  in  finding  possible  solutions  to  the  featured  online  problem.  The  design  of  the  site  encourages  the  teams  to 
take  on  roles  and  approach  solving  the  online  problem  as  a scientist  would.  An  array  of  Problem-Based  Learning  tools  (action 
plan,  basic  design  log,  PBL  questions,  “get  up  and  go”,  problem  board,  problem  log,  reflection  journal,  scientific  investigation 
log,  scientific  method  flowchart,  scientific  process  log,  and  web  site  evaluation)  is  readily  available  for  them  to  employ  as  they 
research  the  problem.  The  web  site  provides  resources  and  guidance  for  the  team’s  culminating  report  and  presentation.  The 
educators’  area  of  the  site  is  structured  so  that  essential  information  is  one  click  away  and  other  information  a maximum  of 
three  clicks  away.  It  contains  an  overview  of  the  NASA  Science  Files  television  broadcast,  web  site,  and  educators’  guide  and 
is  an  excellent  starting  point  for  professional  development  in  the  area  of  PBL.  PBL  instructional  tools  and  strategies, 
worksheets,  classroom  activities,  and  assessment  tools  can  also  be  found  on  the  site.  The  educators’  guide  is  designed  to 
augment  the  television  broadcast  and  can  be  downloaded  from  the  web  site  or  received  in  the  mail.  The  guide  features 
segment  overviews,  program  objectives,  vocabulary,  classroom  activities,  worksheets,  literature  and  Internet  resources,  and 
implementation  strategies.  A team  of  elementary  school  teachers,  NASA  researchers,  and  NASA  Distance  Learning  Center 
staff  develops  the  programs  and  guides. 

Student  Outcomes:  1)  Increase  understanding  of  Problem-Based  Learning  and  science  as  inquiry;  2)  develop  the  literacy  and 
technology  skills  needed  to  use  the  Internet  for  investigation  and  research;  3)  increase  scientific  and  technological  literacy;  4) 
gain  awareness  of  careers  requiring  problem-solving  skills,  mathematics  and  science  proficiency,  and  technology  application; 
5)  view  mathematics,  science,  and  technology  as  a process  requiring  creativity,  critical  thinking,  and  problem-solving  skills;  6) 
gain  awareness  of  NASA:  programs,  projects,  and  facilities;  and  7)  overcome  stereotyped  beliefs  by  learning  about  women 
and  minorities  actively  involved  in  challenging  engineering  and  scientific  tasks. 


C NASA  CONNECT 

Goal:  Support  the  integration  of  mathematics,  science,  and  technology  in  grades  6-8. 

Description;  NASA  CONNECT  establishes  the  “connection”  between  the  mathematics,  science,  and  technology  concepts 
taught  in  the  classroom  and  NASA  research.  Each  program  consists  of  a 30  minute  broadcast  (also  available  on  video  tape),  a 
classroom  activity,  and  a web  activity,  which  are  supported  by  a web  site  and  resource-rich  teacher  guide.  See  the  web  site  at 
connect.larc.nasa.gov  and  published  articles  [3,4]  for  more  information. 

Methodology:  The  three  components  complement  one  another  and  produce  a cohesive,  flexible  unit  for  classroom  use.  The 
broadcast  intersperses  classes  of  students  from  diverse  backgrounds  and  geographical  locations  doing  the  classroom  and 
web  activities  with  NASA  experts  from  diverse  backgrounds  who  describe  their  research  and  use  of  mathematics.  Two 
dynamic  hosts  weave  the  story  together  and  take  the  students  on  an  adventure  in  learning.  Throughout  the  video,  the 
cartoon  character  Norbert  invites  the  viewers  to  pause  and  consider  questions  on  what  they  are  learning.  Short  animated 
mathematics  activities  are  featured  during  the  video  as  reinforcement.  The  script  for  the  video  is  produced  in  conjunction  with 
educators  and  the  scientists  featured  in  the  video  to  be  an  effective  learning  experience  for  middle  school  students.  The 
activities  are  developed  and  tested  by  educators  who  provide  an  overview,  connections  to  national  standards,  resources,  a 
glossary,  a comp  lete  description  of  the  activity,  and  extensions  and  questions.  In  practice  a teacher  would  ideally  first  use  the 
video  with  the  class,  next  do  the  classroom  activity,  and  finally  act  as  a facilitator  for  the  web  activity.  However,  teachers 
often  don't  have  the  time  for  all  three  components  so  they  are  designed  as  independent  units. 

Student  Outcomes:  1)  Make  connections  between  the  mathematics,  science,  and  technology  taught  in  their  classroom  and  the 
real  world  applications  by  observing  NASA  engineers  and  scientists;  2)  increase  understanding  of  mathematics,  science,  and 
technology  concepts  through  hands-on  and  interactive  web  activities;  3)  develop  the  literacy  and  technology  skills  needed  to 
use  the  Internet  for  investigation  and  research;  4)  view  mathematics,  science,  and  technology  as  a process  requiring  creativity, 
critical  thinking,  and  problem-solving  skills;  5)  gain  awareness  of  careers  requiring  problem-solving  skills,  mathematics  and 
science  proficiency,  and  technology  application;  6)  gain  awareness  of  NASA:  programs,  projects,  and  facilities;  and  7) 
overcome  stereotyped  beliefs  by  learning  about  women  and  minorities  actively  involved  in  challenging  engineering  and 
scientific  tasks. 

D.  NASA  LIVE 

Goal:  Increase  the  understanding  of  mathematics,  science,  technology,  and  NASA  among:  K-12  students  and  educators,  13- 
1 8 students  and  faculty,  and  adult  (life  long)  learners  in  formal  and  informal  settings. 

Description:  NASA  LIVE  consists  of  a series  of  synchronous  (two-way),  interactive  videoconferencing  sessions  designed  to 
communicate  NASA  Langley  research,  educational  programs  and  career  opportunities  to  educators  and  students  in  grades  K- 
12,  grades  13-18,  and  adult  learners.  A 30  topic  series  is  currently  available  on  the  NASA  LIVE  web  site  at  live.larc.nasa.gov. 
Methodology:  The  programs  address  the  educational  need  to  connect  students  to  real  problems  and  real  world  applications  of 
mathematics,  science,  engineering  and  technology  (MSET).  Therefore,  program  topics  are  chosen  based  on  an  educational 
need  to  complement  university  courses  relating  to  MSET  as  well  as  aeronautics.  After  completing  the  online  registration  and 
confirming  the  presentation  date  and  time,  participants  (i.e.,  a NASA  researcher  with  faculty  and  students)  are  engaged  in  an 
interactive  videoconference  live  from  NASA  Langley.  Each  videoconference  consists  of  a 60-minute  session  divided  into  a 
30-minute  presentation,  a 20-minute  Q&A  segment,  and  a 10-minute  overview  of  educational  and  professional  opportunities. 
Outcomes:  1)  Stimulate  and  increase  scientific  and  technological  inquiry  and  literacy;  2)  enhance,  enrich,  and 
facilitate  the  learning  process;  3)  model  the  effective  use  and  integration  of  instructional  technology;  4)  raise 
awareness  of  NASA  pre -college,  college/university,  and  distance  learning  programs;  5)  gain  awareness  of  NASA: 
programs,  projects,  and  facilities;  6)  increase  NASA  and  university  collaboration;  7)  increase  outreach  to  the  informal 
education  community;  8)  increase  interest  of  under-represented  groups  in  mathematics,  science,  technology,  and 
NASA;  9)  improve  literacy  of  students  who  do  not  use  English  as  their  primary  language. 

El  NASA's  Destination  Tomorrow 

Goal:  Create  and  heighten  the  interest  of  high  school  students  and  adult  learners  in  mathematics,  science, 
technology,  and  NASA. 

Description:  Destination  Tomorrow  is  a series  of  five  30-minute  nationally  aired  video  programs  designed  to  increase 
the  scientific  literacy  of  educators,  parents  and  lifelong  learners.  The  material  is  drawn  from  current  research  and 
spin-offs  developed  at  NASA  Centers  - primarily  the  NASA  Langley  Research  Center.  See  the  web  site  at 
destination.larc.nasa.gov  for  more  information. 
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Methodology:  Research,  theory  and  principles  of  adult  learning  guide  the  developers.  Topics  are  chosen  to  broaden 
the  public’s  perception  of  NASA  beyond  space  and  flight  by  discussing  less  known  spin-offs  of  NASA  researchers. 

The  30-minute  show  is  divided  into  five  5-minute  segments:  Behind  the  Scenes,  How  It  Works,  On  the  Runway, 

Future/Tech  Watch,  and  Retrospective  and  two  30-second  “Did  You  Know?”  vignettes.  Two  hosts  set  the  tone  of 
the  show  and  introduce  the  segments.  Individual  correspondents  cover  the  segments  in  depth  and  interview  one  or 
two  researchers. 

Outcomes;  1)  Stimulate  and  increase  scientific  and  technological  inquiry  and  literacy;  2)  enhance,  enrich,  and  facilitate  the 
learning  process;  3)  gain  awareness  of  NASA:  programs,  projects,  and  facilities;  4)  interest  NASA  and  university 
collaboration;  5)  increase  outreach  to  the  informal  education  community;  6)  increase  interest  of  under-represented  groups  in 
mathematics,  science,  technology,  and  NASA;  7)  improve  literacy  of  students  who  do  not  use  English  as  their  primary 
language. 

V.  Validation  and  Evaluation  of  Distance  Learning  Programs 

A.  Testing  and  Evaluation  of  Activities  and  Web  Site 

All  NASA  CONNECT  activities  are  tested  by  teachers  during  a special  summer  program  and  during  the  school  year  in  nearby 
school  systems.  During  the  academic  year,  focus  groups  of  middle  school  teachers  select  three  NASA  CONNECT  programs. 
The  teacher  and  students  are  observed  by  NASA  distance  learning  educators  as  the  teachers  use  the  three  program 
components  with  their  students.  The  observations,  informal  discussions,  and  final  evaluation  report  from  the  teachers  serve 
as  tools  to  continually  improve  the  program.  Activities  from  other  NASA  centers  are  tested  by  the  providers.  As  part  of  the 
NASA  Science  Files  web  site  development,  formative  evaluation  and  usability  tests  are  conducted  before  and  during  each 
season  with  the  help  of  public  and  home -based  educators  in  grades  3-5.  The  results  are  evaluated  and  used  to  improve  the 
usability  and  structure  of  the  web  site.  Testing  in  the  development  cycle  is  essential  to  ensure  a quality  product. 

B.  Analysis  of  Survey  Data 

NASA  CONNECT  has  been  evaluated  by  surveys  sent  to  registrants  for  three  years  and  NASA  Science  Files  for  one  year. 
Although  the  original  questions  have  been  included  each  year,  new  questions  have  been  added  to  improve  the  survey.  For 
the  NASA  CONNECT  1999-2000  season  [5],  1000  registrants  were  selected  randomly  and  27%  of  the  responses  were  usable 
(out  of  a 34%  response  rate).  A total  of  99  questions  in  9 categories  were  asked  regarding  instructional  technology  and 
programming  in  the  classroom.  Generally,  the  respondents  agreed  that  instructional  technology  has  a positive  impact  on 
learning,  is  supported  by  the  administration,  and  that  facilities  are  available  for  its  use.  In  particular,  they  strongly  supported 
web-based  activities  and  would  like  to  see  more  NASA  web  activities.  The  assessment  of  the  NASA  CONNECT  programs 
was  very  positive,  with  overall  ratings  ranging  from  4.17  to  4.63  on  a 5-point  Likert  scale  (compared  to  4.09  to  4.61  for  the  1998- 
1999  season  [6]  and  3.97  to  4.56  forthe  2000-2001  season).  The  videos,  lesson  guides,  classroom  activities,  web  activities,  and 
web  site  were  all  very  highly  rated.  The  use  of  the  videos  ranged  from  19%  to  44%,  lesson  guides  from  22%  to  50%,  classroom 
activities  from  1 8%  to  42%,  and  web  activities  from  9%  to  1 3%.  The  low  use  of  NAS  A web  activities  is  in  contradiction  with 
the  high  ratings  for  positive  impact  on  learning,  availability  of  technology,  and  desire  for  more  web  activities.  The  low  usage 
may  simply  reflect  that  the  web  activities  are  the  last  in  the  chain  of  the  NASA  CONNECT  learning  offerings  (video,  classroom 
activity,  web  activity).  Further,  the  three  lowest  ratings  for  the  NASA  web  activities  were:  content  easily  integrated  into 
curriculum  (4.09),  able  to  complete  in  reasonable  time  (3.86),  and  appropriate  for  my  students  (4.04).  Survey  data  for  NASA 
Science  Files  for  2000-2001  have  overall  ratings  from  4.39  to  4.64.  Both  programs  have  been  given  consistently  high  ratings  for 
their  program  offerings. 

Continental  Research  Associates  (Norfolk,  VA)  conducted  an  evaluation  of  NASA's  Destination  Tomorrow.  The 
methodology  included  telephone  interviews  with  400  (out  of  537)  managers  of  cable  access  television  stations  across  the 
United  States  that  aired  the  programs  in  the  2000-2001  season.  Overall,  the  findings  were  extremely  positive.  Significant  among 
the  findings  were  (1)  73.5%  of  the  station  managers  said  that  the  quality  of  the  programs  in  the  series  is  "better  than  average" 
when  compared  to  other  educational  programming  their  station  airs;  (2)  90.5%  indicated  the  five  programs  in  the  2000-2001 
series  have  been  successful  in  educating  people  about  what  NASA  Langley  does;  (3)  64.8%  of  the  stations  airing  NASA's 
Destination  Tomorrow  reported  that  the  programs  have  been  "very  well  received"  by  their  audience;  and  (4)  about  80%  of 
those  television  station  managers  interviewed  stated  that  the  information  provided  by  NASA  Langley  in  the  five  programs  is 
"very  credible." 
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VI.  Future  Directions 


NASA's  KSNN  is  in  its  first  season  and  the  one-minute  newsbreaks  appear  to  be  on  target  for  programming  demand 
in  the  grade  3-5  category.  Spanish  versions  of  NASA's  KSNN  programs  and  web  site  are  complete  for  the  first  year 
and  12  more  are  planned.  Once  we  have  had  experience  developing  the  content,  we  will  open  up  the  newsbreaks  and 
web  content  to  a statewide  competition.  NASA  Science  Files  had  a very  successful  first  two  seasons  and  the  web 
page  will  continue  to  evolve  into  a more  useful  learning  center.  NASA  CONNECT  is  the  most  mature  of  the  programs 
and  will  continue  its  present  course  using  the  survey  results  to  improve  the  programs.  We  are  currently  streaming 
some  of  the  video  broadcasts  from  our  web  site.  In  the  future  a complete  year's  programs  (video  broadcasts,  lesson 
plans,  and  web  activities)  will  be  collected  on  one  DVD  for  dissemination.  NASA  LIVE  is  in  its  infancy  but  promises 
to  be  a very  useful  program  for  on-demand  video  conferencing  and  has  already  progressed  towards  including  the  K- 
12  audience  in  the  future.  The  current  mode  of  video  conferencing  is  directly  through  ISDN  connections.  In  the 
future  we  plan  to  move  to  web  conferencing  and  increase  the  number  of  available  topics.  Faculty  and  students  will 
assess  the  quality  of  the  content,  web-based  materials,  method  of  delivery,  and  the  presenter  by  using  an  online 
evaluation.  NASA's  Destination  Tomorrow  also  had  a very  successful  first  season  and  will  continue  along  the  same 
lines  to  produce  quality  programs  that  we  plan  to  market  to  commercial  TV.  The  5 -minute  segments  are  available  in 
Spanish  and  on  DVD.  For  all  the  programs,  we  will  continue  to  promote  dissemination  and  use  of  the  many  learning 
opportunities  for  all  ages.  It  is  clear  that  it  is  important  to  train  teachers  in  the  use  of  our  technology-based  activities 
in  general  and  to  promote  the  use  of  our  web  activities.  We  plan  to  address  the  need  for  training  in  using  technology 
because  of  the  concern  expressed  in  the  educational  community  [7,8].  We  will  develop  web-based  training,  including 
technology  training  for  teachers  so  that  they  can  to  use  our  program's  components  effectively  in  the  classroom. 
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Abstract:  Electronic  Classrooms  have  become  increasingly  common  over  the  past  decade.  The  combination  of 
data  projectors,  computers,  video  playback,  and  a range  of  ancillary  devices  have  been  found  useful  in  a variety 
of  instructional  formats,  resulting  in  pressures  for  more,  and  with  an  important  institutional  cost  impact. 
Simultaneously,  faculty  recognize  that  the  configuration  of 'smart'  rooms  has  a powerful  influence  on  the 
conduct  of  instruction.  A desire  to  facilitate  collaborative/constructivist  learning  in  such  spaces  has  led 
Dickinson  College  and  this  researcher  to  devise  new  models  for  physical  learning  spaces,  while  simultaneously 
trimming  the  costs  of  creating  them.  This  paper  presents  our  developing  'SmartS*  0rd  generation)  classroom 
designs,  including  seminar,  mid-sized,  and  larger  spaces.  Their  rationale,  underlying  research,  and  the 
technological  and  physical  features  of  these  spaces  are  presented. 

A synopsis:  The  published  version  of  this  paper  provides  a synopsis  of  the  visual  and  verbal  material  to  be 
presented  at  EdMedia  2002. 

Background:  From  their  origins  twenty  or  more  years  ago,  'smart'  classrooms  have  become  increasingly 
routine,  are  demanded  by  many  faculty,  and  used  quite  routinely  for  a range  of  tasks.  It  is  necessary  to 
create  them  expediently  and  economically,  yet  have  them  fulfill  all  routine  demands  made  by  those  using 
them.  This  researcher  and  Dickinson  College  have  created  processes  for  the  analysis  of  needs  and  the 
creation  of  class  spaces  that  are  much  streamlined  from  our  earliest  days.  This  report  and  the 
accompanying  visual  and  supporting  material  (to  be  found  on  line  at  http://www.dickinson.edu/~cavenagh) 
detail  our  progress  to  date. 

Dickinson's  existing  'smart'  classrooms  provide  a computer,  a data  projector,  and  a mixture  of  other 
components.  Some  rooms  add  more  computers  for  group  work,  others  add  enough  computers  for  every 
student.  These  exist  in  several  configurations.  We  also  retain  a few  cart -based  and  laptop  systems  for 
flexibility.  Altogether  we  currently  support  66  facilities  for  a faculty  of  about  200,  and  this  number  is 
growing.  Unfortunately,  the  distribution  of  such  facilities  is  very  uneven.  New  buildings  are  equipped  at  a 
95-100%  level,  while  older  structures  are  at  30-35%.  There  are  pressures  to  increase  support  in  these  older 
buildings,  which  actually  house  a very  large  number  of  classrooms.  These  buildings  are  predominantly 
used  for  humanities  and  social  sciences. 

A number  of  the  humanities  and  social  science  faculty  are  very  much  interested  in  discussion  based 
teaching  with  much  collaborative/constructivist  work,  while  lecturing  is  much  less  common  than  in  the 
past.  They  often  arrange  seating  in  circular  or  group  configurations  and  want  rooms  to  be  flexible. 
Conversely,  the  demands  of  projection  tend  to  force  all  learners  to  face  a single  screen.  Our  'smart'  rooms 
need  to  flex  to  meet  these  demands.  Finally,  groups  quite  often  give  presentations,  so  we  need  a means  of 
allowing  several  students  to  address  a class  at  once,  while  using  technology. 

Needs  analysis:  Dickinson  has  conducted  a formal  analysis  of  needs  using  several  instruments.  The  first, 
a facilities  utilization  study,  was  carried  out  by  our  Registrar  and  an  architectural  consultant.  Student  and 
faculty  surveys  yielded  detailed  insights  into  the  perceived  needs  of  these  groups.  Room-request  data  was 
analyzed  to  reveal  the  spatial  and  time  patterns  faculty  wished  to  meet,  and  in  turn  led  to  a renovation  of  the 
college's  weekly  calendar  to  better  utilize  slack  time  periods. 

The  Smart?  configuration:  Many  of  our  older  smart  classrooms  were  custom-tailored  to  meet  the 
demands  of  specific  departments.  Gradually,  we  recognized  that  their  inherent  variability  made  the  rooms 
difficult  for  others  to  use.  Our  policies  mandate  that  all  classrooms  be  available  for  all  courses,  so  some 
means  to  simplify  and  standardize  was  clearly  necessary.  This  resulted  in  a core  equipment  package  to  be 
used  in  all  new  construction,  and  to  be  provided  when  replacing  olderequipment.  As  this  represented  our 
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third  generation  of  classrooms,  we  labeled  this  core  package  Smart?.  The  package  includes  only  a 
computer  (Mac  or  PC),  a data  projector,  a modest  sound  system,  and  a combination  VCR/CD/DVD  player. 
Much  of  the  equipment  is  housed  in  a custom  lectern  (see  photos  on  web  site),  which  is  itself  designed  to 
be  used  either  by  a faculty  member  or  up  to  four  presenters. 

At  this  writing  these  components  cost  in  the  vicinity  of  $5500,  plus  installation,  plus  any  required 
modifications  to  the  classroom.  (Most  classrooms  are  already  network  equipped,  have  screens,  and 
adjustable  lighting.)  We  then  add  other  computers,  accessories,  and  furniture  as  necessary. 

The  use  of  a combination  VCR/CD/DVD  player  is  an  important  'simplifiei;'  as  it  requires  only  a single 
input  to  the  data  projector.  The  entire  operation  is  controlled  by  two  remotes,  one  for  the  projector  and  one 
for  the  player,  and  the  audio  package  has  a simple  integrated  remote  control. 

The  Smart?  configuration  represents  a breakthrough  in  user  convenience,  and  we  have  been  able  to  create 
custom  instructions  for  each  classroom  by  simply  cutting  and  pasting  and  adding  photos  of  the  rooms. 

Computer-per-user  classrooms  are  expensive  but  fairly  straightforward.  We  try  to  minimize  the  number 
of  these  facilities,  and  then  try  to  maximize  their  use.  We  have  explored  a number  of  layouts  (see  photos 
on  web  site),  and  our  most  recent  room  maximized  desktop  space  by  using  LCD  monitors.  (Surprisingly, 
this  cost  less  than  using  CRTs  because  of  savings  in  furniture  costs.)  The  costs  of  multiple  computer 
installations  cascade  in  sometimes  surprising  ways,  such  as  increased  cooling  costs,  needs  for  better  sound 
control,  and  difficult  demands  on  lighting  systems.  The  control  of 'sight  lines'  in  such  facilities  takes 
particular  care  and  small  variations  from  good  design  practice  can  reduce  the  effectiveness  of  a room  quite 
significantly.  The  SmartJ  core  components  provide  an  instructor's  station. 

A fascinating  challenge;  Groupwork  classrooms  present  our  single  most  interesting  challenge  and 
potentially  our  greatest  pedagogical  innovation.  Making  classrooms  flexible  for  different  seating 
configurations  restricts  both  furniture  and  technology  choices.  At  first  glance  the  use  of  laptops  seems  to 
solve  all  problems,  but  our  analysis  demonstrates  that  two  learners  often  work  badly  at  a single  laptop  and 
three  or  more  face  such  problems  of  visibility  as  to  render  effective  group  interaction  virtually  impossible. 
Further,  moving  laptops  around  within  a class  is  time  consuming  and  places  the  computers  at  risk. 

This  writer  has  engaged  in  analysis  of  group  performances  when  using  technology,  and  this  work  has 
resulted  in  the  creation  of  a range  of  group  workstations  with  curved  fronts  that  provide  eye-to-eye  contact 
among  group  members,  while  a large  monitor  insures  good  visibility.  Distributed  around  our  campus,  such 
stations  have  repeatedly  proved  their  worth.  (This  work  has  been  reported  previously  at  EdMedia  2000, 
Educause  2001,  and  Educause/Baltimore  2002;  see  web  site  for  references.)  Using  this  experience  as  a 
base,  we  have  redesigned  these  workstations  to  be  wall-mounted  in  existing  classrooms. 

Requiring  only  1 0 inches  of  classroom  depth  and  having  folding  desktops,  4-6  such  units  can  be  distributed 
around  the  periphery  of  a classroom  without  consuming  too  much  space.  Combined  with  movable  tables 
and  chairs  in  a classroom,  an  instructor  can  change  room  configurations  without  prior  planning,  and  so  can 
adapt  teaching  modes  to  'the  teachable  moment.' 

Dickinson's  SmartJ  model  for  classroom  evolution  seeks  simplification,  reduction  of  costs,  and 
increased  flexibility.  Simplification  results  in  additional  hidden  savings  such  as  reduced  training  time  for 
users,  reduced  design  time,  and  reduced  construction  time.  Increased  flexibility  of  rooms  means  that 
faculty  have  greater  range  for  their  teaching  styles,  and  like  the  facilities  better.  It  also  means  that  more 
faculty  can  use  rooms  than  was  the  case  when  we  custom-designed  every  facility.  Finally,  our  use  of 
analytic  instruments  means  we  have  a much  improved  understanding  of  real  needs,  and  can  concentrate  our 
development  efforts  on  rooms  where  they  are  most  needed. 

The  particulars  shown  here  meet  the  needs  of  one  campus,  but  we  feel  that  the  underlying  principles  we 
have  followed  have  application  at  any  institution. 

(References,  images,  and  supporting  information  to  be  found  at  http://www.dickinson.edu/^cavenagh.) 
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Introduction 

The  growing  popularity  of  the  World  Wide  Web  (WWW)  and  the  competition  in  electronic  commerce  has  created  a need  for  online 
product  catalogs.  The  conventional  display  approach  arranges  data  in  multiple  pages  and  sections;  users  then  traverse  a hierarchical  tree  structure  to 
discover  the  range  of  products  or  services  that  the  site  offers.  Often,  users  have  to  navigate  within  a complex  world  of  classifications  and  descriptions, 
making  arbitrary  decisions  without  the  slightest  idea  what,  if  anything,  lie  at  the  end  of  the  path.  With  each  choice,  users  are  shut  off  to  perfectly 
viable  options.  Furthermore,  the  experience  of  traversing  multiple  paths  and  viewing  many  pages  may  draw  users  away  from  the  context  of  the  entire 
product  space  causing  disorientation  and  distraction.  In  fact,  after  security  concerns,  navigation  problems  are  the  second  most  cited  reason  for  not 
shopping  online  and  contribute  a significant  aspect  to  the  users’  negative  web  experience  [1].  We  argue  that  it  is  important  to  create  interfaces  that 
support  interactive  product  browsing  on  the  Web,  and  promote  enjoyable  and  productive  user  experiences  as  ©-commerce  sites  continue  to  flourish.  In 
this  paper,  we  describe  the  Glass  Engine,  an  innovative  single-paged  interactive  visualization  developed  at  the  IBM  T J Watson  Research  Center  in 
conjunction  with  the  composer,  Philip  Glass.  This  multi-attribute  electronic  music  catalog  invites  users  to  view,  browse,  and  manipulate  a large 
repository  of  items  with  common  attributes  according  to  their  interests  and  affinities  and  at  their  own  pace. 

Related  work 

For  the  past  two  decades,  research  has  explored  the  application  of  interactive  graphics  and  visualization  techniques  which  facilitate  the 
ready  comprehension  of  large  information  sets.  Some  of  the  new  visualization  techniques  include:  Tree  Maps  (Johnson  & Shneidermarm,  1991)  [2], 
the  Perspective  Wall  (Robertson,  Mackinlay,  & Card,  1991)  [3],  Cone  Trees  (Robertson,  Mackinlay,  & Card,  1991)  [4],  Fisheye  Views  (Furnas, 
1981,  1986)  [5,  6],  and  Hyperbolic  Browser  (Lamping  & Rao,  1995  [7];  Munzner  et  al,  1997)  [8].  A number  of  websites  also  specifically  use  a 
variety  of  visualization  techniques  to  present  large  amounts  of  information  online,  e.g.,  “Map  of  the  Market”  at  Smartmoney.com  [9]  and  the  Sony 
Music  web  site  [10].  Using  a rectangular  grid  of  boxes,  “Map  of  the  Market”  offers  a quick  view  of  over  600  stocks.  Nevertheless,  by  using  two 
dimensions  to  represent  a single  variable,  it  does  not  provide  an  optimal  depiction  of  multidimensional  data.  The  Sony  Music  site  provides  the  user 
with  a Thinkmap  displaying  the  links  among  the  themes  of  songs,  which  are  represented  by  circles.  However,  users  are  not  allowed  to  download  or 
sample  the  music. 

Design  rationale 

Our  project  builds  on  Inselberg's  work  on  the  parallel  coordinate  visualization  technique  [11]  and  Ahlberg,  Williamson,  and  Shneiderman's 
research  on  the  Alphaslider  selection  tool  [12].  In  parallel  coordinates  plots,  a vector  intersects  multiple  parallel  axes  to  define  a point  in  n- 
dimensional  space,  and  multiple  vectors  representing  an  array  of  data  points  can  reveal  interesting  patterns  in  the  data.  One  of  the  goals  for  our  multi- 
attribute online  catalog  is  to  address  the  problem  of  multiple  look-ups.  We  decided  to  adopt  a single-page  arrangement  that  displays  the  entire  musical 
work  of  Philip  Glass  with  all  attributes  sorted  independently . Equipped  with  a flexible  overview  of  all  his  works,  users  can  navigate  the  entire  product 
space  without  following  a pre-set  sequence  of  choices.  By  selecting  the  desired  attributes  and  their  values,  users  can  restrict  the  display  to  the  relevant 
data  set. 


While  we  cannot  anticipate  the  needs  or  intent  of  every'  user,  we  strive  to  present  a dy  namic  multi- attribute  online  catalog  witli  an  open 
structure  that  encourages  discovery- based  browsing.  Thus,  users  are  engaged  to  freely  browse,  explore,  and  manipulate  the  product  space  on  their  own 
term  and  at  their  own  pace. 


Key  features 


The  display  is  dvided  into  two  halves  (Figure  1):  the  top  half  is  composed  of  3 sliders,  each  representing  a single  characteristic  of  the 
whole  musical  work,  while  the  bottom  half  incorporates  5 additional  sliders  representing  the  corresponding  track  numbers  of  the  works  and  the 
attributes  pertaining  to  the  tracks  of  the  works 


Figure  1:  The  Glass  Engine 


The  sliders  on  the  top  half  of  the  display  are: 
Title  - arranged  alphabetically 
Date  - date  of  copyright 
Length  - in  hours,  minutes,  and  seconds 


The  sliders  on  the  bottom  half  of  the  display  are: 

Joy  - emotional  effect  from  low  to  high 

Sorrow  - emotional  effect  from  low  to  high 
Intensity  - emotional  force  from  low  to  high 

Density  - complexity  of  orchestration  from  low  to  high 
Velocity  - perceived  tempo  of  the  track/work 


The  central  interactive  elements  of  the  Glass  Engine  are  sliders,  each  of  which  can  be  considered  a single  dimension  of  the  data  (as  in 
parallel  coordinates);  and  Philip  Glass’s  entire  works  are  plotted  along  each  dimension.  For  instance,  the  work  "Einstein  on  the  Beach"  is  plotted  near 
other  works  starting  with  D-E-F  on  the  Title  slider  and  near  works  composed  in  1 976  on  the  Date  slider.  To  navigate,  a user  moves  a slider  so  that  a 
desired  work  lines  up  with  the  crosshair,  and  music  from  the  work  begins  playing.  Simultaneously,  the  other  sliders  move  automatically  so  that  the 
selected  work  in  each  dimension  also  lines  up  with  the  crosshair,  thus  indicating  the  position  of  the  selected  work  on  thees  other  dimensions. 
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Furthermore,  isers  can  fine-tune  their  navigation  (such  as  accessing  the  previous  work  or  the  next  work)  by  clicking  on  the  back/next 
buttons  above  the  slider.  Users  find  this  particular  function  practical  and  necessary  when  the  plotted  works  are  dense.  The  Glass  Engine  allows  users 
to  perform  two  kinds  of  filtering:  linear  and  categorical.  Linear  filtering  is  accomplished  by  pulling  tabs  from  the  endpoints  of  the  sliders.  For 
instance,  one  might  move  the  endpoint  “2001“  on  the  Date  slider  to  “1990,“  effectively  hiding  all  works  written  after  1990.  Categorical  filtering 
(Operas,  Symphonies,  etc.)  is  accomplished  through  selecting  View:  all  work  type  from  the  toolbar;  users  can  then  select  their  desired  category  from  a 
pull-down  menu. 

In  addition,  the  Glass  Engine  allows  users  to  navigate  all  of  Glass’s  works  by  his  music  characteristics,  (e.g.,  joy  or  tempo).  In  the  bottom 
part  of  the  display,  users  can  navigate  along  the  tracks  that  are,  for  example,  happy  or  slow  or  densely  orchestrated.  In  addition,  using  the  endpoint 
filters,  it  is  possible  to  navigate  solely  through  the  tracks  that  satisfy  multiple  criteria,  e.g.,  happy,  slow,  and  densely  orchestrated  (i.e.,  navigate  or 
browse  the  entire  product  space  with  multiple  criteria  specified.)  Bookmarking  is  another  well-received  feature  of  the  Glass  Engine,  By  selecting 
Favorites  from  the  toolbar,  users  can  bookmark  works  or  tracks  of  interest  to  return  to  them  conveniently.  The  Glass  Engine  also  facilitates 
comparisons  among  works  and  tracks.  Since  the  works  are  displayed  as  patterns  of  sliders,  users  can  compare  the  values  of  musical  attributes  by 
observing  screen  changes  in  slider  patterns.  In  short,  browsing  and  interactive  navigation  with  sliders  simplify  free  exploration;  if  a selected  piece  is 
not  what  the  user  expected,  the  operation  is  reversible  by  just  sliding  the  bars  in  the  opposite  direction.  Finally,  the  Glass  Engine  allows  novice  users 
to  begin  browsing  with  little  training,  while  providing  expert  users  with  powerful  features  such  as  the  ability  to  perform  multip  lO’Criteria  searches. 
Implementation 

The  Glass  Engine  interface  is  a Java  applet.  Since  Java  applets  are  often  large,  slow  to  download,  browser- dependent,  or  require  additional 
plug-ins,  we  restricted  the  applet  to  what  is  supported  in  jdkl.1.8  which  is  a version  commonly  supported  by  most  browsers.  The  applet  can  be 
downloaded  quickly  and  requires  no  additional  plug-ins  to  run.  All  catalog  data  is  downloaded  with  the  applet.  The  music  is  hosted  at  a different  site 
and  is  streamed  on  demand  when  a work  is  selected.  Javascript  in  the  Glass  Engine  HTML  page  serves  as  glue  between  the  applet  and  whatever 
media  player  is  installed  on  the  client  machine. 

Ongoing  usability  study 

A usability  study  comparing  users’  performance  and  experience  with  the  Glass  Engine  and  a hierarchical  interface  was  conducted  at  IBM 
Research  in  August  2001.  Amazon.com  was  chosen  as  the  control  interface  because  of  its  hierarchical  structure  and  diversity  of  products  offered. 
Sixty-one  subjects  participated  in  the  study.  Three  dependent  variables  were  measured:  knowledjge  acquisition,  music  identification,  and  user 
satisfaction.  Results  of  the  usability  study  showed  that  the  Glass  Engine  (GE)  group  achieved  significantly  higher  overall  accuracy  than  the 
hierarchical  interface  (HI)  group  in  both  the  knowledge  acquisition  and  music  identification  tasks  (t(59)— 5.076,  p =.001).  With  respect  to  user 
experience,  the  GE  group  found  it  easy  to  find  the  music  using  the  Glass  Engine  and  gave  significantly  higher  satisfactory  ratings  than  the  HI  group 
on  their  experiences  with  the  interface  (t(59)=^5.294,  p=.000).  Finally,  74%  of  the  participants  in  the  GE  group  would  like  to  learn  more  about  Philip 
Glass  as  opposed  to  60%  of  the  subjects  in  the  HI  group  who  would  like  to  do  so. 

Conclusion  and  Future  Direction  p 

In  this  paper,  we  describe  the  Glass  Engine,  a novel  single-paged,  multi-attribute  electronic  catalog,  which  presents  users  a personalized 
navigation  tool  to  view,  browse,  and  manipulate  a large  repository  of  items  or  products  with  common  attributes.  By  including  as  many  possible 
aspects  of  the  composer's  music,  the  users’  navigation  through  his  works  may  be  guided  by  their  personal  interests,  associations,  and  affinities. 
Empirical  studies  in  progress  point  to  a possible  correlation  between  the  free -form  navigation  made  possible  with  tools  like  the  Glass  Engine,  and 
learning  and  retention.  Another  set  of  empirical  studies  comparing  users’  performance  and  experience  with  the  Glass  Engine  and  other  innovative 
visualization  systems  is  underway  at  IBM  Research.  The  final  analysis  of  the  study  will  hopefully  shed  light  on  the  research  question:  Can  advanced 
user  interfaces  be  successfully  employed  to  give  users  a new  and  comprehensive  view  of  large  product  spaces  on  the  Web? 
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Abstract:  A Web-Based  Learning  Portfolio  (WBLP)  was  evaluated  through  practical  teaching  proc- 
ess to  understand  if  the  WBLP  system  helps  students  to  grasp  the  learning  process  and  enhances 
learning  outcomes.  The  evaluation  results  reveal  that  this  WBLP  system  has  been  more  useful  for 
them  to  obtain  the  feedback  from  other  students  than  from  their  teachers.  It  reflects  the  fact  that 
the  feedback  from  other  students  has  been  helpful  for  them  and  has  thus  become  the  necessary  com- 
ponent to  help  them  in  learning.  Next  step,  we  will  conduct  the  further  empirical  study  in  terms  of 
learning  effects  for  the  system. 


Research  Background 

Portfolio  has  been  used  to  gather  the  learning  activities  of  students  in  various  aspects  as  well  as  the  works  of  students  with  one 
purpose  in  mind.  It  allows  the  teachers,  students,  or  their  parents  to  understand  and  evaluate  the  learning  process,  improvement  situa- 
tion, and  academic  achievement  of  students.  It  may  also  be  used  as  evidence  for  the  pupils  to  reflect  on  their  learning  and  the  changes 
in  their  comprehension  during  the  study  process.  Portfolio  may  provide  the  following  benefits  (Hewitt,  1995): 

1.  It  may  demonstrate  students'  growth  and  improvement  situation. 

2.  It  may  encourage  students  to  set  up  learning  goals. 

3.  It  may  provide  the  hard  evidence  concerning  students'  efforts. 

4.  It  may  show  students'  performance  or  works. 

5.  It  may  serve  the  purpose  for  job  application  or  school  application. 

6.  It  may  help  the  faculty  to  review  students'  learning  progress. 

7.  It  may  serve  to  inspire  the  teacher  and  understand  students’  performance. 

8.  It  stimulates  students'  introspective  thinking  and  enhances  self-assessment  (Smith  & Tillema,  1998;  Wade  & Yarbrough,  1996;  Car- 
roll,  Potthoff  & Huber,  1996;  Vavrus,  1990). 

9.  It  encourages  students'  learning  interest,  builds  up  students'  self-confidence,  and  assists  students  to  know  more  about  themselves,  and 
develops  writing  skill  as  well. 

10.  It  encourages  students'  participation  in  cooperation,  and  increases  their  self-esteem  (Mullin,  1998). 

Portfolio  method  emphasizes  that  a learner  participates  actively  and  thus  establishes  the  learning  results  of  his/her  own  concern. 
Therefore,  its  development  process  whether  it  be  instructed  by  teachers  or  established  by  students’  own  accord,  shall  both  attain  the 
goal  of  encouraging  the  learners  to  be  responsible  for  his/her  own  learning  and  to  grasp  all  the  goals  of  learning  activities.  Furthermore, 
portfolio  assessment  was  originally  designed  as  an  authentic  assessment  approach  meant  for  the  improvement  of  the  traditional  pencil- 
paper  tests.  Thus,  the  assessment  is  real  and  active.  In  contrast  with  the  traditional  way  of  assessment,  portfolio  assessment  pays  at- 
tention not  only  to  the  results  but  also  the  processes  involved.  Since  learning,  teaching,  and  evaluation  comprise  of  the  integrated 
activities  that  shall  go  hand  in  hand,  hence  portfolio  approach  is  actually  a better  way  to  grasp  the  complexity  of  learning  processes.  It 
also  allows  the  learners  or  teachers’  introspection  to  take  place.  It  may  also  help  to  reflect  on  the  overall  learning  and  teaching  proc- 
ess, so  as  to  give  feedback  and  offer  suggestions,  thus  to  benefit  the  learning  in  the  genuine  way.  In  general,  portfolio  possesses  the 
following  characteristics  or  features  (Wade  & Yarbrough,  1996): 

1.  Developmental:  portfolio  represents  a certain  period  of  students’  growth  and  learning.  Portfolio  is  the  long-term  accumulated  learn- 
ing results,  not  meant  for  short-term  goals.  Hence,  it  is  actually  an  on-going  process  in  its  development. 

2.  Dual -valued:  portfolio  offers  both  the  teachers  and  students  the  value  of  two-way  interaction.  It  allows  a learner  the  opportunity  to 
reflect  and  record  his/her  own  learning  process.  It  also  offers  teachers  a good  approach  or  method  to  evaluate  students’  growth  and 
achievements. 

3.  Selective:  portfolio  offers  students  the  opportunity  to  choose,  so  that  students  may  self-determine  what  kind  of  contents  that  they 
want  to  put  in  the  portfolio  and  how  to  organize  the  whole  portfolio.  It  also  allows  students  to  set  up  the  standard  or  basis  for 
evaluation/  assessment. 

4.  Authentic:  portfolio  incorporates  the  genuine  works  or  performances  of  students.  Traditional  tests  normally  cannot  reflect  a student 
growth  or  potential  in  all  aspects,  but  portfolio  may  demonstrate  solid  learning  results  and  thus  offers  an  authentic  way  of  learning. 

5.  Reflective:  portfolio  may  reveal  the  evidence  of  self-reflection.  Thus  learners  may  review  his/her  own  work,  then  to  set  up  further 
goals  through  such  a reflection.  It  may  also  help  to  review  the  previous  efforts,  and  compare  them  with  the  actions  afterward,  for 
better  understanding  of  his/her  improvements  or  growth. 

6.  Individual:  portfolio  is  a personal  learning  and  growing  record  based  on  individual  ways  of  choices  and  organization.  Consequently,  it 
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reflects  personal  contents  and  style  in  students’  personal  portfolio. 

7.  Interactive:  a learner  may  share  with  his/her  teachers  and  peers  through  personal  portfolio,  and  thus  seeking  guidance  and  sugges- 
tions. In  this  way  the  development  and  establishment  of  portfolio  may  be  regarded  as  the  process  of  cooperative  interactions. 

Other  than  that,  portfolio  also  possesses  the  characteristics  of  self-regulated  learning  and  self-directed  learning  (Fischer  & King, 
1995;  Smith  & Tillema,  1998),  this  is  due  to  the  fact  that  a portfolio  allows  the  learner  to  self-determine  his/her  learning  goals, 
strategies,  and  contents  and  then  records  them.  Thus  the  role  of  scholars  is  merely  to  assist  and  provide  feedbacks.  That  is  why  it  would 
encourage  a learner  to  become  an  independent,  self-directed  person  through  the  development  of  personal  portfolio. 

The  Web-Based  Learning  Portfolio  (WBLP)  that  has  been  structured  through  web  interface  and  also  has  gone  through  the  dif- 
ferent stages  of  analysis,  design,  development,  and  test  is  now  being  established.  The  effects  of  the  WBLP  in  students’  learning  are  the 
issue  worthy  of  our  analysis  and  investigation.  With  the  above  pre-requisite  in  mind,  this  research  hopes  to  achieve  the  following  goals: 

1 . To  assess  the  perceived  effects  of  the  WBLP,  and  analyze  its  impact  on  students’  learning  process. 

2.  To  propose  the  suggestions  for  future  research  direction  based  on  the  results  of  evaluation. 


Web-Based  Learning  Portfolio 

This  WBLP  is  mainly  used  to  provide  students  with  a web-based  user  environment.  It  would  help  students  to  complete  their  in- 
dividual portfolios  through  simple,  easy-to-use  interface  for  necessary  guidance.  It  also  allows  the  teachers  and  students  to  browse  their 
classmates’  portfolios,  and  to  give  their  feedback  or  assessment  opinions.  The  contents  of  this  WBLP  include  the  following  seven 
major  items:  the  basic  information  of  students,  the  learning  goals  of  students,  the  works  of  students  in  a course,  the  records  of  students’ 
self- re  fleet  ion  or  self-assessment,  records  of  teachers’  feedback  and  assessment,  records  of  peer  feedback  and  assessment,  personal  web 
page  of  students.  In  addition  to  the  afore-mentioned,  in  order  to  support  students  to  create  and  browse  the  portfolios,  as  well  as  to 
facilitate  and  meet  the  needs  for  teachers  to  inspect  and  evaluate  students’  portfolios,  the  functional  areas  of  WBLP  system  consists  of 
the  following  eight  major  items:  Portfolio  Browse,  Portfolio  Creation,  Portfolio  Guide,  Portfolio  Discussion  Board,  Portfolio  Bulletin 
Board,  Portfolio  Suggestion  Board,  Student  Data  Maintenance,  System  Management  (Chang,  2000;  Chang,  2001).  These  functional 
areas  mainly  provide  the  following  functions  to  the  portfolio  (as  shown  on  Figure  1): 

1 . The  function  for  students  to  conduct  the  various  productions  of  portfolio  contents. 

2.  The  function  for  teachers  and  students  to  browse  the  portfolios  of  other  students  or  peers. 

3.  The  feedback  and  assessments  of  teachers  as  well  as  the  feedback  and  mutual  assessment  mechanism  among  students  themselves. 

4.  The  on-line  guidance  and  illustration  for  students  to  create  portfolio  contents. 

5.  The  searching  and  management  function  of  students’  personal  information. 

6.  The  asynchronous  discussion  channel  of  portfolio  creation  process. 

7.  The  opinion  feedback  channel  for  students  in  using  the  system. 

8.  The  updated  system  announcement  or  course  information  concerned. 

9.  The  function  for  teachers  to  manage  student  information  and  announcement  information. 


Design  of  the  Research 

We  use  an  established  Web-Based  Learning  Portfolio  (WBLP)  on  an  undergraduate  course  in  university.  Through  practical  teach- 
ing process,  we  may  evaluate  if  the  WBLP  helps  students  to  grasp  the  learning  process,  if  it  enhances  learning  outcomes.  The  evalua- 
tion consists  of  user  evaluation,  user  in-depth  interviews,  and  expert  evaluation.  The  evaluation  also  helps  to  understand  the  difficulties 
and  problems  of  this  system  in  its  application  and  its  possible  impact  on  students’  learning. 

The  user  evaluation  method  had  been  conducted  one  and  half  month  after  the  WBLP  system  implementation.  It  was  carried  out 
through  the  evaluation  questionnaire  designed  by  the  researcher.  The  questionnaire  was  distributed  in  several  copies  to  a class  of  35 
students  who  were  taking  the  course  of  “Computer  and  Instruction”  in  a Pre-service  Teacher  Education  Program.  This  research  has 
designed  its  own  questionnaire  for  evaluation  (as  shown  on  Table  3).  The  questionnaire  primarily  asked  the  questions  of  the  WBLP 
impact  on  the  learning  process.  The  contents  of  this  questionnaire  are  based  on  relevant  literatures  and  discussions  of  scholars  and 
experts.  They  have  gone  through  the  process  of  examination  for  validity  and  reliability.  The  questions  have  been  presented  based  on 
Likert’s  five-point  scaling  method.  That  means  the  choice  for  answer  is:  strongly  agree,  agree,  average,  disagree,  and  strongly  disagree. 

In  order  to  understand  the  WBLP  impacts  on  students’  learning  in  depth,  so  we  supplement  the  study  through  the  researcher 
self-designed  “Questions  Listing  of  User  In-Depth  Interviews”  after  the  completion  of  user  evaluation  questionnaire  analysis.  Inter- 
views in-depth  would  then  follow  up  to  get  to  know  the  opinions  and  suggestions  of  the  users  concerning  the  uiresolved  answers  and 
controversial  issues  that  could  never  be  revealed  through  the  previous  questionnaire.  Targeting  for  the  unresolved  questions  and  the 
controversial  issues  of  the  previous  questionnaire  of  user  evaluation,  in-depth  interviews  were  then  conducted  on  5 students  selected 
randomly  after  the  completion  of  questionnaire  analysis  work  in  order  to  deeply  understand  the  users’  opinions  and  suggestions. 

Three  experts  were  being  invited  to  conduct  the  expert  evaluation  (including  the  course  instructor,  a portfolio  assessment 
expert,  and  a web  technology  expert)  to  use  this  WBLP  on-line.  The  evaluation  work  was  then  conducted  through  interview  method 
after  one  week.  In  terms  of  expert  evaluation,  interviews  were  conducted  based  on  the  researcher’s  self-designed  “ Questions  Listing  of 
Expert  Interviews”.  In  this  way  that  we  got  to  understand  the  expert  opinions  in  regarding  the  WBLP  impact  on  learning  and  teaching. 
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Validity  and  Reliability  of  Questionnaire 

Validity  refers  to  whether  a tool  for  measurement/survey  that  may  achieve  its  intended  functions  or  purpose.  The  questionnaire 
of  user  evaluation  in  this  research  has  gone  through  the  processes  of  pre-test  and  pilot-study  to  verify  the  validity  of  the  questionnaire. 

For  attaining  the  goal  of  questionnaire  validity,  it  was  necessary  to  consult  two  expert/scholars  of  portfolio  assessment  and  web 
technology,  and  one  teacher  who  instructed  the  course  after  the  completion  of  the  questionnaires  for  their  opinions  on  revising  the 
framework  of  said  contents,  words  and  expressions  relating  to  the  issues,  etc.  The  suggestions  for  revising  the  questionnaire  include: 
increasing  the  contents  of  the  questionnaire,  more  articulate  in  the  expressions  of  questions,  word  polishing,  etc.  Then  could  the  con- 
tents of  the  questionnaire  be  able  to  attain  the  goal  originally  intended  for  the  effectiveness  in  its  measurement/survey.  Through  the 
implementation  of  such  a method,  this  research  meets  the  needs  of  measurement/survey  tool  efficiency.  The  above  process  involved  is 
the  so-called  expert  validity. 

We  may  achieve  more  complete  validity  verification  through  the  further  process  of  pilot-study.  We  picked  up  5 students  ran- 
domly for  the  purpose  of  this  research  at  that  time  to  conduct  pilot-study  based  on  the  rough  draft  of  the  questionnaire.  We  hoped  to 
go  through  such  a pilot-test  and  render  the  results  to  the  researcher  to  get  to  know  if  the  questionnaire  could  effectively  meas- 
ure/survey the  purpose  intended  for  this  research,  and  to  revise  any  inappropriateness  whatever  necessary.  The  opinions  or  feedback 
for  revision  include  the  clarification  of  some  questions  and  word  polishing,  etc.,  so  as  to  present  the  questions  in  the  way  better  inder- 
stood  by  students  who  were  answering  the  questions  in  the  questionnaire. 

Reliability  refers  to  being  consistent  and  steady  in  terms  of  measurement/survey  method.  In  this  way  we  could  understand  the 
extent  or  degree  of  reliability  for  measurement/survey  instruments.  In  order  to  attain  the  reliability  goal  of  this  questionnaire,  we  con- 
ducted the  reliability  test  of  Cronbach’s  alpha  coefficient  by  using  the  contents  of  the  above  retrieval  questionnaires.  The  extent  or 
degree  of  consistency  within  the  questionnaires  could  thus  be  determined.  Then  we  could  determine  if  the  research  measurement  tool 
possessed  certain  degree  of  reliability.  The  Cronbach’s  alpha  coefficient  of  formal  questionnaire  after  revision  is  equal  to  0,8346  (n  = 
35).  This  shows  that  the  reliability  level  of  this  research  has  attained  a rather  reliable  level. 


Results  and  Discussions 
User  Evaluation 

User  evaluation  consists  of  survey  and  interviews.  Tables  I list  the  statistics  results  of  each  evaluation  item  within  the  question- 
naire. Among  the  25  evaluation  items  of  Table  I,  other  than  the  later  debut  of  the  Discussion  Board  (came  out  only  about  one  week 
prior  to  the  evaluation)  that  has  led  to  the  insufficient  understanding  and  use  of  its  function,  and  thus  resulted  in  the  mean  of  this  two 
items  both  lower  than  4 (3.83  & 3.87  respectively)  for  Item  16  --  the  articles  of  Portfolio  Discussion  Board  enhance  my  growth  in 
learning  and  Item  17  --  the  feedback  on  the  Portfolio  Suggestion  Board  have  been  helpful  for  me  to  overcome  the  difficulty  in  produc- 
ing personal  portfolio.  As  for  the  other  23  items  of  survey,  the  degree  of  agreeability  mean  is  higher  than  4,  and  most  users  are  either 
agree  with  the  statement  or  very  much  agree  with  it.  The  means  of  6 items  (Item,  12,  18,  19,  20,  24,  25)  are  even  higher  than  4,5, 
and  90%  of  the  users  regard  them  very  helpful  for  their  learning. 

From  the  survey  results,  it  appears  to  be  that  most  of  the  users  are  rather  agreeable  that  this  system  has  been  helpful  for  their 
learning  process  or  learning  outcomes,  and  thus  have  come  out  with  positive  appraisal  for  this  system.  As  for  the  questions  in  regarding 
to  whether  the  browsing  of  other  students’  portfolio,  learning  goals,  works,  basic  information,  personal  web  page,  etc.  may  help  them 
in  some  way  (from  Item  6 to  Item  14),  most  students  regard  that  it  has  been  most  helpful  for  them  to  browse  other  students’  works. 
The  mean  for  this  question  is  as  high  as  4.53.  As  high  as  93%  of  the  students  regard  that  the  quality  of  their  own  work  can  be  improved 
by  such  an  emulation  through  the  browsing  of  others’  works  (Item  12).  94%  of  the  users  regard  that  it  helps  to  understand  the  merits 
and  shortcomings  of  other  students’  learning  by  browsing  other  students’  portfolio  (Item  7).  97%  of  the  users  regard  that  it  enhances 
their  own  academic  growth  and  improvement  by  browsing  other  students’  portfolios  (Item  8).  All  these  survey  results  have  revealed  the 
fact  that  the  use  of  portfolio  has  really  helped  the  users  in  learning. 


Table  1:  Percentages  and  means  of  student  agreement  in  WBLP  impacts  on  learning  process 


Impacts  on  Learning  Process 

5 

Percentages  of  Agreement  (%) 
4 3 2 1 

Means 

I Set  up  learning  goals  may  help  me  with  the  direction  in  self-learning 

27 

53 

20 

0 

0 

4.07 

2 The  uploading  and  gathering  of  course  works  helps  to  reveal  my  genuine  learning 
outcomes 

37 

60 

3 

0 

0 

4.33 

3 The  writing  process  for  self-reflection  and  assessment  records  help  me  to  grasp  and 
reflect  on  genuine  learning  process 

37 

53 

10 

0 

0 

4.27 

4 Teacher  feedback  helps  me  to  reflect  on  my  merits  and  shortcomings  in  learning 

53 

40 

4 

3 

0 

4.43 

5 The  feedback  from  peers  help  me  to  reflect  on  my  merits  and  shortcomings  in 
learning 

47 

43 

7 

3 

0 

4.40 

6 Browsing  classmates’  portfolios  helps  me  to  reflect  on  the  merits  and  shortcomings 
in  my  learning 

47 

43 

7 

3 

0 

4.33 
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7 Browsing  classmates’  portfolios  helps  me  to  understand  the  merits  and  shortcom- 
ings of  my  classmates  in  learning 

8 Browsing  my  classmates’  portfolios  helps  me  to  grow  and  improve  in  academic 
achievements 

9 Browsing  my  classmates’  portfolios  helps  to  motivate  me  for  the  learning  of  this 
course. 

10  Browsing  my  classmates’  portfolios  helps  me  to  increase  the  communication  and 
exchanges  with  my  classmates 

1 1 Browsing  classmates’  learning  goals  helps  me  to  understand  classmates’  work 
effort,  and  thus  enhances  that  of  my  own 

12  Browsing  my  classmates’  works  helps  to  upgrade  the  quality  of  my  works 

13  Browsing  my  classmates’  basic  information  helps  me  to  understand  my  classmates 
better 

14  Browsing  my  classmates’  personal  web  page  helps  me  to  better  inderstand  my 
classmates  and  thus  enhances  relationships  with  them 

15  The  information  in  Portfolio  Bulletin  Board  helps  me  to  better  understand  this 
course 

16  The  articles  of  Portfolio  Discussion  Board  help  my  academic  growth 

17  The  message  feedback  in  Portfolio  Suggestion  Board  helps  me  to  solve  the  prob- 
lems that  I have  encountered  in  portfolio  creation 

18  It  helps  me  to  know  about  the  learning  outcomes  of  my  classmates  by  using  this 
system 

19  It  helps  me  to  know  about  my  teacher  feedback  and  suggestions  by  using  this 
system 

20  It  helps  me  to  know  about  my  classmate  feedback  and  suggestions  ly  using  this 
system 

21  It  enhances  my  interactions  and  exchanges  with  my  classmates  by  ifiing  this  sys- 
tem 

22  It  helps  me  to  better  understand  my  growth  and  improvements  in  the  course  by 
using  this  system 

23  It  helps  me  to  better  understand  the  growth  and  improvements  of  other  class- 
mates in  this  course  by  using  this  system 

24  It  helps  me  to  learn  this  course  by  using  this  system 

25  I hope  to  use  this  system  in  other  courses  as  well 

Total  Means  


47 

47 

3 

38 

59 

0 

43 

50 

7 

37 

43 

17 

57 

33 

7 

63; 

30 

4 

50 

33 

14 

40 

40 

13 

57 

33 

7 

17 

57 

20 

30 

43 

10 

67 

33 

0 

63 

27 

10 

60 

37 

3 

40 

47 

7 

44 

43 

13 

43 

40 

14 

70 

27 

3 

60 

33 

7 

46.96 

41.88 

8.4 

3 0 4.37 

3 0 4.31 

0 0 4.37 

3 0 4.13 

3 0 4.43 

3 0 Esi 

3 0 4.30 

7 0 4.13 

3 0 4.43 

6 0 3.83 

17  0 3.87 

0 0 

0 0 

0 0 

6 0 4.20 

0 0 4.31 

3 0 4.23 

0 0 S 

0 0 

2.76  0 4.33 


The  most  positive  feedback  comes  out  in  terms  of  the  overall  benefit  in  using  this  system  (from  Item  18  to  Item  24),  most  of 
the  students  agree  that  by  using  this  system,  it  helps  them  to  better  understand  other  students’  learning  achievements  (mean  = 4.67, 
Item  18),  allowing  them  to  obtain  the  feedback  and  suggestions  of  other  students  (mean  = 4.57,  Item  20),  and  benefit  their  learning  of 
the  course  from  the  portfolio  (mean  = 4.67,  Item  24).  100%  of  the  users  regard  it  useful  to  utilize  the  portfolio  to  get  to  know  the 
learning  achievements  of  other  students  (Item  18).  97%  of  the  users  regard  it  helpful  to  obtain  the  feedback  and  suggestions  of  other 
students  through  this  system  (Item  20).  As  high  as  97%  of  the  users  expressed  that  it  helps  them  to  learn  the  course  by  ifiing  this  sys- 
tem (Item  24). 

Furthermore,  one  very  interesting  phenomenon  is  when  being  presented  with  the  question  (Item  19  and  Item  20)  that  whether 
using  this  system  helps  them  to  obtain  the  feedback  from  teachers  (mean  = 4.53)  or  from  the  other  students  (mean  = 4.57),  the  results 
show  that  this  system  has  been  more  useful  for  them  to  obtain  the  feedback  from  other  students  than  from  their  teachers.  We  deduce 
that  one  of  the  reasons  may  be  that  students  generally  hold  higher  expectation  for  feedback  from  teachers.  Another  reason  may  be 
that  the  contents  of  teachers’  feedback  are  not  as  much  as  that  of  other  students’.  Since  the  teachers  have  to  answer  and  feedback  to 
all  students  while  the  students  only  need  to  feedback  to  a few  selected  peers.  The  above  survey  results  reflect  the  fact  that  the  feedback 
from  other  students  has  been  helpful  for  them  and  has  thus  become  the  necessary  component  to  help  them  in  learning. 

On  the  average,  as  high  as  88.84%  of  the  users  feel  that  the  system  has  been  helpful  in  their  learning  process  and  outcome.  Only 
2.76%  a small  percentage  of  the  users  think  otherwise.  This  shows  that  the  implementation  of  portfolio  truly  helps  most  of  the  stu- 
dents in  academic  growth  and  improvement  with  great  efficiency.  Thus  it  has  created  positive  impact  on  learning  process  and  outcome. 
Figure  2 shows  the  distribution  of  means  for  student  agreement  percentages  in  WBLP  impacts  on  learning  process. 
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Figure  2:  The  distribution  of  means  for  student  agreement  percentages  in  WBLP  impacts  on  learning  process 

To  sum  up  the  results  of  the  above  user  evaluation,  the  appraisal  for  impact  on  learning  process  is  high  (mean  = 4.33).  To  learn 
from  the  lesson  above,  teachers  should  give  much  more  encouragement  to  their  students,  and  pay  more  attention  to  the  difficulties 
that  the  students  may  come  across  in  the  implementation  process  of  future  portfolio  assessment. 


User  In-Depth  Interviews 

We  conducted  in-depth  interviews  with  5 users  (who  were  chosen  randomly  from  the  students  who  had  been  taking  the  course) 
after  the  statistics  that  have  come  out  based  on  the  suggestion  items  and  questionnaire  results  to  deduce  or  conclude  a few  important 
user  opinions  or  suggestions. 

In  terms  of  the  impact  on  learning  process,  most  students  regard  that  the  overall  design  of  the  WBLP  system  is  good.  They  are 
able  to  get  to  know  other  students’  learning  processes,  learning  experiences,  and  thus  enhance  their  learning  of  the  course  through  their 
browsing.  This  WBLP  system  provides  the  gathering  and  browsing  to  access  to  other  students’  works,  and  thus  allows  one  to  get  to 
know  better  about  his/her  merits  and  shortcomings.  Then  one  can  improve  accordingly  in  learning.  Some  students  regard  that  to  set  up 
learning  goals  may  allow  themselves  to  be  ready  for  individual  learning  direction.  However,  there  seems  to  be  no  direct  evidence  re- 
flected through  our  assessment  standard  to  support  the  assumption  that  such  a learning  goal  set  up  may  help  them  in  terms  of  assessing 
their  final  learning  results.  Hence,  its  real  purpose  and  function  seems  to  be  ambiguous.  In  addition,  introspection  and  self-assessment 
help  to  review  the  process  in  students’  learning  or  producing  their  works.  It  would  allow  the  students  to  discover  the  areas  that  they 
may  improve.  As  a result,  it  enhances  learning  for  maintaining  such  a reviewing  process. 


Expert  Evaluation 

Upon  the  completion  of  this  system  establishment,  we  had  invited  3 experts  (including  a portfolio  assessment  expert,  an  Inter- 
net technology  expert,  and  the  course  instructor)  to  use  this  system.  The  researcher  was  then  interviewing  them  one  week  later.  In  this 
way  we  have  obtained  the  valuable  opinions  and  suggestions  of  these  experts. 

These  experts  also  answer  the  following  questions,  such  as:  “whether  this  system  has  been  helpful  for  students’  learning  and 
teachers’  teaching?”  “Is  it  possible  to  provide  an  effective  and  appropriate  portfolio  production  and  browsing  environment  in  order  to 
allow  both  the  teachers  and  the  students  to  better  understand  the  genuine  learning  processes  and  results  of  students?”  Some  experts 
regard  that  this  WBLP  system  provides  the  opportunity  for  gathering  students’  works  for  emulation  among  the  peers.  Consequently, 
the  practice  will  surely  enhance  students’  learning  result.  The  introspection  and  self-assessment  records  of  students  would  allow  the 
teacher  to  better  understand  students’  prerequisite  ability  as  well  as  the  practical  issues  or  problems  that  students  have  encountered,  and 
also  students’  achievements  in  learning.  This  will  help  to  attain  the  goal  of  being  more  objective  in  student  assessments,  and  it  will  also 
help  the  teacher  to  adjust  his  or  her  teaching  method  in  the  future.  Some  experts  point  out  that  the  works  that  students  have  presented 
via  this  WBLP  system  are  mainly  the  final  completed  works  of  students,  but  the  processes  of  such  productions  in  terms  of  gathering 
and  demonstration  shall  also  be  incorporated  as  well.  Hence,  the  production  processes  or  the  on-going  works  shall  be  demonstrated  as 
well.  Finally,  the  system  has  to  improve  its  management  functions  in  terms  of  supporting  the  teachers  to  perform  on-line  assessments 
for  students’  works,  and  to  track  the  status  of  students’  using  the  services  of  this  system  through  on-line  records  and  statistics. 


Summaries  and  Conclusions 

The  characteristics  of  the  WBLP  is  meant  to  allow  the  teacher  and  students’  genuine  understanding  of  the  individual  learner 
learning  process  and  outcome,  and  thus  to  enhance  the  feedback  and  interactions  among  teachers  and  students.  Furthermore,  it  helps 
students  to  get  to  know  the  merits  and  shortcoming  in  his/her  own  learning,  also  the  difficulties  and  problems  involved,  thus  to  pro- 
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mote  the  integration  of  teaching,  learning,  and  assessment. 

As  high  as  88.84%  of  the  users  regard  the  electronic  portfolio  system  has  benefited  their  learning.  As  high  as  93%  of  the  stu- 
dents regard  that  the  quality  of  their  own  work  can  be  improved  by  the  emulation  through  the  browning  of  others’  works  (Item  12). 
94%  of  the  users  regard  that  it  helps  to  understand  the  merits  and  shortcomings  of  other  students’  learning  by  browsing  other  students’ 
portfolio.  97%  of  the  users  regard  that  it  enhances  their  own  academic  growth  and  improvement  by  browsing  other  students’  portfo- 
lios. The  evaluation  results  also  show  that  this  WBLP  system  has  been  more  useful  for  them  to  obtain  the  feedback  from  other  stu- 
dents than  from  their  teachers.  It  reflects  the  fact  that  the  feedback  from  other  students  has  been  helpful  for  them  and  has  thus  be- 
come the  necessary  component  to  help  them  in  learning. 

Whether  the  use  of  web-based  portfolio  system  will  definitely  enhance  students’  self-regulated  learning,  self-directed  learning, 
enhancing  the  ability  for  self-reflection,  encouraging  learning  motivation,  etc.  are  all  the  interesting  issues  worthy  of  our  further  re- 
search and  verification.  Therefore,  we  will  conduct  the  relevant  future  experiment  study  in  terms  of  the  practical  applications  in  teach- 
ing and  learning  efficiency  for  web-based  portfolio  system. 
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Abstract:  As  Web- based  courses  using  videos  become  popular  in  recent  years,  the  issue  of 
managing  audio  -visual  aids  becomes  noteworthy.  In  general,  the  contents  of  audio  -visual  aids 
may  include  a lecture,  an  interview,  a featurette,  or  an  experiment  etc.  Then,  the  audio-visual 
aids  of  Web- based  courses  are  transformed  into  the  streaming  format  that  can  make  the 
quality  of  Internet-based  videos  acceptable  to  learners  using  a limited  bandwidth.  Although 
streaming  technique  enables  the  learners’  accessibility  of  audio-visual  aids  over  Internet,  the 
usage  of  audio-visual  aids  is  still  totally  adhere  to  instructors’  perspectives.  In  fact,  distance 
learners  can  contribute  ideas  not  only  in  text  format,  but  also  in  audio-visual  format.  However, 
previous  researches  did  not  consider  the  feasibility  of  audio-visual  aids  contributed  from 
distance  learners.  The  objective  of  this  paper  is  to  argue  that  the  usage  of  audio-visual  aids 
from  distance  learners’  perspectives  should  be  considered  in  designing  Web  based  course.  To 
demonstrate  this  concept,  this  paper  first  introduces  a screen  camcorder  tool,  which  enables 
learners  to  record  screen  activity  as  videos  in  standard  format  or  streaming  format.  Then,  a 
collaborative  learning  strategy,  called  Jigsaw  II,  is  applied  to  encourage  expertise  group 
contribute  streaming  videos  for  training  other  learners.  Finally,  a preliminary  survey  of 
technology  acceptance  is  implemented  on  37  learners.  Results  confirm  the  feasibility  of 
audio-visual  aids  contributed  from  distance  learners. 


Introduction 

As  Web-based  courses  using  videos  become  popular  in  recent  years,  the  issue  of  managing  audio-visual 
aids  becomes  noteworthy.  It  is  not  a new  idea  of  using  videos  for  distance  learning  (Kozma,  1986;  Zigerell,  1991; 
Lcvnc,  1992).  For  instance,  a distance  learning  course  may  conventionally  transmit  videos  by  videotapes,  cable 
TV,  broadcast  TV,  or  VCD.  Although  researches  indicate  that  the  face-to-face  (synchronous  or  asynchronous) 
situation  is  not  an  essential  factor  of  learning  performance,  some  reports  show  that  distance  learners  prefer  video 
to  other  media.  Hence,  a distance  learning  instructor  may  often  provide  teaching  video  to  distance  learners  by 
Internet  in  recent  years. 

The  contents  of  audio-visual  aids  may  include  a lecture,  an  interview,  a featurette,  the  operation  of  a 
machine,  or  an  experiment  etc.  In  fact,  the  contents  of  audio-visual  aids  are  discipline-dependent.  For  instance, 
the  major  portion  of  the  video  contents  for  a Physics  course  may  be  used  to  demonstrate  an  experiment.  Anyway, 
the  instructor  is  the  only  provider  of  audio-visual  aids  in  most  Web- based  courses.  Furthermore,  very  few 
instructors  were  involved  in  planning  or  designing  the  teaching  videos.  Instead,  audio-visual  aids  are  considered 
as  replacements  of  learning  activities  that  can  not  be  implemented  in  a classroom  situation.  As  Benney  said 
“Most  certainly  video  is  a successful  medium  for  taking  students  into  the  field  to  observe  facts,  processes  and 
emotional  events  which  would  otherwise  be  impossible  to  see  and  experience”  (Benney,  2001). 

To  provide  teaching  video  over  Internet,  the  video  signals  should  be  transformed  into  the  streaming  format 
that  can  make  the  quality  of  Internet -based  videos  acceptable  to  learners  in  a limited  bandwidth.  The  streaming 
format  means  that  the  requested  video  signals  are  stepwise  transmitted  from  server  to  clients.  The  most  famous 
commercial  products  for  providing  streaming-video  functionalities  are  the  windows  media  server  of  MicrosofF*^ 
and  the  realmedia  server  of  RealNetworks**’’^.  Consequently,  a Web-based  course  should  prepare  contents  of 
videos,  transform  videos  into  streaming  format,  and  setup  a media  server  to  provide  teaching  videos  through 
Internet. 

Existing  learning  strategies  of  video-based  instruction  can  be  categorized  by  three  types:  (1)  passive 
watching,  (2)  leam/practice  while  watching,  and  (3)  leam/practice  after  watching  (DeMartino,  2001).  The 
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passive  watching  strategy  means  that  students  do  not  be  engaged  in  a discussion  activity  or  a leam-ty -doing 
activity.  The  passive  watching  strategy  genially  results  in  poor  long-term  outcomes.  The  leam/practice  while 
watching  strategy  means  that  a student  or  a learning  group  can  pauses,  stops,  forwards,  or  rewinds  a video  clip  to 
self-pace  learning  by  joining  a discussion  or  leam-by- doing  activity  with  learning  partners.  The  leam/practice 
after  watching  strategy  means  that  the  discussion  activity  or  leam-by- doing  activity  following  a teaching  video 
provides  a reflective  opportunity  to  deepen  long-term  outcomes. 

No  matter  which  type  of  learning  strategies  for  video-based  instruction  is  used,  the  usage  of  audio-visual 
aids  is  totally  adhere  to  instructors’  perspectives.  Distance  learners  are  not  expected  to  be  able  to  provide  videos 
to  depict  an  idea  or  communicate  with  others.  Hence,  a distance  learning  instructor  tends  to  provide  BBS,  email 
list,  or  WBB  (Web  Bulletin  Board)  to  support  asynchronous,  collaborative  learning  after  watching  an 
instruction -video  (Repenning,  loannidou,  & Phillips,  1999)  In  fact,  distance  learners  can  contribute  ideas  not 
only  in  text  format,  but  also  in  audio-visual  format.  However,  previous  researches  did  not  consider  the  feasibility 
of  audio-visual  aids  contributed  from  distance  learners. 

The  required  computer  hardwares  for  videoconferencing,  such  as  video  camera,  video  capture  card,  and 
netowrk  bandwidth  etc.,  will  be  a considerable  expense  for  a distance  learner.  Consequently,  most  Web-based 
courses  do  not  implement  completely  video-based  communication  environment  because  of  considering  distance 
learners  ’ financial  burden.  Moreover,  video-based  instruction  is  generally  regarded  as  a synchronous 
communication  tool  or  a passive  medium  such  as  recorded  lectures.  However,  a noteworthy  idea  is  that  distance 
learners  can  contribute  audio-visual  aids  in  their  own  way  instead  of  an  instructor’s  way.  In  other  words,  distance 
learners  can  record  computer  screen  activity  as  video  and  oral  explanation  as  audio  in  a video-based 
communication  environment  without  additional  computer  hardware.  Consequently,  distance  learners  can  join  the 
process  of  asynchronous,  video-based  instruction  to  ease  the  general  problem,  that  is  instructors  tend  to  spend 
too  much  time  lecturing,  of  video-based  instruction. 

To  demonstrate  this  idea,  the  followng  section  first  introduces  a screen  camcorder  tool,  which  enables 
learners  to  record  screen  activity  as  videos  in  standard  format  or  streaming  format.  Then,  a collaborative  learning 
strategy  (Jigsaw  11),  which  was  proposed  by  Elliot  Aronson(Slavin,  1995;  Aronson  & Patnoe,  1997),  is  applied 
to  encourage  expertise  group  contribute  streaming  videos  for  training  other  learners.  Finally,  a preliminary 
survey  of  technology  acceptance  was  investigated  and  results  are  shown  and  discussed. 


Computer  Desktop  Camcorder 

Many  commercial  products  can  record  computer  desktop  activity  and  voice  from  a microphone  as  videos 
in  standard  AVI  formats,  for  instance  Lotus™  ScreemCam'^'',  TechSmith™  Camtasia'^',  and  Hyperionics’^” 
HyperCani^^^  so  on.  Computer  “ desktop  activity”  means  that  learners ’every  action,  such  as  open  a file,  execute  a 
program,  show  slides  for  a lecture,  use  browser,  and  even  all  mouse-pointer  movements  so  on,  wil  be  recorded  as 
a video  Furthermore,  learners’  explanations  during  the  aforementioned  interval  of  actions  can  be  recorded  as  the 
audio  track  of  a teaching  video  through  the  microphone  of  a computer.  Because  computer  desktop  or  screen  is 
like  the  viewfinder  of  a video  recorder,  aforementioned  softwares  are  called  desktop  camcorder  or  screen 
camera 

Originally,  desktop  camcorder  tools  are  used  to  train  users  for  new  software  by  an  kind  of  active  style. 
Hence,  most  desktop  camcorder  tools  includes  the  following  features: 

(1)  Users  (here  it  means  learners)  can  determine  the  video  capture  area. 

(2)  Users  can  use  zoom  effects  to  zoom  in  or  zoom  out  during  capture. 

(3)  Users  can  annotate  a caption  on  the  captured  video. 

(4)  Users  can  use  a microphone  to  input  audio  during  capture. 

To  get  an  overview  of  computer  desktop  camcorder,  TechSmitlf*^  Camtasia  is  used  for  demonstration.  Figure  1 
illustrates  use  TechSmith™  Camtasia  to  capture  slides,  mouse  movement,  audio  from  microphone  input  as 
teaching  videos.  Instructors  can  press  the  capture  hotkey  or  elide  on  the  record  button  (the  red  one)  on  the 
toolbar  to  start  recording.  During  recording,  Camtasia  recorder  can  be  minimized  to  aviod  obstructing  learners’ 
lines  of  sight.  Instructors  can  stop  the  capture  by  a hotkey  and  save  the  captured  video  to  a file. 


[1]  Lotus  ScreenCam.  URL:  http://www.lotus.com/home.nsfrightframe/lscreencam 

[2]  TechSmith  Camtasia.  URL:  http://www.techsmith.com 

[3]  Hyperionics  HyperCam.  URL:  http://www.hyperionics.com/hc/index.html 
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Figure  1.  Use  computer  desktop  camcorder  to  produce  lecture  videos. 


In  general,  the  captured  videos  are  produced  in  standard  AVI  formats.  However,  the  captured  videos  can  be 
easily  transformed  into  proper  streaming  formats  that  depends  on  the  technology  for  dreaming  media  services, 
i.e.  the  windows  media  server  of  Microsoft'*’’^  or  the  realmedia  server  of  RealNetworks™.  Both  companies 
provide  freeware  to  transform  videos  of  standard  AVI  formats  into  proper  formats.  Fortunately,  TechSmith™ 
Camtasia  tools  can  support  both  transformations.  Consequently,  users  can  easily  produce  streaming  videos  no 
matter  which  technology  for  streaming  media  services  is  used. 


Collaborative  learning  strategy 

The  elements  for  a collaborative  learning  strategy  include  heterogeneous  grouping,  interdependence, 
individual  accountability,  and  group  processing  (Warschauer,  1997).  Those  elements  are  used  to  make  sure 
learners  will  try  to  help  each  other.  Some  researches  developed  learning  environment  for  implementing 
collaborative  learning  strategy  (Brandon  & Hollingshed^  1999;  Persico&  Manca,  2000).  However,  there  is  not 
any  collaborative  learning  environment  based  on  streaming,  asynchronous,  and  video  -based  technique  instead  of 
Web-based  or  videoconferencing  techniques.  Furthermore,  collaborative  learning  strategies  are  also  required  to 
promote  learning  motivation  and  learning  performance  when  we  apply  the  novel  technique  to  distance  learners. 

Before  developing  prototype  system,  another  question  arises  from  ‘which  one  of  collaborative  learning 
strategies  in  literature  is  more  suitable  for  computer  desktop  camcorder  technique  than  others’.  It  is  difficult  to 
prove  that  a collaborative  learning  strategy  is  more  suitable  than  others  are  because  no  literature  reports  using 
computer  desktop  camcorder  technique  in  collaborative  learning  activities.  Furthermore,  the  selected 
collaborative  learning  strategy  should  be  able  to  encourage  distance  learners  contributing  instructional  videos  to 
training  learning  partners  from  distance  learners’  perspectives.  In  consideration  of  those  issues,  the  Jigsaw  II 
collaborative  learning  strategy  is  choosed  because  the  expertise  groups  mechanism  of  Jigsaw  II  can  support 
distance  learners  to  master  a learning  topic.  Then,  we  can  encourage  members  of  an  expertise  group  to  sharing 
their  knowledge  in  streaming  video  style. 

It  is  assumed  that  an  instructor  would  like  to  construct  a collaborative  learning  strategy,  for  this  example 
Jigsaw  II,  on  Web.  In  general,  there  are  three  steps  in  the  J igsaw  II  collaborative  learning  strategy.  First,  distance 
learners  are  heterogeneously  assigned  to  groups  according  their  cultures,  mother  tongues,  and  grade  point 
averages  so  on.  Hence,  the  groups  of  first  step  are  called  heterogeneous  groups  or  jigsaw  groups.  Then,  every 
distance  learner  of  a heterogeneous  group  is  respectively  assigned  to  a study  group  for  mastering  a specific 
learning  topic.  Distance  learners  collaboratively  study  an  expertise  or  learning  topic  to  support  each  other 
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becoming  an  “expert”  of  a topic  in  a study  group.  Consequently,  the  study  groups  of  step  two  are  called  expertise 
groups  Finally,  every  member  of  an  expertise  group  will  return  to  his/her  own  heterogeneous  group.  After 
distance  learners  return  to  geterogeneous  groups,  every  member  of  a heterogeneous  group  is  acountable  for 
teaching  a learning  topic,  which  he/she  learned  in  an  expertise  group,  to  the  other  members.  In  other  words,  each 
student  should  present  a well-organized  report  to  teaching  a learning  topic  when  he/she  comes  back  to  his/her 
heterogeneous  group. 

The  required  processes,  heterogeneous  groups,  and  expertise  group  for  Jigsaw  II  collaborative  learning  are 
illustrated  in  Figure  2.  From  the  perspective  of  a distance  learner,  computer  desktop  camcorder  can  support 
him/her  to  tutor  learning  partners  in  a heterogeneous  group  by  video-based  instruction.  To  apply  video-based 
instruction,  a distance  learning  instructor  should  setup  a media  server  as  a streaming  video  provider.  Then, 
distance  learners  of  an  expertise  group  should  not  only  study  a learning  topic,  but  also  plan  a teaching  video. 
Finally,  distance  learners  of  an  expertise  group  can  produce  a teahing  video  by  computer  desktop  camcorder  and 
upload  the  teaching  video  to  a media  server.  Consequently,  video-based  instruciton  in  a heterogeneous  group 
becomes  feasible.  The  refining  part  of  Jigsaw  II  collaborative  learning  is  depicted  in  the  right  part  of  Figure  2. 


Video-based  instruction 


Expertise  Expertise  Expertise 

Group  1 Group2  Group3 


Computer  Desktop  Camcorder 

Figure  2.  Integrating  video-based  instruction  by  a desktop  camcoder  supports  Jigsaw  II  collaborative  learning 


Analysis  of  Preliminary  Evaluation 

The  evaluation  presented  in  this  section  attempts  to  analyze  distance  learners*  technology  acceptances  of 
using  desktop  camcorder  for  collaborative  learning.  This  experiment  was  carried  out  in  Da  Yeh  University  and  a 
sample  of  37  students  (18  females  and  19  males)  was  chosen  from  a class  of  information  management 
department.  The  objective  was  to  figure  out  how  useful  and  ease  of  use  the  distance  learners  think  desktop 
camcorder  is. 

A questionnaire,  used  in  Davis  *s  technology  acceptance  model  (TAM),  was  chosen  as  an  evaluation  tool 
(Davis,  1989).  Davis  *s  TAM  can  support  managers  to  understand  the  process  of  adopting  new  technology.  One 
can  realize  the  importance  of  TAM  from  this  description:  “(TAM)... are  widely  accepted  among  the  MIS  research 
community  as  tools  for  evaluating  information  system  applications  and  predicting  usage’*  (Doll , Hendrickson,  & 
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Deng,  1998)  In  other  words,  the  results  of  TAM  can  provide  evidence  to  be  weigh  for  or  against  the  application 
of  adopting  desktop  camcorder  in  collaborative  learning.  Originally,  there  are  twleve  questions  classified  as  two 
categories  (perceived  usefulness  and  perceived  ease  of  use)  in  Davis’s  questionnaire  for  TAM.  In  this  evaluation, 
one  overall  ranking  question  was  added  to  the  original  questionnaire.  The  contents  of  the  questionnaire  used  in 
this  evaluation  are  listed  as  following: 


C/3 

Xfl 

Ql. 

Using  <computer  desktop  camcorder>  in  my  job  would  enable  me  to  accomplish 
tasks  more  quickly. 

Q2. 

Using  <computer  desktop  camcorder>  would  improve  my  job  performance. 

Q3. 

Using  <computer  desktop  camcorder>  in  my  job  would  increase  my  productivity. 

Q4. 

Using  <computer  desktop  camcorder>  would  enhance  my  effectiveness  on  my  job. 

Q5. 

Using  <computer  desktop  camcorder>  would  make  it  easier  to  do  my  job. 

Q6. 

1 would  find  <computer  desktop  camcorder>  useful e in  my  job. 

Q7. 

Learning  to  operate  <computer  desktop  camcorder>  would  be  easy  for  me. 

Q8. 

1 would  find  it  easy  to  get  <computer  desktop  camcorder>  to  do  what  1 want  it  to 
do. 

SE< 

USE 

Q9. 

My  interaction  with  <computer  desktop  camcorder>  would  be  clear  and 
understandable. 

QIO. 

1 would  find  <computer  desktop  camcordei>to  be  flexible  to  interact  with. 

Qll. 

It  would  be  easy  for  me  to  become  skillful  at  using  <computer  desktop  camcorder>. 

Q12. 

1 would  find  <computer  desktop  camcorder>  easy  to  use. 

Q13. 

Overall,  1 find  using  <computer  desktop  camcorder>  useful  in  my  job  and  believe 
that  <computer  desktop  camcorder>  is  easy  to  use. 

Table  1:  Contents  of  the  questionnaire  for  evaluation. 


Every  question  in  Table  1 was  scored  on  a 7-point  scales,  with  strongly  disagree  (1)  and  strongly  agree  (7) 
as  the  two  endpoints.  Table  2 depicts  descriptive  statistics  to  analyze  results  of  this  preliminary  evaluation.  From 
the  mean  of  the  13-th  question  (Q13),  most  learners  agree  that  computer  desktop  camcorder  is  useful  and  easy  to 
use.  Noteworthily,  the  data  show  that,  on  the  average,  learners’  perceived  usefulness  respondents  (Q1~Q6)  had 
relatively  lower  than  learner^  perceived  ease  of  respondents  (Q7~Q12).  Through  an  informal  interview, 
students  indicate  that  video-based  instruction  is  not  very  critical  because  they  are  not  in  a distance  learning 
situation.  One  student  said  “it  (teaching  video)  will  be  very  useful  in  heterogeneous  group  status  if  we  (learning 
partners  of  a heterogeneous  group)  can  not  meet  one  or  two  times  per  weeld’. 


Ql 

Q2 

Q3 

Q4 

Q5 

Q6 

Q7 

Q8 

Q9 

OlO 

Qll 

012 

013 

Mean 

5.27 

5.19 

5.03 

5.22 

5.08 

5.65 

5.86 

5.43 

5.49 

5.11 

5.62 

5.73 

5.68 

SD 

1.52 

1.21 

1.38 

1.46 

1.62 

1.36 

0.98 

1.26 

1.10 

1.43 

1.16 

1.12 

1.42 

Table  2:  Descriptive  statistics  of  preliminary  evaluation  results. 


Conclusion 

Although  streaming  technique  enables  the  learners’  accessibility  of  educational  multimedia  over  Internet  in 
recent  years,  the  usage  of  educational  multimedia  is  still  totally  adhere  to  instructors’  perspectives.  In  other 
words,  all  teaching  videos  and  audio-visual  aids  of  most  instructional  website  are  contributed  by  instructors  or 
instructional  teams.  This  paper  introduces  a tool,  computer  desktop  camcorder,  to  support  di  stance  learners 
contributing  teaching  videos  for  collaborative  learning.  We  integrated  computer  desktop  camcorder  into  a 
famous  collaborative  learning  strategy,  Jigsaw  11,  as  additional  supports  for  heterogeneous  group  learning.  After 
training  distance  learners  for  using  desktop  camcordei;  a preliminary  evaluation  of  technology  acceptance  is 
implemented  on  37  learners.  Results  show  that  most  learners  agree  that  computer  desktop  camcorder  is  useful  for 
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collaborative  learning  and  easy  to  use.  Most  important  of  all,  this  paper  indicates  the  feasibility  of  audio-visual 
aids  contributed  from  distance  learners. 
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An  Interactive  Storytelling  System  for  the  Pearl  of  the  Desert  Exhibit 

Computer  technology  brings  flexibility  and  more  interactive  into  museum  exhibition.  The 
characteristic  of  computer  multimedia  can  present  audio,  video,  graphic,  animation  as  well  as 
text  with  well-designed  program  structure.  Recently,  most  of  the  museums  created  a 
multimedia-based  learning  environment  using  new  computer  technology.  The  goal  is  to  set  up 
new  directions  of  exhibition  for  visitors  to  explore,  to  enjoy,  to  be  inspired  as  well  as  to  improve 
their  knowledge.  This  paper  proposes  an  interactive  storytelling  system  which  reveals  the  art 
collections  of  the  Tunhuang  and  its  historical  information  based  on  film  theory  and  computer 
technology.  The  collection  of  visitors’  motion  as  they  interacted  with  computers  is  also 
investigated. 

Tunhuang  art  is  one  of  the  most  famous  artistic  works  in  Central  Asia  in  which  Buddha 
statues  are  carved  and  painted.  It  consists  of  Tunhuang  Grottoes  from  4^*^  to  the  14^*'  centuries. 
From  the  historical  view,  in  the  past  thousands  of  years,  many  stories  about  these  caves  in 
Tunhuang  were  described  in  books  or  magazines.  An  interesting  story  as  “story  about  the  17^*' 
cave  in  Tunhuang”  would  be  described  briefly  as  following: 

Time  : 1900 
Place  : Tunhuang 

Cast : Wang  Juan~Lu  who  is  a taoist 

Wang  Juan-Lu  was  assigned  to  take  care  of  the  caves  in  Tunhuang  by  Chin  Dynasty 
government.  His  duty  was  to  maintain  the  painting  wall  and  clean  the  sandy  ground.  One 
day,  as  he  cleaned  the  16^^  caves,  the  wall  crashed  suddenly.  He  found  another  caves  where 
many  important  manuscripts  of  literature,  linguistics,  philosophy,  population,  religion, 
science,  medicine,  economy,  society  and  so  on  were  collected. 

He  reported  to  the  Chin  Dynasty  government.  However  the  officers  ignored  that  with 
negative  attitude.  Finally,  Wang  Juan-Lu  traded  most  of  the  manuscripts  with  foreigners 
under  the  table. 

In  order  to  present  the  historical  information  and  art  collections,  National  Museum  of  Natural 
Science  in  Taiwan  conducts  an  exhibition  called  “ The  pearl  of  the  Desert:  The  caves  of 
Tunhuang  Exhibition”  in  1999.  This  exhibition  combines  the  historical  background  of  Tunhuang 
with  beauty  of  arts.  With  the  goal  of  understanding,  computer  played  an  important  role  as  an  art 
works  presenter,  an  interpreter  as  well  as  a story-teller.  The  outcome  is  to  entertain  the  visitors, 
to  enrich  the  visitors’  knowledge,  to  inspire  their  emotion  and  also  to  meet  their  variety  needs. 

The  computer  project  was  well-designed  with  Interactive  storytelling  construction.  Based 
on  film  theory,  time,  place  and  cast  with  animation  text  were  shown  on  the  first  scene.  Next,  the 
cast  Wang  Juan-Lu  appears  and  is  introduced  by  himself  with  speech  and  animation.  The 
audience  are  asked  to  sweep  the  dust  by  clicking  the  mouse  on  the  monitor.  The  cursor  is  also 
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designed  to  look  like  a broom.  Some  incredible  feedback  such  as  spiders,  rats,  and  interesting 
dialogs  appears  on  the  screen  as  the  mouse  is  clicked.  Certainly,  the  wall  would  crash  as  the 
audience  click  certain  area  on  the  screen.  After  the  wall  crashed,  Wang  Juan-Lu  is  moved  near 
the  wall  with  curiosity  and  a new  cave  with  lots  of  manuscripts  can  be  found.  After  that,  he 
would  ask  audience  some  questions  about  the  information  of  this  new  cave  in  dialog-inquiry 
mode.  Some  manuscripts  and  art  works  then  can  be  retrieved  in  detail  with  its  moving  and 
zooming  images  on  the  computer  screen. 

Actually,  computer  not  only  provides  a good  solution  to  preserve  the  objects  in  the 
exhibition,  but  also  provides  the  image  as  clear  and  real  to  satisfy  individual  interests  and  needs. 
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Abstract:  "Smileys"  are  part  of  the  Netspeak  (Crystal,  2001),  which  is  considered  a hybrid  of 
spoken  and  written  languages.  Because  literacy  practices  are  embedded  in  contexts  (Barton, 
1994),  online  literacy  practices  should  differ  depending  on  the  Internet  contexts  or  Internet 
situations.  The  current  paper  intends  to  validate  the  conceptualization  of  "Internet  situations". 
It  also  aims  to  refute  Crystal's  lumping  of  asynchronous  discussion  board  and  synchronous 
chat  as  one  type  of  Net  situation.  Comparisons  were  made  of  the  frequencies  of  smileys  in 
three  online  communication  media:  instant  massager,  email,  and  discussion  boards.  Findings 
indicate  that  "time"  and  "personal"  dimensions  to  be  useful  parameters  of  Internet  situations. 


The  prevalence  of  “faceless”  computer-mediated  communication  (CMC)  (Voiskounsky,  1998)  has 
resulted  in  the  development  of  “smileys”  or  “emoticons”.  Smileys  are  signs  meant  to  mimic  the  human  facial 
expressions  that  reflect  an  individual’s  emotional  state.  It  has  been  postulated  that  the  online  use  of  srrileys  is 
on  the  order  of  compensation  for  the  lack  of  emotional  richness  in  electronic  communication  (Crystal,  2001; 
Voiskounsky,  1998).  Crystal  (2001)  further  stressed  the  essential  role  of  smileys  in  the  emergent  “Netspeak.” 

A discussion  of  Netspeak  - the  language  of  Internet  communications  - could  not  do  without  the 
context  of  “spoken”  and  “written”  languages  because  Netspeak  is  the  meeting  point  of  the  two  (Crystal,  2001). 
In  an  analysis  of  Netspeak  based  on  seven  criteria  that  differentiate  written  and  spoken  languages.  Crystal  found 
that  Netspeak  is  closer  to  written  than  to  spoken  language. 

Since  the  development  of  language  and  literacy  are  inseparable  from  contexts  (Barton,  1994),  the 
diversity  of  possible  Net  discourse  contexts  infers  the  inherent  heterogeneity  in  literacy  practices  via  Internet 
media.  Crystal  (2001)  proposed  four  categories  of  social  situations  in  which  Netspeak  occurs,  calling  them 
“Internet  situations.”  By  comparing  discourse  patterns  among  different  Internet  situations  to  those  of  spoken 
and  written  languages.  Crystal  concluded  that  the  degree  of  similarity  between  Netspeak  and  written  or  spoken 
language  is  dependent  on  the  context.  For  instance,  it  was  found  that  language  used  in  chatroom  discourse  is 
closer  to  spoken  language  than  is  that  used  in  Web-page  presentation. 

As  part  of  Netspeak,  the  evolution  of  smileys  has  also  been  influenced  by  social  and  cultural  contexts. 
Observations  on  the  development  of  smileys  (as  well  as  Netspeak,  itself)  indicate  an  emerging  divergence  - as 
opposed  to  convergence  - in  Net-linguistic  evolution.  A comparison  of  the  English  and  Japanese  smileys 
indicates  that  the  differences  are  deeper  than  in  the  “face”  value  (i.e.,  the  form);  variation  in  socio-cultural 
values  is  also  evident.  For  instance,  since  it  is  impolite  for  a Japanese  lady  to  show  her  teeth  while  smiling  or 
laughing,  a female  smile  is  presented  as  rather  than  Given  that  the  users  and  the  environment  are 

the  developmental  outcome  of  the  social  and  cultural  contexts,  the  development  of  Japanese  smileys  provides  a 
good  example  for  the  social  construction  of  language  and  literacy  practices.  While  the  aforementioned  example 
points  out  the  impact  of  the  cultural  context  on  online  literacy  practices,  the  following  section  examines  the  role 
of  online  situations  on  the  occurrence  of  literacy  events  and  the  defining  dimensions  of  these  situations. 

Since  literacy  practices  are  situation-dependent  (Barton,  1994),  practices  involving  the  use  of  smileys 
should  also  vary  in  accordance  to  the  situation.  Crystal  (2001)  lumped  the  synchronous  chat  and  asynchronous 
discussion  into  one  Internet  situation.  Because  the  lag  between  messages  is  one  major  difference  between 
spoken  and  written  languages,  the  time  factor  should  play  an  essential  part  in  the  ecology  of  online 
communications.  Thus,  given  its  asynchronous  nature,  online  discussion  would  share  a higher  degree  of 
similarity  with  e-mail  than  with  online  chat.  Moreover,  discussion  board  postings  are  viewed  by  the  public  and 
aim  to  communicate  facts  and  ideas.  When  compared  to  discussion  postings,  e-mails  are  more  likely  to  be  used 
for  social  and  unplanned  discourse.  Thus,  given  the  degree  of  “personal”  involvement,  there  should  be  a higher 
degree  of  similarity  between  e-mail  and  chatroom  than  between  discussion  board  and  chatroom.  Furthermore, 


Introduction 


spoken  languages  are  enriched  by  emotional  signals,  like  facial  expressions,  of  which  written  languages  are 
deprived.  Given  that  the  introduction  of  smileys  aims  to  compensate  for  the  lack  of  emotional  richness,  it  is 
postulated  that  there  will  be  a higher  degree  of  smiley  usage  in  those  Internet  situations  most  similar  to  spoken 
discourse.  Thus,  when  comparing  the  usage  of  smileys  in  chat,  e-mail,  and  discussion  situations,  the  highest 
degree  of  smiley  usage  would  be  expected  to  occur  in  the  chatroom  and  the  lowest  on  the  discussion  board. 


Method 


100  recent  postings  in  discussion  boards  for  three  distance  learning  courses;  100  emails  for  both 
business  and  personal  purposes;  and  100  instant  messages  were  collected.  While  18  e-mails  and  57  instant 
messages  contain  at  least  one  smiley  face,  smiley  faces  appear  on  15  of  the  discussion  board  postings.  T-tests 
were  performed  to  examine  whether  there  are  significant  differences  in  the  occurrences  of  smileys  across 
Internet  situations.  While  no  significant  differences  were  found  between  the  e-mail  and  discussion  board 
situations,  smileys  appeared  significant  more  frequently  in  instant  messages  when  comparing  to  the  email 
(t=6.40,  df=198)  and  the  discussion  board  (t=6.85,  df=198)  situations.  Similar  findings  have  been  reported  in  a 
previous  study  (Af  Segerstad  & Ljungstrand,  2002). 


Findings 

The  large  smiley  difference  between  chat  and  discussion  implies  an  inherent  difference  between  the 
two  forms  of  discourse,  which  contradicts  Crystal’s  (2001)  categorization.  The  current  findings  imply  that, 
while  there  is  a large  difference  between  the  chat  situation  (i.e.,  instant  message)  and  the  other  two,  that 
between  the  e-mail  and  discussbn  situations  is  not  as  salient.  Such  findings  imply  a “time”  dimension  for 
Internet  situations  because,  while  the  use  of  instant  messaging  is  mostly  associated  with  synchronous  CMC,  e- 
mailing  and  postings  in  discussion  boards  are  media  for  asynchronous  CMC.  Moreover,  in  the  current  study, 
the  instant  messaging  was  for  personal  purposes,  while  the  use  of  discussion  forums  was  for  public  purposes 
(e.g.,  delivering  facts  and  ideas).  Serving  both  personal  and  business  purposes  puts  e-mailing  in  the  middle 
situation  between  discussion  and  chat.  Such  observation  further  insinuates  a “personal”  dimension  for  Internet 
situations. 

The  current  paper  rests  on  the  assumptions  that  the  emergent  “Netspeak”  exists  and  that  smileys  are  a 
sub-component  of  Netspeak  (Ciystal,  2001).  One  major  purpose  of  the  study  was  to  validate  the 
conceptualization  of  “Internet  situations”  by  conducting  a case  study  about  the  usage  of  smileys.  While 
findings  seem  to  support  the  notion  of  such  conceptualization,  discussions  also  point  out  that  the  “time”  and 
“personal”  dimensions  might  be  two  useful  parameters  to  describe  Internet  situations.  It  is  suggested  that  future 
research  should  focus  on  refining  the  conceptualization  of  Internet  situations  and  their  defining  parameters. 
Such  efforts  would  be  useful  in  establishing  a conceptual  framework,  which  could  aid  in  untangling  the  topic  of 
“Internet  literacy.”  It  should  be  noted  that  the  current  study  was  based  on  data  collected  from  only  1 subject, 
which  cast  doubts  on  the  generalizability  of  the  results.  A larger  sample  will  be  required  to  establish  the 
generalizability  of  the  given  findings. 

Reference 

Af  Segerstad,  Y.  H.  & Ljungstrand,  P.  (2002).  Instant  messaging  with  WebWho.  InternationalJournal  of 
Human-Computer  Studies ^ 56  (1),  147-171. 

Barton,  D.  (1994).  Literacy:  An  introduction  to  the  Ecology  of  Written  Language.  Malden:  Blackwell. 

Crystal,  D.  (2001).  Language  and  the  Internet.  Cambridge,  UK:  Cambridge  University  Press. 

Pollack,  A.  (1996).  Happy  in  the  East  (^_^)  Or  Smiling  :-)  in  the  West. 
http://www.nytimes.com/library/cyber/week/08 1 2smile.html 

Voiskounsky,  A.  E.  (1998).  Internet:  Culture  diversity  and  unification. 
http://www.it.murdoch.edu.au/'^sudweeks/catac98/pdf/07_voiskounsky.pdf. 


318 


Problem-Based  Learning  in  an  On-Line  Biotechnology  Course 
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Abstract;  Biotechnology  in  Agriculture,  Food,  and  Human  Health  is  a World  Wide  Web  course  developed  by  Project  BIO,  a 
program  to  develop  and  share  Internet  biology  education  resources,  at  Iowa  State  University.  This  course  has  been  utilizing 
problem-based  learning  as  a mode  of  instruction  in  a unit  dealing  with  pre-symptomatic  DNA  testing  for  a genetic  disease. 
Problem-based  learning  is  the  use  of  a “real  world”  problem  or  situation  as  a context  for  learning.  Quantitative  and  qualitative 
assessments  of  student  gains  in  critical  thinking,  improvements  in  ability  to  solve  real-world  problems,  acquisition  of  knowledge, 
development  of  the  ability  to  work  productively  as  a team  member,  and  development  of  skills  supporting  life-long  learning  were 
analyzed.  The  current  study  indicates  that  problem-based  learning  is  a valid  and  efficient  means  of  expanding  student  learning  in 
on-line  courses. 


Problem-based  learning  (PBL)  is  an  increasingly  integral  part  of  education  reform  around  the 
world.  The  essence  of  PBL  can  be  summarized  as  the  use  of  a “real  world”  problem  or  situation  as  a 
context  for  learning  (Duch,  1995).  The  purpose  of  PBL  is  to  hcrease  education’s  relevance  to  the 
perceived  needs  of  the  professional  community,  and  to  increase  the  problem-solving  abilities  of  graduates, 
by  having  design  problems  presented  in  class  that  resemble  reality  (Cowdroy,  1994).  Learning  objectives 
in  PBL  situations  include  the  development  of  critical  thinking  skills,  development  of  a high  professional 
competency,  development  of  problem  solving  abilities,  acquisition  of  knowledge,  development  of  the 
ability  to  work  productively  as  a team  member  and  make  decisions  in  unfamiliar  situations,  and  acquisition 
of  skills  that  support  self-directed  life-long  learning,  self-evaluation,  and  adaptation  to  change  (Albanese 
and  Mitchell,  1993;  Ryan  and  Quinn,  1994).  The  student  learns  in  the  context  of  a real-world  problem, 
which  may  lead  to  a desire  to  successfully  solve  the  problem  because  the  student  has  internalized  the 
problem  (Schmidt,  1993).  PBL  has  traditionally  been  conducted  in  cooperative  learning  groups  in  a face- 
to-face  setting.  Less  is  known  about  its  use  in  the  virtual  environment  (Camp,  1996). 

A problem-based  learning  approach  is  being  tested  in  an  on-line  course  entitled  “Biotechnology  in 
Agriculture,  Food  and  Human  Health”  offered  by  Iowa  State  University.  This  is  a three-credit  survey 
course  that  covers  biotechnology  and  its  applications  as  well  as  associated  ethical,  legal  and  social  issues. 
The  biotechnology  course  consists  of  on-line  audio-visual  lectures  that  are  modeled  after  lectures  in  a face- 
to-face  classroom,  experiential  learning  assignments  and  reading  assignments.  About  50%  of  the  grade  in 
the  course  is  based  on  experiential  learning  activities  and  the  other  50%  is  from  on-line  exams  based  on 
content  in  the  on-line  lectures  and  reading  material. 


Design  of  the  Problem-based  Learning  Unit  and  Assessment 

In  the  unit,  students  assume  the  role  of  a 29  year-old  man  whose  mother  died  of  a genetic  disease 
called  Huntington's  disease.  The  ultimate  objective  of  the  activity  is  to  make  a decision  about  whether  the 
man  should  undergo  pre-symptomatic  DNA  testing  for  the  genetic  disease. 

Two  versions  of  the  PBL  unit  were  tested.  Version  1 had  three  assignments  (Defining  the  Issues, 
Gathering  Information,  and  Solving  the  Problem),  which  served  as  guiding  problems  for  the  unit.  All  of  the 
assignments  were  done  as  cooperative  learning  activities  involving  groups  of  2 or  3 students. 

A problem  with  the  first  version  of  the  PBL  unit  was  the  narrow  focus  on  just  one  genetic  disease. 
Because  of  this  the  unit  was  revised  by  replacing  the  Gathering  Information  assignment  with  two  other 
assignments.  Genetic  Diseases  and  Genetic  Testing,  which  gave  the  students  a broader  view  of  the  subject. 
The  other  two  assignments.  Defining  the  Issues  and  Solving  the  Problem,  were  used  unchanged  in  Version 
2 of  the  PBL  unit. 
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Evaluation  of  the  Problem-based  Learning  Unit 


Students  using  PBL  Version  1 took  a non-credit  pre -module  exam  containing  questions  from  the 
same  test  bank  (thus,  the  same  difficulty)  as  the  for-credit  post-module  exam.  Results  indicated  that 
students  e?q)erienced  a significant  gain  in  factual  knowledge  about  genetic  testing  during  the  unit. 

We  also  compared  student  performance  on  the  post-module  exam  with  the  performance  of 
students  who  took  the  course  prior  to  developing  the  PBL  unit.  The  Genetic  Diseases  module  was  initially 
taught  using  an  instructor-centered  approach.  The  exam  scores  for  students  taking  either  version  of  the  PBL 
unit  were  not  significantly  different  from  those  of  students  taking  the  module  in  the  instructor-centered 
mode  (pre-PBL).  This  suggests  that  teaching  the  module  in  a problem-based  mode  does  not  compromise 
the  acquisition  of  factual  knowledge. 

Average  grades  for  the  Genetic  Diseases,  Genetic  Testing  and  Solving  the  Problem  assignments 
were  90%,  91%  and  92%,  respectively.  This  indicates  that  significant  higher  order  processing  of  the  fact- 
based  information  was  occurring. 

Student  evaluations  of  the  PBL  unit  were  positive  when  asked  to  evaluate  the  accomplishment  of 
learning  objectives.  Student  attitudes  toward  the  two  PBL  versions  were  similar  in  both  quantitative  and 
qualitative  measures.  The  cooperative  learning  aspect  of  the  PBL  offered  students  experience  in  time 
management,  schedule  coordination,  and  division  of  labor.  However,  some  students  expressed  concerns 
about  scheduling  difficulties  for  synchronous  electronic  meetings  with  their  cooperative  groups,  and  the 
technical  difficulties  inherent  in  an  on-line  setting  (such  as  Internet  lag,  computer  crashes,  or  bandwidth  or 
browser  problems).  In  addition,  group  activities  decrease  some  of  the  temporal  and  geographic  flexibility 
advantages  that  asynchronous  on-line  courses  offer. 


Conclusions 

Learning  objectives  in  PBL  situations  include  the  development  of  critical  thinking  skills, 
professional  competency,  problem-solving  abilities,  the  acquisition  of  knowledge,  the  ability  to  work 
productively  as  a team  member,  and  the  acquisition  of  skills  supporting  life-long  learning  (Albanese  and 
Mitchell,  1993;  Ryan  and  Quinn,  1994).  The  assignments  and  exams  for  the  PBL  were  designed  to 
evaluate  the  achievement  of  the  first  four  of  these  objectives,  while  student  evaluations  allowed  us  to 
evaluate  the  achievement  of  the  final  two.  The  reports  by  the  student  groups  indicate  a high  level  of 
internalization,  synthesis,  and  discursive  construction  of  knowledge,  integrating  their  learning  of  material 
from  this  course  with  their  individual  background  and  experiences,  fused  to  a common  consensus  within 
the  student  groups.  Evaluation  of  exam  scores  indicates  that  significant  factual  knowledge  is  acquired 
through  the  PBL  format.  While  PBL  did  not  increase  factual  learning  over  the  non-PBL  format,  it  did  not 
appear  to  negatively  compromise  factual  learning,  either.  Student  evaluations  of  this  unit  indicated  that  the 
final  two  objectives  were  met  in  a realistic  fashion.  Although  students  did  note  a few  difficulties  with  the 
PBL  format  in  the  on-line  setting,  we  feel  with  student  performance  on  this  unit  that  the  advantages  of  the 
cooperative  PBL  format  balance  the  disadvantages  experienced  by  certain  students  and  student  groups. 
Our  experience  indicates  that  PBL  is  a valid  means  of  expanding  student  learning  in  on-line  classes. 
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Background 


For  most  students  in  thedepartment  of  Information  Management,  “calculus”  has  always  been  one  of  the 
most  difficult  courses  to  study.  The  largest  part  of  reason  for  their  problems  results  in  their  previous  educational 
experiences.  For  the  pas  t educational  system  in  Taiwan,  students  have  to  pass  the  entry  exam  in  order  to  enter 
colleges.  Therefore,  students  know  how  to  take  tests  without  understand  the  underlying  concepts  of  the  test 
items.  Although  calculus  teachers  often  use  visual  aids  to  demonstrate  concepts,  students  usually  do  not  have 
the  ability  or  enough  information  to  understand.  As  a result,  more  and  more  students  fail  calculus  course,  and 
have  to  take  it  repeatedly.  It  has  always  been  the  major  concern  for  the  calculus  teachers  to  find  a way  to  solve 
the  problem. 

Web-based  learning  environments  have  been  the  main  trend  for  education  in  the  last  few  years.  The 
unique  function  of  web -based  learning  environment  provides  teachers  a space  to  upload  their  teaching  materials, 
a location  to  interact  with  individual  student,  and  to  trace  students’  learning  progress  without  time  constraint, 
web-based  learning  environment  also  offers  students  the  opportunity  to  interact  with  others,  to  review  learning 
materials  whenever  they  needed,  and  to  share  their  thoughts  with  fellow  students  and  teacher.  Due  to  the 
promising  multimedia  and  web  technology,  calculus  teachers  will  be  able  to  demonstrate  abstract  concepts  in  a 
understandable  manner,  to  provide  step  by  step  guiding  on  how  to  solve  calculus  problems,  and  to  monitor 
students’  problem  solving  pattern  in  problem  solving  activities. 

The  purpose  of  this  study  is  to  design  a web-based  calculus  learning  environment  to  supplement  the 
traditional  classroom  teaching,  and  to  exam  the  effectiveness  of  this  web- enhanced  learning  environment  on 
calculus  learning.  It  is  hoped  that  through  detailed  investigation,  this  study  will  contribute  additional  information 
to  the  design  of  web -based  learning  environment,  and  provide  guideline  for  teachers  and  instructional  designers 
to  construct  better  environment  for  procedure  knowledge  learning. 

Theoretical  Framework 

In  the  learning  process,  failures  to  access  and  use  potentially  relevant  information  result  in  failure  to 
transfer  (Bransford,  et  al.,  1990).  A few  studies  suggested  that  when  information  is  introduced  in  a 
problem  -solving  context,  it  is  more  likely  to  be  used  in  new  context  (Adams,  et  al.,  1988;  Lockhart,  Lamon,  & 
Gick,  1988;  Perkins  & Salomon,  1989).  Many  research  have  found  that  working  on  projects  or  problems  is  an 
engaging  activity  with  potential  for  facilitating  learning  (Blumenfeld  et  al.,  1991;  Katz,  1994;  Moursund,  1999). 

Constructivist  theories  support  the  development  of  open-ended  learning  environments,  and  tend  to  favor 
problem  -solving  activities  that  are  linked  to  student  interests,  that  have  some  attributes  of  real-world  problems, 
and  that  are  meaningful  and  satisfying  for  students  to  solve.  From  the  constructivist  perspective,  interest  is  the 
fuel  of  the  constructive  process.  Without  interest,  the  learners  will  never  make  the  constructive  effort  to  make 
sense  out  of  experience  (DeVries,  1993).  Learning  should  be  an  active,  constructive  process  whereby  learners 
generate  meaning  for  information  by  accessing  and  applying  existing  knowledge  (Jonassen,  1988).  The 
majority  of  learning  theories  guiding  web-based  instruction  are  based  on  the  constructivist  principles. 

The  design  of  web- based  instruction  should  integrate  the  constructivist  principles,  instructional 
implications  and  the  Web  features,  such  as  hyperlink,  multimedia,  and  synchronous  and  asynchronous 
communication  (Khan,  1997).  Hyperlink  structure  enables  learners  to  build  their  own  representation  of 
knowledge,  and  control  content  sequencing.  Multimedia  material  provides  a variety  of  presentation  of  learning 
materials.  Different  modes  of  communication  support  collaboration  learning  that  enables  learners  to  construct 
knowledge  (Miller,  & Miller,  2000).  How  to  design  a web-based  learning  environment  that  presents  learning 
materials  meaningfully,  that  provides  authentic  problem-solving  activities,  that  implements  the  unique  web 
technology  in  the  learning  activities,  that  offers  teachers  and  students  a space  to  convey  information  and  construct 
knowledge  is  a big  challenge  for  educators  and  instructional  designers. 
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Wet>* Enhanced  Learning  Environment 

The  content  of  the  web- enhanced  learning  environment  focused  on  the  differential  calculus.  There  are 
seven  sections  in  the  web  environment: 

1.  Learning  Material  Bank: 

This  section  uses  hypertext  and  multimedia  to  present  information  and  to  demonstrate  concepts.  Due 
to  the  nature  of  the  calculus  concepts,  many  animation  clips  were  designed  to  provide  additional  information 
channels  on  problem  solving  process.  Real-world  situations  were  connected  to  more  difficult  and  abstract 
concepts  that  were  demonstrated  by  interactive  multimedia  segments. 

Z Test  Bank: 

This  test- item  database  provides  three  different  levels  of  test  items.  Based  on  their  ability,  students  can 
select  the  appropriate  level  of  tests  to  evaluate  their  understanding  on  learning  materials.  After  the  test,  the 
program  will  provide  a test  report  that  will  indicate  the  test  items  missed  by  the  students.  This  report  also 
has  hyperlink  linking  the  missed  test  item  to  the  relevant  learning  section.  This  test  bank  can  also  be  used  as 
on-line  tests  during  the  class. 

3.  Guided  problem  -solving  section: 

This  section,  based  on  the  scaffolding  principles,  provides  step-by-step  guiding  in  solving  problems. 
The  problems  presented  in  this  section  are  also  related  to  students’  daily  life  experience.  Students  can  solve 
problems  directly  or  choose  guiding  whenever  they  encounter  problems. 

4.  Problem -based  learning  section: 

This  section  requires  cooperative  learning  in  groups.  The  problems  are  real  world  situations  that 
involve  students  in  problem-solving,  decision  making,  give  students  the  opportunity  to  work  autonomously 
over  extended  periods  of  time  with  group  members;  and  to  present  the  solutions . Students  can  use  ©mail  or 
chat  room  or  discussion  cento*  to  communicate  and  interact  with  each  other.  All  the  discussion  and  problem 
solving  process  will  be  stored  as  log  file  for  further  investigation. 

5.  Mathematical  place: 

The  history  of  mathematics,  mathematicians  and  web  links  are  presented  in  this  section,  giving 
students  additional  knowledge  on  mathematics. 

6.  Record  tracking: 

Students’  learning  paths,  test  record,  and  discussion  will  be  stored  in  the  tracking  database.  Teachers 
will  be  able  to  track  down  individual  student,  or  groups  to  monitor  their  learning  progress. 

7.  Idea  sharing: 

Students  use  this  section  to  share  their  ideas,  thoughts,  or  new  problem-solving  skills  with  other 
students  and  teacher,  and  to  express  their  feeling  about  the  course  or  learning  materials.  It  also  gives 
teachers  the  opportunity  to  understand  students’  thinking  and  comprehension  of  the  course. 

The  effectiveness  of  this  learning  environment  will  be  evaluated  in  the  next  school  year. 
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Introduction.  In  this  paper,  we  propose  using  aglets,  a specific  type  of  Java-based  mobile  agents,  to  enhance 
distance  education.  The  Internet  has  become  a ubiquitous  environment  for  students,  researchers,  and  educators  to 
share  resources,  exchange  information,  distribute  reports,  and  collect  data  since  the  creation  of  the  World  Wide 
Web  (WWW).  Among  the  many  factors  that  have  contributed  to  the  unquestionable  success  of  the  Internet,  the 
wide  availability  and  ease  use  of  computer  software  programs  have  played  an  important  role,  in  addition  to  the 
increasing  bandwidth  of  the  communication  networks.  Most  of  the  software  programs  currently  in  use  on  the 
Internet,  however,  are  either  stationary  or  reactive,  meaning  that  during  the  execution  of  a task  the  code  stays  in  a 
given  host  until  the  task  is  completed.  Although  the  code  of  the  agent  can  be  transported  from  one  host  to 
another,  it  is  transferred  without  carrying  any  state.  On  the  other  hand,  mobile  agents  are  software  programs  that 
can  be  instructed  to  halt  its  execution  on  a given  host,  travel  with  the  current  state  to  other  participating  hosts 
across  a network  and  continue  execution  there.  They  can  also  be  retracted  back  to  its  original  host  if  necessary. 
Mobile  agents  offer  certain  capabilities  that  are  not  readily  available  with  stationary  agents. 

What  Are  Aglets.  An  aglet  is  a software  agent  implemented  with  Java  using  the  IBM  Aglets  Workbench, 
developed  at  the  IBM  Tokyo  Research  Laboratory  [LaOs98,  URLOl].  The  term  aglet  is  really  a combination  of 
two  words:  agent  and  applet.  An  agent  is  in  general  referred  to  as  an  autonomous  software  program  that  acts  as  a 
representative  of  some  user,  and  an  applet  is  a Java  class  (a  piece  of  Java  code)  that  can  be  transported,  at  the 
user’s  request,  from  a web  server  to  a web  client  and  then  executes  on  the  client  machine  as  a thread  within  a 
browser.  Although  aglets  can  be  stationary,  the  main  objective  of  the  Aglets  Workbench  is  to  allow  users  to 
develop  mobile  agents  and  the  majority  of  aglets  are  indeed  mobile  agents.  For  this  reason,  aglets  are  often 
referred  to  as  Java-based  mobile  agents.  Aglets  have  three  important  properties:  behavior,  state,  and  location. 
They  differ  from  applets  in  that  applets,  once  downloaded  and  instantiated,  begin  execution  in  the  client  machine 
until  all  specified  tasks  are  completed,  while  aglets  can  travel  (either  reactively  or  proactively)  with  their  current 
state  from  a participating  host  to  another  participating  host  across  a network,  including  possibly  returning  to  its 
origin  and  distribute  their  tasks  among  all  the  hosts  in  the  itinerary.  Aglets  also  differ  from  servlets  in  that  the 
latter  are  uploaded  from  a client,  or  from  some  host  in  the  network,  to  a Java-enabled  web  server  and  get  executed 
in  the  server  during  their  entire  lifetime.  The  relationship  between  a web  browser  and  a web  server  is  basically  a 
client-server  model.  Each  running  aglet  server,  however,  is  a peer-to-peer  process. 

Specific  Aglets  for  Educational  Enhancement.  There  are  many  good  reasons  for  using  mobile  agents. 
Seven  of  them  are  presented  in  [LaOs99],  including  the  potential  for  reducing  network  traffic  and  overcoming 
network  latency,  the  ability  to  encapsulate  protocols,  execute  asynchronously,  and  adapt  dynamically,  and  the 
capability  of  lending  themselves  to  heterogeneous,  robust,  and  fault-tolerant  processing.  Since  aglets  are  in 
general  mobile  agents,  they  enjoy  most  of  the  benefits  that  can  be  offered  by  mobile  agents.  It  should  be 
mentioned,  however,  that  very  few  problems,  if  any,  that  can  be  handled  using  aglets  cannot  be  solved  using 
traditional  stationary  agents  or  mobile  code  such  as  applets  and  servlets.  The  question  of  which  type  of  software 
should  be  used  certainly  depends  on  several  factors:  the  nature  of  the  application,  the  latency  of  the  network,  the 
connectivity  of  the  host,  the  relative  size  of  code  transfer  vs.  data  transfer,  and  the  frequency  of  such  transfer,  just 
to  name  a few.  Mobile  aglets  have  many  applications  on  the  Internet  [KiZi97,  KoGr99].  In  the  following,  we 
briefly  describe  some  specific  aglets  that  need  to  be  developed  in  order  to  achieve  our  goal  to  enhance  distance 
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education.  These  include  a message  broadcast  aglet,  homework  assignment  distribution  aglet,  homework 
assignment  collection  aglet,  grade  report  aglet,  progress  monitoring  aglet,  file  transfer  aglet,  and  communication 
aglet  with  textual,  graphical,  audio  and  video  support. 

The  message  broadcast  aglet  is  for  the  instructor  to  deliver  notices,  announcements,  and/or  other  urgent  messages 
to  all  students  simultaneously  in  a timely  fashion,  one  aglet  for  each  student.  Upon  receiving  such  an  aglet,  the 
student  should  respond  to  the  aglet  indicating  the  receipt  of  such  a message.  This  aglet  then  carries  the  response 
back  to  its  originating  host  and  stores  the  response  information  in  a log  file  to  let  the  instructor  know  if  all 
students  have  received  such  a message.  The  homework  assignment  distribution  aglet  works  in  a way  similar  to 
the  message  broadcast  aglet,  except  that  the  aglet  carries  the  assignment  to  the  student’s  host  in  a form  of  files  and 
store  the  assignment  to  a file  or  files.  The  announcement  of  the  assignment  can  be  sent  using  the  message 
broadcast  aglet  at  any  convenient  time.  The  homework  assignment  collection  aglet  is  sent  to  the  student’s  host  to 
automatically  collect  the  assignment  conducted  by  the  student  when  the  assignment  is  due,  instead  of  asking  the 
student  to  submit  their  assignments  to  some  specified  server.  One  way  of  making  this  automatic  collection 
possible  is  to  specify  the  use  of  a unified  filename.  In  other  words,  all  students  should  perform  their  work  or  store 
their  finished  work  in  a file  and  use  the  same  filename  for  a given  assignment.  At  the  instructor’s  host,  a directory 
can  be  created  for  each  student  and  for  each  assignment.  This  directory  is  used  to  store  the  work  collected  from 
the  student.  The  communication  aglet  is  for  message  exchanges  among  students  and  between  students  and  the 
instructor.  To  initiate  communications,  the  user  must  have  one  of  such  aglets  ready  for  use  at  his/her  own  host. 
It  is  not  required,  however,  that  the  responding  party  preinstall  that  aglet  beforehand  since  the  aglet  that  initiates 
the  communication  can  carry  the  necessary  functionality  to  the  other  party.  Therefore,  each  student  can  have 
his/her  own  favorite  communication  aglet,  if  one  available  to  them.  This  is  an  obvious  advantage  over  the  current 
communication  systems  such  as  NetMeeting,  ICQ,  or  Messenger  because  the  use  of  them  requires  pre-installation 
of  the  same  software  by  each  participating  user  before  a conversation  can  begin.  The  tradeoff  here  is  the  initial 
transfer  of  the  aglet  itself  over  the  network  at  each  initiation  of  the  conversation.  A prototype  communication 
aglet  that  allows  for  textual  and  handwriting  message  exchanges  has  been  developed  in  [ChTuOl].  In  addition  to 
the  aglets  mentioned  above,  a grade  report  aglet  and  a progress  monitor  aglet  can  also  be  designed  to  allow 
students  to  obtain  their  grades  at  the  end  of  the  semester  and  to  allow  the  instructor  to  monitor  the  progress  of  the 
student  during  each  homework  assignment,  respectively. 

Conclusion.  The  Internet  has  become  a ubiquitous  environment  for  resource  sharing,  data  collection,  data 
distribution,  information  exchange,  and  global  communications  nowadays.  Most  of  the  software  agents  currently 
in  use  on  the  Internet,  however,  are  stationary.  In  this  paper,  we  proposed  using  aglets  to  enhance  distance 
learning  and  administration.  Several  specific  aglets  for  educational  use  have  been  addressed.  These  aglets  offer 
certain  capabilities  that  are  not  readily  available  with  stationary  agents 
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Abstract;  Web-based  education  systems  are  becoming  increasingly  popular  in  recent  years; 
however,  most  of  the  systems  developed  so  far  use  static  hypermedia  as  the  manner  of 
representing  contents.  Ifs  a challenge  to  develop  advanced  Web-based  education 
applications  that  can  offer  both  adaptivity  and  intelligence.  This  paper  will  review  some 
related  works  in  this  area  and  present  a design  of  advanced  Web  based  learning  system  that 
support  adaptive  presentation,  adaptive  navigation  and  adaptive  help. 

Keywords:  adaptive  hypermedia,  adaptive  presentation,  adaptive  navigation,  adaptive  help, 
User  Model 


1 Introduction 

Advantages  of  web-based  education  systems  which  are  becoming  increasingly  popular  in  recent  years 
have  been  discussed  in  many  papers.  Learning  resources  are  becoming  unprecedented  abundance  in  the  web. 
This  is  just  the  start  and  the  trend  is  going  to  last  for  a long  time. 

In  spite  of  all  the  popularities,  the  uses  of  web-base  educational  applications  are  still  at  a very  low  level. 
According  to  the  authors  of  [1],  web-based  education  systems  can  be  classified  into  three  levels:  Base  level. 
State-of-the-art  level  and  Research  level.  The  majority  of  education  systems  currently  being  used  are  either 
base  level  or  State-of-the-art  level;  multimedia  resources,  structured  contents  and  multiple  navigating  ways  are 
the  distinct  features  of  these  systems.  In  these  systems,  however,  contents  are  presented  using  static 
hypermedia.  The  advanced  web-based  educational  systems,  which  are  in  research  level,  inherit  some  merits 
from  intelligent  tutoring  system  (ITS),  hypermedia  system  and  adaptive  system.  Their  distinct  features  are 
adaptivity  and  intelligence.  In  this  article,  we  refer  the  former  as  the  first  generation  web-based  education 
systems  and  the  later  as  the  next  generation  systems.  In  fact,  researches  on  web-based  instruction  theory, 
instruction  model  and  advanced  instruction  tools  or  platforms  lagged  the  progress  of  the  Internet  technologies. 

Most  of  web-based  education  systems  in  research  level  have  not  yet  found  their  way  to  real-world  virtual 
classroom.  There  are  only  a few  being  used  in  relatively  small  classes.  There  have  been  many  researchers 
actively  involved  in  this  challenging  area  since  1996.  In  [2],  adaptive  and  intelligent  technologies  applied  in 
Web-based  education  systems  were  reviewed  and  grouped  by  its  origin  as  follows. 

• ITS  technologies  in  Web-based  education  curriculum  sequencing,  problem  solving  support 
technologies. 

• Adaptive  hypermedia  technologies  in  web-based  education:  adaptive  presentation  and  adaptive 
navigation  support. 

• Web-inspired  technologies  in  Web-based  education:  these  technologies  had  little  use  in  educational 
systems  before  the  Internet  age.  In  [2],  such  technology  is  called  student  model  matching  because 
the  essence  of  this  technology  is  the  ability  to  analyze  and  match  student  models  among  many 
students  at  the  same  time.  Two  examples  of  student  model  matching  were  identified,  which  were 
called  adaptive  collaboration  support  and  intelligent  class  monitoring. 

Intelligent  problem  solving  support  technologies  use  AI  extensively.  These  technologies  are  closely 
related  to  application  domains  and  are  difficult  to  generalize,  thus  they  will  stay  on  research  level  for  a long 
time.  On  the  other  hand,  adaptive  presentation,  adaptive  navigation  support  and  model  matching  technologies 
are  relatively  easy  to  implement  in  web,  and  general  authoring  tools  can  be  developed  and  deployed  in  large 
mount  of  education  systems. 

This  paper  is  organized  as  follows:  Section  2 gives  a brief  review  of  previous  researches  on  adaptive 
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presentation,  adaptive  navigation  support  and  model  matching  in  web-based  education  systems.  Section  3 
introduces  some  function  zones  in  web-base  course  pages.  Section  4 describes  the  design  procedure  of  web- 
based  course  which  utilize  the  above  technologies.  These  procedures  can  be  generalized  and  be  made  into 
software  tools  used  in  all  web-based  courses. 


2 Quick  Review  of  Related  Work 

The  first  comprehensive  review  on  adaptive  hypermedia  system  can  be  found  in  [3].  In  [4], [5],  the  author 
continued  tracing  research  progress  in  this  area.  In  [3],  Brusilovsky  distinguished  two  adaptive  methods  in 
hypermedia  systems:  adaptive  presentation  and  adaptive  navigation  support. 

The  following  techniques  for  adaptive  presentation  were  mentioned: 

• Conditional  text 

• Stretch  text 

• Fragment  variants 

• Page  variants 

• Frame-based  techniques 

The  techniques  for  adaptive  navigation  support  listed  in  [Brusilovsky  1996]  [De  Bra  1998]  are  as  follows: 

• Direct  guidance 

• Link  sorting 

• Link  hiding,  link  disabling,  link  removal 

• Link  annotation 

• Map  adaptation  (provide  personalized  overview) 

Most  of  adaptive  hypermedia  are  education  hypermedia.  Such  as  [6], [7], [8], [9], [10],  which  demonstrate 
some  of  the  technologies  mentioned  above. 

Research  on  general  model  of  AHS  and  its  formal  descriptions  can  be  found  in  [11], [12].  In  [11]],  An 
adaptive  hypermedia  application  is  defined  as  a 4-tuple  <DM,  UM,  TM,  AE>  where  DM  is  a domain  model, 
UM  is  a User  Model,  TM  is  a teaching  model,  and  AE  is  an  adaptive  engine. 

Some  AHS  authoring  tools  such  as  Interbook  [13],  AHA[14]  and  NetCoach[15]  have  been  developed  in 
recent  years.  InterBook  is  a tool  for  authoring  and  delivering  adaptive  electronic  textbooks  on  the  World  Wide 
Web.  The  main  feature  of  InterBookis  is  that  it  provides  a technology  for  developing  electronic  textbooks  from 
a plain  text  to  a specially  annotated  HTML.  AHA  is  an  open  Adaptive  Hypermedia  Architecture  whose  core  is 
an  engine  that  maintains  a user-model  based  on  knowledge  about  concepts.  The  content  of  a page  can  be 
adapted  by  means  of  fragment  variants.  The  links  are  annotated  by  changing  the  color  of  the  link  anchor. 
NetCoach  is  designed  to  enable  authors  to  develop  adaptive  learning  courses  without  programming 
knowledge.  It  provides  web-based  tools  for  the  teacher  to  author  course  structure,  exercises  and  to  customize 
user  interface. 

As  for  adaptive  collaboration  support,  an  example  in  this  area  can  be  found  in  [16].  This  paper  reported 
an  Intelligent  Helpdesk  system  which  integrated  two  cognitive  tools:  CPR  (Cooperative  Peer  Response)[17] 
and  Phelps(Peer  Help  System).  The  former  is  a discussion  forum  collaboration  tool  that  is  customizable  to  a 
particular  course,  providing  peer  help  and  FAQ  facilities  via  the  web.  Every  entry  in  the  discussion  forum  and 
FAQ  were  indexed  by  a class-specific  category  or  topic.  Phelps  supports  student  as  they  are  learning,  offers 
assistance  in  finding  peer  helpers  by  model  matching  when  required,  and  mediates  communication  on  related 
topics. 

In  this  paper  we  will  analyze  learning  environment  in  web-based  course  systematically,  then  propose  a 
design  scheme,  which  integrate  adaptive  content  and  adaptive  help. 

3 Function  zones  in  web-based  course 

In  perspective  of  function,  a page  in  the  most  of  web-based  course  can  be  divided  into  three  zones: 

• Function  zone:  most  of  learning  tools  are  shown  in  this  zone.  It  looks  like  menu  or  tool  bar  in 
application  software.  The  content  in  this  area  is  almost  the  same  in  all  pages  of  this  course. 

• Navigation  zone:  it  helps  users  to  learn  about  the  structure  of  the  web-based  course  or  to  guide  them 
through  it.  Usually  it  consists  of  structuring  index  table  and  some  navigation  buttons. 
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• Content  zone:  It  comprises  material  (content,  examples,  exercises  etc.),  which  is  main  part  of  a 
page.  Some  navigation  buttons  are  also  included. 

These  zones  are  separate  in  frames,  and  every  zone  is  an  independent  html  file.  They  can  also  appear  in  a 
single  html  file  with  table  to  separate  them. 

In  the  above  adaptive  hypermedia,  the  adaptivity  is  mainly  embodied  in  navigation  zone  and  content 
zone.  For  example,  in  Interbook,  adaptive  navigation  support  lies  in  annotation  in  index  table  and  direct 
guidance  button  in  content  zone.  In  AHA,  adaptive  presentation  is  shown  in  content  zone  hyperlinks  in 
content  zone  also  included. 

Besides,  we  should  see  that  tools  in  function  zone  are  important  parts  of  the  web-based  course 
environment.  These  tools  can  be  content  table,  search  tool,  bookmark,  help  tool,  communication  tool,  etc. 
Actually,  many  other  tools  can  be  easily  added  to  the  function  zone.  Most  of  them  are  independent  and  can  be 
designed  and  implemented  individually.  When  needed,  they  can  be  linked  into  function  zone.  So,  in  this 
paper,  we  will  not  discuss  design  of  such  tools.  What  we  concerned  is  only  adaptive  help  tools  that  share  the 
same  User  Model  and  domain  model  with  adaptive  hypermedia. 

4 Designs. 

Our  schema  is  a user-model  centered  integrated  learning  system  feature  adaptive  presentation,  adaptive 
navigation  support  and  adaptive  help.  When  a course  is  delivered  via  the  net,  the  system  structure  can  be 
shown  as  Fig.l. 


Fig  1 System  Structure  of  Adaptive  Learning  System 

The  following  are  design  steps  of  an  online  course  for  a teacher: 

1 . Extract  set  of  concepts  (SCs)  from  the  given  domain;  set  of  goals  (SGs)  and  set  of  background 
knowledge  (SB)  are  also  specified  by  the  teacher. 

2.  Design  curriculum  structure.  Usually,  each  curriculum  is  divided  into  chapters,  sections  and  points 
of  knowledge;  these  elements  are  grouped  in  a tree  structure.  Each  node  of  the  tree  has  a 
corresponding  page.  These  pages  are  called  “virtual  pages  (VPs)”  since  they  are  not  real  html  files, 
but  only  comprise  structure  description  of  a page. 

3.  Construct  knowledge  fragments  (KFs).  They  can  be  done  using  web  page  authoring  tools.  These 
fragments  can  contain  all  kinds  of  media  as  well  as  hyperlinks.  They  can  be  explanations  of 
concepts  or  examples;  each  concept  can  be  presented  by  different  explanations  and  different 


327 


examples.  Every  fragment  has  a set  of  attributes  (concept,  role,  object  etc.). 

4.  Virtual  Page  (VP)  consists  of  knowledge  fragments  (KFs).  In  order  to  make  these  pages  show 
different  contents  to  different  user,  construction  rules  must  be  applied  to  these  pages.  These  rules 
will  make  sure  that  different  user  with  different  background  or  different  learning  goal  will  see 
different  contents. 

5.  Each  virtual  page  will  have  its  own  sets  of  prerequisite  concepts  and  output  concepts.  This 
information  will  be  used  to  implement  adaptive  annotation  and  direct  navigation. 

6.  There  will  be  one  set  of  tests  (STs)  for  each  of  the  virtual  page.  These  tests  will  be  used  to  analyze 
the  mastering  state  of  users  on  these  concepts. 

The  data  parts  of  web  based  learning  system  are  made  up  of  SCs,  SGs/SB,  VPs,  KFs,  STs  and  User 
Model.  User  Model  consists  of  user’s  knowledge  level,  learning  goals  as  well  as  their  background.  User’s 
knowledge  level  is  an  overlay  model.  For  each  domain  model  concept,  an  individual  overlay  model  stores 
some  value  which  is  an  estimation  of  the  user  knowledge  level  of  this  concept.  This  can  be  just  a binary  value 
(known-not  known),  a qualitative  measure  (good-average-poor),  or  a quantitative  measure,  such  as  a 
probability  that  the  user  knows  the  concept.  An  overlay  model  of  user  knowledge  can  be  represented  as  a set  of 
pairs  “concept- value”,  one  pair  for  each  domain  concept[3]. 

An  adaptive  engine  will  be  used  to  generate  web  page  dynamically  based  on  values  in  the  User  Model 
and  other  data  listed  above  when  the  curriculum  is  delivered  via  web;  the  pages  demonstrate  the  features  of 
adaptive  content  and  adaptive  navigation.  When  a user  access  a virtual  page  (VP),  an  adaptive  engine  will  act 
as  following: 

Generation  of  content  zone: 

1.  Generate  contents  of  page  (PI):  Content  of  a page  consists  of  KFs.  The  ways  to  combine  these  KFs 
are  decided  by  page  construction  rule.  Goal  value  and  background  in  User  Model  will  be  used  as 
rule  is  executed. 

2.  Analyze  hyperlinks  in  PI;  generate  a new  page  (P2)  using  appropriate  annotation.  Knowledge  level 
in  the  User  Model  will  be  used  in  this  step  to  decide  how  hyperlinks  are  displayed.  For  example,  if  a 
user  is  not  prepared  to  visit  a page,  the  link  for  that  page  will  be  hidden  or  marked  in  red  color; 
Links  to  mastered  contents  will  be  gray  out;  Links  to  accessible  pages  will  be  displayed  in  green. 

Generation  of  navigation  zone: 

Every  page  in  VPs  has  a set  of  prerequisite  concepts  and  ouputt  concepts;  this  makes  it  easy  to  mark  the 
index  table  of  curriculum  contents  separately  based  on  the  knowledge  level  in  User  Model.  User  can  set  the 
way  of  marking  index  table.  Index  table  can  be  marked  in  different  colors  or  different  icons.  It’s  a good 
practice  to  keep  the  style  of  index  table  annotation  in  line  with  the  marking  of  hyperlinks  in  the  content  zone. 
The  button  for  next  step  (direct  navigation  indicator)  can  be  generated  based  on  information  in  the  User  Mode, 
also. 

The  final  page  a user  can  see  is  generated  after  the  processing  of  all  information  mentioned  above. 

Maintenances  of  User  Model 

The  User  Model  can  be  maintained  based  on  the  way  user  browsing  the  pages  (including  pause  time)  as 
well  as  scores  of  tests  on  these  contents.  The  later  will  be  reliable  source  for  judging  user’s  mastering  of 
related  concept. 

Adaptive  help 

Information  of  this  kind  of  Domain  Model  and  User  Model  can  be  used  to  generate  adaptive  help  as  well 
as  make  class  contents  and  navigation  adaptive.  Help  system  is  a tool  which  resides  in  the  function  zone  (on  a 
page).  The  help  system  is  a discussion  forum;  each  topic  of  the  forum  is  indexed  by  concepts  in  Domain 
Model.  When  user  wants  to  submit  a question,  the  user  must  select  one  or  more  concepts  (from  the  list).  The 
system  can  use  the  user  input  to  make  sure  that  all  FAQ  and  contents  of  discussion  forum  are  correspond  to 
the  (user  selected)  concepts.  This  approach  makes  help  system  more  adaptive.  When  a user  click  the  help 
button  while  viewing  a page,  the  help  system  will  act  as  following: 

The  system  know  the  page  from  which  the  help  is  required,  and  then  it  will  know  the  concepts  referred 
to  on  that  page.  The  system  can  generate  a help  page  based  on  that  information. 
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The  indexes  of  prerequisite  knowledge  for  these  concepts  are  displayed  on  top  of  the  generated  help  page. 
User  can  visit  that  prerequisite  knowledge  by  clicking  those  links. 

Answers  to  questions  on  the  concepts  are  also  displayed  on  the  help  page.  These  answers  are  extracted 
from  FAQ  and  discussion  forum. 

User  can  resubmit  question  if  the  desired  help  information  is  not  found  on  the  returned  page. 

Help  information  from  this  help  system  is  generated  according  to  the  position  of  current  question  as  well 
as  knowledge  level  of  the  User  Model.  The  system  utilizes  entries  indexed  by  concepts  of  Domain  Model  and 
User  Model  to  generate  personalized  help  information. 

Based  on  what  we  presented  above,  there  are  a lot  of  works  need  to  be  done  to  design  Web  based  adaptive 
learning  systems.  Our  goal  is  to  design  a set  of  tools  that  can  be  used  by  teachers  to  fulfill  all  those  steps 
mentioned  above.  These  tools  include: 

• Tools  of  define  domain  concepts,  learning  goal  and  background  knowledge 

• Tools  of  define  class  structure 

• Tools  of  define  rules  of  page  construction 

• Tools  of  define  prerequisite  concepts  and  output  concepts 

• Tools  of  create  exercises 

• Adaptive  engine 

• Help  engine 

• Discussion  group  on  indexes  for  domain  concepts 

The  procedures  and  tools  mentioned  above  are  general  purpose,  they  can  be  used  on  all  kind  of  online 
course  design  and  implementation. 

5 Progress  of  the  work 

There  have  being  more  than  thirty  online  universities  in  China,  and  thousands  of  curriculums  are  offered 
on  line.  There  are  more  than  fifty  online  curriculums  being  constructed  in  our  university.  The  purpose  of  our 
work  is  to  formalize  the  creation  of  online  curriculums  and  to  improve  online  education  to  a higher  level.  The 
distinguish  feature  of  our  system  is  adaptive  and  adaptable.  Our  work  is  still  in  the  early  phase;  we  believe  that 
our  work  will  be  beneficial  to  distance  education  . 
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1.  Introduction 

In  21st  century,  information  and  knowledge  are  the  motive  power  changing  our  society  and  the  foundation  cf  our  country’s 
competitive  power.  Information-using  ability  becomes  a criterion  to  decide  the  competitive  power,  and  the  quality  of  knowledge 
and  information  which  human  and  society  possess  determines  the  quality  of  human  life.  Accordingly,  the  Ministry  of  Education  and 
Human  resources  development  decided  on  the  use  of  ICT  (Information  and  Communication  Technology)  in  education  for  students  to 
obtain  and  practically  use  the  fundamental  knowledge  about  information  making,  processing,  treating,  analyzing,  ejploring,  etc. 
Therefore,  the  M inistry  encouraged  every  school  to  use  1 0%  of  class  time  at  least  to  teach  the  knowledge  about  information  in  an 
instruction  & learning  procedure.  Consequently,  schools  try  to  reform  the  classes  according  to  the  request  of  the  Ministry  and  it 
produces  a change  of  educational  environment.  As  one  of  these  efforts,  various  instruction-learning  procedure  ideas  are  being 
developed  in  schools  at  present. 

Many  contents  about  education  are  constructed  with  web-based  type,  but  the  lesson  plans  have  been  written  out  with  Word- 
processor.  The  Ministry  recommends  teachers  to  make  a lesson  plan  using  a Powerpoint.  It  is  true  that  many  lesson  plans  have  been 
made  and  uploaded  with  board-type,  so  they  are  not  saved  in  database  in  accord  with  educational  curriculum.  For  these  reasons,  in 
this  study,  teaching- learning  plans  were  designed  as  a form  using  ICT  in  education  and  developed  as  web-based  electronic  teaching- 
learning  unit  system  that  is  available  to  prepare  and  use  in  web. 

The  contents  and  methods  are  following:  First,  the  electronic  teaching-learning  unit  using  ICT  was  designed  based  on  the  existing 
various  types  of  plans.  Second,  the  system  which  teachers  can  form  the  teaching- learning  plans  in  a web-based  online  situation  was 
reflected  to  the  total  construction  of  this  system.  Third,  according  to  the  7th  educational  curriculum  and  the  type  of  the  teaching- 
learning  plans  using  ICT,  the  database  system  was  developed  for  an  effective  use  of  ICT  materials  when  one  draws  out  the  procedure 
plans  with  an  electronic  teaching-learning  unit  system.  Finally,  new  electronic  teaching- learning  unit  system  was  applied  in  real 
school  fields  and  through  the  questionnaire  analyses,  the  effects  and  improving  points  of  this  system  were  found. 

2.  Web-based  Electronic  Teaching-Learning  Unit  Systems 


Design 

The  whole  structure  of  the  electronic  teaching-learning  unit  system  was  constructed  as  database  to  make  out  the  lesson  plan  and  a 
module  to  manage  the  completed  lesson  plans.  The  database  for  making  a lesson  plan  was  designed  as  3 parts,  curriculum  DB,  ICT 
lesson  plan  DB  and  ICT  materials  DB.  Organically  related  3 Databases  composed  of  RDBMS.  A Client  is  managed  by  dividing  the 
electronic  teaching-learning  unit  system  and  a storehouse  of  ICT  materials  and  these  2 systems  are  managed  with  organic  relationship. 
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Implementation 


To  realize  this  system,  Linux  server  and  MySQL  were  used  and  in  a system  driving  method,  PHP  and  JAVA  script  were  utilized 
based  on  HTML.  This  shows  that  many  useful  programs  can  be  made  with  GNU  Software  suggested  by  GNU  Project  and  utilized  in 
real  fields.  The  system  is  composed  of  the  ICT  electronic  teaching-learning  unit  system  and  a storehouse  of  ICT  materials.  The  first 
screen  is  following.  The  teaching- learning  unit  system  and  the  storehouse  system  have  an  interactive  and  cooperative  relationship. 


[Figure  2]  ICT  electronic  teaching  plan 


[Figure  3]  ICT  materials 


The  completed  lesson  plan  using  this  system  is  like  below.  The  special  point  of  this  teaching  plan  is  to  make  out  a lesson  plan  on  the 
web  easily  and  that  there  is  a list  of  ICT  materials  that  are  supposed  to  use  during  the  class.  These  multi-media  materials  are  linked  so 
that  teachers  can  make  use  of  directly.  In  addition,  all  materials  linked  in  a lesson  plan  are  stored  in  database,  so  they  can  use  only 
materials  and  it  is  easy  to  share  and  search  them. 
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[Figure  4]  Example  of  completed  ICT  electronic  teaching  plan 

3.  Advantages 

Some  elementary  school  teachers  living  in  Inchon,  Korea  actually  used  this  electronic  teaching  unit  system  to  make  out  their 
lesson  plans.  After  using  this  system  in  the  class  time,  most  of  teachers  showed  positive  responses. 

The  result  of  application  of  this  system  is  as  follows. 


The  effect  of  making  electronic  teaching-learning  plan 

1)  Web-based  system  of  the  electronic  teaching- learning  plan  improved  convenience  to  make  the  lesson  plan  through  making 
database  of  current  educational  curriculum  of  an  elementary  school 

2)  Making  the  lesson  plan  and  using  ICT  materials  by  using  this  electronic  teaching- learning  unit  system  are  much  easier  than  by 
using  a word-processor  or  powerpoint.  The  system  maximized  the  variety  and  utilization  of  ICT  materials  necessary  to  make  a 
lesson  plan 

3)  As  all  teachers  made  out  the  lesson  plans  by  themselves  using  the  electronic  teaching-learning  unit  system,  ICT  application 
ability  in  education  was  improved  and  a lesson  model  using  ICT  was  established 
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The  effect  of  utilizing  electronic  teaching-learning  unit  system. 

1)  The  electronic  lesson  plan  is  stored  at  database  in  server,  so  it  makes  possible  to  accumulate  different  types  of  ICT  using  lesson 
plans  though  they  are  same  lesson.  Additionally,  it  makes  available  to  explore  ICT-using  lesson  plans  according  to  the  keyword, 
chapter  and  the  ICT-using  type. 

2)  By  considering  both  circumstances  of  teachers’  study  of  teaching  aids  and  actual  lesson  situation,  they  can  select  either  ‘watching 
the  lesson  plan’  or  ‘doing  lesson’  modes  when  they  use  electronic  teaching-learning  unit  system. 

3)  It  is  a good  try  for  “Doing  lesson”  but  it  needs  improvement  of  interface. 

The  effect  of  contents  library 

1)  It  is  possible  to  link  and  upload  the  various  ICT  materials  and  it  can  accumulate  the  huge  amount  of  ICT  materials 

2)  ICT  referential  materials  are  not  fixed  but  every  new  material  is  added  automatically.  So,  lesson  plans  and  a storehouse  of 
materials  were  interactively  related  to  make  this  system 

4.  Discussions 

The  followings  are  some  suggestions  for  the  growth  of  this  system  that  was  developed  for  activation  of  ICT  -using  education  in 
Korea: 

1)  This  system  will  be  shared  by  many  schools  so  that  more  lesson  plans  can  be  provided  as  well  as  continuously  be  updated  new 
lesson  plans. 

2)  This  system  will  be  modified  for  teachers  to  change  the  procedure  steps  from  3 fixed  steps,  introduction,  development  and 
adjustment  into  more  3 steps  as  well  as  the  names  of  steps  according  to  the  characteristics  of  each  subject 

3)  When  the  actual  lessons  proceed,  teachers  will  be  able  to  choose  the  time  that  they  want  to  show  ICT  materials;  therefore  they 
will  designate  the  time  in  “Doing  lesson”  module  of  this  system  when  they  make  out  the  lesson  plan. 

4)  By  using  FLASH  not  the  JAVA  SCRIPT  for  a tool  of  making  a “Doing  lesson”  module,  more  polished  interfaces  will  be 
needed  and  the  “Doing  lesson”  module  will  have  not  one  frame  but  different  and  various  frames  of  module  according  to  each 
subject. 

5)  This  system  will  be  modified  in  English  and  become  a package  to  use  in  education  practically. 

5.  Conclusion 

Developed  web-based  system  of  an  electronic  teaching- learning  unit  is  practically  used  in  school  fields  now.  It  is  worth  an 
alternative  proposal  of  the  PowerPoint -type  plans  that  the  Ministry  suggests.  The  results  after  the  application  of  this  system  are 
following. 

Web-based  system  of  an  electronic  teaching-learning  unit  is  designed  to  use  database  in  web  considering  accumulation  and  use  of 
the  data  and  convenience.  It  is  managed  through  database  not  only  all  learning  of  7th  educational  curriculum,  but  also  ICT  materials 
which  are  the  most  necessary  to  the  class,  so  it  is  convenient  to  control  the  procedure  plans  easily. 

To  form  the  procedure  plans  easily  for  users,  a module  was  devised  and  to  make  a continually  developing  system,  the  materials  and 
the  teaching-  learning  units  were  organically  connected.  The  development  of  this  system  realized  the  maximization  of  the  teaching- 
learning  plans'  share,  practical  use  and  storage.  In  addition,  it  will  be  generalized  in  real  school  fields  and  contribute  the 
informatization  of  school. 
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Abstract:  With  the  advance  in  Internet  technology,  “e-leaming”  becomes  an  active  research 
topic.  To  make  ©learning  an  effective  learning  mode,  both  technology  and  pedagogy  are 
equally  important.  However,  existing  ©learning  platforms  and  tools  usually  focus  on  the 
technology  aspect  without  much  investigation  on  the  pedagogical  issues.  In  fact,  there  is  still  a 
big  gap  between  pedagogy  and  technology.  The  SPACE  Online  Universal  Learning  (SOUL) 
platform  is  designed  to  fill  this  gap  and  provide  an  effective  e -Learning  platform  for  e-course 
providers.  In  particular,  the  content  engineering  agent  of  the  platform  is  designed  for  e-course 
development,  which  is  based  on  the  Task-Based  Learning  (TBL)  curriculum  development 
framework.  In  this  paper,  we  discuss  the  details  of  this  content  engineering  agent  with  the 
main  focus  on  the  Task-Based  Learning  framework.  As  a remark,  the  content  engineering 
agent  has  already  been  integrated  in  the  SOUL  platform,  which  is  being  used  by  more  than 
17,000  students. 


1.  Introduction 

With  the  growth  in  Internet  and  the  advance  in  Internet  technology,  “e-learning”  provides  an  alternative  learning 
model  for  teachers  and  students  other  than  traditional  classroom  teaching  and  distance  learning.  And  o- learning 
has  become  the  focus  of  research  by  education  professionals  (Leadbeater,  1999).  A lot  of  o-leaming  platforms, 
such  as  BlackBoard  (Blackboard,  2002)  and  (WebCT,  2002),  have  been  developed  by  different  researchers  and 
vendors  (see  also  Gertner  & VanLehn,  2000,  Zhou  et  al,  1996).  However,  these  existing  platforms  usually  focus 
on  the  technology  aspect  without  paying  much  attention  to  other  issues  such  as  pedagogical  concerns  and  quality 
control.  These  issues  are,  in  fact,  critical  in  the  success  of  ©learning  (Harasim,  1999,  Fumell  et  al,  2001).  The 
following  summarizes  some  of  the  drawbacks  of  the  existing  platforms: 

• Big  gap  between  technology  and  teaching  pedagogy:  Existing  platforms  seldom  emphasize  on  this 
aspect.  Consequently,  the  lack  of  pedagogy  makes  these  platforms  perform  like  repositories  of  course 
materials.  They  just  allow  students  to  retrieve  materials  from  the  platform.  As  a result,  they  are  still  far 
away  from  traditional  face-to-face  teaching,  not  to  mention  replacing  it  in  the  near  future. 

• Quality  control:  Existing  platforms  usually  do  not  have  quality  control  in  e-course  curriculum 
development  and  have  poor  feedback  mechanism  to  evaluate  the  effectiveness  of  the  course  materials. 

• Poor  personalization:  Truly  personalization  of  individual  student’s  learning  path  based  on  individual’s 
studying  pace  and  knowledge  background  is  not  implemented  in  existing  platforms.  Specific  advices  for 
individual  students  regarding  personal  learning  problems  are  missing. 

• User-friendliness  (usability):  Teachers  are  required  to  take  a long  time  to  learn  to  use  the  platform  and 
usually  a team  of  technical  support  staff  is  necessary  for  ensuring  smooth  operations  performed  by  the 
teachers.  And  communicating  the  instructional  requirements  between  teachers  and  technical  teams  is  the 
bottleneck  of  e-course  development  provided  the  communication  is  possible  at  all. 

In  order  to  address  these  issues,  the  School  of  Professional  and  Continuing  Education  (SPACE)  of  the  University 
of  Hong  Kong  set  up  a team,  called  the  SPACE  Online  Universal  Learning  (SOUL)  Project  team,  in  1998.  The 
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aim  of  the  project  is  to  provide  online  support  for  educational  purposes;  to  develop  SPACE  online  support 
courses  in  both  Hong  Kong  and  the  Mainland  China;  and  to  carry  out  researches  related  to  online  learning.  The 
SOUL  platform  is  the  major  product  of  the  project  group. 

The  SOUL  platform  is  based  on  the  system  architecture,  called  the  PowerEdBuilder , which  will  be  described  in 
the  ne.xt  section.  Different  components  of  PowerEd Builder  will  tackle  different  issues  regarding  online  learning 
and  will  work  together  as  a single  platform  to  provide  a comfortable  and  user-friendly  environment  to  teachers, 
students  and  administrators.  In  particular,  the  Content  Engineering  Agent  together  with  the  Smart  Tutor 
component,  of  the  PowerEdBuilder  are  designed  to  provide  pedagogical  support  for  teachers  to  create  o course 
as  well  as  tailor-made  personalization  for  students  to  learn  the  material.  For  descriptions  of  different  components 
of  the  PowerEdBuilder,  one  can  refer  to  the  publications  of  the  SOUL  project  team  [7].  In  this  paper,  we  will 
focus  on  the  Content  Engineering  Agent  and  describe  the  Task-based  Learning  framework  of  this  component. 

The  rest  of  the  paper  is  organized  as  follows.  Section  2 provides  an  overview  on  the  SOUL  platform.  The  details 
of  the  Content  Engineering  Agent,  with  focus  on  the  Task-based  Learning  framework,  will  be  given  in  Section  3. 
Section  4 will  contain  the  discussion  and  conclusion. 

As  a remark,  SPACE  is  one  of  the  leading  adult  education  providers  in  Hong  Kong  providing  life-long  learning 
for  the  public.  The  number  of  students  registered  with  SPACE  programs  is  more  than  60,000  and  the  number  is 
expected  to  grow  in  a tremendous  rate.  The  SOUL  platform  has  already  been  put  in  production  while 
improvement  and  enhancement  are  being  made  continuously.  Currently,  more  than  1 7,000  students  and  teachers 
are  using  the  platfomi.  We  are  collecting  feedback  from  the  users  in  different  aspects  which  will  be  used  to 
further  improve  the  platform.  These  evaluation  results  will  be  published  in  another  report  once  available.  We 
hope  that  our  experience  can  help  the  community  to  build  abetter  e- learning  environment  for  students,  teachers, 
and  administrators. 


2.  The  SOUL  Platform 

The  SOUL  platform,  developed  by  the  SOUL  project  team,  is  an  online  learning  platform  especially  for  teachers, 
students,  administrators,  as  well  as  alumni  of  SPACE.  Although  the  platform  is  tailor-made  for  SPACE  users,  the 
design  is  general  and  flexible  enough  to  be  adopted  by  other  e-leaming  institutes.  The  system  architecture,  called 
i\\t  PowerEdBuilder , of  the  SOUL  platform  is  shown  in  Figure  1. 

The  Secure  e-Course  exchange  (eCX)  component  provides  a secure  layer  for  protecting  the  copyrighted  course 
materials  (Yau  et  al,  2002).  The  Communication  and  Searching  Infrastructure  (CSI)  provides  efficient 
communication  channels  among  administrators,  teachers,  and  students.  The  Content  Engineering  Agent  (CEA)  is 
mainly  used  by  instructors  to  create  online  course  materials  and  launch  online  courses.  The  e -Institute  is  the 
administration  center  of  the  platform.  Finally,  the  e-Learning  platform  is  where  students  will  interact  for 
studying  and  downloading  relevant  learning  materials.  Smart  Tutor  is  one  of  the  major  components  inside  the  e- 
Leaming  platform  (Zhang  et  al,  2001).  Smart  Tutor  is  an  intelligent  software  that  can  provide  guidance  to 
students  like  a human  tutor  and  is  mainly  used  by  students.  Expert  personalized  advice  and  learning  path  are 
given  to  the  student  based  on  student  profile  maintained.  All  these  functions  are  the  results  of  application  of 
latest  development  in  the  field  of  artificial  intelligent. 

Each  component  in  the  PowerEdBuilder  contains  different  technologies  and  is  the  result  from  a series  of  research 
studies.  In  this  paper,  we  focus  on  the  Content  Engineering  Agent  and  we  will  mainly  describe  the  Task-based 
Learning  framework  of  the  Agent  which  helps  to  fill  the  gap  between  the  technology  and  pedagogy. 
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SmartTutor 

e-Institute 

Content  Engineering 
Agent  (CEA) 

e-Leaming  Platform 

Communication  and  Searching  Infrastructure  (CSI) 

The  Secure  e-Course  exchange  (eCX) 

Figure  1:  An  Overview  of  PowerEdBuilder 


3.  The  Content  Engineering  Agent 

Content  Engineering  Agent  is  designed  to  bridge  the  gap  between  technology  and  pedagogy  based  on  a common 
teaching  and  learning  framework  called  Task  Based  Learning  (TBL)  (Fensteimacher,  1998,  Raggatt  et  al,  1996, 
Willis,  2000).  We  will  briefly  describe  the  three  stages  of  learning  based  on  the  TBL  framework  in  Section  3.1 
and  show  how  the  Content  Engineering  Agent  incorporate  TBL  framework  in  the  design  in  Section  3.2. 


3.1  Three  Stages  of  Learning 

The  Task  Based  Learning  (TBL)  framework  consists  of  three  stages,  namely  Pre-task,  Task  Cycle  and  Post-task. 
In  our  domain,  that  is,  the  preparation  and  development  of  e-course,  pre-task  is  to  define  the  course  aims,  the 
course  structure,  the  relationships  among  chapters  and  concepts,  and  to  create  the  course  contents  including  the 
Question  bank.  The  task  cycle  will  define  the  tests  and  examinations  by  linking  the  concepts,  course  materials, 
and  the  question  banks,  define  the  weightings  of  these  tests  as  well  as  the  passing  criteria  for  the  task.  The  actual 
generation  of  tests  and  examinations  using  the  Question  bank  is  done  by  the  Smart  Tutor  of  the  e- Learning 
platform.  In  the  post-task  stage,  the  Content  Engineering  Agent  helps  to  collect  feedback  from  students  and 
provide  analysis  of  performance  of  students  to  teachers  in  order  to  identify  problems  in  course  materials, 
students,  questions,  as  well  as  course  structure. 


3.2  Details  of  Content  Engineering  Agent 

Tables  1,  2,  and  3 show  the  summary  of  activities  performed  by  the  Content  Engineering  Agent  as  well  as  the 
Smart  Tutor  (for  completeness,  readers  can  refer  to  (Zhang  et  al,  2001)  for  details  of  Smart  Tutor)  in  the  pre-task, 
task  cycle,  and  post-task  stage,  respectively.  Due  to  the  space  limitation,  we  will  not  give  the  details  of  each 
activity  in  the  pre-task  and  the  task  cycle  stages,  readers  can  refer  to  (Cheung  & Kwok,  2001)  for  more 
information. 


Teaching  and 
Learning  Activities 

Smart  Tutor  (Zhang  et  al,  2001) 

Content  Engineering 

Start  a course  which 
suit  the  learner’s  aims 

1.1 

Get  learner  profile 

1.1 

Define  course  title  and  aims 

Evaluate  the  profile  and 
learning  history  of  the 
leaner  by  teacher 

1.2 

Get  learning  history 

(For  new  user,  learning  history  is 

empty) 

1.2 

Generate  chapter  (i.e.  a set  of 
learning  objectives)  and 
hierarchical  relationship 
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Create  the  course 
materials  and  generate 
the  presentation 
sequence 

1.3 

Get  learning  strategy  chosen  by 
the  learner  (strategy  available: 
topic-oriented;  walk-through;  and 
review)  (Zhang  et  al,  2001) 

1.3 

Set  out  relationship  of  chapter, 
session  and  concepts  (pre  and 
post-link  concepts) 

1.4 

Create  course  contents, 
including  teaching  materials  and 
Question  Bank 

Present  the  course 
materials  to  the  learners 

1.4 

Conduct  ad^tive  learning  sessions 
through  simulation  of  teaching 
activities 

1.5 

Make  use  of  multimedia  (e.g. 
animation,  video)  to  deliver 
tutorials 

1.6 

Conduct  on-line  discussion  and 
question  & answer  session 

Table  1:  Activities  of  Smart  Tutor  and  Content  Engineering  in  Pre-task  Stage 


Teaching  and 
Learning  Activities 

SmartTutor 

Content  Engineering 

Generate  assessment, 
test  and  exam 

2.1 

Generate  assessment,  test  and 
exam  through  Question  Bank 

2.1 

Define  test  and  exam  based  on 
Question  bank  and  materials 

Conduct  tests  (both 
formal  and  informal) 

2.2 

Conduct  online  tests  (both  formal 
and  informal) 

Mark  the  assessment  by 
teacher 

2.3 

Mark  and  assess  the  performance 
of  the  learner 

2.2 

Define  weightings  of  the  test  or 
exam 

Compare  the  result  with 
other  learners 

2.4 

Generate  assessment  result  graph 
with  different  weightings  applied 
among  concepts 

2.5 

Update  the  learner  profile,  learning 
history  and  cognitive  model 

Review  the  course  by 
teacher 

2.6 

Review  the  presentation  sequence 
of  sessions 

Inform  the  learner 
his/her  performance 

2.7 

Report  to  learner,  in  particular,  the 
indications  on  his  concept 
proficiency 

Make  decision  on 
whether  the  learner 
should  repeat 

2.8 

Advise  the  learner  whether  he 
should  repeat  similar  task 

2.3 

Define  passing  criteria  for  the 
task 

Table  2:  Activities  of  Smart  Tutor  and  Content  Engineering  in  Task  Cycle  Stage 


Teaching  and 
Learning  Activities 

SmartTutor 

Content  Engineering 

Review  the  learning 
objectives  with  the 

3.1 

Review  the  learning  objectives 
with  the  learner  and  if  required. 

learner  by  teacher 

learner  can  select  another  strategy 
of  topic-oriented;  walk-through; 
and  review 

Generate  further 

3.2 

Recommend  the  learning  path  by 

practice  for  the  learner 

scheduler 
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Make  decision  on 
whether  the  learner 
could  proceed 

3.3 

Advise  the  learner  whether  he 
could  proceed 

3.1 

Refine  passing  criteria  for  the 
course 

Evaluate  the  course  in 
different  aspects  such  as 
course  structure, 
examination  difficulty 
level  etc. 

3.2 

Provide  the  following 
information: 

- Questionnaire  & Written 
feedback 

- Group  statistics  on  overall 
students*  performance 

- Analysis  of  difficulty  level  of 
test,  concepts  and  the  failure 
pattern  of  students 

- Identify  problematic  clusters  in 
course  structure 

- Correlation  results  on  the 
materials,  course  structure  and 
students’  performance 

Table  3:  Activities  of  Smart  Tutor  and  Content  Engineering  in  Post-task  Stage 


The  post-task  stage  generates  useful  information  which  enables  the  Content  Engineering  Agent  to  help  the 
teachers  to  make  decisions  on  how  to  further  improve  the  e-course.  Besides  the  useful  group  statistics  on  overall 
students’  performance  and  written  feedback,  the  Agent  will  analyze  and  correlate  the  failure  patterns  of  students 
against  the  course  materials,  the  course  structure,  the  difficulty  level  of  test  questions.  As  an  example,  the  Agent 
will  identify  clusters  of  problems/concepts  using  an  rule-based  inference  engine  and  data  mining  technique  (see 
Figure  2 for  a sample  output).  The  agent  will  then  help  teachers  to  relate  concepts,  chapters,  sessions  that  might 
be  missing  currently. 


4.  Conclusion 

The  SOUL  platform  depicted  in  this  paper  is  a joint  effort  from  the  SOUL  project  team  and  e learning 
practitioners  (teachers,  administrators  etc.)  of  the  HKU  SPACE.  In  particular,  the  integration  of  TBL  pedagogy 
into  the  Content  Engineering  Agent  provides  teachers  with  a comfortable  and  user-friendly  environment  to 
develop  high-quality  online  courses.  The  first  phase  implementation  of  Content  Engineering  Agent  has  been 
completed.  Evaluation  on  the  usefulness  and  effectiveness  of  the  Agent  is  being  conducted.  The  evaluation  result 
will  be  analyzed  and  used  as  a guideline  for  further  enhancement  of  the  Agent. 

Interested  readers  can  refer  to  the  web  site  (SOUL,  2002)  and  other  publications  of  the  project  group  for  more 
information  and  other  technologies  used  in  other  components  of  PowerEdBuilder.  We  hope  that  our  experience 
in  building  these  components  can  help  other  e-leaming  researchers 
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Abstract 

In  this  contribution  animated  pedagogical  agents  are  looked  at  from  an  instructional  design  perspective.  A ‘support-typology’  was 
constructed  based  on  different  dimensions  on  which  support  can  be  described  and  different  roles  instructional  agents  can  assume. 
This  typology  is  used  to  analyze  current  available  pedagogical  agents.  This  analysis  indicates  that  current  pedagogical  agents  are 
designed  to  provide  support  on  content  and  problem  solving  aspects  of  the  task  and  that  they  are  capable  of  adapting  their  support  to 
learning  paths.  The  review  of  research  on  the  relation  between  pedagogical  agents  and  learning  indicates  that,  although  only  limited 
research  has  been  done,  certain  agent  characteristics  are  relevant  to  take  into  account  in  further  research  on  the  effect  of  agents  on 
learning.  It  can  be  concluded  that  pedagogical  agents  offer  opportunities  to  be  grasped  in  open  learning  environments. 

Introduction 

Open  learning  environments  with  a high  level  of  learner  control  have  been  argued  to  foster  the  acquisition  of  complex 
problem  solving  skills  (Jonassen,  1997).  However,  research  indicates  that  students  do  not  optimally  use  these  support 
tools  (e.g.,  Clarebout,  et  al.,  2000;  Land,  2000).  It  is  wondered  in  this  contribution  whether  pedagogical  agents  might  be 
able  to  enhance  the  optimal  use  of  support  devices  in  open  learning  environments. 

Pedagogical  agents  are  animated  characters  designed  to  operate  in  educational  settings  for  supporting  learning.  They 
can  provide  non-verbal  feedback  through  means  of  facial  expressions  and  gestures  (Johnson,  et  al.,  2000). 

Adapting  support  to  the  need  of  individual  learners  is  an  important  issue  in  designing  learning  environment,  since 
providing  either  too  many  or  too  less  support  may  be  detrimental  for  learning  (Clark,  1991). 

In  order  to  explore  different  ways  in  which  pedagogical  agents  can  adapt  support  to  learning  needs  and  enhance  the  use 
of  support  devices,  currently  available  agents  were  analyzed,  using  a ‘support’  typology.  After  presenting  this  typology, 
the  analysis  of  the  pedagogical  agents  is  discussed  in  the  second  part.  A final  part  presents  an  overview  of  research  with 
respect  to  pedagogical  agents  and  learning.  This  part  gives  insight  in  the  extent  pedagogical  agents  have  been  proved  to 
be  beneficial  to  learning.  The  discussion  and  conclusion  summarizes  the  results  of  the  analysis  and  tries  to  provide  an 
answer  to  the  question  whether  these  agents  might  enable  to  enhance  the  optimal  use  of  support  devices  in  open  learning 
environments. 

Support  typology  for  pedagogical  agents 

Elen  (1995)  proposed  a scheme  with  five  dimensions  to  describe  support  initiatives:  amount,  formal  object,  topical 
object,  delivery  system  and  timing  of  support.  This  scheme  was  adapted  and  elaborated  to  construct  the  support 
typology  for  pedagogical  agents.  The  support  typology  has  6 dimensions  that  look  as  follows  (see  Figure  1): 

• Object  of  support  support  can  be  directed  to  different  aspects  involved  in  solving  complex  problems  (see  also  Hill 
& Hannafin,  2001):  content  of  the  task,  problem  solving  steps,  metacognitive  strategies  and  handling  the 
technology. 

• Learner  control  with  respect  to  support:  open  learning  environments  imply  by  definition  a great  deal  of  learning 
control  (Hannafin,  1995).  However,  since  students  seem  not  always  capable  of  making  adequate  choices  with 
respect  to  their  learning  process  (e.g..  Large,  1996)  and  given  that  reviews  on  learner  control  do  not  reveal  a 
consistent  positive  effect  (see  for  example  Niemic,  et  al.,  1996)  this  aspect  requires  further  investigation. 
Pedagogical  agents  can  initiate  support  delivery,  students  may  request  support  or  decisions  about  support  provision 
can  be  shared. 

• Adaptability  of  support:  learners  differ  with  respect  to  the  amount  of  support  needed,  depending  on  their  prior 
knowledge  (Ross,  et  al.,  1980)  and  other  learner  characteristics  such  as  aptitude  and  metacognitive  skills  (Clark, 
1990).  However,  not  only  the  amount  of  support  might  be  adapted,  also  the  object  of  support.  Learners  might  need 
support  on  the  level  of  the  content  of  the  task  or  on  a more  metacognitive  level.  Change  in  quantity  and  object  of 
support  can  be  determined  by  the  agent  and  / or  the  learner. 

• Delivery  modalities:  Pedagogical  agents  can  entail  different  delivery  modalities,  which  might  affect  their  effect 
(Moreno,  et  al.,  2000).  Pedagogical  agents  can  communicate  verbally  or  non- verbally.  Reeves  andNass  (1996) 
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indicate  that  students  view  the  interaction  with  a computer  (c.q.  pedagogical  agent)  as  a social  interaction  in  which 
three  features  are  essential,  namely:  image,  voice,  and  personalized  language.  Pedagogical  agents  have  by 
definition  an  image.  Hence,  this  distinction  is  not  included  in  the  typology.  The  other  two  features  are.  A distinction 
will  be  made  between  pedagogical  agents  using  verbal  (text  or  speech)  and  / or  non-verbal  language  (head  nods, 
gestures,  facial  expressions).  Pedagogical  agents  can  also  communicate  in  a personal  way,  engaging  in  a dialogue, 
or  in  a more  formal  way,  trough  a monologue. 

Timing  of  support:  support  can  be  delivered  at  different  moments  during  the  learning  process,  van  Merrienboer 
(1997)  distinguishes  between  information  relevant  to  recurrent  aspects  (rules,  procedures,  prerequisites)  and  to  non- 
recurrent aspects  (system,  approaches,  heuristics).  Support  for  recurrent  aspects  should  consist  of  just-in-time 
information  and  immediate  feedback  on  the  quality  of  the  performance.  Support  for  non-recurrent  aspects  should 
consist  of  both  information  presented  up  front  and  delayed.  A pedagogical  agent  can  deliver  support  either  up  front, 
just-in-time  or  delayed. 

Support  style:  this  dimension  refers  to  different  roles  and  modalities  instructional  agents,  in  this  case  pedagogical 
agents,  can  assume.  Literature  suggests  six  typical  roles  for  instructional  agents: 

supplanting:  instructional  agents  take  over  the  tasks  and  perform  them  for  the  learners.  The  learners  observe 
the  instructional  agents  (e.g.  Salomon,  1994); 

scaffolding:  instructional  agents  perform  those  parts  of  the  task  that  the  learners  are  not  yet  able  to  perform 
independently  (e.g.,  Jonassen,  1996); 

demonstrating:  instructional  agents  show  how  a task  if  performed  after  which  they  observe  how  the  learner 
performs  the  task  (e.g.  Merrill,  1994); 

modeling:  instructional  agents  show  how  a task  is  performed  while  revealing  and  explaining  their  reasoning 
process.  They  solve  a task  while  they  articulate  how  problems  are  solved,  what  strategies  are  used  and  what 
mental  models  are  necessary  to  understand  the  task  (e.g.,  Jonassen,  1996); 

coaching:  instructional  agents  provide  hints  and  feedback,  activate  learners  when  they  perform  the  task. 
Instructional  agents  observe  the  learners  and  provide  guidance  when  they  experience  difficulties  (e.g.,  Barab  & 
Duffy,  2000),  and 

testing:  instructional  agents  tests  the  learners’  knowledge  about  certain  aspects  of  the  task  in  order  to  guide  the 
learning  process  (e.g..  Martens  & Dochy,  1997). 

These  roles  can  be  characterized  by  (a  combination  of)  four  analytical  modalities:  executing,  showing,  explaining 
and  questioning.  For  instance,  if  the  role  of  the  instructional  agents  is  coaching,  support  can  consist  of  a 
combination  of  explaining  and  questioning.  Executing  means  that  an  instructional  agent  performs  the  task  for  the 
learner.  Learners  observe  the  instructional  agent  but  do  not  perform  the  task  themselves.  Showing  involves  the 
demonstration  of  the  task  by  an  instructional  agent  after  which  the  learners  perform  the  task  themselves.  Explaining 
involves  an  instructional  agent  who  clarifies  the  task  while  the  learner  performs  it.  Questioning  means  asking 
questions  about  the  task  to  the  learner. 

These  different  modalities  may  involve  the  task  or  part  of  it  and  instructional  agents  may  shift  between  different 
roles  for  different  parts  of  a task. 


Support  style 


Object  of  support: 

* Content  (c) 

• Problem  solving  steps  (p) 

* Metacognition  (m) 

• Technology  (t) 


Delivery  modalities: 

* Speech  (s) 

* Text  (t) 

* Monologue  (m) 

* Dialogue  (d) 

* Feciel  expressions  (f) 

* Gestures  (g) 


Figure  1:  Support-typology  for  pedagogical  agents 
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Analysis  of  different  pedagogical  agents 
Methodology 

To  identify  current  available  pedagogical  agents,  ERIC,  PSYCINFO  and  the  Internet  (using  the  Google-search  engine) 
were  searched.  Reference  lists  in  texts  generated  through  these  searches  were  also  used  to  find  more  information 
(snowball-method).  This  resulted  in  the  agents  presented  in  Table  1.  Key  words  used  to  find  information  on  these  agents 
were  ‘pedagogical  agenf , ‘animated  agent’,  ‘agent  and  virtual  reality’,  ‘agent  and  multimedia’,  ‘personal  digital  agent’. 
Only  agents  used  in  educational  setting  were  selected.  Previous  reviews  of  pedagogical  agents  have  been  presented  by 
Johnson  et  al.  (2000,  2001).  They  came  to  a similar  list,  although  their  review  was  not  restricted  to  agents  operating  in 
educational  settings  and  was  focussed  on  the  technological  aspects  of  these  agents.  In  the  following  a short  description 
is  provided  of  the  different  agents,  the  actual  analysis  is  presented  in  Table  1. 

Description  of  the  different  pedagogical  agents 

Adele  (Shaw  et  al.,  1999;  Ganeshan,  et  al.  2000):  Adele  supports  distance  learning  in  the  domain  of  medicine  and 
dentistry.  She  exists  in  three  different  interaction  modes.  In  the  advisor  mode,  she  observes  the  students  when 
performing  an  action.  She  interrupts  and  makes  suggestions  when  there  is  an  inconsistency  with  the  standard 
practice.  In  the  practice  mode,  Adele  provides  advice  only  on  request  of  the  student.  A final  mode  is  the 
examination  mode  where  she  provides  only  an  evaluation  after  the  students  have  finished.  Recent  developments 
made  Adele  more  focussed  on  guiding  students  in  their  problem  solving  process. 

Steve  (Johnson  et  al.,  2000):  Steve  is  an  acronym  for  Soar  Training  Expert  for  Virtual  Environments.  He  demonstrates 
‘skills’  involved  in  a specific  task.  The  student  can  walk  around  in  the  environment  while  Steve  demonstrates  part 
of  the  tasks  and  explains  what  he  is  actually  doing  (modeling).  The  student  can  interrupt  the  agent  and  ask  to  finish 
the  task. 

Herman  the  Bug  (Lester,  et  al.,  1999a):  Herman  is  an  alien  with  human-like  movements  and  facial  expressions.  He 
inhabits  a learning  environment  called  ‘design-a-plant’  for  the  domain  of  botanical  anatomy  and  physiology. 
Herman  gives  advice,  encourages  students  and  provides  feedback. 

Cosmo  (Lester,  et  al.,  1999b):  Cosmo  inhabits  the  Internet  Advisor  learning  environment  for  the  domain  of  Internet 
packet  routing.  Cosmo  provides  advice  to  learners  and  explains  content  and  problem  solving  related  issues. 
WhizLow  (Johnson  et  al.,  2000;  Gregoire,  et  al.,  1999):  WhizLow  inhabits  CPU  City,  a three  dimensional  learning 
environment  representing  a motherboard.  Students  are  given  programming  tasks.  WhizLow  carries  out  instructions 
given  by  the  students  and  traces  misconceptions  of  students  that  are  corrected  by  providing  advice. 

PPPersona  (Andre  et  al.,  1999):  The  PPPersona  agent,  is  an  animated  agent  for  presenting  on-line  instruction.  The 
agent  guides  learners  through  web-based  materials,  he  demonstrates  the  information. 

Jacob  (Evers  & Nijholt,  2000):  Jacob  instructs  and  assists  users  performing  a task  in  a virtual  environment  (e.g.  tower 
of  Hanoi).  Jacob  gives  feedback  on  the  user  performance  and  can  demonstrate  the  task  if  necessary. 
Ga/j^/a//(Cassell,  & Thorisson,  1999):  Gandalf  is  an  expert  in  the  solar  system  and  the  user  can  ask  him  questions  about 
specific  planets.  Gandalf  will  virtually  travel  to  these  planets  when  a question  is  asked  and  give  an  explanation 
when  the  student  asks  for  it. 

Autotutor  (Graesser,  et  al.,  1999):  Autotutor  is  designed  to  assist  college  students  in  learning  the  fundamentals  of 
hardware,  operating  systems  and  the  Internet.  Autotutor  presents  questions  to  the  students  to  test  their  knowledge. 

In  Table  1 the  analysis  of  these  different  agents  is  presented  following  the  support  typology. 


Table  I:  Analysis  of  different  pedagogical  agents 


Modality 

Role 

Object 

Adaptation 

Delivery 

modality 

Control 

Timing 

Adele 

ex,  q 

c 

c,  p,  t 

a,  0 

s,  t,  d,  m,  f, 
g 

a,  1 

j,d 

Steve 

s,  ex 

d,  m,  c 

c,  p,  t 

a,  0 

s,  d,  m,  f,  g 

a 

P»j 

Herman  the  Bug 

ex 

c 

c,P 

a,  0 

s,  d,  m,  f,  g 

a 

i 

COSMO 

ex 

c 

c,P 

a,  0 

s,  d,  m,  f,  g 

a,  1 

j 

WhizLow 

e,  ex 

s,  c 

c 

a 

s,  m,  d,  g 

a,  1 

-i 

PPPersona) 

s 

d 

c 

- 

s,  m,  f,  g 

a 

- 

Jacob 

s,  ex 

d,  c 

p 

a 

t,  d,  f,  g 

a,  1 

J 

Gandalf 

s,  ex 

d,  c 

c 

a 

s,  m,  d,  f,  g 

1 

J 

Autotutor 

t,  ex 

_9 

c 

- 

t,  m,  d,  f,  g 

a 

The  analysis  reveals  different  aspects: 
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1 . Most  agents  are  designed  to  act  as  a coach.  The  agent  provides  hints  and  feedback  while  the  learner  solves  a 
problem.  Sometimes  demonstration  of  parts  of  the  task  is  also  included  (e.g.  Steve). 

2.  Explaining  is  most  frequently  used  as  a modality,  complemented  by  questioning. 

3.  For  all  agents,  except  for  Jacob,  the  main  object  of  support  is  the  content.  Some  of  the  agents  also  provide  support 
on  the  level  of  problem  solving.  Jacob  for  example  provides  only  support  on  the  level  of  problem  solving.  Steve 
and  Adele  also  support  the  use  of  the  environment. 

4.  All  agents  are  technologically  able  to  communicate  both  verbally  and  non-verbally.  Most  agents  use  speech.  Some 
use  both  speech  and  text.  Dialogue  as  communication  is  mostly  used. 

5.  Just-in-time  support  is  used  frequently. 

Research  results 

The  analysis  of  pedagogical  agents  reveals  that  agents  are  already  used  in  learning  environments  to  support  learning.  In 
this  part  an  overview  is  presented  of  empirical  research  about  the  relationship  between  learning  and  pedagogical  agents. 
Lester,  et  al.  (1997)  performed  a study  to  measure  agents’  affective  effects  on  learning.  Five  ‘clones’  of  Herman  the  bug 
were  studied:  1)  a fully  expressive  one,  giving  principle-based  animated  advice  as  well  as  task-specific  audio-advice,  2) 
one  who  gave  only  principle-based  animated  advice,  3)  a clone  who  gave  principle  based  but  only  verbal  advice,  4)  one 
giving  task-specific  verbal  advice,  and  5)  a mute  one.  All  other  features  remained  identical.  Results  show  that  all 
students  performed  significantly  better  on  the  posttest.  The  smallest  and  highest  increase  in  performance  was  for  the 
students  in  the  ‘mute’  and  the  fully  expressive  condition  respectively.  Results  on  an  assessment  questionnaire  show 
significantly  higher  scores  for  the  personal  assessment  of  the  agent  (utility  of  the  advice  and  feedback  of  the  agent). 
Given  also  the  increase  on  the  post-test  for  the  'mute’  condition,  Lester  et  al.  conclude  that  there  exists  a persona-effect. 
The  merely  presence  of  an  agent  increases  the  scores.  This  persona-effect  however  was  not  found  by  Andr6  et  al. 
(1999),  who  created  two  versions  of  their  learning  environment,  one  with  PPPersona  and  one  without. 

Moreno,  et  al.  (2000)  compared  two  groups  of  students  working  in  a discovery  learning  environment.  One  group  had  a 
pedagogical  agent  (Herman  the  Bug),  while  the  other  received  text-based  information.  No  significant  difference  was 
found  for  retention.  For  a transfer  test  a significant  difference  was  found  in  favor  for  the  group  with  the  pedagogical 
agent.  This  group  showed  also  a greater  interest  and  motivation.  In  a follow  up  study  Moreno  et  al.  (2000)  experimented 
with  pedagogical  agents  varying  on  the  three  features  of  social  interaction  mentioned  by  Reeves  and  Nass  (1996): 
image,  voice,  and  personalized  language.  The  studies  indicated  that  the  condition  in  which  an  agent  with  voice  was  used 
instead  of  text  resulted  in  better  results  for  retention  and  transfer.  The  sheer  presence  of  an  image  of  the  pedagogical 
agent  had  no  effect.  The  kind  of  communication  (dialogue  versus  monologue)  influenced  the  results  for  the  retention 
test  in  favor  of  the  dialogue  mode  (see  also  Moreno  & Mayer,  2000).  In  other  experiments,  Moreno,  et  al.  (2001) 
experimented  with  the  type  of  image.  The  image  of  Herman  was  compared  to  a video-based  image  of  a real  person.  No 
effects  were  found  for  this  variable  on  retention  and  transfer. 

In  addition  to  these  quasi-experimental  studies  two,  more  informal,  evaluation  studies  can  be  mentioned.  The  first  one 
relates  to  an  evaluation  of  Steve  (Johnson,  et  al.,  1998).  It  suggests  that  interacting  with  agents  in  a virtual  environment 
be  perceived  as  more  natural  than  using  conventional  text-based  tutoring  interfaces.  The  second  study  deals  with  Adele 
(Shaw  et  al.,  1999).  Students  indicated  in  a survey  to  appreciate  the  support  provided  by  Adele.  Moreover,  students 
preferred  to  hear  a rationale  only  when  they  asked  for  it  (practice  mode).  Nevertheless  experience  suggests  that  students 
do  not  ask  for  advice  when  given  the  choice. 

Discussion  and  conclusion 

An  analysis  was  made  of  current  available  pedagogical  agents.  To  do  this  analysis,  first  a support-typology  was 
constructed.  This  typology  provides  a common  framework  to  describe  and  discuss  pedagogical  agents.  The  analysis 
reveals  that  most  pedagogical  agents  act  as  coaches  and  provide  support  on  the  level  of  the  content  and  problem  solving. 
Most  of  the  pedagogical  agents  are  capable  of  adapting  quantity  and  object  of  support  to  learners’  action.  It  can  be 
concluded  that  pedagogical  agents  ai*e  capable  of  adapting  support  to  the  needs  of  the  learner  by  reacting  to  the  learning 
path  followed. 

However,  none  of  these  pedagogical  agents  provide  metacognitive  support,  while  such  support  has  been  recognized  as 
important  (Hannafin,  1995,  Shuell,  1992).  This  might  be  explained  by  the  origin  of  pedagogical  agents.  Pedagogical 
agents  evolved  from  research  on  intelligent  tutoring  systems  (Shaw  et  al.,  1999),  which  are  systems  focussing  strongly 
on  the  acquisition  of  domain  specific  knowledge  (Shute  & Psotka,  1996).  These  tutoring  systems  were  developed, 
starting  from  teaching  goals  and  aiming  at  how  to  replace  a human  tutor  by  an  intelligent  tutoring  system  (Derry  & 
Lajoie,  1993)  and  mostly  work  with  well-structured  problems  for  which  one  solution  exists.  The  analysis  of  pedagogical 
agents  demonstrates  that,  even  now,  most  agents  are  used  in  situations  that  do  not  require  students  to  solve  complex 
problems  but  rather  procedural  tasks  for  which  an  optimal  solution  exists  (e.g.  Jacob,  PPPersona).  So,  the  answer  to  the 
question  ‘where  do  we  stand?’  would  be  that  pedagogical  agents  provide  a more  sophisticated  intelligent  tutoring 
system.  However,  to  develop  pedagogical  agents  able  to  operate  in  open  learning  environments  in  which  students  have 
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control  over  their  learning  and  work  on  a complex  task,  a different  perspective  might  be  indicated.  Rather  than  starting 
from  a teacher-oriented  perspective  (how  to  model  a good  tutor),  a student-oriented  perspective  might  be  advocated. 
Rather  than  starting  from  focussing  on  the  acquisition  of  domain  specific  knowledge,  and  representing  a complete 
domain-model,  metacognitive  processes  could  be  focussed  upon  to  enable  students  to  monitor  their  own  learning 
process. 

Pedagogical  agents  operating  in  open  learning  environments  and  delivering  support  on  a metacognitive  level  might 
encourage  learners  to  make  more  ample  and  deliberate  use  of  support  in  open  learning  environments.  Hill  and  Hannafm 
(2001)  for  example  argue  that  students  lack  the  necessary  skills  to  monitor  their  learning  and  to  make  appropriate 
choices  with  respect  to  support  devices  in  the  learning  environment. 

Research  on  intelligent  tutoring  systems  focussed  on  whether  these  systems  were  a ‘good’  replacement  of  a human  tutor 
(e.g.,  Schofield,  et  al.,  1994),  research  on  pedagogical  agent  should  focus  on  the  influence  of  these  agents  on  (aspects 
of)  the  learning  process,  considering  research  findings  with  respect  to  agent  characteristics  (Lester  et  al.,  1997;  Moreno 
et  al.,  2000)  and  learner  variables  (Shaw  et  al.,  1999). 

Taking  a more  learner-oriented  perspective  into  account  in  the  development  of  pedagogical  agents  and  focussing  on 
supporting  students  on  the  level  of  their  metacognitive  skills  could  result  in  pedagogical  agents  able  to  enhance  the 
(optimal)  use  of  available  support  devices  in  open  learning  environments. 
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Abstract:  Computer-based  simulations  have  the  potential  to  enhance  the  transfer  of  learning  by 
providing  students  with  surrogate  learning  environments.  However,  the  success  of  simulations  to  a 
certain  degree  depend  on  their  designs.  In  order  to  ensure  learning,  simulations  should  be  constructed 
based  on  instructional  design  principles.  As  Clark  (1985)  points  out,  effective  learning  depends  on  the 
design  strategies  implemented  in  a delivery  system  rather  than  on  the  system  itself  Although  different 
designers  have  different  perspectives  when  making  design  decisions  (Goosen,  2001),  there  are  common 
issues  faced  by  designers  at  large.  Serving  to  provide  recommendations  for  practitioners  who  will 
engage  in  designing  and  developing  effective  educational  simulations,  this  paper  surveys  examples  of 
simulations,  examines  design  issues  concerned  with  computer-based  simulations  and  synthesizes  design 
guidelines  suggested  in  literature. 


Introduction 

Designing  and  developing  simulations  can  be  complex  and  costly,  and  thus  it  is  crucial  to  consider  thoroughly 
beforehand  if  simulations  should  be  used  for  particular  educational  purposes.  Once  it  is  determined  that  simulations  can 
help  to  reach  specific  learning  objectives,  to  better  represent  the  selected  phenomena,  and  to  fit  the  instructional  level  of 
the  target  students,  the  next  step  is  to  consider  the  design  aspects.  Although  different  designers  have  different 
perspectives  when  making  design  decisions  (Goosen,  2001),  there  are  common  issues  faced  by  designers  at  large.  In  this 
paper,  major  design  considerations  affecting  the  nature  and  the  effectiveness  of  simulations  are  discussed.  The 
considerations  include  the  advantages  of  simulations  over  other  instructional  alternatives,  fidelity  levels,  and  student 
support. 

Simulations  vs.  Other  Instructional  Alternatives 

Simulations  can  be  used  when  there  is  no  better  alternative  method  to  approach  a significant  learning  objective. 
Simulations  offer  opportunities  for  students  to  understand  phenomena  which  are  otherwise  inaccessible  because  of  the 
costs,  dangers,  or  time  constraints  involved  in  reality  (Dennis  & Kansky,  1984).  For  example,  by  participating  in  a 
financial  management  simulation,  students  learn  how  to  invest  money  and  discover  the  consequences  of  their  actions  in 
a short  time.  They  do  not  need  to  raise  fund  for  the  investment,  take  the  risk  of  losing  their  capital,  nor  wait  for  months 
or  even  years  to  figure  out  the  results. 

The  advantages  of  simulations  over  tutorials  and  drills  include:  (a)  better  transfer  of  learning,  (b)  opportunities 
for  active  exploration,  and  (c)  motivational  appeal  (Alessi  & Trollip,  1991;  de  Jong,  1991).  Simulations  teach  skills  or 
knowledge  in  an  environment  similar  to  the  real  situation  to  which  the  skills  or  knowledge  can  apply.  Moreover,  in 
simulations  students  can  actively  explore  the  content  domain  and  therefore  as  active  participants  they  become  more 
motivated.  Once  a simulation  is  chosen  as  the  appropriate  instructional  mode,  designers  need  to  consider  two  major 
aspects  of  simulation  design:  fidelity  and  student  support. 

Fidelity 

Fidelity  is  concerned  with  to  what  degree  a simulation  replicates  reality  (Alessi,  1988).  Unlike  the  scientific 
models  in  which  high  fidelity  is  always  desirable,  in  instructional  simulations  high  fidelity  is  not  always  a better  choice 
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over  low  fidelity  (Levin  & Waugh,  1988).  Theoretically,  the  relationship  of  fidelity  and  learning  should  be  linear.  That  is, 
the  closer  a simulation  is  to  the  situation  being  simulated,  the  more  learning  transfer  would  occur  in  reality.  However, 
research  findings  indicate  that  high  fidelity  does  not  always  enhance  learning  for  two  reasons:  (a)  high  fidelity  demands 
more  memory  load  and  cognitive  abilities  of  students,  and  (b)  instructional  techniques  which  are  beneficial  to  novices 
usually  decrease  fidelity  (Alessi,  1988).  Details  of  more  realistic  simulations  might  distract  students’  attention  or 
generate  memory  overload,  especially  when  students  are  in  the  initial  stages  of  learning. 

Nevertheless,  the  claim  that  high  fidelity  is  not  always  desirable  in  instructional  simulations  does  not  imply  that 
simulation  design  should  aim  for  low  fidelity.  Rather,  the  claim  indicates  that  different  levels  of  fidelity  are  needed  in  the 
design.  Furthermore,  the  fidelity  concern  is  about  when  and  how  these  fidelity  levels  should  be  adjusted.  Alessi  (1988) 
suggests  that  the  level  should  increase  appropriately  as  students  progress  from  novices  towards  experts.  Levin  and 
Waugh  (1988)  propose  that  a range  of  fidelities  be  provided  in  a simulation  in  which  a content  domain  is  divided  into 
different  sections.  These  sections  can  be  on  the  same  or  different  difficulty  levels.  As  students  progress,  they  deal  with 
more  sections  of  the  same  difficulty  level  or  with  more  challenging  aspects  of  the  domain. 


Student  Support 

A simulation  must  include  appropriate  support  in  order  to  guide  students  to  reach  specific  learning  goals 
(Steinberg,  1991).  Without  the  support,  students  are  not  likely  to  learn  the  concepts  or  principles  embedded  in  the 
simulation  when  they  freely  explore  on  their  own.  They  might  ignore  more  important  aspects  intended  for  them  to  learn 
and  focus  on  rather  insignificant  details.  They  might  not  be  able  to  find  solution  paths  and  thus  the  benefit  they  obtain 
from  the  simulation  is  minimized.  Moreover,  for  a simulation  that  has  high  cognitive  demands  to  be  effective,  appropriate 
support  is  a must.  Although  the  support  is  sometimes  offered  by  an  instructor,  it  can  be  included  in  the  simulation 
program  itself  as  well  (de  Jong,  1991). 

Two  types  of  support  could  be  helpful  to  students:  (a)  directions  about  how  to  interact  with  the  simulation, 
and  (b)  guidance  about  how  to  learn  from  the  simulation  (Steinberg,  1991).  For  the  quantity  of  support  available  to 
students,  Levin  and  Waugh  (1988)  suggest  that  the  notion  of  “dynamic  support”  be  applied  to  simulation  design.  As 
students  advance  from  novices  towards  experts,  the  amount  of  support  they  need  decreases.  Therefore,  the  amount  of 
support  provided  by  a simulation  program  should  be  varied  according  to  students’  instructional  levels.  In  the  initial 
level,  the  program  should  offer  more  extensive  support.  As  the  students  become  more  experienced  in  the  content  domain 
of  the  simulation,  the  support  can  be  gradually  reduced. 

Summary 

Drawing  upon  research  findings  and  examples  of  educational  simulations,  this  paper  synthesizes  simulation 
design  guidelines  including  fidelity  and  student  support.  In  contrast  to  other  modes  of  computer-based  instruction  (e.g., 
tutorials  and  drills),  simulation  environments  are  much  more  complex  in  terms  of  design  and  use.  However,  well-designed 
simulations  can  assuredly  minimize  the  complexity  of  use  and  enhance  their  educational  effectiveness.  The  design 
considerations  can  assist  designers  in  planning  and  constructing  simulations  that  facilitate  learning. 
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Abstract:  This  study  investigated  Charles  Reigeluth’s  Elaboration  Theory  in  ahypertext 
environment.  An  Elaboration-Theory-Based  hypertext  was  designed  to  compare  with  a linear 
hypertext  that  does  not  employ  any  instructional  design  theories  and  principles.  Thirty-three 
college  students  in  a large  U.S.  university  participated  in  the  study.  The  research  results 
showed  statistically  significant  differences  between  the  control  group  and  the  experimental 
group  in  terms  of  students’  achievements  on  learning  facts  and  concepts.  Further  research 
agenda  is  proposed  according  to  the  results  of  item  analysis  in  the  test. 


Introduction 

While  creating  hypermedia  environments  is  becoming  part  of  the  mainstream  in  the  implementation  of 
educational  media  in  most  of  educational  settings,  the  design  and  development  of  hypermedia  materials  are  still 
heavily  based  on  technical  issues  or  simply  designed  by  intuition  (Chen  & Dwyer,  in  press).  One  of  the  most 
referred  and  applied  instructional  theories,  Charles  Reigeluth’s  Elaboration  Theory  (ET),  is  the  target  of 
investigation  in  this  study.  According  to  Reigeluth  (1987),  Elaboration  Theory  prescribes  a macro  level  of 
instruction  by  sequencing  information  in  a simple -to-complex  approach.  More  specifically,  the  theory 
prescribes  that  instruction  starts  with  an  epitome  that  teaches  a few  simple,  general  ideas  in  order  to  link  to 
learner’s  prior  knowledge  (or  experience).  The  remainder  of  the  instruction  presents  progressively  more 
complex  ideas  in  order  to  elaborate  on  the  ones  previously  presented.  The  purpose  of  this  study  is  to  investigate 
whether  Elaboration  Theory  strategies  are  still  able  to  serve  as  effective  guidelines  for  designing  hypermedia 
materials  with  the  latest  emerging  technologies. 

The  study  is  primarily  designed  to  compare  an  Elaboration-Theory-Based  hypertext  with  a linear 
hypertext  that  does  not  employ  any  instructional  theories  and  principles.  The  investigated  instructional 
treatment  is  to  design  a unit  of  hypertext  materials  by  using  one  of  the  major  elaboration  strategies  proposed  by 
Reigeluth  (1987):  sequencing  the  instructional  materials  using  a simple -to-complex  format.  In  compliance  with 
the  purpose  of  the  study,  this  research  study  is  to  determine  whether  the  Elaboration-Theory -Based  hypertext, 
designed  by  using  Reigeluth’s  instructional  strategies,  more  effective  than  a non-theoiy  hypertext  in  facilitating 
student  achievement  of  different  educational  objectives. 


Instructional  Treatments:  The  instructional  content  used  for  designing  the  instructional  treatments  is  the 
2000- word  heart  content  module  developed  by  Dwyer  and  Lamberski  (2000). 

1.  Control  Group:  Linear  hypertext  as  a control  group 

In  this  design  of  treatment,  the  original  heart  content  was  converted  into  HTML  format  and  put  onto 
the  World  Wide  Web.  Learners  determined  their  own  learning  paces  by  dragging  the  scrolling  bar  up  and  down 
on  the  right  side  of  the  browser. 

2.  Experimental  Group:  Elaboration-Theory -Based  hypertext 

This  treatment  employed  one  of  the  major  elaboration  strategies:  sequencing  the  instructional  materials 
by  using  a simple -to-complex  format.  The  first  frame  (lesson)  of  the  instruction  was  designed  to  represent 
central  theme  of  the  instruction.  One  epitome  was  created  to  serve  as  an  advanced  organizer  to  provide  learners 
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with  the  most  relevant  and  inclusive  introductory  materials.  The  subsequent  frames  (lessons)  were  designed  to 
elaborate  on  different  parts  of  the  epitome . 


Research  Design 

The  research  design  in  the  study  is  a randomized  posttest  with  a control  group  design.  The  independent 
variable  is  the  type  of  hypertext.  The  dependent  variables  are  three  criterion  tests:  terminology  test, 
identification  test  and  comprehension  test  that  are  given  immediately  after  the  students  finish  reading  the  online 
instructional  treatments.  A t-test  statistics  measure  is  used  to  analyze  collected  data. 


Procedure  and  Data  Analysis 

Thirty-three  college  students  participated  in  this  study.  Sixteen  of  them  were  randomly  assigned  to  the 
control  group,  seventeen  in  the  experimental  group.  The  results  of  the  study  (see  Table  1)  indicate  statistically 
significant  differences  between  the  control  group  and  the  experimental  group.  That  is,  treatment  designed  by 
using  elaboration  theory,  was  superior  to  the  control  treatment  (non-theory  and  linear  hypertext)  for  the 
identification  and  terminology  tests,  but  not  for  the  comprehension  test. 


Table  1;  Results  of  the  study 


Measures 

Mean  Score 
(Control:  16) 

Mean  Score 
(Experiment:  17) 

p- value 

Identification  Test 

11.67 

15.76 

*0.0077  (<  .05) 

Terminology  Test 

7.06 

11.50 

*0.0163  (<  .05) 

Comprehension  Test 

8.20 

11.15 

0.1804 

* indicates  the  significant  differences  found  between  the  two  treatments 


Concluding  Comments 

According  to  the  above  results,  the  Elaboration-Theory-Based  hypertext  indeed  improved  students’ 
learning  achievements  in  terms  of  identification  and  terminology  tests.  However,  it  did  not  enhance  students’ 
comprehension  capabilities.  To  investigate  potential  solutions  to  the  aforementioned  problem,  item  analysis  for 
the  comprehension  test  was  conducted.  The  results  indicated  the  item  difficulties  of  11  out  of  20  items  were  not 
desirable  (less  than  0.5).  Therefore,  a more  thorough  unit  of  hypertext  instruction  designed  by  using  more 
elaboration  strategies  should  be  further  investigated  to  improve  those  identified  comprehension  items. 
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Abstract:  This  paper  will  discuss  Computer  Based  Virtual  Field  Trips  that  use  technologies 
commonly  found  in  public  schools  in  the  United  States.  The  discussion  will  focus  on  the 
advantages  of  both  using  and  creating  these  field  trips  for  an  instructional  situation.  A virtual  field 
trip  to  Cumberland  Island  National  Seashore,  St.  Marys,  GA  will  be  used  as  a point  of  discussion 
for  the  technologies  involved  and  the  value  of  their  use  for  instruction  in  a science  classroom. 

While  this  field  trip  is  being  used  as  a discussion  point,  the  techniques  and  advantages  identified 
can  be  applied  over  a P-16  grade  range  in  all  subject  areas.  If  the  decision  is  made  by  an  instructor 
to  create  a virtual  field  trip,  the  technologies  used  can  be  as  simple  as  using  digital  pictures  and  text 
in  Microsoft  PowerPoint  or  HyperStudio  or  as  complex  as  using  digital  video  and  panoramas  in 
Macromedia  Director.  The  level  of  complexity  will  depend  on  the  technology  available  and  skills 
of  the  developers. 


Virtual  field  trips  are  a hot  buzzword  for  integrating  technology  into  the  curriculum.  If  one  does  a web 
search  using  “Virtual  Field  Trip”  as  a key  phase,  hundreds  of  sites  will  be  identified.  These  sites  are 
overwhelmingly  web  based  and  provide  an  abundance  of  pictures  with  limited  information  about  locations  that 
are  deemed  to  have  educational  value.  Krupnick,  states  that,  “The  W eb  is  now  crowded  with  sites  that  are 
considered  ‘Virtual  Field  Trips’  and  they  vary  a great  deal  in  content  and  usefulness.”  She  further  states  that, 
“For  teachers  who  already  have  a curricular  unit  assembled  and  are  looking  for  online  enhancements,  these  are 
adequate.  But  for  teachers  who  are  looking  for  a source  of  new  and  exciting  curriculum  such  sites  are  only  a 
starting  point.  In  order  to  make  them  useful,  an  instructor  would  need  to  develop  curriculum.”  (2000,  p.  43) 

The  approach  taken  in  this  paper  is  that  a virtual  field  trip  is  much  more  than  a web-based  presentation  of  a 
location.  Our  focus  is  to  show  how  to  design  a field  trip  based  on  curriculum  content  that  allows  the  student  to 
“learn”  from  the  field  trip  in  a way  that  is  similar  to  actually  taking  the  field  trip.  Coulter  states,  "The  key  is  for 
(instruction)  to  drive  the  technology  implementation  and  not  vice  versa,  despite  pressures  to  integrate 
technology  into  the  classroom....  virtual  field  trips  ...  enable  students  to  refine  and  extend  their  growing 
understanding  as  they  explore  other  parts  of  the  world.”  (2000,  p.  49) 
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In  educational  situations  actual  field  trips  need  to  be  more  than  “just  a day  away  from  school,”  with  limited 
tangential  educational  purpose.  They  should  be  as  real  a connection  to  the  “real  world”  as  possible.  A normal 
day  field  trip  does  not  offer  sufficient  time  to  take  full  advantage  of  all  a site  has  to  offer.  Bellan  and  Sheurman 
(1998)  discuss  ideas  about  actual  and  virtual  field  trips  and  state  pitfalls  of  both.  Their  article  outlines  the 
teacher  involvement  necessary  to  alleviate  some  of  the  pitfalls.  Among  these  are  a teacher  visiting  both  the 
actual  and  virtual  sites,  the  teacher’s  plan  for  student  use  of  a virtual  site,  student  use  of  the  virtual  site  before 
visiting  actual  sites,  and  follow  up  instruction  using  both  the  virtual  and  actual  trip  experiences.  Taking  the  field 
trip  on  the  computer  does  not  eliminate  all  the  concerns  about  how  field  trips  are  used.  The  virtual  field  trip  is 
usually  not  sufficient  by  itself.  Pre-planning  the  trip  and  the  follow-up  experiences  should  be  addressed  as  part 
of  the  teacher’s  curriculum.  In  addition  we  suggest  a virtual  field  trip  should  be  developed  by  the  teacher  to  be 
used  in  connection  with  an  actual  field  trip  to  the  site. 


Definition  of  a Virtual  Field  Trip 

Virtual  field  trips  as  referred  to  in  this  paper,  are  computer-based  simulations  of  an  actual  field  trip,  which 
allows  the  user  to  vicariously  experience  the  environment  of  the  intended  location.  They  provide  the  teacher  and 
learner  the  opportunity  to  explore  aspects  of  an  actual  trip  without  leaving  the  classroom.  They  should  include 
all  elements  of  a welMesigned  field  trip  and  provide  the  student  with  experiences  that  are  beyond  those  that 
could  be  obtained  from  a pamphlet  about  or  a photo  display  of  the  location.  Stainfield,  Fisher,  et.  al.  (2000)  feel 
that  virtual  refers  to  “digital  alternative  representations  of  reality”  and  that  a virtual  field  trip  is  not  an  attempt  to 
create  a virtual  reality. 


The  Power  of  Instructor  Designed  Virtual  Field  Trips 

Instructors  can  create  their  own  virtual  field  trip  with  their  students  by  using  a digital  camera.  By  planning 
what  they  would  like  to  include,  the  field  trip  can  be  designed  to  meet  the  objectives  of  the  specific  instructional 
situation.  The  site  used  for  the  virtual  trip  can  be  one  that  is  within  the  student’s  local  community.  Choosing  a 
site  that  can  be  visited  by  the  class  or  a student’s  family  will  vastly  increase  the  educational  value  of  the 
experience.  This  requires  that  teachers  must  have  control  in  creating  the  trip  to  meet  their  curricular  needs. 
Student  involvement  in  the  creation  of  the  virtual  field  trip  can  give  power  to  the  experience.  This  can  take  the 
form  of;  collecting  information  at  school,  using  aspects  of  the  technology  in  the  construction  of  the  product, 
taking  the  actual  field  trip  and  gathering  the  images  and  information  that  is  presented  in  the  virtual  field  trip. 
This  type  of  involvement  focuses  student’s  attention  and  learning  on  the  intricate  aspects  of  the  content 
presented.  In  addition  it  provides  ownership  by  the  students  of  the  project  and  focuses  their  learning  of  the 
curricular  content.  Last,  but  not  least,  it  provides  opportunities  for  students  to  use  the  technology  in  a real  world 
situation,  which  addresses  the  NETS  standards  by  preparing  them  to  function  in  a technology  rich  information 
society. 


Advantages  of  Creating  a Virtual  Field  Trip 

The  main  advantage  of  a virtual  field  trip  is  that  it  can  be  used  to  meet  the  objectives  of  the  curriculum  and 
the  needs  and  ability  levels  of  the  students.  Other  advantages  include: 

• providing  opportunities  for  repeated  visitations  to  the  site  for  continued  study. 

• allowing  the  teacher  to  focus  on  one  specific  aspect  of  the  trip  at  a time. 

• providing  for  the  presentation  of  a wider  variety  of  experiences  than  may  be  possible  on  one  trip. 

• illustrating  time  sensitive  issues  that  could  not  be  viewed  on  a single  actual  trip. 

• allowing  classes  in  a different  geological  location  to  visit  and  compare  with  resources  in  their  own  area. 

• providing  integration  of  the  multiple  aspects  of  the  field  trip  into  a number  of  different  curriculum  areas. 

• allowing  for  commonality  of  experiences  by  all  participatents. 

• allowing  students  to  take  a closer  look  at  areas  that  could  not  be  fully  explored  during  an  actual  field  trip. 

• using  for  assessment  purposes. 

• sharing  with  colleagues  and  parents. 
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• using  repeatedly  by  the  teacher  year  after  year. 

When  teachers  use  a virtual  field  trip  in  conjunction  with  an  actual  field  trip  to  the  same  site,  a virtual  trip  can 
serve  as  a motivator  for  the  trip,  or  as  an  advanced  organizer  for  the  day’s  activities.  It  also  encourages  learners 
to  plan  and  prepare  for  activities  to  be  carried  out  on  the  trip  or  can  provide  a simulation  for  students  who  may 
not  have  been  able  to  attend  the  actual  field  trip. 


Limitations  of  a Virtual  Field  Trip 

There  are  two  major  limitations  to  using  a virtual  field  trip.  The  first  is  the  availability  of  a field  trip  that 
meets  the  objectives  of  the  curriculum.  Commercially  available  and  Web  based  field  trips  are  designed  for  a 
large  audience  and  may  need  to  be  adapted  to  meet  an  individual  instructors  curriculum  needs.  The  second 
limitation  focuses  on  the  designer’s  expertise  in  the  content  area  and  appropriate  pedagogy  for  the  field  trip. 


Barriers,  and  the  Paths  over  Them 

The  main  limitation  in  creating  a virtual  field  trip  is  the  time  needed  to  create  the  experience.  The  teacher 
who  wants  to  create  a virtual  field  trip  needs  to  find  ways  of  involving  others  such  as;  colleagues,  community 
members, “ students  and  parents.  If  the  teacher  views  the  design  of  the  field  trip  as  dynamic,  constantly  evolving, 
construction  of  both  the  actual  and  virtual  field  trips  will  grow  over  time. 

Because  curriculum  design  requires  content  expertise,  another  limitation  is  the  creator’s  knowledge  of  the 
content  area  of  the  field  trip  and  the  curricular  objectives.  As  teachers  become  involved  with  the  design  they 
often  need  to  extend  their  content  knowledge,  thus  increasing  the  time  necessary  to  design  the  experience.  On 
the  positive  side,  they  can  get  so  involved  that  their  personal  knowledge  increases  and  it  becomes  a professional 
development  experience. 

A third  limitation  is  the  availability  of  the  technology.  However  with  the  increasing  availability  of 
technology  in  the  schools,  homes  and  work  places  this  is  becoming  less  of  a barrier.  Coupled  with  the 
availability  of  the  technology  is  the  teacher’s  ability  to  use  the  technology  effectively.  Again  this  can  become  a 
learning  situation  and  thus  increases  the  time  to  create  the  product.  This  limitation  can  be  minimized  by  the 
involvement  of  colleagues,  community  members,  students  and  parents. 


Creating  a Virtual  Field  Trip 

The  general  steps  for  creating  a virtual  field  are: 

1.  Examine  objectives  of  the  course  and  choose  a field  trip  that  includes  experiences  that  fit  within  the 
realm  of  the  objectives  and  enhances  learning. 

The  purpose  of  a virtual  field  trip  may  meet  multiple  objectives:  to  learn  specific  content,  to  experience  a 
new  venue,  or  to  develop  skills  in  documenting  experiences  and  using  technology. 


2.  Create  a concept  map  of  the  experiences  to  be  included  in  the  Iicldtrip. 

A concept  map  provides  an  overview  of  all  the  elements  to  be  included  in  the  virtual  field  trip  and  acts  as  an 
organizing  framework  to  build  upon  in  construction  of  the  final  product.  The  concept  map  should  not  be 
considered  the  final  blueprint  for  the  field  trip;  rather  it  should  be  seen  as  a dynamic  overview  and  a starting 
point  from  which  the  product  evolves. 


3.  Select  the  appropriate  technology  to  be  used  based  on  the  content  and  the  curricular  objectives  of  the 
trip.  The  technology  includes  the  organizing  program  and  tools  for  collecting  and  presenting  data. 
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The  equipment  and  software  used  will  depend  on  the  complexity  of  the  trip  and  the  grade  level  of  the 
students.  Virtual  field  trips  can  vary  in  their  complexity  based  upon  the  experience  of  the  creator  with  the 
technology  available. 


4.  Collect  and  organize  materials  to  be  included  based  on  the  curriculum  objectives  to  be  accomplished 
and  the  concept  map. 

Examples  of  desired  materials  might  be:  a)  still  pictures,  both  digital  and  photographs;  b)  videos;  c.  text;  d) 
databases  and  graphs;  and  e)  sound  clips. 

As  materials  are  collected  and  organized  decisions  must  be  made.  For  example,  panoramic  videos  can  be 
used  to  give  an  overview  of  the  location.  If  panoramas  are  to  be  used,  decisions  will  have  to  be  made 
beforehand  as  to  the  location  and  the  concept  to  be  illustrated. 


5.  Convert  all  materials  to  digital  format. 

When  converting  materials  to  digital  format  the  question  of  designing  for  dual  platforms  (Windows  and 
Mac  OS)  must  be  considered.  While  designing  for  dual  platforms  is  not  more  difficult  or  time  consuming,  the 
decisions  made  about  the  format  of  the  individual  parts  (pictures  and  text)  must  be  considered  to  insure  that  they 
are  appropriate  for  the  platform(s)  chosen.  Making  these  decision  early  eliminates  converting  work  later. 
Materials  not  in  digital  format  need  to  be  digitized. 


6.  Assemble  all  elements  in  the  organizing  program  based  on  the  concept  map. 

Here  decisions  on  the  format  that  the  field  trip  will  be  presented  in  must  be  made.  Will  it  be  one  stand-alone 
program  presentation  or  a series  of  parts  that  can  be  accessed  separately?  Also,  before  assembling  the  materials 
one  must  consider  the  platforms  that  can  present  the  materials . Will  the  field  trip  best  meet  the  needs  of  the 
teacher  if  it  is  dual  platform  or  is  one  platform  sufficient?  These  decisions  along  with  the  assembler’s 
knowledge  of  presentation  programs  will  determine  if  Macromedia  Director,  HyperStudio,  Power  Point  or  any 
of  the  other  available  authoring  programs  meet  the  needs  of  the  designed  field  trip.  The  choice  of  the  authoring 
program  used  is  often  the  assemblers  personal  choice  based  on  familiarity  and  skill  with  the  program. 


7.  Evaluate  the  finished  product  to  be  sure  it  meets  the  objectives. 

In  a teacher  created  product,  evaluation  should  be  formative  in  nature,  providing  a continuously  changing 
and  evolving  product.  There  will  always  be  questions  of;  how  could  we  do  this  better?  what  can  be  added  to  it? 
what  are  the  differing  needs  of  the  learners  using  it?  and  how  can  each  learner  group  add  to  the  information 
presented? 

When  evaluating  the  effectiveness  of  a virtual  field  trip  one  should  consider  both  the  appropriateness  and 
the  effectiveness  of  the  technology  used  in  the  presentation  of  the  content.  Children  frequently  experience 
learning  situations  indirectly  thought  technology  by  the  use  of  pictures,  time  -lapse  photography  and  interactive 
non-linear  experiences  that  allow  the  learner  to  follow  their  interest  and  to  revisit  locations  in  the  program  as 
their  interest  grows.  Technology  should  not  be  seen  as  a replacement  for  experiences  but  as  an  enhancement  of 
these  experiences.  Technology  can  be  used  to  enhance  the  senses,  build  interest  and  excitement  or  to  review  and 
analyze  experiences.  Digital  cameras  provide  the  technology  necessary  to;  record  experiences  for  later  analysis 
without  damaging  the  environment,  document  cause  and  effect  relationships,  record  changes  over  time, 
document  unique  events,  record  images  that  give  historical  perspectives,  open  vistas  of  limited  access  to  all 
learners,  and  reviewing  and  analyzing  experiences. 

We  chose  to  illustrate  the  process  of  creating  a virtual  field  trip  within  a course  for  Middle  Grades  pre- 
service teachers.  Its  objectives  are  to  prepare  the  pre -service  teachers  to;  1)  design  and  use  field  trips  effectively 
in  their  classrooms,  2)  effectively  develop  technology  use  and  skills  in  their  students  and  3)  illustrate  an 
integrated  curriculum  approach  to  learning. 
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The  Cumberland  Island  National  Seashore:  Virtual  Field  Trip  (Clark,  et  al,  2002)  has  been  effective  in 
motivating  our  middle  school  pre -service  teachers  in  several  ways.  The  students  actually  look  forward  to  getting 
up  at  5 a.m.  for  a 6 a.m.  departure  and  walking  in  the  sun/rain/wind/heat/cold  for  twelve  or  more  hours  before 
returning  home.  The  students  are  more  attuned  to  the  sites  they  will  see  and  participate  in  activities  on  the 
island.  For  example; 

• upon  reaching  the  beach  instead  of  thinking  of  swimming,  they  look  for  live  buried  shells. 

• they  have  thought  about  the  mathematics  of  figuring  out  a tree’s  canopy  coverage,  or  a fountain’s  volume, 
and  proceed  with  the  task. 

• they  know  what  plants  and  animals  are  to  be  found  and  set  about  searching  for  them. 

• they  know  how  to  use  the  digital  camera  to  document  necessary  aspects  of  an  unknown  phenomena  for 
later  study  or  life  form  for  later  identification  in  their  field  manuals. 

• they  look  for  signs  of  man’s  influence  on  the  environment  and  history  of  the  area. 

• they  look  for  signs  of  the  destructive  effects  of  the  feral  horse  and  hog  population  on  the  environment. 

• in  the  marsh  and  bogs  they  look  more  closely  at  the  smaller  forms  of  wildlife. 

• they  have  a stronger  awareness  of  what  goes  into  a quality  learning  experience. 

• upon  returning  from  the  actual  trip  they  note  what  they  saw,  what  they  did  not  see,  and  organize  their  data 
and  experience  into  a technology  enriched  presentation. 


Conclusion 

We  feel  that  a virtual  field  trip  that  resides  on  a storage  medium  and  not  on  the  WWW  has  advantages  in  its 
effectiveness.  The  program  is  not  dependent  on  access  to  or  the  vagaries  of  the  Web,  it  runs  faster,  it  is  always 
accessible,  and  it  focuses  the  student’s  attention  on  the  task  at  hand. 

This  paper  uses  the  terms  teacher  created,  creator,  and  assembler  in  an  apparently  interchangeable  manner. 
This  is  due  to  the  principle  that  while  the  teacher  is  the  driving  force  in  the  design  and  creation  of  the  virtual 
field  trip;  the  tasks  can  be  done  in  conjunction  with  students,  parents,  interested  community  members  and 
colleagues.  The  technologies  used  to  develop  and  create  virtual  field  trips  are  commonly  found  in  public 
schools  today.  This  process  of  creating  virtual  field  trips  can  be  as  simple  as  using  digital  pictures  and  text  in 
Microsoft  Power  Point  or  HyperStudio  or  as  complex  as  using  digital  video  and  panoramas  in  Macro  Media 
Director.  The  technology  skills  necessary  in  the  creation  of  the  field  trip  all  are  within  the  scope  of  the  National 
Educational  Technology  Standards  (NETS)  for  Teachers,  and  many  are  within  the  scope  of  the  NETS  for 
students. 


References 

Bellan,  J.M.,  & Sheurman,  G.  (1998).  Actual  and  virtual  reality:  Making  the  most  of  field  trips.  Social 
Education,  62 (\\  35-40. 

Clark,  K.,  Hosticka,  A.,  Schriver,  M.,  & Bedell,  J.  (2002).  Cumberland  Island  National  Seashore:  Virtual  Field 
Trip  [Computer  software,  CI>-ROM].  Atlanta:  Georgia  Middle  School  Association. 

Coulter,  B.  (2000).  What’s  it  like  where  you  live.^  Science  and  Children,  f <?(6),  48-50. 

Krupnick,  K.  (1998).  Dog  sleds  online:  Creating  a virtual  field  trip.  Social  Studies  Review,  i<?(l),  43-46. 

Stainfield,  J.,  Fisher,  P.,  Ford,  B.,  & Solem,  M.  (2000).  International  virtual  field  trips:  A new  direction? 
Journal  of  Geography  in  Higher  Education,  24^  255-262. 


354 


BEST  COPY  AVAILABLE 


A Typology  for  Identifying  Teachers’  Progress  in  ICT  uptake 


Barney  Clarkson,  Edith  Cowan  University,  Multimedia  Department,  Australia 
b.clarkson@ecu.edu.au 

Ron  Oliver,  Edith  Cowan  University,  Multimedia  Departmert,  Australia 
r.oliver@ecu.edu.au 


Abstract:  This  paper  describes  the  design  of  an  instrument  to  help  identify  teachers’  level  of 
ICT  uptake.  The  instrument  takes  the  form  of  a typology  matrix  comprising  four  stagps  in  ICT 
uptake  across  a continuum  describing  teachers’  levels  of  dependence.  Identification  of 
teachers’  positions  in  the  typology  matrix  is  determined  by  their  affective,  cognitive  and 
demonstrated  states  of  ICT  application  and  use.  The  paper  describes  the  process  of  determining 
the  position  of  two  elementary  school  teachers  within  the  typology  and  discusses  the  reliability 
and  validity  of  the  instrument  and  the  placement  process. 


Introduction 

Schools  and  teachers  are  in  midst  of  massive  changes  since  the  pressure  to  use  information  and  communication 
technologies  (ICT)  began  in  the  late  1980s.  Addressing  teachers’  needs  has  become  an  important  element  for 
many  school  and  educational  systems  around  the  Western  world  (Elmore,  1992).  Unfortunately  educational 
institutions  under  pressure  to  utilise  ICT,  have  often  found  it  easier  to  see  the  technology  issues  rather  than  the 
human  ones  (eg.  Hodas,  1993;  McClintock,  1988;  Papert,  1998).  The  resulting  outcomes  have  tended  to  be 
applications  of  ICT  based  on  motives  and  directions  that  often  fail  to  capitalise  on  the  opportunities  and 
affordances  the  new  technologies  can  bring  to  teaching  and  learning. 


The  Need  For  a Scale  of  ICT  Uptake 

Much  of  the  professional  development  for  educators  to  aid  in  their  ICT  uptake  has  not  had  the  expected  effect, 
even  though  there  have  been  many  attempts  at  providing  support  for  teachers  during  their  ICT  progression  and 
many  attempts  at  describing  various  ICT  diffusion  or  uptake  processes  (Cheung,  1999;  Christensen,  1997; 
Mahajan  & Peterson,  1985;  Marcinkiewicz,  1994;  Russell,  1995;  Sandholtz,  Ringstaff,  & Dwyer,  1992). 
Holloway  (1996)  suggests  the  processes  may  be  too  techno-centric  and  need  to  consider  a more  humanist 
perspective. 

Writers  concerned  with  change  in  education  argue  that  most  effective  approaches  to  professional  development 
consider  both  the  whole  system,  and  the  people  in  it  (eg.  Fullan,  1 999) . Part  of  the  challenge  of  orchestrating  any 
significant  and  sustained  change  in  educational  systems  is  that  these  entities  have  significant  natural  inertia,  and 
much  of  the  change  is  illusory  and  confused  with  movement  — ^the  never-ending  and  sometimes  chaotic 
movement  that  is  characteristic  of  large  organizations  (Huberman,  1992).  Teachers  need  to  learn  and  adapt  in 
such  an  environment.  In  fact  Fullan  (1999)  argues  "that  learning  occurs  on  the  edge  of  chaos,  where  a delicate 
balance  must  be  maintained  between  too  much  and  too  little  structure"  (p.ix).  This  makes  the  argument  that 
improving  the  balance  between  individual  teachers  needs  and  their  professional  development  is  important,  since 
the  divergence  of  an  individual’s  circumstances,  pressures,  resources  and  abilities  could  be  vast. 

One  of  the  problems  that  often  faces  professional  development  for  ICT  uptake  is  that  teachers  come  to  the 
programs  with  vastly  differing  backgrounds  and  predispositions  to  learning.  Their  previous  experiences  and 
existing  expertise  plays  a large  part  in  influencing  the  form  of  professional  development  they  need.  Currently  it 
is  difficult  to  know  how  to  differentiate  among  the  teachers  so  that  appropriate  learning  settings  may  be 
contrived  (eg.  Knowles,  Holton,  & Swanson,  1 998)  A number  of  researchers  have  discussed  the  training  needs 
of  teachers  looking  to  use  ICT  as  aids  to  their  teaching  and  described  strategies  that  can  take  teachers  beyond 
their  current  zones  of  comfort  (eg.  Schrum,  1998).  Within  much  of  this  discussion  is  an  implicit  understanding 
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of  the  need  to  be  able  to  identify  teachers’  current  development  level  with  respect  to  ICT  uptake  along  some 
typology.  It  was  this  need  that  provided  impetus  to  the  project  described  in  this  paper  which  sought  to  develop 
and  validate  such  a typology. 

The  Research  Setting  and  The  Model 

The  principal  aim  of  the  research  reported  in  this  paper  was  to  develop  a framework  by  which  teachers’ 
pedagogies  and  capabilities  with  ICT  could  be  mapped  onto  some  multi- stepped  scale  as  part  of  assessing  their 
ICT  uptake.  The  study  was  undertaken  with  interested  teachers  in  two  metropolitan  elementary  schools  in 
Western  Australia,  over  the  second  semester  of  their  school  year.  This  paper  reports  the  four-stage  typology  of 
ICT  uptake  which  was  developed. 

The  typology  was  derived  from  a series  of  models  of  learning  described  by  Brundage  (1980)  and  Boud  (1988) 
and  with  considerable  input  from  studies  of  teachers  and  their  teaching  practices  with  ICT.  In  this  sense  it  was 
grounded  in  the  teachers’  data  as  well  as  reliant  on  previous  research.  The  model  describes  four  stages: 
Dependence,  Counter-Dependence,  Independence  and  Interdependence  (shown  as  Figure  1).  These  stages  reflect 
typical  phases  through  which  all  learners  pass  through  as  they  achieve  mastery  on  a new  topic.  ICT  is  simply 
another  topic  for  learners  to  master,  in  this  less  techno-centric  conception  of  ICT  uptake  as  a learning  issue. 


Dependence  Counter-Dependence  Independence  Interdependence 

Figure  1:  The  four  stages  for  learning  new  material  (from  Boud,  1988)  and  adapted  to  this  model  of  ICT  uptake 

Within  each  stage  of  the  model,  the  relevant  literature  provides  descriptions  that  can  help  to  identify  how 
teachers  feel  and  react.  Through  these  descriptions  it  would  seem  possible  to  allocate  any  teacher  to  a location 
on  the  scale.  To  improve  triangulation  and  to  ensure  consistency  of  reaction  from  teachers,  descriptions  for  the 
typology  were  further  developed  to  provide  a more  sensitive  means  to  identify  teachers’  positions.  Three 
domains  were  developed  for  each  of  the  stages  to  enable  different  aspects  of  teachers’  experiences  and 
predispositions  to  inform  their  placement.  These  three  domains  are  described  as:  feelings,  understandings  and 
behaviours.  The  domains  were  chosen  to  match  the  domains  of  human  activity  proposed  by  Bloom  in  the  1950s 
and  still  a useful  distinction  (Krathwohl,  Anderson,  & Bloom,  2001).  The  stages  describe  teachers’  affective 
states,  their  cognitive  states  and  the  ways  these  are  manifest  in  their  actual  teaching.  If  these  stages  were  truly 
distinct  and  credible,  then  it  was  expected  in  developing  the  typology  that  teachers  would  be  located  at  one  stage, 
with  their  ratings  for  feelings,  understandings  and  behaviours  falling  roughly  into  roughly  the  same  stage. 
Following  this  line  of  reasoning,  the  typology  seemed  to  promise  a means  by  which,  in  theory,  ICT  uptake 
could  be  measured  by  progress  along  the  four  stages  and  within  the  three  domains  simultaneously.  The  typology 
was  presented  as  a 4x3  matrix  figure  2)  with  cells  defining  the  basic  layout.  It  was  named  the  ADL  uptake 
model  in  an  attempt  to  capture  the  role  of  Autonomy,  Dependence  and  Learning  in  the  ICT  uptake  process. 


Domam"'''^^ 

Dependence 

Counter-  Dependence 

Independence 

Interdependence  | 

Feelings 

Uhdersia^^ 

Behaviours 

Figure  2:  Four  stages  of  ICT  uptake  proposed  as  the  ADL  model 


To  enable  the  typology  to  be  applied  within  general  settings,  distinct  and  unique  descriptions  for  each  of  the  cells 
were  developed.  These  descriptions  needed  to  provide  ways  to  describe  teachers  across  the  4 stages  within  the  3 
domains.  Fig.  3 demonstrates  Descriptors  taken  from  the  Understandings  row  across  the  4 stages. 


Survival  issues  with  ICT 

Limited/local 

Directed, 

ICT  thinking,  usage 

Understandings 

dominate;  Concerned 

thinking  about 

focussed  ICT 

now  second  nature; 

more  with  own  mastery 

ICT;  Teaching 

thinking; 

Learning  oriented, 

and  learning 

oriented 

Teaching  and 
learninq  oriented 

student  focussed 

Figure  3:  Example  of  Descriptors  for  the  four  stage  model,  in  this  case  over  the  domain  ’Understandings’ 


Each  cell  of  the  4x3  table  was  further  elaborated  with  Pointers  that  are  representative  of  that  stage  and  domain. 
For  example  at  Dependence^  the  teachers’  Understanding  might  typically  include  some  but  probably  not  all  of 
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these:  of  ICT’s  flexibility,  instability;  questions  own  role;  cannot  yet  comprehend  ways  to  use  ICT 

broadly;  believes  ICT  is  an  object  to  be  examined,  learned,  taught  about;  seeks  explicit  even  unequivocal 
standards,  regards  explicit  step-by-step  guidance  as  critical;  prefers  single  solutions. 

These  Descriptors  and  Pointers  are  not  indicative  of  fixed  immutable  steps  that  every  learner  passes  through  but 
general  and  overall  predictable  patterns  to  their  feelings,  thoughts  and  behaviours,  which  are  representative  of 
each  stage.  Some  will  pass  through  them  more  quickly  than  others  for  example,  but  the  distinctions  are,  in 
Brundage  & MacKeracher’s  terms,  “an  aid  to  analysis  and  not  meant  to  reflect  universal,  absolute 
differences”  (1 980,  p.  1 1 ) 

Mapping  the  Data — Two  Teachers’  Records 

To  test  the  utility  and  efficacy  of  the  typology,  a number  of  elementary  teachers  were  interviewed  and  observed 
whilst  teaching  as  a means  to  discover  their  understandings,  feeling  and  behaviours  in  relation  to  their  use  of  ICT 
in  the  classroom.  The  teachers  were  then  mapped  into  the  various  cells  of  the  typology  and  their  positions 
investigated. 

Using  the  data  obtained  for  each  of  the  teachers  in  the  study,  the  ADL  matrix  was  completed  and  teachers’ 
positions  noted.  It  was  a relatively  simple  process  to  fit  the  teachers  into  the  foupstage  ICT  uptake  scale  and 
thus  to  give  them  some  form  of  quantitative  index  that  could  be  compared  at  a Ider  date  to  ascertain  if  progress 
in  terms  of  advancing  their  positions  could  be  determined. 

The  findings  indicated  that  credible  roles  like  novice  through  to  sustained  ICT  user  were  categories  that  could  be 
identified  for  teachers  fitted  to  this  typology,  although  it  was  decided  to  avoid  using  role  names  as  the  interest 
was  on  classifying  their  current  location  or  stage,  and  allowing  them  to  move  stages  as  they  needed  and  as  they 
progressed. 

It  was  found  that  the  quickest  way  to  rate  the  teachers  was  to  review  their  data  record  from  interviews,  notes  etc 
and  decide  which  cells  represented  them  best.  For  validation  purposes  in  the  study,  the  teachers  were  mailed  a 
copy  of  the  completed  table  and  each  was  asked  to  circle  positions  on  the  table  which  they  felt  were  relevant  and 
descriptive  of  themselves.  Positions  where  the  teachers  marked  multiple  pointers  provided  sufficient  evidence  to 
suggest  that  they  perceived  themselves  to  be  ‘in’  that  cell.  Remarkable  commonality  between  the  researchers’ 
predicted  and  teachers’  own  ADL  matrices  was  found. 

Teachers’  positions  within  the  typology  were  indicated  on  summary  charts  in  which  the  cells  that  represented 
their  status  were  shaded  grey.  For  example  a novice  might  have  a chart  (called  their  location  c/iar/)  with  all 
the  left  column  shaded,  and  nothing  else  (see  Fig.  4). 
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Figure  4:  Sample  ADL  location  chart  of  ICT  uptake  for  a Novice  ict- using  teacher 


Using  The  Typology 

The  following  sections  describe  several  teachers  who  were  investigated  and  whose  ICT  uptake  was  judged  and 
located  on  the  ICT  uptake  location  chart. 

Teacher  A:  Paula. 

At  the  end  of  the  research  period  Paula  was  very  much  in  control  of  her  learning  environment.  She  had  realised  a 
few  years  earlier  that  she  was  becoming  irrelevant,  and  decided  that  she  wasn’t  prepared  to  continue  unchanged 
in  her  teaching.  When  asked  if  the  advent  of  the  computer  lab  in  her  school  had  put  extra  pressure  on  her,  she 
replied: 

PAULA: I can  only  speak  for  me  but  it  should  have  put  pressure  on  everybody.  It  did  to  an  extent,  but  on 
myself. . .because  of  my  age...  I wanted  to  last  the  distance,  and  I saw  a lot  of  people  {here  at  school}  that  I didn’t  want 
to  be  like.  I didn’t  want  to  have  their  attitude,  and  I knew  that  I haven't  perfected  the  craft  of  teaching  because  it  is 
always  changing...  according  to  the  demands  of  society,  A lot  of  them  haven’t  noticed  that!  (Inti  :line56) 

This  became  the  impetus  for  her  to  rethink  her  whole  approach  to  teaching.  She  started  telling  her  students  that 
school  was  their  job,  not  hers.  She  began  to  attend  professional  development  courses  that  were  going  to  help  her 
change  the  way  she  thought  and  taught,  and  it  had  a kick-back  effect  on  her  usage  of  ICT: 
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♦INT:  So  you  agreed  to  a plan  that  put  a block  of  computers  into  a lab  first,  and  class  machines  later?  And  you  thought 
that  was  a good  idea,  but  then...? 

PAULA:  It  still  is,  but  it  is  frustrating  having  to  wait.  With  the  way  I think,  and  maybe  a few  others  think,  as  subjects 
become  more  integrated  and  we  go  that  way,  and  we  bring  aboard  thinking  skills,  allow  the  kids  to  learn  how  to  think, 
to  set  their  goals  and  so  on.  Whatever.  It  means  that  they  have  to  work  more  independently,  they  need  to  be  more  self- 
motivated,  and  given  the  opportunity  to  set  their  own  goals.  This  is  as  opposed  to  someone  playing  entertainer,  and 
pulling  surprises  out  of  a hat  for  the  students  each  day.  As  kids  set  their  own  goals  they  learn  to  start  saying  I want  to 
search  on  the  Inlerael  for  something,  I want  to  scan  this  picture  in  for  this  project,  takes  some  digital  photos  and  so  on. 
Now,  over  three  years  we  have  gone  from  ’^a’^  lesson  in  the  computer  lab  - and  it  was  reallymy  lesson  I was  the  one  on 
the  learning  journey  — and  now  it’s  them  telling  me  what  they  are  doing  and  planning.  {Inti  :line86} 

By  the  time  Paula  assembled  her  ICT  uptake  location  chart,  she  was  not  only  making  use  of  ICT  as  a learning 
tool  in  her  class,  but  she  was  also  encouraging  her  students  to  do  the  same.  In  fact  some  of  her  Year  4 students 
were  the  school  experts  on  various  pieces  of  ICT: 

PAULA:  In  my  class  there  was  a handful  who  were  experts  at  taking  photos,  printing  and  down-loading.  Another  little 
kid  decided  he  was  interested  in  scanning.  Now  I haven't  got  a clue  how  to  do  those...  I know  they're  not  hard...  but  why 
should  I bother?  So  I handed  over  to  them.  It’s  more  of,  me  standing  back,  getting  time  to  understand  and  observe 
(while  they  do  the  work} . The  kids  know  that  I can't  down-load  pictures.  It  might  take  me  just  five  minutes  to  learn,  but 
it's  not  as  important  to  me  as  it  is  to  them.  When  I get  my  own  digital  camera  then  it  will  be.  So  the  kids  don't  come 
along  [to  school]  to  watch  me  work,  instead  I stand  back  and  watch  them.  They  are  more  in  control  of  what  they  do. 

{Intl:line212} 

Although  she  could  be  described  as  highly  student-centric  in  her  approach,  she  had  become  a competent  user  of 
ICT;  and  it  was  harder  still  to  fault  her  use  of  it  in  her  teaching.  She  was  willing  to  share  ideas  with  others,  was  a 
mentor  to  Nora,  the  adjacent  Year  3 teacher,  and  they  left  the  room  divider  open  permanently,  so  that  they  could 
share  students,  ideas  and  support..  Her  ICT  planning  and  usage  had  become  second  nature  to  her,  and  she  was 
focussed  on  her  students’  performance  not  her  own.  Her  use  of  the  Internet  and  e-mail  as  tools  were  indications 
that  she  had  become  not  only  Independent  but  also  Interdependent  in  her  ICT  usage.  Her  ICT  uptake  bcation 
chart  was  very  predictable  (see  Figure  5). 
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Figure  5:  Final  ICT  uptake  location  chart  for  Paula 


Very  few  of  the  teachers  chose  pointers  in  more  than  two  columns,  and  very  few  did  not  choose  all  the  cells  in  a 
column  either.  These  facts  provided  support  for  the  principle  that  there  was  some  underlying  pattern  to  the 
categoris^ions. 

Nevertheless,  this  did  not  explain  all  the  charts.  Some  teachers’  charts  did  clearly  cover  two  columns-^erhaps 
Paula’s  chart.  Figure  5,  was  an  example  of  this — and  it  was  felt  that  there  could  be  an  explanation  for  this. 
Teachers  who  straddled  multiple  categories  along  the  scale,  it  was  decided,  might  be  in  the  process  of  transition 
between  categories.  Support  for  this  notion  was  strengthened  by  the  observation  that  teachers  rarely  straddled 
more  than  two  stages  when  they  completed  their  chart.  In  Paula’s  case  she  may  be  showing  evidence  that  she 
was  covering  two  categories,  or  stages,  but  the  fact  that  she  was  performing  at  the  upper  of  the  two  stages  (in 
other  words,  she  had  circled  sufficient  pointers  in  her  Interdependence  behaviours  cell,  the  one  at  the  bottom 
right  of  her  chart)  was  good  reason  to  allow  her  this  categorisation.  David  was  a better  example  of  someone  in 
transition. 

Teacher  B:  David 

Although  he  was  negative  about  ICT,  David  bought  a computer  during  the  research  period  and  began  to  make 
use  of  it  at  home.  His  interest  levels  were  higher  than  his  performance  levels,  as  he  found  them  initially 
confusing  and  was  unwilling  to  try  any  but  the  most  mundane  activities  with  his  students.  He  believed  that  the 
school  and  the  Department  of  Education  (his  employer)  expected  him  to  teach  students  about  ICT,  and  that  they 
had  never  provided  enough  support  to  help  him  even  start  the  process.  He  admitted  that  his  usage  was 
unsystematic,  and  felt  that  he  was  exp  ected  to  show  high  levels  of  skill  with  computers  all  the  time: 
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DAVID:  Oh,  Tm  quite  sure  that  Tm  required  to  have  every  kid  completely  computer  literate,  and  doing  very  complex 
computer  type  things.  I'm  sure  that’s  happening  here.  {Inti  ;line  46} 

Not  only  does  he  have  unrealistic  expectations  of  his  own  use,  he  puts  pressure  on  himself  as  well.  When  one 
parent  offered  to  help  on  Fridays,  he  was  pleased  and  pressured  in  equal  measures: 

DAVID: ....  I had  the  head  of  the  P & C in  my  classroom,  an  absolutely  wonderful  person  called  Jaccques,  very  very 
computer  literate,  and  he's  actually  been  in  the  classroom  on  Friday  mornings,  taking  loads  of  kids  doing  word 
processing... 

*INT:  I saw  your  thanks  to  him  in  the  newsletter. 

DAVID:  yes,  but,  I can  see  in  his  eyes,  that  he  knows  I don't  know  what  the  hell  I'm  doing.  He’s  done  extra  things  like 
buying  a set  of  headphones,  brought  in  a few  CDs;  and  every  Friday  morning  I wonder  what  the  hell  am  I going  to  get 
Jacques  to  do  on  computers.  So  I feel  pressurised  there,  I feel  anxious  and  worried. . . (Inti  :line70} 

It  is  apparent  that  David  feels  that  he  must  provide  all  the  direction  when  it  comes  to  computer  use.  It  is  a focus 
of  his  attention  and  not  a tool  or  a conduit  to  a learning  purpose.  These  are  the  characteristics  of  a Dependent 
person  on  the  ICT  uptake  typology.  At  the  same  time  he  admits  that  computers  definitely  have  a place  in 
schools,  and  that  he  would  like  to  improve.  He  doesn’t  know  how  he  will  become  an  adopter  of  computers  in  his 
class,  but  he  has  seme  ideas  which  seem  to  simultaneously  guide  and  stress  him: 

*INT:  So  how  will  you  become  this  person  moving  from  considering  to  adopting? 

DAVID:  ...  I don’t  know.  I don’t  have  a clear  idea  in  my  head,  or  goal,  of  how  the  computer  can  be  used  in  the 
classroom.  To  me  it's  always  been  a token  gesture.  Where  you’re  rostering  a child  on  a computer,  and  we  were  seen  to 
be  doing  some  something,  but  I can’t  see  how  it's  integrated  into  the  learning  environment.  I do  actually  like  the  idea 
very  much  of  using  it  for  remedial  purposes,  like  teaching  phonics,  that  sort  of  stuff  And  I like  the  idea  of  using  the 
Internet  in  the  classroom. ...  ahhh,  open-ended  research  type  stuff;  and  I like  the  idea  of  being  able  to  network  with 
other  teachers  through  the  Internet.  (inrl  :line91 } 

David,  with  no  clear  idea  of  how  computers  can  be  used,  and  who  regards  their  usage  as  part  of  a ‘token 
gesture’,  is  relatively  low  on  the  typology  of  ICT  uptake. 

In  summary  he  perceives  it  as  a novelty  or  even  a threat,  is  defensive  about  his  lack  of  progress  and  seems 
unlikely  to  make  any  progress  without  specific  and  localised  support.  His  understanding  is  limited,  but  as  well  as 
views  about  ICT  as  a teaching  tool,  he  also  envisages  some  more  flexible  and  open-ended  uses  of  ICT  which 
suggests  that  his  thinking  at  least  is  a little  more  towards  Counter-Dependence  than  either  his  Feelings  or  his 
Behaviours.  Hence  his  chart  has  a peak  as  shown  in  Figure  6. 
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Figure  6:  David's  Initial  ICT  uptake  location  chart 


Discussion 

The  placement  of  the  teachers  on  the  chart  provided  a number  of  interesting  findings  which  merit  some 
discussion.  In  the  first  instance,  the  typology  was  found  to  provide  quite  consistent  descriptions  for  the 
participating  teachers.  The  teachers’  positions  in  the  table  tended  to  be  within  the  discrete  stages  across  the  3 
domains  being  used.  There  was  a high  degree  of  consistency  in  this  suggesting  that  the  typology  was  able  to 
reliably  place  teachers  in  a stable  position. 

Furthermore  there  was  a high  degree  of  consistency  between  the  positions  which  the  researchers  placed  the 
teachers  and  where  they  placed  themselves.  This  consistency  provided  some  sense  of  triangulation  of  the  data 
and  provided  some  assurance  that  the  typology  could  accurately  place  teachers. 

In  a number  of  instances  teachers  were  found  to  sit  in  one  stage  but  to  have  some  connection  with  a cell  in  an 
adjacent  stage.  These  instances  tended  to  suggest  teachers  whose  position  on  the  typology  was  not  stable  and 
was  indicative  of  a teacher  who  was  perhaps  moving  along  the  typology  to  an  advanced  level  of  ICT  uptake. 
This  finding  was  confirmed  on  several  occasions  when  such  teachers  were  found  to  have  moved  positions  along 
the  typology  to  stages  indicative  of  greater  levels  of  ICT  use  and  uptake. 
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Summary 


This  paper  has  described  the  design  of  an  instrument  to  help  identify  teachers’  level  of  ICT  uptake.  It  was  less 
techno- centric  than  many  previous  versions,  and  was  shown  to  have  some  utility.  Two  aspects  of  describing  a 
teacher’s  location  were  evident.  Some  fitted  easily  to  the  three  domains  of  a category  suggesting  that  they  were 
characterised  by  that  stage  and  some  stability;  others  tended  to  straddle  two  (but  rarely  more)  stages,  suggesting 
that  they  were  in  the  process  of  change. 

All  the  possibilities  for  placement  on  the  scale  were  expected  to  describe  people  with  very  different  neecfe  about 
ICT  and  expectations  about  ICT  training.  In  fact  it  was  evident  that  these  teachers  with  their  diverse  views  and 
different  ratings  ion  the  ICT  uptake  typology,  had  quite  different  support  needs  and  diverse  professional 
development  needs,  even  though  they  may  have  been  in  the  same  schools  and  similar  circumstances. 

It  was  supposed  that  teachers  other  than  those  in  the  two  case  schools  would  be  undergoing  similar  stresses  and 
also  need  a variety  of  support  and  professional  development  services.  Thus  the  typology  could  be  applied  more 
widely.  A useful  next  step  could  be  the  development  of  an  instrument  or  questionnaire  to  investigate  an 
alternative  way  to  finetune  the  placement  of  people  onto  the  grid,  as  the  current  one  might  provide  too  many 
cues  to  users  who  had  seen  it  before. 

The  typology  appears  to  provide  a reliable  and  valid  way  to  determine  teachers’  positions  as  users  of  ICT  in  their 
teaching  and  learning.  Further  study  will  be  carried  out  to  ensure  the  typology  can  be  used  effectively  in 
mainstream  settings.  If  and  when  this  is  possible,  the  typology  would  appear  to  provide  considerable  assistance 
as  a means  to  helping  to  ensure  that  professional  development  programs  can  be  targeted  in  meaningful  ways  to 
the  recipients  based  on  their  needs  and  their  current  capabilities  as  users  of  ICT  in  their  teaching. 
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Abstract:  Action  learning  is  effective  learning  and  development  through  the  use  of  real 
life  context  based  problems,  applications,  and  solutions.  At  Grant  MacEwan  College,  in 
Edmonton,  Alberta,  Canada,  emphasis  in  faculty  instruction  is  on  practical,  real  life 
situations  that  faculty  encounter  in  their  classrooms  and  in  teaching.  Moving  from  a 
program  of  presentations  and  passive  participation,  professional  development  at  Grant 
MacEwan  now  incorporates  practical,  hands-on  experiences  that  reflect  the  current  needs 
of  faculty  through  action  learning.  This  new  directive  for  professional  development 
focuses  on  making  changes  to  the  learning  environment  rather  than  forcing  a personal 
change  in  the  faculty  members  teaching  style. 

Action  Learning  began  in  the  corporate  sector  as  a method  for  solving  real  life  problems  in  a creative 
and  practical  manner.  It  moved  into  educational  settings,  fitting  well  into  graduate  level  programs  that 
based  their  course  activities  on  constructivist  and  situated  cognition  learning  theory  (Ritland-Bannan, 
1999).  Similarly,  it  fit  well  with  educators  attempting  to  implement  changes  to  their  teaching  methodology 
(Beaty,  1999:  Gold,  2001).  Action  learning  is  based  on  a team  approach  whose  function  it  is  to  solve  real 
problems  by  trying  various  solutions,  reflecting  upon  the  impact  of  those  solutions,  and  creating 
improvements  to  those  solutions. 

If  action  learning  can  help  companies  solve  problems,  students  learn  through  practical  experience,  and 
teachers  try  new  learning  methodologies,  can  it  help  college  instructors  learn  new  ways  of  teaching  with 
technology?  Grant  MacEwan  College  educational  technology  designers  thought  it  could. 

Formerly  a program  of  occasional  one  or  two  day  presentations  and  passive  participation,  faculty 
training  in  using  technology  for  teaching  and  learning  now  focuses  on  practical,  hands-on  experiences  that 
reflect  current  needs  of  faculty.  Using  principles  of  action  learning,  college,  program,  and  individual 
faculty  needs  are  addressed  through  continuous  workshops  and  individual  consultations  that  incorporate  (a) 
Context  -based  learning,  (b)  team  support,  (c)  reflective  questioning  and  (d)  action  based  solutions. 

A key  concept  in  action  learning  is  that  of  working  on  real  problems  with  real  people,  in  real  situations 
(Gallant,  2000).  Workshops  are  marketed  as  possible  solutions  to  problems  faculty  may  be  facing.  For 
example,  the  workshop  on  Web-based  discussion  forums  is  described  in  terms  of  addressing  the  problem  of 
promoting  greater  interactivity,  deeper  levels  of  analysis,  more  critical  thinking,  and  improved 
collaboration  in  course  discussions.  Instructors  who  attend  these  workshops  come  in  response  to  a problem 
they  want  to  address.  Attendance  at  these  workshops  is  often  a response  to  “just-in  time”  support  required 
for  a project  need.  However,  faculty  members  can  and  do  request  immediate  one-on-one  support  if  the 
workshop  schedule  does  fit  their  needs.  This  is  not  only  required  to  meet  the  immediate  need  of  a project. 
It  is  also  desirable  because  of  the  many  different  skill  levels  found  within  members  of  a team.  Faculty 
members  have  to  feel  comfortable  asking  for  remedial  as  well  as  advanced  skill  development  in  the  use  of 
technical  applications  (Schifter,  2000).  Much  of  the  training  that  goes  on  at  MacEwan  is  comprised  of 
office  visits  to  give  immediate  support  and  training  to  group  and  individual  technology  initiatives.  Our 
motto  is  “Yes,  we  make  house  calls!” 

Context-based  learning.  Training  is  hands-on.  Participants  sit  down  at  computers  and  work  with 
actual  applications,  with  content  they  can  use  in  their  own  teaching  projects.  The  sessions  are  not 
demonstrations  but  rather  a combination  of  sequential  skill  building  and  peer  assistance  that  enables  them 


361 


to  use  a certain  technology  to  solve  their  teaching  problems.  An  illustrated  manual  is  produced  for  each 
workshop;  however  it  may  or  may  not  be  used  by  all  participants.  It  is  not  uncommon  to  see  one  or  two 
members  in  each  group  trying  things  out  by  trial  and  error  and  ignoring  the  manual  and  the  facilitator 
completely.  Regardless  of  learning  styles,  the  goal  for  all  participants  is  to  leave  the  session  with  a usable 
product  that  they  can  test,  make  changes  to,  and  implement  in  their  courses  (Baylor,  2000). 

Team  support  This  brings  up  the  concept  of  teamwork  and  community.  Whether  instructors  are  in  the 
same  program  or  not,  the  experience  of  planning  new  actions  to  address  a problem  bind  them.  One  of  first 
activities  in  a Grant  MacEwan  workshop  is  getting  to  know  the  people  around  you  in  the  training  room. 
What  program  are  they  in?  Where  are  their  offices?  What  are  they  doing  to  solve  a common  problem? 
What  are  they  trying?  What  is  working?  What  is  not?  How  are  they  using  technology  to  solve  this 
problem?  Participants  are  given  a list  of  the  colleagues  who  attended  the  workshop  so  they  can  go  back 
down  the  hall  a few  days  later,  and  ask  that  person,  “How  did  you  do  that  thing  in  WebCT  to  allow  your 
students  to  post  their  assignments  again?”  Forming  a supportive  network  of  colleagues  is  part  of  the  plan  in 
faculty  training.  When  training  is  given  to  specific  programs,  a team  approach  is  used  to  identify  problems 
and  evaluate  solutions. 

Reflective  questioning  and  action  based  solutions.  Faculty  training  based  on  action  learning  requires  a 
team  to  constantly  think  about  the  results  of  a decision  and  how  it  will  impact  learning.  MacEwan  training 
is  based  on  decisions  that  programs  and  instructors  make  and  the  actions  that  are  taken  as  a result  of  that 
decision.  Furthermore,  once  an  action  is  implemented,  instructors  question  and  reflect  upon  the 
effectiveness  and  efficiency  of  that  decision.  Problems  and  mistakes  are  shared  within  workshops  as 
learning  opportunities.  Because  teaching  with  technology  is  new  and  unfamiliar  to  most  faculty  members, 
questions  must  be  raised  so  that  fears  and  concerns  are  addressed.  Reflection  about  decisions,  and  changes 
to  those  decisions,  are  based  on  questions  and  consultation  with  team  members.  Final  decisions,  if  there 
ever  is  such  a thing,  are  made  within  the  context  of  the  instructors’  real  situation  after  questions  and 
reflection  about  initial  actions. 

As  facilitators  and  trainers  for  our  faculty,  we  try  to  be  the  contact  that  instructional  teams  use  for 
linking  to  other  groups.  We  try  to  be  the  catalyst  to  move  them  to  evaluate  their  current  mode  of  teaching. 
We  try  to  monitor  their  process  to  see  when  and  where  our  intervention  can  help.  We  try  to  set  a climate 
that  encourages  trust  and  willingness  to  try  new  initiatives.  And  we  try  to  be  coaches  who  encourage  and 
support  faculty  to  take  responsibility  for  their  own  professional  development  in  the  field  of  educational 
technology.  This  facilitation  model  (Marquardt,  1999)  required  by  action  learning  principles  helps  faculty 
try  new  methodologies  in  teaching... .new  methodologies  that  incorporate  technology  and  that  are  based  on 
action  learning. 

References 


Baylor,  A.  L.  (2000,  June).  Cognitive  strategies  for  training  with  technology.  TechTrends,  44,  13-15. 

Beaty,  L.  (1999,  Fall).  Consultation  through  active  learning.  New  Directions  for  Teaching  and  Learning, 
79,  51-58. 


Gallant,  G.  M.  (2000,  Winter).  Professional  development  for  web-based  teaching:  Overcoming  innocence 
and  resistance.  New  Directions  for  Adult  and  Continuing  Education  y 88,  69  -77. 

Gold,  S.  (2001,  May).  A constructivist  approach  to  online  training  for  online  teachers.  Journal  of 
Asynchronous  Learning  NetworkSy  5,  35  -57. 

Marquardt,  M.  J.  (1999).  Action  learning  in  action:  Transforming  problems  and  people  for  world-class 
organizational  learning.  Palo  Alto,  CA:  Davies -Black  Publishing. 

Ritland-Bannan,  B.  (2000).  Teaching  instructional  design:  An  action  learning  approach.  Retrieved  March 
1 , 2002  from  http://itech  1 .coe.uga.edu/itforum/paper37/paper37.html 

Schifter,  C.  (2000,  March-April).  Faculty  motivators  and  inhibitors  for  participation  in  distance  education. 
Educational  Technologyy  43-46. 


3-62 


Promoting  Peer-to-Peer  Discourse  for  Collaborative  Mathematics  in 
Canadian  Grade  7 Classrooms 


Jonathan  Cohen,  James  Dai,  Michael  Wu,  Troy  Wu  and  Maria  Klawe 
E-GEMS  Research  Lab 
University  of  British  Columbia 
Canada 

egems@cs.ubc.ca 


Abstract:  Whereas  traditional  research  on  collaborative  educational  systems  has 
primarily  focused  on  how  to  define  better  modes  of  digital  interaction,  this  approach  is 
found  lacking  when  applied  to  developing  collaborative  systems  for  elementary  school 
aged  children.  It  is  aeatively  and  collaboratively  restrictive  to  filter  the  enthusiastic 
interactions  of  these  excited  12-year-old  children  through  progressively  more 
complicated  GUIs.  Current  &GEMS  research  examines  design  factors  of  educational 
systems  that  recognize  and  facilitate  the  social  context  of  the  classroom  and  hopes  to 
encourage  peer-to-peer  social  discussion.  By  correlating  observed  interactions  in  the 
digital  domain  with  those  in  the  social  domain,  we  hope  to  shed  light  on  design  factors  of 
collaborative  systems  that  can  be  an  integral  and  exciting  part  of  a child’s  mathematical 
education.  This  paper  describes  the  vessel  of  &GEMS  research  into  socially  engaging 
design  factors  - the  collaborative  educational  game  PrimeClimb. 


Introduction 


PrimeClimb,  as  shown  in  the  left  image,  is  a collaborative  game  in  which  a pair  of  children  attempt  to  climb  to 
the  top  of  a mountain  of  numbers  together.  In  order  to  reach  the  top  of  each  mountain,  students  must 
collaboratively  avoid  falling-  a feat  accomplished  by  recognizing  shared  common  factors  between  the  numbers 
on  which  they  stand.  Collaboration  is  reinforced  through  the  fact  that  in  order  to  reach  the  goal  each  player 
must  observe  their  partner’s  number  in  order  to  move  without  falling.  There  is  a tool  - the  magnifying  glass - 
that  can  be  used  to  shed  light  on  the  factors  of  any  number.  Magnifying  glasses  are  hint-giving  resources  that 
are  limited  and  shared  amongst  the  two  players.  Our  observation  sessions  were  used  to  provide  feedback  in 
developing  the  current  design  factors  of  the  magnifying  glass. 

The  Study 

The  sessions  of  the  study  proceeded  as  follows:  the  pairs  of  children  were  first  given  a pre- 

questionnaire  to  access  their  mathematic  factorization  skills  and  general  attitudes  toward  collaboration  in  math 
and  computer  games.  They  then  played  PrimeClimb  together  for  20  minutes  and  the  session  ended  with  a post- 
questionnaire that  reassessed  their  math  skills  and  attitudes.  The  focus  of  the  PrimeClimb  study  has  been  to 
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analyze  the  social  context  of  collaboration  in  a digital  environment  in  terms  of  verbal  and  non-verbal 
interactions.  Our  data  capture  and  analysis  methodology  must  reflect  this  focus.  Each  play  session  was 
computer-logged  and  video-recorded.  In  order  to  correlate  the  digital  events  in  the  computer  log  with  social 
discussion  and  interactions,  an  absolute  clock  was  displayed  on  the  game  screen  at  all  times.  Each  video 
captured  a player’s  upper  body  along  with  her  game  screen;  thus  two  tapes  were  produced  from  each  session 
(see  the  right  image,  the  figure  has  been  blurred  to  preserve  the  subject’s  identity).  Captured  audio  transcripts 
have  proved  successful  in  correlating  digital  events  to  facilitate  design  analysis  in  ethnography  (Suchman,  2000) 
and  math  education  research  (Pirie,  1996).  We  hope  to  use  our  captured  video  data  to  correlate  the  social 
interactions  with  the  PrimeClimb  game  events. 

The  detailed  analysis  of  the  PrimeClimb  study  data  is  underway.  A brief  glance  already  reveals  a 
couple  of  encouraging  signs.  Firstly,  different  categories  of  social  interactions  of  children  playing  PrimeClimb 
are  becoming  increasingly  clear.  As  in  past  research  from  mathematics  education,  the  categorization  of  peer-to- 
peer  discussion  in  a mathematical  context  must  take  place  first  and  foremost  before  the  analysis  of  the 
pedagogical  values  of  those  categories  can  be  carried  out  (Pirie,  1998).  A few  possible  patterns  of  interactions 
have  emerged  that  include:  a player  explaining  to  the  other  player  why  a move  is  not  allowed  using 
mathematical  language,  a player  explaining  the  reason  for  a failed  move  using  mathematical  knowledge  and  a 
player  strategizing  and  logically  planning  a series  of  moves.  An  example  of  the  latter  is  when  a player  states 
that  he/she  is  on  a prime  number  and  encouraging  her  partner  to  move  as  high  as  possible  (this  is  an  easy 
strategy  but  could  prove  faulty  as  multiples  of  prime  numbers  share  common  factors).  We  hope  to  apply  this 
categorization  and  consequent  analysis  of  peer-to-peer  discussion  about  math  t)  our  computer  supported 
collaborative  environment.  Secondly,  we  have  noted  that  all  the  children  had  very  little  difficulty  grasping  the 
interface  to  the  game  - a result  of  multiple  prototype  sessions  that  have  shaped  the  game  interface.  This  means 
PrirreClimb  is  able  to  be  a stable  foundation  upon  which  other  research  questions  about  collaboration  can  be 
launched.  It  has  been  argued  that  educational  games  should  not  be  specially  designed  for  girls  at  the  risk  of 
marginalizing  them,  instead  games  should  be  designed  non-gender  specifically  in  which  the  players  can  seek 
their  own  identities  through  collaborative  tackling  of  the  challenge  at  hand  (Cassel  2000).  PrimeClimb  seems  to 
have  succeeded  in  this  respect  so  far  as  boys  and  girls  employed  a vast  array  of  strategies  and  were  all  readily 
accommodated  by  the  game. 

Conclusion 

The  data  collection  part  of  the  PrimeClimb  study  has  been  completed  and  the  data  analysis  has 
begun  with  promising  signs.  Our  ultimate  vision  is  to  build  collaborative  educational  systems  that  fit 
seamlessly  into  and  facilitate  the  social  context  of  the  classroom  - this  study  has  so  far  shown  that  ethnography 
and  math  education  research  have  methodologies  that  can  provide  valuable  insights. 
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Distance  Learning  Instruction:  A New  Model  of  Assessment 
By  Vicki  L.  Cohen,  Fairleigh  Dickinson  University 


Background 

Distance  learning  is  rapidly  gaining  acceptance  as  a valid  means  of  course  delivery  in 
educational  institutions  across  the  country.  With  distance  learning  becoming  an  accepted 
form  of  instruction,  these  institutions  are  struggling  to  enhance  the  quality  of  teaching, 
learning  and  scholarship  over  the  Internet  while  trying  to  effectively  integrate  this  type  of 
instruction  into  the  curriculum.  Increasingly  educational  institutions  are  employing  this 
type  of  learning  as  part  of  their  course  offerings;  in  addition,  many  “virtual  universities” 
sponsored  by  state  departments  or  through  commercial  endeavors  are  offering  a variety  of 
programs  and  degrees  via  computer-mediated  distance  learning  courses.  Is  this  an 
effective  way  to  offer  instruction?  What  is  the  best  way  to  assess  this  type  of  instruction? 

As  more  and  more  institutes  of  higher  education  start  offering  computer-mediated 
distance  learning  courses,  it  becomes  increasingly  important  that  these  courses  are 
evaluated  effectively.  This  new  paradigm  of  learning  requires  a new  model  of 
assessment.  This  paper  presents  a model  that  focuses  on  six  constructs  that  computer- 
mediated  distance  learning  courses  should  examine:  Teaching  and  Learning,  Developing 
a Community  of  Learners,  the  Instructor,  The  Student,  Implementation  of  the  Course,  and 
Technology  Use.  These  constructs  become  important  elements  in  the  design  and 
evaluation  of  distance  learning  classes. 

The  Model  for  Assessing  Distance  Learning 

When  looking  at  the  first  construct.  Teaching  and  Learning,  research  supports  that 
students  of  all  ages  who  are  provided  easy  access  to  well-crafted  computer-mediated 
distance  learning  classes  generally  have  a positive  experience  taking  this  type  of  course 
(Abbott  and  Paris,  2000;  Baron  and  McKay,  2001;  Gagne  and  Shepherd,  2001;  Mitra  and 
Steffensmeier,  2000).  However,  these  studies  also  point  out  that  an  increase  in  positive 
attitude  is  directly  related  to  instructional  approaches  used,  meaningful  assignments 
required,  supportive  faculty,  and  involvement  in  meaningful  discussion  groups  (Abbott 
and  Paris,  2000).  As  Baron  and  McKay  (2001)  point  out,  having  the  requisite  computer 
proficiency  also  affected  students’  perceptions  of  the  course. 

Research  also  suggests  that  distance  learning  can  provide  the  same  level  of  academic 
excellence  as  courses  taught  in  traditional  modes.  Moore  and  Thompson  (1990,  1997) 
reviewed  much  of  the  research  from  the  1980’s  and  1990’s  on  the  effectiveness  of 
distance  learning,  which  included  not  only  computer-mediated  learning,  but  two-way 
interactive  video  and  a variety  of  other  technologies,  as  well.  They  concluded  that 
distance  education  was  considered  effective  when  measured  by  the  achievement  of 
learning,  by  the  attitudes  of  students  and  teachers,  and  by  return  on  investment. 

Developing  a Community  of  Learners  is  the  second  construct  to  consider  when  assessing 
distance  learning  instruction.  Educators  are  increasingly  emphasizing  the  importance  of 
electronic  communities  in  distance  learning  courses  as  a key  to  effective  implementation 
(Creed,  1996;  Herrmarm,  1998;  Illinois  Online  Network,  2000;  MacKirmon,  2000;  Palloff 
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and  Pratt,  1999;  Poole,  2000).  As  Palloff  and  Pratt  (1999)  say:  “In  distance  education, 
attention  needs  to  be  paid  to  the  developing  sense  of  community  within  the  group  of 
participants  in  order  for  the  learning  process  to  be  successful”  (p.29).  The  role  of  the 
community  is  an  essential  component  for  the  success  of  any  distance  learning  class. 

When  considering  the  third  construct.  The  Instructor,  in  an  online  distance  learning 
course,  the  role  of  the  instructor  needs  to  be  examined.  The  instructor  no  longer  stands  in 
front  of  the  room  imparting  information,  directing  all  interaction,  and  facilitating 
activities.  Instead,  instructors  must  take  advantage  of  the  constmctivist  online  course  and 
create  an  environment  that  is  learner-centered,  collaborative,  and  egalitarian  in  approach. 
The  governing  principle  is  that  learning  is  an  active  process  in  which  learners  construct 
rather  than  acquire  knowledge  and  the  instmctor’s  role  is  to  support  that  construction 
rather  than  impart  knowledge  (Grossman  and  Wagner,  2000).  Palloff  and  Pratt  (1999) 
categorize  the  various  tasks  and  roles  required  of  an  online  instructor  into  four  general 
areas:  pedagogical,  social,  managerial,  and  technical.  In  the  pedagogical  area,  the 
instructor  must  know  how  to  design  and  implement  an  effective  hands-on  course  and 
create  an  environment,  which  promotes  interaction  and  exchange  between  all  involved. 

In  the  social  area,  the  instmctor  must  direct  all  social  intercourse  that  occurs  online  and 
ensure  that  communication  is  not  just  between  instructor- student  but  is  a multifaceted 
pattern,  involving  student-student-instmctor.  In  the  managerial  area,  the  instmctor  must 
know  how  to  manage  a multitude  of  tasks  and  organize  a multitude  of  files.  Oftentimes, 
the  managerial  function  can  become  overwhelming  and  an  effective  instmctor  utilizes 
organizational  skills  and  timesaving  techniques.  Lastly,  in  the  technical  area,  the 
instmctor  must  be  technically  proficient  so  that  the  course  runs  smoothly.  The  instmctor 
must  provide  technical  feedback  to  students  on  basic  requirements  for  exchanging  and 
reading  files,  and  using  software,  browsers,  and  operating  systems.  In  a distance  learning 
course,  the  role  of  the  instmctor  changes  dramatically. 

In  reviewing  the  next  constmct.  The  Student,  the  role  of  the  student  also  changes 
significantly  in  a distance  learning  course.  Many  students  will  find  the  transition  to  this 
new  type  of  online  learning  to  be  difficult,  especially  if  they  are  used  to  a more  traditional 
course  where  they  are  “fed”  knowledge  and  evaluated  using  a test.  Students  in  an  online 
course  must  be  active  and  engaged  in  knowledge  generation.  They  cannot  sit  passively 
and  expect  the  course  to  be  given  to  them  by  the  instmctor.  Palloff  and  Pratt  (1999) 
categorize  the  role  of  the  leaner  in  an  online  course  into  three  areas:  knowledge 
generation,  collaboration,  and  process  management.  In  knowledge  generation  the  student 
must  learn  how  to  constmct  and  generate  his/her  own  knowledge  base.  In  collaboration, 
the  student  must  leam  how  to  collaborate  electronically  with  other  students  via  e-mail 
and  discussion  forums.  In  process  management  the  student  must  take  the  responsibility 
for  managing  his/her  time,  as  well  as  taking  the  initiative  for  researching  topics  and 
concepts.  The  student  must  leam  how  to  manage  the  process  of  learning  in  a 
constmctivist  online  course.  This  is  an  important  constmct  to  consider  in  assessing 
distance  learning  courses  due  to  the  low  completion  rate  of  students  enrolled  in 
computer-mediated  distance  learning  courses. 
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One  of  the  more  important  aspects  of  computer-mediated  distance  learning  is  effective 
implementation  of  the  course  which  in  the  fourth  construct,  Implementation.  This  covers 
the  basic  instructional  design  of  the  course,  the  infrastructure  available  to  support  the 
course,  the  quality  of  the  access  to  the  course,  and  the  level  of  technical  support  available. 
These  factors  affect  each  student’s  ability  to  access  and  successfully  complete  an  online 
course. 

The  last  construct.  Technology  Use,  looks  at  how  often  students  use  various  components 
of  the  course,  and  find  each  component  useful.  It  is  highly  recommended  that  different 
aspects  of  the  course  be  assessed  for  frequency  of  use  and  degree  of  usefulness.  Such 
aspects  that  could  be  assessed  are:  the  Discussion  Forum,  the  Announcement  Feature,  the 
chat  room,  the  white  board,  the  Student  Drop  Box  (where  assignments  are  uploaded),  the 
learning  links,  the  lecture  notes,  the  schedule  of  assignments,  the  syllabus,  and  the  e-mail 
feature.  This  information  will  be  very  useful  in  the  revision  and  implementation  of  the 
course. 

Discussion 

Computer-mediated  distance  learning  involves  a whole  new  array  of  factors  to  consider 
in  the  teaching-learning  process.  However,  although  distance  learning  courses  involve 
various  new  factors  to  consider  in  their  design  and  assessment,  distance  learning  courses 
are  fundamentally  the  same  as  any  other  course  offered  to  students—  the  basic 
denominator  is  effective  instruction.  The  modality  of  delivering  the  course  may  change, 
but  the  basic  tenants  of  good  pedagogical  theory  remain  the  same.  Although  we  are 
transforming  education  into  a 21®‘  century  enterprise  that  utilizes  the  latest  technological 
advances,  the  basic  issues  remain  constant:  what  is  effective  instruction  and  how  can  we 
best  teach  to  students  who  possess  different  learning  styles  and  different  competencies? 
How  can  we  better  prepare  instructors  to  be  effective?  These  are  the  questions  we  need 
to  keep  in  mind  as  we  conduct  further  research  into  the  field  of  distance  learning.  In 
terms  of  effective  course  design,  distance  learning  continues  to  represent  one  point  along 
a continuum  of  design  options,  one  other  mode  of  delivery  to  consider. 

The  constructs  outlined  in  this  paper  are  those  that  should  be  considered  in  all  courses; 
they  are  applied  in  somewhat  different  contexts  within  a distance  learning  course.  As  we 
move  forward  technologically,  our  theories  of  learning  and  instruction  need  to  be  applied 
consistently  and  persistently  to  the  development  of  all  courses,  and  our  attention  needs  to 
be  on  the  quality  of  the  instruction,  not  on  the  modality  of  delivery.  This  “new” 
paradigm  of  learning  needs  to  be  grounded  in  time-tested  and  research-based  theories  of 
learning  to  ensure  that  our  students  receive  the  quality  instruction  they  deserve. 

Overview  of  This  Session 

The  session  at  Ed-Media  will  present  this  new  model  of  assessment  by  examining  what 
the  research  recommends  for  each  construct.  A visual  model  will  be  presented  and  each 
construct  will  be  reviewed  according  to  what  is  recommended  to  assess  this  area.  The 
session  will  demonstrate  an  assessment  form  that  has  been  developed  to  assess  the 
author’s  distance  learning  class.  Literacy  and  Technology  Across  the  Curriculum,  that  is 
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being  offered  at  Fairleigh  Dickinson  University.  The  assessment  form  will  be  examined 
and  copies  will  be  distributed  to  participants.  Preliminary  data  collected  from  the  course 
will  be  presented.  Specific  examples  of  how  this  model  helped  the  author  to  design  and 
implement  this  course  will  be  demonstrated.  Questions  and  discussion  on  assessment  in 
distance  learning  instruction  will  conclude  this  session. 
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Abstract:  Numerous  studies  have  been  conducted  to  evaluate  the  effect  of  animated 

visualizations  on  students  learning  data  structures,  but  few  have  attempted  to  evaluate  the 
retention  of  information  gained  from  visualizations.  Our  work  focuses  on  comparing  different 
media  used  for  teaching  data  structures,  particularly  as  they  affect  the  learning  process  over 
time.  Results  from  our  empirical  studies  suggest  that  a combination  of  text  and  visualization 
helps  students  retain  knowledge  better  than  either  approach  alone. 


Introduction 

In  computer  science  education,  algorithms  and  data  structures  are  complex  topics  and  difficult  for 
students  to  learn.  Animated  visualizations  that  graphically  depict  the  processing  steps  of  data  structure 
operations  are  frequently  developed  as  instructional  media  to  help  students  learn.  Many  studies  have  been 
conducted  to  evaluate  the  effectiveness  of  these  visualizations  for  learning  (see  Hundhausen  2002  or  Wilson 
1996  for  a review).  However,  results  have  generally  been  negative,  typically  showing  no  significant  difference 
between  visualizations  and  traditional  teaching  methods.  Minor  design  features  seem  to  have  major  impact  on 
effectiveness.  Furthermore,  few  studies  have  attempted  to  evaluate  students’  long-term  retention  of  information 
gained  from  data  structure  visualizations,  the  ultimate  goal  of  education.  A related  study  (Palmiter  1991)  found 
that  animated  demonstrations  to  teach  software  user  interfaces  improved  student  performance  over  a textual 
manual  in  the  short  term,  but  did  not  maintain  performance  in  the  long  term.  It  is  possible  that  students  merely 
mimicked  the  steps  of  the  animations  at  first  without  actually  understanding  the  steps  as  required  for  long  term 
retention.  We  present  a series  of  studies  aimed  to  examine  these  issues. 


Results 

The  goal  of  the  first  study  was  to  compare  different  interaction  strategies  of  data  structure 
visualizations.  60  undergraduate  CS  students  studied  the  AVL  Tree  data  structure  for  15  minutes  using  one  of  3 
methods:  (1)  a textual  material  derived  from  Shaffer’s  textbook  (2001),  (2)  the  Arsen  visualization  (Gogeshvili 
2001)  which  simply  animates  the  tree  operations  initiated  by  users,  or  (3)  the  BinaryTreesome  visualization 
(Gustafson  1998)  which  quizzes  students  to  perform  the  operation  steps  themselves  while  providing  hints. 
Students  then  answered  a 15-minute  closed-book  test  containing  3 procedural  questions  about  the  ‘insert’, 
‘delete’,  and  ‘find’  tree  operations,  and  2 conceptual  questions  about  higher-level  concepts. 

The  data  (table  1)  indicate  that  all  3 methods  resulted  in  similar  student  performance  on  the  test 
questions,  except  for  the  ‘delete’  operation  question  which  had  significantly  better  scores  from  the  students  who 
used  the  text  material  (p<0.05).  This  may  be  due  to  the  open-ended  nature  of  the  visualization  tools,  which 
required  users  to  discover  and  explore  their  full  functionality  to  learn  all  the  AVL  tree  operations.  Many 
students  simply  may  have  forgotten  to  explore  the  ‘delete’  operation  in  the  visualizations  or  ran  out  of  time, 
whereas  the  text  ensured  coverage  of  the  material  and  helped  students  to  pace  themselves.  This  indicates  that 
open-ended  visualizations  should  be  accompanied  by 
guidance  material,  and  led  to  our  examination  of  a combined 
text  and  visualization  approach  in  the  next  study.  The  Arsen 
visualization  received  significantly  better  subjective 
satisfaction  ratings  from  students  than  the  other  two  methods 
(p<0.05)  and  a weak  effect  over  BinaryTreesome  on  the 
‘delete’  question  (p<0.1),  indicating  some  advantage  of  the 
simple  animated  visualization  method. 


(1)  Text 

(2)  Arsen 

(3)BT 

Test 

73% 

66% 

57% 

‘delete’ 

40% 

15% 

0% 

Subj.  Sat. 

66% 

87% 

77% 

Table  1:  Mean  scores  and  ratings,  U*  study. 
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The  goal  of  the  second  study  was  to  examine  the  effect  of  data  structure  visualization  on  longer-term 
knowledge  retention.  49  undergraduate  CS  students  studied  the  Depth -First  and  Breadth-First  Search  graph 
traversals  for  20  minutes  using  one  of  3 methods:  (1)  a textual  material  from  Shaffer’s  textbook,  (2)  Saraiya’s 
visualization  (Saraiya  2001)  which  lets  users  step  through  the  operations,  or  (3)  the  combination  of  (1)  and  (2). 
Students  then  answered  a 10-minute  closed-book  test  containing  3 procedural  and  2 conceptual  questions.  After 
a period  of  15  days,  students  again  answered  a second,  similar  test  without  reviewing  the  study  material. 

Overall,  students  scored  similarly  on  the  first  test  across  all  methods  (table  2).  Significant  effects 
between  methods  were  not  found  on  either  test.  However,  on  average,  students’  scores  decreased  on  the  second 
test  by  21%  with  the  text  method,  by  1 1%  with  the  visualization  method,  and  by  only  7%  with  the  combined 
method.  The  15-day  waiting  period  had  a significant  effect  on  test  scores  (p<0.05).  It  had  a significant 
detrimental  effect  on  the  text  users  (p<0.01)  and  a weak  effect  on  the  visualization  users  (p<0.1).  No  effect  was 
detected  for  the  combined  method  users.  This  data  indicates  a non-significant  trend  towards  the  combined 
method  providing  better  retention.  In  future  work,  student  tests  should  contain  many  more  questions  to  enable 
higher  resolution  scores  and  more  conclusive  statistics.  Strangely,  on  one  conceptual  analysis  question, 
students’  scores  actually  increased  remarkably  by  25%  on 
average  for  the  visualization  and  combined  methods,  while 
the  text  method  scores  remained  constant.  In  subjective 
satisfaction  ratings,  the  text  method  was  the  clear  loser 
(p<0.01).  On  average,  students  most  preferred  the  combined 
method. 


(1)  Text 

(2)  Vis. 

(3)  Comb. 

Test  1 

14% 

73% 

75% 

Test  2 

53% 

62% 

68% 

Subj.  Sat. 

62% 

80% 

87% 

Table  2:  Mean  scores  and  ratings,  2 study. 


Conclusions  and  Future  Work 

For  teaching  data  structures,  we  recommend  a learning  method  that  combines  visualization  and  text. 
Students  are  clearly  dissatisfied  with  usual  text  materials,  but  visualization  alone  can  lead  to  gaps  in  knowledge. 
Since  subjects  perceive  that  visualizations  are  very  helpful,  visualizations  will  stimulate  and  motivate  them  to 
learn  new  topics.  Visualizations  can  help  to  drive  concepts  into  long-term  memory.  Accompanying  text  will 
ensure  adequate  coverage  of  the  concepts.  This  approach  simultaneously  enables  focus  as  well  as  open-ended 
exploration  by  students.  Even  in  time  pressure  learning  periods,  students  were  able  to  juggle  both  learning 
methods  to  produce  good  performance,  including  taking  time  to  learn  the  visualization  tool  itself 

Current  research  in  algorithm  visualizations  suggests  that  actively  engaging  students  while  they  are 
watching  visualizations  can  increase  their  effectiveness  (Hundhausen,  et  al.  2002).  This  would  support  our 
hypothesis  that  the  visualizations  in  the  first  experiment  failed  because  students  were  not  guided  to  study  the 
‘delete’  operation.  One  approach  to  active  engagement  is  to  supply  guide  questions  to  be  answered  while  the 
visualization  is  being  studied.  In  addition  to  active  engagement,  we  believe  that  there  is  an  identifiable  set  of 
key  features  that  are  incorporated  into  nearly  all  successful  visualizations.  We  currently  hypothesize  that  this 
feature  set  includes:  User  control  of  the  pace  of  the  visualization  animation;  the  ability  to  test  hypotheses  by 
allowing  users  to  enter  input;  display  of  both  logical  and  physical  views  of  a data  structure;  the  ability  to  backup 
the  animation  to  replay  critical  steps;  and  a pseudocode  display  of  the  algorithm,  keyed  to  the  steps  being 
visualized.  We  are  presently  conducting  studies  to  measure  the  magnitude  of  the  contribution  of  each  of  these 
features  using  a controlled  visualization  toolset. 
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Abstract  While  the  idea  of  reusing  objects  in  digital  learning  environments  is  not  new,  continual  strides  are  being 
made  toward  improving  the  prospects  of  reusability.  A major  trend  in  company  training  settings  is  to  think  of 
reusability  in  terms  of  a LMS  (learning  management  systems),  but  where  instructor  use  and  pedagogies  are  little 
considered.  We  describe  an  approach  to  re-use  based  on  a pedagogical  model  that  puts  learner  interaction  and 
contribution  in  the  driving  role  and  which  sees  LMSs  and  LCMSs  as  tools  for  the  instructor  or  learner. 


LMSs,  LCMSs:  Where  is  the  Instructor? 

There  is  currently  much  attention  in  the  business  world  to  the  re-use  of  learning  objects. 
Frequently  this  occurs  in  the  context  of  the  introduction  of  "e-leaming"  where  the  latter  runs 
separately  from  "classroom  courses".  E-leaming  is  typically  seen  as  being  instructor-free  or  - 
neutral,  in  order  to  capitalize  on  an  "any  time,  anywhere"  motivation.  Complex  systems, 
called  learning  management  systems  (LMSs)  and  learning  content  management  systems 
(LCMSs),  are  proliferating,  generally  based  on  the  underlying  assumption  that  the  system 
itself  will  select  and  deliver  the  learning  experience,  based  on  some  level  of  sophisicated  user 
modelling.  LMSs  are  defined  as  systems  "to  manage  learners,  keeping  track  of  their  progress 
and  performance  across  all  types  of  learning  activities"  while  LCMSs  manage  content  or 
learning  objects  to  "serve  up  to  the  right  learner  at  the  right  time"  (Chapman  & Hall,  2001,  p. 
11).  LCMSs  typically  include  content-development  tools,  being  in  effect  a new  iteration  of 
the  long  series  of  attempts  to  bring  authoring  tools  into  mainstream  use  for  computer- 
supported  learning.  "Content  assembly"  and  "publish  learning"  into  different  "output  formats" 
are  key  tasks  of  LCMSs  (Chapman  8l  Hall,  p.  16). 

Meanwhile  at  the  same  time  as  commercial  LCMSs  and  LMSs  are  being  taken  up  for  "e- 
leaming"  in  company  training  settings,  the  use  of  Web-based  course-management  systems, 
also  called  online  educational  delivery  systems  (Landon,  http://www.c2t2.ca/landonline/). 
continues  to  grow  in  importance  particularly  in  support  of  instructor-led  courses  with  or 
without  a classroom  component.  Course-management  systems  (CMSs,  not  to  be  confused 
with  content-management  systems,  also  sometimes  called  CMSs)  integrate  content  delivery, 
communication,  learner  activities,  collaborative  work  support,  feedback,  testing,  portfolio 
development,  groupware  tools,  and  administrative  tools  for  the  instructor.  Selection  of 
content  objects  is  only  part  of  the  use  of  an  online  educational  delivery  system,  and  in  many 
cases  a minor  part. 

t 

The  relationship  of  LMSs  and  LMCSs  to  instructor-led  classroom  or  blended  learning  is  not 
yet  much  studied.  Partially  this  is  because  the  use  of  LMSs  and  LCMSs  is  still  new,  and  most 
companies  making  these  major  purchases  are  still  in  the  phasing- in  stages.  More 
fundamentally,  it  is  because  those  responsible  for  LMS  and  LMCS  use  in  the  organization 
tend  to  operate  separate  from  the  "business  as  usual"  instructor- led  courses  which  are 
typically  the  candidates  for  extension  via  an  online  delivery  system.  The  fundamental  issue 
is:  Where  is  the  heart  of  the  learning  process?  In  the  delivery  of  pre-made  learning  objects  or 
in  the  support  of  learning  activities  involving  human  interaction  and  problem  solving  in 
workplace  contexts  making  use  of  in-house  experience?  We  believe  that  in  a company 
learning  context  the  emphasis  should  be  on  the  latter  (Collis,  2001;  Collis  & Winnips,  2001). 
When  we  start  with  a pedagogy  based  on  the  solving  of  real  workplace  problems  ("authentic 
learning",  Harrington,  Reeves,  & Oliver,  2001)  where  the  interaction  with  others  is  critical  we 
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come  to  a different  view  on  reusability,  and  the  technologies  that  support  it.  This  relationship 
is  part  of  a research  project  involving  the  Faculty  of  Educational  Science  and  Technology  at 
the  University  of  Twente  and  the  Shell  Learning  Centre.  This  paper  will  discuss  some  aspects 
of  the  on-going  work. 

Pedagogical  Base;  Focus  on  Contribution  Intended  for  Reuse 

What  is  a pedagogical  aim  that  is  highly  related  to  re-use?  We  offer  the  proposition  that  it  is 
not  an  aim  primarily  related  to  finding  "instructional"  content  from  elsewhere,  but  rather  one 
with  a strong  orientation  toward  learning  from  experiences,  from  one's  own  and  from  those  of 
others.  This  involves  a pedagogical  shift,  away  from  an  emphasis  on  learning  as  acquisition 
of  predetermined  content,  toward  a balance  that  includes  or  even  emphasizes  learning  as 
participating  and  contributing  to  the  learning  experience  in  a way  which  can  be  captured  and 
reused  by  others  (Collis  & Moonen,  2001;  Collis  & Strijker,  2002;  Sfard,  1998).  The  basis  of 
this  pedagogy  is  educational  (see  also  for  example,  Kearsley  & Shneiderman,  1998)  but  it 
also  is  based  on  strategic  and  practical  considerations.  By  an  approach  in  which  participants 
themselves  contribute  resources  in  a way  that  is  intended  for  use  by  others  during  the  course 
itself  and  in  subsequent  cycles  of  the  course,  a large  collection  of  locally  relevant  resources  is 
accumulated  on  an  on-going  basis  based  predominately  on  the  work  of  the  participants  and 
thus  less  on  the  preliminary  work  of  instructor  or  learning-object  designer.  This  involves 
shifting  of  the  costs  in  terms  of  preparation  time,  from  the  content  producer  or  instructor  to 
the  participants  (Collis  & Moonen,  2001).  The  resources  accumulated  fit  with  the  style  and 
level  of  the  local  participants  (avoiding  the  "not  invented  here"  problem),  and  in  the  company 
setting  particularly,  the  tacit  knowledge  of  the  participants  is  made  available  to  others  in  the 
institution  (Collis  & Winnips,  2001).  An  important  point  in  this  pedagogy  is  that  many  of  the 
reusable  resources  are  not  to  be  seen  as  professionally  made  self-contained  study  materials; 
their  creation,  use,  and  reuse  comes  from  the  way  the  instructor  or  learner  sees  them  as 
helpful  for  learning  activities  based  on  actual  workplace  problems. 

The  reuse  of  submitted  contributions  however  is  not  automatic  simply  because  the 
contributions  are  available  to  all  in  the  course  environment.  Their  reuse  comes  primarily 
from  the  pedagogy  used  by  the  instructor.  As  an  example,  participants  can  be  asked  to  study 
one  or  more  of  the  contributions  of  others  in  the  current  course  or  previous  courses  if 
available  for  reuse,  and  as  the  next  activity  compare  and  contrast  their  own  workplace 
situations  with  those  of  their  peers.  Many  such  guided  activities  have  been  in  use  at  the 
University  of  Twente  for  several  years  (De  Boer  & Collis,  2001);  in  the  company  context  the 
pedagogical  experience  is  only  at  the  starting  point. 

For  the  products  of  such  activities  to  be  reusable  as  digital  resources  in  other  learning  settings, 
the  underlying  database  and  system  technology  as  well  as  the  user  interface  must  meet  several 
requirements. 

Technology  Requirements 

The  technology  required  for  support  of  this  pedagogy  requires  an  underlying  object-oriented 
database.  We  suggest  that  gradually,  an  experience  management  architecture  (Layton,  1999) 
will  need  to  integrate  the  now-familiar  course  management  systems  with  systems  such  as 
LMSs  and  LCMS  as  well  as  others.  Figure  1 shows  the  architecture  that  we  are  currently 
researching. 
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Figure  1,  Integrated  architectui’e 


For  contribution-based  reusability  to  grow  in  an  institution,  specific  technical  tools  and  user 
interface  functionalities  are  critical.  First,  there  must  be  a simple  way  for  participants 
(instructors  and  learners)  to  enter  new  resources  into  a common  database.  Figure  2 shows  one 
of  the  templates  we  make  directly  available  in  the  course  environments  running  under  our 
local  TeleTOP  course-management  system. 
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Figure  2.  Template  for  resource  creation 

At  the  moment  that  an  object  is  thus  submitted  into  the  database,  metadata  based  on  SCORM 
standards  can  be  automatically  assigned  for  some  of  the  SCORM  fields.  A new  document 
containing  a metadata  record  is  created  automatically  for  each  submitted  item  and  stored  in  a 
metadata  database. 


In  the  TeleTOP  system,  the  submitted  items  can  be  directly  made  available  to  all  course 
participants  for  re-use  in  subsequent  learning  activities.  The  instructor  decides  this,  and  can 
decide  that  all  participants  have  direct  read  and  write  rights  at  one  extreme,  or  at  another,  that 
only  an  object  chosen  by  the  instructor  is  re-used  in  a read-only  format  at  the  other  extreme. 


For  the  submitted  objects  to  be  reused  outside  of  the  immediate  course  context  as  well  as 
within  it,  the  instructor  needs  to  make  a selection  of  which  materials  are  good  candidates  for 
reuse.  For  these  selected  objects,  the  instructor  can  then  add  a broader  range  of  metadata  to 
help  in  the  later  retrieval  process.  Figure  3 shows  the  combination  of  automatically  affixed 
metadata  and  the  possibilities  for  local  entries. 
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Note  that  it  is  not  every  item  submitted  during  the  course  which  requires  this  localizing 
attention;  only  the  relatively  select  and  small  sample  that  the  instructor  sees  as  good 
candidates  for  reuse.  There  is  automatic  XML  representation  of  metadata  documents 
generated  within  the  TeleTOP  system. 

Subsequent  searches  of  the  objects  in  the  database  to  look  for  candidates  for  reuse  are  made 
via  the  metadata  document  forms,  which  are  in  turn  related  to  the  actual  submitted  objects  and 
can  faciliate  their  being  copied  into  the  next  desired  environment.  A simple  interface  is 
available  to  support  the  search  process  using  any  keywords  on  the  metadata  documents,  and 
to  facilitate  the  copying  process,  to  move  the  items  selected  for  reuse  into  a new  course 
environment.  In  the  instructor-led  course,  it  is  the  instructor,  not  a LCMS  or  LMS,  that 
makes  these  decisions.  The  LCMS  can  help  the  instructor  be  aware  of  resources  that  are 
available  and  the  LMS  can  help  the  instructor  be  better  attuned  to  the  learning  histories  of  the 
participants.  But,  in  this  sort  of  contribution  approach,  it  is  the  instructor  (or  course  manager 
or  facilitator  or  even  the  learner)  who  decides  what  learning  objects  are  most  helpful  for  the 
problem-based  activities  that  form  the  heart  of  the  course. 


Through  the  combination  of  these  sorts  of  technologies  and  pedagogies,  and  in  the 
implementation  context  of  a course  as  a activity  and  contribution-oriented  setting,  a number 
of  the  barriers  limiting  the  potential  reuse  of  digital  resources  can  be  addressed,  particularly 
those  that  relate  to  fit  with  the  local  context.  In  a (multinational)  company  training  setting, 
the  reuse  aspect  also  brings  a gradual  increase  in  the  sharing  of  in-house  knowledge  and 
experience  (Collis  & Winnips,  2001). 
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Abstract:  This  paper  analyzes  the  uses  of  the  Inquiry  Page  (www.inquirv.uiuc.edu)  in  the 
National  Science  Foundation  GK-12  Fellowship  Program*,  where  scientist- fellows  and 
K-12  teachers  partner  to  integrate  the  use  of  computer-based  modeling,  scientific 
visualization,  and  informatics  in  science  and  mathematics  education.  The  Inquiry  Page  is 
a web-based,  knowledge -building  tool  that  facilitates  and  fosters  real-world  application 
of  inquiry -based  learning.  By  creating  “Inquiry  Units”  teachers  and  learners 
collaboratively  engage  in  the  inquiry  path,  building  new  knowledge  as  well  as  a resource 
for  other  teachers  and  learners.  By  offering  structures  for  inquiry  teaching  and  learning, 
a-synchronous  community-building  is  occurring  with  teachers/leamers  being  able  to  draw 
upon  and  share  their  own  applications  of  inquiry -based  teaching.  The  Inquiry  Page,  then, 
becomes  the  nexus  between  theory  and  application.  It  emerges  as  a “portal”  for  the 
scientist-fellow,  teacher,  and  students  in  the  classroom;  facilitates  communication 
between  the  teacher/scientist  team  and  other  educators  both  at  the  school  and  in  the  larger 
education  community;  supports  inquiry -based  teaching  in  the  real  world  of  the  classroom; 
and  nurtures  the  development  and  professional  growth  of  the  teacher  and  scientist/fellow 
by  engaging  in  reflective  educational  theory  and  practice. 

*The  EdGrid  GK-12  Fellowship  Program  supports  annual  teaching  fellowships  for  graduate  student 
scientists  from  the  University  of  Illinois,  Urbana-Champaign  and  is  administered  by  the  National 
Center  for  Supercomputing  Applications, 


Evaluation  methods:  The  GK-12  Evaluation  team  is  adopting  a situated  evaluation 
approach  to  gathering  data  at  each  Wgli  school  site,  using  online  formative  and  summative 
evaluations  of  students,  teachers,  and  fellows  in  addition  to  observation,  videotaping,  and 
interviews.  Teacher  and  fellow  reflections  are  thus  captured  quantitatively  and  qualitatively. 
Artifact/products  of  students,  teachers,  and  fellows  are  reviewed.  Key  to  the  evaluation 
process  is  an  understanding  of  and  sensitivity  to  each  site’s  context  and  dynamics.  Of 
particular  interest  for  the  Inquiry  Page  is  identifying  its  uses  and  utility,  and  this  is 
determined  by  units  created  and  interviews  with  teachers/fellows. 

Sites:  Site  1 is  located  in  a suburb  of  a large  city.  It  was  formed  to  offer  alternative  (i.e., 
vocational)  education  for  students  grades  10-12  to  supplement  their  regular  high  school 
curriculum.  Site  2 has  approximately  2,325  students  and  offers  a traditional  college- 
prepatory  high  school  curriculum.  It  is  located  in  a comparatively  wealthy  cosmopolitan 
suburb  of  a large  city.  Site  3 has  approximately  1600  students  and  is  located  in  an  industrial 
district.  The  school  offers  a diverse  curriculum  that  includes  two  “academies”  specializing  in 
medical  science  or  math/engineering/information  technology  for  accelerated  students.  Site  4 
is  located  in  a twin-city  that  is  home  to  a large  university.  It  offers  a traditional  college- 
prepatory  curriculum. 

The  Inquiry  Page:  The  Inquiry  Page  (www.inquiry.uiuc.edu)  is  a five- year-old  project 
developed  initially  as  a supportive  tool  for  teacher  development  by  Dr.  Bertram  Bruce  at  the 


3VS 


BEST  C0P¥  AVAILABUE 


University  of  Illinois,  Urbana  Champaign.  It  has  since  evolved  into  a web-based  tool  and 
collaboratory  to  facilitate  inquiry-based  learning  and  teaching  within  much  broader  contexts. 
Evaluators,  teachers,  and  scientist- fellows  participating  in  the  GK-12  Program  have  used  the 
Inquiry  Page,  identifying  its  utility  for  their  needs. 

Facilitating  Inquiry  in  Teaching/Learning:  Each  site  differed  in  its  adoption  of  the  Inquiry 
Page.  Two  of  the  fellows  found  it  unusable  in  their  specific  classrooms:  one  fellow  noted 
teacher  disinterest  in  learning  methodologies  or  new  technology,  while  another  fellow  noted 
a sense  of  it  not  being  relevant  to  his/her  teachers’  subject/leaming  context.  Each  of  the 
other  fellows,  however,  employed  the  Inquiry  Page  as  (1)  a lesson  planning  tool  for  teachers, 
including  tutorials,  resources,  and  personal  reflections  in  their  role  as  fellow/teacher  in  the 
classroom;  (2)  an  archive  for  their  own  work  and  reflections  on  their  particular  GK-12  site, 
including  site-specific  resources;  and  (3)  as  a way  to  more  deeply  understand  inquiry-based 
learning  and  engage  in  a “co- learner”  relationship  with  their  teacher.  The  model  identified  by 
the  fellows  in  their  mid-year  evaluation  as  most  successful  was  one  in  which  the  teacher  and 
fellow  worked  collaboratively  in  teaching/leaming.  The  Inquiry  Page  facilitated  this  type  of 
relationship  in  and  out  of  the  classroom;  with  units  serving  as  new  points  of  communication 
with  educators  both  within  and  outside  of  the  individual  district/sites.  The  GK-12  Evaluation 
team  initially  thought  the  Inquiry  Page  could  be  used  to  document  their  observations; 
however,  they  found  that  it  was  more  useful  as  an  artifact  of  the  teacher/fellow  teams’ 
evolving  teaching  models.  Unexpectedly,  the  Inquiry  Page  has  been  used  as  a presentation 
tool  with  the  units  being  demonstrated  at  NSTE,  NABT,  TeachIT,  and  various  district 
meetings.  Perhaps  most  interesting  is  the  Inquiry  Page’s  role  in  introducing  inquiry-based 
learning  to  the  fellow/scientists  and  giving  a “name”  to  what  teachers  were  already  doing  or 
wanted  to  do  in  the  classroom.  Although  all  had  heard  the  term,  and  had  an  intuitive  sense  of 
what  it  meant,  it  was  during  the  process  of  creating  units  that  points  of  recognition  and 
understanding  of  real- world  application  occurred.  Essential  features  of  classroom  inquiry  in 
the  sciences  as  defined  by  the  National  Research  Council  are  identifiable  at  each  of  the  sites; 
particularly  at  one  site,  where  the  scientist  fellow/teacher/students  are  generating  new 
research  questions  that  are  being  incorporated  into  the  scientist-fellow’s  dissertation  work. 
The  Inquiry  Page  is  bridging  the  gap  between  education  pedagogy  and  real- world 
applications  in  the  classroom,  creating  a framework  for  scientists  and  teachers  to 
collaborate — to  the  benefit  of  both. 
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Who’s  Teaching  Whom?  Scientist/Teacher  Teams  and 
Administrative  Strategies 
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Abstract:  As  one  of  the  research  university  teams  participating  in  the  National  Science 
Foundation  (NSF)  GK-12  Fellowship  Program*,  site  educators  and  scientists  are 
realizing  collaborative,  technology -rich  learning  environments  that  have  implications  for 
teacher  development,  administrative  planning  for  technology  and  curriculum,  and  the 
professional  growth  of  young  scientists.  Scientist  fellows  in  biology,  chemistry, 
mathematics,  computer  and  information  science  are  strategically  partnering  with  high 
school  teachers  and  administrators,  integrating  the  use  of  problem-based  learning  (PBL), 
scientific  visualization,  and  informatics  in  science  and  mathematics  education.  Initial  data 
thus  collected  by  the  GK-12  Evaluation  Team  of  six  teams  at  four  sites  across  Illinois 
offer  context  and  contrast  in  professional  development,  support,  planning,  and 
implementation.  This  paper  addresses  the  evolving  collaborations  between  administration 
strategists,  the  science  and  mathematic  teachers,  and  scientist-fellows  at  these  sites  and 
the  implications  for  success. 

*The  EdGRJD  GKI2  Fellowship  Program  supports  annual  teaching  fellowships  for  graduate 
student  scientists  from  the  University  of  Illinois,  Urbana^  Champaign  and  is  administered  by  the 
National  Center  for  Supercomputing  Applications. 


Evaluation  methods:  The  GK-12  Evaluation  team  is  adopting  a situated  evaluation 
approach  to  gathering  data  at  each  high  school  site,  using  online  formative  and  summative 
evaluations  of  students,  teachers,  and  fellows  in  addition  to  observation,  videotaping,  and 
interviews.  Teacher  and  Fellow  reflections  are  thus  captured  quantitatively  and  qualitatively. 
Artifact/products  of  students,  teachers,  and  fellows  are  reviewed.  Key  to  the  evaluation 
process  is  an  understanding  of  and  sensitivity  to  each  site’s  context  and  dynamics. 

Sites:  Site  1 is  located  in  a suburb  of  a large  city.  It  was  formed  to  offer  alternative  (i.e. 
vocational)  education  for  students  grades  10-12  to  supplement  their  regular  high  school 
curriculum.  Site  2 has  approximately  2,325  students  and  offers  a traditional  college - 
prepatory  high  school  curriculum.  It  is  located  in  a comparatively  wealthy  cosmopolitan 
suburb  of  a large  city.  Site  3 has  approximately  1600  students  and  is  located  in  an  industrial 
district.  The  school  offers  a diverse  curriculum  that  includes  two  “academies”  specializing  in 
medical  science  or  math/engineering/information  technology  for  accelerated  students.  Site  4 
is  located  in  a twin-city  that  is  home  to  a large  university.  It  offers  a traditional  college- 
prepatory  curriculum. 

Roles  and  relationships  thus  identified  in  the  GK-12  Project  partnerships  that  are  addressed 
here  are  that  of  the  (1)  educators  and  fellows  as  teachers/learners  and  (2)  administrative 
decision-makers  and  their  role  in  planning  and  support. 

Educator  and  Fellow  Relationship;  Each  site  differed  in  the  roles  the  teachers  and  the 
fellows  adopted.  The  model  identified  by  the  fellows  in  their  mid-year  evaluation  as 
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successful,  is  one  in  which  the  teacher  and  fellow  are  peers,  collaboratively  learning  and 
teaching.  This  model  allows  for  the  educator  and  fellow  to  both  be  learners  and  teachers;  and, 
in  one  problem-based  learning  (PBL)  environment,  for  the  students  to  be  “teachers.”  Initial 
data  indicates  that  successful  partnerships  are  less  rigid,  allowing  for  role  boundaries  to 
change  and  accommodate  professional  growth.  Math  and  science  visualization  tools,  course- 
specific  technology,  and  computers  were  real- world  points  for  collaborative  learning  by  both 
the  teachers  and  fellows. 

Administration’s  Role  in  Planning  and  Support:  The  sites  differed  drastically  in  the  role 
that  administration  has  played  in  both  implementing  technology  in  the  science  cumculum, 
and  the  support  it  offers  its  educators.  For  example,  at  one  site,  the  principal  was  not  aware  of 
who  the  scientist- fellow  was  or  what  s/he  was  doing  there,  meeting  only  once  during  an 
eight- month  period.  This  colored  the  relationship  the  teacher  had  with  the  fellow,  and 
appeared  to  be  a block  to  introducing  new  ways  to  integrate  computer-based  learning  (CBL) 
and  PBL.  Another  site  lacked  communication  avenues  between  the  teacher,  the 
administration,  and  the  fellow,  with  the  administration  adopting  a laissez  faire  approach.  This 
put  the  responsibility  for  implementing  CBL  and  PBL  in  the  classroom  on  the  shoulders  of 
the  scientist- fellow  with  little  guidance  from  tlie  administration. 

These  sites  contrast  with  one  site  where  the  administrative  team  has  taken  a pro-active  role 
both  in  district-wide  strategic  planning  and  long-term  support.  The  district  is  invested  in 
learning  how  the  GK-12  program  fits  into  the  professional  development  models  that  already 
exist;  is  growing  a teacher-to-teacher  mentoring  program,  building  a culture  of  continued 
professional  growth;  sees  data  gathering  as  critical  to  informing  the  decision-making  process; 
and  actively  seeks  funding  to  accommodate  the  needs  of  a technology-rich  science 
curriculum.  The  administration  genuinely  solicits  teachers’  input,  meeting  formally  and 
informally  with  teachers  and  fellows;  and  sends  teachers  to  national  conferences  (i.e.,  NABT, 
NSTE,  & NCTM).  In  practical  terms,  the  administration  demonstrates  their  support  by 
arranging  schedules  to  allow  teachers  “extra”  time  in  order  to  plan  and  become  comfortable 
with  new  digital  tools  in  order  to  integrate  PBL  methods. 
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ABSTRACT 

Adaptive  Hypermedia  Systems  are  capable  of  delivering  personalized  learning  content  to  learners  across  the  WWW.  Learning  Environments 
provide  interfaces  and  support  services  to  aid  tutors  in  course  construction  and  aid  learners  in  navigating  those  courses.  However,  most  Learning 
Environments  deliver  content  sourced  from  local  repositories.  This  content  tends  not  to  offer  adaptive  features  that  Adaptive 

Hypermedia  ^sterns  are  capable  of  delivering.  Adaptive  Hypermedia  Systems,  which  are  generally  Web-based,  could  be  viewed  as  personalized 
content  services,  with  the  capacity  to  deliver  content  to  many  Learning  Environments.  However,  in  order  to  co-ordinate  the  correct 
cooperation  of  the  Adaptive  Hypermedia  Service  and  the  Learning  Environment  knowledge  and  information  models  need  to  be  exchanged,  e.g. 
learner  profile,  assessment  information,  pedagogical  constraints.  There  does  not  exist,  however,  a standardized  mechanism  for  integrating 
Adaptive  Hypermedia  Services  with  Learning  Environments. 

In  this  paper  the  requirements  for  the  generic  integration  of  Adaptive  Hypermedia  Services  with  Open  Learning  Environments  is  explored.  The 
main  aim  of  such  integration  should  be  to  work  within  a framework  that  allows  for  minimal  impact  on  current  Adaptive  Hypermedia  Services  and 
Learning  Environment  implementations,  while  allowing  for  maximum  standards-based  interworking.  This  paper  proposes  an  architecture  and 
interface  to  support  the  collaboration  of  Adaptive  Hypermedia  Service  and  Learning  Environments. 

Categories  and  Subject  Descriptors 

D.2.2  [Tools  and  Techniques]:  Modules  and  interfaces 

General  Terms 

Human  Factors,  Standardization. 

Keywords 

Adaptive  Hypermedia  Services,  Learning  Environments,  Integration,  e-Learning,  Content  Interworking,  Data  Model 

1.  INTRODUCTION 

In  the  current  educational  electronic  landscape  the  most  prevalent  architecture  for  delivering  content  to  a learner  is  via  a Learning  Environment. 
These  environments  (sometimes  referred  to  Learner  Management  Systems  or  Courseware  Support  Systems  [13])  holds  all  the  content  centrally 
and  delivers  it  to  the  learner  over  the  Internet,  typically  via  web  browser  technology 


Developers  and  publishers  of  content  must  design  their  content  for  a particular  Learning  Environment  (LE),  restricting  the  reuse  and 
marketability  of  that  content.  Typically  this  restriction  is  based  on  the  local  repository  used  by  the  LE  or  the  content  management  services 
offered  by  the  LE,  e.g.  TopClass,  WebCT.  Advances  are  being  made  in  the  packaging  of  content  to  facilitate  reuse  and  integration  of  content 
into  different  Learning  Environments  (see  Section  4.3  IMS  Content  Packaging),  but  these  advances  are  concerned  currently  with  static 
content.  If  developers  of  content  wish  to  produce  material  with  complex  play  rules  or  high  levels  of  learner  interaction  they  are  restricted  to 
the  facilities  provided  by  individual  Learning  Environments. 

An  alternative  is  to  produce  stand  alone  Adaptive  Hypermedia  Services  (AHS)  which  are  capable  of  providing  these  rich  play  rules  and  learner 
interaction.  These  services  would  reside  as  separate  servers  on  the  Internet.  The  benefit  in  integrating  LEs  and  AHSs  lies  in  combining  the 
strengths  of  both  systems.  LEs  provide  administration  and  support  facilities,  while  AHSs  provide  better  quality  education,  personalization  and 
pedagogical  control  to  learners. 

This  paper  explores  how  current  open  specifications  may  be  used  to  achieve  seamless  integration  between  such  AHSs  and  LEs.  Section  2 
discusses  the  Learning  Environments  requirements  which  an  AHS  should  meet  to  integrate  with  the  LE.  In  section  3 Adaptive  Hypermedia 
Services  and  their  information  requirements  are  discussed.  Section  4 deals  with  current  standards  and  specifications  which  may  be  utilized  and 
enhanced  to  enable  integration  between  LEs  and  AHSs.  Section  5 continues  to  propose  an  architecture  for  this  integration.  In  section  6 the 
implementation  of  an  example  system  is  discussed.  Section  7 provides  a summary  of  the  aspects  discussed  in  the  paper. 

2.  LEARNING  ENVIRONMENTS 

Most  current  Learning  Environments  [3]  integrate  and  store  all  content  to  be  delivered  to  learners  into  a proprietary  data  storage  format. 
Publishers  of  content  have  to  author  their  content  for  a particular  LEs  proprietary  format  or  import  it  into  that  format.  For  many  of  the  LEs 
the  content  is  static,  interspersed  with  assessments.  Some  LEs  use  the  learner’s  performance  in  the  assessment  information  as  a control 
mechanism  to  determine  the  flow  of  the  content.  Few  LEs,  however,  modify  the  flow  of  content  through  monitoring  the  learner’s  progress 
through  non-assessment  material,  i.e.  they  do  not  generally  take  time  factors  or  multiple  visits  to  a piece  of  content  into  account.  Lack  of 
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standardization  on  how  these  content  flow  mechanisms  are  represented  leaves  the  publisher  of  content  with  a problem  - they  can  either 
author  content  with  complex  play  rules  for  a specific  LE  and  risk  making  it  unfeasible  to  export  that  content  to  other  LEs  or  they  can  author 
static  content  that  is  easily  exported  between  different  LEs  with  no  play  rules. 

This  static  approach  to  the  content  leads  to  a ‘one  size  fits  all’  approach,  where  the  learner’s  are  all  presented  with  the  same  course  content 
and  assessments.  The  main  advantage  that  LEs  offer  over  the  classroom  situation  is  that  the  learner  can  set  a pace  that  best  suits  their 
learning.  In  academic  and  commercial  learning  situations  it  is  rarely  preferable  that  the  learner  decides  the  pace  at  which  they  digest  the 
material  fully,  as  constraints  such  as  formal  examinations,  which  tend  to  occur  at  fixed  points  in  time,  dictate  when  material  needs  to  be 
learned  by. 

Some  LEs  offer  facilities  to  include  remote  content,  i.e.  the  content  resides  on  a remote  (probably  HTTP)  server.  These  LE  do  not,  however, 
offer  standards-based  mechanisms  to  the  remote  server  to  request  or  pass  information  back  to  the  LE.  This  facility  is  usually  used  to  include 
static  content  from  a remote  server  because  for  a remote  server  to  provide  adapted  content  requires  integration  with  a proprietary 
communications  protocol. 

From  a pedagogical  perspective  LEs  do  not  offer  many  services  to  assist  the  learner  in  how  they  learn.  They  do  tend  to  offer  good 
management  facilities  to  the  course  coordinator,  such  as  - 

• Learner  and  class  management 

• Course  assembly  and  publishing 

• Learner  tracking  across  courses 

• Summative  assessment  information 

The  Learning  Environment,  therefore,  can  offer  learner  management  facilities  to  the  Adaptive  Hypermedia  Services. 

2.1  Requirements  of  the  Learning  Environment 

2.1.1  Control  Information 

The  Learning  Environment  needs  to  be  informed  of  when  the  learner  pauses  their  learning  within  the  Adaptive  Hypermedia  Service.  This 
control  information  may  be  used  by  the  LE  to  determine  whether  the  learner  should  be  launched  back  into  the  AHS  upon  logging  into  the  LE 
next  time.  Some  LEs  operate  on  a free  navigation  system,  where  the  learner  can  browse  to  any  section  within  the  course  and  the  course  is 
only  considered  complete  when  the  learner  has  received  an  assessment  for  each  section. 

LEs  using  a content  management  API  (see  4.4  Content  Interworking  below),  such  as  that  in  ADL  SCORM  vl.l  [15]  have  access  to  the 
required  data  model  element  cmi.core.lesson_status  to  indicate  whether  the  content  has  been  completed.  The  AHS  can  cater  for  learners  that 
have  completed  the  assessment  aspects  associated  with  their  required  objectives  and  received  an  assessment  score  from  the  AHS,  but  have  not 
completed  their  optional  objectives  yet  by  setting  the  value  of  this  data  model  element  to  incomplete  (which  is  part  of  the  restricted 
vocabulary  for  this  element). 

2.1.2  Assessment  Information 

Learning  Environments  tend  to  gather  assessment  information  on  the  course  section  level.  As  different  content  may  return  assessment 
information  in  a variety  of  ways  the  LE  often  requires  the  assessment  score  to  be  normalized.  The  AHS  may  have  to  return  assessment 
information  in  a particular  format. 

3.  ADAPTIVE  HYPERMEDIA  SERVICES 

Adaptive  Hypermedia  Services  allow  hypermedia  content  to  be  delivered  to  learners  in  an  adaptive  manner.  In  order  to  be  called  a service  the 
AHS  must  facilitate  ease  of  integration  in  order  that  learners  are  able  to  seamlessly  launch,  through  any  LE,  and  use  the  AHSs  adaptive 
content.  This  process,  as  far  as  the  learner’s  interaction  with  the  LE  is  concerned,  should  appear  no  different  from  regular  static  content.  It 
may  be  the  case  that  the  learner  is  studying  content  from  both  static  and  adaptive  sources  to  achieve  a learning  objective.  The  AHS  may, 
therefore,  be  called  as  part  of  a larger  course. 

3.1  Requirements  of  an  Adaptive  Hypermedia  Service 

AHS  information  requirements  depend  on  the  type  of  adaptivity  it  is  providing  and  on  whether  it  insists  on  gathering  this  information  for 
itself.  The  AHS  may  require  course  objectives,  learner  information  and  assessment  information  from  the  Learning  Environment  in  order  to 
successfully  produce  content  specific  to  the  learner’s  and  tutor’s  requirements. 

3.1.1  Objectives 

An  Adaptive  Hypermedia  Service  may  be  capable  of  delivering  more  content  than  the  tutor  requires  the  learner  to  study  in  order  to  achieve 
particular  learning  objectives  within  a curriculum.  It  is  necessary  that  the  AHS  be  able  to  determine  what  the  required  learning  objectives  are 
before  generating  a body  of  content  for  the  learner. 

Learners  tend  to  prefer  systems  where  they  maintain  control  of  their  learning  [10].  If  learners  have  total  control  over  their  objectives  they 
may  ignore  elements  of  the  content  that  are  required  within  the  curriculum.  To  cater  for  both  approaches  the  AHS  could  support  two  levels  of 
objectives  - required  and  optional.  When  the  learner  has  completed  (and  been  successfully  assessed  on)  all  of  the  required  learning  objectives 
set  by  the  tutor  they  are  considered  to  have  completed  the  AHS  courseware.  They  may  have  selected  optional  objectives  which  are  of  interest 
to  them,  or  that  they  feel  aid  their  learning.  They  may  continue  to  interact  with  the  AHS  until  all  of  these  objectives  are  also  completed. 

This  two-tiered  approach  to  objectives  gives  the  learner  control  of  their  learning,  while  assuring  the  tutor  that  they  have,  at  least,  coved  the 
core  objectives. 
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3.1.2  Learner  Information 

There  is  a large  amount  of  information  that  could  be  passed  between  the  Learning  Environment  and  the  Adaptive  Hypermedia  Service 
regarding  the  learner.  The  main  difficulty  in  passing  this  information  is  to  represent  it  in  a way  that  all  AHSs  and  LEs  can  understand. 
Candidate  specifications  include  PAPI  [12]  and  IMS  Learner  Information  Packaging  (LIP).  As  yet  neither  of  these  specifications  deal  with  the 
representation  of  pedagogical  aspects  of  the  learner,  such  as  learning  styles,  prior  knowledge  or  life  long  learning  goals  (i.e.  career  path).  Even 
if  there  was  a recognized  standard  for  passing  this  information  there  is  no  common  vocabulary  to  which  AHSs  could  adhere  to  process  the 
information.  Individual  AHSs  will  however  have  their  own  mechanisms  and  vocabularies  for  storing  pertinent  learner  information. 

The  primary  requirement  of  the  LE  with  respect  to  learner  information  is,  therefore,  that  they  can  uniquely  identify  learners  within  the  AHS. 
Globally  unique  identifiers  of  any  nature  is  a non- trivial  problem.  The  approach  suggested  here  is  similar  to  most  ‘solutions’  to  this  problem  - 
if  the  LE  is  running  from  a given  URL  and  has  internal  identifiers  for  learners,  then  the  AHS  could  use  a combination  of  the  LEs  URL  and 
internal  learner  identifier  to  produce  an  identifier  for  internal  AHS  use.  This  approach  does  have  one  major  shortcoming  — if  the  learner  is 
enrolled  in  a number  of  LEs  the  AHS  has  no  mechanism  to  recognize  that  the  learner  may  have  accessed  it  from  another  LE.  IMS’s  LIP 
Information  Model  [9]  document  discusses  this  issue  concluding  that  the  source  of  the  information  record  is  responsible  for  the  uniqueness  of 
the  learner  identifier  and  that  the  ‘uniqueness  of  the  source  (the  LE)  label  is  outside  of  the  scope  of  this  specification’. 

Another  issue  with  which  the  AHS  may  have  to  contend  is  whether  the  learner  is  already  engaged  in  another  course  which  also  utilizes  this 
AHS.  If  they  are  the  AHS  will  need  to  differentiate  between  the  two  by  asking  the  LE  for  a section  identifier  of  some  description.  This  will 
allow  the  AHS  to  use  the  correct  set  of  objectives  required  and  optional)  for  the  learner. 

If  the  LE  cannot  reasonably  be  asked  to  pass  pedagogical  information  about  the  learner  to  the  AHS  then  it  is  the  responsibility  of  the  AHS  to 
pre-test  the  learner  to  acquire  this  information.  This  pre-test  is  used  to  determine  the  learners  competencies  (e.g.  prior  knowledge)  and 
learning  preferences  (e.g.  learning  styles  and  display  preferences).  This  is  where  the  importance  of  identifying  the  learner  comes  to  the  fore.  If 
we  can  identify  the  learner  then  if  they  re-enter  the  AHS  we  do  not  need  to  perform  a full  pre-test  as  some  of  the  information  determined  in 
previous  tests  is  domain  independent,  such  as  some  learning  preferences.  Assessment  information  may  also  be  used  to  aid  the  determining  of 
the  learner’s  competencies. 

3.1.3  Assessment  Information 

As  mentioned  above  in  2.1  Requirements  of  the  Learning  Environment  LEs  generally  require  summative  assessment  information  from  each 
section  of  a course.  In  [14]  one  of  the  sixteen  design  guidelines  for  courseware  is  ‘specifying  entry  level  learner  competencies’  for  the 
content.  Within  the  Adaptive  Hypermedia  Service,  however,  there  are  no  fixed  competencies  required  as  the  AHS  delivers  content  that  best 
suits  the  learner’s  competencies.  The  AHS  must,  first  of  all,  determine  these  competencies. 

The  AHS  can  use  assessment  information  to  determine  at  what  level  to  pitch  the  competency  pre-testing  of  the  learner.  If  the  AHS  can  ask 
the  LE  for  the  assessment  information  of  both  the  learner  and  their  peers  for  sections  of  the  course  completed  then  it  is  possible  for  the  AHS 
to  determine  how  the  learner  is  performing  in  relation  to  his  peers  and  thus  gauge  the  difficulty  of  the  pre-test.  Assessments  should  not  be  too 
difficult  at  the  beginning  of  the  course  or  they  will  discourage  the  learner  [6].  It  is  also  undesirable  for  prior  knowledge  assessments  to  be  too 
easy  for  the  learner  as  the  assessing  of  the  learners  prior  knowledge  may  take  longer,  thus  increasing  the  time  before  the  learner  feels  they  are 
learning  new  (relevant)  material. 

The  aim  of  using  previous  assessment  information  is  to  determine  the  point  within  a difficulty  scale  at  which  the  pre-testing  should  beginning. 
At  this  point  certain  assumptions  may  be  made  about  the  learners  prior  knowledge.  The  task  of  the  pre-test  is  to  confirm  these  assumptions 
and  determine  if  the  learner  has  further  (non-assumed)  prior  knowledge  that  is  pertinent  to  their  objectives  and  the  content  the  AHS  will  be 
presenting.  Ideally  this  start  point  should  be  positioned  just  below  where  the  learner’s  competencies  actually  lie,  so  as  to  confirm  the  learner 
possesses  these  competencies  and  determine  where  the  boundary  of  their  competencies  lies. 

4.  STANDARDS  AND  SPECIFICATIONS 

4.1  IMS  Learner  Information  Package 

The  IMS  Learner  Information  Package  [9]  specification  address  interoperability  between  internet -based  learner  information  systems.  Learner 
information  concerns  Learners  (individual  or  group)  or  Producers  (creators,  providers  or  vendors).  LIP  includes  facilities  for  the  Learner  to 
determine  which  aspects  of  their  information  are  sharable  with  other  systems.  LIP  has  been  designed  with  four  basic  requirements  in  mind  - 
Distributed  Information,  Scalability,  Privacy  and  Data  Protection  and  Flexibility  and  External  References. 

The  last  requirement  is  described  in  [9]  as  Learner  information  includes  many  constructs,  such  as  learning  objectives  and  learning  history, 
which  are  in  practice  represented  by  different  structures  in  different  contexts.  Learner  information  data  models  must  be  flexible  enough  to 
accommodate  this  need.  An  external  reference  may,  in  the  future,  be  used  by  both  Learning  Environments  and  Adaptive  Hypermedia  Services 
to  share  learning  objectives  and  learning  style  information.  IMS  LIP  vl.O  is  currently  available  as  a public  draft. 

4.2  PAPI 

PAPI  [12]  is  the  IEEE  Public  and  Private  Information  Specification  which  is  a standard  format  for  the  representation  and  communication  of 
student  profiles.  The  purpose  of  the  specification  is  to  allow  the  creation  of  student  records  which  can  be  communicated  between  educational 
systems  over  the  lifetime  of  a learner. 

The  PAPI  specification  is  divided  into  four  areas  - Personal  Information,  Preference  Information,  Performance  Information  and  Portfolio 
Information  and  it  also  incorporates  the  Dublin  Core  metadata  element  set.  The  information  used  to  construct  the  user  profile  is  inferred  by 
the  system,  directly  input  by  the  user  or  is  constructed  by  the  user  and  system  in  collaboration.  PAPI  also  intends  to  address  the  privacy  and 
security  issues  involved  in  the  storage  and  communication  of  user  profile  [3] 
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4.3  IMS  Content  Packaging 

IMS  Content  Packaging  is  an  interoperability  specification  to  allow  content  creation  tools,  learning  management  systems  and  run-time 
environments  to  share  content  in  a standardized  set  of  structures.  Version  1.1  of  the  specification  is  focused  on  defining  interoperability 
between  systems  that  wish  to  import,  export,  aggregate,  and  disaggregate  packages  of  content  [8]. 

The  primary  goal  of  the  IMS  Content  Packaging  specification  is  to  provide  a mechanism  which,  once  implemented  by  producers  and  vendors, 
will  allow  content  to  be  exported  between  systems  with  the  minimum  of  effort.  Version  1.1.  of  this  specification  is  currently  available  as  a 
public  draft. 

4.4  Content  Interworking 

The  Content  Interworking  API  was  initially  developed  as  part  of  the  AICC  CMI,  more  recently  developed  as  part  of  ADL  SCORM  and  the 
University  for  Industry  Content  Interworking  Specification. 

It  is  proposed  that  the  IMS  Content  Management  group  will  adopt  the  Content  Interworking  API  as  specified  in  the  AICC  CMI  specification 
v3.0.1  and  currently  under  implementation  by  the  ADL  [15]  and  by  the  University  for  Industry  [16]. 

4.4.1  Aviation  Industry  CBT  Committee  Computer  Managed  Instruction  Guidelines 

The  AICC  guidelines  [1]  provide  a method  for  seamless  data  flow  between  different  computer  based  training  (CBT)  lessons  and  Computer 
Managed  Instruction  (CMI)  systems,  between  different  CMI  systems,  and  from  CBT  lessons  created  with  different  authoring  systems  to  a 
common  data  store  and  off-the-shelf  analysis  tools. 

The  driving  force  behind  the  development  of  the  AICC  CMI  was  to  allow  content  developed  by  different  authors  to  be  used  with  any  CMI 
system  that  supports  the  guidelines.  To  this  end  an  API  was  designed  that  allowed  the  content  to  connect  to  the  CMI  system.  The  API 
principally  facilitates  the  getting  and  setting  of  data  in  the  CMI  systems  data  model. 

There  are  two  aspects  of  the  AICC  approach  to  enabling  interoperability  of  CMI  systems  with  different  CBT  systems  - 

1.  Lesson  launch:  The  CM!  should  have  a standard  approach  to  CBT  lesson  initiation,  and 

2.  Communication:  The  CMI  should  have  a standard  approach  to  providing  information  to  the  CBT  lessons,  and  receiving 

information  from  the  CBT  lessons. 

4.4.2  Advanced  Distributed  Learning  SCORM 

ADL  SCORM  [15]  (Sharable  Courseware  Object  Reference  Model)  is  based  on  the  AICC  CMI  guidelines.  ADL  was  established  with  the  purpose 
of  developing  a (US)  Department  of  Defense  wide  strategy  for  using  learning  and  information  technologies  to  modernize  education  and 
training.  Version  1 of  SCORM  defines  a reference  model  to  facilitate  the  interworking  of  Learning  Management  Systems  (LMS)  and  content 
providers  material.  (LMS  is  used  in  the  SCORM  documentation  in  place  of  CMI).  The  SCORM  is  based  directly  on  the  runtime  environment. 

The  ADL  collaborated  with  AICC  members  and  participants  to  develop  a common  Launch  and  API  specification. 

4.4.3  IMS  Content  Management 

The  IMS  Content  Management  [7]  specification  is  currently  under  development.  The  goal  of  the  IMS  Content  Management  specification  is 
to  establish  a standard  for  data  interchange  and  communication  between  instructional  content  and  run-time  environments.  This  specification 
should  provide  cost-effective  content  interoperability  for  platform,  tool  and  content  developers  when  implementing  the  specification  in  their 
products. 

It  is  proposed  that  the  Content  Interworking  API  to  be  used  within  IMS  Content  Management  will  also  be  based  on  AICC  SME  guidelines  and 
ADL  SCORM. 

5.  ARCHITECTURE 

The  mechanisms  required  of  an  open  architecture  for  integrating  Adaptive  Hypermedia  Services  with  Learning  Environments  are  standards- 
based  approaches  to  - 

• Importing  the  location  of  the  AHS  into  the  LE 

• Launching  the  AHS  from  the  LE 

• Communicating  between  the  AHS  and  LE 

A commonly  understood  data  model  is  also  required  to  facilitate  the  communication  of  learner  information,  learning  objectives  and  assessment 
information. 

5.1  Importing  the  AHS  location  into  the  LE 

IMS  Content  Packaging  will  provide  a mechanism  for  static  content  to  be  integrated  into  a Learning  Environment.  By  importing  content  as 
part  of  an  IMS  Content  Packaging  archive,  with  an  XML  manifest  included,  it  is  envisaged  that  the  content  can  be  automatically  installed  into 
the  LEs  content  repository. 

With  Adaptive  Hypermedia  Services  the  content  is  not  shipped  in  discrete  units  of  material.  The  AHS  is  a remote  service  that  the  LE  has  an 
entry  point  to.  As  the  LE,  such  as  Microsoft’s  LRN  [11],  may  already  have  an  IMS  Content  Packaging  import  facility  a reasonable  approach 
would  be  to  utilize  this  and  enhance  it  to  allow  for  the  importing  of  the  location  of  the  AHS  as  part  of  an  IMS  Content  Package.  This  could 
be  imported  into  a LE  in  a similar  way  to  static  content.  The  manifest  would  describe  the  adaptive  service  and  also  contain  a URL  which  the 
LE  would  use  to  launch  the  service. 
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Another  consideration  on  the  importing  of  the  Content  Package  is  that  the  learning  objectives  that  the  AHS  is  capable  of  achieving  should 
also  be  imported  into  the  LE.  Provision  should  be  given  to  the  tutor  to  specify  which  learning  objectives  they  would  like  the  learners  to 
achieve  within  the  AHS. 


5.2  Launching  the  AHS  from  the  LE 

When  the  Adaptive  Hypermedia  Service  is  launched  from  the  Learning  Environment  the  content  it  delivers  will  be  coming  form  a different 
server  than  the  Learning  Environment.  The  AHS  firstly  initializes  its  connection  with  the  LE.  The  AHS  needs  to  then  determine  if  the 
learner  has  been  engaged  in  this  course  component  before.  If  they  have  then  it  can  rebuild  the  adaptive  course  component  for  the  learner  from 
a snapshot  taken  prior  to  the  content  being  paused. 

If  the  learner  has  not  utilized  this  adaptive  course  component  before  then  the  AHS  must  determine  two  things  - what  the  learning  objectives 
are  for  the  learner  and  whether  the  learner  has  been  engaged  in  any  course  component  offered  by  the  AHS  before.  The  learning  objectives  are 
used  by  the  AHS  to  determine  the  required  objectives  for  the  content  it  is  about  to  deliver.  Optional  objectives  can  also  be  offered  based  on  the 
required  objectives  and  the  learner’s  prior  knowledge.  If  the  learner  has  used  the  AHS  before  from  the  launching  LE  this  will  alleviate  the 
necessity  to  determine  the  learning  style  and  possibly  some  prior  knowledge  aspects  of  the  learner.  It  will  still  be  necessary  to  pre-test  the 
learner  as  their  assumed  prior  knowledge  may  have  degraded  with  time  and/or  they  may  have  learned  further  relevant  knowledge. 


5.3  Communication  between  the  AHS  and  the  LE 

Communication  between  the  Adaptive  Hypermedia  Service  and  the  Learning  Environment  can  be  achieved  using  the  SCORM  Runtime 
Communication  API  as  used  in  SCORM  vl.l.  A subtle  modification  to  the  HTML  frame  layout  is  required  at  the  AHS  to  enable  calls  to  API 
functions  residing  on  the  LE  from  AHS  content.  The  actual  API  calls  used  are  the  same  as  those  used  in  SCORM  vl.l  as  the  API  is  designed  to 
get  and  set  values  that  are  separately  defined  by  an  external  data  model  [15]. 


Figure  1.  The  AHS  accessing  the  LE  data  model  using  the  Content  Interworking  API 
The  remote  AHS  calls  the  Content  Interworking  API  to  access  the  data  model  on  the  LE  using  the  following  process  - 

a)  The  learning  content  (right  browser  frame)  and  JavaScript  API  (left  browser  frame,  hidden)  are  delivered  to  the  learner’s  browser. 

b)  An  API  function,  in  the  left  hand  API  frame,  ( e.g.  LMSGetValue(“cmi.core.leason_status”))  is  called  from  the  content  frame. 

c)  The  API  frame  communicates  the  request  to  the  Learning  Environment. 

d)  The  Learning  Environment  returns  the  value  (in  this  case  of  cmi.core.leason_status)  to  the  API  Frame. 

e)  The  function  returns  the  value  to  content  frame. 

f)  The  value  may  be  passed  back  to  the  Adaptive  Hypermedia  Service. 

5.3.1  Common  Data  Model 

The  data  model  required  for  AHS/LE  interaction  is  similar  to  that  proposed  in  SCORM  vl.l.  The  key  requirements  of  the  data  model  for  this 
interaction  are  - 

• Learning  Environment  identifier  (to  uniquely  identify  the  LE,  possibly  it’s  URL). 

• Learner  identifier  (to  uniquely  identify  a learner  from  a LE). 

• Section  identifier  (to  determine,  using  the  elements  above,  whether  the  student  has  attempted  this  content  in  the  AHS  before). 

• Section  status  (completed,  incomplete,  attempted  etc.) 

• Section  objectives  (allows  the  AHS  to  query  the  LE  for  the  learning  objectives  the  tutor  wishes  learners  to  achieve). 

• Score  information  (across  all  sections,  for  the  learner  and  averages  for  all  enrolled  students). 

• Score  range  (the  LE  may  not  use  percentile  scoring  and  the  AHS  may  need  to  normalize  the  scores). 

The  number  of  SCORM  data  model  elements  has  been  significantly  reduced  between  version  1.0  and  version  1.1.  This  reduction  was  to  aid 
harmonization  between  independent  developments  of  the  Content  Interworking  API  (see  4.4  Content  Interworking),  i.e.  many  groups  where 
using  the  same  API,  but  had  subtly  different  data  models.  Many  of  the  optional  elements  defined  in  SCORM  vl.O  had  not  been  implemented 
by  other  consortia.  A complete  list  of  the  data  model  changes  between  SCORM  versions  is  available  [15]. 

Most  of  the  required  data  model  elements  for  AHS/LE  interaction  already  exist  in  some  guise  in  the  CMI  data  model  used  in  SCORM  vl.l. 
Learner  identifier,  section  identifier,  section  status,  section  objectives  and  score  range  all  have  equivalents.  The  LE  identifier  and  summative 
score  information  is  not  available  within  the  data  model. 
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6.  IMPLEMENTATION 

The  mechanism  for  interfacing  LEs  and  AHSs  described  in  this  paper  has  been  implemented  using  Fretwell-Downing  Education’s  LE  [5]  and 
Trinity  College,  Dublin’s  AHS  [3]  within  the  European  Commission  funded  EASEL  [4]  project.  The  data  model  used  is  based  on  SCORM  vl.l 
[15]  with  the  addition  of  LE  identifier  and  summative  score  elements.  Using  this  model  it  is  possible  for  the  AHS  and  LE  to  interact,  sharing 
information  about  the  learner  including,  but  not  limited  to  - 

• Learner  Identification 

• Interface  Preferences 

• Pedagogical  Preferences 

• Performance  Information 

The  first  piece  of  information  that  is  requested  by  the  AHS  from  the  LE  is  the  LE  identifier.  This  should  be  unique,  possibly  URI  based,  and 
allows  the  AHS  to  contextualize  all  further  communications.  For  example,  when  it  asks  for  the  learner’s  identification  it  might  assume  that 
this  identifier  is  unique  within  the  LE,  but  not  across  other  Learning  Environments. 

Learner  identification  enables  the  AHS  to  ascertain  if  the  learner  has  visited  the  service  before  and  if  they  have  determine  whether  it  is 
necessary  to  pre-test  them.  For  example,  if  the  learner  completed  another  course  within  the  AHS  already  then  it  should  have  pedagogical 
information  that  is  sufficient  to  adapt  to  the  learners  requirements  and  abilities.  If  the  AHS  has  no  information,  or  no  appropriate 
information  about  the  learner  then  it  is  necessary  to  pre-test. 

Prior  to  administering  the  pre-test  the  AHS  can  query  the  LE  for  assessment  scores  for  any  material  the  learner  has  attempted  in  the  course 
(assuming  the  AHS  is  a component  of  a larger  course)  and  also  request  the  assessment  scores  for  the  learner’s  peers  in  the  course.  These  values 
are  returned  in  as  normalized  values  in  a range  specified  by  the  LE.  Using  this  information  the  AHS  can  determine  the  difficulty  at  which  to 
set  the  pre-test,  if  it  has  multiple  pre-tests  aimed  at  increasing  levels  of  knowledge.  Such  a pre-test  can  be  used  to  determine  the  prior 
knowledge  about  a domain  before  generating  the  personalized  course. 

Other  extensions  that  may  be  made  to  the  LEs  data  model  could  cater  for  - Cultural  Background,  Preferences  & Learning  Culture, 
Communication  Style  and  Needs,  Cognitive  and  Learning  Style,  Prior  Knowledge  & Expertise,  Communication  Style  & Needs,  Learning 
History  and  Objectives  and  Goals. 

The  primary  constraint  to  adding  these  complex  elements  within  the  data  model  is  that  of  a commonly  understood  and  accepted  vocabulary  to 
describe  each  extension.  This  is  why  the  approach  taken  within  EASEL  was  to  allow  the  Adaptive  Hypermedia  Service  to  use  a proprietary 
vocabulary  internally,  but  to  return  assessment  information. 

Assessment  information  is  returned  to  the  LE  using  the  normalized  range  mentioned  above.  This  allows  the  AHS  to  inform  the  LE  how  the 
learner  has  performed  in  any  assessments  they  have  completed,  but  this  is  only  a summative  score  for  all  assessments  completed  in  the  AHS. 
The  IMS  Question  and  Test  specification  was  investigated  as  a candidate  for  richer  communication  of  learner  assessment  results,  but  the 
uncertainty  as  to  whether  an  individual  assessment  would  be  present  in  a personalized  course  meant  the  information  for  different  learners 
might  be  inconsistent. 

7.  SUMMARY 

This  paper  has  described  a mechanism  for  integrating  Adaptive  Hypermedia  Services  and  Learning  Environments  and  illustrated  this  approach 
with  an  implementation  from  the  EASEL  project.  The  mechanism  described  is  based  in  current  and  emerging  learning  technology 
specifications  and  requires  minimal  modifications  to  LEs  that  utilize  these  specifications  for  their  intended  purposes. 

By  enabling  AHSs  to  be  successfully  integrated  into  LEs  this  mechanism  provides  the  potential  for  more  learners  to  avail  of  the  pedagogical 
benefits  possible  using  customized  adaptive  content. 
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Abstract:  Contemporary  humanities  students  tend  to  think  that  human  expression  occurs  in  a vacuum. 
They  do  not  understand  that  human  expression  in  any  given  time  period  is  produced  by,  and  remadcs,  the 
Zeitgeist  of  that  period  while  at  the  same  time  exhibiting  characteristics  of  human  nature  that  have  existed 
since  the  beginning  of  human  history.  Humanities  students  need  connection  among  disciplines,  among 
cultural  forces,  with  history,  and  to  themselves.  This  paper  illustrates  how  technology  and  Internet  access 
can  change  the  roles  of  instructor  and  learner  alike,  as  well  as  improve  the  pattern  of  learning  behaviors  in  a 
classroom.  Through  online  resources  students  can  gain  a better  sense  of  Zeitgeist  in  a given  culture  at  a 
given  time;  at  the  same  time  they  can  recognize  the  universality  of  the  human  experience.  The  Internet  and 
Web  page  design  tools  also  provide  opportunities  for  students  to  conduct  research  themselves  and  to 
publish  their  research  efforts  in  new  and  engaging  ways. 


Introduction 

Because  of  movements  toward  specialization  and  compartmentalization  in 
education  through  the  course  of  the  twentieth  ceitury,  many  of  my  students  have  a 
fragmented  view  of  the  world.  To  some  extent,  they  have  been  taught  art,  literature,  and 
music,  but  they  do  not  understand  any  historic  or  contemporary  connections  among  these 
disciplines,  nor  do  they  recognize  relationships  among  these  and  science,  technology, 
religion,  politics,  and  mathematics.  Thus,  they  tend  to  think  that  human  expression 
occurs  in  a vacuum.  They  do  not  understand  that  human  expression  in  any  given  time 
period  is  produced  by,  and  remarks,  the  Zeitgeist  of  that  time  period.  Human  expression, 
however,  also  exhibits  those  characteristics  of  human  nature  that  have  existed  both  before 
and  since  the  beginning  of  recorded  history.  Students  in  the  humanities  really  need 
connection  among  disciplines,  among  cultural  forces,  with  history,  and  to  themselves. 
With  this  rationale  in  mind,  and  after  a brief  overview  of  my  evolving  pedagogical 
method,  I seek  in  this  presentation  to  illustrate  how  technology  and  Internet  access  can 
change  the  roles  of  instructor  and  learner  alike,  as  well  as  improve  the  pattern  of  learning 
behaviors  in  a classroom  (Dalgamo,  2001).  Technology  can  provide  opportunities  to 
interweave  resources  so  that  students  can  gain  a better  sense  of  Zeitgeist  in  a given  culture 
at  a given  time;  at  the  same  time  they  can  recognize  the  universality  of  the  human 
experience.  The  Internet  and  Web  page  design  tools  also  provide  opportunities  for 
students  to  conduct  research  themselves  and  to  publish  their  research  efforts  in  new  and 
engaging  ways. 

Pedagogical  Methods 

Before  discussing  interweaving  strategies  with  an  example  text,  I must  comment 
on  my  changing  method  of  teaching  the  course  from  which  the  example  comes.  The 
course  is  pre- modem  world  literature.  I used  to  teach  this  course  in  a lecture  and  class 
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discussion  pattern,  but  in  recent  years  I have  come  to  recognize  that  students  learn  best 
when  they  are  active  collaborative  learners  (Anstendig,  Meyer,  & Driver,  1998;  Barab, 
Thomas,  & Merrill,  2001;  Lu,  Zhu,  & Stokes,  2000;  Scheurman,  1997)  and  that  “[t]he 
internalization  of  capabilities  that  constitutes  learning  is  ...  as  much  a social  as  an 
individual  cognitive  phenomenon”  (Borthick,  2001).  Therefore,  I first  modified  the 
course  by  having  my  students  do  research  and  oral  presentations  on  works  during  the 
course  of  the  semester  (Mintz,  1998).  However,  I realized  that  this  alteration  was  not 
sufficient  to  change  the  learning  patterns  in  the  class  and  to  engage  and  enable  my 
students  on  the  path  to  learning.  Thus,  I decided  to  have  my  students  prepare  for  class  by 
reading  the  assigned  text  and  answering  directed  questions  that  I now  place  on  the  course 
calendar  in  WebCT.  The  questions  relate  to  structure,  themes,  archetypes,  metaphors, 
etc.,  of  that  text  in  relation  to  earlier  works  as  well  as  to  contemporary  culture,  and  they 
build  upon  each  other  through  the  course  of  the  semester.  At  the  beginning  of  class,  I 
give  a daily  quiz  to  ensure  that  students  have  read  the  work.  Then  students  give  their 
scheduled  presentations  on  that  day’s  text.  After  these  presentations,  students  break  up 
into  groups,  physically  moving  their  chairs  to  face  each  other  and  thus  altering  the 
monolithic,  instructor- centered  structure  of  the  classroom.  They  share  their  responses  to 
the  online  questions  and  decide  on  the  best  answers  to  each  question  regarding  the  work. 
During  this  time  I wander  through  the  classroom,  listening  to  group  exchanges,  asking 
more  guided  questions  to  help  them  “flesh  out”  their  better  responses,  and  expanding 
upon  their  insights  by  making  reference  to  historical  information  or  to  past  texts  with 
which  they  are  now  familiar.  Finally  the  class  comes  together  to  answer  the  questions, 
and,  using  Internet  sources  to  supplement  the  discussion,  I reinforce  those  answers  with 
the  information  that  I once  imparted  through  lecture.  Frequently,  however,  students  end 
up  telling  me  what  I would  have  told  them  in  a lecture.  In  other  words,  they  discover  and 
own  the  knowledge.  Also,  quieter  students  respond  very  positively  to  this  format  because 
it  tends  to  give  them  voices;  the  quiet  student  learns  that  if  I call  on  him  or  her  to  answer 
when  the  class  is  working  together,  then  I have  heard  his  or  her  answer  and  thought  it  a 
good  one. 

As  an  illustration  of  effective  Internet  use,  in  this  presentation  I will  employ  the 
medieval  play,  Everyman,  as  an  example.  To  instill  in  my  students  an  understanding  of 
the  medieval  world  view  vis-a-vis  the  play,  I provide  links  to  sites  in  both  WebCT  as  an 
accompaniment  to  the  questions  and  on  the  online  syllabus  that  is  produced  in  FrontPage 
and  linked  from  my  home  page.  For  religious,  political,  economic,  and  other  reasons,  the 
common  medieval  European  was  encouraged  to  think  of  his  station  in  life  as  an  act  of 
God’s  will.  In  his  relatively  harsh  world  he  was  encouraged  to  focus  on  the  afterlife  and 
on  the  purgation  of  sin  in  this  life  in  order  to  avoid  the  harsh  judgment  of  God.  To  show 
that  medieval  architecture  reflected  this  thinking  by  drawing  the  eye  upward  to  heaven 
and  to  beauty  at  great  heights,  I furnish  links  to  the  cathedral  at  Chartres.  To  illustrate  the 
fact  that  art  production  and  the  transcription  of  religious  texts  were  considered  by  many 
as  acts  of  spiritual  devotion  and  thus  were  quite  elaborate,  I provide  links  to  medieval  art 
and  to  illuminated  manuscripts.  I also  furnish  links  to  audio  sources  of  medieval  music 
including  Gregorian  chants,  as  well  as  to  sites  that  treat  medieval  quotidian  life  and 
demonstrate  not  only  cultural  Zeitgeist  but  also  how  very  human  and  like  us  the  medieval 
Europeans  were.  My  students  view  these  sites  on  their  own,  but  by  the  time  we  revisit 
them  together  in  class,  students  can  really  begin  to  grasp  how  these  different  means  of 
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expression  and  cultural  behaviors,  as  well  as  universal  behaviors,  are  tied  together.  In 
addition,  in  order  to  help  them  to  understand  the  dynamic  quality  of  the  English  language 
especially  during  the  rise  of  the  printing  press,  I link  to  an  early  printing  of  Everyman,  in 
which  ‘u’  and  ‘v’  were  used  each  others’  places  when  compared  to  modem  usage.  This 
recognition  serves  to  alter  students’  misguided  perspectives  that  English  has  historically 
been  static. 

Having  witnessed  the  positive  effects  of  Internet  use  in  teaching  this  world 
literature  course,  I then  decided  to  let  my  students  use  the  Web  to  share  some  aspects  of 
what  they  had  learned.  In  lieu  of  the  traditional  research  assignment,  in  recent  semesters 
I have  given  my  students  the  option  of  designing  research  Web  sites.  Since  both 
traditional  essays  and  research  sites  were  assessed  by  Clayton  College  and  State 
University’s  established  writing  criteria,  those  who  chose  traditional  essays  were  by  no 
means  penalized  for  their  choice.  Because  of  difficulties  with  hypertext  readability,  the 
only  difference  in  the  assignment  was  that  the  traditional  essays  were  to  be  five  to  seven 
pages  in  length,  while  online  essays  were  to  be  approximately  five  pages  in  length.  As 
examples  provided  during  this  conference  presentation  showed,  students  who  made  the 
research  site  choice  found  great  benefits  in  publishing  on  the  Web.  Those  benefits 
included  learning  to  use  established  criteria  for  authoritative  Web  sites;  reasoning  out  for 
themselves  how  interdisciplinary  forces  were  at  work  in  a culture  by  using  visual,  audio, 
and  textual  sources;  linking  directly  to  sources  from  their  sites  and  even  within  the  text  of 
their  essays;  and  using  their  own  creativity  in  site  design.  There  were  also  problem  areas. 
Those  included  problems  with  coherence  in  the  multimedia  environment  and  with 
transference  of  text  from  one  application  to  another.  For  example,  many  HTML  editors 
do  not  permit  or  recognize  indents,  so  proper  paragraphs  and  MLA  “Works  Cited”  lists 
cannot  be  composed  in  them  nor  pasted  into  them.  Students  also  ran  into  problems  with 
changing  or  alternative  rules  for  citation  of  sources  on  the  Internet.  In  addition,  some 
students  lacked  Web  design  experience,  and  their  sites  verified  this  lack.  However,  a few 
of  my  novice  designers  produced  the  most  remarkable  sites  with  well-composed  essays. 

Conclusion 

My  methods  in  this  course  are  still  progressing  toward  a those  that  facilitate  the 
best  learning  for  the  student,  but  the  rate  of  student  information  retention  is  markedly 
higher,  as  is  the  positive  awareness  of  learning  among  students;  both  superior  anecdotal 
exam  evidence  and  more  positive  student  responses  to  questiormaires  in  my  classes 
versus  those  in  the  traditionally  taught  courses  indicate  these  improvements.  Students 
utilize  and  recognize  their  knowledge  and  empowerment. 

As  I am  discovering,  from  one  semester  to  the  next,  authoritative  sources  for  tlie 
humanities  are  appearing  at  an  exponential  rate  on  the  Internet.  Our  use  of  these  sources 
with  irmovative  teaching  strategies  can  serve  to  alter  the  roles  of  instructors  and  learners. 
This  use  can  serve  to  enhance  our  students’  grasp  of  a work  within  its  cultural  Zeitgeist \ it 
can  also  illustrate  more  clearly  the  universality  of  human  characteristics  from  the 
begirming  of  recorded  history.  In  short,  these  resources  can  recormect  our  students  to  the 
world  around  them  in  encompassing  and  interdisciplinary  ways.  In  addition,  Web 
publishing  offers  new  means  for  students  both  to  synthesize  research  information  and  to 
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compose  innovative  hypertext  essays.  The  cyber-age  is  a breakthrough  time  for  new 
media  and  new  methods  of  teaching  and  learning  in  the  humanities. 
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Abstract:  In  an  online  course  the  role  of  tutor  is  fundamental  and  the  choice  of  people  to  hold 
this  role  is  very  important.  In  this  speech  I shall  look  in  detail  at  the  skills,  requirements  and 
criteria  used  to  choose  the  three  tutors  for  the  Metodi  99  project  (an  online  course  in 
Methodology  and  Techniques  of  Social  Research  delivered  in  1998/1999  by  the  Ctu  of  Milan 
University  and  Faculty  of  Political  Science),  how  this  choice  was  made,  what  strategies  were 
adopted  and  what  results  were  obtained. 


Context 

Metodi  99  was  organised  in  two  different  environments:  a Web  site  for  the  publication  of  teaching  materials  and 
communication  (forum)  among  professor  tutors  and  students  and  a First  Class  software  where  participants 
communicated  with  each  other  in  five  spaces  (Technical  Aid,  Mailbox,  Secretary,  Student  Area  and  a Project 
Area  limited  to  the  project  team).  Due  to  some  problems  relating  to  the  financing  of  the  project,  the  confirmation 
of  the  feasibility  immediately  preceded  the  production  phase,  making  it  necessary  to  find  an  urgent  solution  to 
several  very  important  questions  only  a few  days  before  the  start  of  the  course.  All  these  considerations  had  great 
repercussions  on  the  tutoring,  first  and  foremost  on  the  choice  of  people  suitable  for  assuming  this  role  and, 
second,  on  the  preparation  of  an  appropriate  training  programme. 


Requirements  and  Role 

In  an  online  environment  where  there  is  no  verbal  and  physical  language  to  help  the  dialogue,  interpersonal 
communication  techniques  are  particularly  important.  As  D.  Rowntree  says  “Tutoring  an  online  course  is  not  an 
easy  option.  There  is  a lot  to  do”.  There  are  many  potential  roles  for  the  tutor  (pedagogical,  social,  managerial 
and  technical)  in  different  areas  (organisational,  stuctural,  social  and  conceptual).  In  our  course  the  tutor  was 
halfway  between  professor  and  students.  In  the  first  place,  the  tutor  had  to  be  an  “expert  in  the  subject”  to 
provide  students  with  didactic  and  methodological  support,  to  give  advice  on  materials  and  clear  up  any  doubts. 
Secondly,  he  or  she  had  to  possess  good  “interpersonal  skills”  to  communicate  with  students  in  simple  terms 
choosing  a suitable  register,  and  with  the  professor  and  project  team.  Lastly,  a “reasonably  good  training  in 
computer  skills”  was  necessary  in  order  to  be  at  ease  with  the  use  of  a computer,  the  network  and  the  hardware 
and  software  necessary  to  follow  the  course. 

The  tutor’s  main  task  was  to  look  after  a group  of  about  twenty  students  both  from  a didactic  and  an 
organizational  point  of  view,  explaining  the  activities  to  be  carried  out,  allocating  assignments,  sending 
reminders  of  deadlines  and  assisting  the  professor  in  deciding  grades.  In  terms  of  time,  the  role  involved  at  least 
one  hour  online  every  day  including  weekends. 


Choice  and  Training 

Three  young  people  were  chosai:  Francesca,  Marco  and  Matteo.  None  of  them  had  ever  had  experience  of 
tutoring  online  but  all  of  them  were  proficient  in  their  subject.  Their  backgrounds  were  substantially  different 
and  this  in  tern  had  a strong  influence  on  their  attitude  to  their  new  role.  Francesca  was  a political  science  student 
at  the  end  of  her  university  studies  and  working  occasionally  for  her  professor;  Marco  was  a recent  graduate 
hoping  to  do  a doctorate  but  without  didactic  experience;  Matteo  (the  oldest)  had  had  several  teaching  and 
training  experiences  in  the  research  methodology  field.  These  different  “life  experiences”  also  resulted  in  a 
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difference  in  the  amount  of  time  available  for  the  course:  Matteo’s  work  kept  him  occupied,  so  that  he  was  more 
distant,  at  times  absent. 

A brief  training  period  was  arranged  to  explain  the  role  of  the  tutor  with  some  documentation  and  examples  of 
messages  sent  by  the  tutors  of  the  previous  courses;  a session  was  also  held  with  the  course  designer  who 
clarified  their  role  and  responsibilities.  Compensation  for  the  curtailed  training  period  was  provided  in  the  form 
of  a tutor  co-ordinator  with  previous  experience  as  the  “observer”  of  an  online  course  and  a good  knowledge  of 
the  role,  particularly  from  a communicative  point  of  view.  Together  with  the  course  designer,  this  figure  was  of 
vital  importance  in  helping  the  three  tutors  to  assume  their  role. 


The  experience 

Before  the  start  of  the  course,  the  tutor  co-ordinator  made  contact  through  a series  of  messages  of  a theoretical 
nature,  soliciting  reflection  on  the  role  of  the  tutor.  A firm  but  flexible  attitude  was  advocated,  in  order  to 
provide  students  with  suggestions  and  encourage  them  to  reflect  on  the  content.  In  doing  this,  the  students 
needed  to  bear  in  mind  that  the  learning  environment  was  not  a traditional  one  but  based  on  words,  where  written 
communication  had  a set  of  different  rules,  characteristics,  advantages  and  limits  compared  with  spoken 
communication  and  these  could  not  be  ignored  (Ong,  1982).  When  the  course  started,  the  three  tutors  soon 
realized  that  theory  was  very  different  from  practice  and  they  were  obliged  to  face  up  to  a kind  of  role  crisis, 
with  doubts  as  to  their  identity  as  tutors.  A theoretical  knowledge  of  their  tasks  was  insufficient  and  the  standard 
support  outlined  in  the  documentation  handed  out  at  the  start  of  the  course  broke  down  in  practice. 

All  three  tutors  experienced  the  same  sense  of  unease  and  inadequacy  and  regretted  the  lack  of  a longer  training 
period  of  a more  practical  nature.  Francesca  felt  the  most  disoriented:  she  was  strongly  tempted  to  place  herself 
on  a level  with  the  students,  as  she  still  felt  part  of  this  category,  rather  than  assuming  more  authority. 

Reactions  differed:  whilst  Francesca  and  Marco  attempted  to  look  for  support  in  the  project  team,  Matteo  tended 
to  remain  outside,  limiting  himself  to  observing  his  own  class.  This  did  not  imply  lack  of  interest  in  what  was 
going  on  but  the  urge  to  reflect  on  his  role  and  question  his  decisions.  Of  the  three  tutors,  he  was,  in  fact,  the  one 
who  revealed  the  greatest  maturity  from  a professional  point  of  view  and  best  covered  his  role. 

In  this  initial  phase  of  general  disorientation,  the  advice  of  the  tutor  co-ordinator  proved  to  be  a great  help,  since 
he  helped  solve  specific  practical  problems,  making  suggestions  as  to  the  style  of  communication  to  be  adopted. 
The  tutors'  ways  of  relating  to  the  students  were  different:  Francesca  was  always  more  than  ready  to  help,  since 
she  felt  closer  to  them  in  terms  of  age  and  experience,  and  the  style  of  her  messages  was  very  colloquial  and 
informal.  Marco  gave  the  impression  of  being  stricter  and  less  willing  to  intervene  by  encouraging  the  students 
to  participate  and  learn,  being  aware  that  at  university  each  of  them  was  responsible  for  his  own  learning  route. 
In  his  view,  the  tutor  should  merely  provide  the  tools  and  students  decide  whether  to  take  advantage  of  them  or 
not.  Lastly,  Matteo  proved  to  be  the  most  formal  and  detached;  certainly  the  most  thoughtful.  From  him  came  the 
most  interesting  observations  on  the  difficulties  encountered:  “assuming  your  role  is  different  from  assuming 
responsibility,”  he  told  us;  “it  is  essential  to  compare  yourself  with  other  people  in  order  to  learn  how  to  behave”. 
These  comments  were  inspired  by  the  difference  between  the  role  of  the  traditional  university  assistant  and  the 
role  of  the  online  tutor.  In  conclusion,  Metodi  99  taught  us  a great  deal  about  tutoring:  it  is  not  easy  to  find 
people  for  this  role;  a training  course  before  the  start  of  the  course  is  essential;  personal  background  can  have  a 
strong  influence  both  on  the  way  the  role  is  approached  and  on  the  style  of  communication. 
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Abstract:  Data  are  presented  on  the  effects  of  Animated  Agents  on  multimedia  learning  environments  with 
specific  concerns  of  split  attention  and  modality  effects.  The  study  was  a 3 (agent  properties:  agent  only, 
agent  with  gestures,  no  agent)  x 3 (picture  features:  static  picture,  sudden  onset,  animation)  factorial  design 
with  outcome  measures  of  mental  load  rating  scale,  a persona  rating  scale,  multiple-choice  questions,  a 
matching  test,  a retention  test,  and  transfer  tests  involving  creative  solutions.  Overall,  there  were  no  split 
attention  or  modality  effects  found  with  integrating  the  agent  into  the  display. 


KEYWORDS;  animated  pedagogical  agents,  multimedia  learning,  split  attention  effects, 
attentional  control 

Interest  in  the  use  of  animated  pedagogical  agents  in  instructional  design  involving  multimedia  in 
virtual  learning  environments  has  increased  recently,  as  new  technologies  have  made  them  more  accessible 
(Craig,  Hu,  Marks,  & Graesser,  1999;  Johnson,  Rickel,  & Lester,  2000).  An  animated  pedagogical  agent  is 
a computerized  character  (either  humanlike  or  otherwise)  that  can  interact  with  a user  in  order  to  impart 
some  type  of  information.  Because  these  agents  are  relatively  new,  there  has  been  little  research  into  their 
proper  construction,  capabilities,  use,  or  limitations. 

Animated  agents  can  be  seen  as  logical  extensions  of  the  development  and  customization  of  new 
learning  interfaces.  A line  of  research  conducted  by  Byron  Reeves  and  Clifford  Nass  (1996)  provides  a 
context  for  exploring  these  interfaces.  They  put  forth  the  basic  principles  of  what  they  call  the  "Media 
equation  theory."  This  model  holds  that  people  naturally  interact  with  various  forms  of  media  in  the  same 
ways  they  interact  with  other  people.  In  this  context,  media  can  include  anything  from  written  text  to 
television  to  computer  programs. 

If  people  tend  to  anthropomorphize  media  such  as  computers  and  the  programs  that  run  on  them, 
there  may  be  real  advantages  to  implementing  pedagogical  agents  in  computer  interfaces.  First,  agents 
increase  the  bandwidth  of  communication  by  the  addition  of  a direct  conversational  partner,  a partner  who 
is  potentially  capable  of  showing  varied  emotional  states  and  patterns  of  deixis.  Second,  agents  may 
increase  the  computer's  ability  to  engage  learners  and  motivate  them.  Furthermore,  appropriate  lifelike 
behaviors  make  agents  appear  knowledgeable,  attentive  to  the  learner,  and  helpful  (Johnson  et  al.,  2000). 

Animated  pedagogical  agents  would  seem  to  provide  a challenge  to  multimedia  environments, 
given  the  precepts  of  the  cognitive  theory  of  multimedia  learning  (Moreno  & Mayer,  1 999).  If  the  agents 
are  integrated  as  a part  of  an  illustration  or  animation,  their  presence  could  cause  split-attention  effects 
(Sweller  & Chandler,  1994),  or  modality  effects  (Moreno  & Mayer,  1999).  Modality  effects  could  result 
from  both  the  agent  and  learning  materials  being  presented  in  the  visual  modality.  Such  effects  might  not 
be  overcome  entirely  by  the  integration  of  spoken  text  with  a picture  or  animation,  because  learners  might 
concentrate  on  the  agent  and  ignore  the  learning  materials.  What  might  be  required  is  to  direct  attention 
away  from  the  agent  who  provides  the  spoken  text  and  toward  the  appropriate  visual  materials,  for  example 
gesturing  by  the  agent  (Johnson  et  al.,  2000)  or  attention  capture  within  the  animation  itself  (Yantis  & 
Hillstrom,  1994).  There  is  currently  evidence  that  animated  pedagogical  agents  used  in  virtual  learning 
environments  can  promote  baseline  problem  solving  skills  (Johnson  et  al.,  2000;  Moreno,  2001;  Moreno, 
Mayer,  Spires,  & Lester,  2001).  However,  it  is  not  clear  what  role  agents  play  in  learning  environments 
(Andre  et  al.,  1999). 
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One  of  the  first  studies  (Lester,  Voerman  et  al.,  1997;  Lester,  Towns,  Fitzgerald,  1999)  of 
animated  agents  led  to  what  is  called  the  ’’Persona  effect."  The  claim  is  that  the  presence  of  a lifelike 
character  in  the  environment  has  a positive  impact  on  the  learner's  interactive  experience.  The  study  also 
revealed  that  more  expressive  agents  are  given  higher  ratings  on  clarity  and  utility  than  the  less  expressive. 
Herman  the  bug  was  the  agent  in  the  study.  Participants  gave  ratings  on  how  helpful  Herman  was  as  an  aid 
to  their  learning  experience.  However,  it  is  important  to  note  that  the  Persona  effect  is  silent  on  whether 
participants  learn  more  when  interacting  with  agents  (Lester,  Voerman  et  al.,  1997).  It  only  states  that 
learners  enjoy  the  experience  more. 

The  present  study  was  designed  to  investigate  issues  related  to  attention  by  manipulating  qgent 
properties  and  features  of  the  pictorial  information.  Two  possibilities  present  themselves  in  this  context. 
First,  consider  the  agent,  deictic  gesture  can  be  used  to  direct  the  learner’s  attention  while  integrating  the 
animated  agent  with  a picture  or  animation.  Pointing  and  gesturing  are  a natural  way  in  which  both  adults 
and  children  attempt  to  direct  attention  (Alibali  & DiRusso,  1999;  Krauss,  1998).  Gestures  should  occur 
simultaneously  or  prior  to  the  onset  of  the  speech  act  that  they  signify  (Momel,  & Krauss,  1992).  A second 
option  is  to  capture  attention  by  using  parts  of  a picture  or  animation  itself.  According  to  attention  research, 
an  excellent  way  capture  attention  is  by  an  abrupt  onset  and  motion  (Jonides  & Yantis,  1988;  Yantis  & 
Hillstrom,  1994). 

The  design  was  a 3 (agent  properties:  agent  only,  agent  with  gestures,  no  agent)  x 3 (picture  features: 
static  picture,  sudden  onset,  animation)  factorial.  If  the  use  of  an  agent  leads  to  split  attention  effects  and  if 
these  effects  are  reduced  by  agent  gesture,  then  we  would  expect  to  see  the  no  agent  condition  > the  agent 
condition  < agent  with  gesture  condition.  If  there  were  no  split  attention  effect,  then  no  differences  would 
be  seen  between  agent  properties.  The  cognitive  theory  of  multimedia  learning  would  predict  that  within 
the  picture  features  animation  > static  picture.  Also,  within  the  picture  features  sudden  onset  > static 
picture  if  attention  capture  is  sufficient  for  learning. 

Methods 

Participants 

Participants  in  this  experiment  were  135  students  drawn  from  an  undergraduate  psychology  students 
at  the  University  of  Memphis  who  volunteered  from  a pool  of  participants.  This  pool  consisted  of  all 
students  taking  either  of  two  levels  of  introductory  psychology  courses. 

Materials 

The  materials  for  the  experiment  were  of  two  kinds:  computerized  materials  and  pencil  and  paper. 
The  computerized  materials  consisted  of  the  visual  and  narrative  information  presentation  (training 
section).  The  pencil  and  paper  materials  consisted  of  a questionnaire  for  domain  knowledge,  a test  of 
spatial  ability,  a mental  load  rating  scale,  a persona  rating  scale,  multiple -choice  questions,  a matching  test, 
a retention  test,  and  transfer  tests  involving  creative  solutions. 

The  computerized  materials  were  created  using  three  different  computer  application  packages.  The 
agent  and  voice  were  created  using  the  Microsoft  Agent  software  package  (Microsoft,  1998).  The 
multimedia  animations  were  created  using  Macromedia  Flash  3.0  (Macromedia,  1998).  These  packages 
were  integrated  using  a Program  called  Xtrain  (Hu,  1998;  Hu  & Craig,  2000). 

Likert-type  scales  were  used  for  the  Persona  test  and  mental  load  rating.  The  persona  test  ranged 
from  1 to  6 with  1 being  extremely  enjoyable  and  6 being  extremely  not  enjoyable.  The  scale  was  similar 
to  those  that  were  described  in  previous  research  literature  (Johnson  et  al.,  2000;  Lester  et  al.,  1997).  The 
mental  load  rating  ranged  from  1 to  6 with  1 being  extremely  easy  and  6 being  extremely  difficult.  This 
subjective  rating  has  been  used  in  previous  research  as  a measure  of  the  cognitive  load  of  a task  (Kalyuga, 
Chandler,  & Sweller,  1999;  Paas  & Van  Merrienboer,  1993,  1994). 

Participants  also  received  two  pencil  and  paper  tests  at  the  outset  of  the  experiment  session.  Both  of 
these  tests  were  brief  and  were  given  prior  to  the  multimedia  information  presentation.  The  tests  were  for 
domain  knowledge  and  spatial  ability. 

The  test  of  domain  knowledge  was  a standard  screening  test  used  in  related  research  (Mayer  & 
Moreno,  1998;  Moreno  & Mayer,  1999).  This  questionnaire  consisted  of  a seven  item  activity  checklist 
containing  statements  concerning  weather  knowledge,  with  one  point  added  for  each  checked  item  and  a 
five  level-self  assessment  from  less  than  average  domain  knowledge  (1)  to  very  much  (5).  The  cut-off 
criterion  adopted  was  six  (out  of  a maximum  11)  in  order  to  be  consistent  with  related  research  (Mayer, 
1997;  Mayer  & Moreno,  1998;  Moreno  & Mayer,  1999).  The  test  of  spatial  ability  was  a standard  paper- 
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folding  task  with  scores  used  as  a covariate.  In  the  task,  participants  were  simply  given  twelve  minutes  to 
correctly  answer  as  many  questions  as  they  could  (Bennet,  Seashore,  & Wesman,  1972). 

The  information  presentation  was  concerned  with  the  process  of  lightning  firmation.  These 
materials  have  been  shown  to  be  effective  for  achieving  learning  gains  in  previous  research  (Mayer  & 
Moreno,  1998;  Moreno  & Mayer,  1999).  The  scenario  presented  followed  a causal  path  from  how  a storm 
front  forms  to  the  creation  and  display  of  lightning. 

The  three  remaining  tests  have  been  used  in  previous  research  (Mayer  & Moreno,  1998;  Moreno  & 
Mayer,  1999).  These  involved  retention,  matching,  and  transfer.  The  retention  test  consisted  of  one 
question,  "Please  write  down  an  explanation  of  how  lightning  works"  (See  Appendix  D for  example). 
These  tests  were  collected  after  five  minutes.  The  matching  test  consisted  of  four  frames  with  instructions 
that  ask  the  participants  to  circle  and  label  the  cool  moist  air,  warmer  surface,  updraft,  freezing  level, 
downdraft,  gusts  of  cool  wind,  stepped  leader,  and  the  return  stroke.  These  were  collected  after  three 
minutes.  The  test  for  creative  solutions  consists  of  four  questions  presented  one  at  a time  for  three  minutes 
each. 

In  addition  to  the  tests  used  by  Mayer,  participants  were  also  presented  with  a series  of  six  multiple- 
choice  questions  requiring  a forced  choice  among  four  possibilities,  one  correct  with  three  other  foils. 
These  six  questions  assessed  three  categories  of  knowledge  (explicit  shallow,  explicit  deep,  and  implicit 
deep).  The  explicit  shallow  questions  focused  on  the  shallow  surface  level  information  from  the 
presentation  (e.g..  The  upper  portion  of  the  cloud  is  made  up  of  what?).  The  explicit  deep  questions  focus 
on  understanding  of  the  concepts  what  were  presented  in  the  information  delivery  (e.g..  When  do 
downdrafts  occur?).  The  Implicit  deep  questions  focus  on  the  application  of  the  underlying  concepts  to 
problems  that  were  not  focused  on  in  the  information  delivery  (e.g..  Why  does  it  get  colder  right  before  it 
rains?). 

Procedure 

The  basic  procedure  was  as  follows.  When  the  participants  first  entered  the  laboratory,  they  were 
issued  a packet  of  materials.  This  packet  contained  their  informed  consent,  test  of  domain  knowledge,  and 
the  test  of  spatial  ability.  After  these  were  completed,  those  eligible  for  the  study  (those  scoring  under  half 
on  the  domain  knowledge  test)  received  instructions  for  part  two  of  the  experiment.  They  were  presented 
with  the  information  delivery,  which  took  about  three  minutes.  Afterward,  they  were  given  the  retention 
question  (5  minutes),  the  Multiple -choice  questions  (2  minutes),  the  matching  test  (3  minutes),  and  4 
transfer  questions  (3  minutes  each). 

Results  and  Discussion 

Persona  effect 

A 3 (agent  properties:  no  agent,  agent,  agent  with  gesture)  x 3 (picture  features:  picture,  onset, 
motion)  ANOVA  was  performed  on  the  persona  data.  There  was  no  evidence  for  a persona  effect  in  this 
study.  There  could  be  several  explanations  for  this.  Participants  were  only  exposed  to  one  condition  and 
thus,  they  had  nothing  to  make  a comparison  with.  If  this  is  the  case,  a within  subject  design  would  be  an 
accurate  measure.  Also,  the  agent  was  displayed  for  only  180  seconds  and  that  might  not  have  been  long 
enough  to  produce  an  effect.  Since  the  persona  effect  is  based  on  the  agent  making  the  learning  experience 
more  enjoyable  (Johnson  et  al.,  2000),  an  agent  interaction  within  the  learning  environment  for  180  seconds 
was  probably  not  enough  to  produce  an  effect. 

Even  though  the  persona  effect  (Andre  et  al.,  1999;  Lester,  Convers  et  al.,  1997)  was  not 
significant,  a trend  can  be  seen  in  the  data.  In  the  scale,  a lower  score  indicated  a more  positive  rating.  The 
means  ratings  for  the  three  groups  were  M = 3.44  for  no  agent  present,  M = 3.42  for  agent  present,  and  M = 
3.07  for  agent  with  gestures  with  lower  numbers  indicating  a more  positive  rating.  A Cohen’s  f effect  size 
was  calculated  for  the  persona  data.  These  analyses  yielded  effect  size  score  of  .42  in  the  agent-with- 
gestures  vs.  no-agent  comparison,  and  a score  of  .02  in  the  agent-only  vs.  no-agent  comparison.  This 
shows  a supportive  trend  for  the  persona  effect. 

Cognitive  Load 

A 3 (agent  properties:  no  agent,  agent,  agent  with  gesture)  x 3 (picture  features:  picture,  onset, 
motion)  ANOVA  was  performed  on  the  cognitive  load  data  (Paas  & Merrienboer,  1993).  It  yielded  a 
significant  effect  between  the  picture  features,  (F  (2,126)  = 3.737,  p < .05).  A post  hoc  test  performed  on 
the  three  picture  features  groups  yielded  significant  differences  in  perceived  comprehension  ratings 
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between  the  picture  condition  (M  = 3.62)  and  the  motion  condition  (M  = 3.09),  ^ < .05).  This  finding  is 
in  line  the  cognitive  theory  of  multimedia  learning.  It  predicts  a decrease  in  the  difficulty  of 
comprehension  for  the  motion  condition  over  the  picture  condition  when  synchronization  of  the  display  is 
attained  that  ensures  temporal  and  spatial  contiguity  (Moreno  & Mayer  1999). 

Matching 

A 3 (agent  properties:  no  agent,  agent,  agent  with  gesture)  x 3 (picture  features:  picture,  onset, 
motion)  ANOVA  was  performed  on  the  matching  task  data.  It  yielded  a significant  effect  only  for  picture 
features,  F(2,133)  = 12.434,  £ < .001.  Tukey  contrasts  revealed  that  both  onset  (M  = 4.42)  and  motion  (M 
= 4.56)  conditions  performed  significantly  better  than  the  picture  condition  (M  = 3.13,  £ < .001),  but  that 
the  onset  and  motion  conditions  do  not  differ  from  each  other. 

Although  these  results  differ  from  previous  results  fliat  did  not  find  differences  in  the  matching 
data  (Mayer,  1997;  Moreno  & Mayer,  1999),  they  support  the  cognitive  theory  of  multimedia  learning. 
According  to  this  theory,  in  order  for  successful  learning  to  occur,  there  must  be  an  integration  of  the 
verbally  based  and  the  visually  based  models  of  the  material  (Mayer,  1984).  This  integration  may  be 
enhanced  by  the  use  of  animations  over  pictures  (Moreno  & Mayer,  1999).  Both  the  motion  and  the  onset 
conditions  provided  the  verbal  and  visual  integration  reeded  for  matching  in  the  present  study.  The 
difference  was  possibly  due  to  the  decreased  presentation  time  that  prevented  the  ceiling  effects  found  in 
previous  research  by  Moreno  and  Mayer  (1999). 

Retention 

A 3 (agent  properties:  no  agent,  agent,  agent  with  gesture)  x 3 (picture  features:  picture,  onset, 
motion)  ANOVA  was  performed  on  the  retention  data.  It  yielded  a significant  effect  only  for  Picture 
features,  F(2,  133)  = 25.73,  £ < .001.  Tukey  contrasts  yielded  a difference  between  the  picture  condition 
(M  = 1.93)  and  both  the  onset  (M  = 4.20)  and  motion  ^ = 5.07)  conditions.  There  was  no  difference 
between  the  latter  two  groups. 

Transfer 

The  transfer  task  probed  the  extent  to  which  participants  applied  the  concepts  they  learned  to  other 
problems  and  exhibited  creative  solutions.  A 3 (agent  properties:  no  agent,  agent,  agent  with  gesture)  x 3 
(picture  features:  picture,  onset,  motion)  ANOVA  was  performed  on  the  transfer  task  data.  This  analysis 
yielded  a significant  effect  of  picture  features  only,  F(2,133)  = 4.03,  £ < .05.  Tukey  contrasts  yielded  a 
difference  between  the  onset  (M  = 2. 13)  and  picture  (M  = 1 .44)  conditions  (^  < .05).  The  motion  condition 
(M  = 1 .89)  was  intermediate  and  did  not  differ  from  either  of  the  other  two  groups. 

The  differences  between  the  onset  and  picture  conditions  provides  support  for  claim  the  that  a sudden  onset 
of  a color  singleton  directs  attention  as  needed  to  assist  the  construction  of  mental  models  that  facilitate  the 
implicit  inferences  necessary  to  construct  creative  solutions. 

Multiple-Choice  Questions 

A 3 (agent  properties:  no  agent,  agent,  agent  with  gesture)  x 3 (picture  features:  picture,  onset, 
motion)  ANOVA  was  performed  on  the  total  score  obtained  from  the  multiple -choice  questions.  This 
yielded  a significant  difference  only  between  picture  features,  F(2,133)  = 7.58,  £<  .001.  Both  the  onset  (M 
= 2.13)  and  motion  (M  = 1.89)  groups  significantly  outperformed  the  picture  (M  = 1.44)  group  (^  < .001). 
There  was  no  difference  between  onset  and  motion  conditions. 

Explicit  Shallow.  The  multiple-choice  questions  attempted  to  evaluate  three  types  of  knowledge 
(explicit  shallow,  explicit  deep,  and  implicit  deep).  A 3 (agent  properties:  no  agent,  agent,  agent  with 
gesture)  x 3 (picture  features:  picture,  onset,  motion)  ANOVA  performed  on  the  data  from  the  questions 
that  tapped  explicit  shallow  knowledge  yielded  only  a significant  effect  of  picture  features,  F(2,  133)  = 
3.08,  £ < .05.  Tukey  contrasts  showed  that  participants  in  the  motion  condition  (M  = 1.64)  outperformed 
the  picture  condition  (M  = 1.36),  .05).  This  result  is  consistent  with  findings  from  the  matching  task, 

which  tested  for  shallow  knowledge. 

Explicit  Deep.  A 3 (agent  properties:  no  agent,  agent,  agent  with  gesture)  x 3 (picture  features: 
picture,  onset,  motion)  ANOVA  was  performed  on  the  data  for  the  explicit  deep  questions.  It  yielded  only  a 
significant  effect  for  the  picture  features,  F(2,133)  = 7.93,  p<.001.  Tukey  contrasts  revealed  differences 
between  both  the  onset  condition  (M  = -71)  and  the  motion  condition  ^ = .79)  when  compared  to  the 
picture  condition  (M  = .29).  The  participants  in  the  motion  condition  outperformed  those  in  the  picture 
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condition  fe  < .001).  Similarly,  those  in  the  onset  condition  outperformed  participants  in  the  picture 
condition  (p  < .01).  These  findings  suggest  that  by  directing  attention  appropriately  the  onset  of  the  color 
singleton  (Yantis  & Hillstrom,  1994)  facilitated  deeper  learning  of  core  concepts  as  effectively  as  an 
animation  with  motion  in  the  present  relatively  brief  presentation.  This  would  seem  to  indicate  that  the 
onset  provided  the  temporal  contiguity  that,  according  to  Moreno  and  Mayer  (1999),  was  required  to  get 
full  integration  of  the  verbal  and  visual  representations. 

Implicit  Deep.  The  final  multiple -choice  questions  were  designed  to  tap  implicit  deep  knowledge. 
A 3 (agent  properties:  no  agent,  agent,  agent  with  gesture)  x 3 (picture  features:  picture,  onset,  motion) 
ANOVA  that  was  performed  on  the  data  yielded  no  significant  effects,  but  picture  features  was  marginally 
significant,  F(2,133)  = 2.738,  p = .068.  Furthermore,  the  means  (picture  M = .96,  onset  M = 1-20,  motion 
M = .91)  were  in  a somewhat  similar  direction  as  the  transfer  data,  which  attempted  to  tap  the  same  pool  of 
knowledge. 

The  means  and  standard  deviations  for  all  conditions  and  measures  are  presented  in  Table  1 below. 


Persona 

Test 

Matching 

Test 

Retention 

Question 

Transfer 

Questions 

Multiple 

Choice 

(Total) 

Explicit 

Shallow 

Explicit 

Deep 

Implicit 

Deep 

Agent  Properties 

No  agent 

3.44 

4.04 

3.91 

1.69 

3.40 

1.56 

0.67 

1.18 

Agent  Only 

3.42 

4.04 

3.80 

1.76 

2.96 

1.53 

0.49 

1.00 

Agent  w/ 
Gesture 

3.07 

4.02 

3.49 

2.02 

3.02 

1.52 

0.59 

0.89 

Picture  features 

Picture 

3.62 

3.13 

1.93 

1.44 

2.60 

1.36 

0.29 

0.96 

Onset 

3.24 

4.42 

4.20 

2.13 

3.47 

1.56 

0.71 

1.20 

Motion 

3.09 

4.56 

5.07 

1.89 

3.31 

1.64 

0.79 

0.91 

Table  1.  Table  of  Means  and  standard  deviations 


Summary  and  conclusions 

The  study  revealed  several  findings.  First,  there  were  no  differences  due  to  agent  properties  and, 
thus,  there  was  no  evidence  of  split-attention  effects.  The  presence  of  the  agent  in  the  learning  environment 
was  not  detrimental  to  learning.  It  appears  that  agents  can  be  safely  integrated  into  brief  multimedia 
presentations  without  fear  of  interference,  as  proposed  by  Johnson  et  al  (2000).  Second,  most  of  the 
analyses  revealed  differences  among  the  image-type  conditions  that  supported  the  cognitive  theory  of 
multimedia  learning  (Moreno  & Mayer,  1999).  Close  temporal  synchronization  of  the  narration  with  the 
animated  display  enhanced  learning,  presumably  by  establishing  relationships  between  the  visual  and 
verbal  representations.  Third,  the  findings  suggest  stimulus  onset  is  just  as  effective  and  in  some  cases 
more  effective  in  directing  attention  to  appropriate  parts  of  the  display  as  motion  in  a fully  animated 
display. 
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Abstract 

A common  "mistake"  made  by  lecturers  teaching  online  is  to  do  little  more  than  put  their  notes  online, 
and  to  support  it  with  some  sort  of  electronic  mailing  list.  Many  online  learning  environments  such  as 
WebCT  and  Blackboard  are  also  designed  to  support  this  instructional  model.  This  paper  discusses 
alternative  activities  that  have  been  used  to  put  the  ACTION  into  INTERACTION  in  an  online 
teaching  and  learning  environment.  Activities  discussed  include  virtual  discussions,  building  virtual 
desks,  and  even  a virtual  student  rag  procession.  The  paper  also  shows  which  message  acts  are  most 
likely  to  be  engaged  in  by  instructors,  and  what  message  acts  are  carried  out  by  learners. 

Introduction 

The  word  “virtual”  that  has  become  associated  with  what  is  done  over  the  Internet,  always  implies 
comparison  “virtual  reality”  is  like  reality,  “virtual  teaching”  is  like  actual  teaching.  An  implied 
comparison  of  items  is  known  as  a metaphor  (Aristotle,  s,a,).  Much  of  the  World  Wide  Web  takes  the 
form  of  a metaphorical  book  with  “web  pages”.  Teaching  on  the  web  occurs  in  metaphorical  “virtual 
classrooms”.  An  extended  metaphor  is  a model.  The  traditional  contact  lecture  is  the  model  for 
activities  in  a metaphorical  classroom,  by  way  of  text -based  lecture  notes  and  ennail  and  chat  room- 
based  questions  and  answers.  Much  of  this  unfortunately  lacks  creativity.  The  same  boring  stuff 
produced  as  lectures  become  boring  web  pages. 

This  paper  explores  the  dynamics  between  the  design  of  internet-based  instruction  and  the  response  of 
learners  to  the  activities  that  they  are  asked  to  carry  out.  Two  questions  are  addressed:  (1)  To  what 
extent  can  activity -based  learning  be  conducted  over  the  Internet;  and  (2)  What  message  acts  do 
lecturers  and  learners  execute  over  the  Internet? 

Context 

This  on-going  research  has  been  taking  place  in  a two-year  tutored  masters'  degree  in  computer-based 
education  at  the  University  of  Pretoria,  Students  range  from  23  to  55,  and  all  work  full  time.  Reactions 
of  both  the  enrolled  students  and  alumni  are  reported.  Teaching  is  constructivist  rather  than 
instructivist.  Students  meet  once  a month  for  three  days  and  keep  in  contact  via  the  Internet.  One 
module,  “Teaching  on  the  Internet”  is  presented  entirely  over  the  Net, 
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To  what  extent  can  activity-based  learningbe  conducted  over  the 
Internet? 

Russel,  (s.a.)  shows  that  the  medium  of  instruction  causes  no  significant  difference  in  student 
performance.  Similarly,  Clark  (1994)  advocates  the  separation  of  medium  and  methodology  in  research 
on  educational  technology,  and  feels  that  it  is  the  method,  rather  than  the  medium  that  influences 
learning.  Kozma  (1994)  on  the  other  hand,  argues  that  certain  media  attributes  demand  certain 
methods.  Mabry  (1997)  describes  models  for  "providing  communication  resource  opportunities  similar 
to  those  expected  from  face-to-face  interaction". 

Conversation 

Email  bulletin  boards  and  chat  rooms  are  the  equivalent  of  classroom  conversation,  providing  a " 'talk' 
metaphor,  which  compares  on-line  interaction  to  face-to-face  conversation"  (Baym,  1995).  Moreover 
"An  informal,  everyday  quality  is  created  through  the  use  of  smileys,  non-standard  spelling  reflective 
of  vernacular  pronunciation,  punctuation  to  indicate  pauses  rather  than  speech  clauses,  special  symbols 
borrowed  from  programming  languages  and  an  extensive  special  vocabulary"  (Marvin,  1995). 

Collaborative  and  co-operative  learning 

According  to  Carton  (1997)  "When  a computer  network  connects  people  or  organizations,  it  is  a social 
network". Although  Anderson  and  Kanuka  (1997)  point  out  that  an  online  forum  appears  to  hold  little 
or  no  relative  advantage  compared  to  face-to-face  forums  with  regard  to  socialization  and  ability  to 
communicate,  there  are  other  advantages.  Employees  "would  have  minimal  work  time  loss  and  no 
travel  time  loss".  For  Johnson  and  Johnson  (1991)  successful  co-operative  learning  has  three 
prerequisites,  a mutual  goal,  individual  responsibility  and  positive  interdependence.  From  the  literature 
then  it  would  seem  that  the  methods  of  contact  teaching  could  be  replicated  since  medium  and  method 
should  be  separated;  but  different  media  may  be  necessary  to  simulate  different  aspects. 

What  message  acts  do  lecturers  and  learners  do  over  the  Internet? 

Learners  are  likely  to  adapt  their  behaviour  based  on  the  that  they  are  given  because  of  material 
behaviour:  "The  impact  of  the  relationship  between  material  and  interpersonal  behavior  is  termed 
material  behavior"  (Jones,  1997),  because  "material  acts  as  a non-verbal  message -system  with  rules  that 
provide  some  degree  of  predictability  which  manages  human  interaction".  Bonnycastle  (1997)  sees  a 
duality  between  our  signs  and  ourselves  "Depending  on  a variety  of  factors,  we  choose  forms  of 
representation  that  are  both  individually  and  culturally  determined  to  facilitate  that  commmication.  In 
turn,  that  painting,  book,  idea  become  part  of  the  environment  and  the  cycle  begins  again". 

Teaching  and  learning  in  a metaphorical  classroom 

The  digital  classroom  in  this  article  used  the  metaphor  of  a junior  school  classroom  to  teach  "Internet- 
based  learning"  to  adult  learners.  For  Branscomb  (1996)  "Metaphors  serve  as  a map  for  sorting  out 
what  is  similar  and  what  is  different  when  confronting  a new  problem".  1 wanted  to  see  how  closely  1 
could  mimic  the  events  of  a real,  physical  classroom  in  a web-based  environment.  Because  of  this  the 
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visual  similarity  in  my  initial  digital  classroom  had  to  be  as  strong  as  possible  since  the  physical 
differences  were  enormous.  The  students  never  met  face -to« face,  and  neither  were  they  able  to 
communicate  face-to-face. 

The  classroom  had  two  components:  A website  (representing  the  “physicar  classroom)  and  an 
electronic  mailing  list  that  represented  interpersonal  (speaking)  interaction  that  occurs  in  a classroom. 
Jones  (1997)  distinguishes  between  ”a  virtual  community's  cyber-place  and  the  virtual  community 
itself.  A virtual  community's  cyber-place  will  be  termed  a virtual  settlement".  The  classroom  with  its 
associated  mailing  list  therefore  formed  the  virtual  settlement,  and  had  the  following  characteris  tics  as 
identified  by  Jones  (1997):  "Minimum  Level  of  Interactivity,  variety  of  communicators,  a common- 
public -space  where  a significant  portion  of  a community's  interactive  group-cmc  occurs;  and  a 
minimum  level  of  sustained  membership"  (Jones,  1997). 

The  website  "classroom"  can  be  viewed  at  <http://hagarMp,ac,za/rbo/classrm,html>,  It  had  four 
sections.  First  there  was  the  blackboard  and  notice  boards.  Clicking  on  these  revealed  timetables  and 
tasks;  the  blackboard  revealed  comprehensive  on-line  study  guides,  as  suggested  by  Van  Brakel  (1996). 
The  chalkboard  was  a graphic  that  students  could  modify,  producing  graffiti  as  in  a real  classroom. 
Second  was  the  administrative  area  with  the  presenter's  desk  (linking  to  my  own  home  page),  resource 
cupboard  (with  links  to  subject  matter  and  construction  programs  such  as  graphic  and  HTML  editors) 
and  alumni  association  (with  links  of  a general  nature,  regarding  computer-assisted  education). 

Third  was  the  poster  wall  with  links  to  projects  by  previous  students,  and  links  to  the  "posters"  created 
by  current  students  co-operatively  as  the  course  progressed. 

Lastly  came  the  learners'  desks.  Each  student  had  a WWW  directory  linked  to  a desk.  The  student  had 
to  replace  the  generic  picture  of  a desk  with  a personalized  one,  and  then  "fill  the  desk"  with  things 
typically  found  in  a school  desk:  (1)  Your  ears  (Mail  to:  ...),  (2)  Your  utility  bag  (Links  to  handy 
utilities  such  as  HTML  editors.  Search  Engines,  Clipart  libraries,  etc.),  (3)  Your  textbooks  (Links  to 
useful  sites),  (4)Your  work  (Interesting  work  you  have  done  in  other  M.Ed.  modules),  (5)  Your  hobbies 
(Links  to  sites  of  special  interest  to  you),  (6)  Your  class  work  (Your  answers  to  all  the  objectives  of  the 
course);  and  (7)Your  portfolio  (A  link  to  the  portfolio  of  your  examination  project). 

In  this  classroom  learners  did  not  only  find  information,  but  generated  their  own  web  pages.  The 
students  immediately  took  on  the  roles  of  learners  in  a schoolroom.  They  addressed  me  as  "Sir"  or 
"Teacher"  in  all  their  emails , tattled  on  one  another,  interrupted  proceedings  with  jokes,  and  generally 
acted  with  an  air  of  light  heartedness.  One  student,  Mari,  went  as  far  as  "bunking"  class.  Instead  of 
sticking  to  the  "desk"  metaphor  that  she  had  to  create,  she  replaced  it  with  a beach  metaphor.  She  did 
not  have  a desk.  She  had  a surf  kit  instead.  Ari,  did  not  show  a picture  of  his  desk,  but  of  his  two  feet 
resting  on  it.  These  results  show  how  the  adult  learners,  when  placed  inside  the  metaphor  of  the 
schoolroom,  exercised  typical  schoolroom  “naughtiness".  The  other  desks  show  and  interesting  blend 
of  students*  creativity  or  lack  of  it  - much  the  same  as  one  would  find  in  a real  classroom. 

As  a special  challenge  students  were  allowed  to  add  graffiti  to  the  chalkboard.  This  was  a reasonably 
complicated  task  but  they  found  the  environment  sufficiently  motivating,  and  by  the  second  week  of 
the  course,  all  had  written  on  the  board. 
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We  were  able  to  reproduce: 

• Posters:  i.e.  vignettes  of  information,  examples  of  similar  projects,  and  examples  of  the  work 
of  other  students. 

• Resource  cupboards  and  toolboxes:  i.e.  repositories  for  additional  reference  information  and 
imaging  and  website  construction  tools. 

® Presence  of  other  learners  in  the  form  of  "desks”,  and 

• Conversation  by  means  of  a dedicated  mailing  list. 

• What  we  could  not  create  at  that  stage  was  real-time  interaction,  and  that  made  us  miss  the 
element  of  urgency  thus  created. 

Humor  figured  strongly  in  the  class,  supporting  Baym's  (1995)  contention  that  "...humorous 
performance  can  be  used  to  create  group  solidarity,  group  identity,  and  individual  identity  in  CMC". 
An  analysis  of  the  messages  indicates  that  the  most  frequent  actions  of  the  instructor  were  to  (1)  make 
suggestions,  (2)  give  encouragement,  (3)  explain  how  to  do  something;  and  (4)  give  directives.  The 
main  actions  of  the  learners  were  to  (a)  give  information,  (b)  ask  questions,  (c)  present  a problem,  (d) 
initiate  or  contribute  to  the  discussion;  and  (e)  express  amusement  or  joke.  The  message  acts  in  the 
classroom  were  very  similar  to  speech  acts  in  a face-to-face  classroom,  showing  that  "Computer- 
mediated  groups  can  create  solidarity  through  developing  interpretive  consensus"  (Baym,  1995). 

A virtual  rag  procession 

Building  on  the  success  with  humour  in  the  classroom  I wanted  to  extend  the  virtual  community  to 
masters'  students  in  the  first  and  second  and  course  alumni.  According  to  Baym,  (1995)"analysis  of 
humor  is  important  because  CMC  research  has  been  slow  to  address  the  formation  of  group  identity 
and  solidarity,  though  such  phenomena  occur  in  on-line  groups  and  are  negotiated,  in  part,  through 
humor".  The  model  I was  the  University  rag,  a carnival,  culminating  in  parade  through  Pretoria. 
Although  graduate  students  never  take  part,  my  rag  was  exclusive  to  my  graduate  students  and  alumni 
The  url  is  <http://hagarMpMC.za/catts/jool/jool.html>.  Unlike  my  students'  other  work  this  was  not  for 
marks,  but  the  model  was  a competition.  The  first  prize  Microsoft  Front  Page  sponsored  by 
Microsoft  South  Africa.  It  was  not  aimed  at  building  new  skills  but  at  community  building.  Kozar 
( 1 995)  says  that  "Not  only  does  this  kind  of  communication  allow  people  to  renew  and  strengthen 
friendships,  but  it  also  provides  them  with  an  interface  for  play,  and  through  play  for  the  creation  of 
new  patterns  for  the  performance  of  tradition". 

When  the  students  recognised  the  metaphor,  they  were  able  to  expand  it.  They  found  corporate 
sponsors  whose  pages  were  linked  to  the  'floats’  and  credit  was  given  as  would  have  been  done  on  a 
physical  float.  Students  asserted  their  right  to  express  their  own  opinions.  Although  the  theme  'Coin 
the  barrel'  was  intended  as  and  anti-gun -ownership  campaign,  Dolf,  sponsored  by  Magnum  gun- 
collectors  magazine,  constructed  a float  giving  "Good  reasons  why  you  should  spend  coins  on  barrels". 
This  supports  Baym's  (1995)  contention  that  "Humor  also  serves  as  a means  of  creating  individuality 
when  the  cues  we  use  to  define  ourselves  in  face-to-face  groups  are  unavailable". 

Mignon,  took  revenge  on  the  sexist  idea  of  having  a "Rag  Queen"  beauty  pageant,  and  constructed  her 
own  "Rag  Man"  float.  She  invited  her  fellow  students  to  enter  her  competition  and  put  their  pictures  on 
her  site  as  additional  floats.  In  this  way  she  demonstrated  that  "humor  is  a joint  production,  which  the 
audience  not  only  understands  but  helps  create.  This  joint  authorship  enhances  group  identity  and 


solidarity”  (Baym,  1995).  She  put  my  picture  at  the  top  of  the  float-  an  act  reminiscent  of  the 
manipulation  of  teachers’  pictures  so  prevalent  inschools.  The  school  metaphor  created  in  the  initial 
classroom,  therefore,  had  survived  even  beyond  the  duration  of  the  module. 

Mari  crept  into  the  hearts  of  the  virtual  community  with  a very  touching  float,  depicting  the  first  sonar 
scan  of  her  unborn  baby,  together  with  a poem  explaining  why,  for  the  sake  of  the  unborn  child,  we 
should  'Coin  the  barrel'.  She  demonstrated  that  "it  is  in  part  through  humorous  performance  that 
particular  posters  overcome  the  seeming  anonymity  of  the  computer  medium  to  develop  their  own 
voices”  (Baym,  1 995). 

Conclusion 

This  section  will  briefly  state  the  conclusions  reached  and  lessons  learnt  in  answer  to  the  two  original 
questions. 

To  iv/iaf  extent  can  activity-based  learning  be  conducted  over  the 
Internet? 

The  physical  attributes  of  an  actual  classroom,  i.e.  chalkboard,  resource  cupboard,  teacher’s 
workstation,  learners'  workstations,  posters  and  portfolios  can  easily  be  metaphorically  recreated  in 
cyberspace.  Learners'  reactions  to  these  creations  are  similar  to  those  of  learners  in  a physical 
environment.  Group  cohesion  can  be  created  through  humorous  interchange,  large  amounts  of 
information  can  be  sent  across  by  e-mail  based  "virtual  lectures".  Student  and  teacher  activities  in 
metaphorical  classrooms  correspond  closely  with  their  activities  in  face-to-face  classrooms. 

What  message  acts  do  lecturers  and  learners  do  over  the  Internet? 

Learner  response  to  the  metaphor  was  to  "down  age"  and  engage  in  the  role-play  of  being  in  a school 
classroom.  This  spontaneous  role-play  activity  assisted  greatly  developing  a team  spirit  in  the  class,  but 
while  they  engaged  in  the  role  -play  exercises,  nevertheless  remained  adult  in  their  decision-making. 
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Critical  evaluation  skills  are  central  to  education  as  reliance  on  Web-based  information  increases. 
This  is  particularly  true  now  that  the  editorial  and  peer  review  processes  are  implicitly  removed  from 
experts  and  textbook  authors  and  placed  into  the  hands  of  students  searching  the  Web  for  material  related  to 
research  projects.  Since  it  is  increasingly  difficult  for  Web  sites  to  attract  readers’  attention,  the  Web  site 
designers  increasingly  resort  to  exaggerations,  encouraged  by  the  anonymity  afforded  by  this  technology. 
In  the  sciences,  for  example,  students  must  sift  through  false  or  exaggerated  claims,  non-refereed  research, 
and  various  forms  of  misinformation.  These  critical  evaluation  skills  are  no  less  important  in  other  fields 
where  students  cite  non-refereed  research  or  quote  papers  posted  on  the  Web.  In  response  to  this  need,  a 
number  of  authors  have  posted  criteria  for  evaluating  Web  sites,  although  little  research  has  been  done 
regarding  this  process.  Furthermore,  teachers  frequently  assume  it  was  the  responsibility  of  other  teachers 
(e.g.,  the  English  teacher)  to  have  covered  this  topic  ^ding,  Landsman,  & Nguyen,  2001).  The  strong  need 
for  both  research  and  the  development  of  effective  instructional  interventions  in  this  area  provides  the 
impetus  for  this  paper,  which  examines  aspects  of  Web-site  evaluation  among  high  school  biology  students, 
university-level  pre-service  teachers  and  computer  science  students.  We  represent  the  perspectives  of 
faculty  working  in  different  disciplines . By  addressing  these  topics  in  an  interdisciplinary  forum,  we  hope 
to  engage  educators  and  res  earchers  from  other  disciplines  in  constructive  dialogue  in  which  we  co-develop 
instructional  goals  and  research  agendas  to  increase  the  critical  evaluation  skills  of  ourselves  and  the 
teachers  and  students  with  whom  we  work. 

The  emergence  of  powerful  Web  searching  tools  enables  K-12  teachers  and  their  students  to  locate 
a plethora  of  potential  resources  for  teaching  and  learning,  but  the  operative  term  is  “potential”,  especially 
when  considering  possibilities  for  these  resources  in  teaching  and  learning  environments.  As  an  initial  step 
in  addressing  these  needs,  we  first  discuss  research  relevant  to  teaching  critical  evaluation  skills,  first  by 
examining  the  credibility  ratings  of  information  sources  by  pre-service  teachers,  and  secondly,  by 
describing  an  instructional  intervention  carried  out  with  high  school  biology  students  who  were  taight  to 
evaluate  Web-sites.  For  example,  Klemm,  Iding  & Speitel  (2001)  compared  pre-ser\dce  teachers’  and 
scientists’  evaluations  of  the  credibility  of  31  information  sources  for  science  information.  They  found 
some  noteworthy  discrepancies.  For  example,  elementary  pre-service  teachers  rated  material  from  weekly 
investigative  television  newsmagazines  like  20/20  as  among  the  highest  in  credibility,  while  scientists  rated 
them  among  the  lowest.  In  general,  pre-service  teachers  tended  to  rate  information  sources  as  being  more 
credible  than  did  scientists.  These  discrepancies  clearly  demonstrate  the  need  for  critical  evaluation  skills 
among  the  general  public  and  educators  in  particular.  In  another  study,  Iding,  Landsman,  & Nguyen  (2001) 
carried  out  an  instructional  intervention  with  high  school  biology  students,  rather  than  teachers.  Students 
received  instruction  in  critical  Web-site  evaluation  and  developed  their  own  criteria  for  evaluating  science 
Web-sites  generally  and  the  science  information  contained  in  them,  more  specifically.  Results  indicated 
that  students’  lists  of  criteria  were  more  extensive  after  instruction.  They  reported  that  they  would  devote 
more  time  to  evaluating  Web-material  in  the  future  and  that  their  confidence  in  their  ability  to  evaluate 
Web-based  scientific  information  had  increased 

Next,  wc  will  mention  instructional  and  research-related  issues  that  emerge  from  these  studies  and 
other  work  (Lynn,  Bell,  & Hsi,  1998).  For  example,  what  factors  should  be  provided  in  order  to  facilitate 
the  development  of  critical  evaluation  skills?  Suggestions  include  providing  an  orientation  to  some  basic 
categories  for  their  evaluation  (e.g.,  validity,  credibility),  or  providing  computer-based  scaffolding  or 
tutorials,  working  collaboratively  to  discuss  and/or  create  actual  criteria  for  evaluating  Web  sites,  and 
finally,  gaining  sufficient  skills  evaluating  real  Web  sites,  both  individually  and  collaboratively.  Other 
basic  research  questions  include  the  need  to  elucidate  the  professional  needs  of  teachers  and  the  adequacy 
of  tools  for  preparing  and  supporting  teachers  in  selecting  and  teaching  about  selecting  valid,  credible,  and 
useful  Web-based  resources.  Secondly,  in  addition  to  addressing  the  needs  of  the  teachers  and  students, 
research  needs  to  address  design  and  credibility  aspects  of  Web-sites  and  how  these  factors  affect  teachers’ 
and  students’  judgments  about  information  and  information  sources.  For  example,  questions  driving  this 
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research  could  include  the  following:  Do  design  aspects  outweigh  actual  information  content  in  influencing 
credibility  judgments?  What  aspects  of  Web-based  information  affect  users’  judgments?  How  confident 
are  students  and  teachers  about  their  ability  to  make  judgments  about  information  in  different  content 
areas?  In  the  next  section,  we  describe  some  of  the  issues  around  Web-based  information  and  Web -site 
evaluation  that  are  pervasive  among  university-level  computer  science  students. 

Computer  Science  faculty  find  their  students  struggling  with  information  on  the  Web.  Students  in 
courses  spanning  lower -division  general  education  critical  thinking  to  graduate  research  technique  are 
frustrated  and  mislead.  Students  in  critical  thinking  courses  can  be  overwhelmed  with  the  abstract  notions 
of  logic  and  argument,  as  well  as  the  detailed  activities  of  gleaning  Web  material  and  thinking  logically 
about  it.  We  believe  that  teaching  critical  thinking  in  the  context  of  introductory  computer  science  courses 
complements  “information  competence”  training  provided  by  many  university  libraries.  For  example.  The 
California  State  University  Information  Literacy  Fact  Sheet  describes  several  ways  to  incorporate 
information  competency  into  university  curriculums:  freshman  orientation  courses,  general  education 
courses,  cornerstone  classes  in  major  areas,  and  competency -based  mastery.  We  also  observe  that  graduate 
students  are  unable  to  determine  the  trustworthiness  of  materials  found  on  the  Web.  We  speculate  that 
although  the  students  have  an  abstract  understanding  of  peer  review  and  the  scientific  publication  cycle,  the 
students  miss  cues  to  correctly  determine  the  trustworthiness  of  researchers’  Web  pages,  technical  reports, 
industry  publications,  and  peer-reviewed  on-line  conference  proceedings,  magazines,  and  journals. 

We  also  believe  that  recent  research  on  Web  usage  provides  insights  into  teaching  information 
competence.  For  example,  Morrison,  Pirolli,  and  Card  (2001)  surveyed  thousands  of  Web  users  about  why 
they  use  the  Web  ^urposi),  how  they  use  the  Web  Method)  and  what  they  are  looking  for  (content). 
Purpose  was  broken  dawn  into  “finding  information”  (25  percent),  “compare/choose”  (51  percent), 
“understand”  (24  percent).  Users’  methods  were  “explore”  (two  percent),  “monitor”  (two  percent),  goal 
directed  “find”  (25  percent),  and  goal  directed  “collect”  (71  percent).  Note  that  96  percent  of  user  activities 
were  goal -directed.  Nielsen  (2001)  summarizes  this  research:  “Users’  most  important  Web  tasks  involve 
collecting  and  comparing  multiple  pieces  of  information,  usually  so  they  can  make  a choice.”  That  is,  Web 
users  generally  do  tasks  similar  to  what  students  are  asked  to  do  in  the  classroom.  We  speculate  that  this 
general  usability  research  may  provide  insights  into  teaching  skills  for  critical  Web  use.  Future  directions 
for  research  include  collecting  more  data  of  how  students  use  material  obtained  from  the  Web  and 
analyzing  the  information  derived  from  studies  both  qualitatively  and  quantitatively.  However,  many 
questions  still  remain  on  how  and  what  to  measure  in  order  to  collect  meaningful  informatioa  Students 
must  be  trained  to  evaluate  Web-based  information  critically  and  we  want  to  develop  software  tools  or 
metrics  that  can  facilitate  this  training.  We  also  must  have  a better  understanding  of  how  humans  decide  if 
information  is  valid.  In  order  for  this  to  happen,  however,  we  need  to  discover  what  dimensions  form  a 
basis  for  value  judgments  of  veracity.  We  expect  that  results  from  this  research  will  be  able  to  provide 
some  insights  into  students'  beliefs  and  attitudes  about  the  veracity  of  information  found  on  the  Web.  In 
addition,  we  hope  this  research  can  address  how  those  beliefs  and  attitudes  are  related  to  task  performance. 

’Lies,  damned  lies  and  Web-based  information  is  a play  on  the  famous  quote  “lies,  damned  lies  and  statistics”  that  is 
typically  attributed  to  Benjamin  Disraeli.  (See  the  autobiography  of  Mark  Twain  by  Samuel  Clemens) 
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Abstract:  Getting  lost  is  a common  phenomenon  for  anyone  who  has  ever  navigated 
through  a confusing  Web  site.  What  makes  a Web  site  confusing?  Why  do  people  get 
stuck  in  the  World  Wide  Web?  In  this  study  we  looked  at  how  people  reacted  when 
presented  with  Web  sites  that  differed  in  level  of  familiarity.  We  found  that  when  people 
viewed  Web  sites  with  an  unfamiliar  structure,  they  became  confused  and  they  tended  to 
take  more  time  when  navigating  Future  research  will  examine  what  factors  are  involved 
in  successfully  extricating  oneself  from  the  World  Wide  Web. 


Introduction 

One  highly  documented  phenomenon,  which  exemplifies  a potential  pitfall  of  allowing  users  to 
control  their  own  navigation,  is  getting  stuck  or  lost  in  hyperspace  (Boechler  2001).  No  one  really  knows 
for  sure  why  people  seem  so  prone  to  this  problem.  One  possible  reason  that  people  get  lost  is  that  they  rely 
on  some  preformed  concept  of  how  to  find  their  way  through  hyperspace.  For  example,  many  users  seem  to 
navigate  from  top  to  bottom,  left  to  right,  page  to  page  (Eveland  & Dunwoody  1998;  Niederhauser, 
Reynolds,  Salmen,  & Skolmoski  2000).  Another  possibility  is  that  the  cues  as  to  relevance  and  temporal 
precedence  are  not  the  same  for  a Web  page  as  they  are  for  books  or  other  print-based  media  (Sundar, 
Narayan,  Obregon,  & Uppal  1999;  Tewksbury  & Althaus  2000).  It  is  left  up  to  the  user  to  select  which 
information  is  important  to  attend  to  without  the  benefit  of  familiar  cues.  A user  who  is  not  familiar  with 
the  structure  of  a particular  Web  site  may  not  recognize  the  significance  or  relevance  of  the  cues,  and  may 
instead  look  for  cues  that  they  have  seen  and  used  before. 

We  are  initiating  a program  of  research  to  begin  to  determine  how  it  is  that  people  navigate 
through  hyperspace  and  why  this  process  breaks  down  so  frequently.  The  first  study,  which  is  reported 
here,  examines  what  impact  level  of  familiarity  has  on  navigation.  We  compared  two  Web  sites  with 
different  levels  of  familiarity  classified  as  rexr-based  and  metaphor-hdiStd  respectively.  The  metaphor- 
based  Web  site  was  designed  to  represent  links  metaphorically  with  pictures.  The  text-based  site 
represented  links  with  text. 

We  hypothesized  that  participants  would  spend  different  amounts  of  time  navigating  through  the 
Web  site  based  on  their  familiarity  with  the  structure  of  the  Web  site  that  they  viewed  because  the  cues  that 
they  expected  would  not  be  there.  We  believed  that  people  would  spend  more  time  navigating  the 
unfamiliar  site.  We  also  believed  that  there  would  be  differences  in  the  amount  of  information  recalled 
based  on  the  type  of  Web  site  that  they  were  viewing,  because  getting  stuck  and  being  confused  by  the  Web 
site  might  actually  compel  users  to  internalize  the  structure  of  the  unfamiliar  Web  site  and  thus  lead  to  an 
increase  in  the  amount  of  information  recalled. 


Method 

Participants  and  Materials 

Participants  were  93  undergraduate  psychology  students  from  the  University  of  Alberta.  They 
received  course  credit  for  their  participation.  The  majority  of  participants  (85%)  were  18  to  20  years  of 
age,  6%  were  younger  than  18,  and  9%  were  older  than  20.  Of  the  93  participants,  62%  were  female  and 
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38%  were  male.  The  majority  of  participants,  66%,  were  in  their  first  year  of  study,  22%  of  participants 
were  in  their  second  year  of  study  and  the  other  12%  where  either  in  their  third  or  fourth  year  of  study. 
There  were  45  participants  in  the  metaphor-bdiSQd,  condition  and  48  in  the  ^ex^-based  condition.  The 
materials  were  two  Web  sites,  an  online  questionnaire,  and  a free  recall  task. 


Procedure 

Participants  were  randomly  assigned  to  view  either  one  type  of  Web  site  or  the  other  and  were 
asked  to  evaluate  the  Web  site.  They  were  given  a questionnaire  to  fill  out  after  they  were  finished  looking 
at  the  Web  site  The  participants  were  then  were  asked  to  complete  a free  recall  task,  about  which  they  had 
no  prior  knowledge.  There  was  no  time  limit  given  for  any  of  the  tasks,  and  the  participants  were  not  told 
of  the  hypotheses  of  the  study. 


Results  and  Discussion 

Participants  viewing  the  metaphor-bdiSQd  site  rated  the  Web  site  significantly  more  confusing  and 
less  understandable  than  participants  viewing  the  ^ex^-based  site.  Participants  also  spent  more  time  on  the 
less  familiar  site.  The  lack  of  a familiar  structure  seemed  to  be  a hndrance  for  the  participants  in  the 
metaphor-bdistd.  condition.  Although  participants  in  the  /e^xr-based  condition  often  reported  that  the  site 
they  were  viewing  was  boring,  and  rated  their  site  as  less  interesting  than  participants  in  the  ^ex^-based 
condition,  they  were  able  to  navigate  more  quickly  through  the  site  than  participants  in  the  me^qp/ior-based 
condition.  This  suggests  that  even  though  the  familiar  structure  may  not  have  been  as  visually  appealing  as 
the  less  familiar  one,  the  cues  were  transparent  and  participants  did  not  have  to  have  to  discover  the 
meaning  of  the  cues. 

While  lack  of  a familiar  structure  did  seem  to  be  a problem  for  navigation,  it  did  not  impact 
participants’  abilities  to  recall  information  implying,  perhaps,  that  participants  did  not  internalize  the 
structure  of  the  Web  site.  There  was  no  difference  in  the  amount  of  information  participants  were  able  to 
recall,  either  about  the  structure  of  the  Web  sites  or  the  specific  content  that  was  located  on  the  Web  sites. 

People  come  to  Web  sites  with  preconceived  notions  as  to  what  Web  sites  should  look  like  and  the 
conventions  that  should  be  used.  When  these  preconceived  notions  are  not  upheld,  it  can  lead  to  an 
increased  level  of  confusion  and  frustration  when  navigating  online.  When  we  are  driving,  or  otherwise 
navigating  through  a physical  environment  we  rely  on  conventional  cues  such  as  street  signs  or  traffic 
signals  to  aid  us  in  arriving  at  our  desired  destination.  Similarly,  when  navigating  through  Web  sites,  we 
have  expectations  about  how  navigational  cues  should  be  implemented  In  unfamiliar  environments  people 
have  a hard  time  recognizing  the  significance  of  navigational  cues.  We  plan  to  extend  our  research  to 
examine  what  factors  are  the  most  important  for  successful  navigation  and  to  gather  further  evidence  as  to 
why  people  get  stuck  in  the  World  Wide  Web. 
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Abstract:  In  order  to  teach  K-12  inservice  and  preservice  teachers  to  use  computers 
and  instructional  technology  to  augment  their  instruction,  Internet  use  was  integrated 
into  a face-to-face  master’s  level  education  class  at  St.  John’s  University.  After  setting 
course  outcomes,  a curriculum  was  developed  working  backward  to  create  an  ordered 
series  of  instructional  tasks  to  achieve  those  outcomes.  The  procedures  didn’t  require 
sophisticated  software  and  can  be  replicated  in  the  classroom  using  a word  processor. 


Introduction 

Are  today’s  teachers  ready  to  integrate  technology  into  their  classrooms?  According  to  the 
International  Society  for  Technology  in  Education,  ISTE  (2000)  a major  problem  in  achieving  this  goal  is 
the  lack  of  teacher  expertise.  According  to  Jayroe,  Ball  and  Novinski  (2001)  the  difficulty  faced  by  many 
educators  is  how  to  introduce  technology  into  the  curriculum  in  a meaningful  and  constructive  manner. 
Inservice  teachers  need  to  be  able  to  understand  the  usefulness  of  how  technology  can  be  effectively  used 
in  the  classroom  (Franklin,  Turner,  Kariuki  & Duran,  2001). 


Discussion 

Many  new  teachers  leave  undergraduate  schools  with  limited  knowledge  of  how  to  use  technology 
in  the  classroom,  and  in  some  cases  the  university  faculty  are  “neither  modeling  the  use  of  technology,  nor 
requiring  students  to  use  technology”  (O’Bannon,  Matthew  & Thomas,  1998).  Before  teachers  enter  a 
classroom  they  must  be  technologically  literate.  Professors  must  be  proficient  in  the  use  of  technology,  and 
integrate  technology  into  their  existing  courses  (Kahn,  1997).  There  are  schools  of  higher  education  that 
presently  do  not  adequately  integrate  technology  into  their  teaching  (Rosenthal,  1999).  Inservice  teachers 
use  the  methodology  that  they  feel  comfortable  with,  when  presenting  a lesson.  When  a prospective 
teacher  is  taught  with  technology,  this  becomes  embedded  in  their  schema,  and  makes  it  easier  to  use  when 
they  teach  (Plotnick,  1995).  Technology  is  not  central  to  teacher  preparation  experience,  and  most 
technology  instruction  is  teaching  about  technology,  not  teaching  with  technology  across  the  curriculum 
(Beck  & Wynn,  1998).  Albion  and  Gibson  (2000)  observed  that  if  teachers  are  to  become  successful  at 
adapting  technology  into  their  curriculum,  they  require  an  understanding  beyond  that  of  being  confident  in 
using  computers  in  their  daily  lives. 


Implementation 

It  was  important  to  engage  learners  in  meaningful  hands  on  projects  from  the  onset  of  classes. 
Through  the  use  of  project-based  building  block  constructivist  methodology  a course  was  designed  that 
gave  learners  new  ideas  on  ways  to  infuse  technology  into  their  curriculum  while  learning  about 
instructional  technology.  The  course  of  study  was  divided  into  projects.  Each  project  required  mastery 
before  attempting  the  ensuing  project.  Projects  included;  e-mailing  a graphic  attachment  in  a word 
processing  document,  locating  and  evaluating  five  curriculunvbased  Web  sites,  creating  classroom/teacher 


material  using  Microsoft  Word,  presenting  subject  matter  using  PowerPoint  and  creating  a Project-Based 
thematic  unit  using  Inquiry-Based  Learning.  A rubric  was  given  to  each  student  delineating  what  was 
required  for  each  project.  Grades  were  posted  to  the  class  Blackboard  Web  site.  This  required  the  learners 
to  monitor  Blackboard  where  relevant  announcements  and  projects  were  posted. 


Conclusions 

It  is  important  in  any  course  of  study  where  instructional  technology  is  used  to  have  students 
become  engaged  in  meaningful  hands  on  projects.  Summers,  Lohr  and  O’Neil  (2002)  discuss  the 
importance  of  a learner  being  able  to  transfer  what  they  have  learned  in  the  classroom  to  real  life  situations. 
Through  connecting  technology  with  active  involvement  in  learning  Kommers  and  Mizzoguchi  (2000) 
believe  that  teachers  create  a knowledge  base  that  is  described  as  a key  issue  by  constructivists,  Lloyd, 
Merkley  and  Dennenbring  (2001)  contend  that  learners  must  feel  comfortable  and  understand  the  relevance 
of  using  various  technologies  upon  graduation  in  order  for  it  to  become  part  of  the  educational  process. 
Technology  has  an  important  role  in  teaching.  Technology  should  not  be  taught  in  isolation  but  used  to 
augment  traditional  teaching  methodologies  by  providing  alternative  means  of  obtaining  information. 
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Abstract:  This  paper  is  a report  of  the  early  findings  of  a study  conducted  within  Tamid's 
Civics  Teachers  On-Line  Community,  operating  at  the  Open  University  of  Israel.  At  "Tamid", 
we  aim  to  improve  the  standards  of  civics  teaching  in  High  schools  by  developing  a specific 
environment  that  enhances  democratic  values  among  in-service  teachers  for  the  benefits  of  their 
students.  Democracy  is  dynamic  and  evolves  and  improves  according  to  social,  cultural  and 
technological  changes.  Accessibility  and  transparency  are  two  qualities  of  the  World  Wide  Web 
that  may  deeply  influence  tomorrow’s  democracy.  At  Tamid,  the  interactions  between  the 
teachers  are  examined  from  the  pedagogical  content  knowledge  building  perspective.  Early 
findings  show  that  On-Line  relevant  resources  and  assignments  encouraged  some  teachers  to 
relate  to  their  peers'  work,  to  share  their  own  resources  and  to  use  them  with  their  students; 
others  witnessed  they  used  resources  and  enjoyed  discussions  in  a passive  mode. 


Introduction 

Israel's  internal  cleavages  are  deepening  and  the  teaching  of  civics  became  a very  important  challenge 
for  Israel’s  education  system.  A new  civics  curriculum  that  is  for  the  first  time  aimed  at  the  Secular,  Religious  and 
Arab  young  Israeli  citizens  was  introduced  in  2000.  This  new  curriculum  sees  democracy  as  a way  of  life  rooted 
in  values  and  principles  and  not  only  as  formal  set  of  institutions  and  processes.  This  was  our  opportunity  to 
meet  the  teachers'  needs  and  provide  them  a specific  on-line  educational  environment  in  order  for  them  to  cope 
with  these  heavy  conceptual  changes.  The  debate  on  the  democratization  influence  of  the  web  convinced  us  to 
examine  civics  teachers  on-line  learning  patterns.  Accessibility  and  transparency  are  tools  that  enable  the 
implementation  of  democratic  values,  it  was  interesting  to  check  if  a widely  open  and  accessible  on-line 
environment  could  enhance  a meaningful  and  practically  democratic  process  of  learning:  pluralistic  debate, 
freedom  of  expression,  respectful  interactions  etc.... 

Our  project  started  with  20  Civics  teachers  in  spring  2000  as  a new  branch  of  the  Tamid  environment  for 
Science  Teachers.  Since  October  2001,  we  train  120  teachers  in  5 geographical  groups  by  way  of  an  on-line 
course.  Apart  from  the  on-line  course,  the  teachers  enjoy  valuable  resources  both  at  the  knowledge  and 
pedagogical  levels.  Tamid's  site  is  wide  open  to  all  with  no  need  for  password  as  part  of  our  genuine  desire  to 
provide  a design,  accessible  and  transparent,  to  all  Civics  Teachers.  We  were  interested  to  check  if  the  on-line 
wide-open  Tamid  environment  was  motivating  teachers  to  share  their  resources/knowledge  with  their  peers  and 
engage  reflective  dialogue  around  specific  and  general  teaching  problems/dilemmas.  Was  there  a correlation 
between  the  fact  that  our  environment  was  accessible  and  transparent  to  all  civics  teachers  and  the  quality  of  the 
interactions  between  teachers  and  moderators  and  among  peers  teachers? 


The  Study:  Building  together  practical  disciplinary  knowledge 


**There  may  be  some  new  form  of  community  developing  among  the  myriad  solitaries  perched  in  front 
of  their  screens  and  connected  only  by  their  fingertips  to  the  new  web  defined  by  the  internet.  But  the  politics 
of  that  **community”  has  to  be  invented..  " Benjamin  Barber,  1997. 
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Inventing  the  “politics”  of  our  civics  teachers’  community  was  not  an  easy  job  since  we  had  to  create  a 
climate  of  mutual  trust  as  a condition  for  a meaningful  learning  process.  Coordinating  the  aspirations  of  each  one 
of  different  participants,  thinking  over  what  is  a meaningful  feed-back,  discussing  the  advantages  and  the  risks 
of  a transparent  environment  in  which  everyone's  assignment  is  subject  to  the  group's  criticism  or  praise,  dealing 
with  others’  opinions  and  experience  the  suffering  that  may  imply  tolerance.  These  are  only  a tiny  part  of  the  on- 
going questioning  which  the  developing  team  and  the  teachers  are  experiencing  day  after  day  at  Tamid. 

Our  research  is  based  on  one  On-Line  course  that  has  been  running  in  five  geographical  groups  from 
October  2001  to  April  2002.  120  in-service  teachers  attended  the  course.  Electronic  discourse  was  taking  place  in 
both  a “national"  discussion  group  - where  actuality  issues  where  discussed  from  the  point  of  view  of 
practitioners  that  needed  to  use  up  to  date  material  for  explaining  curricular  concepts  - and  local  groups  - where 
assignments  were  mainly  oriented  toward  analysis  of  students’  texts  and  knowledge  building  trough  academic 
articles  and  their  "translation”  into  didactical  practice. 

As  part  of  the  learning  process  the  teachers  had  to  develop  their  own  sense  of  responsibility  and 
autonomy  since  there  was  no  external  due-date  for  the  diverse  assignments.  Still,  all  of  the  assignments  implied 
interaction  and  discussion.  Each  teacher  had  to  lead  the  discussion  one  academic  article  within  an  on-line 
discussion  group.  This  dialogue  was  a very  fascinating  alchemy  since  it  was  an  occasion  to  enhance  the 
connection  between  the  subject  matter  and  its  didactics  besides  inducing  reflective  thinking  among  the  teachers. 
Who  started  the  discussion?  Who  reacted?  How  many  "reactions"  to  a topic?  What  made  a topic  turning  into  a 
highly  reflective  learning  process?  To  which  extent  did  "passive"  teachers  witnessed  the  use  they  made  of 
materials  or  arguments  that  came  up  during  the  discussion?  Which  patterns  of  participation:  pedagogical  issues 
or  ideological  views,  passive  versus  active  behaviors,  consumer  or  contributor  etc.. 


Early  Findings 

The  building  of  an  on-line  teachers  community  is  a process.  But  our  early  findings  (evaluations, 
questionnaires,  interviews  and  discussion  groups  analysis)  show  that  for  most  of  the  teachers,  Tamid  site 
became  a professional  meeting  place:  to  submit  a test  to  peers  and  moderators  appreciation,  to  view  links  to  up- 
to-date  articles  on  relevant  issues,  to  ask  specific  questions  that  came  up  at  school,  to  take  part  on  discussions 
on  political  discussions  and  to  think  together  with  peers  of  new  strategies  to  teach  highly  controversial  topics, 
to  inform  peers  of  the  use  that  was  made  with  students  of  material  or  arguments..  These  occur  in  a highly 
respectful  climate,  accessibility  to  all  but  still  no  curses  and  no  anonymous  participants.  Thanks  to  Tamid's  web 
environment,  teachers  from  the  periphery  could  enjoy  the  same  high  quality  resources,  experts  and  facilitators  as 
teachers  from  the  center  of  the  country  but  this  is  still  a theoretical  access  since  most  of  these  teachers  usually 
lack  the  technological  tools  to  make  their  participation  in  the  on-line  learning  process  meaningful. 

Teachers  are  practitioners  that  need  to  update,  confront  and  articulate  their  subject  matter  knowledge 
and  their  teaching  skills.  Therefore,  building  practical  knowledge  (from  home)  through  an  easily  accessible  site 
designed  to  meet  their  needs  is  highly  motivating  for  in-service  training  teachers. 
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Abstract:  In  recent  years,  the  view  that  Information  and  Communication  Technology 
(ICT)  is  vital  in  K12  education  has  become  widespread.  ICT  use  in  schools  has 
increased  and  various  professional  bodies  have  set  ICT  standards  for  students  and 
teachers.  Schools  of  education  are  under  pressure  to  produce  teachers  who  are  able 
to  effectively  integrate  technology  into  their  teaching.  However,  most  teacher 
preparation  programs  do  not  adequately  prepare  teachers  in  ICT,  nor  assess 
candidates  relative  to  ICT  standards.  This  paper  discusses  the  development  of  a 
computerized  system  to  assess  ICT  declarative  and  procedural  knowledge  and  to 
provide  a profile  to  the  participant. 


Introduction 

In  recent  years,  governments,  education  organizations,  and  researchers  have  increasingly  supported  the  view 
that  incorporating  ICT  into  learning  and  teaching  is  an  important  aspect  of  keeping  the  curriculum  relevant  and 
preparing  students  for  their  future  in  a complex  knowledge-based  world  (Alberta  Education,  1999b;  CEO  Forum 
on  Education  and  Technology,  1997;  Jonassen,  1995;  Logan,  1995;  Milken  Exchange  on  Educational 
Technology,  1999;  Thornburg,  1991).  Data  that  provide  insight  into  tfie  computer  literacy  level  of  incoming 
undergraduate  education  students  would  be  helpful  to  faculty  designing  appropriate  curriculum.  However,  there 
is  currently  no  assessment  tool  that  establishes  the  ICT  literacy  level  of  current  or  prospective  students.  The 
predictive  validity  of  a high  school  transcript  or  grade  12  English  or  mathematics  marks  is  insufficient.  How  do 
the  ICT  skills  of  recent  undergraduate  students  compare  with  the  skills  indicated  at  the  Grade  12  level  of  the 
Provincial  technology  outcomes?  Although  the  Best  Practices  in  Technology  documents  (Alberta  Education, 
1999a)  indicate  that  much  ICT -related  activity  is  occurring  in  schools,  and  an  optional  ICT  K-12  curriculum  has 
been  available  since  June  1998,  there  are  currently  a few  students  entering  educational  technology  courses  at 
this  university  who  have  literally  never  turned  on  a computer  (based  on  an  informal  “hands-up”  survey  of 
students  in  September  1999),  while  some  students  are  familiar  with  some  aspects  of  ICT,  and  a few  are  quite 
adept.  This  paper,  part  of  a larger  study  (Davies,  2002)  discusses  the  development  of  several  Web-based 
instruments  (Research  Consent  Form,  Attitude  Survey,  Background  Survey,  Knowledge  and  Performance  Tests) 
which  assess  the  computer  literacy  level  of  incoming  undergraduate  education  students. 


Development  of  the  Instruments 

The  Background  Survey  and  Knowledge  Test  were  implemented  using  ASPs  written  in  the  VBScript  (Microsoft 
Corporation,  2000b)  programming  language.  The  ASP  dynamically  generated  a Web  page  containing  an  HTML 
form  with  question  and  response  data  stored  in  a Microsoft  Access  2000  relational  database  on  a Windows 
server.  The  Background  Survey  included  a variety  of  form  field  types  that  enabled  implementation  of  different 
question  types:  radio  button  (multiple -choice/single  response),  drop-down  list  (multiple-choice/single  response 
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with  a large  set  of  possible  answers),  check  box  (multiple -choice/multiple  response),  and  text  box  (short  answer). 
The  Knowledge  Test  was  composed  entirely  of  multiple-choice  questions  with  five  possible  radio  button 
answers. 

The  ASP  also  inserted  client-side  JavaScript  (Netscape  Communications  Corporation,  2000)  code  into  these 
HTML  pages  for  the  purpose  of  quick  data  validation.  When  the  participant  clicked  on  the  form  “Submit” 
button,  the  JavaScript  local  editing  procedure  was  invoked.  If  unacceptable  data  were  found,  an  error  message 
window  was  displayed  on  the  screen  and  the  form  was  not  submitted.  For  example,  the  Background  Survey 
asked  for  the  year  of  high  school  graduation.  Acceptable  responses  had  to  be  a four-digit  year  not  greater  than 
the  current  year.  Upon  acknowledgment  of  the  error  message,  the  participants’  display  was  automatically 
scrolled  to  the  question  where  the  error  was  found.  The  JavaScript  procedure  also  checked  to  see  whether  all 
applicable  questions  had  been  answered.  If  missing  responses  were  found,  a warning  message  window  was 
displayed  on  the  screen.  Since  it  was  unethical  to  demand  that  the  participant  answer  all  questions,  the 
participant  was  then  given  the  choice  to  return  to  the  first  missing  response  or  to  submit  the  data.  The 
Knowledge  Test  responses  were  immediately  scored  by  comparing  with  the  correct  response  stored  in  the 
database  for  each  item. 

Observations  of  participants  filling  in  the  Web-based  forms  indicated  that  the  JavaScript  validation  routines 
contributed  to  the  completeness  and  accuracy  of  the  data  collected.  For  example,  one  participant  was  observed 
to  receive  the  warning  message  stating  that  not  all  questions  were  answered.  The  individual  read  the  warning 
message,  returned  to  the  form,  stated  “Oops,  I missed  that  question,”  clicked  on  a response,  and  proceeded  to 
submit  the  form  with  all  questions  completed.  In  all  of  the  Background  Survey  data  collected  during  the  pre- 
course pilot  test  (34  participants  x 35  questions  each  for  a total  of  1190  items)  only  1 missing  response  and  no 
invalid  responses  occurred.  It  would  have  been  extremely  time-consuming  to  ensure  this  fevel  of  data 
completeness  and  accuracy  with  paper-based  forms.  In  the  corresponding  Knowledge  Test  data  (952  items), 
there  were  45  unanswered  items.  These  were  treated  as  incorrect  responses  when  computing  the  overall  test 
score.  The  vast  majority  of  the  missing  responses  (39)  occurred  because  two  participants  attempted  the  first  few 
questions  and  then  submitted  the  form  without  checking  responses  for  the  remaining  questions.  Some  additional 
messages  were  added  to  the  Knowledge  Test  to  ensure  that  students  would  be  aware  that  there  was  no  penalty 
for  guessing. 

The  Performance  Test  was  a much  more  technically  complex  instrument  than  the  Knowledge  Test  since  it 
required  automating  the  analysis  of  files  that  participants  manipulated  on  their  local  computers.  The  actual 
applications  tested  were  established  based  upon  required  ICT  skills,  but  were  delimited  by  criteria  such  as  time 
constraints  for  the  initial  instrument  development,  minimizing  problems  in  collecting  data,  and  allowing  students 
to  take  the  Performance  Test  with  varying  versions  of  application  software  on  either  the  Windows  or  Macintosh 
platform.  The  solution  chosen  was  to  create  a Visual  Basic  for  Applications  (VBA)  (Microsoft  Corporation,  2000) 
procedure  within  the  same  Access  database  described  earlier.  Web-based  (especially  client-side)  programming 
techniques  were  avoided  because  of  variable  client  computer  setup  and  security  issues  involved  in  attempting 
to  examine  files  on  a client  computer  over  the  Internet.  This  part  of  the  system  required  exchanging  a set  of  files 
(compressed  into  a single  archive)  between  each  participant’s  computer  and  the  database  server. 

During  the  Performance  Test,  the  VBA  procedure  was  continually  running,  monitoring  a certain  file  directory 
every  60  seconds  for  arriving  submissions  and  executing  an  automated  scoring  routine.  The  VBA  scoring 
procedure  implemented  programming  techniques  (e.g.,  use  of  Microsoft  Automation  objects,  methods  and 
properties)  which  enabled  automated  execution  of  file  system  commands  (e.g.,  file  searches  and  directory 
listings),  reading  of  text  stream  files,  interfacing  with  external  applications  (e.g.,  Microsoft  Word  and  Excel), 
opening  files  in  these  programs,  and  examining  their  object  hierarchy. 

The  automated  scoring  routines  were  subjected  to  several  iterations  of  testing  and  refining.  The  first  author 
created  about  a dozen  test  cases  - sets  of  computer  files  representing  completed  student  practical  tests  - with  a 
mixture  of  completely  correct,  partially  correct  and  completely  incorrect  solutions  for  each  task,  completed  using 
different  software  versions  on  different  computer  platforms.  These  were  subjected  to  the  automated  routines  and 
the  scores  for  each  item  of  each  test  case  were  manually  checked  for  accuracy.  This  was  repeated  until  all  of  the 
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test  cases  were  being  properly  scored.  These  same  processes  were  repeated  with  files  created  by  various  expert 
reviewers  as  well  as  the  entire  group  of  student  files  resulting  from  the  performance  pilot  test.  Programming 
efficiency  was  also  examined  and  improved  until  the  time  required  to  score  the  performance  test  averaged  less 
than  a second  per  test  case. 

The  procedure  that  scored  the  spreadsheet  activity  of  the  Performance  Test  highlights  the  flexibility  in  the 
software  used  by  participants  allowed  by  the  programming  solution  chosen.  Spreadsheet  files  created  in  Excel  5 
and  98  for  Macintosh  and  Excel  95,  97,  and  2000  for  Windows  formats  were  all  scored  without  technical 
problems.  In  addition,  the  procedure  also  worked  for  files  created  in  other  programs  such  as  Apple/ClarisWorks 
or  Corel  Quattro  Pro  then  saved  in  Excel  format.  Multi-format  flexibility  assumes  that  the  spreadsheet  activities 
chosen  for  the  test  are  limited  to  common  features  available  in  the  different  spreadsheet  file  formats.  Only 
common  tasks  such  as  basic  text  or  number  formatting,  cell  alignment,  and  formulas  were  used.  This  was 
adequate  for  the  level  of  expertise  being  measured  in  the  target  population. 


Pilot  Test 

The  instruments  were  pilot-tested  on  a thirty-five  volunteer  undergraduate  education  students  prior  to  the  main 
data-gathering  period.  These  students  were  drawn  from  registrants  in  two  Summer  2000  sections  of  the 
recommended  educational  computing  option  course.  The  students  com.pleted  the  Web-based  foims  and  the 
Performance  Test  in  a campus  lab  on  Pentium  450  MHz  computers  with  Microsoft  Windows  98  and  Office  2000 
(Microsoft  Corporation,  2000a)  installed.  No  student  chose  the  options  of  using  a Macintosh  computer,  or  to  fill 
in  paper-based  copies  of  the  online  forms.  The  same  group  of  students  was  later  invited  to  participate  in  a 
Knowledge  and  Performance  post-test,  which  was  held  on  the  second-last  day  of  the  term.  After  completing  the 
test,  the  students  were  given  a list  of  answers  to  the  multiple -choice  Knowledge  Test,  computer  files  providing 
correct  solutions  to  the  Performance  Test,  and  personal  help  with  any  questions  they  had. 


Instrument  Validity 

Content  validity  of  the  instruments  was  independently  judged  by  three  individuals  who  have  expertise  in 
educational  technology:  a faculty  member,  a PhD  student,  and  a senior  undergraduate  student  who  had  worked 
for  a year  as  a marker  in  the  educational  technology  undergraduate  course.  A number  of  modifications  were 
made  to  the  instruments  based  on  the  feedback  from  these  initial  reviewers.  For  example,  some  items  that  were 
deemed  inappropriate  were  deleted  from  the  instruments,  the  wording  of  some  items  was  clarified,  computer 
displays  were  improved,  and  new  items  suggested  by  the  reviewers  were  added.  These  reviewers  also  served  to 
verify  that  the  system  was  operating  without  technical  errors.  Additional  educational  technology  experts  were 
called  upon  to  similarly  review  the  instruments  after  the  pre -course  pilot  test.  Improvements  to  the  instruments 
were  made  as  a result  of  this  second  round  of  validation  activities. 

Feedback  on  the  instruments  was  obtained  from  the  pilot-test  students  in  a number  of  ways.  First,  during  the 
pre-test,  the  researcher  asked  the  students  to  raise  their  hand  if  at  any  time  during  the  testing  they  found  any 
information  on  the  consent  form  or  any  question  on  the  instruments  to  be  unclear  or  inappropriate.  A few  such 
questions  occurred  and  were  discussed  privately  with  the  participant.  These  inquiries  were  noted  on  the 
researcher’s  printed  copies  of  the  instruments.  Second,  after  reviewing  the  pre-test  data,  several  students  were 
contacted  by  email  and  asked  for  more  information  concerning  their  answers  to  certain  items  on  the  Background 
Survey.  This  resulted  in  some  ideas  for  additional  changes  to  the  survey.  Third,  during  the  post-test,  the 
students  were  asked  to  fill  in  a short  feedback  sheet.  They  were  asked  whether  there  were  items  that  they  felt 
were  unclear  or  inappropriate  on  either  the  Knowledge  or  Performance  Test,  and  whether  they  had  any 
suggestions  for  additional  items  that  could  be  included.  All  of  these  sources  of  student  feedback  were  reviewed 
and  resulted  in  modifications  to  the  instruments. 
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Statistical  correlations  were  computed  as  indicators  of  test  validity  (see  Table  1).  A high  correlation  (r(33)  = . 460, 
/7<.01)‘  between  the  Knowledge  pre-test  and  Performance  pre-test  scores  provided  evidence  of  concurrent 
criterion-related  validity.  That  is,  there  was  logically  some  commonality  in  the  underlying  constructs  that  these 
two  tests  measured.  Strong  correlations  between  the  course  midterm  exam  which  occurred  two  weeks  after  pre- 
test and  the  pre-test  Knowledge  (r(30)  = . 533,/?<.01)  and  Performance  (K^O)  = .606,/?<.001)  scores  were  evidence 
of  predictive  criterion-related  validity. 

Correlations  between  the  course  final  exam  and  the  post-tests  (run  1 day  before  the  exam)  were:  Knowledge 
post-test  K21)  = -409,  p =.059)  and  Performance  post-test  ^23)  = .673,  /?<.001).  The  latter  correlation  was 
significant  and  offers  strong  evidence  of  concurrent  criterion-related  validity.  The  first  correlation,  while  not 
quite  statistically  significant  at  the  .05  level  (p=.059),  still  offered  some  evidence  of  validity.  It  should  be  noted 
that  the  course  exams  during  the  Summer  2000  term  were  entirely  performance-based,  thus  it  was  not  surprising 
that  the  correlations  between  the  course  exams  were  stronger  with  the  Performance  Test  than  with  the 
Knowledge  Test.  Also,  it  was  easier  to  obtain  some  marks  by  sheer  guessing  on  a multiple -choice  test  than  it 
was  on  a performance-based  test.  Correlations  between  the  course  final  exam  and  the  pre-tests  were:  Knowledge 
Test  ^28)  = .522,  p<.0\)  and  Performance  Test  (r  (28)=  .688,  /?<.001).  All  of  these  correlations  were  further 
evidence  of  predictive  criterion-related  validity. 


Know 

Pre 

PerfPre 

Know 

Post 

Perf  Post 

Course 

MExam 

Course 

FExam 

Course 

Assign 

Course 

Total 

Know  Pre 

1.000 

PerfPre 

.460** 

1.000 

Know  Post 

.556** 

.332 

1.000 

Perf  Post 

.342 

.678** 

.220 

1.000 

Course  MExam 

.533** 

.606** 

.211 

.162 

1.000 

Course  FExam 

.522** 

.688** 

.409 

.673** 

.627** 

1.000 

Course  Assign 

.506** 

.486** 

.141 

.427* 

.579** 

.592** 

1.000 

Course  Total 

.592** 

.707** 

.341 

.543** 

.804** 

.917** 

.808** 

1.000 

**  Correlation  significant  at  the  .01  level  * Correlation  significant  at  the  .05  level 
Table  1:  Pilot  Test  Instrument  Validity  - Correlation  Statistics 


Instrument  Reliability 

The  Background  Survey  was  not  constructed  as  a scale  where  all  items  contributed  to  an  overall  score.  Rather 
than  computing  internal  consistency  statistics,  reliability  of  this  instrument  was  established  by  selective  re- 
testing. Four  of  the  students  who  had  volunteered  to  participate  in  the  initial  instrument  evaluation  were  re- 
tested using  a different  format  for  presenting  the  questions  (as  an  interview  rather  than  online  written 
questions).  No  differences  in  the  responses  from  the  two  forms  were  found,  indicating  high  reliability  (Fraenkel 
& Wallen,  1996)  although  these  participants  did  offer  a few  suggestions  for  clarifying  the  wording  of  a few  items. 
The  consistency  of  responses  was  not  surprising,  since  most  of  the  questions  on  the  survey  would  be 
considered  objective  (mainly  factual  information  such  as  whether  or  not  they  own  a home  computer).  Answers 
to  questions  like  these  are  likely  to  be  answered  the  same  in  a test-retest  situation  where  there  is  little  time 
between  tests. 

Cronbach's  alpha  coefficient  was  computed  for  the  pre-test  Attitude  Survey  (.63),  the  Knowledge  Test  (.79),  and 
the  Performance  Test  (.91).  The  Attitude  Survey  reliability  was  judged  too  low  for  meaningful  data  interpretation 
(a  widely  accepted  lower  limit  for  alpha  is  .7),  the  second  was  acceptable,  and  the  latter  was  exceptionally  high, 
being  at  the  level  of  marketed  achievement  tests  (Fraenkel  & Wallen,  1996,  p.l63). 


‘ In  this  notation,  r(33)  means  r(df),  where  df  is  the  degrees  of  freedom  (equal  to  n-1) 


Improvements  to  the  Attitude  Survey  were  essential  to  establish  solid  reliability.  Analysis  of  the  inter-item 
correlation  matrix  identified  three  items  that  were  poorly  associated  with  the  other  items  and  thus  did  not 
contribute  well  to  the  overall  test  score;  these  items  were  modified.  In  addition,  since  the  survey  originally 
consisted  of  only  1 2 items,  reliability  could  be  easily  raised  by  increasing  the  number  of  related  items  (Fraenkel  & 
Wallen,  1996,  p.l63).  A target  of  20  items  was  established. 

The  alpha  coefficients  for  the  post-test  Knowledge  Test  and  Performance  Test  (run  again  at  the  end  of  term  as  a 
post-test)  were  not  as  high,  .63  and  .84  respectively.  This  was  because  the  tests  included  some  questions 
deemed  by  experts  to  be  easy  (equivalent  to  the  stated  prerequisites  for  the  recommended  educational 
computing  course),  yet  which  stumped  many  students  in  the  pre-test,  effectively  screening  individuals  with  very 
low  knowledge  or  practical  skills.  Tests  are  not  always  equally  effective  in  different  situations  (Murphy  & 
Davidshofer,  1991);  these  tests  were  less  effective  as  a post-test  after  completing  a course  which  covers  much  of 
the  content  of  the  tests  and  provides  remediation  for  missing  prerequisite  skills. 

In  the  pre -course  Knowledge  Test,  no  questions  (out  of  28)  were  answered  correctly  by  all  students.  In  fact,  the 
easiest  question  was  answered  correctly  by  88%  of  students.  By  contrast,  in  the  post  Knowledge  Test,  there 
were  4 questions  answered  correctly  by  all  students,  and  another  8 questions  answered  correctly  by  at  least  75% 
of  students.  There  was  less  overall  variance  in  the  post-test  scores  than  in  the  pre-test  scores.  Also,  items  of 
zero  variance  (same  score  for  all  students)  cannot  be  correlated  with  other  test  items  and  thus  do  not  enter  into 
the  reliability  calculations,  which  reduces  the  reliability  coefficient  (a  measure  of  average  inter-item  correlation). 

The  course  appears  to  have  been  effective  in  raising  the  student  scores.  The  mean  pre-to-post  gain  on  the 
Knowledge  Test  was  1 7.52.  A one-sample  t-test  on  the  gain  scores  (post  - pre),  comparing  them  to  a test  value 
of  zero  (equivalent  to  a dependent  or  paired  samples  t-test  using  the  pre  and  post  scores)  found  the  difference 
statistically  significant.  On  the  Performance  pre-test,  no  questions  (out  of  24)  were  answered  correctly  by  all 
students,  while  on  the  post-test,  6 questions  were  answered  correctly  by  all  students.  The  mean  pre-to-post  gain 
on  the  Performance  Test  was  26.07;  the  one-sample  t-test  on  the  gain  scores  compared  to  zero  found  the 
difference  statistically  significant.  It  should,  however,  be  noted  that  the  difference  in  cases  between  the  pre  and 
post-tests  must  be  considered;  the  students  who  didn’t  participate  in  both  tests  were  not  part  of  the  gain 
analysis  and  may  have  differed  from  those  that  did. 

The  pre-to-post  gains  on  these  two  tests  affirmed  the  effectiveness  of  the  instruction  and  were  additional  pieces 
of  evidence  for  the  validity  of  the  tests.  It  demonstrated  that  participants  who  have  had  more  training  or  practice 
with  ICT  tools  (i.e.,  the  students  at  post-test  time)  scored  much  higher  than  those  with  less  (i.e.,  the  students  at 
pre-test  time).  This  is  logically  consistent  with  what  the  tests  purport  to  measure. 

Time  Required  to  Administer  Instruments 

The  pilot  pre-test  also  served  to  verify  that  the  instruments  could  be  completed  within  a reasonable  timeframe  on 
a single  day.  The  actual  time  required  for  students  to  complete  all  of  the  forms  and  tests  was  approximately  1.5 
hours,  about  0.5  hour  for  all  of  the  online  forms  (consent,  attitudes,  and  knowledge)  and  1 hour  for  the 
performance  tasks.  The  start  and  end  time  for  each  participant’s  work  on  each  online  form  was  stored  in  the 
database,  making  it  simple  to  calculate  the  average  time  required  to  complete  a form.  In  the  case  of  the 
Background  Survey,  the  average  time  was  about  8 minutes,  and  the  maximum  time  was  12  minutes.  The  maximum 
time  required  for  the  12  Attitude  items  was  around  4 minutes.  For  the  Knowledge  Test,  the  minimum  time 
required  was  3 minutes,  the  maximum  25  minutes,  and  the  average  13  minutes.  For  the  Practical  test  the  minimum 
time  required  was  12  minutes,  the  maximum  65  minutes,  and  the  average  36  minutes. 


Observations  from  the  Pilot  Test 

A number  of  statistics  were  computed  to  provide  a general  picture  of  the  pilot  data.  Histograms  of  the  pre  and 
post  Knowledge  and  Performance  test  scores  revealed  that  these  distributions  were  approximately  normal,  with 
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the  post-tests  having  much  higher  means  and  lower  variances  than  the  corresponding  pre-test.  Overall,  the 
achievement  on  the  pre-course  tests  was  quite  low  with  the  means  of  both  tests  being  below  50%.  Comparing 
the  test  questions  against  the  recommended  educational  computing  course  curriculum  and  stated  prerequisites, 
the  researchers  concluded  that  marginally  acceptable  course  prerequisite  skills  and  knowledge  (basic  computer 
operation,  file  management  and  word  processing)  would  be  indicated  by  a score  of  at  least  50%  on  the  pre-tests. 
On  the  Knowledge  Test,  44%  of  students  did  not  meet  this  standard  and  59%  did  not  meet  it  on  the  Performance 
Test,  indicating  that  many  students  do  not  possess  adequate  course  prerequisites.  A more  comfortable  level  of 
prerequisite  skills  would  be  indicated  by  scores  of  at  least  60%.  On  the  Knowledge  Test,  77%  of  students  did 
not  reach  the  60%  level  and  71%  did  not  meet  it  on  the  Performance  Test.  At  the  other  end  ofthe  spectrum,  a few 
students  performed  well  enough  on  the  pre-test  to  be  likely  candidates  for  successfully  challenging  the  course 
or  taking  courses  that  require  skills  equivalent  to  completing  that  course  as  a prerequisite.  Obviously  more 
testing  and  validity  evaluation  would  be  required  to  establish  a standard  for  this,  but  in  the  authors’  opinion, 
80%  seems  like  a level  that  would  reasonably  indicate  mastery.  If  this  were  the  case,  3%  of  students  (1  individual 
in  the  pilot  group)  would  have  qualified.  A summary  report  on  the  pilot  test  group  performance  on  the  pre- 
course  tests  was  provided  for  informational  purposes  to  the  course  instructors  and  senior  teaching  assistants. 


Full  Implementation  of  the  Tests 

The  instruments  were  successfully  incorporated  into  the  regular  offering  of  the  course  beginning  in  the  Fall  2000 
(approximately  1000  students).  Validity  and  reliability  tests  gave  similar  results  to  the  pilot  test.  In  addition,  the 
implementation  of  online  testing  substantially  reduced  the  amount  of  time  required  to  evaluate  students, 
particularly  in  the  Performance  test.  The  data  from  these  tests  have  established  a baseline  of  data  on  the  ICT 
skills  and  knowledge  of  students  entering  our  Faculty  of  Education.  We  expect  the  average  ICT  skills  and 
knowledge  of  education  entrants  to  rise  over  the  next  few  years.  At  some  point  a certain  level  of  ICT  Literacy 
could  be  an  admission  requirement.  These  automated  instruments  could  be  used  as  an  efficient  admission 
screening  tool.  The  teacher  education  program  could  then  focus  more  resources  on  improving  teacher 
candidates’  abilities  to  integrate  technology  into  their  teaching  instead  of  development  of  basic  ICT  skills  and 
knowledge. 
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Abstract:  Two  popular  learning  environments,  TeleTOP  and  Blackboard,  are 
implemented  for  specific  educational  contexts  in  many  universities  and  other 
institutions.  Goal  is  to  increase  the  use  of  information  and  communication 
technology  (ICT)  in  education,  particularly  network  technology  or  Web-based 
systems.  These  electronic  learning  environments  do  not  seem  to  differ  a lot  in 
functionalities  and  use.  Also,  the  problems  that  instructors  have  to  deal  with  are 
similar.  Instructors  must  learn  how  to  work  with  these  new  tools,  and  discover  how 
they  effect  their  education.  Two  similar  best  practice  days  successful  experiences 
with  functionalities  offered  and  functionalities  used  by  the  instructors  learn  that 
instructors  can  make  education  as  flexible  (in  time  and  place)  for  students,  but 
should  be  aware  of  time  consuming  activities  that  cause  dissatisfaction  and 
frustration.  The  target  should  be  that  students  should  benefit  from  the  added  value 
of  electronic  learning  environments,  use  functionalities  that  are  useful  for  the 
students  and  the  use  is  consistently. 


Introduction 

The  inevitable  expansion  of  the  opportunities  of  technology  and  the  growing  awareness  of  the 
possibilities  for  using  the  technology  in  a specific  educational  context  are  causing  many  universities 
and  other  institutions  to  increase  the  use  of  information  and  communication  technology  (ICT)  in 
education,  particularly  network  technology  or  Web-based  systems.  In  relation  to  this  many 
organizations  are  using  or  starting  to  use  electronic  learning  environments.  Two  of  such  learning 
environments,  TeleTOP  and  Blackboard,  are  described  here.  After  a brief  overview  of  the  general 
concept  of  an  electronic  learning  environment  the  two  environments  will  be  described,  considering 
the  differences,  but  also  the  similarities  between  the  two  systems.  Based  on  two  similar  best  practice 
days  the  successful  experiences  with  functionalities  offered  and  functionalities  used  by  the  instructors 
will  be  described,  concluded  with  some  important  outcomes. 

Electronic  learning  environments 

TeleTOP  and  Blackboard  are  two  examples  of  tailor-made  Web  applications,  also  known  as  course 
management  systems,  virtual  learning  environments  or  electronic  learning  environments.  Mirande, 
Riemersma  and  Veen  (1997)  give  a very  broad  definition  of  an  electronic  learning  environment.  They 
describe  it  as  a learning  environment  that  uses  technology.  Jansen,  Fisser,  and  Terlouw  (2000) 
describe  an  electronic  learning  environment  as  an  environment  that  supports  and  enriches  the 
traditional  environment  by  functionally  using  new  forms  of  ICT.  Droste  (2000)  defines  e-leaming 
platforms  as  the  technical  facilities  that  facilitate  interaction  in  the  learning  process,  communication, 
and  organization.  The  three  main  characteristics  of  an  e-leaming  platform  in  her  analysis  are  that  it 
contains  a subject-matter  section,  a communication  section  and  an  organization  section. 

Electronic  learning  environment  are  thus  comprehensive  software  packages  that  support  some  or  all 
aspects  of  course  preparation,  delivery  and  interaction  and  are  accessible  via  a network  (Collis  & 
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Moonen,  2001).  Most  electronic  learning  environments  focus  upon  separate  courses  and  do  not 
collaborate  with  other  systems  such  as  the  personnel  database  and  a system  for  exams.  It  is  believed 
that  a system  such  as  Blackboard  or  TeleTOP  in  combination  with  existing  systems  can  offer  a 
comprehensive  electronic  learning  environment  with  one  portal  for  instructors  and  students  from 
which  they  can  find  all  necessary  information.  However,  this  paper  focuses  on  the  electronic  learning 
envrionment  used  in  education.  In  the  next  sections  TeleTOP  and  Blackboard  are  described  as  two 
examples  of  such  an  environment  and  a comparison  between  the  two  systems  is  made. 

TeleTOP 

TeleTOP  is  a course  management  system,  developed  at  the  Faculty  of  Educational  Science  and 
Technology  of  the  University  of  Twente.  Since  1997,  the  system  has  been  developed  and 
implemented.  The  implementation  started  at  the  faculty  of  Educational  Technology  and  other 
departments  soon  followed  (Collis  & De  Boer,  1 999).  At  this  moment  TeleTOP  is  used  throughout 
the  University  of  Twente  and  is  also  in  use  in  other  Universities,  companies  and  training  settings.  The 
core  ideas  to  start  building  a Web-based  environment  were  to  support  learning  and  make  it  more 
flexible.  Furthermore  the  best  of  “old  values  of  good  teaching”  and  an  attractive  campus  life  with  new 
didactics  and  advanced  technologies  were  taken  into  account  to  optimize  these  new  possibilities  in 
learning. 

The  TeleTOP  system  was  build  upon  a powerful  database  (Lotus  Notes)  and  used  by  all  staff 
members/teachers  in  the  Faculty  of  Educational  Science  and  Technology;  it  was  made  to  be  WWW- 
based  (for  both  students  as  well  as  instructors)  and  it  was  made  accessible  through  an  ordinary  Web- 
browser  everywhere  in  the  world.  The  TeleTOP  system  became  a very  easy  to  use  system  and  it 
allowed  to  a certain  extent  fit  with  the  instructional  practices  of  an  on-campus  University  and  extent 
these  to  distance  students  as  well.  It  also  did  engage  change  to  new  didactics.  The  starting-point  was 
that  WWW-based  learning  environments  should  not  replace  existing  teaching  and  learning,  but  should 
add  to  them.  As  many  systems  TeleTOP  is  able  to  create  and  display  all  sorts  of  earning  materials,  as 
video,  text,  glossaries,  web  links,  polls,  quizzes,  and  so  on.  An  important  aspect  of  the  environment  is 
the  Roster,  a table-like  environment  in  which  study  materials,  assignment  instructions,  student 
submissions,  and  instructor  feedback  are  integrated.  This  tool  allows  the  instructor  to  model  his 
course,  create  activities,  relate  assignments  and  study  materials.  It  also  helps  instructors  to  think  of 
effective  ways  to  communicate  with  the  students,  and  helps  them  to  set-up  the  structural 
communication  models. 


Blackboard 

Blackboard  is  the  commercial  electronic  learning  environment,  developed  by  Blackboard  Inc.,  which  is 
used  at  the  University  of  Amsterdam.  Within  Blackboard  each  course  is  delivered  through  a website. 
This  website  contains  all  the  content  and  tools  required  to  teach  a course.  Each  course  has  an  instructor 
who  manages  the  course  through  the  Instructor  Control  Panel. 

For  this  article  a description  of  the  functionalities  of  and  the  experiences  with  Courseinfo  v4.0  is 
given,  the  version  of  the  electronic  learning  environment  developed  by  Blackboard,  which  was  used 
during  the  best  practice  day  (see  last  section).  The  University  of  Amsterdam  has  been  using  this 
version  since  1999,  and  is  now  migrating  to  Blackboard  5,  Level  Three,  the  software  platform  that 
delivers  a course  management  system,  with  a customizable  institution-wide  portal  and  online 
communities  and  advanced  integration  tools  and  APIs  to  integrate  Blackboard  5 with  existing 
institutional  systems. 


Differences  and  similarities 


423 


Before  taking  a closer  look  at  the  actual  use  of  two  electronic  learning  environments,  lets  first  look  at 
some  general  characteristics.  Table  1 shows  the  Differences  and/or  similarities  of  two  of  them: 
Blackboard  and  TeleTOP. 


Table  1 . Differences  and/or  similarities  between  Blackboard  and  TeleTOP. 


Topic 

Blackboard 

TeleTOP 

Differences 

and/or 

similarities 

Instructors 

interface 

Web-based, 

The  instructor  is  able  to  place  materials 
through  a control  panel.  In  the  ‘view’ 
mode  he  is  able  to  see  how  the  students 
see  the  course. 

web-based  en  based  on  the 
principle:  what  you  see  is  what 
you  get.  An  instructor  is  able  to 
see  how  the  student  will  see  the 
site  immediately. 

Difference  in  that 
prospects  that  BB 
works  with  a 
control  panel,  TT 
with  what  you  see 
is  what  you  get. 

Instructor 

Tools 

All  course  administration  is  done 
through  the  Instructor  Control  Panel. 
This  area  is  only  available  to  users  with 
one  of  the  following  defined  roles  as 
instructor,  teaching  assistant,  grader  and 
system  administrator.  It  contains  the 
following  tools:  User  management; 
Communication  center;  Page  editors  to 
add  text  and  files;  Creating  assessment 
materials,  including  quizzes;  Managing 
the  online  gradebook;  Online  support 
and  documentation  and  Site 
management 

The  instructor  can  choose  his 
tools  from  the  menu,  per 
functionality  (i.e.  news,  or  roster) 
the  forms  will  able  the  instructor 
to  easily  put  materials  in  the 
environment.  The  tools  ai*e 
organized  around  the  following 
themes:  course  organization; 
communication;  group  work  and 
educational  resources. 

Student 

interface 

Web-based,  the  student  is  able  to  find 
materials  using  a Web  browser. 

Web-based,  the  student  is  able  to 
find  materials  using  a Web 
browser. 

Similar. 

Student 

Tools 

Web-based,  accessed  through  ‘My 
Blackboard’  (a  listing  of  all  the 
announcements,  courses,  and  calendar 
items  that  are  created  or  associated  with 
it  in  a central  location)  or  ‘course 
gateway  listing’  (a  listing  of  all  courses 
in  which  the  student  is  enrolled) 

and  contains  tools  for  Announcements  , 
course  information  , staff  information  , 

course  documents,  assignments, 

communication  and  external  links. 

Furthermore  student  tools  as  a digital 
dropbox,  edit  home  page  and  personal 
information,  course  calendar,  check 
grade,  manual,  tasks,  a resource  center , 
a course  map  are  available 

Information  tools  (news,  course 
info);  Communication  tools  (mail; 
FAQ,  discussion)  and  Resources 
(web-link,  MM,  quizzes,  polls, 
etc) 

Special  is  the  Roster:  a schedule 
tool  to  combine  al  resources  into 
web  or  face-to-face  sessions. 

Here,  the  course  activities  are 
described,  and  the  educational 
resources  are  connected  to  these 
activities.  For  example  the 
assignments  and  feedback  to  these 
assignments. 

The  student  tools,  and  also  the 
student  interface  is  similar  to  that 
of  the  instructors.  The  student  has 
fewer  rights  for  adding  materials, 
but  instructors  can  modify  these 
rights,  so  that  students  are  able  to 
for  example  add  resources. 

The  Roster  of 
TeleTOP  is 
typical  for  the 
environment 
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For  a more  detailed  overview  of  differences  and  similarities  between  TeleTOP  and  Blackboard,  as 
well  as  other  systems,  visit  the  site  that  Bruce  Landon  has  set  up:  http://www.c2t2.ca/landonline/ 


Use  of  electronic  learning  environments  in  education,  learning  from  experience. 

In  June  2001  a TeleTOP  Best  Practice  day  was  organized  at  the  University  of  Twente.  A similar  day 
was  organized  in  September  for  Blackboard  users  at  the  University  of  Amsterdam.  In  both  universities 
instructors  were  invited  to  present  their  experiences  with  the  system  in  their  education.  There  was  a 
broad  variety  in  instructors  and  courses,  as  instructors  of  different  programs  participated.  In  the 
presentations  of  the  use  of  the  electronic  learning  environments,  instructors  showed  how  the  system 
was  set-up,  which  functionalities  were  used,  what  goals  were  achieved  and  how  this  was  experienced 
in  the  courses.  The  participants  of  both  Best  Practice  days,  all  instructors  also  using  the  systems,  were 
invited  to  nominate  the  presentation  that  they  thought  was  the  best  example  of  good,  useful  and/or 
interesting  use. 

The  best  use  of  a TeleTOP  site,  selected  by  the  instructors  of  the  University  of  Twente,  was  a typical 
example  of  how  electronic  learning  environments  are  used  in  the  education  at  that  university.  Within 
the  course  the  environment  supports  different  students:  students  on  campus  and  distance  students  who 
attend  at  face-to-face  meetings  ever^^  other  Friday.  Figure  1 shows  a screen  dump  of  the  TeleTOP 
environment. 
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Figure  1.  View  of  the  Instructional  Design  course. 

The  instructor  has  used  the  environment  to  give  a detailed  overview  of  the  schedule.  A limited  number 
of  assignments  were  part  of  the  course.  The  students  could  find  these  rights  from  the  start  of  the  course 
and  were  flexible  in  the  submission  of  the  assignments,  though  they  needed  it  to  submit  for  a fixed 
date.  A part  of  these  assignments  were  test  in  which  the  scoring  and  feedback  was  organized  and  given 
by  he  system.  The  instructors  who  liked  this  example,  argued  that  the  instructor  managed  to  make  use 
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of  ICT  in  his  education  to  make  in  as  much  flexible  for  students  to  participate,  but  also  cared  for  his 
time,  so  it  would  not  get  out  of  hand. 


The  Blackboard  course  that  was  evaluated  as  best  practice  at  the  University  of  Amsterdam  was  the 
course  “Design  of  modem  foreign  languages”,  a course  for  students  from  the  Graduate  School  of 
Teaching  and  Learning,  in  which  the  students  have  to  make  a design  plan  for  a language  course. 
Students  participating  in  this  course  are  regular  on-campus  students.  Functionalities  of  the  Blackboard 
learning  environment  that  are  used  by  the  instructor  to  set  up  the  course  are  Announcements,  Course 
Information,  Staff  Information,  Course  Documents,  Assignments,  Communication  and  External  Links. 
Students  have  their  own  tools  as  described  in  Table  1.  This  can  be  seen  in  Figure  2. 
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Figure  2.  View  of  the  Design  of  modem  foreign  languages  course. 

The  options  course  information  and  assignments  are  the  two  options  that  are  used  most  extensively. 
Students  are  informed  about  the  course  in  general,  including  the  goals,  the  schedule,  and  the  way  of 
assessing  and  grading  the  products  of  the  student.  The  students  have  to  do  seven  separate  assignments. 
These  assignments  lead  to  a ‘master  test’,  the  final  assignment.  In  this  assignment  the  students  have  to 
apply  all  knowledge  and  skills  that  they  have  used  in  the  previous  assignments. 

The  way  the  instructor  designed  the  course  and  the  way  the  functionalities  were  used  were  not 
evaluated  as  extremely  original,  but  as  very  useful  and  consistent  for  the  students.  The  instructor 
showed  that  by  using  the  course  in  Blackboard  next  to  the  lectures  students  can  benefit  from  an  added 
value,  especially  related  to  time  and  place  flexibility. 


Conclusions 

It  is  interesting  to  see  what  the  institutions  in  the  Netherlands  that  have  decided  to  start  working  with 
electronic  learning  environments  experience  working  with  these  systems.  On  one  hand  institutions 
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decide  to  organise  a full  institution-wide  implementation  of  these  systems.  On  the  other  had  we  see 
that  instructors  are  still  struggling  with  the  question  how  they  should  use  these  new  flexible  tools  and 
how  this  will  change  their  teaching.  They  are  concerned  with  how  they  can  use  the  opportunities  and 
functionalities  that  will  make  their  teaching  and  their  students’  learning  more  flexible,  efficient  and 
enriched.  In  those  institutions  were  a clear  innovation  goal  has  been  set,  these  questions  are  more 
easier  to  answer  than  when  the  instructors  have  to  deal  with  this  question  themselves.  However, 
despite  of  the  differences  in  electronic  learning  environments  we  see  that  instructors  are  creative, 
eager  to  learn  how  to  use  the  tools  and  like  the  new  possibilities.  It  looks  like  that  best  practice  days  as 
we  have  described  in  this  paper  are  very  fruitful  opportunities  for  instructors  to  exchange  ideas.  Most 
important  outcomes  of  the  recent  best  practice  days  at  the  University  of  Twente  and  the  University  of 
Amsterdam  are: 

Make  education  as  flexible  (in  time  and  place)  for  students  as  possible,  but  be  aware  of  time 
consuming  activities  that  cause  dissatisfaction  and  fhistration. 

Let  students  benefit  from  the  added  value  of  an  electronic  learning  environment,  use 
functionalities  that  are  useful  for  the  students  and  use  them  consistently. 

Further  research  should  clarify  if,  and  how,  best  practice  days  focussed  on  the  exchange  of 
experiences  with  an  electronic  learning  environment  have  effect  on  the  positive  use  of  electronic 
learning  environments  in  current  and  future  education. 
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A Multimedia  Web-Supplement  to  a Course  on  Beethoven 


Anthony  Paul  De  Ritis,  Ph.D. 

Department  of  Music  and  Multimedia  Studies 
Northeastern  University,  Boston,  MA,  USA 
a.deritis@neu.edu 

Abstract:  www.beethoven.neu.edu  is  the  result  of  a faculty  initiative  sponsored  by  Northeastern 
University’s  College  of  Arts  & Sciences,  with  a goal  of  understanding  better  the  implications  of 
technology  upon  teaching.  This  website  was  featured  in  the  "Learning"  section  of  the  Boston  Globe  on 
July  15,  2001,  and  may  be  used  as  a case  study  for  faculty,  staff,  and  administration  interested  in 
developing  web-based  supplements  to  their  classes.  This  session  describes  the  content  and  design  of 
the  Beethoven  web-supplement,  its  use  of  e-leaming  technologies,  and  an  evaluation  and  assessment  of 
its  effectiveness. 


Issues  with  E-learning 

The  Beethoven  web-supplement  attempted  to  grapple  with  a number  of  current  e-leaming  problems  and  issues,  a few  of 
which  are  stated  below: 

• How  can  we  better  understand  the  implications  of  technology  upon  teaching? 

• Do  pre-  and  post-  surveys  shed  light  on  students  computing  habits?  Their  acceptance  of  online  learning  methods? 

• Do  online  pre  -assessment  tests  enhance  a students  understanding  of  course  material? 

• Does  streaming  audio  of  musical  examples  relieve  the  stress  on  our  university  media  labs  costs  and  resources? 

• Does  an  online  gradebook,  and  the  ability  for  students  to  monitor  their  grade  progress  throughout  a course, 
diminish  their  grade  anxieties? 

• How  much  do  students  really  know  about  multimedia  plug-ins  and  are  they  really  interested  in  using  the  Internet  as 
an  educational  learning  environment? 


Description 

The  web-supplement  to  MUS  1 145:  Beethoven,  was  the  result  of  a collaborative  effort  between  Professor  Anthony  De 
Ritis,  his  undergraduate  assistant,  Aris  Amanatidis,  the  Northeastern  University  EdTech  Center,  and  the  College  of  Arts 
and  Sciences  and  Northeastern  University.  Originally  developed  in  1999,  this  course  has  been  offered  at  the  university 
on  three  separate  occasions.  The  College  of  Arts  & Sciences  offered  faculty  an  opportunity  to  develop  web-supplements 
through  a Faculty  Development  Grant  that  included  about  $7,500  of  funding.  This  included  a one-course  stipend,  a 
laptop  computer,  a digital  camera  and  scanner. 

This  Beethoven  web-supplement  was  developed  partly  as  an  on-line  companion  to  Maynard  Solomon’s  book  on 
Beethoven,  the  required  text  for  the  course.  Each  student  purchased  this  text,  and  was  expected  to  take  on-line  quizzes - 
one  quiz  per  chapter  (now  often  called  “pre-assessment  tests”)  - that  required  the  student  to  read  material  for  class 
discussions  prior  to  attending  class.  At  that  time,  through  Web  Course  in  a Box,  all  quizzes  were  automatically  graded. 


This  website  consists  of: 

• Pre-  and  post-  surveys  for  evaluation  and  assessment 

• Syllabus 

• Lecture  schedule 

• Description  of  in-class  lectures,  including  class  business,  topics  to  be  covered,  due  dates  for  readings  and 
assignments,  and  a glossary  of  important  terms. 

• Online  readiness  assessment  quizzes 

• An  asynchronous  discussion  forum 

• An  online  gradebook 
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• Maps 

• Required  listening  in  both  streaming  and  mp3  formats 

• An  index  of  Beethoven’s  music  compositions 
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Example  lecture  schedule 


Outcome 

This  Beethoven  web-supplement  was  featured  in  the  “Learning”  section  of  the  Boston  Globe  in  July  2001.  This  article, 
titled  “The  ELearning  Road”  chronicled  how  universities  need  to  be  aware  of  the  “disruptive  technology”  of  the 
Internet  and  the  web  in  order  keep  their  student  “clientele”. 

Clearly,  students  felt  a personal  pressure  to  be  prepared  for  class.  Because  their  chapter  quizzes,  after  they  were 
automatically  graded,  also  included  a time -stamp  that  was  e-mailed  to  me,  I was  able  to  monitor  who  was  prepared  for 
class,  and  who  was  not.  I found  that  my  class  lectures  were  more  interesting,  in-depth  and  had  participation  from  a 
larger  portion  of  the  student  body  (there  are  sixty  students  in  each  of  the  Beethoven  classes  offered  with  this  web- 
supplement). 

Survey  data  showed  that  over  80%  of  our  average  Northeastern  students  owned  a computer  with  Internet  access,  and 
that  some  students  actually  upgraded  their  personal  home  computer  systems  during  the  course  in  order  to  take  advantage 
of  the  web-based  services.  In  particular,  students  were  very  please  to  be  able  to  have  access  to  all  of  the  musical 
examples  required  for  listening  at  their  disposal  24/7  — instead  of  having  to  be  held  to  the  media  library  hours.  In 
addition,  I was  able  to  customize  an  exact  listening  list  that  I felt  important  for  each  student  (often  these  musical 
examples  do  not  appear  on  a single  CD  purchase).  Web-based  streaming  audio  allows  for  the  easy  organization  of 
supplementary  musical  examples,  which  saves  time  and  money  for  both  the  student  and  faculty. 


Relevance 


www.beethoven.neu.edu  is  a case  study  that  has  be  used  for  all  university  faculty,  staff,  and  administration,  for  the 
impact  of  e -learning  technologies  on  education.  The  success  of  this  web-supplement  has  led  the  way  for  additional  grant 
opportunities  at  Northeastern  University  up  to  $25,000,  and  has  stimulated  the  development  of  hybrid-courses  which 
consist  of  courses  offered  half  “on-line”  and  half  “in-class”.  In  addition.  Northeastern  University  has  now  examined  a 
variety  of  possibilities  of  offering  complete  courses  and  degree  programs  online. 
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There  Is  Still  Hope  for  ICT  in  Flanders  Fields. 

ICT  in  Education:  the  Use,  Benefits,  Barriers  and  Expectations  as 
Perceived  by  Educators  at  Flemish  Universities. 


Eric  De  Vos  & Katie  Goeman 
Vrije  Universiteit  Brussel  (Belgium) 
Faculty  of  Arts 

Department  of  Communication  Sciences 
Eric.De.Vos@vub.ac.be 
Katie.Goeman@vub.ac.be 

Nathalie  Blocry 
Faculty  of  Applied  Sciences 
Department  of  Computer  Sciences 
Nathalie.Blocrv@vub.ac.be 


Abstract:  In  this  paper  we  present  the  main  research  findings  of  a large  scale  investigation 
about  the  state  of  the  art  in  ICT  use  for  educational  purposes  in  Belgium,  and  specifically  the 
current  situation  in  Flemish  universities.  The  main  focus  is  on  user  characteristics  and  the 
attitudes  of  educators.  Questions  asked  include:  How  do  educators  use  ICT  in  their  teaching 
practices?  How  do  they  perceive  the  support  given  by  faculties  within  the  famework  of 
overall  university  policy?  Which  problems  and  barriers  do  they  encounter?  By  means  of  an 
online  survey  we  collected  opinions  in  order  to  unravel  which  factors  play  a role  in  the 
decision  to  adopt  ICT.  This  investigation  reveals  how  ICT  is  used  in  a non  innovative  way  in 
Flemish  universities. 


Theoretical  Background 

Educators  play  a crucial  role  in  the  adoption  and  implementation  of  new  technologies  in  universities.  New 
means  of  transmitting  knowledge  are  only  used  if  they  offer  clear  advantages  with  respect  to  the  old  familiar 
teaching  methods  and  enhance  flexibility,  efficiency  and  effectiveness  (Collis  et  al.,  1999),  means  to  cope  with 
information  sources  (Veen  et  al.,  1999),  ways  to  develop  individual  learning  paths  (Gehl,  1996),  better  student- 
educator  communication  (Twigg,  1995)  and  responsiveness  (Hatfield,  1997).  Previous  research  indicates  that 
teachers  reveal  a variety  of  attitudes  towards  the  introduction  of  ICT: 


Attitude 

Short  description 

Neutral  itari  an 

Usually  users  of  ICT  as  a tool 

Booster 

Unconditional  believers  and  pioneers 

Oppositional 

Technophobe,  fearing  encroaching  on  human  values 

Sceptic 

Those  waiting  for  clear  results  before  engaging 

Transformationalist 

Those  considering  changes  as  necessary  but  studying  ICT  carefully 
before  its  implementation 

Table  1:  Categories  of  educators’  attitudes  (Source:  Evans,  2000) 

The  fear  of  the  unknown,  the  tendency  to  consider  the  existing  situation  as  a good  one,  a feeling  of  solitude 
caused  by  a lack  of  support  and  knowledge,  and  a reluctance  towards  self  reflection  can  influence  these 
intermixed  attitudes  (Hagner,  2000).  Collis  et  al.  0999)  developed  the  4E  model  to  predict  the  chance  an 
educator  will  adopt  ICT,  based  on  the  following  concepts:  (1)  environment,  (2)  (perceived)  educational 
effectiveness,  (3)  ease  of  use  and  (4)  (personal)  engagement. 
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Evidently,  institutional  policy  plays  also  an  important  role  in  the  motivation  of  the  educator.  The  university 
approach  can  be  either  top-down  or  bottom-up  and  based  on  different  points  of  view  (Collis  et  al.,  1999).  We 
quote  (1)  proactive  (driven  by  image  building  and  market  share),  (2)  reactive  (driven  by  response  to  demand),  (3) 
transformative  (student  centred)  and  (4)  speculative  (driven  by  hope  in  improvement  efficiency  and/or 
effectiveness)  professional  contexts. 

The  barriers  opposing  the  introduction  of  ICT  in  higher  education  were  thoroughly  investigated  in  The 
Netherlands.  According  to  Veen  et  al.  (1999)  the  most  relevant  ones,  in  order  of  importance,  are:  a lack  of  time, 
skills,  adequate  infrastructure,  support  and  rewards. 

Research  objectives  and  methodology 

Besides  general  computer  use  and  training,  the  following  themes  were  studied  at  six  Flemish  universities:  the 
perception  of  university  and  faculty  policy,  educators*  types  of  ICT  use  and  attitudes  towards  ICT 
implementation,  including  perceived  advantages  and  benefits.  A self-administered  questionnaire  was  delivered 
via  a database  driven  website  to  all  teachers  in  the  humanities  faculties  (N=  2244).  They  belonged  to  the 
departments  of  arts,  law,  economy,  applied  economy,  business,  political  and  social  sciences,  communication, 
psychology,  educational  sciences  and  theology. 


Results 

Profile  of  the  respondents 

442  candidates  completely  filled  in  the  questionnaire  (approximately  20%  response).  Two  thirds  of  the 
respondents  are  teaching  assistants,  one  third  lecturers.  61%  of  the  respondents  are  male,  39%  female.  It  is 
noteworthy  that  the  male/female  proportion  is  about  fifty-fifty  up  to  the  age  of  forty.  The  age  distribution  is  as 
follows:  20-30  51%,  30-40  21%,  40-50  15%,  older  than  50  13%.  Female  respondents  are  almost  absent  in  the 
highest  age  category. 


ICT  skills  and  training 

A very  small  fraction  of  the  respondents  is  not  able  to  handle  computers,  operation  systems  and  ^neral 
applications  like  word  processing.  Similarly,  e-mail,  digital  information  systems  and  internet  applications  are 
unknown  to  only  a very  small  group  (respectively  4.3%,  3.2%  and  5%).  The  overwhelming  success  of  the 
internet  is  not  induced  by  educational  puposes  but  used  for  research.  CD  Rom  use  is  not  frequent  (as  expected), 
except  in  situations  where  it  is  imposed  by  research.  The  consultation  of  electronic  teaching  media  (databases, 
courseware,  communication  facilities,  etc.)  is  not  common.  Merely  8.9%  of  the  teachers  use  them  daily,  while 
76%  almost  never  or  seldom  get  in  touch  with  these  media. 


Internet  resources 

CD  Rom 

Electronic  teaching  material 

almost  never 

0.9% 

28.0% 

41.9% 

from  time  to  time 

7.2% 

44.9% 

34.2% 

weekly 

20.6% 

18.2% 

14.7% 

daily 

71.3% 

8.9% 

8.9% 

Table  2:  Use  of  internet  resources,  CD-ROM  and  electronic  teaching  material 

These  ICT  skills  are  mainly  acquired  via  personal  study  and  practical  experience.  Mainly  younger  educators  had 
specific  computer  training  at  school  oruniversity.  On  encountering  problems  they  tend  to  rely  on  colleagues. 


Influence  of  university  and  faculty 

Both  the  general  use  of  computers  and  e-mail  is  heavily  stimulated  by  the  university  or  faculty  (73,5%  and  76%). 
On  the  other  hand,  educators  do  not  report  receiving  much  support  when  experimenting  with  ICT  for  educational 
goals.  Significant  differences  between  universities  exist. 


General  computer  use 

Use  of  e-mail 

Use  of  ICT  for  educational  goals 

no 

4.1% 

1.4% 

26.4% 

some 

9.3% 

8.6% 

35.4% 

yes 

73.5% 

76.0% 

36.6% 

imposed 

13.1% 

14.0% 

1.6% 

Table  3:  Stimulation  by  the  university  or  faculty 


ICT  policy 

Although  all  Flemish  universities  have  official  ICT  policy  plans  and  coordinators,  only  a part  of  the  educators 
are  well  informed  about  these  initiatives.  Clearly,  a lack  of  communication  lies  behind  this. 


Docs  the  university  or 
ICT  policy  plan? 

faculty  have  an 

Is  there  an  ICT  coordinator  or 
manager? 

yes 

33.8% 

yes 

60.2% 

no 

5.4% 

no 

4.8% 

don’t  know 

60.8% 

don’t  know 

35.0% 

Table  4:  Policy  plan  and  ICT  coordinators 


Educational  use  of  ICT 

One  third  of  the  educators  use  ICT  for  educational  purposes.  Table  5 shows  the  present  use  and  objectives: 


Use 

Objectives 

not  used 

32.4% 

eliminate  shortcomings 

11.0% 

as  an  help  tool 

33.7% 

extend  exercises 

29.7% 

in  the  learning  process 

8.3% 

substitution  for  exercises 

10.7% 

tool  and  learning  process 

23.1% 

extend  seminars 

22.4% 

don’t  know 

2.5% 

substitution  for  seminars 

6.6% 

extend  courses 

26.2% 

substitution  for  courses 

8.3% 

other 

13.4% 

Table  5:  Present  use  and  objectives 

The  left  half  of  table  5 must  be  interpreted  with  caution  because  it  appears  that  many  respondents  are  not  aware 
of  the  distinction  between  the  use  of  ICT  as  an  help  tool  and  its  capacity  as  a learning  support. 


Motivation 

When  considering  the  motivations  teachers  rate  as  important  and  very  important  for  adopting  ICT  in  teaching 
environments,  they  mainly  want  to  increase  the  effectivity  of  their  education.  Secondly,  they  intend  to  promote 
autonomous  study,  enhance  flexibility  and  create  new  educational  contexts  by  means  of  ICT.  Thirdly,  they  are 
motivated  by  enhancing  the  students’  motivation  and  stimulating  contacts  between  student  and  educator. 


432 


Figure  1:  Motivation  for  ICT  use 


Desired  use  in  the  future 


Substituting  traditional  classes  is  very  unpopular.  A clear  majority  of  the  respondents  do  not  want  to  replace 
lectures,  seminars  and  exercises  by  ICT  supported  activitities.  They  are  more  likely  to  use  ICT  to  extend  these 
teaching  activities.  As  far  as  testing  and  remedying  deficiencies  are  concerned  it  can  be  noted  that  about  half  the 
educators  are  not  willing  to  apply  ICT. 
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Barriers 


When  asked  to  point  out  the  barriers  for  adopting  ICT  in  teaching  environments,  the  educators  reported  the 
following  problems  (in  order  of  importance): 

1.  lack  of  time; 

2.  fear  of  loosing  personal  contacts  with  students  (wich  surprisingly  appears  not  te  be  age  related) 

3.  lack  of  skills,  manpower  support  and  adequate  infrastructure; 

4.  problems  caused  by  technical  items  and  software; 

5.  lack  of  interest  of  students; 

6.  high  costs; 

7.  introduction  of  ICT  felt  as  a bad  educational  evolution 

8.  lack  of  university  policy. 

Only  items  (2),  (5)  and  (7)  refer  to  pedagogical  aspects  of  computer  supported  education,  the  other  are  related  to 
personal  and  materialistic  factors. 


Figure  3:  Perceived  disadvantages  of  ICT  (barriers) 


Conclusions 

This  study  shows  that  ICT  is  used  in  a rather  non-innovative  way  in  Flemish  universities.  The  advantages  of 
different  technologies  are  well  understood  as  far  as  communication,  information  gathering  and  distribution  are 
concerned.  But  a vast  majority  of  the  educators  surveyed  do  not  grasp  the  rich  possibilities  ICT  can  offer  in 
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supporting  or  improving  their  teaching  practices-.  Moreover,  the  teachers  investigated  perceive  many  practical 
and  technical  barriers  for  a succesful  integration  of  ICT  in  educational  settings.  Universities  should  play  a more 
important  role  in  offering  relevant  information  and  stimulating  educators,  not  only  in  a passive  but  in  an  active 
way.  Possible  “solutions”  include:  awareness  raising,  appropriate  training  and  support  - both  technical  and 
pedagogical  - and  new,  strong  policies  with  a long-term  vis  ion  on  educational  renewal  rather  than  a policy  based 
on  ad  hoc  decisions  and  actions. 

There  is  still  hope  for  Flanders  Fields.  Small  groups  of  innovators  are  disseminating  their  knowledge  and 
positive  experiences  to  their  colleagues  and  helping  to  influence  university  policymakers. 
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Abstract:  Recent  brain-based  research  has  generated  renewed  interest  in  pedagogical  practices 
of  the  classroom.  This  development  has  drawn  attention  to  lesson  plan  development  since  the 
development  of  lesson  plans  guides  the  learning  process  of  students.  To  address  the  “whole- 
brain”  in  the  instructional  process,  technology  application  should  be  incorporated  in  lesson  plan 
design  since  it  can  effectively  activate  attributes  of  both  the  left  and  right  hemispheres  of  the 
brain.  This  paper  will  provide  a brief  review  of  brain-based  research  and  link  it  to  pedagogical 
practices  in  the  classroom  through  the  design  and  implementation  of  technology -integrated 
lesson  plans  to  optimize  learning.  Following  this  discussion,  the  authors  will  discuss  how  the 
University  of  Houston-Clear  Lake  (UHCL)  Preparing  Tomorrow’s  Teachers  to  Use  Technology 
(PT3)  Project  integrates  technology -integrated  lesson  plans  based  on  brain  research  into  the 
UHCL  teacher  preparation  program  to  provide  quality  learning. 


Introduction 

Brain-based  theory  indicates  that  learning  is  a process  of  active  constmction  of  meaningful  ideas  by 
the  learner  in  an  enriched  environment  (Lowery,  1998;  Wolfe  & Brandt,  1998).  Dake  (1977)  asserts  that 
traditional  pedagogical  avenues  address  functions  in  the  left  hemisphere  of  the  brain  (e.g.,  verbal  reasoning, 
logic,  analysis)  and  not  the  attributes  of  the  right  hemisphere  (e.g.,  intuition  and  simultaneity).  Lowery  (1998) 
concurs,  identifying  teaching  methods  such  as  lecture,  demonstration  and  textbook  narrative  as  only  minimally 
impacting  the  many  avenues  to  the  brain  and  learning.  Teachers  should  use  multiple  methods  of  instruction  to 
stimulate  both  brain  hemispheres  to  optimize  student  learning.  Technology  applications  have  this  capability.  In 
an  interview  by  Tell  (2000),  Healy  warns  that  educators  are  dangerously  close  to  limiting  the  use  of  technology 
on  low-level  content  when  research  indicates  that  technology  can  drastically  impact  teaching  and  learning, 
especially  in  the  development  of  higher-order  thinking  skills.  An  effective  lesson  design,  which  can 
incorporate  technology  usage,  helps  students  to  explore  ideas,  acquire  and  synthesize  information,  and  frame 
and  solve  problems  (Seamon,  1999). 

Brain  Based  Research  And  Its  Implications  On  Classroom  Practice 


Brain-based  research  on  teaching  and  learning  impacts  not  only  teachers’  delivery  of  instruction,  but 
also  the  students’  acquisition  of  knowledge  and  skills.  Researchers  (Caine  & Caine,  1998;  Wolfe  & Brandt, 
1998)  caution  educators  not  to  use  brain  research  as  a template  of  prescriptive  teaching  strategies,  but  to  use 
the  general  findings  as  guide  to  direct  teaching  and  learning  practices.  Brain  research  provides  educators  with  a 
framework  for  developing  optimal  situations  for  meaningful  learning.  This  framework  identifies  general 
concepts  about  quality  teaching  and  learning  activities  and  situations  to  ensure  student  learning.  Chief  among 
the  concepts  is  the  notion  of  having  an  "enriched”  environment.  Enriched  environments  required  learners  to 
engage  in  a variety  of  inquiries  within  powerful  content  contexts,  thereby  increasing  the  probability  that 
students  will  construct  meaningful  knowledge  and  thinking  capabilities  (Lowery,  1998).  The  environment  in 
which  a brain  operates  determines  to  a large  degree  the  functioning  ability  of  that  brain  (Wolfe  & Brandt, 
1998).  Because  the  brain  strives  to  “make  meaning”  from  its  surroundings,  a classroom  with  many  stimuli 
gives  students  several  opportunities  for  comprehension  of  knowledge  and  skills  at  higher  levels  of 
understanding. 

The  notion  that  the  brain  changes  physiologically  through  the  learning  process  is  supported  by  Wolfe 
and  Brandt  (1998),  who  argue  that  educators  must  develop  an  understanding  of  the  brain  and  its  processes  to 
support  meaningful  teaching  and  learning,  while  disregarding  the  fads  and  ambiguous  programs.  To  do  this, 
educators  need  to  stimulate  different  parts  of  the  brain  (Perry,  2000).  Neural  systems  when  exposed  to  only  one 
form  of  stimuli  fatigue  quickly  and  need  to  rest.  For  example,  after  only  a few  minutes  of  lecture,  the  brain 
seeks  other  stimuli.  When  constructing  knowledge,  the  brain  requires  data  that  can  only  be  obtained  through 


sensory  perceptions.  Show-and-tell  teaching  methods,  such  as  lectures  and  textbook  narratives,  activate  only  a 
few  of  the  many  possible  avenues  to  the  brain  (Lowery,  1998).  The  proper  use  of  technology  affords  educators 
the  opportunities  to  prescribe  learning  experiences  in  which  students  can  experience  multiple  cultures,  research 
planets  in  solar  system,  and  converse  with  students  on  another  continent,  in  a non-threatening  way.  The  use  of 
technology  provides  teachers  with  the  ability  to  find  and  provide  connections  between  new  content  and  a 
variety  of  contexts,  thereby  giving  meaning  to  material  and  concepts  to  be  learned,  since  the  brain  does  not 
effectively  absorb  meaningless  data  (Wolfe  & Brandt,  1998). 

Several  studies  (Carr  & Harris,  2001;  Haigh,  1981;  Wolfe,  1998)  indicate  that  quality  lesson  planning 
has  a beneficial  impact  on  the  teaching  and  learning  in  the  classroom.  A lesson  plan  provides  a sequential 
structure  describing  what  and  how  specific  information  will  be  learned  (Cruickshank,  Bainer,  & Metcalf, 
1995).  Educators  should  construct  lesson  plans  that  incorporate  technology  applications  that  can  stimulate 
different  parts  of  the  learner’s  brain.  Perry  (2000)  argues  that  optimal  learning  in  a classroom  occurs  when 
teachers  are  able  to  activate  the  student’s  entire  brain. 

UHCL  PT3  Project 

The  central  focus  of  the  UHCL  PT3  Project,  funded  by  the  U.S.  Department  of  Education,  is  to  create 
an  educational  process  in  preparing  teacher  candidates  to  develop  and  deliver  technology-integrated  lesson 
plans  so  classroom  students  (Pre -Kindergarten  to  12^*^  grade)  use  technology  in  demonstrating  mastery  of 
lesson  plan  objectives.  In  its  training  methodology,  the  UHCL  PT3  Project  addresses  brain -based  researchers’ 
recommendations  of  activating  both  brain  hemispheres  by  using  a two -pronged  approach.  First,  the  teacher 
candidates  experience  left  and  right  brain  stimulations  through  the  training  module  activities.  Teacher 
candidates  work  left-brained  attributes  in  “pointing  and  clicking”  the  various  links  on  the  UHCL  PT3  Website 
rhttp://pt3.cl.uh.eduk  but  engage  in  right  brain  development  when  evaluating  lesson  plans  posted  on  the  PT3 
Website.  The  second  ‘prong’  of  the  approach  is  helping  teacher  candidates  identify  software  applications  that 
can  stimulate  left  and  right  brain  attributes  of  classroom  students.  For  example,  software  applications  such  as 
Inspiration  and  Kid  Pix  can  encourage  higher  level  thinking  processes  such  as  brainstorming,  concept 
mapping,  and  webbing;  but  can  also  be  used  for  math  and  spelling  drill  activities.  After  developing  the 
technology -integrated  lesson  plans,  teacher  candidates  deliver  these  lesson  plans  in  their  respective  grade  level 
classrooms  during  their  internship  assignments.  These  classroom  instructions  allow  teacher  candidates  to 
experience  first-hand  the  educational  inplications  of  brain  based  research,  affecting  the  classroom  students’ 
learning  as  well  as  their  own. 
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Abstract 

The  development  of  a set  of  educational  web  pages  is  a complex  task,  because  it  involves 
multiple  abilities:  to  be  a good  content  author,  a graphist,  an  ergonomist,  following  the  advice 
of  the  webmaster.  Moreover,  this  task  has  to  be  redone  every  time  an  author  (or  the  academic 
site  in  which  his  pages  are  included)  change  the  graphical  or  the  navigational  appearance. 
Last,  you  cannot  use  the  advanced  mechanisms  of  HTML  (which  can  help  in  managing  styles 
and  content),  since  they  are  not  fully  supported  by  most  frequently  used  web  browsers.  Our 
goal  is  to  propose  a language  to  be  used  by  professors,  helping  them  to  structure  the  content 
of  their  hypertext  pedagogical  documents,  and  to  free  them  from  all  the  other  aspects.  The 
definition  of  a set  of  web  pages  is  done  by  describing  both  a structural  model  (using  our 
HDSL  language),  and  a graphical  model  (using  completely  normal  HTML).  Using  this  pair 
of  models,  our  HDSL  engine  will  build  automatically  the  XSL  conversion  sheets  able  to  con- 
vert any  of  the  documents  built  using  the  structural  model  into  (for  example)  web  pages  using 
the  most  currently  accepted  HTML.  HDSL  also  offers  capabilities  of  inheritance  between 
structural  models,  which  allows  a professor  responsible  of  a whole  pedagogical  web  site  to 
define  a look  and  feel  for  it,  and  derive  it  according  to  the  needs  of  the  various  authors  writing 
the  hypertext  supports  of  their  courses.  Moreover,  the  HDSL  engine  is  able  to  accept  presen- 
tation models  in  specific  languages  other  than  HTML,  such  as  PDF,  WML,.. .starting  from 
the  content  of  a single  document. 


Context 

Developing  a coherent  set  of  web  pages  to  produce  a pedagogical  document  is  a complex  task,  since  the  pages  can 
usually  be  described  using  multiple  approaches.  One  can,  for  example,  organize  them  by  dividing  the  main  sub- 
ject into  sections  (introduction,  chapters,  bibliography,  index,...),  or  classifying  them  by  kind  of  information  (pic- 
tures, sounds,  icons,...),  by  author.  This  list  is  not  limited,  and  you  have  often  to  use  many  of  these  classification 
criteria  simultaneously.  This  leads  you  to  handle  a great  number  of  heavily  interdependent  documents,  like  a real 
software  project  manager.  But  the  content  of  web  pages  has  a much  more  dynamic  structure  than  the  source  files 
of  a software  project:  to  be  pertinent,  the  content  of  a pedagogical  document  has  to  be  frequently  updated. 

Moreover,  one  has  also  to  give  a coherent  look  and  feel  to  this  complex  set  of  documents.  This  aspect  is  of  great 
importance  in  a pedagogical  context,  since  it  allows  students  not  to  be  bothered  by  presentation  or  navigational 
considerations.  Unfortunately,  one  has  also  to  do  with  content  authors  who  can  either  be  HTML  fluent,  and  these 
will  take  a lot  of  time  to  elaborate  their  own  look  and  feel,  or  on  the  contrary  they  may  be  completely  focused  on 
the  content  of  their  document,  and  they  will  deliver  them  in  various  formats. 

Another  problem  is  the  ability  to  describe  the  page  content  semantics  for  search  tools,  either  your  own,  or  com- 
mercial ones.  The  keywords  HTML  offers  with  its  meta  tags  are  also  too  simplistic,  since  they  lack  the  ability  to 
describe  the  context  in  which  they  are  used,  and  are  attached  to  the  whole  document,  when  we  need  a fine  grain 
semantic  description.  A powerful  and  semantically-based  search  tool  is  a fundamental  mechanism  for  helping  stu- 
dents to  browse  your  document,  and  find  the  definition  or  explanation  they  are  looking  for. 

The  cost  of  managing  a huge  set  of  HTML  documents  could  be  greatly  reduced  by  the  use  of  advanced  tools 
available  in  HTML  4 (Ragget  1999).  For  example,  CSS  styles  sheets  can  be  used  to  insure  a homogenous  look, 
navigation  toolbar  and  menus  located  in  frames  to  standardize  the  interface  presented  to  the  internet  reader. 
Unfortunately,  most  of  these  advanced  tools  are  not  universally  supported  by  web  browsers,  or  by  other  medium 
specific  presentation  languages  (PDF,  WML,  restricted  HTML). 
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Almost  all  these  problems  can  be  solved  by  using  our  Hypertext  Document  Structuring  Language,  HDSL,  based 
on  XML  (Bray,  2000),  that  allows  to  describe  the  structure  of  families  of  hypertext  documents,  independently  of 
their  presentation  and  the  medium  on  which  they  are  displayed.  That  language  allows  also  to  add  semantics  to  the 
items  of  the  document,  either  on  their  content,  or  on  their  usage  (navigation  toolbar,  menu,...).  Separation  of  con- 
tents and  presentation  in  HDSL  give  the  ability  to  describe  the  look  and  feel  of  the  pedagogical  document  which  is 
different  from  mixing  them  with  contents,  as  done  with  HTML. 

To  achieve  this  goal,  we  propose  a mechanism  based  on  pairs  of  documents.  For  every  subset  of  the  pedagogical 
document  (course,  chapter,  section,...),  which  has  its  own  look  and  feel,  the  content  author  will  supply  a model 
using  HDSL,  which  describes  its  different  parts  (required  or  optional,  describing  the  contents  or  the  relations  with 
other  documents,...).  According  to  this  structural  model,  the  graphist  will  supply  a presentation  model,  which 
organizes  the  look  and  usage  of  theses  parts.  This  presentation  model  can  use  various  presentation  languages, 
standard  HTML,  reduced  HTML  (for  text-only  browsers,  low-bandwidth  clients,...),  WML,  PDF,  or  others, 
according  to  the  medium  used  to  display  the  document.  Our  HDSL  engine  will  take  all  these  models,  and  gener- 
ates automatically  as  many  XSL  stylesheets  as  needed  to  produce  «displayable»  forms  of  the  concrete  HDSL  doc- 
ument written  by  content  authors,  for  every  needed  medium.  It  allows  to  use  a unique  version  of  the  document  that 
defines  the  content,  which  is  of  a great  importance  to  insure  the  coherence  of  the  various  displayable  forms. 

The  solutions  XML  currently  offers,  such  as  pure  XSL  sheets  (Clark  1999),  has  multiple  drawbacks:  you  have  to 
fully  master  XSL,  to  write  a XSL  sheet  for  every  presentation  medium,  for  every  subset  of  your  pedagogical  doc- 
ument. Moreover,  this  XSL  writing  task  must  be  redone  every  time  you  change  the  look  and  feel.  And  this  way  of 
producing  web  pages  doesn’t  allow  re-utilization  or  genericity  which  our  tool  includes,  nor  any  possibility  for 
handling  interrelations  between  documents.  It  doesn’t  provide  either  any  may  to  manage  the  versioning  of  the 
document  the  HDSL  engine  supports,  and  which  is  of  great  help  in  managing  the  production  of  many  contents 
author. 

Related  Work 

The  WebML  language  (www.oasis-open.org/cover/webML.html)  is  a notation  for  specifying  complex  Web  sites 
at  the  conceptual  level.  It  allows  the  description  of  navigation  and  composition  abstractions,  which  are  assembled 
into  pages  and  interconnected  by  links.  WebML  is  devoted  to  the  construction  of  web  sites  based  on  very  dynamic 
information  fetched  from  multiple  databases,  and  linked  by  complex  relationship,  described  using  E/R  model, 
ODMG  object-oriented  model  in  a structural  model.  The  presentation  model  defines  the  appearance  of  pages, 
independently  of  the  output  devices  and  the  rendition  language,  like  HDSL,  except  for  the  data  described  in  the 
structural  model.  The  application  domain  of  WebML  is  rather  e-commerce  oriented  sites  than  educational  or  aca- 
demic ones. 

The  Percussion’s  products  (Rhythmyx  Content  Manager)  (www.percussion.com)  is  a commercial  set  of  tools, 
devoted  to  the  management  of  web  pages  by  the  content  authors,  without  asking  them  to  mastering  HTML,  or 
even  site  rules  and  structures.  It  allows  the  same  information  to  be  presented  in  multiples  pages,  as  HDLS  does, 
and  clearly  allows  the  separation  between  content  and  presentation.  But  it  is  a relatively  cumbersome  tool,  more 
dedicated  to  commercial  sites  than  educational  or  academics  ones.  HDSL,  on  the  contrary,  is  a rather  straightfor- 
ward tool,  which  pushes  the  versatility  of  XML  in  domains  such  as  the  expression  of  the  content  semantics.  Per- 
cussion also  develops  a XML  to  HTML  tool,  Xsplit,  but  which  presents  only  some  of  the  capabilities  of  HDSL 
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Abstract 

This  paper  describes  the  first  stage  of  a four  year  research  project  on  the  design,  development 
and  use  of  web  based  digital  learning  material  for  food  chemistry  education.  The  paper 
discusses  design  guidelines,  based  on  principles  that  were  selected  from  theories  on  learning 
and  instruction,  and  illustrates  in  detail  how  these  guidelines  were  used  for  the  design  and 
implementation  of  digital  learning  objects  (LO). 

Six  cases,  a set  of  presentational  LO  and  a dozen  interactive  exercises  have  been  designed, 
developed,  implemented,  and  imported  in  different  learning  environments.  The  design 
guidelines  proved  to  be  useful  during  the  design  process.  The  digital  learning  material  has 
been  evaluated  positively  by  students  and  lecturers.  The  material  forms  now  a set  of  inspiring 
examples  for  food  chemistry  in  higher  education. 


Introduction 

The  Food  and  Biotechnology  (FBT)  programme  is  a research  programme  on  design  of  digital  learning  material. 
The  programme  was  initiated  at  Wageningen  University  in  September  2000  and  currently  counts  5 large 
projects.  The  digital  learning  objects  (LO)  that  are  developed  within  the  FBT  programme  should  meet  the 
following  requirements.  First,  the  quality  of  the  learning  material  must  be  considered  highly  satisfactory  by 
experts  in  the  respective  fields  as  well  as  by  students.  Second,  it  must  be  possible  to  manage  the  LO  with  the 
mainstream  Learning  (Content)  Management  Systems  (LMS).  Third,  the  LO  must  be  useful  in  a wide  range  of 
different  instructional  settings  in  higher  education.  Finally,  the  development  costs  for  oie  credit  point  (one 
weeks  study)  material  should  be  within  the  60.000  $ (US). 

Within  the  FBT  programme  a four  year  research  project  on  the  design,  development  and  use  of  web 
based  digital  learning  material  for  food  chemistry  education  is  carried  out.  Food  chemistry  is  a disciplinary 
science,  which  masters  and  generates  knowledge  and  insight  mainly  related  to  a)  components  of  food,  b) 
(bio) chemical  reactions  occurring  in  food,  c)  effect  of  treatments  on  (bio)chemical  reactions  and  d)  quality 
characteristics  of  food.  The  challenge  of  the  project  is  to  select  and  experience  with  design  guidelines,  taken 
from  different  disciplines  in  order  to  design  effective  LO  for  food  chemistry  education. 

Theories  on  learning  and  instruction  offer  important  clues  to  design  effective  learning  material, 
including  digital  learning  material.  In  particular  there  are  good  textbooks  on  course  design  and  instructional 
design  (Posner  1997,  Smith  1993,  Merrienboer  1997).  Furthermore,  there  is  much  research  on  effectiveness  of 
computer  assisted  learning.  This  research  is  mostly  conducted  in  a laboratory  context  where  a number  of 
practical  constraints  can  be  ignored.  On  the  other  hand,  much  literature  describes  the  use  of  digital  learning 
material  in  practice.  Nevertheless,  practical  detailed  guidelines  for  the  design  of  LO  within  realistic  budgetary 
limits,  which  also  satisfy  technical  and  organisational  constraints,  are  missing.  In  fact,  there  is  a strong  need  for 
paradigm  examples  for  specific  scientific  disciplines  such  as  food  chemistry. 


The  work  presented  in  this  paper  describes  the  design  guidelines  for  digital  learning  material  for  the 
introductory  course  Food  Chemistry,  which  is  a 4 credit  point  course  for  second  year  food  technology  students 
at  Wageningen  University.  Material  has  been  developed  to  support  two,  for  this  course,  important  learning 
objectives:  acquisition,  construction  and  retention  of  (declarative)  food  chemistry  knowledge  and  a quantitative 
understanding  of  the  relative  importance  of  the  chemical  reactions  occurring  during  food  processing  and 
storage. 

The  next  section  presents  which  pedagogical  principles  were  selected.  Section  3 describes  how  a set  of 
digital  LO  have  been  designed  and  developed  according  to  guidelines  based  in  the  selected  principles.  Section  4 
gives  the  evaluation  of  the  use  of  the  LO  by  students  and  lecturers. 


Selected  Design  Principles 

From  theories  on  learning  and  instruction,  five  principles  were  selected  as  the  basis  for  every  LO.  This 
selection  was  based  on  three  selection  criteria.  First,  the  selected  principles  are  relatively  insensitive  to 
changes  of  educational  or  instructional  concepts.  Second,  the  need  to  reduce  cognitive  load  was  especially 
considered  relevant  for  food  chemistry,  where  the  amount  of  detail  is  initially  overwhelming  for  many 
students.  Finally,  the  selection  focuses  on  the  strong  points  of  digital  material  as  compared  with  lecture 
notes.  The  principles  are  not  completely  mutually  independent.  The  principles  are: 

a.  Aim  at  optimisation  of  cognitive  load 

Cognitive  load  is  the  demand  placed  upon  a person's  cognitive  abilities  when  performing  a task.  Effective 
training  should  reduce  the  cognitive  load,  since  learning  is  typically  constrained  by  limited  processing 
capacity  (Merrienboer  1997).  Ignoring  this  principle  will  reduce  the  students'  confidence.  Digital  learning 
material  provides  more  possibilities  to  reduce  cognitive  load  than  lecture  notes. 

b.  Aim  at  motivation  of  students 

Motivated  students  learn  better,  faster,  and  more  and  also  remember  better  (Schmidt  1998).  According  to 
the  ARCS  model  (Keller  1988)  instruction  should  capture  the  Attention  of  the  student,  should  be  perceived 
as  Relevant  by  the  student,  and  should  induce  Confidence  and  Satisfaction,  to  motivate  students.  From  this 
principle  many  guidelines  for  the  design  of  learning  material  can  be  derived.  Digital  material  provides  new 
options  for  motivation  (Malone  1987). 

c.  Present  information  Just  In  Time  (JIT) 

Providing  information  just  prior  to  application  is  a form  of  Just-in-Time  Education  (JiTEdXHudspeth 
1992).  An  advantage  of  JiTEd  based  learning  material  b that  the  cost  of  learning  is  directly  turned  into 
benefit.  JIT-information  is  also  an  instrument  to  prevent  cognitive  load  as  well  as  to  motivate  students. 

It  is  important  to  be  aware  that  students  who  work  with  digital  material  expect  JIT  presentation,  since  they 
know  that  JIT  presentation  is  one  of  the  strong  points  of  digital  material. 

d.  Visualise  whenever  possible: 

There  is  much  evidence  suggesting  that  retention  of  knowledge  will  be  enhanced  when  images  accompany 
words  (Denis  1994).  Additionally,  images  will  often  increase  quick  understanding  and  reduce  cognitive 
load  (Anderson  1995,  Larkin  1987).  Lecture  notes  seldom  take  advantage  of  presentational  formalisms 
other  than  printed  text  and  formulas.  Visualisation  in  digital  material  is  ultimately  cheaper  than  in  lecture 
notes. 

e.  Activate  whenever  possible: 

Students  should  be  activated  to  elaborate  on  recently  acquired  declarative  knowledge.  Evidence  is  found 
that  practice  or  elaboration  on  a subject  strengthens  the  retention  and  the  retrieval  of  the  subject  (Anderson 
1995)  as  well  as  understanding.  Better  retention  reduces  cognitive  load  in  later  stages  (Merrienboer  1997). 
Clearly  information  and  communication  technology  offers  much  more  possibilities  to  activate  students  than 
lecture  notes. 


Applying  Design  Principles  within  Food  Chemistry  Education 

Digital  material  is  designed  according  to  several  criteria,  for  instance  the  intended  learning  objectives  and  the 
requirements  of  the  FBT  programme,  which  are  all  considered  as  important  to  design  successful  digital  learning 
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material.  The  above  mentioned  design  principles  are  translated  into  design  guidelines  that  match  with  the  other 
criteria.  The  design  guidelines  are  therefore  different  for  different  kinds  of  digital  learning  material. 


Presentational  Learning  Objects  and  Interactive  Exercises 


Presentational  learning  objects  (presentational  LO)  and  interactive  exercises  have  been  designed  to  support  the 
acquisition,  construction  and  retention  of  (declarative)  food  chemistry  knowledge. 

A typical  example  of  presentational  LO  or  interactive  exercises  is  shown  in  the  figures  1 and  2,  which 
display  two  parts  of  one  interactive  exercise  (see  Demo  Site).  In  this  exercise  students  first  have  to  look  at  the 
molecule  of  anthocyanin  (a  natural  colorant).  By  moving  the  mouse  pointer  over  a specific  part  of  this  molecule 
an  explanation  of  a specific  property  pops  up.  Figure  1 shows  the  situation  where  the  mouse  pointer  points  at  a 
place  that  is  known  to  make  a complex  with  a multiple  charged  metal  ion.  When  students  have  read  the 
information  they  can  go  to  the  exercise.  This  part,  which  is  shown  in  figure  2,  is  based  on  the  game  mah-jong. 
The  student  has  to  click  two  pieces  that  belong  together  and  press  the  OK-button.  If  the  two  belong  together  the 
pieces  disappear.  The  goal  of  the  game  is  to  get  rid  off  all  pieces.  The  information  linked  to  the  molecule  in  the 
first  part  is  related  to  the  words  on  the  pieces  in  the  mah-jong  game. 
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Figure  1:  Information  about  properties  that 
influence  the  stability  of  anthocyanin. 
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Figure  2:  Exercise  to  elaborate  on  the  previously 
acquired  information. 


The  most  important  detailed  design  choices  to  design  presentational  LO  and  interactive  questions,  based  on  the 
guidelines  listed  in  part  2,  are  listed  in  table  1 . 


Principles 

Design  guidelines 

Minimisation  of 
cognitive  load 

The  LO  and  exercises  should 

Be  small:  it  takes  at  most  30  minutes  to  go  through  the  presented  information. 
Be  independent:  presented  information  is  perfectly  understandable  on  its  own 
Support  acquisition  or  elaboration  of  one  distinct  knowledge  structure 
Use  powerful  key-symbols 

Motivation  of 
students 

Realise  a large  variation  in  the  type  of  the  LO 
Incorporate  games 

JIT  presentation 

Provide  links  to  presentational  LO  in  the  interactive  exercises 
Present  information  which  has  to  be  used  in  actions  in  the  same  LO 

Visualisation 

Consist  of  high  impact  visual  components 

Activation 

Induce  active  behaviour  of  the  student 

TaUe  1:  Design  guidelines  specific  for  the  design  of  presentational  LO  and  interactive  exercises 


To  minimise  cognitive  load  the  presentational  LO  and  interactive  exercises  are  small,  complete  and  powerful. 
The  LO  are  small  since  they  are  designed  to  take  at  most  30  minutes.  In  the  case  of  the  anthocyanin  example, 
the  whole  exercise  will  approximately  take  15  minutes.  Second,  the  LO  are  independent  and  support  the 
construction  or  the  elaboration  of  a complete  and  distinct  piece  of  knowledge.  For  instance,  the  example  deals 
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only  with  those  properties  of  anthocyanin  which  are  important  for  the  stability  of  the  molecule.  The  third  way  to 
minimise  cognitive  load  is  by  using  powerful  key-symbols  that  explain  the  situation  in  a direct  way.  For 
instance,  different  parts  in  the  molecule  structure  of  anthocyanin  (Fig.  1),  the  key-symbols,  are  highlighted  (see 
Demo  Site).  Every  student  will  understand  that  the  information  linked  to  a part  belongs  to  that  specific  part  of 
the  molecule.  This  is  essential  to  explain  something  without  using  words.  The  advantage  of  the  natural 
relationship  between  (part  of)  an  image  and  the  meaning  of  that  image  is  used  in  all  LO  and  exercises. 

To  motivate  the  students,  the  range  of  different  types  of  LO  and  exercises  is  large  (see  Demo  Site). 
Different  exercises  are  for  example:  a)  choose  the  right  option  between  two  options,  b)  choose  between  several 
options,  c)  match  propositions  d)  complete  chemical  reaction  pathways  by  dragging  molecules  to  the  right  place 
in  the  reaction,  e)  choose  between  different  areas  in  a graph  with  different  reaction  rates,  and  f)  complete  the 
construction  of  a molecule  by  dragging  the  right  component  to  each  molecule.  Examples  of  different 
presentational  LO  are:  a)  movies  explaining  or  showing  reactions,  b)  molecules  that  give  information  when 
clicking  on  it,  and  c)  small,  easy  exercises  providing  additional  information. 

Another  important  tool  to  motivate  students  is  by  incorporating  games  (Malone  1987).  The  exercise  in 
figure  1 and  2 is  a typical  example  of  a game.  Several  exercises  have  a game -like  appearance. 

JIT-presentation  of  information  is  mainly  provided  in  two  ways.  First,  the  interactive  exercises  have 
links  to  corresponding  presentational  LO.  In  this  way,  students  can  easily  look  up  information  to  answer  a 
question  or  learn  more  about  the  subject.  Second,  like  in  the  anthocyanin  example,  necessary  information  is 
presented  within  the  LO. 

Visualisation  is  an  important  part  of  the  presentational  LO  and  interactive  exercises.  Within  these  LO 
colour  is  used  in  a functional  way,  especially  to  highlight  important  elements.  The  visual  LO  vary  from  static 
pictures  — for  example  a molecule  or  a photograph  of  a fruit  — to  animations  and  screen  recordings  — for 
example  an  animation  of  a chemical  reaction  in  detail  — . The  use  of  text  is  minimised.  Again  the  anthocyanin 
exercise  is  a good  example. 

The  presentational  LO  and  interactive  exercises  induce  active  behaviour.  Both  figures  1 and  2 show 
how  students  can  be  activated.  In  the  first  part  of  the  exercise  (Fig.  1)  the  student  will  only  learn  something 
about  the  molecule  if  (s)he  points  the  mouse  pointer  at  the  molecule.  The  student  is  forced  to  pay  attention  to 
what  (s)he  sees  and  reads.  In  the  second  part  of  the  exercise  (Fig.  2)  the  student  is  activated  to  play  a game  and 
probably  this  activates  the  student  to  go  back  to  the  first  part  to  study  those  parts  (s)he  didn't  remember.  In  all 
presentational  LO  and  exercises  students  are  forced  to  do  something:  mouse-over  actions,  click  the  mouse, 
move  objects,  fill  in  numbers,  answer  short  questions,  play  a game,  etc. 


Digital  Cases 

Digital  cases  have  been  designed  to  promote  the  transfer  of  knowledge  of  mathematics,  reaction  kinetics  and 
reaction  equilibrium,  which  students  already  acquired  in  previous  courses,  in  order  to  be  able  to  translate 
practical  problems  within  Food  Chemistry  to  arithmetical  equations  and  to  solve  them.  The  intended  learning 
outcome  is  a quantitative  understanding  of  relative  importance  of  reactions  occurring  during  food  processing 
and  storage. 

The  digital  cases  are  divided  in  a presenting  and  an  activating  part.  The  presenting  part  contains  all 
necessary  information  about  e.g.  reaction  kinetics  and  information  about  working  with  Excel.  This  information 
is  structured  in  one  place,  which  is  denoted  a library.  The  activating  part  exists  of  six  different  cases.  Every  case 
applies  a certain  concepts  in  the  field  of  Food  Chemistry,  for  example  the  hydrolyses  of  saccharose  (sugar)  in  a 
fruit  drink.  On  average,  each  case  contains  10  guiding  questions.  The  first  questions  (2  to  3)  are  introducing  the 
subject,  asking  about  basic  knowledge  necessary  to  understand  the  topic  of  the  case.  The  following  questions 
are  mainly  about  making  calculations  or  graphs,  and  they  require  for  instance  to  fill  in  a (calculated)  number  or 
an  interpretation  of  the  calculation/graph.  Calculations  are  assisted  by  the  information  in  the  library  and  by 
feedback  and  hints.  The  most  important  design  guidelines  for  the  digital  cases,  based  on  the  principles  in  section 
2,  are  listed  in  table  2. 

The  activating  part  within  each  case  is  built  around  a practical  situation.  For  example,  the  student  plays  the 
role  of  a researcher  who  has  the  task  to  solve  a problem  in  a company.  The  calculations  on  chemical  reactions, 
kinetics,  etc.  are  leading  to  a solution  of  the  problem  or  a clarification  of  the  situation,  which  shows  the 
usefulness  of  the  calculations. 
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Principles 

Design  Guidelines 

Minimisation  of 
Cognitive  load 

Provide  only  necessary  information 
Minimise  search  for  information 

Motivation  of  students 

The  student  plays  a role  during  the  case 
Show  the  usefulness  of  calculations 
Give  the  student  a feeling  of  independence 

JIT  presentation 

Provide  hints/feedback/introductory  questions 
Provide  movies  on  how  to  work  with  Excel 

Table  2:  Design  guidelines  specific  for  the  design  of  the  digital  cases 


The  introductory  questions  have  the  objective  to  activate  (make  available)  prior  knowledge.  The 
availability  of  much  relevant  prior  knowledge  improves  the  learning  capacity  (Schmidt  1998). 

To  minimise  cognitive  load,  only  necessary  information  that  students  need  to  calculate  on  reaction,  like 
formulas,  is  provided  in  the  library.  Because  students  have  to  use  some  advanced  tools  of  Excel,  the  library  also 
contains  information  in  the  form  of  audio  visual  screen  recordings.  The  students  can  put  the  information  from 
the  library  immediately  into  practice,  which  is  a form  of  JIT -presentation. 

Cognitive  load  is  also  reduced  by  minimising  the  search  for  information,  which  is  needed  to  solve  a 
question  within  a case.  Therefore,  questions  contain  hyperlinks  that  point  to  specific  information  in  the  library. 
Also  background  information  about  food  chemistry,  for  example,  why  a product  turns  brown  during  heating,  is 
provided  when  needed.  Especially  the  introductory  questions  refer  to  background  information.  Both  the  links 
and  the  background  information  are  a form  of  JIT-presentation. 

The  information,  the  links  to  the  information  and  the  hints  are  applied  in  such  a way  that  a student  should 
be  able  to  finish  a case  without  the  help  of  a teacher.  The  hints  help  students  to  work  towards  the  solution. 
Students  can  ask  for  a hint  themselves,  but  after  submitting  an  incorrect  answer  a hint  is  given  automatically. 
Also,  after  submitting  an  answer  feedback  on  the  answer  is  given.  This  will  probably  give  the  student  the 
feeling  that  (s)he  is  able  to  solve  the  case  by  him  or  herself,  which  strengthens  the  student’s  self-confidence  and 
satisfaction. 


Evaluation  of  the  Design  Principles  within  Food  Chemistry  Education 

The  students  used  and  judged  the  digital  learning  material  during  the  time  of  the  course.  Table  3 gives 
some  results  from  questionnaires  students  had  to  fill  in  after  using  the  digital  learning  material.  The  results  are 
categorised  according  to  the  principle  they  represent.  Not  all  principles  were  evaluated,  since  some  guidelines 
do  not  need  to  be  judged  by  the  students;  these  guidelines  were  just  used  by  the  designer. 

It  seems  that  the  success  of  the  digital  cases  is  due  to  the  JIT-presentation  of  information  like  hints, 
links  to  the  theory,  and  feedback.  Because  of  this  information  students  seem  to  learn  faster,  to  be  more  self- 
confident  and  to  be  more  motivated  to  solve  the  questions. 

The  results  of  the  questionnaires  for  presentational  LO  and  exercises  strengthen  the  belief  that  it  is  very 
well  possible  to  provide  knowledge  by  mainly  using  visuals  accompanied  by  a minimum  of  textual  parts, 
especially  for  explaining  reactions  or  explaining  intricate  events  in  food  chemistry.  For  food  chemistry, 
visualisation  of  different  aspects  is  a strong  tool  to  help  students  understand  and  remember  the  sometimes 
intricate  information. 

For  the  designer,  the  guidelines,  which  are  described  according  to  theories  on  learning  and  instruction, 
supported  the  design  of  digital  learning  material  based  on  theories  rather  than  based  on  common  sense.  The 
guidelines  keep  the  designer  on  the  road  during  the  project.  Every  piece  of  material  that  was  designed  and 
developed  was  judged  to  be  parallel  with  the  guidelines  and  principles.  Although  it  is  not  easy  to  test  if  the 
principles  are  correctly  implemented  there  has  been  no  indication  of  the  opposite. 

The  pedagogical  principles  seem  to  be  translated  successfully  in  guidelines  that  were  used  to  design 
digital  learning  material.  ^ this,  a set  guidelines  based  on  general  accepted  principles  are  described  that  are 
useable  in  different  educational  settings  within  food  chemistry  and  related  settings. 
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Principle 

Presentational  LO  and  exercises 

Cognitive  load 

-None  of  the  objects  give  too  much  information  at  once 

- Time  to  finish  each  exercise  ranged  from  1 minute  to  9 minutes  (stdev  = 60%) 

- The  interactive  exercises  are  clear:  95% 

Motivation 

- Exercises  are  useful:  86% 

- Students  learned  what  was  supposed:  86% 

- Students  enjoyed  the  exercises:  86% 

Visualisation 

- Screen  lay-out  was  judged  positive 

- Objects  that  use  more  text  than  general  are  evaluated  less  positive  by  some  students 

Activation 

- Activating  parts,  like  clicking  through  an  explanation  of  a chemical  reaction  or  mouse- 
over  effects,  are  Judged  useful 

Principle 

Digital  cases 

Cognitive  load 

- There  is  enough  information  provided:  100% 

- The  links  are  useful:  100% 

Motivation 

- The  difficulty  of  the  cases  was  judged  as  good:  91% 

- Questions  show  the  usefulness  of  the  calculations:  91% 

- Questions  give  more  insight  in  how  to  use  the  theory:  100% 

- Playing  a role  is  motivational:  91% 

- Playing  a role  is  useful:  100% 

JIT 

presentation 

- Introductory  questions  help  to  understand  the  cases:  100% 

- Movies  about  Excel  are  useful:  76% 

- Theory  is  useful:  95% 

- Hints  are  useful:  90% 

- Feedback  was  sufficient  to  carry  out  the  cases:  86% 

Table  3 : Results  from  students'  questionnaires. 


Final  Remarks 

During  the  academic  years  1999/2000  and  2000/2001  the  digital  LO  were  evaluated.  Both  students  and  lecturers 
judged  the  digital  LO  positive.  The  digital  LO  will  now  be  used  as  an  inspiration  for  faculty  staff.  Because  of 
the  independent  character  of  the  LO  they  can  be  imported  in  a range  of  different  LMS  such  as  Blackboard  and 
Hyperwave  ELS.  The  proof  of  the  principle  that  it  is  possible  to  design  and  develop  a rich  body  of  LO  for  food 
chemistry,  which  satisfy  the  design  criteria  described  in  the  introduction,  is  close  at  hand.  Further  research  will 
be  directed  to  articulation  of  design  criteria  for  digital  LO  in  food  chemistry  based  on  a match  between  intended 
learning  outcomes  of  food  chemistry  courses  and  the  possibilities  and  limitations  of  state  of  the  art  digital  LO. 
This  should  be  a first  step  towards  a methodology  of  digital  learning  object  design  for  food  chemistry  and  to  a 
handbook  of  best  practices. 
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Abstract:  The  University  of  Nebraska  Medical  Center  is  a leader  among  the  US 
Medical  Schools  with  graduates  who  choose  Primary  Care  Medicine.  The  ability 
to  accurately  diagnose  an  ear  disease  is  an  important  skill  of  a Primary  Care 
graduate.  Prior  to  graduation  all  medical  student  receive  didactic  lectures 
covering  ear  disease.  Upon  inquiry  the  Department  of  Otolaryngology  found 
that  30%  of  the  students  graduate  with  clinical  “hands-on”  experience  of 
diagnosing  ear  disease.  A Blackboard  course  was  created  to  give  the  students  an 
opportunity  to  experience  diagnosing  ear  disease  by  offering  a patient-based 
course  with  supporting  visuals.  The  course  creation,  successes  and  failures  will 
be  discussed.  A course  demonstration  will  occur  during  the  presentation.  The 
instructional  design  and  the  demonstrated  ability  to  create  a course  that  is 
multimedia  rich  without  a heavy  commitment  to  programming  with  html  are 
points  of  interest. 


The  University  of  Nebraska  Medical  Center,  College  of  Medicine  is  a leader  among  the  U.S.  medical 
schools  with  graduates  who  choose  to  Primary  Care  medicine.  A Primary  Care  physician  should  have  the 
principal  skills  of  ability  to  recognize,  diagnose  and  treat  common  illnesses  including  acute,  chronic,  or 
emergent  conditions  in  patients  of  all  ages.  The  ability  to  accurately  diagnose  an  ear  disease  is  an 
important  ability  of  a Primary  Care  graduate.  Prior  to  graduation  all  medial  students  receive  didactic 
lectures  covering  ear  disease.  Upon  inquiry  the  Department  of  Otolaryngology  - Head  and  Neck  Surgery 
found  that  30%  of  the  students  graduate  with  clinical  “hands-in”  experience  of  diagnosing  ear  disease. 

A multimedia  rich  Blackboard  course,  based  on  patient-based  model  was  created  to  give  the  students  an 
opportunity  to  experience  diagnosing  ear  disease.  The  course  is  also  intended  for  third  year  medical 
students  whom  are  engaged  in  the  clinical  experience  and  are  in  need  of  elaboration  or  just-in-time  training. 

The  initiative  was  funded  by  the  Department  of  Otolaryngology.  A student  worker  managed  the  content 
research,  multimedia  capturing,  and  creation  of  the  course.  The  student  worker  was  employed  for  forty 
hours  a week  for  three  months.  The  department  also  purchased  the  necessary  equipment  to  capture  the 
supporting  course  multimedia  (videos  and  still  images). 


Prior  to  the  course  creation  and  gathering  of  the  multimedia  a context  analysis  was  completed.  The  first 
component  of  the  analysis  placed  the  online  course  within  the  third  year  of  the  College  of  Medicine 
curriculum.  The  curriculum  for  the  first  two  years  of  medical  school  is  predetermined  for  the  medical 
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student.  The  third  year  of  the  curriculum  consists  of  six  required  clinical  clerkships  (Internal  Medicine, 
Family  Practice,  Surgery,  Pediatrics,  OB/GYN,  and  Psychiatry).  The  fourth  year  of  the  curriculum  consists 
of  a required  basic  science  elective  and  eight  electives.  Within  the  third  or  fourth  year,  with  the  input  of  an 
advisor,  the  student  is  required  to  select  a primary  area  of  interest.  Upon  the  selection  the  students  is 
required  to  choose  five  of  eight  electives  from  their  primary  area  of  interest  (out  of  the  8 electives).  These 
primary  area  electives  have  been  determined  to  be  appropriate  for  students  entering  that  field.  The 
remaining  three  electives  can  be  chosen  from  any  other  primary  interest  area. 

During  the  third  year  of  the  curriculum,  traditionally  only  30%  of  the  medical  students  have  selected 
Otolaryngology  - Head  and  Neck  Surgery  during  the  Surgery  clinical  experience.  During  the  second  and 
third  year  of  medical  school  a student  attends  a minimal  number  of  hours  of  didactic  instruction  covering 
Otolaryngology  - Head  and  Neck  Surgery.  This  analysis  was  very  helpful  prior  to  researching  and  writing 
the  content,  acting  as  a barometer  to  set  the  level  of  content.  The  audience  analysis  highlighted  the 
importance  of  closely  simulating  the  traditional  clinical  experience. 

Prior  to  being  the  course  creation  process,  some  instructional  design  aspects  were  identified.  In  effort  to 
mimic  the  clinical  setting  a patient-based  model  was  selected  for  the  construct  of  the  course.  The  course 
includes  video  of  the  ear  examination,  supporting  still  images,  test  results  and  the  patient’s  history.  The 
video  examination  is  an  essential  piece  of  the  instruction  that  exhibits  the  technique  of  using  the 
examination  instrument,  orientation  within  the  human  anatomy,  and  the  ability  to  show  specific  character 
tics  to  aid  the  process  of  ear  disease  diagnosis.  The  students  will  be  instructed  to  participate  in  the  course 
via  the  campus  network  or  a high-speed  Internet  connection.  An  alternative  option  of  writing  the 
multimedia  to  CD-ROM  and  distributing  the  CD-ROM  to  off  campus  (non-networked/modem  based) 
students  is  being  assessed 

Once  these  instructional  design  aspects  where  identified  the  faculty  member  began  the  research  of  content 
and  capturing  supportive  multimedia  as  each  the  ear  diseases  were  diagnosed  and  treated  in  the  clinic.  The 
faculty  member  obtained  consent  forms  from  patients,  outlining  the  rights  to  summarize  the  patient’s  case 
and  to  capture  necessary  multimedia  (digital  images,  video  of  examination,  and  test  results). 

Each  level  is  supported  with  the  multimedia,  patient  history  and  examination  and  the  student  is  instructed  to 
complete  within  a 15  to  20  minute  interval.  During  the  course  creation  process  some  message  design  issues 
addressed. 

The  overall  structure  of  the  course  consists  of  eight  levels  of  severity  of  the  ear  disease  diagnosis  based  on 
physical  findings.  The  students  can  work  through  the  entire  course  or  to  jump  to  the  desired  content, 
offering  the  just-in-time  training.  Simple  buttons  and  hyperlinks  were  constructed  in  Blackboard  to  create 
these  two  options.  During  the  creation  process,  the  message  design  was  modified  numerous  times.  The  final 
design  includes  patient  history,  examination,  still  images,  and  a video  ear  examination. 

Other  elements  of  interest  are  prescriptive  instructions,  the  importance  of  limiting  the  students’ 
expectations,  and  the  ability  to  create  an  online  multimedia  rich  course  without  a heavy  commitment  to 
programming  with  .html  (the  language  of  the  internet). 


Following  Faraday’s  steps  at  high  school, 
using  microcomputer  based  labs. 

L.Papatsimpa,  P.Dimitriadis,  K.Papamichalis,  K.Kabouris,  G.Kalkanis 
University  of  Athens,  Pedagogical  Dept.  P.E.,  Navarinou  13A,  Athens,  GREECE 

Abstract 

The  new  information  technologies  can  create  a more  productive  educational  environment  for  physics  teaching.  In  this  paper  we  describe  the  use 
of  microcomputer  based  labs  (MBL)  for  the  studying  of  electromagnetic  induction.  In  MBL  labs,  the  students  — by  participat  ing  in  hands-on 
experiments — have  the  opportunity  to  measure,  directly  and  simultaneously,  the  physical  quantities  that  are  involved  in  the  induction 
phenomenon.  The  computer,  equipped  with  an  interface  and  sensors,  serves  as  a measurement  device.  From  the  evaluation  of  the  learning 
outcomes  we  can  deduce  that  there  is  a major  contribution  to  students’  understanding  of  the  physical  concepts  and  laws. 

Introduction 

Today,  more  than  any  other  period  in  the  past,  and  due  to  the  exponential  growth  of  technological  progress,  people  are  greatly  influenced  by  the 
development  of  scientific  knowledge  and  the  advancement  of  information  and  communication  technologies* . The  relationship  between  science- 
technology  and  education  is  far  more  complicated  today  than  any  other  time  in  history^.  Nowadays,  there  is  a close  relation  between  science  and 
technology,  where  both  fields  share  a constant  feedback  process.  The  expansion  of  scientific  knowledge  leads  to  new  technological  applications, 
while  technology  provides  scientists  with  tools,  techniques  and  devices  that  allow  expansion  of  their  capabilities  and  effectiveness,  thus  leading 
to  a further  development  of  science.  Yet,  as  technology  gets  more  and  more  complicated,  there  is  no  sharp  boundary  between  science  and 
technology  since  the  area  of  their  overlapping  gets  constantly  widei^. 

Traditionally,  the  teaching  of  science  and  technology  in  schools  focused  mainly  on  providing  the  knowledge  of  theories  and  facts  or  the 
obtainment  of  technical  skills.  This  mechanistic  approach  of  teaching  does  not  represent  neither  an  effective  nor  an  appropriate  strategy  towards 
science -literate  and  technology -literate  pupils.  So,  there  is  a growing  necessity  to  develop  a new  educational  environment  that  will  allow  students 
to  collect,  visualize,  analyze  and  model  the  experimental  data  so  that  they  will  be  able  — through  their  active  involvement  and  participation — to 
understand  the  laws  of  physics  and  the  present  relationship  between  science  and  technology  A new  suggestion  that  will  facilitae  the 
obtainment  of  the  proposed  objectives  is  the  use  of  microcomputer- based  laboratories  (MBL  - labs)  in  education.  In  a MB-lab  students  do  real 
experiments  not  simulated  ones,  using  sensors  connected  to  a computer  via  an  interface. 

Subject  /Problem 

The  phenomenon  of  the  electromagnetic  induction  holds  a significant  and  central  rank  not  only  concerning  its  connection  with  the  development 
of  classical  physics,  but  also  with  the  technological  utilization  of  a major  physical  discovery  that  led  to  the  production  of  electric  energy  — a 
process  that  had  a radical  impact  in  modem  civilization.  Furthermore,  it  gives  us  the  opportunity  to  understand  the  function  of  devices  that 
transform  mechanical  energy  into  electric  energy  and  vice  versa,  like  electric  generators,  motors  or  transformers. 

The  traditional  instructional  approach  to  the  phenomenon  of  electromagnetic  induction  is  substantially  limited  in  the  repetition  of  the  experiments 
conducted  independently  by  Michael  Faraday  and  Joseph  Henry  in  the  middle  of  the  1 9**  century^  — a practice  which  is  followed  by  the  sudden 
and  inexplicably  appearance  of  the  Faraday’s  law  of  induction,  that  relates  the  induced  emf  with  the  rate  of  the  magnetic  flux  change.  But, 
naturally,  the  vast  minority  of  children  carmot  follow  Faraday’s  exemplary  line  of  reasoning  that  led  to  the  statement  of  the  law  of  induction.  So, 
they  just  memorize  the  law  with  no  functional  understanding  of  it  and  without  having  the  opportunity  to  apply  it  at  real-world  situations  or 
problem^. 

A laboratory  approach,  based  on  the  new  information  technologies,  could  contribute  to  the  improvement  of  the  educational  outcomes  concerning 
the  principles  of  electromagnetic  induction  and  Faraday’s  law. 

More  specifically,  the  use  of  sensors  for  collecting  voltage  and  magnetic  field  measurements  and  the  real-time  display  of  this  data  in  graphs 
presented  at  the  computer  screen,  permits: 

• The  direct  measurement  of  the  strength  of  the  magnetic  field  — something  especially  difficult  to  be  done  with  the  use  of  traditional 
devices — but  without  which,  one  carmot  experimentally  discover  and  verify  Faraday’s  law. 

• The  comparison  between  the  change  of  the  strength  of  the  magnetic  field  and  the  induced  voltage. 

• The  focus  of  students’  interest  in  the  development  of  the  phenomenon. 

• The  detection  of  students’  preconceptions  and  the  use  of  socratic  dialogues  between  teacher  and  students  throughout  the  experimental 
procedure. 

Design  and  procedure 

By  utilizing  the  new  information  technologies  we  designed  and  applied  an  instructional  intervention  for  the  phenomenon  of  electromagnetic 
induction  at  the  laboratory  by  connecting  sensors  through  interface  with  the  computer.  The  procedure  involved  the  historical  induction 
experiments  of  Faraday  and  Henry.  Three  high  schools  in  Athens  were  selected  to  be  school  units  of  application,  so  that  the  sample  would 
represent  the  average  Greek  high  school  student.  75  high  school  students  of  the  1 2***  grade  from  3 different  school  classes  (of  vocational-technical 
education),  participated  in  our  research.  T he  students  had  already  been  recently  taught  about  the  induction  phenomenon,  through  the  traditional 
instructional  method. 

Two  experimental  activities  were  conducted.  In  the  first  one,  we  used  a coil  and  a magnet  that  was  moving  relative  to  the  coil,  while  in  the  second 
we  used  two  coils  wrapped  around  a soft  iron  core.  With  the  use  of  a voltage  and  a magnetic  field  sensor — we  obtained  a reakime  collection  of 
the  corresponding  data  and  a graphical  display  of  their  change  with  time  at  the  computer  screen. 

The  first  learning  outcome  was  that  students  realized  than  an  induced  voltage  in  the  coil  occurs  only  during  the  movement  of  the  magnet  and  that 
the  polarity  of  the  voltage  depends  on  the  direction  of  the  magnet’s  movement,  whereas,  with  the  magnet  inverted,  the  changes  of  the  induced 
voltage  with  time  vary  accordingly.  From  all  the  above,  students  came  to  the  conclusion  that  polarity  is  determined  by  the  absolute  value  of  the 
magnetic  field  change.  Next,  the  students  started  wondering  about  the  factors  that  determine  the  magnitude  of  the  induced  voltage  Vn  and 
concluded  that  Vio  does  not  depend  simply  from  the  change  of  the  strength  of  the  magnetic  field,  but  from  its  rate  of  change.  By  using  printedout 
diagrams,  each  student  would  calculate  the  slope  of  the  curve  that  shows  the  voltage  versus  time;  in  other  words,  they  would  calculate  the  rate 

dB 

at  the  peak  of  the  voltage  waveform.  Next,  students  determined  the  ratio  and  realized  that  this  ratio  remains  conrtant  or  that  the 

dt  ClB 

dt 

induced  voltage  is  proportional  to  the  rate  of  change  of  the  magnetic  field  strength,  i.e.  with  the  rate  of  change  of  the  magnetic  flux.  Finally,  the 
fact  that  the  induced  voltage  is  proportional  to  the  rate  at  which  the  magnetic  field  changes  (i.e.  to  the  rate  of  change  of  the  magnetic  flux),  was 
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also  verified  by  the  students  through  the  software  which  allows  the  simultaneous  screen  display  of  how  and  Vin  vary  with  time  — a display 

at 

that  gives  students  the  opportunity  of  a direct  comparison  between  the  two. 

In  the  second  experimental  procedure  we  used  three  sensors  at  the  same  time.  The  first  sensor  was  measuring  the  strength  of  the  current,  the 
second  the  strength  of  the  magnetic  field,  and  the  third  was  taking  measurements  of  the  induced  voltage  in  the  second  coil.  The  learmng  outcome 
was  that  the  student  understood  (or  realized)  that  an  induced  voltage  is  produced  in  the  second  coil  when  the  strength  of  the  electric  current  varies 
with  time  and  that  the  voltage  magnitude  is  proportional  to  the  rate  of  change  of  the  current  in  the  first  coil, 
lilts 

Before  the  instruction,  students  were  given  a pre-test  that  consisted  of  several  close-typed  multiple-choice  questions  and  they  were  asked  to  fill-in 
some  sections  of  a diagram.  Two  weeks  later,  after  the  instruction,  students  were  given  a post  -test,  that  consisted  of  similar  style  questions  with 
that  of  the  pre-test,  but  of  higher  difficulty  level. 

The  results  are  shown  on  Table  I.  They  show  three  basic  elements  of  the  dectromagnetic  induction  phenomenon,  namely  the  reasons  for  the 
production  of  emf,  the  polarity  of  the  induced  voltage,  and  Faraday’s  law.  The  table  illustrates  the  percentage  of  successful  students’  answers 
concerning  their  conceptual  understanding  of  these  subjects. 


Table  I 


Pre-test 

Post  -test 

Gain 

Causes  of  electromagnetic 
induction 

92 

95 

3 

Polarity  of  the  induced 
voltage 

20 

42 

22 

Faraday’s  law 

20 

51 

31 

Results  of  pre-  and  post-testing  of  students’  conceptual  unders  tandingof  electromagnetic  induction 
As  we  can  see  from  the  obtained  results,  the  reasons  for  the  production  of  emf  are  we  11- understood  by  the  students  even  through  the  traditional 
instruction.  But,  with  the  use  of  MB-Iabs  there  is  a substantial  improvement  on  the  results  concerning  the  polarity  of  the  induced  voltage  and  the 
connection  of  its  magnitude  with  the  rate  of  change  of  the  magnetic  flux.  Gain  is  observed  mostly  on  those  features  that  are  illustrated  more 
thoroughly  through  the  graphical  transformation  of  the  data  into  time-varying  diagrams  (for  example,  the  slope  of  the  curve  represents  the  rate  of 
change  of  the  corresponding  physical  quantity). 

Furthermore,  student  course  evaluation  and  comments  by  students  show  the  MBL  - labs  have  been  very  we  11  received. 

Discussion  i-  • • u 

In  spite  of  the  extensive  implementation  of  MBL-  programs  in  the  teaching  of  Mechanics,  the  relevant  literature  about  their  application  in  the 

study  of  electromagnetic  phenomena  is  very  limited  and  it  is  characterized  only  by  the  indirect  verification  of  Faraday’s  law  of  induction  . From 
the  results  of  our  research  it  is  evident  that  MBL-  labs,  serves  as  an  effective  tool  for  the  development  of  a good  conceptual  understanding  of  the 
electromagnetic  induction.  Our  research  support  s the  view  that  the  utilization  of  MBL-labs  assists  students  ability  to  describe,  understand  and 
predict  the  development  of  physical  phenomena.  Similar  conclusions  have  been  reached  from  the  implementation  of  MBL  - labs  for  the  teaching 
of  other  content  areas  in  junior  and  high  school  students^*’. 

Our  research  team  aims  at  the  expansion  of  MBL  - labs  utilization  into  other  phenomena  included  in  the  junior  and  high  school  science  curricula. 
A wider  and  more  general  application  of  the  microcomputer- based  laboratories  in  various  content  areas  and  school  grades,  may  be  the  key  factor 
for  the  necessary  reform  that  aims  in  boosting  students’  interest  and  improving  the  effectiveness  of  science  teaching. 
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Abstract:  This  session  will  include  critical  information  for  administrators  and  faculty  in  higher 
education  to  develop  and  implement  an  on-line  master's  degree  program  in  Technology/Human 
Resource  Development.  Students  enrolling  in  the  MS  degree  program  can  pursue  the  Master  of  Science 
in  Technology/Human  Resource  Development  degree  or  students  can  add  Trade  and  Industrial  Teacher 
certification  to  their  degree  plan  or  they  may  elect  to  complete  only  the  T&I  Teacher  Certification 
portion  of  the  program  without  pursuing  the  Master  of  Science  degree.  The  degree  program  is  offered 
through  The  University  of  Texas  TeleCampus  which  provides  a virtual  university  setting  complete  with 
library  resources,  student  support,  and  tutorial  help  for  participating  students. 

Today’s  information  technology  and  instructional  telecommunications  enable  the  U.  T.  System  to 
extend  its  collective  educational  reach  to  students  in  Texas  and  beyond  through  the  UT  TeleCampus,  an 
Internet-based  central  support  unit  for  distance  education.  The  UT  TeleCampus  was  launched  in  May  1998 
as  a central  support  system  for  the  online  educational  initiatives  of  the  15  component  campuses  and 
research  facilities  that  comprise  the  U.  T.  System  (UT  TeleCampus).Extensive  support  services  are 
provided  by  TeleCampus  staff  in  the  way  of  instructional  design  and  course  development,  as  well  as  faculty 
training  in  bringing  courses  from  the  lecture-based  classroom  to  the  Web -based  classroom.  Technological 
support,  policy,  marketing  research  and  external  communications  are  among  the  other  essential  support 
services  provided  to  the  campus  from  the  UT  TdeCampus  (UT  TeleCampus). 

The  goal  of  the  TeleCampus  in  placing  programs  online  was  the  creation  of  collaborative  degrees, 
utilizing  the  best  resources  in  faculty  expertise  from  all  campuses  (UT  TeleCampus).  This  was  the  model 
followed  in  developing  the  Master  of  Science  Degree  in  Technology  in  Human  Resource  Development 
(HRD). 

Overview 

The  Department  of  Technology,  in  support  of  the  mission  of  The  University,  is  a student -centered 
department  committed  to  conducting  multi- option  programs  targeted  at  preparing  technical  professionals, 
and  educators.  In  developing  this  online  degree  program,  the  department  is  further  supporting  the  mission 
of  the  institution  that  will  “establish  the  university  as  a leader  in  the  teaching  and  use  of  informational 
technology.”  Providing  a Master  of  Science  degree  in  Technology  in  the  Human  Resource  Development 
(HRD)  option  in  an  online  format  so  that  it  is  accessible  to  all  the  citizens  of  Texas  and  beyond  will  achieve 
the  mission  of  The  University  of  Texas  at  Tyler  and  the  Department  of  Technology.  In  the  proposed  third 
year  of  the  phase-in  program,  additional  courses  will  be  developed  to  enhance  the  HRD  option  and 
complete  the  State  Board  of  Educator  Certification  (SBEC)  Trade  and  Industrial  (T  & I)  certification  to 
help  address  the  critical  shortage  of  teachers  in  this  discipline. 

Human  Resource  Development  is  a professional  field  of  practice  that  focuses  on  education  and 
training  of  adults  in  diverse  settings.  It  is  organized  learning  activities  arranged  within  an  organization  in 
order  to  improve  performance  and/or  personal  growth  for  the  purpose  of  improving  the  job,  the  individual, 
and/or  the  organization.  HRD  includes  the  areas  of  training  and  development,  career  development, 
organization  development,  adult  literacy,  and  distance  education. 

The  online  Master  of  Science  degree  in  Technology  with  the  Human  Resource  Development 
option  will  utilize  the  resources  of  The  University  of  Texas  at  Tyler,  The  University  of  Texas  at  Galveston 
(UTG),  The  University  of  Texas  at  Permian  Basin  (UTPB),  The  University  of  Texas  at  Arlington  (UTA), 
and  the  UT  TeleCampus.  The  Department  of  Technology  will  assume  responsibility  for  developing  six 
courses  in  the  major  of  the  degree  option  and  will  utilize  existing  component  online  TeleCampus  courses  to 
round  out  the  degree  requirements  in  the  major  and  support  areas.  The  degree  offers  access  to  higher 
education  attainment  by  professionals  in  the  field  by  flexible  scheduling  and  asynchronous  delivery.  In  the 
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proposed  optional  3^  year  of  the  program,  UTT  will  develop  the  three  additional  courses  needed  to 
complete  the  T&  I certification  program  for  Texas  teachers  in  the  trade  and  industrial  discipline,  and  the 
courses  will  be  utilized  to  strengthen  the  HRD  degree  option. 


Benefits 

1.  A major  benefit  of  this  online  degree  is  the  meeting  of  the  increased  demand  in  business  and 
industry  for  personnel  trained  and  educated  in  the  human  resource  area.  By  providing  the  access 
necessary  for  adult  learners,  the  demand  for  a highly  skilled  workforce  can  be  met. 

2.  Another  benefit  of  the  degree  is  to  meet  the  unique  needs  of  the  adult  learner,  the  nontraditional 
student,  and  the  geographically  bound  student.  The  elimination  of  time,  space,  and  boundaries 
allows  The  Department  of  Technology  to  meet  its  mission  and  the  mission  of  the  institution. 

3.  The  cooperative  nature  of  the  degree  program  allows  each  component  institution  to  leverage  its 
strengths  while  at  the  same  time  minimize  the  inability  to  offer  every  course  that  is  necessary  in 
developing  a quality  degree  option.  The  increased  visibility  of  each  component  makes  it  a win- 
win  for  all  participants 

4.  The  online  degree  is  now  the  standard  vehicle  for  attracting  and  retaining  the  student  population 
that  today  has  more  options  for  higher  education  selections  than  ever  before. 

5.  The  online  degree  will  allow  The  Department  of  Technology  to  continue  to  serve  students  in 
remote  locations  who  have  been  previously  served  by  interactive  television  courses  being  phased 
out  of  the  institution  because  of  cost-cutting  measures  and  increased  line  charges. 

6.  The  online  degree  provides  three  of  the  six  courses  needed  for  trade  and  industrial  teacher 
certification  in  public  schools,  an  alternative  certification  program  in  a high  demand  area  in  the 
state. 

7.  In  the  third  year  of  the  grant  program,  the  final  three  courses  needed  for  trade  and  industrial 
certification  will  be  developed  helping  meet  the  demand  for  teachers  in  this  discipline. 


Summary  of  Program 

The  Master  of  Science  degree  in  Technology  with  the  Human  Resource  Development  (HRD)  non- 
thesis option  is  a 36  semester  credit  hour  degree  program  requiring  two  three  semester  credit  hour  core 
courses.  Research  Techniques  dXid  History  and  Philosophy  of  Technology,  18  semester  hours  in  the  area  of 
concentration  of  HRD  and  12  semester  hours  of  support  courses.  During  the  last  semester  of  the  program, 
the  candidate  must  successfully  complete  a written  and/or  oral  comprehensive  examination  on  the 
course  work  of  the  degree  plan. 

The  program  is  administered  by  The  Department  of  Technology,  The  University  of  Texas  at  Tyler 
who  will  serve  as  the  degree  granting  institution  as  authorized  by  the  Texas  Higher  Education  Coordinating 
Board.  UTT  personnel  will  develop  6 courses  to  be  used  in  the  major  of  the  degree  in  a two-year  period. 
Identified  support  courses  will  come  from  the  components  of  UT  Galveston,  UTPB,  UTA,  and  UTT  and 
other  online  TeleCampus  courses. 

Students  must  meet  the  admission  requirements  of  UTT  graduate  school  as  outlined  in  the  current 
catalog  that  can  be  accessed  at  WWW.Uttvler.edu. 

Students  completing  the  certification  courses  must  meet  the  requirements  of  the  State  Board  of 
Educator  Certification  for  T&I  teachers.  Current  guidelines  for  this  program  can  be  accessed  at  their  web 
site  of  http://www.sbec.state.tx.us/certreq/certreq.htm  or  through  the  Texas  Education  Agency  web  site  for 
Career  and  Technology  at  http://www.tea.state.tx.us/Cate/ti/index.html. 
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Abstract:  Why  do  so  many  instructional  design  projects  struggle?  Perhaps  one  reason  is  that 
the  projects  are  managed  by  instructional  designers  who  have  little  project  management 
experience.  Experienced  instructional  designers  frequently  perform  the  role  of  project 
managers  yet  ID  graduate  programs  rarely  require  project  management  courses.  Even 
instructional  designers  who  are  not  officially  “project  managers”  need  to  manage  projects, 
their  own  or  those  with  other  team  members.  Often,  however,  they  have  no  training  in  project 
management. 


Introduction 

If  an  instructional  designer  does  not  have  training  and  experience  in  project  management,  both  the  project 
and  the  instructional  designer  will  suffer.  Students  enrolled  in  Master’s  of  Business  Administration  (MBA) 
graduate  programs  spend  months,  perhaps  years,  studying  project  management  yet  instructional  designers  often 
have  little  experience  in  how  to  manage  a project  development  schedule  to  insure  that  they  are  on  schedule  and 
on  budget.  Ironically,  instructional  designers  are  often  offended  when  a subject  matter  expert  is  given  the  title 
“instructional  designer.”  They  argue  that  this  person’s  area  of  expertise  is  the  content,  not  instructional  design. 
Similarly,  an  instructional  designer  is  a subject  matter  expert  on  learning  and  the  design  of  instruction.  Why 
would  anyone  assume  that  a successful  instructional  designer  will  be  a successful  project  manager  if  that  person 
has  no  training  in  project  management? 

In  other  words,  this  is  a very  important  topic  for  instructional  designers  as  they  will  probably  be  required 
to  perform  some  or  many  project  management  responsibilities.  The  goal  of  this  paper  is  to  reflect  on  our 
experiences  as  instructional  designers  and  project  managers  and  to  provide  practical  suggestions  for  how  to 
insure  that  successful  instructional  designers  are  also  successful  project  managers. 


Expert  Opinions 

There  are  three  members  of  this  panel  plus  a chair.  Jackie  Dobrovolny,  Rod  Sims,  and  Tim  Spannaus  are 
the  panel  members.  Joe  Lamos  is  the  chair. 


Jackie  Dobrovolny 

Jackie  began  her  career  in  instructional  design  with  no  instructional  design  or  project  management 
experience  or  training.  That  was  25  years  ago.  Currently,  she  has  a Ph.D.  in  instructional  design,  has  taken 
several  project  management  (MBA)  courses  and  managed  numerous  projects.  In  2001,  she  conducted  an 
informal  study  of  instructional  designers  and  project  managers  at  the  annual  conference  of  the  International 
Society  for  Performance  Improvement.  In  three  separate  round  table  discussions,  she  asked  a total  of  38 
conference  participants  to  sort  43  index  cards,  each  of  which  contained  a brief  description  of  one  knowledge  or 
skill  (see  Table  1).  The  43  cards  were  divided  among  participants  at  each  round  table  and  participants  then 
sorted  their  cards  into  one  of  four  stacks:  instructional  designer  only,  project  manager  only,  both  instructional 
designer  and  project  manager,  or  neither  instructional  designer  nor  project  manager.  Each  person  at  each  of  the 
three  round  tables  received  approximately  four  cards  to  sort.  Once  all  43  cards  were  sorted,  the  participants  at 
each  round  table  discussed  and  altered  each  of  the  four  stacks  of  cards. 

The  results  (see  Table  1)  indicated  that  of  the  43  knowledge  and  skills,  the  participants  at  all  three  round 
tables  labeled  22  of  them  in  the  same  way.  That  is,  they  agreed  that  six  knowledge  and  skills  were  unique  to 
IDs,  three  were  unique  to  project  managers,  and  13  were  the  responsibilities  ofboth  IDs  and  project  managers. 


ID  Only 

Knowledge  & Skills 

PM  Only 

Knowledge  & Skills 

Both  ID  & PM 
Knowledge  & Skills 

1.  Apply  current  research  and 
theory  to  the  practice  of 
instructional  design 

1 . Develop  and  manage  budgets 

1.  Conflict  management  skills 

2.  Apply  principles  of  message 
design  to  page  layout  and 
screen  design 

2.  Hiring,  firing,  and 

performance  appraisal  skills 

2.  Decision  making  skills 

3.  Create  visuals  that  instruct, 
orient,  or  motivate. 

3.  Use  project  management 
software 

3.  Identify  and  resolve  ethical 
and  legal  implications  of 
instructional  design  in  the 
work  place 

4.  Develop  effective  tests  and 
surveys 

4.  Knowledge  of  the  political 
aspects  of  a project  and/or 
organization 

5.  Knowledge  of  various  learning 
theories. 

5.  Plan  and  conduct  informative 
and  efficient  meetings 

6.  Knowledge  of  various 
instructional  media,  their 
advantages  and  disadvantages 

6.  Plan  and  conduct  interviews 

7.  Problem  solving  skills 

8.  Time  management  skills 

9.  Use  active  listening  skills 
effectively 

10.  Use  project  evaluation 
strategies 

11.  Verbal  communication  skills 

12.  Work  with  reviewers,  e.g., 
subject  matter  experts,  editors, 

managers  to  insure  their  input 
improves  the  product  within 
the  allocated  time  and  budget 

13.  Written  communication  skills 

Table  1:  Instructional  Designer  (ID)  and/or  Project  Manager  (PM)  Knowledge  and  Skills 


Interestingly,  the  three  groups  completely  disagreed  on  two  responsibilities.  The  three  round  table  participants 
categorized  ‘‘Manage  revisions  cycles. ...”  as  ID  only,  PM  only,  and  both  ID  and  PM.  They  categorized  “Team 
building...  as  PM  only,  both  ID  and  PM,  and  neither  ID  nor  PM.  They  also  disagreed  on  the  following 
responsibilities:  knowledge  of  effective  motivation  and/or  reward  strategies,  knowledge  of  individual 
differences  research  and  diversity  literature,  knowledge  of  various  instructional  design  models  and  theories, 
and  use  of  return  on  investment  formulas  and  strategies  to  evaluate  a project.  This  data  suggest  that  there  is 
significant  disagreement  on  the  responsibilities  of  a project  manager.  Instructional  designers  need  to  discuss 
this  disagreement  and  at  least  clarify  the  overlap. 


Rod  Sims 

Rod  has  been  an  instructional  designer  for  several  different  organizations  and  wrote  Authorware  With 
Style.  Recently,  he  took  a position  at  Deakin  University  as  manager  of  technology-based  instruction.  His  team 
develops  both  e-leaming  courses  and  degree  programs  and  his  responsibilities  include  hiring  and  evaluation, 
staff  training,  budgeting,  and  internal  marketing.  He  rarely  exercises  his  instructional  design  skills  and  he  finds 
that  he  misses  that  activity.  While  his  management  position  is  challenging  and  fulfilling,  he  longs  to  exercise  his 
creativity  designing  instruction. 

Perhaps  it  is  possible  to  be  effective  at  and  enjoy  both  project  management  and  instructional  design.  The 
creativity  that  fuels  a good  instructional  designer  is,  however,  neglected  in  project  management  and  thus,  we 
need  to  consider  the  possibility  that  good  (creative)  instructional  designers  should  only  be  promoted  to  project 
manager  when  their  creativity  needs  a rest. 


Tim  Spannaus 

Tim  is  President  of  ibstpi  (International  Board  of  Standards  for  Training,  Performance  and  Instruction)  and 
contributing  author  of  Instructional  Design  Competencies:  The  Standards  (ERIC/IT,  2001).  The  Competency 
book  resulted  from  a study  over  several  years  of  the  required  competencies  of  instructional  designers.  By  broad 
agreement  among  the  respondents  to  the  validation  study,  management  skills  were  rated  as  highly  critical  to  the 
success  of  a designer.  Most  of  the  management  skills  were  regarded  as  advanced  skills,  in  recognition  that 
novice  designers  were  likely  to  work  as  members  rather  than  as  leaders  of  teams. 

In  a separate,  not  yet  published  study  of  training  manager  competencies,  it  has  been  found  that  managers 
need  to  have  some  instructional  design  competencies.  This  finding  confirms  the  idea  that  in  an  ideal  world,  the 
competencies  of  training  managers  and  instructional  designers  overlap. 

Tim  is  Senior  Lecturer  and  Research  Fellow  at  Wayne  State  University  (USA)  and  Chairman  and  Chief 
Learning  Architect  of  The  Emdicium  Group,  Inc. 


Joe  Lamos 

Joe  has  been  in  the  field  of  instructional  design  and  development  for  32  years.  For  more  than  20  years  he 
has  managed  instructional  design  projects  and  teams  of  instructional  designers  and  developers.  Joe  holds  a 
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Ph.D.  in  Educational  and  Psychological  Studies  from  the  University  of  Colorado.  He  is  currently  Acting 
Director  of  the  Cooperative  Program  for  Operational  Meteorology,  Education  and  Training  (COMET),  a 
division  of  the  University  Corporation  for  Atmospheric  Research  (UCAR)  in  Boulder,  Colorado. 

Most  worthy  education  or  training  efforts  today  require  a team  to  accomplish.  If  the  project  is  to  provide 
an  instructional  solution  to  a performance  problem,  because  that  is  what  is  needed,  then  project  leadership 
should  preferably  come  from  someone  who  can  articulate  a vision  of  the  learning  environment,  learning  tools  or 
materials  that  need  to  be  created.  This  doesn't  necessarily  need  to  be  someone  trained  as  an  instructional 
designer  but  it  should  be  if  the  ID  discipline  is  to  continue  to  be  relevant  relative  to  the  content  disciplines 
(which  traditional  perceptions  hold  as  the  core  need  in  doing  training  or  education).  An  important  distinction 
that  needs  to  be  made  is  that  between  project  leadership  and  project  management.  For  practical  reasons  these 
two  foci  can  exist  in  a single  individual  but  that  is  not  a necessity. 
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Advanced  Learning  Object  Hub  Application  {ALOHA) 


Introduction 

A research  and  development  project  directed  at  e-leaming  has  taken  on  the  design  and  build  of  an 
educational  object  repository.  As  part  of  this  undertaking,  various  tools  for  indexing,  sharing, 
multipurposing,  and  repurposing  learning  objects  are  also  under  development.  Advanced  Learning  Object 
Hub  Application  {Aloha)  is  one  of  these.  It  is  a suite  of  tools  used  for  the  discovery,  management, 
customization,  and  delivery  of  learning  objects.  For  use  by  indexers,  educators  and  learners,  Aloha 
includes  versatile  and  powerful  indexing  tools  for  different  metadata  standards,  inter-repository  searching 
and  content  and  object  packaging.  These  management  tools  include  the  ability  to  customize,  or  repurpose 
learning  objects  by  educators  and  their  students.  The  content  packages  of  educational  material  are  created 
in  a standardized  way  that  will  allow  them  to  be  multipurposed  between  the  desktop,  the  web,  and  CD- 
ROM  using  other  existing  software.  Development  is  ongoing  and  contributions  from  other  projects  are 
being  sought.  What  follows  in  an  outline  of  some  of  Aloha  *s  features. 

IMS,  CanCore,  DublinCore  Metadata  Indexing 

The  current  primary  function  ofi4/o/zfl  is  indexing  educational  objects.  It  is  a Java-based  client  tool.  The 
decision  was  based  on  the  power  and  flexibility  Java  has  demonstrating  in  interacting  with  the  world  wide 
web  (Jones,  1997).  Its  flexible  interface  is  friendly  for  amateur  users  and  customizable  for  the  professional 
indexer.  It  is  easy  to  create,  share  and  customize  indexing  templates  and  forms,  It  is  able  to  automatically 
parse  and  fill  in  the  metadata  for  over  200  file  types. . It  makes  marking  up  IMS  or  other  forms  of 
metadata  much  easier.  Administration  tools  managing  workflow  issues  with  multiple  indexers  including 
the  librarian,  the  educator,  and  the  media  developer  are  also  available.  This  supports  the  idea  of  modularity 
where  different  users  can  index  objects  in  context  specific  ways  and  share  their  metadata  with  other  users 
and  metadata  schemas  (Duval  et  al  2002). 


Any  valid  xml  schema  can  be  brought  \nio  Aloha  and  users  can  begin  to  use  the  Aloha  interface  to  help 
them  build  productive  tools  for  indexing  tasks.  Learning  Objects  and  metadata  can  be  uploaded 
simultaneously  with  a touch  of  a button  to  an  appropriate  media  server,  handling  the  job  of  a FTP  program. 

Inter-repository  Web  Searching 

A stand-alone  application  that  serves  as  a client  for  the  world -wide  web.  Aloha  allows  multiple  resources  to 
be  searched.  Like  Apple’s  Sherlock,  its  open  architecture  allows  the  addition  of  search-plugs  to  leading 
learning  object  repositories.  It  comes  equipped  with  plug-ins  for  several  existing  learning  object 
repositories.  Using  xml-rpe  other  repositories  can  be  added  with  ease. 

There  are  currently  peer-to-peer  options  available  to  the  user  of  ALOHA  through  the  SPLASH  peer-to-peer 
network  being  established  by  the  POOL  (what  does  this  stand  for  and  is  there  a web  link)  project.  The 
peer-to-peer  network  will  be  like  any  other  repository  that  you  want  to  search,  you  need  pick  it  as  a search 
option.  Aloha 's  search  capabilities  are  simple  to  use,  easy  to  expand,  and  more  effective  than  standard  web 
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searches  as  they  are  based  on  well-defined  standardized,  metadata.  The  current  search  engines  on  the 
Internet  are  powerful  but  they  are  accessing  an  unmanaged  resource  which  often  returns  irrelevant  material 
(Pollock  & Hockley  1997) 


The  best  centralized  learning  object  repositories  will  be  vetted  and  restricted  in  their  access,  in  so  far  as 
uploading  of  learning  objects.  The  ability  to  set  the  access  level  of  your  uploaded  object  allows  you  to 
create  trusted  groups  that  can  only  view  your  objects  if  you  give  them  permission.  This  will  allow  personal 
collections  of  learning  objects  to  be  shared  amongst  ad  hoc  communities  of  educators  and  learners. 

Desktop  repositories  can  be  batch  uploaded  to  centralized  repositories  if  desired. 

Content  Management  and  Multipurposing 

Once  a user  has  found  a resource,  whether  it  is  downloaded  or  not,  or  if  they  have  created  a learning  object 
themselves.  Aloha 's  future  features  include  content  management  and  multipurposing  functions.  These 
features  are  important  to  enable  users  to  download  objects,  manage  courses,  and  deliver  those  objects  in  a 
revolutionary  way.  Aloha  will  allow  users  to  create  SCORM -based  edulists  and  courses  and  organize  these 
for  personal  use  or  sharing  with  educators  or  students  (ADL  2002).  Once  arranged,  Aloha  will  let  users 
share  their  edulists,  classes,  and  courses  with  students  and  other  educators  through  the  web,  email,  or 
uniquely  through  The  will  also  be  able  to  publish  their  course  packages  with  any  software  that  will 

read  these  standardized  packages.  Learning  objects  can  be  visualized  uniquely,  customized  to  the  users 
needs  allowing  content  re-purposing. 

Summary 

This  paper  descnbcs  Aloha  in  brief  detail.  The  presentation  will  demonstrate  some  of  its  features.  One  of 
the  most  important  principles  of  Aloha  is  to  open  its  architecture  even  further  to  allow  a sustainable 
development  environment  and  promote  its  usage  among  educators. 
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Abstract 


In  the  work  reported  here,  we  have  set  ourselves  the  goal  of  creating  cost-effective  high- 
quality  automated  on-line  learning  environments  for  procedural  tasks.  The  result  is  an  initial 
implementation  of  a system  called  the  Flowchart  Task  Tutoring  and  Tracing  Toolkit  (F'T*).  The 
procedure  language  developed  for  FT*  is  significantly  richer  than  that  used  in  most  previous 
scenario-based  procedural  Intelligent  Tutoring  Systems  (ITSs).  On  the  other  hand,  it  is  less 
expressive  than  the  plan  languages  used  in  previous  model-based  procedural  ITSs.  Our  aim  is  to 
share  some  of  the  authorability  advantages  of  scenario-based  systems,  while  achieving  much  of 
the  generative  power  of  model-based  systems.  To  date,  we  have  designed  and  implemented  the 
language  with  an  ASCII  encoding,  built  and  packaged  an  interpreter  that  can  interact  with 
external  simulations  and  prototyped  a first  application,  and  begun  to  explore  the  interaction  of  the 
interpreter  with  a student  modeling  component. 

1.  Goals  and  Definitions 

In  the  work  reported  here,  we  have  set  ourselves  the  goal  of  creating  cost-effective  high-quality 
automated  on-line  learning  environments  for  procedural  tasks.  The  result  is  an  initial  implementation  of  a 
system  called  the  Flowchart  Task  Tutoring  and  Tracing  Toolkit  (FT"*).  We  start  by  elaborating  on  what, 
exactly,  our  goal  description  is  intended  to  mean. 

By  procedural  tasks,  we  mean  any  process  to  be  carried  out  (or  super\dsed)  by  a person,  where 
that  process  calls  for  a reasonably  well-specified  (though  by  no  means  rigidly  fixed)  sequence  of 
operations.  Examples  might  include: 

• controlling  complex  technological  systems  (e.g.  industrial  plants,  satellite  clusters), 

• dealing  with  narrow  social  or  business  interactions  (e.g.  staffing  a help-desk  or  a check-out  line), 

• or  even  mastering  the  cognitive  aspects  of  complex  constellations  of  physical  skills  (e.g  learning 

to  perform  cardiac  life  support  techniques,  or  machine  maintenance  activities). 

By  high-quality  learning  environments,  we  mean  settings  that  incorporate,  and  effectively 
interrelate  adaptive  instruction,  practice,  and  coaching.  In  on-line  environments,  practice  is  typically 
provided  by  interaction  with  simulations.  Automated  instruction  and  coaching  call  for  artificial 
intelligence  (AI)  components  that  typically  go  under  the  heading  of  intelligent  tutoring  systems  (ITSs). 

By  cost-effective,  we  mean  our  goal  is  to  lower  the  costs  of  fielding  these  systems.  We  believe 
that,  at  least  for  procedural  tasks,  raw  technological  capability  is  no  longer  the  barrier  to  widespread 
adoption  of  effective  advanced  learning  environments.  We  know  how  to  build  interesting  and  useful 
systems,  but  typically  it  takes  too  much  time,  expertise  (and  therefore  money)  to  build,  distribute,  and 
support  them.  Such  systems  require  a mix  of  domain,  instructional,  and  computer-science  expertise. 

Thus,  in  summary,  we  are  interested  in  building  a particular  class  of  simulation-based  intelligent 
tutoring  systems,  and  doing  so  in  a way  that  makes  these  systems  more  affordable  in  both  the  short  and 
long  run.  We  believe  this  is  the  best  way  to  make  truly  engaging  and  effective  instruction  available  to  the 
widest  range  of  students.  In  addition,  the  architecture  we  propose  extends  neatly  to  provide  on-line  task 
support  in  those  cases  where  the  procedural  task  is  naturally  performed  with  computer-support. 
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2.  Comparing  Procedure-Oriented  ITSs 

There  have  been  a number  of  systems  developed  to  support  intelligent  tutoring  on  procedural 
tasks,  taking  both  scenario-based  and  model-based  approaches.  Prior  to  the  work  described  here,  we  at 
SHAI  had  developed  a library  module  called  the  Task  Tutor  Toolkit  (T^)  (Ong  and  Noneman,  2000)  that 
emphasized  authorability  of  relatively  simple  scenarios.  Likewise,  Guralnik  (1996)  describes  an 
authoring  tool,  called  MOPed-II,  that  applies  a content  theory  of  task  knowledge  to  enable  the  tutoring 
system  to  generate  replies  to  common  procedure-oriented  questions.  The  RIDES  system  (Munro  and 
Pizzini,  1995)  and  its  successors  are  probably  the  best-developed  systems  extant  for  authoring  graphical 
procedure-oriented  training  simulations  (frequently,  of  devices)  integrated  with  the  intelligent  tutor;  it  too 
takes  primarily  a scenario-based  approach.  In  contrast,  we  could  cite  more  ambitious  and  elaborate 
model-based  (and  dialog-oriented)  systems  such  as  the  BE&E  tutor  (Core,  Moore,  and  Zinn,  2000),  or  the 
sequence  of  systems  (i.e.  TOTS,  STEVE,  PAT,  and  PACO)  described  by  Rickel,  et.  al,  (2000). 

These  systems  vary  on  several  dimensions: 

• Type  of  task  environment  simulation  and/or  degree  of  integration  with  a particular  simulator; 

• Complexity  of  procedures  that  can  be  specified/trained; 

• Relationship  between  procedure  specifications  and  “problems”  or  “scenarios”; 

• Relationship  between  procedure  specifications  and  student  models; 

• Attention  to  and  support  for  authoring; 

• Support  for  lessons/tasks  other  than  procedure  practice. 

There  are  two  major  stances  on  the  issue  of  how  a procedural  ITS  package  might  relate  to  a 
simulation  environment.  Either  the  simulator  is  integrated  into  the  ITS  package  as  with  RIDES,  or  the 
tutoring  component  is  explicitly  packaged  as  a module  designed  to  be  interfaced  with  a variety  of  external 
simulators  as  with  T^  and  the  TOTS  et.  al.  line  of  work.  The  first  approach  allows  for  a highly  integrated 
system  that  can  fully  support  all  aspects  of  ITS  development.  The  second  approach  makes  more  sense  in 
domains  where  simulators  are  either  already  extant,  or  are  likely  to  be  more  complicated  than  (or 
inappropriate  for)  the  simulation  and  authoring  tools  in  the  integrated  environment.  Our  current  work  on 
FT^  follows  in  the  tradition  of  T^  in  being  separate  from  any  particular  simulator. 

The  issue  of  procedure  complexity  is  currently  the  strongest  point  differentiating  FT"^  from  prior 
work  on  procedure-oriented  ITSs.  Scenario-based  ITSs  such  as  RIDES  and  T^  adopt  something  like  a 
simple  tree  representation  for  procedures,  where  internal  tree  nodes  can  represent  either  ordered 
sequences  of  children  or  unordered  collections  of  children,  and  external  leaf  nodes  represent  primitive 
actions.  T^  adds  the  capability  of  generalizing  the  parameters  to  primitive  actions  to  ranges  or  sets  rather 
than  fixed  values.  FT'^  for  its  part,  allows  a much  more  general  procedure  representation  that  includes  not 
only  ordered  and  unordered  sets  of  steps,  but  also  (1)  named  sub-procedures  with  parameters  and  local 
variables  to  supplement  global  state;  (2)  conditional  branches,  and  thus  potentially  loops;  (3)  exception 
conditions;  (4)  alternative  steps;  and  (5)  optional  steps.  Section  3 presents  the  full  procedure  language  in 
detail.  While  a significant  generalization  over  other  scenario-based  tutors,  procedures  remains  less 
expressive  than  the  plan  language  used  in  most  model-based  tutors. 

Procedure  specifications  in  scenario-based  systems  are  created  as  part  of  defining  a particular 
lesson  or  problem.  Each  procedure  specification  essentially  represents  a particular  scenario  a student  may 
be  asked  to  work  through.  Supported  by  the  more  complex  procedure  language,  FT"^  takes  the  stance  that 
a procedure  specification  may  represent  a particular  scenario,  or  a large  chunk  of  the  curriculum  (from 
which  many  different  scenarios  might  be  generated).  Thus  FT^  aims  to  bridge  the  gap  between  scenario- 
based  and  model-based  procedural  ITSs  with  a representation  of  intermediate  complexity.  The  basic  idea 
is  that  mastery  of  the  full  procedure  requires  demonstrated  ability  to  execute  all  the  right  steps  at  the  right 
time  (essentially,  to  run  the  student  through  a set  of  scenarios  that  exhaustively  cover  all  the  branches  and 
transitions  in  the  procedure). 

The  current  FT"^  prototype  does  not  yet  fiilly  implement  this  approach,  but  many  of  the  underlying 
facilities  are  in  place.  In  particular  the  system  maintains  student  models  that  reflect  the  transition 
structure  of  the  encoded  procedures.  The  system  scores  a student  on  three  dimensions:  (1)  how  often  they 
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correctly  execute  an  action  indicating  they  followed  a valid  transition  in  the  procedure  graph;  (2)  how 
often  they  fail  to  execute  an  action  that  is  required  by  a forced  transition  in  the  procedure  graph;  (3)  how 
often  they  incorrectly  execute  an  action  that  represents  an  incorrect  transition  out  of  a conditional  test. 

This  strongly  procedure-centered  view  of  student  model  is  significantly  different  from  the  learning 
objectives  or  principles  centered  view  taken  by  other  systems. 

One  of  the  nice  features  of  a simpler  procedure  language  is  that  it  is  more  easily  authorable  by 
subject  matter  experts  who  are  not  computer  science  experts.  Both  RIDES  and  aim  to  ease  scenario 
authoring  by  supporting  a scripting  by  demonstration  capability  (in  both  cases  complemented  by  a post- 
editing phase  allowing  somewhat  different  degrees  of  procedure  generalization,  as  described  earlier).* 
Currently  FT^  offers  no  authoring  support  for  its  procedures,  and  this  is  a recognized  gap  in  its  aim  to 
address  the  problem  of  ITS  cost.  However,  we  have  plans  to  integrate  FT"*  both  with  the  enacted 
sequence  recording  capabilities  of  T^  and  with  a graphical  flowchart  editor  that  enables  much  more 
significant  generalization  of  enacted  sequences.  We  note  that  the  design  of  the  FT*  procedure  language 
was  shaped  by  the  expectation  that  a simple  flowchart  interface  would  be  created;  it  relies  on  a primitive 
branch  and  link  representation  rather  than  more  modem  stmctured  programming  language  constmcts. 

We  note  also,  that  flowcharts  have  been  exploited  in  instructional  and  task  aiding  system  in  other 
contexts  and  for  other  purposes.  For  instance,  FAST  (Thompson,  Ockerman,  Najjar,  & Rogers,  1997)  is  a 
wearable  electronic  performance  support  system  intended  to  offer  factory  personnel  just-in-time  advise. 
One  form  of  advice  available  is  explicit  flowchart  representations  of  tasks  to  be  accomplished  presented 
to  workers  in  response  to  questions  such  as  “How  do  I do  this?”  Slightly  further  afield,  Puerta  (1993) 
describes  how  flowcharts  can  be  used  to  support  authoring  of  task-specific  support  tools.  In  his  examples, 
physicians  design  protocol  flowcharts,  that  then  serve  to  help  stmcture  system  behavior  and  interfaces. 

3.  The  Frt  Procedure  Flowchart  Language 

In  this  section  we  provide  a detailed  specification  of  the  FT*  procedure  flowchart  language.  Each 
flowchart  represents  a named  procedure.  There  can  be  any  number  of  procedures  defined  for  an  ITS,  and 
steps  in  one  procedure  can  invoke  other  procedures  (including  themselves  recursively,  either  directly  or 
indirectly). 

Each  procedure  contains  a set  of  steps,  linked  to  one  another  through  “next”  pointers.  Basic  steps 
have  just  a single  “next”  pointer,  while  test  steps  can  have  two  (or  more)  “nexts”.  Steps  (or  branches  out 
of  test  steps)  without  an  explicitly  defined  “next”  are  taken  to  constitute  the  end  of  a path.  A path  may  be 
a route  through  a whole  procedure,  or  may  be  the  end  of  one  chain  in  a parallel-step  (which  will  be 
discussed  at  greater  length  below). 

In  addition,  a procedure  may  specify  exception  conditions.  When  an  exception  condition 
becomes  tme,  all  other  steps  become  unavailable,  and  control  jumps  to  the  step  specified  by  the  exception 
handler.  When  the  chain  of  steps  beginning  with  this  initial  step  of  the  exception  handler  is  completed, 
control  can  either  return  to  the  point  where  the  exception  occurred  (indicated  by  a special-purpose  return- 
step),  or  it  can  jump  to  a statically  specified  point  in  the  flowchart. 

Ignoring  exception  return-steps,  procedure  start-steps  and  other  such  special-purpose  step  types, 
there  are  five  main  kinds  of  steps: 

• Primitive-Step:  A primitive-step  corresponds  to  a real-world  action.  In  order  to  move  past  such 
a step,  some  action  must  be  taken. 

• If-Step:  An  if- step  is  a kind  of  test-step  that  is  a simple  if-then-else  two-way  branch.  If  the  test 
expression  evaluates  to  true,  then  the  first  path  should  be  taken,  otherwise,  the  second  path  should 
be  taken.  Thus  an  if-step  has  two  next  steps,  but  only  one  or  the  other  is  ever  available. 

• Cond-Step:  A cond-step  is  a more  flexible  kind  of  test-step  than  the  if-step;  it  resembles  the 
“cond”  form  of  the  Lisp  or  Scheme  programming  languages  in  that  it  can  have  any  number  of 
distinct  tests,  each  of  which  controls  a possible  branch.  The  tests  are  evaluated  in  order,  and  the 


* An  interesting  feature  of  T^  is  that,  contrary  to  the  approach  in  RIDES,  it  does  not  require  tight  integration  with  its 
own  simulator  system  to  achieve  this  scenario  scripting  by  demonstration  capability. 


branch  paired  with  the  first  test  that  evaluates  to  true  is  considered  active.  A cond-step  has  as 
many  next  steps  as  it  has  tests,  but  again,  only  one  is  ever  available. 

• Procedure-Step:  A procedure-step  represents  an  invocation  of  a nested  procedure.  When  a 
procedure-step  is  reached,  control  jumps  to  the  nested  procedure.  When  that  nested  procedure 
completes,  control  resumes  with  the  procedure-step’s  designated  next  step. 

• Parallel-Step:  A parallel  step  consists  of  several  possible  chains  of  execution,  some  of  which 
may  be  specially  designated  as  optional.  A parallel  step  specifies  a minimum  and  a maximum 
number  of  the  main  (non-optional)  chains  that  must  be  completed.  A chain  is  completed  by 
reaching  a step  in  the  chain  that  has  no  next  step.  The  user  may  jump  back  and  forth  between 
steps  in  different  chains,  and  can  start  new  main  chains,  up  to  the  designated  maximum  number. 
Once  the  minimum  number  of  main  chains  are  completed,  and  so  long  as  no  other  chains  in  the 
parallel-step  are  in  progress,  the  parallel-step’s  own  next  step  may  be  executed. 

Parallel-steps  are  the  most  complicated  type  of  step.  They  were  designed  to  support  unordered  or 
parallel  actions  (i.e.  a set  of  N main  chains,  with  the  minimum  and  maximum  chain  counts  both  set  to  N), 
alternative  actions  (i.e.  a set  of  N main  chains,  with  the  minimum  and  maximum  both  set  to  1),  and 
optional  actions  (i.e.  allowance  for  a set  of  optional  chains).  But  the  minimum/maximum  capability  also 
allows  for  the  somewhat  odder  possibility  of  unordered  subsets  of  the  main  chains  (i.e.  a set  of  N main 
chains,  with  the  minimum  set  to  J and  the  maximum  set  to  K,  where  1 <=  J <=  K <=  N). 

The  FT"*  procedure  flowchart  language  can  be  cast  in  several  different  forms.  Currently,  we  have 
defined  an  ASCII  file  format  (based  on  Lisp-style  S-expressions),  and  an  in-memory  Java  data  structure 
format  (suitable  for  tracing  through  procedures  and  determining,  at  each  point,  what  are  the  current 
allowable  set  of  actions  a user  might  take).  As  suggested  by  the  title  of  the  system,  the  language  was  also 
designed  to  have  a direct  mapping  to  graphical  flowcharts.  Such  a graphical  language  has  been  designed, 
but  not  yet  implemented.  Figure  1 shows  a sample  procedure  for  making  cappuccino,  primarily  intended 
to  illustrate  the  majority  of  features  of  the  language  (and  to  highlight  possible  problems  as  well). 


Figure  1.  A Flowchart  View  of  the  Sample  “Cappuccino”  Procedure. 
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The  outermost  heavy-lined  box  in  Figure  1 represents  the  entire  ‘‘Cappuccino”  procedure.  It  is 
divided  into  two  parts:  (1)  the  mainstream  procedure  flow  (the  larger  area  with  the  darker  background), 
and  (2)  the  exception  conditions  (the  smaller  area  to  the  right  with  a lighter  background).  The  main 
procedure  is  composed  of  three  major  sets  of  steps,  where  each  set  is  grouped  into  a complex  parallel- 
step.  The  parallel-steps  are  the  boxes  with  medium-weight  lines,  and  with  Min/Max  counts  shown 
attached.  Again,  the  parallel-steps  are  potentially  broken  up  into  two  parts:  (1)  the  set  of  main  chains 
(shown  with  a darker  background),  and  (2)  the  optional  chains  (present  only  in  the  middle  parallel-step, 
and  shown  with  a lighter  background).  The  thin-lined  rectangles  represent  primitive-steps.  The  thin-lined 
rounded  rectangle  “Grind  Beans”  represents  a procedure-step  (the  invocation  of  some  other  named 
procedure,  whose  detailed  structure  is  not  shown  in  this  figure).  The  thin- lined  diamond  “Enough 
Foam?”  represents  an  if-step  that  has  the  effect  of  creating  a loop. 

From  the  start  state  of  this  procedure,  the  only  reachable  step  is  the  first  parallel-step  that 
packages  the  two  “Select  Cup”  primitive-steps  as  the  starting  (and  ending)  points  of  two  alternate  chains. 
These  are  alternate  chains  because  the  min  and  max  constraints  are  both  set  to  1.  That  means  that  as  soon 
as  one  of  the  “Select  Cup”  steps  is  executed,  a first  chain  has  been  started,  and  since  the  maximum 
number  of  allowed  chains  is  1,  no  other  chain  can  be  executed.  Whichever  “Select  Cup”  primitive-step  is 
executed  also  completes  the  first  parallel-step. 

From  that  point,  it  is  legal  to  start  any  of  the  three  chains  in  the  second  parallel  step.  The  chain 
consisting  solely  of  “Pour  Syrup”  is  an  optional  chain,  and  so  its  execution  does  not  count  towards  the 
min/max  counts  of  the  parallel-step.  With  min  and  max  both  set  to  2,  the  two  main  chains  must  both  be 
executed  to  fully  satisfy  the  parallel-step.  The  two  main  chains  are  a bit  more  complex  than  those  in  the 
first  parallel-step.  The  chain  that  starts  with  “Grind  Beans”  is  a straight  sequence,  except  that  the  “Grind 
Beans”  step  is  shown  as  a procedure -step — a reference  to  another  procedure — and  therefore  could  require 
some  arbitrarily  complicated  set  of  actions  before  it  is  possible  to  advance  to  the  “Load  Grounds”  step.  In 
the  second  chain,  we  illustrate  the  use  of  an  if-step,  and  the  creation  of  a loop;  the  point  here  is  simply  to 
show  that  tests  of  environmental  conditions  are  allowed  and  can  be  used  to  determine  what  steps  ought  to 
be  performed  next.  Finally,  as  an  aside,  we  note  that  the  chains  of  this  parallel-step  adhere  to  the 
structural  rule  that  each  chain  is  self-contained:  there  can  be  no  crossing  over  from  the  steps  of  one  chain 
to  those  of  another. 

The  third  parallel-step  is  similar  to  the  first,  but  since  its  min  and  max  are  both  2,  it  amounts  to  an 
unordered  “and”  rather  than  an  “or” — it  is  necessary  to  both  serve  the  cappuccino  and  wash  up  to 
complete  the  parallel-step,  and  thereby  to  complete  the  entire  “Cappuccino”  procedure. 

The  only  aspect  of  the  procedure  we  have  not  yet  discussed  is  the  exception  condition  set  to 
watch  for  attempts  to  steam  milk  when  the  pitcher  is  empty.  Structurally,  the  interesting  point  is  the 
return-step  at  the  end  of  the  exception-handler;  this  should  cause  control  to  return  to  “Steam  Milk”  step 
(which  is  the  only  place  the  exception  could  have  been  noticed).  Of  course  an  exception  that  can  occur  in 
only  one  place  might  be  defined  as  an  explicit  test  and  branch  at  the  point  in  question.  Alternately,  this 
exception  might  better  be  defined  in  the  scope  of  the  second  parallel-step  rather  than  in  the  scope  of  the 
procedure;  however,  our  current  syntax  does  not  allow  such  distinctions  in  exception  scope. 

As  noted  earlier,  this  procedure  language  is  less  expressive  than  a plan-oriented  language;  this 
leads  to  procedures  that  are  less  flexible  than  they  logically  could  be.  For  instance,  there  is  no  real  reason 
to  force  the  cup  selection  to  be  performed  before  starting  in  on  preparing  the  components  of  the  drink. 
Logically,  the  cup  only  needs  to  be  selected  before  the  first  pour  or  ladle  step  (to  ensure  there  is 
something  to  pour  our  ladle  into).  On  the  other  hand,  in  procedural  training  applications,  it  is  quite 
common  for  die  “by  the  book”  way  to  accomplish  a task  to  be  defined  with  limited  flexibility  (compared 
to  all  the  possible  sequences  that  might  accomplish  the  goal  in  question).  At  least  for  the  applications  we 
have  considered,  we  are  not  very  concerned  that  the  FT^ procedure  language  is  too  restrictive. 

4.  The  SatCon  Project 

The  work  reported  here  was  carried  out  in  the  context  of  a project  called  SatCon  (for  Satellite 
Control).  The  point  of  this  project  has  been  to  demonstrate  how  AI  technologies  can  be  applied  to  build  a 
system  for  US  Air  Force  satellite  controllers  that  provides  both  intelligent  simulation-based  training  and 
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job  task-support  during  actual  operations.  The  project  ran  for  six  months,  resulting  in  a first  prototype 
that  links  the  current  FT'*  with  a simulator  and  a GUI  through  an  agent  community.  We  expect  to  start 
development  of  a more  complete  operational  system  in  the  summer  of  2002,  as  part  of  a two-year  follow- 
on  project.  Section  5 on  future  work  lays  out  some  of  our  goals  for  the  ongoing  effort. 

Given  the  dual  application  requirement,  the  multiplicity  of  potentially  relevant  legacy  systems, 
and  the  expected  overall  system  complexity,  an  agent  architecture  was  early  identified  as  particularly 
appropriate.  Given  the  nature  of  the  satellite  control  task — complex  and  somewhat  variable,  with  a wide 
range  of  contingencies,  and  with  tremendously  valuable  resources  at  stake — operators  in  current  (and 
projected)  practice  are  largely  restricted  to  acting  within  the  bounds  of  pre-specified  scripts  or  procedures. 
Those  procedures  are  far  too  conylex  to  be  expressible  in  the  T^  task  language,  thus  our  invention  of  the 
significantly  more  expressive  FT"^  language. 

To  date,  we  have  (1)  designed  the  F'T*  procedure  language  (including  internal  data  structures  and 
external  representations  in  both  ASCII  and  graphical  form),  (2)  built  a parser  to  transform  ASCII 
specifications  into  data  structures,  and  implemented  a tracing  algorithm  that  tracks  allowable  next  actions, 
(3)  designed  a complementary  student  modeling  formalism,  and  mechanisms  to  use  such  models  to  drive 
selection  and/or  generation  of  training  scenarios,  (4)  implemented  student  model  creation  and  update 
during  the  procedure  tracing  process,  (5)  packaged  the  existing  FT‘*  components  so  as  to  separate  the 
training  and  task-support  capabilities  from  any  simulation  or  operational  environment,  and  (6)  constructed 
a first  demonstration  application  by  embedding  F'T*  in  a distributed  agent  environment. 

5.  Future  Work 

Future  work  will  focus  on  authorability.  This  will  include  porting  the  existing  T^  capability  for 
creating  straight-line  procedure  specifications,  and  combining  it  with  the  interactive  flowchart  editor.  In 
addition,  we  will  complete  implementation  of  the  mechanism  for  automatically  generating  individual 
scenarios  based  on  general  procedure  specifications  combined  with  student  models.  With  these  additions, 
we  expect  to  fulfill  our  goal  of  cost-effective  high-quality  automated  on-line  learning  environment  for 
procedural  tasks. 
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Abstract:  A number  of  educational  and  societal  trends  are  analyzed  for  their  potential  impact  on 
the  practice  of  distance  education.  The  trends  reflect  the  various  ideologies  and  values  currently 
found  within  professional  communities  and  publics  at  large.  How  the  various  trends  push  and  pull 
in  different  directions,  and  how  they  may  lead  to  alternative  futures  for  distance  education — These 
are  the  subject  of  the  paper. 


Distance  education  has  been  described  as  a disruptive  technology — an  innovation  that,  while  initially  posing  no 
threat  to  established  institutions,  over  time  challenges  conventional  practices  and  contributes  to  new  ways  of 
thinking  (Archer,  Garrison,  & Anderson,  1999).  I agree  with  this  assessment.  Distance  education  has  already 
become  more  than  an  alternative  form  of  delivery.  It  has  shaken  up  the  educational  establishment,  especially  at 
higher  education  and  corporate  levels.  Precisely  because  of  this  success,  though,  assessing  potential  et  midstream 
can  be  a challenge.  At  times  like  these,  instructional  designers  and  providers  can  benefit  enormously  by  stepping 
back,  reviewing  broad  trends,  and  forecasting  likely  scenarios  based  on  those  trends.  That  is  the  purpose  of  this 
paper.  A number  of  current  trends  are  outlined  and  their  likely  impact  on  education  considered.  Then  some  brief 
pointers  toward  the  future  are  presented. 

Observers  of  distance  education  can  point  to  a number  of  discernible  trends  affecting  practice  over  the  past 
several  years.  Trends  do  not  determine  the  future,  but  they  can  provide  a basis  for  present  action  and  an 
understanding  of  possible  futures.  The  trends  discussed  below  are  pulling  in  different  directions — some  fostering 
change,  others  reinforcing  the  status  quo;  some  using  technology  in  a controlling  way,  others  using  technology  to 
empower  individuals. 

Technologizing  of  School  Systems 

School  systems,  particularly  American  K12  public  schools,  are  facing  pressures  to  modernize  and 
“technologize”  their  processes  by  establishing  more  predictable  outputs  and  methods  (Tyack  & Cuban,  1995). 
Although  American  public  schools  are  the  immediate  point  of  discussion  for  this  section,  the  principles  extend  to 
any  schooling  or  educational  system. 

Standardized  competencies.  The  standards  movement  has  resulted  in  a common  set  of  learning  outcomes, 
presented  in  quasi-behavioral  language,  but  at  a fairly  high  level  of  generality  to  accommodate  different  teaching 
methods. 

System-side  assessments  and  accountability.  Standardized  assessments  are  part  of  an  overall  move  to  make 
schools  more  accountable  to  the  public  or  to  the  government.  Test  scores  are  thus  indicators  of  tax  dollars  and 
government  resources  being  well-spent. 
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Incentivization  of  funding.  Increasingly,  operational  funds  are  being  tied  to  compliance  with  specific  mandates 
and  regulations.  These  mandates  are  made  at  levels  beyond  individual  schools,  intended  to  bring  schools  into  line 
with  desired  teaching  and  assessment  practices. 

Regulated  processes  and  methods.  With  increased  emphasis  on  high-stakes  testing,  teachers  are  being  asked  to 
fit  their  teaching  methods  more  closely  to  the  larger  system  of  goals  and  assessments.  Often,  methods  are  established 
and  prescribed  by  schools,  districts,  and  even  states,  leaving  less  room  for  professional  judgment  and  variations  in 
teaching  style. 

Alignment  of  outcomes,  assessments,  and  methods.  In  a well -coordinated  schooling  system,  an  alignment  exists 
between  processes  and  outputs — in  particular,  between  standardized  outcomes,  assessment  measures,  and  acceptable 
teaching  methods. 

De-professionalizing  the  teacher  *s  role.  Tighter  alignment  of  processes  and  outputs  has  a definite  impact  on  the 
teacher’s  role.  As  suggested  above,  the  teacher  often  assumes  a “technician’s”  role  of  implementing  prescribed 
rules,  as  opposed  to  a professional  ’ s role  of  exercising  judgment. 

Learner-  and  User-Centered  Philosophies 

At  the  same  time  schools  are  moving  toward  efficiency  and  control,  the  mood  among  many  educators  is 
definitely  learner-centered.  The  constructivist  movement  in  education  stresses  individual  and  collaborative 
constmction  of  meaning.  While  many  teachers  wish  they  could  teach  in  more  learner-centered  ways,  the  system  can 
make  it  difficult.  Teachers  and  trainers  thus  face  a certain  tension  between  efficiency  and  control  on  the  one  hand, 
and  learner-centered  flexibility  on  the  other. 

Convenient,  anytime/anywhere  access.  Instead  of  students  going  to  class,  learning  is  coming  to  the  student — in 
the  workplace  or  at  home.  Just-in-time,  just-in-place  learning  resources  are  increasingly  available  to  learners  in  their 
normal  living  settings. 

Constructivism.  Constructivist  teaching  strategies  give  students  complex  and  engaging  projects  and  tasks  to 
perform,  with  scaffolding  and  support  from  colleagues  or  a teacher/facilitator.  Learning  happens  via  meaningful 
experiences  and  direct  encounters.  Examples  include  guided  inquiry  activities  such  as  Webquests,  or  problem-based 
learning  cases. 

Field-based  and  informal  learning.  Informal  learning  happens  by  virtue  of  participation  in  some  other  valued 
activity,  e.g.,  work  or  play.  Field-based  learning  refers  to  semi -structured  activities  such  as  internships,  practicum 
experiences,  expeditions  and  trips,  etc. 

High-touch  connectivity.  Many  technology  innovators  maintain  a dual  focus  in  their  dissemination  efforts — 
providing  advanced  information  tools  coupled  with  high  levels  of  personal  support  and  connectivity  among 
individuals  (Naisbitt,  1982;  Spitzer,  2001).  The  commitment  to  both  high-tech  and  high-touch  suggests  a need  to 
make  tools  people-centered  rather  than  the  reverse  (Norman,  1993). 

How  does  distance  education  fit  within  these  conflicting  forces  of  standardization  versus  learner-centered 
values?  Ironically,  distance  education  can  be  seen  to  support  both  movements.  A well-conceived  distance  education 
program  can  fit  squarely  within  a strictly  controlled  standards -based  curriculum.  At  the  same  time,  some  aspects  of 
the  distance  learning  experience  are  completely  learner-centered,  especially  the  access  and  convenience  afforded 
students. 

Moves  to  Streamline  and  Automate  Instructional  Design 

In  this  section  several  trends  are  outlined  that  relate  to  making  instructional-design  processes  more  efficient  and 
effective  through  a process  of  streamlining  or  automation  of  tools  and  processes. 

Standardized  taxonomies  for  learning  outcomes  and  instructional  strategies.  A basic  precept  of  instructional 
design  is  known  as  the  “conditions-of-leaming”  assumption  (Ragan  & Smith,  1996).  Before  producing  instruction, 
you  determine  what  you  need  to  teach,  as  well  as  your  audience  and  situation.  Then  and  only  then  can  good 


O 


instructional  strategies  be  determined.  According  to  this  view,  rules  connecting  goals  with  instructional  strategies 
become  essential  to  the  systematic  practice  of  instructional  design. 

Data-driven  generation  of  rule-based  instruction.  An  automated  version  of  this  line  of  thinking  seeks  to  make 
instructional  development  largely  a data-driven  activity.  Automated  instructional  design  is  an  ambitious  concept: 
Just  plug  in  data  concerning  learning  outcomes,  learners,  and  situation,  and  the  rule  system  spits  out — not  only  a set 
of  recommended  strategies — but  draft  instructional  materials.  While  research  on  automating  instruction  has 
continued  over  two  decades,  the  agenda  is  still  in  its  infancy,  and  may  prove  increasingly  viable  in  coming  years. 

More  flexible,  adaptable  authoring  tools.  Powerful  authoring  tools  break  from  linear  processes  and  allow  late 
and  iterative  changes  in  design,  more  in  keeping  with  natural  design  thinking  and  envisioning.  Examples  include  3- 
D modeling  programs  that  allow  for  various  uses  once  the  model  has  been  created;  or  authoring  tools  that  allow 
prototyping  and  creation  of  dummy  interfaces,  to  test  out  concepts  at  early  stages  before  investing  in  full-design 
development. 

More  modular,  re-usable  design.  The  learning-objects  movement  is  about  reusing  content  to  make  efficiency 
gains  in  the  instructional  development  process.  Because  digital  content  is  “non-rival”  in  nature  (i.e.,  copies  of  equal 
quality  can  easily  be  made  from  originals),  reusability  for  various  purposes,  media,  and  occasions  become  an 
inviting  possibility  (Wiley,  2002). 

The  Digital  Shift:  Advances  in  Information  Technologies 

After  their  initial  emancipating  impact,  new  technologies  eventually  come  to  constrain  our  thinking  and  actions, 
especially  after  heavy  investment  in  their  use.  Advances  in  information  technologies  over  the  last  twenty  years  are 
so  profound  that  they  are  affecting  every  area  of  our  professional  lives.  I term  this  condition  the  “digital  shift” 
because,  as  we  convert  our  thinking,  knowledge,  and  communication  to  digital  and  informational  form,  a whole  new 
set  of  possibilities  opens  up  (Brown,  2000;  Brown  & Duguid,  1996). 

Digitized  information  is  traceable  and  archivable.  Exchanges  and  interactions  are  more  easily  captured,  at  least 
on  a digital  level.  Digital  databases  are  searchable  to  a degree  that  we  can  often  retrieve  needed  resources  when 
solving  problems  in  real  time.  Because  of  their  non-rival  nature,  digital  resources  are  replicable  in  that  they  can  be 
copied  and  distributed  an  infinite  number  of  times  at  zero  or  extremely  low  cost.  Using  webs  to  link  via  hypertext 
interconnecting  information  has  become  commonplace  in  the  problem-solving  practices  of  information  workers. 
Accompanying  digital  information  is  a stronger  suite  of  tools:  online  communication  tools  will  increasingly  allow 
higher  resolution,  more  modalities,  more  choice,  and  more  fidelity  to  everyday  encounters.  These,  along  with  more 
powerful  representation  tools,  are  leading  to  virtual  worlds  that  allow  for  increasingly  rich  experience  and 
interaction. 

Global  Marketplace 

In  recent  years,  economies  worldwide  have  moved  toward  greater  linkages  and  interdependencies.  This  move  is 
called  the  global  economy  or  the  global  marketplace  (see  also  Collis  & Gommer,  2001a  and  Collis  & Gommer, 
2001b  for  a helpful  analysis  on  this  general  issue). 

Economies  of  scale.  By  virtue  of  the  Web  and  the  shift  to  online  learning,  markets  for  learning  resources  have 
shifted  from  local  to  global.  Thus  a school  in  Australia  may  offer  a course  that  attracts  students  from  all  over  the 
world.  A portal  or  website  may  compete  against  an  office  of  student  services  within  a community  college.  These 
shifts  in  markets  and  audiences  create  new  economies  of  scale — allowing  larger  investment  and  larger  outreach — 
but  they  can  also  threaten  locally  developed  providers. 

Globally  distributed  labor  pool.  A company  based  out  of  India  may  hire  an  experienced  PhD  at  $3/hour  to 
facilitate  a graduate-level  computer  science  course.  This,  in  turn,  may  force  a competitive  response  by  a local  school 
or  learning-resource  provider.  By  simple  virtue  of  the  Web,  salary  scales  and  hiring  practices  for  online  resource 
providers  are  starting  to  become  more  globalized 

Disaggregation  of  products  and  services.  It  can  be  hard  to  put  a price  tag  on  residential  school  experiences. 
What  is  a Harvard  MBA  worth,  and  where  does  the  value  lie?  Many  graduates  would  place  great  value  on  non- 
traditional  outcomes,  e.g.,  the  network  of  friends  and  contacts;  the  exposure  to  a company’s  work  practices  via  an 


internship;  the  rite-of-passage  and  developmental  roles  of  schooling;  the  opportunity  to  take  personal  risks  and  test 
oneself.  Online  learning  providers  will  need  to  somehow  differentiate  the  valued  outcomes  of  a schooling 
experience. 

Commoditization  of  instruction . As  suggested  earlier,  instruction  can  be  seen  either  as  a mass-produced  product 
or  as  a unique  experience.  Because  online  learning  resources  require  rrore  up-front  development  than  typical 
classroom  experiences,  and  because  online  instruction  is  still  seen  as  an  entrepreneurial  enterprise,  there  is  a 
tendency  to  see  online  instruction  as  a commodity.  Once  investment  has  been  made  in  the  product,  providers  often 
want  to  distribute  that  product  as  far  as  the  market  will  allow.  This  could  also  be  called  a shift  from  a craft  to  an 
industrial  model  of  production  and  delivery.  A view  of  instruction  as  commodity,  of  course,  is  compatible  with 
viewing  education  in  input-output  terms. 

Mixing  of  commerce  and  education.  Many  enthusiasts  are  disheartened  at  the  commercialization  of  the  Web, 
but  it  was  a predictable  effect  coincident  with  increasing  choice  and  individual  control.  A similar  tendency  is  seen  in 
schools  as  greater  choices  and  perspectives  are  accommodated  through  charter  and  private  schools.  Commercial 
investment  can  provide  the  needed  stimulus  to  innovation  and  development,  but  it  can  also  reduce  innovation  and 
variation,  especially  small -niche  perspectives  at  the  fringes.  Commercial  appropriation  of  learning  can  result  in 
some  confusion  through  blurring  of  boundaries  between  consumption  and  education,  between  entertainment  and 
learning.  In  an  open  market,  where  satisfaction  of  desire  plays  a critical  role,  learning  outcomes  may  suffer  from 
neglect. 

Radical  Forces  Inspired  by  Global  Connectivity 

Web  as  democratizing,  emancipating,  empowering  force.  Early  literature  about  the  Internet  was  infused  with 
optimism  and  idealism  about  universal  sharing  and  access.  The  Web  indeed  can  be  an  empowering  force  that  gives 
information  access  to  users  who  are  physically  remote  from  resources  (Ryder,  1995).  At  the  same  time,  the  Web, 
like  so  many  other  tools,  reflects  our  own  values  and  ideas.  A divide  still  exists  between  the  privileged  and  the 
disenfranchised,  but  the  rules  have  changed  somewhat.  Principal  barriers  now  include  lack  of  access  and  lack  of 
cultural  or  personal  fit  with  the  technology.  Age  can  even  be  a barrier  to  empowerment,  with  younger  people 
tending  to  have  more  time  and  familiarity  with  technology  than  older  generations. 

Open  source.  The  commercial  model  of  technology  advancement,  exemplified  in  the  software  industry  by 
Microsoft,  is  being  challenged  right  now  by  the  open  source  movement.  Linux,  an  operating  system  whose  source 
code  is  open  for  the  world  to  see  and  costs  nothing  to  download  and  use,  has  become  a major  movement  in  the 
software  development  world.  Open-source  advocates  are  trying  to  create  a world  where  software  is  freely  available 
and  a living  is  made  through  continuing  relationships  of  service  and  support.  Open-source  ideas  may  be  applied  to 
online  learning  and  education:  Challenge  commercial  ownership  by  making  resources  freely  available,  for  example, 
on  the  Web.  If  communities  of  practice  can  be  organized  around  openly  available  tools  and  resources,  then  the 
system  can  become  self-sustaining  and  reinforcing  to  participants  (Schrage,  2000). 

Selfpublishing  and  knowledge  sharing.  Self-publishing  is  to  knowledge  management  as  open  source  is  to 
Microsoft — an  alternative  to  a hierarchically  controlled  system.  Instead  of  fixed  search  categories  and  a company- 
designed  form,  end  users  themselves  can  publish  solutions  and  locally  valued  resources.  The  Web  epitomizes  this 
growing  trend,  to  the  occasional  chagrin  of  copyright  owners  and  librarians  (Ryder  & Wilson,  1997). 

Peer-to-peer  networking.  The  Napster  phenomenon  taught  us  that  downloading  from  central  servers  is  not  the 
only  way  to  perpetuate  an  online  enterprise.  Peer-to-peer  networking  refers  to  individual  users  sharing  resources  by 
opening  up  their  hard  drives  to  each  other.  The  core  concept  is  even  more  radical  than  Napster’s,  because  once  out 
of  the  bag  and  in  the  hands  of  end  users,  true  peer-to-peer  usage  cannot  be  controlled.  In  this  way  peer-to-peer 
networking  constitutes  a classic  form  of  self-organizing  system,  using  the  technology  to  bypass  every  form  of 
central  control. 

Self-organized  learning-  and  performance-support  groups.  Peer-to-peer  connectivity  is  the  extreme  end  of  self- 
organizing on  the  Web,  but  there  are  other  forms.  Interest  groups,  listservs,  support  groups  of  all  kinds — Each  of 
these  is  a self-organizing  system  that  draws  on  distributed  energy  and  participation  for  its  survival.  Slashdot.org  is  a 
primary  example  (Wiley  & Edwards,  2002). 
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Threats  to  credentialing,  degree-granting  institutions.  In  the  last  decade,  competency-based  approaches  are 
increasingly  offered  as  an  alternative  to  seat-time  approaches  for  credentialing  institutions,  thanks  largely  to  growth 
in  online  and  self-directed  learning.  For-profit  outreach  institutions  like  the  University  of  Phoenix,  once  ridiculed  for 
giving  credit  for  “life  experience,”  continue  to  gain  market  share  against  residential  institutions.  In  spite  of  reliability 
problems,  professional  portfolios  are  increasingly  used  for  competency  demonstration  and  evaluation.  Online 
learning,  where  seat  time  loses  much  of  its  meaning,  continues  to  improve  its  services  and  learning  outcomes,  along 
with  market  share.  These  “disruptive  technologies”  and  accompanying  competency-based  tools  are  truly  disrupting 
the  status  quo. 

Global  education  as  an  alternative  to  a national  curriculum.  Global  education  refers  to  a new  philosophy  of 
learning  that  seeks  to  create  responsible  citizens  of  the  world.  Transcending  national  interests,  the  global  education 
curriculum  takes  broadly  based  positions  on  issues  of  non-violence  and  conflict  resolution;  sustainable  growth 
policies;  treatment  of  rich  and  poor;  and  protection  of  the  global  environment  (McEneaney,  Kolker,  & Ustinova, 
1998). 


Reflections 

How  do  these  various  trends  add  up?  I hope  the  reader  engages  in  some  reflection  and  conversation  about  that 
question.  What  follows  is  my  best  effort  at  generalizing  upon  the  trends.  Rather  than  paint  a specific  scenario,  I 
highlight  a few  principles  suggesting  to  how  the  trends  may  combine. 


. Each  of  the  trends  listed  above  will  continue  to  play  a role  in  future  developm.ents 


of  education  and  training,  where  distance  technologies  will  play  an  expanding  role.  The  trends  may  compete  with 
one  another,  or  sometimes  cancel  each  other  out.  But  they  all  represent  significant  aspects  of  the  problem  space 
within  which  distance  education  of  the  future  will  take  shape. 


Open  systems  trump  closed  ones.  This  is  my  way  of  saying,  learners  and  communities  will  find  a way  to 
appropriate  emerging  tools  and  technologies,  rather  than  the  reverse.  I have  a bias  that  says  that  open  systems  (self- 
directed  learners,  self-organizing  groups  of  learners  and  workers)  constitute  the  most  vital  and  thriving  unit  for 
understanding  human  actions  and  choices  (cf.  Hill,  1999).  Process  efficiencies  and  mass-produced  tutorials  can  be 
appropriated  and  put  into  service  by  these  learners  and  groups,  and  that  is  good.  Where  a group  can  appropriate  a 
tool  or  technology  and  use  it  to  learn  from,  let  it  do  so.  Where  the  technology  breaks  down,  the  group  will  adapt  and 
make  do.  This  is  not  a Utopian  faith  in  the  goodness  of  people;  rather,  it  is  an  acknowledgement  of  the  power  and 
priority  of  groups  that  identify  us  and  guide  our  behavior.  Schools  as  collectivized  learning  institutions  will  not  go 
away.  Teachers  or  guides,  responsible  for  the  growth  of  novices,  will  not  go  away.  Collective  learning  in  real  time 
will  not  go  away.  These  practices  are  in  place,  not  because  we  lack  alternatives,  but  because  we  are  social  beings 
who  invest  considerable  time  and  resources  toward  local  interactions  and  support.  I am  confident  that  the  same 
groups — schools,  classrooms,  families,  workgroups,  professional  organizations — will  find  ways  for  distance- 
education  resources  to  work  in  their  service. 


Technologies  are  still  reflections  of  us.  Through  technologies  and  new  ideas,  we  are  always  in  the  process  of  re- 
inventing ourselves.  Technologies  serve  as  mirrors  of  our  values  and  aspirations,  as  well  as  our  weaknesses  and 
intractable  problems.  This  truth  about  technologies  underscores  the  importance  of  subjecting  our  plans  to  continuing 
scrutiny.  Whenever  possible,  we  want  our  technologies  to  reflect  our  best  selves  and  our  highest  ambitions. 

Tedjndo^  ard  idais  wR  ojntdmie  to  ayeudw  toother.  Historians  of  technology  tell  us  that  a technology  often 
based  on  the  best  thinking  available,  in  turn  stimulates  new  thinking  and  new  possibilities.  This  is  certainly 
true  of  the  Web  and  networked  information  systems.  A huge  spike  of  promising  ideas,  models,  and  R &D 
efforts  has  accompanied  the  new  technology,  ^^en  these  new  efforts  are  seen  as  artifacts  themselves,  we  see 
how  one  technology  pronq)ts  the  development  of  another,  and  how  the  cycle  repeats  itself  through  new 
iterations  of  technology,  design,  theorizing,  and  practice.  Thus  we  can  be  sure  that,  as  technology  continues 
its  onward  march,  new  models  and  ideas  will  surely  follow—  and  in  some  cases,  precede  the  technology  itself. 
As  John  Dewey  said  more  than  seventy  years  ago: 

Many  are  the  conditions  which  must  be  fulfilled  if  the  Great  Society  is  to  become  a Great 
Community...  The  highest  and  most  difficult  kind  of  inquiry  and  a subtle,  delicate,  vivid  and 
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responsive  art  of  communication  must  take  possession  of  the  physical  machinery  of  transmission 
and  circulation  and  breath  life  into  it.  When  the  machine  ^e  has  thus  perfected  its  machinery,  it 
will  be  a means  of  life  and  not  its  despotic  master.  (Dewey,  1954/27) 
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Economy  of  Scale 

The  University  of  Wisconsin  System  (UWS)  consists  of  27  individual  campuses 
(two  research  institutions,  eleven  4-year  universities,  thirteen  2-year  colleges,  and  an 
extension)  plus  Learning  Iimovations  (LI)  (a  UWS  organization  that  focuses  on  total 
online  programs).  Several  years  ago  UWS  found  itself  inundated  with  requests  from 
these  individual  campuses  and  LI  for  financial  assistance  for  the  following. 

• Web-enhanced  and  web-based  learning 

• Enhancement  of  their  respective  libraries 

• Expanded  curricular  redesign  efforts 

Since  it  was  not  possible  for  UWS  to  support  such  endeavors  on  each  campus  and  since 
such  fimding  would  basically  re-invent  the  wheel  28  different  times,  UWS  decided  to 
implement  a Distributed  Learning  System  Environment.  The  philosophical  assumption 
behind  this  type  of  environment  is  that  the  27  campuses  of  UWS  plus  LI  could  receive  a 
much  greater  return  on  investment  if  they  collectively  take  responsibility  for 
infrastructure  investments  (with  infrastructure  being  defined  as  a base  that  is  universally 
accessible,  potentially  used  by  all  and  has  value  in  its  commonality). 

The  University  of  Wisconsin  System  has  attempted  to  establish  a well-defined  set  of 
support  services  for  internet-based  teaching  and  learning  for  its  27  campuses  and  LI.  The 
objectives  of  these  services  are:  1)  to  provide  all  University  of  Wisconsin  (UW) 
campuses  with  the  opportunity  to  use  distributed  learning  technologies  to  enhance 
teaching  and  learning,  2)  to  guarantee  a learning  and  teaching  system  infrastructure 
(including  software,  hardware,  and  staffing)  that  enables  on-campus  and  off-campus 
network-based  courseware,  and  3)  to  provide  a longer  term  funding  model  that  will 
ensure  core  distributed  learning  services  for  all  faculty  and  teaching  staff 

UWS’s  Distributed  Learning  System  Environment  includes  such  services  as  the 
following. 

• Instructional  Design,  Course  Development  and  Faculty  Support 

• Copyright  Clearance  Center 

• Video  Production  and  Digital  Television 

• Asynchronous  Library  Services 

• Network  Services 

• Student  Support  Services 

• Statewide  Distance  Education  Communication,  Coordination,  Support,  and 
Web  Maintenance 

• Web-Based  Learning  System  (WBLS)  Hosting,  Help  Desk,  Training,  and 
Support 
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It  is  the  last  service  on  which  this  presentation  will  focus.  UWS  sent  out  RPPs  to  all 
27  campuses  and  LI  to  provide  this  service.  The  successful  campuses  were  the 
University  of  Wisconsin  - Eau  Claire  which  focused  on  LeamingSpace,  the  University  of 
Wisconsin  - Madison  which  focuses  on  WebCT,  and  the  University  of  Wisconsin  - 
Milwaukee  (dot.edu)  which  focused  on  Web  Course  in  a Box  and  Blackboard. 

Therefore,  instead  of  providing  this  service  at  each  UW  campus  and  LI,  the  Web-Based 
Learning  Systems  provide  this  support  from  these  three  central  locations  for  all  UW 
campuses  in  a much  more  cost-effective  fashion. 

Web-Based  Learning  Systems  (WBLS) 


Services 

For  whichever  of  the  products  chosen,  the  following  services  are  provided. 

1.  Hardware:  server  access,  disk  space  as  necessary 

2.  Software:  for  course  development  (license,  vendor  support,  upgrades,  renewals, 
separate  instance);  for  course  delivery  to  students 

3.  Staffing:  professional  development;  instructional  workshops  in  the  use  of  the 
software;  accessible  and  sustained  support  to  institutions,  educators,  and  students; 
instructional  design  consultation;  24x7  help  desk  (toll  free  number)  for  faculty 
and  students;  Network  and  System  Administrators 

Thus,  WBLSs  provided  faculty  and  teaching  academic  staff  throughout  UWS  a 
choice  of  four  web-based  learning  tools  for  course  presentation. 

Faculty  Benefits 

The  faculty  using  these  tools  report  many  benefits:  versatile  courseware,  time  for 
active  learning,  online  courses  that  can  mirror  classwork,  problem  solving  activities,  and 
collaboration.  These  tools  vary  in  ease  of  use  and  make  it  possible  for  faculty  to  create 
meaningful  courses  that  range  from  the  very  basic  to  the  highly  interactive,  from  web- 
enhanced  to  distance  education. 

Student  Benefits 

Student  benefits  are  numerous.  Web-based  and  web-enhanced  learning  provides 
a consistent  look  and  feel  to  course  information  and  resources;  provides  anytime, 
anyplace,  any  pace  instruction  (most  beneficial  to  non-traditional  students);  allows 
students  to  continue  the  conversation  beyond  the  traditional  classroom's  constraints  of 
time  and  space  through  collaborative  tools  such  as  discussion  forums  and  virtual  chats. 
Students  now  have  the  ability  to  begin  the  process  of  becoming  lifelong  learners. 

dot.edu 

dot.edu  is  an  Application  Service  Provider  for  the  State  of  Wisconsin  and  beyond. 
The  impact  of  dot.edu  has  been  tremendous:  we  have  6880  courses  in  some  phase  of 
development;  4500  faculty  have  been  trained;  and  131,000  students  are  enrolled  in  these 
courses.  We  are  in  both  UW-System  research  institutions,  10  of  the  eleven  4-year  UW- 
System  universities,  all  thirteen  2-year  UW-System  colleges,  UW-Extension,  and 
Learning  Innovations.  We  have  29  non  UW-System  partners  who  pay  to  use  this  service. 
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This  involves  30  Dell  servers;  4 larger  Sun  servers;  a very  large  environment  for 
the  most  recent  web-authoring  tool  that  we’ve  added  (Prometheus);  a video  server;  a 
tremendous  amount  of  travel,  coordination,  collaboration,  strategic  partnerships  and,  at 
present,  a cost  of  approximately  $1,000,000. 

dot.edu  Partnership  Model 


Process 

The  partners  of  dot.edu  include  all  faculty,  teaching  academic  staff,  and 
administrators  in  the  following  categories: 

• The  UW  System  campuses 

• Wisconsin  public  and  private  higher  education  institutions 

• Wisconsin  public  and  private  schools  and  districts  as  well  as  education  agencies 
in  the  state 

• Non-educational  institutions  and  those  institutions  outside  the  State  of  Wisconsin 
Programs  can  be  selected  and  uniquely  organized  to  meet  the  needs  of  the  particular 
partner.  The  Utility  model  is  structured  as  a 3-stage  implementation  plan  that  eventually 
transfers  responsibility  for  course  development  instruction  from  dot.edu  to  the  partners. 

Stage  1 : dot.edu  provides  direct  instruction  and  support  to  the  on-site  trainers 
Stage  2:  dot.edu  provides  direct  instruction  and  support  to  first  set  of  instructors, 
with  support  from  on-site  trainers 

Stage  3:  On-site  trainers  provide  all  direct  instruction  and  support  to  subsequent 
sets  of  instructors,  with  support  from  dot.edu. 

The  course  development  instructional  program  design  and  implementation 
timeline  are  determined  by  the  needs  of  the  institution.  A written  implementation  plan  is 
designed  jointly  by  dot.edu  and  the  respective  institution.  It  is  developed  with  the 
flexibility  necessary  to  meet  changing  needs.  The  campus  eventually  sustains  the  course 
development  instruction. 

Pricing 

Pricing  is  constantly  being  revisited  but,  in  general,  contains  an  annual  fee  of 
approximately  $30,000  and  a per  course  section  fee  of  approximately  $150.  Firm  quotes 
are  given  upon  request  and  after  review  of  services  needed.  The  fee  includes  the  hosting, 
24  hours  of  training  and/or  instructional  design  consultation,  the  24x7  Solution  Center, 
unlimited  users,  unlimited  courses,  and  unlimited  disk  space.  Additional  training  can  be 
obtained  for  $200/hour  plus  expenses. 

dot.edu  does  provide  an  Incubator  for  Prometheus  (for  those  customers  who  might 
need  time  to  determine  if  online  education  is  for  them,  for  customers  to  try  out  the 
dot.edu  services,  and  for  customers  to  try  out  the  Prometheus  software).  A per  course  fee 
is  charged  for  this  service  (prices  given  upon  request;  typical  charge  would  be 
$500/course  section). 

UWS  currently  subsidizes  the  UW- System  27  campuses  and  LI  so  that  each 
campus  does  not  have  to  budget  for  these  services  - therefore,  these  services  are  provided, 
sponsored,  and  funded  centrally.  Non-UWS  customers  pay  the  fees  mentioned  above. 
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New  Developments 


It  was  realized  that  the  WBLS,  WiscNet,  Wisconsin  Public  Television,  the  Pyle 
Center,  Learning  Innovations  and  others  were  providing  their  services  to  all  educational 
institutions  in  Wisconsin.  In  order  to  make  it  easier  for  these  educational  institutions 
(with  regard  to  costs,  contact  information,  services  provided,  and  so  forth),  it  was 
determined  that  a type  of  clearing  house  be  established  to  act  as  the  conduit  for  the 
various  state  Service  Providers.  Thus,  ITS@Wisconsin  was  formed  (Instructional 
Technology  Services  @Wisconsin),  providing  a single  point  of  contact  for  educational 
institutions  in  the  State  of  Wisconsin.  In  doing  so,  the  old  name  of  Web-Based  Learning 
Systems  (WBLS)  was  abandoned. 

As  time  progressed,  UW-Eau  Claire  ceased  offering  LeamingSpace  and  Web 
Course  in  a Box  was  purchased  by  Blackboard.  In  addition,  a new  web-authoring  tool 
(Prometheus)  was  added  to  the  mixture  in  December  of  2000.  Thus,  UW-Madison  offers 
WebCT  and  dot.edu  offers  Blackboard  and  Prometheus. 

dot.edu  has  formed  strategic  partnerships  with  Dell,  Sun  Microsystems,  WiscNet, 
Blackboard  and  Prometheus.  In  addition,  dot.edu  has  recently  received  the  Sun  Center  of 
Excellence  award.  Together  with  Sun  Microsystems,  Dell  and  Prometheus,  dot.edu 
intends  to  take  the  dot.edu  concept  nationally  to  show  other  states  that  education  can  take 
care  of  education  and  provide  a seamless  PK-16  education. 

Goals  and  Mission 

The  goals  of  dot.edu  are  not  stagnant  but  ever  changing. 

• We  will  be  partnering  with  the  Student  Accessibility  Center  at  UW-Milwaukee  to 
accompany  us  on  all  our  training  sessions  to  educate  all  faculty  and  students  with 
regard  to  accessibility  issues 

• We  will  be  instrumental  in  the  formation  of  standards  through  the  Wisconsin  Co- 
Lab 

• Disseminate  knowledge 

• Leverage  resources  to  provide  an  economy  of  scale 

• Support  web-enhanced,  hybrid,  and  online  education 

• Provide  appropriate  high-quality  web-based  learning  systems 

• Remove  barriers  preventing  interaction  between  the  UWS,  PK-12,  and  other 
appropriate  members  of  Wisconsin’s  educational  community 

The  mission  of  dot.edu  is  also  constantly  expanding  but  has  always  remained  the 
same  - provide  educational  communities  with  robust  and  up-to-date  e-leaming  system 
infrastructure  including  technology,  training,  support,  and  instructional  design  and  to 
effectively  apply  these  resources  to  enhance  education,  dot.edu  hopes  to  be  able  to 
provide  a seamless  e-leaming  experience  to  PK-16  and  beyond. 
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What  We  Have  Learned  and  Reasons  We've  Succeeded 


Lessons  Learned 

Much  has  been  learned  in  the  process  of  developing  dot.edu.  Since  nothing  like 
this  existed,  we  are  constantly  "making  it  up"  as  we  go  along.  Continual  refinement  of 
policies  and  procedures,  pricing  stmctures,  service  agreements,  and  services  offered  takes 
place.  Some  of  the  specific  lessons  we've  learned  follow. 

1 . Identity  issues  - each  of  these  Service  Providers  exist  on  a specific  central  UW 
campus  but  each  is  funded  and  sponsored  by  the  University  of  Wisconsin  System 

2.  Customer  support  - this  is  a very  intense  and  expensive  aspect  of  the  dot.edu, 
much  more  than  anticipated  at  the  beginning 

3.  Dealing  with  unknowns  on  each  campus  - this  comes  into  play  when  the  training 
takes  place  on  our  partners'  respective  campuses 

4.  New  products  - since  this  is  such  a changing,  volatile  field  at  present,  new 
products  are  continually  being  analyzed  and  tested 

5.  Legalities  - copyright  and  intellectual  property  issues 

6.  Instmctional  design  - it  is  quite  a challenge  to  provide  instructional  design  and 
pedagogical  consultation  through  a distributed  system 

7.  Measuring  capacity  and  demand  - regarding  disk  space,  number  and  types  of 
servers,  personnel  needed 

8.  Students  - staffing  has  to  be  supplemented  by  student  workers 

9.  External  funding  - it  is  imperative  that  partnerships  are  developed  and  external 
funding  is  sought 

1 0.  Administrative  systems  - it  is  becoming  more  and  more  apparent  that  it  is 
necessary  that  these  web-authoring  tools  integrate  with  individual  campus 
Administrative  back-office  systems 

Why  We\e  Been  A Success 

dot.edu  has  been  a tremendous  success  because  of  a few  very  simple  reasons. 

• Customer  service  - fast,  friendly,  and  thorough 

• A great  product  which  is  continually  being  improved  and  enhanced 

• A buy-in  by  central  administration,  campus  administration,  and  faculty 

• Having  as  an  end  product  of  all  of  this  - true  collaboration  and  teamwork  throughout 
the  State  of  Wisconsin  and  beyond 

• A willingness  to  go  that  extra  mile 

® An  adherence  to  the  mission  of  the  dot.edu  and  ITS@Wisconsin 

• A willingness  to  take  the  risk  and  go  where  no  one  has  gone  before 

• dot.edu  is  Education  Serving  Education 

Conclusion 

Several  years  ago  the  UW-System  had  the  foresight  to  see  that  a problem  existed 
and  they  designed  a solution.  They  then  had  the  tenacity  to  actually  try  their  solution.  Its 
success  has  exceeded  everyone's  greatest  expectations.  Through  dot.edu,  the  rest  of 
ITS@Wisconsin  and  the  entire  Distributed  Learning  System  Environment,  the  UWS 
faculty  and  campuses  can  now  take  education  in  the  State  of  Wisconsin  to  that  next  level. 


Agent  Technologies  in  the  Electronic  Classroom:  Some  Pedagogical 

Issues 


Abstract:  The  use  of  intelligent  software  agents  within  computer  mediated  learning  environments  has  become 
an  important  focus  of  research  and  development  in  both  AI  and  educational  contexts.  Some  of  the  roles 
envisaged  and  implemented  for  these  electronic  entities  involve  direct  interactions  with  students,  participating 
in  the  ‘social’  dimension  of  the  classroom  that  is  of  such  importance  in  contemporary  pedagogical  theory. 
Others  contribute  to  the  many  background  tasks  that  support  the  teaching/leaming  process.  Each  type  of 
activity  raises  its  own  special  challenges  in  relation  to  the  capabilities  of  the  software  and  to  our 
understandings  of  teaching  and  learning.  Through  discussion  of  both  theoretical  perspectives  and  practical 
examples,  this  paper  explores  a selection  of  these  issues. 


One  of  the  fastest  growing  applications  of  AI  research  is  the  implementation  of  computer  programs 
commonly  referred  to  as  ‘agents’.  Among  the  features  distinguishing  this  type  of  software  from  more  traditional 
programs  are  a high  degree  of  autonomy  in  decision  making  and  action,  the  ability  to  ‘learn’  from  experience  and 
to  adapt  their  behaviour  accordingly,  and  often  a highly  personified  interface.  Many  are  specifically  designed  to 
process  complex  information,  make  decisions  and  initiate  actions  in  ‘mission  critical’  areas  of  human  endeavour 
including  health,  scientific  research,  business,  defence  and  increasingly  in  education.  While  in  some  cases  we 
are  aware  of  our  interactions  with  these  electronic  entities,  in  many  contexts  their  activity  takes  place  ‘behind  the 
scenes’,  at  a level  not  apparent  to  the  user.  Agent  roles  currently  implemented  in  educational  projects  around 
the  world  include  record  keepers,  information  seekers,  testers,  facilitators  of  collaboration,  tutors  or  instructors, 
fellow  learners,  and  tutees. 

While  the  usefulness  of  this  type  of  software  in  many  contexts  is  already  well  proven,  a number  of 
issues  in  relation  to  its  uses  in  education  are  not  yet  fully  resolved.  The  substitution  of  computer  programs 
possessed  of  varying  degrees  of  artificial  intelligence  and  ‘personality’  for  various  aspects  of  human  presence  in 
the  computer  based  classroom,  raises  some  quite  fundamental  questions  concerning  the  processes  through 
which  knowledge  is  socially  constructed,  and  the  qualities  which  are  necessary  to  ensure  successful 
participation  in  those  processes.  To  what  extent  can  socially  interactive  roles  such  as  tutor  and  fellow  learner  be 
effectively  and  appropriately  fulfilled  by  a computer  program,  however  ‘intelligent’?  Where  agents  undertake 
less  interactive  tasks  such  as  record  keeping  and  testing,  other  issues  arise  including  that  of  privacy  and  of 
students’  control  over  their  own  learning. 


Agents  in  the  electronic  classroom 

Typical  of  the  broad  vision  held  by  many  researchers  and  educators  for  the  implementation  of  agent 
technology  in  the  online  classroom  is  that  promulgated  by  Johnson,  who  writes: 

“Pedagogical  agents  are  autonomous  agents  that  support  human  learning,  by  interacting  with 
students  in  the  context  of  interactive  learning  environments.  They  extend  and  improve  upon 
previous  work  on  intelligent  tutoring  systems  in  a number  of  ways.  They  adapt  their  behaviour  to 
the  dynamic  state  of  the  learning  environment,  taking  advantage  of  learning  opportunities  as 
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they  arise.  They  can  support  collaborative  learning  as  well  as  individualized  learning,  because 
multiple  students  and  agents  can  interact  in  a shared  environment.  Given  a suitably  rich  user 
interface,  pedagogical  agents  are  capable  of  a wide  spectrum  of  instructionally  effective 
interactions  with  students,  including  multimodal  dialog.  Animated  pedagogical  agents  can 
promote  student  motivation  and  engagement,  and  engender  affective  as  well  as  cognitive 
responses”  (Johnson  1998,  p.  13). 

Arguably  the  most  obvious  choice  of  interactive  role  for  an  agent  would  be  that  of  teacher.  While  one 
might  reasonably  fear  a resurgence  of  the  heavily  instructionist  model  of  ‘the  computer  as  tutor*  prevalent  in  the 
early  days  of  educational  computing,  there  are  indications  that  developers  are  attempting  to  base  their  agent- 
based  manifestations  on  aspects  of  pedagogical  theory  more  acceptable  to  current  thinking. 

As  Solomos  and  Avouris  (1999)  write,  for  instance: 

“The  user  mental  model  of  the  system  should  be  based  on  the  metaphor  of  the  ‘invited 
professor’  rather  than  the  ‘knowing  everything  own  tutor’.  . . . Our  first  findings  confirm  the 
observation  that  today’s  users,  accustomed  to  hypertext -like  interaction,  are  more  likely  to  accept 
this  collaborative  teaching  metaphor,  according  to  which  their  tutoring  system  is  viewed  as  an 
intelligent  hypertext  browser,  offering  links  to  other  tutoring  systems  with  the  right  content  and 
at  the  right  time”  (Solomos  & Avouris  1999,  p.  259). 

The  image  of  the  teacher  as  a facilitator  of  learning  rather  than  as  the  ‘sage  on  the  stage’  is  also 
reflected  in  such  statements  as:  “Each  student  working  on  the  project  will  have  an  agent,  operating  in  the 
background,  watching  progress,  measuring  it  against  the  plan,  and  taking  remedial  action  when  necessary” 
(Whatley  et  al  1999,  p.  362). 

The  substitution  of  agents  for  one  or  more  of  a student’s  classmates  or  fellow  learners  is  an  interesting 
concept  being  explored  by  a number  of  developers.  Successful  implication  of  this  idea  would  add  considerable 
appeal  to  online  learning  environments.  Currently,  while  electronic  interactions  between  students  can  be 
encouraged  and  facilitated  up  to  a point,  the  quality  and  timing  are  far  less  subject  to  the  control  of  the  ‘teacher’ 
than  is  the  case  in  a face  to  face  situation.  The  presence  of  ‘classmate  agents’  would  not  only  enhance  the 
general  social  ambience  of  the  online  classroom,  but  could  enable  more  constructive  interactivity  to  take  place  at 
pedagogically  appropriate  times. 

Since  the  1980s  Chan  (Chan  1996,  1998)  and  colleagues  have  been  working  on  a range  of  models  of 
socially  interactive  agents  for  learning  environments,  perhaps  the  best  known  being  the  ‘learning  companion*  - 
a software  entity  having  limited  knowledge  of  the  domain  in  question,  conceptualised  as  a fellow  learner  with 
whom  the  student  may  collaborate  and  even  disagree.  As  in  real  life,  some  of  these  learning  companions  may  be 
better  informed  than  the  student  in  the  relevant  domain  of  knowledge,  while  others  may  know  less.  Perhaps  not 
surprisingly,  in  learning  environments  for  younger  students,  animals  are  a popular  choice  of  persona  for  such 
agents,  as  in  this  example  of  a networked  learning  environment  for  Taiwanese  high  school  students,  as 
described  by  Chan: 

“The  Dalmation  is  having  the  same  performance  as  the  student.  ...  Another  animal  companion  is 
Dragon,  like  one  of  those  animal  companions  in  Mulan,  a Disney  cartoon  of  this  summer.  This 
dragon  will  “learn”  (mainly  rote  learning)  from  the  student  and  also  from  other  students  on  the  Net 
and  so  may  know  more  than  the  student.  At  certain  point  it’ll  stop  learning  and  come  back  to 
teach  the  student.  In  a way.  Dragon  is  protecting  the  student.” 

(Chan  1998) 

An  interesting  development  of  this  concept  is  suggested  by  Sheremetov  and  Nunez  (1999,  p.  310),  who 
describe  the  function  of  a ‘monitor  agent’  as  being  to  modify  the  role,  behaviour  or  expertise  of  learning 
companions.  A learning  companion’s  personality  could,  for  examples,  be  changed  from  that  of  strong  group 
leader  to  a weaker  companion  or  even  a passive  observer,  depending  on  the  monitor  agent’s  interpretation  of  the 
degree  of  guidance  required  by  the  learner. 

A third  socially  interactive  role  that  may  be  enacted  by  an  agent  is  that  of  ‘tutee*.  We  are  all  familiar 
with  the  common  wisdom  that  we  learn  best  through  teaching  others.  This  principle  was  invoked  in  the  early 
days  of  educational  computing  through  Logo,  where  ‘teaching  the  turtle’  was  the  chosen  metaphor  for  the 
activity  of  programming.  More  recently,  a number  of  researchers  have  explored  the  translation  of  this  concept 
into  electronic  learning  contexts  where  agents  exist  to  be  ‘taught’  by  the  student  user,  as  in  the  example  from 
Chan  quoted  above.  A further  example  is  described  by  Ju  (1998)  who  writes  of  a computer  based  peer  tutoring 
system  employing  two  categories  of  agent  - an  ‘expert’,  and  a ‘learner’: 
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students  become  active  learners  who  are  guided  to  learn  by  teaching  a computer.  After  the 
students  watch  how  the  computer  expert  solves  a set  of  linear  equations  [the  program]  helps  the 
human  student  act  as  a teacher  in  order  to  learn  more  about  the  subject  matter.  At  this  time,  the 
computer  plays  the  role  of  a student ...” 

(Ju  1998,  p.  559) 

In  addition  to  these  highly  visible  and  interactive  ‘social’  roles,  there  is  considerable  scope  for  agents 
to  undertake  a range  of  support  activities  including  monitoring  the  activities  and  responses  of  students, 
administering  tests,  recording  results  and  seeking  information  on  behalf  of  students  or  teachers.  The  latter  task 
is,  of  course,  familiar  to  us  in  relation  to  the  operation  of  standard  Internet  search  engines. 


Issues  for  consideration 

Each  of  these  implementations  of  agent  technologies  raises  issues  that  demand  consideration.  While 
some  are  basically  pedagogical,  others  straddle  the  boundaries  between  pedagogy  and  other  areas,  such  as 
ethics.  While  some  relate  to  whether  or  not  the  achievement  of  certain  ends  is  a practical  possibility,  others  go 
further  and  question  whether  what  is  possible  is  necessarily  desirable. 

A key  element  in  the  implementation  of  socially  interactive  agents  is  a high  degree  of  personification  or 
‘character’.  It  is  well  accepted  that  an  element  of  personification  of  program  interfaces  is  inevitable.  As  Shirk  puts 
it: 

“Although  there  is  some  dispute  among  software  critics  concerning  the  advisability  of  having 
‘personalities’  in  computer  programs,  their  presence  seems  unavoidable.  Any  time  there  is  communication 
between  a computer  and  a human,  the  information  presented  by  the  computer  has  a certain  style,  diction, 
and  tone  of  voice  which  impact  upon  the  human’s  attitude  and  response  toward  the  software”  (Shirk  1988, 
p.  320). 

Deliberate  personification  is,  however,  more  problematic.  As  Masterton  notes:  “A  common  problem  with 
AI  programs  that  interact  with  humans  is  that  they  must  present  themselves  in  a way  that  reflects  their  ability. 
Where  there  is  a conflict  between  the  ability  of  the  system  and  the  users’  perception  of  that  ability  a breakdown 
occurs  and  users  may  either  fail  to  exploit  its  full  potential  or  become  frustrated  with  its  shortcomings” 
(Masterton  1998,  p.  215).  He  suggests  the  implementation  of  a style  of  pedagogical  agent  with  a degree  of 
anthropomorphism  intended  to  convey  qualities  such  as  friendliness  and  usefulness,  without  the  implication  of 
possession  of  full  human  capabilities.  He  describes  the  development  and  role  of  such  an  entity,  a VTA  (Virtual 
Teaching  Assistant),  which  is  able  to  introduce  topics  and  answer  simple  questions,  the  more  complex  types  of 
exposition  and  interaction  being  left  to  the  human  teacher.  In  terms  of  a traditional  scenario  at  university  level, 
the  VTA  functions  somewhat  like  a tutor  or  demonstrator  as  distinct  from  a lecturer.  “In  this  way  faculty  is  left 
free  of  the  guiding  and  assisting  issues  of  the  course  and  is  able  to  concentrate  on  more  complex  questions  and 
higher  level  issues  generated  during  the  course”  (Masterton  1998,  p.  211).  The  ‘learning  companions’  of  Chan 
mentioned  above  can  be  seen  as  further  instances  of  this  principle.  Our  expectations  in  regard  the  cognitive  skills 
of  animals  may  well  be  more  appropriate  to  the  capabilities  of  software  agents  than  are  our  experiences  of  human- 
to-human  interactions. 

An  agent’s  capacity  to  demonstrate  some  equivalent  to  the  emotional  responses  of  a human  being,  and 
to  appropriately  recognise  and  respond  to  the  emotions  of  users  is  becoming  recognised  as  an  important  element 
both  in  personification  generally  and  in  educational  contexts  in  particular.  As  Frasson  writes,  “Emotions  play  an 
important  role  in  the  learning  process  and  new  strategies  have  to  take  into  account  this  human  factor  for 
improving  knowledge  acquisition.  Intelligent  agents  can  hep  in  this  process,  adding  emotional  behavior  to 
believability  of  their  actions”  (Frasson  2000,  p.  60). 

Apart  from  the  possibility  that  less  fully  personified  agents  might  be  more  educationally  effective,  there 
are  clear  ethical  issues  attached  to  the  presentation  of  these  programs  in  a form  which  students  are  unable  to 
distinguish  from  that  of  a human  participant  in  the  learning  experience. 

Another  important  characteristic  of  agent  software  is  its  capacity  for  autonomous  or  self-directed 
decision  making  and  action  in  the  pursuit  of  its  goals.  This  raises  the  question  of  the  extent  to  which  a 
pedagogical  agent  should  be  furnished  with  pre-existing  goals  which  could  lead  it  to  undertake  actions  without 
instruction  from  the  learner,  and  even  contrary  to  what  the  learner  might  perceive  as  his  or  her  interests  and 
wishes.  Loeffler  (1996)  notes  that  the  unpredictability  resulting  from  significant  autonomy  might  well  result  in 


agents  who  are  less  ‘helpful’  to  us  than  we  might  hope  or  indeed  expect.  As  Minsky  puts  it,  “There’s  the  old 
paradox  of  having  a very  smart  slave.  If  you  keep  the  slave  from  learning  too  much,  you  are  limiting  its 
usefulness.  But,  if  you  help  it  to  become  smarter  than  you  are,  then  you  may  not  be  able  to  trust  it  not  to  make 
better  plans  for  itself  than  it  does  for  you”  (Minsky,  1994,  p.  25).  It  is  easy  to  slip  from  such  considerations  into 
the  need  for  a contemporary  version  of  Asimov’s  laws  of  robotics  as  conceived  in  fictional  terms  more  than  30 
years  ago. 

In  educational  contexts  there  is  potential  for  some  degree  of  conflict  between  the  usefulness  of  an  agent 
possessing  a high  degree  of  autonomy,  and  contemporary  understandings  of  the  importance  of  individuals  being 
able  to  exercise  control  over  various  aspects  of  their  own  learning.  It  could  be  argued  that  current  trends  in 
educational  thinking  which  favour  giving  more  control  and  autonomy  to  the  learner  would  appear  to  be  more  in 
line  with  the  thinking  of  researchers  such  as  Schneiderman  (1983)  who  favour  ‘direct  manipulation’  over  the 
development  of  interactive  agents  with  a significant  degree  of  independence  of  action.  There  is  certainly  a case 
for  suggesting  that  pedagogical  agents  be  configured  so  as  to  be  particularly  sensitive  to  individual  user  models, 
more  responsive  to  instruction  from  the  user/student,  and  that  their  characteristics  and  capabilities  should  be 
more  transparently  presented. 

A high  degree  of  transparency  in  relation  to  the  functioning,  indeed  to  the  existence  of  agents  within  the 
electronic  classroom  is  also  important  in  relation  to  maintaining  a level  of  trust  which  many  educators  believe  to 
be  an  important  component  of  any  learning  environment,  whether  computer  based  or  face  to  face. 

A further  concern  in  regard  to  the  autonomy  of  pedagogical  agents  relates  to  the  issue  of  intervention  in 
the  learning  process.  It  is  well  accepted  that  a high  degree  of  unsought  assistance  whether  from  a human  teacher 
or  an  excessively  diligent  and  proactive  agent  can  be  quite  detrimental,  in  particular  to  the  metacognitive  aspects 
of  learning.  The  fact  that  this  is  also  an  issue  for  teachers  and  learners  in  face  to  face  educational  contexts 
underlines  its  complexity,  and  degree  to  which  it  is  unlikely  to  handled  with  the  requisite  sensitivity  through  prior 
programming  of  an  agent. 

An  interesting  aspect  of  most  agent  based  educational  systems  is  their  use  of  a multiplicity  of  agents, 
many  of  them  capable  of  a complex  range  of  interactions  with  the  student,  with  one  another,  and  increasingly 
with  agents  associated  with  other  programs.  These  interactions  range  in  nature  from  collaboration  to  competition, 
and  their  purposes  are  derived  from  theoretical  analyses  of  the  various  component  tasks  and  activities  that  make 
up  the  human  activity  of  ‘teaching’.  An  example  is  the  Multiple  Agent  Tutoring  System  (MATS)  described  by 
Solomos  and  Avouris: 

“ MATS  is  a prototype  that  models  a “one  student-many  teachers”  learning  situation.  Bach  MATS  agent 
represents  a tutor,  capable  of  teaching  a distinct  subject.  All  MATS  tutors  are  also  capable  of 
collaborating  with  each  other  for  solving  learning  difficulties  that  their  students  may  have”  (Solomos  & 
Avouris  1 999,  p.  243). 

While  on  the  one  hand,  the  variety  of  functions  of  agents  within  a multi-agent  environment  is  an  attempt 
to  realise  the  type  of  rich  user  interface  which  Johnson  suggests  is  necessary  if  the  pedagogical  interactions 
within  electronic  learning  environments  are  to  approximate  to  any  degree  to  the  face  to  face  educational 
experience,  some  educators  have  concerns  in  regard  to  the  assumptions  underlying  these  practices.  They  argue 
that  such  developments  are  underpinned  by  a reductionist  rather  than  a holistic  understanding  of  the  processes 
and  relationships  involved  in  teaching  and  learning.  In  separating  out  the  different  components  of  pedagogical 
interactions,  are  we  enabling  each  part  to  be  realised  more  effectively,  or  are  we  failing  to  acknowledge  that  the 
global  act  of  human  teaching  may  in  fact  be  more  than  the  sum  of  its  component  parts? 

Central  to  the  work  of  many  theorists  and  researchers  is  the  belief  that  it  is  possible  for  agents  to 
participate  effectively  in  the  social  aspects  of  knowledge  construction.  Sheremetov  and  Nunez,  for  example, 
whose  works  derives  overtly  from  the  theoretical  frameworks  of  Piaget  and  Vygotsky,  argue  that: 

“The  design  of  learning  environments,  virtual  or  not,  aims  to  promote  productive  interactions.  In  this  type 
of  learning  a student  changes  from  being  a passive  information  receiver  to  an  active  collaborator, 
interacting  with  the  tutors  and  colleagues  in  the  learning  process.  Learning  does  not  only  result  from 
acquiring  knowledge,  solving  problems  or  using  tools,  but  also  from  interacting  about  these  on-going 
activities  with  persons  and  agents”(  Sheremetov  and  Nunez  1999,  p.305  - 306). 

But  however  personified  and  autonomous  the  software  agent,  can  it  really  be  said  to  participate  fully  in 
the  social  construction  of  knowledge?  It  has  been  argued  quite  extensively  that  even  the  most  heavily 
personified  of  computer  programs  suffer  from  an  intrinsic  lack  of  ability  to  participate  in  the  metacognitive 
aspects  of  learning.  Pufall  (1988),  for  instance,  expresses  a strong  belief  that  a computer  program  is  unable  at  any 
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level  commensurate  with  human  capacities  to  modify  its  own  knowledge  structures  or  cognitive  processes,  and 
so  cannot  be  regarded  as  a co-constructor  of  knowledge  in  a meaningful  sense.  While  this  might  well  have  been 
the  case  in  relation  to  earlier  computer  based  learning  environments,  can  we  continue  to  make  the  same  claims 
with  confidence  today  or  in  the  future?  The  capacity  of  software  to  ‘learn’  and  adapt  to  experience  through  the 
incorporation  of  new  information,  the  appropriate  modification  of  its  representation  of  the  context  in  which  it 
functions  (its  ‘world’)  and  of  its  inference  mechanisms,  is  undoubtedly  increasing.  If  our  test  of  full  participation 
depends  on  an  understanding  that  the  agent  has  ‘learnt’  in  precisely  the  same  way  that  the  human  has  learnt, 
then  we  will  have  difficulty  accepting  the  electronic  entity  as  genuine  co-constructor  of  knowledge.  If,  however, 
we  make  our  claim  on  the  grounds  that  it  appears  to  the  human  learner  that  the  agent  has  partic^ated  in  the 
learning  that  has  taken  place,  then  perhaps  we  can  at  least  tentatively  admit  such  a piece  of  software  to 
membership  of  the  social  milieu  that  has  mediated  the  educational  experience. 


Conclusion 

The  widespread  implementation  of  agent  technologies  within  the  electronic  classroom  is  bound  to  raise 
a large  number  of  issues  of  which  those  listed  above  are  a sample.  While  it  is  critical  that  pedagogical  concerns 
rather  than  simple  technical  capabilities  are  the  driving  force  behind  the  development  of  these  new  learning 
environments,  it  is  also  important  to  look  beyond  the  extent  to  which  agents  can  simply  replicate  existing 
classroom  conditions.  Where  agent  based  educational  systems  can  be  identified  as  differing  from  the  traditional 
classroom,  careful  research  and  critical  evaluation  is  required  if  we  are  to  distinguish  between  those  which 
simply  undermine  valued  aspects  of  the  educational  enterprise  as  we  understand  it,  and  those  which  have  the 
potential  to  create  fruitful  new  possibilities  in  regard  to  how  we  teach  and  how  we  learn. 
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Creativity  and  Efficacy  in  Implementing  a Guiding  Partner  Approach: 
Thinkquest  for  Tomorrow’s  Teachers  PT3  Project 


The  ThinkQuest  for  Tomorrow's  Technology  project  funded  by  the  USDOE  is  in  its  second  year  of  work 
as  a catalyst  grant.  ThinkQuest  and  fourteen  universities  are  partners  in  the  implementation  of  work  to 
prepare  preservice  teacher  education  candidates  to  use  technology  effectively  to  enhance  teaching  and 
learning. 

The  ThinkQuest  for  Tomorrow's  Teachers  project  informs  university  students  of  the  Guiding  Partner 
Approach  (GPA)  based  on  student-centered  constructivist  pedagogy  that  requires  student -to-student  and 
student-to-  faculty  “elbow  to  elbow"  learning.  As  students  explore  their  understanding  of  important 
academic  content  for  K-12  learning,  they  also  decide  on  how  to  use  internet-enriched  activity  design  to  help 
to  display  interesting  resources  from  their  research  on  curricular  topics.  They  also  work  with  their  peers 
and  teachers  as  collaborators,  researchers,  and  co-facilitators,  so  that  they  experience  the  benefits  of 
collaboration,  rather  than  individualism  and  isolation.  This  approach  to  learning  will  provide  a model  to 
students  of  a new  ideological  foundation  for  work  in  systems  such  as  K-12  education. 

In  the  second  year  of  implementation,  faculty  at  each  of  the  fourteen  universities  are  incorporating  digital 
and  internet  technologies  as  skills  to  be  learned  and  as  opportunities  for  teams  of  students  to  work  together 
in  creating  educational  web  sites  with  information  and  activities  of  interest  to  youth,  teachers,  and 
university  pre -service  teachers.  At  the  lead  university,  the  University  of  the  Pacific,  teams  of  students,  from 
team  sizes  of  two  to  six  students,  in  selected  courses  in  the  School  of  Education  have  prepared  web  sites 
with  content  relevant  to  topics  they  are  learning  in  their  course  work  as  well  as  content  and  pedagogy 
pertinent  to  state  academic  content  standards. 

Key  to  this  understanding  of  the  guiding  partner  approach  are  insights  provided  by  Judi  Harris  in  her  book, 
Virtual  Architecture:  Designing  and  Directing  Curriculum-Based  Telecomputing  {\99S,  ISTE)  as  she 
states,  “How  to  apply  the  tools  in  curriculum-based  educational  activities,  although  less  frequently  the 
target  of  careful  thought  and  in-depth  investigation,  is  a much  richer,  more  complex,  longer-term,  and  more 
critical  area  for  educators  to  explore  at  this  point  in  time.  In  short,  it's  not  about  how  teachers  use  the 
TOOLS — it's  about  the  USE  of  the  tools"  (pp.5-6). 

This  paper  presentation  will  focus  on  the  results  of  collaborative  work  by  university  students  on  digital  web 
sites  to  understand  their  sense  of  efficacy  in  researching  academic  content  and  in  learning  to  use  the  digital 
and  internet  technologies.  Also,  the  paper  will  focus  on  a relationship  between  imaginative  and  creative 
development  of  content  and  students'  skill  in  using  the  technology.  We  have  observed  that  as  students 
became  creative  in  their  thinking  about  the  content,  they  correspondingly  became  more  interested  in  finding 
interesting  ways  to  display  the  content  in  the  digital  environment. 

As  an  example,  students  in  an  Introduction  to  Language  and  a Children's  Literature  course  were  required  to 
develop  web  sites  as  courses  assignments.  Students  in  the  Introduction  to  Language  course  had  course 
paper  assignments  leading  to  the  web  page  assignment  that  required  them  to  investigate  information  about  a 
topic  of  interest  to  them  in  several  areas  of  language  study.  Students  were  invited  to  research  any  one  topic 
among  several  topics  such  as  research  in  second  language  acquisition,  modern  and  classical  languages, 
logographic  representations  of  language,  language  and  reading,  and  dialects  in  American  language 
communities.  Students  first  developed,  individually,  annotated  bibliographies  of  ten  sources  of  information 
from  scholarly  publications.  Then,  they  prepared  conventional  five  page  papers  on  the  topic,  using  sources 
uncovered  in  the  annotated  bibliography  assignment.  Students  were  then  asked  to  group  themselves  with 
others  in  the  course  who  wrote  on  topics  in  the  same  area  in  order  to  develop  teams  to  work  on 
development  of  a web  site  by  each  team. 

When  students  first  developed  ideas  about  their  web  site,  they  organized  very  brief  planning  outlines  or 
story  boards  for  their  web  sites.  Students  then  participated  in  a brief  presentation  on  the  use  of  a web  editor 
software  product.  In  the  teams,  students  were  assured  that  no  one  had  to  be  an  expert  on  the  content  or  the 
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digital  technology  skills,  and  they  were  encouraged  to  work  together.  Students  then  investigated  how  to 
apply  the  technological  skills  to  design  the  artistic  backgrounds  for  their  web  sites  and  to  include  content 
for  the  sites.  Students  also  discovered  that  some  of  their  peers  had  some  background  in  using  web  editors, 
and  those  students  took  on  a role  of  teaching  others  how  to  use  the  web  editor  and  its  functions.  We 
observed  that  as  students  became  more  confident  about  the  use  of  the  web  editor  and  some  of  its  functions, 
they  also  designed  creative  presentations,  with  a range  of  sophistication  in  the  look  and  content  for  their 
sites.  Some  sites  were  novice  web  sites,  providing  presentations  of  information  from  paper  based 
materials,  and  others  were  more  imaginative  web  sites  with  interesting  graphic  design  and  use  of  the  web 
functions  and  capabilities. 

Students  in  a Children’s  Literature  course  also  worked  in  teams  to  design  and  implement  a web  site  on  a 
children’s  book  and  author.  Similarly,  students  planned  out  the  organization  of  their  web  site  with  story- 
board-like plans.  They  worked  on  ideas  for  the  web  site  to  be  a comprehensive  tool  for  a child  so  that  the 
child  could  develop  comprehension  skills  and  higher  level  thinking  through  activities  and  features  of  the 
web  site.  These  students  also  became  more  creative  and  imaginative  about  their  web  site  and  the  potential 
for  interactivity  and  design  elements  as  they  became  more  comfortable  with  the  web  editor  and  as  they 
thought  more  creatively  about  the  content. 

This  paper  will  present  work-in-progress  with  a sample  of  web  sites  from  student  teams  and  excerpts  from 
their  written  reflective  papers  about  their  progress  in  the  creative  process  and  change  in  their  sense  of 
efficacy  as  they  worked  with  academic  content  m.aterial  and  the  web  editor  and  its  capabilities. 

The  experiences  of  these  students  is  illustrative  of  the  purpose  of  the  ThinkQuest  for  Tomorrow’s  Teachers 
grant:  To  prepare  new  prospective  teachers  to  be  able  to  combine  practice  in  the  Guiding  Partner 
Approach  and  the  use  of  internet  technologies  and  content  knowledge,  so  that  they  develop  an  ideological 
basis  for  professional  work  that  encourages  exploration,  collaboration,  and  constructivist  pedagogy. 
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The  New  Scholar's  Interface:  Virtual  Image  Technologies  Compared 

James  M.  Duncan,  M.L.I.S.,  Hardin  Library  for  the  Health  Sciences,  The  University  of 
Iowa  Libraries;  Fred  R.  Dee,  MD,  Department  of  Pathology,  The  University  oflowa 
College  of  Medicine 

Abstract 

Virtual  slide  and  virtual  image  technology  has  the  potential  to  revolutionize  the  way  we 
teach  and  share  microscopy  and  other  kinds  of  high- resolution  images.  This  technology 
applies  to  not  only  student-  focused  instructional  media,  but  also  to  historical  research  and 
web-based  publishing.  In  this  paper,  the  authors  compare  fiinctional  aspects  of  "virtual 
image"  technologies  based  on  Flashpix,  QuickTime  VR  and  other  formats.  Advantages 
and  disadvantages  of  each  format  will  be  discussed  with  regard  to  the  content  of  images, 
the  target  audience  and  the  pedagogical  purpose.  This  paper  also  will  discuss  some  details 
about  technical  requirements  and  basics  of  authoring. 

Why  Use  Virtual  Images? 

Increasingly,  libraries  and  academic  departments  are  digitizing  and  delivering  image 
content  for  use  by  researchers  and  students.  In  many  cases  such  projects  involve  web 
delivery  of  static  images  no  greater  than  72  dpi.  This  may  be  sufficient  if  the  content  of 
the  image  and  end-use  of  the  image  does  not  require  more  than  low  resolution.  However, 
for  intensive  analysis  and  study,  higher  resolution  alternatives  are  often  preferred. 
Desirable  ftmctional  mechanisms  for  interacting  with  such  high-resolution  images 
include  the  ability  to  magnify  or  "zoom  in"  on  an  image,  pan  across  an  image,  or  point  to 
landmarks  or  highlighted  areas  of  an  image.  Interaction  with  digital  images  introduces 
opportunities  that  can  contribute  to  greater  understanding,  particularly  when  added- value 
contextual  content  can  be  naturally  incorporated  within  the  image  file  itself  or  the 
interface  within  which  the  image  is  displayed. 

For  several  years,  staff  and  faculty  members  at  the  University  oflowa  Libraries  and  the 
University  oflowa  College  of  Medicine  have  been  working  with  high- resolution  image 
content  and  various  methods  of  delivering  that  content.  In  one  project  involving  a 
formative  evaluation  of  virtual  slides  used  during  a first-year  histology  laboratory, 
students  rated  these  images  equal  to  the  traditional  microscope  in  terms  of  image  quality 
and  utility,  and  rated  them  superior  in  terms  of  learning  efficiency  and  ease  of  access. 
(Harris,  et  al.,  2001).  In  another  project,  high-resolution  representations  of  rare 
anatomical  illustrations  were  rendered  in  a QTVR  format  and  delivered  to  the  web 
(http://www.lib.uiowa.edu/hardin- www/mascagni/Q.  Project  plamiers  are  continuing  to 
explore  mechanisms  for  studying  and  evaluating  how  this  format  (and  its  interactive 
functions)  impacts  the  way  in  which  individuals  conduct  historical  research. 

The  question  of  when  to  utilize  one  particular  virtual  image  format  over  another  depends 
on  several  factors:  file  size  and  resolution  of  the  digital  source;  degree  of  detail  required 
for  end  use;  server  resources  available  for  storage  and  delivery;  availability  of  technical 
expertise;  and  budget. 
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Microscopy  Images 

In  the  case  of  virtual  microscope  images,  typical  image  acquis  ition  involves  the  capture 
of  contiguous  fields,  which  are  then  tiled  or  stitched  together  to  form  one  seamless  large 
image.  In  cases  involving  a QTVR  solution,  producers  have  created  virtual  image  files 
with  starting  source  files  as  large  as  3,472  x 2,784  pixels  (Trelease,  2000).  "Hot  spots" 
provide  linkages  between  virtual  images.  For  example,  one  virtual  image  captured  at  lOx 
magnification  could  be  linked  to  another  captured  at  40x.  The  benefit  to  using  QTVR  is 
that  it  is  based  on  an  industry  standard  technology,  Apple  Computer's  QuickTime 
('http://www.aDple.com/quicktime/qtvr/).  This  is  free  system  software  already  used  by 
millions  for  playing  and  authoring  multimedia.  For  the  virtual  slide  project  at  The 
University  of  Iowa,  producers  captured  up  to  1,200  images  at  640  x 480  resolution,  each 
300  dpi.  After  tiling  these  images,  the  source  image  for  a virtual  slide  would  be 
approximately  18,000  x 18,000  pixels,  and  in  TIFF  (tagged  image  file  format)  would 
consume  close  to  1 GB  per  slide.  After  compression  in  a FlashPix  format,  each  slide  uses 
approximately  125  MB  of  storage.  Image  quality  is  impressive  in  the  end  delivery  of 
virtual  microscope  images,  which  can  be  accessed  via  a Java-enabled  web  browser. 
Regardless  of  the  approach,  a microscope  equipped  with  a high-end  digital  camera  or 
video  camera  is  required  to  produce  professional  quality  virtual  slides.  Of  the  two  options 
presented  above,  QTVR  is  the  most  approachable  in  terms  of  technical  expertise 
required;  off-the-shelf  software  is  widely  available  for  producing  such  images. 

Production  of  microscopy  images  using  the  FlashPix  approach  involved  significant 
storage  capability,  a customized  server,  and  skilled  personnel  for  media  production  and 
technical  support,  which  translates  to  higher  costs. 

Historical  Images 

A library-based  project  where  rare  images  from  a special  collection  were  delivered  in  the 
QTVR  format  is  being  considered  the  first-pass  at  working  with  this  source  content. 
Starting  with  600  dpi  images  captured  from  professionally  photographed  35mm  slides, 
producers  imported  the  images  into  QTVR  Authoring  Studio,  an  off-the-shelf  software 
package.  Using  a variety  of  compression  settings  and  exporting  techniques,  each  image 
was  produced  as  a QTVR  object,  providing  zoom  and  pan  capabilities.  Hot  spots  were 
pre-authored,  linking  to  more  focused  views  of  select  portions  of  each  image.  One  of  the 
drawbacks  to  the  format  was  that  any  zooming  became  centered  in  the  middle  of  the 
image;  to  see  a magnified  portion  of  an  image  at  the  edge  of  the  picture,  an  user  must  first 
zoom  in  and  then  pan  across.  Future  project  plans  include  re-authoring  the  virtual  images 
using  a different  authoring  tool,  Zoomifyer,  which  will  fix  the  zooming  issue  and  will 
provide  alternate  file  format  delivery  options,  such  as  Java. 
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Abstract:  This  article  deals  with  a significant  challenge  in  education  today:  the  challenge  to 
teach  life  long  learning  and  critical  thinking  skills  in  web-based  environments.  This  is  a 
challenge  for  two  reasons.  First,  business  and  government  are  pressuring  educational 
institutions  to  prepare  employees  who  can  think  critically,  solve  a range  of  problems,  move 
easily  from  one  task  to  another,  work  in  team  situations,  and  continuously  enhance  their 
knowledge  and  skills.  Second,  consumers  (i.e.,  students)  desire  more  distant  learning 
strategies.  Most  strategies  that  develop  these  skills  emphasize  small  group  work,  collaboration, 
and  teacher/group  interactions.  One  approach  to  this  challenge  is  to  apply  the  Rich 
Environments  for  Active  Learning  (REAL)  model  to  web-based  learning  environments. 
REALS  are  interactive,  student -centered  learning  environments  that  rely  on  intentional 
learning,  authentic  contexts,  generative  learning  activities,  collaboration,  and  reflection  to 
address  the  learning  of  content  and  life  long  learning  skills. 


The  Collision  of  Two  Educational  Goals 

Goal  #1:  Preparing  People  for  an  Ever-changing  World 

Changing  global  economic  circumstances  and  increasingly  complex  societal  needs  place  greater  and 
greater  pressure  on  education  systems  to  develop  learners  who  can  apply  knowledge  and  skills  in  new  domains 
and  situations.  Both  public  and  private  institutions  expect  and  demand  employees  who  can  think  critically  and 
solve  a range  of  problems,  move  easily  from  one  task  to  another,  work  efficiently  and  effectively  in  team 
situations,  and  constantly  adjust  and  enhance  their  knowledge  and  skills  to  meet  emerging  needs;  yet,  these 
institutions  claim  that  those  people  are  difficult  to  find. 

This  isn’t  too  surprising.  Conventional  instruction  often  utilizes  simplified  and  decontextualized 
examples  and  problems.  This  leads  to  inert  knowledge  (Whitehead,  1929)  - knowledge  that  cannot  be 
transferred  to  real  problems  and  situations.  Learners  are  not  asked  to  take  responsibility  for  their  own  learning  - 
they  do  not  set  learning  goals,  ask  questions  to  direct  learning  activities,  assess  their  learning  strategies  and 
approaches,  or  reflect  on  what  they  have  learned.  This  lack  of  focus  on  metacognitive  and  self-directed  learning 
skills  interferes  with  their  ability  to  transfer  their  knowledge  and  skills  to  future  needs.  Therefore,  to  meet  the 
goal  of  “preparing  people  for  an  ever-changing  world”,  instructional  programs  need  to  apply  methods  that  focus 
on  the  development  of  higher-order  learning  skills  including  critical  thinking,  problem  solving,  research,  and 
life  long  learning. 


Goal  #2:  Learning  at  a Distance  on  Demand 

The  learning  audience  shapes  the  second  goal  — the  demand  for  more  distance  education 
opportunities.  Generally,  most  definitions  of  distance  education  include  the  concept  of  time-  and  space- 
independent  teaching  and  learning.  This  includes  the  use  of  information  and  communications  technologies, 
interactive  video,  and  computer  networks  to  enable  asynchronous  and  synchronous  leamer-to-facilitator, 
leamer-to-leamer,  and  leamer-to-content  interaction.  Distance  learning  alternatives  help  learners  deal  with  a 
number  of  the  personal  constraints,  obstacles,  and  needs: 

• People  live  in  remote  geographic  areas  far  from  educational  institutions. 

• Local  educational  institutions  may  have  a limited  number  of  program  options  from  which  to  choose. 

• People  have  work  schedules  that  conflict  with  campus-bound  course  schedules.  This  includes  people  who 
work  shifts,  travel  frequently  on  business,  work  long  hours,  and  are  in  the  armed  forces. 
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• People  have  personal  and  family  commitments  that  conflict  with  campus-bound  course  schedules  including 
children  at  home  and  aging  parents. 

• After  a long  day  at  work,  people  don’t  want  to  battle  traffic  or  parking  to  get  to  a campus  on  time  (and 
encountering  the  stress  caused  by  trying  to  is  not  the  best  way  to  start  a learning  activity). 

• Some  learners  may  simply  prefer  a distance  format  over  a face-to-face  format  because  of  their  learning 
styles  and  preferences  (e.g.,  more  comfortable  sharing  ideas  asynchronously). 


Coming  to  Terms  with  These  Goals 

The  tension  created  when  these  two  goals  collide  is  an  instructional  design  one:  On  the  one  hand,  how 
do  you  help  people  learn  in  a manner  that  enables  them  to  transfer  their  skills  and  knowledge  to  a wide  variety 
of  situations;  and  on  the  other  hand,  how  do  you  address  the  learning  audience’s  expectation  for  learning 
experiences  available  on  demand  in  highly  individualized  ways?  One  approach  to  this  conflict  is  the  application 
of  the  Rich  Environments  for  Active  Learning  (REAL)  model  to  web-based  learning  environments. 


Rich  Environments  for  Active  Learning  (REALs) 

A model  for  applying  the  concepts  of  constructivism  to  instructional  practice,  REALs  are 
comprehensive  instructional  systems  that  engage  learners  in  dynamic,  authentic  learning  activities  that  increase 
their  control  and  responsibility  over  the  learning  process  while  they  learn  problem-solving  and  collaborative 
skills  and  content  (Dunlap  & Grabinger,  1995;  Grabinger  & Dunlap,  1995;  Grabinger,  Dunlap,  & Duffield, 
1997;  Kommers,  Grabinger,  & Dunlap,  1996).  The  REAL  model  utilizes  the  following  instructional  strategies: 

• intentional  learning  and  student  responsibility 

• authentic  contexts  and  relevant,  meaningful  learning 

• dynamic,  generative  learning  activities 

• collaboration  and  the  social  negotiation  of  meaning 

• extensive  reflection  and  self-assessment 


Encouraging  Intentional  Learning:  Taking  Ownership 

Intentional  learning  refers  to  the  cognitive  processes  that  have  learning  as  a goal.  Students  engaged  in 
intentional  learning  are  purposeful,  effortful,  self-regulated,  and  active  learners  (Palincsar,  1990;  Palincsar  & 
Klenk,  1992;  Scardamalia  & Bereiter,  1985;  Scardamalia,  Bereiter,  McLean,  Swallow,  & Woodruff,  1989; 
Scardamalia  & Bereiter,  1991;  Scardamalia  & Bereiter,  1997).  Encouraging  students  to  take  “an  intentional 
stance  toward  cognition”  helps  students  learn  how  to  monitor  their  own  thinking  and  learning  processes  (i.e., 
metacognitive  skills),  and  to  pursue  individually  determined  learning  goals  (i.e.,  self-directed  learning).  When 
students  take  responsibility,  or  ownership,  over  their  own  learning,  they  develop  metacognitive  and  lifelong 
learning  abilities  (Honebein,  1996). 

REALs  help  students  manage  their  own  learning  by  identifying  their  learning  needs,  setting  learning 
objectives,  selecting  and  employing  learning  strategies,  using  appropriate  resources,  and  assessing  their  overall 
process.  Research  (Scardamalia  & Bereiter,  1991)  indicates  that  students  can  assess  what  they  know  and  don’t 
know  and  learn  to  ask  questions  to  guide  their  knowledge  building,  thus  assuming  a “higher  level  of  agency” 
and  more  ownership  for  their  learning.  To  teach  for  intentional  learning  means  to  cultivate  those  general 
abilities  that  make  it  possible  to  become  independent,  life  long  learners  (Palincsar,  1990). 

Authentic  Contexts 

It  is  difficult  to  transfer  learning  from  one  situation  to  another.  Learning  is  more  likely  to  be  transferred 
if  instruction  is  situated  within  a realistic  context  (Brown,  Collins,  & Duguid,  1989).  Anchoring  learning  in 
larger,  more  complex  contexts  helps  prevent  the  acquisition  of  inert  knowledge  (Cognition  and  Technology 
Group  at  Vanderbilt,  1993).  Because  understanding  is  developed  as  a natural  consequence  of  interaction  with  a 
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complex  environment,  learning  activities  should  be  authentic,  reflecting  the  types  of  interactions  students  are 
likely  to  face  in  the  “real  world”  (Honebein,  1996). 

Authenticity  is  an  important  part  of  REALs  for  four  reasons.  First,  realistic  problems  hold  more 
relevance  to  students*  needs  and  experiences  because  they  can  relate  what  they  are  learning  to  problems  and 
goals  that  they  see  every  day  (Blumenfeld,  Soloway,  Marx,  Krajcik,  Guzdial,  & Palincsar,  1991;  Pintrich,  Marx, 
& Boyle,  1993).  Second,  authentic  situations  that  reflect  the  true  nature  of  problems  enable  students  to  develop 
deeper  and  richer  knowledge  structures  (Albanese  & Mitchell,  1993)  leading  to  a higher  likelihood  of  transfer  to 
novel  situations.  Third,  authenticity  encourages  interaction  through  collaboration,  and  negotiation  (Johnson  & 
Johnson,  1979;  Lowry  & Johnson,  1981).  Finally,  ill-structured,  complex  problems  require  a team  approach  that 
provides  natural  opportunities  for  learners  to  seek  out  information,  test  and  refine  their  ideas,  and  help  each 
other  understand  the  content. 


Dynamic,  Generative  Learning  Activities 

Learners  are  active  constructors  of  knowledge  - not  just  passive  receptors  of  information.  Generative 
learning  activities  require  students  - individually  and  collaboratively  - to  be  responsible  for  creating, 
elaborating,  and  representing  domain  knowledge  in  an  organized  manner  (Cognition  and  Technology  Group  at 
Vanderbilt,  1990,  1993;  Hannafin,  1992;  Scardamalia,  Bereiter,  McLean,  Swallow,  & Woodruff,  1989; 
Scardamalia  & Bereiter,  1991).  Through  generative  learning  activities,  learners  take  an  active  role  in  forming 
new  understandings  through  the  creation  of  products  and  solutions  to  authentic  challenges.  This  process  of 
“generating”  knowledge  - instead  of  passively  receiving  information  - helps  learners  develop  transferable 
knowledge  structures,  strategies,  and  the  skills  for  life  long  learning. 

Some  generative  learning  activities  provide  students  with  a context  or  situation  requiring  them  to  take 
action  (e.g.,  a problem  that  needs  to  be  solved  or  a case  that  needs  to  be  analyzed).  For  example,  Schank,  Fano, 
and  Jona  (1993)  describe  a generative  learning  environment  in  their  discussion  of  the  research  method  of 
teaching.  Under  the  research  method  of  teaching,  students  research  a particular  topic  and  then  present  their 
results  to  others  (e.g.,  the  class,  a collaborative  group,  etc.).  In  this  way,  students  take  over  the  responsibilities 
of  information  gathering,  synthesis,  and  dissemination  from  the  teacher.  To  be  successful,  students  need  to  be 
allowed  to  select  their  own  topics  to  research  and  report  on,  so  that  they  have  a real  interest  in  proceeding  with 
the  assignment  and  have  more  control  over  their  learning.  Because  the  learning  is  student-directed,  the  learning 
is  more  meaningful.  Bruner  (1961)  states,  “...in  general,  material  that  is  organized  in  terms  of  a person’s  own 
interests  and  cognitive  structures  is  material  that  has  the  best  chance  of  being  accessible  in  memory”. 


Collaboration 

Learning  takes  place  in  a social  context;  higher  cognitive  processes  originate  from  social  interactions 
(Vygotsky,  1978).  Lebow  (1993,  p.  6)  states  that  knowledge  acquistion  is  “firmly  embedded  in  the  social  and 
emotional  context  in  which  learning  takes  place”.  Thus,  collaboration,  conversation,  communication,  and 
establishing  a community  of  learners  are  critical  to  the  teaching  and  learning  process  (Pask,  1975). 

REALS  demand  collaboration  among  students  to  achieve  complex  goals.  By  employing  collaboration 
strategies,  REALs  help  learners  engage  in  a number  of  activities  that  support  successful  learning: 

• Collective  problem  solving.  Groups  give  rise  synergistically  to  insights  and  solutions  that  would  not  come 
about  individually.  While  working  in  collaborative  groups,  learners  are  more  willing  to  take  on  the  risk 
required  to  tackle  complex,  ill -structured,  authentic  problems  because  they  have  the  support  of  others 
(Brown  etal.,  1989;  Vygotsky,  1978). 

• Displaying  multiple  viewpoints.  Students  experience  and  develop  an  appreciation  for  multiple  perspectives 
when  working  with  others.  They  may  also  play  different  roles  within  the  group  to  gain  additional  insights. 
Conceptual  growth  comes  from  sharing  perspectives  and  testing  ideas  with  others  - a negotiating  process 
that  modifies  internal  representations  (Bednar,  Cunningham,  Duffy,  & Perry,  1991). 

• Confronting  ineffective  strategies  and  misconceptions.  In  collaborative  work,  group  members  draw  out, 
confront,  and  discuss  both  misconceptions  and  ineffective  strategies.  Through  collaborative  participation, 
students  also  refine  their  knowledge  through  argumentation,  structured  controversy,  and  the  sharing  and 
testing  of  ideas  and  perspectives. 
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• Providing  collaborative  work  skills.  Students  leam  to  work  together  in  a give-and-take  interaction  rather 
than  just  dividing  the  workload.  By  participating  in  collaborative  learning  activities,  learners  gain  an 
appreciation  for  the  value  of  cooperation  and  the  individual  strengths  that  members  of  the  team  bring  to  the 
group. 


Reflection  and  Self-Assessment 

Self-reflection  activities  are  embedded  into  REALs  in  order  to  support  both  the  development  of 
knowledge  and  metacognitive  skills.  Von  Wright  (1992)  defines  metacognitive  skills  as  “the  steps  that  people 
take  to  regulate  and  modify  the  progress  of  their  cognitive  activity:  to  leam  such  skills  is  to  acquire  procedures 
that  regulate  cognitive  processes.”  Glaser  (1984)  describes  this  as  knowing  what  one  knows  and  does  not  know, 
predicting  outcomes,  planning  ahead,  efficiently  apportioning  time  and  cognitive  resources,  and  monitoring 
one’s  efforts  to  solve  a problem  or  leam.  More  specifically,  metacognitive  skills  that  are  required  for  life  long 
learning  include  (Ridley,  Schutz,  Glanz,  & Weinstein,  1992): 

• recognition  of  content  and  skill  limitations; 

• setting  goals  and  creating  action  plans  based  on  those  defined  limitations; 

• activating  the  appropriate  prior  knowledge  to  achieve  set  goals; 

• assessing  progress  in  learning  and  task  performance  and  effectiveness  of  learning  resources  selected; 

• awareness  of  what  still  needs  to  be  completed  to  reach  a goal,  and  how  best  to  allocate  time  and  resources; 
and 

• modification  of  strategies,  tactics,  processes,  and  resource  selection  based  on  the  needs  of  the  task  at  hand. 

Even  though  reflective  activity  is  important,  it  is  possible  for  students  to  be  so  caught  up  in  completing 
a task  that  they  fail  to  reflect,  impeding  the  learning  process.  “We  can  keep  students  so  busy  that  they  rarely 
have  time  to  think  about  what  they  are  doing,  and  they  may  fail  to  become  aware  of  their  methods  and  options” 
(Wheatley,  1992,  p.  536).  Schon  (1983)  refers  to  this  as  being  “in  the  action”  rather  than  reflecting  “on  the 
action.”  If  students  do  not  have  opportunities  to  examine  their  methods  and  options,  they  will  not  develop  the 
metacognitive  skills  needed  for  life-long  learning.  Therefore,  learning  activities  need  to  support  students  in 
reflecting  on  their  own  learning  and  problem-solving  processes,  as  well  as  on  what  they  have  learned  (Schon, 
1987). 


Conclusion 

A consistent  theme  within  each  of  the  REAL  guidelines  b “interaction.”  Interaction  refers  to  the 
engagement  of  learners  in  the  learning  process.  By  engaged  learning,  we  mean  that  all  student  activities  involve 
active  cognitive  processes  including  creating,  problem  solving,  reasoning,  decision  making,  and  evaluation 
(Kearsley  & Shniederman,  199$.  Interactivity  involves  the  learners  in  options:  watching,  browsing,  finding, 
doing,  using,  linking,  annotating,  constructing,  creating,  and  elaborating  (Ambron  & Hooper,  1988;  Sims, 
1997).  The  use  of  technology  does  not  diminish  the  importance  of  good  pedagogy  and  good  pedagogy  demands 
interaction.  This  is  especially  true  for  web-based  instruction  with  the  critical  need  for  leamer-to-content, 
leamer-to-leamer/s,  and  learn er-to-facilitator  interaction. 
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Abstract:  This  paper  describes  how  Open  Source  philosophy  - a movement  that  has 
developed  in  opposition  to  the  proprietary  software  industry  - has  influenced  our  educational 
practice  in  the  pursuit  of  scholarly  freedom  and  authentic  learning  activities  for  our  students 
and  ourselves.  This  paper  provides  a brief  overview  of  the  Open  Source  movement,  and 
describes  three  Open  Source-inspired  Web-based  tools/environments  developed  to  promote 
open  sharing  and  constructing  of  scholarly  work  on  the  Web. 


Overview 

Do  you  know  the  story  about  the  discovery  of  the  structure  of  DNA?  Or,  the  discovery  of  the  Rosetta 
Stone  cipher?  Both  are  stories  of  not  only  brilliance,  but  also  competition.  Relying  on  the  foundational  work  of 
Linus  Pauling,  Francis  Crick  and  James  Watson  discovered  the  double  helix.  Then  separately,  relying  on  the 
foundational  work  of  Thomas  Young,  Jean  Francois  Champollion  deciphered  the  Stone’s  script.  However,  due 
to  competition  and  secrecy,  which  kept  all  parties  from  openly  sharing  their  findings  (and  possibly  sharing  the 
rewards  of  discovery... after  all,  whose  name  do  you  remember?),  the  progress  of  science  may  have  been 
delayed.  This  type  of  competition  is  in  direct  conflict  with  the  nature  of  science  and  discovery  - a process  of 
evolving  exploration  founded  on  the  work  of  others. 

Educators  are  concerned  with  the  impact  this  type  of  competition  and  secrecy  can  have  on  their  practice  as 
well;  educational  innovation  can  be  stymied  by  industry  and  individuals  that  perpetuate  the  politics  and  control 
over  professional  sharing  and  advancement  - such  as  the  publishing  industry  or  the  proprietary  communication 
and  collaboration  software  industry.  Although  some  educators  may  hold  secret  the  work  they  are  doing  to 
advance  knowledge  and  practice  around  good  teaching  and  learning,  most  are  interested  in  advancing  the 
collective  understanding  - in  sharing  insights,  ideas,  solutions,  and  materials  that  can  help  us  all  do  our  work 
more  effectively.  Educators  are  also  interested  in  creating  tools  and  structures  that  support  the  type  of  learner- 
centered,  collaborative  knowledge  sharing  that  can  lead  to  discovery  and  innovation.  Often,  educators  are 
stymied  in  these  pursuits  due  to  excessive  proprietary  software  costs  and  dwindling  institutional  budgets. 

This  paper  describes  how  Open  Source  philosophy  — a movement  that  has  developed  in  opposition  to  the 
proprietary  software  industry  — has  influenced  our  educational  practice  in  the  pursuit  of  scholarly  freedom  and 
authentic  learning  activities  for  our  students  and  ourselves.  This  paper  provides  a brief  overview  of  the  Open 
Source  movement,  and  describes  three  Open  Source-inspired  Web-based  tools/environments  we  have 
developed  with  our  students  and  colleagues  to  promote  open  sharing  and  constructing  of  scholarly  work  on  the 
Web. 


Open  Source  Movement 

Encouraging  innovation  through  open  sharing  to  facilitate  replication  and  discovery,  the  Open  Source 
movement  is  a grassroots  revolution  that  has  taken  shape  in  the  world  of  information  technology.  The  Open 
Source  movement  is  challenging  the  commercial  model  of  technology  advancement,  exemplified  in  the 
software  industry  by  Microsoft.  Linux,  an  operating  system  whose  source  code  is  open  for  the  world  to  see  and 
costs  nothing  to  download  and  use,  has  become  a significant  force  for  change  in  the  software  development 
world.  Other  information  technology  products  emerging  from  the  movement  include  Netscape/Mozilla, 
Apache,  Perl,  and  GIMP.  Open  Source  advocates  are  trying  to  create  a world  where  software  is  openly 
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available  and  a living  is  made  through  continuing  relationships  of  service  and  support.  These  same  Open  Source 
ideas  are  influencing  our  teaching  and  learning  practice.  The  following  are  three  examples  of  how  Open  Source 
has  impacted  our  work  with  students  at  the  University  of  Colorado  at  Denver. 


Self-Publishing:  NOVAtions  Online  Journal  [in  Joni^s  voice] 

A year  ago,  I was  informed  that  it  was  my  turn  to  teach  a doctoral  course  focused  on  instructional 
technology  during  the  up-coming  fall  semester.  This  course  is  part  of  a doctoral  program  in  educational 
leadership  and  innovation,  and  - as  such  - the  participating  doctoral  students  were  interested  in  a wide  range  of 
educational  foci:  teacher  education,  paraprofessional  preparation,  teaching  and  learning  in  K12  and  adult 
settings,  and  technology -enhanced  learning.  In  preparing  for  the  course  I anticipated  insightful,  yet  challenging, 
discussions  and  collaborations  given  the  diverse  backgrounds  and  interests  of  the  students.  I realized  that  there 
was  no  way  for  me  to  be  the  “expert”  in  all  of  their  interest  areas  during  our  experience  together,  and  that  I 
didn’t  need  to  be  because  the  students  were  emerging  experts  in  the  topic  foci  they  were  studying.  So,  I looked 
for  a way  to  create  a unifying  theme  and  project  while  at  the  same  time  taking  advantage  of  the  developing 
expertise  of  the  students  to  create  a valuable  learning  experience  for  everyone  - including  myself.  To  this  end, 
we  built  a grassroots  online  professional  journal. 

NOVAtions  is  an  online  journal  for  emerging  scholars  built  using  the  Slashdot.org  Open  Source  code  (see 
http://novatioiis.opeiiceiitfic.coni).  Siashdol.org  is  an  online  self-organizing  social  system  (OSOSS)  (Wiley  & 
Edwards,  2002)  that  is  a Web-based  news  source  in  which  the  readership  contributes  and  reviews  news  items, 
and  has  significant  influence  and  control  over  the  direction  of  and  themes  covered  by  the  site.  Our  NOVAtions 
journal  is  designed  to  provide  a forum  for  a community  of  practice  - scholars  interested  in  educational 
innovation  in  teaching  and  learning.  Contributors  to  the  journal  also  function  as  the  editorial  review  board,  in 
much  the  same  was  as  Amazon  and  Barnes  and  Noble  enable  the  book  buying  public  to  serve  as  reviewers. 

The  doctoral  students  in  my  course  built  the  NOVAtions  journal  from  scratch  (see  Figure  1).  Self- 
publishing  the  online  journal  provides  the  doctoral  students  with  an  authentic,  enculturating  leadership 
experience  in  which  they: 

• publish  articles  and  collaborative  book  reviews  to  learn  more  about  the  process  of  sharing  scholarly 
ideas  with  others  and  publishing  ideas  in  a journal,  and 

• serve  as  the  editorial  review  board  to  learn  about  the  editing  process  and  improve  their  own  writing. 
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Figure  1:  Students  self-publishing  \nNOVAtions  online  journal 


This  fledgling  journal  was  launched  in  Ebcember  2001,  and  plans  are  now  in  the  works  to  extend  the 
journal  participation  to  the  larger  community  of  practice  in  educational  leadership  and  innovation. 


Knowledge  Sharing:  Web  Resource  Collaboration  Center  (WRCC)  [in  Joni*s  voice] 


The  Web  Resource  Collaboration  Center  (WRCC)  (Dunlap,  1999)  is  a Web-based  tool  which  empowers 
learners  to  build  their  own  Web-based  resource  centers  - using  the  Web  to  provide  on -demand  access  to 
integrated  information,  guidance,  advice,  assistance,  training,  and  tools  - to  support  their  learning,  professional 
development,  and  performance. 

A few  years  ago,  I was  hired  by  an  information  technology  organization  to  “get  to  the  bottom**  of  why  its 
elaborate  electronic  performance  support  system  (EPSS)  which  was  available  on  the  company *s  intranet  was 
not  being  utilized  by  employees.  The  company  had  used  its  training  and  development  resources  to  build  this 
EPSS  to  help  employees  keep  up  with  all  of  the  new  technologies  they  were  expected  to  master  for  the  various 
contractual  projects  of  the  organization.  Since  a front-end  analysis  was  not  actually  conducted  before  the 
development  of  the  EPSS,  this  is  where  I started.  The  employees  were  pleased  that  the  EPSS  provided  a variety 
of  resources  (e.g.,  tutorials,  white  papers,  job  aids,  business  cases,  etc.)  to  support  their  various  learning  needs 
and  preferences.  Instead  of  conventional  training,  they  wanted  access  to  learning  and  professional  development 
resources  that  would  help  them  keep  their  knowledge  and  skills  “cutting  edge**. 

Although  they  were  not  against  the  idea  of  an  EPSS,  the  employees  did  not  believe  that  the  developers  of 
the  EPSS  understood  what  resources  they  needed.  In  addition,  they  certainly  believed  that  developers  did  not 
know  how  to  present  them  in  contextualized  ways  (e.g.,  resources  that  would  help  with  one  type  of  project  vs. 
another  type  of  project).  They  were  also  concerned  that  there  was  no  way  to  capture  the  “here*s  how  I did  it** 
expertise  of  the  people  in  the  organization,  and  in  the  external  community  of  practice.  Finally,  the  EPSS  was 
static.  This  was  in  sharp  contrast  to  the  constantly  evolving  information  and  tools  these  employees  were  using. 
The  most  up-to-date  information  was  being  distributed  on  the  Web.  The  EPSS  was  not  dynamic  enough  to 
capture  those  changes,  so  the  employees  were  using  the  Web  to  support  their  learning  and  professional 
development  - albeit  not  very  efficiently  which  led  to  frustration.  Bottom  line,  the  employees  had  been  cut  out 
of  the  process,  and  believed  that  they  were  better  judges  of  what  was  needed  to  support  their  learning  and  work. 

This  consultation  led  to  my  interest  in  developing  a tool  that  would  (1)  take  advantage  of  some  of  the 
structural  qualities  of  EPSS,  (2)  harness  the  resources  on  the  Web  (since  it  was  a distribution  source  for  some  of 
the  most  up-to-date  information  and  tools),  and  (3)  provide  a structure  for  learning  communities  and 
communities  of  practice  to  build  their  own  unique  content  to  support  both  lifelong  learning  and  professional 
development  activities.  To  meet  this  challenge,  we  created  a Web-based  development  tool  called  the  Web 
Resource  Collaboration  Center  (WRCC). 

By  creating  a structure  that  supports  individualized  and  collaborative  knowledge  building  by  the  people 
who  will  actually  be  using  the  knowledge,  the  higher-order  thinking,  problem-solving,  and  decision-making 
involved  in  the  selection  and  utilization  of  appropriate  learning  materials  and  performance  support  is  done  by 
those  who  can  get  the  most  out  of  the  process.  Additionally,  because  these  activities  occur  in  the  workplace  and 
are  driven  by  the  needs  of  the  job  at  hand,  the  learning  activities  are  contextualized,  authentic,  and  meaningful. 
The  WRCC  meets  the  following  goals: 

• The  users  learn  about  the  domain  while  they  are  locating,  evaluating  (which  requires  utilization  of 
resources),  organizing,  and  creating  resources  to  support  their  learning  and  job  performance  activities 
- making  the  process  relevant  and  productive; 

• The  content  of  the  WRCC  is  information  that  has  been  applied/articulated  from  the  perspective  of 
reflective  practice,  making  the  WRCC  a knowledge  management  forum; 

® The  WRCC  is  developed  by  and  for  the  people  involved  in  the  project,  challenge,  or  domain;  and 

• The  WRCC  can  change  and  adapt  based  on  the  changing  organizational  and  leaming/performance 
needs  because  the  end-users  control  the  content. 

In  this  way,  the  WRCC  not  only  enables  learners  to  build  a learning  and  performance  resource  that  will  provide 
them  with  immediate  support  and  guidance,  but  also  helps  fliem  develop  structure,  strategies,  and  skills  for 
subsequent  learning  activities. 

To  provide  a structure  for  these  activities,  the  WRCC  is  broken  into  three  functional  areas:  the  Discussion 
Forum,  the  Link  Manager,  and  the  Resource  Construction  System  (see  Figure  2).  These  tools  - written  entirely 
in  Perl  - are  not  unique  - there  are  similar  tools  available  from  a variety  of  sources.  The  impact  is  in  the  use 
and  integration  of  the  tools,  and  the  fact  that  they  are  Open  Source  and  support  learner-centered  knowledge 
sharing. 
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Figure  2:  Examples  of  WRCC’s  Link  Manager,  Discussion  Forum,  and  Resource  Construction  System 

Once  built,  I recognized  the  value  of  this  type  of  tool  for  learning  communities  in  general,  including  the 
students  I work  with  at  the  university.  The  WRCC  is  being  used  to  support  instructional  activity  in  both  face- 
to-face  and  online  courses,  and  as  a performance  support  and  knowledge  management  tool  by  self-organizing 
learning  communities  (see  http://carbon.cudenver.edu/public/wle/wrcc/techforkl2/  for  an  example  of  a WRCC). 
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Self-Organized  Learning  and  Performance  Support  Groups:  Electronic  Knowledge 

Base  (EKB) 

In  our  Information  and  Learning  Technologies  (ILT)  Master’s  program,  we  organize  students  into  cohorts. 
As  a cohort,  students  progress  through  the  program  together,  and  quickly  develop  of  professional  bond  that  lasts 
beyond  the  program.  One  ILT  Master’s  cohort  of  20  K-12  teachers  shared  a common  purpose  of  learning  how 
to  best  integrate  technology  into  their  instruction,  classrooms  and  schools.  This  group  was  constantly  sharing 
their  views,  understandings,  and  opinions,  as  well  as  information  and  research  they  unearthed.  This  collective 
base  of  knowledge  was  the  lifeblood  of  their  common  endeavor,  and  they  wanted  to  “capture”  the  knowledge 
they  were  discovering  and  constructing  with  each  other. 

Inspired  by  Jonassen,  Peck,  and  Wilson’s  (1997)  note  that  ’’learning  and  knowledge-building  communities 
depend  heavily  on  ...  a rich  collection  of  information  and  learning  resources  t)  support  them,"  this  cohort 
designed  and  developed  Electronic  Knowledge  Bases  (EKBs)  using  adapted  Perl  scripts  available  at  no  charge 
to  educational  institutions  (Young,  2000;  see  http://carbon.cudenver.edu/public/ilt/pages/).  They  used  the 
EKBs  as  a way  to  formalize  the  process  of  collecting  a rich  source  of  knowledge  resources.  These  databases 
served  as  an  easily  accessible  and  amendable  repository  of  knowledge  assisting  them  in  their  search  for  specific 
information  and  research  related  to  teaching  and  learning.  The  EKBs  supported  their  efforts  as  a dynamic 
learning  community,  and  became  a source  of  knowledge  for  other  educators  not  participating  in  the  ILT 
Master’s  program  (see  Figure  3).  Dave  Young,  then  a student  and  more  recently  a faculty  colleague  and  co- 
author, was  the  primary  developer  of  the  resource. 
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Figure  3:  EKB  Homepage,  Detailed  View  of  Contribution,  and  Resource  Review  Feature 
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Peer-to-peer  networking:  Our  Next  Adventure 


Although  we  haven’t  implemented  peer-to-peer  networking  in  our  environment,  we  are  very  intrigued  by 
the  possibilities  and  planning  it  as  a next  step  in  our  adventure  in  reclaiming  learner  control.  Peer-to-peer 
networking  refers  to  individual  users  sharing  resources  by  opening  up  their  hard  drives  to  each  other  - a group 
of  computers  communicate  directly  with  each  other,  rather  than  through  a central  server.  Besides  potentially 
having  a positive  impact  on  collaboration  and  knowledge  sharing,  it  can  also  make  good  technical, 
infrastructure  sense  - if  computing  power  and  resources  are  distributed  then  no  one  server  or  network 
connection  is  over-taxed. 

The  Napster  phenomenon  taught  us  that  downloading  from  central  servers  is  not  the  only  way  to  perpetuate 
an  online  enterprise.  The  core  concept  is  even  more  radical  than  Napster’s,  because  once  out  of  the  bag  and  in 
the  hands  of  end  users,  true  peer-to-peer  usage  cannot  be  controlled.  In  this  way  peer-to-peer  networking 
constitutes  a classic  form  of  self-organizing  system,  using  the  technology  to  bypass  every  form  of  central 
control  and  put  the  control  in  the  hands  of  learners. 


Conclusion 

The  Open  Source  movement  is  a reenergizing  catalyst  for  our  reclamation  of  learner  control.  Influencing 
how  we  think  about  supporting  collaboration,  knowledge  sharing,  and  teaching  and  learning  in  general,  we  are 
embracing  the  message  of  Open  Source  with  open  arms.  The  three  examples  described  above  - as  well  as  the 
work  being  done  by  colleagues  at  other  institutions,  such  as  Utah  State  University’s  SlashLEARN  project 
(Irving,  2001;  see  http://www.slashleam.org/)  and  StorageTek’s  COSS  project  (Ryder  & Wilson,  1997)  - have 
inspired  us  to  continue  to  enrich  our  practice  and  our  students’  learning  experiences  through  the  integration  of 
Open  Source-inspired  tools  and  environments.  We  bok  forward  to  sharing  our  projects  and  implementation 
results  with  our  ED -MEDIA  colleagues. 
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Abstract:  Oklahoma  Christian  University  has  implemented  a ubiquitous  computing 
program  where  every  student  and  faculty  member  are  equipped  with  IBM  ThinkPad 
laptops  that  connected  to  a wireless  network.  The  technological  enhancements  provided 
by  this  program  helped  to  create  an  environment  where  collaboration  between  students 
and  faculty  could  be  increased.  During  the  first  full  year  of  implementation,  one  course 
typically  taught  in  a lecture-based  format  was  re-designed  to  foster  more  collaboration 
and  active  learning.  The  instructor  enhanced  the  course  with  collaborative  technology, 
delivered  most  of  the  first  exposure  to  the  materials  online,  and  created  collaborative 
assignments  to  be  done  during  the  classroom  time.  A survey  and  several  interviews  were 
conducted  to  glean  student  feedback.  Students  found  the  course  challenging  and  they  rose 
to  meet  that  challenge. 


Introduction 


Oklahoma  Christian  University  is  blazing  a new  trail  with  the  o-campus  concept  instituted  in  the 
Fall  of  2001.  “The  new  technologies  provide  opportunities  for  creating  learning  environments  that  extend 
the  possibilities  of  “old”  (Bransford  et  al.,  1999).  Oklahoma  Christian  launched  its  wireless  e-Campus  in 
the  Fall  of  2001  by  distributing  IBM  ThinkPads  with  wireless  networking  capabilities  to  every  full-time 
student.  It  is  the  first  university  in  Oklahoma  and  one  of  the  few  nationwide  that  offer  to  all  students  a 
wireless  campus  and  laptop  computers.  Students  can  access  the  Internet  or  myOC  (the  university’s 
personalized,  web -based  portal)  from  virtually  anywhere  on  campus.  Such  a culture  change  affects  not  only 
students,  but  also  faculty.  For  some,  this  culture  change  is  an  easy  adjustment;  for  others,  a seemingly 
insurmountable  hurdle.  Yet,  the  change  affects  everyone  and  it  demands  adjustments  in  the  way  teachers 
teach  and  students  learn. 

The  use  of  technology  does  not  guarantee  that  effective  learning  will  occur  in  a course.  Bransford, 
Brown  and  Cocking  (1999)  say  that  technology  can,  in  fact,  hinder  learning  if  used  inappropriately. 
However,  when  used  appropriately  technology  can  connect  teachers,  the  instructional  experience,  and 
learners  in  ways  that  enhance  learning  (Newby,  et.  al.  2000).  The  goal  of  this  collaborative  effort  is  to  use 
technology  as  an  instructional  intervention  in  the  creation  of  an  effective  learning  environment  that  is  based 
on  current  research  regarding  how  people  learn. 

Integration  of  Pedagogy  and  Technology 

The  Office  of  Educational  Research  and  Improvement  of  the  U.S.  Department  of  Education 
charged  a 16  member  committee  with  the  task  of  evaluating  new  developments  in  the  science  of  learning 
(Bransford,  et  al.,  1999).  This  committee  reviewed  research  and  consulted  with  everts  in  the  process  of 


reporting  the  most  current  information  regarding  how  people  learn.  Their  recommendations  are  instructive 
for  all  educators. 

These  new  conceptions  of  learning  have  created  a shift  from  an  emphasis  on  rote  learning  and 
repetition  to  a focus  on  depth  of  understanding  and  the  ability  to  apply  new  knowledge.  The  implications  of 
this  new  focus  require  new  or  renewed  instructional  interventions  and  learning  environments  (McLaughlin 
and  Shepard,  1995). 

The  foundation  of  the  classroom  model  that  follows  is  a learner-centered  approach  to  education. 
This  model  is  steeped  in  collaborative  learning  and  has  a strong  emphasis  on  formative  assessment 
(Poindexter  et.  al,  2001).  This  discussion  continues  by  exploring  each  aspect  of  this  model  course 

“Learner- centered”  environments  value  the  knowledge,  skills,  attitudes,  and  beliefs  that  learners 
bring  with  them  to  the  classroom  (Bransford  et  al.,  1999).  The  conceptual  knowledge  and  beliefs  that 
students  bring  to  the  classroom  are  the  starting  point  of  instruction.  The  pre-assessment  in  the  model  class 
purposefully  investigates  the  knowledge  that  students  are  initially  bringing  to  this  course. 

Learning  is  greatly  influenced  by  the  context  in  which  it  occurs.  Instructors  are  charged  with 
creating  a learning  community  that  facilitates  students  sharing  their  preconceptions,  questions,  and  taking 
learning  risks  in  pursuit  of  academic  growth.  Cooperative  and  collaborative  learning  are  strongly  supported 
instructional  tools  in  the  research  literature.  They  have  been  shown  to  lead  to  cognitive  development 
(Johnson  and  Johnson  1984;  Slavin,  1991;  Sharan  and  Sharan,  1989/90;  Bransford  et  al.,  1999). 

The  model  course  that  follows  requires  students  to  read  and  listen  to  lectures  outside  of  the 
classroom  so  that  classroom  time  can  be  spent  on  collaborative  pursuits.  Students  are  placed  in  learning 
teams  as  they  work  through  multiple  milestone  assignments.  They  will  also  produce  a capstone  project.  The 
instructional  technology  embedded  in  the  course  encourages  collaboration  beyond  the  four  walls  of  the 
classroom.  Students  may  use  Microsoft  NetMeeting  to  work  together  online,  as  well  as  communicate  and 
collaborate  with  the  instructor.  Synchronous  chats,  message  boards  and  email  will  provide  students  with  the 
opportunity  to  receive  feedback  from  peers  outside  of  their  learning  team.  The  instructor  will  serve  as  a 
guide,  but  students  will  be  charged  with  directing  their  own  learning.  Student -involved  learning  has  a 
strong  link  to  academic  achievement  (Stiggins,  2001). 

A unique  and  innovative  aspect  of  this  course  is  its  use  of  formative  assessment.  Educators  of  the 
twenty-first  century  are  functioning  in  an  assessment  climate  that  is  almost  completely  focused  on 
summative  assessments  (Stiggins,  2001).  While  these  summative  assessments  provide  very  specific 
information  about  what  students  know,  they  do  little  to  promote  learning  (Gallagher,  2000;  Stiggins,  2001). 
Dylan  Wiliam  (2000)  defines  summative  assessment  as  assessments  that  are  “used  to  certify  student 
achievement  or  potential”  (p.  1).  He  goes  on  to  describe  formative  assessments  as  assessments  that 
“provide  feedback  to  learners  about  how  to  go  about  improving”  (p.  1).  Crooks  (2001)  talks  about 
formative  assessment  as  assessment  for  learning.  Several  research  projects  highlight  the  connection 
between  formative  classroom  assessment  and  academic  achievement.  Professor  Terry  Crooks  (1988)  of  the 
Univeristy  of  Otago,  New  Zealand  conducted  a synthesis  of  over  200  studies  on  the  use  of  formative 
assessment.  He  concludes  that  formative  classroom  assessment  can  have  a very  positive  impact  on  student 
achievement.  He  further  chronicles  the  connection  between  classroom  assessment  and  student  decision- 
making. “The  decisions  students  themselves  make,  he  (Crooks)  contends,  are  the  decisions  that  determine 
what  is  learned  (Stiggins,  2001).  “It  [classroom  assessment]  guides  students’  judgments  about  what  is 
important  to  learn,  affects  motivation  and  self-perception  of  competence,  structures  their  approaches  to  and 
timing  of  personal  study... consolidates  learning  and  affects  the  development  of  enduring  learning  strategies 
and  skills”  (Crooks,  1988,  p.  267). 

English  researchers  Paul  Black  and  Dylan  Wiliam  (1998)  conducted  a meta-analysis  of  more  than 
40  controlled  studies  centered  on  the  impact  of  improved  classroom  assessment  on  student  achievement  as 
reflected  in  summative  assessments.  They  asked  three  questions  in  this  project: 

• Is  there  evidence  that  improving  the  quality  and  effectiveness  of  use  of  formative  assessments  raises 
student  achievement? 

• Is  there  research  evidence  that  formative  assessments  are  in  need  of  improvement? 

• Is  there  evidence  about  the  kinds  of  improvements  that  are  most  likely  to  enhance  student 
achievement? 

The  results  conclude  a resounding  “yes”  to  all  three  questions.  The  results  cross  all  content  areas, 
knowledge  and  skill  types  and  all  levels  of  education.  Results  reveal  unprecedented  gains  in  achievement. 
They  report  effects  between  0.4  to  0.7. 
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An  effect  size  of  0.4  would  mean  that  the  average  pupil  involved  in  an  innovation  would 
record  the  same  achievement,  as  a pupil  in  the  top  35%  of  those  not  so  involved.  An 
effect  size  gain  of  0.7  in  the  recent  International  comparative  studies  in  math  would  have 
raised  the  score  of  a nation  in  the  middle  of  the  pack  of  41  countries  (e.g.,  US)  to  one  of 
the  top  five. 

(Black  and  Wiliam,  1998,  p.  141). 

Strong  evidence  supports  the  use  of  formative  classroom  assessment.  This  model  course  provides  multiple 
opportunities  for  students  to  receive  feedback  from  the  instructor  and  fellow  students  so  they  can  adjust 
their  learning.  It  also  provides  opportunity  for  the  instructor  to  assess  students  and  then  make  instructional 
decisions  based  on  those  assessments. 

Current  research  encourages  educators  to  “Explore  the  potential  of  new  technologies  that  provide 
the  opportunity  to  incorporate  formative  assessment  into  teaching  in  an  efficient  and  user-friendly  manner” 
(Bransford  et.  al.,  1999,  p.  39).  This  model  course  is  an  exploration  of  that  potential. 

Case  Study  Context 


The  model  course  consisted  of  37  freshman-level  students.  Eighty-three  percent  of  the  students 
were  between  18  and  20  years  old;  10  percent  were  between  23  and  24  years  old;  and  7 percent  were  25 
years  old  or  older.  Sixty  percent  of  the  class  was  male,  40%  were  female.  There  were  only  three 
international  .students  in  the  course,  one  from  Venezuela,  one  from  Australia,  and  the  third  from  Africa. 
Ninety  percent  of  the  students  lived  on-campus  and,  therefore,  had  24/7  access  to  the  technology  described 
above. 

The  model  course  was  designed  to  make  heavy  use  of  technology,  yet  the  technology  was  meant 
to  be  “transparent,”  used  as  a tool  of  the  trade  (much  like  a pencil  or  textbook).  Priority  was  given  to 
student  learning.  If  the  technology  inhibited  student  learning,  it  was  changed  or  removed.  This  course  was 
designed  so  that  it  could  be  offered  as  a hybrid  course  (i.e.,  a course  that  has  both  F2F  and  online 
components)  or  as  a distance  education  course. 

The  course  followed  a 20/80  rule  (sometimes  10/90),  meaning  only  be  about  20%  of  face-to-face 
(F2F)  time  was  used  for  lecture  while  about  80%  of  the  F2F  time  was  designated  for  students  to  collaborate 
on  assignments  or  for  in-class  presentations  (Poindexter  et.  al.,  2001).  To  accomplish  this,  every  lecture 
meant  to  give  the  students  first  exposure  to  the  material  was  moved  outside  the  classroom  and  into  the 
Blackboard  course  management  system  (Walvoord  & Anderson,  1998).  Each  lecture  was  accompanied  by 
the  instructor’s  notes.  The  learners  logged  into  the  Blackboard  course  site,  clicked  into  the  lecture  area,  and 
listened  to  the  lecture  prior  to  the  class  meeting.  These  lectures  consisted  of  digital  audio  of  the  instructor 
(or  guest  lecturer),  accompanied  by  PowerPoint  slides  which  were  synchronized  with  the  recording.  The 
student  could  stop,  play,  pause,  or  resume  these  presentations.  Moreover,  s/he  could  jump  ahead  or  back  in 
the  presentation  by  clicking  a hyperlink  to  the  desired  slide.  These  lecture  materials  were  available  to  the 
students  for  the  entire  semester  (and  beyond)  and  could  be  reviewed  at  the  discretion  of  the  student.  After 
listening  to  the  lecture,  the  students  took  quizzes  over  the  material  that  was  introduced  (all  quizzes  will  be 
given  online  using  the  built-in  assessment  feature  in  Blackboard).  The  instructor  reviewed  the  scores  of 
this  assessment  and  adjusted  the  10%-20%  classroom  lecture  time  for  remediation  as  necessary.  Other  than 
these  quizzes,  no  exams  were  given  in  this  course. 

The  students  were  grouped  into  teams  in  order  to  collaborate  on  assignments  and  the  final  project. 
The  groupings  were  based  on  several  criteria: 

o results  of  the  Keirsey  version  of  the  Myers-Briggs  temperament  analysis 

• learning  styles 

• self-selection  (limited) 

(Poindexter  et.  al.,  2001) 

Teams  collaborated  on  multiple  milestone  assignments  and  one  final  project.  The  milestone  assignments 
demonstrated  the  learners’  abilities  to  identify  and  accomplish  the  steps  in  the  interpretive  process.  Each 
team  chose  a passage  from  The  Acts  of  the  Apostles  and  they  worked  on  that  text  throughout  the  semester. 
These  passages  were  the  subject  of  the  milestone  assignments  as  well  as  the  final  project.  The  milestone 
assignments  were  primarily  written  assignments,  but  occasionally  students  were  asked  to  present  their  work 
in  class. 
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The  final  project  had  two  parts.  First,  each  team  submitted  an  interpretation  paper  to  their 
instructor  in  which  they  offered  and  defended  an  interpretation  of  the  passage  from  Acts  they  chose.  Next, 
each  team  gave  an  in-class  presentation  of  their  interpretation.  This  presentation  required  the  use  of  a 
technological  medium  for  its  delivery.  The  student  could  have  chosen  from  PowerPoint,  digital  video,  web 
sites,  or  some  other  digital  medium.  Their  presentations  could  not  exceed  20  minutes  in  length,  in  order  to 
allow  time  for  critical  peer  review.  The  team  of  students  filled  the  role  of  teacher  during  these 
presentations. 

The  instructor  provided  several  ways  for  students  to  communicate  with  him  and  even  collaborate 
on  assignments  as  necessary.  First,  Blackboard  has  many  built-in  features  to  allow  this:  o-mail,  message 
boards,  and  the  virtual  classroom  (i.e.,  synchronous  chat),  to  name  a few.  Moreover,  the  instructor  had 
embedded  Microsoft  NetMeeting  into  a web  page  and  uploaded  it  to  the  Blackboard  course  site.  He  pre- 
conflgured  the  software  to  call  his  computer  at  the  click  of  a button.  NetMeeting  allowed  the  learners  to 
call  the  instructor  over  the  Internet  and  collaborate  with  the  him  on  the  assignment  or  to  simply  receive 
feedback.  For  example,  if  a student  needed  feedback  on  a milestone  assignment,  s/he  may  call  the 
instructor  using  NetMeeting.  NetMeeting  allows  application  sharing  over  the  Internet,  so  s/he  could  open 
the  word  processing  application,  share  that  application  through  NetMeeting,  and  give  control  to  the 
instructor.  Then  the  instructor  could  actually  mark  up  the  student’s  document  with  comments,  suggestions, 
and  corrections  as  necessary.  This  allowed  for  more  than  just  feedback;  it  also  allowed  the  student  (or 
team)  to  actually  collaborate  on  an  assignment  with  the  instructor. 

The  instructor  provided  several  ways  for  the  team  members  to  collaborate  as  well.  First,  they,  too, 
could  have  used  NetMeeting  for  collaboration  in  the  same  manner  described  above  (NetMeeting  allows 
multiple  users  to  connect  with  each  other).  Next,  the  instructor  enabled  the  groups  feature  in  Blackboard. 
Each  group  could  collaborate  using  message  boards,  chat,  o-mail,  and  file  sharing — all  of  which  are  built 
into  Blackboard.  Each  team  had  its  own  message  board,  chat  room,  o-mail  address,  and  digital  drop  box, 
and  those  tools  were  only  open  to  members  of  the  same  team. 

The  instructor  provided  all  reading  and  research  materials  to  the  student  in  at  least  one  of  the 
following  ways: 

• digital  format  via  Blackboard  (articles  will  be  scanned  to  PDF  and  uploaded  to  the  content  areas  of  the 
course  web  site  in  Blackboard) 

• on  reserve  in  the  university  library 

• links  to  content  available  on  the  Internet  and  World  Wide  Web. 

Those  items  that  could  not  be  provided  were  listed  in  a digit  al  bibliography  so  the  students  could  order 
items  from  the  library  using  the  Interlibrary  Loan  service  or  found  them  on  Net  Library  in  o-Book  format. 

Findings  and  Discussion 

A survey  was  conducted  at  mid-term  of  the  course  and  several  interviews  with  students  were 
conducted  at  various  times  throughout  the  semester.  Each  of  these  assessments  revealed  some  interesting 
information.  First,  50%  of  the  students  agreed  or  strongly  agreed  that  the  technology  used  in  this  course 
enhanced  their  ability  to  learn  the  material.  The  students  found  themselves  interacting  with  the  material  and 
with  each  other  more  in  this  course  than  in  any  of  their  other  courses.  Because  the  lectures  were  online  and 
because  the  technology  allowed  them  to  communicate  with  each  other  so  much  easier,  they  found 
themselves  engaged  in  the  course  material  even  on  days  the  class  was  not  meeting. 

The  survey  and  the  interviews  uncovered  another  interesting  fact.  On  the  survey,  the  students 
were  asked  if  they  agreed  or  disagreed  with  the  following  statement:  “If  given  the  opportunity  to  take  this 
class  again,  I would  choose  an  Acts  class  with  a more  traditional  lecture  format.”  Fifty-four  percent  of  the 
students  agreed  or  strongly  agreed  with  this  statement,  20%  were  neutral,  and  27%  disagreed  or  strongly 
disagreed.  However,  several  interviews  with  different  students  enrolled  in  the  course  revealed  at  least  one 
reason  why  they  would  have  chosen  lecture-based  course:  it  is  easier.  Every  student  interviewed 
commented  that  this  course  was  very  demanding,  whereas  lecturo-based  courses  are  not  as  demanding 
because  “all  you  have  to  do  is  show  up,  memorize  facts  for  the  exam,  regurgitate  those  answers  on  the 
exam,  and  that  is  it.  There  are  really  no  assignments  where  we  have  to  work,  like  there  are  in  this  class.” 
Students  will,  however,  rise  to  meet  the  expectation  of  a course.  While  there  are  many  factors  contributing 
to  one  class’s  grades  being  higher,  it  is  still  interesting  to  compare  grades  from  this  course  and  the  same 
course  taught  in  a lecture-based  format  (see  Tab.  1). 
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Lecture-based 

Technology - 
enhanced 

A 

8 (25%) 

9 (26%) 

B 

10(31%) 

20  (57%) 

C 

5 (16%) 

4(11%) 

D 

2 (6%) 

0 (0%) 

F 

7 (22%) 

2 (6%) 

Table  1 


It  appears  that  students  in  the  technology -enhanced,  collaboration -heavy  course  performed  better  than  those 
in  the  passive,  lecture-based  course. 

Finally,  it  was  also  discovered  that  students  enjoyed  working  on  the  assignments  in  teams.  As  one 
student  put  it,  “There  was  still  pressure  to  do  the  assignment  and  to  do  it  well,  but  pressure  on  a team  is  a 
lot  different  than  pressure  one  a single  individual.”  In  the  beginning,  there  were  grumblings  about  the  way 
teams  were  chosen,  but  those  grumblings  ceased  and  even  turned  into  positive  results.  One  student 
remarked,  “At  first,  I didn’t  like  the  team  I had  been  placed  on,  but  as  the  semester  went  on,  I developed  a 
strong  relationship  with  my  team  members — I trusted  them  to  do  their  best  and  they  trusted  me  to  do  my 
best.  When  I needed  help,  they  helped;  when  they  needed  help,  I helped.”  Yet,  as  is  normal,  there  were  a 
couple  of  dysfunctional  teams.  Usually,  the  root  of  the  dysfunction  was  related  to  external  circumstances 
and  not  issues  within  the  group. 

There  were  also  several  lessons  learned  during  this  course.  One  lesson  shows  that  while 
technology  makes  it  easier  to  collaborate,  students  still  desire  face-to-face  contact  with  the  instructor  and 
other  students.  Interestingly,  the  students  enjoyed  and  even  wanted  the  collaborative  technology  in  place — 
and  they  used  it.  However,  they  also  wanted  a high  level  of  face-to-face  collaboration.  Many  of  the  teams 
worked  out  schedules  of  when  they  would  meet  online  and  when  they  would  meet  face-to-face.  When 
meetings  were  online,  usually  every  team  member  was  “present”  at  the  meeting;  face-to-face  meetings 
were  not  as  well  attended,  but  the  quality  of  the  meeting  at  the  face-to-face  meetings  was  often  better  than 
those  conducted  online.  According  to  those  who  reported  this  fact  thought  it  was  due  to  the  level  of  focus 
or  distraction:  there  tended  to  be  a greater  focus  at  the  face-to-face  meetings  and  more  distraction  at  the 
online  meetings. 

In  terms  of  the  collaborative  technology  provided  for  the  students  (e.g.,  Microsoft  NetMeeting), 
the  students  tended  to  find  their  own  technology  for  collaboration.  By  far  the  most  popular  was  instant 
messenger  software  that  the  students  obtained  on  their  own.  A close  second,  however,  were  the 
collaboration  tools  built  into  the  Blackboard  course  management  system. 

Implications  of  this  work  fo  r Oklahoma  Christian  University 


This  course  provides  faculty  with  one  possible  teaching  model  that  integrates  technology  and 
pedagogy.  This  model  is  based  on  current  pedagogical  research.  Moreover,  the  model  serves  as  a stimulus 
for  conversations  regarding  best  teaching  practices  in  classroom  instruction  at  the  college  level. 

The  course  also  provides  a model  for  teaching  a course  via  distance  education.  As  mentioned,  the 
model  course  was  designed  in  such  a way  that  it  could  be  taught  in  a face-to-face  setting  or  a distance 
education  setting. 

Implications  for  Other  Institutions 


We  would  like  to  encourage  other  institutions  of  higher  learning  to  have  conversations  about  best 
pedagogical  practices  and  how  technology  can  facilitate  learning  in  classroom  settings. 
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Abstract:  Qualitative  data  collected  from  24  lead  faculty  members  at  a regional 
comprehensive  university  were  analyzed  to  discover  barriers  and  incentives  to  the  adoption  of 
educational  technology.  Leading  barriers  to  adoption  of  educational  technology  are;  time, 
resources,  and  lack  of  confidence  in  the  benefit  of  educational  technology.  Incentives  that 
were  mentioned  included;  because  it’s  the  right  thing  to  do,  personal  satisfaction,  and  student 
demand.  The  effect  of  mentor  role  models,  modeled  use  of  educational  technology,  and  other 
exposure  to  educational  technology  methodology  is  also  discussed. 


Introduction 

One  of  the  challenges  for  implementing  technology -based  learning  on  college  campuses  is  providing  support  for 
instructors  who  are  already  motivated  to  engage  in  new  approaches  to  teaching  and  learning.  Another  challenge 
is  to  provide  the  initial  motivation  that  will  convince  faculty  that  the  time  and  effort  required  by  these  new 
approaches  is  justified.  The  University  of  Southern  Coloradol  in  Pueblo,  Colorado  is  a regional  comprehensive 
institution  that  has  begun  to  address  both  of  these  challenges  with  an  innovative  program  which  supports  faculty 
who  are  early  adopters  with  the  time  and  resources  necessary  to  develop  showcase  projects.  These  projects  in 
turn  will  be  used  to  attract  and  recruit  other  faculty  within  their  colleges,  hopefully  inspiring  them  to  take  the 
plunge  into  the  world  of  technology-enhanced  teaching  and  learning. 

The  use  of  educational  technology  on  the  USC  campus  has  been,  until  recently,  determined  on  an  individual 
basis,  or  departmentally.  In  2000,  Blackboard  course  management  software  was  installed  on  campus.  In  the 
same  year,  the  University  developed  a strategic  plan  that  recognized  the  importance  of  educational  technology 
to  teaching  and  learning.  And  most  recently,  the  faculty  handbook  was  modified  to  include  appropriate  use  of 
educational  technology  as  a criterion  on  which  faculty  performance  is  evaluated. 

A national  survey  found  that  almost  47%  of  classes  were  taught  utilizing  web  page  supplementation  nationwide 
at  public  universities  (Green,  2001).  Currently,  approximately  33%  of  the  University  of  Southern  Colorado’s 
classes  employ  web  pages  as  adjunct  information  for  student  use.  At  55%  of  participating  universities,  including 
USC,  on-line  course  management  tools,  e.g..  Blackboard,  WebCT,  or  e-education,  are  provided.  On  other 
campuses  with  course  management  capabilities,  22%  of  classes  were  offered  using  these  tools,  whereas,  at  USC 


[1]  The  University  of  Southern  Colorado  is  a member  of  the  Colorado  State  University  System.  During  the 
2001-2002  academic  year,  the  University  enrolled  4,000  students  in  twenty-nine  undergraduate  and  six  graduate 
programs  with  approximately  500  full-time  faculty  and  staff.  The  University  has  been  an  Hispanic  Serving 
Institution  since  1998,  and  is  accredited  at  the  bachelor’s  and  master’s  levels  by  the  Commission  on  Institutions 
of  Higher  Education  of  the  North  Central  Association  of  Colleges  and  Schools. 
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approximately  18%  of  all  courses  utilized  the  on-line  supplemental  interface.  Nationwide,  34%  of  all  faculty 
members  have  their  own  web  pages,  whereas  at  USC  less  than  20%  of  faculty  members  do  so.  USC  is  among 
those  institutions  that;  require  all  students  to  take  a mandatory  introductory  computer  class,  offers  supportive 
instructional  training  for  faculty,  and  provides  computer  access  for  every  faculty  member.  The  comparative 
results  would  suggest  that  USC  shares  the  number  one  challenge  highlighted  by  the  national  participants,  that 
“assisting  faculty  to  integrate  technology  into  instruction  is  the  single  most  important  Instructional  Technology 
issue  confronting  their  campuses  over  the  next  two  to  three  years”  (Kenneth  Green,  personal  communication, 
November,  26,  2001). 

It  is  in  this  context  that  the  University  received  a $1.6  million  Title  III  grant  from  the  U.S.  Department  of 
Education  to  create  the  Instructional  Technology  Center  (ITC) — a facility  charged  with  faculty  training  in 
education  technology  methodologies.  A major  goal  is  to  increase  faculty  use  of  educational  technology  for  the 
purpose  of  enhancing  student  performance  and  academic  success.  To  accomplish  this  goal,  the  grant  enables  the 
creation  of  an  environment  on  campus  where  faculty  leaders  or  “champions”  are  empowered — through  training, 
release  time,  and  access  to  current  technology — to  develop  discipline-specific  projects  that  will  supplement 
traditional  avenues  of  student  learning  through  the  appropriate  use  of  educational  technology.  At  the  same  time, 
these  projects  will  serve  as  incentive  models  for  other  faculty  members  in  their  departments,  colleges,  and  in  the 
university  at  large.  According  to  Morgan  (1 998),  “Using  faculty  more  proficient  with  the  technology  to  support 
their  less  knowledgeable  or  more  reluctant  peers  is  another  way  to  focus  on  teaching  rather  than  just  the 
technology.  A faculty  mentor  program,  with  release  time  or  other  special  incentives,  is  often  an  excellent  way  to 
jump-start  innovation.” 

As  part  of  this  mission,  the  ITC  conducted  a survey  in  the  autumn  of  2001  of  representative  faculty  members 
from  each  department  to  establish  a base  of  information  on  attitudes  toward  educational  technology,  current  use 
patterns,  philosophies  of  education  related  to  computer-assisted  education  and  factors  affecting  the  adoption  of 
technology  by  faculty  members. 


Theoretical  Foundation 

The  adoption  or  “diffusion  of  innovation”  curve  developed  by  Rogers  (1995)  illustrates  the  way  that  an 
innovation  is  first  adopted  by  a few  members  of  a community,  and  how,  over  time,  other  members  join  in  and 
begin  to  use  or  deploy  the  innovation.  According  to  Rogers,  an  innovation  is  a concept  or  practice  that  is 
perceived  as  new  by  an  individual,  and  diffusion  is  the  process  by  which  the  innovation  moves  through  the 
system.  Members  of  a community  who  are  more  receptive  to  the  innovation  and  who  lead  the  other  members 
are  labeled  “innovators”  and  “early  adopters.”  These  are  followed  by  the  “early  majority,”  the  “late  majority” 
and,  finally,  the  “laggards.” 

One  of  the  ways  that  innovation  spreads  through  a community  is  via  interpersonal  communication  and 
information  sharing.  Personal  contact  with  an  esteemed  colleague,  often  including  a personal  demonstration,  is 
key  to  the  diffusion  process  in  higher-education  settings  (Hutchinson  & Huberman,  1993).  Perceived  leaders 
who  adopt  an  innovation  are  particularly  instrumental  in  the  promotion  of  a new  idea  or  practice.  According  to 
one  study  of  mid-sized  to  large  research  universities,  chemistry  and  mathematics  professors  favor  different 
dissemination  approaches  depending  on  their  current  level  of  understanding  and  exposure  (Foertsch,  et.  al., 
1997).  The  same  study,  however,  indicated  that  in  most  cases  personal  contact  with  an  esteemed  colleague  was 
strongly  preferred  by  the  adoptee. 

It  has  been  posited  that  university  faculty  who  fit  the  “innovator”  model  are  self-motivated  and,  given  the 
appropriate  infrastructure,  will  integrate  instructional  technology  even  when  incentives  or  rewards  are  absent 
(Jacobsen,  1998).  With  the  sense  that  innovators  are  already  “on  board,”  proponents  of  change  have  often 
focused  on  those  in  the  early  adopter  and  early  majority  categories  because  of  a perception  that  they  hold  the 
most  promise  for  advancement  with  the  investment  of  the  limited  resources  at  hand.  In  contrast,  those 
predisposed  to  the  other  end  of  the  scale  are  oftentimes  viewed  as  a poor  investment  in  terms  of  the  time  and 
energy  that  is  required  to  move  them  towards  adoption. 

Kurt  Lewin’s  (1951)  “field  theory”  also  provides  a framework  to  analyze  incentives  (“driving  forces”)  and 
barriers  (“restraining  forces”)  that  faculty  encounter.  Identification  of  barriers  to  adoption  is  the  first  step  in  the 
process.  Once  barriers  to  change  are  identified,  it  is  possible  to  remove  the  barriers  or  give  the  faculty  the  tools 
or  resources  that  they  need  to  overcome  them. 
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Methodology 


Full-time  faculty  from  24  campus  departments  were  interviewed  to  elicit  responses  to  closed  and  open-ended 
questions  about  their  use  of  educational  technology.  For  the  purpose  of  this  study,  educational  technology  was 
defined  as,  “recent  developments  in  computer-based  technologies  (hardware  and  software)  used  to  facilitate 
learning.”  The  survey  garnered  responses  from  one  representative  within  each  department,  with  three 
exceptions  where  representatives  could  not  be  reached  and  did  not  return  telephone  calls  or  e-mail  contacts.  By 
interviewing  a population  of  24  (representing  1 1.8%)  of  the  204  full-time  faculty,  the  researchers  obtained  a 
significant  cross-section  of  opinion  which  is  believed  to  be  representative  of  general  campus  sentiment  among 
faculty. 


Results 

Faculty  interviewees  were  assigned  a category  of  educational  technology  use  based  on  their  response  to  three 
questions.  The  first  question  asked  them  to  self-evaluate  their  personal  level  of  use  on  a five-part  scale.  The 
second  question  asked  them  to  identify  the  educational  technology  that  they  used  in  their  teaching.  And  the  third 
question  asked  them  the  frequency  of  their  use  of  educational  technology.  Based  on  their  responses  to  these 
three  questions  each  of  the  24  respondents  was  assigned  to  one  of  three  categories;  below  average  (w=8), 
average  (n=\  1),  and  above  average  (n=5). 


Incentives 

Next,  respondents  were  asked  to  identify  factors  that  convinced  them  to  become  (and  remain)  users  of 
educational  technology.  Five  choices  were  offered,  along  with  the  opportunity  to  name  additional  factors.  The 
five  choices  were;  personal  satisfaction  (selected  by  58.3%  of  respondents),  support  from  administration 
(selected  by  16.6%  of  respondents),  required  by  administration  (selected  by  4.2%  of  respondents),  student 
demand  (selected  by  37.5%  of  respondents),  and,  because  it’s  the  right  thing  to  do  (selected  by  62.5%  of 
respondents]?.  Respondents  were  also  asked  to  indicate  which  of  the  factors  was  the  most  important  factor  for 
them.  The  most  common  response  was  “because  it’s  the  right  thing  to  do”  (n=6),  followed  by  “student  demand” 
(n=2).  Other  factors  mentioned  as  being  influential  in  adoption  of  the  use  of  educational  technology  are: 

• more  student  friendly* 

• efficiency  and  efficient  use  of  class  time* 

• convenience* 

• good  source  of  info  and  students  have  access  to  it* 

• it  raises  student  achievement* 

• immediacy  of  communication*,  sense  of  teamwork  with  students  and  between  students 

• I'm  hoping  that  after  I've  been  through  it  once,  that  my  updating  of  material  will  be  easier 

• I think  it's  the  right  thing  to  do  to  familiarize  students  with  resources 

• richness  and  convenience  of  resources* 

• kinds  of  material  that  can  be  accessed* 

• found  that  students  were  gaining  employable  skills,  need  for  visuals  to  provide  relief... they  are  so  visually 
oriented,  students  have  to  be  at  cutting  edge  of  technology* 

• students  learn  many  different  ways,  and  this  type  of  technology  allows  you  to  use  all  three  areas  (visual, 
auditory,  kinesthetic)  and  it  enhances  student  learning* 

• accommodating  various  student  learning  styles  and  achieving  the  desired  learning  outcome*  I think  it  made 
me  a better  teacher. 

• personal  decision* 

Note:  * indicates  factors  that  were  considered  to  be  the  “most  important”  factor  for  the  respondent’s  choice  to 
adopt  educational  technology 

When  considering  the  response  to  this  question  in  relation  to  the  faculty  member’s  designation  as  a below 
average,  average,  or  above  average  user,  it  is  interesting  to  note  that,  with  one  exception,  the  “below  average” 


[2]  Multiple  responses  were  allowed,  resulting  in  total  in  excess  of  100%. 
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users  selected  as  their  most  important  factor  either  “personal  satisfaction”  or  another  response  that  focused  on 
student  or  instructor  convenience.  In  contrast,  all  of  the  “above  average”  users  selected  either  “because  it’s  the 
right  thing  to  do”  or  a responses  that  focused  on  student  outcomes,  e.g.,  “it  enhances  student  learning.”  This 
sense  of  almost  a moral  obligation  to  provide  the  best  possible  experience  for  the  learner  is  consistent  with  the 
belief  that  educational  technology  can  accommodate  a wider  range  of  learning  styles  and  can  result  in  higher 
levels  of  student  achievement. 


Barriers 

In  an  attempt  to  better  understand  what  respondents  perceived  to  be  impediments  to  their  adoption  of 
educational  technology,  we  asked  them  to  list  three  barriers  and  the  one  that  they  considered  the  greatest  barrier. 
Because  of  the  wide  range  of  responses,  we  attempted  to  group  the  responses  into  three  main  categories;  time 
constraints,  resource  restraints,  and  questions  about  effectiveness.  Responses  that  fit  into  the  first  category  are: 

• time  to  learn  technology 

• time  to  develop,  time  required 

• too  busy* 

• planning  time* 

• time  constraints  on  prep  time* 

• time  constraints  within  class  itself 

• time  to  do  it* 

• because  it  wastes  time 

• time  to  experiment  and  discipline  to  learn  the  techniques  that  I don't  know* 

• time  to  prepare* 

• time  it  takes  to  find  out  what's  out  there,  find  out  what 's  most  effective  and  time  to  integrate  it  into  what  you 
do* 

Note:  * indicates  factors  that  were  considered  to  be  the  ‘'most  important  ” barrier  to  adoption  of  educational 
technology 

A sense  that  resources  are  not  available  to  support  development  and  implementation  of  educational  technology 
is  also  an  issue  as  indicated  by  the  following  responses: 

• cost/availability  of  equipment 

• resources 

• lack  of  knowledge 

• training/knowledge 

• access* 

• convenience  of  access* 

• cost  of  purchase 

• lack  of  hardware  and  facility* 

• cost* 

• funding 

• lack  of  access 

• shortage  of  classrooms  that  are  equipped 

• lack  of  help/facilitators 

• money  to  purchase  it* 

• there  was  no  one  to  provide  assistance  (when  I got  started)* 

Note:  * indicates  factors  that  were  considered  to  be  the  “most  important  ” barrier  to  adoption  of  educational 
technology 

There  continues  to  be  concern  that  educational  technology  does  not  deliver  on  its  promise  or  that  its  usefulness 
is  still  to  be  proven.  The  following  statements  reflect  these  views: 

• doesn't  see  how  it  always  benefits  students* 

• not  significant 

• not  so  important  to  me. . .my  current  methods  still  work 

• it  doesn't  help* 
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• part  of  the  technology  would  be  harmful  to  my  pedagogy. . .some  of  the  technologies  out  there,  e.g., 
PowerPoint,  are  used  poorly.  Something  is  lost  when  you  get  away  from  FTP  interaction 

• technology  in  general  has  not  always  been  accommodating  for  people  in  the  visual  arts. . .a  lot  of  the 
technology  is  not  image  friendly* 

• wondering  if  this  is  legitimate  and  achieving  the  desired  outcomes 

• not  convinced  of  its  usefulness* 

Note:  * indicates  factors  that  were  considered  to  be  the  **most  important  ” barrier  to  adoption  of  educational 


And  finally,  there  are  additional  barriers  that  do  not  fit  into  the  above  categories.  The  following  statements 
reflect  these  concerns: 

• apprehension 

• doesn't  always  work 

• comfort  level* 

• obsolescence 

• difficulty  of  planning 

• inertia 

• difficulty  of  use 

• copyright  problems  have  created  access  problems 

Note:  * indicates  factors  that  were  considered  to  be  the  ''most  important"  barrier  to  adoption  of  educational 
technology 


When  faculty  were  asked  if  a mentor  had  used  technology,  and  whether  that  experience  had  contributed  to  their 
use  of  technology,  those  in  the  “below  average  use”  group  generally  stated  “no”,  even  if  the  mentor  had  used 
technology.  In  the  “average  use”  group,  there  was  equal  indication  that  mentors  had  or  had  not  used  technology. 
Among  the  faculty  who  were  mentored  by  technology  users,  more  faculty  indicated  that  they  used  technology 
as  a result,  in  part,  of  the  mentor  model.  In  the  “high  use”  group,  a majority  of  respondents  enjoyed  the  tutelage 
of  mentor  users,  and  indicated  that  this  influence  was  a strong  encouragement  for  them  to  model.  Statements 
regarding  mentor  influences  included: 

® every^thing  I've  done  is  because  of  my  mentor. . . since  he  has  left  I've  backed  away  from  it  somewhat 

• my  mentor  created  positive  attitude  toward  technology 

• positive  modeling  of  use 

• that  particular  person  led  me  to  make  that  a large  part  of  my  doctoral  dissertation  and  although  the  technology 
has  changed,  it  has  had  long  lasting  effects 

• I had  two  wonderful  mentors  were  I was  before,  and  that's  why  I became  interested  in  technology. . .it  made 


Other  Influences 

Faculty  were  queried  on  their  observation  of  technology  use  in  other  venues,  such  as  at  conferences,  and  if  that 
experience  affected  their  personal  view  and  use  of  technology.  All  responders  indicated  that  they  had  seen 
technology,  such  as  PowerPoint,  employed.  However,  the  “below  average”  and  “average”  user  categories 
indicated  more  reservations  about  the  effectiveness  of  technology  as  it  was  experienced.  More  “below  average” 
users  indicated  complete  dissatisfaction  with  the  experience  than  did  “average”  users.  Also,  more  “below 
average”  users  indicated  that  they  did  not  use  this  technology,  or  used  it  with  reservations,  than  did  “average” 
users.  In  contrast  to  these  two  groups,  the  “above  average”  users  unanimously  indicated  a positive  experience, 
and  that  the  use  had  favorably  impacted  their  own  use  of  technology.  Respondents  offered  the  following 
comments  about  viewing  the  use  of  technology  by  others,  and  its  effectiveness: 

• Sometimes  it  was  good,  but  talking  heads  w/slides  are  also  present  so  quality  may  be  questionable 

• it  made  me  more  committed  not  to  use  it 

• limited  helpfulness 

• some  are  and  some  are  not. . .I've  really  grown  to  dislike  PowerPoint. 
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all  the  difference  for  me 
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• If  person  is  not  organized,  presentation  can  reflect  this 

• as  long  as  the  technology  worked 

• inspired  me 

• I've  gotten  ideas  from  people.,  .usually  through  discussion  groups  or  looking  at  their  web  sites 

• generally  not. . .I've  run  into  dull  PowerPoint  presentations 

• that  is  where  T get  ideas  about  what  to  try.  I belong  to  the  ASEE  (American  Society  of  Engineering 
Educators)  and  they  talk  a lot  about  instructional  topics 

Faculty  exposure  to  articles  and  on-line  sources  of  information  related  to  technology  in  education  was  also 
explored.  Faculty  were  asked  if  they  had  viewed  such  content,  and  if  so,  how  had  it  influenced  their  use  of 
technology.  Most  respondents  indicated  that  they  had  read  about  technology  and  issues  related  to  its  use,  and 
had  found  the  information  helpful.  Most  faculty  responded  favorably  when  asked  about  the  positive  model  that 
such  material  presented.  Comments  included: 

• I refer  students  to  tech  resources  that  I find 

• it  opens  up  so  many  sources  and  resources  for  information,  e.g.,  on  the  Internet 

• it  puzzles  me 

• it  has  made  me  see  the  value  and  importance  of  it 

• those  discussions  are  ongoing  in  our  field... effective  vs.  non-effective  uses  for  my  discipline.  One  thing  that 
you  get  from  these  public  discussions  is  a sense  of  the  value  of  the  choices 

• provided  ideas  for  newsletter,  forms 

e hearing  other  faculty  present  ideas 

• distance  learning  techniques  (Chronicle  of  Higher  Ed) 

• it  influenced  me  to  a part  that  it  has  convinced  me  that  it  is  a very  important  tool. . .even  though  I cannot  keep 
up  with  the  innovation 

• reference  again  to  papers  at  conferences.  I like  to  see  how  others  do  things  and  then  incorporate  them  in  my 
classes. 

• garnered  some  info,  but  not  very  persuasive 

• ideas  on  what  it  does  and  how  it  works 

• minimally,  articles  on  teaching  & pedagogy,  not  major  input 

• journals  like  Syllabus  and  others  give  me  ideas  about  hardware  and  ideas  about  use,  e.g.,  using  handheld 
devices  for  education 

• not  really 

• reading  those  sorts  of  things  gives  you  an  idea  of  what’s  going  on  in  the  rest  of  the  country.  Being  involved  in 
TLTR , Steve  Gilbert,  etc.  they  encourage  one  to  use  technology  when  it  is  appropriate,  because  it  will  add 
value  if  used  appropriately. 

Participants  were  also  asked  their  opinion  about  the  level  of  educational  technology  use  and  implementation  on 
the  use  campus  as  compared  to  other  institutions  of  comparable  size  and  mission.  There  was  clearly  a 
correlation  between  users’  own  use  of  educational  technology  and  their  perception  of  wheie  the  university  in 
general  was  in  relation  to  peer  institutions.  Using  a scale  of  1-3,  with  1 being  “behind”,  2 being  “equal  to”,  and 
3 being  “ahead”,  faculty  members  in  the  “below  average”  user  group  thought  that  USC  was  ahead  of  peer 
institutions  (mean  = 2.67),  while  faculty  in  the  “average”  group  thought  that  USC  was  “average”  as  compared 
to  similar  institutions  (mean  = 1.94),  and  faculty  in  the  “above  average”  user  group  believed  USC  to  be  behind 
similar  institutions  in  terms  of  use  and  implementation  of  educational  technology. 


Discussion 

University  faculty  members  enjoy  a high  degree  of  autonomy  and  self-determination.  Change  in  behavior  is 
seldom  achieved  by  the  use  of  top-down  approaches  (Noblitt,  1997).  Rather,  change  is  usually  the  result  of  the 
removal  of  barriers  and  the  presence  of  incentives.  Identification  of  barriers  is  one  possible  starting  point  for 
adoption  and  diffusion  research.  Interviews  with  faculty  leaders  on  our  campus  suggest  that  the  leading  barriers 
to  adoption  of  educational  technology  are;  time,  resources,  and  lack  of  confidence  in  the  benefit  of  educational 
technology.  Our  grant  allows  us  to  directly  address  the  first  barrier  by  providing  6 credit-hours  of  release  time 
and  a summer  stipend  for  faculty  who  successfully  propose  a project  that  uses  educational  technology  to 
enhance  teaching  and  learning.  The  second  barrier,  perceived  limitations  of  resources,  is  one  that  the  University 
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has  been  addressing  and  continues  to  address  using  a variety  of  means.  Campus  computing  infrastructure  is 
currently  being  enhanced  both  in  terms  of  functionality  and  performance.  The  decision  to  purchase  Blackboard 
as  our  course  management  software  has  resulted  in  a growth  from  zero  to  165  courses  that  utilize  Blackboard  in 
just  the  first  year  of  use.  A student-supported  technology  fee  generates  approximately  $220,000  each  year  for 
technology  infrastructure  that  directly  affects  students.  All  of  these  efforts  are  attempts  to  address  concerns  by 
faculty  that  their  working  environment  is  supportive  of  their  efforts  to  acquire,  create  and  deploy  technology- 
enhanced  resources.  Perhaps  the  last  barrier,  faculty  members’  concerns  about  the  effectiveness  of  educational 
technology,  is  the  one  area  where  much  of  the  work  remains  to  be  done.  This  is  an  area  where  effective 
modeling  by  peers  and  respected  colleagues  shows  promise  for  attitude  changes  that  can  lead  to  changes  in 
behavior.  Jacobsen  (1998)  found  that  faculty  members  prefer  to  receive  help  and  instruction  from  colleagues  on 
campus.  While  responses  to  our  question  suggests  that  demonstrations  of  successful  use  of  educational 
technology  by  a respected  peer  early  in  one’s  career  is  a positive  influence,  it  may  also  be  that  those  who  are 
early  adopters  of  educational  technology  are  more  likely  to  recall  such  experiences.  Other  types  of  exposure  to 
educational  technology  seem  to  send  mixed  messages.  Exposure  to  use  of  educational  technology  in  external 
settings  and  in  academic  and  scholarly  publications  was  as  often  as  not  seen  as  a disincentive.  Dull  or  poorly 
executed  PowerPoint-enhanced  presentations  were  noted  by  several  respondents. 

And  finally,  the  presence  of  incentives  for  faculty  to  adopt  and  deploy  educational  technology  is  an  area  where 
both  resources  and  faculty  development  play  a role.  As  mentioned  earlier,  stipends  are  being  offered  to  faculty 
whose  projects  are  supported  by  the  Instructional  Technology  Center  on  campus.  Other  faculty  mentioned  the 
convenience  and  efficiency  that  educational  technology  affords.  But  more  often  than  not,  the  incentives  that 
respondents  mentioned  were  the  benefits  that  they  believed  were  accrued  by  their  students.  The  belief  that 
technology -enhanced  instruction  supported  a wider  variety  of  learning  styles,  and  that  students  were  learning 
both  course  content  and  picking  up  employable  skills  at  the  same  time,  were  cited  as  incentives.  While  this 
study  is  exploratory  in  nature,  it  appears  to  support  the  idea  that  adoption  of  innovation,  in  this  case  the  decision 
to  employ  educational  technology,  is  multi-faceted  and  efforts  to  increase  participation  by  faculty  members 
must  take  an  equally  multi-faceted  approach  if  it  is  to  be  successful. 
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Abstract:  This  paper  describes  Amanda*,  a method  for  mediating  asynchronous  discussions 
among  distant  learners.  Amanda  is  intended  to  help  tutors  achieve  better  results  from  group 
discussions  and  improve  knowledge  transfer  among  the  participants.  The  method  consists  of 
launching  a set  of  issues  for  collective  debate  and  involving  the  participants  in  successive 
discussion  cycles  with  negligible  human  mediating  effort.  Along  the  discussion,  specific 
nicchanisiVis  search  for  polential  inter ac lions  that  might  improve  the  debate  and  propose  new 
interactions  among  the  group.  This  paper  also  presents  the  software  prototype  that  implements 
the  method,  as  well  as  some  sample  screens  ftom  tests  carried  out  in  actual  training  situations. 


Introduction 

Collaborative  learning  is  about  promoting  knowledge  transfer  among  the  apprentices  through  a series  of 
learning  interactions.  Among  these  learning  interactions  is  the  group  discussion,  a collective  process  of 
articulating  knowledge  by  means  of  argumentative  statements.  Several  works,  such  as  (Quignard,  99)  and 
(Veerman,  00)  investigate  the  role  of  argumentative  discussions  in  learning.  In  the  knowledge  creation  theory  by 
(Nonaka,  99),  group  discussions  are  characterized  as  externalization  spaces,  where  the  implicit  knowledge  (or 
mental  models)  of  each  participant  is  made  explicit  by  means  of  the  articulation  of  ideas.  In  traditional 
classrooms,  such  a dialoguing  activity  is  achieved  by  face-to-face  discussions  and  teamwork.  In  distance 
learning  environments,  asynchronous  and  collective  discussions  are  implemented  by  discussion  forums. 

The  experience  with  discussion  forums , however,  shows  they  often  fail  to  promote  group  learning.  They 
either  suffer  from  the  lack  of  participation  or  grow  two  much  to  be  followed  by  the  tutor.  The  reason  is  the 
absence  of  a method  that  articulates  the  discussion  and  promotes  group  interaction.  In  this  paper,  we  propose  a 
method  and  a system  for  mediating  discussions  among  a group  of  distant  learners  with  negligible  or  no 
interference  of  a human  tutor. 


Method  Overview 

The  AMANDA  method  is  a computational  solution  for  mediating  asynchronous  collective  discussions 
over  multiple  issues.  Initially,  a set  of  issues  (questions)  is  distributed  among  the  participants.  The  resulting 
answers  give  rise  to  successive  discussion  cycles,  during  which  the  participants  argue  over  their  peers’ 
propositions.  The  collective  discussion  is  mediated  by  a set  of  intelligent  mechanisms  that  reason  over  the 
current  discussion  and  propose  new  interactions  among  the  group.  The  participants  provide  answers  to  the 
proposed  questions  and  argue  over  their  peer’s  propositbns  by  means  of  discussion  forms  (Fig.  6). 

* Amanda  (Agent  de  Modelisation  et  ANalyse  de  Discussions  Argumentees)  is  a joint  R&D  effort  between  the  Pontifical 
University  of  Parang  Brazil  (PUCPR),  the  Technology  University  of  Compiegne  (UTC)  and  CEGOS,  France. 
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Internally,  the  interactions  among  the  group  are  organized  in  a discussion  tree  (Fig,  1),  a collection  of 
questions,  answers  and  argumentations  representing  the  current  state  of  the  discussion.  The  mediating  effort  of 
Amanda  is  focused  on  intentionally  expanding  the  discussion  tree  with  new  nodes  and  thus  creating  new 
interactions  among  the  group. 


The  Discussion  Tree 

The  discussion  tree  is  the  underlying  model  of  the  discussion  in  AMANDA,  It  is  an  extension  of  the 
argumentation  model  proposed  in  (Karacapilidis,  98)  and  the  Simari-Loui  model  presented  in  (Chesnevar,  00),  In 
AMANDA,  the  discussion  tree  represents  multi-issue,  multiparty  argumentative  discussions.  It  is  composed  of 
question  nodes  (Q -nodes),  answer  nodes  (Alt-nodes)  and  argumentation  nodes  (Arg-nodes),  as  shown  in  figure  1. 


Fig.  1:  The  discussion  tree  (an  example) 

A Q-node  represents  an  issue  to  be  debated.  It  contains  a ratural  language  question,  such  as  “Which 
types  of  connection  elements  may  exist  in  a computer  network?”,  A Q-node  can  be  classified  as  a content- 
expected  interrogative  speech  ac/ (Foray ska -Pompa,  00),  for  which  one  expects  an  answer  that  conveys  a certain 
“content”  as  response.  In  AMANDA,  a question  can  be  either  typed  by  the  tutor  or  automatically  generated  from 
the  domain  models^,  A Q-node  can  be  linked  to  a number  of  answers,  represented  by  the  Alt -nodes. 

An  Alt-node  is  an  alternative  answer  to  a given  question.  The  answer  contained  in  an  Alt- node  is  the 
“content”  expected  by  its  Q-node,  In  Karacapilidis’  model,  an  Alt -node  is  a position  over  an  issue.  Every  Alt- 
node  is  assigned  to  a given  participant,  who  is  the  author  of  the  answer.  Formally,  an  Alt -node  is  a triple 
where  a is  the  textual  content  of  the  answer,  Q is  the  related  question  and  / is  the  author  of  the  answer. 

An  Arg-node  is  a supporting/refuting  reaction  from  a given  participant  over  a position  expressed  by  his 
peer.  It  conveys  bah  an  intention  (supporting  or  refuting)  and  a proposition  (a  pro  - or  contra- argument).  In  fact, 
according  to  (Searle,  70),  an  argumentation  is  an  illocutionary  speech  act  composed  of  two  elements:  an  essential 
content  (intention  to  justify/refute  a proposition)  and  a propositional  content  (the  proposition  that  holds  the 
intention).  In  AMANDA,  the  essential  content  is  expressed  in  four  levels:  total  agreement  (Arg-H-),  partial 
agreement  (Arg+),  partial  disagreement  (Arg-  )and  total  disagreement  (Arg- - ),  while  the  propositional  content 
is  a free  text  provided  by  the  participant.  Every  Arg-node  is  assigned  to  a participant,  who  is  the  author  of  the 
argumentation.  Formally,  and  Arg-node  is  a 4-tuple  <g,  //,  w,/>,  where  gis  the  propositional  content,  H is  the 
corresponding  parent  node,w  e {+  + ,+,  - , - - } is  the  essential  content,  and  / is  the  author  of  the  argumentation. 


The  Advance  of  the  Discussion 

Amanda  advances  the  discussion  by  expanding  the  discussion  tree  with  new  nodes,  based  on  the 
current  interactions  among  the  participants.  Figure  2a  shows  an  overview  of  the  proposed  method.  In  stage  I 
{Launching),  the  set  of  issues  (Q)  is  distributed  among  the  set  of  participants  (P),  which  creates  the  first 
configuration  of  the  discussion  tree  (DT)  and  the  first  generation  of  discussion  forms  (DF).  In  stage  2 (pF 

^ Although  the  generation  of  natural  language  questions  from  domain  models  is  an  important  feature  of  Amanda,  it  will  not 
be  explored  in  further  details,  since  it  does  not  directly  relate  to  the  discussion  mediation. 
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Reception),  the  discussion  forms  are  received  from  the  participants  and  the  discussion  tree  is  updated 
accordingly.  In  stage  3 (l^ew  Cycle),  a new  discussion  cycle  is  opened,  which  corresponds  to  the  aggregation  of 
new  nodes  to  the  discussion  tree  and  the  generation  of  the  corresponding  discussion  forms.  New  discussion 
cycles  are  successively  opened  until  the  discussion  cannot  be  advanced  any  further. 

Most  of  the  reasoning  employed  in  the  method  occurs  in  the  New  Cycle  stage.  The  opening  of  a new 
discussion  cycle  occurs  in  five  steps,  as  illustrated  in  figure  2b.  It  starts  by  filtering  all  nodes  of  the  discussion 
tree  that  are  not  worth  re-launching,  i.e.  nodes  with  empty  textual  content,  highly  supported  nodes  or  nodes 
belonging  to  depth  levels  above  a certain  threshold.  The  Filtering  step  produces  a set  of  re-launchable  nodes  (F). 

In  the  next  step  {Extraction  & Ordering),  each  node  in  F is  assigned  a re-launch  score  (RS).  The  RS 
reflects  the  importance  of  a node  for  the  discussion,  being  expressed  as  the  average  among  four  attributes:  (i)  the 
agreement  level  of  the  node;  (ii)  the  depth  level  of  the  node;  (iii)  the  degree  of  support/refutation  of  the  node  in 
relation  to  its  parent  and  (iv)  the  percentage  of  participants  covered  by  the  node.  The  higher  the  RS  of  a node,  the 
more  likely  it  is  that  this  node  will  contribute  to  the  advance  of  the  discussion.  The  output  of  this  step  is  an 
ordered  list  of  nodes  (O)  sorted  according  to  the  RS  values. 


Fig.  2 : Overview  of  the  AMANDA  method 

The  Assignment  stQp  analyses  each  node  in  O to  search  for  potential  interactions  that  might  advance  the 
discussion.  To  handle  this  heuristic  and  multiple-criteria  procedure,  we  propose  the  use  of  independent 
assignment  mechanisms  (AM),  which  attempt  to  find  coherent  matching  relations  between  the  set  of  nodes  and 
the  set  of  p articipants.  Each  mechanism  applies  its  own  assignment  rules  to  find  the  most  suitable  participant  to 
work  on  a given  node  of  the  discussion  tree.  If  a given  participant  1 is  considered  ‘suitable*  to  work  on  a given 
node  N e O,  then  N is  re-launched  (an  ‘empty*  child- node  N*  is  created)  and  N*  is  assigned  to  1.  The  pair  <N,  1>, 
resulting  from  this  matching,  is  called  an  assignment . Five  assignment  mechanisms  are  currently  proposed  (EXT, 
REPLY,  Buddy,  Equal  and  VLD-ATCIQ,  each  one  attempting  to  address  specific  objectives  in  the  discussion. 

The  Ext  mechanism  proposes  assignments  to  assure  that  all  issues  (Q -nodes)  are  answered  by  all  the 
participants.  The  REPLY  mechanism  assures  the  right  of  response  to  every  participant  whose  proposition  was 
refuted  in  prevbus  cycles.  The  BUDDY  mechanism  allows  the  participants  to  argue  over  answers  on  issues  they 
have  already  answered.  The  EQUAL  mechanism  aims  at  spreading  the  participants  over  the  discussion,  so  that 
every  argumentation  is  equally  argued  among  the  group.  Finally,  the  VLD-ATCFC  mechanism  assures  that  all 
refuting  argumentations  are  validated  by  the  mediators/tutors  (if  any). 

Each  assignment  mechanism  proposes  its  own  set  of  assignments.  However,  not  all  proposed 
assignments  (PA)  can  be  aggregated  to  the  discussion  tree.  We  must  respect  the  maximum  workload  per 
participant  and  avoid  undesirable  assignments.  This  is  done  by  the  Assignment  arbitration  step,  which  filters  the 
proposed  assignments  (PA)  to  produce  the  set  of  final  assignments  (FA). 
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The  assignments  contained  in  FA  are  finally  turned  into  new  nodes  and  aggregated  to  the  discussion 
tree  by  the  Re-launch  step.  This  is  achieved  by  re-launching  each  assignment  <N,  I>  in  FA  and  assigning  each 
new  ‘empty’  node  N’  to  the  corresponding  participant  I.  In  addition,  the  next  generation  of  discussion  forms 
(DF)  is  made  available  to  the  participants  and  the  method  goes  back  to  stage  2 {DF  Reception). 


Method  Implementation 

The  Amanda  method  described  above  was  implemented  and  tested  in  various  training  situations.  The 
corresponding  system  (Fig.  3)  is  composed  of  a central  Coordination  ModuleX\i^X  implements  the  flowchart  of 
figure  2.  The  system  has  two  user  interfaces;  a local  interface  that  allows  full  control  over  the  discussion  and  a 
learner  interface  that  generates  the  discussion  forms  via  the  Internet.  In  addition,  the  system  is  provided  with  a 
Knowledge  Base  (KB)  module  that  allows  the  construction  of  the  domain  models  (ontologies  and  task  models) 
and  a natural  language  generator  (f!L  Generator),  which  generates  questions  based  on  the  domain  models. 


GUI 

HTML 

forms 


Fig.  3:  System  overview 


System  interfaces 


The  local  interface  (Fig.  4)  enables  all  actions  on  the  discussion,  such  as:  creating  a new  discussion, 
adding  participants/questions,  viewing/editing  the  discussion  tree,  updating  the  discussion  (pF  Reception), 
inspecting  parameters  and  opening  discussion  cycles.  For  research  purposes,  it  also  allows  to  simulate  discussion 
scenarios  with  the  aim  of  validating  the  method  in  a wide  range  of  situations. 
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COWf  AVAEABIiS 


From  the  local  interface,  we  can  advance  the  discussion  to  the  next  cycle  by  clicking  on  the  New  cycle 
button.  This  calls  the  New  Cycle  interface  (Fig.  5).  On  this  interface,  the  assignment  mechanisms  can  be 
individually  triggered  and  the  proposed  assignments  appear  in  the  “To  be  assigned  to”  column  beside  the  node  to 
be  re-launched.  The  Adjust  button  serves  to  filter  undesirable  assignments  and  to  limit  the  number  of 
assignments  according  to  the  maximum  workload  per  participant  (see  Assignment  arbitration).  The  Apply  button 
finally  opens  the  new  cycle,  by  aggregating  the  new  nodes  to  the  discussion  tree  {Re-launch  step). 
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Fig.  5:  The  New  Cycle  interface 


As  a new  discussion  cycle  is  opened,  a new  generation  of  discussion  forms  is  made  available  to  the 
participants.  The  discussion  form  for  a given  participant  is  built  by  grouping  all  ‘just- re  launched’  nodes  assigned 
to  him  and  formatting  them  appropriately,  so  that  they  can  be  answered  and  sent  back  to  the  ystem.  Figure  6 
shows  a real  discussion  form  in  HTML  format,  as  it  appears  through  the  Internet. 
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Fig.  6:  The  learner  interface  (HTML  discussion  forms) 

The  discussion  forms  are  answered  by  the  respective  participants  and  are  sent  back  to  be  processed  by 
the  system  iPF  Reception ).  The  answers  and  argumentations  contained  in  the  discussion  forms  are  used  to  fill 
the  corresponding  ‘empty’  nodes  created  by  the  latest  re-launch  and  the  discussion  cycle  is  terminated.  At  this 
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moment,  a new  cycle  can  be  opened  and  new  interactions  among  the  group  can  be  proposed  based  on  the  latest 
inputs  from  the  participants. 

Conclusion 

In  this  paper  we  proposed  a computational  method  for  mediating  group  discussions,  with  the  objective 
of  relieving  the  tutor  from  the  time-consuming  task  of  articulating  the  discussion  and  improving  the  knowledge 
transfer  among  the  group.  This  method  was  fully  implemented  and  has  been  applied  in  actual  distance  training 
situations,  where  it  has  been  the  test-bed  for  various  algorithms  and  coordination  strategies. 

The  results  obtained  in  the  field  so  far  are  promising.  They  show  that  AmanDA  enables  to  conduct 
collective  discussions  among  large  groups  of  participants  over  a large  set  of  issues  with  negligible  or  no  human 
interference.  It  is  capable  to  autonomously  articulate  a discussion  and  find  interesting  patterns  of  interaction 
among  the  group.  This  yields  in  a ‘disciplined’  discussion,  where  the  discussants  are  given  equal  chances  to 
answer  the  issues,  defend  their  refuted  viewpoints  and  participate  equally  throughout  the  discussion.  Another 
effect  of  this  articulation  is  that  the  debate  is  progressively  focused  on  polemical  propositions  rather  than  on 
common  agreements,  yielding  in  a better  use  of  the  effort  employed  by  the  participants  and  improving  their 
motivation. 

In  addition  to  the  discussion  mediation,  we  also  conducted  experiences  on  domain  modeling  and  NL 
generation.  We’ve  had  surprisingly  positive  results  on  generating  natural  language  questions  from  domain 
models  (ontologies  and  task  models)  and  using  the  resulting  questions  as  discussion  elements.  We  noticed  that 
system- generated  questions  can  be  even  ‘better’  than  those  proposed  by  human  tutors,  specially  when  the 
domain  models  are  well  constructed. 

Although  it  is  still  unclear  how  well  the  proposed  method  performs  in  comparison  with  a discussion 
mediated  by  a human  tutor,  we  have  reasons  to  believe  that  AMANDA  can  be  a valuable  aid  for  distant  learning 
and  a start  point  for  further  research. 
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Abstract:  An  application  specific  style  guide  was  produced  for  a large  web-based  multi- 
media and  hyper-media  learning  project.  It  was  introduced  to  15  different  sets  of  developers 
at  Universities  in  disparate  locations  to  promote  consistency  and  good  HCI  practice  for  the 
whole  project.  This  paper  discusses  the  reactions  to  the  use  and  practicality  of  the  style  guide 
through  the  findings  of  a survey  undertaken  one  year  later.  In  general,  many  negative  and 
neutral  attitudes  towards  the  style  guide  were  recorded.  However,  specific  trends  were 
observed  due  to  exposure  and  use  of  the  style  guide,  the  background  and  experience  of  the 
developers  and  task  involvement  of  developers.  The  paper  discusses  the  understanding  of  HCI 
in  the  project  and  the  need  to  encourage  developers  to  use  a style  guide  in  multi-media  and 
hypermedia  learning  projects. 


Introduction 

During  the  development  of  multi-media  and  hyper-media  based  learning  material,  there  are  often  many 
Human-Computer  Interaction  (HCI)  issues  that  arise.  Developers  therefore  need  to  pay  attention  to  good  HCI 
practice  when  developing  learning  material.  This  is  particularly  important  when  an  e-leaming  project  is  large 
and  involves  many  developers,  since  it  is  often  impractical  for  HCI  checks  to  be  made  on  every  piece  of 
learning  material  produced.  One  way  to  help  achieve  consistency  in  interface  design  is  the  use  of  a project- 
wide style  guide.  The  use  of  a style  guide  can  positively  influence  projects  that  are  geographically  dispersed, 
even,  as  in  this  case,  when  there  is  only  a small  group  of  Human  Factors  experts  working  on  the  software 
development  project  (Colbert,  1997). 

Style  guides,  in  one  form  or  another,  are  used  extensively  in  many  industrial  projects  that  use  human- 
machine  interfaces.  They  are  also  common  place  in  many  large  web-site  developments.  However,  the  use  of 
style  guides  in  e-leaming  projects  has  not  been  widely  documented.  Universities  and  colleges  often  maintain  a 
corporate  style  guide  for  their  externally  focused  informational  web  sites,  yet  there  are  few  examples  of  Higher 
Educational  organisations  using  style  guides  during  the  design  of  user  interfaces  for  computer-based  learning 
materials.  In  places  where  they  are  used,  it  is  normally  in  the  form  of  look  and  feel  or  corporate  guides,  rather 
than  helping  the  developer  with  task  specific  application  style  guides. 

Much  of  the  research  in  the  area  of  styles  guides  that  has  been  done  to-date  has  been  in  commercial 
and  industrial  settings.  There  have  been  accounts  from  a number  of  authors  on  the  benefits  and  problems  of 
using  style  guides  (Colbert,  1997;  Hill,  1995;  Kostelnick,  1998;  Simpson,  1999;  and  Washington,  1993).  The 
reported  benefits  of  style  guides  are  often  more  about  the  outcomes  of  applying  good  HCI  practice,  rather  than 
empirically  studying  the  implementation  of  the  style  guide.  Those  that  have  studied  the  development  and  use  of 
style  guides  report  that  developers  benefit  from  having  more  specific  advice  at  hand  (Simpson,  1999),  that 
guidelines  can  be  distributed  quite  cheaply  and  can  be  practical  without  the  need  for  a HCI  specialist  and  that 
user  testing  may  not  be  necessary  in  aspects  of  the  interface  that  are  covered  by  the  style  guide  (Hill,  1995). 
Accounts  of  specific  problems  with  the  implementation  and  use  of  style  guides  are  more  easy  to  pin-point.  A 
re-occurring  theme  is  that  of  the  difficulty  in  managing  the  style  guide.  The  process  for  managing  the 
development,  implementation,  use  and  revision  of  a style  guide  is  not  well  defined  (Simpson,  1999). 
Washington  has  also  reported  that  the  “requirements  analysis”  and  “promotion”  of  the  style  guide  are  often 
ignored  or  overlooked  which  often  has  detrimental  affects.  Hill  identifies  that  guidelines  can  (1)  be  difficult  to 
interpret;  (2)  often  give  contradictory  advice  and  therefore  need  to  be  interpreted  in  context;  and  (3)  often 
consider  theoretical  problems  which,  in  practice,  are  not  experienced  often.  Similarly,  Kosteinick  reported  that 
developers  can  become  frustrated  deciding  which  style  to  adopt,  particularly  in  the  event  that  two  or  more  styles 
conflict. 
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Use  of  a Style  Guide  in  a Large  Web-Based  Educational  Project 

The  authors  of  this  paper  were  brought  in  to  advise  on  the  Human-Computer  Interaction  (HCI)  issues 
of  a large  web-based  educational  project.  It  was  decided  to  introduce  a style  guide  that  all  learning  material 
developers  could  use.  The  team  began  by  performing  a “requirements  analysis”.  A review  was  carried  out  of 
all  fifteen  University  partners  who  were  producing  chemistry  related  distance  learning  material.  As  was 
expected,  each  partner  was  offering  learning  material  in  a format  that  they  had  developed  independently. 
Therefore,  inconsistencies  existed  in  the  whole  look  and  feel  of  material,  in  the  organisation  of  the  content  and 
in  the  way  interaction  between  the  learner  and  the  information  was  initiated  and  completed.  More  importantly, 
there  was  an  extensive  amount  of  what  could  be  deemed  as  “bad  practice”  being  adopted  across  the  University 
partners.  Therefore,  paramount  was  the  construction  of  a style  guide  that  would  address  the  variations  and 
contradictory  signals  being  sent  to  the  learner  and  to  instill  “good  practice”  principles  in  the  developers.  The 
purpose  was  to  produce  a style  guide  that  would  help  application  developers  at  the  University  partners  - so  that 
learners  could  concentrate  upon  learning  the  material,  rather  than  wrestling  with  the  interface. 

Style  Guide  Development 

A style  guide  should  advise  when  and  how  to  use  controls  and  input  methods  and  should  encourage 
consistency  and  re-use  within  the  aims  of  the  project  (Colbert,  1 997).  Therefore,  the  goal  of  our  style  guide  was 
led  by  the  need  to  provide  clear  direction  on  consistency  and  good  practice  during  the  development  of  learning 
material.  This  required  guidance  on  application  specific  HCI  design  elements  such  as  the  design  of  interactive 
exp  eni  lie  fits,  affordance  and  system  navigation,  as  w'cll  as  expected  features  such  as  text  layout,  design  of 
primitive  elements  and  the  use  of  colour.  In  all,  44  separate  elements  that  affect  the  interface  design  for 
websites  were  addressed.  Particular  features  of  the  application  style  guide  included: 

■ a section  of  readable  prose  explained  why  a style  guide  was  necessary  and  how  ensuring  a similar  look 

and  feel  throughout  the  project  would  be  of  help  to  the  learner; 

■ individual  recommendation  sections  on  each  of  44  different  HCI  design  elements; 

■ a pull-out  A3  size  “pictorial  reference”  guide  sheet  at  the  back  of  the  style  guide  that  provided 

synopses  of  guidelines,  with  references  to  the  main  sections  in  the  guide  itself; 

■ full  use  of  laser  printed  colour  copying  to  offer  the  best  clarity  possible; 

■ softcopy  production  on  CDROM  in  PDF  format  for  convenience  and  accessibility. 

In  each  of  the  44  individual  recommendations  sections  on  design  elements,  the  guide  offered 
background  information  and  a description  of  features,  together  with  a diagram,  and  then  outlined  explicit 
recommendations  or  design  guidance.  The  recommendations  were  tailored  specifically  towards  the  project  and 
the  use  of  the  design  elements  for  learning  materials.  Recommendations  were  provided  on  what  developers 
should,  and  should  not  do  with  the  item  and  indicated  the  degree  of  flexibility  in  the  use  of  the  item  in  the 
project.  Following  these  recommendations,  a set  of  examples  was  provided  for  each  section  of  the  style  guide. 
These  examples  were  screen-dumps  of  existing  practice  within  the  project  or  associated  learning  material  and 
were  accompanied  by  notes  explaining  any  good  points  and  possible  usability  issues. 

The  style  guide  was  offered  with  the  full  backing  and  consent  of  the  project  management,  the  technical 
team  and  the  educational  content  advisory  group.  It  was  introduced  to  the  project  partners  and  the  learning 
material  developers  at  a conference.  The  partners  took  hardcopy  and  softcopy  guides  away  with  them,  in 
anticipation  that  they  would  be  used  in  the  further  development  of  the  learning  material  in  the  project. 

Research  into  the  Use  of  the  Style  Guide 

During  the  following  year,  the  use  and  influence  of  the  style  guide  was  noted  whenever  visits  were 
made  to  developers.  One  project  partner  made  ftill  colour  copies  of  the  style  guide  for  each  member  of  the 
project  team,  which  showed  a commitment  (at  least  in  monetary  terms)  to  the  use  of  the  guide.  In  other  partner 
locations,  the  hardcopy  of  the  guide  was  on  the  shelf  of  the  project  leader,  apparently  unused.  Therefore,  the 
authors  of  the  application  style  guide  sought  to  find  out  the  usage  of  the  guide,  its  practicality  and  any 
recommendations  for  improvements  that  could  be  made.  At  a conference,  one  year  after  the  launch  of  the 
original  guide,  feedback  forms  were  handed  out  to  all  conference  attendees.  The  form  included  simple  answer, 
Likert-based  rating  scales  and  detailed  answer  questions,  on  a two-sided  sheet  of  A4.  Responses  were  collected 
and  collated.  The  remainder  of  this  paper  discusses  the  results  obtained  and  the  implications  both  for  this 
specific  project  and  also  for  others  seeking  to  use  a style  guide  in  educational  multimedia  developments. 
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Investigation  into  the  Effective  Use  of  the  Style  Guide 


Response  Rate 

The  response  rate  of  relevant  conference  participants  was  approximately  one  third  (n=18).  Of  those 
who  responded,  2 people  had  not  had  any  exposure  to  the  style  guide  and  did  not  answer  any  specific  rating- 
scale  questions.  A farther  4 people  only  answered  the  first  section  of  rating  questions,  relating  to  initial 
impressions  but  did  not  provide  responses  to  their  current  attitudes  towards  the  style  guide.  Therefore,  only 
around  65%  of  respondents  (n=12)  fully  answered  all  of  the  questions  on  the  feedback  questionnaire. 
Respondents  comprised  chemistry  teachers  (n=3,  all  completed  all  questions),  other  chemists  (n=8,  only  5 
completed  all  questions)  and  computer  scientists  (n=7,  only  4 completed  all  questions). 

General  Attitudes 

One  of  the  main  findings  from  this  feedback  was  that  very  few  developers  had  truly  used  or  referred  to 
the  style  guide  in  the  one  year  period.  One  third  of  respondents  answered  that  they  had  only  seen,  but  never 
used  the  style  guide.  The  remaining  two  thirds  had  used  the  guide  a few  times  during  the  course  of  the  year  but 
only  two  of  these  people  said  that  they  were  currently  using  the  style  guide  in  their  work. 

Generally,  trends  were  observed  in  the  respondent’s  use  of  the  style  guide  and  how  positive  their 
answers  were.  However,  usage  made  no  difference  to  the  responses  for  some  of  the  questions.  In  particular, 
most  respondents  provided  negative  feedback  when  asked  if  the  guide  offered  them  any  new  information  or 
improved  their  knowledge  of  interface  design  issues.  Feedback  from  the  questiorniaires  also  indicated  a lack  of 
understanding  about  the  use  of  a style  guide.  Rating  scale  responses  on  initial  impressions  were  mixed  and 
included  a lot  of  negative  attitudes  towards  the  style  guide,  even  before  it  had  been  used.  Free  text  answers 
from  respondents  suggested  that  they  believed  the  style  guide  to  be  just  something  of  interest,  rather  than  to  be 
actively  used  in  day  to  day  development  of  learning  material.  Many  believed  that  the  technical  management 
group  should  sift  through  the  recommendations  and  then  present  a standard  set  of  icons  and  buttons  and  controls 
and  should  force  all  developers  to  do  exactly  the  same  thing  with  their  material  every  time.  Feedback  also 
showed  that  they  did  not  consider  the  style  guide  useful  when  it  offered  choice.  In  general,  developers  did  not 
want  general  comments  and  ideas,  they  said  that  they  wanted  to  be  told  what  to  do  in  each  circumstance.  They 
assumed  that  this  would  be  the  only  way  to  provide  the  learner  with  a feeling  of  seamless  interchange  between 
the  learning  material  in  different  modules,  that  had  been  developed  by  different  partners  in  the  consortium. 

Respondents  were  asked  about  their  attitudes  towards  each  presentation  idea  used  in  the  style  guide. 
They  were  asked  to  rate  the  usefulness  of  the  look  and  feel  section,  the  detailed  recommendations,  the  good  and 
poor  usability  examples  and  the  A3  pictorial  reference  guide.  A nearly  equal  mix  of  positive,  negative  and 
neutral  responses  were  recorded  for  every  presentation  idea  except  the  use  of  examples,  which  had  five  times  as 
many  positive  responses  as  negative  ones.  Free  text  answers  also  offered  support  for  the  use  of  good  and  poor 
usability  examples  and  indicated  that  developers  found  these  a good  foundation  for  internal  discussions,  so  that 
mistakes  were  not  repeated  in  future.  However,  there  was  sense  of  egocentricity  in  responses.  The  project 
involved  sixteen  different  partner  groups  developing  learning  material  for  a single  learning  environment,  yet 
most  respondents  only  wanted  to  see  views  on  their  part  of  the  development.  Respondents  did  not  want  to  see 
examples  of  errors  that  others  had  made,  indicating  that  the  believed  they  would  not  learn  from  others  mistakes. 

Trends  in  Negative  and  Positive  Attitudes  Towards  the  Style  Guide 

Importantly,  the  exposure  and  use  of  the  style  guide  appeared  to  be  related  to  the  attitudes  offered  in 
the  responses  to  the  rating  scales.  The  most  negative  responses  were  from  the  one  third  of  people  who  had  not 
actually  used  the  style  guide.  Most  of  those  who  had  not  used  the  style  guide  provided  few  responses  to 
questions.  Where  there  were  responses  from  non-users,  they  were  either  negative  opinions  or  indifferent 
attitudes.  This  was  in  contrast  to  those  who  had  used  the  guide,  who  offered  many  positive  responses.  Also, 
the  most  positive  responses  came  from  the  person  who  was  currently  using  it  in  their  day  to  day  work. 

Results  showed  some  trends  in  attitudes  for  those  with  similar  backgrounds.  In  particular,  chemistry 
teachers  offered  the  most  negative  responses.  All  chemistry  teachers  provided  negative  reactions  to  the  use  of 
the  style  guide  in  helping  the  development  of  learning  materials,  solutions  to  problems  during  development  and 
the  production  of  multi-media  elements.  . They  also  said  that  it  did  not  improve  their  knowledge  of  interface 
design  or  help  improve  the  learning  material  that  they  were  developing.  In  contrast,  the  computer  scientists 
offered  much  more  favourable  responses.  They  were  positive  about  the  guide  being  clear,  initially  helpful  and 
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still  helpful  one  year  on.  They  indicated  that  it  was  helpful  in  finding  solutions  to  interface  problems,  helping 
develop  learning  materials  generally,  and  was  specifically  helpful  in  the  development  of  multi -media  elements. 

Responses  were  also  considered  in  terms  of  the  tasks  that  people  performed  and  the  computer  packages 
they  used  during  development.  Trends  in  attitudes  towards  the  helpfulness  of  the  style  guide  were  observed  for 
these  task  groups.  Figure  1 summarises  each  group’s  attitude  towards  the  6 questions  that  were  asked  involving 
the  helpfulness  of  the  style  guide  during  development. 


Figure  1:  the  attitudes  of  different  task  groups  in  response  to  6 questions  on  the  helpfulness  of  the  style 

guide  in  design  and  development. 

The  first  distinct  task  group  were  the  “all-rounders”  (n=6)  who  had  very  high  involvement  in  the 
development  of  content  and  the  design  and  who  also  used  a number  of  packages  including  editors  (HTML  or 
other  Web  enabling  package),  graphics  and  animation  or  video  packages.  These  were  all  chemists  or  chemistry 
teachers,  with  a mix  of  qualification  levels.  No  particular  pattern  was  observed  in  their  exposure  or  use  of  the 
guide  with  some  never  having  used  it,  some  using  it  a few  times  during  the  year  and  one  currently  using  it.  This 
task  group  offered  different  opinions  on  their  general  attitudes  about  the  clarity,  usefulness  and  whether  they 
were  happy  using  the  guide.  However,  they  all  provided  similar,  negative,  attitudes  towards  the  guide  in  terms 
of  helpfulness  in  finding  solutions  to  problems  and  developing  learning  material.  The  negative  responses  in 
figure  1 relate  only  to  attitudes  about  the  helpfulness  of  the  style  guide  for  design,  development  and  review  of 
resources. 

The  second  task  group  were  those  who  were  “content  driven”  (n=4).  These  people  were  involved  in 
the  development  of  text,  graphics  and  some  animation  or  video  work,  but  did  not  use  as  many  packages  as  the 
“all-rounders”.  They  were  all  chemists  and  one  was  a teacher  as  well  as  a chemist.  As  seen  in  figure  1,  the 
views  of  this  group  were  less  extreme,  although  generally  not  positive.  This  group  also  did  not  think  that  the 
style  guide  offered  them  any  new  information  and  did  not  improve  their  knowledge  of  interface  design  issues. 

The  third  task  group  identified  were  the  “technical  designers”  (n=3).  These  had  no  influence  over  the 
development  of  the  content  and  therefore  concentrated  upon  the  design  of  text,  graphics,  animations  and  video 
using  a number  of  appropriate  computer  packages.  Again,  there  was  a mix  of  exposure  and  use  of  the  style 
guide  in  this  group.  Interestingly,  this  group  provided  mostly  positive  attitudes  towards  the  style  guide.  They 
believed  that  the  style  guide  was  helpful  in  developing  learning  material  and  multimedia  elements  and  helped 
them  find  solutions  to  interface  design  problems. 

The  fourth  task  group  could  be  defined  as  “just  doing”  their  job  (n=3).  Two  of  these  group  were 
computer  scientists,  the  other  was  a chemist,  all  had  lower  level  qualifications.  They  were  not  involved  in  the 
development  of  the  content  and  were  limited  to  the  production  of  text  and  graphics.  The  only  computer 
package  used  was  an  editor.  This  group  offered  more  neutral  responses  than  anything  else  - they  were  neither 
positive  nor  negative  in  their  attitudes  towards  the  usefulness,  helpfulness  and  ease  of  use  of  the  style  guide. 
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Discussion 


The  negative  and  neutral  attitudes  recorded  in  the  feedback  questionnaire,  together  with  a high  non- 
response rate  indicate  that  this  group  of  learning  material  developers  had  many  problems  with  the  style  guide. 
The  lack  of  exposure  and  use  of  the  style  guide  emerged  as  one  possible  reason  for  the  unenthusiastic  reaction. 
Indeed,  initial  impressions  (before  the  guide  had  been  used)  were  often  not  positive.  Therefore,  as  a 
consequence  of  the  original  disapproving  attitudes  together  with  the  style  guide  not  being  employed  by  the 
developers,  the  opinions  one  year  on  were  understandably  negative.  The  following  discussion  suggests  three 
possible  reasons  why  this  situation  occurred  in  this  project.  These  are  (1)  the  lack  of  understanding  of  HCI  by 
the  developers;  (2)  the  workload  and  background  of  developers;  and  (3)  the  way  that  the  style  guide  was 
managed  in  the  project. 

The  Understanding  of  HCI  in  the  Project 

Responses  to  the  free  text  questions  in  the  feedback  questionnaire  gave  the  impression  that  there  was  a 
lack  of  understanding  on  how  to  use  a style  guide,  and  of  HCI  in  general.  Very  few  developers  had  come  across 
a style  guide  before,  which  was  reflected  in  their  search  for  exact  standards,  rather  than  wanting  a set  of 
recommendations  that  they  could  apply  in  the  right  circumstances.  Since  a style  guide  should  not  de-skill  the 
designers  but  should  allow  them  to  spend  more  time  meeting  their  main  objective  of  supporting  user  tasks 
(Browne,  1994)  our  guide  was  designed  to  offer  some  freedom  to  designers,  whilst  ensuring  that  they  did  not 
infringe  any  standard  rules  of  good  HCI  practice.  However,  the  developers  on  this  project  did  not  appear  to 
understand  this  and  often  felt  that  there  was  too  much  information  available.  The  comprehensive  nature  of  our 
style  guide  may  have,  therefore,  been  an  error,  as  producing  unnecessarily  complex  style  guides  caused 
problems  for  developers  in  the  past  (Colbert,  1997).  Yet,  the  production  of  detailed  methodological  guidance 
and  compliance  rules  for  a development  environment  for  complex  learning  material  is  not  easy.  Therefore, 
providing  more  information,  rather  than  less,  and  allowing  designers  the  freedom  to  choose  should  still  be 
considered,  before  offering  a smaller  sub-set  of  guidelines. 

There  was  also  a general  attitude  that  most  HCI  information  offered  the  developers  nothing  new.  Most 
believed  that  they  would  not  learn  anything  from  the  information  contained  in  the  style  guide  and  that  it  was  all 
“common  sense”.  Many  thought  that  past  experience  as  a teacher  and  personal  “feelings”  about  the  design 
would  ensure  that  learning  material  would  be  correct  for  all  learners.  These  attitudes  reflect  very  little  change  in 
developers  from  those  reported  by  other  authors  in  the  past  towards  acceptance  of  style  guides  (Lowgren  and 
Lauren,  1993;  Tetzlaff  and  Schwartz,  1990).  These  findings  are  particularly  discouraging,  since  the  style  guide 
was  introduced  at  a conference  of  the  developers  and  also  included  a section  explaining  why  consistency  in  the 
project  and  the  use  of  a style  guide  was  necessary.  Perhaps  the  case  for  consistency  to  help  convince  the  reader 
of  the  value  of  the  style  guide  document  (Browne,  1994)  has  not  been  presented  fully  enough  in  this  project.  In 
contrast,  a particularly  encouraging  finding  was  the  apparent  relationship  between  exposure  to  the  style  guide 
and  the  positive  feedback  received,  generally.  Therefore,  the  conviction  exists  that  if  only  developers  could  be 
encouraged  to  actually  use  the  style  guide,  they  would  find  it  a more  helpful  document. 

Workload  and  Background  of  Developers 

The  most  positive  responses  came  from  those  who  only  had  a technical  involvement  with  the  project. 
These  people  were  not  involved  in  the  production  of  the  actual  learning  material  content,  but  developed 
animations,  graphics  and  text  to  support  it.  Therefore,  task  orientation  may  be  a reason  for  the  findings.  It  is 
possible  that  “technical  designers”  are  happier  using  a style  guide  because  they  have  a better  understanding  of 
the  use  of  it,  or  have  previous  practice  and  education  in  using  such  materials.  Also,  they  understood  the 
computer  packages  used  and  had  a better  working  knowledge  of  such  applications,  so  they  have  more  time  to 
spend  thinking  about  the  design  of  the  material.  We  recorded  a number  of  feedback  answers  that  implied  that 
“all-rounders”  and  “content-driven”  people  felt  that  they  had  no  time  to  look  at  the  guide  - taking  a step  back 
and  learning  about  HCI  design  principles  and  their  application  was  not  a high  priority  for  them.  The  design  of 
the  material  was  frequently  viewed  as  one  of  the  easiest  parts  of  the  project  by  teachers  and,  as  such,  developing 
the  content  took  up  most  of  their  time  and  effort.  Many  “all-rounders”  had  to  learn  how  to  use  the  package  as 
well  as  develop  the  content  to  the  learning  material,  so  time  pressures  may  have  meant  that  they  ignored  the 
style  guide.  Our  results  have  shown  a trend  that  indicates  negative  views  of  the  style  guide  are  held  by  those 
with  many  things  to  do  in  the  development  of  multi-media  and  hyper-media  educational  resources,  whereas 
those  who  have  specific  design  problems  have  more  positive  attitudes  towards  the  guide.  This  implies  that 
more  assistance  is  required  to  help  every  developer  get  the  most  out  of  a style  guide. 
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Style  Guide  Management 

The  style  guide  was  produced  in  hardcopy  and  in  softcopy.  However,  there  was  little  regular  use  of  the 
style  guide  in  any  form.  Unfortunately,  each  partner  University  was  issued  with  only  one  hardcopy  of  the 
guide.  In  some  cases  there  were  up  to  15  different  developers  working  on  the  project  at  each  partner  University 
and  visibility  of  the  style  guide  was  certainly  less  than  we  expected.  In  one  partner  University  copies  were 
produced  for  every  developer  but  these  were  very  expensive  (136  pages,  full  colour).  In  most  cases  only  the 
one  hardcopy  existed  and  this,  more  often  than  not,  resided  in  one  place  with  the  team  leader.  PDF  file  format 
softcopies  were  not  copied  to  everyone,  or  used  as  often  as  expected,  even  though  the  project  was  being 
produced  on  the  computer  network.  We  were  not  aware  of  these  issues  until  our  one  year  feedback 
questionnaire  was  collected  and  analysed.  We  expect  different  results  if  there  were  copies  on  every  developers 
desk.  Therefore,  it  is  recommended  that  a follow-up  on  the  use  of  the  style  guide  and  how  well  the  information 
is  being  disseminated  to  all  developers  is  examined  in  e-leaming  projects. 

There  is  also  one  particular  latent  issue  in  the  management  of  the  entire  project  that  seems  to  have  had 
an  adverse  affect  upon  the  use  of  the  style  guide.  During  the  year,  the  technical  management  group 
implemented  a cascading  style  sheet  (CSS)  to  provide  consistency  in  the  colour,  font  size  and  style  of  the  text  in 
web  (HTML)  pages.  The  use  of  the  CSS  was  simply  to  ensure  that  standard  text  was  not  formatted  differently 
between  different  project  partners.  However,  the  partners  saw  the  development  and  application  of  this  style 
sheet  as  the  answer  to  all  of  their  interface  consistency  and  re-use  problems.  It  did,  of  course,  help  the  layout  of 
text  but  could  not  affect  anything  within  graphics,  animations  or  other  applets  placed  outside  of  the  standard 
HTML  code.  One  year  on,  the  pages  look  similar,  but  there  are  just  as  many  HCT  mistakes  taking  place  in  areas 
that  are  not  formatted  HTML  text.  The  implications  of  the  use  of  a CSS  system  for  web  design  is  that  those 
who  are  not  well  informed  on  HCI  issues  may  have  their  perceptions  on  the  meaning  of  “look  and  feel”  and 
“interface  consistency”  changed  detrimentally. 


Conclusions 

The  style  guide  produced  specifically  for  this  e-leaming  project  was  not  well  used  by  developers. 
Exposure  to  the  style  guide  did  prove  to  be  a positive  experience  for  those  developers  who  tried  to  use  it. 
However,  many  developers  did  not  use  the  guide  because  their  prior  understanding  of  HCI  issues,  the  use  of  a 
style  guide  and  the  meaning  of  “look  and  feel”  did  not  encourage  them.  The  background,  workload  and 
familiarity  with  the  use  of  computer  packages  also  affected  their  attitudes  towards  the  potential  helpfulness  of 
the  style  guide.  We  have  learned  that  in  order  to  succeed,  the  implementation  of  a style  guide  needs  to  be 
closely  managed  and  factors  that  affect  the  exposure  and  use  of  the  style  guide  by  the  developers  considered. 
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Abstract:  This  paper  examines  how  teachers,  as  educational  designers,  can  utilize  universal  design 
for  learning  (UDL)  concepts.  UDL  is  a comprehensive  approach  to  the  design  of  educational 
systems  that  addresses  elements  necessary  for  the  achievement  of  desired  educational  goals  and 
objectives:  elements  such  as,  equity  among  the  participants,  environmental  supports,  and  the 
coupling  between  participant  abilities  and  task  requirements.  The  essential  principles  of  UDL, 
which  work  synergistically,  are:  equitability,  ergonomic  soundness,  perceptibility,  cognitive 

soundness,  error  management,  flexibility,  and  stability/predictability.  The  UDL  principles 
presented  in  this  paper  draw  from  Enabling  Technology  Laboratory  experiences  as  well  as  the 
knowledge  and  experience  of  many  individuals,  ranging  from  educators  to  engineers.  Educational 
designers  can  systematically  apply  UDL  principles  to  create  more  efficient  and  effective 
educational  environments. 


Introduction 

Teachers  commonly  refer  to  their  preparation  work  as  “lesson  plans”.  Yet  what  they  are  really  doing  is 
design  work:  they  are  designing  educational  activities,  materials,  and  curricula.  Thus,  teachers  are  educational 
designers.  As  such,  educational  designers  have  one  simple  goal:  to  create  the  best  possible  design.  Universal  design 
for  learning  is  a conceptual  approach  that  enables  educational  designers  to  more  fully  realize  that  goal. 

Universal  design  for  learning  (UDL)  is  a comprehensive  approach  to  the  design  of  educational  systems  that 
addresses  elements  necessary  for  the  achievement  of  desired  educational  goals  and  objectives:  elements  such  as, 
equity  among  the  participants,  environmental  supports,  and  the  coupling  between  participant  abilities  and  task 
requirements.  The  application  of  UDL  principles  targets  the  educational  needs  of  all  students  while  addressing 
different  learning  styles.  Truly  every  student,  from  the  gifted  to  the  at-risk  to  the  one  with  physical  and  cognitive 
disabilities,  benefits  from  UDL.  Technology  plays  a pivotal  role  in  these  endeavors  by  enabling  increased 
adaptability  and  flexibility.  The  International  Technology  Education  Association  states  that  technology  is  “the 
diverse  collection  of  processes  and  knowledge  that  people  use  to  extend  human  abilities  and  to  satisfy  human  needs 
and  wants”  (ITEA,  2002). 

UDL  builds  on  universal  design  principles  developed  by  architects  and  engineers,  but  goes  on  to  add 
principles  gleaned  from  psychology  and  educational  research.  UDL  strives  to  create  environments,  educational 
materials,  and  activities  that  are  more  physically  and  cognitively  accessible.  Physical  accessibility  includes 
wheelchair  accessibility  and  computer  access  through  the  use  of  assistive  technology  such  as  alternative  keyboards 
or  screen  readers.  Perceptual  accessibility  includes  dealing  with  disabilities  such  as  color  blindness,  hearing  loss,  or 
visual  impairments.  While  the  distribution  of  IQs  associated  with  cognitive  disabilities  must  be  considered, 
cognitive  accessibility  is  not  limited  to  individuals  with  disabilities.  UDL  must  also  include  the  creation  of  materials 
that  support  different  learning  styles  and  strengths,  providing  students  with  multiple  ways  of  engaging  the  material, 
presenting  material,  and  reporting  results.  In  short,  UDL  should  be  regarded  as  simply goot/  design  practice. 

The  movement  toward  universal  design  has  world-wide  following.  In  the  United  States  IDEA  97  (the  1997 
amendments  to  the  Individuals  with  Disabilities  Education  Act)  is  a major  impetus  for  utilizing  universal  design 
principles  in  education,  in  that  it  mandates  a fuller  inclusion  of  individuals  with  disabilities  in  general  education 
classrooms  and  activities.  In  a recent  resolution  the  Council  of  Europe,  Committee  of  Ministers,  state  the  need  to 
integrate  people  with  disabilities  into  the  community  using  universal  design  or  “design  for  all”  principles.  Education 
was  one  of  the  specifically  mentioned  areas  of  need  (Council  of  Europe,  Committee  of  Ministers,  2001). 

The  European  Concept  for  Accessibility  Network  makes  the  case  for  equating  universal  design  with  good 
design.  In  discussing  diversity  in  the  years  2000  and  beyond  they  urge  Europeans  to  “no  longer  talk  about  the 
specific  needs  of  certain  categories  of  people,  but  talk  about  human  functioning.  We  should  look  at  every  aspect  of 
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human  functioning,  without  categorizing.  ...  Accessibility  will  loose  its  stigma  and  become  a mainstream  issue.  We 
won’t  need  terms  like  Design  for  All  or  Universal  Design  anymore.  We  will  only  refer  to  good  design  and  bad 
design”  (European  Concept  for  Accessibility  Network,  2001). 

The  Enabling  Technologies  Laboratory  (ETL)  at  Wayne  State  University  embraces  the  definition  of 
technology  as  “the  diverse  collection  of  processes  and  knowledge  that  people  use  to  extend  human  abilities  and  to 
satisfy  human  needs  and  wants”  (ITEA,  2002).  This  definition  reinforces  and  supports  the  principles  of  universal 
design  in  that  it  allows  technology  to  be  not  only  devices,  or  things,  but  also  processes,  quality  tools,  and 
organizational/planning  techniques.  Through  its  partnership  with  eight  southeastern  Michigan  Intermediate  School 
Districts,  ETL  has  worked  with  teachers  to  bring  UDL  applications  into  numerous  classrooms.  The  UDL  principles 
presented  in  this  paper  draw  from  ETL  experiences  as  well  as  the  knowledge  and  experience  of  many  individuals, 
ranging  from  educators  to  engineers. 


UDL  Principles 

The  application  of  UDL  should  yield  products  and  processes  that  are:  equitable,  ergonomically  sound, 
perceptible,  cognitively  sound,  error  managed,  flexible,  and  both  stable  and  predictable. 

Equitable:  Educational  products  and  processes  should  provide  the  same  means  of  use  for  all  users:  identical 
whenever  possible;  equivalent  when  not.  The  products  and  processes  should  avoid  segregating  or  stigmatizing  any 
users,  making  the  design  appealing  to  all  users  (Connell,  1997).  Figure  1 exemplifies  this  aspect  of  universal  design. 
If  a science  classroom  has  binoculars  available  for  field  trips,  then  at  least  one  of  them  should  provide  image 
stabilization. 


Figure  I.  An  example  of  universal  design.  The  binoculars  contain  microprocessor  technology  that 
stabilizes  the  image  for  all  users.  This  allows  a tired  person  or  a person  with  tremors  or  other  motor 
problems  to  use  the  same  device. 

Ergonomically  Sound:  The  physical  demands  of  educational  activities  must  be  within  acceptable  limits  for  a wide 
range  of  users.  All  individuals  must  be  able  to  physically  access  the  activity.  Appropriate  space  for  wheelchairs  must 
be  provided.  The  relative  placement  of  materials,  equipment,  and  users  should  be  ergonomically  sound.  Students 
should  be  able  to  easily  lift  items,  carry  objects,  turn  knobs,  press  keys,  and/or  move  computer  mice;  if  they  cannot 
easily  perform  such  ergonomic  tasks,  alternative  means  of  physical  engagement  must  be  available. 

Perceptible:  The  product,  system,  and  environment  must  effectively  communicate  necessary  information  to  the 
user,  regardless  of  ambient  conditions  or  the  user's  sensory  abilities  (Connell,  1997).  The  design  should  utilize 
redundant  modes  of  information  presentation  (e.g.,  verbal,  iconic,  pictorial,  tactile).  The  design  should  maximize 
legibility  by  providing  adequate  contrast  between  the  information  and  its  surroundings.  The  expanded  use  of  sound 
field  systems  exemplifies  this  principle.  The  teacher  wears  a cordless  microphone  and  a portable 
amplifier/transmitter  that  transmits  a signal  to  a receiver  that  then  distributes  the  signal  to  speakers  placed  around  the 
classroom.  The  sound  field  technology  started  in  Special  Education  classes,  but  research  shows  that  at  any  given 
time  a high  percentage  of  general  education  classroom  students  have  hearing  problems  due  to  colds,  flu,  allergies, 
fatigue,  and  other  temporary  conditions  (Crandell,  Smaldino,  & Flexor,  1999).  Research  also  shows  that  students  in 
sound  field  classes  perform  better  academically  than  those  not  in  such  a class  (Crandell  et  al.,  1999).  Hence,  there  is 
a growing  trend  to  install  sound  field  technology  in  all  classrooms. 
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Cognitively  Sound:  The  cognitive  demands  of  educational  activities  must  be  within  acceptable  limits  for  a wide 
range  of  users.  The  task  structure  should  be  appropriate  to  the  task  - neither  too  wide  nor  too  deep.  The  material 
must  support  different  learning  styles  and  human  intelligences  (Forrester,  2001). 

Understanding  the  pivotal  role  of  representations  is  central  to  the  application  of  UDL  principles.  The 
educational  material  must  provide  different  representations,  which  in  turn  appeal  to  different  learning  styles.  Figure 
2 shows  a concept  map  illustrating  habitats.  This  map  was  produced  by  a computer  program  that  allows  one  to  flip 
back  and  forth  between  a pictorial  representation  to  a words-only  outline  form,  thereby  supporting  different  learning 
styles. 


Figure  2.  An  example  of  a concept  map  illustrating  habitats.  This  conceptual  map  was  produced  using 
Inspiration®  a software  tool  to  help  develop  ideas  and  organize  thinking.  A menu  option  allows  the  user 
to  express  the  map  in  an  outline  form  - words  rather  than  images. 

Creating  appropriate  representations  of  thoughts  and  concepts  is  also  critical  for  reflection  and  discovering 
higher-order  relationships.  The  process  of  a learner  reflecting  and  forming  ever  more  complex  mental  models  of 
reality  is  central  to  the  constructivist’s  learning  theory  (Jonassen,  Peck,  & Wilson,  1999).  Educational  designers  use 
the  laws  of  calculus  to  formulate,  simulate,  model,  and  test  hypotheses  concerning  the  physical  behavior  of 
elements.  The  ability  to  represent  physical  concepts  such  as  force  and  electrical  activity  with  mathematics  allows  us 
to  analyze  systems  and  design  products.  Graphs,  charts,  flow-diagrams,  cause-effect  diagrams,  affinity  diagrams, 
and  conceptual  maps  are  also  examples  of  representations. 

The  representation  should  facilitate,  not  hinder,  the  reflective  process  of  the  learner.  For  example.  Figure  3 
shows  two  ways  of  representing  numerical  density  data.  The  shading  represents  the  data.  In  Row  1 there  is  no 
correspondence  between  the  numerical  density  data  and  the  visual  shading  density.  In  Row  2 the  shading  deepens  as 
the  numbers  increase;  thus,  there  is  a relationship  between  the  shading  and  the  data.  Row  2’s  shading  is  more 
intuitive  than  Row  I’s  shading  because  the  visual  shading  becomes  denser  as  the  numerical  density  data  it  represents 
increases  (Norman,  1993).  The  representation  in  Row  1 forces  one  to  think  about  and  compare  the  visual  shading  to 
the  numerical  data  values;  there  is  no  natural  mapping  between  the  two  representations.  If  the  educational  objective 
is  to  analyze  the  data  and  draw  inferences  about  some  physical  process,  then  the  extra  cognitive  processing 
introduced  by  Row  1 *s  encoding  interferes  with  learning. 


Figure  3.  Example  representations.  The  numbers  represent  some  elements  per  unit  area.  Row  1 *s 
representation  is  less  intuitive  than  Row  2’s  representation  because  in  Row  2 the  visual  density 
increases  as  the  numerical  data  density  increases. 
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Flexible:  The  Center  for  Applied  Special  Technology  (CAST)  has  formulated  three  principles  that  capture  essential 
aspects  of  UDL  in  a simple  and  straightforward  manner.  Educators  should  provide  students:  1)  multiple  means  of 
representation,  2)  multiple  means  of  expression,  and  3)  multiple  means  of  engagement  (CAST,  1999).  Multiple 
means  of  representing  concepts,  ideas,  data,  and  information  allow  people  with  different  learning  styles  or 
disabilities  to  more  readily  comprehend  the  essential  concepts  presented  by  the  representations.  Multiple  means  of 
expression  provide  people  with  a variety  of  ways  to  express  themselves  with  respect  to  communicating,  reporting, 
assessment,  and  evaluation.  Lastly,  and  in  some  respects  most  importantly,  multiple  means  of  engagement  enable 
people  with  different  interests,  learning  styles,  or  disabilities  to  be  more  readily  motivated  to  pursue  and  maintain 
engagement  with  the  material.  Motivation  is  a critical  factor  and,  in  that  respect,  providing  multiple  means  of 
engagement  is  an  essential  element  of  UDL. 

Flexibility  is  the  key  to  providing  multiple  means  of  representation,  expression,  and  engagement. 
Flexibility  can  be  achieved  in  many  ways  across  several  dimensions.  Table  1 presents  five  key  dimensions  of 
flexibility. 


Table  1.  Dimensions  of  flexibility  as  modified  from  (Collis  & Moonen,  2001) 


Dimension 

Fixed  ^ element  Flexible 

Time 

Times  for  starting  and  finishing  a course. 

Times  for  submitting  assignments  and  interacting  within  the  class. 
Tempo/pace  of  studying. 

Moments  of  assessment. 

Content 

Topics  of  the  course. 

Sequence  of  different  parts  of  a course. 

Orientation  of  the  course  (theoretical,  practical). 

Key  learning  materials  of  the  course. 

Assessment  standards  and  completion  requirements. 

Requirements 

Conditions  for  participation. 

Instructional  approach  & 
Resources 

Social  organization  of  learning  (face-to-face,  group,  individual). 

Language  to  be  used  during  the  course. 

Learning  resources:  modality,  origin  (instructor,  learners,  library,  WWW). 
Instructional  organization  of  learning  (assignments,  monitoring). 

Delivery  & Logistics 

Time  and  place  where  contact  with  instructor  and  other  students  occur. 
Methods  and  technology  for  obtaining  support  and  making  contact. 

Types  of  help,  communication  available,  technology  required. 

Location  and  technology  for  participating  in  various  aspects  of  the  course. 
Delivery  channels  for  course  information,  content,  and  communication. 

Figure  4 demonstrates  flexibility;  it  shows  a student  in  an  adjustable,  mobile,  seating  system.  When  the 
student  was  provided  with  this  seating  system,  rather  than  her  wheelchair,  her  teacher  remarked  that  she  “was  now 
part  of  the  class.”  She  could  sit  at  the  same  tables,  at  the  same  height  as  her  classmates.  The  student  was  excited 
and  her  classmates  were  also  excited  that  she  could  now  participate  in  classroom  activities  just  as  they  could.  (Also 
note  that  the  seating  system  allows  her  to  remain  in  her  seating  orthosis,  which  is  critical  for  her  ergonomic 
stability.)  Thus,  figure  4 exemplifies  the  UDL  principal  of  flexibility  because  the  special  seating  system  enables  this 
student  to  participate  in  “regular”  classroom  activities  that  were  precluded  when  she  is  in  her  wheelchair. 


Figure  4.  Use  of  an  agile  device  - a movable,  adjustable  chair  made  of  C re  form,  a pipe  and  joint 
technology  for  creating  agile  devices.  The  student  is  out  of  her  wheelchair  and  at  the  same  level  as  her 
classmates,  able  to  sit  at  the  same  tables. 


Error  Managed:  Individuals  leam  by  making  mistakes;  hence,  UDL  must  manage  errors.  Educational  activity 
associated  with  desired  learning  can  be  termed  value-added  activity.  Activity  not  associated  with  the  desired 
learning  activity  can  be  termed  non-value-added  activity.  The  non -value-added  portions  of  the  process  must  be 
error-proofed,  while  the  value-added  portion  of  the  process  must  allow  errors  to  occur  (Erlandson,  Greenwood, 
Perrin,  & Zapinski,  1997).  Figure  5 illustrates  this  idea  for  the  educational  objective  of  improving  writing  skills. 
The  writing  objective,  inside  the  heavily  outlined  box,  could  just  as  easily  be  a computer  program  for  mathematics 
or  physics  instruction. 


UDL  Concerns  Associated  with  Writing  a Report 
at  Schooi  in  a Ciassroom 


Physicel  Access: 

>key  board  si  mouse 
>dictation 
>lnteltikeys 
>Dynavox 

>switchlvisual  scanning 
>mouth  stick 
>head  pointer 
>eye  gaze 


Access  to: 

>Operating  system 
>Words  I Icons  I Color 
>Simplified  Structure 
-Overlays 
*lntellltools 
+ Dynavox 
-Directly  to  word 
processor 


>MS  Word 
>Appleworks 
>Co-Writer 
> Write  Out  Loud 


Access  to  functions  & 
features  of  the  program. 
>Built  into  the  progrem 
How  easy  to  set-up? 
*font  size 
*color 
*spelling 
^grammar 
*word  prediction 
^dictation 
>External  I Added 
*lntelHTools 
*JAWSI screen  reader 


Retrieving  & 
Delivering  the 
Document 


>Physically  get  and  deliver 

> email 

>fax 


Printing 

Access  to 

the 

Docume n t 

Printers 

How  is  the  print  function 
envoked  and  executed? 
Cable  I Infra  Redi  Wireless? 


Writing  Process; 
Learning/Creetive  • 

Act 

This  is  the  value-added 
part  of  the  process. 


Figure  5.  If  the  educational  objective  is  to  develop  writing  skills,  the  activities  contained  in  the  heavily 
outlined  box,  then  everything  outside  the  outlined  box  must  be  considered  non-vaiue-added  activity.  If 
students  experience  errors  in  setting-up  or  getting  to  the  writing  process,  they  may  be  so  frustrated  that 
they  do  not  learn  what  is  desired.  If  they  experience  errors  and  cannot  print  or  otherwise  obtain  a copy 
of  their  work  - this  might  also  diminish  the  learning  process. 


Stable  and  Predictable:  Educational  environments  need  to  be  stable  and  predictable  in  that  the  learners,  teachers, 
and  other  participants  can  expect  an  environment  that  supports  the  desired  learning.  A school  classroom  requires  an 
environment  different  from  a field  trip,  or  community  based  experience.  A metal  shop  will  have  different 
requirements  than  a library.  The  desired  educational  objective  dictates  what  is  or  is  not  acceptable. 

While  each  environment  presents  unique  requirements  for  stability  and  predictability,  a common  theme 
across  all  environments  is  the  need  to  reduce  the  inherent  variability  of  the  educational  processes  or  activities  that 
take  place  in  the  environment.  The  inherent  variability  of  an  educational  process  is  not  to  be  confused  with 
providing  variation  and  variability  of  curriculum  material  or  activities.  These  are  very  different  conditions. 

The  reduction  of  the  inherent  variability  of  a process  is  a fundamental  principle  of  universal  design  and  key 
to  the  creation  of  stable,  predictable  systems.  Every  task  and  process  naturally  has  a certain  amount  of  variability  or 
variance.  Deming  termed  this  variability  common  cause  (Deming,  1982).  Everyday  occurrences,  such  as  traffic 
volume  and  the  timing  of  traffic  signals,  contribute  to  the  time  variability  associated  with  one’s  drive  to  work.  An 
accident  or  severe  storm  is  an  exceptional  event  that  is  not  common  to  the  process.  Deming  termed  such  events 
special  causes  (Deming,  1982).  In  terms  of  educational  activities  and  the  educational  environment,  educational 
designers  need  to  plan  for  the  occurrence  of  special  cause  events  (fires,  tornados),  but  they  typically  have  no  control 
over  these  special  cause  events.  On  the  other  hand,  educational  designers  do  have  the  ability  to  reduce  common 
cause  events  and  problems  associated  with  such  variability. 

The  task  of  balancing  a broom  on  the  open  palm  of  one’s  hand,  broom  bristles  up,  provides  an  example  of 
common  cause  variability.  The  laws  of  physics  introduce  a great  deal  of  variability  and  can  make  this  task  quite 
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difficult.  An  accomplished  balancer  has  the  expertise  and  skills  necessary  to  balance  the  broom;  however,  most 
people  would  scurry  around  trying,  with  no  avail,  to  keep  the  broom  balanced.  Most  individuals  simply  do  not 
possess  the  skills  necessary  to  overcome  the  variation  inherent  to  the  process  to  successfully  perform  this  job. 

However,  if  the  process  is  redefined  by  holding  the  broom  in  two  hands,  and  introducing  some  technology, 
such  as  bracing  the  broom  handle  against  a desktop,  many  more  people  can  successfully  perform  the  task.  The 
redefined  process  and  use  of  enabling  technology,  technology  that  enables  improved  performance,  makes  the  task 
more  accessible  by  reducing  the  variation  inherent  to  the  process.  This  illustrates  a typical  strategy  for  reducing  the 
variability  associated  with  processes:  redefine  the  processes  and  incorporate  enabling  technology. 

Workplace  organization,  standardized  work  procedures,  and  the  use  of  visual  controls  all  help  reduce 
common  cause  variability.  For  example,  the  parking  place  lines  and  directional  arrows  present  in  parking  structures 
organize  the  parking  environment.  As  long  as  people  understand  and  comply  with  these  visual  controls,  the 
environment  is  largely  self-managing.  The  knowledge  built  into  the  environment  helps  everyone  perform  correctly 
and  allows  optimal  utilization  of  parking  resources.  The  self-managing  features  of  workplace  organization, 
standardization,  and  visual  controls  built  into  the  environment  reduce  the  variability  of  human  performance  and 
create  more  stable,  predictable  environments  for  everyone  (Erlandson,  Noblet,  & Phelps,  1998;  Erlandson,  2001b; 
Erlandson  et  al.,  1997). 

Likewise,  educational  designers  need  to  utilize  design  principles  that  reduce  the  common  cause  variability 
inherent  in  educational  environments.  Providing  a safe,  clean,  and  comfortable  facility  is  an  obvious  and  critical 
step.  Creating  a classroom  flow  that  minimizes  movement  around  the  room  and  hence  the  opportunity  for  students  to 
bump  into  each  other  and  cause  disciplinary  problems  is  another  technique  for  variation  reduction.  Providing 
environments  that  are  more  self-managing  is  another  step.  Figure  6 shows  a shadow  diagram  in  a shop.  This  simple 
technology  provides  significant  self-managing  features  as  well  as  error-proofing.  Everyone  knows  by  looking  at  the 
board  if  a tool  is  present.  Everyone  knows  to  put  the  tools  back  over  the  corresponding  shadow  diagram. 


Figure  6.  A shadow  board  builds  knowledge  into  the  environment 
and  creates  a more  self-managing  environment. 

Conclusions 

The  UDL  principles  of  equitability,  ergonomic  soundness,  perceptibility,  cognitive  soundness,  error 
management,  flexibility,  and  stability/predictability  work  synergistically.  For  example.  Figure  7 shows  three  sets  of 
dials.  If  the  educational  objective  is  to  conduct  an  experiment  wherein  the  dials  (representing  some  part  of  the 
experimental  setup)  must  be  in  the  proper  range  for  the  experiment  to  proceed,  then  reading  the  dials  is  a non-value 
added  activity  not  the  essential  learning  objective.  UDL  principles  would  suggest  use  of  the  Level  4 arrangement, 
rhe  experimenter  can  quickly  see  if  the  dial  readings  are  appropriate.  Not  only  are  they  within  the  marked  area,  but 
they  are  all  vertical.  This  is  a quick  pattern  recognizing  cognitive  operation,  one  that  people  are  very  good  at.  The 
Level  1 arrangement  requires  interpreting  the  three  dial  positions,  a highly  error  prone  operation  with  a large  amount 
of  inherent  variability.  The  Level  3 arrangement  is  less  error  prone  and  possesses  less  inherent  variability  but  still 
requires  more  cognitive  effort  than  Level  4.  In  this  example  the  visual  controls  build  knowledge  into  the 
environment;  thereby  facilitating  a more  self-managing  environment.  Since  the  visual  controls  work  for  everyone, 
they  are  non-stigmatizing  as  they  provide  error  proofing. 

The  systematic  application  of  UDL  principles  creates  educational  environments  that  support  competent 
participation  by  all  students.  UDL  principles  are  good  design  principles  that  support  a variety  of  learning  theories. 
Educational  designers  can  use  the  UDL  principles  to  create  more  efficient  and  effective  educational  environments, 
curricula,  materials,  and  activities  (Erlandson,  2001a;  Erlandson,  2001b). 
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Figure  7.  An  example  of  error-proofing  using  visual  control,  drawn  from  the  General  Motors  Quality  Network  Workshop  series 
the  application  of  quality  and  process  improvement  tools. 
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Abstract: 

Despite  the  recent  developments  in  the  field  of  instructional  design  and  technology,  assessing  learning  complex 
domains  remains  quite  challenging.  This  case  study  explores  the  potential  of  causal  influence  diagrams  used  by 
System  Dynamicists  for  assessing  the  progress  of  learning  in  complex  domains.  The  performance  of  eleven  novice 
instructional  designers  is  compared  with  the  improvement  in  their  causal  influence  diagram  before  and  after  the 
course.  The  results  of  this  case  study  suggest  that  the  evaluation  methodology  tesed  on  causal  influence  diagram  is 
promising  for  assessing  progress  of  learning  in  complex  domains;  however,  more  research  is  necessary  to  validate  the 
methodology. 

Introduction 

An  important  yet  little  understood  question  remains  as  how  to  facilitate  learning  (and  promote  expertise)  in 
complex  domains  (Spector  & Andersen,  2000).  Our  knowledge  about  the  facilitation  of  learning  has  significantly 
increased  in  the  past  two  decades.  Now,  we  know  quite  well  how  to  facilitate  learning  in  simple,  well-defined  domains. 
Unfortunately,  we  are  not  yet  able  to  effectively  facilitate  learning  in  domains  that  possess  highly  interrelated  dynamic 
variables  and/or  components. 

A number  of  researchers  propose  different  instructional  approaches  that  may  facilitate  learning  in  complex, 
dynamic  domains.  Situated  learning,  cognitive  apprenticeship,  and  cognitive  flexibility  theory  are  some  of  the 
perspectives  that  are  being  emphasized  in  this  regard.  On  the  other  hand,  some  researchers  also  experiment  with 
advance  learning  technologies  hoping  that  they  will  uncover  part  of  the  mystery  with  learning  and  expertise  in  complex 
domains  (Achtenhagen,  2000;  Alessi,  2000;  Spector,  2001).  The  biggest  challenge  in  this  line  of  inquiry  is  that  a 
validated  methodology  for  assessing  learning  in  complex  domains  does  not  exist. 

A review  of  System  Dynamics  literature  suggests  that  causal  influence  diagrams*  (CID)  is  a very  useful 
methodology  to  graphically  represent  complex,  dynamic  systems  (Davidsen,  1996;  Davidsen  & Spector,  1997;  Spector 
et  al.,  2001;  Sterman,  1994).  This  paper  presents  the  results  of  a case  study,  which  explores  the  potential  of  causal 
influence  diagrams  for  the  purposes  of  assessing  progress  of  learning  in  complex  domains. 

Method 

A total  of  1 1 graduate  students  at  a large  university  in  eastern  United  States,  enrolled  in  introductory  level 
instructional  design  theory  and  practice  course  participated  in  this  study.  Of  these,  63%  of  the  participants  were  female. 
All  students  were  pursuing  either  a masters  or  a certificate  degree  in  the  field  of  education.  None  of  the  students  had  any 
prior  professional  or  academic  knowledge  or  experience  in  the  domain  of  instructional  design.  Moreover,  the  students 
were  not  familiar  with  systems  thinking.  System  Dynamics,  or  causal  influence  diagrams. 

At  the  beginning  of  the  course,  the  students’  learning  preferences  were  assessed  by  VARK  questionnaire 
developed  by  Fleming  (2001).  The  results  suggest  that  63%  of  the  students  are  not  strong  in  visual  cues.  This  means  that 
they  do  not  prefer  the  depiction  of  information  in  charts,  graphs,  flow  charts,  and  all  the  symbolic  arrows,  circles, 
hierarchies  and  other  devices  for  the  purposes  of  learning  or  teaching.  The  rest  of  the  students,  as  well  as  the  instructor, 
were  multimodal  with  varying  degrees  of  preferences  in  all  of  the  visual,  oral,  read/write,  and  kinesthetic  modes. 

To  assess  the  degree  to  which  participants  would  demonstrate  an  understanding  of  the  complex  problem 
domain,  causal  influence  diagram  was  used.  A causal  influence  diagram  can  be  viewed  as  a kind  of  concept  mapping 
technique  tailored  especially  for  dynamic  and  complex  domains.  It  visually  represents  the  dynamic  influences  and 
interrelationships  that  exist  among  a collection  of  variables.  A causal  influence  diagram  consists  of  a collection  of  nodes 
(key  factors)  and  links.  The  nodes  maybe  entities  that  can  be  counted,  such  as  products  or  people,  or  they  might  be  rates 
or  processes  that  represent  how  things  are  changing.  An  annotated  causal  diagram  has  a description  associated  with  each 
node  to  indicate  the  specific  nature  of  that  factor.  The  links  connecting  nodes  are  directional  to  show  how  one  conceives 
of  the  causal  relationship  among  the  factors  (e.g.,  a rate  of  change  in  weight  might  have  a causal  influence  on  blood 
pressure-  the  arrow  in  this  case  would  point  from  rate-of-weight-change  to  rate-of-blood  pressure-change).  The  links 


* Some  System  Dynamicists  use  the  term  “Causal  Loop  Diagram”  instead  of  “Causal  Influence  Diagram”.  The  author  prefers  to  use 
the  latter  but  also  acknowledges  that  the  two  are  synonymous. 
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are  also  annotated  as  either  SAME  (+)  or  OPPOSITE  (-)  to  indicate  whether  a change  in  the  causal  factor  tends  to  create 
a change  in  the  same  or  opposite  direction  in  the  effected  factor. 

During  the  first  week  of  the  course,  the  students  were  asked  to  choose  an  instructional  design  problem  in  an 
area  with  which  they  are  familiar  and  submit  a short  description  of  the  context  as  well  as  the  problem.  Upon  their 
submission  of  the  problem  description,  they  received  feedback  from  the  instructor  and  revised  their  initial  description  of 
the  problem.  Afterwards,  the  students  were  asked  to  draw  out  a causal  influence  diagram  showing  the  relationships 
between  the  variables  or  concepts  related  to  the  problem  of  their  choosing.  After  the  students  submitted  their  CID,  they 
were  asked  to  reflect  on  their  experiences  with  this  initial  exercise  in  an  online  discussion  forum.  The  students  were  also 
asked  to  post  their  CID  in  the  forum  and  reflect  on  each  other’s  CIDs.  Then,  the  instructor  provided  feedback  to  each 
student. 

Throughout  the  course,  the  students  were  encouraged  to  practice  with  a number  of  different  concept  mapping 
techniques  while  working  with  the  course  materials  and  their  course  projects.  After  learning  each  instructional  design 
step,  the  students  submitted  that  part  of  their  project  assignment,  received  feedback  and  moved  on  to  the  next  step.  At 
the  end  of  the  course,  the  students  were  asked  to  submit  the  final  version  of  the  total  instructional  design  solution  for 
their  project.  Also,  at  the  end  of  the  course,  they  were  given  a different  instructional  design  problem  and  were  asked  to 
draw  a causal  influence  diagram  for  this  problem  domain. 

Results  and  Discussion 

After  their  first  encounter  with  CID,  the  students  reported  the  overwhelming  feeling  of  confusion.  They  had 
problems  while  trying  to  represent  the  domain  of  their  choice  even  though  they  felt  they  knew  the  domain  very  well. 
Also,  some  of  them  reported  they  were  not  used  to  represent  their  thinking  in  graphics;  they  were  simply  not  visual. 
These  students  were  not  also  familiar  with  any  of  the  concept  mapping  methodologies. 

While  reviewing  each  other’s  CID,  the  majority  of  the  students  thought  their  friends  had  done  a fine  job 
representing  the  domain  (even  though  they  were  not  good  representations  at  all)  so  they  assigned  high  grades  to  each 
other.  However,  as  some  students  stated  the  reason  of  these  high  grades  were  not  just  for  the  work  itself  but  also  mainly 
due  to  the  fact  that  they  knew  how  much  effort  each  put  into  that  work.  Most  of  the  students  also  expressed  their 
inability  to  judge  the  content  of  the  diagrams. 

The  results  were  analyzed  in  terms  of  the  improvement  in  their  thinking  on  the  instructional  design  problem.  In 
general,  their  causal  influence  diagrams  still  lacked  comprehensiveness  but  an  overall  improvement  was  noted. 
Afterwards,  the  students’  letter  grades  (which  excluded  CID  exercises  completely)  were  correlated  with  the 
improvement  in  their  CID.  The  results  suggest  that  causal  influence  diagrams  do  reflect  the  improvement  in  student 
learning.  Students  who  showed  improvement  in  the  way  they  think  about  instructional  design  case  as  evidenced  by  their 
project  work  also  showed  improvement  in  their  causal  influence  diagrams.  However,  one  may  argue  a better  system  for 
comparison  is  needed  in  order  to  be  able  to  better  judge  how  novice  CID  compares  to  those  of  experts.  Also,  one  may 
question  the  reasoning  behind  the  letter  grading  during  the  course  projects.  Indeed,  the  grading  method  used  was 
subjective  although  the  instructor  tried  to  be  fair  in  her  judgment.  In  fact,  it  is  difficult  to  induce  a more  objective 
assessment  tool  to  assess  learning  in  the  domain  of  instructional  design  due  to  its  complex  nature.  In  such  complex 
domains,  CID  emerges  as  a very  promising  assessment  tool  of  learning  progress  provided  that  it  is  further  researched 
and  validated  in  a variety  of  complex  domains. 
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Digital  literacy:  A new  terminology  framework  and  its  application  to  the 
design  of  meaningful  technology-based  learning  environments 


Abstract:  Digital  literacy  is  more  then  just  using  software.  It  includes  reading  instructions  from  graphical 
interfaces  (photo-visual  literacy),  utilizing  digital  reproduction  in  learning  (reproduction  literacy),  constructing 
knowledge  from  non-linear  navigation  (lateral  literacy),  and  evaluating  information  (information  literacy). 
These  literacies  serve  as  a measure  of  the  learners’  work  quality  in  digital  environments.  The  present  paper 
proposes  a terminology  framework  for  digital  literacy.  Discusaon  of  every  literacy  type  is  accompanied  by 
results  from  a study  that  examined  the  ability  of  learners  to  effectively  utilize  digital  literacy  in  educational 
contexts. 


Introduction 

The  fast  developments  in  digital  technologies  during  the  last  decades  confront  members  of  the 
technological  society  with  situations  that  require  the  utilization  of  an  ever  growing  assortment  of  cognitive 
skills , termed  ‘digital  skills’.  Examples  that  are  commonly  given  to  these  skills  include  the  ability  to  access  or 
retrieve  data,  to  use  computer  programs,  and  to  operate  digital  appliances.  Using  digital  skills  require  the  users 
to  own  a relatively  new  kind  of  literacy,  termed  ‘digital  literacy’  (Lanham,  1995;  Pool,  1997;  Inoue,  Naito  and 
Koshizuka,  1997).  However,  digtal  literacy  is  much  more  then  just  the  physical  proficiency  in  operating 
computer  programs,  as  many  studies  describe.  It  is  a special  kind  of  mindset;  a special  kind  of  thinking.  For 
example:  using  a computer  program  involves  the  user’s  ability  to  communicate  by  deciphering,  or  ‘reading’ 
messages  embedded  in  its  interface.  It  also  involves  retrieving  data  from  the  Web,  which  requires  the  user’s 
knowledge,  not  only  of  how  to  use  search  engines,  but  mainly  of  how  to  evaluate  the  retrieved  data  and 
distingiish  between  its  relevance  and  non-relevance.  It  also  requires  a smart  use  of  the  hypermedia  technology 
that  involves  lateral-associative  thinking,  which  is  very  different  from  the  traditional,  linear  reading  of  a book. 
The  digital  reproduction  (copy  and  paste)  capability  of  computers  face  learners  with  new  horizons  and  offers  a 
new  meaning  to  creativity  and  ingenuity.  All  the  above  require  the  utilization  of  a new,  very  elaborate  and 
flexible  way  of  thinking,  that  is  typical  of  digitally  literate  learners. 

This  paper  proposes  a terminology  framework,  and  describes  the  major  dimensions  of  digital  literacy.  It 
presents  some  research  findings  on  the  utilization  of  these  types  of  literacy  among  youth  and  adult  learners. 

Methodology 

In  order  to  investgate  the  various  aspects  of  digital  literacy,  three  groups  of  participants  were  selected.  The 
three  groups  were: 

• 10  high-school  students  (age  14-15) 

• 10  college  students  (age  24-28) 

• 10  adults  (age  30-40) 

All  participants  were  given  assignments  that  required  performing  the  ability  to  use  different  kinds  of  digital 
literacy:  each  assignment  designed  to  address  a different  literacy.  Details  on  each  assignment  are  given  below. 
The  participants  worked  on  the  assignments  individually.  Each  age- group  worked  separately,  but  at  the  same 
time.  The  assignments  required  the  participants  to  perform  work  with  computer  programs.  The  participants  were 
observed  as  they  worked  on  the  assignments,  and  notes  were  taken  regarding  the  nature  of  their  work  while 
performing  the  tasks.  The  outcomes  of  the  assignments,  as  well  as  the  documentation  of  the  processes  of  each 
participant’s  work,  were  used  to  assess  and  comment  on  their  digital  literacy.  At  the  moment,  the  research 
findings  are  not  yet  completely  analyzed,  and,  therefore,  only  the  clearest  and  most  prominent  results  are 
presented.  For  the  sake  of  coherence,  in  the  paper,  every  dimension  of  digital  literacy  is  presented  separately,  and 


Yoram  Eshet 

Tel  Hai  Academic  College 
Tel  Hai,  Israel 
cshct@nctvision.nct.il 


then  discussed  with  respect  to  the  relevant  research  findings,  that  are  used  to  elaborate  on  its  educational 
significance  and  utilization . 


Photo-Visual  literacy  - Learning  to  read  from  pictures 

Writing  is  a means  of  communication  using  symbols.  It  has  evolved  through  history  from  a pictorial  alphabet, 
that  used  meaningful  symbols  to  represent  words  or  letters,  and  therefore  required  a low  level  of  cognitive 
mediation,  to  an  abstract  alphabet  that  uses  “meaningless”  symbols  (letters)  and  therefore  requires  a higher  level 
of  cognitive  mediation.  The  evolution  of  computer  interfaces  show  a Eciprocal  trend,  evolving  from  non- 
interactive, text-based,  hard-to  learn  batch  interfaces,  into  intuitive  picture -based,  easy-to  learn  Graphic  User 
Interfaces  (Nielsen,  1993).  The  first  interfaces  (either  in  CP/M  or  in  DOS  operating  systems)  were  operated  by 
text-based  commands  and  required  a very  high  cognitive  mediation  on  the  user’s  side  (e.g.  memorizing 
commands  and  understanding  commands*  syntax).  In  the  90*s,  under  Windows  environments,  interfaces  evolved 
into  the  icon  graphic- based,  highly- interacti/e  Graphic  User  Interfaces,  that  are  highly-intuitive  and  easy  to 
learn.  They  appeal  to  the  user’s  intuition  and  do  not  require  a high  cognitive  mediation  (Shneiderman,  1998). 
The  evolution  of  interfaces  marks  three  major  trends:  (1)  an  increase  in  the  fcvel  of  visualisation,  (2)  an 
increase  in  the  level  of  interactivity,  and  (3)  a decrease  in  the  level  of  cognitive  mediation  required  to  reach 
proficiency  in  the  use  of  the  interface.  Since  computer  interfaces  communicate  between  the  user  and  the  program 
as  they  contain  the  information  necessary  to  operate  the  program,  using  an  interface  should  be  regarded  as  a 
reading  skill  (Mullet  and  Sano,  1995).  Therefore,  the  evolution  of  interfaces  can  be  regarded  as  an  evolution  of  a 
new  kind  of  writing,  from  abstract  text -based  to  concrete  icon- based  communication.  We  suggest  that  in  order  to 
perform  well  with  this  new  means  of  communication,  one  must  have  a good  command  of  a unique  form  of 
digital  literacy  - the  ‘photo- visual  literacy’  - that  the  graphic  interfaces  have  brought  about:  a literacy  that 
requires  the  ability  to  use  visuals  (icons)  as  messages  or  “text”,  as  described  by  Snyder  (1999). 

In  order  to  examine  how  different  users  cope  with  different  types  of  interfaces,  we  ran  a usability  test  of 
two  programs  that  share  the  same  major  objective  (to  design  a stage),  but  are  designed  under  a completely 
different  approach:  One  (Stage  Struck)  has  a pure  graphical  interface:  almost  no  menus  and  no  ‘formal’,  pre- 
defined target.  It  is  the  user’s  responsibility  to  find  out  how  and  what  should  be  done  in  the  program.  The  other 
program  (fhe  Opening  Night)  has  a very  structured  and  quite  ‘expected’  design  (though  it  is  also  graphic).  It  is 
menu-driven;  menus  are  textual;  the  user’s  task  is  well  explained  and  determined  in  a text  form. 

The  present  research  results  show  that  the  Adults  group  performed  better  with  the  more  traditional 
interface  of  Openning  Night,  whereas  the  youngest  participants  performed  better  with  the  highly  intuitive  and 
graphic  interface  of  Stage  Struck.  In  addition,  the  research  participants  included  three  dyslexic  persons,  who 
were  not  fluent  readers  of  a regular  text.  It  was  interesting  to  find  that  they  scored  among  the  highest  with  the 
graphic  interface  of  Stage  Struck^  but  were  very  unsuccessful  with  the  traditional,  more  text  -based  interface  of 
The  Openning  Night. 

Our  research  has  pointed  to  another  aspect  of  photo  visual  literacy  that  involves  the  way  young  children 
learn  how  to  read  a new  language.  During  interviews  with  the  research  participants,  we  came  across,  and 
documented  two  cases  of  4-5  year  old  children  (Chilian  and  Israeli)  who  learned  to  read  and  speak  some  basics 
of  a foreign  language  (English)  from  ‘living  books’  types  of  programs  such  as  Just  Grandma  and  Me 
(http ://www.therevi ewzone.com/grandmame.htmn.  In  these  programs,  the  text  is  highlighted  phrase  by  phrase, 
simultaneousely  with  the  narration.  From  discussions  with  the  children,  and  then  with  their  parents,  we 
discovered  that  these  children  were  able  to  pick  up  the  words,  and  to  formulate  preliminary  spoken  sentences 
that  are  beyond  the  sentences  read  to  them  in  the  program.  In  doing  so,  they  were  tracking  the  highlighted 
words  , and  combined  the  visual  shape  of  each  word  with  its  narration.  This  case  seems  to  suggest  another  kind 
of  digital  literacy,  that  enables  learning  new  languages  through  perceiving  words  as  ‘pictures’,  rather  then  a 
combination  of  letters,  and  by  synthesizing  audiovisual  stimuli  (Beavis,  1999). 
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Reproduction  literacy:  The  copy  and  paste  culture  - a new  interpretation  of  creativity 
and  ingenuity 

The  invention  of  the  press  by  Gutenberg  (1455)  marked  the  first  big  leap  in  the  ability  of  humans  to 
copy,  reproduce  and  distribute  information  in  a large  scale.  The  second  big  leap  was  made  possible  by  the  digital 
reproduction  capabilities  (e.g.  the  Copy  and  Paste  feature)  of  computers,  that  enabled  people  to  reproduce  or  edit 
digital  texts  or  visuals,  (Benjamin,  1994).  The  unlimited  possibilities  to  reproduce  and  distribute  digital 
information  led  to  new  interpretations  of  originality,  creativity  and  ingenuity  of  artwork  as  well  as  of  academic 
text.  These  interpretations  ask  questions  such  as  “How  far  can  one  copy  or  edit  artwork  or  text,  so  that  it  is  still 
considered  an  original  work  and  not  plagiarism?”  “What  are  the  limits  of  creativity?  When  does  it  become  a 
technical  act  of  reproduction?”  A classical  case  that  pushed  the  limits  of  meaning  ful  art  in  the  reproduction  era 
was  the  case  of  Drako  Maver  (1998)  fhttp://www. kapelica.org/maver/main.htm).  Maver  was  a non-existent 
person  who  was  invented  as  an  Internet  artist,  and  whose  works  were  a collage  of  pictures  of  corpses  copied 
from  the  Web  and  edited.  This  faked  artist  was  invented  and  introduced  by  a group  of  Italian  students  as  part  of  a 
large  -scale  effort  to  challenge  the  limits  of  art  in  the  reproduction  era.  So  successful  was  this  effort  that  Maver 
even  won  some  prestigous  awards  for  his  pioneering  Internet  art  works...  (of  course,  the  referees  were  not  aware 
of  his  non  existent  nature).  Can  these  works  be  considered  art,  real,  original,  and  creative?  Coping  with  this 
challenge  requires  a special  kind  of  digital  literacy  on  the  side  of  the  digitally-literate  user.  We  tentatively  term 
this  literacy  ‘Reproduction  Literacy’. 

In  the  present  study,  we  investigated  people’s  attitute  towards  reproduction  literacy  in  two  ways:  (1)  by 
presenting  them  with  the  the  Maver  case  and  asking  whether  they  would  consider  it  art  work  and  (2)  by  giving 
them  a digital  essay,  asking  them  to  edit  it  (rearrange  words;  add/delete  text),  so  that  it  can  be  used  in  a 
completely  different  context  Results  clearly  show  the  attitude  differences  between  younger  and  older  people  The 
recognition  of  Maver ’s  work  as  a legitimate  authentic  artwork  decreased  from  75%  among  the  youngest  group 
(high -school  students)  to  39%  among  the  college  students,  and  to  only  13%  by  the  adult  group.  The  opposite  was 
found  for  the  second  task,  that  required  participants  to  reuse  text  in  order  to  create  something  original.  Here,  the 
adult  group  performed  the  best  (43%  of  them  succeeded  in  fulfilling  the  task).  Success  dropped  to  35%  with  the 
college  students,  and  to  only  19%  with  the  high-school  students.  This  finding,  of  decreasing  success  from  older 
to  younger  learners,  conforms  with  reports  of  Labbo,  Reinking  and  McKenna  (1998),  that  described  problems 
of  treating  digital  text  by  young  students  compared  with  that  of  adults. 


Lateral  literacy:  Hypermedia  and  thinking 

Besides  improving  people’s  performance  with  computer  programs,  the  hypermedia  technology 
introduced  computer  users  with  new  dimensions  of  thinking  and  new  challenges  of  digital  literacy,  that  are 
necessary  in  order  to  make  an  educated  use  of  this  elaborate  technology.  In  the  past,  the  limited  non-hypermedia- 
based  computer  environments  enhanced  a more  linear  thinking  that  was  dictated  by  the  the  non-flexible 
operating  systems,  and  the  fact  that  users  were  used  to  books,  and  expected  to  work  with  computer-based 
environments  in  the  same  way  thay  read  through  books.  From  an  educational  perspective,  the  major  importance 
of  the  hypermedia-based  environments  is  not  so  much  the  multi-tasking  work  they  allow  the  user,  but  the  ability 
to  use  these  environments  for  navigating  laterally,  in  a non-linear  way,  through  knowledge  domains.  This 
capability  enhances  a lateral,  multi-dimentional  thinking,  and  has  led  to  the  evolution  of  a new  kind  of  digital 
literacy  - ‘lateral  literacy’.  Rouet  and  Levonen  (1996)  discussed  the  impact  of  the  hypermedia  technology  on 
learning.  They  suggested  that  this  technology  helps  learners  to  move  away  from  linear  thinking  into  a rich- 
associative  lateral  thinking.  According  to  them,  in  order  to  perform  demanding  multi-level  tasks,  learners  must 
be  able  to  think  laterally  and  synthesize  knowledge  from  pieces  of  information  that  are  collected  in  different, 
sometimes  independent,  domains  of  knowledge.  Spiro,  Feltovitch,  Jacobson,  and  Coulson.  (1991)  discussed  the 
role  of  hypermedia  in  forming  a multi-dimentional  knowledge,  based  on  flexible  cognition.  Some  authors  (e.g. 
Salomon,  1996)  raised  doubts  as  to  the  ability  of  learners  to  form  meanungful  knowledge  by  surfing  in 
hypermedia  environments  as  the  Internet. 

In  order  to  examine  the  level  of  lateral  literacy  among  learners,  the  participants  of  our  research  were 
asked  to  perform  a similar  task:  To  plan  a trip  in  a foreign  country  by  using  information  from  the  Internet.  This 
task  requires  the  learners  to  utilize  lateral  literacy  in  order  to  combine  independent  pieces  of  information 
(collected  by  laterally  surfing  in  the  Internet),  in  order  to  construct  knowledge  (the  trip).  Although  analysis  of 
results  is  not  yet  completed,  findingss  seem  to  point  to  the  high  ability  of  the  youngest  participants  (high-school 
students)  to  perform  the  task.  The  weakest  performers  were  the  adults,  who  tended  to  stick  to  linear  surfing,  that 
is,  they  made  very  limited  use  of  the  hypermedia  technology,  which  led  to  a very  poor  outcome  (a  plan  of  a trip 
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to  a foreign  country).  These  findings  probably  reflect  the  extensive  exposure  of  young  people  to  hypermedia 
environments.  It  may  suggest  that  the  exposure  of  learners  to  rigid,  linear  learning  methods  harm  their  cognitive 
flexibility,  and  affect  their  ability  to  cope  with  ill-structured  problems  (Spiro  et  al.,  1991).  On  the  other  hand,  the 
research  findings  may  indicate  that  the  extensive  exposure  of  young  students  to  the  lateral,  non-linear  surfing  on 
the  Web,  improved  their  ability  to  utilize  multi-level  constructivistic  cognitive  skills. 


Information  literacy:  Trust  nobody 

Today,  with  the  exponential  growth  of  available  information,  the  consumers’  ability  to  evaluate  and 
assess  information  has  become  a key  issue  in  training  people  to  become  educated  consumers  of  information 
(Kerka,  1999;  Salomon,  2000).  Information  can  be  be  classified  into  the  following  general  groups:  (1)  primary 
objective  information  (e.g.,  all  kinds  of  census  data,  migration  routes  of  birds,  distribution  of  volcanoes,  satellite 
images,  and  weather  information).  It  is  usually  easy  to  assess  the  validity  of  this  information;  it  is  original  and 
relatively  unbiased.  (2)  primary  subjective  information  (e.g.,  much  of  the  news  in  newspapers  and  diaries).  These 
are  usually  biased  to  some  extent,  and  it  is  usually  hard  to  assess  the  quality  and  validity  of  this  type  of 
information  because  the  average  consumer  is  not  fully  aware  of  the  fact  that  the  term  ‘truth’  is  very  subjective, 
and  that  even  ‘objective’  information  (what  people  call  ‘fact’)  can  be  easily  treated  in  a biased  way  to  produce  a 
very  subjective  piece  of  information.  (3)  synthetical  information  (e.g.  journalist  articles,  scientific  hypotheses 
and  models).  This  type  of  information  is  usually  professional,  more  mature  in  its  nature,  and  usually  attracts 
relatively  educated,  sometimes  professional,  consumers,  who  are  capable  of  assessing  the  quality  of  information 
properly. 

\Vc  suggest  that  the  ability  of  infomiation  consuniers  to  make  educated,  smart,  infoimation  assessments 
requires  a special  kind  of  literacy,  termed  here  ‘Information  Literacy’.  This  literacy  acts  as  a filter:  it  identifies 
false,  irrelevant,  or  biased  information,  and  avoids  its  penetration  into  the  learner’s  cognition  (Minkel,  2000).  It 
is  true  that  information  literacy  is  not  unique  to  the  digital  age  only.  It  was  always  a crucial  trait  of  successful 
scholars,  even  before  the  information  revolution  of  the  digital  age.  But  in  the  digital  age,  with  the  unlimited 
exposure  of  humans  to  digital  information,  this  has  become  a crucial  prerequisite  that  enables  learners  to  make 
an  educated  use  of  digital  information.  Without  a good  command  of  information  literacy,  how  can  one  decide 
which,  of  the  endless  pieces  of  contradicting  information  found  on  the  Web,  to  believe?  Which  of  the  news  on 
the  Web  to  trust?  Which  political  opinion  posted  on  the  Web  to  adopt? 

In  the  present  research  we  examined  information  literacy.  All  participants  were  asked  to  access  seven 
different  Internet  sites  that  contain  Middle  East  news  (the  Internet  site  of  a main-stream  Israeli  newspaper;  a 
terrorist  organization  information  Internet  site;  a site  of  an  Israeli  government  news  agency;  two  private, 
politically -oriented  news  sites;  an  Arab  country  news  site,  and  a site  of  a European  news  agency).  Participants 
were  given  several  events  (news)  that  were  reported  in  all  of  the  examined  sites.  They  were  asked  to  assess  and 
rate  the  quality  and  objectivity  of  the  news.  Analysis  of  results  is  yet  incomplete,  but  one  trend  is  evident:  The 
younger  the  participants,  the  lower  their  information  literacy.  In  other  words,  the  oldest  participants  showed  a 
significantly  higher  ability  (49%)  to  apply  critical  thinking  to  information,  compared  to  the  youngest  participants 
(15%).  These  findings,  especially  in  the  light  of  the  fact  that  youngsters  are  among  the  most  extensive  consumers 
of  digital  information  (in  the  Internet  or  in  other  forms  of  digital  information  and  communication),  illustrate  the 
critical  importance  of  emphasizing  the  inclusion  of  teaching  information  literacy  in  education  systems.  The  low 
49%  for  adults  is  also  very  worrying.  It  doubly  suggests  that  information  literacy  be  taught  in  schools. 


Conclusions 

This  paper  presents  various  aspects  of  digital  literacy  that  are  usually  neglected  in  educational  research 
concerning  the  impact  of  digital  information  on  our  society  in  general,  and  our  education  systems  in 
particular.  It  proposes  a terminology  framework  for  digital  literacy,  i.e.  photo -visual  literacy,  lateral 
literacy,  reproduction  literacy,  and  information  literacy.  The  following  (preliminary)  research  results 
were  found  for  each  type  of  literacy: 

• The  youngest  participants  (14-15  years  old)  were  the  best  utilizers  of  photo-visual  literacy; 
they  were  capable  of  working  very  well  with  graphic  interfaces  while  the  older  participants 
were  better  with  more  traditional,  menu-driven,  or  text-based  interfaces. 

• The  dyslexic  participants  performed  better  with  graphic  interfaces,  then  with  traditional, 
more  text -based  ones. 

• The  older  participants  were  the  most  reproduction-literate  learners;  they  were  best  in 

rearranging  or  re-using  prior  information  into  new  forms  of  knowledge.  On  the  other  hand. 
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the  youngest  (14-15  years  old)  participants  were  more  flexible  in  their  perception  of  the 
originality  and  ingenuity  of  works  made  using  digital  reproduction  methods  (albiet  a 
sophisticated  ‘copy  and  paste’). 

• The  youngest  participants  (high-school  students)  showed  the  highest  level  of  lateral 
literacy,  which  seemed  to  decrease  with  the  older  age  groups.  This  finding  is  probably 
strongly  related  to  the  extensive  exposure  of  youngsters  to  the  hypermedia  environments  of 
the  Internet. 

• The  oldest  participants  in  the  research  (>30  years  old)  had  the  highest  level  of  information 
literacy,  whereas  the  younger  participants  were  not  overly  capable  of  making  good 
evaluations  and  assessments  of  information.  This  finding  illustrates  the  importance  of 
emphasizing  teaching  critical  information  literacy  in  educational  systems,  especially  in 
schools. 

In  general,  two,  different,  modes  of  approaching  digital  literacy  were  evident  among  the  research  participants: 
The  technological  mode  (using  modem  graphic  interfaces;  surfing  the  Web  in  non-linear  ways),  and  the 
pedagogical  mode  (creating  meaningful  digital  reproductions  of  knowledge;  critically  evaluating  information). 
The  younger  participants  were  better  performing  technological  tasks  whereas  the  older  participants  performed 
better  in  pedagogical  tasks. 

More  research  and  data  analysis  are  required  in  order  to  examine  the  validity  and  aplicability  of  our  findings  in 
wider  populations. 
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Abstract:  Now  that  online  distance  education  has  outgrown  its  infancy,  increasing 
attention  is  being  paid  to  evaluating  online  course  offerings.  A stable,  easy-to-use  web- 
based  course  evaluation  system  is  a necessary  part  of  any  distance  education  evaluation 
effort.  This  presentation  describes  the  design,  development,  implementation  and  revision 
of  a worldwide-web-based  system  for  creating  and  administering  student  evaluations  of 
online  distance  education  courses  and  on- campus  courses  utilizing  Internet  technology. 
The  system  was  developed  after  assessing  instructor  needs  for  such  a system  and 
reviewing  the  existing  and  potential  options  for  meeting  those  needs.  The  system  was 
designed  to  provide  for  both  traditional  summative  (end-of-term)  evaluation  and  mid- 
semester formative  evaluation. 


Introduction 

With  the  incredible  growth  of  online  distance  education  over  the  past  decade,  with  over  1 .5  million  students 
earning  college  credit  online  in  the  1997-1998  school  year,  according  to  a National  Center  for  Education 
Statistics  (NCES)  study,  and  82%  of  the  institutions  in  the  NCES  study  planning  to  start  or  increase  their 
online  offerings,  and  the  need  for  faculty  and  administrators  to  evaluate  these  new  courses  and  degree 
programs,  it  is  essential  that  tools  designed  expressly  for  collecting  student  feedback  to  this  new  type  of 
college  course  are  created.  This  presentation  describes  how  the  instructional  support  office  staff  at  a large 
midwestem  state  university’s  School  of  Education  designed  and  developed  (with  faculty  input)  an  online 
course  evaluation  system  called  EvalOnline.  The  system  can  viewed  online  at  http://ic.educ.indiana.edu. 

Overview  of  the  Project 

In  our  presentation,  we  will: 

• Share  the  results  of  our  faculty  needs  analysis  regarding  their  requirements  for  an  online  course 
evaluation  system. 

• Provide  a technical  overview  of  the  system,  and  the  related  decisions  made. 

• Describe  how  the  system  is  currently  being  used. 

• Describe  our  plans  for  further  development  of  the  system. 


Needs  Analysis 

The  EvalOnline  system  was  created  in  response  to  feedback  from  our  faculty  and  associate  instructor 
clients  about  their  dissatisfaction  with  the  university’s  existing  online  course  evaluation  system  and  how  it 
failed  to  meet  their  needs  as  distance  educators.  In  a series  of  informal  discussions  and  during  one-to-one 
consultations  with  them,  we  discovered  the  following  instructor  needs: 

• Accessibility  and  Ease  of  Use  for  Students 

• Ease  of  Use  for  Instructors 

• Compatibility  with  Existing  School  Requirements 

• Flexibility 

• Relational  Database  Capabilities 
Technical  Overview 

In  the  initial  stage  of  the  development  of  EvalOnline  system,  the  design  team  researched  the  benefits  and 
disadvantages  of  using  a Microsoft  Windows-based  server  compared  with  the  Linux/Unix-based  server 
alternative.  Once  the  platform  was  chosen,  the  design  team  started  work  on  the  database  infrastructure  and 
the  web  interface  with  which  the  users  would  interact.  Instructors  and  students  were  both  given  the  same 
web  address  to  access  the  system  (http://ic.educ.indiana.edu).  On  that  page,  they  saw  buttons  for  Students, 
Faculty  and  Administrators.  Students  clicked  on  the  Students  button,  and  then  were  shown  a page  that 
allowed  them  to  login,  using  the  password  given  to  them  by  their  instructor.  The  next  page  shown  to  them 
was  the  course  evaluation,  which  incorporated  both  Likert  Scale  (Strongly  Disagree  to  Strongly  Agree)  and 
open-ended  questions  (more  question  types  will  be  added  soon).  They  chose  their  responses  and  then 
clicked  the  Submit  button,  which  added  their  course  evaluation  information  to  the  system.  For  instructors, 
the  system  allo\red  for  a wider  variety  of  activities:  they  could  create  or  edit  their  account,  create  or  edit  a 
course  evaluation,  or  view  the  results  of  their  course  evaluations.  Administrators  (departmental  secretaries, 
primarily)  could  view  course  evaluations  of  their  instructors. 

Current  Use 

During  the  Spring  2001  semester,  the  first  semester  of  wide  Q)ost-pilot)  availability,  50%  of  the  instructors 
teaching  onlne  courses  for  the  School  of  Education  utilized  the  EvalOnline  service  to  create  and  deliver 
their  end-of-term  course  evaluations.  The  rest  of  the  online  instructors  either  used  the  previously  existing 
university  testing/grading  system  or  did  not  administer  a course  evaluation  to  their  students.  We  hope  to 
increase  this  percentage  by  describing  and  demonstrating  the  use  of  the  system  to  the  rest  of  the  instructors. 
We  also  expect  that  the  faster  system  response  time  allowed  by  the  new  900mhz  Pentium  III  server  and  the 
greater  ease-of-use  due  to  interface  and  documentation  refinements  will  increase  instructor  acceptance  of 
the  system.  As  the  School’s  distance  education  program  continues  to  grow,  adding  additional  courses  and 
programs,  we  expect  use  of  the  system  to  continue  to  rise.  We  have  also  had  several  on-campus  instructors 
utilize  the  system  and  we  expect  this  type  of  use  to  rise  as  well. 

Future  Development 

In  the  near  future,  we  plan  to  survey  a sample  of  the  students  and  instructors  who  utilized  EvalOnline 
during  Fall  2001  and  Spring  2002,  to  get  their  feedback  regarding  the  system’s  strengths  and  weaknesses. 
After  analyzing  the  responses,  we  plan  to  initiate  a new  round  of  revisions  and  refinements  to  the  program. 
We  are  also  planning  extensive  changes  to  the  administrative  interface,  to  allow  administrators  to  save  and 
print  more  advanced  reports,  and  to  make  general  administrative  features  easier  and  more  time-efficient 
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On-line  Net.art  Gallery  and  Educational  Resource 


With  the  guidance  of  Professor  Mark  Amerika,  a group  of  students  within  the  CU 
Boulder  Fine  Arts  Department  have  completed  an  on-line  art  gallery  entitled  Histories  of 
Internet  Art:  Fictions  and  Factions.  This  gallery  features  Internationally  acclaimed 
works  of  Internet  art,  hot  new  works  by  up  and  coming  net  artists,  as  well  as  the  best 
student  efforts  coming  out  of  the  CU  Boulder  Techne  Lab.  Nearly  all  of  these  pieces  are 
comple  mented  by  student  written  reviews,  which  serve  to  help  clarify  many  of  the  new 
concepts  presented  by  contemporary  artists.  In  conjunction  with  the  visiting  artist 
program  at  CU,  prominent  net.artists  have  been  invited  to  CU  to  be  interviewed  and  lead 
workshops  with  students  who  are  participating  in  this  program.  Interview  text  and  clips 
as  well  as  art  generated  from  these  sessions  are  also  presented  in  this  gallery.  In 
addition,  there  is  an  area  for  researched  net  theory  and  links  to  other  collections  of  net.art. 

My  proposal  is  to  present  this  gallery  to  the  participants  of  the  ED- Media 
Conference  and  give  a short  lecture  and  demonstration  of  what  can  be  found  within  this 
gallery  and  how  it  can  serve  as  an  essential  tool  for  University  art  departments  that  are 
interested  in  exploring  this  new  art  form.  Art  departments  across  the  country  will  likely 
be  including  this  site  in  their  digital  arts  research  list.  Altliough  official  press  releases 
have  only  begun  to  be  circulated,  there  has  already  been  international  attention  from  The 
Whitney  Biennial,  Alt-X,  Naural.it,  and  others. 

One  of  the  most  significant  aspects  of  this  gallery  is  that  it  is  student  created  and 
maintained  and  will  continue  to  grow  and  offer  important  original  content.  Of  much 
interest  to  the  art  community  and  art  enthusiasts  in  general  is  the  filtration  of  the  artwork 
and  the  resource  created  by  bringing  these  works  to  a central  location.  The  interviews 
are  also  extremely  popular  as  they  are  original  content  that  can  only  be  found  on  this  site. 
As  an  educational  resource,  students  can  view  and  explore  this  artwork  as  well  as  study 
some  of  the  essential  theory  that  has  influenced  this  art  movement.  The  reviews  are  also 
helpful  in  determining  which  pieces  are  important  topics  of  research  and  many 
suggestions  and  points  of  view  are  offered  as  to  the  significance  of  these  works.  Faculty 
and  staff  in  academic  art  departments  who  are  interested  in  starting  or  enhancing  a digital 
arts  program  can  look  to  this  as  a resource  and  a link  to  pertinent  information. 

Although  net.art  is  still  fairly  new  in  the  art  scene,  it  has  made  its  way  into 
mainstream  acceptance  and  there  are  on-line  galleries  now  being  offered  by  the  Whitney, 
The  Tate,  SFMOMA,  and  many  other  prestigious  institutbns.  Net.art  may  be  the  only 
art  movement  that  has  gained  this  much  acceptance  in  such  a short  time  ever  before  in  the 
history  of  art,  much  being  owed  to  the  accessibility  of  the  medium  in  which  it  resides. 
Because  this  medium  is  such  an  open  and  leveling  platform,  the  history  of  Internet  art  is 
being  written  and  argued  by  prominent  artists,  critics,  and  art  historians  simultaneously 
while  the  movement  is  gaining  popularity.  It  has  been  accepted  that  anyone  can  publish 
their  findings  and  opinions  on  the  Internet  and  although  this  creates  a means  of  having  a 
voice  for  all  individuals,  the  question  arises  as  to  who  will  actually  look  at  what  has  been 
published.  This  has  created  a new  form  of  economy:  the  attention  economy.  The 
attention  economy  is  already  a formidable  force  and  may  yet  become  one  of  the  most 
influential  economies  yet.  As  any  advertiser  well  knows,  the  attention  economy  has  a 
direct  bearing  on  its  monetary  counterpart  as  it  is  only  by  gaining  the  attention  and  trust 
of  a consumer  that  a product  can  sell.  Everything  on  the  Internet  is  clamoring  for  the 


•539 


attention  of  the  surfing  audience,  and  there  is  a great  need  for  a trusted  filtration 
mechanism  to  highlight  some  of  the  more  important  information.  The  Histories  of 
Internet  art  gallery  hopes  to  become  a popular  filter  and  an  important  hub  for  linking  to 
the  best  that  the  International  community  of  net.artists  has  to  offer.  In  also  having  a 
section  for  student  works,  it  is  possible  for  students  to  gain  the  attention  from  the  art 
community  while  attending  school.  This  type  of  design  can  also  serve  as  a working 
template  for  other  student-run  on-line  galleries. 

Many  argue  that  Internet  art  is  only  a combination  of  audio,  video,  and  text  based 
art  forms.  There  is  a new  aesthetic  that  is  emerging  from  the  blend  of  these  mediums 
combined  with  interactivity  and  the  network  space.  It  is  possible  to  use  the  rest  of  the 
Internet  as  a dynamic  entity  within  this  new  form  of  art.  Interactive  elements  blur  the 
divisions  between  artist,  viewer,  critic,  theorist,  and  historian.  Commercial  development 
of  web  based  tools  and  software  is  fueling  this  new  form  of  visual  creative  expression, 
and  it  is  important  to  demonstrate  that  there  are  alternative  ways  of  viewing  the  Internet 
as  a medium.  The  Internet  can  be  seen  as  a digital  canvas  that  has  tremendous 
accessibility  potential.  It  does  away  with  issues  pertaining  to  physical  space, 
transportation,  collections,  and  formation  of  an  established  canon  by  the  powers  that  be. 
There  has  been  much  theorizing  as  to  how  this  affects  present  culture  and  how  it  will 
influence  art  movements  in  the  future.  It  is  of  great  importance  to  illuminate  and  explore 
these  implications  as  they  cannot  help  but  influence  all  forms  of  art.  Even  the  most 
traditional  artisans  are  turning  to  digital  aids  to  more  efficiently  produce  and  manipulate 
their  work.  Not  only  does  this  create  new  possibilities  in  imaging  but  also  a new 
aesthetic  sensibility  has  emerged  as  a bi-product.  It  is  critical  that  this  is  not  overlooked. 

In  the  short  span  of  the  lecture  I wish  to  present,  I plan  to  raise  some  of  these 
issues  in  relation  to  the  on-line  gallery  that  we  have  created  here  at  the  University  of 
Colorado  at  Boulder.  Histories  of  Internet  Art:  Fictions  and  Factions  gallery 
(http://blurr2.colorado.edu/~hiaff)  is  being  developed  as  part  of  the  TECHNE  practice- 
based  research  initiative  with  the  support  of  the  University  of  Colorado  at  Boulder 
Department  of  Fine  Art,  the  blurr_  lab,  and  ATLAS  (a  campus -wide  technology 
initiative).  From  its  initial  inception,  it  has  brought  together  talent  from  the  college  of 
Fine  arts.  Journalism,  and  Engineering.  It  is  my  hope  and  intention  that  other  disciplines 
within  academia  will  become  interested  in  participating  in  and  embracing  this  method  of 
on-line  presentation  and  this  new  field  of  creative  expression  and  communication.  It  is 
also  my  hope  that  I can  generate  interest  and  stir  excitement  in  this  new  form  of  fine  art. 
Thank  you  for  your  consideration. 
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Introduction 

Here  is  Edward  Bear,  coming  downstairs  now,  bump,  bump,  bump,  on  the  back  of  his  head  behind 

Christopher  Robin.  It  is,  as  far  as  he  knows,  the  only  way  of  coming  downstairs,  but  sometimes  he 

feels  there  really  is  another  way,  if  only  he  could  stop  bumping  for  a moment  and  think  of  it. 

Milne,  1996 

The  quote  from  Milne  speaks  to  the  way  many  teachers  function  in  the  K12  arena.  There  is  so 
much  to  do  and  so  little  time  to  do  it  in,  they  continue  to  do  work  in  the  same  way  it  has  been  done  for 
decades.  Teachers  generally  know  there  may  be  better  methods  available,  but  do  not  have  the  time  to 
investigate  what  they  are.  As  assessments  and  data-driven  instruction  become  increasingly  important, 
teachers  must  stop  "bumping”  down  the  stairs  in  order  to  become  more  efficient,  effective  users  of  the 
equipment  provided  for  them.  Integrating  technology  into  a Constructivist  learning  environment  is  one  way 
that  is  currently  being  advocated  as  a method  that  will  move  more  students  toward  functioning  in  the  real 
world. 

The  technological  changes  that  have  swept  through  our  society  have  left  educational  systems 
largely  unchanged.  Over  the  past  20  years,  an  immense  division  appeared  between  the  process  of  teaching 
and  learning  in  schools  and  the  ways  we  obtain  knowledge.  This  division  has  been  made  more  obvious  by 
the  fact  that  the  process  of  teaching  has  not  changed  substantially  over  the  past  100  years.  The  result  is  an 
estrangement  of  children  in  schools  from  society  as  a whole.  Educators  now  question  how  we  can  best 
educate  students  raised  in  a world  of  instant  information,  where  interactive  technologies  have  led  them  to 
believe  they  can  act  on  the  world  with  the  press  of  a button.  What  is  needed  is  a guiding  philosophy  for 
educational  curriculum  and  effective  uses  of  technology.  It  has  been  proposed  that  this  philosophy  could  be 
Constructivism  (von 
Glaserfeld,  1989). 

Technology  and  Constructivism 

Constructivism's  central  idea  is  that  human  learning  is  constructed,  and  that  learners  build  new 
knowledge  on  the  footing  of  previous  learning.  In  a Constructivist  atmosphere,  the  learner  constructs 
knowledge  through  discovery  and  exploration  in  order  to  solve  real  world  problems  with  peers.  According 
to  Bruner,  Constructivism  is  a teacher-facilitated  process  that  places  students  at  the  center  of  active 
learning,  rather  than  in  a passive  role.  Instead  of  simply  absorbing  ideas  spoken  by  teachers,  or  somehow 
trying  to  internalize  thoughts  through  endless,  repeated  rote  practice,  constructivism  suggests  children 
actually  invent  their  own  ideas.  They  assimilate  new  information  and  modify  their  understanding  in  light  of 
new  data.  In  the  process,  their  ideas  gain  in  complexity  and  power,  and  with  appropriate  support,  children 
can  develop  insight  into  their  own  thinking  processes.  Advocates  of  constructivist  teaching  say  its  practice 
is  vital  if  America  hopes  to  cultivate  and  maintain  a work  force  capable  of  competing  globally  in  the  21®* 
century. 

Schools  have  always  mirrored  the  values  of  our  society,  however  the  technological  changes  that 
have  swept  through  society  have  left  the  educational  system  virtually  unchanged.  It  is  frightening  to  note 
that  as  our  children  are  growing  up,  they  may  not  be  receiving  the  instruction  and  knowledge  necessary  to 
function  in  a technological  world.  The  task  of  the  educational  system  should  be  to  embrace  the  future  and 
empower  children  to  learn  with  the  tools  available  to  them.  Trying  to  think  up  clever  ways  to  use  computers 
in  a traditional  classroom  setting  will  not  do  the  job.  If  we  assign  technology  to  a secondary  role,  which 
does  not  fully  utilize  its  potential  strengths,  we  are  failing  to  use  it  to  its  fullest  potential. 

By  using  technology  as  part  of  a constructivist  philosophy  in  the  classroom,  we  can  give  our 
children  a powerful  set  of  tools.  Technology  should  be  thought  of  as  an  integral  component  of  any 
curriculum.  Computers  in  a constructivist  classroom  can  be  used  as  a writing  tool,  to  complete 
spreadsheets,  to  compile  databases,  for  concept  mapping,  to  create  multimedia,  to  author  Internet  projects, 
to  complete  research,  and  mathematical  problem-solver.  Computers  also  make  it  possible  to  exchange 
information  between  classrooms  and  access  online  information. 
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"The  introduction  of  computers  into  the  classroom  changes  the  teachers'  role,  leading  to  decreased 
teacher-directed  activities  and  a shift  from  didactic  approaches  to  a constructivist  approach"  (Schofield  & 
Verban  1988).  Research  has  shown  that  teaching  in  classrooms  with  computers  involved  more:  a)  project 
work,  b)  extensive  projects,  c)  motivation  for  the  writing  process,  d)  group  work  and  cooperative  learning, 
e)  interdisciplinary  activities,  f)  opportunities  for  students  to  make  choices  and  g)  a different  philosophy  of 
teaching  (Dwyer,  et  al.,  1990).  A classroom  with  a technology  rich  environment  has  also  been  shown  to 
involve  less  structure  and  less  teacher  presentation  (Schofield  & Veiban,  1988).  The  trick  to  teaching  is  to 
entice  and  motivate  the  students'  excitement  and  interest  in  the  topic,  and  then  to  give  them  the  proper  tools 
to  reflect,  explore,  compare,  and  contrast.  Current  interactive  multimedia  technologies  have  the  potential  to 
represent  ideas  in  almost  any  form  so  students  view  the  resources,  creating  their  own  meanings  and 
understandings  of  the  information  they  encounter. 

Increasingly,  researchers  and  educators  are  linking  Constructivism  and  learning  with  technology. 
This  is  not  surprising  since  many  see  a strong  support  for  the  principles  of  Constructivist  philosophy  in 
computer-based  learning  environments.  Computer-based  Constructivist  projects  can  take  many  forms,  both 
online  and  off.  They  do,  however,  all  have  one  thing  in  common;  they  are  based  on  a problem-solving 
format.  CSILE,  CoVis,  Computer  Clubhouse,  JASPER,  and  Web  Quests  are  all  examples  of  such  formats. 

Using  a Building  Block  model  (Dodge,  2000;  March,  1995;  March  & Reed,  1999)  integrated 
technology  projects  can  be  created  that  will  allow  students  to  gather  information,  use  critical  thinking  skills 
and  communicate.  A constructivist  technology-integrated  lesson  plan  should  be  designed  to  bridge  the 
transition  between  teacher-led  instruction  and  self-directed  learning  by  students.  The  hope  is  that  through 
scaffolding,  both  teachers  and  students  will  gain  experience  in  a learning-centered  approach.  One  way  these 
projects  can  be  created  has  four  steps:  a)  determine  outcomes,  b)  draft  the  project  framework,  c)  develop 
the  evaluation  tool,  and  d)  design  the  task. 

Summary 

The  goal  for  education  should  not  be  simply  to  access  information,  but  to  understand  and  use  it  to 
solve  real  world  problems.  Technology  offers  educators  the  opportunity  to  move  away  from  instructional 
strategies  that  focus  on  presenting  abstract  information  to  a passive  learner,  to  an  active  process  where 
meaning  is  developed  based  on  experience.  This  way  of  teaching  and  learning  supports  the  Constructivist 
point  of  view  where  the  learner  is  building  an  internal  representation  of  knowledge  and  a personal 
interpretation  of  experiences.  Technology  also  offers  students  the  opportunity  to  communicate  and 
collaborate  with  millions  of  worldwide  users  on  the  Internet. 

Technology  has  effectively  revolutionized  American  society.  An  unexpected  by  product  of  this 
revolution  has  been  the  emergence  of  a generation  of  children  weaned  on  multidimensional,  interactive 
media  sources,  a generation  whose  understanding  and  expectations  of  the  world  differ  profoundly  from  that 
of  the  generations  preceding  them.  We  need  to  give  these  children  the  education  necessary  to  succeed  in  a 
technological  future.  We  need  to  build  on  children's  native  learning  abilities  and  technological  competence 
with  an  instructional  model  that  allows  learners  to  be  actively  involved  and  engaged  in  a real-life  arena, 
which  includes  technology. 
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A Game  Editor  for  Virtual-Prismaker  Learning  Environment 
to  Improve  Teaching  and  Learning  in  Classroom 


Abstract:  This  paper  describes  a personal  game  editor  developed  to  improve  teaching  and  learning 
strategies  in  classroom  trough  a learning  environment  called  Virtual-Prismaker.  Most  educational 
games  are  closed  in  the  sense  that  the  teacher  is  not  permitted  to  introduce  new  activities,  in  the 
context  of  Virtual-Prismaker  project  we  have  faced  the  need  to  open  the  game’s  educational 
possibilities  by  enabling  to  incorporate  the  educator’s  experiences  and  needs,  and  by  enforcing  the 
motivation  children  find  in  playing  computer  games. 


1.  Introduction 

When  preparing  activities  that  are  to  be  carried  out  in  classroom,  the  teacher  tries  to  maximize  the  use  of 
available  resources.  Prismaker  is  a construction  game  composed  of  reduced  number  of  kinds  of  available 
pieces.  The  main  construction  block  is  a semicube.  Apart  from  these  basic  construction  blocks,  there  are  a 
series  of  additional  pieces:  wheels,  axis,  small  cylinders  and  logos.  In  our  context,  logos  are  very  useful 
and  greatly  improve  the  construction  game’s  cognitive  capacities.  Indeed,  logos  are  used  to  assign  blocks 
a meaning.  Teachers  are  able  to  create  a greater  number  of  different  games  for  distinct  school  matters  by 
using  this  highly  interesting  feature.  Information  technologies  provide  new  tools  for  the  creation  of 
contents  that  adapt  to  children  characteristics,  to  their  learning  capacities  and  to  their  special  needs.  Our 
research  team  is  involved  in  the  development  of  an  interactive  learning  environment  based  on  the  physical 
educational  game  Prismaker*™  [Prismaker,  2001].  The  game  is  called  Virtual-Prismaker  [Gonzalez  et  al., 
2001;  Lopez  et  al.,  2001].  Introducing  new  technologies  in  classroom  does  not  mean  substituting  the 
teacher’s  work.  It  rather  allows  creating  new  games  according  to  the  students  needs.  This  is  the  cue  we 
are  approaching  in  this  paper. 

2.  A Game  Editor  to  Improve  Teaching  and  Learning  Strategies 

Computer  games  designed  for  education  in  classroom  should  allow  the  teacher  to  create  and  to  modify  the 
existent  playing  possibilities  to  adapt  the  material  to  the  learning  necessities  of  the  different  students. 
Thus,  the  central  idea  of  our  project  is  to  offer  the  possibility  to  personalize  games.  Observing  similar 
games  that  exist  in  the  marketplace,  we  have  noticed  that  their  playing  possibilities  are  totally  closed.  Our 
solution  is  to  offer  teachers  an  open  game.  For  the  currently  available  types  of  games,  the  teacher  is  in 
charge  of  creating  those  activities  he  considers  to  be  convenient,  as  well  as  their  characteristics.  Here  is 
where  the  teacher’s  educational  role  begins.  Our  virtual  game  editor  tool  is  not  sought  to  substitute  the 
teacher’s  labor,  but  to  offer  him  to  be  a decisive  part  in  a new  teaching  means. 

A characteristic  incorporated  to  the  game  is  the  division  in  difficulty  levels.  This  also  bears  a motivation 
aspect  for  the  player  as  he  is  challenged  to  complete  all  the  levels.  The  difficulty  levels  are  based  on  the 
age  of  the  player  and  the  teaching  matters  (mathematics,  history,  language,  etc).  The  student  has  to 
overcome,  according  to  his  age,  the  different  difficulty  levels  to  complete  each  matter.  It  is  the  teacher’s 
Job  to  associate  any  playing  activity  to  a certain  level. 
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In  our  solution  the  main  raw  material  needed  to  create  activities  are  images.  We  have  also  created  a 
letter/word  editor  that  allows  us  to  create  simple  images  containing  letters,  numbers  and  words  in  a quick 
and  easy  way. 

3.  The  Game  Editor  Capacities 

3.1.  Meaning  related  games 

These  kinds  of  games  have  been  incorporated  as  they  throw  two  desirable  characteristics  for  learning.  The 
first  characteristic  is  its  memory  training  capacity.  And,  respect  to  the  concept  of  association,  it  offers  a 
great  potential  applied  to  any  matter.  Some  examples  are  offered  now.  In  a Foreign  Language  course  it  is 
possible  to  give  Ae  native  word  and  the  foreign  word  for  it.  An  example  in  Geography  is  to  show  the 
name  (or  the  map)  of  a state  and  the  name  (or  a representative  monument)  of  its  capital.  There  is  no 
limitation  to  the  teacher’s  imagination.  Thus,  it  is  feasible  to  create  identical  couples,  to  create  couples 
with  the  same  concept  or  even  to  create  couples  with  some  preset  relationship.  This  way,  children  learn  to 
relate  concepts.  The  association  possibilities  are  infinite.  For  example  we  can  enumerate  the  following 
ones:  learning  words  in  several  languages,  relating  two  numbers  associated  somehow,  comparing  an 
image  to  a written  expression,  associating  the  same  images. 

3.2.  Order  related  games 

This  second  kind  of  games  is  based  on  composing  an  original  image  where  the  important  thing  is  the 
order  of  the  parts  that  compose  the  global  image.  The  student  is  given  the  original  image  and  the 
disordered  pieces.  Considering  the  fact  that  the  important  thing  is  the  order,  we  have  added  to  this  type  of 
game  the  possibility  to  create  a sequence  of  images.  We  can  create  this  way,  for  example,  an  alphabet,  a 
sequence  of  images  that  describe  a story,  etc.  Another  possibility  is  to  create  a sequence  of  images  where 
the  order  is  the  important  thing.  The  editor  offers  a multiple  image  viewer  for  this  case.  Once  we  have  the 
images,  the  creation  of  an  activity  game  of  this  type  is  very  simple.  We  can  create  image  sequences  such 
as  an  alphabet,  a story  where  each  drawing  represents  a part  to  the  story,  etc.  Series  or  patterns  are 
allowed  too.  When  a pattern  is  being  created  the  teacher  decides  the  elements  the  pattern  will  consist  of, 
along  with  the  number  of  times  the  pattern  must  be  repeated.  A mathematical  sequence  could  be  created 
defining  the  initial  value,  the  expression  to  calculate  the  next  element  and  the  minimum  number  of 
elements  to  play  successfully  that  game. 

3.3.  Meaning  plus  order  related  games 

Meaning  plus  order  has  been  used  in  a game  based  on  mathematics,  so  that  children  are  engaged  to  learn 
about  the  basic  operators  in  a simple  way.  In  this  kind  of  game,  the  student  is  offered  a disordered 
mathematical  expression,  where  each  piece  contains  an  image  of  an  operand  or  an  operator.  The  student’s 
mission  is  to  place  the  pieces  so  that  the  expression  is  correct.  Therefore,  both  the  order  of  the  pieces  and 
the  result  calculated  to  evaluate  the  expression  has  to  be  correct.  Each  piece  will  be  an  integer  number  or 
one  of  the  following  operators:  sum,  subtraction,  division,  multiplication,  right  parenthesis  or  left 
parenthesis. 

Acknowledgements 

This  work  is  supported  in  part  by  the  CICYT-FEDER  1FD97-1017  grant  and  by  the  2000020264  JCCM 
and  European  Social  Fund.  The  authors  would  like  to  thank  Corebo  Moreno  Rodenas  and  Jose  Martinez 
Esparcia  for  their  very  helpful  contribution  to  Virtual -Prismaker. 

References 

[Gonzalez  et  al.,  2001]  Gonzalez,  P.,  Montero,  F.,  Lopez,  V.,  and  Femandez-Caballero,  A.  (2001).  A 
virtual  learning  environment  for  short  age  children,  In  Proceedings  IEEE  International  Conference  on 
Advanced  Learning  Technologies,  ICALT  2001,  pp.  283-284. 

[Lopez  et  al.,  2001]  Lopez,  V.,  Femandez-Caballero,  A.,  Gonzalez,  P.,  Montero,  F.,  and  Molina,  J.P. 
(2001).  Collision  handling  and  shadow  casting  in  Virtual-Prismaker,  In  Proceedings  of  the  lASTED 
International  Conference  on  Visualization,  Imaging  and  Image  Processing,  VI IP 200 1,  pp.  113-116. 

[Prismaker,  2001]  www.prismaker.com 


PDA  Strategies  for  Preservice  Teacher  Technology  Training 


Candace  Figg 
College  of  Education 
University  of  Louisiana  at  Lafayette 
United  States 
cfigg@louisiana.edu 

Jenny  Burson 
College  of  Education 
University  of  Texas  at  Austin 
United  States 

jenny  burson@teachnet.edb.utexas.edu 


Abstract:  Making  technology  integration  relevant  by  eliminating  the  access  barrier  for 
preservice  teachers  is  a problem  that  may  be  solved  for  educators  of  preservice  teachers  by 
the  utilization  of  PDA  (Personal  Digital  Assistant)  devices.  For  approximately  $250,  students 
can  be  equipped  with  the  PDA,  keyboard,  and  software  necessary  to  read,  write,  collect  data, 
and  create  presentations.  This  paper  outlines  the  perceptions  of  the  preservice  teachers  in 
their  beginning  technology  class  that  utilized  PDAs  to  support  their  learning  experiences  and 
discusses  the  appropriate  use  of  PDAs  to  integrate  technology  into  classroom  activity 
structures. 


Introduction 

Surveys  regarding  the  current  state  of  technology  use  in  classrooms  indicate  that,  although  more  students  have 
access  to  computers  than  ever  before,  either  in  the  classroom  or  at  home,  78%  of  classroom  teachers  still  view  lack  of 
access  to  computers  in  the  classroom  as  a barrier  to  their  efforts  to  integrate  technology  into  their  daily  teaching 
practices  (U.S.  Department  of  Education,  2000).  In  a comprehensive  survey  that  reviewed  teacher  use  and  teacher- 
directed  use  of  computers  in  classrooms  around  the  United  States,  Becker,  Ravitz,  and  Wong  (1999)  found  that  “regular 
use  of  computers  with  students  is  highly  dependent  on  access  to  computers’*  and  that  frequent  use  of  computers 
occurred  more  frequently  with  5-8  computers  in  the  classroom  rather  than  with  15-30  computers  in  a lab  (executive 
summary  statement  on  website). 

The  challenge,  then,  to  educators  responsible  for  providing  preservice  and  inservice  teachers  with  the  skills 
necessary  to  infuse  technology  into  their  daily  teaching  practices  is  to  provide  teachers  with  options  to  overcome  the 
access  barrier  as  well  as  inform  them  of  emerging  technologies  and  practices  that  strengthen  and  support  their  inclusion 
of  technology  into  daily  classroom  routines  and  practices. 

Meeting  this  challenge  may  have  received  a helping  “hand”  with  the  recent  drop  in  prices  of  the  Personal 
Digital  Assistant  (PDA),  a handheld  device  that  allows  a portable  and  relatively  inexpensive  means  of  gathering, 
recording,  and  organizing  information  into  meaningful  forms  of  communication.  For  approximately  $250  per  student,  the 
teacher  is  able  to  provide  a student  with  a PDA,  black  and  white  camera,  and  a keyboard  that  will  enable  the  student  to 
take  notes,  jot  down  ideas,  record  thoughts,  memories,  words,  and  responses  to  readings/experiences,  collect  pictures 
for  creative  projects,  as  well  as  support  time  management  skills  needed  for  the  successful  student. 

The  research  described  here  provided  preservice  teachers  currently  participating  in  the  beginning  technology 
course.  Teaching  with  Technology,  with  an  opportunity  to  explore  the  application  of  handhelds  to  elementary  classroom 
instruction,  and  then  investigated  their  perceptions  as  to  the  factors  perceived  as  most  valuable  in  influencing  teachers 
to  use  this  emerging  technology  as  a part  of  their  students'  everyday  learning  activities. 

The  Study:  The  Participants  and  Data  Generation/Analysis 
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Investigating  the  perceptions  of  preservice  teachers  as  to  the  factors  that  will  influence  teachers  to  use 
handhelds  as  part  of  everyday  learning  and  teaching  activities  that  integrate  technology  into  the  instruction  required 
two  components.  First,  the  study  required  the  selection  of  a purposive  sample  of  participants  with  knowledge  and 
understanding  of  the  capabilities  and  functionality  of  the  handheld,  as  well  as  participants  knowledgeable  about  the  use 
of  technology  seamlessly  applied  to  instructional  design.  The  participants  in  this  study  were  chosen  from  among  the 
preservice  teachers  currently  enrolled  in  a university  technology  integration  class,  Teaching  with  Technology,  that  is 
one  of  three  courses  that  comprise  the  first  block  of  the  Elementary  Preservice  Teacher  Certification  Program.  These 
participants  also  expressed  an  interest  in  learning  to  use  emerging  technologies  as  a part  of  daily  classroom  instruction, 
and  had  constant  access  to  computers  as  they  were  currently  working  in  a technology-rich  environment  as  a part  of  the 
technology  course.  These  parameters  were  established  so  that  the  perceptions  of  informants  with  similar  technology 
training  backgrounds  and  technology  support  could  be  explored.  The  sampling  was  designed  to  include  a broad  variety 
of  informant  backgrounds  based  on  life  experiences,  age,  unique  observation  field  situations,  and  a variety  of  teaching 
expertise  (beginners  vs.  novice  preservice  teachers). 

Secondly,  the  research  required  a study  design  that  placed  the  voices  of  the  participants  and  the  telling  of  their 
perceptions  at  the  center  of  the  inquiry,  one  of  the  assumptions  key  to  constructing  research  in  the  constructivist 
paradigm.  Naturalistic  Inquiry,  the  strategy  chosen  for  this  study,  provided  the  researcher  with  appropriate  methods 
necessary  to  explore  preservice  teachers'  perceptions  of  technology -based  activities  deemed  valuable  to  their  teaching 
practices.  Defined  as  a collection  of  methods  with  "the  commitment  to  studying  human  action  in  some  setting  that  is  not 
contrived,  manipulated,  or  artificially  fashioned  by  the  inquirer”  (Schwandt,  1997,  p.  102),  Naturalistic  Inquiry  relies  upon 
a study  design  which  allows  the  findings  to  "emerge"  from  the  data  as  the  value  systems  of  the  researcher  and 
participants  "interact  in  unpredictable  ways  to  influence  the  outcome"  (Lincoln  & Cuba,  1985,  p.  41). 

The  constructivist  design  occurred  in  two  stages.  During  the  first  stage,  participants  were  given  the 
opportunity  to  explore,  handle,  and  acquire  competency  with  the  use  of  handhelds,  as  well  as  probe  the  application  of 
handhelds  within  the  instructional  design  process.  To  do  this,  the  participants  attended  two  workshops.  The  first 
workshop  instructed  participants  in  the  basic  use  of  handhelds;  the  second  workshop  instructed  participants  in  the 
integration  of  handhelds  into  the  instructional  design  process.  Between  the  workshops,  the  participants  were  given  a 
handheld  and  a keyboard  and  asked  to  use  the  handhelds  on  a daily  basis  by  taking  notes,  using  the  calendar  feature, 
maintaining  an  address  book,  and  beaming  documents  between  study  participants.  Participants  provided  the  researcher 
with  weekly  reflective  journals  that  documented  their  learning  experiences  during  this  first  stage. 

After  the  second  workshop,  participants  participated  in  a series  of  unstructured,  open-ended  interviews 
initiated  with  a single  broad-focused  question:  What  are  the  factors  you  perceive  as  most  influencing  elementary 
teachers  use  of  emerging  technology,  such  as  the  handheld,  as  a part  of  students'  everyday  learning  activities,  as  well 
as  a part  of  the  teacher's  everyday  teaching  activities?  The  interviews  were  tape-recorded,  transcribed,  and  member- 
checked  through  follow-up  interviews  with  the  informants.  Other  data  in  the  form  of  related  documents,  such  as  e-mail 
conversations  with  participants,  reflexive  journals  of  researcher  and  participants,  examples  of  lesson  plans  and  student 
activities  created  during  the  process,  were  collected  and  analyzed.  A peer-debriefmg  group  met  weekly  for  the  purpose 
of  reviewing  the  analysis  of  documents  and  interpretation  of  resulting  interview  data. 

Data  analysis  in  a audy  that  uses  naturalistic  inquiry  methods  is  inseparable  from  data  generation  and 
collection;  thus  the  process  of  data  analysis  began  with  the  first  contact  between  researcher  and  participant.  Data 
generated  from  each  contact  with  participants,  through  interview,  observation,  or  the  creation  or  review  of  documents, 
helped  to  form  several  tentative  findings,  which  led  the  researcher  to  probe  some  areas  of  discussion  more  deeply  or 
change  strategies  for  observation  and  data  generation.  Adjusting  data  generation  procedures  continued  as  additional 
data  were  generated  and  tested  against  the  evolving  themes.  The  process  of  generating  data,  unitizing  data,  and 
developing  categories  and  themes  from  the  data  continued  until  reaching  thematic  saturation,  or  the  point  at  which  no 
more  new  categories  and  themes  made  themselves  apparent  (Erlandson  et  al.,  1993). 

The  Findings 

These  findings  are  preliminary  as  the  final  interviews  are  still  being  member-checked,  but  two  main  themes 
(subject  to  refinement  as  the  data  collection/analysis  continues)  have  emerged: 

Theme  1:  Participants  perceived  handhelds  as  an  invaluable  tool  for  collaborative  learning  environments 
and  a catalyst  for  development  of  learning  communities.  Participants  overwhelmingly  expressed  satisfaction  with  the 
capability  of  handhelds  to  collect  data/resources  and  ease  with  which  that  information  could  be  shared  with  others, 
either  through  traditional  printing  methods  or  beaming  files  to  others  to  use.  Participants  perceived  this  ease  of 
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distribution  of  data  and  the  process  of  collaborative  sharing  as  promoting  the  development  of  a learning  community  that 
supported  their  learning  needs. 

Theme  2:  Participants  perceived  handhelds  as  an  invaluable  instructional  tool  because  it  provided 
technology  that  was  easy  to  use,  supported  the  learning  needs  of  students  often  supported  by  technology,  and  was 
portable  and  accessible.  Participants  valued  the  handheld  as  an  instructional  tool  for  students  because  the  skill  level 
required  to  use  the  tools  was  minimal,  could  be  acquired  with  minimum  classroom  time,  and,  student  work  could  never  be 
“lost”  due  to  the  student  carelessly  closing  an  application  without  saving.  In  addition,  with  the  view  that,  as  students, 
they  personally  used  technology  mostly  for  taking  class  notes,  writing  short  paragraphs  for  various  purposes,  and 
keeping  up  with  the  assignments,  participants  viewed  the  capability  of  the  handheld  to  provide  word  processing,  editing, 
and  beaming  functions  as  supporting  the  majority  of  their  learning  needs  that  they  were  currently  supporting  with 
desktop  computers.  And,  finally,  the  fact  that  this  capability  could  be  taken  anywhere  and  used  anytime  was  perceived 
as  providing  users  with  a portability  that  could  not  be  provided  by  laptops  or  desktop  computers. 


Conclusion 

Although  the  purpose  of  this  inquiry  was  not  to  provide  findings  that  are  representative  of  all  preservice 
teachers,  but  to  explore  the  perceptions  of  these  12  participants  as  they  reflected  upon  the  value  of  handhelds  in  the 
learning  environment,  these  findings  may  suggest  possible  consideration  to  the  various  contexts  of  using  emerging 
technologies  as  enhancements  for  learning  environments.  Strategies  for  using  handhelds  as  a part  of  a successful 
learning  environment  may  include  integrating  handhelds  into  instructional  designs  that  require  the  following: 

> Collaborative  learning  processes  or  the  development  of  a learning  community, 

> Writing  Process  or  intensive  development  of  writing  pieces,  or 

> Portable  tools  for  remote  data  collection. 
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Abstract:  This  paper  is  a report  of  user  studies  for  developing  learning  strategy 
tool  software  for  children.  The  prototype  software  demonstrated  is  designed  for 
children  with  learning  and  behavioral  disabilities.  The  tools  consist  of  easy-to-use 
templates  for  creating  organizational,  memory,  and  learning  approach  guides  for  use 
in  classrooms  and  at  home.  User  studies  that  were  conducted  with  software 
prototypes  included  two  rounds  of  expert  review,  usability  testing  in  a lab  setting 
with  teachers  and  parents,  a focus  group  with  parents  and  educators,  and  “think- 
aloud”  observations  of  children  using  a sample  of  the  tools.  Results  were  used  in 
improving  the  software,  preparing  the  teacher  orientation  module,  and  designing 
online  supports  for  users.  The  most  useful  dimensions  of  evaluation  were  found  to  be 
direct  observation  of  children  using  the  software  and  messages  contributed  by  a high 
school  student  with  learning  disabilities  to  the  Online  Focus  Group. 

Service  to  children  with  behavioral  and  learning  disorders  is  a high  priority  in  the  U.S.  (Elam,  Rose,  & 
Gallup,  1998;  Kay,  1999).  Currently,  federal  mandates  require  that  such  children  be  served  in  the  “least 
restrictive  environment”  by  providing  appropriate  supports  and  effective  strategies  to  ensure  success  (IDEA97). 
Common  problems  faced  by  these  students  in  regular  classrooms  are  difficulties  maintaining  attention, 
organizing  work,  managing  time  and  materials,  and  responding  with  accuracy  (Lewis  & Doorlag,  1999).  With 
these  deficiencies  students  are  at-risk  for  increased  behavioral  problems,  failure,  and  ultimately,  removal  from 
these  classrooms.  Research  shows,  however,  that  strategy  instruction  in  these  skills  can  help  students  become 
successful  learners  and  managers  of  their  own  behavior  (Swanson  & Hoskyn,  1998). 

Electronic  Performance  Support  for  Children  with  Learning  and  Behavior  Problems 

Computer-based  training  and  support  mechanisms  are  an  innovative  approach  for  helping  children  gain 
control  over  personal  behaviors.  While  electronic  performance  support  systems  have  been  used  in  education  and 
industry  to  support  adults  in  learning  new  skills  “at  the  right  place,  right  time,  right  form”  (Laffey,  1995),  work 
is  scant  in  applying  these  principles  to  the  development  of  software  tools  for  children.  If  the  goal  of  EPSS 
software  is  appropriate  for  adults — to  provide  whatever  is  necessary  to  ensure  performance  and  learning  at  the 
moment  of  need  in  a seamless  activity — then  it  offers  the  same  potential  for  helping  children  improve 
performance  in  their  learning  environments.  If  effective,  EPSS  support  tools  for  children  improve  their 
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independent  functioning,  reduce  teacher  time  re-focusing  their  mistakes  and  energies,  and  offer  lifelong  skills 
for  success. 

This  paper  reports  the  design  process  followed  by  the  developers  in  creating  a new  set  of  EPSS  tools  to 
help  young  children  learn  and  use  these  school  survival  strategies.  These  unique  materials  include  four 
components:  (1)  strategy  software  tools  for  children,  (2)  an  information  resource  database  for  teachers,  (3)  a 
web-based  orientation  module  for  teachers,  and  (4)  online  resources  and  discussion  lists  to  support  teachers  and 
parents  (refer  to  conceptual  framework  in  Figure  1)  (Fitzgerald  & Koury,  2001-2002).  During  the  development 
period,  a variety  of  user  studies  were  conducted  with  children,  parents  and  educators  to  ensure  that  the  materials 
represent  best  practices  in  the  field.  This  paper  summarizes  the  user  studies  and  initial  findings  and  highlights 
important  issues  in  software  design  for  children. 

Related  Work:  Previous  Applications  with  EPSS  Tools  with  Children 

Computer-based  training  and  support  mechanisms  are  an  innovative  approach  for  helping  children  gain 
control  over  personal  behaviors.  Although  there  are  limited  data  on  the  use  of  computer-based  instruction  to 
support  behavior  change  in  children  to  date,  research  results  are  promising.  Fitzgerald  and  Werner  (1996) 
reported  success  with  a computerized  verbal  mediation  essay  as  a cognitive  retraining  procedure  to  assist  a 
student  with  significant  behavioral  disorders  in  changing  Ks  behavior;  the  computerized  essay  provided 
consistent  practice  and  focused  the  child’s  attention  and  thoughts  on  behavioral  choices  and  consequences.  In 
another  case  study,  the  same  researchers  reported  a procedure  in  which  software  templates  were  developed  for  a 
student  to  create  self-monitoring  materials. 

Following  these  early  studies,  the  developers  created  a series  of  EPSS  tools  for  children  that  supported 
children  in  using  self-management  skills  in  classrooms  (Fitzgerald  & Semrau,  1998-2000).  The  children’s 
software  programs.  First  Step  KidTools  for  ages  7-10,  and  Second  Step  KidTools  for  ages  11-14,  provide  a 
series  of  progressively  complex  templates  to  be  used  as  tools.  In  using  the  tools,  children  identify  behaviors  for 
self-improvement,  identify  specific  self-control  strategies  they  want  to  use,  prepare  self-talk  statements  to  guide 
their  use  of  the  strategies,  and  create/print-out  materials  to  support  their  plans.  Both  software  programs  consist 
of  tool  templates  that  are  kid-friendly  with  colorful  graphics,  text-with-audio  directions,  multiple  examples, 
simple  formats,  and  automatic  record-keeping  capabilities.  To  use  the  tools,  the  child  simply  clicks  on  “hot 
words”  on  the  template  form  to  enter  personalized  content  and  then  print  out  the  completed  form  for  use  in  the 
classroom  or  home. 

Qualitative  methodologies  were  used  to  gather  multiple  data  on  the  usefulness  of  the  approach  and 
important  design  features  in  the  software.  Several  important  design  decisions  were  made  based  on  these  studies 
that  have  been  applied  to  the  production  of  the  new  strategy  software:  (1)  include  the  option  of  spoken  text  for 
poor  readers;  (2)  use  children  as  on-screen  guides  through  the  tools;  (3)  use  children’s  voices  for  narration;  (4) 
provide  access  to  help  screens  that  can  be  accessed  while  using  the  tools;  (5)  provide  multiple  options  to  re-start 
or  make  changes;  and  (6)  put  the  child  in  charge  of  creating  his/her  tools  using  natural  language.  Pilot  testing 
with  KidTools  in  two  states  demonstrated  that  children  were  able  to  independently  use  the  tools  and  were 
motivated  by  the  computerized  tool  format.  Teachers  reported  improved  motivation  and  self-control  in 
classrooms.  The  findings  from  KidTools  are  extended  to  the  production  of  KidSkills  with  its  new  focus  on 
learning  strategies.  The  previous  work  ensures  appropriate  literacy  and  ease-of-navigation  features  are  included 
in  the  KidSkills  design  (Fitzgerald,  Watson,  Lynch  & Semrau,  1999). 

Conceptual  Framework  for  the  Design  oi  KidSkills 

Recognizing  the  importance  of  ecological  variables  surrounding  an  innovation  systems  (Peled,  Peled,  & 
Alexander,  1994),  multiple  products  are  being  developed  in  a systems  approach.  The  primary  components  of 
this  framework  are  illustrated  in  Figure  1:  development  of  the  EPSS  software,  development  of  a resource 
information  database  for  educators  and  parents,  development  of  an  orientation  module,  and  use  of  online  focus 
groups  and  a web-based  hotline  to  share  information,  provide  support,  and  disseminate  materials  during 
implementation. 


KidSkills,  the  children’s  software,  offers  strategies  based  on  cognitive -behavioral  approaches  that  help 
youngsters  change  cognitions  (thoughts,  beliefs,  self-talk,  cues)  and  behaviors  (actions)  within  a problem- 
solving framework  to  gain  successful  outcomes.  These  approaches  have  increased  in  popularity  during  the  last 
two  decades  as  researchers  have  documented  their  effectiveness  (Gresham,  1985)  and  curriculum  developers 
have  designed  and  disseminated  teachable  programs  for  implementation  in  schools  (Anderson,  1981;  Nichols, 
1999).  To  date,  there  have  been  no  known  precursors  to  this  work  that  combines  electronic  performance  support 
tools  along  with  strategy  training  for  teaching  cognitive-behavioral  strategies  to  young  children. 


Fig.  1 Conceptual  Framework  for  Development  of  KidSkills 


The  KidSkills  Prototype 

There  are  two  levels  of  KidSkills.  First  Step  KidSkills  is  designed  for  ages  7-10  and  Second  Step  KidSkills 
for  ages  11-14.  There  are  18  tools  in  First  Step  KidSkills  organized  in  the  categories  of:  Getting  Organized, 
Learning  New  Stuff,  Doing  Homework,  and  Doing  Projects.  There  are  30  tools  in  Second  Step  KidSkills 
organized  in  the  categories  of:  Getting  Organized,  Learning  New  Stuff,  Organizing  Information,  Preparing  for 
Tests,  Doing  Homework,  and  Doing  Projects.  Information  on  these  strategies  is  provided  in  a resource 
information  database  program.  Skill  Resources,  to  provide  educators  and  parents  resources  to  assist  children 
during  implementation.  A teacher  orientation  module  is  currently  being  developed,  including  a PowerPoint 
presentation,  demonstration  videos,  and  a web-based  hotline  for  assistance. 

The  software  contains  colorful  graphics,  text  -with-audio  directions,  and  simple  formats.  Graphic  characters 
serve  as  “guides”  to  the  different  tools  and  provide  audio  directions  in  children’s  voices.  The  text  and  audio  in 
KidSkills  allow  for  children  to  enter  text  in  their  own  words.  The  audio  directions  supplement  the  simplified  text 
instructions  and  can  be  turned  on  or  off.  To  use  the  tools,  the  child  simply  clicks  on  “hot  words”  on  the  template 
form  to  enter  his  or  her  content  and  then  prints  the  completed  form  for  use  in  the  classroom  or  home.  Children 
have  instant  access  to  examples  with  directions  and  flexible  ways  to  re-start  when  desired.  The  program 
automatically  enters  the  child’s  name,  date,  and  establishes  an  audit  trail  for  record-keeping  purposes.  These 
audit  trails  are  instrumental  in  tracking  the  development  of  children’s  thinking  and  skill  decisions. 

Figure  2 displays  the  main  menu  screen  for  Second  Step  KidSkills.  Children  select  tools  from  categories  by 
clicking  on  the  hot  areas  on  the  screen  or  by  using  the  pull-down  menu.  Figure  3 shows  an  example  of  a 
completed  Second  Step  tool.  In  this  example,  the  KWL  ^ows... Wants  to  know... Learned)  tool  provides  a 
learning  strategy  to  help  the  learner  focus  on  important  information  when  learning  new  material  and  integrate 
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new  information  with  his/her  current  information.  This  helps  the  learner  re-structure  knowledge.  The  child 
would  enter  the  following  information:  the  class,  topic,  source  of  information,  what  the  child  knows,  wants  to 
learn,  and  what  was  learned.  The  tool  could  be  used  prior  to  studying  to  guide  learning,  or  as  a follow-up  to 
review  information. 


Figure  2.  Main  Menu  from  Second  Step  KidSkills  Figure  3.  Sample  KWLTool 

User  Studies  in  the  Developmental  Process 

The  software  development  process  is  recursive,  going  through  several  phases  of  development-^  testing-^ 
revision  based  on  procedures  recommended  by  designers  of  children’s  software  (Druin,  1999)  and  evaluators  of 
interactive  learning  systems  (Reeves  & Hedberg,  2002).  The  first  step  of  formative  evaluation,  design  testing, 
includes  three  processes:  1)  review  of  content  and  interface  design  by  experts  in  learning  strategies  and 
children’s  software  design,  2)  observations  of  adults  working  with  the  prototypes,  and  3)  focus  group  meetings 
with  parents  and  educators  to  discuss  the  tools  and  consumer  training  needs.  The  second  step  of  formative 
evaluation,  usability  testing,  includes  three  processes:  4)  observations  of  children  using  a sample  of  the  tools 
while  collecting  “think-aloud”  transcripts,  5)  examination  of  tool  artifacts,  and  6)  expert  reviews  of  the 
completed  software.  These  six  developmental  steps  lead  to  a full  beta  testing  of  the  software,  training,  and 
support  modules. 

Expert  Review  of  Paper  Prototypes 

Paper  copies  of  all  the  tools  were  reviewed  by  eight  content  experts,  including  teachers,  administrators, 
university  personnel  and  a parent  of  a learning  disabled  child.  Each  reviewer  used  a structured  questionnaire  to 
provide  feedback  on  screen  design,  content  of  the  tools,  terminobgy  for  children,  and  other  literacy  features. 
Some  of  the  reviewers  had  prior  experience  using  the  KidTools  program  and  were  able  to  give  suggestions  for 
operability.  Based  on  these  reviews,  some  of  structural  elements  and  terminology  of  the  tools  were  changed  to 
make  them  more  understandable  by  children  with  learning  disabilities;  some  tools  were  re -conceptualized;  color 
coding  was  added  to  guide  entries;  and  two  new  tools  were  created. 

Design  Testing  in  a Lab  Setting 

An  open  lab  time  was  scheduled  at  a statewide  special  education  conference  for  educatora  and  parents  to 
“play”  with  the  KidSkills  prototype.  During  this  time,  two  graduate  research  assistants  staffed  the  lab  and 
observed  46  adults  using  the  software,  watching  for  difficulties  or  unusual  routines.  Field  notes  were  made  to 
record  comments  of  participants,  and  feedback  forms  were  collected.  Overall  ratings  were  8 to  9 on  a 9-point 
Likert  scale  (9  being  the  highest).  Positive  comments  were  that  the  program  would  be  fiin  and  easy  to  use,  the 
strategies  would  be  useful  for  students  with  learning  disabilities,  the  audio  directions  were  helpfiil,  and  they 
appreciated  the  interactivity  and  multisensory  approaches.  Some  potential  problems  identified  were  difficulty  in 
using  pull-down  menus  for  children,  desire  for  more  graphics  and  larger  hot  spots,  use  of  tab  key  to  save  text 
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entries,  desire  for  spell  checker  and  thesaurus,  and  navigation  back  and  forth  between  screens.  Problems  that 
were  identified  for  implementation  included  teacher  training,  scheduling  use  within  the  school  day,  access  to 
computers  at  home  and  school,  and  need  for  quick  instructional  resource. 

Online  Focus  Group  with  Consumer  Group  Members 

An  online  focus  group  was  held  with  members  of  the  target  consumer  groups,  including  one  parent,  three 
classroom  teachers,  three  teacher-trainers,  and  a high  school  student  with  learning  disabilities.  Project  staff  also 
participated  in  the  discussions.  Four  open-ended  questions  were  used  to  engage  participants:  (1)  acceptance  and 
use  of  the  software,  (2)  potential  implementation  problems,  (3)  recommendations  for  training  and  support,  and 
(4)  other  suggestions.  Discussion  threads  emerged  within  these  broad  areas.  The  messages  were  coded  for 
themes  using  the  qualitative  analysis  program,  NVivo.  With  this  text  analysis  program,  information  can  be 
linked  and  examined  across  multiple  sources,  leading  to  an  integration  of  the  results  (Richards,  1999).  Some  of 
the  concerns  that  were  raised  were  earmarked  for  observation  of  tool  usage  with  children,  particularly  use  of  the 
tab  key,  size  of  text  entry  fields,  quality  of  audio  narration,  and  navigation  problems.  Discussion  of 
implementation  issues  were  helpful  in  designing  the  orientation  module  and  planning  the  web  hotline. 

Think -Aloud  Observations  of  Children  Using  Working  Prototypes 

Twelve  children  with  mild-moderate  disabilities  were  selected  to  participate  in  usability  testing.  These 
students  all  had  learning  disabilities  or  behavioral  disorders,  grades  levels  27,  ages  8-14,  and  received 
academic  programming  in  resource  or  inclusive  classrooms.  The  selected  children  used  the  software  in  a one- 
on-one  setting  with  project  staff.  After  an  initial  training  period  with  one  of  software  tools,  children  were  asked 
to  “think  aloud”  as  they  worked  with  the  software  to  produce  materials  using  three  different  tools  (Smith  <& 
Wedman,  1988;  Someren,  Barnard,  & Jacobijin,  1994).  An  observer  audiotaped  the  sessions  and  made  field 
notes  in  order  to  record  the  children’s  reactions  to  the  interface  and  the  overall  operability  of  the  software  using 
observation  and  interview  guidelines.  These  results  were  analyzed  using iVF/vo. 

Examination  of  Artifacts  Created  by  Children  Using  Working  Prototypes 

The  artifacts  produced  by  the  children  were  examined  to  answer  any  questions  raised  during  earlier  stages 
of  review  or  suggestions  made  about  the  tools  during  the  Online  Focus  Group.  The  software  contains  a 
complete  audit  trail  that  records  out  to  files  all  tool  choices  and  entries  made  during  usage. 

Expert  Reviews  of  Completed  Software 

The  completed  software  tool  programs  will  be  sent  to  a panel  of  experts  in  July,  2002.  The  reviewers  will 
have  background  in  special  education  (cognitive-behavioral  interventions  and  learning  strategies)  and  two  of  the 
reviewers  will  have  expertise  in  human-interface  design  of  software  (EPSS  architecture  and  design  for 
children).  A systematic  interface  design  will  be  conducted  using  an  assessment  guidelines  based  on  procedures 
for  user  interface  studies  (Schneiderman,  1998).  The  guidelines  include  overall  reactions,  screens,  sound, 
graphics,  content,  terminology,  and  operability.  Reviewers  will  submit  a written  report  with  recommendations. 

Putting  the  Pieces  Together 

Although  each  formative  evaluation  procedure  yielded  valuable  information  and  useful  suggestions,  we 
found  summation  and  integration  of  the  information  to  be  most  critical.  As  discussed  by  Maslowski  <& 
Visschler,  all  the  possible  dimensions  of  formative  and  summative  evaluation  are  rarely  conducted  due  to 
financial  and  time  constraints  (1999).  Their  suggestion  is  to  evaluate  the  dimensions  where  designers  face 
questions  for  the  greatest  benefit.  It  is  easy  to  respond  to  each  procedure  with  revisions  and  further  testing,  as 
recommended  in  the  recursive  testing  process,  yet  this  approach  can  lead  one  down  false  paths  based  on  partial, 
rather  than  complete,  findings.  For  example,  adults  viewed  the  tools  and  raised  concerns  about  navigation  and 
the  use  of  the  tab  key,  yet  observers  of  children  found  that  with  very  little  guidance,  children  quickly  learned  to 
navigate  and  enter  information  into  tools.  Adults  questioned  the  clarity  of  the  children’s  voices  for  narration,  yet 
observers  reported  that  children  found  the  voices  appealing.  Adults  who  tested  the  software  found  it  easy  to  use, 
yet  our  high  school  student  strongly  recommended  a hotline  for  teachers  to  answer  computer-use  questions. 


It  was  clear  that  conflicting  feedback  was  provided  by  different  “voices”  of  expertise.  The  most  useful 
dimensions  to  us  were  the  direct  observations  of  children  using  the  tools  and  the  messages  to  the  Online  Focus 
Group  provided  by  our  high  school  student  with  learning  disabilities.  He  grasped  how  the  tools  could  be  used  in 
classroom  settings  and  the  abilities  and  limitations  of  teachers  in  supporting  roles.  Above  all,  we  learned  to 
integrate  the  voices  of  children  during  the  design  process  (Druin,  1999). 
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1.  Introduction 

Inquiring  databases  through  the  Web,  in  terms  of  visual  interface,  is  currently  an  active  area  of  research 
[15,16].  The  main  problem  is  to  provide  a user-friendly  interface  to  query  remote  database.  Most  systems  provide 
three  kind  of  querying:  text  search  engines,  form-based  interfaces  and  natural  interfaces.  An  alternative  way  to  query 
is  based  on  icons.  Starting  from  previous  works  [1,2,4,5,13],  a client-server  Iconic  Visual  Query  System  (IVQS), 
called  IVQS  Server,  has  been  developed.  This  system  allows  the  user  to  query  remote  and  different  multimedia 
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where  an  icon  represents  a query,  and  entity  icon,  where  an  icon  represents  an  entity  (a  table  in  a relational 
Database).  An  iconic  representation  of  a query  can  make  easier  for  the  end-user  or  the  inexpert  user  to  query  a 
remote  database  without  knowing  the  internal  structure  of  the  database  or  compiling  exhausting  forms.  Our  system 
has  been  used  in  education  fields  obtaining  good  results.  IVQS  Server  has  been  developed  as  an  applet  Java  on 
client  side  and  a Java  application  on  server  side,  so  it’s  completely  platform  independent  and  it  doesn’t  need  any 
installation  on  client  side. 


2.  System  Overview 

The  IVQS  Server  prototype  is  a Visual  Query  System  developed  at  ENEA's  Usability  Lab  to  access  and  query 
remote  databases  through  the  Web.  The  system  allows  to  create  and  execute  queries  to  remote  databases  and  to  have 
a multimedia  results  visualisation.  The  system  has  an  architecture  client-server  where  the  client  is  a Java  Applet  and 
the  server  side  is  a Java  application,  both  executable  directly  through  a Java-enabled  Web  Browser  (without  needing 
any  installation).  The  system  provides  two  kind  of  iconic  querying  (fig.l): 

• Query  icon.  Each  icon  represents  a query.  More  icons  can  be  grouped  inside  a folder  so  that  the  icons  are 
structured  in  a tree.  The  user  can  execute  the  query  double  clicking  on  the  icon. 

• Entity  icon.  Each  icon  represents  an  entity  (a  table  in  a relational  Database).  Between  two  or  more  entities  a 
query  can  be  associated.  Dragging  an  icon  entity  and  dropping  it  on  to  another  one,  the  associated  query,  if 
exists,  is  executed.  For  example,  if  there  are  the  entities  (tables)  DEPARTMENT  and  EMPLOYEE,  it  can 
be  associated  a query  that  is  the  join  on  the  common  field  DEPARTMENTJD. 

An  Icon  Editor  has  been  integrated  both  to  create  and  to  associate  an  icon  to  a query  or  to  an  entity.  In  this  way  the 
user  can  create  his  own  icon  or  import  an  existing  image  specifying  an  URL.  The  system  provides  a multi-user 
environment  with  user  identification/password  authentication.  Each  user  can  have,  in  a list  of  his  own  favourite 
queries  in  a specific  area.  Two  kind  of  user  are  handled: 

• Expert  User  {Super  User).  This  kind  of  user  can  manage  the  system  to  create  and  modify  queries,  entities 
and  can  set  up  the  result  display  layouts.  The  user  has  a good  knowledge  of  relational  databases  and  of  their 
structure.  The  user  can  create  queries  visually  selecting  databases,  tables,  attributes  and  conditions. 

The  system  can  execute  queries  to  the  most  known  relational  databases  such  as  Oracle  8.x,  9.x,  MS  SQL 
Server,  MS  Access,  Informix,  Sybase  and  ODBC/JDBC-compliant  data  sources).  In  particular,  choosing 
two  or  more  different  database,  he’s  able  to  build  a query  involving  tables  from  different  remote  RDBMS. 
The  user  can  use  the  system  by  himself  or  as  an  administrator  for  a set  of  end  users. 


• End  User  {User).  This  kind  of  user  uses  a structure  already  set  up  by  a super  user.  The  user  deals  with 
predefined  queries  and  entities.  Typically  he’s  neither  a computer  nor  a database  expert;  the  user  can  use  the 
system  without  knowing  the  internal  structure  of  the  databases.  The  user  can  have  his  own  set  of  favourite 
queries. 

Every  data  related  to  the  user  is  stored,  by  means  of  IVQS  server  side,  in  the  system  database. 

IVQS  Server  provides  two  kind  of  result  display:  table  and  form.  In  the  form  result  display  it’s  possible  to  customise 
the  visualisation  layout  in  terms  of  object  position,  font  type,  size,  style  and  colour.  Objects  can  be  text  labels, 
images,  Web  resource  links  (web  pages,  e-mail  address,  executable  files)  as  URLs.  In  the  latter  case  the  user  can 
open  the  resource  that  will  be  handled  by  the  browser.  Multimedia  objects,  such  as  movies  or  sounds,  are  placed  in  a 
specific  area  Media  Area. 


Typical  uses  of  IVQS  Server: 

• IVQSS  as  a visual  database  interface  for  the  end  user,  who  can  access  a set  of  remote  databases  in  a quick 
and  easy  way  using  predefined  queries  pre-built  by  the  administrator. 

• IVQSS  as  a tool  for  long  distance  learning,  both  for  the  administrator  who  can  manage  the  students 
database  and  for  the  students  as  an  interface  to  access  structured  information  collected  by  multimedia  cards. 
In  particular  with  IVQSS  applications  in  natural  science  and  materials  have  been  implemented. 


3.IVQS  SERVER  Architecture 

The  system  architecture  has  three  modules:  IVQS  Server,  IVQS  applet,  the  client,  that  is  stored  in  the  Server  and  the 
IVQS  Web  Site.  When  the  client  Web  browser  sends  the  request,  the  Web  Server  sends  back  the  applet,  which  starts 
being  executed  on  the  client.  The  applet,  because  of  its  security  limitations  [14,19,20],  can  open  socket  connections 
only  to  the  originating  Server.  We  need  a server  side  application,  which  can  access  remote  database  and  talks  with 
the  applet.  We  used  a Java  application  at  server  side,  it  can  access  various  kinds  of  RDBMS  (Oracle,  MS  Access, 
Informix,  etc.).  In  this  way  the  applet  communicates  with  the  application  server  to  send  query  and  to  get  back  results 
in  XML  format. 

After  the  first  generation  of  IVQS  [22]  has  been  implemented  the  IVQS  Server  release  with  the  following 
functionalities:  User  management.  User  data  management.  Query  management.  Query  Optimisation,  Asynchronous 
Query,  Query  Repository,  Query  Statistics,  Query&E-mail  Services  and  Dynamic  Web  Page  Services. 


4. Experimental  Results 

IVQS  Server  has  been  used  on  a set  of  existent  databases,  each  one  very  different  from  the  other  in  terms  both  of 
information  and  target  users.  Here  is  the  list  of  the  databases  where  IVQS  Server  has  been  applied: 

1.  FAD  Database  (Long  distance  learning),  ENEA  web  site  for  long  distance  learning  on  the  Internet. 

2.  G7  Global  Inventory  Project  (GIP).  An  Internet  inventory  of  G7  IT  projects.  The  target  users  are  Internet 
ones. 

3.  Prosoma  (PLUS),  EU  Web  site,  a multimedia  database  containing  the  most  relevant  project  results  in 
Europe. 

The  target  users  are  researchers,  enterprises,  technology  operators  and  the  EU. 

In  all  these  experiences,  IVQS  Server  has  been  tested  by  the  users  to  measure  its  Usability  [5].  The  results  obtained, 
in  terms  of  user  satisfaction  and  user  efficiency  [10],  showed  that  a query  iconic  representation  allows  a better 
perception  in  user's  mind  making  it  easier  to  memorise  [6,7,8]. 

5. Conclusions  and  future  work 

The  increasing  number  of  Web  users  is  looking  for  convenient  ways  to  search  information  on  the  Net  and  to  collect 
and  query  data  from  different  sources  through  the  Web  [2,12,16,18].  It’s  necessary  to  develop  new  alternative  tools 
for  such  tasks.  It  has  been  introduced  IVQS,  a Visual  Query  System  to  query  remote  database  through  the  Web 


using  an  iconic  interface,  developed  and  tested  at  ENEA  Usability  Lab  in  collaboration  with  University  of  Rome  "La 
Sapienza".  The  iconic  interface  that  is  alternative  to  the  form-based  web  pages  which  are  the  most  common  interface 
nowadays.  The  system  has  proved  that  an  iconic  interface  is  really  easier  to  use  for  large  end-user  groups.  As  future 
features  it's  planned  to  produces  an  increasing  on  multimedia  web  services  provided. 
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Abstract 

This  action  research  program  involved  implementation  of  distance 
education-based  developmental  studies  programs  for  mathematics 
using  PLATO  via  the  Internet  at  8 participating  institutions.  Twelve 
characteristics  were  found  to  be  in  common  over  the  most  successful 
of  the  programs.  We  present  the  twelve  characteristics  and 
summarize  overall  effectiveness  of  the  programs. 


The  Internet  and  distance  learning  hold  untapped  potential  to  change  the  culture,  curriculum,  and  course  of 
educational  institutions  around  the  world.  But,  most  distance  eductation  models  assume  highly  skilled  and  self- 
motivated  learners:  exactly  the  opposite  of  the  typical  developmental  studies  learner.  The  League  for  Innovation  in 
the  Community  College  and  PLATO'^  Learning,  Inc.  initiated  an  action  research  project  exploring  the  questions  and 
challenges  of  implementing  successful  distance  learning  developmental  math  programs  for  community  colleges 
across  the  country. 

College  Participants 

Selection  of  participants  from  League  colleges  and  League  Alliance  members  was  based  on  specific  commitment 
criteria.  Each  participating  college  was  asked  to  designate  two  faculty  members  and  commit  training  and  service 
time  to  research  and  program  development.  The  research  consortium  initially  included  nine  colleges;  however,  eight 
fully  implemented  PLATO  On  the  Internet*  (POI)  as  part  of  their  developmental  mathematics  program:  “ 

Central  Florida  Community  College,  Ocala,  FL 
Delta  College,  University  Center,  MI 
Kapiolani  Community  College,  Honolulu,  HI 
Kirkwood  Community  College,  Cedar  Rapids,  lA 
Moraine  Valley  Community  College,  Palos  Hills,  IL 
Miami-Dade  Community  College,  Miami,  FL 
Santa  Fe  Community  College,  Gainesville,  FL 
Sinclair  Community  College,  Dayton,  OH 

PLATO  on  the  Internet  Courseware 

PLATO  on  the  Internet  (POI)  courseware  is  a modular,  self-paced,  computer-based  learning  system  that  offers 
students  interactive  learning  opportunities  in  mathematics,  reading,  English,  and  core  work  skills  with  over  2,000 
hours  of  instructional  content  available.  As  a comprehensive  academic  and  applied  skills  courseware  system, 

PLATO  uses  computer-adaptive  integrated  learning  processes  for  student  assessment,  prescriptive  placement, 
interactive  instruction,  and  evaluative  testing  and  feedback. 

Project  Goals 

The  purpose  of  this  project  was  to  explore  critical  success  factors  for  computer-based  distance  learning  in 
developmental  math  programs  during  a summer  trial  implementation  session  and  a full  fall  semester  term.  College 
participants,  League  research  team  members,  and  PLATO  service  teams  outlined  four  areas  of  investigation: 

• Development  of  effective,  individualized,  open  entry/open  exit  programs  for  developmental  students  via 
distance  education 

• Cultivation  of  learners*  motivation  through  the  use  of  technology  in  developmental  studies  programs  using 
distance  education 


• Exploration  of  successful  developmental  student  profiles  using  distance  learning  technology 

• Effective  combinations  of  campus-based  support  service  and  distance  learning  delivery  systems  as  models 
of  success  for  developmental  learners 

Formative  Conclusions 

The  study  identified  these  critical  success  factors: 

• Easy  Access  to  Internet  and  Easy  Navigational  Courseware. 

• Technical  Support. 

• Alignment  of  Online  Courseware  and  Course  Objectives. 

• Individualized  Instructional  Format. 

• Student  Recruitment  and  Counseling. 

• Orientation. 

• Student  Connections 

• Faculty  Development. 

• High  Standards  of  Quality  and  Content  Development. 

• College  Leadership  and  Program  Support. 


Project  Outcomes,  at  end  of  the  study 


Total 

Participants 

(n=) 

Course  Completers 
(at  the  dose  of  the 
project) 

In-Progress 

Anticipated 
Success  Rate 

185 

89  (48%) 

27  (14%) 

116(62%) 
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the  consortium,  but  college  and  departmental  obstacles  prevented  implementation  of  the  PLATO  on  the  Internet® 
curriculum. 
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Abstract.  The  Electronic  Portfolio  initiative  in  the  College  of  Education  at  the  University  of  Florida 
represented  two  years  of  our  best  efforts  at  matching  the  current  research  on  portfolios  to  a 
systemic  project  for  all  future  teachers.  It  was  therefore  surprising  for  one  of  the  authors,  while  on  a 
Fulbright  to  the  University  of  Porto  in  Portugal,  to  find  an  independently  developed  electronic 
portfolio  project  that  had  so  many  things  in  common  with  the  project  we  developed  at  the 
University  of  Florida.  In  this  session,  rather  than  describing  the  projects  in  detail,  pointing  out 
similarities  and  differences,  we  have  decided  to  try  to  isolate  the  factors  that  caused  our  surprise. 

The  Electronic  Portfolio  initiative  in  the  College  of  Education  at  the  University  of  Florida  represented  two 
years  of  our  best  efforts  at  matching  the  current  research  on  portfolios  to  a systemic  project  for  all  future  teachers.  It 
was  therefore  surprising  for  one  of  the  authors,  while  on  a Fulbright  to  the  University  of  Porto  in  Portugal,  to  find  an 
independently  developed  electronic  portfolio  project  that  had  so  many  things  in  common  with  the  project  we 
developed  at  the  University  of  Florida.  In  this  session,  rather  than  describing  the  projects  in  detail,  pointing  out 
similarities  and  differences,  we  have  decided  to  try  to  isolate  the  factors  that  caused  our  surprise.  After  all,  it  isn’t 
everyday  that  two  independent  teams  of  research-aware  educators  come  to  the  same  conclusions  about  the  way  any 
educational  project  should  be  run!  Seriously,  we  believe  a brief  investigation  of  the  issues  facing  electronic 
portfolio  implementers  may  prove  informative.  We  have  worked  and  conferred  with  several  groups  who  are  either 
actively  working  with  electronic  portfolios  or  working  on  developing  an  electronic  portfolio  initiative.  We  have  also 
spoken  with  professors  who  have  lost  faith  in  portfolios  as  a meaningful  educational  experience.  Some  or  our 
anecdotal  experiences,  which  we  maintain  are  insights,  frame  the  views  that  follow. 

Mention  the  ^oxd portfolio  to  a room  full  of  educators  and  many  will  complete  the  phrase  with  the  word 
assessment.  Portfolio  assessment  - the  words  seem  to  be  inextricably  linked.  Yet  a vast  amount  of  research  related 
to  portfolio  initiatives  (Lyons,  1998,  Paulson  & Paulson,  1990,  Wolf,  1991)  is  skeptical  about  applying  any 
standardized  or  generic  assessment  procedures  to  the  evaluation  of  a student’s  portfolio.  This  is  due  in  part  to  the 
fact  that  unlike  an  assignment,  a portfolio  can  be  thought  of  as  belonging  to  the  student  and  maturing  over  time. 
Electronic  portfolios  can  be  maintained  and  updated  dynamically  over  time  and  there  are  many  reasons  to  promote 
portfolio  development  that  transcends  individual  courses  or  even  school  years.  By  encouraging  students  to  continue 
the  revision  of  the  work  they  have  done,  and  in  particular  to  revise  their  rationale  for  their  work  (as  in  the  case  of  an 
evolving  philosophy  statement,  for  example)  professors  directly  apply  research  findings  on  reflective  practice. 
Indeed,  if  portfolios  are  viewed  from  the  perspective  promoted  in  much  of  the  portfolio  research,  they  provide  an 
extremely  rich  environment  for  reflective  practice.  In  addition,  portfolios  can  be  used  to  make  connections  between 
disciplines  and  between  student  experiences  and  generally  accepted  accomplished  practices.  In  these  cases, 
connections  are  being  encouraged  in  a way  that  helps  a student  link  specific  scenarios  or  experiences  to  more  global 
practices.  Since  portfolio  building  is  the  construction  of  a virtual  representation  of  one’s  ideas  and  experience, 
connections  between  a student’s  work  and  the  larger  educational  themes  s/he  is  studying  make  sense.  We  would 
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argue  that  working  from  experience  to  more  general  theories  is  more  effective  than  studying  theories  waiting  for  an 
exemplar,  and  portfolios  encourage  the  former. 

Similarly,  we  feel  strongly  that  predefined  rubrics  and  checklists  limit  the  power  of  personal  portfolios.  If  a 
student  is  going  to  reexamine  positions  or  ideas  s/he  promoted  in  the  past  in  a meaningful  way,  can  that  evaluation 
be  scripted?  Can  it  be  scheduled?  How  does  one  evaluate  an  individual  student’s  portfolio  in  a generic  way? 

Despite  the  strong  connection  between  a portfolio  and  reflective  practice,  most  systemic  initiatives  require  a clearly 
defined,  almost  formulaic  way  of  communicating  the  assessment  process.  Consequently,  many  professors  begin  to 
create  checklists,  or  two  dimensional  checklists  and  rubrics.  Unfortunately,  description  becomes  prescription 
(Clark,  1979)  and  soon  students  begin  to  loose  ownership  of  the  portfolio,  then  they  loose  interest,  and  ultimately, 
the  portfolio  becomes  a typical  assignment.  If  portfolio  development  is  to  be  a personal  experience,  it  must  be 
evaluated  on  a personal  level,  if  it  is  to  be  evaluated  at  all.  Providing  rubrics,  in  addition  to  removing  student  level 
choices  about  portfolio  makeup,  can  easily  lower  the  bar  with  respect  to  quality  of  the  finished  product.  According 
to  Spiro  (1991),  “instead  of  retrieving  from  memory  a previously  packaged  ‘prescription*  for  how  to  think  and  act, 
one  must  bring  together  from  various  sources,  an  appropriate  ensemble  of  information  suited  to  the  particular 
understanding  or  problem  solving  needs  of  the  situation  at  hand.” 

On  the  other  hand,  in  order  for  a system  level  portfolio  initiative  to  work  effectively,  faculty  have  to 
understand  what  is  expected  of  students.  More  than  that,  faculty  should  be  instrumental  in  envisioning  what 
students  will  produce.  In  most  universities  it  is  difficult  to  coordinate  a system  wide  effort  with  subjective 
evaluation  procedures.  A..lthough  it  is  possible,  it  requires  setting  up  subsy stems  to  handle  a process  which  may 
include  time  consuming  interviews  and  quality  communication  between  staff  members.  It  is  much  easier  to  publish 
guidelines,  or  rubrics,  or  checklists  than  it  is  to  evaluate  in  context,  on  a personal  level.  Consequently,  most 
systemic  portfolio  initiatives  either  bow  to  the  difficulties  and  simplify  the  assessment  process,  or  set  up 
straightforward  procedures  from  the  outset.  Of  course,  this  “conspiracy  of  convenience”or  reductive  bias  (Spiro, 
1994)  is  related  to  larger  issues  involving  any  movement  towards  experiential  or  problem-based  learning  (Schon, 
1983).  According  to  Rams  den,  “if  enhancing  the  quality  of  student  learning  is  the  primary  goal,  it  is  imperative  to 
prevent  the  task  of  collecting  and  demonstrating  from  overwhelming  the  process  of  reflection  and  change.” 

Ramsden  goes  on  to  say: 

There  is  now  a widespread  view  in  academic  development  circles,  derived  directly  from  the 
student  learning  research,  that  we  should  concentrate  on  learning,  on  what  the  learner  does 
and  why  the  learner  thinks  he  or  she  is  doing  it,  rather  than  what  the  teacher  does  (Shuell, 

1986:  Biggs,  1990,1993).  And,  if  teaching  is  about  helping  to  make  learning  possible, 
assessment  becomes  defined  as  being  about  understanding  students  and  what  they  have  learnt. 

Effective  assessment  helps  students  develop  the  skills  of  self-assessment.... Luckily,  if  we  get 
the  improvement  part  right,  the  accountability  part  is  generally  sure  to  follow.  Good  evidence 
of  improvement  is  automatic  evidence  of  accountability  (Ramsden,  1994). 

In  this  light,  we  can  see  that  in  order  to  develop  a truly  student  centered  program,  professors  (or  project 
directors)  must  focus  on  improvement  in  student  performance  and  link  accountability  to  related  indicators.  In  other 
words,  professors  must  trust  themselves  and  their  students  to  do  quality  work.  It  is  the  acceptance  of  this  fact  that  is 
somewhat  rare,  and  it  is  this  we  found  in  the  portfolio  project  in  Porto.  There  was  general  agreement  about  the  idea 
of  nurturing  quality  over  time  and  avoiding  rigid  guidelines  “handed  down”  to  students.  We  both  agreed  that  our 
problems  had  to  do  with  the  difficulties  and  potential  of  a single,  particular  student  at  a specific  time,  to  say  the 
things,  to  discover  the  things,  to  communicate  the  things,  to  build  consensus  about  the  things  that  allow  and 
empower  that  particular  student  to  understand. 

It  is  often  difficult  for  professors  and  faculties  to  operationalize  this  student-centered  approach  even  though 
most  philosophically  embrace  it.  But  using  a student-centered  approach  to  portfolios  has  several  consequences  that 
directly  affect  learning  and  ultimately,  program.  For  example,  in  both  Florida  and  Porto,  rather  than  portfolio  entries 
being  selected  by  the  teacher,  students  select  their  own  entries  and  are  then  asked  to  provide  a rationale  for  their 
choices  that  explains  why  they  selected  the  work  to  be  included.  As  it  happens,  students  have  difficulty  providing 
this  rationale  (professors  from  other  universities  have  also  noted  this  difficulty).  As  a result  of  the  students* 
difficulty,  in  Florida  faculty  dialogues  took  place,  which  led  to  discussions  about  course  adjustments.  Also  student 
interviews  were  modified  to  focus  on  rationale  statements,  thus  providing  the  student  with  a forum  to  delineate  his 
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or  her  thinking.  We  embraced  and  promoted  what  Ehrmann  and  Lewis  (1997)  called  DIATing  (Doing  It  Again 
Thoughtfully).  Significantly,  during  the  design  phase  of  the  portfolio  initiative  in  Florida,  a large  percentage  of 
faculty  held  to  the  belief  that  professors  rather  than  students  should  specify  what  appears  in  the  portfolio.  If  that 
view  had  carried  the  design,  student  weaknesses  would  have  not  been  identified,  and  steps  to  address  the 
weaknesses,  that  is,  modifications  to  the  program,  would  have  never  been  made. 

Our  experiences  in  Florida  and  the  supportive  comments  made  by  professors  in  Porto  and  elsewhere  have 
strengthened  our  commitment  to  student-centered  education  based  on  the  development  of  an  electronic  portfolio. 
We  consider  the  portfolio  to  be  a perfect  vehicle  to  promote  the  transition  from  teacher  directed  to  learner-centered 
education. 
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Abstract:  This  paper  reviews  some  of  the  key  issues  of  what  we  believe  should  be  the  comer 
stone  of  a project -based  learning  approach  with  ICT  and  net -based  multimedia  learning 
resources.  It  refers  to  the  LAVA  Learning  project  where  pedagogic,  learning  resources  and 
computer-based  tools  have  been  developed  to  support  a complete  learning  environment.  The 
project  ended  up  creating  a new  project  tool  for  this  approach,  and  learning  material  and 
specialized  content  were  made  available  to  users  of  this  tool.  Content  of  all  media  types  was 
provided  a primary  focus  on  text  and  video  material.  Curriculum  from  different  content 
providers  was  linked  together  to  constitute  a source  of  learning  materials. 


Introduction 

In  Norwegian  public  schools  there  is  a considerable  focus  on  project  based  learning  as  it  is  required  by 
the  Norwegian  National  School  plan,  L97,  see  (L97).  The  schools  were  also  to  start  using  information  and 
communication  technology  (ICT)  in  all  subjects.  At  that  time  the  teacher  experience  was  low  in  both  areas, 
especially  ICT.  Internet  was  there,  but  net -based  learning  resources  lacked  and  so  did  broadband  networks. 

The  LAVA*  Learning  research  project  was  launched  trying  to  design  a prototype  learning  suite  consisting 
of  what  could  be  the  next  generation  net -based  curriculum  and  the  ne?t  generation  learning  technology  on  the 
premises  of  the  pedagogy  of  project  based  learning.  The  research  team  has  been  given  a total  funding  of  nearly  3 
million  US$  over  a three  year  period  to  determine  critical  factors  in  the  nexus  of  these  three  areas. 

At  the  start,  we  found  considerable  prior  work  in  pedagogy  in  the  literature,  some  on  appropriate  content, 
and  a large  body  of  work  in  technology  and  tools  for  ICT  use  in  schools  (with  links  to  pedagogy).  Much  of  the 
work  targets  mathematics  and  science.  However  we  didn’t  find  any  projects  where  the  three  areas  of  research 
were  conducted  synchronously. 

Our  focus,  and  the  purpose  of  this  paper,  is  the  combined  development  of  pedagogy,  net-based  curriculum 
for  project  work,  and  computer  technology  to  support  this  pedagogical  approach  in  the  humanities  and  social 
sciences  at  a school  level.  The  objective  of  the  work  has  been  to  create  a learning  environment  that  stimulates  the 
pupils  towards  explorative  discussions  in  a given  topic.  We  therefore  have  chosen  to  use  the  physical  arena  of 
collaboration  that  exists  in  the  schools  instead  of  creating  a virtual  community  on  the  net.  We  have  also  decided 
to  let  the  pupils  in  groups  of  4 share  the  main  computer  in  the  project  work  to  stimulate  discussion  and 
cooperative  creation  of  the  learning  projects. 

This  paper  presents  some  of  the  major  decisions  made  and  results  of  the  work  done  in  the  project  LAVA 
Learning’s  second  field  trial  (of  three).  We  start  with  the  pedagogical  approach  to  project  based  learning  and  its 
impact  on  content.  We  then  point  out  the  requirements  for  suitable  ICT  tools  that  we  found  to  be  a consequence 


* LAVA  is  a Norwegian  acronym  for  “delivery  of  video  over  ATM  networks”,  a collection  of  projects  with  a technology 
viewpoint  on  video.  The  ATM  is  left  now  focusing  on  learning  as  one  application  of  high  quality  video  streaming. 


of  the  approach.  Commonly  used  software  tools  were  evaluated  but  found  to  fall  short  or  our  requirements 
leading  us  to  develop  our  own  net- centric  multimedia  production  tool  called  “Slime”. 


Project  work  as  a pedagogical  approach 


As  already  indicated  the  pedagogic  focus  in  Norwegian  schools  has  turned  towards  more  use  of  project - 
based  learning  and  use  of  ICT  in  all  subjects.  As  discussed  by  Roschelle  et.al.  in  (Roshelle  00),  learning  is  a 
complex  task.  Pedagogic  approaches,  curriculum  and  its  use,  the  teacher’s  experience,  the  social  and  physical 
environment  and  several  other  factors  affect  how  children  learn.  Changing  one  factor,  like  infusing  PC’s  into  the 
classroom,  will  not  by  itself  dramatically  change  learning.  Getting  the  most  from  the  use  of  computers  should 
also  involve  changes  in  the  rest  of  the  learning  environment,  especially  the  teaching  methods  used  and  the 
curriculum.  In  the  endeavor  to  improve  learning  conditions,  one  should  also  pay  attention  to  results  from  recent 
cognitive  research,  for  instance  in  (Bransford  99).  It  has  been  shown  that  the  four  fundamental  characteristics — 
active  engagement,  participation  in  groups,  frequent  interaction  and  feedback,  and  connections  to  real-world 
contexts  enhances  the  learning  processes  significantly. 

In  project  work  learning  takes  place  during  the  creative  process  of  producing  a project  report  and  during 
the  discussions  among  the  pupils  when  they  are  engaged  in  doing  so.  The  pupils  formulate  a problem  statement 
to  be  investigated,  and  find  and  compile  information  that  can  be  used  to  elaborate  and  answer  the  principle 
questions  raised  by  the  topic  of  concern.  Learning  facts  and  relations  between  concepts  takes  place  as  the  pupils 
pose  questions,  confirm  or  reject  each  other’s  suggestions  and  explain  their  position.  The  teacher,  the  content 
available  to  the  students,  and  the  tools  used  to  compile  and  document  students’  efforts,  must  all  contribute  to 
keeping  the  right  focus  on  the  pupil’s  discussions.  When  focus  is  lost,  learning  will  be  less  effective  and 
subsidiary  or  irrelevant  threads  evoive. 

The  pedagogical  researchers  involved  in  LAVA  Learning  have  been  pro-active  in  designing  a learning 
scenario  that  incorporates  the  use  of  net -based  curriculum  and  computer  based  project  tools  in  a natural  way. 
The  scenario  based  upon  was  the  following: 

1.  Go  through  the  start  page  designed  by  the  project  (see  below)  and  follow  all  linked  material  that  you 
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find  intffesting. 

2.  Discuss  the  subjects  of  the  start  page  and  decide  upon  a main  problem  to  be  investigated  (had  to  be 
accepted  by  the  teacher). 

3.  Brainstorm  the  actual  subject 
identifying  possible  sources  of 
information. 

4.  Iteratively  build  an  electronic 
project  report  using  a 
combination  of  net -based  and 
self-produced  content. 

5.  Finalize  the  project  as  a class 
presentation. 

6.  Present  the  project  to  the  teacher 
and  the  rest  of  the  pupils  in  the 
class. 

To  give  the  pupils  a good  start  of  the 
project,  we  have  had  numerous  discussions 
of  how  the  start  page  should  be,  and  which 
role  it  should  play.  It  ended  with  a web-site 
providing  an  introductory  preamble  to  the 
theme  of  the  project  with  some  provoking 
elements  like  “why  don’t  Norwegians  eat 
dogs?” 

Figure  1 shows  the  start  page  of 
2001.  Each  of  the  pictures  in  the  collage  is  a 
link  and  leads  to  a short  video,  audio  file  or 
text.  Links  are  provided  at  the  bottom  of  the 
screen  to  material  prepared  by  content 


Figure  1 : The  start  page  of  the  project  work  can  be  seen  at 
http://lava.nr.no/undervisningsopplegg200 1 /startside/ 
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providers  participating  in  the  project  (described  in  the  next  sectioi^. 


Net-based  content 

Pupils  normally  define  the  problem  areas  of  their  work  in  groups,  not  necessarily  focused  on  traditional 
subject  boundaries.  We  observed  that  they  (11  - 15  years)  often  had  problems  finding  good  and  relevant  content. 
Their  sources  were  often  very  ad-hoc,  and  much  of  what  exists  is  in  English  and  often  written  for  adults,  see 
(Ludvigsen  01).  It  also  appears  to  us  that  the  material  often  supports  the  pupils’  problem  areas  very  poorly.  We 
found  a need  for  more  material  that  is  more  available  to  the  pupils  in  language,  in  actual  themes  and  so  on., 
without  being  a study  book  on  the  net. 

Norwegian  schools  are  using  the  Internet  intensively  in  project  work.  They  combine  this  with  printed 
materials  like  study  books,  lexical  books  etc.  Printed  materials  are  usually  very  fact  oriented  and  often  appear  to 
give  answers  to  all  relevant  questions.  Our  opinion  is  that  existing  syllabus  books  often  discourages  rather  than 
encourages  discussion  among  the  students  and  close  subjects. 

The  Internet  on  the  other  hand  is  completely  open.  Here  you  find  lots  of  mainly  textual  material,  but  the 
relevance,  organization  and  language  is  often  in  a form  not  suited  for  children  and  project  work.  There  are  some 
exceptions,  but  in  the  amount  of  relevant  material  in  Norwegian  was  found  to  be  limited. 

We  therefore  asked  our  content  providers  to  try  to  develop  the  next  generation  net -based  curriculum  as 
open  learning  sources  but  well  coordinated  and  targeted  on  “Norway  as  a multi  cultural  society”  with  main  focus 
on  food  and  culture  - food  and  religion.  The  material  was  developed  according  to  the  following  guidelines: 

• We  would  have  rich  media  content  — text,  pictures,  audio,  video  and  web  pages. 

• The  content  should  be  directed  towards  stimulate  curiosity  and  discussion  and  not  an  accumulation  of  facts. 

• Texts  should  be  concise,  structured  to  appeal  to  intuition,  and  should  not  follow  a book  like  structure. 

• Content  from  each  of  the  three  content  providers  (The  National  Library  in  Norway,  The  Norwegian 
Broadcasting  Corporation  and  Aschehoug,  a large  Norwegian  publisher)  should  be  structured  to  support  the 
materials  provided  by  the  others  within  the  selected  subject  and  should  be  mutually  linked. 

• All  content  should  be  available  in  a web -browser. 

• In  addition  to  the  content  providers  services,  audio,  video  and  pictures  should  be  available  for  editing  and 
manipulation  by  the  students. 

Project  tools 

In  project  work,  the  pupils  are  producers.  Learning  takes  place  as  an  integrated  and  iterative  process  of  a 
number  of  tasks;  Searching  (Internet,  books  aso.),  collecting,  editing,  creating  and  merging.  This  normally 
requires  several  computer  tools  to  support  the  students’  various  work  processes. 

In  the  first  field  trial  (of  three)  in  LAVA  Learning  see  (Ludvigsen  01),  we  used  a tool  called  Syncrolink, 
see  (Syncrolink),  in  combination  with  the  MS  Frontpage  html  editor  and  MS  Internet  Explorer.  Syncrolink  is  a 
tool  that  allows  the  user  to  annotate  videos  with  hyperlinks  to  web  pages.  In  the  trials,  the  pupils  used  Syncrolink 
to  see  the  video,  remove  the  parts  they  didn’t  think  was  relevant  and  add  links  to  web  pages  they  had  created  or 
that  was  found  on  the  Internet.  Their  work  process  evolved  to:  Find  the  interesting  places  to  put  links  in  the 
video  (using  Syncrolink),  create  or  find  web-pages  that  cover  the  issues  being  discussed  (using  Internet  Explorer 
or  Frontpage),  store  these  web-pages  and  finally  connect  the  links  using  Syncrolink. 

The  problem  with  this  mode  of  work  appeared  to  be  that  the  students  lost  track  of  their  contexts  and  hence 
that  the  different  parts  of  what  they  produced  did  not  harmonize  very  well. 

Another  problem  that  was  observed  in  this  first  trial  was  that  the  children  lacked  a placeholder  for 
interesting  material  they  came  across  on  the  Internet.  Adding  the  URL’s  to  the  Favorites  list  in  the  browser  only 
solved  the  problem  on  a specific  machine.  In  cases  where  the  pupils  used  different  computers  for  content 
collection,  they  had  problems  transferring  correct  URL’s.  Solving  the  problem  involved  writing  down  the  URL 
or  iBing  a third  tool  to  type  it  in,  and  store  this  on  a networked  disk. 

In  studying  the  pupil’s  work  we  were  convinced!  Tools  can  never  be  neutral  in  this  context.  Inappropriate 
tools  will  easily  distract  the  pupils’  attention.  Our  opinion  is  that  supporting  a good  working  process  is  the  best 
way  the  tool  can  contribute  to  good  learning  processes. 

As  for  MS  Frontpage,  results  (we  took  lots  of  video  of  the  pupils  during  the  project  work)  show  that  the 
pupils  spent  lots  of  time  discussing  the  appearance  of  their  web  pages  rather  than  discussing  the  content  and 
subject  matter. 


In  collecting  preferable  properties  for  a project  tool  we  also  wanted  the  ability  to  handle  all  media  types  in 
a transparent  manner  being  it  net -based  or  locally  produced.  Within  the  actual  subject  of  the  pupils  project  work, 
the  Norwegian  Broadcasting  Corporation  and  the  National  Library  had  lots  of  relevant  video  material  that  they 
wanted  to  make  available  over  the  Internet.  They  wanted  to  experiment  with  new  net -based  services  targeted  for 
schools.  TV  is  quite  a different  medium  from  video  on  the  web,  and  TV  programs  are  self-contained  and  have  a 
longer  duration  than  acceptable  as  part  of  a multimedia  presentation.  We  needed  to  provide  mechanisms  for  the 
children  to  include  only  those  parts  of  the  video  that  were  relevant  to  their  field  of  inquiry. 

Experience  from  early  trials  of  the  LAVA  project,  showed  that  when  students  edited  a selected  video 
from  30  to  about  5 minutes,  the  exercise  gave  them  a much  deeper  grasp  of  the  subject  matter  in  the  video.  In 
later  trials  we  wanted  to  preserve  this  ability. 

Permissions  to  do  this  editing  of  copyrighted  material  were  needed  and  given  by  the  content  owners.  We 
wanted  to  test  our  earlier  observations  that  the  editing  process  led  to  better  understanding  of  the  video  content. 
To  avoid  copying  and  changing  the  original  video  material,  and  avoiding  having  to  copy  and  handle  large  video 
files,  the  video  editing  needed  to  be  carried  out  on  streamed  video  with  caching  capabilities. 

We  started  looking  for  a convenient  project  tool  with  a clear  focus  on  the  need  for  support  of  the  pupils 
work  processes.  Having  evaluated  easily  available  tools  like  MS  PowerPoint  with  MS  Media,  Macromedia 
Dreamweaver,  Real  and  others,  we  found  that  none  provided  good  support  for  our  requirements.  Having  the 
resources  of  developing  our  own  tool,  we  decided  to  create  what  became  the  Slime  tool  shown  in  figure  2 and  3. 

The  main  approach  behind  Slime  was  to  provide  a set  of  canvases  or  scenes  where  all  media  types  could 
be  collected  and  edited.  Since  the  scenes  needed  to  handle  both  static  and  time- varying  media  types,  we  decided 
to  tag  all  media  types  with  time  tags.  This  meant  that  in  playback  mode,  a text,  a webpage  and  a picture  would 
all  be  assigned  a time  slot  when  each  object  was  displayed  on  the  canvas,  and  a time  where  it  each  object  would 
disappear. 

In  Slime  content  objects  are  displayed  in  parallel  or  in  sequence  or  overlapping  in  time  during  playback 
of  edited  material.  The  contents  of  a “scene”  can  be  played  as  a synchronized  whole.  When  all  media  objects  in  a 
scene  have  been  played,  the  tool  jumps  to  the  next  “scene”. 

To  navigate  between  the  scenes  of  a project,  users  can  use  both  the  scene  graph,  the  navigation  box  to  the 
lower  left,  and  the  overall  timeline  shown  on  top  of  the  tool.  The  scene  graph  is  a graphical  display  of  the  object 
names  and  structure  of  scenes. 

Students  can  easily  switch  the  search  modus  of  the  tool  by  clicking  on  the  switch  button.  The  search  mode 
allows  the  user  to  search  the  Internet  using  a standard  web-browser,  or  collect  local  files  or  search  in  given 
content  providers’  databases  for  annotated  material.  Interesting  materials  found  can  be  collected  and  included  in 
a given  scene.  Material  collated  in  this  fashion  can  be  thereafter  edited  and  synchronized  with  the  other  content 
objects. 
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Figure  2:  The  Slime  tool  in  production  modus 
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A special  object  model  underlying  the  Slime  tool  that  makes  it  possible  for  content  providers  to  control 
content  distribution  and  use  after  it  has  left  the  server.  These  security,  IPR  and  copyright  issues  are  beyond  the 
scope  of  this  paper,  but  are  documented  in  (Diesen  00). 

The  snapshots  in  figures  2 and  3 indicate  some  of  the  functionality  of  the  Slime  tool.  For  a more  detailed 
description,  please  see  (Slime), 


Results  and  discussion 

To  document  the  school  trials,  researchers  have  been  in  the  classrooms,  and  about  40  hours  of  video  of 
pupils  working  together  using  the  Slime  tool  are  available.  The  pedagogic  results  will  be  comprehensively 
documented  in  papers  by  other  researchers  in  the  LAVA  Learning  project,  and  will  be  available  on 
http://www,nr,  no/lav  a/1  ava-le/  as  they  are  published.  We  will  focus  here  on  some  of  the  content  and  tool  issues. 

We  found  that  the  Slime  tool  was  able  to  support  the  entire  project  oriented  learning  process  of  the  pupils. 
Many  of  the  pupils  started  their  production  by  creating  a structure  of  empty  scenes.  This  worked  as  a way  of 
storing  their  common  understanding  of  what  their  project  would  result  in.  Project  structure  was  frequently 
revisited  and  discussed  during  the  project  work.  This  showed  us  that  this  kind  of  mutable  structuring  element 
provided  good  work  support.  It  generated  considerable  discussion  about  the  organization  of  their  material  for 
example  by  serving  as  a focal  point  to  identify  what  was  lacking  and  so  on. 

Despite  the  excellent  support  of  student  work-processes,  we  clearly  saw  that  Slime  is  a poorer 
presentation  tool  than  PowerPoint.  We  had  observed  problems  with  timing  of  the  display  of  text  boxes.  This 
indicates  the  need  for  a “proceed”  button  or  a revision  in  our  approach  to  the  time  model. 

What  we  also  found  was  that  material  made  available  directly  into  the  project  tool  worked  well  and  was 
used  to  some  extent,  but  the  video  material  not  was  sufficiently  annotated.  The  pupils  had  to  go  through  what 
they  assumed  to  be  relevant  in  order  to  evaluate  it,  and  the  annotations  did  not  give  a sufficiently  accurate 
description  of  video  content.  Lots  of  pupils  ended  up  omitting  the  use  of  video,  or  just  using  privately  produced 
video.  In  the  next  field  trial  (the  third  and  last  of  LAVA  Learning),  we  will  have  to  annotate  the  videos  with  a 
higher  granularity.  This  will  ensure  that  search  results  will  turn  up  only  the  relevant  parts  of  a video. 

We  also  observed  a problem  with  content  resources  in  general.  The  problem  focus  of  the  different  project 
groups  of  students  was  very  broad.  Teachers  will  either  have  to  restrict  focus,  or  huge  amounts  of  content  must 
be  made  available.  Students  found  lots  of  material  on  the  Internet  on  relatively  ad  hoc  web  sites,  and  the  content 
was  used  relatively  uncritically  with  regard  to  the  sources.  It  is  obvious  to  us  that  there  is  a pressing  need  for 
more  content,  and  an  educational  emphasis  in  critical  evaluation  of  sources  available  on  the  Web  is  very 
important 
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Figure  3:  The  Slime  tool  in  search  modus 
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Conclusions 

We  feel  that  we  have  managed  to  combine  research  in  the  three  areas  of  pedagogy,  content  production 
and  technology  to  create  a good  working  learning  environment  using  a project  based  learning  approach.  The 
main  emphasis  is  on  the  pedagogy,  which  defined  the  framework  for  a project  tool  that  would  support  effective 
learning.  The  issue  was  to  keep  student  focus  on  the  subject  matter  of  the  project  and  not  on  the  underlying 
technology. 

Our  observations  from  field  studies  indicate  that  a specially  designed  project  tool  like  Slime  enables 
students  to  keep  their  focus  on  the  subject  matter  to  a much  greater  extent  than  the  traditional  off  the  shelf  tools. 
Integration  of  search  and  production  facilities  turned  out  to  be  important  to  the  learning  process. 

The  content  providers  of  LAVA  Learning  gave  access  to  their  raw  material,  which  enabled  us  to  link  it 
directly  into  the  project  tool  of  the  pupils.  This  turned  out  to  be  successful,  but  the  amount  of  available  and 
structured  content  for  educational  needs  will  need  to  be  provided  to  keep  up  with  demand  for  electronically 
available  material.  It  is  of  great  importance  to  get  reliable  sources  of  material  available  on  the  Internet  for  the 
children  working  with  an  approach  like  project  based  learning. 

Do  children  learn  better  in  this  way?  This  really  is  a question  with  no  simple  answer.  What  we  have  seen 
is  that  motivation  increases  a lot.  Some  children  would  hardly  leave  school.  On  the  other  hand,  we  believe  that  a 
good  learning  result  requires  well  functioning  project  groups  and  a good  teacher.  The  teacher  needs  to  help  the 
groups  in  finding  good  problems  to  work  with  and  in  testing  the  relevance  and  quality  of  the  material  they  use. 
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(Diesen  00)  Diesen  D.  and  Oskal  A.:  Using  object  oriented  information  distribution  to  present  and  protect 
information.  Advances  in  infrastructure  for  Electronic  Business,  Science,  and  Education  on  the 
Internet,  L'Aquila,  Italia,  6.  - 12.  August  2001 

(Slime)  Description  of  Slime  can  be  found  in:  http://www.nr.no/lava/lava-le/slime/ 
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Abstract 

This  presentation  provides  guidance  on  the  issue  of  universal  web  accessibility  for  individuals  with 
physical  challenges  who  need  accommodations  to  fully  participate  in  the  electronic  world.  Four 
categories  of  information  facilitate  understanding  the  complexity  of  the  issues.  These  categories 
are: 

• U.S.  legal  mandates; 

• International  accessibility  policy  and  legislation; 

• Resources  and  information;  and 

• Web  site  accessibility  validation. 

These  four  are  explored  in  depth  and  provide  the  tools  for  meeting  today’s  challenges.  As  the  Web 
evolves,  new  accessibility  issues  will  emerge.  Once  you  think  you  have  it  figured  out,  it  will 
change,  requiring  continual  self-study  to  keep  abreast  of  the  how  these  changes  effect  your 
situation.  Helpful  resources  can  be  accessed  from  the  following  site:  Http://www.e- 
linkagesgIobaI.com 


Introduction. 

This  presentation  provides  guidance  on  the  issue  of  universal  web  accessibility.  Lack  of 
information  concerning  web  accessibility  not  only  contributes  to  the  digital  divide,  but  can  lead 
to  serious  legal  problems  as  well.  Lawsuits  can  arise  because  of  U.S.  legislation  that  required  all 
web  sites  receiving  federal  funds  to  meet  accessibility  directives  by  the  spring  of  2001.  The  goal 
of  this  paper  is  to  provide  clarification  of  the  accessibility  statutes  and  offer  resources  and 
information  to  assist  in  successfully  developing  accessible  web  sites. 

The  circular  process  of  comparing  and  contrasting  information  from  the  Internet,  expert 
testimony  and  personal  expertise  were  the  primary  components  fueling  the  engine  of  this 
qualitative  inquiry.  It  goes  without  saying  that  constant  and  rapid  change  makes  it  extremely 
difficult  to  study  any  phenomenon  on  the  Internet. 

Many  web  developers  are  ignoring  the  accessibility  issue  or  just  do  not  know  how  to 
make  their  sites  universally  accessible.  They  are  bewildered  in  the  face  of  the  vast  amount  of 
information  available  on  applying  accessibility  techniques  to  the  design  of  their  web  sites. 

Four  categories  of  information  facilitate  understanding  the  complexity  of  the  issues. 
These  categories  are: 

• U.S.  legal  mandates; 

• International  accessibility  policy  and  legislation; 

• Resources  and  information;  and 

• Web  site  accessibility  validation 

These  four  areas  provide  the  tools  for  meeting  today’s  challenges.  As  the  Web  evolves, 
new  accessibility  issues  will  emerge.  Once  you  think  you  have  it  figured  out,  it  will  change, 
requiring  continual  self-study  to  keep  abreast  of  the  how  these  changes  effect  your  situation. 

The  Legal  Mandates  for  Accessible  Internet  Sites 

The  Americans  and  Disability  Act  (ADA),  which  was  signed  in  1990,  prohibits 
discrimination  “on  the  basis  of  disability  in  employment,  programs  and  services  provided  by 
state  and  local  governments,  goods  and  services  provided  by  private  companies  and  in 
commercial  facilities.”  However,  the  ADA  doesn't  just  apply  to  the  physical  world.  Waddell 
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(1998)  points  out  that  according  to  the  Justice  Department,  the  ADA  also  applies  to  cyberspace. 
In  an  opinion  letter  dated  September  9,  1996,  The  Justice  Department  stressed  that  covered 
entities  under  the  ADA  are  required  to  provide  effective  communication,  regardless  of  whether 
they  generally  communicate  through  print  media,  audio  media  or  computerized  media  such  as 
the  Internet.  Covered  entities  that  use  the  Internet  for  communications  regarding  their  programs, 
goods  or  services  must  be  prepared  to  offer  those  communications  through  accessible  means  as 
well. 

Section  508  of  the  Rehabilitation  Act  Amendments  of  1999  (http://www. access- 
board. gov/sec508/nprm.htm)  provided  the  basis  for  the  Electronic  and  Information  Technology 
Accessibility  Standards,  enacted  August  7,  1998.  "Effective  (two)  years  after  the  date  of 
enactment,  any  individual  with  a disability  may  file  a complaint  alleging  that  a federal  agency 
fails  to  comply  with  section  508  in  providing  accessible  electronic  and  information  technology.” 
Camevale  (1999)  notes:  "As  colleges  and  universities  expand  their  distance- education 
programs,  they  are  finding  that  they  must  include  the  virtual  equivalent  of  ramps  when  building 
their  website.  Higher  education  institutions  know  they're  obligated.  "It's  not  that  web  site  creators 
are  ignoring  the  accessibility  issue,”  says  Jane  Jarrows,  president  of  Disability  Access  and 
Information.  “It  is  that  they  don't  always  realize  how  important  accessibility  is."  While  the  U.S. 
Department  of  Education's  Office  for  Civil  Rights  (OCR)  has  specific  guidelines  for  compliance 
on  traditional  campuses,  the  agency  has  not  yet  issued  accessibility  rules  for  online  education. 

United  States  Legal  Cases  Related  to  Education 

According  to  Waddell; 

Not  surprisingly,  web  accessibility  issues  are  now  being  faced  by  educational 
institutions.  Library  reference  services  are  being  transforrhed  by  the  efficiency  of 
Internet  access  to  information  systems  and  search  engines.  Professors  are  teaching 
long  distance  learning  courses  over  the  Internet  and  even  if  a student  is  physically 
in  class,  homework  assignments  and  resources  are  being  posted  on  class 
homepages.  Yet,  even  if  a library  terminal  has  assistive  computer  technology 
installed  for  students  with  disabilities,  Internet  research  by  students  with 
disabilities  is  not  possible  with  inaccessible  web  page  design. 

A blind  student  from  Long  Beach  California  filed  a complaint  to  the  Office  of  Civil 
Rights  (OCR).  The  university  provided  OCR  with  a volimtary  resolution  plan,  which  resolved 
the  issues  raised  in  this  case.  The  plan  included  the  following  commitments; 

1 . )  Develop  and  implement  a systematic  method  for  ensuring  that  the  issue  of 
accessibility  to  persons  with  disabilities,  particularly  blind  persons,  is  taken  into  account  when 
colleges  purchase  computer  software  and  hardware. 

2. )  Develop  and  implement  a systematic  method  for  informing  campus  employees  who 
design/select  web  pages  for  use  by  students  to  make  sure  the  web  pages  are  in  accordance  with 
principles  known  to  maximize  accessibility  to  users  with  disabilities,  including  visual 
impairments 

Waddell,  1998,  provides  more  information  about  this  case  by  citing  Brummel’s  1994 
white  paper,  “ The  Information  Infrastructure:  Reaching  Society’s  Goals  Report  of  the 
Information  Infrastructure  Task  Force  Committee  on  Applications  and  Technology  Published  by: 
U.S.  Department  of  Commerce.” 


. .(T)he  issue  is  not  whether  the  student  with  the  disability  is  merely  provided  access, 
but  the  issue  is  rather  the  extent  to  which  the  communication  is  actually  as  effective  as  that 
provided  to  others.  Title  II  (of  the  Americans  with  Disabilities  Act  of  1990)  also  strongly  affirms 
the  important  role  that  computer  technology  is  expected  to  play  as  an  auxiliary  aid  by  which 
communication  is  made  effective  for  persons  with  disabilities”  (Pages  1-2,  1996  Letter;  28 
C.F.R.  35.160(a))  (Waddell,  1998  citing  Brummel,  1994). 

In  further  clarifying  what  is  meant  by  “effective  communication,”  OCR  has  held  that  the 
three  basic  components  of  effective  communication  are  “timeliness  of  delivery,  accuracy  of  the 
translation  and  provision  in  a maimer  and  medium  appropriate  to  the  significance  of  the  message 
and  the  abilities  of  the  individual  with  the  disability”  (Page  1,  1997  Letter)  (Waddell,  1998, 
citing  Brummel,  1 994). 

On  June  25,  2001,  the  accessibility  requirements  for  U.S.  electronic  and  information 
technology  took  effect  under  Section  508.  This  law  mandates  that  this  technology  be  accessible 
according  to  standards  developed  by  the  Access  Board,  which  are  now  part  of  the  federal 
government’s  procurement  regulations. 

Section  508  and  its  enforcement  provisions  apply  to  products  procured  by  U.S.  federal 
agencies  after  June  25,  2001.  This  law  relies  heavily  on  the  procurement  process  to  miake  sure 
there  is  compliance  with  the  new  standards.  Compliance  with  the  standards  is  required  unless  it 
would  pose  an  "undue  burden" — as  defined  in  the  standards — or  if  no  complying  product  is 
commercially  available.  The  law  permits  individuals  with  disabilities  to  file  a complaint  with  the 
appropriate  federal  agency  concerning  access  to  products  procured  after  the  effective  date. 

International  Accessibility  Policy  and  Legislation 

Accessibility  issues  are  considered  to  be  one  of  the  “target  areas  for  equal  participation” 
at  an  international  level.  Late  in  1993 — at  the  infancy  of  the  World  Wide  Web — the  United 
Nations  General  Assembly  adopted  the  Standard  Rules  on  the  Equalization  of  Opportunities  of 
Persons  with  Disabilities  [1].  Rule  5 of  the  Standard  Rules  addresses  accessibility  in  terms  of  the 
physical  environment  and  with  reference  to  information  and  communications  services.  Among 
other  points.  Rule  5 recommends:  'States  ...  develop  strategies  to  make  information  services  and 
documentation  accessible  for  different  groups  of  persons  with  disabilities." 

Although  not  a legally  binding  instrument,  the  Standard  Rules  represent  a strong  moral 
and  political  commitment  of  governments  to  take  action  to  attain  equalization  of  opportunities 
for  persons  with  disabilities.  The  rules  serve  as  an  instrument  for  policy  making  and  as  a basis 
for  technical  and  economic  cooperation.  In  the  past  few  years,  a special  rapporteur  appointed  by 
the  U.N.  Secretary  General  to  monitor  the  implementation  of  the  Standard  Rules  has  encouraged 
governments  to  consider  accessibility  of  information  and  communications  services  within  their 
country’s  social  development  policies. 

The  unique  governance  structure  of  the  Internet  makes  it  difficult,  if  not  impossible,  for  a 
country  to  impose  accessibility  legislation  on  its  own  citizens.  For  example,  it  is  not  easy  for  one 
government  to  impose  penalties  for  poor  accessibility  in  a web  site  of  one  of  its  own  citizens  if 
the  web  site  is  hosted  in  another  country.  Governments  have  used  either  policy  or  limited 
legislation  in  an  effort  to  ensure  that  public  information  is  an  accessible  public  good. 

Policies  that  encourage  accessibility — stating  its  benefits  to  the  producers  and  consumers 
of  information — are  used  by  most  of  the  early  adopters  of  the  importance  of  information 
accessibility.  These  policy/guidelines  can  only  implement  minor  penalties  such  as  the  notoriety 
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of  non-conformance.  Australia,  Canada  and  Portugal  have  issued  policy  statements,  and  have 
mostly  limited  policy  application  to  their  own  federal  governments. 

A good  example  of  a clear  and  concise  policy  statement  comes  from  the  "Guide  to  the 
Internet"  of  the  Government  of  Canada.  It  states: 

"Since  the  end  user  carmot  count  on  either  standard  tecluiology  or  helping  devices  to 
ensure  access  to  information  on  the  [World  Wide]  Web,  the  onus  is  on  the  web  page 
developer  to  deliver  the  message  in  a way  that  allows  everyone  to  benefit. 

"It  is  every  Canadian's  right  to  receive  Government  information  or  service  in  a form  that 
can  be  used,  and  it  is  Government  of  Canada's  obligation  to  provide  it." 

Only  the  U.S.  has,  so  far,  turned  information  accessibility  into  legislation.  It  does  so 
using  both  “push”  and  “pull”  regulations.  "Push"  regulations — such  as  Section  255  of  the 
Telecommunications  Act  of  1996 — require  industry  to  consider  accessibility.  This  type  of 
regulation  is,  however,  only  effective  in  production  of  accessible  products  such  as  kneeling  buses 
and  accessible  teller  machines.  It  is  difficult  to  regulate  delivery  of  online  information  using 
“push”  regulation  because  World  Wide  Web  information  is  available  globally,  and  there  are 
many  countries  that  do  not  consider  information  accessibility  an  issue. 

Section  508  of  the  Rehabilitation  Act  is  an  example  of  "pull"  legislation  because  it 
requires  that  the  government  purchase  accessible  products,  but  does  not  require  that  industry 
produce  them.  This  provides  a market  pull  to  industry.  Obviously,  industry  prefers  pull 
regulations  to  push  regulations.  In  practice.  Section  508  may  also  be  more  effective  because 
suppliers  stand  to  lose  major  federal  contracts  if  they  are  perceived  not  to  adopt  corporate 
accessibility  policies.  By  imposing  accessibility  regulation  on  its  own  ranks,  the  U.S.  federal 
government  is  hoping  that  corporations  will  adopt  accessibility  policies  as  a matter  of  example 
and  as  a sign  of  good  corporate  citizenship. 

The  European  Union  seems  convinced  that  “pull”  regulation  and  standardizing  on  WAI  is 
their  best  option.  In  its  Europe  Action  Plan,  the  E.U.  provides  a special  section  titled  “e- 
accessibility:  Participation  for  all  in  the  knowledge -based  economy.”  This  action  plan 
optimistically  targets  the  year  2002  as  the  deadline  for  all  member  countries  to  make  their  federal 
websites  follow  the  WAI  Content  Accessibility  Guidelines. 

In  Canada,  federal  websites  have  mostly  complied  with  the  Treasury  Board  Secretariat's 
Common  Look  and  Feel  (CLF)  Guidelines.  The  successful  implementation  probably  makes 
Canada's  web  sites  the  most  accessible  among  industrialized  nations.  But  the  Treasury  Board  is 
not  satisfied.  The  CLF  Guidelines  are  currently  being  used  in  the  formulation  of  legislation 
similar  to  Section  508,  and  with  a compliance  date  of  December  31,  2002.  The  difference 
between  U.S.  regulations  and  the  Canadian  and  E.U.  regulations  is  that  Canada  and  the  E.U.  are 
adopting  the  W3C  Web  Accessibility  Initiative  as  a reference  material  to  keep  with  new 
technologies  and  developments.  This  move  will  likely  be  the  same  approach  as  other  countries 
determine  how,  if  ever,  they  formulate  information  accessibility  policy. 

Unfortunately,  many  countries  may  never  have  to  deal  with  web  accessibility  for  mostly 
economic  reasons.  One  reason  is  that  country  priorities  for  information  and  communication 
services  will  naturally  begin  with  the  telecommunications  infrastructure  and  the  cost/benefit 
analysis  of  various  platforms.  For  example,  the  e-Mexico  project  concentrates  on  making 
telecommunications  facilities  available  to  the  rural  areas,  and  the  term  accessibility  is  used  to 
refer  to  the  access  of  these  facilities. 
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Another  reason  web  accessibility  might  not  be  adopted  is  that  in  countries  where  labor 
costs  are  low,  care  giving  for  persons  with  disabilities  is  the  norm,  and  independent  living  is  not 
common.  Caregivers  can  be  asked  to  provide  other  services  such  as  reading  books  and  online 
information. 

Verification  of  Web  Site  as  Universally  Accessible  and  Related  Services 
Web  pages  need  to  be  validated  to  ensure  they  meet  the  minimum  requirements  for 
universal  accessibility.  There  are  two  major  validation  services — available  free  online — which 
evaluate  web  pages  and  help  identify  potential  accessibility  problems  to  correct.  The  first  is 
Bobby,  which  is  part  of  the  Center  for  Applied  Special  Technology  (CAST),  a non-profit 
organization,  whose  mission  is  to  expand  opportunities  for  individuals  with  disabilities  through 
the  development  of  innovative  technology.  The  Bobby  validation  system  is  simple  to  use  to 
validate  a web  site.  The  web  site  URL  is  submitted  by  entering  the  information  onto  the  web 
page  and  then  clicking  a submit  button.  Http://www.Cast.org/ 

The  process  for  obtaining  the  report  is  easy.  Copy  and  paste  the  URL  to  insert  into  the 
verification  program.  Copying  and  pasting  keeps  from  making  a typing  error.  Tip:  If  a Windows- 
based  computer  is  used,  select/highlight  the  URL  and  use  "Ctrl”  and  “C"  keys  to  copy  the  URL 
and  then  "Ctrl”  and  “V"  keys  to  paste  the  URL  to  the  address  box.  Then  submit. 

Web  sites  should  have  at  least  a Priority  One  Accessible  rating.  Wlien  users  get  feedback 
from  the  site,  they  will  be  able  to  see  the  entire  page.  Tiny  question  marks  by  the  images  means 
for  users  to  check  their  information  manually — this  will  not  stop  those  from  getting  validated.  If 
major  problems  are  experienced,  a question  mark  appears  by  each  image. 

W3C  Validation.  The  second  major  validation  site  is  W3C’s  HTML  validation  service,  a 
free  service  that  checks  documents  like  HTML  and  XHTML  for  conformance  to  W3C 
recommendations  and  other  standards,  http://validator.w3.org/ 

Approval  Icon.  The  NCAM  accessibility  icon  was  selected  from  17  symbols  as  a Web 
Access  Symbol  for  people  with  disabilities.  Web  masters  can  use  this  to  denote  their  site  contains 
accessibility  features  for  disabled  users.  The  symbol  should  always  be  accompanied  by  its 
description— a globe,  marked  with  a grid,  tilts  at  an  angle;  a keyhole  is  cut  into  its  surface — and 
alt-text  tag,.  This  image  was  created  by  Stromship  Studios  of  Boston.  There  is  no  charge  to  use 
this  symbol,  and  it  may  be  used  in  electronic  or  printed  form.  It  can  be  copied  from  the  NCAM 
web  site  and  pasted  it  into  a document,  http://ncam.wgbh.org/accessncam.html 

Conclusions 

"The  difficulties  of  studying  the  Internet  can  only  be  described  in  the  present  moment  due  to  the 
constant  state  of  change  on  the  Internet"  (Rizia,  1999).  The  future  will  hold  many  exciting 
technological  innovations,  making  computers  adapt  to  individuals  rather  than  the  reverse. 
Technology  that  allows  access  to  computer  resources  for  a person  with  disabilities  is  known  as 
adaptive  technology.  These  "electronic  curb-cuts"  are  a combination  of  hardware  and  software. 
The  challenge  is  making  adaptive  technology  part  of  any  base  configuration  and  making  all 
components  work  in  a seamless  fashion.  This  will  increase  the  access  to  computers  and 
applications  without  making  major  modifications. 

\^en  designing  for  the  Internet,  a global  approach  needs  to  be  considered.  As  of  now,  the  U.S. 
is  the  only  country  that  mandates  accessibility  for  anyone  doing  business  with  the  government. 
Of  course,  that  includes  most  higher  education  institutions.  Meeting  accessibility  guidelines  for 
most  situations  means  adding  descriptive  tags  to  any  image  and  text  links  for  audio  or  video 
files.  Anyone,  anywhere  can  use  the  Bobby  site  to  check  for  these  simple  things.  Having 


accessibility  sites  makes  good  economic  sense  as  the  number  of  individuals  with  disabilities  on 
the  Net — who  could  emerge  as  consumers  of  vast  numbers  of  products — continues  to  increase. 

Resources  and  Information 

Helpful  resources  can  be  accessed  from  the  following  site:  Http://www.e- linkagesglobal.com 
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Abstract:  Quality  is  an  elusive  and  ill  defined  concept.  We  have  become 

accustomed  to  associating  quality  with  fancy  features,  high  prices,  zero  defects  and 
conformance  to  specifications.  How  do  we  interpret  Quality  in  the  field  of  higher 
education  in  general,  and  E-education  in  particular?  In  the  field  of  higher  education, 
we  speak  of  "best  practices",  "quality  learning  interventions",  "design  standards", 
"quality  instructional  design",  "quality  services  and  products"  etc.  Although  there  is 
a fair  amount  of  literature  on  quality  management  in  higher  education,  there  appears 
to  be  little  which  applies  the  principles  of  Quality  Assurance  and  Quality 
Management  in  the  field  of  E-education.  This  paper  describes  the  implementation  of  a 
formal  Quality  Management  System  in  E-education  at  a tertiary  institution  in  South  Africa. 


The  Council  for  Higher  Education  (CHE)  in  South  Africa  is  a statutory  body  established  by  the  Higher 
Education  Act  of  1997  to  advise  the  Minister  of  Education  on  all  matters  pertaining  to  higher  education. 
The  CHE  is  responsible  for  establishing  a quality  assurance  system  for  higher  education  through  the 
Higher  Education  Quality  Committee  (HEQC),  which  was  constituted  in  March  2001. 

There  is  a real  need  for  ETD  (Education,  Training  and  Development)  practitioners  to  research  the 
implementation  of  QA  systems  and  to  contribute  to  the  work  of  the  HEQC.  The  founding  document  of 
the  HEQC  states  that  "The  HEQC  is  committed  to  a quality  driven  higher  education  system  that 
contributes  to  socio-economic  development,  social  justice  and  innovative  scholarship  in  South  Africa" 
(CHE,  2000,  p.5). 


The  Department  6 Telematic  Learning  and  Education  Innovation  (TLEI)  is  a service  department, 
which  was  established  at  the  University  of  Pretoria,  South  Africa,  in  1997.  We  provide  support  to 
academic  staff  members  who  wish  to  embrace  innovative  learning  models,  including  computer  ^assisted 
assessment,  multimedia,  web-supported  learning,  interactive  television  and  various  other  delivery 
alternatives  and  combinations.  We  provide  faculty  and  student  training  in  online  learning,  as  well  as 
technical  support. 


This  project  focuses  on  the  quality  of  the  products  and  services  offered  by  the  E-education  division  of 
the  Department  of  TLEI.  The  research  brief  of  this  study  is  to  implement  a Quality  Management 
System  (QMS),  which  will  formalise  processes,  procedures  and  measurement  instruments. 

A Quality  Management  System  (QMS)  is  a way  of  formally  ensuring  that  an  organisation  is 
consistently  in  control  of  the  product  or  service  which  it  provides  to  its  customers.  The  British 
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Standards  Institute  (BSI)  defines  a QMS  as  “the  organisational  structure,  responsibilities,  procedures, 
processes  and  resources  for  implementing  quality  management”  (Boyd,  2001).  Documenting  a quality 
system  demonstrates  how  each  of  the  aspects  interacts  to  ensure  success  in  improving  the  efficiency, 
performance  and  cost  effectiveness  of  the  organisation.  A QMS  should  not  be  perceived  as  separate 
from  the  way  ‘things  are  done  around  here’,  but  a way  of  evolving  better  and  better  business  practices 
and  a part  of  normal  everyday  routines. 

Although  a conscious  decision  not  to  seek  certification  to  IS09000  standards  has  been  made  at  present, 
it  is  useful  to  review  the  provisions  of  IS09000  both  in  application  to  education,  and  to  software 
systems  development,  so  that  the  system  is  suitable  for  refinement  and  auditing  to  IS09000  standards  in 
the  future,  if  required. 

For  the  purposes  of  the  QMS,  we  define  our  “customers”  to  be  lecturers  and  students,  and  we  describe 
our  “products”  as  learning  experiences. 


Quality  Management  System  ■”  progress  so  far 

We  began  by  documenting  our  intuitive  ideas  for  monitoring  and  evaluating  the  learning  experiences 
we  design  and  develop.  Our  Instructional  Design  “toolkit”  consists  of  design  standards  for  multimedia, 
video  and  web-based  products,  QA  checklists  used  to  evaluate  prototypes,  guidelines  for  screen  design 
and  other  technical  functions,  as  well  as  client  service  satisfaction  surveys. 

Our  Project  Timeline  reflects  the  traditional  ADDIE  instructional  design  model,  namely  Analysis, 
Design,  Development,  Implementation  and  Evaluation.  Following  these  five  stages,  we  have  defined 
our  overall  processes  to  be:  A:  Initiate  Eeducation  projects;  B:  Manage  Eeducation  projects;  C: 

Design  and  Develop  Instructional  Materials;  D:  Implement  learning  experiences  (p  roducts);  E:  Evaluate 
learning  experiences  (products).  These  five  processes  are  broken  down  into  a total  of  17  procedures, 
each  represented  by  a box  in  the  project  timeline. 

We  conducted  introductory  QA  training  workshops,  which  were  attended  by  all  staff  members  in  E 
education.  Process  diagrams  indicating  the  suppliers,  customers  and  their  requirements  at  each  stage 
have  been  used  to  gain  insight  into  the  five  processes  and  how  they  may  be  measured  effectively.  The 
importance  of  measurements  in  completing  the  feedback  loop  is  vital,  and  these  are  yet  to  be  defined. 

We  have  formulated  a Quality  Policy,  which  is  an  essential  component  of  a QMS.  The  policy  reflects 
our  commitment  to  fitness  for  purpose,  client  satisfaction,  cost  effectiveness  and  continuous 
improvement  of  all  our  processes  and  functions.  The  formal  QMS  will  reside  on  the  departmental 
intranet,  in  order  to  facilitate  access  to  and  control  of  documentation. 


Conclusion 

This  is  a work- in-progress  project,  which  contributes  to  the  field  of  Quality  Assurance  in  Higher 
Education,  and  B-education  in  particular.  It  will  provide  input  to  the  HEQC  in  South  Africa,  in  their 
brief  to  establish  a framework  for  Quality  Assurance  in  Higher  Education.  It  will  provide  insight  into 
the  practical  dilemmas  of  implementing  a formal  Quality  Management  System  in  Eeducation  at  a 
tertiary  institution. 
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Abstract 

A collaborative  software  development  project  designed  to  maximize  the  skill  sets  and  interests  of 
school  children  and  teachers,  educational  software  technologist  and  researchers,  and  college  undergraduates 
is  presented.  The  work  brings  together  elementary  school  children  with  college  seniors  and  technology 
consultants  to  implement  a problem-solving  methodology  within  a collaborative  environment  to  design, 
develop  and  implement  a multimedia  software  application  that  enhances  the  spatial  orientation  abilities  of 
children  and  puts  the  programming,  interface  design  and  multimedia  systems  capabilities  of  college 
students  into  action.  This  effort  in  project-based  learning  offers  young  students  the  opportunity  to  learn 
mapping  skills,  problem-solving  techniques,  and  participatory  design  methods  while  planning  and 
conducting  virtual  tours  of  their  city. 
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Introduction 

Familiarity  with  geography,  the  science  of  space  and  place  on  the  Earth’s  surface,  helps  people 
visualize  and  understand  their  home  and  orients  their  relationships  vis-a-vis  other  cultures  and 
environments.  Maps  help  to  show  children  where  they  are,  where  they’ve  been  and  where  they  can  go, 
while  creating  a sense  of  belonging  to  a community,  a history  and  a path  to  the  future.  The  urban 
environment,  in  particular,  is  populated  with  many  young  students  who  do  not  often  perceive  their  world 
beyond  the  boundaries  of  their  immediate  neighborhood. 

We  describe  here  a software  development  collaboration  project  designed  to  maximize  the  skill  sets 
and  interests  of  elementary  school  children  and  teachers,  educational  software  technologist  and  researchers, 
and  college  undergraduates.  Through  the  implementation  of  project-based  learning,  problem-solving  and, 
participatory  design  methodologies,  and  community  involvement,  software  containing  interactive  video, 
calculation  programs  and  spatial  orientation  tools  offers  4^-  through  ^^-grade  students  in  Newark,  New 
Jersey  the  opportunity  to  learn  mapping  skills  while  planning  and  conducting  virtual  tours  of  their  city. 

Cognitive  Development,  Spatial  Orientation  and  Technology 
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Learning  theorists  have  articulated  unique  developmental  predispositions  for  different  kinds  of 
learning.  David  Sobel  (1998)  states  that  between  ages  five  and  seven,  children  start  to  move  away  from 
home  and  parents  and  explore  the  natural  world.  From  ages  seven  to  eleven,  children  are  predisposed  to 
merging  with  nature  and  making  geographic  sense  of  the  world  around  them.  Although  computers  cannot 
replace  the  human  contact  and  feedback  that  only  a teacher  can  provide,  they  are  tools  that  can  be  used  to 
significantly  enhance  students’  educational  experiences  at  a time  when  children  have  a propensity  to 
explore.  Through  the  process  of  participatory  design  (Druin,  1999),  10-year  old  students  have  an  equal,  if 
not  greater  stake  in  the  development,  testing,  refinement  and  use  of  the  software  as  compared  to  the  college 
students  who  are  interacting  with  the  youngsters  at  each  stage  and  step  of  the  software  development 
process. 


Software  companies  have  developed  a wide  range  of  software  applications  geared  to  geographic 
inquiry  and  map  making.  The  current  strategies  used  in  mapping  software  programs  permit  students  to  be 
active  mapmakers.  Neighborhood  Map  Machine,  Trudy's  Time  and  Place,  Carmen  Sandiego,  Where  Are 
We,  Map  Makers  Tool  Kit,  Geo  Safari  - all  introduce  geography  skills  to  students  grade  4-8,  geared  to 
providing  geographic  discovery  and  knowledge.  What  these  programs  cannot  offer,  however,  is  any 
significant  or  concrete  local  context  for  the  user.  Moreover,  educational  software  applications  generally  do 
not  satisfactorily  support  the  urban  environment.  Their  symbols  are  suburban  in  orientation,  and  many  of 
their  activities  use  rural  landscapes  as  a backdrop  to  learn  navigating  skills.  However,  children  often  do  not 
have  an  opportunity  to  explore  their  neighborhoods.  Urban  lifestyles  sometimes  include  dangers  in 
neighborhoods  and  deplorable  conditions  of  many  buildings,  causing  parents  to  be  reluctant  to  have  their 
children  explore  on  their  own. 

We’ve  developed  a partnership  among  students  and  faculty  at  the  New  Jersey  Institute  of 
Technology,  Little  Bytes,  a technology -integrated  curriculum  development  firm,  and  St.  Philip’s  Academy, 
an  independent  elementary  school  in  Newark,  to  serve  as  the  foundation  for  a comprehensive  multiyear 
program  in  multimedia  learning  systems  that  brings  college  seniors  studying  software  engineering  and 
multimedia  design  together  with  elementary  school  students  and  teachers  in  an  effort  to  provide 
community-based  educational  software  that  introduces  children  to  the  landmarks  and  cultural  facilities  of 
Newark  through  the  collaborative  development  of  mapping  skills  instructional  software.  A major  goal  of 
the  program  is  to  build  on  the  participatory  design  model  of  software  design,  articulated  most  clearly  for 
application  in  an  educational  environment  by  Druin  (1999),  and  a problem-solving  methodology  that  has 
been  successfully  implemented  at  NJIT  and  in  four  Newark  public  high  schools  (Deck,  1997;  Deck  & 
Friedman  2001).  We  are  testing  the  hypothesis  that  integrating  these  two  models  will  promote  positive 
change  in  the  academic  climate  of  classrooms  by  incorporating  teachers,  students  and  skilled  college-level 
software  engineers  to  create  educational  multimedia  applications  that  accommodate  the  specific  needs  of 
the  younger  students,  provide  teachers  with  design-level  access  to  appropriate  instructional  materials  and 
educational  technologies,  and  give  youngsters  hands-on  experience  in  the  design,  development,  testing  and 
use  of  computer  software  tools. 

Our  project  began  with  Little  Bytes,  a client  of  NJIT’s  business  incubator  providing  the  content 
specifications  to  begin  building  the  software  application.  The  NJIT/Little  Bytes  collaboration  has  opened 
the  door  to  increased  involvement  with  community-based  and  civic  organizations  such  as  the  Newark 
Museum,  Newark  Bears  baseball  team,  New  Jersey  Performing  Arts  Center,  the  Newark  Housing  Authority 
and  the  Newark  Department  of  Engineering,  all  of  whom  have  contributed  to  the  development  of  a 
mapping  skills  software  application  that  is  focused  on  Newark’s  neighborhoods  and  landmarks.  Having 
already  placed  several  college  students,  earning  service  learning  credits,  at  St.  Philip’s  Academy,  where 
Little  Bytes  serves  as  educational  technology  consultant,  we  were  well  positioned  to  expand  our 
involvement  to  include  a small  class  of  seniors  to  focus  on  a specific  outcome. 

Project  Implementation 

During  the  spring  2002  semester,  NJIT  seniors  majoring  in  multimedia  information  technology, 
information  systems  and  computer  science,  were  more  enthusiastic  to  be  involved  in  this  innovative 
project.  The  college  students*  educational  experience  is  project-based,  as  it  moved  them  out  of  the  lecture 
hall  and  into  the  community  and  the  computer  lab  as  they  worked  with  learning  systems  researchers,  4^-6*^ 


grade  students  and  educational  technology  consultants  to  implement  a software  design  that  was  created 
collaboratively  among  St.  Philip’s  Academy  4^*^  grade  school  students,  their  teacher  and  Little  Bytes.  The 
software’s  three  major  components  begins  with  a series  of  fun  tutorials  facilitated  by  an  animated  character, 
during  which  the  user  learns  basic  geography  skills;  putting  those  skills  to  use  as  they  conduct  four  searches 
throughout  different  locations  in  Newark,  and  finally  being  able  to  design  virtual  tours  of  the  city  by 
building  efficient  routes  between  landmarks  that  are  in  video  format. 

The  protocols  of  the  project  are  oriented  to  maximize  a problem-solving  methodology  found 
effective  in  computing,  mathematics  and  composition  courses  at  the  pre-college  and  college  levels  (Deek  & 
Friedman,  2001),  combined  with  Druin’s  approach  to  software  engineering  (Druin,  1999).  Children’s 
interaction  with  technology  expands  beyond  their  end-user  status  and  into  the  conceptual  design, 
development,  usability  testing  and  debugging  phases  as  well.  By  involving  the  children,  we  supply  an 
alternative  method  of  software  design,  development  and  evaluation,  one  that  is  accessible  to  our  end  users, 
the  elementary  school  students.  Through  the  use  of  visually  oriented  software  design  and  increased 
opportunities  to  team  children,  teachers  and  software  engineering  students,  educational  researchers  and 
software  designers  in  the  development  of  new  applications,  software  evaluation  becomes  the  province  of  all 
the  participants. 

Application  development  through  participatory  design  has  three  main  goals:  to  develop  integrated 
learning  environments  that  support  visual  and  verbal  literacy;  to  encourage  student  centered  learning;  and, 
to  develop  methodologies  that  offer  a better  understanding  of  what  children  want  and  need  when  using 
technology.  The  program  provides  a combined  laboratory  and  classroom  environment  in  which 
participants  design,  develop,  test  and  refine  software  tools  intended  to  inculcate  spatial  awareness  and 
mapping  skills  while  also  attaining  positive  results  with  students. 

Problem-Solving 

Through  participatory  design,  there  are  extended  interactive  discussions  among  teachers,  students 
and  software  engineers  during  each  phase  of  software  development.  We  began  with  a meeting  of  all  the 
people  involved:  the  4^  graders,  their  teacher  and  principal;  the  college  students  and  their  instructor,  two 
representatives  of  a community  based  youth  group,  Do  Something  Newark;  and  two  educational 
technology  consultants  from  Little  Bytes.  After  explaining  to  the  youngsters  briefly  what  we  had  in  mind, 
the  children  were  asked  to  describe  and  draw  a picture  of  a character  that  would  act  as  their  guide  through 
the  computer  game.  The  college  students  formed  several  groups:  graphical  user  interface  designers, 
database  and  systems  programmers,  videographers  and  3D  designers,  and  each  began  to  storyboard  specific 
components  of  the  intended  product. 

There  are  three  major  components.  Section  1,  Navigator,  consists  of  1 1 skill  building  interactive 
exercises  that  teach  basic  map  reading  skills.  Here,  an  animated  character  that  acts  as  a guide,  Mapper,  is 
introduced.  Section  2,  Mad  Mysteries,  presents  three  scenarios  in  which  students  find  answers  to  clues 
offered  in  each  to  solve  mysteries  that  take  place  in  Newark.  They  are  awarded  certificates  of  proficiency 
that  allow  them  to  move  on  to  the  final  component.  Section  3,  Create  a Tour.  The  children  are  given  a 
problem  to  solve:  how  to  structure  a tour  of  Newark  for  four  different  groups  of  tourists,  all  with  different 
time  constraints  as  well  as  interests.  The  skills  garnered  in  Section  1 and  tested  in  Section  2,  are  applied  in 
Section  3.  After  two  weeks  of  preliminary  design,  the  college  students  settled  on  Macromedia  Flash  as  the 
animation  and  movie  creation  software  best  suited  for  sections  1 and  2,  and  Adobe  Digital  Video  Suite 
served  as  the  main  technology  behind  section  3. 

Employing  a consistent  methodology  that  is  common  to  all  the  participants  and  able  to  be 
documented  added  to  the  sense  of  partnership  among  the  participants  as  they  shared  ways  to  solve 
problems  specific  to  the  areas  that  each  encounter.  Bringing  theory  into  practice,  as  Filho  (2001)  suggests, 
demands  that  a process  architecture  be  used  when  students  are  engaged  in  complex  software  engineering 
projects  such  as  the  project  describe  here.  We  used  SOLVEIT,  which  is  such  an  architecture,  and  has  been 
successfully  employed  in  high  school  and  college  settings.  (Deek  & Friedman,  2001)  The  SOLVEIT 
environment  is  collection  of  tools  supporting  a six-step  process  for  problem  solving  and  software  design 
and  implementation.  These  steps  are:  formulating  the  problem,  planning,  designing,  and  translating  the 
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solution,  and  finally,  testing  and  delivering  the  product.  The  combination  of  a participatory  design 
approach  and  SOLVEIT  as  a software  engineering  methodology  produced  a highly  accessible  interactive 
software  application.  NJIT  students  used  applications  such  as  Flash,  3D  StudioMax,  Adobe  Digital  Video 
Suite,  as  well  as  C++,  Java  and  Access  to  provide  the  younger  students  with  a game -like  learning 
environment.  This  combination  of  methodologies  and  tools  not  only  broadens  all  the  students’  educational 
experience,  but  also  provides  a more  meaningful  context  within  which  to  integrate  the  knowledge  and  skills 
they  have  garnered. 

Student  Response 

While  the  college  students  were  initially  hesitant  to  break  free  of  their  traditional  classroom 
environment,  they  were  quick  to  appreciate  the  benefits  of  working  within  the  structure  provided  by 
SOLVEIT  as  well  as  those  that  problem-based  learning  experiences  offer,  particularly  when  the  solution  to 
a problem  takes  the  form  of  interactive  software,  and  the  design  and  development  of  that  software  is  based 
in  the  methodology  of  participatory  design.  Ritika,  who  was  responsible  for  the  scaling  activity  in  the 
Navigator  group,  used  Flash  5 

because  the  class  had  agreed  on  doing  most  of  the  project  with  it.  Also,  it  was  something 
new  to  me,  and  I wanted  to  learn  it.  I tried  to  think  of  the  kids'  demands  first  when  I was 
making  this,  so  I tried  to  make  it  as  fun  as  possible.  I put  in  animations,  and  will  put  in 
sounds,  which  I think  they  will  enjoy.  From  the  hst  meeting  with  the  kids,  I could  tell 
they  wanted  something  very  challenging  with  degrees  of  difficulty,  but  I am  not  sure  how 
to  incorporate  that  into  my  part  of  the  game.  (Instructor’s  notes) 

Marc  was  part  of  the  audio/video  team  responsible  for  the  planning,  shooting,  editing,  and  integration  of 
video  footage  and  voice-overs.  In  a mid-semester  report  he  noted. 

The  recent  visit  by  the  St.  Philip’s  students  to  our  NJIT  lab  led  to  the  continued 
development  of  software  segments  that  have  thus  far  been  pieced  together.  The  students’ 
ideas  reflect  their  upbeat  attitude  toward  the  city  of  Newark  and  their  role  in  the  project. 
An  example  of  their  input  during  this  meeting  was  the  idea  to  change  the  background 
color  at  the  welcome  screen  from  a black  night  to  a more  optimistic  daytime.  Jerri, 
Ursula  [of  Little  Bytes]  and  AJ  [Zenkert,  the  grade  teacher]  have  all  been  a big  help. 
Ursula  is  good  at  clearing  up  confusing  development  issues  between  Jerri  and  our  NJIT 
class.  I look  forward  to  our  ftiture  collaborations.  (Instructor’s  notes) 

Rajesh,  who  in  addition  to  the  credits  is  responsible  for  several  of  the  Navigator  components,  used 
Macromedia  Flash,  Adobe  Illustrator,  Microsoft  Paint,  and  Microsoft  Image  Composer  as  development 
tools. 


These  were  chosen  for  their  development  capabilities  and  ease  of  use.  On  average,  in 
addition  to  class  time,  I spent  about  5 hours  a week  on  the  project.  Initial  progress  was 
slow  because  of  the  learning  and  investigating  time  involved  in  starting  the  project. 
Meeting  with  Jerri,  Ursula,  AJ,  and  the  St.Philip's  kids  proved  that  the  users  need  a fun, 
challenging,  and  educational  application.  (Instructor’s  notes) 

Gosia  & Tim  began  their  work  by  developing  the  introduction  sequence  for  the  software. 

We  used  traditional  storyboarding  and  drawing  techniques  to  develop  the  intro  Space 
animation.  We  tried  to  make  an  entertaining  and  fun  animation  that  hoped  to  show  the 
kids  from  St.  Philip’s  their  perspective  by  traveling  from  space,  down  through  the  solar 
system,  into  New  Jersey.  Music  was  added  to  give  a little  flavor  to  the  work  using  a 
hardware  synthesizer,  along  with  other  software.  Working  in  the  studio  environment  with 
all  the  very  diverse  people  was  unarguably  the  best  experience  I have  had  since  being  at 
NJIT.  (instmetor’s  notes) 
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Nainsi  and  Parag  found  that, 


This  class  has  given  a wonderful  opportunity  very  close  to  the  real  world  without  entering 
a REAL  world.  Our  project  was  mainly  created  by  Flash;  I have  learned  lot  about 
Macromedia  Flash  and  its  implementing  capabilities.  Drawing  with  details  and  putting 
every  piece  together  is  sometimes  tedious.  But  overall  it  is  fiin.  Our  discussions  among 
ourselves,  Jerri  and  the  kids  have  shaped  our  requirements  tightly.  I really  like  the 
involvement  of  kids  in  our  project,  as  I find  them  smart  is  noticing  things  and  accurate  in 
demanding  what  exactly  they  will  like  to  see.  What  I don’t  like  sometimes  is  not  having  a 
set  rule,  not  knowing  what  actually  is  needed  until  it  is  done  halfway  or  all  the  way.  But  I 
guess  that  is  part  of  learning,  and  I don’t  have  many  complaints  about  that.  (Instructor’s 
notes) 


St.  Philip’s  Academy  students  and  teachers  have  also  benefited  from  this  project  in  many  different 
ways.  The  current  Social  Studies  curriculum  does  not  contain  many  lessons  about  mapping  and  map 
usage.  Quality  supplemental  mapping  skills  material  are  not  easy  to  find,  particularly  material  that  involves 
the  type  of  mapping  that  is  most  pertinent  to  urban  students’  world:  the  city.  When  students  understand 
how  to  use  maps  and  concepts  of  scale  and  distance,  they  have  skill  sets  that  can  be  applied  in  subject  areas 
such  as  history,  politics,  wars,  migration,  and  immigration. 

For  the  students,  the  opportunity  to  participate  in  the  development  of  software  is  invaluable. 
Being  a part  of  a team,  whether  it  is  a sports  team  or  a debate  team,  teaches  children  about  working 
together,  solving  problems,  listening  to  others’  ideas,  and  communicating.  Throughout  this  process,  the 
students  have  had  to  use  and  improve  all  of  these  skills.  What  they  did  not  realize  was  that  they  learned 
and  reinforced  many  mapping  skills  without  ever  sitting  through  a traditional  classroom  lesson.  The 
students  were  very  eager  and  excited  to  be  a part  of  the  project,  and  they  worked  very  hard  to  be  helpful, 
honest  and  creative  in  the  process.  They  began  to  understand  that  they  could  not  have  everything  they 
wanted  in  the  software,  but  that  it  did  not  hurt  to  offer  ideas  and  attempt  to  help,  with  the  hopes  that  their 
ideas  would  be  used  in  the  final  product.  The  teamwork,  brainstorming,  and  creative  thinking  that  the 
students  used  during  this  process  provided  a wonderful  challenge  and  learning  experience  for  them. 

During  a recent  usability  testing  session,  14  4^  graders  reviewed  several  components  of  the 
Navigator  section,  including  a parade  that  asks  them  to  measure  distance  and  height,  and  assess  whether  a 
float  will  be  able  to  pass  under  a bridge,  a “walk  in  the  park,”  which  tests  students’  ability  to  orient 
themselves  to  shifting  directions,  and  a “legend”  exercise  that  asks  students  to  identify  icons  and  move 
them  in  their  appropriate  places  on  a map.  Feedback  was  obtained  concerning  levels  of  difficulty, 
“coolness,”  as  well  as  descriptions  of  what  the  students  learned.  These  results  have  direct  bearing  on  the 
nature  of  the  modifications  the  college  students  will  do  over  the  summer,  but  also  suggest  that  we  are  on 
the  right  path,  not  only  in  terms  of  integrating  software  into  the  4*^  grade  curriculum,  but  also  that  involving 
the  children  in  the  design  and  development  of  the  software  has  proven  beneficial  to  the  younger 
participants  of  the  project.  The  majority  of  children  found  the  software  to  be  “cool,”  the  instructions  clear 
and  the  navigation  easy.  There  was  a nearly  unanimous  call  for  the  components  to  be  more  challenging, 
but  at  the  same  time,  more  children  found  the  software  to  be  fiin  than  boring.  When  asked,  “What  did  you 
learn  while  playing  this  activity?”  ten  of  the  14  respondents  answered,  “How  to  measure,”  while  the 
remaining  four  replied,  “Nothing.”  When  asked,  “If  you  could  change  this  activity  in  any  way,  what  would 
you  change?”  about  half  answered,  “Make  it  easy,”  two  said,  “Make  it  more  challenging,”  and  four  did  not 
respond.  Generally,  the  students  liked  the  interactivity  and  that  the  activities  require  concentration. 

Conclusion  and  Future  Work 

Development  and  utilization  of  customized  educational  software  empowers  the  teacher  and  takes 
advantage  of  rapidly  emerging  multimedia  technologies.  We  expand  our  scope  and  vision  by  bringing 
together  the  classroom  teacher,  students,  the  business  community,  civic  institutions  with  the  universities  to 
create  technology  integrated  products  designed  to  promote  student-centered  learning  which  serves  the 


community  and  ensures  educational  success  of  the  child.  The  first  version  of  City  Mapping  is  designed 
primarily  for  4^  graders.  It  is  our  plan  to  produce  a module  for  and  6*^  graders  as  well  during  the  fall 
2002  semester.  At  the  time,  we  will  also  expand  our  technology  integration  into  the  elementary  school 
curriculum  by  implementing  an  international  media  sharing  collaboration  among  students  in  Indonesia, 
Poland  and  Newark,  with  NJIT  students  building  the  infrastructure  that  will  allow  the  exchange  and 
collaborative  development  of  music,  art  and  narrative  via  the  Internet. 


o 
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Abstract:  Many  instructors  are  expected  to  design  and  create  web  courses.  The  design  of 
web  courses  can  be  a difficult  task  for  educators  who  lack  experience  in  interaction  and 
instructional  design.  Design  patterns  have  emerged  as  a way  to  capture  design  experience  and 
present  design  solutions  to  novice  designers.  Design  patterns  are  a widely  accepted  method  of 
providing  design  support  to  software  engineers,  and  they  have  been  proposed  to  support 
designers  of  web-based  courses.  However,  end-users’  abilities  to  use  design  patterns  are 
unclear  especially  since  web-course  design  and  computer  programming  are  different 
activities.  We  present  a methodology  for  supporting  novices’  use  of  patterns  during  web- 
based  instructional  design.  This  methodology  consists  of  a pattern  language  for  web-based 
instruction  and  a design  environment  that  scaffolds  the  process  of  finding,  selecting,  and 
applying  patterns  to  design  problems. 


1.  Introduction 

Design  patterns  provide  a way  to  capture  and  present  solutions  to  design  problems  and  to 
facilitate  communication  among  the  many  members  of  a design  team.  The  use  of  patterns  for  design 
originated  in  the  field  of  architecture  during  the  late  1970s  as  a way  to  describe  solutions  to 
reoccurring  problems  encountered  in  architectural  design  (Alexander  et  al.,  1977).  The  goal  was  to 
support  both  architects  and  the  general  public  in  designing  quality  towns,  neighborhoods,  and  homes. 
Building  upon  this  work,  the  concept  moved  into  the  software  engineering  discipline  as  a way  to 
document  design  experience  for  less  experienced  software  developers  (Beck  & Cunningham,  1987; 
Gamma  et  al.,  1995).  These  patterns  have  since  been  used  as  an  educational  tool  to  teach  computer 
science  curriculum  (Gelfand  et  al.,  1998).  Design  pattern  usage  has  also  become  a relatively  new 
idea  in  the  field  of  human-computer-interaction  and  educational  technology  to  support  designers  in 
interaction  and  instructional  design  (Borchers,  2001;  Frizell,  2001;  Rossi  et  al.,  1996). 

The  popularity  of  design  patterns  can  be  attributed  to  their  ability  to  capture  design 
experience  and  their  concrete  nature.  Patterns  are  more  flexible  than  static  templates  and  more 
concrete  than  abstract  guidelines.  This  suggests  that  end-users  can  better  apply  them  to  their  design 
problems.  Although  the  literature  on  design  patterns  makes  claims  to  their  effectiveness  in  helping 
novice  designers,  there  is  not  a lot  of  empirical  evidence  with  end-users  utilizing  patterns  in  actual 
design  projects.  The  current  research  is  from  the  area  of  software  engineering,  whose  design 
solutions  are  often  in  the  form  of  classes  (in  the  object  oriented  paradigm)  and  describe  how  the 
classes  relate  to  and  communicate  with  each  other  and  objects  to  solve  a problem.  The  original 
architectural  design  patterns  are  somewhat  different  from  software  engineering  patterns  with  the 
latter  being  more  concerned  with  implementation  than  design.  Just  as  the  goal  of  the  architectural 
patterns  is  to  help  architects  build  structures  that  possess  a certain  quality  for  their  inhabitants,  the 
goal  of  WBI  patterns  is  to  assist  educators  in  designing  web-based  courses  that  are  instructionally 
effective  for  students.  Web-based  instructional  (WBI)  design  patterns  are  more  closely  related  to 
architectural  design  patterns  because  the  focus  is  more  on  the  user’s  experience  with  the  final 
product.  Patterns  can  be  a powerful  resource  tool  for  designers  within  these  domains,  but  our 


working  hypothesis  is  that  novice  designers  need  additional  support  in  the  process  of  selecting  and 
applying  patterns  to  their  design  problems. 

This  paper  presents  our  approach  for  scaffolding  the  pattern  user’s  design  tasks.  In  Section 
2,  we  take  a closer  look  at  design  patterns  and  current  research  results.  Section  3 discusses  design 
environments  that  have  been  proposed  to  support  design  pattern  usage.  In  Section  4,  we  present  our 
methodology  for  providing  support  for  the  application  of  patterns  in  web-course  design  and  the 
resulting  design  environment.  Section  5 concludes  with  a brief  overview  of  our  research  objectives 
and  future  research  directions. 

2,  Design  Patterns 

A design  pattern  captures  a solution  to  a problem  and  presents  it  in  such  a way  that  the 
solution  can  be  adapted  and  used  repeatedly.  Patterns  go  beyond  simply  presenting  a solution;  they 
also  tell  the  user  why  the  solution  is  needed  and  the  context  in  which  the  solution  can  be  applied. 
This  information  is  presented  in  a format  usually  organized  into  several  sections  including  the 
pattern  name,  which  describes  the  design  problem  and  provides  a way  to  communicate  about  the 
pattern,  the  problem,  which  describes  the  design  issues  the  pattern  addresses,  the  context,  which 
explains  when  to  apply  the  pattern,  the  solution,  which  tells  the  user  how  to  solve  the  problem,  and 
the  forces,  which  describe  the  trade-offs  of  applying  the  solution.  Patterns  usually  exist  within  a 
language  with  other  patterns  addressing  design  problems  in  that  same  domain.  A pattern  language  is 
a collection  of  patterns  that  are  connected,  thus  allowing  the  designer  to  see  how  all  the  patterns 
within  the  language  work  together. 

Our  research  is  concerned  with  design  patterns  for  WBI  design.  The  need  for  design  support 
has  become  a major  issue  in  the  design  of  web  courses.  The  poor  design  of  the  instructional  materials 
in  web  courses  is  one  of  the  key  problems  with  learning  from  the  web  (Bork  & Britton,  1999; 
Kessler  et  al.,  1999).  Design  patterns  provide  a mechanism  for  capturing  pedagogical  strategies  and 
good  design  practices  in  a way  that  can  assist  educators  in  designing  instructionally  sound  web 
courses.  WBI  design  patterns  vary  in  content  from  dealing  with  navigation  design  issues  to 
addressing  problems  with  student  learning  activities  (Frizell,  2001;  Rossi,  1996;  Anthony,  1995).  As 
in  other  areas  of  pattern  usage,  the  goal  is  to  support  less  experienced  designers.  Figure  1 gives  an 
abbreviated  example  of  a design  pattern  that  has  been  proposed  for  web-course  design  (Frizell, 
2001).  This  pattern  addresses  the  isolation  problem  some  web-based  students  have,  and  the  solution 
describes  to  the  designer  a method  for  alleviating  the  problem. 

There  have  been  some  preliminary  studies  on  the  effectiveness  of  pattern  usage  during  the 
design  process  (Prechelt,  1997;  Schmidt,  1995;  Beck,  1987).  Although  this  evidence  has  been  in  the 
realm  of  software  engineering,  preliminary  results  from  these  studies  do  show  that  patterns  may 
prove  beneficial  to  the  design  process.  The  benefits  of  design  patterns  may  be  greatly  harnessed  by 
coupling  the  patterns  within  an  environment  that  supports  the  designer  in  working  with  the  patterns 
in  the  language.  We  believe  this  to  be  especially  true  in  a domain  such  as  web-based  instruction 
where  the  course  designer  may  not  have  any  experience  in  interaction  and  instructional  design,  and 
also  in  domains  where  the  pattern  language  has  grown  to  include  a large  number  of  patterns. 
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Name:  Learning  Community 

Problem: 

Some  students  have  a need  to  feel  connected  with  other  students  enrolled  in  the  course.  How  to 
facilitate  a sense  of  community  for  on-line  students? 

Context: 

Web-based  courses  where  students  are  feeling  isolated. 

Forces: 

Students  are  not  all  on-line  at  the  same  time. 

Some  students  prefer  anonymity. 

The  difficulty  of  the  medium. 

Solution: 

Provide  an  environment  that  encourages  students  to  get  to  know  other  students  in  the  course  and 
to  communicate  with  each  other.  This  can  be  facilitated  by  having  students  post  information  about 
themselves.  You  can  make  the  first  assignment  called  ‘Introduce  Yourself,  It  can  be  in  the  form  of  a 
web  page  and  contain  information  such  as  the  student’s  name,  email  address,  interests,  and  a picture. 
This  information  can  encourage  interaction  and  dialogue  among  the  students. 

Web  discussion  tools  such  as  bulletin  boards  can  also  be  added  to  the  course  to  encourage  students  to 
discuss  topics.  Students  can  go  to  the  bulletin  board  to  post  problems  and  share  ideas.  You  can  also 
post  questions  on  the  bulletin  board  and  require  students  to  respond.  This  type  of  student  interaction 
creates  a sense  of  community  for  students. 

Including  Group  Projects  as  a part  of  the  instructional  activities  provides  another  mechanism  to 
encourage  a connection  among  students. 

Rationale: 

Creates  a learning  environment  that  encourages  participation  and  interaction  among  the  students.  A 
course  design  that  provides  for  a high  level  of  interaction  may  alleviate  some  of  the  issues  of  learners 
who  feel  isolated  and  non-connected  during  on-line  courses.  In  distance  education  courses,  an  attrition 
rate  of  50%  is  common.  By  making  students  feel  like  they  are  part  of  a community,  they  may  be  more 
likely  to  participate  and  complete  the  course. 

Related  Patterns:  Group  Projects 


Figure  1:  WBI  Design  Pattern 


3.  Pattern  Support  Environments 

Within  the  software  engineering  community,  there  has  been  some  research  on  providing 
computer  support  for  working  with  design  patterns.  Budinsky  (1996)  presents  a design  tool  to 
support  the  implementation  of  design  patterns  in  object-oriented  software.  Their  tool  supports 
browsing  the  patterns  on-line  and  automatically  generates  the  code  for  the  pattern’s  implementation 
based  on  information  collected  from  the  software  designer.  The  objective  was  to  automate  the 
implementation  of  design  patterns  for  the  software  developer.  However,  with  this  work  they  found 
that  the  problem  that  may  exist  in  pattern  application  is  not  in  the  coding  or  development;  it  is  in  the 
designer  understanding  his  problem  and  deciding  which  pattem(s)  helps  solve  it  best  (Chambers  et 
al.,  2000). 

Web-course  design  environments  such  as  Shih  (2000)  and  Thomson  (2000)  recognize  the 
need  for  design  support  for  pattern  usage,  but  these  environments  mostly  provide  support  for  the 
navigational  structure  of  web-based  courses.  Patterns  are  used  to  model  optimal  paths  through  the 
course  and  a design  tool  guides  the  designer  in  developing  courses  based  on  these  paths. 
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Design  environments  such  as  the  ones  described  above  are  heavily  implementation  oriented; 
a support  environment  for  pattern  usage  in  web-course  design  must  also  provide  support  for  pattern 
selection. 

4.  Supporting  the  application  of  patterns  in  web-course  design 

Our  research  with  design  patterns  is  two-fold.  We  are  working  on  a pattern  language  for 
web-based  instruction  to  support  designers  in  the  creation  of  web  courses  and  on  a design 
environment  to  support  our  language.  We  focus  the  remainder  of  this  paper  on  our  methodology  for 
providing  this  support  and  the  resulting  design  environment. 

Our  goal  is  to  provide  support  for  three  main  activities,  and  selecting  the  appropriate  patterns, 

and  the  application  of  the  selected  patterns  into  the  course’s  design. 

• Finding  Patterns:  We  utilize  a hypertext  rendition  of  our  pattern  language  to  support  browsing 
and  quickly  navigating  through  the  patterns  in  the  language.  Patterns  are  grouped  into  categories 
based  on  the  type  of  problems  they  address  with  cross-referencing  between  related  patterns.  This 
allows  quick  access  to  the  patterns. 

• Selecting  Patterns:  To  support  course  designers  in  selecting  the  appropriate  pattem(s)  to  solve 
their  design  problems,  we  provide  support  for  pattern  selection  in  two  ways.  We  provide  a menu- 
based  approach,  which  allows  the  designer  to  search  for  patterns  based  on  their  course  goal  or 
problem.  We  also  use  a decision  support  system  to  guide  the  designer  through  the  process  of 
designing  a course  using  a standard  instructional  design  process  combined  with  the  patterns  that 
address  problems  within  each  phase. 

• Applying  Patterns:  After  the  course  designer  chooses  the  patterns  he  wants  to  use  in  his  course’s 
design,  a design  specification  for  the  web-course  is  developed  based  on  these  selected  patterns. 
The  course  design  specification  acts  as  a bridge  from  design  to  implementation.  It  gives  the 
instructor  an  idea  of  how  the  course  is  structured  based  on  his  particular  design  decisions. 

A designer  that  is  experienced  with  working  with  the  pattern  language  may  not  only  know 
which  patterns  to  select  and  how  to  apply  them,  but  also  follows  an  effective  design  process  that 
includes  the  use  of  patterns  as  a subtask.  It  is  important  that  the  novice  designer  learns  to  not  only 
work  with  the  pattern  language,  but  also  gains  a better  understanding  of  design  in  general  and  design 
with  patterns  specifically.  The  process  of  finding,  selecting,  and  applying  patterns  should  be 
scaffolded  for  the  novice  designer.  One  of  the  characteristics  of  scaffolding  is  that  support  can  be 
faded  away  over  time  giving  the  learner  the  control  to  apply  their  new  knowledge  and  skills 
(Vygotsky,  1978/ 

We  are  developing  a design  tool  named  PattemEdge  to  explore  the  combination  of  a pattern 
language  coupled  with  a method  that  scaffolds  pattern  usage  for  the  designer.  PattemEdge  employs  a 
dynamic  web-based  interface  implemented  with  PHP  and  uses  a relational  database  for  storing 
patterns.  A mle-based  system  supports  the  designer  in  the  selection  of  patterns  taking  into 
consideration  the  characteristics  of  the  learners  and  the  goals  and  objectives  of  the  course  designer. 

The  overall  approach  is  depicted  in  Figure  2.  The  patterns  within  the  language  are  annotated 
for  how  and  when  they  are  best  used.  The  inference  engine  can  then  use  this  information  during  the 
pattern  selection  process.  When  interacting  with  the  system,  the  designer  is  able  to  browse  through 
the  patterns  and  select  and  apply  them  on  their  own  as  they  would  if  the  patterns  were  presented 


without  design  support.  Users  also  have  the  option  of  being  able  to  seek  design  advice  for  working 
with  the  patterns.  The  system  will  collect  information  about  the  course  to  be  designed  such  as  learner 
characteristics  and  the  instructor’s  goals  and  objectives.  Based  on  the  information  known  about  the 
course  and  interaction  with  the  instructor,  the  tool  suggests  design  solutions  (i.e.  patterns)  to  the 
instructor.  The  solution  given  could  be  a pattern  to  solve  a particular  design  issue  or  a collection  or 
subset  of  design  patterns  from  the  language  to  cover  all  aspects  of  a course’s  design.  After  the 
selection  of  patterns,  the  tool  generates  a design  specification  for  the  web-based  course  based  on 
these  patterns.  We  believe  this  design  specification  could  be  integrated  into  an  authoring  tool  to 
support  the  instructor  in  developing  the  course  based  on  the  design  decisions. 


5.  Conclusion  and  Future  Work 

An  objective  of  our  research  is  to  develop  an  environment  that  fosters  educators  in  designing 
instructionally  sound  web-based  courses  through  the  development  of  a WBI  pattern  language  and  a 
method  to  support  the  effective  use  of  this  language.  We  believe  end-users  can  benefit  from  having  a 
design  environment  that  scaffolds  design  pattern  usage  for  them.  Instead  of  merely  focusing  on 
implementation  issues,  this  environment  should  provide  support  for  finding  and  selecting  the 
appropriate  patterns.  With  PattemEdge,  we  provide  this  support.  Future  work  includes  evaluations 
with  users  to  assess  the  effectiveness  of  the  design  environment  in  supporting  pattern  usage. 
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Abstract:  Since  the  inception  of  the  World  Wide  Web,  faculty  members  have  been  developing  online  course 
materials.  However,  there  is  little  careful  analysis  of  how  students  use  these  Webs.  In  particular,  do  more 
successful  and  less  successful  students  use  course  webs  and  associated  materials  differently?  Are  usage  patterns 
similar  to  those  of  printed  resources,  or  do  students  explore  materials  differently  on  the  Web?  To  answer  questions 
like  these,  educators  and  researchers  need  tools  that  allow  them  to  closely  examine  student  use  of  Web  materials. 
Building  upon  user  tracking  tools  that  gather  information  on  student  usage,  including  the  time  each  reader  arrives 
at  and  spends  on  each  page,  the  use  of  multiple  windows,  and  the  links  followed  from  page  to  page,  we 
implemented  Clio 's  Assistants  a customizable  suite  of  tools  that  permit  exploration  of  student  Web  usage  patterns 
both  graphically  and  textually.  The  graphical  tools  include  simple  bar  charts,  customizable  directed  graphs,  and 
“replays”  of  student  sessions.  Textual  tools  include  simple  statistical  summaries,  human-readable  log  files, 
database  queries,  and  an  advanced  pattern  matching  language.  Through  these  tools,  one  can  identify  and  explore 
patterns  of  Web  usage. 


1 Introduction 

Although  a number  of  tools  are  used  to  create  educational  computing  resources,  the  World  Wide  Web  (Berners- 
Lee  et  al.  1994)  is  perhaps  the  most  popular  mechanism  for  creating  computerized  educational  resources.  The 
Web  provides  many  advantages,  including  easy  design  of  documents,  “universal”  access  (most  Web  pages  can 
be  accessed  from  anywhere  on  the  Internet),  and  the  ability  to  incorporate  local  and  remote  documents  in  a 
course  Web. 

Students  use  course  Webs  in  a variety  of  ways.  For  example,  some  students  explore  course  webs  using  only 
one  window  while  others  open  multiples  windows  (e.g.,  one  to  hold  the  current  problem  being  studied,  another 
to  hold  reference  materials,  and  a third  to  hold  current  news).  Similarly,  some  students  will  visit  each  link  on  a 
page  while  others  will  be  very  careful  in  their  selection  of  links.  Are  there  patterns  that  successful  students  seem 
more  likely  to  use?  Do  students’  usage  patterns  evolve  over  time?  And,  perhaps  most  importantly,  can  less 
successful  students  benefit  from  using  the  patterns  of  more  successful  patterns? 

Scholars  cannot  answer  any  of  these  questions  until  there  are  ways  to  identify  and  explore  these  usage 
patterns.  The  goal  of  Project  Clio  is  to  provide  tools  that  allow  analysts  to  identify  and  experiment  with  usage 
patterns.  Clio  works  in  three  phrases:  gathering  (Clio’s  Watchers),  synthesis  (Clio’s  Accountants),  and  analysis 
(Clio’s  Assistants).  Clio’s  Watchers  are  a collection  of  tools  that  gather  information  while  a student  is  browsing 
the  Web.  By  using  the  Web  Raveler  architecture  (Kensler  and  Rebelsky  2000),  Clio’s  Watchers  are  able  to 
gather  information  on  student  usage  whether  students  are  on  a local  or  remote  site.  Because  the  data  gathered  by 
Clio’s  Watchers  are  repetitious  and  make  some  information  (such  as  time  on  page)  implicit,  Clio’s  Accountants 
convert  the  raw  data  into  more  useful  data  that  are  then  stored  in  a relational  database.  Finally,  Clio’s  Assistants 
provide  customizable  ways  to  explore  those  data.  The  key  aspects  of  Clio’s  Watchers  are  that  they  gather  data 
for  a group  of  individuals  (e.g.,  all  members  of  a course)  and  that  the  gather  data  for  all  pages  those  individuals 
visit,  whether  they  are  local  or  remote. 

Clio’s  Assistants  provide  the  focus  of  this  paper.  In  Section  2,  ve  describe  the  primary  assistants.  In 
Section  3,  we  describe  a typical  interaction  with  Clio’s  Assistants.  In  Section  4,  we  compare  Clio  and  Clio’s 
Assistants  to  other  hypertext  analysis  tools.  In  Section  5,  we  revisit  the  need  for  Clio  and  Clio’s  Assistants. 
Finally,  in  Section  6,  we  consider  future  directions  for  Clio. 
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2 Clio’s  Assistants:  The  Suite  of  Tools 


The  Clio’s  Assistants  Tool  Suite  is  a collection  of  tools,  both  graphical  and  textual,  that  allows  analysts  to 
explore  the  ways  in  which  students  use  the  Web.  At  the  core  of  this  exploration  is  the  notion  of  classification. 
Although  Clio’s  Assistants  permit  analysts  to  explore  usage  of  individual  pages  (e.g..  How  long  did  the  average 
student  spend  looking  at  exercise  3?),  most  assistants  allow  analysts  to  consider  groups  of  pages  (e.g..  When 
confronted  with  an  exercise,  what  kind  of  page  did  student  X most  likely  visit  before  entering  an  answer?  Or 
From  what  pages  were  students  more  likely  to  leave  the  local  course  Web?).  The  classification  of  pages  is 
relatively  straightforward:  For  each  class  of  pages,  the  analyst  enters  (1)  one  or  more  patterns  (e.g.,  “all  pages 
whose  URL  begins  with  http://www.cs.grinnell.edu/'^rebelsky/Examples/”),  (2)  a classification  (e.g., 
“Examples”)  and  (3)  a shorthand  for  the  classification  (e.g.,  “E”).  Classifications  are  stored  permanently  in  the 
system.  Links  may  be  classified  in  two  ways:  by  the  internal  link  type  (the  REL  attribute)  or  by  pairs  of 
patterns.  Because  links  depend  closely  on  the  particular  pages  used  (e.g.,  while  a link  from  sect3. 5.html  to 
sect2.4.html  in  the  same  directory  is  probably  a “prerequisite  information”  link,  not  all  links  between  sections 
are  prerequisite  links),  internal  link  types  are  preferred. 

Once  analysts  have  classified  pages,  they  may  explore  the  usage  logs  with  both  graphical  and  textual  tools. 
These  tools  range  from  simple  summaries  (textual  and  bar  charts)  to  complex  representations  of  the  data  (e.g., 
as  a directed  graph  or  animation  over  time).  Direct  access  to  the  “nearly  raw”  data  is  also  available.  In  the 
following  paragraphs,  we  describe  the  tools  in  more  detail. 

Bar  Chart:  Simple  Graphical  Summaries  Often  the  best  way  to  begin  exploration  of  data  is  with  a 
simple  overview  of  the  most  “popular”  parts  of  a site.  Bar  charts  concisely  represent  a large  amount  of  data  in  a 
simple  and  quick  way.  The  Bar  Chart  tool  permits  analysts  to  quickly  explore  a number  of  comparative 
relations.  For  example,  bar  charts  can  be  generated  for  number  of  visits  versus  URLs  (for  a group  of  students) 
to  get  a feel  for  which  pages  students  visit  most  often,  or  time  spent  versus  classification  to  explore  how 
students  are  dividing  their  time.  We  are  currently  exploring  ways  to  let  analysts  explore  the  data  within  each  bar 
of  the  chart.  For  example,  upon  finding  that  students  spend  a lot  of  time  on  example  pages,  an  analyst  might 
then  ask  to  see  a bar  chart  of  the  more  popular  URLs  for  example  pages. 

Summary  Statistics:  Simple  Textual  Summaries  While  the  Bar  Chart  tool  provides  a general  overview 
of  popularity,  the  overview  is  limited  to  one  type  of  thing  (URLs,  Classification,  etc.).  The  Summary  Statistics 
tool  generates  a variety  of  statistical  information  about  the  browsing  session  of  a particular  user  or  multiple 
users.  While  we  are  currently  exploring  the  most  appropriate  information  to  provide,  it  currently  provides  the 
mean  and  median  time  spent  on  pages,  most  visited  URL,  most  visited  Web  domain,  most  visited  classification, 
and  similar  data. 

Slideshow:  Graphical  Replays  When  exploring  the  patterns  of  a single  student,  it  is  sometimes  most 
useful  to  watch  exactly  what  the  student  did:  what  pages  did  she  visit  and  in  which  order,  with  which  on  the 
screen  simultaneously.  While  it  would  be  best  to  be  able  to  watch  over  the  students  shoulder,  the  Slideshow  tool 
provides  a reasonable  substitute  in  that  it  “replays”  the  original  Web  pages  visited  by  the  students 
chronologically,  with  multiple  simultaneous  pages  shown  in  different  frames.  This  display  is  supplemented  with 
additional  information,  such  as  time  spent  on  each  page  and  the  referring  page.  This  tool  is  particularly  useful 
when  the  content  of  the  page,  and  not  just  the  classification,  is  of  interest. 

Directed  Graph:  Complex  Graphical  Summaries  The  tools  described  so  far  provide  only  basic 
information  about  usage.  How  can  an  analyst  find  more  complex  patterns?  The  Directed  Graph  tool  provides 
users  with  a more  sophisticated  graphical  means  of  examining  a browsing  history.  A directed  graph  consists  of 
nodes  (dots)  and  directed  edges  (arrows)  that  are  used  to  illustrate  Web  pages  and  link  between  those  pages, 
respectively.  Different  characteristics  of  the  nodes  and  edges  can  be  set  to  correspond  to  different  aspects  of  the 
pages  and  links.  For  example,  different  colors  might  represent  different  classifications  and  different  node  sizes 
might  represent  different  amounts  of  time  spent  on  a page.  Similarly,  the  color  of  a link  might  represent  its 
classification.  Alternately,  the  color  of  a node  might  represent  the  sequential  time  at  which  it  was  visited 
(making  it  easier  to  consider  patterns  involving  multiple  windows;  similarly  colored  nodes  are  likely  onscreen 
at  the  same  time). 

Each  of  a node’s  four  characteristics  can  be  associated  with  one  of  eight  attributes  of  the  corresponding 
Web  page.  The  node  characteristics  are  color,  size,  horizontal  position,  and  textual  label.  Page  attributes  include 
URL,  classification,  site,  title,  sequence  number  (in  terms  of  pages  visited  in  current  window),  arrival  time, 
maximum  time  on  page,  and  total  time  on  page.  At  present,  each  link  has  only  one  customizable  characteristic, 
its  color.  The  color  may  represent  link  type,  sequence  number,  or  number  of  times  the  link  was  followed. 
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Each  node  of  a directed  graph  represents  a distinct  (window,  URL)  pair.  The  directed  graph  is  displayed  in 
one  of  two  frames,  with  the  second  frame  being  reserved  for  node  information.  When  a node  is  clicked  on, 
information  such  as  number  of  page  visits,  timestamps  and  URL  all  become  visible  in  the  second  frame.  Also 
located  in  this  frame  are  controls  for  zooming  in  and  out  on  the  entire  graph. 

The  directed  graph  may  prove  beneficial  to  those  who  can  most  easily  see  patterns  with  the  aid  of  visuals. 
Because  many  nodes  and  edges  can  all  be  examined  quickly,  patterns  of  usage  become  apparent  to  the  user. 
Combining  the  graphical  means  of  pattern  exploration  with  the  highly  customizable  attributes  of  the  graph,  the 
directed  graph  can  be  an  excellent  tool  for  pattern  analysis. 

Pattern  Matching:  Complex  Exploration  of  Log  Files  Once  an  analyst  has  identified  a pattern  through 
one  of  the  previous  tools  (e.g.,  they  may  see  one  student  who  follows  many  links,  but  often  backtracks,  as  if 
they  followed  the  wrong  link)  or  through  speculation  (e.g.,  one  might  postulate  that  good  students  always  check 
prerequisite  links;  the  best  students  quickly  realize  that  they  know  the  prerequisite  material  and  return  to  the 
page),  analysts  may  then  wish  to  explore  when  and  how  often  that  pattern  occurs  within  the  browsing  history  of 
a single  student  or  the  class.  The  Pattern  Matching  tool  is  particularly  useful  for  searching  through  the  logs  of 
all  students  being  tracked  and  letting  the  analyst  see  how  many  and  who  are  following  a certain  pattern.  For 
example,  consider  the  following  sequence  of  events: 

1.  The  student  went  to  a 'Search  Engine'  page. 

2.  The  student  went  to  a 'Search  Results'  page  for  less  than  ten  seconds. 

3.  The  student  went  back  to  a 'Search  Engine'  page. 

This  pattern  would  most  likely  indicate  a failed  search.  Because  the  Pattern  Matching  tool  relies  on 
classifications,  failed  searches  can  be  found  at  Yahoo  and  Lycos,  not  just  Google.  The  Pattern  Matching  tool 
has  a small  language  embedded  in  it  that  shortens  queries  to  just  a few  characters.  For  example,  the  pattern 
above  would  be  written  in  this  language  as  ’ SE/SR{  <10 } /SE  ' . SE  here  stands  for  Search  Engine,  SR  for 
Search  Results,  the  '/'  indicates,  "and  then,"  and  the  brackets  indicate  a time  interval.  The  Pattern  Matching 
language  provides  a number  of  other  special  patterns,  including  an  “or”,  an  “at  the  same  time  as”  and  even  a 
way  to  classify  pages  on  the  fly.  The  results  are  returned  as  time  intervals  for  a particular  user,  and  links  are 
provided  to  the  directed  graph,  the  log  file,  and  the  slideshow. 

We  consider  the  time -on-page  pattern  particularly  useful  as  it  helps  one  distinguish  between  apparently 

similar  by  actually  dissimilar  navigation  strategies.  Consider  the  pattern  “Problem,  Reference  Pege, 

Example”.  A student  who  follows  that  pattern  may  be  one  who  wants  to  read  the  reference  completely  and  then 
view  some  examples.  It  could  also  be  one  who  is  simply  using  the  reference  page  as  a quick  way  to  get  to  an 
example  because  there  is  no  link  to  an  example  from  the  problem.  How  can  you  tell  the  difference?  If  the 
student  spends  a lot  of  time  on  the  reference  page,  the  student  is  probably  reading  the  page.  If  the  student 
spends  only  a short  amount  of  time  on  the  reference  page,  the  student  is  probably  using  the  page  as  a quick  way 
to  get  to  an  example. 

“Nearly  Raw  Data”:  Log  Files  and  Database  Queries  As  we  worked  t)  develop  these  tools,  we 
interviewed  a number  of  faculty  members  and  instructional  multimedia  technology  specialists  to  see  what 
features  they  might  want.  A few  noted,  in  effect,  “While  Td  probably  use  your  tools  for  my  initial  exploration,  I 
really  want  access  to  the  raw  data  for  my  final  analysis”.  Viewing  the  log  files  is  another  means  of  analyzing 
the  information  available  in  our  database.  The  log  file  can  be  downloaded  in  CSV  format  or  viewed  online. 
CSV  files  can  be  viewed  in  many  spreadsheet  programs.  Online  viewing  allows  for  the  creation  of  links  and  the 
ability  to  link  between  the  various  Clio  Analysis  tools.  One  can  sort  and  select  the  data  chronologically  or  by 
any  other  information  displayed. 

Because  the  data  are  stored  in  a relational  database,  we  also  expect  to  provide  a mechanism  for  direct 
database  queries  using  SQL.  However,  that  remains  a future  goal. 


3 Extended  Example 

Let  us  consider  a sample  session  in  which  one  uses  Clio’s  Assistants  to  explore  students  Web  use.  Professor 
Peabody  is  interested  in  studying  where  and  how  his  students  find  their  information  for  a particular  assignment. 
Peabody  asked  his  students  to  turn  on  Clio  while  searching  for  answers  on  the  Web.  (For  issues  of  privacy,  he 
first  discusses  the  uses  of  Clio  with  the  students  and  obtains  their  permission.)  Several  days  later,  the  professor 
has  his  students  turn  in  their  solutions  to  their  assignments  as  well  as  stop  using  Clio.  He  then  accesses  the 
analysis  tools  with  his  favorite  Web  browser  and  begins  using  Clio’s  Assistants. 
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Peabody  begins  by  viewing  the  log  files  of  his  students.  The  information  presented  to  him  appears  as  a 
series  of  rows  with  information  about  each  page  the  students  visited.  Having  received  permission  to  correlate 
homework  scores  with  student  identifiers,  Peabody  notes  that  Student  5 did  a wonderful  job  on  his  homework 
and  he  now  views  the  log  file  just  for  Student  5.  Because  Peabody  believes  that  the  log  file  contains  the  most 
information,  he  begins  there.  A quick  scan  of  the  beginning  of  the  file  shows  that  Student  5 spent  a lot  of  time 
on  the  reference  pages.  However,  as  Peabody  jumps  to  the  end  of  the  file,  he  sees  that  most  of  the  entries  there 
are  for  examples.  Two  questions  quickly  arise:  Did  Student  5’s  exploration  change  over  time  and  what  kinds  of 
pages  did  Student  5 use  most,  the  early  reference  pages  or  the  later  example  pages? 

The  second  question  is  perhaps  the  best  place  to  start.  Professor  Peabody  selects  the  Bar  Chart  tool  and 
asks  for  a comparison  of  number  of  visits  vs.  classification.  Both  examples  and  reference  pages  appear  quite 
frequently.  However,  Professor  Peabody  is  stunned  to  see  that  the  mostwisited  classification  was  “Course  Front 
Door”  and  the  second  most-visited  classification  was  “Search  Engine”.  Two  new  questions  arise:  Why  did 
Student  5 revisit  the  course  front  door  again  and  again?  Was  he  lost  in  cyberspace?  Perhaps  more  importantly, 
why  did  Student  5 use  search  engines  so  frequently? 

Peabody  asks  for  summary  statistics  for  Student  5 on  the  front  door.  He  sees  that  the  average  time  spent  on 
the  page  is  less  then  twenty  seconds  and  that  the  most  common  referring  page  is  “no  referrer”.  What  does  that 
mean?  It  probably  means  that  Student  5 makes  it  a habit  to  jump  back  to  the  start  of  the  course  web  when 
moving  on  to  new  information.  Perhaps  Student  5 likes  to  ground  his  exploration  from  a comfortable  place.  A 
quick  check  of  the  directed  graph  shows  a pattern  that  looks  very  much  like  what  he  postulated:  lots  of  long 
straight  trails  that  begin  with  the  front  door.  Is  this  answer  conclusive?  No,  the  best  way  to  find  out  more  is  to 
speak  with  the  student,  as  in  the  work  described  in  (Jones  and  Berger  1996). 

Now  Professor  Peabody  wonders  what  the  student  was  searching  for  outside  of  the  course  web.  He  sets  up 
a pattern  to  request  all  sequences  of  the  form  “Search  Engine,  Results  Page,  Any  Other  Page”.  He  expects  that 
by  scanning  the  pages  received,  he  will  obtain  additional  information  on  what  kinds  of  things  Student  5 was 
looking  for.  He  notices  two  things.  First,  that  many  of  the  search  results  that  Student  5 selected  are  within  his 
course  web.  That  suggests  that  the  student  is  just  using  the  search  engine  to  find  pages  with  the  course  web.  Is 
that  an  indication  that  the  links  on  his  pages  aren’t  good,  or  that  Student  5 prefers  to  search  rather  than  to  use 
links?  Another  reason  to  talk  to  the  student.  The  other  thing  Professor  Peabody  notices  is  that  many  of  the  other 
results  are  all  within  a certain  domain.  It’s  probably  worth  exploring  whether  he  should  make  direct  links  to 
those  pages. 

Before  going  on  to  other  things,  Peabody  decides  to  revisit  the  question  of  whether  students  need  a search 
engine  to  find  pages  within  his  course  web.  He  looks  through  all  students  for  the  pattern  “Search  Engine, 
Results  Page,  Page  in  My  Course  Web”.  He  finds  that  Student  5 is  the  only  student  who  uses  that  pattern.  Now 
it’s  time  to  consider  whether  or  not  that’s  a technique  he  should  suggest  to  other  students. 


4 Conclusions 

Most  discussions  of  the  World  Wide  Web  in  education  focus  on  the  direct  benefits  the  Web  provides  for 
students:  Easier  access  to  material,  ability  to  explore  material  at  the  speed  and  in  the  manner  that  best  suits  the 
student,  availability  of  a greater  variety  of  materials.  Project  Clio  provides  a very  different  benefit:  Qio’s  goal 
is  to  help  faculty  understand,  through  data  analysis,  the  pattern  and  personality  of  students’  usage  of  the  World 
Wide  Web.  For  Clio  to  fully  assist  in  the  analysis  of  data,  Clio  must  allow  the  analyst  to  explore  the  various 
patterns  of  learning  that  individuals  undergo.  Such  learning  patterns  include:  active  and  passive  approaches  to 
discovery  (Dufresne  and  Turcotte  1997)  and  different  searching  methods  (Jones  and  Berger  1996).  By 
providing  a suite  of  tools  that  take  learning  styles  into  account,  we  are  expanding  the  boundaries  of  this 
underdeveloped  area  of  research.  We  hope  that  once  scholars  are  able  to  identify  successful  patterns,  those 
patterns  can  be  incorporated  into  Web  sites  and  taught  to  less  successful  students. 

The  greatest  strengths  of  the  Clio’s  Assistants  tool  suite  are:  (1)  the  diversity  of  tools  available;  (2)  the 
interconnectedness  of  the  tools,  which  permits  the  results  of  one  tool  to  be  used  in  another;  (3)  the  ability  to 
record  information  for  a number  of  students  on  arbitrary  sites,  (4)  the  customizability  of  the  tools,  particularly 
the  directed  graph  tool;  (5)  the  support  for  information  not  commonly  available,  such  as  time  on  page  and  the 
use  of  multiple  windows;  and  (6)  the  richness  of  the  pattern  matching  language. 
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5 Future  Goals 


The  first  goal  of  the  Clio  team  is  to  try  the  tools  in  a real  classroom  situation.  We  are  currently  negotiating  with 
some  faculty  to  classify  their  pages  and  require  their  students  to  use  Clio.  However,  such  trials  require  a more 
robust  version  of  Web  Raveler  (currently  under  development)  and  a careful  consideration  of  the  impact  on 
students.  We  expect  to  gather  our  first  data  in  Fall  2002  and  analyze  it  in  the  subsequent  semester. 

Many  of  the  Clio  Assistant  tools  need  to  be  expanded.  For  example,  the  Log  File  tool  should  be  expanded 
to  allow  an  analyst  to  select  portions  of  the  log  file  so  as  to  “dig  in”  to  particular  parts  or  patterns.  For  example, 
one  might  first  select  just  the  entries  for  a particular  day,  then  just  a particular  classification.  At  the  more 
advanced  end  of  the  spectrum,  we  are  hoping  to  animate  the  directed  graphs  so  that  one  can  see  the  graph 
expand  over  time,  providing  an  alternative  to  the  slideshow  and  giving  another  node  characteristic  (that  is, 
when  the  node  appears  on  ftie  screen).  We  also  expect  that  an  animated  directed  graph  will  help  reveal 
important  aspects  of  sessions  in  which  students  return  to  certain  pages  repeatedly. 

We  are  also  considering  other  appropriate  tools.  As  suggested  earlier,  we  are  considering  ways  to  give 
direct  SQL  access  to  the  data  and  looking  for  interesting  queries  that  one  might  want  to  make  using  SQL.  We 
are  also  considering  whether  it  might  be  appropriate  to  present  logs  in  some  form  of  3-dimensional  “world”,  in 
which  there  would  be  additional  opportunities  to  represent  attributes  of  pages  and  links  (Kmiec  et  al.  2002). 
Since  the  current  tools  rely  on  the  intuition  of  those  using  the  tools  to  find  patterns,  we  are  also  looking  at  ways 
to  use  data-mining  techniques  to  automatically  find  patterns  (Pinchback  et  al.  2002). 

Finally,  we  need  to  consider  some  subtle  issues  in  more  depth.  For  example,  there  is  a question  as  to 
whether  a page  can  receive  multiple  classifications.  For  the  purpose  of  the  pattern -matching  tool,  it  m.akes  sense 
to  permit  a page  to  be  classified  in  many  ways.  However,  for  the  directed  graph  tool,  it  seems  that  only  one 
classification  can  be  used  at  a time  (if  an  example  is  red  and  a page  about  topic  X is  green,  what  color  should  an 
example  about  topic  X be?).  We  expect  that  these  kinds  of  issues  will  be  revealed  and  clarified  as  the  Clio 
Assistants  are  used  by  a variety  of  analysts. 
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Abstract:  Despite  major  differences  in  course  delivery,  the  critical  components  of  effective 
online  mathematics  teacher  education  may  not  be  different  from  the  critical  components  of 
effective  face-to-face  mathematics  teacher  education.  Research  indicates  that  components  of 


i mathematics  tcachcr  cducatiG 


nces  with  mathematics  (b) 


confronting  personal  beliefs  about  mathematics,  (c)  engaging  in  practical  inquiry,  and  (d) 
discussing  pedagogical  implications  in  the  context  of  mathematics  education  literature.  This 
paper  describes  the  case  of  an  online  mathematics  education  course  for  in-service  elementary 
teachers,  and  discusses  issues  of  design  and  effect  of  online  experiences. 


Introduction 

McGowen  & Davis  (2001a)  suggest  that  we  need  an  “antidote”  to  teachers’  conceptions  of  mathematics 
as  learning  procedures  and  getting  right  answers.  Findings  show  that  such  conceptions  are  consistently 
associated  with  observed  practice  of  teachers  (McGowen  & Davis  2001a,  2001b;  Stipek  et  al  2001)  and  that 
teachers  who  hold  such  conceptions  of  mathematics  have  lower  teacher  self-confidence  and  enjoy  mathematics 
less  than  teachers  who  hold  inquiry -oriented  conceptions  (Stipek  et  al  2001).  Attempts  to  change  teachers’ 
mathematics  conceptions  and  teaching  practice  through  teacher  education  and  professional  development 
programs  focusing  on  inquiry -oriented  mathematics  instruction  “are  minimally  effective,  in  part  because  teachers 
filter  what  they  learn  through  their  existing  beliefs”  (Stipek  et  al  2001,  214).  Teachers  assimilate  new  ideas 
without  substantially  altering  existing  beliefs  that  drive  their  practice  (Cohen  & Ball  1990).  In  terms  of  affecting 
classroom  practice,  the  new  ideas  are  either  ignored  or  they  are  interpreted  and  distorted  through  the  lens  of 
existing  beliefs.  In  this  paper  we  posit  that  instruction  needs  to  provide  critical  experiences  that  lead  teachers  to 
pedagogical  thinking  and  practice  that  enable  them  to  move  towards  an  inquiry-oriented  conception  of 
mathematics. 


Critical  Experiences 

Critical  experiences  may  be  defined  as  those  experiences  that  cause  teachers  to  reflect  on  their 
knowledge,  experience  and  beliefs  and  help  them  see  mathematics  and  mathematics  teaching  in  a new  light. 
When  moments  of  epiphany  occur,  mathematics  education  artifacts  - such  as  past  experiences,  curriculum 
documents,  classroom  situations,  ideas  from  professional  development  workshops,  from  education  articles,  and 
so  forth  - can  be  seen  as  inkblots  in  which  the  image  (or  idea)  appears  to  shift  and  teachers  see  something  new, 
something  that  was  not  apparent  to  them  before.  As  one  teacher  in  one  of  our  studies  commented,  “I  feel  like 
[this  experience]  has  cleaned  my  spectacles  and  I am  reading  the  [curriculum]  document  with  new  vision”. 
Similar  findings  are  reported  in  a study  by  McGowen  & Davis  (2001b)  where  one  teacher  noted  that  the 
experiences  provided  in  a course  of  study  “opened  my  eyes  to  a new  outlook  on  mathematics”  (444). 


In  our  research  we  have  concentrated  on  aesthetic  aspects  of  critical  experiences.  Contraiy  to  the  usual 
alignment  of  "critical”  with  "rigorously  intellectual”,  participants  in  our  study  demonstrate  that  critical 
engagement  occurs  within  an  aesthetic  context  (Gadanidis,  Hoogland  & Hill  2002).  This  was  true  for  teachers 
with  diverse  mathematical  backgrounds  and  attitudes  that  ranged  between  positive  and  negative.  It  is  interesting 
to  note  that  although  aesthetic  qualities  of  critical  experiences  for  mathematics  teachers  are  not  explicitly 
identified  in  related  studies,  aesthetic  qualities  are  in  some  cases  implied.  For  example,  McGowen  & Davis 
(2001b)  make  use  of  phrases  such  as  “we  focused  on  a [...]  beautiful  experience  in  establishing  connections” 
(439)  and  “the  atmosphere  [...]  was  electric”  (440)  to  describe  aspects  critical  experiences  for  mathematics 
teachers. 

Research  on  mathematics  teacher  development  (Cohen  & Ball  1990;  Gadanidis,  Hoogland  & Hill  2002; 
McGowen  & Davis  2001a,  2001b;  Stipek  et  al  2001)  indicates  that  the  following  are  integral  components  of 
critical  experiences: 

■ Teachers  confront  their  beliefs  about  mathematics. 

■ Teachers  have  aesthetic  experiences  with  mathematics. 

■ Teachers  engage  in  practical  inquiry. 

■ Teachers  consider  pedagogical  implications  in  the  context  of  relevant  mathematics  education  literature. 


Design  and  Effect  of  an  Online  Course 


The  discussion  of  the  design  and  effect  of  a fiill-credit  additional  qualification  online  mathematics 
education  course  for  elementary  in-service  teachers  [referred  to  hereafter  as  Online  Course]  is  organized  around 
the  four  components  of  critical  experiences  outlined  above.  It  should  be  noted  that  although  the  design  of  the 
course  incorporated  these  components,  the  components  were  explicitly  identified  after  the  course  was  designed 
and  taught.  The  original  design  of  the  Online  Course  was  based  on  successful  learning  experiences  for  teachers 
that  emerged  from  face-to-face  professional  development  sessions.  In  the  discussion  below  we  highlight  the 
design  similarities  between  the  Online  Course  and  the  face-to-face  professional  development  sessions  that 
inspired  the  design  of  the  course.  We  hope  to  illustrate  that  the  components  of  critical  experiences  for 
mathematics  do  not  depend  upon  a particular  environment  - be  that  face-to-face  or  online.  Rather,  that  critical 
experiences  depend  upon  the  quality  and  nature  of  instruction  as  identified  above. 

Prior  to  developing  the  Online  Course  the  instructor  was  a mathematics  consultant  for  a large  school 
district,  dealing  mostly  with  elementary  mathematics  program  design  and  teacher  development.  This  experience 
reinforced  research  findings  that  many  elementary  teachers  view  mathematics  as  procedures  to  be  learned  for 
getting  right  answers  (McGowen  & Davis  2001a,  2001b;  Stipek  et  al  2001).  One  of  the  goals  of  district-wide 
professional  development  was  to  help  teachers  become  aware  of  these  beliefs  and  to  examine  them  critically. 
Bringing  teachers  face-to-face  with  their  unexamined  beliefs  about  mathematics  involved  more  than  simply 
telling  or  showing  teachers  what  mathematics  is  really  like  or  how  it  may  be  different  from  their  personal  beliefs. 
Teachers  were  provided  with  opportunities  to  personally  experience  aesthetically-rich  mathematical  contexts, 
which  were  different  than  the  teachers’  historical  experiences  with  mathematics  or  the  experiences  they  may 
have  been  providing  for  their  own  students.  Likewise,  teachers  in  the  Online  Course  were  provided  with 
similarly  aesthetic  mathematics  experiences.  Such  experiences  created  a reflective  context  for  examining 
personal  in  both  the  face-to-face  and  the  online  environments. 

Experiencing  aesthetically-rich  mathematics  does  not  have  to  involve  complex  mathematics,  especially 
for  elementary  teachers.  Some  of  the  mathematics  experiences  in  the  Online  Course  involved  teachers  in 
mentally  solving  arithmetic  problems  such  as  16  x 24  and  156  + 78  + 9 (see  Activities  1 & 2 in  Figure  1).  These 
activities  were  chosen  based  on  their  positive  effect  in  previous  face-to-face  workshops  conducted  for 
elementary  teachers  and  parents.  In  such  workshops,  typically  half  the  people  in  each  group  were  asked  to  solve 
a problem  like  16  x 24  or  156  + 78  + 9 in  their  heads  and  half  the  people  to  use  pencil  and  paper.  After  a few 
minutes,  people  shared  and  explained  the  methods  they  used  in  their  groups.  Then  the  discussion  was  opened  up 
and  people  shared  and  explained  other  methods.  It  quickly  became  apparent  that  the  people  who  used  paper  and 
pencil  methods  had  little  to  say.  One  reason  for  this  was  that  most  people  used  the  same  procedure.  Another 
reason  was  that  although  they  were  able  to  describe  the  procedure  they  followed,  they  often  were  not  able  to 
explain  why.  Some  people  reverted  to  statements  like  “this  is  how  it  works  — it’s  just  a rule”.  On  the  other  hand, 
people  who  solved  the  problem  in  their  heads  shared  a variety  of  methods  and  they  understood  what  they  were 
doing  and  why  they  were  doing  it.  They  displayed  pride  in  their  individual  approaches  to  problem  solving.  There 
was  an  excitement  about  mathematical  thinking  in  the  room,  with  people  eager  to  share  their  personal  methods 
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and  quick  to  express  surprise  and  praise  for  unique  methods  that  others  shared.  A palpable  energy  was  created  in 
this  exchange. 

We  suggest  that  the  experience  of  mentally  solving  16  x 24  or  156  + 78  + 9 is  aesthetically  rich  in  that 
the  mental  processes  involved  do  not  demand  rule -based  procedures.  How  people  solve  16  x 24  depends  greatly 
on  how  they  personally  interpret  the  problem.  For  example,  some  people  may  multiply  16  and  25  and  then 
subtract  the  extra  16.  Others  may  deconstruct  the  problem  as  10  x24  + 6 x 24.  Many  other  solutions  processes 
are  possible  - even  ones  that  use  algebraic  structures  like  (20  - 4)(20  + 4).  Given  such  problems,  people  are 
eager  to  share  their  solutions,  they  express  interest  and  sometimes  surprise  in  the  solutions  of  others,  and  are 
motivated  to  try  to  come  up  with  different  solution  processes.  Open-ended  inquiry,  interest,  surprise  and 
motivation  are  characteristics  of  an  aesthetic  approach. 

Teachers  in  the  Online  Course  noticed  that  their  mental  solution  processes  were  “different  than  when  I 
did  it  with  paper  and  pencil  because  I solved  my  problem  by  starting  with  the  bigger  numbers  first  (left  to  right, 
not  right  to  left!)”.  Such  experiences  appear  to  have  helped  teachers  move  towards  questioning  traditional  views 
of  mathematics  and  developing  a deeper  understanding  of  what  constitutes  mathematical  activity  and 
mathematical  understanding  in  the  context  of  addition  and  multiplication.  “To  me,  the  implications  are  that  doing 
arithmetic  mentally  requires  real  understanding.  The  traditional  way  (on  paper,  doing  the  “ones”  first)  is  more  of 
a procedure  to  be  memorized  that  requires  little  understanding”. 

In  face-to-face  professional  development,  practical  inquiry  was  facilitated  through  a double -session 
structure.  Between  sessions  tried  out  new  ideas  in  their  classrooms  and  shared  their  experiences  and  reflections 
in  the  second  session.  Teachers  were  encouraged  to  bring  to  the  second  session  samples  of  student  work.  Many 
of  the  insights  that  teachers  gained  and  shared  arose  from  observations  of  students  doing  mathematics  and 
thinking  mathematically  in  the  context  of  the  new  ideas  that  teachers  tried  in  their  classrooms.  The  Online 
Course  involved  teachers  in  practical  inquiry  in  that  teachers  were  asked  to  explore  the  thinking  of  others, 
including  the  thinking  of  their  own  students,  in  mentally  solving  problems  like  16  x 24  and  156  + 78  + 9 (see 
Activity  3 in  Figure  1).  They  shared  and  reflected  on  these  observations  in  online  discussions.  Many  of  the 
teachers  tried  the  problems  with  their  students  and  discovered  that  they  too  used  a variety  of  methods,  and 
usually  not  standard  paper  and  pencil  procedures.  This  helped  teachers  realize  that  their  mathematical  thinking  as 
adults  was  similar  to  that  of  their  students  and  different  from  the  standard  paper  and  pencil  procedures.  Teachers 
were  impressed  by  the  creativity  of  student  answers  and  questioned  their  reliance  on  paper  and  pencil 
procedures. 

Asking  teachers  in  the  Online  Course  to  mentally  solve  problems  like  16  x 24  and  156+78  + 9 and  to 
share  their  solution  processes  also  offered  opportunity  for  practical  inquiry  into  the  nature  of  mathematics  and 
doing  mathematics.  This  set  the  context  for  discussions  of  related  pedagogy.  However,  one  would  expect  that 
practical  inquiry  would  also  involve  experimenting  with  teaching  practice,  which  was  not  a requirement  of  the 
Online  Course.  Unlike  the  face-to-face  professional  development  described  above,  the  Online  Course  did  not 
explicitly  ask  teachers  to  experiment  with  new  teaching  ideas  in  their  own  classrooms.  This  is  something  that 
will  be  reconsidered  when  redrafting  the  Online  Course. 

In  face-to-face  professional  development  sessions,  ideas  from  mathematics  education  literature  were 
shared  and  discussed.  The  Online  Course  gave  teachers  the  opportunity  to  read  such  literature.  Two  articles 
about  children  inventing  personal  algorithms  for  arithmetic  operations  (Bums,  1994;  Kamii  et  al,  1993)  provided 
a context  for  teacher  reflections  on  their  thinking  when  mentally  solving  problems  like  16  x 24  and  156  + 78  + 9 
and  for  considering  pedagogical  implications.  “To  guide  your  thinking”  questions  (see  Figure  1)  directed  teacher 
attention  to  pedagogical  issues.  In  contrast  to  the  face-to-face  professional  development  sessions  where  ideas 
verbalized  may  be  forgotten,  an  advantage  of  the  Online  Course  was  the  ‘permanent’  record  of  discussions. 
Many  teachers  revisited  past  discussions  and  created  scrapbooks  of  ‘good  ideas’  by  copying  sections  of  online 
transcripts  in  word  processing  documents.  As  was  the  case  in  the  study  by  McGowen  & Davis  (2001b),  teachers 
made  important  connections  between  their  experiences  and  ideas  in  the  articles  they  read. 

I do  agree  with  Kamii  and  Bums'  points  of  view.  I think  that  by  having  the  student  discover  a successful 
method  they  will  be  more  likely  to  internalize  and  understand  the  concept.  In  coming  up  with  their  own 
methods  they  are  doing  the  thinking  the  way  their  mind  works.  We  can  see  [in  our  discussion]  that 
everyone  processes  things  differently. 
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Discussion 


Research  indicates  that  many  teachers  are  unaware  that  an  alternative  to  a procedural  view  of 
mathematics  exists  (McGowen  & Davis  2001b).  The  interplay  between  the  mathematics  experiences,  journal 
readings,  online  reflections,  and  discussions  created  for  many  of  the  teachers  in  the  Online  Course  a critical 
experience  that  helped  them  ‘see’  mathematics  and  mathematics  teaching  in  a new  light.  As  one  teacher 
commented. 

After  seeing  how  different  people  calculate,  I better  understand  this  last  overall  expectation  in  [the 
curriculum  document].  Hmph!  It  is  not  until  I do  something  myself,  do  I more  fully  understand  the 
language  and  what  the  curriculum  is  really  driving  at.  Thus  students  need  to  explore  different  ways  of 
doing  calculations,  talk  about  it,  communicate  their  ideas  in  a variety  of  ways.  I feel  like  [this 
experience]  has  cleaned  my  spectacles  and  I am  reading  the  document  with  new  vision. 

A critical  aspect  of  the  Online  Course  was  that  teachers  confronted  their  personal  beliefs  about  mathematics  and 
mathematics  teaching  in  the  context  of  practical  inquiry  with  aesthetically-rich  mathematics  problems.  In  solving 
such  problems,  in  observing  students  solve  them,  and  in  sharing,  comparing  and  discussing  solutions  teachers 
realized  that  mathematics  problems  might  be  solved  in  many  different  ways.  Another  critical  aspect  was  the 
reading  and  discussing  journal  articles  that  placed  such  experiences  in  the  broader  context  of  mathematics 
pedagogy.  Online  discussions  allowed  the  personal  kind  of  sharing  that  created  community.  This  is  not  to 
suggest  that  such  critical  experiences  created  changes  in  teachers’  perceptions  of  mathematics  and  mathematics 
teaching  that  were  comprehensive  or  permanent  or  that  significantly  affected  their  classroom  practice.  How 
teachers  teach  is  also  greatly  affected  by  accepted  teaching  practices  in  the  wider  school  community  (Buzeika, 
1999;  Ensor,  1998)  and  by  conflicting  priorities  (Skott,  1999).  However,  such  critical  experiences,  whether  they 
are  in  online  or  face-to-face  teacher  education  or  professional  development  settings,  may  be  important  starting 
points  for  change  in  classroom  practice. 


Conclusion 

There  are  important  characteristics  of  online  learning  such  as  text -based  communication  and 
asynchronous  discussion  that  distinguish  it  from  face-to-face  learning.  However,  the  cases  of  the  Online  Course 
and  the  face-to-face  professional  development  discussed  above  indicate  that  the  components  of  critical 
experiences  transcend  such  differences.  These  components  include  (a)  aesthetic  experiences  with  mathematics, 
(b)  confronting  personal  beliefs  about  mathematics,  (c)  engaging  in  practical  inquiry,  and  (d)  discussing 
pedagogical  implications  in  the  context  of  mathematics  education  literature.  We  believe  that  in  the  design  of 
online  courses  for  mathematics  teachers,  the  primary  focus  of  teacher  educators  should  be  not  on  the 
technological  differences  but,  rather,  on  the  quality  and  the  components  of  the  critical  experiences. 
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A.  How  do  you  think  mathematically? 

Let’s  take  a close  look  at  how  your  mind  performs  arithmetic  operations.  Try  the  following  activities: 
Activity  #1 : 

• Multiply  16  and  24  in  your  head. 

■ Record  the  process  you  followed. 

■ Now  multiply  16  and  24  on  paper  using  the  procedure  you  were  taught  in  school. 

Activity  #2: 

■ Add  156,  78  and  9 in  your  head. 

“ Record  the  process  you  followed. 

■ Now  add  1 56,  78  and  9 on  paper  using  the  procedure  you  were  taught  in  school. 

Activity  #3; 

■ Ask  someone  else  (a  student,  a teacher,  a friend)  to  solve  the  following  and  then  tell  you  the  processes 
they  used: 

■ Multiply  16  and  24  in  their  head. 

■ Add  156,  78  and  9 in  their  head. 

To  guide  your  thinking: 

How  are  the  processes  you  use  to  multiply  or  add  in  your  head  similar  to  or  different  from  the  paper  and 
pencil  procedures  you  were  taught  in  school  for  multiplying  or  adding? 

B.  Jean  Piaget 

Jean  Piaget,  who  spent  a lot  of  time  researching  the  mathematical  thinking  of  young  children,  said  that  there 
are  three  types  of  knowledge  (Kamii  et  al  1993): 

1.  Physical  knowledge:  This  is  the  knowledge  of  our  physical  environment.  For  example,  colours  of 
objects,  or  the  fact  that  an  apple  falls  when  you  release  it  from  your  hand. 

2.  Social  knowledge; These  are,  in  part,  the  rules  and  conventions  that  we  make  up  to  make  our  social  life 
run  smoothly.  For  example,  we  all  agree  that  a red  traffic  light  means  stop.  Social  knowledge  is  to  some 
degree  arbitrary.  For  example,  we  could  all  agree  to  stop  on  a green  light  and  go  on  a red  light. 

3.  Logico-mathematical  knowledge:  This  is  the  knowledge  of  relationships.  This  knowledge  allows  us  to 
see  patterns  and  make  connections  in  skuations.  For  example,  we  can  see  two  circles  as  representing  a 
pair  of  glasses,  the  number  eight,  or  a snowman.  Or,  when  considering  the  following  question,  we  can 
see  different  relationships  that  lead  to  different  answers:  Which  one  of  the  numbers  3,  4, 5,  and  7 does 
not  belong? 

To  guide  your  thinking: 

Think  about  the  procedures  you  learned  in  school  for  multiplying  or  adding  numbers  on  paper.  Which  type 
of  knowledge  do  they  represent?  Why? 

Think  about  the  processes  you  used  earlier  for  multiplying  or  adding  numbers  in  your  head.  Which  type  of 
knowledge  do  they  represent?  Why? 

Think  about  the  mathematics  you  learned  in  school.  How  would  you  classify  it  in  terms  of  Piaget’s 
knowledge  categories?  Can  you  identify  some  examples  in  each  category? 


Figure  1.  Online  course  experiences. 
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Asynchronous  Video-Based  Instruction  with  Variable  Speed  Playback: 
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Abstract:  In  the  summer  of  2000,  Variable  Speed  Playback  functionality  was  added  to  an 
asynchronous  video-based  accounting  course  at  Brigham  Young  University.  This  paper  reports  the 
results  of  student  surveys  on  the  use  and  value  of  VSP  in  the  course.  Student  feedback  was 
exceptionally  positive  on  the  value  of  being  able  to  dynamically  control  the  speed  of  their  lectures-on- 
demand  up  to  2.5  times  normal  speed  with  no  pitch  distortion.  On  average,  student  time  viewing 
lectures  was  reduced  by  over  half,  with  no  reduction  in  average  quiz  and  exam  scores. 


Introduction 

In  recent  months  there  has  been  a proliferation  in  higher  education  institutions  of  online  lecture - 
based  steaming  media.  New  digital  and  streaming  video  technologies  steadily  facilitate  the  potential  for 
increased  user  control  over  audio  and  video  media  (Galbraith  2000).  Variable  Speed  Playback  (VSP) 
represents  a significant  increase  in  user  control  over  this  linear  medium.  VSP,  also  known  as  Time-Scale 
Modification  (fSM),  or  time  compression,  involves  adjusting  the  duration  of  temporal  content.  From  a 
student’s  perspective,  VSP  can  be  used  to  “speed  through  boring  or  redundant  material  and  slow  down  for 
interesting,  challenging  or  new  portions  of  audio  and  video  presentations.” 

In  the  Fall  of  2001,  tests  began  at  Btigham  Young  University  (BYU)  on  the  2xAV  VSP  plugin 
from  Enounce  Inc.  (www.enouncc.comT  The  2xAV  plugin  provided  a degree  of  control  over  their  video- 
based  accounting  lectures  that  students  had  requested  over  previous  semesters.  When  surveys  were 
distributed  to  assess  if  student  needs  were  adequately  met  with  this  tool,  developers  were  caught  off  guard 
by  both  the  overwhelmingly  positive  feedback  received,  and  the  average  rate  of  acceleration  employed. 
This  feedback  forced  developers  to  research  whether  or  not  student  claims  of  increased  learning  and 
sustained  attentiveness  at  higher  speeds  were  substantiated  by  previous  research. 

The  research  (Harrigan  2000,  1995)  appears  to  bear  out  the  students  consensus  that  VSP  abilities 
helped  them  stay  more  focused,  learn  more,  and  increased  their  exam  scores — all  at  a significant  time 
savings.  One  student  responded,  “In  order  to  grasp  everything  [at  higher  speeds]  it  is  necessary  for  me  to 
pay  closer  attention.  This  fosters  greater  understanding  of  the  course  material.  Since  I started  using  the 
plugin,  my  grades  have  gotten  better.” 

The  Study 

VSP  abilities  were  made  available  to  2,400  students  enrolled  in  a video-based,  entry-level 
accounting  course  at  BYU.  Learners  could  view  lectures  at  up  to  2.5x  the  normal  speed.  Significantly,  both 
speeding  up  and  slowing  down  the  audio  is  accomplished  without  pitch  distortion  (i.e.,  the  sound  does  not 
sound  like  chipmunks  when  accelerated).  A 6-item  survey  was  made  available  to  the  students  via  the  web- 
based  course  management  system  (Blackboard"!  Approximately  40%  of  course  enrollees  responded  to  the 
survey  over  the  last  two  semesters,  totaling  924  respondents  (n=924). 

Findings 

Ninety-seven  percent  of  students  reported  using  the  VSP  functionality  (acceleration)  and  found  it 
very  useful.  The  majority  found  that  by  accelerating  the  lecture  speed  to  between  1.3x  and  2.3x  normal 
speed,  comprehension  was  retained  while  time  to  move  through  the  lesson  content  was  decreased.  Most 
reported  accelerating  during  lengthy  explanations  or  examples  they  felt  irrelevant  or  redundant.  One 
student  commented  that  “when  my  attention  started  to  wander  I would  speed  it  up  and  sort  of 'skim*  until  I 
got  to  a part  that  I needed  to  pay  closer  attention  to  where  I would  slow  it  down  again.”  The  ability  to 
individually  control  the  rate  of  acceleration  through  the  lesson  materials  appeared  to  be  very  valuable. 
Rather  than  present  the  lectures  at  a predetermined  rate — perhaps  at  the  1 .6x  normal  speed,  students  were 
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able  to  find  their  own  “sweet  spot.**  Three  non-native  English  speaking  students  were  among  the  very  few 
who  reported  not  using  VSP  and  would  have  been  disadvantaged  by  fixed  acceleration.  One  student 
remarked,  “Initially,  I listened  at  1.5,  then  at  about  1.7.  . .1  hypothesize  that  my  brain  was  adjusting  to 
intaking  so  much  information.  Shortly  after  adjusting  to  1.7,  I bumped  up  to  2.5  with  no  problems 
whatsoever.  I feel  that  I have  really  been  intellectually  stimulated  while  listening  at  this  high  speed.**  This 
VSP  ability  is  further  supported  by  Harrigan  (2000),  Omoigui  (1999)  and  Short  (1977). 

The  study  took  an  interesting  turn  upon  evaluating  data  from  the  second  semester.  In  the  first 
semester,  respondents  reported  using  speeds  from  1.3x  to  1.8x,  averaging  out  at  1.6x  normal  speed. 
Interestingly,  the  most  recent  data  shows  students  reporting  an  average  acceleration  rate  of  2. lx  normal 
speed — reducing  the  time  to  view  a lesson  by  over  half.  The  highest  number  of  students  reported  an 
acceleration  rate  of  2.5x.  This  startling  increase  might  be  attributed  to  a number  of  factors,  with  “ear 
training*’  being  perhaps,  the  single  most  influential.  “I  found  that  the  more  I used  it,  the  faster  I could 
listen,’*  wrote  a student.  “I  started  at  1.5  and  worked  up  to  1.8.**  A 1965  study  (Orr  et  al,  1965)  noticed  that 
listeners  could  tolerate  acceleration  up  to  2. Ox,  but  that  with  8-10  hours  of  training,  significantly  higher 
speeds  were  possible.  “He  grows  on  you,**  said  another  student,  “and  you  can  understand  him,  even  when 
it  is  at  twice  the  normal  speed.  The  brain  is  an  amazing  thing.**  Voor  (1965)  also  found  that 
comprehension  levels  of  time -compressed  speech  increased  with  practice.  Harrigan  (2000),  on  the  other 
hand,  concluded  that  comprehension  declines  significantly  with  time-compression  in  the  range  of  33-50 
percent.  Intelligibility  will  decline  for  users  at  some  point  of  acceleration,  directly  compromising 
comprehension  (Heiman  et  al  1986).  It  would  appear,  however,  that  after  using  VSP  for  3.5  months, 
processing  roughly  52  hours  of  lx  video  instruction,  this  group  of  students  became  quite  conditioned  to 
viewing  lectures  at  unusually  high  rates  of  speed. 

Other  probable  factors  that  contributied  to  this  average  speed  increase  between  semesters,  include 
qualities  of  the  instructor,  students  and  technology.  Is  the  instructor  naturally  an  unusually  slow  speaker? 
Does  he  have  exceptionally  good  diction?  Students  complained  about  his  repetition  in  the  lectures.  Does 
the  fact  that  concepts  were  repeated,  and  multiple  examples  presented,  encourage  students  to  accelerate? 
They*ll  hear  it  twice  more  before  he  moves  on  to  a new  topic  anyway!  Is  it  because  rewinding  was  so 
easy?  When  complicated  or  new  concepts  were  presented,  students  reported  that  they  could  easily  step 
back  and  hear  it  again— slower?  Was  it  because  students  procrastinated,  and  were  forced  to  view  content  at 
highly  accelerated  rates  in  order  to  pass  quizzes  or  exams?  Finally,  did  the  combination  of  multiple  lecture 
enhancements  palbraith  2000)  employed  in  the  course  (video  of  professor-body  language,  supporting 
synchronized  slides  and  animations,  custom  navigation  features)  contribute  to  students*  ability  to 
comprehend  the  material  at  such  high  rates?  The  level  to  which  all  of  these  factors  contributed  is  not 
known,  but  combined,  they  may  proffer  an  explanation. 

An  important  question  yet  to  be  analyzed  is  whether  or  not  students  are  accelerating  the  lectures 
beyond  their  ability  to  fully  comprehend  the  material.  As  if  anticipating  such  a question,  one  student  wrote, 
“very  rarely  would  I jeopardize  understanding  for  speed  or  timeliness.  I found  that  at  a faster  rate,  my  mind 
wandered  less  than  when  I listened  at  a slower  rate,  because  I was  forced  to  pay  attention.**  While  mean 
scores  did  not  go  down  for  all  participants  in  the  class,  was  there  a correlation  between  individual  student 
exam  scores  and  their  reported  acceleration  rate?  This  remains  to  be  seen,  although  Harrigan  (1995)  and 
Short  (1977)  indicate  that  learning  and  scores,  in  fact,  increased  with  moderate  acceleration. 

Conclusion 

Studies  have  shown  that  VSP  is  primarily  desirable  on  information-based  content  rather  than 
entertainment  (Omoigui  et  al,  1999).  Survey  results  clearly  show  the  high  value  of  the  2xAV  plugin  and 
VSP  to  the  students  in  our  study,  where  a high  number  of  users  reported  viewing  the  lecture  material  at  the 
plugin  limit  of  2.5x  normal  speed.  Furthermore,  the  ability  to  intermittently  decelerate  and/or  accelerate 
lectures  according  to  individual  comprehension  level  and  interest,  has  the  potential  to  fundamentally 
change  the  nature  of  video  from  a traditionally,  “lean  back*’  and  “passive”  medium,  to  a “lean  forward,” 
“active”  instructional  medium. 

It  is  perhaps  arguable  whether  such  an  amount  (52  hrs)  of  lecture -format  material  represents  sound 
instructional  design  in  any  setting  or  course.  While  this  may  be  true  given  some  circumstances,  there  are  a 
number  of  good  reasons  to  incorporate  such  material  in  many  courses.  Regardless  of  one’s  philosophy  or 
preference  on  this  matter,  if  the  lecture  format  is  to  be  used  in  asynchronous  instruction,  no  learner  should 
be  with  out  the  tremendous  power  of  variable  speed  playback  control.  “It  is  pure  information  flowing  into 
my  head  without  air  bubbles.”  concluded  one  student. 
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Educational  Potential  of  eBook  Technology 

Abstract 

The  presenter  will  discuss  and  demonstrate  the  current  state  of  eBook 
technology.  More  importantly,  he  will  share  experiences  with  using  eBook 
technology  within  the  framework  of  existing  college  courses.  Characteristics  of 
this  new  technology,  both  positive  and  negative,  in  relationship  to  accessibility 
and  effectiveness  will  be  presented.  Implications  for  educators,  such  as  content 
access  and  intellectual  property  concerns,  will  be  discussed. 

Proposal 

The  World  Wide  Web  has  emerged  as  a powerful  force  for  education.  Courses  and 
expertise  can  be  accessed  outside  of  the  traditional  bounds  of  a school  or 
university.  In  the  classroom,  the  web  provides  a seemingly  endless  source  of 
information  and  resources.  Although  the  Internet  is  increasingly  becoming  a 
delivery  system  for  multimedia,  educators  continue  to  rely  on  it  to  deliver  text- 
based  materials. 

Unfortunately,  the  web  lacks  the  portability  of  the  most  traditional  educational 
medium,  the  book.  A popular  solution  is  to  envision  students  equipped  with 
laptop  computers  and  Internet  access  provided  by  wireless  and/or  wired 
additions  to  a school's  network.  Unfortunately,  this  is  an  expensive  solution, 
especially  when  the  student  can  acquire  a more  powerful,  cheaper  desktop  system 
(albeit  a less  portable  one). 

In  the  meantime,  students  and  faculty  achieve  portability  by  printing  copies  of 
needed  web  content  and  assignments  (a  wasteful,  inefficient  use  of  resources). 
Outside  of  the  computer  classroom  and  away  from  a home  system,  the  student  is 
no  more  "connected"  or  "empowered"  than  a previous  generation  carrying  a 
similar  load  of  textbooks  and  papers  around  campus. 

An  emerging,  potential  solution  is  the  development  of  handheld  electronic  books 
(or  eBooks).  The  term  “ebook”  is  often  used  to  describe  portable  reading  devices 
and  the  content  they  contain.  Current  models  are  somewhat  bigger  and  slightly 
heavier  than  their  paperback  counterparts.  Connectivity  with  a computer  and/or 
the  Internet  provides  a ready  source  of  material.  In  addition  to  electronic 
versions  of  traditional  books,  existing  web  pages  can  be  downloaded  and  read  by 
these  devices. 

Many  eBooks  have  accompanying  software  that  allows  the  presentation  of  the 
same  material  on  any  desktop  or  laptop  system.  The  price  of  some  eBook 
hardware  has  dropped  dramatically  over  the  past  few  years  and  is  likely  to 
continue  to  decrease  as  the  devices  become  more  common.  One  current  model, 
which  sells  for  just  under  $300,  can  hold  55,000  pages  of  text  and  at  least  20 
hours  of  battery  use  without  recharging.  It  is  likely  that  the  use  and  availability  of 
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eBooks  will  increase  dramatically,  regardless  of  their  application  (or  not)  in 
education. 

Unfortunately,  the  book  and  electronic  media  have  often  been  portrayed  as 
competitors  for  the  hearts  and  minds  of  students.  Traditionally,  books  and 
literacy  have  been  closely  associated  with  education.  Electronic  media  are  a vast 
wasteland  and  contribute  to  the  shortening  of  attention  spans.  Consuming 
technology  requires  isolating  yourself  in  front  of  a television  screen  or  computer 
monitor.  On  the  other  hand,  young  turks  promote  the  new  media  as  providing 
richer,  alternative  representations  of  knowledge.  Dusty  books  contain  outdated 
information;  in  the  information  age  only  the  uninformed  have  to  resort  to  RTFM 
("reading  the  manual"). 

The  integration  of  these  apparently  divergent  viewpoints  can  be  realized  through 
the  proper  use  of  eBooks.  Although  a traditionalist  will  likely  bemoan  the  loss  of  a 
certain  tactile  pleasure  in  the  reading  of  an  electronic  version  of  a book,  it  is  hard 
to  argue  that  a portable,  virtual  library  of  readings  in  the  students'  hands 
(whether  in  the  classroom,  on  a bus,  or  under  a convenient  shady  tree)  does  not 
enable  the  depth  of  learning  formerly  confined  to  a library  or  well- stocked  study. 
Electronic  books  do  not  provide  the  high-tech  frills  associated  with  current 
desktop  and  laptop  systems.  However,  the  lower  cost  and  increased  portability 
extend  the  reach  of  available  content.  This  economy  and  accessibility  are 
important  concerns  for  the  equitable  use  of  technology  in  education. 

Presumably,  they  are  more  environmentally  friendly  than  the  laser-printing 
alternative.  It  has  also  been  estimated  that  at  least  50%  of  the  books  printed  by 
current  publishers  are  destroyed  after  failing  to  sell  (hopefully,  a significant 
number  are  recycled  and  not  totally  wasted). 

The  presenter  will  discuss  and  demonstrate  the  current  state  of  eBook 
technology.  More  importantly,  he  will  share  experiences  with  using  eBook 
technology  in  existing  college  courses.  Characteristics  of  this  new  technology, 
both  positive  and  negative,  in  relationship  to  accessibility  and  effectiveness  will 
be  presented.  Implications  for  educators,  such  as  content  access  and  intellectual 
property  concerns,  will  be  discussed. 
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Abstract:  SUMMIT  (Stanford  University  Medical  Multimedia  and  Information  Technologies) 
is  currently  developing  hypermedia  applications  for  remote  anatomy  and  surgical  training. 
These  applications  include  interactive  hi^-resolution  3-D  imaging,  3-D  streaming  video  and 
haptic  (force  feedback)  tools  to  be  used  in  both  self  and  collaborative  study  modes.  These 
applications  are  network-intensive  and  the  perceived  performance  quality  of  each  one  changes 
in  a different  manner  to  varying  network  conditions.  Part  of  the  research  conducted  at 
SUMMIT  has  been  focused  on  determining  the  network  requirements  for  medical  training 
applications  like  these.  This  paper  first  presents  the  results  of  experiments  that  study  the 
usability  of  these  applications  as  the  network  conditions  change  and  then  discusses  the 
implications  that  these  results  have  on  the  development  of  similar  networked  hypermedia 
educational  applications  and  on  the  design  of  networks  to  be  used  for  them. 


Introduction 

SUMMIT  is  currently  developing  SD  applications  and  haptic  tools  that  will  be  used  for  remote 
anatomy  and  surgical  training.  The  3-D  imaging  applications  allow,  for  example,  anatomy  students  to  view 
high -resolution  images,  such  as  those  of  particular  body  structures,  in  3-D  with  the  use  of  special  glasses.  The 
3-D  streaming  video  applications  allow  the  broadcast  of  surgical  procedures.  These  applications  are  greatly 
enhanced  by  adding  haptic  tools  to  them,  which  provide  the  user  with  a force  feedback  mechanism  that  creates 
the  illusion  of  a sense  of  touch  on  the  images  or  models.  Thus,  the  student  actually  receives  tactile  feedback 
through  a computerized  model  of  a particular  organ  or  structure,  or  when  practicing  a surgical  procedure. 

The  ultimate  goal  is  to  be  able  to  run  these  applications  over  the  Internet.  Today’s  Internet,  however, 
cannot  meet  the  demands  generated  by  this  new  generation  of  applications  as  it  does  not  offer  the  networking 
functionality  and  Quality  of  Service  (QoS)  guarantees  neces  sary.  As  a result,  a good  amount  of  research  today, 
such  as  the  Next  Generation  Internet  (NGI)  and  Intemet2  initiatives,  is  focused  on  developing  an  advanced 
Internet  that  can  provide  enhanced  functionality.  While  most  people  think  of  these  improved  networks  in  terms 
of  additional  bandwidth,  it  is  important  to  note  that  there  are  other  parameters  such  as  packet  loss,  delay  and 
jitter,  which  significantly  affect  the  performance  of  sophisticated  applications  such  as  these.  Part  of  the  ongoing 
research  at  SUMMIT  is  to  study  the  effects  these  network  parameters  have  on  hypermedia  applications.  The 
remainder  of  this  paper  discusses  the  methodology  and  results  from  tests  conducted  at  SUMMIT  to  determine 
what  minimum  network  characteristics  are  required  to  support  this  kind  of  applications.  These  results  are  of 
importance  both  to  the  network  planners  and  to  the  developers  of  future  remote  educational  applications. 


Background 

The  two  applications  studied  in  this  paper  are  the  JPEG  Server  (3D -imaging)  and  the  AutoHandshake 
(haptics).  The  JPEG  Server  application  provides  interactive  access  to  high -resolution  stereo  image  sets.  An 
example  of  such  a set  is  the  full  36(f  rotation  views  of  a dissection  through  several  layers  of  the  hand  (Fig.  1). 
This  application  allows  students  to  perform  a virtual  dissection  of  the  organ  or  structure  being  studied  (such  as 
hand,  leg,  skull),  by  removing  different  layers  (such  as  skin,  fat  or  tissue).  It  thus  helps  anatomy  students  to 
gain  a better  understanding  of  organs  and  structures  at  different  layers  and  to  create  better  mental  models  of 
them.  The  JPEG  Server  uses  the  TCP  (Transmission  Control  Protocol)  network  transport  protocol  to  establish 
the  session,  do  initial  setup  procedures  and  transmit  user  requests.  The  actual  image  data  is  transmitted  using  a 
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modified  UDP  {User  Datagram  Protocol)  that  implements  reliability.  This  reliability  feature  ensures  that  every 
received  image  is  correct  when  displayed.  Due  to  the  high -resolution  characteristic  of  the  images,  a high 
bandwidth  network  connection  is  required.  Depending  on  the  size  and  complexity  of  the  image,  the  required 
bandwidth  may  go  up  to  40  Mbps  (megabits  per  second). 


(a)  (b)  (c) 

Figure  1:  Sample  3D-imaging  pictures:  (a)  Hand  with  all  layers,  (b)  Hand  with 
some  layers  (skin,  fat,  etc.)  removed,  (c)  Same  image  as  in  (b)  rotated. 

The  other  application  described  here,  the  Auto  Handshake,  is  a variant  of  the  Handshake  application, 
which  is  intended  to  train  students  remotely  in  surgical  procedures.  This  is  done  by  using  a haptic  device  at  each 
end  and  having  the  instructor  guide  the  movements  of  the  student  remotely.  The  AutoHandshake  variant  is  used 
when  no  instructor  is  available  and  the  movements  have  been  previously  recorded  on  a simulator.  This 
application  uses  the  TCP  protocol  for  initial  setup  in  the  same  way  as  the  JPEG  Server  application  and  then 
UDP  to  transmit  the  haptic  data.  For  this  particular  application  however,  the  UDP  protocol  is  not  modified  to  be 
reliable,  since  there  is  no  time  to  correct  network  transmission  errors  while  doing  a movement.  Unlike  the  JPEG 
server,  the  AutoHandshake  is  a low-bandwidth  application  that  uses  only  up  to  128  Kbps. 


Method 

The  purpose  of  the  experiments  was  to  study  the  performance  of  the  3-D  imaging  and  haptic 
applications  under  varying  network  conditions.  Since  it  is  difficult  to  actually  test  these  applications  on 
different  networks  in  different  geographic  locations,  it  was  necessary  to  recreate  the  network  conditions  in  the 
laboratory.  To  do  so,  software  tools  were  used  to  investigate  different  networks  in  order  to  get  an  idea  of  how 
much  network  parameters  (throughput,  packet  loss,  delay  and  jitter)  differ  among  different  networks.  This 
provided  ranges  of  values  for  the  parameters  of  each  of  the  networks  examined.  A network  emulator  was  then 
used  to  recreate  these  conditions  in  the  lab.  Then,  by  connecting  the  applications  across  the  emulator,  the 
perceived  quality  of  the  applications  was  evaluated  as  the  network  parameters  were  varied. 


Network  Characterization 

Network  performance  can  be  described  quantitatively  using  four  basic  parameters.  These  are:  (i) 
throughput,  the  available  bandwidth,  (ii)  packet  loss,  how  many  IP  (fnternet  Protocol)  data  packets  are  lost 
during  transmission  due  to  congestion,  link  failures  or  other  problems,  (iii)  delay,  the  time  it  takes  for  a packet 
to  get  Irom  one  end  to  the  other  and  (iv)  jitter,  the  statistical  variance  of  the  delay.  These  parameters  vary 
constantly,  even  on  the  same  network.  For  example,  depending  on  the  route  a packet  takes  to  get  from  one 
point  to  another,  it  might  need  to  go  through  a different  number  of  routers  along  the  way  leading  to  different 
delays.  Also,  the  amount  of  traffic  that  goes  through  a particular  router  changes  constantly.  Thus,  the  amount  of 
packets  queued  waiting  to  be  serviced  at  each  router  varies  constantly,  causing  different  per  packet  delays  and, 
when  the  buffers  are  full,  packet  loss.  Due  to  the  best  effort  design  of  the  IP  protocol,  packet  loss  is  not  unusual. 

A commercial  software  package  by  NetlQ  Corp  called  Chariot  was  used  to  characterize  various 
network  connections.  Chariot  operates  as  follows:  small  memory  resident  programs  called  Performance 
Endpoints  are  installed  on  computers  that  are  located  on  the  ends  of  the  connection  being  evaluated.  There  is 
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also  a controlling  module  residing  on  one  of  the  endpoints  or  on  a third-party  computer.  This  module 
communicates  with  the  endpoints  and  provides  them  with  a script  that  simulates  a particular  network 
application,  which  can  have  a particular  network  transport  protocol  (TCP,  UDP  or  others),  data  rate,  session 
duration,  packet  size,  etc.  During  the  test,  the  endpoints  measure,  among  other  things,  the  average  throughput, 
packet  loss,  delay  and  jitter  and  report  these  back  to  the  controlling  module. 

For  this  experiment,  network  characterization  was  performed  for  four  different  types  of  two-endpoint 
network  connections:  a)  connected  across  a 100  Mbps  local  area  network,  b)  connected  across  a wireless  local 
area  network  (WLAN)  using  the  IEEE  802. 1 1 standard,  c)  across  an  Internet  connection  with  one  endpoint  at 
Stanford  University  and  the  other  at  the  University  of  Wisconsin  - LaCrosse,  and  d)  across  an  htemet 
connection  with  one  endpoint  at  Stanford  University  and  the  other  at  Sweden’s  Royal  Institute  of  Technology 
(KTH).  The  latter  two  connections  go  through  Intemet2  research  and  education  infrastructures,  which  provide 
higher  performing  connections  than  across  the  commercial  Internet.  Table  1 shows  the  results  that  characterize 
the  four  networks  using  the  throughputs  and  protocols  used  by  the  two  applications  under  study. 


JPEG  Server  - 10  Mbps,  Reliable  UDP 

AutoHandshake  - 128  Kbps,  UDP 

WLAN 

100  Mbps 
LAN 

Wisconsin 

Sweden 

WLAN 

1 00  Mbps 
LAN 

Wisconsin 

Sweden 

Packet  Loss  (%) 

N/A 

0.000 

0.006 

0.130 

0.000 

0.000 

0.000 

0.024 

Delay  (ms) 

N/A 

0.200 

29.500 

82.500 

1.750 

0.200 

29.500 

82.500 

Jitter  (ms) 

N/A 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

Table  1:  Network  characterization  results  for  different  local  and  global  connections 


Network  Emulation 

Having  obtained  the  values  in  Table  1,  the  next  step  was  testing  the  perceived  quality  of  each 
application  under  varying  network  conditions.  To  perform  these  tests  using  different  values  of  packet  loss, 
delay  and  jitter,  a network  emulator  was  used.  The  setup  consisted  of  a gigabit  fiber-optic  connection  between 
the  server  hosting  the  applications  and  the  client  computer.  The  network  emulation  occurred  on  a computer  in 
the  middle  of  the  two  endpoints.  This  computer  used  a public-domain  network  emulator  software  called 
NISTNet  to  manipulate  the  data  traffic  going  between  the  two  endpoints.  NISTNet  allows  the  emulator  to 
behave  like  different  networks  by  changing  the  values  of  packet  loss,  delay  and  jitter  in  a controlled  fashion. 
Thus,  using  the  values  of  the  packet  loss,  delay  and  jitter  obtained  from  the  real  networks,  the  emulator 
parameters  were  varied  while  running  the  applications  to  determine  the  boundaries  of  acceptable  performance. 


Application  Evaluation 

As  mentioned  earlier,  the  two  applications  studied  were  the  JPEG  Server  (3-D  imaging)  and  the 
AutoHandshake  application  (haptics).  To  determine  the  application  quality,  a conventional  subjective  grading 
scheme  was  used.  The  values  for  packet  loss,  delay  and  jitter  were  changed  one  at  a time,  and  the  users  were 
asked  to  grade  the  performance  of  the  application.  It  is  worth  noting  here  that  the  users  were  familiar  with  its 
performance  under  normal  conditions.  Each  user  graded  their  experience  on  a scale  of  1 to  5.  A score  of  5 
indicated  “Excellent:  network  degradation  is  not  noticeable":  a score  of  4 indicated  "Good:  network 

degradation  is  noticeable,  but  not  annoying";  a score  of  3 indicated  "Fair:  network  degradation  is  evident  and 
annoying";  a score  of  2 indicated  "Poor:  network  degradation  is  severely  affecting  the  user  experience  and  the 
application  is  minimally  useable";  and  1 indicated  "Bad:  network  degradation  has  made  the  application 

unusable".  This  grading  scheme  is  similar  to  what  has  been  extensively  used  previously  for  subjective  video 
coding  and  transmission  quality  assessments  (Webster  et.  al  1993,  Watson,  Sasse  1998,  Jones,  Atkinson  1998). 

To  test  the  JPEG  Server,  a set  of  3-D  images  of  the  human  hand  was  used  (Fig.  I).  Six  test  subjects 
familiar  with  the  JPEG  Server  were  asked  to  rate  the  performance  of  the  application  as  the  values  of  packet  loss, 
delay  and  jitter  were  randomly  changed.  Similarly,  for  the  AutoHandshake  application,  a subjective  evaluation 
was  performed  using  six  test  subjects  that  were  fimiliar  with  the  application  and  its  capabilities  under  no 
network  degradation.  A Phantom  Desktop  haptic  device,  developed  by  SensAble  Technology,  was  used  during 
these  tests.  The  AutoHandshake  evaluation  involved  having  the  user  make  a certain  movement  with  the 
Phantom  device,  and  then  having  the  server  repeat  the  movement.  The  user  would  then  determine  if  the 
accuracy  and  smoothness  of  the  movement  being  repeated  was  equal  to  that  of  the  original. 
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Results 


Figures  2(a)-2(f)  display  the  results  for  the  JPEG  Server  and  AutoHandshake  under  different  packet 
loss,  delay  and  jitter  network  conditions.  In  these  figures,  the  characteristic  parameter  values  for  each  of  the 
four  different  network  connections  mentioned  in  Table  1 are  indicated  with  markers  (“lOOLAN”,  “WLAN”, 
“U.Wisc”  and  “Sweden”).  In  Figures  2(a>2(c),  the  “WLAN”  marker  is  crossed  out  since  this  technology 
currently  does  not  provide  the  necessary  bandwidth  for  the  JPEG  Server  application. 
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Figure  2;  JPEG  Server  performance  affected  by  (a)  packet  loss,  (b)  delay  and  (c)  jitter. 
AutoHandshake  performance  affected  by  (d)  packet  loss,  (e)  delay  and  (f)  jitter. 

Figure  2(a)  shows  how  the  JPEG  Server  performance  degrades  with  increasing  packet  loss,  reaching 
“Bad”  performance  levels  at  about  .8%  packet  loss.  Since  the  application  retransmits  packets  that  are  in  error, 
packet  loss  causes  the  images  to  be  delayed  or  not  shown  at  all.  Figure  2(b)  illustrates  how  the  performance  also 
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degrades  gradually  with  increasing  delay,  but  never  reaches  the  “Bad”  level.  It  never  reaches  this  level  because 
the  application  will  always  eventually  show  the  desired  image,  unlike  the  situation  caused  by  packet  loss,  where 
it  may  never  be  shown  due  to  continued  retransmission  requests  until  it  fails.  Figure  2(c)  shows  how  the 
application  is  fairly  insensitive  to  reasonable  values  of  jitter.  Jitter  causes  the  JPEG  server  to  have  to  wait  just  a 
few  milliseconds  more  before  displaying  the  image.  This  effect  is  barely  noticed  by  the  end  user. 

Figures  2(d)- 2(f)  show  the  results  for  the  AutoHandshake  application.  Figure  2(d)  shows  that  the 
application  is  very  resilient  to  packet  loss,  starting  to  degrade  only  at  values  higher  than  1%.  Packet  loss  simply 
causes  a few  misses  on  points  that  describe  the  path  that  the  haptic  tool  should  be  following.  Since  the  sampling 
of  points  along  the  movement  path  is  high,  these  missed  points  are  averaged  from  the  ones  that  do  arrive, 
making  the  application  lairly  insensitive  to  packet  loss.  Figure  2(e)  is  different  from  the  other  ones,  since  the 
effect  that  network  delay  has  on  the  application  is  different:  it  either  works  fine  or  it  does  not  work  at  all.  This  is 
due  to  the  force  feedback  loop  in  this  application  (Niemeyer,  Slotine  1998).  When  the  network  parameters 
force  the  feedback  loop  into  an  unstable  mode,  the  application  effectively  becomes  unusable.  In  the  studies,  the 
application  was  run  10  times  under  a given  delay  and  verified  if  it  would  work  or  not.  The  markers  indicate  the 
number  of  times  the  application  did  and  did  not  become  unstable.  The  results  differ  depending  on  whether  the 
movement  is  gentle  or  abrupt.  Again,  this  dependence  occurs  because  of  the  dynamics  of  the  force  feedback 


JPEG  Server 

AutoHandshake 

Bandwidth 

40  Mbps 

128  Kbps 

Packet  Loss 

<.01  % 

< 10% 

Delay  (one  way) 

< iOO  ms 

< 20  ms  (abrupt  movements) 

< 80  ms  (gentle  movements) 

Jitter 

Not  sensitive  to  jitter 

< 1 ms 

Table  2:  Minimum  QoS  requirements  per  application 


loop.  The  application  performance  is  particularly  sensitive  to  jitter,  as  Figure  2(f)  illustrates,  reaching  “Bad” 
performance  levels  at  around  15  ms  of  jitter.  Jitter  causes  this  degradation  as  the  arrival  of  data  points  out  of 
order  causes  the  path  to  loose  smoothness,  which  is  easily  perceived  by  the  end  user. 

The  graphs  on  Figures  2(a) -2(f)  can  be  used  to  determine  the  minimum  network  requirements  for  these 
applications.  Table  2 summarizes  the  requirements  that  these  two  applications  impose  on  the  network.  These 
requirements  have  been  taken  as  the  values  where  the  evaluated  performance  is  in  rated  “good”  or  better.  The 
threshold  for  the  JPEG  Server  is  at  about  0.01%  packet  loss  and  100  ms  one  way  delay,  as  can  be  seen  in 
Figures  2(a)  and  2(b).  For  the  AutoHandshake,  the  thresholds  are  at  10%  packet  loss  and  1 ms  jitter  as  Figures 
2(d)  and  2(f)  illustrate.  For  its  delay  threshold,  it  depends  on  the  movement:  for  abrupt  movements  it  would  be 
at  around  20  ms,  while  for  gentle  movements  it  would  be  at  around  80  ms,  as  Figure  2(e)  illustrates.  It  is  worth 
noting  here  that  this  evaluation  has  been  made  on  a per  parameter  basis:  that  is,  each  parameter  was  treated 
independently.  The  results  that  different  combinations  of  these  parameters  may  bring  have  not  yet  been  studied. 

Table  2 illustrates  one  important  point:  simply  providing  more  bandwidth  is  not  sufficient  to  support 
these  applications  or  others  like  them.  For  example,  even  though  the  AutoHandshake  application  needs  only 
128  Kbps  to  work  properly,  it  has  minimum  delay  and  jitter  requirements  that  may  be  hard  to  meet  even  for  east 
coast  to  west  coast  connections  within  the  U.S.  This  delay  constraint  for  a haptic  real-time  application  results  in 
a limitation  on  the  geographical  distance  allowable  between  a client  and  server  or  between  two  peers.  This  can 
become  a serious  problem  since  irrespective  of  the  network  functionality  available,  certain  delay  requirements 
cannot  be  met  for  geographically  dispersed  locations  as  the  transmission  speed  is  bound  by  the  speed  of  light. 


Discussion 

The  previous  results  illustrate  how  these  two  applications  impose  a series  of  requirements  on  the 
network  being  used.  These  results  can  be  used  along  with  the  ones  obtained  for  network  characterization  to 
estimate  the  performance  of  the  applications  on  different  real  networks.  Table  3 illustrates  this.  For  example, 
the  network  connection  from  Stanford  University  to  University  of  Wisconsin-La  Crosse  would  support  good  to 
excellent  performance  of  the  JPEG  Server  application,  but  only  inconsistent  use  of  the  AutoHandshake  with 
abrupt  movements.  The  connection  from  Stanford  to  KTH,  Sweden  would  not  be  good  enough  for  the  JPEG 
Server  due  to  packet  loss,  and  delay  requirements  would  not  allow  the  normal  operation  of  the  AutoHandshake 
application.  It  is  worth  noting  that  both  of  these  connections  go  through  the  Intemet2  infrastructure,  but  even  so. 
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the  application  requirements  are  higher  than  what  they  can  provide.  The  only  network  connection  of  the  ones 
characterized  that  can  fully  support  both  applications  is  the  100  Mbps  LAN,  which  is  local. 

The  delay  requirement  for  haptic  applications  is  particularly  hard  to  handle  from  the  point  of  view  of 
the  network  implementers.  First,  there  is  a theoretical  minimum  delay,  bounded  by  the  speed  of  li^t.  Then 
there  is  the  delay  that  each  router  along  the  path  causes  during  its  normal  operation  due  to  queuing  and 
switching.  A QoS  implementation  that  would  allow  minimizing  the  number  of  routers  along  a path  or 
guaranteeing  a maximum  per  router  delay  would  be  needed. 


JPEG  Server 

AutoHandshake 

Connection  Type 

Packet 

Loss 

Delay 

Jitter 

Packet 

Loss 

Delay 

Jitter 

Gentle 

Movement 

Abrupt 

Movement 

WLAN 

N/A 

N/A 

N/A 

Excellent 

Excellent 

Excellent 

Excellent 

100  Mbps  LAN 

Excellent 

Excellent 

Excellent 

Excellent 

Excellent 

Excellent 

Excellent 

U.Wisc 

Good 

Excellent 

Excellent 

Excellent 

Excellent 

Excellent  60%; 
Bad  40% 

Excellent 

Sweden 

Bad 

Good 

Excellent 

Excellent 

Excellent  80%; 
Bad  20% 

Bad 

Excellent 

Table  3:  Application  performance  in  some  characterized  network  connections 


Conclusions 

These  experiments  have  shown  how  applications  for  remote  medical  training  have  stringent  network 
requirements.  Contrary  to  common  belief,  it  is  not  just  a matter  of  higher  bandwidth.  Other  network 
parameters,  such  as  packet  loss,  delay  and  jitter,  are  also  very  important.  There  is  a need  to  better  understand 
the  limitations  that  current  and  near-future  networks  have  (Tab.  1)  and  design  the  applications  accordingly.  The 
sensitivities  and  requirements  that  a particular  application  may  have  on  a given  network  may  be  modified  by 
adjusting  a variety  of  design  factors,  including  whether  the  application  is  real  time  or  not,  the  kind  of  media  that 
is  being  transmitted,  the  expectations  of  the  end  users  and  the  network  protocols  being  used. 

A public  network  infrastructure  that  is  expected  to  support  remote  medical  training  will  need  to 
provide  QoS  guarantees  in  terms  of  packet  loss,  delay  and  jitter  (Tab.  2)  and  mechanisms  that  go  beyond  those 
provided  today.  Intemet2  infrastructures  today  are  not  capable  of  appropriately  handling  these  applications. 

Some  of  these  applications  are  currently  in  use  at  the  Medical  School,  such  as  the  JPEG  Server  for  the 
course  “Surg  219:  Human  Anatomy  and  Development”.  Future  work  at  SUMMIT  includes  the  evaluation  of 
other  network  intensive  applications,  such  as  streaming  3-D  video  and  peer-to-peer  haptic  applications,  studying 
the  network  traffic  patterns  that  are  generated  when  the  applications  are  in  use  and  evaluating  their  scalability. 
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Abstract 

This  paper  focuses  on  examining  the  various  kinds  of  pedagogical,  organisational  or  technical  elements  and 
constraints  of  successful  application  of  collaborative  learning  in  distance  education.  Our  research  relies  on 
case  studies  concerning  collaborative  learning  and  work  in  several  courses  in  virtual  learning  environments. 
Firstly,  based  on  the  implementation  and  analysis  of  educational  practices  with  clearly  differentiated  goals, 
contents  and  methodologies,  our  research  revealed  that  the  progress  of  virtual  learning  groups  goes  across 
four  critical  moments:  Group  formation,  consolidation,  development  and  dissolution.  Finally,  we  propose, 
explore  and  evaluate  tools  and  technical  functionalities  that  mediate  collaboration  and  support  effective 
virtual  collaborative  work  and  learning 


Introduction 

Net  learning  based  on  global  interactivity  is  emerging  as  an  important  element  of  a new  educational  paradigm  which 
puts  collaborative  learning  and  lifelong  learning  at  the  center  of  the  stage  (Harasim,  2000).  The  net  facilitates  access  to 
resources  asynchronously  without  geographical  and  temporal  limitations,  promote  interactions  and  information 
exchanges,  due  to  laiger  opportunities  of  interaction. 

In  this  context,  one  of  the  basic  present  educational  challenges  is  to  prepare  people  to  be  able  to  participate  in  an 
information  society  in  which  knowledge  is  the  critical  source  of  social  and  economic  development  (Cornelia,  1999).  In 
this  society,  productive  collaboration  is  the  key  issue  of  interactive  organization  networks  that  are  open  to  constant 
changes. 

A group  of  professors  from  the  Multimedia  and  Information  Sciences  department  at  the  Open  University  of  Catalonia 
(UOC),  from  Barcelona,  we  are  looking  for  ways  to  enrich  the  learning  capabilities  of  the  virtual  classrooms  in  our 
university  campus.  Within  our  teaching  practices,  we  explored  strategies  for  putting  in  place  virtual  teamwork,  thinking 
that  this  learning  methodology  will  provide  and  promote  agile  and  quality  interaction  between  professor,  students  and 
resources. 

Pursuing  this  aim,  we  focused  our  work  in  two  main  objectives:  1.  To  analyze,  from  a pedagogical  viewpoint, 
cooperative  working  processes  in  a virtual  campus  environment  to  facilitate  learning  and  apprenticeship  processes  to 
student  and  professor.  2 To  identify  the  requirements  that  communication  tools  have  to  fulfill  in  order  to  be  adequate  for 
learning  cooperatively  in  a virtual  campus. 

Description  of  reviewed  cases:  A pilot  experience 

In  this  article  we  present  a Case  Study  experience.  It  is  the  result  of  our  practice  with  teamwork  learning  in  the  virtual 
campus  of  UOC,  within  four  courses  with  different  goals,  activities,  and  methodologies,  h each  case,  the  general 
methodology  had  to  be  adapted  depending  on  size  and  group  composition  (heterogeneous  or  homogeneous),  rules  that 
should  direct  the  activity,  types  of  organization  of  working  spaces,  evaluation  strategies,  and  roles  that  students  can 
play.  Four  courses  that  included  team  project  curricular  design  were  chosen.  We  followed  154  groups  of  the  different 
subjects 
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Experience  Analyze 


By  implementing  cooperative  work  in  these  courses,  our  research  allowed  us  to  identify,  of  a methodological  point  of 
view,  that  the  progress  of  learning  groups  in  a virtual  environment  goes  across  four  critical  phases  (Guitert,  2002)  that 
require  defining  specifications  which  are  quite  different  from  those  applied  in  individual  learning  in  virtual 
environments.  In  particular,  based  on  the  two  main  objectives  defined  before,  we  obtained  the  following  results: 

In  virtual  cooperative  work,  a group  goes  through  four  phases  or  stages  during  its  learning  process.  These  phases  can  be 
more  or  less  evident  o flexible  depending  on  the  working  goal  and  methodology  of  the  course  at  hand. 

A.  Group  formation;  This  is  the  period  in  which  working  groups  are  formed.  It  is  important  that  students  themselves 
are  actively  engaged  in  forming  the  groups,  being  conscious  that  this  phase  forms  part  of  their  working  process.  At  this 
phase,  the  guidance  the  tutor  may  give  to  the  students  can  be  materialised  in  the  following  tasks:  Information:  guides 
and  resolves  issues  that  a group  may  set  as  regards  group  work  in  a virtual  environment.  Regulation:  influences  on  the 
direction  task  organisation  of  the  group  should  take;  inform  to  the  group,  if  it  is  necessary,  on  task  organisation,  how  to 
initiate  an  activity,  how  to  adapt  the  planning  system  better,  etc.  Support:  promotes  the  proper  conditions  that  enable 
group  consolidation  and  better  task  development;  encourages  group  organisation,  shows  his/her  support  explicitly  when 
the  group  needs  it  and  tries  to  be  always  aware  of  how  the  group  proceeds  its  work  in  the  shared  space. 

B.  Consolidation:  At  this  stage,  the  groups  have  been  formed  and  they  are  prepared  to  initiate  the  laboratory  activities 
proposed  to  them.  Before  real  work  starts,  though,  it  is  the  time  when  the  group  members  should  break  the  ice  and  get  to 
know  each  other  more  deeply. 

In  a virtual  environment,  this  moment  is  important  for  two  main  reasons: 

1.  It  constitutes  a warm-up  activity  before  the  group  starts  working  on  real  learning  activities. 

It  gives  members  the  necessary  time  to  make  initial  contacts  with  each  other.  In  a non-distance  educational 
environment,  once  formed,  a group  sets  up  a meeting  and  can  start 

2.  working  immediately;  instead,  in  a virtual  environment  it  is  advisable  for  a group  to  spend  some  time  trying  to 
establish  good  organisation  and  planning  of  the  work  before  starting  working. 

In  this  stage,  the  group  has  to  decide  who  is  going  to  be  in  charge  of  co-ordinating  tasks.  It  has  also  to  make  decisions 
about  the  organisation  of  the  shared  space,  decision  making  processes,  the  wok  planning,  as  well  as,  the  frequency  the 
group  is  going  to  communicate 

C.  Development:  This  is  the  phase  when  the  group  applies  and  puts  into  practice  all  the  decisions  taken  as  regards 
scheduling  and  organisation  forms,  task  sharing,  etc.  and  when  each  member  work  cooperatively  to  carry  out  the 
assigned  tasks.  Once  a group  has  started  working  on  the  task  at  the  Development  phase,  the  tutor  will  just  wait  to  give 
his/her  support  whenever  students  will  ask  for  it.  She  or  he  will  give  support  in  case  they  have  a doubt  about  the 
content,  or  get  stuck  with  a process  or  even  when  a conflict  arises  among  them. 

D.  Closing;  Once  a group  has  reached  the  definite  resolution  of  the  project,  every  member  realises  a self-assessment  of 
the  work  process  carried  out.  The  aim  of  this  self-assessment  is  to  give  the  chance  to  each  member  to  evaluate  the 
process  they  have  been  through,  the  results  obtained  by  comparing  them  with  the  effort  made,  and  the  way  they  handled 
media  and  information  and  communication  resources  to  accomplish  the  learning  goals. 

Finally,  at  this  phase  of  the  group  work,  it  is  important  that  the  group  would  be  able  to  close  the  process  followed  by 
evaluating  not  only  the  resulting  work  but  also  the  attitudes  and  relationships  that  have  been  maintained  among  all 
members.  For  this  reason,  the  tutor  establishes  and  foments  a common  discussion  space  where  students  can  consolidate 
their  cooperative  work  process  by  generating  common  feed-back  capable  of  assessing  both  positive  and  negative  aspects 
of  the  experience. 

In  relation  to  the  second  objective  of  this  case  study,  we  identify  the  functionalities  that  tools  have  to  have  to  help 
collaborative  learning  processes  in  virtual  environments.  The  following  are  the  main  characteristics:  focus  on  student 
needs,  user-friendly,  easy  to  learn  and  to  use,  a clear  interface,  and  an  Internet  tool. 

Main  functions 

The  tools  we  used  allowed  these  two  essential  processes  take  place: 

1.  Communication  and  information  exchange  (between  students  and  between  students  and  teachers) 

2.  Organization  and  management  of  information  in  the  team 

1 . Students  of  the  Virtual  Campus  use  very  common  tools  for  interacting  and  communicating  the  electronic  mail 
(asynchronous  communication)  and  Chat  (Synchronous  communication).  Chat  is  frequently  used  for  quickly  make 
decisions.  In  very  specific  moments,  synchronous  communication  may  be  necessary.  Instead,  electronic  mail,  among 
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other  functions,  allows  information  exchange  in  a more  elaborated  way.  It  allows  answers  to  questions  be  more 
analyzed. 

2.  The  Virtual  Campus  offers  the  group  a common  disk.  Therefore,  the  group  has  an  instrument  for  exchanging  and 
storing  information.  This  way,  group  members  can  share  documents  and  elaborate  them  together.  Each  member  of  the 
group  is  allow  to  create  or  eliminate  fields. 

The  case  study,  clearly  showed  beyond  its  own  scope,  that  for  learning  processes  take  fully  advantage  of  the  new  media 
capabilities,  it  is  crucial  to  understand  how  a virtual  team  exchange  information  and  builds  on  knowledge.  Moreover,  it 
is  not  only  necessary  to  have  technological  support  for  allowing  team  cooperation  and  interaction,  but  it  is  fundamental 
to  better  handle  the  learning  process,  and  control  all  the  elements  that  intervene  on  cooperative  work  on  the  net 
environments. 
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Abstract  In  building  an  online  course  or  transforming  a traditional  course  for  online  delivery,  the  most 
critical  element  is  not  the  selection  of  appropriate  media  but  the  selection  of  the  most  effective  teaching 
strategies.  Competitive  online  courses  are  those  that  are  designed;  1)  to  involve  students  in  the  learning 
process,  2)  to  engage  them  in  conducting  learning  activities,  and  3)  to  enable  them  to  readily  apply  what 
they  learned  in  the  real  world.  This  paper  presents  a case  scenario  where  an  undergraduate  course  was 
designed  based  on  the  instructional  theory  model  of  collaborative  problem  solving  (CPS).  It 
demonstrates  the  rational  for  choosing  the  theory  and  the  design  of  the  course  following  the  guidelines 
suggested  by  the  model.  Student  feedback  and  suggestions  on  improvement  are  also  presented  in  the 
paper. 


Collaborative  Problem  Solving 

Collaborative  problem  solving  (CPS)  is  a newly  developed  instructional  theory  that  combines  two  instructional 
approaches:  cooperative  learning  and  problem-based  learning  (Nelson,  1999).  It  emphasizes  cooperation — the  key  of 
cooperative  learning — in  the  context  of  “a  carefully  constructed  problem  scenario”,  which  is  the  essence  of  problem- 
based  learning  (Savery  & Duffy,  1995).  As  a theory  model,  CPS  provides  guidelines  that  address  the  whole  process  of 
collaborative  learning  including:  a)  building  a readiness  in  students  to  learn  collaboratively;  b)  developing  group  skills; 
c)  forming  groups;  d)  engaging  in  collaborative  problem  solving;  e)  finalizing  the  process  through  appropriate  analysis, 
synthesis,  assessment,  and  closure  activities  (Nelson,  1999). 

The  goal  of  CPS  theory  is  to  develop  knowledge  of  a content  area  that  consists  of  complex  domains.  In  the 
meantime,  it  emphasizes  the  development  of  problem-solving,  critical  thinking,  and  collaboration  skills.  CPS  holds  the 
pedagogical  values  of  maximizing  the  natural  collaboration  processes  of  learners;  creating  a situated,  learner-centered 
learning  environment;  honoring  ownership  of  the  learning  experience  for  students;  encouraging  content  analysis  and 
exploration  from  multiple  perspectives;  acknowledging  the  importance  of  social  context  for  learning;  and  cultivating 
supportive  relationships  among  learners  (Nelson,  1999).  With  group  problem  solving  being  a common  practice  in  this 
age  of  information,  collaborative  problem  solving  prepares  learners  with  the  most  necessary  skills  in  the  workplace  - the 
ability  to  collaborate  and  a desire  for  lifelong  learning. 


Using  Collaborative  Problem  Solving  in  INS  344 

INS  344,  Commercial  Liability  Risk  Management  and  Insurance  was  offered  by  the  School  of  Business  at  Indiana  State 
University  as  a required  courses  for  undergraduate  students  majoring  in  insurance  and  risk  management  and  as  an 
elective  course  for  other  business  majors.  It  is  also  part  of  the  DegreeLink  program,  which  offers  students  with  associate 
degree  an  opportunity  to  get  a baccalaureate  degree  by  completing  two  additional  years  of  coursework  in  the  areas  of 
insurance  and  business  administration.  Since  most  of  the  DegreeLink  students  are  working  adults  who  are  not  able  to 
attend  classes  on  campus,  DegreeLink  courses  are  delivered  over  the  distance  using  the  Web  as  the  major  modality.  INS 


344  that  previously  was  offered  as  a face-to-face  class,  became  available  online  in  the  fall  semester  of  2001.  Sixteen 
students  enrolled  in  this  solely  web-based  course,  seven  students  were  majoring  in  Insurance  and  Risk  Management  and 
nine  identified  other  areas  of  business  including  Business  Administration,  Finance,  and  Commercial  Law  and 
Accounting. 

The  design  and  development  of  INS  344  from  face-to-face  to  the  online  format  took  place  a year  before  the 
delivery.  The  transformation  brought  dramatic  changes  to  the  course  not  only  in  terms  of  its  delivery  media  but  also  the 
instructional  approach.  Instead  of  using  the  traditional  textbook  chapter-by-chapter  coverage,  the  instructional  designer 
and  the  instructor  decided  to  change  the  instructional  design  to  a project-driven  course  with  strong  emphasis  on 
interaction  and  collaboration  among  the  learners  themselves.  Among  the  various  available  instructional  theories, 
collaborative  problem  solving  (CPS)  was  chosen  because  of  the  good  match  between  the  course  and  the  CPS  conditions 
including  type  of  content,  learning  environment,  and  characteristics  of  both  learners  and  instructor. 

Course  Content 

INS  344,  Commercial  Liability  Risk  Management  and  Insurance,  focused  on  an  examination  of  the  major  commercial 
liability  loss  exposures  including  premises,  products,  completed  operations,  contractual,  workers’  compensation,  and 
other  miscellaneous  liability  coverage.  During  this  class,  students  were  asked  to  identify  and  analyze  different  corporate 
liability  ris  ks  and  then  to  select  optimal  combinations  of  risk  treatment  and  insurance  for  financial  protection  against 
liability  losses.  The  assignments  in  this  course  were  mostly  heuristic  in  that  they  required  a complex  system  of 
knowledge  and  skills  that  could  be  combined  in  a variety  ways  to  complete  the  task  successfully.  The  content  of  the 
course  therefore  suited  the  conditions  for  using  CPS,  which  is  found  to  be  most  appropriate  with  heuristic  tasks  as 
opposed  to  procedural  tasks  (Nelson  & Reigeluth,  1997). 

In  addition  to  the  complex  nature  of  the  project  scenarios  and  the  variety  of  knowledge  and  skills  needed  for 
completing  the  assignments  and  obtaining  project  solutions,  the  individual  projects  also  varied  from  one  group  to 
another.  Thus,  there  would  not  be  a single  question  or  one  best  way  of  doing  something  - a situation  in  which  CPS  is 
found  to  be  most  appropriate  methodology  to  use  (Nelson,  1999). 

Learning  Environment 

A learning  environment  conducive  to  collaboration,  experimentation,  and  inquiry  was  found  to  be  the  most  effective 
environment  for  CPS  (Nelson,  1999).  Therefore,  building  this  type  of  learning  environment  was  the  intent  of  the 
instructor  and  the  designer  of  INS  344,  including  a component  that  would  provide  sufficient  time  for  individual  groups  to 
become  established.  To  support  interaction  among  project  team  members,  group  space  was  created  to  enable  online 
communication  via  email,  discussion  board,  file  exchange  area,  and  virtual  chat  room. 

Learner  Characteristics 

Nelson  (1999)  pointed  out  that  having  self-directed  learners  who  are  comfortable  with,  and  willing  to  take  responsibility 
for  their  own  learning  is  an  essential  factor  for  the  successful  implementation  of  CPS.  The  characteristics  of  a CPS 
learner  coincide  with  that  of  a successful  distance  learner  in  terms  of  independence  and  self-discipline  (Nipper,  1989  & 
Guan,  2000).  Other  findings  indicate  that  students  who  choose  to  take  distance  courses  demonstrated  lesser  need  for 
control  and  affection.  They  had  more  need  to  initiate  actions  than  to  wait  passively  for  actions  (Guan,  2000). 

Instructor  Characteristics 

Having  taught  three  different  online  courses,  the  instructor  was  comfortable  in  his  role  as  a facilitator  who  would  spend 
most  of  his  time  observing  students*  interactions  and  would  intervene  only  when  it  was  absolutely  necessary.  He  was 
therefore  well  suited  to  the  CPS  approach,  which  requires  that  instructors  be  flexible  and  tolerant  of  a certain  degree  of 
ambiguity  in  what  exactly  is  to  be  learned  and  how  the  learning  will  take  place  (Nelson  1999). 


Designing  and  Developing  INS  344  with  CPS 

As  a comprehensive  instructional  theory,  CPS  provides  two  general  categories  of  guidelines  to  assist  the  implementation 
process:  (a)  comprehensive  guidelines,  which  support  the  entire  process,  and  (b)  process  activities,  which  provide  step- 
by-step  guidance  for  designing  the  appropriate  learning  activities. 

Based  on  the  comprehensive  guidelines,  the  roles  and  the  responsibilities  of  the  instructor  and  the  learners  were 
clarified  as  the  following: 
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Instructor: 


Provide  resources  as  needed  by  the  learners 
Identify  roles  in  each  project  team 
Establish  project  timeline 

Formulate  questions  to  focus  learner  on  the  critical  elements  of  the  content 
Facilitate  group  building 

Collect  feedback  from  students  and  modify  learning  activities 
Offer  just-in-time  instruction  when  requested  by  learners 

Learners: 

Prepare  for  team  project  by  completing  a mini  project 
Select  roles  in  the  project  team 

Collaborate  with  team  members  in  identifying  a problem  pertinent  to  the  subject  area 
Collect  background  information  of  the  company  where  the  problem  was  identified 
Propose  and  conduct  cost-benefit  analyses  for  alternative  solutions 
Complete  online  learning  exercises  over  commercial  liability  content  areas 
Present  recommended  solutions  to  the  problem 
Share  the  final  project  report  with  the  whole  class 

Instructor  & Learners: 

Collaborate  to  determine  learning  issues 
Exchange/share  learning  resources 
Conduct  formative  and  summary  evaluations 
Provide  group  and  individual  evaluations 
Offer  suggestions  for  course  revision 

In  contrast  to  the  comprehensive  guidelines,  which  help  clarify  each  party’s  responsibilities  during  the  learning  process, 
process  activities  provide  guidelines  for  building  the  learning  events.  As  a project-driven  course,  the  content  of 
Insurance  344  was  structured  around  the  need  to  recommend  solution(s)  for  a major  business  problem  facing  a large 
corporation.  As  shown  in  Table  1,  the  learning  process  consists  of  seven  learning  modules,  each  carrying  at  least  one 
process  activity  suggested  by  CPS. 


Table  1.  INS  344  Learning  Modules  & Collaborative  Problem  Solving  Process  Activities 


Learning  Modules 

Process  Activities 

Module  1 : Introduction 

1 : Building  Readiness  (as  an  individual) 

Module  2:  Team  Building,  Role  Selection 

2:  Form  and  Norm  Groups 
4:  Define  and  Assign  Roles 

Module  3:  Warming  Up  Exercise  for  Team 
Project:  A Mini-Case 

1 : Building  Readiness  (as  a group) 

Module  4:  Company  Background 

3.  Determine  a Preliminary  Problem  Definition 

Module  5:  Problem  Identification 

3.  Determine  a Preliminary  Problem  Definition 
5:  Engage  in  Iterative  Collaborative  Problem-Solving 
Process 

Module  6:  Solution  Alternatives - 
Cost/Benefit  Analyses 

5:  Engage  in  Iterative  Collaborative  Problem-Solving 
Process 

6.  Finalize  Solution  Alternatives 

7.  Conduct  Financial  Analysis 

Module  7:  Recommended  Solutions 

8.  Synthesize  and  Reflect 

Course  Survey 

9.  Assess  Products  and  Processes 

10.  Provide  Closure 
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Although  the  learning  modules  were  in  accord  with  the  CPS  process  activities,  the  sequence  of  procedures  was 
slightly  altered.  Because  the  course  was  offered  totally  online,  adjustment  was  made  in  a few  areas:  a)  introduction  to 
online  learning,  b)  pre -defined  roles  of  project  members,  and  c)  instructor-built  project  timeline. 

Introduction  to  Online  Learning 

An  introductory  module  was  created  for  the  very  beginning  of  the  course  to  prepare  students  for  online  learning.  In  this 
module,  each  student  was  asked  to  find  results  for  a given  problem  related  to  risk  measurement.  To  complete  this 
assignment,  students  would  learn  to  navigate  the  course  site,  review  online  documents,  open  an  attached  sample  report, 
and  use  the  Class  Discussion  Board  to  post  their  own  responses. 

Pre-defmed  roles  of  Pro  iect  Members 

Instead  of  letting  students  define  their  own  roles  in  the  project  team,  the  roles  were  defined  by  the  instructor  before  the 
team  was  built  up.  Each  project  team  consisted  of  four  members:  a project  manager,  a risk  manager,  an  insurance 
analyst,  and  a financial  analyst.  Job  descriptions  and  responsibilities  of  each  role  were  posted  on  the  course  site  for 
students  to  review.  Position  selection  took  place  in  an  online  discussion  forum  that  contained  four  discussion  threads 
named  after  the  roles,  such  as  Project  Manager,  etc.  Students  were  asked  to  claim  their  roles  by  replying  to  the 
appropriate  thread.  Since  there  was  a total  of  sixteen  students  signed  up  for  the  course,  four  project  teams  were  to  be 
organized,  with  only  four  opportunities  available  for  each  role.  In  other  words,  once  the  role  of  Project  Manager,  for 
example,  had  been  claimed  by  four  people,  that  position  was  closed.  A rule  was  made  clear  that  "the  person  with  the 
first  date  and  time  claiming  a position  has  priority  in  the  selection  process." 

Identifying  the  principal  roles  needed  to  complete  a design  plan  is  a critical  part  of  the  learning  process  (Bridge, 
1992;  Johnson  & Johnson,  1997;  West,  1992).  It  is  also  a time  -consuming  task  that  requires  significant  interaction  and 
communication  and  negotiation  among  the  team  members.  Considering  the  overall  semester  time  limit  and  the  amount  of 
content  to  be  covered  in  the  course,  the  activity  of  identifying  the  separate  groups  was  conducted  by  the  instructor 
instead  of  the  students.  Thus,  students  could  quickly  find  their  position  and  be  ready  for  the  forthcoming  tasks. 

Instructor-built  Project  Timeline 

The  combination  of  the  web-based  format  and  project-driven  approach  doubles  the  chance  for  students  to  lose  track  of 
time.  So,  instead  of  allowing  project  team  to  develop  their  own  individual  timeline  for  their  project,  a mile -stone  chart 
was  shared  with  the  students  at  the  beginning  the  semester  to  provided  them  with  a clear  understanding  of  due-dates  and 
deliverables.  Within  individual  modules,  there  was  also  a timeline  chart  that  highlighted  the  time  frame  expectations  for 
that  module. 


Researching  Design  of  INS  344 

Since  collaboration  is  “the  hallmark”  of  the  CPS  theory  and  is  built  upon  interaction  (Nelson,  1999),  investigating  the 
collaboration  and  interaction  among  students  within  project  groups  became  the  main  focus  of  the  research.  By  tracking 
the  quantity  and  quality  of  online  interaction  and  by  surveying  students,  the  course  instructor  and  instructional  designer 
intended  to  examine:  a)  students  participation  of  interaction,  b)  the  effectiveness  of  interaction,  c)  students  perception  of 
interaction,  d)  students  reaction  to  CPS,  e)  students  preferences  of  interaction  media,  and  f)  suggestions  for 
improvement. 

Students  Participation  and  Interaction 

In  general,  interaction  among  students  in  this  class  was  quite  intensive  with  a total  of  133  messages  posted  on  the  class 
discussion  board.  Additionally  within  the  separate  group  space,  the  intensity  of  group  interactions  was  strong  in  three  of 
the  four  groups,  although  as  seen  in  Table  2,  the  quantity  of  interaction  turned  out  to  be  quite  unbalanced. 


Table  2,  Data  from  Group  Page 


Number  of  Messages  Posted 

On-line  Chat 

File  Exchange 

Group  1 

103 

5 

0 

Group  2 

86 

5 

39 

Group  3 

92 

3 

2 

Group  4 

6 

0 

0 
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Effectiveness  of  Interaction 

All  of  the  students  agreed  or  strongly  agreed  (28%)  that  they  learned  more  by  doing  the  course  project  collaboratively 
with  other  team  members  than  they  would  have  by  working  by  themselves.  Comparing  with  groups  in  other  classes,  53% 
of  the  students  rated  their  INS  344  project  group  as  excellent,  33%  rated  good,  and  13%,  fair. 

Students  Perception  of  Interaction 

The  survey  showed  that  60%  of  the  students  expected  to  have  the  greatest  amount  of  interaction  with  their  team  members 
while  27%  expected  to  interact  mostly  with  their  team  project  manager.  One  student  expected  to  have  the  greatest 
amount  of  interaction  with  the  instructor  and  another  one  chose  the  textbook  as  the  major  interaction  object.  In  reality, 
73%  of  the  students  said  that  they  ended  up  having  the  greatest  amount  of  interaction  with  their  team  members  while 
some  (13%)  doing  so  with  classmates  outside  their  team. 

Students  Reaction  to  CPS 

When  asked,  “how  do  you  feel  about  problem-based-leaming  as  the  instructional  methodology  used?”,  93%  of  the 
students  chose  “Favorable”  and  7%  chose  “Very  strongly  in  favor.”  All  but  one  student  was  in  favor  of  the  course  being 
primarily  project  driven.  Some  of  the  comments  made  by  students  in  regard  to  the  project: 

“loved  my  group!” 

“I  liked  the  course  because  the  project  gave  me  a chance  to  see  a real  world  thing.” 

“I  liked  the  structure  and  the  project  based  learning.” 

“I  really  enjoyed  the  group  project  for  this  class.  It  was  really  nice  to  know  the  group  members.  This  way  it  was  easy  to  get  ahold 
of  them  and  be  able  to  communicate  with  them.  I know  that  my  group  was  taking  the  class  to  learn  and  it  wasn’t  Just  a class  to 
blow  off.” 

“ Communicating  with  member  was  easy” 

“Our  Instructor  was  very  helpful,  and  he  put  us  in  groups  with  people  that  we  all  work  really  well  with.” 

“ I liked  the  group  project.” 

“ Got  to  work  on  actual  real  world  company.” 

All  the  students  either  agreed  (60%)  or  strongly  agreed  (40%)  that  they  were  motivated  by  the  effort  their  team 
members  had  put  into  the  group  project.  No  one  was  discouraged  by  the  lack  of  involvement  of  any  team  members. 

They  all  believed  that  their  group  was  able  to  work  effectively  as  a team  and  that  their  team  members’  contribution  were 
equally  distributed.  Of  their  own  contribution  to  the  team  project,  80%  of  the  students  described  themselves  as  “a  strong 
worker  with  fair  amount  of  input”  while  20%  called  themselves  “THE  major  contributor”. 

Students  Preferences  of  the  Interaction  Media 

In  terms  of  the  communication  channels  used  for  interaction,  the  discussion  board  was  used  most  extensively,  followed 
by  email  and  then  face-to-face  meetings. 

Suggestions  for  Improvement 

A content  analysis  from  students’  feedback  offered  the  following  themes: 

1)  Information  & Resource  Sharing 

- Several  students  indicated  that  they  “had  difficulty  finding  some  information”  and  were  “wondering  if  I was 
getting  a broad  enough  knowledge  base  for  achieving  objectives.”  Suggestions  for  improvement  include 
“giving  an  example  of  the  final  project  at  the  start  of  class”  and  “post  a suggested  reading  list/schedule.”  It  was 
also  suggested  that  “(Instructor  should)  require  postings  of  other  group  materials,  so  that  we  way  (will)  learn 
from  others.” 

2)  Project  Direction,  Schedule,  and  Pace 

--  53%  of  the  students  said  that  “keeping  up  the  project  schedule”  had  been  the  most  challenging  aspect  of  the 
course.  Some  student  felt  that  it  is  difficult  to  “complete  everything  by  due  dates.  It  was  tough  to  lay  it  out  at 
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beginning.  Falling  behind,  then  catching  up,  quite  sporadic.**  One  student  found  it  difficult  “having  to  follow  a 
schedule,  rather  than  work  at  our  own  pace.” 

3)  Web  Access  and  Navigation 

-Learning  the  structure  of  the  site  was  found  to  be  difficult  by  some  students.  Others  encountered  technical 
problems  accessing  the  course  and  the  learning  exercises. 


Conclusion 

As  indicated  in  the  Leaner-Centered  Psychological  Principles  (APA  Online,  1999),  learning  can  be  enhanced  when 
learners  have  the  opportunity  to  interact  and  to  collaborate  with  others  on  instructional  tasks.  Collaborative  problem 
solving  (CPS)  provides  a theoretical  framework  for  instructors  and  course  designers  to  build  a learning  environment  that 
allows  for  social  interactions,  and  that  encourages  flexible  thinking  and  social  competence.  By  using  the  CPS  model,  the 
undergraduate  business  course,  INS  344  broke  down  the  traditional  lecture -and-exam  structure  and  brought  students 
closer  to  the  real  world  situation.  This  approach  cultivated  a teamwork  environment  that  mirrors  the  most  common  and 
natural  situations  in  the  age  of  information. 

This  paper  outlined  the  design  and  delivery  of  an  online  business  course  using  Collaborative  Problem  Solving 
instructional  approach.  It  has  been  demonstrated  from  the  learning  outcomes  and  students’  feedback  that  the  CPS  offered 
effective  instructional  methods  that  were  well  suited  to  the  content  and  the  context  of  the  course.  Also,  from  analysis  of 
the  groups’  learning  process,  it  was  concluded  that  there  is  need  for  continuing  evolvement  and  more  creative  usage  of 
the  CPS  guidelines. 
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Abstract:  This  paper  briefly  describes  the  WebNize  system  and  how  it  applies  a Metro  Map 
metaphor  for  organizing  guided  tours  in  Web  based  resources.  Then  experiences  in  using  the 
Metro  Map  based  tours  in  a Knowledge  Sharing  project  at  the  library  at  Aarhus  School  of 
Business  (ASB)  in  Denmark,  is  discussed.  The  Library  has  been  involved  in  establishing  a 
Learning  Resource  Centre  (LRC).  The  LRC  serves  as  an  exploratorium  for  the  development 
and  the  testing  of  new  forms  of  communication  and  learning,  at  the  same  time  as  it  integrates 
the  information  resources  of  the  electronic  research  library.  It  is  the  objective  to  create  models 
for  Intelligent  Knowledge  Solutions  that  can  contribute  to  form  the  learning  environments  of 
the  School  in  the  21st  century.  The  WebNize  system  is  used  for  sharing  of  knowledge  through 
metro  maps  for  specific  subject  areas  made  available  in  the  Learning  Resource  Centre  at  ASB. 
The  metro  maps  now  serve  as  the  entrance  to  learning  resources  for  several  hundred  students. 


1.  Introduction 

Support  for  structured  navigation  in  large  bodies  of  information  has  been  an  issue  for  hypermedia 
research  since  Bush  (1945)  introduced  the  notion  of  Trails.  In  hypermedia  research  (Trigg,  1988)  the  trail 
concept  is  called  guided  tours  and  a number  of  systems  have  been  implemented  to  support  trails  and  guided 
tours  also  for  the  Web. 

Among  the  systems  supporting  guided  tours  for  the  Web,  are:  Pooh’s  Guided  Tours  Service 
(www.infosys.tuwien.ac.at/GuidedTour/GuidedTour.html),  Footsteps  (Nicol  et  al.,  1995),  Walden’s  Paths 
(Furuta  et  al.,  1997),  and  Ariadne  (Juhne  et  al,  1998).  All  of  these  systems  provide  value-adding  services  for  the 
WWW  making  it  easier  for  the  prospective  users  of  the  WWW  to  find  their  way  through  relevant  information. 
This  paper  presents  a hypermedia  based  guided  tour  system  for  the  WWW  (Berners-Lee  et  al.,  1992),  which 
was  developed  as  an  integrated  part  of  the  open  hypermedia  system  called  Webvise(Gr0nbaek  et  al.,  1999), 
which  is  now  a commercial  product  called  WebNize.  The  WebNize  guided  tours  can  be  used  by  readers  in 
plain  browsers  without  using  Java  or  any  plug-in.  The  WebNize  system  is  a full-blown  open  hypermedia 
system,  with  an  integrated  guided  tour  editor.  WebNize  generates  guided  tours  in  plain  HTML  and  PNG 
formats  for  access  through  a plain  browser. 

The  WebNize  Guided  Tour  System  is  inspired  from  open  hypermedia  (Gronbaek  & Trigg,  1999),  classical 
hypertext  (Trigg,  1988;  Zellweger,  1989),  and  the  recent  Web  initiatives  mentioned.  WebNize  improves 
support  for  users  in  navigating  through  prepared  presentations  of  subjects.  In  addition  to  inspiration  from  the 
classical  hypertext  systems  we  have  been  inspired  from  the  ideas  of  simplified  metro  or  bus  maps  to 
communicate  information  on  routes  in  a complex  city.  This  idea  came  across  when  our  pilot  users  pointed  to  a 
handcrafted  web  page  at  Chalmers,  Sweden  (educate2.lib.chalmers.se/demopath.html).  The  WebNize  system 
including  the  Guided  Tour  System  as  described  in  this  paper  is  available  from  www.hypergenic.com. 

The  paper  is  organized  as  follows:  First  we  describe  the  context  in  which  WebNize  has  been  brought  into 
use  for  organizing  learning  materials.  Secondly,  we  describe  the  how  WebNize  support  the  organization  of 
learning  materials.  Then  we  describe  experiences  and  results  and  finally  we  conclude  the  paper. 
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2.  Learning  Resource  Centre  (LRC) 


The  Library  at  the  Aarhus  School  of  Business  presents  the  results  of  co-operation  between  the  Library  and 
the  Faculty  of  Modem  Languages  at  the  Business  School.  The  purpose  of  this  co-operation  has  been  the 
development  of  a knowledge  sharing  platform  and  tools  to  facilitate  knowledge  sharing  in  a modem 
international  environment  of  education  and  research  at  the  Aarhus  School  of  Business  (Bang,  2001). 

With  the  Library  as  the  centre,  the  co-operation  resulted  in  the  establishment  of  a Learning  Resource  Centre 
(LRC)  primarily  aiming  at  servicing  students,  teachers,  and  researchers  at  the  Faculty  of  Modem  Languages. 

The  LRC  has  been  established  in  close  co-operation  between  the  three  environments:  research, 
teaching  and  library.  This  co-operation  has  been  prompted  by  an  urgent  need  of  initiating  experiments  with  new 
forms  of  learning  and  education.  Experience  shows  that  new  teaching  methods  generate  quite  new  demands  for 
developing  new  ways  of  giving  students,  teachers,  and  researchers  access  to  quality-assessed  information 
resources  and  to  technology  support.  A modem  research  library  must  be  capable  of  supporting  the  new  learning 
environment  with  a wide  selection  of  services  matching  the  needs  of  the  users. 

Thus,  the  LRC  has  been  organized  as  an  “exploratorium”  for  students  in  new  forms  of  learning  and  for 
training  of  teachers  who  are  to  act  as  facilitators  in  the  virtual  learning  environment.  The  LRC  centre  both 
functions  as  a physical  meeting  place  and  as  a place  where  electronic  information  resources  and  learning  tools 
are  available  to  the  users  via  well-arranged  and  clear  user  interfaces  accessible  24  hours  a day. 

2.1  Learning  portal  based  on  Metro  maps 

The  Metro  map  metaphor  has  proved  to  be  a suitable  user  interface  for  at  learning  portal  for  students  and 
teachers  in  a LRC  exploratorium.  The  Metro  metaphor  is  fascinating,  familiar  to  most  people  and  easy  to 
comprehend.  It  provides  a general  view,  is  useful  for  stmeturing  a very  large  amount  of  web  sites  and  it  is  next 
to  impossible  to  get  lost  in.  The  METRO  allows  a graphically  oriented  user  an  easy  navigation  round  the 
information  and  learning  resources  displayed  in  the  show  window  of  the  electronic  study  of  the  LRC.  All  along 
you  can  follow  a predefined  search  route  for  the  drive  of  your  journey  from  station  to  station  on  a key  map. 
You  can  even  get  off  and  on  as  you  like,  without  losing  track  of  where  you  are. 

The  Library  uses  the  metro  map  metaphor  as  a basis  for  constmeting  paths  in  the  form  of  guided  tours  to 
the  learning  and  information  resources  in  the  LRC  (Jensen  & Harbo,  2000).  The  Metro  in  the  LRC  at  the 
Library  of  the  Aarhus  School  of  Business  is  based  on  the  use  of  the  open  hypermedia  system  called  WebNize. 

2.2  New  Learning  Environments  and  Guided  Tour  Systems 

In  recent  years  the  study  and  learning  environments  at  the  institutions  of  higher  education  in  Denmark 
have  changed.  The  frames  earlier  set  by  the  traditional  class-  and  lecture-room  education  do  not  suffice.  The 
students  still  follow  conventional  lectures,  but  the  responsibility  for  own  learning  is  growing.  To  a large  extent 
they  work  problem-based  and  project-orientated  and  need  to  meet  in  seminar  rooms  with  facilities  giving  access 
to  a wide  range  of  quality-assessed  information  resources  - electronic  and  printed  - and  user  support  tools  side 
by  side  with  teaching  materials  and  learning  tools.  New  and  different  physical  and  intellectual  demands  on 
facilitation  of  the  study  and  working  processes  arise  as  a natural  consequence  of  modem  teaching  methods. 
This  only  draws  a picture  of  the  facilities  supporting  the  traditional  daytime  students  at  the  universities  and  the 
institutions  of  higher  education.  Distance  learners  and  part-time  students  have  special  needs. 

Teachers  and  the  research  library  only  reach  the  students  in  these  environments  via  an  electronic  user 
interface  on  the  Internet.  The  support  for  studies  and  learning  of  such  dimensions  is  a pedagogical  challenge  to 
institutions  of  higher  education  and  to  research  libraries.  The  development  of  entirely  new  tools  and  methods 
for  presenting  library  resources  and  guiding  the  students  is  necessary  to  ensure  optimal  use  of  information 
content. 

At  the  LRC  of  the  Business  School  we  have  chosen  to  use  the  Metro  metaphor  and  the  concept  of  ‘Guided 
Tours’  as  the  educational  concept,  which  is  to  support  user  access  to  electronic  information  resources,  to  the 
learning  content  and  to  the  tools  supporting  the  learning  process  itself.  The  concept  of  ‘Guided  Tours’  in  the 
learning  portal  offers  the  opportunity  of  guiding  the  user  via  pre-defined  routes  in  the  shape  of  linear  courses. 
The  users  may  also  choose  to  plan  their  own  individual  course  based  on  their  needs. 
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3.  The  WebNize 

hypermedia  System 


We  believe  that 
support  for  guided  tours  is 
needed  in  many  application 
domains.  Teachers,  libra- 
rians, Web  journalists, 
portal  editors,  publishers, 
governmental  admini- 
stration etc.  often  wish  to 
present  procedures  or 
collections  of  information  in 
a homogeneous  manner. 

This  is  supported 
by  the  fact  that  the  notion  of 
a guided  tour  has  become 
quite  common  on  the  Web. 

The  search  terms  ’’guided 
tour"  gave  in  December 
2001  more  than  550.000  hits 
on  Google 

(www.google.com).  Many 

of  these  hits  actually  represent  sites  that  present  a certain  topic  in  a sequence  of  web  pages  and  commented 
links  like,  e.g.  the  Visual  Human  (www.madsci.org/~lynnATI),  Wind  Energy  (www.windpower.dk/tour)  and 
the  Chalmers  Library  guide  (educate2.lib.chalmers.se/demopath.  html).  The  WebNize  system  has  been 
designed  to  overcome  problems  and  limitations  of  manually  created  guided  tours  and  existing  guided  tour 
systems. 


3.1  The  metro  map  metaphor 

As  described  in  (Sandvad  et  al.,  2001)  the  metro  map  has  been  used  in  WebNize,  because: 

• It  is  an  intuitive  metaphor  - navigation  resembles  familiar  ways  of  navigating  metro  and  bus  maps 

• It  creates  an  overview  of  a large  complex  of  related  guided  tour  modules  - the  Metro  Map 

• It  maintains  focus  and  overview  for  the  user  navigating  a specific  guided  tour  module  - the  Route  Map 

Usage  of  the  metro  map  metaphor  in  guided  tours  has  shown  to  be  an  efficient  technique  for  providing 
overview  over  a set  of  web  documents  because  the  metaphor  is  intuitive  and  simple.  Many  people  around  the 
world  know  how  to  read  route  maps  for  metros  or  busses.  Given  the  destination  station  it  is  easy  to  find  which 
route  to  take  even  in  large  cities  like  Paris  and  London.  Designing  metro  maps  in  the  real  world  is,  however, 
not  a simple  matter  (Garland,  1994;  Tufte,  1997),  and  it  is  not  either  simple  on  the  Web,  where  we  don't  have  a 
fixed  underlying  map  of  the  structure  to  traverse  similar  to  a city  map.  But  we  provide  a general  tool  that  allow 
map  designers,  i.e.  the  domain  experts,  a large  degree  of  freedom  in  the  design  of  simplified  maps  of  arbitrary 
complex  underlying  structures. 

The  main  idea  in  using  the  metro  map  metaphor  in  guided  tours  is  to  consider  each  major  subject  in 
the  web  information  as  a route  in  the  metro  map.  When  the  reader  wants  to  know  something  about  a certain 
subject  s/he  just  has  to  “take  the  metro”  on  the  route  that  deals  with  that  subject.  If  the  reader  is  very  unfamiliar 
with  the  subject  s/he  should  start  at  the  central  metro  station  whereas  more  experienced  readers  can  enter  the 
metro  farther  out  on  the  route  (see  Figure  1). 

Hence,  metro  maps  are  used  to  provide  overview  when  entering  the  information  infrastructure  by 
showing  an  overview  over  all  routes.  But  the  metro  map  metaphor  has  more  to  give.  Inside  the  real  metro 
wagons  a horizontal  route  map  is  shown  with  all  the  stations  on  the  route,  and  every  time  the  metro  stops  at  a 
station  the  traveller  can  see  where  s/he  is  on  the  route,  which  stations  have  already  been  visited,  and  which 
stations  are  yet  to  be  visited  (see  Figure  2).  This  has  inspired  the  presentation  of  a route  map  above  the  Web 
document  as  will  be  shown  in  the  following. 
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To  take  full  advantage  of  the  structure  of 
the  guided  tour,  the  user  should  be  able  to  view  the 
relevant  part  of  the  tour  while  following  a route.  The 
user  should  be  able  to  monitor  the  stations  in  the 
tour  that  have  been  visited  and  where  the  user  is 
currently  located  within  the  tour.  Finally,  the  author 
of  a tour  should  help  the  user  by  adding  names 
or/and  annotation  to  the  stations. 

This  is  supported  in  the  WebNize  Guided  Tour 
System.  It  helps  users  keep  their  orientation  in  the 
tour,  e.g.  if  the  user  has  lost  orientation  because  s/he 
has  gone  off  the  tour  by  following  links  from  a page 
in  the  tour,  a quick  look  at  the  graphical  presentation 
of  the  tour  helps  the  user  regain  the  focus.  The 
annotation  support  gives  the  author  the  possibility  to 
add  comments  about  Web  pages  within  the  tour. 

This  can  be  used  in  a variety  of  use  situations,  e.g.  if 

the  author  wants  to  review  the  sites  in  the  tour,  draw  the  readers  attention  to  a specific  topic  in  the  page  or  — in 
an  educational  situation  — adding  questions  to  the  topics  touched  in  the  page. 
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Figure  2:  A route  map  generated  by  WebNize 


4.  Usage  of  WebNize 
generated  tours 

This  section  presents  the 
metro  map  metaphor  and  shows 
an  example  of  how  it  has  been 
used  in  a user  guidance  system  at 
the  Library  at  the  Aarhus  School 
of  Business.  We  present  a metro 
map  and  one  of  the  route  maps 
that  are  generated  from  the 
WebNize  guided  tour  editor.  The 
generated  metro  and  route  maps 
can  be  visited  at 
www.hba.dk/metro. 

4.1  The  generated  metro 
and  route  maps 
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Figure  1 shows  the  f'>£“*‘e  3:  WebNize  Metro  Map  Editor  Interface 

metro  map  of  the  user  guidance  system.  The  different  routes  in  the  map  present  different  recommended  ways 
through  the  user  guidance  system.  Each  route  represents  a certain  topic.  The  topics  are:  literature,  library 
orientation,  study  guidance,  project  writing,  Internet,  vocabularies,  writing  laboratory,  virtual  libraries  and 
information  retrieval.  If  the  user  wants  to  follow  the  recommended  sequence  s/he  just  has  to  click  on  the  oval 
figure  of  a route.  Otherwise  s/he  can  jump  directly  to  a particular  station  by  clicking  on  it.  When  clicking  a 
station  in  the  metro  map,  a route  map  is  shown  together  with  the  contents  of  the  parti-cular  station.  If  the 
‘Virtual  Libraries’  station  is  clicked  in  the  yellow  route  the  frameset  in  Figure  2 is  presented. 

The  frameset  consists  of  two  frames:  a navigation  frame  and  a contents  frame.  The  navigation  frame 
shows  the  selected  route  and  the  position  on  the  route  by  red  colouring  of  the  current  station.  The  contents 
frame  shows  the  contents  of  the  current  station,  i.e.  the  URL  that  the  station  represents.  Each  station  can  be 
selected  directly  by  clicking  on  it,  and  the  route  map  highlights  the  current  station.  The  Metro  and  Route  maps 
helps  maintaining  an  overview  of  where  the  user  is  in  a large  body  or  related  learning  resources. 


4.2  The  user  interface  of  the  WebNize  Guided  Tour  System 
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The  guided  tour  editor  is  a drawing  editor  (see  Figure  3)  that  makes  it  possible  to  define  the  sequencing  of 
stations  in  the  guided  tour  in  a general  graph.  Stations  can  be  added  to  the  graph  in  different  ways.  One  way  is 
adding  a URL  to  the  guided  tour  by  using  the  command  ‘Add  to  guided  tour’  in  an  extended  popup  menu  of  the 
Internet  Explorer.  URLs  (from  the  Address  field  and  links  in  the  web  pages)  can  also  be  dragged  into  the 
guided  tour  editor.  More  facilities  are  described  in  details  in  (Sandvad  et  al.,  2001)). 


Figure  4:  The  Danish  Electronic  Research  Library 
instruction  Metro  map  running  in  the  LRC. 


5.  Results  and  experiences 

Till  now,  WebNize  has  been  used  as  a developing  platform  for  the  Metro  concept  and  to  create  Guided 
tours  for  the  use  of  establishing  learning  platforms  in  the  LRC  exploratorium  at  the  Business  School.  Feedback 
from  students  and  teachers  confirm  that  the  concept  presenting  large  amounts  of  information,  courses  and 
teaching  material  in  a linear  and  transparent  manner  is  valuable.  This  gives  occasion  for  the  further 
development  and  use  of  the  concept  in  professional  learning  platforms  of  other  Danish  research  libraries. 

The  Business  School  Library  has  recently  made  use  of  the  WebNize  tool  in  designing  a portal  for 
knowledge  sharing  for  a cross-departmental  research  team  working  with  e-Business 
(www.hba.dk/webnize  lOOserver/tours/e  business/index.htmL  E-Business  Forum  is  a platform  for  knowledge 
sharing  across  professional  environments  and  frontiers,  and  it  will  develop  into  a dynamic  site  with  researchers 
sharing  their  knowledge,  commenting  on  research  results,  and  automatically  ‘reap’  information  from  databases. 

The  Business  School  Library  has  also  utilized  the  Metro  concept  for  experiments  with  user  instruction  for 
The  Danish  Electronic  Research  Library  Project  (DEF).  Here  the  Guided  Tours  have  been  used  for  structuring 
the  content  of  the  DEF  Project  as  search  instructions  for  users  (www.hba.dk/uecA  as  well  as  Figure  4 and  5. 


6.  Conclusion  and  future  work 

This  paper  has  described  the  WebNize  guided  tour  system,  which  consists  of  an  editor/viewer  helper 
application  and  a generator  to  create  a stand-alone  guided  tour  website  consisting  of  only  HTML  and  PNG 
files,  which  can  be  used  through  plain  browsers.  The  status  of  the  WebNize  guided  tour  system  is  that  it  is  now 
a product,  which  is  being  developed  and  used  for  a number  of  portal  projects  on  consulting  basis  by 
Hypergenic. 

The  Metro  Map  based  guided  tours  have  been  in  use  at  the  Aarhus  School  of  Business  since  September 
2000  (www.hba.dk/metro).  The  guided  tours  have  been  used  as  basis  for  introducing  all  first  year  students  to 
library  use  and  information  search.  In  addition  the  Metro  Map  is  the  start  page  on  all  PCs  (70)  in  the  Learning 
Resource  Centre  with  1250  students  and  130  teachers.  The  response  is  very  positive  from  the  students  as  well 
from  the  teachers  and  the  Aarhus  School  of  Business  are  using  to  the  approach  in  other  domains  beyond  library 
introduction.  A new  route  ‘Courses’  with  a station  for  each  course  is  currently  being  developed  in  collaboration 
with  the  teachers  and  the  use  of  the  system  is  by  teachers  as  well  as  students  expected  to  increase  in  the  future. 
To  a larger  extent  the  Metro  Map  is  being  used  for  structuring  ‘Virtual  Libraries’,  that  is,  targeted  vertical 
portals  holding  parts  of  the  information  resources  of  the  research  library,  targeted  at  a department,  a team  of 
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researchers,  students  or  a company.  Next  step  is  developing  the  tool,  thus  enabling  the  user  to  download  it  and 
use  it  for  creating  personal  learning  platforms  ‘my  courseware*  or  personal  libraries  ‘my  library*.  In  co- 
operation with  Hypergenic  A/S,  the  LRC  is  experimenting  on  the  development  of  such  facilities. 
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Introduction 

Companies  are  increasingly  becoming  aware  of  the  importance  of  knowledge  for  their  competitiveness  and 
success.  Knowledge  should  be  kept  and  distributed  in  the  company.  For  this  reason,  the  preservation  of  expert 
knowledge  is  becoming  an  important  challenge  for  knowledge  management.  We  argue  that  knowledge 
acquisition  techniques  can  help  to  meet  this  challenge:  we  are  using  expert  interviews  to  elicit  knowledge  from 
experts  and  compile  this  knowledge  into  a XML-based  information  system. 

Up  to  now  the  contribution  of  video  components  in  the  process  of  knowledge  presentation  in  information 
systems  has  been  underestimated.  Therefore  we  present  a concept  for  including  automatically  generated  context- 
dependent  video  sequences  from  the  recording  of  expert  interviews.  We  explicitly  focus  on  the  requirements  for 
attribution  of  video  sequences  and  information  concepts  with  metadata  and  outline  the  idea  of  a “fuzzy” 
metadata  attribution  scheme  which  is  used  to  control  automatic  selection  and  combination  of  video  sequences. 

Process  of  Knowledge  Acquisition 

Our  research  group  raise  knowledge  in  cooperative  sessions  with  experts  of  a company.  We  use  structured 
interviews,  a well  known  knowledge  elicitation  technique  (cf.  [4]).  Just  before  the  interview  the  expert  obtains  a 
special  structured  questionnaires  for  preparation.  The  interview  is  recorded  with  a video  camera  but  so  far  only 
audio  information  is  (semi-)automatically  attached  to  the  questionnaire  and  retrieved  later  in  the  information 
system.  During  the  interview  mindmaps  are  generated  by  the  knowledge  engineer  (cf.  [3])  whose  information 
modules  are  filled  up  by  information  gathered  out  of  the  interviews.  The  preparation  of  questionnaires,  the 
presentation  of  interviews  and  the  identification  of  knowledge  are  also  conducted  by  the  knowledge  engineer. 


Structure  of  an  Information  System 

An  information  system  can  be  understood  as  an  electronic  reference  book  for  browsing  and  finding  knowledge 
of  practical  experience  from  experts.  This  knowledge  could  hardly  be  made  explicit  so  far.  As  stated  in  [6]  the 
basis  for  such  an  information  system  is  a knowledge  map  consisting  of  hierarchically  arranged  knowledge  items 
which  are  called  concepts  or  information  modules.  These  modules  may  contain  contextual  information,  images, 
links  and  other  resources  which  are  needed  for  a comprehensive  presentation  of  the  concepts. 

These  different  entities  are  tagged  with  metadata  which  are  an  essential  requirement  for  connecting  the 
information  system  with  other  systems,  e.g.  learning  systems  or  tutor/training  systems.  So  far  we  use  a custom 
metadata  scheme  and  encode  the  descriptors  into  a XML-representation. 

Automatic  Generation  of  Context  Dependent  Video  Information  Resources 

In  the  previous  sections  we  have  motivated  the  usefulness  of  video  sequences  for  technical  oriented  information 
systems.  A realization  which  tries  to  generate  context  dependent  video  sequences  from  the  expert  interview 
recordings  needs  to  address  the  following  topics: 

■ linking  and  synchronization  of  video  recordings  with  concepts  from  the  knowledge  pool 

• necessary  metadata  for  automatic  recombination 

• integrated  and  interactive  visualization  of  concepts  and  video  presentation 
“Fuzzy”  Video  Attribution 

The  problem  with  temporal  media  - like  video  or  audio  recordings  - is  to  find  a precise  and  sufficient  attribution 
of  presented  concepts.  Normally  knowledge  experts  won’t  describe  a single  concept  from  beginning  to  end 
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before  they  start  discussing  the  next  one,  but  they  jump  between  describing  interrelated  concepts.  An  attribution 
mechanism  for  such  interrelated  descriptions  should  be  able  to  define  a temporal  varying  degree  of  relevance  of 
presented  semantic  concepts  in  the  recording.  We  therefore  suggest  a method  for  attribution  of  video  recordings 
called  “fuzzy”  video  attribution.  It  refines  the  concept  of  overlapping  attributes  or  conceptual  structures  (cf  [1, 

2])  by  specifying  a time-dependent  relevance  curve  for  each  attribute. 

Metadata  Requirements  for  Automatic  Video  Information  Resource  Generation 

A metadata  scheme  which  can  be  used  for  automatic  video  resource  generation  has  to  address  mainly  the  issues 
of  video  sequence  selection  and  combination.  We  therefore  suggest  the  creation  of  four  additional  metadata 
descriptors  for  video  sequences: 

■ Content  Type:  Characterization  of  video  sequence,  e.g.  description,  example 

■ Content  Level:  Characterization  of  complexity,  e.g.  introduction,  detailed  technical  description 

■ Audience:  Specification  of  user  roles  for  which  recording  is  intended,  e.g.  beginner,  technical 
experienced  user 

■ Concept  Relevance  Level:  Time-dependent  degree  of  relevance  of  recording  for  specific  topic 

The  first  three  descriptors  can  easily  be  mapped  to  Learning  Object  Metadata  (LOM,  cf  [5])  educational 
category  data  elements  (5.2  Learning  resource  type,  5.5  Intended  end  user  role,  5.10  Description).  Unfortunately 
there  is  no  possible  mapping  for  ConceptRelevanceLevel  as  this  descriptor  needs  to  encode  time-dependent 
values  for  different  concepts.  Possible  implementations  might  be  to  specify  a value  for  each  second  in  the 
corresponding  video  sequence  or  to  conduct  a Fourier  transformation  for  the  specified  relevance  curve  and 
assign  calculated  parameter  values  to  the  descriptor. 

Integrated  and  Interactive  Visualization  of  Concepts  and  Video  Presentation 

Presentation  of  video  sequences  is  only  useful  if  it  is  tightly  integrated  into  the  underlying  information  system. 
This  implies  that  navigation  between  different  topics  must  be  possible  from  within  the  video  stream  and  that  the 
semantic  context  for  the  presented  video  sequence  should  be  indicated.  Furthermore  the  relevance  level 
information  needs  to  be  presented  to  the  user  (e.g.  by  showing  a “relevance  meter”). 

The  constructed  video  presentation  also  needs  to  be  customizable  by  the  user,  e.g.  in  order  to  tell  the  system  to 
generate  only  technical  video  sequences.  Another  possible  customization  would  be  to  specify  a lower  limit  for 
the  relevance  level  which  therefore  gives  the  user  some  control  over  the  length  of  the  generated  video  sequences. 

Outlook 

Information  modules  should  be  processed  efficiently  by  a reasonable  linking  to  templates  of  an  task-oriented 
tutor  system.  Through  this  it  is  firstly  possible  to  develop  specified  task  types,  secondly  to  adjust  the  proposals  of 
solution  delivered  by  the  students.  Taken  the  presented  requirements  into  consideration  an  implementation  of  a 
prototype  video  sequence  construction  system  based  on  the  Synchronized  Multimedia  Integration  Language 
(SMIL,  cf  [7])  specification  is  currently  planned.  Navigational  elements  to  different  topics  and  presentation  of 
semantic  context  will  be  realized  by  producing  a SMIL  document. 
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Orienting  Students  to  Distance  Learning:  Strategies  for  Success 

Since  1995,  the  University  of  Northern  Colorado  has  offered  a graduate  program  in  speech- 
language  pathology  that  is  delivered  through  distributed  learning  environments.  Two  cohort  groups 
have  completed  the  program  , a third  group  will  graduate  in  August,  2002,  and  a fourth  group  began 
the  program  in  August,  2001 . One  of  the  challenges  in  designing  and  implementing  a program  that 
uses  distance  technologies  is  insuring  that  the  students,  many  of  whom  have  not  participated  in 
formal  education  for  several  years,  have  basic  competencies  in  using  technologies  to  access  courses, 
find  reference  materials,  prepare  and  transmit  class  assignments,  and  communicate  with  faculty, 
library  personnel,  administrators,  and  fellow  students.  A second,  and  no  less  important,  challenge  is 
to  assist  students  in  evaluating  their  own  learning  styles  and  strategies  to  facilitate  their  success  as 
independent,  adult  learners  in  nontraditional  learning  environment.  Introductory  orientation 
materials,  which  incorporate  basic  information  about  program  procedures  (e.g.,  class  registration) 
and  an  introduction  to  technology  are  provided  prior  to  the  initial  semester;  however,  the  faculty  felt 
a more  in-depth  experience  was  required. 

This  session  describes  the  objectives,  content  and  outcomes  of  the  class  that  was  designed  to 
meet  these  challenges.  An  earlier  version  of  this  course,  which  focused  primarily  on  development 
of  graduate  level  writing,  was  offered  to  the  first  two  cohorts.  Evaluation  of  student  and  faculty 
experiences  revealed  that  the  course  needed  to  be  redesigned  to  orient  students  to  distance  learning 
strategies,  skills,  and  technologies.  While  a writing  component  is  incorporated  in  the  course,  it  is 
used  as  a mechanism  to  meet  the  orientation  goals,  as  well  as  introduce  students  to  faculty 
expectations  for  graduate  writing.  This  course  was  offered  to  the  third  and  fourth  cohorts. 

The  redesigned  class  incorporated  the  following  six  objectives  that  students  meet  as  they 
participate  in  the  class: 

1 . Students  will  evaluate  their  learning  needs,  learning  styles,  learning  strategies,  study  habits, 
and  technology  experiences  to  facilitate  participation  as  successful  adult  learners  in 
distributed  learning  environments. 

2.  Students  will  demonstrate  competencies  in  accessing  and  transmitting  course  information 
and  assignments  online  and  using  email,  chat  rooms,  and  listservs  to  communicate  with 
fellow  students  and  the  instructor. 

3.  Student  will  develop  appropriate  topics  for  review  papers  in  communication  disorders. 

4.  Students  will  use  appropriate  strategies  to  access  and  obtain  reference  materials  through  the 
UNC  library. 

5.  Students  will  demonstrate  strategies  for  accessing  and  evaluating  internet  documents  of 
research  purposes. 

6.  Students  will  write  a review  paper  that  demonstrates  competence  in  using  a word-processing 
program  and  APA  editorial,  writing,  and  manuscript  styles. 

Activities  designed  to  help  students  meet  these  objectives  included: 

1 . Accessing  the  course  webpage  and  using  it  to  retrieve  course  information  and  communicate 
with  other  students  and  the  instructor. 

2.  Readings  and  activities  in  How  To  Be  a Successful  Distance  Student,  Marguerita  McVay 
(2000),  including  completion  of  surveys  for  self-analysis  of  learning  needs,  study  skills, 
learning  strategies,  and  technology  experiences. 

3.  Maintaining  a learning  journal  on  at  least  a weekly  basis  to  facilitate  development  of 
reflective  learning  strategies. 

4.  Developing  annotated  bibliographies  of  print  and  internet  references  using  OVID  databases 
(e.g.,  Psychinfo,  Medline,  ERIC)  and  online  search  strategies. 
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5.  Evaluating  internet  references  using  criteria  presented  in  the  McVay  text. 

6.  Selecting  a topic  in  speech-language  pathology  of  appropriate  scope  for  a short  (8-10  page 
review  paper). 

7.  Completing  the  paper  using  APA  style  guidelines. 

Students  are  instructed  how  to  submit  assignments  online  through  the  course  webpage  or  as 
attachments  to  email  and  strongly  encouraged  to  do  so. 

Finally,  examples  of  student  products  and  communication,  from  the  third  and  fourth  cohorts, 
were  presented  along  with  an  analysis  of  learning  journal  entries  from  the  third  and  fourth  cohorts. 
These  entries  provide  insights  into  students'  unique  learning  styles,  strategies  and  challenges,  their 
perceptions  of  readings,  their  development  in  use  of  technologies,  and  their  reactions  to  the 
assignments.  Patterns  of  entries  will  be  presented,  as  they  provide  a clear  picture  of  student 
outcomes  for  the  class.  For  example  many  students  expressed  increased  confidence  in  using 
technology.  Midway  through  the  course  one  student  wrote. 

This  class is  really  helping  me  to  relate  what  I've  learned  to  my  professional 

environment.  In  times  past,  I would  never  have  attempted  to  even  turn  on  a computer  at 
work.  I've  always  relegated  myself  to  "working"  at  home  where  I could  be  under  the 
watchful  eye  of  my  husband  who  is  always  quick  to  lend  a hand  with  helping  me  understand 
and  maneuver  the  computer. 

As  I am  acquiring  more  knowledge  of  the  computer,  I am  excited  about  it. 

(PB,  10/22/99) 

At  about  the  same  time,  another  student  wrote,  "I  have  become  more  confident  using  the  Internet, 
finding  different  links,  getting  focused  on  what  information  I am  looking  for."  (CR,  10/23/99). 
Patterns  that  reflect  Ihistrations  and  successes  with  using  the  tools  of  technology,  developing 
communicative  competence  in  an  on-line  environment,  and  building  interpersonal  relationships 
among  students  and  faculty  were  highlighted. 
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Abstract:  Students  construct  meaning  from  text  through  conversation  and  social  interaction.  The  aim  of  this  research  was  to  examine  how 
the  use  of  an  online,  threaded  discussion,  as  a discourse  support,  can  facilitate  deeper  levels  of  student  understanding.  The  basic  assumption 
being  tested  is  that  using  discourse  supports,  applying  various  discussion  strategies  like  reciprocal  teaching,  creates  an  underpinning  to  do 
intellectual  work  and  that  these  supports  can  be  employed  in  an  online  discussion  environment.  Four  online  discussions  were  conducted 
over  a period  of  two  months,  two  in  my  Advanced  Placement  American  Government  classes,  and  two  in  my  college-prep  American 
Government  classes.  In  an  online  discussion,  the  students  and  the  teacher  have  a visual  record  of  the  conversation  and  the  quantitative  and 
qualitative  aspects  of  the  conversations.  The  online  discussion  environment  allowed  more  students  to  participate  in  a class  discussion  and 
increased  the  amount  of  their  participation. 

Introduction 

Students  constructing  meaning  from  text  through  conversation  and  social  interaction,  teachers  creating  scaffolds  to 
support  those  activities  - these  are  ideas  that  I have  only  recently  begun  to  consciously  apply  to  my  teaching  strategies. 
Intuitively  and  through  my  model  for  teaching,  the  Socratic  method,  1 know  I have  been  basically  doing  the  good  things  for 
my  students.  However,  I realize  that  I have  missed  the  theoretical  underpinnings  that  might  allow  me  do  more  or  be  more 
consistent  in  what  I do.  Equipped  with  a better  theoretical  understanding,  1 am  energized  by  the  prospect  of  creating  a 
stronger  environment  for  my  students  in  both  classroom  work  and  the  use  of  the  Internet  as  supports  for  learning. 

My  experience  as  a member  of  the  CHSSP-TRIP  project,  in  early  February  2001,  provided  the  inspiration  to 
research  the  use  of  online  discussions  as  a way  to  support  deeper  understanding  of  course  content.  It  motivated  me  to 
investigate  the  literature  about  constructivist  ideas  and  learn  about  the  use  of  discourse  strategies  like  reciprocal  teaching. 

Many  teachers  have  “ah  ha*’  experiences  in  their  classrooms  without  a lot  of  background  in  the  learning  theory  to 
explain  the  results.  They  have  difficulty  repeating  the  same  outcome  in  other  elements  of  their  teacher  strategies;  successful 
activities  become  hit  and  miss.  However,  if  more  teachers  had  access  to  examples  of  teacher  research,  they  could  use  these  as 
templates  for  their  own  research.  Far  more  beneficial  than  obtaining  sample  lesson  plans,  teacher  research  models  provides 
teachers  with  ways  to  apply  sound  learning  theory  while  transforming  their  classrooms  into  learning  laboratories.  This  is  an 
exciting  way  to  teach. 

Research  Question 

How  can  the  use  of  an  online,  threaded  discussion,  as  a discourse  support,  facilitate  deeper  levels  of  student 
understanding? 

Assumptions 

1.  Discussion  engages  students  in  more  thinking  than  listening  to  lectures  or  just  responding  to  teacher  questions. 

a.  The  discussion  experience  contributes  to  deeper  understanding  and  retention  of  concepts.  Studies  into 
the  effects  of  reciprocal  teaching  strategies  confirm  the  relationship  between  discussion  and 
understanding  (ncrel.org). 

2.  The  use  of  discourse  supports,  like  reciprocal  teaching,  create  an  underpinning  to  do  intellectual  work  in  ways 
that  provide  assistance  in  deeper  levels  of  thinking  and  understanding.  As  Professor  Robert  Bain  has  stated, 
“Until  internalization  occurs,  all  performance  must  be  assisted”  (Bain,  TRIP). 

a.  The  reciprocal  teaching  strategies,  as  I used  them  to  support  content  comprehension  were  presented 
and  explained  to  the  students  in  the  following  order:  prediction,  clarifying,  summarizing,  and 
questioning. 

3.  The  use  of  a internet  discussion  board  creates  the  opportunity  to  extend  a discussion  outside  the  class  time, 
allowing  for  more  reflective  responses  and  create  a record  of  the  discussion  that  can  be  referred  to  at  a later 
time. 

The  Research  Plan  and  Observations  During  the  Process 

Phase  One  - Instruction  in  the  reciprocal  teaching  strategies 

First,  I modeled  the  four  parts  of  reciprocal  teaching  strategies  for  them:  prediction,  clarifying,  summarizing,  and 
questioning.  Second,  we  used  the  reciprocal  teaching  steps  to  read  another  short  selection  of  text  together.  Finally,  after 
organizing  into  groups,  I assigned  the  class  to  do  another  short  part  of  the  text  applying  the  strategies. 
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Phase  Two  - Checking  for  understanding  of  the  short  readings  and  feedback  on  their  first  use  of  these  strategies 

The  initial  reaction  from  all  six  of  my  classes  was  positive;  students  recognized  that  support  from  the  group  aided 
their  individual  understanding  and  extended  their  thinking  to  other  government  related  topics.  These  are  all  seniors.  Not  all  of 
them  are  particularly  willing  to  volunteer,  to  the  whole  class,  that  they  just  learned  something  helpful.  Ironically,  the  shy  or 
“too  cool”  groups  of  students  will  later  find  their  voice  in  the  online  discussions. 

Phase  Three  - Integrating  reciprocal  teaching  strategies  into  the  daily  life  of  the  class  and  then  practiced  in  online 
discussions 

Before  I could  give  them  an  assignment  to  use  the  discussion  strategies  online,  I wanted  to  be  sure  my  students  were 
comfortable  with  the  reciprocal  teaching  process.  Based  on  my  observations  of  their  notes,  group  conversations,  and  the 
follow-up  whole -class  discussion,  more  of  the  students  had  a better  understanding  of  the  pertinent  concepts  than  if  they  had 
read  the  article  by  themselves  before  discussing  it  in  class. 

At  the  conclusion  of  the  two  online  discussions,  all  of  the  classes,  approximately  170  students,  completed  an 
evaluation  of  their  two  online  experiences. 

Findings 

Despite  having  used  a reciprocal  teaching  strategy  to  facilitate  an  online  discussion  of  text  materials  only  once  with 
the  American  Government  classes,  the  test  scores  in  two  of  three  classes  slightly  exceeded  the  test  scores,  on  the  same  exam, 
of  the  class  the  first  semester.  There  are  many  variables  that  can  account  for  the  differences,  however,  the  online  experiences 
of  my  classes  demonstrated,  to  me,  the  usefulness  of  online  discussions.  In  an  online  discussion,  the  students  and  the  teacher 
have  a visual  record  of  the  conversations,  both  in  the  quantity  of  talk  and  quality  of  the  conversations.  While  the  teacher  can 
give  students  “robust”  comprehension  strategies  for  face-to-face  talk,  the  teacher  does  not  usually  require  a transcript  of  the 
conversations.  The  online  record  creates  links  between  assessment  and  student  accountability.  The  students  also  noticed  the 
value  of  both  the  reciprocal  teaching  strategies  and  the  online  discussion.  One  of  the  student  comments  was: 

“At  first,  I didn’t  understand  how  to  predict,  clarify,  or  summarize,  the  documents  correctly.  Yet,  with  practice,  I 
was  better  able  to  understand  the  document.  Now  that  I know  how  to  read  and  study  compositions,  I have  a better 
understanding  of  homework  and  what  I read  everyday.  After  reading  and  discussing  articles  online  each  night,  I was 
able  to  come  to  class  and  discuss  and  share  with  my  table  what  I took  to  be  the  meaning  of  the  article.  Table 
discussions  allow  people  to  share  extra  information  with  others  giving  a deeper  insight  into  the  significance  of 
writings.” 

Concluding  Thoughts  about  Teacher  Research 

One  important  question  is  how  to  bring  in  the  scaffolding  tools  to  “do”  political  science  as  students  cover  course 
content.  This  is  an  area  requiring  more  research. 

Another  question,  which  I continue  to  explore,  is  whether  or  not  the  online  conversation  about  a reading  can 
improve  a subsequent,  whole  class  discussion?  Can  the  students’  understanding  of  the  reading  enable  the  teacher  to  add 
additional  things  for  the  students  to  think  about?  There  might  be  issues  and  ideas  that  the  students  would  not  otherwise  been 
able  to  see  or  reflect  on  without  a good  understanding  of  what  they  had  read.  This  is  particularly  important  with  readings  that 
are  more  difficult  than  the  basic  textbook. 

This  project  has  made  an  impact  on  my  teaching  and  creates  important  implications  for  the  teaching  strategies  of 
other  social  science  teachers.  If  discourse  supports  are  effective  in  increasing  student  understanding  of  content,  teachers  not 
using  discourse  supports  in  their  strategies,  need  to  alter  what  they  are  doing  with  students.  If  online  use  of  discourse  supports 
can  extend  those  effective  strategies  even  further,  teachers  need  to  rethink  their  use  of  the  Internet. 
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Abstract:  We  present  a slide  presentation  software  that  incorporates  a concept  map,  which  explicitly  shows 
how  the  various  slides  (and  other  multimedia  components)  presented  are  related  to  each  other.  Furthermore, 
presentations  are  conceived  as  hypermedia  systems,  where  the  presenter  can  navigate  among  slides  (and  the 
concept  map)  instead  of  the  typical  linear  ordering  of  slides.  This  approach  will  alleviate  the  load  on  the 
audience  to  memorize,  recognize  and  process  perceived  information.  TTiis  tool  can  be  extremely  useful  to 
enhance  understanding  of  presented  material,  which  is  particularly  valuable  in  the  classroom  environment. 


Introduction 


Slide  presentation  software  systems  are  very  widely  used  teaching  aids.  Tools  like  PowerPoint  have 
contributed  enormously  by  making  the  preparation  of  slides  efficient  and  convenient,  and  by  providing 
numerous  features  that  enable  high  quality  display  of  slides.  Nonetheless,  they  offer  limited  facilities  to 
automatically  organize  the  slides  in  a logically  coherent  manner.  These  tools  typically  provide  a linear  ordering 
of  the  slides,  and  the  presenter  usually  goes  forward  from  one  slide  to  the  next,  and  occasionally  goes  back  to 
previous  slides.  This  approach  has  its  limitations,  especially  when  the  subject  matter  presented  is  complex.  It  is 
a very  common  experience  to  feel  “lost”  while  attending  a presentation  because  the  information  currently 
perceived  (visually  or  orally)  cannot  be  placed  in  its  appropriate  context.  Furthermore,  a complete  grasp  of  the 
information  being  presented  often  necessitates  considerable  memorization  and  processing  of  material  contained 
in  previous  slides.  But  the  Model  Human  Processor  (Card  et  al.  1986)  describes  the  limits  of  the  human  mind  in 
perception,  memorization,  processing,  recognition,  discrimination,  etc. 

Also,  a linear  ordering  of  slides  often  encourages  linear  thought  (Zuhn  1995)  i.e.  following  a sequential 
association  of  ideas,  linking  each  concept  with  a single  pre-requisite  concept  and  the  next  logical  concept, 
instead  of  the  more  instructive  global  thought  (i.e.  choosing  from  divergent  paths  of  multiple  linkages  of 
concepts  to  reach  a conclusion). 

Typically  the  individual  slides  in  a presentation  are  logically  connected  to  each  other,  and  the  relation 
among  slides  can  be  represented  by  a network.  This  can  be  viewed  a small-scale  hypermedia  system.  The 
objective  in  this  research  project  is  to  design,  develop  and  evaluate  a slide  presentation  tool  that  makes  the 
relation  of  individual  slides  to  the  overall  network  visible  throughout  a presentation.  That  is,  each  side  is  placed 
in  a wider  context.  This  will  make  understanding  easier  for  the  audience  as  well  as  enhance  the  expressive 
capability  of  the  presenter. 


Related  Work 


Computer-Human  Interaction  research  can  enormously  contribute  to  the  effective  dissemination  of 
knowledge  in  the  classroom  environment,  especially  when  the  design  of  hardware  and  software  systems  is 
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sensitive  to  human  and  psychological  factors,  such  as  capabilities  and  preferences  of  users.  In  (1995),  Zuhn 
describes  the  use  of  concept  maps  in  a hypermedia  system  to  help  teachers  organize  and  deliver  teaching 
material.  The  system  encourages  global  thinking  about  concepts  and  skills  to  be  learned  by  students.  Franklin  et 
al.  (2000)  show  how  PowerPoint  can  be  enhanced  with  a speech -based  interface  in  the  context  of  the  Intelligent 
Classroom  Project.  The  use  of  multiple  input  modes  (particularly  visual  ones)  to  eliminate  reliance  on  a single 
constraining  input  mode  in  a classroom  environment  is  described  in  (Flachsbart  et  al.  2000).  Visualization  of 
information,  particularly  in  the  context  of  hypermedia  systems,  is  a related  research  area.  Conventional 
interactive  systems  provide  inadequate  views  of  the  information  space  being  explored  and  hide  most  of  the 
information,  especially  about  the  organization  of  the  information.  Many  researchers  have  suggested  graphical 
overviews  for  the  users  (Nielsen  1990,  Zizi  1996).  For  example,  Zizi  (1996)  describes  topic  and  document 
mapSy  which  provide  an  abstract  global  overview  of  documents  and  their  contents.  Other  research  on 
information  visualization  includes  the  Visualizer  Project  (Card  et  al.  1991),  Tree  Maps  (Shneiderman  1992), 
Starfield  Displays  (Ahlberg  et  al.  1994). 


Design  and  Implementation 


The  system  we  present  consists  of  mainly  the  following:  (1)  a graphical  representation  of  the  network  of 
slides,  and  (2)  individual  slides  which  essentially  contain  the  material  to  be  presented  with  some  added 
information,  such  as  hyperlinks  to  other  slides.  We  use  a concept  map  to  represent  the  network.  The  nodes  in 
the  network  can  be  slides  or  documents  (document  map)  or  topics  (topic  map).  The  links  are  logical  relations 
among  nodes.  These  links  can  be  represented  in  various  ways  (such  as  directed  lines)  to  depict  different  types  of 
relations  among  slides  or  topics.  We  also  group  nodes  into  concept  regions  which  associate  logically  related 
slides  based  on  proximity  on  a concept  map.  We  can  use  scrolling  facilities  when  there  are  many  slides,  and  a 
hierarchy  of  maps  when  the  subject  matter  can  be  more  naturally  presented  at  various  levels  of  abstraction, 
thereby  avoid  cluttering  one  map  with  too  many  details.  As  the  presentation  progresses,  the  nodes  on  the 
concept  map  are  updated  (by  changing  their  color,  for  instance)  to  show  that  they  have  already  been  presented. 
The  concept  map  also  contains  hyperlinks  to  connect  to  individual  slides. 

The  individual  slides  also  have  hyperlinks  to  the  concept  map  and  other  directly  related  slides. 
Furthermore,  a smaller  version  of  the  concept  map  (without  any  text)  is  displayed  on  individual  slides.  This 
scaled-down  concept  map  is  made  up-to-date  i.e.  it  reflects  where  we  have  reached  in  the  presentation  and  how 
it  is  related  to  the  slide  currently  been  displayed.  The  presenter  has  the  flexibility  of  moving  directly  from  one 
slide  to  the  next  or  via  the  concept  map.  It  should  be  pointed  out  that  the  design  of  the  system  should  be  simple 
and  intuitive,  such  that  its  cognitive  load  does  not  outweigh  the  gains  in  enhancing  understanding.  We  now 
describe  in  detail  the  precise  design  choices  we  made  for  a prototype  of  the  system  that  was  developed. 


Design  of  the  Concept  Map 

Fig.  1 is  a concept  map  of  a presentation  on  this  paper  [1].  The  concept  map  is  based  on  a combination  of 
the  document  map  and  topic  map  approaches.  That  is,  each  node  in  the  concept  map  can  be  a slide  or  a topic. 
For  instance,  “Related  Work”  is  a topic,  whereas  “The  Organizer  Project”  is  a slide.  Nonetheless,  these  two 
types  of  nodes  are  not  graphically  differentiated,  so  as  to  avoid  confusing  the  audience  with  too  much 
information.  The  title  of  the  presentation  is  also  displayed  at  the  top,  which  is  a useful  reminder  of  what  the  talk 
is  about.  The  links  are  represented  by  lines  of  various  colors  and  thicknesses  to  show  various  types  of  relations. 
In  Fig.  1,  only  two  types  of  links  are  shown:  (1)  black,  uni-directional  lines  for  parent-child  relationship  in  a tree 
structure,  and  (2)  a brown,  bi-directional  line  to  connect  two  slides  at  different  parts  of  the  tree.  Other  types  of 
links  can  also  be  used,  but  they  should  be  clear  and  intuitive  or  based  on  standard  diagrammatical  notations. 
Otherwise  they  should  be  explained  to  the  audience. 


[1]  Although  colors  are  not  shown  in  Figures  1-4,  the  use  of  colors  is  essential  in  this  system. 
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Concept  regions  group  a few  logically  related  slides/topics  together  and  show  them  in  a blue  area  (Fig.  1).  A 
concept  region  is  shown  on  single-clicking  anywhere  in  the  region  (except  on  the  boxes  for  nodes).  Only  one 
concept  region  is  shown  at  a time.  If  there  are  too  many  slides  to  fit  one  screen,  then  we  can  have  a multi- 
level concept  map,  where  a topic  node  at  a higher  level  map  opens  up  another  map.  This  enables  display  of 
information  at  different  levels  of  abstraction.  Another  option  is  making  the  concept  map  large,  which  can  be 

viewed  through  a scrolling  facility.  At  the  beginning  of  a presentation,  all  nodes  are  shown  in  green  on  the 
concept  map,  to  indicate  that  the  corresponding  slides  have  not  been  completed  yet.  The  presenter  can  open  an 
individual  slide  by  single-clicking  on  its  box.  If  a topic  box  is  clicked,  then  it  connects  to  its  leftmost,  nearest 
descendant  which  is  a slide.  If  no  such  slide  exists,  then  nothing  happens  on  the  mouse  click.  When  a slide  is 
opened,  then  it  is  said  to  be  active.  On  returning  to  the  concept  map  after  showing  an  active  slide,  the  latter 
becomes  completed.  Once  a node  is  active,  it  remains  active  as  long  as  at  least  one  of  its  descendants  is  active 
or  non -completed.  If  all  its  descendants  are  completed,  then  the  node  becomes  completed.  Active  slides  are 
indicated  in  a yellowish  color  and  completed  ones  in  deep  pink  (Fig.  2). 

If  the  text  in  a node  box  is  not  adequate  to  convey  what  it  contains,  then  we  have  a longer  narration  on  a 
mouse-over  action  on  the  box  (Fig.  3).  Clicking  on  the  narration  will  remove  the  narration.  The  relation  between 
particular  pairs  of  nodes  can  be  emphasized  by  a mouse-over  action  on  the  corresponding  links,  whereby  the 
related  nodes  are  highlighted  (Fig.  3).  In  fact,  a link  can  be  further  emphasized  by  opening  a slide  to  illustrate 
the  relation.  We  can  also  connect  a node  directly  to  a graphic,  audio  or  video  component.  In  our  example 
presentation,  we  include  an  audio  component  and  indicate  it  by  an  appropriate  icon  on  the  concept  map. 


Design  of  Individual  Slides 

Individual  slides  are  made  of  four  frames  (Fig.  4).  One  central  frame  contains  the  material  being 
presented.  The  three  frames  on  the  right  respectively  contain:  (1)  a smaller  version  of  an  up-to-date  concept  map 
to  reflect  the  current  status  of  the  presentation;  this  map  does  not  have  any  text,  but  shows  the  concept  region 
which  contains  the  current  slide;  (2)  named  hyperlinks  to  all  slides  connected  to  the  current  slide;  as  a 
convention,  we  opted  for  the  topmost  hyperlink  to  be  the  parent  node  (if  any);  (3)  a hyperlink  to  the  concept 
map  (or  maps  at  various  hierarchical  levels).  The  presenter  can  always  choose  to  show  only  the  information 
being  presented  (i.e.  omit  the  last  three  frames)  in  situations  where  more  space  is  required  on  the  screen,  or 
additional  information  like  hyperlinks  are  redundant. 

The  slides  have  a default  ordering.  The  presenter  can  show  the  next  slide  through  one  of  the  typical 
PowerPoint-like  actions:  single-click,  PageDown  or  Return.  Besides  these  options,  the  presenter  can  choose  to 
go  to  any  slide  by  doing  one  of  the  following  : (1)  click  on  a named  hyperlink,  (2)  click  on  the  appropriate  box 
on  the  smaller  version  of  the  concept  map,  or  (3)  go  via  the  concept  map.  The  presenter  should  choose  the 
option  which  will  make  the  presentation  clearer  and  the  flow  smooth  and  intuitive. 


Implementation 

The  prototype  was  implemented  in  PowerPoint,  using  macros  and  modifying  the  Visual  Basic  code 
generated.  In  the  current  prototype,  many  of  the  functionalities  have  been  hard-coded  to  suit  a particular 
presentation.  We  can  envision  a more  generic  tool,  where  most  of  the  features  are  automated  and  the  user  has 
the  flexibility  to  customize  the  presentation  by,  for  instance,  choosing  color  schemes,  disabling  some  features, 
choosing  templates  for  organization  of  nodes  on  the  concept  map,  automating  the  creation  and  maintenance  of 
maps  and  slides,  etc. 


Extensions 

There  are  many  ways  in  which  this  prototype  can  be  extended,  and  individual  features  can  be  variously 
implemented.  Some  examples  of  new  features  and  variations  of  existing  ones  have  already  been  discussed.  A 
few  more  suggestions  follow: 
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• Place  anchors  within  the  presented  text  in  individual  slides; 

• Flexibility  of  organizing  the  contents  (and  sizes)  of  frames  on  individual  slides; 

• Explicitly  distinguish  among  nodes,  which  can  be  topics,  slides  or  other  entities  like  audio,  graphic 
or  video  objects;  suitable  graphic  and  auditory  icons  can  also  be  used; 

• Differentiating  between  the  user  actions  (like  mouse  clicks)  that  the  presenter  wants  the  audience  to 
see  and  those  which  should  be  hidden,  and  using  short-cut  keys  for  the  latter; 

• Different  ways  of  showing  the  current  status  of  the  presentation.  For  instance,  a node  can  be  shown 
as  completed  as  soon  as  any  of  its  descendants  is  completed; 

A good  design  of  the  system  would  have  to  be  based  on  factors  like  feedback  from  target  users,  the 
particularities  of  the  teaching  environment  (type  of  learning  institution,  subject  matter,  etc.),  familiarity  of  the 
audience  with  the  features,  etc.  One  approach  is  to  provide  a wide  range  of  features,  options  and  templates,  and 
let  the  presenter  customize  the  system. 


Evaluation 


Our  prototype  was  used  for  actual  formal  presentations  in  graduate  classes  in  the  Department  of 
Computer  Science  at  Texas  A&M  University.  The  system  was  evaluated  through  a questionnaire,  which 
essentially  queried  students  about  the  effectiveness  of  individual  features  added  to  PowerPoint  to  help 
understanding  the  presented  material.  More  evabation  was  performed  informally  with  students  and  faculty  of 
the  department.  The  response  obtained  was  largely  positive.  Over  85%  of  the  people  queried  felt  that  the  use  of 
concept  maps  has  a potential  to  facilitate  learning  during  a presentation.  Around  D%  of  the  students  who 
attended  the  formal  presentations  found  our  prototype  useful.  10%  felt  that  the  system  made  it  more  confusing. 
The  remaining  20%  saw  no  significant  benefit  of  our  extensions  to  PowerPoint.  We  obtained  many 
suggestions  during  the  evaluation  phase,  some  of  which  were  subsequently  incorporated  in  our  design.  But  there 
were  some  criticisms  as  well;  the  most  common  ones  are  summarized  below: 

• Invoking  the  concept  map  can  be  obtrusive  to  the  general  flow  of  the  presentation; 

• The  audience  can  be  confused  by  the  extra  information  provided;  familiarity  with  the  features  is 
necessary; 

• The  smaller  version  of  the  concept  map  and  the  named  hyperlinks  take  up  too  much  space  on 
individual  slides; 

• The  presenter  should  fully  master  the  tool;  it  is  very  easy  to  perform  an  incorrect  action  (such  as 
clicking  at  the  wrong  spot),  which  can  send  the  presentation  in  an  unintended  direction; 

• The  presenter  may  be  confused  when  faced  with  too  many  optional  actions.  (This  problem  can  be 
partly  solved  by  a subtle  indication  of  one  preferred  action  at  each  stage  of  the  presentation); 

• The  current  status  of  the  presentation  (indicated  by  non -completed,  completed  and  active  nodes, 
using  different  colors)  is  not  always  obvious; 

It  should  be  noted  that  the  proposed  system  focuses  on  the  audience  as  the  “user*’.  As  far  as  the  presenter 
is  concerned,  it  is  clear  that  he/she  has  to  spend  more  effort  in  the  preparation  of  a presentation,  in  mastering  the 
various  features  and  in  organizing  the  presentation  to  ensure  a smooth  flow  and  logical  coherence. 

The  benefits  of  the  system,  as  reported  to  us  during  the  evaluation,  are  summarized  in  the  following: 

• The  ability  to  constantly  perceive  the  “big  picture”  rather  than  merely  focus  on  individual  slides; 

• More  awareness  of  what  the  presenter  is  really  thinking  about; 

• Induces  the  presenter  to  organize  his  material  more  logically; 

• Questions  can  be  answered  more  effectively  by  conveniently  accessing  relevant  slides; 
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• Decisions  about  which  slides  to  show  and  which  ones  to  pass  over  during  a presentation  can  be  made 
on-the-fly  in  a convenient  and  logical  way  - for  instance,  parts  of  the  tree  or  hierarchy  of  nodes  can  be  left  out 
from  the  presentation,  without  the  inconvenience  of  opening  slides,  examining  them  and  finally  deciding  to  skip 
them; 

• The  possibility  of  catching  up  with  the  talk  even  if  you  come  in  late  or  doze  off  for  a while! 


Conclusion  and  Future  Work 


We  hypothesize  that  the  slide  presentation  tool  described  in  this  paper  has  considerable  potential  to 
facilitate  learning  in  the  classroom,  especially  since  slide  presentation  systems  are  so  widely  used.  The  key 
advantages  of  the  system  we  propose  are:  (1)  The  ability  to  graphically  perceive  the  inter-relationships  among 
various  fragments  of  information,  and  (2)  the  possibility  of  smoothly  and  logically  navigating  through  the 
network  of  ideas,  topics  and  documents. 

Myriad  variations  and  extensions  of  the  design  we  have  presented  can  be  thought  of  The  tool  will  need 
to  be  customized  to  suit  the  target  audience:  classrooms  with  different  types  of  students,  and  other  audiences 
like  seminars,  conferences,  marketing,  etc.  In  fact,  we  advocate  a high  degree  of  automation  and  customization, 
so  as  to  reduce  the  effort  required  to  prepare  a presentation,  and  to  efficiently  and  effectively  tailor  the  tool  for 
different  situations.  As  future  work,  we  envisage  the  development  of  a fully  generic,  customizable  prototype, 
and  its  evaluation  by  objectively  assessing  how  different  features  (or  sets  of  features)  impact  understanding  (as 
well  as  ease  of  presentation)  in  various  environments. 
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Figure  1:  A concept  map  with  a concept  region  grouping  4 nodes;  uni-directional  and  bi-directional  links  are  also 
shown.  “System  Overview*’  node  is  active;  all  other  nodes  are  non- completed. 
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Figure  2:  Different  colors  are  used  to  display  completed  nodes  (like  “Related  Work”),  active  ones  (like 
“Limitations”)  and  non-completed  ones  (like  “Conclusion”). 
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Figure  3:  This  concept  map  shows  a narration  and  highlights  two  linked  nodes. 


Summary  of  benefits 

• “Where  are  we?” 

• Reduce  “cognitive  load” 

• Ease  understanding  and 
presentation 

• Navigation,  global  thought  and 
abstraction 

• Encourages  logical  organization 

• Answers  to  queries 

• Managing  many  slides 


[Proposed  System] 
[Limitations  of  .771 


[ Map  1 


Figure  4:  A typical  individual  slide  with  four  frames. 
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Abstract:  This  paper  describes  an  innovative  study  that  collected  data  through  personal  interviews 
from  one-hundred  members  of  a faculty  of  education  to  investigate  the  barriers  and  enablers  to 
using  technology  in  communication,  teaching  and  research.  The  findings  and  recommendations  are 
of  value  to  faculty  who  are  interested  in  learning  and  using  technology  and  to  those  responsible  for 
implementing  technology  and  its  infrastructure . 


Introduction 

Through  research  and  development,  a plethora  of  technologies  and  applications  are 
available  for  faculty  to  use  in  their  communications,  in  their  teaching  and  in  their  research. 
Researchers,  such  as  Chizmar  and  Williams  (2001),  support  the  need  for  studies  to  investigate 
technologies,  services  and  products  that  faculty  members  want  and  can  successfully  use.  Chizmar 
and  Williams  compared  their  assertions  with  the  results  of  a faculty  survey  to  determine  faculty 
needs.  With  a range  of  interests  and  abilities  from  early  adopters  to  wary  adopters,  this  study  was 
implemented  to  investigate  the  enablers  to  faculty  members’  use  of  technology. 

The  Study 

The  research  study  was  conducted  to  investigate  the  faculty  member’s  use  of  technology. 
A convenience  sample  was  used.  Each  participant  was  personally  interviewed  by  one  of  three 
experienced  graduate  student  interviewers.  Of  the  120  invited  for  interviews,  100  participated. 

The  interview  followed  a guided  question  approach  with  an  open-ended  question  to  capture  any 
input  from  the  participants  that  the  participants  deemed  necessary  to  include.  The  data  was  then 
carefully  read  and  sorted  and  entered  into  an  electronic  database  for  sorting,  resorting,  queries  and 
classifications.  The  findings  from  the  study  were  condensed  and  presented  to  the  faculty  at  an 
open  invitation  meeting. 

Findings 

Several  themes  emerged  from  the  participants*  comments  in  the  study  including  barriers 
and  enablers  to  implementing  technology  in  their  communication,  in  their  teaching  and  in  their 
research.  All  of  the  participants  in  the  study  indicate  that  they  use  technology  to  communicate,  and 
most  participants  are  interested  in  learning  more  about  technology  and  how  it  can  be  incorporated 
into  their  communication,  teaching  and  research.  Two  barriers  to  incorporating  technology  that 
emerge  from  the  study  are  a lack  of  time  and  an  increasing  lack  of  contact  with  colleagues. 

Participants  comment  that  their  most  complex  barrier  is  their  lack  of  time  and  that 
evolving  technologies  are  demanding  of  their  time  to  continuously  learn.  Participants  suggest  it  is 
not  just  a barrier  to  wary  adapters,  but  it  is  also  a barrier  to  those  who  are  adept  at  using 
technologies.  Participants  comment  that  learning  new  and  updated  technologies  is  time 
consuming,  and  that  self-teaching  or  troubleshooting  is  often  the  least  effective  use  of  their  time. 
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Recognizing  that  technology  drives  learning,  and  their  learning  will  never  end,  participants  look 
for  a supportive  technology  infrastructure,  templates  and  technical  services,  and  access  to  ongoing 
help  to  enable  them  to  use  their  time  more  efficiently  to  learn  and  use  technologies. 

Participants  comment  that  technology  allows  them  to  work  from  home  or  elsewhere;  and 
to  communicate  with  students,  colleagues  and  administration  locally,  or  with  colleagues  with 
similar  interests  globally,  all  without  leaving  their  desks.  Participants  also  comment  that  as  they 
gain  confidence  in  using  existing  technologies,  they  become  less  dq)endent  on  their  colleagues  for 
help.  Although  being  able  to  work  off  campus,  and  communicate  without  leaving  their 
workstations  increases  their  productivity,  it  limits  opportunities  to  personally  communicate  with 
faculty  of  education  colleagues.  Participants  recognize  that  limited  access  to  colleagues  within  the 
faculty  of  education  can  be  a barrier  to  discussions  on  the  impact  of  technology  on  education  and 
learning,  and  to  the  sharing  of  new  technologies  and  best  application  practices.  To  overcome  this 
barrier,  participants  suggest  faculty  of  education  colleagues  need  to  meet  to  enable  discussions. 
Participants  look  for  opportunities  for  faculty  of  education  members  to  meet  to  discuss  technology 
issues  such  as  ethics,  intellectual  property,  copyright,  permanence  of  electronic  data,  and  impact 
on  education  and  learning  as  a broad  issue  and  from  a perspective  from  their  disciplines. 
Participants  also  look  for  opportunities  to  learn  about  new  technologies  and  best  practices  from 
early  adapters  of  technology  through  casual  conversations  or  from  demonstrations  in  order  to 
determine  the  relevance  of  the  technology  to  their  work.  Participants  ask  early  adapters  not  to 
make  any  assumptions — colleagues  are  relying  on  early  adapters  to  communicate  with  them,  to 
show  new  and  emerging  technologies  and  to  make  suggestions  how  technology  could  apply. 

As  technology  drives  learning,  learning  never  ends.  To  this  end,  participants  suggest 
enablers  such  as  a strong  infrastructure  that  supports  access  and  technical  support,  and  an 
interconnected  support  system  to  provide  help  at  the  right  time  for  the  right  needs.  Participants 
also  suggest  open  communication  with  colleagues  to  discuss  issues  relevant  to  technology  in 
education,  and  best  practices.  Specific  findings  and  recommendations  from  this  study  are  of  value 
to  the  users  of  the  applications  and  to  those  that  are  responsible  for  implementing  the  technologies 
and  infrastructure. 
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Abstract:  In  this  paper  we  present  the  main  results  of  a study  concerning  students’  e-leaming 
needs  at  a Belgian  universtiy  (VUB).  Our  analysis  focuses  on  the  type  of  academic  education 
they  prefer,  which  e-leaming  facilities  they  expect  and  how  they  perceive  the  consequences  of 
internet  based  education.  The  data  reveal  that  students  want  a blended  educational  system, 
with  considerable  eleaming  aspects  but  no  genuine  distance  education.  They  show  a 
multitude  of  negative  and  positive  attitudes  toward  the  impact  of  e-leaming. 


Introduction,  objectives  and  methodology 

In  October  2000,  the  staff  of  the  Vrije  Universiteit  Brussel,  a mid-sized  Dutch  speaking  university  in  Belgium, 
accepted  the  new  concept  of ‘competence  based  learning  in  flexible  education’.  By  2005  university  policymakers 
expect  a aeries  of  powerful  learning  and  teaching  environments,  where  information  and  communication 
technologies  (ICT)  play  a significant  and  role.  Initiatives  have  been  planned  which  will  be  characterized  by 
(Derks,  2000):  (1)  an  extension  of  the  ICT  infrastructure  (e.g.  wireless  campus)  and  system  administration,  (2) 
more  and  better  ICT  support  for  educators,  (3)  an  integral  ICT  approach  and  (4)  bottom-up  and  top-down 
methods.  In  order  to  fulfil  the  various  educational  functions,  i.e.  ‘facilities  for  stimulating  specific  student 
learning  activities’,  e-leaming  modes  are  to  be  implemented.  According  to  Webb  (1997)  an  Internet  based 
learning  environment  consists  of  different  major  players:  the  teacher,  the  learner,  the  technical  support  officer 
and  the  central  administration.  To  be  successful  when  implementing  any  educational  innovation  one  has  to  keep 
in  mind  the  role  of  the  student,  otherwise  the  university  risks  to  make  heavy  but  useless  capital  investment. 
Consequently,  it  is  cmcial  to  conduct  exploratory  research,  because  there  is  a strong  need  ‘to  consider  the 
characteristics  of  our  target  audience  when  designing  and  exploiting  educational  provision’  (Valcke,  2000). 

In  this  study  we  attempt  to  explore  the  future  prospects  of  e-leaming  at  our  university.  We  investigated  three 
research  domains:  (1)  which  kind  of  academic  education  do  students  prefer  - traditional,  blended  or  pure  online?, 
(2)  which  aspects  of  e-leaming  are  most  wantedl  and  (3)  how  do  students  perceive  the  outcomes  of  e-leaming? 
Data  were  collected  through  an  anonymous  questionnaire  administered  to  470  undergraduate  and  graduate 
students  from  different  education  levels  and  disciplines  (representative  multiple  stratified  sample,  N=  9451). 


University  Type 

The  majority  of  learners  (88.6%)  of  learners  have  positive  associations  with  a combination  of  a traditional  and  a 
virtual  study  environments.  Only  3 (exclusively  male)  students  think  pure  e-leaming  is  convenient,  while  11% 
want  to  stick  to  the  traditional  approach. 


Results 


ERIC 


E-learning  facilities 

Despite  these  rather  conservative  attitudes  toward  e-leaming,  students  do  want  a lot  of  practical  and  pedagogic 
facilities  to  be  online.  By  means  of  the  data  modes  we  defined  what  students  most  prefer  (table  1).  Practical 
study  organisation,  access  to  iiformation  and  communication  are  wanted  online  but  not  elements  like  distance 
courses,  chat  sessions  or  interactive  classes.  Surprisingly,  students  do  not  favour  a compulsory  computer  and 
internet  course. 


Coursematerial  on  Internet 

Mode 

Practical  information  via  Internet 

Mode 

Course  outlines/self  study  guides 

yes 

Electronic  information  board 

yes 

Exercises 

yes 

Exam  timetables 

yes 

Sample  assignments 

yes 

Exam  regulation 

yes 

Sample  exams 

yes 

Exam  results 

yes 

Self-assessment  tests 

yes 

Course  timetable 

yes 

Computermanagcd  exams 

yes 

Complete  list  of  staff  (tel.,  e-mail,  etc.) 

yes 

1 web  page  per  course 

yes 

University  web  site  for  enrollment 

yes 

Online  purchase  of  course-notes  and  books 

yes 

University  web  site  for  entering  choice  of  optional  courses 

yes 

Infrastructure 

Mode 

Communication  via  Internet 

Mode 

More  computer  rooms 

yes 

Discussion  forum  about  courses  and/or  exercises 

maybe 

Laptop  purchase  via  the  university  (at  reasonable  price) 

yes 

Tutoring  timetable  via  e-mail 

yes 

Wireless  internet  access  on  campus 

yes 

Chat  sessions  with  educators/teaching  assistants 

maybe 

Software  service  on  campus 

yes 

Chat  sessions  with  fellow  students 

maybe 

Helpdesk  on  campus 

yes 

Distance  education 

no 

Computer  renting 

yes 

Interactive  classes  via  computers  to  replace  lectures 

maybe 

Financial  help  with  purchase  of  computer  material 

Cheap  internet  access 

Compulsory  computer  and  internet  course 

yes 

yes 

maybe 

Assignments  via  e -mail 

yes 

Table  1:  Students’  preferences  for  specific  e-leaming  facilities 

Perceived  outcomes  of  e-learning 

Using  a 5-point  Likert -type  scale  ranging  from  strongly  disagree  to  strongly  agree,  respondents  were  asked  to 
provide  their  opinions  on  a variety  of  issues  related  to  the  impact  of  e-leaming.  The  majority  of  students  believe 
that  e-leaming  will  bring  some  educational  benefits  like  more  attractive  education  and  better  preparation  for  the 
job  market.  However,  a significant  proportion  of  students  are  undecided  about  the  potential  of  o-leaming  for 
increasing  the  quality  of  the  learning  experiences.  Almost  six  out  of  ten  (57%)  do  not  know  if  more  enrollments 
can  be  expected,  whereas  the  remaining  43%  are  almost  equally  distributed  over  the  disagree  and  agree 
categories.  The  benefit  of  lowered  inhibitions  (where  education  is  more  accessible)  is  recognised  by  46%  of  the 
respondents.  More  than  50%  of  the  sample  fear  encroachment  on  social  contacts  and  life  on  campus.  On  the 
other  hand  students  do  not  think  it  will  threaten  their  privacy.  Students  anticipate  difficulties  with  self  discipline 
and  pace  of  study.  They  express  more  or  less  neutral  feelings  toward  control  and  security  issues.  Opinion  is 
divided  over  whether  e-leaming  is  more  cost  effective  than  other  forms  of  learning. 


Conclusions 

Students  at  the  Vrije  Universiteit  Brussel  reject  o-leaming  as  a substitute  for  other  forms  of  training.  They 
strongly  opt  for  a blended  model  of  online  and  face-to-face  education.  E-leaming  is  thought  to  be  convenient, 
allowing  to  use  online  course  material,  organize  and  communicate  easily  about  their  studies  in  a flexible  way. 
Our  findings  contradict  the  opinion  of  the  educators  adopting  a booster  opinion  towards  ICT  in  education. 
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Abstract:  The  Discetech  project,  which  has  started  in  1996  in  Northern  Italy,  aims  at 
experimenting  the  introduction  of  new  technologies  within  the  teaching-learning 
activities  of  high  school  classrooms.  Since  1998,  a similar  project,  Bimbotech,  has 
bccii  developed  for  iiitial  classrooms  (from  4 to  10  years  old  children).  Several 
hundreds  experiences  involving  several  thousands  pupils  have  been  carried  on  by 
the  teachers  who  have  been  previously  instructed  and  assisted  by  the  project’s  staff. 
This  paper  briefly  introduces  a case  study,  one  of  the  most  interesting  Bimbotech 
experiments  analyzing  the  organization,  the  phases,  the  results  and  the  learned 
lessons.  The  final  section  wants  to  suggest  a few  useful  remarks  upon  mutual 
connection  between  multimedia  and  knowledge. 


1 . An  overview  of  the  pro j ect 

DISCETECH  and  BIMBOTECH  are  twin- projects,  which  try  to  experiment  new  technologies  to 
teach  and  learn  in  the  Italian  school  system.  Discetech  started  in  the  city  of  Como*  in  1996,  while 
Bimbotech  in  1998,  under  the  initiative  of  Politecnico  di  Milano^;  in  1997  it  also  branched  out  in 
Lecce^,  for  the  initiative  of  the  Faculty  of  Engineering  of  the  local  University.  Finally  last  year  a 
new  branch  was  opened  in  Milan. 

The  Discetech  project  as  target  has  high  school  teachers  and  students  while  the  Bimbotech  one  aims 
at  children  from  4 to  10  years  of  age  (this  is  equivalent  in  Italy  to  “scuola  matema”  and  “scuola 
elementare”). 

First  of  all,  we  attempt  to  support  teachers  to  achieve  the  basic  ICT  knowledge  necessary  to  their 
teaching  activities  using  computers.  In  a second  time,  on  the  other  hand,  teachers  are  asked  to  plan 
and  realize  a didactical  experience  in  the  classroom  in  order  to: 

• Try  to  employ  multimedia  technologies  within  the  class  curriculum  to  find  out  their  useful 
potential; 


* Como  is  a town  near  Milan,  where  another  headquarters  of  Politecnico  has  been  placed. 

^ The  largest  technical  University  in  Italy,  with  Faculties  of  Engineering  and  Architecture. 
^ Lecce  is  a town  in  the  south  of  Italy. 
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• Evaluate  the  efficiency  of  the  above  goals  regarding: 

• Content  acquisition 

• Knowledge  building  process  (e.g.  focusing  on  and  selecting  information); 

• Behavior  skills  acquisition  (e.g.  co-operative  skill  and  teacher- student 
communication). 

Our  visbn  aims  at  improving  the  teacher’s  background  and  skills  with  the  purpose  to  experiment 
new  didactical  solutions  with  students  where  new  technologies  must  be  involved  and  projects 
carried  on  collecting  data  all  the  school- year  long.  These  attitude  answers  the  purpose  to  acquire 
expertise  mainly  in  experimenting  with  computer  and  interacting  with  multimedia  instruments 
rather  than  discussing  what  should  be  done  and  so  stopping  just  on  theory. 

Besides,  there’s  no  intention  to  judge  the  relationship  between  teachers  and  pupils  or  to  give  advice 
about  the  pedagogical  approach  that  teachers  choose  to  make  use  of:  they  have  total  autonomy  on 
selecting  the  one  they  prefer  and  wish. 

The  project  would  require  a three  years  commitment: 

• the  first  year  (basic  level)  looks  after  the  off-line  and  on-line  multimedia  employment; 

• the  second  one  (advanced  level)  focuses  on  planning  and  carrying  out  a simple  hypermedia 
product  (off-line,  e.g.  a power -point  presentation,  as  well  as  on-line  e.g.  a small  HTML 
hypertext); 

• the  third  and  last  one  Experimental  level)  encourages  teachers  to  realize  (with  Discetech 
staff  support)  a complex  and  complete  multimedia  project. 

The  first  two  years  are  organized  in  three  phases: 

• formation  (from  October  to  January):  teachers  acquire  the  ICT  knowledge  necessary  to  the 
experience  that  Discetech  suggest  them  to  realize  in  the  classroom; 

• modeling  (February):  teachers  are  guided  by  the  staff  to  think  about  a proper  personal 
project; 

• working  within  the  class  (from  March  to  April):  each  plan  is  developed  during  the  school 
time. 

Overall  600  teachers  joined  the  Discetech  program  (some  of  them  for  more  than  1 year)  and  more 
than  300  educational  projects  have  been  carried  out,  involving  nearly  9000  students  up  till  now. 

In  the  next  section  we  will  present  one  of  the  best  experiences,  according  to  the  originality  and  the 
quality  of  results,  made  during  last  year  (2000/2001)  in  Como. 

2.  A Case  Study:  T\itPinocchio^%  tale 

This  experience  has  been  carried  on  by  one  of  the  most  active  group  of  teacheis  taking  part  to  the 
Bimbotech  program."^  The  classes  involved  were  two,  both  consisting  of  thirty  7-8  years  old 
children. 

i 7 years  old  children  , 30  ^ 

8 years  old  children  30 


* The  school  is  Scuola  Elementare  di  via  Giussani  - circolo  Como  4,  via  Giussani  - Como  The  teachers  involved  are  Raffaella  Ciceri  and  Francesca 
Rossi. 
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The  overall  aim  of  the  experience  can  be  summarized  as  it  follows: 

a)  Get  children  used  to  the  PC  in  the  day  by  day  class  activities,  promoting  self- interaction 
with  computer. 

b)  In  particular,  knowing  and  using  the  Microsoft  program  called  Power  Point  as  a tool  to 
realize  a slide  each  of  the  children  as  a part  of  a global  classroom  work. 

c)  Employing  the  new  technologies  to  unify  all  the  different  experiences  of  the  knowledge 
building  process  including  both  the  traditional  teaching  method  (like  frontal  lesson)  and  new 
attempts  concerning  theatrical  activities,  drawing,  creative  writing. 

The  project  took  up  several  subsequent  months  trying  to  make  the  Pinocchio  ’s  tale  the  topic  of  the 
entire  school  year. 

The  teachers  started  with  the  reading  of  the  tale,  followed  by  the  comprehension  of  the  text.  In  a 
second  time  they  made  the  children  split  it  up  in  several  sequences,  one  for  each  of  them  so  that 
every  child  has  been  able  to  make  a little  abstract  and  a drawing  in  color,  interpreting  the  characters 
in  theatrical  laboratories  and  thinking  over  the  whole  tale  plot,  interacting  successively  each  other 
in  the  classroom.  After  that,  the  last  step  of  the  experience  concerned  the  employment  of 
computers:  children  are  put  in  the  position  to  scanner  their  drawing  and  to  realize  a Power  Point 
slide.  In  this  phase  teachers  just  guided  and  supported  them  on  demand  to  do  themselves  every  step, 
letting  them  using  entirely  the  PC. 

The  Power  Point  presentation  contains: 

• The  complete  tale  developed  slide  by  slide  in  order  of  time,  each  one  made  by  a singular 
pupil,  which  includes  the  insert  of  a colored  background,  some  buttons,  a text,  a scanned 
drawing  and  an  animation  (see  the  picture  below  as  example  - figure  1-). 


promises  he  will 


Pinocchio  after 
been  saved  by 
the  fairy, 


never  say  lies 
again. 


Figure  1 : The  screenshot  of  an  explicative  slide. 
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• The  cast  of  Pinocchio's  tale  developed  in  some  slides,  one  for  each  character,  realized  by 
children  who  thought  about  the  most  relevant  qualities  that  marked  them  out,  as  you  can  see  in 
the  next  picture,  illustrating  the  cat  and  the  fox,  two  bad  characters  who  want  to  lead  Pinocchio 
astray.  According  to  the  hypertext  structure  every  slide  has  a special  button  that  allows  to  turn 
back  to  the  home  page. 


The  cat  is  blind 

while  the  fox  is 

limping.  They 

are  two  scamps 

and  that’s  why 

they  try  to 

cheat 

Pinocchio:  they 

want  him  to 

plant  his  money 

in  the  miracles 

field  because 

the  morning 
after  he  could 

Wi 

find  a gold  tree. 

gzitto  e deco,  la 
[ . volpc  e zoppa,  Sonu 
due  lurfanti.  per 
questo  ccrcono  di 
imbrodiare  Pinocchio 
diccnduuli  chc  sc 
pianta  Ic  sue  5 inuncte 
’ . ns\  campo  dei 
. >:iniracoiL.  la  mamna 
dope  trovera  una 
pianla  d'oro. 


Figure  2:  The  cat  and  the  fox. 

• The  theatrical  laboratories:  according  to  the  school  guidelines,  which  in  Italy  are  fixed  by 
P.O.F.^  since  1999  with  the  beginning  of  the  self-governing  school,  the  whole  work  program 
during  the  school  year  has  been  based  on  the  novel  of  Pinocchio.  That’s  why  teachers  chose  to 
stage  it  involving  several  classes  and  let  the  children  socialize  from  a classroom  to  another.  The 
hypermedia  is  a precious  support  to  relate  this  experience  as  you  can  see  in  the  following 
picture. 


We  imagined  to 
live  in  a 
beautiful  and 
magic  town. 

We  liked 
everything 
about  this 
experience 
especially  the 
ballet. 


Figure  3:  The  description  of  the  theatrical  lab  with  photo. 
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^ P.O.F.  is  the  Plan  of  the  Formative  Offer,  which  defines  the  cultural  and  planning  school  identity  and  helps  to  explain 
and  share  the  pedagogical  and  didactical  directions.  This  plan  dlows  to  fix  a specific  goal  trying  out  a new  strategic 
lineup  to  get  to  it. 
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Different  conclusions:  as  a creative  writing  exercise,  teachers  asked  children  to  invent  some 
gripping  imaginary  ends  and  put  them  in  a slide  in  accordance  with  each  one  wish  (see  the 
picture  showing  the  opening  slide  to  the  different  courses). 


Figure  4:  The  screenshot  where  youVe  supposed  to  explore  alternative  conclusions. 


3.  Conclusion 

The  importance  of  this  experience  consists  on  the  attempt  to  combine  different  activities,  which 
culminate  in  a multimedia  product:  its  creation  allowed  a new  manner  for  children  to  relate  each 
other  mutually.  Furthermore  it  helps  to  renew  learning  and  teaching  contents:  indeed,  Pinocchio ’s 
tale  has  been  approached  in  a different  way  and  this  permits  to  create  a new  manner  of  knowing  by 
the  development  of  a radically  fresh  work.  The  fact  that  pupils  have  found  out  several  different 
conclusions  to  the  story  (see  previous  picture  - figure  4 -)  is  exemplary  for  us  to  explain  the 
innumerable  creative  powers  of  a project  like  this  one. 

Discetech  highlights  every  intelligence  style:  in  this  particular  perspective  technologies  and 
especially  multimedia  are  very  useful  to  improve  all  learning  and  teaching  process  structures, 
exploiting  a richer  interaction  across  the  activities.  They  keep  concentration  up  as  well,  feeding  a 
reticular  knowbdge  based  on  the  association’s  reasoning  and  powering  every  human  talent  up 
contributing  to  make  the  classroom’s  relationship  stronger  and  more  complete  than  usual  since 
computers  naturally  encourage  cooperation,  as  Maragliano  and  Moretti  say^. 

We  think  that  an  experience  like  the  one  we  reported  could  be  easily  and  successfully  exported  in  a 
lot  of  other  primary  schools.  That’s  why  we  are  strongly  interested  to  spread  out  this  project 
offering  the  possibility  to  share  this  work  and  the  useful  information  about  it  and  to  approach  other 
works  too  by  Internet. 


^ R.  Maragliano,  G.  Moretti,  Valutare  nel  multimedia^  in  R.  Maragliano,  / media  e la  formazioney  Roma,  La  Nuova 
Italia  Scientifica,  1994. 
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This  experience  is  a meaningful  example  about  the  role  that  multimedia  languages  could  hold 
inside  the  school  system.  The  computer-assisted  education  is  important  for  several  reasons:  besides 
the  merely  technical  and  application-oriented  aspects,  new  technologies  are  means  for 
communication’s  increase  and  teaching  method’s  strengthening  widening  knowledge  searching  a 
new  and  stronger  interconnection  between  different  disciplines.  Their  employment  allow  to  get 
over  the  traditional  teaching  statute  promoting  a different  way  of  being  together  and  communicate, 
which  is  possible  thanks  to  their  ability  to  actively  involve  students  in  learning  processes. 
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Abstract:  Web- VOCAL  is  a Vocabulary  Concordance  and  Academic  Lexis  CALL  design 
with  integrated  web  browser  functions.  Web- VOCAL  has  concordance  and  collocation 
features  particularly  relevant  for  study  in  EFL  courses  oriented  toward  the  comprehension  of 
factual  information:  using  web-VOCAL  a student  is  able  to  sort  a text  for  vocabulary  types, 
find  word-frequencies  and  collocations,  and  use  up  to  four  default  corpora.  Word-families  can 
be  displayed  throughout  a text.  Words,  phrases  and  concepts  can  be  searched  in  dictionaries 
and  through  the  Internet  utilizing  site  links  defined  in  the  program.  Internet  web  pages  can  be 
stripped  of  html  tags,  converted  to  text  (.txt)  files  and  collocated  in  various  ways.  The  goal  of 
the  program  is  to  enable  vocabulary  study  to  be  further  integrated  into  English  for  Academic 
Purposes  (EAP)  reading  classes,  encouraging  accelerated  comprehension  of  the  target 
academic  language  by  the  student.  Current  EFL  vocabulary  research  and  pedagogical 
considerations  related  to  the  Web-VOCAL  project  are  summarized. 


Introduction 

The  Web-VOCAL  program  began  development  two  years  ago.  At  that  time,  research  was  oriented  to  program 
design  considerations  (Gilbert  8l  Matsuno,  2001).  We  have  now  developed  a revised,  functioning  prototype,  for 
the  first  time  incorporating  web-resource  functions.  In  this  article  we  will  discuss  the  prototype  CALL  software 
implementation,  and  also  examine  some  of  the  pedagogical  issues  relevant  to  the  project.  Web-VOCAL  is  a 
multimedia  “Vocabulary  Concordance  and  Academic  Lexis  incorporating  Web  resources.”  The  program  has 
been  designed  to  provide  English  for  Academic  Purposes  (EAP)  students  with  options  allowing  them  to  discern 
quickly,  from  computerized  text,  words  in  three  parsed  hierarchies  of  vocabulary;  to  search  quickly  for 
definitions  and/or  translations  of  terms;  to  examine  collocations  from  selected  corpora;  and  to  acquire  new 
vocabulary  though  a tripartite  General  Service  List/Academic  Word  List/Technical  Word  List  approach  to 
vocabulary  acquisition. 

Using  Web-VOCAL,  internet  web  pages  and  site  information  can  be  quickly  found,  automatically  stripped  of 
html/meta-tags  and  converted  to  Web-VOCALrUsable  text  (.txt)  files.  Online  dictionaries,  thesauri, 
encyclopedias  and  translation  dictionaries  capable  of  translating  both  web  pages  and  text  can  be  accessed  from 
within  the  pop-up  Web-VOCAL  browser  window  (with  one  mouse-click).  Web-VOCAL  allows  selected  parts  of 
reading  texts,  collocations,  and  word  definitions  to  be  combined,  organized  and  saved.  Internet  bookmarking 
features  allow  site  bookmarks  to  be  saved/exporte d/imported  on  floppy  or  hard  disk  by  the  student. 

The  overall  purpose  of  Web-VOCAL  is  to  create  a single  comprehensive  CALL  environment  in  which 
necessary  reading-comprehension  study  tools  are  easily  available  to  students  as  they  read  the  study -text  in  the 
“source  reading  text”  (SRT)  frame  of  the  program  (Fig.  2).  Novel  CALL-based  tools  have  been  created  in  order 
to  potentially  accelerate  a student’s  reading  comprehension  by  focusing  on  methods  of  accessing  unfamiliar 
vocabulary,  in  both  contextual  (collocational  and  encyclopedic)  formats  and  non-contextual 
(dictionary/translation  dictionary)  formats. 

The  SRT  module  is  composed  of  three  frames:  the  “SRT”  frame  (largest  frame),  the  “Word  Lisf’  frame  (left 
side),  and  the  “Collocations”  frame  (lower  frame).  “Site”  and  browser  buttons  displayed  on  the  main  window 
activate  Web  browser  functions. 


Accessing  new  vocabulary 


It  should  be  noted  that  vocabulary  knowledge  and  reading  comprehension  are  very  closely  related  (Nation, 
1999).  In  fact,  the  correlation  is  high  and  reliable  enough  that  TOEFL  researchers  at  one  point  considered 
whether  both  vocabulary  items  and  reading  items  needed  to  be  included  inthe  TOEFL  test,  “and  if  not,  which  of 
the  two  could  be  dispensed  with”  (Read,  1997).  Three  vocabulary  lists  are  utilized  in  our  current  concordance 
design.  They  are:  1)  the  2000  highest  frequency  word  families  included  in  West’s  (1953)  General  Service  List  of 
English  Words  (GSL),  further  sub-divided  into  two  subgroups  of  1000  word  families;  2)  Coxhead’s  1998 
Academic  Word  List  (AWL)  (Coxhead,  1998),  a multi-disciplinary  academic  word  list  of  570  word  families — 
note  that  “academic  words”  will  not  include  all  of  the  terms  denoted  by  “semi/sub-technical  vocabulary”  as  used 
in  ESP  nomenclature,  but  are  approximately  equivalent,  as  discussed  in  Nation  (Nation,  1999);  and  3)  technical 
vocabulary,  representing  words  or  terms  that  are  largely  field-specific.  Nation  points  out  that,  “The  division  of 
the  vocabulary  of  academic  texts  into  three  levels  of  (l)general  service  or  basic  vocabulary,  (2)sub-technical 
vocabulary,  and  (3)technical  vocabulary  is  a commonly  made  distinction”  (Nation,  1999). 

We  might  ask,  why  select  particular,  limited  vocabulary  sets  for  study,  while  ignoring  the  bulk  of  the  English 
vocabulary?  The  answer  lies  in  a real-world  situation  having  to  do  with  the  number  of  word  families  in  English, 
the  disparity  of  vocabulary  knowledge  between  typical  undergraduate  LI  (source  language,  mother  tongue)  and 
L2  (target  language)  English  users,  and  the  limited  time  available  to  educate  students  in  order  to  achieve 
academic  goals  in  English,  e.g.,  reading  essays  and  research  papers  in  their  academic  field,  the  overwhelming 
majority  of  which  are  in  English: 

Even  many  journals  of  .smaller  nations’  scientific  societies,  like  those  of  Slovenia  for  example,  publish  also  in  English. 
When  abstracted  more  widely  these  are  then  accessible  to  a world  audience.  For  a scientist  to  publish  in  a language 
other  than  English  therefore  is  increasingly  to  cut  herself  off  from  the  worldwide  community  of  scientists  who  publish 
in  English.  The  work  may  then  be  ignored  simply  because  it  is  published  in  a language  unknown  to  the  rest  of  the 
world  (Wood,  1997). 

Webster’s  Third  New  International  Dictionary  contains  approximately  114,000  word  families,  excepting 
proper  names  (Goulden,  Nation  & Read,  1990);  a number  well  beyond  the  knowledge  of  most  LI  or  L2  students. 
Though  past  measurements  of  vocabulary  size  have  been  found  to  be  erroneous  (Nation,  1999),  recent  reliable 
studies  (Goulden,  Nation  & Read,  1990;  Zechmeister  et  al,  1995),  indicate  that  educated  native  English  speakers 
know  about  20,000  word  families;  the  educated  undergraduate  L2  learner  of  English  may  only  have  10% -15%  of 
this  vocabulary,  a comprehension  of  2,000-3,000  or  so  word  families.  This  places  them  at  a great  disadvantage, 
as  “to  read  with  a minimal  disturbance  from  unknown  vocabulary,  [English]  language  users  probably  need  a 
vocabulary  of  15,000  to  20,000  words”  (Nation,  1999). 

Researchers  have  attempted  to  determine  what  percent  of  the  vocabulary  in  a given  text  needs  to  be 
comprehended  by  an  L2  learner  of  English  in  order  to  (a)  adequately  comprehend  the  text,  (b)  comprehend  a 
new  vocabulary  item  by  context,  and  (c)  read  comfortably/read  for  pleasure.  Research  (Liu  & Nation,  1999) 
suggests  that  at  least  95%  of  the  mnning  words  in  a text  need  to  be  known  to  the  learner,  in  order  for  unknown 
vocabulary  to  be  comprehended  by  context.  This  95%  figure,  which  indicates  that  a reader  will  encounter,  on 
average,  one  unknown  word  for  every  two  lines  of  (10  words  average  per  line)  text,  and  20  unknown  words  on  a 
(400 -word  average)  page,  seems  as  well  to  be  a lowest  threshold  for  successful  reading  comprehension.  Laufer 
(1988),  examining  successful  readers  of  an  English  for  Academic  Purposes  text  in  the  “First  Certificate  in 
English”  exam,  found  that  scores  in  the  95%  percentile  and  above  had  a higher  number  of  successful  readers. 
Readers  who  scored  at  90%  (one  unknown  word  per  line  of  text)  did  not  do  significantly  better  or  worse  than 
others  scoring  up  to  94%;  these  scoring-groups  did  not  have  many  successful  readers.  A recent  study  (Liu  & 
Nation,  1999)  of  reading  comprehension,  examining  adequate  vocabulary  coverage  related  to  reading  for 
pleasure,  found  that  when  80%  of  the  words  of  a fiction  text  (i.e.  two  unknown  words  per  line)  were  familiar  to 
readers,  none  gained  adequate  comprehension;  at  90%  to  95%  some,  but  not  many,  gained  comprehension.  It 
was  found  that  only  at  about  98%  coverage  (eight  unknown  words  per  400  word  page)  was  “unassisted 
comprehension”  gained.  A general  conclusion  is  that: 

Knowing  academic  vocabulary  is  a high  priority  for  [L2]  learners  who  wish  to  do  academic  study  in  English.  After 
gaining  control  of  the  2,000  high  frequency  words,  learners  need  to  then  focus  on  academic  vocabulary.  Knowing  the 
2,000  high  frequency  words  and  the  Academic  Word  List  will  give  close  to  90%  coverage  of  the  running  words  in 
most  academic  texts.  When  this  is  supplemented  by  proper  nouns  and  technical  vocabulary,  learners  will  approach  the 
critical  95%  coverage  threshold  needed  for  reading”  (Nation,  1 999). 
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Thus,  this  area  of  research  has  been  able  to  establish  that  the  95%  figure  is  workable  as  a threshold  for 
known/unknown  vocabulary  in  EAP  study. 

Nation’s  2,000  highest  frequency  words  (headwords/word  families)  are  taken  from  West’s  1953  general 
service  list,  which,  though  aging,  is  defended  as  remaining  “the  best  of  the  available  [general  service  vocabulary] 
lists”  (Nation  & Waring,  1997).  Coxhead’s  AWL  is  composed  of  the  most  frequent  word  families  from  a 
3,500,000  running  word  academic  corpus  containing  a balance  of  the  four  general  fields  of  science,  arts, 
commerce  and  law — with  each  of  these  fields  further  divided  into  seven  subject  areas.  Each  word  contained  in 
the  list  has  a range  that  includes  all  28  (4  x 7)  subject  areas  (the  corpus  texts  are  divided  equitably  across  all 
subject  areas),  and  a frequency  in  the  complete  corpus  of  at  least  100.  The  third  list,  the  technical/other  word  list, 
must  be  either  imported  or  composed  by  the  teacher:  this  list  may  be  designed  as  field-specific,  syllabus-specific, 
or  as  a multi-disciplinary  list,  at  the  teacher’s  discretion.  A breakdown  of  coverage  by  the  different  kinds  of 
vocabulary  in  a typical  academic  corpus  is  seen  in  Table  1,  below. 


Type  of  Vocabulary 

% of  Coverage 

1**  10(X)  word  families: 

71.4% 

2"^^  1000  word  families: 

4.7% 

AWL  570  word  families: 

10.0% 

Others : 

13.9% 

Total: 

100.0% 

Table  1:  Percent  word-coverage  in  the  academic  corpus  (Coxhead,  1998) 

In  terms  of  EAP  coverage,  as  a rule  of  thumb  the  most  frequent  1,000  GSL  words  cover  approximately  up  to 
77%,  the  second  1,000  cover  approximately  up  to  5%,  AWL  (semi -technical)  vocabulary  will  account  for 
approximately  8.5%  -10%,  and  technical  words  (words  very  closely  related  to  the  topic  and  subject  area  of  the 
text)  typically  cover  about  5%  of  the  running  words  of  a text.  The  total  percent-coverage  then,  is  95.5% -97%, 
which  provides  the  target  goal  for  academic  vocabulary  acquisition,  leading  to  the  unaided  comprehension  of 
unknown  vocabulary  by  context,  and  successful  reading  comprehension,  within  a particular  academic  field.  The 
additional  “missing”  5%  represents  vocabulary  of  lower  frequency  than  the  first  2,000  GSL  words,  and/or  words 
which  are  semi-technical  but  not  in  the  AWL  list. 

We  can  note  the  specificity  of  the  AWL  to  academic  texts.  The  third  group  of  1,000  GSL  word  families 
(words  with  a frequency  of  2,000  to  3,000)  covers  only  about  4.3%  of  the  AWL  corpus.  Though  there  has  been 
little  comparative  research,  one  study  has  shown  an  AWL  coverage  of  only  1.7%  in  fiction  texts,  compared  to 
10%  coverage  in  academic  texts  (Nation,  1999).  These  results  indicate  a high  degree  of  specificity  of  academic 
vocabulary,  and  suggest  that  while  the  typical  EFL  student  will  have  a goal  of  comprehending  the  first  3,000 
GSL  word  families,  it  is  beneficial  for  the  academically-oriented  student  to  focus  on  semi-technical  vocabulary 
after  they  have  succeeded  in  comprehending  the  first  2,000  GSL  word  families.  For  the  “English  for  academic 
purposes”  (EAP)  student  then,  the  GSL  2,000  highest-frequency  words  represent  core  vocabulary,  while  the 
AWL  words  can  be  considered  high-frequency  vocabulary. 

By  focusing  on  words  within  the  academic  word  list,  students  will  utilize  their  study  time  efficiently,  as  AWL 
vocabulary  has  relevance  across  academic  disciplines.  Students  entering  an  academic  program  can  be  assessed 
using  a measure  such  as  the  Vocabulary  Levels  Test  to  detemiine  whether  they  have  a satisfactory 
comprehension  of  the  first  2,000  GSL  words,  and  to  what  extent  semi-technical/AWL  vocabulary  is  known.  As 
well,  AWL  vocabulary,  studied  within  the  context  of  academic  texts,  should  provide  an  efficient  approach  to 
TOEFL  preparation. 


EFL  design  concept 

Emerging  technologies  are  leading  to  the  development  of  many  new  opportunities  to  guide  and  enhance  learning  that 
were  unimagined  even  a few  years  ago.  — Robbie  McClintock  (1999) 

The  idea  for  the  Web-VOCAL  program  began  with  classroom  observations  made  over  several  years  with 
students  in  several  fields  (science,  economics,  social  welfare,  and  liberal  arts)  who  were  attempting  to  read 
content-oriented  English  texts  assigned  both  in  EFL  reading  classes  and  in  non-EFL  classes  taught  in  Japanese. 
Typically,  students  read  with  a translation  dictionary  nearby,  looked  up  unknown  words  and  wrote  the  Japanese 
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translation  above  the  word  in  the  text,  and/or  in  a notebook,  paired  with  the  English  term.  This  process  was 
laborious  and  time-consuming.  In  some  native-language  (LI)  course  curricula,  particularly  in  science  and 
English-language  literature  classes,  students  were  assigned  readings  in  difficult  content-related  English-language 
texts  (e.g.  Walden,  Silent  Spring,  scientific  essays,  literary  review  essays).  In  such  cases,  students  would 
sometimes  be  looking  up  several  vocabulary  items  per  line  in  their  attempts  to  comprehend  the  texts. 

One  of  the  principles  defining  the  purpose  of  reading  in  “Biglish  for  specific  purposes”  (ESP)  study  as 
opposed  to  general  EFL  study  is  the  functional  shift  from  ‘text  as  a linguistic  object  (TALO)’  to  ‘text  as  a 
vehicle  for  information  (TAVI)’  (Dudley-Evans  & St.  John,  1998):  “The  key  principles  [are]  that,  for  ESP 
learners,  extracting  information  accurately  and  quickly  is  more  significant  than  language  details;  that 
understanding  the  macrostructure  comes  before  language  study;  and  that  application  of  the  information  in  the 
text  is  of  paramount  importance.”  Web-VOCAL’s  basic  goal  is  to  promote  the  TAVI  process  by  simplifying, 
accelerating  and  enhancing  the  process  by  which  students  extract  meaningful  information  from  texts — through 
aiding  comprehension  of  targeted  vocabulary  items,  presenting  collocations  and  providing  web  resources.  The 
software  has  been  designed  to  have  “low-impact  imposition”  in  that  software  function  avoids  complex  user- 
routines  (difficulty  of  use)  or  conceptual  hurdles  (high  software  learning-curves)  that  might  cause  the  student  to 
have  greater  difficulty  utilizing  the  CALL  resources  than  the  non-computational  resources  already  at  hand. 

Upon  the  above  pedagogical  foundation,  cognitive  enhancements  to  the  reading  process,  including  (a) 
vocabulary  acquisition,  (b)  memorization,  (c)  organization,  and  (d)  novel  research  strategies,  become  possible  in 
the  CALL  design  environment.  Corpora/collocation  features  quickly  yield  sample  usage  sentences,  which  can  be 
easily  saved  for  later  study.  Hierarchic  GSL/AWL/Technical  vocabulary  parsing  helps  the  student  categorize  and 
“rate”  a particular  vocabulary  item  for  generic  relevance.  Vocabulary  analysis  of  texts  provides  a rapid  means  to 
roughly  access  texts  for  difficulty. 

CALL  in  the  classroom  also  has  the  potential  to  change  the  dynamic  between  student  and  teacher: 

With  digital  information  technologies,  what  is  pedagogically  possible  changes.  Possible  educational  attainments  are 
different  and  greater  than  they  have  hitherto  been.  . . . Educators  are  engaging  in  the  social  construction  of  a new 
educational  system  ...  in  this  case  from  the  potentialities  arising  through  the  use  of  information  technology  in 
education  (McClintock,  1999). 

One  of  the  several  benefits  of  CALL  in  the  classroom  is  that  learners  are  given  “freedom  to  explore 
alternative  pathways — to  find  and  develop  their  own  style  of  learning”  (Berge  & Collins,  1995).  Synergy 
between  the  user,  CALL  program,  classroom  environment,  curricula,  and  teacher  cannot  be  determined  without 
pilot  and  outcomes  studies.  Nevertheless,  certain  scenarios  become  possible  which  are  unattainable  outside  of 
the  Web-VOCAL  environment.  Students  are  able  to  input  their  own  texts  and  use  the  program  to  analyze  any 
text -based  correspondence  (e.g.  email,  internet);  they  can  share  study-files  and  bookmarks.  The  program  may 
also  be  used  in  independent/distance-leaming  environments.  The  program  is  individually  customizable,  giving 
students  power  and  autonomy:  for  instance,  students  can  build  up  large,  subject/field -tailored  vocabulary 
databases,  store,  organize,  and  (yet)  easily  access  them  over  long  periods  of  time;  as  well,  students  can  be  shown 
how  to  create  their  own  micro -corpora  (see  section  4.1).  Web-VOCAL  is  also  effective  for  the  second-language 
reader  engaged  in  independent  study:  academic  researchers/professors  needing  to  read  and/or  publish  (in 
English)  in  international  journals  can  make  use  of  the  program. 


Program  Description 

Using  a browser  (e.g.  IE-5/6)  students  download  Web-VOCAL  from  the  class  homepage  (Fig.l).  Following 
simple  instructions,  the  program  self-installs,  creating  two  folders  (in  ” C:\Program  Files”)  and  a desktop  shortcut. 
All  files  except  corpus  files  are  contained  in  the  “WebVocal”  folder.  A second  folder  (‘WebVocCorpus”) 
contains  the  corpus  files.  This  allows  students  to  export  Web-VOCAL  to  another  computer  on  a single  floppy 
disk,  as  the  program  and  essay  files  combined  are  unlikely  to  exceed  1.2  megabytes.  The  main-program  view 
can  be  seen  in  Figure  2.  After  Web-VOCAL  installs,  students  click  the  desktop  shortcut  link,  the  program  opens 
and  the  student  presses  the  (large  orange)  “BROWSER”  button  in  the  main  window  (as  mentioned  on  the  class 
homepage).  The  default  browser  URL  is  the  class  homepage,  so  the  Web-VOCAL  browser  returns  the  student  to 
the  class  homepage  where  the  essays  (as  .txt  file  links)  are  located.  Clicking  on  an  essay  link  automatically 
downloads  the  file  and  places  it  within  the  “Source  Reading  Text”  (SRT)  frame,  where  it  can  be  analyzed  for 
word -frequencies  and  vocabulary  sets.  Collocations  for  words/phrases  can  be  found  by  using  one  of  using  four 
different  corpus  sets  (highlight  the  word/phrase,  right-click  the  mouse,  select  “Collocations”).  Relevant  “micro- 
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corpora”  corpus  sets,  designed  by  teacher  or  student  for  Web-VOCAL,  can  be  created  from  a CD-ROM 
encyclopedia  in  little  time  (see  section  4.1);  individual  corpora  can  be  selected  through  the  “Tools”  menu. 

There  are  two  methods  for  incorporating  Web  information  into  the  SRT.  The  Web-VOCAL  browser  will 
automatically  strip  all  tags  (creating  a .txt  file)  of  any  HTML  page  and  display  the  new  file  in  the  SRT  window 
(by  pressing  the  “ANALYZE”  button  in  the  Web-VOCAL  browser).  All  resulting  files  can  be  saved;  any  .txt  file 
or  HTML  file  can  be  imported  into  the  program  as  well. 

Vocabulary  items  found  in  the  reading-text  or  collocation  sentences  in  the  “Collocations”  frame  (below  the  SRT 
frame)  can  be  searched  using  one  of  the  four  “Internet  link’  buttons  found  on  the  upper  right-hand  side  of  the 
main-program  window.  Experience  has  shown  that  often,  Japanese  students  have  little  idea  of  the  web  resources 
available  which  may  aid  them  in  English  comprehension.  The  four  “Internet  link’  buttons  each  contain  several 
URLs:  “E  Site”  contains  Eiglish  dictionaries,  encyclopedias,  thesauri;  “J  Site”  contains  similar  Japanese 
resources;  “Trans.”  Contains  English-Japanese/Japanese-English  translators  (there  are  translator  sites  providing 
real-time,  free,  side-by-side  bilingual  translations  of  English/Japanese  web  sites).  “Search”  contains  search 
engine  URLs,  both  Japanese  and  English.  URLs  defined  by  the  “Internet  link’  buttons  can  be  easily  changed  by 
the  teacher.  Voice  synthesis  is  also  available,  providing  text-to-speech  ability,  aiding  in  English  pronunciation 
(right-click  mouse  menu.  The  “Tools”  menu  contains  a “Web  Resources”  button;  this  button  links  to  a teacher- 
defined  set  of  URLs  designed  to  be  subject/course  specific  links. 

When  students  click  on  a headword  in  the  “Word  Lisf’  frame,  all  words  in  that  word-family  will  be 
automatically  highlighted  throughout  the  main  text.  Individual  words  within  a word  family  can  likewise  be 
highlighted  in  the  main  text.  Often  students  are  unable  to  save  their  data  on  the  university’s  LAN  between 
sessions,  aid  use  different  computers  each  day.  For  this  reason,  Web-VOCAL  is  designed  to  be  easily 
downloaded  onto  a floppy,  and  includes  a save/delete/export/import  feature  for  internet  bookmarks,  so  students 
can  keep  track  of  their  internet  research  and  maintain  both  a web  and  an  essay  database.  There  are  several  other 
features  at  the  student’s  disposal.  Any  text  can  be  sorted  to  discover  high-frequency  vocabulary  (“GSL  button”), 
academic  vocabulary  (‘AWL  button”),  and  extra  vocabulary  (‘XTRA  button”),  which  contains  low-frequency 
vocabulary  absent  from  any  other  category.  The  “TECH  button”  is  designed  to  provide  vocabulary  created  by 
the  teacher  to  display  low-frequency  vocabulary:  1)  relevant  to  the  course,  2)  the  essay(s)  under  study,  3)  an 
upcoming  lecture,  4)  scientific  terms,  5)  proper  names,  6)  archaic  spellings,  etc. 


Creating  Micro  Corpora 

Web-VOCAL  does  not  include  its  own  corpora  at  this  time,  advocating  (and  instructing  the  user  on)  the 
construction  of  micro -corpora.  As  shown  by  Tribble  (and  our  own  exp  erience)  a teacher  is  able  to  construct  a 
micro -corpus  of  some  50,000  words,  drawn  from  a CD-ROM  encyclopedia,  in  about  30-40  minutes: 

The  non-standard  corpus  which  I am  advocating  for  students  with  an  interest  in  learning  how  to  write  factual  texts  . . . 
is  the  widely  available  multimedia  encyclopedia.  I am  proposing  the  use  of  such  a corpus  with  students  who  are 
beginning  to  learn  how  to  write  formal,  professionally  oriented  texts. . . The  Encarta  Encyclopedia  [Deluxe  2002  CD- 
ROM  contains  60,000]  separate  articles  listed  in  the  "search"  facility.  These  contain  anywhere  between  200  and  5000 
words.  If  we  take  a pessimistic  mean  of  1000  words,  then  the  biggest  possible  "corpus"  based  on  this  source  of  texts 
could  contain  [60,000,000]  words.  . . . [One  is]  able  to  construct  themed,  twenty  to  thirty -thousand  word  micro- 
corpora in  fifteen  to  twenty  minutes.  Although  such  a corpus  sounds  insignificantly  tiny  when  compared  with  the 
huge  corpora  which  can  now  be  accessed,  I would  argue  that  if  one  wishes  to  investigate  the  lexis  of  a particular 
content  domain  (e.g.  health)  a specialist  micro- corpus  can  more  often  be  useful  than  a much  larger  general  corpus. 
(Tribble,  1997)  [Encarta  Encyclopedia  information  updated  to  2002.] 

The  corpus  files,  which  may  grow  large,  can  be  exported  on  a second  floppy  disk.  With  regard  to  .txt  file  sizes, 
the  Canterbury  Corpus  (Powell,  2001)  shows  that  a work  of  fiction  containing  140,000  words  yields  a .txt  file  of 
approximately  750Kb  while  a non-fiction  work  of  101,000  words  yields  a .txt  file  of  600Kb.  A complete  Web- 
VOCAL  corpora  “set”  of  four  50,000  word  corpuses  (containing  “general”,  “academic”,  “journaF  and  “user” 
corpora)  will  thus  fit  on  a 1.2  megabyte  floppy  disk;  if  larger  corpus  sets  are  needed  they  can  be  accommodated 
on  extra  floppies. 
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Discussion 


We  have  found  that  a fruitful  CALL  design-approach  weds  pedagogical  considerations  to  interface  metaphors 
in  a co-emergent  design  process,  giving  equal  weight  to  educational  conceptual  structures  and  application 
control  structures.  The  psychological  space  of  user-CALL  program  interaction  is  a virtual  space,  which  remains 
largely  unexplored  in  terms  of  how  the  design  of  educational  tools  should  best  be  implemented.  In  order  to 
further  investigate  the  potential  of  the  program,  Web- VOCAL  is  being  readied  for  pilot  testing  later  this  year. 
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Teaching  mathematics  and  science:  Using  global  learning  for  the  effective 
preparation  of  prospective  elementary  teachers 
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Department  of  Curriculum  and  Instruction,  Wichita  State  University 
kav.gibson@wichita.edu  mara.alagic@wichita.edu  connie.haack@wichita.edu 

Abstract:  Research  and  experience  suggest  that  structured  reflections  and  metacognitive  thinking 
positively  influence  prospective  teachers’  pedagogical  content  knowledge  (PCK),  and  beliefs  and 
attitudes  (dispositions)  about  mathematics/science  learning  and  teaching.  To  provide  a challenging 
learning  environment  for  such  activities,  prospective  teachers  from  different  educational  cultures 
will  be  paired  in  this  global  learning  project.  They  will  engage  in  interactive  (online)  reflective 
journaling  based  on  experiences  that  combine  content  and  pedagogy;  and  design  a sequence  of 
activities  to  teach  a mathematics/science  concept  that  will  be  displayed  on  joint  web  pages.  The 
activities  will  be  assessed  for  evidence  of  positive  growth  in  PCK  and  dispositions  related  to  the 
teaching  and  learning  of  mathematics/science.  Specifically,  the  project  is  guided  by  the  question: 

How  do  cross-cultural  shared  experiences  affect  the  development  of  specific  PCK  and  influence 
prospective  teachers*  dispositions? 

There  is  a basic  need  to  develop  greater  self-efficacy  in  elementar}'  education  teachers  related  to 
mathematics  and  science  learning  and  teaching.  Many  of  these  teachers  have  negative  beliefs,  attitudes  and  anxieties 
about  learning  mathematics  and  science  that  constitute  a significant  barrier  to  teaching  these  subjects  in  a 
meaningful  way.  Prospective  teachers  have  developed  their  beliefs  over  time  before  they  enter  the  university.  To 
change  undesirable/negative  dispositions  therefore  will  take  time  (Hart  2002).  Educators  in  the  USA  and  Australia 
are  recognizing  this  barrier  and  making  efforts  to  overcome  it  (Mulholland  & Wallace  2001;  Ramey-Gassert  & 
Shroyer  1992;  Schoon  & Boone  1998;  Watters  & Ginns  1995). 

A number  of  studies  support  the  belief  that  reflective  teachers  positively  effect  the  education  experiences  of 
their  students  both  in  the  development  of  positive  dispositions  and  pedagogical  content  knowledge  (Abell  & Bryan 
1997;  Clarke  2000;  Fennema  & Romberg  1999;  Francis,  Tyson  & Wilder  1999;  Hart  2002).  Abell  and  Bryan 
designed  their  elementary  science  methods  course  around  '\t)hc  reflection  orientation  ...  grounded  in  the  belief  that 
learning  to  teach  science,  like  learning  science  itself,  is  a process  of  re-evaluating  and  reforming  one’s  existing 
theories  in  light  of  perturbing  evidence  (p.l54).” 

At  Wichita  State  University,  prospective  elementary  teachers  take  an  integrated  mathematics  and  science 
methods  class  in  the  first  semester  of  their  senior  year,  just  prior  to  their  student  teaching  semester.  Reflective 
practice  is  nurtured  to  ensure  a supportive  environment  that  addresses  the  barrier  of  teachers’  negative  beliefs, 
attitudes  and  anxieties.  Student  responses  to  weekly  online  reflective  questions  have  focused  attention  on  a number 
of  issues  related  to  teacher  dispositions  and  topics  broached  during  classroom  activities.  The  first  result  is  that, 
faculty  gain  insight  into  the  thinking  of  the  students,  and  the  insights  guide  ongoing  efforts  for  cultivating  effective 
mathematics  and  science  teaching.  In  addition,  students  have  increased  opportunities  to  form  the  habit  of  reflective 
journaling.  Anecdotal  evidence  suggests  that  students  are  particularly  motivated  to  examine  their  thoughts  in  relation 
to  those  of  their  classmates.  Online  synopses  of  student  responses  have  often  raised  the  level  of  interest  and  resulted 
in  further  discussions  about  students’  positive  growth  in  pedagogical  content  knowledge  and  dispositions. 

Reflective  thinking  is  fostered  in  lesson  preparation  and  teaching  assignments.  Lesson  planning  and  peer 
teaching  routinely  contain  a required  reflection  on  the  process  and  product.  The  course  culminates  in  a five-week 
field  experience  with  many  opportunities  and  demands  for  reflection  with  respect  to  teaching  experiences  in  the 
elementary  classroom.  Often  the  primary  concern  expressed  in  the  latter  reflections  is  about  classroom  management, 
but  it  is  still  felt  that  the  reflective  piece  is  needed  that  is  more  directly  connected  to  mathematics  and  science  related 
pedagogical  content  knowledge. 

For  example,  Teacher’s  Inquiry  into  Children’s  Knowledge  and  Learning  Evolvement  (TICKLE)  is  an 
assignment  designed  to  make  the  connection  between  content  learning  and  teaching  practice  more  apparent  to  the 
students.  The  TICKLE  assignments  are  given  as  a springboard  to  ideas  about  pedagogical  content  knowledge.  Each 
student  works  with  a child  between  the  ages  of  four  and  twelve  and  explores  the  child’s  thinking  about  concepts 
within  a broader  topic  such  as  volume  or  light.  Before  each  TICKLE  assignment,  a collection  of  activities  is 
introduced  to  provide  the  background  needed  to  design  activities  that  they  deem  developmentally  appropriate  for  the 
child  with  whom  they  are  working.  After  analyzing  individual  results,  students  come  back  to  class  with  new 
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pedagogical  content  knowledge.  Together  they  build  a deeper  understanding  of  the  developmental  sequence 
necessary  to  teach  a given  concept.  As  a result  they  begin  to  answer  for  themselves  the  question:  what  is  appropriate 


for  the  grade  level  that  I’ll  be  teaching? 

Faculty  have  found  that  the  diversity  of  learners,  even  within  a single  classroom,  demands  a diversity  of 
teaching  approaches.  Already  the  challenges  of  negative  student  dispositions  have  been  addressed  in  a variety  of 
ways.  As  expected  in  the  presence  of  student  diversity,  our  multiple  strategies  have  met  with  mixed  success.  Some 
students  respond  to  a given  strategy  and  other  students  respond  to  significantly  different  strategies.  However,  the 
common  successful  thread  among  the  strategies  has  been  an  emphasis  on  reflective  thinking  about  learning  and 
learning  to  teach  mathematics  and  science  within  the  existing  learning  environment.  With  this  in  mind,  we  are 
attempting  to  extend  this  reflective/metacognitive  environment  by  pairing  students  at  Wichita  State  University  with 
their  counterparts  at  an  Australian  university  to  determine  how  cross-cultural  reflection  and  communication  affects 
students’  self-efficacy. 

Initially,  students  will  engage  in  guided  reflections  centered  on  questions  related  to  their  personal  beliefs 
about  teaching  and  learning  mathematics  and  science.  To  become  acquainted  with  each  other’s  dispositions,  they 
will  co-reflect  via  email  on  such  questions  as:  What  do  you  know  about  reflective  practice?  Based  on  past 
experience,  how  do  you  feel  about  learning  science  and  mathematics?  What  are  your  concerns  about  teaching 
mathematics/science?  What  would  help  you  become  a better  science/mathematics  teacher?  This  ongoing  reflective 
practice  will  provide  a supportive  learning  environment  to  encourage  the  changing  of  negative  dispositions  and  the 
development  of  mathematics  and  science  activities  for  teaching  with  understanding  (Fennema  & Romberg  1999). 

For  part  of  the  semester,  students  at  both  universities  will  be  involved  with  identical  mathematics  and 
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fi.  Based  on  that  common  curriculum,  pairs  of  students,  one  from  each  country,  will  collaboratively 
develop  and  publish  on  the  web  a collection  of  activities.  In  the  process  of  designing  these  activities,  students  will  be 
guided  to  reflect  on  their  individual  learning  as  well  as  the  collaborative,  creative  process  in  order  to  support  meta- 
cognitive  thinking  and  learning  in  the  development  of  PCK  and  positive  dispositions.  The  intent  then  is  to  have  the 
students  publish  their  reflections  on  the  web  along  with  their  activities.  Such  sharing  will  allow  both  current 
classmates  and  future  students  in  these  methods  classes  to  observe  and  learn  from  other  students’  experiences. 

A global  learning  environment  designed  in  this  manner  will  provide  reflective  opportunities  for  faculty  and 
students  through  content -specific  dialogue.  These  cross-cultural  shared  experiences  should  positively  effect 
prospective  teachers’  development  of  specific  pedagogical  content  knowledge  and  bring  about  positive  changes  in 
their  evolving  dispositions. 
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Abstract:  How  is  technology  (IT)  integrated?  When  is  IT  integrated?  Where  is  IT 
integrated?  How  much  is  IT  integrated?  Too  often,  little  thought  is  given  to  determine  a 
school  district’s  response  to  these  questions  or  to  the  analysis  of  current  IT  usage  or 
future  IT  needs.  The  approach  to  planning  for  the  integration  of  educational  IT  into  the 
learning  environment  described  in  this  paper  focuses  on  the  collaborative  efforts  of 
school  district  personnel,  and  faculty  and  graduate  students  from  a local  university  in  the 
mid-western  region  of  the  United  States. 


Introduction 

Forward  looking  school  leaders  concerned  about  their  ability  to  effectively  lead  schools  into  the 
IT-based  learning  environment  of  the  21st  century  are  aware  of  certain  verities  of  educational  life  that 
should  impact  school  leader  behavior.  Bracci  (1999)  suggested  one  of  those  truths  related  to  the  integration 
of  information  IT  into  learning  environments.  The  questions  became  not  whether  IT  should  be  integrated, 
but;  How  is  IT  integrated?  When  is  IT  integrated?  Where  is  IT  integrated?  How  much  is  IT  integrated? 

Another  truth  in  the  world  of  education  suggests  that  the  mere  presence  of  IT  has  became  a 
measure  by  which  some  school  districts  assessed  their  effectiveness.  Many  administrators,  parents,  and 
teachers  believed  that  if  they  had  a large  number  of  computers  in  the  classroom,  they  were  "cutting  edge" 
and  therefore  effective.  However,  too  often,  little  to  no  thought  was  given  to  determine  a district’s  current 
IT  usage  or  future  IT  needs.  Jukes  (2000)  observed,  "The  key  is  how  you  use  the  technology,  not  how 
often"  (p.  73).  It  has  been  made  clear  that  effective  use  of  IT  in  the  learning  environment  begins  with 
deliberate  and  futures -based  planning. 

The  approach  to  planning  for  the  integration  of  IT  into  the  learning  environment  described  in  this 
paper  focuses  on  the  collaborative  partnership  efforts  of  school  district  personnel,  and  faculty  and  graduate 
students  from  Wichita  State  University  in  the  mid-west  of  the  United  States.  These  educators  formed  a 
partnership  to  collaborate  on  the  development  of  an  approach  to  strategic  planning  for  IT  integration  that 
has  been  effective  in  revisiting  and  re-enlivening  a school  district  IT  plan  that  had  lost  its  way. 


Research  Design 

In  a recent  paper,  Gibson  (2002)  claimed  that  school  leaders  must  be  the  focus  of  new  efforts  to 
guarantee  the  effectiveness  of  IT  integration  efforts.  While  agreeing  that  significant  attention  should 
continue  to  be  paid  to  the  IT  development  of  tomorrow’s  teachers,  he  suggested  an  increasing  realization 
that  teachers  were  not  as  effective  in  their  IT  integration  efforts  without  the  support  of  school  leaders 
capable  of  motivating,  collaborating,  and  supporting  IT  integration  efforts.  The  graduate  program  in 
Educational  Administration  at  Wichita  State  University  attempted  to  address  these  issues  through 
construction  of  a program  of  school  leader  preparation  designed  to  produce  visionary  leaders  for  twenty- 
first  century  schools.  The  study  reported  here  emphasizes  the  collaborative  research  orientation  of  this 
program  and  the  need  to  use  current  data  on  technology  usage  in  planning  for  IT  use.  The  study  was 
designed  to  examine  the  use  of  IT  in  the  instructional  environment  of  a local  school  district  and  make 
recommendations  for  updating  the  district  IT  plan. 

Multiple  research  methods  used  to  explore  the  research  questions  relating  to  current  use  of  IT, 
perceptions  of  IT  needs  from  students',  teachers’,  and  administrators’,  and  suggested  improvements 
regarding  the  integration  of  IT  included  focus  groups  with  district  students  and  faculty,  interviews  with 
administrators  and  IT  coordinators,  and  an  analysis  of  district  documents.  Inductive  and  constant 
comparative  analysis  of  these  triangulated  data  resulted  in  a series  of  conclusions  and  recommendations 
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designed  to  assist  district  staff  in  revising  their  district  IT  plan,  and  a process  that  could  be  used  in  the 
future  for  continued  data-based  analyses  of  district  IT  needs. 


Conclusions  and  Recommendations 

The  study’s  conclusions  were  grouped  into  seven  major  themes:  current  support,  parent  usage, 
administrator  usage,  teacher  usage,  student  usage,  perceived  needs,  and  proposed  needs.  Recommendations 
related  to  these  findings  were: 

• Formative  evaluations  of  the  IT  plan  should  be  conducted  yearly  and  focus  on  integrating  IT  into 
the  learning  environment  emphasizing  student-centered  activities  and  targeted  staff  development. 

• The  allocation  of  funds  for  IT  should  continue  to  increase. 

• Funds  should  be  allocated  to  support  full-time  coordination  and  assistance  throughout  the  district. 

• Resources  (personnel,  time,  financial,  equipment,  and  facilities)  should  support  the  plan. 

• Access  to  Internet  should  be  made  available  for  all  students  and  staff. 

• Training  should  be  provided  for  students,  parents  and  teachers  to  enhance  the  effective  use  of  IT. 

• Staff  development  should  increase  awareness  of  the  potential  of  IT  to  transform  the  learning 
process  to  focus  on  student-centered  activities. 

• Student-free  time  should  be  regularly  scheduled  to  assist  staff  in  learning  about  IT. 

• Results-based  staff  development,  designed  for  IT  integration,  should  be  targeted  specifically  to 
learner  skill  levels. 

• Support  and  encouragement  should  be  provided  to  all  staff  as  they  integrate  IT  into  the 
instructional  environment. 

• Emphasis  should  be  placed  on  innovative  ways  to  integrate  the  Internet  into  the  learning  process. 

• Specific  resources  (websites,  professional  library,  examples  of  lesson  plans,  network  contacts,  site 
visits,  expert  colleagues)  should  be  developed  to  assist  teachers  to  integrate  IT  into  learning 
environments. 


Conclusion 

As  a result  of  this  process,  school  district  personnel  identified  IT  needs  crucial  to  future  growth  within  this 
district  using  baseline  data  comprising  a realistic  evaluation  of  existing  usage  patterns,  an  understanding  of 
existing  attitudes  of  students,  teachers,  and  administrators  towards  IT  and  its  connection  to  the  learning  process. 
A clear  picture  of  the  gap  between  what  could  be  currently  accomplished  with  the  existing  IT  infrastructure  and 
skill  levels,  and  what  was  considered  to  be  realistic  future  needs  of  the  district  resulted. 

Further,  in  the  process  of  participating  in  this  program,  graduate  students  conducting  the  research  gained 
valuable  insight  into  their  responsibilities  as  IT  leaders,  at  the  same  time  as  exploring  a process  of  data 
gathering,  future  planning,  and  realistic  program  evaluation  that  will  benefit  them  as  school  leaders.  With  data 
derived  from  studies  like  this,  and  an  understanding  of  the  process  of  data  gathering  and  analysis  derived  from 
participation  in  this  program,  these  school  leaders  have  come  to  understand  the  purpose  of  identifying  the  IT 
realities  of  the  district  as  the  first  step  in  the  process  of  district  decision-making  based  on  assessing  the  status  of 
district  level  IT  usage  merged  with  an  understanding  of  future  district  IT  needs. 

Most  importantly,  these  students  discovered  a critical  criterion  required  for  IT  planning  was  vision.  As 
Jukes  (2000),  suggested,  integrating  IT  is  far  more  than  just  doing  computers’  or  allowing  students  to  spend 
time  online.  Effective  integration  occurs  only  when  IT  adds  real  value  to  recognized  educational  needs  of 
individual  learners.  These  graduate  students  gained  experience  suggesting  that  future  minded  educators  must 
have  a vision  that  integrates  IT  not  just  into  a school  or  a school  district,  but  into  the  learning  process. 
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Abstract;  Challenging  tomorrow’s  school  leaders  to  develop  deep  understanding  and 
respect  for  alternative  conceptions  and  responses  to  everyday  leadership  issues  is  a major 
undertaking.  This  paper  presents  a description  of  the  rationale  and  development  of  the 
Global  Forum  on  School  Leadership.  Conceived  as  an  extension  to  existing  reflective  and 
dialogic  practices  of  a pre-service  leader  preparation  program  in  the  mid-west  of  the 
United  States,  the  forum  attempts  to  incorporate  the  perspectives  of  global  colleagues  on 
issues  related  to  everyday  practices  and  school  leadership  conundrums  in  a mediated, 
asynchronous,  web-based  discussion  format.  This  project  adds  a needed  global 
orientation  to  leadership  preparation  in  a post  September  1 1th  international  context. 


Project  Rationale  for  the  Global  Forum  on  School  Leadership 

Developing  leadership  expertise  requires  foundational  knowledge  and  use  of  inquiry  and  reflective 
practice  skills  applied  in  context  specific  experiences.  These  experiences  provide  opportunity  for  the 
integration  of  theory  and  practice  focusing  on  school  improvement.  Yet,  the  contextualizing  of  leadership 
experiences  in  traditional  and  unvarying  cultural  contexts  often  presents  its  own  limitations,  particularly 
during  the  formative  period  of  leadership  philosophies,  perspectives,  and  practices.  For  that  reason,  the  idea 
of  incorporating  a global  orientation  to  leadership  preparation,  particularly  in  a post  September  1 national 
and  international  context,  has  merit  on  many  fronts. 

The  Global  Forum  on  School  Leadership  challenges  tomorrow’s  school  leaders  to  develop 
deeper  understanding  of  areas  of  responsibility  and  of  professional  behaviors  related  to  topics  of 
professional  concern  through  involvement  in  collegial  analyses  of  views  and  experiences  that  vary  from 
the  norm  or  the  expected.  Intellectual  and  professional  challenges  to  the  status  quo,  to  standard  and 
expected  culturally  derived  responses,  will  result  from  the  global  interactions  contained  within  this  shared 
international  forum. 


Description  of  the  Context  of  the  Project 

The  Masters  in  Educational  Administration  at  Wichita  State  University  has  been  designed  to 
prepare  visionary,  reflective  leaders  for  Kansas  schools.  This  integrated  program  prepares  future  school 
leaders  to  be  competent  in  leadership,  interpersonal  relationships,  school  law,  school  finance,  school 
personnel,  curriculum  management,  learning  theory,  inquiry,  human  development,  school  improvement 
processes,  and  supervision,  and  grooms  them  to  become  administrators  who  create  and  maintain  effective 
schools.  This  program  is  a clinically  oriented,  field-based,  applied  inquiry  program  that  distinguishes 
itself  from  traditional  administrator  preparation  programs  through  a design  that  emphasizes  the  integration 
and  contextualization  of  the  curriculum,  incorporation  of  field -based  research  studies,  a collaborative, 
team-based  approach  to  teaching  and  learning,  a mediated  internship  program,  and  a cohort-based  student 
support  structure.  Students  enrolled  in  this  program  participate  in  mediated  on-line  discussions  as  part  of 
the  assessment  of  their  performance  in  the  program. 

Based  upon  a theoretical  framework  derived  from  problem-based  learning  (PBL)  (Boud  & 

Feletti,  1991;  Bridges,  1992X  this  program  functions  with  the  belief  that  authentic  problems  of  practice, 
explored  in  collaborative  team  settings  lead  to  learner-directed  and  setting-enhanced  learning.  The 
underlying  propositions  of  constructivism  (Savery  & Duffy,  1995),  including  the  ideas  of  cognitive 
dissonance  and  negotiation  of  meaning,  are  core  program  components.  Studies  of  student  thinking  during 


the  initial  problem  analysis  phase  of  PBL  (De  Grave,  Boshuizen,  & Schmidt,  1996)  corroborate  the  belief, 
clearly  in  evidence  within  this  program,  that  exposure  to  different  ideas  in  a group  leads  to  conceptual 
change  and  that  group  interactions  serve  to  encourage  activation  and  elaboration  of  existing  knowledge 
and  integration  of  alternative  views. 


The  Global  Forum  for  School  Leadership 

Pre-service  school  administrators  enrolled  in  this  program  will  participate  in  mediated, 
asynchronous  discussions  with  local  and  international  colleagues  to  explore  issues  of  professional 
relevance.  Exploration  of  authentic  problems  of  practice,  analyses  of  philosophical  stances  on  leadership 
behaviors  and  orientations,  dissection  of  the  rationales  governing  daily  actions  and  decisions,  and 
challenges  to  strongly  held  beliefs,  attitudes,  and  dispositions  will  constitute  the  form  and  the  substance  of 
interactions  in  the  Global  Forum. 

Existing  groups  of  neophyte  administrators  in  training  and  their  professors  and  mentors  will 
interact  on  self  selected  issues  relevant  to  their  profession  and  to  their  studies.  Topics  of  discussion  would 
include  an  analysis  of  cultural  perspectives  on  any  areas  of  leadership  responsibilities  such  as  strategic, 
organizational,  instructional,  political,  and  community  leadership. 

The  technology  infrastructure  necessary  to  support  this  project  relies  on  accessibility  to  the 
internet  via  appropriate  computer  interfaces.  The  telecommunications  needs  of  the  project  requires  access 
to  Blackboard  and  to  a website  via  the  internet  for  participants  at  both  ends  of  the  interaction.  Threaded 
discussions,  and  a wxb-mcdiatcd  approach  will  be  supplemented  by  full  text  access  to  a variety  of  field - 
based  action  research  reports.  A trial  website  has  been  established  to  demonstrate  the  potential  of  sharing 
practitioner  conducted  research  (http://education.wichita.edu/aces/admin). 

Graduate  program  activities  focus  around  authentic  problems  of  practice,  explored  in  collaborative 
team  settings,  and  generating  learner-directed  and  setting-enhanced  learning.  Within  this  environment, 
technology  integration  is  modeled  by  faculty  (Blomeyer  & Clemente,  1997;  Gibson,  2001;  Gibson,  2000a). 
The  core  tenets  of  constructivism,  including  the  ideas  of  cognitive  dissonance  and  negotiation  of  meaning 
(Savery  & Duffy,  1995),  are  integral  components  of  this  program.  Program  graduates  acquire  an 
understanding  of  the  impact  of  information  technology  on  their  roles  as  visionary  leaders  of  schools  of  the 
future  and  experience  the  transformational  potential  of  technology  on  their  own  learning  process  (Gibson, 
2000b).  Students  engage  in  interactive  (online)  reflective  dialog  centered  upon  specific  school  leadership 
topics.  This  mediated  dialog  will  be  assessed  on  a continuing  basis  for  evidence  of  leadership  insight, 
reference  to  the  integration  of  theory  and  practice,  content  knowledge,  and  meta-cognitive  development. 
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Abstract:  This  paper  provides  the  rationale  and  description  of  a unique  programming 
environment  to  teach  the  fundamental  concepts  of  programming  (data  types,  input/output, 
control  structures,  functions,  arrays,  files,  and  the  mechanics  of  running,  testing,  and 
debugging  programs)  to  introductory  students  using  either  VBScript  or  JavaScript.  This 
environment  is  earmarked  for  courses  that  precede  the  first  programming  course  in  computer 
science.  Students  can  compose  and  execute  scripts  from  within  a text  editor  that  has  been 
configured  to  invoke  the  Windows  Script  Host  (VBScript),  or  a web  browser  (JavaScript). 
This  creates  an  interactive  environment  that  matches  the  functionality  of  an  integrated 
development  environment.  The  target  audience  includes  students  with  little  or  no 
programming  background  who  need  a brief  introduction  to  programming  before  immersion  in 
their  first  rigorous  course  in  C-H-,  Java,  C#,  or  Web  programming.  The  advantages  of  using  a 
scripting  language  are  many:  text -based  code-writing  requires  students  to  construct  each 
programming  component  rather  than  setting  check-boxes  or  using  drag-and-drop  strategies; 
the  syntax  is  often  friendlier  than  the  aforementioned  languages;  the  interpreted  environment, 
where  partial  execution  can  take  place,  can  encourage  both  comprehension  and  debugging 
skills;  the  scripts  may  be  executed  directly  or  embedded  within  an  HTML  template  file  and 
viewed  through  a browser  with  a single  click.  A complete  description  and  downloadable 
sample  files  may  be  found  at  http://www.uwsp.edu/cis/dgibbs/EDMEDIA2QQ2/index.htm. 


Introduction 

Students  emerging  from  today’s  secondary  schools  have  more  experience  with  computers  than  ever 
before.  That  experience,  however,  is  typically  limited  to  software  applications  (word  processors,  spreadsheets, 
database  managers)  and  Internet  browsing.  Too  few  have  received  instruction  in  how  computers  work  from  the 
point-of-view  of  software  instructions.  Even  fewer  know  how  to  create  a program  and  execute  it.  Those  with 
some  exposure  to  programming  often  struggle  with  the  breadth  of  material  encountered  in  the  first  course  in 
programming  (CSl)  at  the  university  or  community  college.  The  reasons  for  the  struggle  are  understandable 
when  one  stops  to  consider  all  that  is  taking  place  in  the  typical  CSl  course:  the  semantics  of  introductory 
programming,  the  syntax  of  the  language,  the  software  development  life -cycle  (analyze,  design,  code,  test  and 
debug,  maintain),  use  of  built-in  data  structures,  algorithm  development,  and  problem  solving.  In  order  to  write, 
test,  and  debug  programs,  students  must  also  learn  a programming  environment. 

To  make  matters  worse,  facets  of  the  CSl  course  mentioned  above  have  evolved  in  the  wrong 
direction,  i.e.  they  have  been  made  more  difficult  rather  than  easier  for  a variety  of  reasons.  Demands  of  the 
marketplace  and  availability  of  programming  tools  have  resulted  in  educational  institutions  adopting 
syntactically  complex  languages  in  their  introductory  courses.  C,  C++,  and  Java  all  fit  in  this  category. 
Mastering  their  difficult  syntax  actually  interferes  with  the  learning  of  programming  fundamentals.  LOGO  and 
Pascal,  both  much  fnendlier  syntactically,  were  fine  languages  for  learning  how  to  program  but  have  fallen  by 
the  wayside  because  they  are  not  used  to  any  significant  degree  in  industry.  Further,  the  programming 
environments  of  C,  C++,  and  Java,  currently  in  use  in  instructional  situations  are  those  provided  by  the  major 
language  developers.  They  have  been  created  for  professional  programmers  and  as  such  include  many  powerful 
features  that  the  novice  may  never  use. 


ERIC 


Solutions 


What  about  Visual  Basic?  While  it  is  true  that  VB  does  retain  the  friendly  syntax  of  its  predecessor 
(BASIC),  the  graphical  environment  encourages  users  to  “point-and-click’*  their  way  to  applications,  which  is 
exactly  what  novices  should  not  be  doing.  Computing  educators  have  long  resisted  using  graphical 
programming  environments  for  teaching  the  fundamental  concepts.  The  fear  is  that  in  using  forms,  drag-and- 
drop,  and  check-boxes,  students  do  not  learn  the  fundamentals  of  input/output,  decision  structures,  looping, 
subprograms,  files,  arrays,  and  data  structures. 

One  of  the  solutions  to  the  problems  outlined  above  has  been  to  insert  a course  into  the  curriculum 
designed  to  ease  entry  into  the  CSl  course.  Described  as  CSO  (“see-es-zero”)  the  course  typically  includes  the 
concepts  of  computing  and  information  processing,  algorithm  development  using  pseudocode  or  flowcharting 
(often  with  paper-and-pencil  techniques),  and  implementing  simple  programming  concepts  in  a user-friendly 
environment.  The  CSO  concept  has  grown  to  the  point  where  the  recent  release  of  the  Association  for 
Computing  Machinery’s  Computing  Curricula  2001  (ACM  2001)  included  exactly  such  a course  in  its 
description. 

Why  a Scripting  Language?  Why  an  Interactive  Environment? 

Why  learn  introductory  programming  with  a text -based  scripting  language  rather  than  a graphical 
forms -based  language?  To  construct  a program  in  a scripting  language,  students  must  enter  the  instructions, 
usually  by  keying  them  in  (although  copying  and  pasting  can  be  encouraged  later  on)  rather  than  pointing  and 
clicking.  There  are  no  mysteries;  both  the  code  and  any  error-messages  are  apparent  from  the  interpreted 
environment.  The  programming  environment  is  very  easy  to  use.  A freeware  text  editor  can  be  used  to  prepare 
the  programming  statements.  The  editor  is  configured  to  invoke  the  Windows  Script  Host  (VBScript)  or  the 
web  browser  itself  (JavaScript).  Error  messages  are  quite  accurate,  and  generally  indicate  the  line  and  character 
position  of  the  error. 

Interaction  with  an  electronic  text  is  made  possible  through  the  use  of  hyperlinks  within  a Word 
document.  (A  hypertext  document  would  achieve  the  same  functionality.)  Script  files  saved  with  a .vbs  or  .js 
extension  are  associated  through  the  Windows  operating  system  with  the  editor,  meaning  that  double  clicking 
the  hyperlinked  files  from  the  Word  document  invokes  the  editor  and  loads  the  file.  The  scripts  can  be  executed 
by  clicking  the  configured  menu  pick. 

In  summary,  the  interactive  environment  includes  an  electronic  text  with  clickable  links  that  load  code 
samples  right  into  the  editor  where  they  can  be  modified  and  executed.  Students  can  view,  modify,  and  execute 
existing  code  samples,  and  then  create  solutions  to  problem  exercises,  link  them  to  their  own  o-text  thus 
forming  an  e-portfolio  of  their  work. 

Beneflts  for  the  teaching  and  learning  of  CSO 

The  case  for  the  introductory  course  (CSO)  has  been  made  in  the  computing  curricula  throughout 
universities  and  community  colleges.  This  paper  argues  that  the  scripting  languages  of  VBScript  or  JavaScript 
meet  the  programming  needs  of  these  courses,  and  does  so  in  a manner  fruitful  for  students  and  teachers.  In 
addition  to  the  arguments  already  made,  there  are  other  benefits  to  using  scripting  and  the  interactive 
environment  described  here.  All  software  is  free;  there  are  no  compiler  or  interpreter  licensing  costs.  (The 
editor  is  freeware.  The  “interpreter”  is  provided  in  the  web  browser  which  is  also  free.)  VBScript  has  become 
the  default  scripting  language  for  writing  server  side  code  in  the  Microsoft  (Active  Server  Page)  environment, 
and  JavaScript  is  the  default  scripting  language  for  client  side  code.  While  beginning  students  may  be  a few 
semesters  away  from  client-server  programming,  the  initial  experience  will  be  beneficial.  Students  can  use  the 
electronic  book  as  the  basis  for  building  their  programming  e-portfolio,  a collection  of  programming  concepts 
and  their  VBScript  implementation.  In  creating  their  own  version  of  the  electronic  text,  students  are  linking 
files,  copying  and  pasting  code  samples,  and  developing  skills  in  managing  a hypertext  document. 
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Abstract:  Academic  dishonesty  is  a concern  at  any  educational  level.  However,  many 
faculty  members  feel  uncomfortable  with  delivering  courses  in  the  online 
environment  due  to  a concern  that  students  may  find  it  easier  to  participate  in 
academic  dishonesty  than  they  would  in  a traditional  classroom.  This  paper 
looks  at  factors  that  are  considered  to  influence  academic  dishonesty  and  how 
these  factors  can  be  considered  in  the  design  of  online  courses  to  promote 
academic  integrity. 


Introduction 

Academic  dishonesty  is  a concern  for  many  instructors  whether  they  teach  in  high 
school,  two-year  or  four-year  institutions.  Academic  dishonesty  may  include  cheating  on 
examinations,  plagiarizing,  falsifying  sources  or  bibliographies,  knowingly  helping  other 
students  cheat,  working  together  on  projects  that  should  be  completed  independently,  or 
turning  in  the  same  assignment  for  more  that  one  course  (Dean,  2000).  With  the  advent 
of  the  Internet  and  the  World  Wide  Web,  there  seems  to  be  an  attitude  among  instructors 
that  academic  dishonesty  is  easier  because  of  the  availability  of  material  that  can  easily 
be  cut  and  pasted  (Renard,  2000).  Course  delivery  through  the  online  environment  may 
also  make  it  easier  for  students  to  cheat  since  students  and  instructors  do  not  have  the 
same  relationship  in  an  online  course  as  they  do  in  a face-to-face  course.  Investigating 
reasons  for  academic  dishonesty  in  face-to-face  and  online  environments  may  help  shed 
some  light  on  ways  to  develop  online  coursework  that  encourages  academic  integrity 
rather  than  leaving  students  to  their  own  devices  in  which  they  may  be  tempted  toward 
academic  dishonesty.  This  paper  will  investigate  the  following:  1)  reasons  for  academic 
dishonesty;  2)  academic  dishonesty  in  the  online  environment;  3)  possible  ways  to  design 
online  courses  so  that  academic  integrity  is  encouraged  and  academic  dishonesty  is 
discouraged. 
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Reasons  for  academic  dishonesty 


Measuring  the  incidence  of  academic  dishonesty  is  usually  done  through  self- 
report  surveys  given  to  students.  McCabe  and  Trevino  (1996)  reported  that  from  a 
sample  of  1 ,800  students  at  nine  different  state  sponsored  universities,  seventy  percent  of 
students  surveyed  admitted  to  cheating  on  exams.  Additionally,  it  was  reported  that 
eighty-four  percent  admitted  to  cheating  on  exams  and  almost  fifty  percent  admitted  to 
working  with  others  on  assignments  intended  to  be  independent  (McCabe  & Trevino, 
1996). 

Several  factors  seem  to  be  associated  with  the  incidence  of  academic  dishonesty. 
Dean  (2000)  has  identified  four  of  these  as  patterns  seen  in  the  literature;  individual 
characteristics,  peer  group  influences,  instructor  influences,  and  institutional  policies. 

Individual  characteristics  include  ideas  such  as  age,  gender,  social  activities  and 
level  of  academic  achievement  (Crown  & Spiller,  1998;  McCabe  & Trevino,  1997).  Peer 
group  influences  indicate  that  general  student  disapproval  of  cheating  is  most  likely  to 
discourage  it  while  peer  group  acceptance  of  cheating  is  likely  to  encourage  it  (Crown  & 
Spiller,  1998).  Students  who  take  courses  with  instructors  who  are  perceived  as  being 
actively  involved  and  concerned  about  students  are  less  likely  to  be  involved  in  academic 
dishonesty  (Crown  & Spiller,  1998).  Institutional  policies  that  are  communicated  clearly 
to  students  along  with  the  penalties  for  academic  dishonesty  are  likely  to  reduce  the 
occurrence  of  academic  dishonesty  (Crown  & Spiller,  1998;  McCabe  & Trevino,  1996). 

Considering  these  factors  in  regular  academic  classrooms,  it  would  seem  that  the 
same  kinds  of  academic  dishonesty  would  occur  in  the  online  environment  for  the  same 
reasons.  Additionally,  it  may  be  more  tempting  or  easier  to  engage  in  academic 
dishonesty  in  an  online  course  than  in  one  that  is  face  to  face.  Renard  (2000)  states  that 
although  plagiarism  is  not  new,  Internet  “cheat  sites”  have  made  cutting  and  pasting 
written  sections  or  even  entire  papers  easier.  Students  involved  in  an  online  course  would 
be  using  the  Internet  consistently  so  the  ease  of  using  these  sites  could  be  tempting. 
Additionally,  the  perception  of  the  instructor  being  less  involved  in  an  online  course 
could  encourage  academic  dishonesty  since  that  is  one  of  the  factors  that  can  influence  a 
student’s  decision  to  cheat. 

Academic  dishonesty  in  the  online  environment 

In  recent  years  the  popularity  of  the  Internet  has  continued  to  grow  among  the 
general  population  around  the  world.  In  the  United  States  alone,  what  was  once 
considered  a communication  novelty  is  rapidly  being  perceived  as  an  educational 
necessity  in  many  homes.  Along  with  the  increased  popularity  of  the  Internet,  especially 
the  World  Wide  Web,  many  institutions  of  higher  education  are  rushing  to  convert  an 
increasing  number  of  courses  to  online  delivery.  With  the  rush  to  offer  courses  online, 
there  has  arisen  questions  as  to  the  reasons  for  both  higher  education  institutions  and  their 
students  to  desire  this  type  of  course  delivery.  Many  of  these  questions  devolve  down  to 
a question  of  the  quality  versus  quantity. 

There  are  varied  opinions  about  the  correct  answers  to  the  questions  that  arise 
from  the  use  of  the  Internet  to  deliver  online  courses.  In  order  to  be  able  to  answer  these 
questions,  we  must  have  a basis  of  understanding  of  what  both  students  and  instructors 
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perceive  and  desire  about  the  use  of  online  course  delivery.  Many  of  these  perceptions 
are  nothing  more  than  myths  or  folklore  taken  from  past  popular  beliefs  and  science 
fiction.  Others  come  from  a misimderstanding  of  the  perceptions  held  from  the  various 
entities  (students,  instructors,  and  administrators)  associated  with  online  course  delivery 
(Moore  & Kearsley,  1996). 

Faculty  who  design  courses  to  be  delivered  online  may  evaluate  the  time  a student 
should  spend  in  a traditional  classroom  for  an  equivalent  course,  add  to  it  time  that  should 
be  spent  outside  of  class,  and  in  some  cases,  add  extra  requirements  due  to  the  student  not 
having  to  physically  drive  to  a campus-based  classroom.  Faculty  may  see  these  students 
as  being  able  to  connect  to  the  online  course  and  go  right  to  work  with  the  course 
materials.  Often  faculty  do  not  consider  that  the  physical  environment  in  which  the 
student  is  taking  the  course  may  not  be  the  optimal  environment  for  student  success.  As 
student  work  begins  to  be  turned  in,  possibly  at  a level  below  the  expected  standard,  the 
instructor  may  feel  that  the  rigor  of  the  course  is  being  challenged  and  therefore  try  to 
compensate  by  adding  additional  work.  On  the  other  hand,  if  the  work  submitted  by  the 
student  is  of  a higher  than  expected  standard,  the  faculty  member  may  suspect  that  the 
student  may  be  using  inappropriate  aids  in  completing  course  assignments. 

In  contrast,  students  may  see  themselves  as  being  somewhat  timid  with  the 
technology  needed  for  a particular  course.  Additionally,  these  students  may  not  see 
themselves  as  having  the  time  to  make  traditional  classroom  courses,  especially  since 
many  online  students  manipulate  their  course  schedule  aroimd  work  or  family  obligations 
(Gibson,  1999).  Some  students  feel  that  the  online  courses  may  be  easier  than  a campus 
delivery  due  a perception  about  having  to  only  interact  with  course  content,  not  with  the 
instructor.  Students  who  have  done  poorly  with  a particular  class  may  feel  that  taking  the 
professor  out  of  the  loop  will  in  some  way  help  them  pass  the  course.  In  this  case, 
students  who  may  not  have  the  strongest  work  ethic  may  be  enrolled  in  courses  requiring 
a stronger  work  ethic.  With  all  cases,  it  is  easy  for  the  student  to  become  overextended  in 
their  time  and  commitment  to  the  course.  This  can  lead  to  a situation  where  students  feel 
that  they  are  required  to  do  too  much  work  for  the  credit  earned  in  the  course. 

The  above  mentioned  scenarios  for  both  faculty  and  students  set  the  stage  for 
many  of  the  factors  that  can  lead  to  discomfort  and  the  temptation  for  academic 
dishonest.  Faculty  members  who  are  working  to  move  to  the  online  environment  may 
feel  detached  from  their  students,  feeling  that  they  really  do  not  have  the  kind  of 
relationship  that  they  would  have  if  the  same  students  were  in  a traditional  classroom.  In 
a traditional  classroom,  faculty  can  “look-in-the-eyes”  of  their  students  and  make  some 
determination  about  a students  feelings  and  needs.  Not  having  this  direct  contact  may 
lead  the  faculty  member  to  begin  to  be  suspicious  about  the  rigor  of  the  course  and  then 
ultimately  the  quality  and  originally  of  student  assignments. 

The  lack  of  direct  contact  and  a feeling  of  detachment  also  may  profoimdly  affect 
students.  Students  who  already  have  very  busy  schedules  may  be  compelled  to  take 
online  courses  due  to  the  belief  that  they  can  add  their  academic  work  on  top  of  an 
already  busy  lifestyle.  Once  the  demands  of  the  course  become  overwhelming,  especially 
in  cases  where  the  instructor  may  feel  that  the  rigor  of  the  course  is  being  challenged,  the 
student  may  feel  that  the  course  requirements  are  unreasonable  for  the  credit  to  be 
awarded.  Once  students  begin  to  feel  that  the  course  requirements  are  unreasonable,  the 
temptation  to  use  inappropriate  resources  to  complete  course  assignments  may  begin  to 
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grow.  The  temptation  can  become  even  stronger  when  students  develop  the  feeling  that 
they  are  all  alone  in  their  course  with  little  contact  or  interaction  with  the  instructor  of 
peers. 

Designing  online  courses  that  promote  academic  integrity 

The  scenario  and  concerns  mentioned  above  should  not  be  considered  a simple 
reality  of  online  course  deliver.  In  truth,  academic  integrity  is  something  that  all  faculty 
members  must  work  to  promote  in  any  instructional  environment.  However,  with  regard 
to  online  course  delivery,  there  are  design  features  that  can  specifically  promote 
academic  integrity  in  this  environment. 

First,  online  course  materials  should  clearly  state  that  academic  dishonesty  is  not 
acceptable  and  then  clear  examples  should  be  given  to  illustrate  the  kind  of  activities  that 
the  student  should  avoid.  Since  institutional  policies  are  seen  as  one  factor  that  has  an 
influence  on  a student’s  view  of  academic  dishonest,  it  is  important  to  clearly  outline  for 
the  student  what  the  policy  is  in  all  online  courses.  Further,  by  giving  specific  examples, 
faculty  can  help  students  avoid  misunderstandings  about  what  the  policy  actually  means. 
While  many  students  would  never  turn  in  someone  else’s  paper  as  their  own,  these  same 
students  may  be  tempted  to  copy  and  paste  sections  of  their  assignments  from  the 
Internet,  justifying  this  activity  as  research. 

Secondly,  online  course  materials  should  include  a high  degree  of  interaction. 
Interaction  may  be  developed  in  several  ways  and  should  work  to  increase  the  student’s 
contact  with  the  content,  the  instructor,  and  peers.  A variety  of  tools  can  be  used  to 
increase  interaction  such  as  chat,  bulletin  boards,  MOOs,  email,  etc.  Faculty  using  these 
tools  can  achieve  variety  in  their  courses  and  decrease  the  perception  by  students  that  the 
instructor  is  really  not  there  or  is  not  paying  attention  to  the  work  they  are  doing. 
Further,  an  instructor  can  use  these  tools  to  facilitate  cooperative  instructional  strategies 
where  students  work  in  peer  groups.  Using  cooperative  strategies  in  online  courses  help 
to  enculturate  the  student  into  the  course  by  developing  a support  structure  where 
students  can  become  knowledgeable  of  course  norms.  In  interactive,  well-supported 
cooperative  environments,  student  peers  help  to  encourage  academic  integrity  and 
adherence  to  course  policies.  Faculty  who  make  good  use  of  interaction  strategies  will 
reduce  the  temptation  of  the  student  to  seek  outside  help  and  increase  student  satisfaction 
with  the  course. 

Lastly,  online  courses  should  be  designed  to  include  a variety  of  evaluation 
methods.  With  a single  evaluation  method,  a multiple-choice  test  for  example,  a true 
picture  of  a student’s  understanding  is  difficult  to  achieve.  By  using  multiple  evaluation 
methods,  a trend  can  be  identified  with  regard  to  student  performance  and  understanding. 
This  is  equally  true  in  both  online  and  traditional  environments.  In  a traditional 
classroom  where  a student  may  be  very  quiet  during  the  semester  and  three  multiple 
choice  exams  and  a final  examination  are  used  for  the  course  grade,  it  can  be  difficult  to 
determine  if  the  student  understands  the  material,  simply  is  good  at  memorization,  or 
cheats.  In  courses  that  use  multiple  evaluation  methods,  the  trend  for  a student’s  work 
can  be  compared  to  individual  assignments.  If  a situation  occurs  where  one  paper  is 
turned  in  at  a higher  level  of  understanding  or  quality  than  has  been  seen  in  other  course 
interactions,  the  possibility  of  academic  dishonesty  may  be  present.  Likewise,  a student’s 


work  trend  can  be  used  to  guide  the  student  to  higher  levels  of  understanding.  As 
students  come  to  realize  that  their  work  is  being  viewed  and  evaluated,  and  that  feedback 
is  being  given,  they  will  feel  more  attached  to  the  course  environment,  and  academic 
integrity  will  be  promoted. 

Final  thoughts 

It  has  been  said  that  perception  often  becomes  reality.  In  the  context  of  online 
course  delivery,  a person’s  perception  about  this  type  of  delivery  can  directly  shape  the 
satisfaction  and  performance  in  the  course.  With  regard  to  academic  integrity,  there  are 
many  different  views  as  to  the  type  of  activities  that  students  should  or  should  not  be 
allowed  to  use  to  complete  their  course  assignments.  Likewise,  there  are  differences  of 
opinion  as  to  the  ease  in  which  students  could  chose  to  use  inappropriate  aids  to  complete 
course  assignments  online  as  compared  to  traditional  courses.  However,  this  paper  has 
attempted  to  outline  three  components  of  online  course  design  that  may  promote 
academic  integrity.  Courses  that  clearly  specify  the  institution’s  and  instructor’s  policy 
about  academic  dishonesty,  along  with  examples,  help  students  to  be  knowledgeable  and 
understand  the  need  for  academic  integrity.  Courses  that  have  a high  degree  of 
interaction  between  the  student  and  course  content,  the  instructor,  and  peers  can  promote 
a sense  of  being  connected  and  will  help  encultureate  students  into  acceptable  norms  for 
course  performance.  And  finally,  courses  that  use  a variety  of  evaluation  methods  can 
mark  trends  in  a student’s  performance  that  can  help  a faculty  member  assist  in  a 
student’s  growth  while  at  the  same  time  helping  to  identify  potential  dishonesty. 

There  are  no  easy  answers  to  the  many  questions  that  are  associated  with  online 
course  delivery.  It  is  important  to  understand  that  many  different  people  who  become 
involved  in  online  learning  do  so  for  many  different  reasons.  By  understanding  these 
reasons,  courses  can  be  designed  in  a manner  that  can  facilitate  a faculty  members 
satisfaction  as  well  as  the  success  students.  Through  the  process  students  will  have 
greater  opportunity  to  grow,  be  successful,  and  have  a high  degree  of  academic  integrity. 
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Abstract;  The  explosive  growth  in  IT  capabilities  and  extensive  use  of  IT  systems  has 
satisfied  the  increasing  desire  of  organizations  to  gain  competitive  advantage.  Advances  in 
information  technology  and  its  inherent  devices  have  led  to  profound  changes  in 
organizational  rules.  This  study  attempts  to  build  an  introduction  to  evaluate  IT  effects  on 
control  as  a functions  of  management.  Modem  organizations,  based  on  using  modem 
technology,  have  to  make  major  modifications  to  the  entire  organizations.  This  study  begins 
by  examining  concepts  of  control  in  organizations  following  a short  review  of  past  work  on 
IT  effects  on  concept  of  hierarchy.  Then  present  a comparison  of  conventional  control  and 
new  attitude  in  traditional  and  modem  organization. 


Introduction 

According  to  management  literature  one  of  the  most  important  and  constant  functions  of  management  is 
controlling.  Controlling  is  the  process  of  establishing  objectives  and  specifying  how  they  are  to  be 
accomplished  in  an  uncertain  future  (Cook  & Coffey,  1997).  Researches  have  indicated  that  in  the 
organizational  hierarchy,  each  level  of  management  tends  to  place  a different  emphasis  on  the  functions  of 
management.  However  controlling  is  a function  that  is  widely  applied  by  managers  at  all  levels  of  management 
(Rue  & Byars,  1997).  Unfortunately  formal  control  may  have  some  side  effects  (Schermerhom,  hunt,  & Osborn, 
1991)  such  as  imbalance,  lack  of  patience,  confusing  documentation  with  action  (concern  for  performance  may 
lead  to  a stack  of  impressive-sounding  plans  but  no  result),  panic  and  so  on.  These  reasons  have  forced 
managers  to  consider  other  control  mechanism  that  is  social  control.  The  purpose  of  social  controls  is  to  get 
people  to  commit  themselves  to  the  organization.  For  social  system  control  to  work,  people  need  to  know  that 
someone  in  authority  knows  what  they  are  doing  and  is  willing  to  call  attention  to  gaps  between  performance 
and  objectives.  Based  on  a survey  and  information  gathered  from  22  manufacturing  and  service  firms  in  the 
UK,  Li  (1997)  highlights  a radical  change  in  a basic  rule  of  organizational  design.  He  argues,  most  existing 
organizational  designs  are  based  on  a series  of  compromises  (for  example,  between  central  control  and 
empowerment).  Researchers  have  identified  two  categories  of  control  methods:  behaviour  or  personal  control 
that  is  based  on  direct,  personal  surveillance  ( Rue  & Byara,  1997).  First-line  supervisors  usually  apply  this  kind 
of  control.  Output  or  impersonal  control  is  based  on  the  measurement  of  output,  e.g.  monitoring  sales  figures. 
IT  advances  facilitate  and  improve  both  control  method. 


Control  and  hierarchy,  traditional  view  versus  modern  approach 

Miles  and  Snow  (1992)  identified  four  broad  forms  of  organization:!)  The  functional  organizational: 
this  encouraged  internal  production  of  parts  and  components  to  assure  control  The  logic  of  that  is  centrally 
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coordinated  specialization.  2)  The  divisional  form:  The  operating  logic  of  the  divisional  form  is  the  coupling  of 
divisional  autonomy  with  centrally  controlled  performance  evaluation  and  resource  allocation.  3)  The  Matrix 
Form:  The  operating  logic  of  the  matrix  form  is  assure  control  and  centrally  coordinated  specialization.  4) 
Network  organization:  the  combination  of  central  evaluation  and  local  operating  autonomy  is  reflected  in  the 
dynamic  network  where  independent  firms  are  linked  together  for  the  one-time  (or  short-time)  production  of  a 
particular  good  or  services.  From  their  viewpoint  underlying  all  of  the  positive  characteristics  of  network 
structure  is  the  dynamic  of  Voluntarism.  Applegate  (1994)  identified  two  major  differences  of  control  issue  in 
traditional  and  modem  company:  a)  In  traditional  company  the  role  of  control  was  standardizing  behaviour  and 
supervisory  control  of  people,  but  now  in  modem  firm,  the  role  of  control  is  to  sharing  information,  which 
enabled  control  of  the  operating  processes  directly,  b)  In  traditional  hierarchical  organizations,  managers 
automatically  inherited  responsibility  and  accountability  for  decisions  made  by  subordinates,  but  in  modem 
organization  all  the  employees  are  responsible  for  decision  making. 


Summary  and  conclusion 

Over  the  past  decade  a great  number  of  papers  have  been  written  about  organisations  in  advanced  IT  age.  One 
major  characteristics  of  IT  is  separating  work  from  location  (Groth,  1999),  IT  devices  such  as  email,  the  Internet,  and  etc 
permit  tight  cuuruination  of  geographically  dispersed  workers  across  time  zones  and  cultures.  Much  attention  must  be 
placed  on  trust  and  coordination  mechanism  rather  than  control  mechanisms.  Here  we  distinguish  some  reasons:  l)As 
organisations  perform  most  of  their  processing  using  electronic  forms  and  images,  workflow  systems  will  be  used  to  route 
documents  electronically  to  individuals  and  work  groups  that  need  access  to  them.  2)  Number  of  employees  has  reduced  so 
it  is  much  easier  to  control  them  by  informal  ways.  3)The  other  reason  for  trust  arises  from  this  fact  that  the  nature  of  work 
and  naturally  employees’  expertise  have  changed.  In  the  IT  age,  an  individual  who  contributes  value  by  adding  or 
interpreting  information  is  called  a knowledge  worker.  4)The  struct  ure  and  functioning  of  the  organization  will  no  longer  be 
determined  only  by  living  patterns.  Programmed  patterns,  will  increasingly  influence  the  structure  and  functioning  of  the 
organization  (Groth,  1999),  5)  Outsourcing:  Successful  organisations  in  2P*  firstly  must  identified  new  imperatives  and  then 
build  a proper  organisational  structure  to  cope  with  them.  One  of  these  imperatives  is  outsourcing. 
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Abstract 

The  jagged  study  zone  model  helps  designers  to  create  educational  environments  which  challenge  the  learner  and 
make  the  learner  to  commit  to  the  learning  process.  The  characteristics  of  a jagged  study  zone  can  be  explained  by 
showing  how  its  features  are  related  to  traditional  closed  and  open  learning  environments.  The  requirements  for  a 
jagged  study  zone  are  identified  by  extracting  the  strengths  and  opportunities  of  the  closed  and  open  learning 
environments,  and  by  combining  them  to  the  opposites  of  weaknesses  and  threats  of  those  traditional  environments. 
We  show  examples  how  the  jagged  study  zones  can  be  supported  and  discuss  the  role  of  the  teacher  when  considering 
the  jagged  study  zone  model. 

1.  Introduction 

In  the  recent  discussion  concerning  meaningful  technology  for  learning,  the  concept  of  “learning  landscape”  or  open 
learning  environment  seems  to  refer  to  an  environment  that  provides  a learner  with  an  activating,  yet  safe  learning 
experience.  The  learning  itself  has  become  slightly  dull  and  unexciting.  From  the  arctic  Finnish  perspective, 
generations  before  us  have  considered  learning  not  an  easy  or  simple  fiin-like  activity,  but  rather  a painful  experience 
that  changes  the  learner  at  a fundamental  level.  Instead  of  referring  to  a tamed,  village -like  landscape  with  occasional 
woods  and  meadows  for  lovely  picnics,  learning  can  also  be  a process  of  blood,  sweat,  and  tears.  In  our  experience 
these  dull  learning  landscapes  rarely  offer  enough  excitement  and  realistic  challenges  for  every  student.  We  do  not 
argue  that  learning  or  education  should  always  be  extremely  exciting,  but  on  some  occasions  the  appropriate  amount 
of  exhilaration  might  motivate  the  student  to  cope  with  very  difficult  domains. 

At  the  same  time  the  learning  landscapes  or  open  learning  environments  do  not  necessary  offer  enough  guidance  and 
counseling  to  the  poorly  motivated  student.  If  the  student  is  not  capable  of  making  the  right  choices,  the  learning 
environment  most  likely  will  not  help  her.  Hence,  the  jagged  study  zone  should  take  also  care  of  a poor  student; 
facilitating  her  with  efficient  learning  products  and  support. 

In  this  paper  we  will  propose  a model  for  building  educational  environments  by  introducing  the  concept  of  jagged 
study  zones.  Applying  a jagged  study  zone  one  can  cover  most  of  the  demands  listed  above. 

2.  Deflnition  of  the  Jagged  Study  Zone 

We  begin  our  definition  of  a jagged  study  zone  by  examining  the  properties  of  two  well-know  learning  contexts:  a 
closed  learning  environment  and  an  open  learning  environment.  In  a closed  learning  environment  the  teacher  sets  the 
goals  and  dictates  for  the  tools  used,  pace,  materials,  schedule  and  the  learning  goals.  Wilson  (1995)  characterizes  the 
closed  environment  to  be  a construct,  in  which  the  students  work  by  using  facilities  to  collect  and  interpret 
information.  The  closed  environment  is  restricted  and  rigid;  the  students  are  not  able  to  work  or  express  themselves 
freely.  The  metaphor  for  closed  learning  environment  could  be  a product  line  of  a factory.  Student  is  a product 
traveling  through  the  product  line  and  the  factory  (educator)  provides  him  with  all  the  necessary  information  and 
knowledge. 

To  get  free  from  the  limitations  of  the  closed  learning  environment  Miller  (1997)  and  Levi  (1994)  proposed  an  open 
learning  environment,  which  is  characterized  by 

1 being  learner  centered, 

2 rich  communication, 

3 exploratory  learning, 
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4 the  use  of  learning  goals,  and 

5 possibilities  to  make  decisions  individually  and  in  groups  as  they  encounter  problems. 

Learning  is  directed  and  scaffolded,  but  not  controlled  or  enforced  to  a certain  model  (Wilson  1995).  The  advocates  of 
open  learning  environments  claim  that  the  motivation  for  learning  comes  from  the  learner  herself. 

We  can  combine  the  strong  points  of  the  closed  and  open  learning  environments  into  a novel  concept.  We  begin  by 
making  a SWOT  analysis  of  the  closed  and  open  learning  environments  (see  tables  1 and  2)  and  combining  the 
strengths  and  opportunities  identified  with  the  analysis.  Then  we  will  reinforce  our  concept  by  incorporating  the 
opposites  of  the  weaknesses  and  threats  found  in  both  approaches  by  introducing  supporting  facilities. 


Strengths 

Weaknesses 

+ goal  awareness 
+ skill  learning 
+ efficient  in  closed  tasks 

- does  not  support  initiative 

- suppresses  creativity 

- narrow  applicability 

Opportunities 

Threats 

+ learning  by  gaming 
+ utilizes  external  motivation 

- trains  learners  into  robots 

- promotes  tunnel  vision 

Table  1:  SWOT-analysis  of  a closed  learning  environment  concept. 


Strengths 

Weaknesses 

creative  problem  solving 

- hard  to  teach  (lack  of  teacher  control) 

-1-  own  choices 

- can  be  boring 

4-  variety  of  tools 

- lack  of  plot  or  narrative  organization 

-H  individual  paths 

Opportunities 

Threats 

+ develops  autonomy 

- traps,  deadlocks 

+ coaching  - tutoring  guidance 

- dependent  variables 

+ challenging  for  a motivated  student 

- learning  topologies 

+ internal  motivation 

Table  2:  SWOT-analysis  of  an  open  learning  environment  concept. 

Learning  environments  based  on  adaptive  hypermedia  (Brusilovsky  2001)  are  good  examples  when  closed  learning 
environments  have  been  transformed  to  be  more  flexible.  Despite  the  introduction  of  adaptation,  systems  based  on 
adaptive  hypermedia  are  still  confined  environments,  where  system  designers  or  learning  material  authors  have  to 
decide  the  rules  by  which  educational  materials  are  presented  to  the  learner. 

The  open  market  model  (Meisalo,  Sutinen  & Tarhio,  2000,  69-7 1 ; Meisalo  & Lavonen,  2000)  is  a typical  construct 
based  on  the  open  learning  environment  paradigm.  This  model  structures  the  open  environment  and  the  activities  of 
the  learner  therein.  In  the  original  market  model,  the  student  is  considered  to  be  a consumer  in  a large  supermarket 
(computer).  In  the  supermarket  the  customer  makes  highly  individual  choices  that  are  relevant  only  for  her.  In  the 
supermarket  you  can  forget  all  the  other  customers  or  people,  and  the  only  important  thing  is  to  think  about  your  own 
welfare  and  choices.  In  the  open  market  model,  referred  above,  both  positive  interaction  of  learners  in  the  environment 
and  the  access  to  the  world  outside  the  supermarket,  even  wild  nature,  electronic  workshops,  etc.,  are  emphasized.  In 
the  original  field  of  application  the  interest  has  been  primarily  in  science  and  technology  education.  Both  the  nature 
and  the  technological  applications  show  often  features  that  are  far  from  smooth  and  ideal.  This  demonstrates  that  the 
idea  of  jagged  study  zones  brings  about  new  features  to  learning  tools,  like  in  our  example  of  extending  the  original 
market  model  into  the  open  market  model. 

The  jagged  study  zone  concept 

We  see  Vygotsky’s  (1978)  zone  of  proximal  development  as  a component  for  jagged  study  zone,  in  which  the  student 
is  encouraged  to  take  the  learning  experience  to  the  outer  limits.  The  jagged  study  zone  can  be  a harsh  environment 
for  the  student,  and  demands  serious  efforts  to  explore  and  reach  its  limits.  The  characteristics  of  the  jagged  study 
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zone  are  derived  from  the  closed  and  open  learning  environments.  We  identify  four  main  requirements  for  a jagged 
study  zone  by  combining  the  strengths  and  weaknesses  of  the  closed  and  open  learning  environments  and  by 
countering  their  weaknesses  and  threats. 

Features  derived  from  closed  learning  environments: 

The  goal  awareness  and  skill  learning  identified  as  strengths  of  a closed  learning  environment  and  the  corresponding 
opportunities  of  learning  by  gaming  and  utilization  external  motivation  translate  into  collaborative  tools  for  achieving 
meaningful  goals  in  the  jagged  study  zone  (first  requirement).  The  tools  help  students  to  understand  their  goals  and  to 
scaffold  them  in  reaching  the  goals.  External  motivation  and  the  competitive  gaming  element  can  be  attained  by 
introducing  a collaborative  element  into  the  learning  process,  where  the  students  help  each  other  and  may  get  inspired 
by  each  others’  progress  and  skills. 

The  closed  learning  environment  tends  not  to  support  initiative  and  it  suppresses  creativity.  From  this  emerges  the 
threat  of  training  students  into  tunnel-visioned  robots  without  inquiring  minds.  To  counter  these  weaknesses  and 
threats  the  jagged  study  zone  promotes  the  students  to  learn  creative  problem  solving  and  versatile  meta-skills  for 
survival  such  as  self-evaluation  in  a harsh  learning  environment.  This  can  be  conducted  by  allowing  the  students  the 
opportunities  to  solve  fuzzy  problems  and  to  deal  with  uncertainty  in  the  jagged  study  zone  (second  requirement). 

Features  derived  from  open  learning  environments: 

When  combining  the  best  features  of  open  learning  environments,  we  can  extract  the  third  requirement  for  jagged 
study  zones.  The  open  learning  environment  promotes  creative  problem  solving  by  letting  the  students  to  make  their 
own  choices  and  select  their  own  paths.  In  jagged  study  zones  the  student  must  take  challenging  actions  by  self- 
initiated  and  self- regulated  construction  during  the  learning  process. 

The  weakness  of  the  open  learning  environment  is  that  it  often  leaves  the  student  alone  with  his  choices.  There  is 
plenty  of  space  to  go,  but  there  is  no  guidance  for  a student  who  gets  lost.  To  counter  this  threat  we  need  adaptive 
methods  for  getting  out  of  deadlocks  and  traps  (fourth  requirement).  If  we  do  not  want  to  rely  on  automated  methods, 
it  is  always  possible  to  use  peer-help  in  identifying  the  traps.  This  can  be  achieved  for  example  by  social  navigation 
(Munro  1999),  i.e.  making  the  actions  of  every  individual  visible  to  other  individuals. 

Summary  of  requirements  for  a jagged  study  zone: 

1 . In  the  jagged  study  zone  there  are  collaborative  tools  for  achieving  meaningful  goals. 

2.  The  jagged  study  zone  equips  the  students  with  an  opportunity  to  solve  fuzzy  problems  and  deal  with  uncertainty 
and  unexpected  phenomena. 

3.  During  the  learning  process  the  student  must  take  challenging  actions  by  self-initiated  and  self-regulated 
construction. 

4.  Adaptive  methods  for  getting  out  of  deadlocks  and  traps,  or  peer-help  based  on  for  example  social  navigation. 
Example  of  a jagged  study  zone  as  a landscape 

The  jagged  study  zone  concept  can  be  illustrated  by  a jagged  desert  landscape  where  one  can  find  for  example 
following  things; 

Oasis: 

An  oasis  is  a place  where  learner  can  stop  at  to  find  inspiration,  guidance  and  even  amusement. 

In  the  oasis  the  student  can  find  refugee  if  she  gets  lost  during  her  explorations  in  the  jagged  landscape.  There  can  be 
attractions  and  guides  at  the  Oasis  to  motivate  student.  The  Oasis  can  for  example  include  a hut  with  a mirror,  which 
the  student  can  use  as  a tool  for  self-evaluation. 

Inspiration  points: 

Inspiration  points  provide  the  student  with  fresh  stimuli,  for  example  as  distance  thought  models  operate  in  idea 
generation.  These  can  appear  to  the  student  as  a mirage  in  the  desert.  Inspiration  points  may  include  ideation  tools  and 
they  can  may  be  created  and  inserted  by  a teacher. 

Workshops: 
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The  workshop  is  a place  where  the  student  can  find  tools  that  help  the  student  in  exploration.  The  tool  may  include 
visualization,  writing  applications  and  collaborative  devices.  For  example,  learning  seeds  (Kurhila  and  Sutinen  1999) 
can  be  processed  in  a workshop. 

Learning  seeds: 

In  the  landscape,  the  learner  can  find  learning  seeds,  which  are  planted  by  the  teacher  or  fellow  learners.  These  can  be 
parts  of  learning  material,  exercises,  ideas  and  visual  presentations.  The  seeds  can  “grow”  from  the  actions  other 
learners  make,  i.e.  the  learners  work  on  a written  assignment  by  commenting  each  other’s  input.  The  seeds  can  also 
grow  by  some  other  function,  for  example  by  some  pre-defined  rules.  This  is  particularly  suitable  if  the  seeds  are  a part 
of  an  artificial  world  consisting  simulated  events  in  time  and  space.  It  is  possible  that  a prosperous,  well-surviving  and 
well-nourished  learning  seed  can  grow  to  be  an  inspiration  point  in  the  landscape. 

Viewpoints: 

The  learner  can  use  the  viewpoint  to  gaze  over  the  landscape.  In  this  way  the  learner  can  review  his  journey  in  the 
landscape.  It  is  also  possible  that  a learner  can  view  his  journey  in  respect  to  other  learners’  journeys,  incorporating 
social  dimension  also  to  the  evaluation  of  the  learning  process.  In  addition,  the  more  traditional  way  to  exploit  the 
learning  history  is  that  the  teacher  can  use  the  viewpoints  to  track  and  possibly  intervene  (help,  guide,  advise)  learners 
as  they  explore  the  landscape. 

3.  Examples  of  supporting  jagged  study  zones 

Context  1:  Virtual  platforms.  In  the  current  course  delivery^  systems,  such  as  WebCT  or  Learning  Space,  the  jagged 
study  zone  idea  is  not  supported.  The  systems  offer  rigid,  often  similar,  solutions  to  the  design  of  learning  situations. 
According  to  the  jagged  study  zone  concept,  the  virtual  platforms  should  offer  ways  to  implement  stimulating  learning 
situations.  At  the  current  time  the  construction  of  stimulating  features  to  learning  landscapes  remains  almost  entirely 
on  the  hands  of  the  teacher. 

Context  2:  Computer  Science  teaching.  In  the  Virtual  Approbatur  project  at  the  Department  of  Computer  Science, 
University  of  Joensuu  in  Finland,  the  design  and  implementation  of  virtual  courses  is  based  on  the  analysis  of 
students’  needs  according  to  the  Candle  scheme  (Haataja  et  al.  2001).  On  the  other  hand,  the  Candle  scheme  is 
derived  from  the  concept  of  jagged  study  zones.  For  example  the  visual  tools,  like  algorithm  animator  Jeliot  (Haajanen 
et  al.  1997),  offer  a student  a way  to  actively  strengthen  the  understanding  of  programming  concepts.  Visual  tools  give 
students  the  aid  for  grasping  the  theoretical  aspects  of  programming.  Furthermore,  one  of  the  ways  to  realize  the 
jagged  study  zone  concept  would  be  to  use  students’  unfinished  learning  materials  and  blurred  learning  goals  (tasks). 
In  fact,  the  starting  point  might  be  an  entirely  incorrect  version  of  an  algorithm.  The  following  trouble-shooting 
activity  by  Jeliot  is  an  utmost  characteristic  of  a jagged  study  zone.  Thus,  Jeliot  inspires  the  students  to  work  with  the 
material  deeply  to  make  it  more  suitable  for  achieving  the  learning  goals. 

The  Empirica  Control  environment  gives  a student  tools  to  study  a phenomenon  that  takes  place  in  his  physical 
neighborhood.  Thus,  the  observed  circumstances  are  far  from  ideal  ones  describing  abstract  models.  This  noise  - 
representing  the  jagged  feature  of  Empirica  - supports  him  towards  his  own  efforts  in  the  learning  process. 

Context  3:  Adaptive  learning  environments.  Adaptive  learning  environments  have  evolved  from  intelligent  tutoring 
systems  (ITS)  towards  more  comprehensive  hypermedia-based  environments  with  adaptive  presentation  and  adaptive 
navigation  support  (Brusilovsky  2000).  Most  of  the  intelligent  tutoring  systems  do  not  provide  the  learner  with 
learning  materials:  the  knowledge  required  is  acquired  outside  the  system,  and  the  ITS  provides  only  a platform  for 
helping  the  learner  in  the  process  of  problem  solving.  Today  the  trend  is  to  offer  an  environment  where  the  learning 
material  is  browsed  in  an  adaptive  manner.  It  should  be  noted  that  in  reality  only  a few  systems  can  offer  this,  since 
providing  adaptation  is  never  a straightforward  task  to  design  and  implement. 

When  fitting  the  jagged  study  zone  model  to  a hypothetical  adaptive  learning  environment,  there  is  a need  to  think  the 
concept  of  adaptive  learning  environments  more  broadly.  For  example,  collaboration  can  be  achieved  when  users  are 
not  modeled  as  individuals  but  as  groups.  This  poses  a noted  problem  (Beck  et  al.  1996),  but  one  solution  is  to  use 
probabilistic  models  to  form  or  just  suggest  groups  of  individuals,  where  the  group  has  sufficient  potential  to  tackle  the 
learning  tasks  (requirement  1 of  the  jagges  study  zone  model).  Requirement  2 of  the  jagged  study  zone  can  be 
supported  by  standard  adaptive  navigation  support  if  the  learning  environment  is  based  on  hyperlinked  material,  or 
with  adaptive  scaffolding  which  can  mean  intelligent  and  interactive  problem  solving  support.  Requirement  3 can  be 
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supported  with  unpredictable  components  generated  by  random  algorithm  for  inspiration  points  thus  bringing  up  the 
motivation. 

To  provide  any  kind  of  individual  adaptation  requires  the  learner  to  be  modeled  in  some  way.  The  model  is  always  an 
approximation  of  reality  and  thus  imperfect.  Therefore,  an  adaptive  learning  environment  can  easily  be  a “sweatshop” 
if  the  environment  is  too  rigid  and  the  modeling  is  conducted  inappropriately.  However,  slightly  imperfect  learner 
modeling  is  in  complete  harmony  with  the  fourth  requirement  listed  for  the  jagged  study  zone  model.  If  there  is  a need 
to  shift  the  emphasis  from  modeling  the  learner  artificially  to  human  (peer)  evaluation,  we  can  incorporate  the  idea  of 
social  navigation  to  the  environment  to  identify  when  learners  or  groups  of  learners  are  facing  traps  and  deadlocks. 

Context  4:  Woven  Stories.  The  concept  of  woven  stories  can  be  briefly  described  as  several  authors  writing  a story  in 
a shared  virtual  space  (Harviainen  et  al  1999,  Gerdt  et  al,  2001).  The  difference  between  the  woven  stories  idea  and 
collaborative  writing  becomes  evident  in  their  functions:  the  writers  of  a woven  story  engage  in  a dialogue  of 
storytelling  rather  than  concentrating  on  an  effort  to  produce  a joint  document.  The  shared  virtual  space,  where  the 
authors  write  the  woven  story  is  a place  where  the  writers  may  be  inspired  by  the  writings  of  the  others,  take  part  in 
writing  a twist  into  an  existing  storyline,  compare  one’s  writings  to  other  contributions  or  simply  write  a story  for 
others  to  read. 

A system  realizing  the  concept  of  woven  stories  can  be  used  as  a collaborative  learning  environment.  The  concept  of 
woven  stories  fits  well  with  the  jagged  study  zone  model.  The  shared  functions  of  the  virtual  space  offer  tools  to 
authors  or  learners,  with  which  they  can  produce  and  link  texts  to  achieve  goals  which  are  of  communal  or  individual 
importance  (requirement  1).  Writing  and  re-writing  text  is  a good  way  to  reiterate  an  original  problem  definition  and 
to  elaborate  better  and  better  solutions  via  partial  or  preliminary  ones  (requirement  2).  A system  for  woven  stories  can 
include  mechanisms  monitoring  the  students  to  notice  any  lost  students.  These  mechanisms,  for  example  periodical 
reports  and  alarm  triggers,  can  be  used  to  inform  teachers  and  tutor  students  that  intervention  is  needed  (requirement 
4).  This  information  can  be  relayed  to  the  lost  student  too,  so  that  she  realizes  her  situation  and  may  benefit  from  the 
scaffolding  facilities  of  the  environment. 

The  authors  of  a woven  story  may  link  their  story  passages  to  other  authors’  passages,  thus  including  their 
contribution  into  a story  that  is  being  written  by  other  authors.  From  an  another  point  of  view  they  may  include  a piece 
of  somebody  else’s  story  into  one’s  own  story.  When  the  students  link  their  stories  to  others’  stories  and  participate  in 
the  creation  of  a storyline  they  must  make  challenging  choices  (requirement  3).  The  students  must  relate  their 
creations  to  writings  of  others,  maybe  adapting  their  contributions  somewhat. 


4.  The  roles  of  students  vs.  teachers  in  learning  environments  w^ith  jagged  study  zones 

It  is  important  that  there  is  a balance  between  the  student  autonomy  and  the  teacher  guidance  in  jagged  study  zones.  In 
principle,  there  is  always  a responsibility  for  a teacher  to  make  sure  that  the  students  are  working  towards  the  goals  set 
for  the  course  or  equivalent.  On  the  other  hand,  student  autonomy,  especially  the  decision  making  skills  and  the 
readiness  to  take  responsibility  do  not  develop  if  they  do  not  have  situations  where  to  get  relevant  experience.  These 
aspects  can  be  readily  observed  in  certain  types  of  computer  games,  but  in  jagged  study  zones  they  appear  in  more 
realistic  ways. 

In  a modem  learning  environment  it  is  important,  that  the  learner  is  able  to  work  creatively  and  join  various  teams  for 
brainstorming  and  other  forms  of  ideation  processes.  It  is  also  important,  that  the  learner  is  able  to  co-operate  with  a 
number  of  specialists,  one  of  them  the  teacher,  but  some  being  fellow  students  possibly  reached  over  the  Web  etc.  Even 
motivating  and  guiding  feedback  is  available  in  natural  forms  in  workshops  of  jagged  study  zones. 

The  teacher’s  role  in  a jagged  study  zone  is  that  of  a wilderness  guide,  a ranger  who  is  aware  of  potential  threats, 
risks,  opportunities,  and  challenges  meeting  all  explorers  in  the  unknown.  Students  can  be  seen  as  scouts,  at  a 
proximal  zone  of  new  findings,  reorganizing,  and  changing  earlier  knowledge  structures.  While  a single  hunter  and 
trapper  may  sometimes  have  reasonably  good  life,  high  mountain  tops  or  most  extreme  wilderness  can  be  reached  only 
by  teams.  Positive  group  dynamics  within  the  trekking  team  is  essential  - collaborative  learning  of  higher  order 
(thinking)  skills  is  dependent  truly  on  the  success/survival  of  every  team  member.  The  team  makes  use  of 
heterogeneous  and  versatile  skills  of  each  one.  This  metaphor  relates  also  directly  to  research  work,  but  any  learner  is 
equally  exploring  her  environment  for  knowledge  and  skills  that  are  new  to  her. 


5.  Concluding  remarks 


Today,  in  our  society  indifferent  attitudes  are  common.  In  many  occasions  the  attitudes  towards  learning  and 
education  are  similarly  indifferent.  The  student  will  not  succeed  with  this  type  of  unconcerned  attitude  in  jagged  study 
zones.  The  students  have  to  make  a stand,  form  an  opinion,  to  act,  to  really  stick  their  hands  on  the  mud. 

It  is  possible  to  identify  jagged  traces  in  the  existing  learning  environments,  as  the  examples  taken  from  Empirica, 
Jeliot,  and  Woven  Stories  indicate.  However,  to  bring  the  idea  of  jagged  study  zones  not  just  a feature  of  a computer- 
supported  learning  process,  but  an  all-embracing  principle  thereof  needs  determined  action.  This  will  be  our  agenda  in 
both  designing  novel  educational  environments  as  well  as  evaluating  their  power  and  usefulness. 
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Qualitative  and  quantitative  methods  are  often  used  for  understanding  interactions  in 
asynchronous  learning  spaces.  Traditionally  researchers  in  these  fields  have  not 
collaborated  Research  on  asynchronous  discussions  provides  an  ideal  arena  for 
collaboration  because  of  the  large  volume  of  data  available  and  the  relatively  under- 
theorised nature  of  the  field.  The  presenters  are  examining  communication  and  decision 
making  in  computer  mediated  teamwork.  They  seek  to  bridge  the  qualitative/quantitative 
divide  by  using  both  approaches  to  better  understand  what  is  happening. 


The  purpose  of  this  paper  is  to  describe  a research  project/design,  which  investigates  decision  making  modes  in 
an  asynchronous  computer-mediated  communication  (CMC)  environment.  The  multidisciplinary  research  team 
(representing  the  fields  of  artificial  intelligence,  international  management  and  education)  will  employ  a design 
which  has  iterative  quantitative  and  qualitative  elements. 

Research  Questions  and  Motivations 

We  have  three  primary  research  questions: 

1.  What  are  the  dominant  decision  making  modes  used  by  teams  in  the  computer -mediated  environment? 

2.  What  is  the  mix  of  process,  social  and  task  communication  used  in  computer  -mediated  teamwork? 

3.  How  can  quantitative  and  qualitative  approaches  be  combined  to  investigate  these  phenomena? 

There  are  three  motivations  for  this  study:  1)  To  extend  past  findings  from  three  predominant  models  of  decision 
making  (rational,  constrained  and  emergent)  to  the  CMC  context,  2)  to  integrate  qualitative  and  quantitative 
findings  in  an  iterative  process  to  better  understand  CMC  decision  making,  and  3)  to  attempt  to  develop  a 
software  tool  for  practitioners  that  allows  simplified  analysis  and  depth  of  meaning. 

The  Conceptual  Framework 

Asynchronous  Computer  Mediated  Communication:  The  literature  in  CMC  positively  characterises  the 
environment  as  one  of  time  and  space  flexibility,  many  to  many  communication,  computer  mediated  interaction 
and  text  based  nature  (Harasim,  et  al.  1996).  Contributions  have  a high  degree  of  plasticity  in  the  sense  that  they 
can  be  reshaped  (Hammond,  1998).  Garrison  et  al  (1998)  theorise  that  there  could  be  a connection  between  this 
text  based  communication  and  higher  order  thinking.  There  is  also  the  potential  for  equal  opportunity  for 
participation  (Eastmond,  1994).  The  literature  has  also  negatively  characterised  the  CMC  environment  as 
including  dependence  upon  technology  which  may  not  always  work  (Hara  and  Kling,  1999),  the  absence  of 
physical  presence  which  can  result  in  message  ambiguity  and  lack  of  clarity  (Eastmond,  1994),  and  perceptions 
of  the  medium  as  an  act  of  publishing,  rather  than  speech,  which  may  result  in  communication  anxiety  (Grint, 
1989).  These  features  represents  a significantly  different  context  for  research  on  decision  modes. 

Group  Decision  Making:  Over  the  years,  the  study  of  decision  making  has  focused  on  two  primary  questions: 
‘how  do  decisions  get  made?’  and  What  is  the  most  effective  approach  to  making  decisions?’.  This  research 
focuses  on  the  first  question  of  decision  making  modes  rather  than  performance.  There  are  at  least  three  modes  of 
decision  making  that  can  be  taken  from  the  literature:  the  rational  (Gulick  & Urwirk,  1937),  constrained  (Simon, 
1960),  and  emergent  modes  (Allison,  1971).  To  identify  which  mode  is  used  in  any  particular  group  exchange 
requires  careful  analysis  of  the  group’s  dialogue.  Dialogue  in  face-to-face  environments  generally  contain  three 
types  of  statements:  task,  process,  and  social.  These  will  be  operationalised  for  our  study  in  the  methodology 
section. 
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A group  of  senior-level  university  students  worked  in  dyads  to  analyse  and  make  decisions  in  a business  case 
solution.  In  this  assessed  assignment,  the  groups  were  instructed  to  conduct  all  their  interaction  through 
electronic  media  which  could  include  e-mails,  electronic  chats  or  electronic  discussion  forums.  These  CMC 
media  provide  a rich  and  complete  transcript  of  the  participants’  interaction  and  these  transcripts  will  serve  as  the 
basis  of  our  analysis.  The  participants  in  this  study  have  some  experience  in  decision  making  and  less 
experience  in  working  in  electronic  environments. 

In  our  qualitative  inquiry,  content  analysis  will  be  used  to  analyse  the  CMC  transcript  data  (Henri,  1991)  using  a 
‘gestalt’  approach  (Gunawardena  et  al.  1997).  Statements  will  be  identified  as  task,  process  and  social  and 
decisions  will  be  identified  as  rational,  resource  constained,  or  emergent.  The  resultant  description  will  be 
merged  with  further  quantitative  analysis  and  then  used  as  part  of  the  machine  learning  cycle  of  the  project.  In 
our  quantative  inquiry,  we  will  use  Knowledge  Discovery  fom  Databases  (KDD)  as  a means  of  automatic 
quantitative  investigation.  KDD  includes  a broad  number  of  methods  for  extracting  knowledge  from  data.  In  this 
case  we  are  interested  in  attempting  to  discover  ways  of  automatically  classifying  discussions,  and  identifying 
key  points  in  the  discussion. 

We  will  employ  an  iterative  process  of  analysis.  The  quantitative  analysis  relies  heavily  on  the  coding  scheme 
devised  for  the  qualitative  analysis.  Initially,  the  quantitative  analysis  attempts  to  identify  suitable  parameters  for 
classification.  The  results  of  the  initial  qualitative  analysis  are  then  applied  in  the  development  of  the  quantitative 
analysis.  Similarly,  novel  relationships  or  associations  revealed  by  the  quantitative  analysis  can  inform  further 
iterations  of  the  qualitative  approach  by  indicating  possible  classifications  or  associations  between  statements 
diid  decisions. 


Anticipated  Results 

We  anticipate  findings  relating  to  decision  making  modes  and  the  mix  of  communication.  With  regard  to  the 
decision  making  modes,  we  anticipate  finding  evidence  of  all  three  in  the  asynchronous  environment.  What  will 
be  particularly  interesting  will  be  the  patterns  that  come  into  view.  With  regard  to  the  types  of  communication, 
we  expect  to  find  significant  attention  being  given  to  social  comment  and  to  the  establishment  of  identity  and 
team  cohesion  as  compensatory  mechanisms.  In  terms  of  process,  the  absence  of  physical  presence  may  have  a 
positive  impact  on  giving  and  receiving  feedback  and  metacognitive  act  ivity.  Some  intermediate  categories  of 
process  related  to  task  and  process  related  to  social  factors  may  also  emerge.  Finally,  with  regard  to  methods  of 
analysis,  we  hope  that  the  dialogue  between  qualitative  and  quantitative  approaches  will  provide  new  insight  into 
CMC  decision  making  and  how  we  conduct  research  on  it. 

References 

Allison,  G.  1971.  The  Essence  of  Decision:  Explaining  the  Cuban  Missile  Crisis.  Boston:  Little-Brown. 

Eastmond,  D.  (1994).  “Adult  Distance  Study  through  Computer  Conferencing.”  Distance  Education  ISHL  128-152. 

Grint,  K (1989)  Accounting  for  Failure  : Participation  and  Non-Participation  in  CMC,  in  Mindweave  : Communication, 
Computers  and  Distance  Education,  Mason,  R.,  and  Kaye,  A (eds)  Oxford  Pergamon  Press. 

Gulick,  L.  & Urwirk,  L.  1937.  Papers  on  the  Administration.  New  York:  The  Free  Press. 

Gunawardena,  C.,  (1997).  “Analysis  of  a Global  Online  Debate  and  the  Development  of  an  Interaction  Analysis  Model  for 
examining  Social  Construction  of  Knowledge  in  Computer  Conferencing.”  Journal  of  Educational  Computing 
Research  17(4):  397-431. 

Hammond,  M.  (1998).  “Learning  Through  On-Line  Discussion  : what  are  the  opportunities  for  professional  development  and 
what  are  the  characteristics  of  on-line  writing.”  Journal  of  Information  Technology  for  Teacher  Education  7f3L  33 1 - 
346. 

Hara,  N.  and  R.  Kling  (1999).  “Students'  Frustrations  with  a Web- Based  Distance  Education  Course.”  First  Monday  4(12). 
Harasim,  L.,  R.  Hiltz,  et  al.  (1996).  Learning  Networks  : A Field  Guide  to  Teaching  and  Learning  On  Line.  Cambridge, 
England,  MIT  Press. 

Henri,  F.  (1991).  Computer  Conferencing  and  Content  Analysis.  Collaborative  Learning  Through  Computer  Conferencing. 

The  Naiaden  Papers.  A.  Kaye.  Berlin,  Springer -Verlag  1 17-  136. 

Merriam,  S.  (1998).  Qualitative  Research  and  Case  Study  Applications  in  Education.  San  Fransisco,  Jossey-Bass. 

Sallis,  P.  and  D.  Kasabova  (2000).  “Computer-Mediated  Communication,  Experiments  with  e-mail  readability.”  Information 
Sdences(123):  43-53. 

Simon,  H.  1 960.  T he  New  Science  of  Management  Decision.  New  York:  Harper  & Row. 


678 

♦The  research  team  comprises  3 colleagues  and  consequently  all  authors  are  equal  contributors  to  this  paper. 


SPLACH:  a Computer  Environment  Supporting 
Distance  Project-Based  Learning 


Sebastien  George 

Computer  Science  Laboratory  of  the  Universite  du  Maine  (LIUM) 
Avenue  Rene  Laennec  - 72085  Le  Mans  Cedex  9 - France 
email:  george@lium.univ-lemans.fr 


Abstract:  Our  research  deals  with  collaborative  learning  at  a distance  and  takes  place  in  the 
field  of  CSCL  (Computer-Supported  Collaborative  Learning).  To  promote  communications 
between  people  learning  in  a distance  context,  we  think  that  it  is  important  to  involve  tfiese 
learners  in  collective  activities.  We  suggest  to  set-up  activities  using  project -based  learning  in 
order  to  stimulate  and  facilitate  collective  learning.  In  order  to  support  distance  project -based 
learning,  we  designed  and  developed  a computer  environment  called  SPLACH.  This 
environment  comprises  specific  tools  for  different  actors:  the  project  designer,  the  learners  and 
the  project  leader.  SPLACH  was  investigated  in  two  different  contexts:  with  pupils  in  a 
secondary  school  in  the  domain  of  technology  and  with  adult  students  learning  programming  at 
the  Tele-university  of  Quebec.  This  article  presents  the  computer  environment  SPLACH  and 
some  of  the  results  of  its  evaluation. 


Introduction  and  Context  of  Work 

A major  interest  of  computer-supported  distance  learning  is  to  allow  communication  among  students 
and  to  employ  a constructivist  approach  for  learning:  according  to  the  CSCL  (Computer-Supported 
Collaborative  Learning)  research  domain  (Bannon  1989).  Our  main  hypothesis  is  that  conditions  have  to  be 
created  to  stimulate  interesting  interactions  between  learners.  A study  of  numerous  distance  learning 
environments  made  us  realize  that  the  main  attention  revolves  around  computer  tools  that  must  be  provided  for 
distant  learners  to  communicate.  Learners  working  together  would  theoretically  need  only  communication  tools. 
However,  tools  that  simply  offer  technical  possibilities  for  communication  would  appear  to  be  insufficient  to 
ensure  collective  learning.  We  feel  this  is  not  always  sufficient.  Indeed,  although  the  learners  are  offered  a 
possibility  to  communicate,  there  is  no  reason  for  them  to  do  so  if  there  is  no  common  interest  bringing  them 
together.  The  lack  of  social  cohesion  renders  truly  collective  learning  impossible. 

In  order  to  create  social  cohesion  in  distance  learning,  our  main  concern  is  to  concentrate  on  the 
instructional  activity  and  to  the  computer  environment  to  support  collective  learning.  Thus,  distance  should  not 
be  seen  as  an  obstacle  to  which  the  computer  tools  provide  a solution,  but  rather  as  an  interesting  learning 
situation  in  which  the  computer  tools  are  designed  to  support  new  learning  activities.  These  were  the  principles 
governing  the  development  of  our  distance  collective  activities  based  on  project-based  learning  practices. 
Project-based  learning  (PBL)  is  an  educational  method  of  creating  collective  classroom  learning  situations  which 
has  been  tried  and  tested  for  many  years  (Dewey  1902,  Kilpatrick  1918,  Johnson  and  Johnson  1991).  It  is 
particularly  interesting  to  use  in  a distance  learning  context  to  encourage  collective  learning.  However,  existing 
distance  learning  platforms  are  not  really  suited  to  support  PBL.  Even  if  there  are  several  experiments  of  PBL 
for  platforms  like  WebCT  or  FirstClass  (Christiansen  and  Dirckinck-Holmfeld  1995),  these  platforms  distort  the 
pedagogical  situation  (only  tools  to  communicate  and  to  exchange  data,  lack  of  tools  for  the  teacher,  lack  of 
synchronous  work  tools,  lack  of  group  awareness  tools,  etc.).  We  stress  the  importance  of  two  interdependent 
conditions  for  the  success  of  distance  PBL:  (1)  projects  need  to  be  set  up  carefully  in  order  to  promote 
interdependence  in  a distance  learning  situation  (2)  the  computer  environment  has  to  support  all  he  actors 
involved  in  PBL.  In  this  way  we  decided  to  create  a specific  computer  environment  dedicated  to  PBL,  founded 
on  particular  projects  and  team  organization.  The  organization  of  each  of  our  projects  is  structured  around  a 
project  designer^  role  played  by  a teacher,  a project  leader^  role  played  by  a tutor,  and  a team  of  learners  sharing 
tasks  in  the  same  project.  The  project  designer’s  role  is  to  define  the  projects  by  providing  specification 
documents,  structuring  the  projects  and  forming  the  teams,  tasks  usually  pertaining  to  an  instructional  designer. 
The  subject  of  each  project  should  provide  the  learners  with  the  opportunity  to  share  tasks  among  themselves. 
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Indeed,  in  a distance  learning  situation  it  is  not  advisable  to  make  the  learners  collaborate  all  the  time,  that  is  to 
say  carry  out  all  the  tasks  together  (Henri  and  Lundgren-Cayrol  2001).  The  project  subject  is  presented  as  a list 
of  tasks  specifications,  chosen  so  that  it  can  be  subdivided  by  the  learners  into  separate  parts  which  should,  of 
course,  not  be  independent  from  each  other  but,  on  the  contrary,  closely  dependent.  Moreover,  in  order  to  take 
into  account  the  synchronous  and  asynchronous  aspects  of  any  distance  work,  the  project  is  divided  into  stages 
where  each  stage  is  broken  up  into  an  asynchronous  work  phase  followed  by  a synchronous  phase.  The  intention 
of  structuring  collective  work  is  to  facilitate  the  learners’  collective  work  and  to  stimulate  the  learning  of  how  to 
work  collectively.  More  information  about  the  instructional  design  for  the  collective  activities  can  be  found  in 
(George  and  Leroux  2001).  The  work  of  the  project  leader  consists  in  monitoring  the  evolution  of  the  project 
and  helping  the  learners  both  technically  and  pedagogically.  The  project  leader  is  seen  as  the  facilitator  and 
consultant.  The  teams  of  learners  work  to  reach  the  goal  set  for  the  project.  Each  team  was  composed  of  three 
learners  in  order  to  promote  high  involvement  in  the  project  (Caplow  1968).  In  this  paper,  we  focus  on 
characteristics  of  the  computer  environment  enhancing  collective  project-based  learning  (PEL). 


Development  of  a Computer  Environment  to  Support  Distance  Project-Based  Learning 

The  main  aim  of  our  work  is  to  provide  tools  for  the  different  actors  involved  in  project-based  learning. 
The  actors  defined  in  the  previous  section  should  have  computer  environments  allowing  them  to  do  their  specific 
tasks:  setting-up  the  projects  for  the  project  designer,  carrying  out  the  project  in  team  for  the  learners  and 
monitoring  and  intervening  for  the  project  leader.  So  the  SPLACH  [1]  environment  was  developed  to  support 
PEL;  it  includes  three  computer  environments  which  is  customized  for  each  of  the  actors,  that  is  the  project 
designer,  the  project  leader  and  the  learner.  SPLACH  is  entirely  developed  in  Java  and  is  built  on  a client/server 
architecture  (George  2001).  We  use  the  Java  Shared  Data  Toolkit  (JSDT)  library  for  the  data  exchange  between 
applications  (Eurridge  1999).  We  describe  thereafter  the  three  specific  SPLACH  environments. 


Project  Designer’s  Environment 

With  SPLACH,  the  project  designer  has  tools  to  define  and  to  set-up  projects  for  learners.  Firstly,  the 
project  designer  can  create  new  projects  and  specify  teams  of  learners  and  their  project  leader  who  will  monitor 
the  learners.  Then,  the  project  designer  structures  each  project  in  steps  according  to  the  pedagogical  objectives  to 
be  reached.  The  project  designer  defines  preformatted  documents  that  will  appear  as  a project  progresses. 
Finally,  the  project  designer  plans  project  giving  a preliminary  schedule  that  could  be  discussed  among  the 
learners  and  their  project  leader  (Fig.  1).  Indeed  this  environment  can  be  viewed  as  a project  editor  facilitating 
the  creation  of  project-based  learning  in  a distance  context. 


Structure  of  the  project  in  stages  Plan  of  the  project  and  of  Its  stages  Link  preformatted  documents  to  the  stage 

Figure  1 : Interface  of  the  project  designer 


[ 1 ] SPLACH  is  the  French  acronym  for  '"'‘Support  d'une  pedagogie  de  Projet  pour  L ’Apprentissage  Collectif  Humain  " which 
can  be  translated  as  “Support  Project-Eased  Learning  for  Collective  Human  Learning”. 


r«ST  COIP?  AVAILABtK 


680 


Learner’s  Environment 


Concerning  the  learner’s  environment,  our  aim  was  to  design  an  environment  that  incorporates  all  the 
tools  necessary  for  the  learners’  collective  activities.  While  common  practice  dictates  the  use  of  different  tools 
(for  instance  Word®  for  editing  documents,  Eudora®  for  Bmailing  and  Netmeeting®  for  synchronous  meetings) 
when  learners  have  to  use  numerous  software  products  at  the  same  time,  the  cognitive  workload  will  be 
increased.  In  a learning  context,  this  extra  load  would  not  be  favorable.  The  simpler  the  interface  and  use  of  the 
tools,  the  more  learners  will  be  able  to  concentrate  on  their  activities.  This  simplicity  calls  for  an  environment 
comprising  all  the  necessary  tools  to  be  designed  and  available  in  one  environment.  The  amount  of  time  and 
energy  spent  on  the  development  will  of  course  be  greater,  but  this  is  the  only  way  to  develop  a system  that  any 
learner  can  use,  irrespective  of  his/her  competence  in  computing.  Moreover,  there  is  another  advantage  to  this 
investment.  Existing  software  is  often  quite  prohibitive,  and  difficult  to  analyze  in  terms  of  usage  traces.  Another 
of  our  aims  was  to  facilitate  collective  learning  based  on  information  obtained  from  the  computer  system.  The 
design  and  development  of  our  own  tools  allow  for  collection  of  data  concerning  the  learners’  and  the  teams 
activities.  These  data  are  then  analyzed  and  used  by  the  system  to  promote  the  collective  activities  (Despres  and 
George  200 1 ). 

Thus  the  learners’  SPLACH  environment  integrates  asynchronous  communication  tools  (Ermail  and 
discussion  forum),  a synchronous  meeting  tool  which  allows  textual  discussion  and  application  sharing,  a 
scheduling  tool  in  the  form  of  a team  calendar  which  provides  the  learners  with  coordination  of  the  project,  a 
tool  to  write  reports  during  the  project  (documents  are  preformatted  by  the  project  designer)  and,  finally,  specific 
tools  of  the  subject  matter.  These  specific  tools  are  the  only  part  of  SPLACH  that  varies  from  one  subject  matter 
to  another.  A lot  of  attention  was  particularly  paid  to  the  design  of  the  synchronous  meeting  tool,  intending  to 
support  awareness  (Dourish  and  Bellotti  1 992)  among  the  participants  of  a synchronous  distance  meeting  and  to 
stimulate  the  teamwork.  Since  good  quality  multipoint  videoconference  tools  are  not  yet  accessible  with  standard 
networks,  we  chose  to  display  photos  of  participants  during  the  meeting  sessions.  Moreover,  participants  can 
share  a document  and  work  collectively  on  it  using  an  easy  visual  system  of  turn  taking  (Fig.  2).  During  a 
meeting,  participants  communicate  within  a semi-structured  interface  that  helps  to  keep  threads  of  discussion 
visible. 
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Figure  2:  Interface  of  the  learners 
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Project  Leader’s  Environment 

The  project  leader  is  a central  key  in  our  conception  of  distance  PBL.  S/He  has  to  have  tools  to 
communicate  with  learners  and  to  monitor  their  activities.  For  the  moment,  the  project  leader’s  environment  is 
very  close  to  the  learner’s  environment;  however  including  some  additional  functionalities  and  specific  rights. 
Thus,  the  project  leader  can  modify  the  planning  of  a project,  manages  rights  of  turn  taking  during  synchronous 
meeting.  We  are  presently  researching  and  investigating  specific  tools  to  assist  the  tutor  in  monitoring  both 
individual  and  collective  learner  activities.  To  reach  our  objectives  to  refine  the  leader/tutor  environment,  an 
iterative  design  process  is  employed  by  implementing  the  SPLACH  environment,  interview  and  keep  traces  of 
how  tutors  use  it,  and  then  adapt  and  enrich  the  SPLACH  environment  accordingly.  One  of  these  tools  is  a 
system  designed  to  assist  the  tutor  in  his  monitoring  tasks  (George  2001). 


Evaluation  of  SPLACH 

One  of  our  major  concerns  was  to  develop  a computer  environment,  which  can  support  projects  in  a 
large  range  of  educational  fields.  In  order  to  prove  the  reusability,  SPLACH  has  been  investigated  in  two 
different  contexts:  with  junior  high  school  students  in  the  technology  field  and  with  adult  students  learning 
programming.  In  the  following  sections,  we  discuss  some  results  focusing  on  the  use  of  SPLACH.  These  results 
are  based  on  observations,  questionnaires  and  automatic  data  traces  recorded  by  the  computer  system. 


With  Junior  High  School  Students  in  Educational  Robotics 

Each  year  a robotics  competition  is  organized  for  junior  high  school  students  in  the  department  of  the 
Sarthe  (France).  All  teams  taking  part  are  given  a list  of  specifications  at  the  beginning  of  the  school  year;  during 
the  year  the  students  work  on  designing  robots  in  order  to  compete  in  the  festival  at  the  end  of  the  school  year. 
This  festival  provided  an  interesting  situation  to  design  the  collective  distance  projects.  The  students  on  each 
team  usually  come  from  the  same  school.  For  our  investigation,  we  created  two  teams  made  up  of  students  from 
geographically  remote  schools.  These  teams  of  learners  used  our  SPLACH  environment  from  their  classroom  to 
work  collectively  on  the  robotics  project.  We  assumed  the  role  of  project  designer  and  project  leader  ourselves 
from  our  laboratory  in  order  to  better  understand  how  it  felt  to  supervise  and  monitor  teams  of  learners  at  a 
distance.  For  these  evaluation,  we  added  specific  tools  from  the  educational  robotics  fields  to  SPLACH:  a tool  to 
view  electronic  course  books  that  embed  all  notions  introduced  in  the  technological  context,  a tool  to  describe 
micro -robots  [2]  to  computers  which  automatically  generates  elementary  programs,  and  a tool  to  program  and 
drive  micro-robots.  The  Fig. 3 shows  screen  shots  of  these  different  tools. 


Figure  3;  Specific  educational  robotics  tools  integrated  in  the  SPLACH  environment 


The  evaluation  was  conducted  with  fifteen  students  (aged  13/14)  in  three  different  schools.  The  list  of 
specifications  for  this  project,  defined  by  the  organizers  of  the  robotics  challenge,  required  the  teams  to  design  a 
robot  which  could  carry  out  several  functions  (follow  a black  line,  knock  over  a skittle,  put  a ball  into  a hole, 
etc. . .).  The  students  defined  the  modules  it  would  take  to  build  a robot  and  then  determined  who  was  going  to  do 
what.  Obviously,  the  modules  were  interrelated,  which  in  turn  promotes  collective  learning,  since  they  depended 
on  each  other  to  bring  the  modules  together  as  one  robot,  which  would  enter  the  competition. 


[2]  The  robot  used  are  Fischertechnik®  micro-robots. 
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The  evaluation  lasted  for  about  three  months,  with  the  students  using  the  SPLACH  environment  for 
about  two  or  three  hours  each  week.  Every  Friday  lunchtime,  ninety  minutes  were  set  aside  for  synchronous 
meeting  sessions  with  the  teammates.  Outside  these  sessions  the  teams  could  work  on  the  project  whenever  they 
wanted  to  by  communicating  asynchronously  via  Email  or  the  discussion  forum.  Two  weeks  before  the 
competition,  the  students  met  physically  for  the  first  time  in  order  to  assemble  the  different  parts  of  the  robots. 
On  the  day  of  the  festival,  one  of  our  teams  shared  first  place  in  the  competition.  The  results  of  a post- 
questionnaire show  how  the  learners  felt  the  importance  to  work  collectively,  and  that  they  found  this  type  of 
collective  work  highly  motivating.  Moreover,  the  results  of  the  questionnaire  show  that  the  learners  perceived 
that  they  were  part  of  a team  throughout  the  project,  which  was  also  apparent  during  the  competition.  We  can 
therefore  say  that  our  main  educational  goal,  which  was  to  create  social  cohesion  and  interdependence  among 
distance  learners,  has  been  achieved.  The  learners  had  no  particular  difficulty  to  use  SPLACH  even  if  they  were 
children.  It  was  one  of  our  concerns  to  make  SPLACH  so  user-friendly  that  it  would  cater  to  a wide  public  of 
learners,  including  young  ones. 


With  Adult  Students  Learning  Programming 


A second  evaluation  of  SPLACH  was  carried  out  at  the  Tele -university  of  Quebec.  Six  adult  students  in 
a programming  course  used  SPLACH  during  six  weeks  to  work  in  teams,  each  student  working  from  her/his  own 
home.  The  list  of  specification  of  the  projects  corresponded  to  a practical  work  of  their  course.  The  main  goal 
was  to  permit  the  students  to  work  collectively  on  a program  in  Pascal,  each  student  being  responsible  for 
specific  functions  of  the  program.  A tutor  at  the  Tele -university  assumed  the  role  of  project  leader. 

We  have  integrated  to  SPLACH  a specific  tool  to  allow  the  learners  to  work  in  team  on  a same 
computer  program.  This  tool  makes  a link  between  SPLACH  and  the  Delphi®  environment.  Thus  the  computer 
functions  and  procedures  found  in  a source  program  written  with  Delphi®  are  automatically  added  into 
SPLACH.  In  this  way  learners  share  their  programs.  The  Fig.  4 shows  this  tool  during  the  display  of  a 
programming  procedure  in  SPLACH.  In  the  lower  part  of  the  screen,  the  learners  can  access  the  main  program, 
their  personal  programs  or  their  teammates  programs. 


Altogether,  students  connected  to  the  SPLACH  server  for  a total  of  70  hours.  Students’  answers  to  the 
questionnaire  bring  out  the  fact  that  they  appreciated  the  possibility  to  communicate  and  to  help  each  other. 
Furthermore,  the  students  considered  the  synchronous  phases  to  be  necessary  to  the  organization  of  the  collective 
activity  even  if  they  place  temporal  constraints  not  always  easy  to  solve.  This  type  of  project  appears  to  help 
students  better  understand  the  necessary  conditions  to  a good  cooperation.  They  like  the  environment  because  it 
allows  them  to  easily  gather  documents  and  programs  together  in  the  same  place.  Furthermore,  they  felt  that  the 
project  leader  had  a crucial  role,  although  some  of  them  regretted  that  he  was  not  more  actively  involved  and 
more  directive.  Moreover,  some  of  the  students  would  like  to  take  on  the  role  of  the  project  leader.  It’s  an  option 
that  we  foresee  for  future  investigations. 
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Outcomes  From  the  Evaluations 


For  the  two  evaluations,  the  goal  of  the  projects  were  reached:  building  and  programming  a robot  for 
the  junior  high  school  students,  writing  a computer  program  for  the  students  of  the  Tele -university.  An 
observation  of  the  students’  activities  shows  that  final  products  have  really  been  the  result  of  collective  work. 
These  evaluations  also  made  it  possible  to  substantiate  the  validity  of  the  SPLACH  environment  from  both  a 
pedagogical  and  technical  point  of  view.  This  environment  is  capable  to  adequately  supporting  distance  project- 
based  learning.  The  students  found  it  easy  to  use,  most  likely  because  it  integrates  all  the  tools  necessary  for 
collective  work.  We  have  also  shown  that  the  SPLACH  environment  can  be  used  in  different  context. 


Conclusions  and  Futures  Directions 

The  SPLACH  environment  was  developed  to  support  project-based  learning  at  a distance.  It  adapts  its 
tools  according  to  the  actors:  the  project  designer,  the  learner  and  the  project  leader.  For  a distance  learning 
designer,  SPLACH  can  be  seen  as  a computer  platform  facilitating  the  setting-up  of  project  based-learning  in 
which  you  only  have  to  integrate  the  tools  pertinent  to  the  subject-matter.  If  existing  specific  tools  are  developed 
in  the  java  language,  they  could  easily  be  integrated  to  SPLACH.  In  other  cases,  a link  can  be  made  in  SPLACH 
to  launch  external  specific  tools.  For  learners,  a strong  integration  of  the  collective  work  tools  leads  to  make 
SPLACH  accessible  to  a wide  public  of  learners  as  the  evaluations  corroborated.  It  was  noticed  from  our 
evaluations  that  the  project  leader/tutor  had  a much  more  active  role  compared  to  traditional  tutoring  of 
individual  distance  learning.  The  collective  activity  is  more  complex  to  supervise  and  monitor,  which  is  more 
time  consuming  for  the  tutor.  Thus,  a future  direction  of  our  work  lies  in  providing  particular  tutor  tools  to  assist 
her/him  in  the  supervising  and  monitoring  of  learners’  collective  activities.  In  this  way,  we  work  on  integrating  a 
multi-agent  system  into  SPLACH  aiming  at  helping  both  the  learners  and  the  project  leader. 
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Utilizing  Electronic  Portfolios  in  Teacher  Education  Programs 

Constance  Geier,  Ed.D. 

Northern  State  University,  Aberdeen  SD 

The  School  of  Education  at  Northern  State  University  in  Aberdeen,  South  Dakota  has 
implemented  the  electronic  portfolio  as  an  assessment  tool  and  as  a means  of  integrating  technology 
throughout  teacher  education.  The  electronic  portfolio  offers  a unique  opportunity  to  build  preservice 
teachers’  proficiency  with  technology  as  well  as  showcase  their  expertise  in  teaching. 

The  portfolio  concept  is  very  popular  in  education  today  - and  for  good  reason.  First,  today’s 
educators  have  embraced  constructivism  - the  belief  that  teaching  is  an  active  and  learner-centered  process. 
This  philosophy  recognizes  that  students  build  their  own  understanding  of  the  world  by  using  what  they 
already  know  to  interpret  new  ideas  and  experiences.  Constructivists  emphasize  not  only  what  students 
know,  but  what  they  do. 

Secondly,  the  growing  interest  among  colleges  of  education  in  performance  assessment  makes  a 
transcript  of  grades  and  a score  on  the  National  Teachers*  Exam  (NTE)  seem  inadequate  indicators  of 
competence.  A wise  person  once  said:  “There  is  a lot  of  difference  between  naming  the  tools  and  building 
the  house.” 


And  thirdly,  there  is  stiff  competition  for  teaching  jobs  in  most  areas  of  the  count!y^  It  is 
imperative  that  prospective  teachers  be  able  to  demonstrate  their  teaching  competence  in  concrete  ways  - to 
university  faculty,  to  prospective  employers,  to  policy  makers  at  the  state  and  national  levels,  to  parents,  the 
media  and  the  general  public. 

Why  electronic  portfolios  rather  than  the  paper  versions?  Electronic  portfolios  emphasize  process 
as  well  as  product  and  are  multi-sensory  in  nature,  including  images,  sound,  video,  text,  and  multimedia 
products.  It’s  more  fiin  to  see  a great  bulletin  board  than  to  read  about  it,  and  it’s  more  effective  to  hear  a 
university  supervisor  talking  about  a preservice  teacher’s  strengths  than  to  read  a letter  of  recommendation. 
In  addition,  electronic  portfolios  facilitate  the  integration  of  technology  throughout  the  teacher  education 
program;  they  provide  students  with  exposure  to  a wide  variety  of  technology  experiences  - all  in  the 
context  of  teaching  and  learning. 

The  critical  phases  of  portfolio  development  are  collecting,  selecting,  and  reflecting  - but  the 
process  actually  has  many  phases.  Burke,  Fogarty,  and  Belgrad  (1994)  proposed  ten:  projecting,  collecting, 
selecting,  interjecting,  reflecting,  inspecting,  perfecting,  connecting,  injecting  (and  ejecting),  and 
respecting. 

As  students  think  about  what  entries  they  will  collect,  how  to  select  those  that  best  convey  their 
abilities,  and  how  to  present  what  they  have  learned,  they  are  constantly  reflecting.  Reflective  thinking  (as 
defined  by  Dewey,  1933)  is  the  ability  to  give  serious  and  persistent  consideration  to  a subject  in  order  to 
act  deliberately  and  intentionally  rather  than  routinely  and  impulsively.  If  teacher  educators  want  preservice 
teachers  to  move  beyond  non-reflective  reliance  on  impulse,  tradition,  and  authority,  opportunities  for 
reflection  must  be  provided  throughout  the  teacher  education  program. 

After  piloting  the  use  of  electronic  portfolios  with  twelve  teacher  education  students  during  the 
1998-1999  academic  year,  the  School  of  Education  at  NSU  began  implementing  electronic  portfolio 
components  into  teacher  education  methods  courses.  The  model  used  for  electronic  portfolio 
implementation  solicited  proposals  from  faculty  members  interested  in  integrating  electronic  portfolio 
components  into  their  methods  classes.  Faculty  received  monetary  compensation  for  the  creation  and 
integration  of  electronic  portfolio  components  into  their  courses. 

Each  student-created  electronic  portfolio  is  original;  portfolio  components  are  designed  to  address 
one  or  more  of  the  ISTE  National  Educational  Technology  Standards  (NETS)  and  performance  indicators 
for  teachers.  In  addition,  students  must  address  each  of  the  five  categories  of  the  knowledge  base  for 
teacher  education  at  NSU:  Knowledge  of  Self  as  an  Individual,  Knowledge  of  Content,  Knowledge  of  the 


Learner,  Knowledge  of  Pedagogy,  and  Knowledge  of  Self  as  a Teacher  and  Member  of  a Learning 
Community. 

Electronic  portfolios  have  been  a part  of  the  teacher  education  programs  at  Northern  State 
University  for  four  years;  current  efforts  have  focused  on  assessment  of  the  electronic  portfolio  process  and 
product.  Teacher  education  graduates  are  asked  to  complete  a self-assessment  survey  of  their  technology 
expertise  and  their  comfort  level  with  technology  integration.  During  their  professional  semester, 
preservice  teachers  showcase  their  electronic  portfolios  and  receive  feedback  from  university  faculty 
members.  Electronic  portfolio  components  are  then  assessed  to  determine  whether  appropriate 
opportunities  have  been  provided  for  students  to  meet  the  ISTE  National  Educational  Technology 
Standards  (NETS)  and  performance  indicators  for  teachers  as  well  as  the  required  program  outcomes  for 
teacher  education  graduates  of  NSU. 

This  presentation  will  focus  on  data  collected  from  the  self-assessment  surveys  as  well  as  the 
results  of  the  electronic  portfolio  assessment.  Specific  components  of  original  student-created  electronic 
portfolios  addressing  one  or  more  of  the  ISTE  standards  and  meeting  one  or  more  program  outcomes  will 
be  shared  along  with  accompanying  assessment  tools,  including  a newly-developed  holistic  rubric. 

Burke,  K.,  Fogarty,  R.,  Belgard,  S.  (1994).  The  mindful  school:  The  portfolio  connection. 
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Abstract:  This  demonstration  presents  a constructivist  approach  to  teaching  web 
design  skills,  2D  animation,  basic  scripting  and  programming  concepts  using 
Macromedia  Flash  and  ActionScript.  The  overall  curriculum  is  discussed  followed 
by  an  examination  of  two  courses  that  focus  on  teaching  Flash  and  ActionScript 
accompanied  by  demonstrations  of  tutorials  and  projects,  which  incorporate 
meaningful  content  provided  by  on  campus  institutions.  The  teaching  approach  is 
informed  by  theories  of  multiple  intelligence  and  address  different  styles  and  rates  of 
learning  by  featuring  learner  centered,  hands-on  tutorials,  which  emphasize  learning 
by  doing. 


Background 

In  the  fall  2001  Quinnipiac  University  introduced  a new  major  in  Interactive  Digital  Design  to 
complement  the  major  in  Computer  Science.  Both  programs  are  under  one  department  in  the  College  of  Liberal 
Arts.  The  Computer  Science  Majors  are  provided  with  a foundation  in  Computer  Science  within  the  framework 
of  a traditional  Liberal  Arts  education  that  prepare  graduates  for  success  in  a wide  variety  of  Internet  and 
computer  related  fields. 

The  Interactive  Digital  Design  major  provides  students  with  in-depth  hands-on  experience  in  creating, 
designing  and  authoring  original  interactive  media  for  CD-ROM,  DVD,  the  World  Wide  Web,  and  desktop 
presentation.  This  major  emphasizes  artistic  creativity,  experimentation  and  technological  competence  grounded 
in  a critical  understanding  of  the  arts,  design  and  related  areas  of  the  humanities. 


Approach 

The  program  features  a core  set  of  requirements  open  to  majors,  minors  and  qualified  non-majors. 
Because  of  this  diverse  clientele,  the  tutorials  must  address  different  styles  and  rates  of  learning.  Teaching  is 
informed  by  Howard  Gardner  (Gardner,  1985)  theory  of  multiple  intelligences  and  Robert  Sternberg’s  Triarchic 
Theory  of  Intelligence  (Sternberg  et  al.,  2000).  Tutorial  projects  incorporate  content  based  on  university- 
affiliated  institutions,  which  develop  Sternberg’s  triad  of  practical,  creative  and  analytical  skills. 
Demonstrations  include  sample  projects  created  for  the  Albert  Schweitzer  Institute  and  the  Service  Learning 
Program  at  Quinnipiac,  which  is  part  of  the  national  Campus  Compact.  Experience  shows  that  the  integration  of 
this  value  rich  content  leads  to  a greater  student  engagement  and  commitment.  Students  enrolled  in  IDD  301  2D 
Animation  thereby  develop  cognitive  tools  for  working  with  image,  text,  motion,  sound  and  interactivity  guided 
by  design  principles.  Project  based  learning  is  incremental  trial  and  error  i.e.  learning  by  doing.  IDD:  315 
Scripting  for  Interactivity  introduces  advanced  scripting  including  control  structures,  objects  and  classes, 
properties  and  methods,  expressions  and  variables,  conditionals,  arrays,  functions,  handlers,  smart  movie  clips, 
while  using  debugging  and  content  management  tools  of  FLASH. 

Each  new  programming  concept  is  always  incorporated  in  a tutorial  as  part  of  a larger  design  problem. 
In  turn  the  final  goal  is  to  instill  an  overall  cyberliteracy  such  as  advocated  by  Laura  J.  Gurak  (Gurak,  2001). 
This  instills  recognition  that:  “on  the  internet,  communication  is  a blend  of  oral,  written,  and  visual 
information.”  This  cyberliteracy  is  matched  an  critical  awareness  that  is  fostered  by  working  with  real  world 
content  such  as  that  provided  by  the  Albert  Schweitzer  Institute  as  part  of  a true  liberal  arts  education. 
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Abstract  Computer  programming  is  a domain  of  knowledge  that  is  generally  considered 
difficult  by  students,  many  of  whom  experience  low  levels  of  achievement  and  become 
disillusioned.  This  paper  suggests  that  cognitive  load  theory  needs  to  be  taken  into  account 
when  designing  instructional  materials  for  this  domain.  The  cognitive  load  that  is  experienced 
by  a student  can  be  considered  to  be  made  up  of  three  types:  intrinsic,  extraneous,  and  germane. 
Computer  programming  has  a high  intrinsic  load  and  it  is  therefore  necessary  to  reduce  the 
extraneous  bad  as  much  as  possible  by  using  techniques  such  as  the  study  of  programming 
examples.  Germane  cognitive  load  can  then  be  applied  by  removing  certain  parts  of  the 
solutions  to  the  examples  and  then  requiring  students  to  complete  these  part  -complete  solutions 
thereby  encouraging  schemata  creation  in  longterm  memory.  A new  software  tool  called 
CORT  (Code  Restnichiring  Tool)  has  been  Created  which  utilises  this  part -completion  method. 


Introduction 

In  education,  certain  subjects  require  problem  solving  skills  and  are  considered  by  many  students  to  be 
inherently  difficult.  An  example  of  such  a subject  is  software  development  and  this  requires  students  to  be  able 
to  analyse  problems  and  then  design  and  implement  solutions  in  a computer  programming  language.  It  has  been 
observed  that  a large  number  of  students  achieve  only  low  grades  and  become  disillusioned  with  the  subject,  for 
example  Perkins,  Schwartz,  & Simmons  (1988)  state  that: 

Students  with  a semester  or  more  of  instruction  often  display  remarkable  naivete  about  the 
language  that  they  have  been  studying  and  often  prove  unable  to  manage  dismayingly  simple 
programming  problems. 

Also,  King  (1994)  states: 

Even  after  two  years  of  study,  many  students  had  only  a rudimentary  understanding  of 
programming. 

One  of  the  reasons  for  the  above  is  that  students  experience  a very  high  cognitive  load  during  their 
learning  and  this  paper  proposes  that  cognitive  load  theory  needs  to  be  carefully  taken  into  account  in  the  design 
of  learning  materials  and  tools  for  such  problem  solving  domains.  The  paper  then  describes  a software  tool 
called  CORT  (Code  Restructuring  Tool)  that  has  been  built  by  the  author  and  used  with  novice  programmers  at  a 
university  in  Australia. 


Cognitive  Load  Theory 

Cognitive  load  theory  is  built  upon  the  idea  that  working  memory  is  limited  to  around  seven  chunks  of 
material  (Miller,  1956)  and  that  people  can  only  deal  with  two  or  three  elements  simultaneously.  The  degree  of 
interactivity  between  the  elements  also  affects  the  capacity  of  working  memory. 

Chess  playing  can  be  considered  a problem  solving  domain  and  research  (Chase  & Simon,  1973) 
showed  that  the  main  difference  between  novices  and  experts  was  the  fact  that  the  latter  had  thousands  of  board 
configurations,  as  many  as  100000  (Simon  & Gilmartin,  1973),  stored  in  long-term  memory  within  schemata. 
The  consequence  is  that,  unlike  less-skilled  players,  experts  do  not  have  to  spend  as  much  time  searching  for 
good  chess  moves  using  their  limited  working  memory.  Similarly,  research  into  problem  solving  (Carroll,  1994) 
confirmed  that,  compared  to  novices,  experts  have  knowledge  of  an  enormous  number  of  problem  states  and 
their  associated  moves.  Such  states  are  within  long-term  memory  and  such  research  indicates  that  human 
problem  solving  comes  from  stored  knowledge  and  not  from  complex  reasoning  within  working  memory.  It  is 


suggested  that  humans  are  poor  at  complex  reasoning  unless  most  of  the  elements  with  which  we  reason  are 
already  in  long-term  memory,  working  memory  being  incapable  of  highly  complex  interactions  using  novel 
elements  (Sweller,  van  Merrienboer,  & Paas,  1998).  This  means  that  novices  who  are  attempting  a problem  must 
engage  in  complex  chains  of  reasoning  using  their  working  memory  and  in  doing  so  it  is  likely  that  working 
memory  will  be  overburdened.  In  other  words  the  cognitive  load  on  novices  is  too  great. 

Ways  in  which  cognitive  load  can  be  reduced  for  novice  problem  solvers  are  therefore  very  important. 
In  the  schema  theory  of  model  representation,  a schema  can  be  anything  that  can  be  treated  as  a single  entity  or 
element  such  as  a mathematical  formula  or  a particular  programming  algorithm.  Schemata  have  the  function  of 
storing  knowledge  and  reducing  the  burden  on  working  memory. 

Experts  can  process  information  relevant  to  their  domain  automatically,  novices  however  having  to 
process  information  consciously  (Schneider  & Shiffrin,  1977;  Tindall- Ford,  Chandler,  & Sweller,  1996)  An 
example  of  such  automatic  processing  is  that  of  the  expert  driver  who  can  drive  their  car  without  apparently 
thinking,  whereas  a learner  driver  has  to  consciously  think  of  several  things  at  the  same  time  such  as  depressing 
the  clutch  and  shifting  to  a new  gear,  observing  the  road  ahead,  moving  the  steering  wheel  etc.  Any  instructional 
design  for  a domain  has  to  therefore  not  only  encourage  the  construction  of  sophisticated  schemata  but  also 
encourage  the  automatic  processing  of  those  schemata.  This  is  important  because  of  the  limited  capacity  of 
working  memory  that  can  only  deal  with  a few  schemata  at  the  same  time.  The  ease  with  which  information  can 
be  processed  in  working  memory  is  the  main  thrust  of  cognitive  load  theory. 

Working  memory  may  be  affected  by  intrinsic  cognitive  load  and  extraneous  cognitive  load  (Sweller, 
1994).  In  recent  research,  a further  distinction  is  made  with  the  inclusion  of  germane  cognitive  load  (Sweller  et 
al.,  1998) 


Intrinsic  Cognitive  Load 

Intrinsic  cognitive  load  is  determined  by  the  mental  demands  of  the  task  (Chandler  & Sweller,  1996). 
Some  material  has  very  low  cognitive  load  and  an  example  is  the  learning  of  the  basic  vocabulary  of  a foreign 
language.  Each  element  or  schema  is  independent  from  the  others  with  no  interactivity  and  subsequently  the 
required  mental  processing,  or  intrinsic  cognitive  load,  is  low.  Tasks  that  have  low  element  interactivity  can  be 
learnt  serially  rather  than  simultaneously.  Tasks  with  a high  degree  of  element  interactivity  have  a heavy 
intrinsic  cognitive  load  and  an  example  is  the  learning  of  the  grammar  of  a foreign  language  as  all  the  words  in 
phrases  need  to  be  considered,  that  is  processed,  at  once. 

Computer  programming  is  a doma  in  with  a high  intrinsic  cognitive  load  and  this  needs  to  be  recognised 
in  any  instructional  design.  The  intrinsic  cognitive  load  cannot  be  reduced,  however  something  can  be  done 
about  the  extraneous  cognitive  load. 


Extraneous  Cognitive  Load 

Extraneous  cognitive  load  is  generated  by  the  instructional  format  used  in  the  teaching  and  learning 
process  and  poor  design  leads  to  a high  extraneous  cognitive  load.  If  a high  extraneous  cognitive  load  is 
combined  with  a high  intrinsic  cognitive  load  then  this  can  lead  to  working  memory  overload.  This  is  often  what 
happens  with  novice  programmers  when  the  instructional  design  is  poor. 

The  important  point  is  that  when  the  intrinsic  cognitive  load  of  the  material  is  high,  then  it  is  incumbent 
on  the  instructional  desgner  to  think  very  carefully  and  ensure  that  the  extraneous  cognitive  load  is  as  low  as 
possible.  A lot  of  research  has  been  done  in  looking  at  ways  of  reducing  extraneous  cognitive  load,  for  example 
(Chandler  & Sweller,  1991;  Kalyuga,  Chandler,  & Sweller,  1998;  Marcus,  Cooper,  & Sweller,  1996;  Sweller, 
1994;  Tindall  -Ford  et  al.,  1997).  These  include:  integrating  diagrams  and  text  so  as  to  reduce  the  “split-attention” 
effect;  goal-free  problem  solving;  and  the  use  of  worked  examples  in  problem  solving. 


Germane  Cognitive  Load 

More  recently,  the  concept  of  germane  cognitive  load  has  been  introduced  into  cognitive  load  theory 
(Sweller  et  al.,  1998)  It  is  thought  that  if  the  instructional  design  is  such  that  the  extraneous  cognitive  load  is 
kept  to  a minimum,  and  the  intrinsic  cognitive  load  is  not  too  high,  then  there  may  be  some  unused  working 


memory  available.  This  could  then  be  used  by  learners,  with  appropriate  instructional  design,  to  engage  in 
conscious  processing  that  helps  in  the  construction  of  schemata  in  the  particular  domain  of  interest.  This 
conscious  processing  is  the  germane  cognitive  load.  An  example  is  the  use  of  part -complete  solutions  in  the 
learning  of  problem  solving  (Paas,  1992;  van  Merrienboer,  1990;  Van  Merrienboer  & De  Croock,  1992).  The 
studying  of  complete  worked  examples  by  students  is  seen  as  one  way  of  reducing  the  extraneous  cognitive  load. 
When  students  have  to  complete  an  incomplete  worked  example  then  they  have  to  “mindfully  abstract”  the 
schemata  from  the  example  in  order  to  understand  it.  That  is,  they  have  to  consciously  process  it  and  this 
increases  the  germane  cognitive  load.  Figure  1 shows  the  relationship  between  the  various  cognitive  loads  in  the 
domain  of  programming. 


Figure  1:  Cognitive  load  relationships  in  programming 


Use  of  Worked  Examples  and  Part -Complete  Solutions  in  the  Teaching  and  Learning  of 
Programming 

The  "Reading"  approach  to  the  learning  of  programming  makes  use  extensive  of  worked  examples  in 
an  attempt  to  reduce  the  extraneous  cognitive  load  on  novices.  A lot  of  the  work  in  this  area  has  been  carried  out 
by  van  Merrienboer  and  his  colleagues  (van  Merrienboer,  1990a;  van  Merrienboer,  1990b;  van  Merrienboer  & 
Paas,  1990c;  van  Merrienboer  & De  Croock,  1992;  van  Merrienboer,  Krammer,  & Maaswinkel,  1994).  They 
argue  that  the  traditional  approach  to  the  teaching  and  learning  of  programming  is  ineffective  and  that  the 
“Reading”  approach  is  a better  one  to  follow.  However,  they  also  suggest  that  presenting  worked  examples  to 
students  is  not  sufficient  as  the  students  may  not  “abstract”  the  programming  plans  from  them.  Programming 
plans  can  be  considered  analogous  to  the  chess  of  board  configurations  mentioned  earlier.  “Mindful”  abstraction 
of  plans  is  required  by  the  voluntary  investment  of  effort  and  the  question  then  arises  as  to  how  we  can  get 
students  to  study  the  worked  examples  properly.  In  practice,  students  tend  to  rush  through  the  examples,  even  if 
they  have  been  asked  to  trace  them  in  a debugger,  as  they  often  believe  that  they  are  only  making  progress  in 
their  learning  when  they  are  attempting  to  solve  problems. 

Such  conscious  processing  by  students  places  germane  cognitive  load  upon  them.  One  suggestion  is  that 
students  should  annotate  worked  examples  with  information  about  what  they  do  or  what  they  illustrate 
(Lieberman,  1986).  Another  suggestion  is  to  use  incomplete,  well-structured  and  understandable  program 
examples  that  require  students  to  generate  the  missing  code  or  “complete”  the  examples.  This  latter  approach 
forces  students  to  study  the  incomplete  examples  as  it  would  not  be  possible  for  their  completion  without  a 
thorough  understanding  of  the  examples’  workings.  An  important  aspect  is  that  the  incomplete  examples  are 
carefully  designed  as  they  have  to  contain  enough  “clues”  in  the  code  to  guide  the  students  in  their  completion. 
In  other  words,  the  germane  cognitive  load  must  not  be  made  too  large.  It  is  suggested  that  this  method  facilitates 
both  automation,  students  having  blueprints  available  for  mapping  to  new  problem  situations,  and  schemata 
acquisition  as  they  are  forced  to  mindfully  abstract  these  from  the  incomplete  programs  (van  Merrienboer  & 
Paas,  1990). 
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In  one  study,  two  groups  of  28  and  29  high-school  students  from  grades  10  to  12  participated  in  a ten 
lesson  programming  course  using  a subset  of  COMALrSO  (van  Merrienboer,  1990b).  One  group,  the 
“generation”  group,  followed  a conventional  approach  to  the  learning  of  programming  that  emphasised  the 
design  and  coding  of  new  programs.  The  other  group,  the  “completion”  group,  followed  an  approach  that 
emphasised  the  modification  and  extension  of  existing  programs.  It  was  found  that  the  completion  group  was 
better  than  the  generation  group  in  constructing  new  programs.  It  was  found  that  the  percentage  of  correctly 
coded  lines  was  greater  and  that  looping  structures  were  more  often  combined  with  correct  variable  initialisation 
before  a loop  together  with  the  correct  use  of  counters  and  accumulators  within  the  loop.  It  would  appear  that  the 
completion  strategy  had  indeed  resulted  in  superior  schemata  formation  for  those  students  within  that  group.  In 
addition,  the  completion  group  used  superior  comments  in  connection  with  the  scope  and  goals  of  the  programs, 
indicating  that  they  had  developed  better  high-level  templates  or  schemata.  It  was  noted  in  the  study  however 
that  both  groups  were  equal  in  their  ability  to  interpret  programs  and  that  this  might  indicate  that  students  in  the 
completion  group  do  not  understand  their  acquired  templates. 

A side  effect  of  the  research  was  also  noted.  The  drop-out  rate  from  the  completion  group  was  found  to 
be  lower  than  for  the  generation  group,  particularly  for  female  students  with  low  prior  knowledge.  It  was 
suggested  that  perhaps  the  generation  of  complete  programs  is  perceived  as  a difficult  and  menacing  task  and 
that  the  completion  strategy  overcomes  this  difficulty. 


CORT  (Code  Restructuring  Tool) 

A tool  has  been  designed  by  the  author  that  is  based  upon  the  above  ideas  of  the  completion  of  part- 
complete  programming  solutions.  Three  methods  of  using  CORT  have  been  identified  that  allow  various  degrees 
of  germane  cognitive  load  to  be  applied  and  the  tool  has  been  utilised  with  students  at  Edith  Cowan  University  in 
Australia,  student  feedback  being  extremely  positive. 

CORT  has  been  used  within  a software  development  unit  in  a Business  degree  course.  This  unit  is  an 
introductory  unit  to  programming  in  Visual  BASIC,  the  majority  of  students  having  no  previous  programming 
experience.  The  unit  runs  over  a period  of  14  weeks,  the  students  having  a two -hour  lecture  and  a one -hour 
computer  laboratory  session  in  which  they  attempted  various  “CORT”  problems.  The  students  would  then  finish 
their  problems  if  necessary  in  their  own  time. 


A Typical  “CORT”  Computer  Laboratory  Session 

At  the  beginning  of  a computer  laboratory  session,  the  students  are  given  a hard  copy  of  a problem 
statement  that  they  have  to  try  and  solve  using  CORT.  The  students  would  then  run  the  CORT  program  and  load 
in  the  CORT  file  corresponding  to  that  particular  problem.  Two  windows  display  a part -complete  solution  to  the 
problem  together  with  possible  lines  to  be  used  as  shown  in  figure  2. 

The  part -complete  solution  on  the  right  is  then  completed  by  moving  certain  lines  from  the  left  hand 
window.  Lines  can  also  be  moved  up  or  down,  and  indented  or  outdented  in  the  right  hand  window.  Some 
problems  may  have  too  many  lines  in  the  left  hand  window,  some  of  those  lines  being  incorrect.  In  addition, 
there  is  a simple  editor  to  allow  the  amendment,  addition,  and  deletion  of  lines  of  code. 

When  the  solution  has  been  completed,  the  code  from  the  right-hand  window  is  copied  to  the  Windows 
clipboard.  Visual  BASIC  is  then  run  and  a file  is  loaded  which  contains  no  programming  code  but  does  contain 
the  necessary  Visual  BASIC  interface  for  the  problem  under  consideration.  The  CORT  code  is  then  pasted  into 
Visual  BASIC  and  the  program  tested.  Use  is  made  of  the  trace  and  debugging  facilities  of  Visual  BASIC,  these 
facilities  providing  an  insight  to  the  workings  of  the  notional  machine. 

If  the  program  does  not  work,  a student  can  switch  back  to  CORT  and  make  any  necessary  changes 
before  retesting  that  code  in  Visual  BASIC. 


Methods  of  using  CORT 

Three  methods  of  using  CORT  were  identified  and  these  different  methods  were  varied  throughout  the  unit  so 
that  different  levels  of  germane  cognitive  load  could  be  applied.  The  first  method  is  to  supply  all  the  missing 
lines  of  code  in  the  left 4iand  window  without  any  extra  lines.  The  second  method  also  supplies  all  the  missing 
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lines  of  code  but  also  includes  extra  "distracter”  lines  of  code  that  are  not  required  to  complete  the  program 
solution.  Finally  the  third  method  is  to  supply  only  some  of  the  lines  of  code  in  the  left-hand  window,  students 
having  to  key -in  some  lines  themselves. 


Figure  2:  CORT  interface 


Preliminary  Feedback  from  Students 

Data  was  collected  on  the  use  of  CORT  during  semester  2,  2001.  Preliminary  results  suggest  the  following. 

• Students  commented  on  the  fact  that  CORT  provides  a starting  point  for  solving  a problem.  Many  suggested 
that  they  would  not  know  how  to  go  about  tackling  some  of  the  problems  if  CORT  was  not  provided. 

• Method  2 of  using  CORT  proved  to  be  most  popular.  Method  1 was  seen  by  students  as  being  too  easy 
while  method  2 required  students  to  think  a great  deal  about  the  problem  solution.  Method  3 was  not  as 
popular  as  that  required  students  to  have  knowledge  of  the  exact  syntax  of  statements. 

• It  was  observed  that  students  engaged  in  significant  reflection  and  higher  order  thinking  when  using  method 

2. 

• Students  commented  that  they  were  encouraged  and  motivated  by  the  fact  that  they  could  get  their  programs 
to  work  in  a relatively  short  time  frame. 


Conclusions 

This  paper  argues  that  cognitive  load  theory  needs  to  be  carefully  taken  into  account  when  designing 
instructional  materials  for  computer  programming.  Programming  has  a very  high  intrinsic  cognitive  load  and 
therefol*e  the  extraneous  cognitive  load  should  be  made  as  low  as  possible.  However  we  still  need  to  ensure  that 
students  “think”  and  are  encouraged  to  create  the  necessary  schemata  in  long  term  memory  and  this  can  be  done 
by  applying  germane  cognitive  load  upon  them.  One  way  of  doing  this  is  to  carefully  design  solutions  to 
problems  that  are  only  part  complete  and  that  require  completion  by  students.  This  approach  has  been  built  into  a 
new  software  tool  called  CORT  (Code  Restructuring  Tool)  (Gamer,  2000)  that  reduces  the  extraneous  cognitive 
load  by  supporting  the  completion  approach  to  learning  programming.  Three  methods  of  CORT  have  been 
identified  and  preliminary  results  suggest  that  method  2,  in  which  all  missing  lines  plus  distracters  are  available 
in  the  left-hand  window,  has  great  potential  for  providing  the  necessary  amount  of  germane  cognitive  load  to 
help  students  develop  the  necessary  programming  schemata. 

References 


1IS§T  COPY  AVAILABLE 


6&3 


Carroli,  W.  (1994).  Using  worked  examples  as  an  instructional  support  in  the  algebra  classroom.  Journal  of  Educational 
Computing  Psychology,  86 y 360-367. 

Chandler,  P.,  & Sweller,  J.  (1991).  Cognitive  load  theory  and  the  format  of  instruction.  Cognition  and  Instruction,  5, 293- 


Chandler,  P.,  & Sweller,  J.  (1996).  Cognitive  Load  while  Learning  to  use  a computer  program.  Applied  Cognitive 
Psychology,  10  y 151-170. 

Chase,  W.  G.,  & Simon,  H.  A.  (1973).  The  Mind's  Eye  in  Chess.  In  W.  G.  Chase  (Ed.),  Visual  Information  Processing.  New 
York:  Academic. 

Gamer,  S.  (2000,  21-24  Nov).  >4  Code  Restructuring  Tool  to  help  Scaffold  Novice  Programmers.  Paper  presented  at  the 
International  Conference  in  Computer  Education,  ICCE2000,  Taipei,  Taiwan. 

Kalyuga,  S.,  Chandler,  P.,  & Sweller,  J.  (1998).  Levels  of  expertise  and  instructional  design.  Human  Factors,  40  y 1-17, 

King,  J.,  J.  Feltham,  et  al.  (1994).  “Novice  Programming  in  High  Schools:  Teacher  Perceptions  and  New  Directions.” 
Australian  Educational  ComputinelSep  1994,):  17-23. 

Lieberman,  H.  (1986).  An  Example  Based  Environment  for  beginning  Programmers.  Instructional  Science,  I4(3)y  277-292. 

Marcus,  N.,  Cooper,  M.,  & Sweller,  J.  (1996).  Understand  instructions.  Journal  of  Educational  Psychology.  88,  49-63. 

Miller,  G.  A.  (1956).  The  Magical  Number  Seven,  Plus  or  Minus  Two:  Some  Limits  on  our  Capacity  to  Process  Information. 
Psychological  RevievJfi3)y  81-97. 

Paas,  F.  G.  W.  C.  (1992).  Training  strategies  for  attaining  transfer  of  problem-solving  skill  in  statistics:  A cognitive  load 
approach.  Journal  of  Educational  Psychology,  84  y 429-434. 

Perkins,  D.  N.,  Schwartz,  S.,  & Simmons,  R.  (1988).  Instructional  Strategies  for  the  Problems  of  Novice  Programmers.  In  R. 
E.  Mayer  (Ed.),  Teaching  and  Learning  Computer  Programming:  Multiple  Research  Perspective{pp.  153-178):  Hillsdale, 
NJ:  Erlbaum. 

Schneider,  W.,  & Shiffrin,  R.  (1977).  Controlled  and  automatic  human  information  processing:  Detection,  search  and 
attention.  Psychological  Review,  84 y 1-66. 

Simon,  H.,  & Gilmartin,  K.  (1973).  A Simulation  of  memory  for  Chess  Positions.  Cognitive  Psychology,  5,  2946. 

Sweller,  J.(1994).  Cognitive  load  theory,  learning  difficulty,  and  instructional  design.  Learning  and  instruction,  4y  295-312. 
Sweller,  J.,  van  Merrienboer,  J.  J.  G.,  & Paas,  F.  G.  W.  C.  (1998).  Cognitive  Architecture  and  Instmctional  Design. 
Educational  psychology  review,  10(3  SEP  01  1998),  25 1-. 

Tindall-Ford,  S.,  Chandler,  P.,  & Sweller,  J.  (1997).  When  two  sensory  modes  are  better  than  one.  Journal  of  Experimental 
Psychology:  Applied,  3,  257-287. 

van  Merrienboer,  J.  J.  G.  (1990a).  Instructional  Strategies  for  Teaching  Computer  Programming:  Interactions  with  the 
Cognitive  Style  Reflection-Impulsivity.yowrna/  of  research  on  computing  in  education,  23  Fall  I990(\)y  45-. 

van  Merrienboer,  J.  J.  G.  (1990b).  Strategies  for  Programming  Instruction  in  High  School:  Program  Completion  vs.  Program 
Generation.  Journal  of  educational  computing  research.,  6(3),  265-. 

van  Merrienboer,  J.  J.  G.,  & Paas,  F.  (1990c).  Automation  and  Schema  Acquisition  in  learning  elementary  computer 
programming.  Computers  in  Human  Behavior(6')y  273-289. 

van  Merrienboer,  J.  J.  G.,  & De  Croock,  M.  B.  M.  (1992).  Strategies  for  computer-based  programming  instruction:  program 
completion  vs.  program  generation.  Journal  of  Educational  Computing  Research,  5(3),  365-394. 

van  Merrienboer,  J.  J.  G.,  Krammer,  H.  P.  M.,  & Maaswinkel,  R.  M.  (1994).  Automating  the  planning  and  construction  of 
programming  assignments  for  teaching  introductory  computer  programming.  In  R.  D.  Tennyson  (Ed.),  Automating 
Instructional  Design,  Development,  and  Delivery  (NATO  ASI  Series  F,  Vol.  119)  (pp.  61  -77):  Springer  Verlag,  Berlin. 


332. 


CardioMeeting:  A Learning  Environment  to  Support  the  Discussion  of 

Scientific  Papers  in  Cardiology 


Ana  Claudia  Garcia^  Kathia  Oliveira^,  Mariella  Monton?,  Carla  Valle^  Viviane  Costal 
Ana  Regina  Rocha^,  Lisia  Rabelo^,  Alvaro  Rabelo  Jr^ 

'Federal  University  of  Rio  de  Janeiro,  Graduate  School  of  Engineering,  Computer  Science  Department, 

P.O.  Box  68511  -Zip  Code  21945-970  Rio  de  Janeiro,  Rio  de  Janeiro,  Brazil 

darocha@xos.ufri.br 

^ Unit  of  Cardiology  and  Cardiovascular  Surgery,  Funda9ao  Bahiana  de  Cardiologia,  Federal  University  of  Bahia, 
Rua  Augusto  Viana  S/N  - Zip  Code  40140-060  Salvador,  Bahia,  Brazil 
arabelo(^,u fha.br.  lmrabelo(^.uol. com.br 

3 

Catholic  University  of  Brasilia,  Graduate  School  of  Computer  Science  Department,  Campus  I (Taguatinga),  EPCT 
Q.S.  7,  lote  1,  Suite:  B-108.9-  Zip  Code  72030-170  Aguas  Claras,  Taguatinga,  DF,  Brazil 

kathia@ucb.br 

Abstract 

In  a Medical  School  the  future  cardiologist’s  education  is  made  being  considered  theoretical 
aspects  of  the  specialty  and  of  the  medical  practice.  In  what  it  refers  to  the  theoretical  aspects 
of  the  teaching,  an  important  resource  is  the  discussion  of  scientific  papers  in  cardiology.  These 
papers  describe  real  researches,  approaching  the  used  methodology,  obtained  results,  statistical 
analysis  and  works  related  to  the  same.  This  work  describes  a support  environment  to  the 
discussion  of  papers  in  cardiology,  called  CardioMeeting.  That  environment  is  integrated  into 
an  education  meta-environment  for  cardiology,  CardioEducar  also  described  in  this  work. 

] , Introduction 

Within  the  medicine  course,  the  cardiology  teaching  takes  place  in  different  moments,  h the  first  years  of  the 
course,  the  student  is  exposed  to  the  methods  diagnoses.  In  the  fifth  and  sixth  year  they  participate  in  medical  aid  in 
the  internship  over  different  areas  in  a hospital. 

Besides  that,  a fundamental  moment  in  the  training  of  the  future  cardiologist  is  the  medical  residence.  After  two 
years  of  residence  in  a medical  clinic,  the  resident  is  linked  into  a unit  of  cardiology  where  they  develop  health  care 
activities,  receiving  specific  education  in  cardiology.  This  education  and  training  feels  through  patients’  care,  and 
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the  attendance  of  classes  and  of  sessions  where  students,  residents  and  cardiologists  discuss  patients'  cases  and 
current  themes  of  the  specialty.  The  Unit  of  Cardiology  and  Cardiovascular  Surgery  / Fundagao  Bahiana  de 
Cardiologia  (UCCV/FBC)  of  the  Federal  University  of  Bahia  (UFBA)  participates  in  the  cardiologists'  education 
through  the  Medical  Residence,  several  undergraduate  disciplines  of  the  Medical  School,  and  in  the  internship. 
Many  classes  are  for  discussion  of  scientific  papers  previously  distributed  to  the  students. 

With  the  purpose  of  supporting  these  different  aspects  of  cardiology  training,  we  have  launched  a project  - 
CardioEducar  - whose  purpose  is  to  develop  a learning  environment  based  on  the  Internet.  CardioEducar,  an 
educational  meta- environment,  offers  an  integrated  environment  where  professors,  cardiologists,  physicians  and 
students  have  access  to  several  educational  environments.  As  a meta-environment  CardioEducar  comprises  several 
working  environments  for  cardiology  education. 

Within  these  environments  of  CardioEducar  we  have  a set  of  environments  we  call  Learning  Environments  that  aim 
to  support  the  different  meetings  where  cardiologists,  residents  and  medical  students  participate  in  order  to  teach  and 
learn. 

One  of  the  most  important  aspects  of  medical  students’  education  deals  with  reading  and  discussion  of  scientific 
papers.  In  our  Medical  School  this  education  is  performed  through  specific  meetings  with  the  following  objectives: 
(i)  to  teach  medical  students  how  to  read  and  to  criticize  cardiology  papers;  (ii)  to  teach  the  students  the  information 
they  should  look  for  as  a justification  (evidence)  for  their  research  and  medical  decisions;  (iii)  to  show  the  students 
how  to  co-relate  results  from  different  works  and  to  relate  them  to  medical  practice.  This  kind  of  meeting  takes  place 
every  week  at  the  University  Hospital  and  it  was  ministered  without  technological  support.  The  students  were 
supposed  to  read  the  papers  previously  to  a classroom  meeting,  to  take  notes  and  to  prepare  some  questions  for 
discussion.  During  the  meeting,  one  of  the  students  makes  a summary  of  the  paper  and  the  professor  leads  the 
discussion  and  answers  the  queries.  Usually,  the  professor  relates  the  paper  under  discussion  with  other  papers  and 
researches  results  already  discussed  in  previous  meetings.  However,  there  was  no  registration  of  the  whole 
discussion  and  the  students  of  subsequent  groups  cannot  benefit  from  discussions  already  accomplished. 

This  was  the  motivation  for  CardioMeeting,  an  Internet  based  environment  whose  main  goal  is  to  support  medical 
education  on  this  specific  meeting.  In  the  following  sections  this  environment  will  be  described. 

2.  CardioMeeting 

The  main  goal  of  the  CardioMeeting  system  is  to  provide  a technology  support  to  the  discussion  session,  since  the 
planning  of  the  discussion  to  the  classroom  meeting.  It  is  composed  of  four  main  environments:  the  Authoring 
Environment,  the  Discussion  Environment,  the  Meeting  Environment  and  the  Tutorial  Environment  (Figure  1). 
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Figure  1:  The  homepage  of  CardioMeeting  with  the  four  environments 

The  Authoring  Environment  supports  a professor  (the  coordinator  of  a paper  discussion  meeting)  in  the 
organization  of  a discussion  meeting  It  offers  facilities  for  introducing  data  of  a single  paper  or  a theme  that 
involves  more  than  one  paper  related  to  a subject.  These  data  concerns  in  indicate  the  papers  (by  its  reference)  and  to 
prepare  some  questions  for  discussion.  The  coordinator  may  also  assign  a student  to  use  this  environment  to  provide 
more  details  about  the  papers  to  other  participants  (for  example,  the  abstract).  Figure  2 shows  a screen  of  the 
Authoring  Environment. 
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Figure  2:  Authoring  Environment:  Sessions’  Planning 
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The  Discussion  Environment  monitors  an  asynchronous  discussion  among  the  students  and  the  professor  for  a 
certain  period  of  time  as  a preparation  for  the  classroom  meeting.  This  discussion  may  involve  the  questions  posed 
by  the  professor,  doubts,  and  debate  about  the  limitations  of  the  work  presented  in  the  paper,  comments  about  the 
methodology  and  indication  of  related  works.  The  discussions  are  asynchronous  to  provide  more  flexibility  for  the 
students  to  use  the  environment  according  to  their  possibilities  and  preferences.  Figure  3 shows  a screen  of  the 
Discussion  Environment. 
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Figure  3:  Preparation  for  Discussion  Environment:  methods  and  Methodologies 

The  Meeting  Environment  supports  the  classroom  meeting,  which  aims  to  synthesize  the  discussion  that  took  place 
during  the  preparation  stage,  and  to  reach  a conclusion  about  the  paper  or  the  theme  proposed.  In  this  sense,  this 
environment  permits  to  register  the  final  analysis  of  the  discussion  and  also  to  retrieve  information  of  past 
discussions.  Figure  4 shows  a screen  of  this  environment 
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Figure  4:  Discussion  Environment:  Final  Analysis  of  a paper 
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The  Tutorial  Environment  organized  in  html  pages  supports  medical  students'  education  giving  directions  on  how 
to  read  medical  papers  and  is  specifically  concerned  on  topics  related  to  the  material  and  methods  used  in  the 
research  being  described  in  a the  paper. 

3.  Conclusion 

CardioMeeting  was  developed  by  a multidisciplinary  team  of  cardiologist,  and  computer  science  specialists,  as  part 
of  a collaborative  project  (CardioEducar)  sponsored  by  the  Brazilian  government.  Other  Learning  Environments 
developed  and  integrated  to  the  meta- environment  CardioEducar  are  already  in  use  with  great  acceptance  (Gama  et 
al.,  1997),  (Rocha  et  al.,  2000),  (Rabelo  et  al.,  2001) 

All  the  environments  were  implemented  using  ASP  (Active  Server  Pages)  as  the  web  programming  language.  The 
cases  are  stored  in  a Microsoft  SQL  Server  Database.  CardioMeeting  was  developed  according  to  a software  process 
with  an  evolutionary  approach,  specifically  defined  for  the  CardioEducar  project.  A first  prototype  was  built  and 
after  its  evaluation  by  faculty  members,  cardiologists  and  students  we  have  implemented  the  first  operational  version 
that  is  now  in  use  in  our  University  Hospital. 

The  utilization  of  the  CardioMeeting  showed  us  an  increase  of  the  participation  by  the  students.  The  promotion  of  a 
computer-based  discussion  demonstrated  more  acceptances  for  some  people  than  the  one  face-to-face.  This  aspect 
encouraged  some  of  students  to  raise  more  questions  before  the  classroom  meeting.  This  environment  also  permitted 
some  physicians  of  the  institution  to  participate  of  the  discussion,  in  case  of  they  were  not  able  to  be  present  on  the 
classroom  meeting. 
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“What  You  Do  (in  teams)  Speaks  Louder  than  What  You  Say”:  Establishing  Rapport  in 
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It  is  ironic  that  often  the  biggest  problems  computer  information  system  students 
encounter  in  graduate  school  is  not  with  their  software  development  projects.  This  paper 
addresses  one  of  the  biggest  problems  that  computer  information  system  students 
encounter. . .how  to  work  with  other  members  of  a group  on  a project.  Research  was 
conducted  over  eighteen  months  with  two  instructors,  one  from  the  computer  information 
system  department,  and  one  from  the  instructional  technology  department  of  a public 
university  located  in  the  southwestern  United  States.  This  is  an  ethnographic  research 
project,  with  qualitative  methodology.  The  authors  chronicled  the  events  of  each  team 
and  analyzed  the  events  and  results  of  each  group  studied  tlu'oughout  that  period.  The 
results  of  the  study  can  contribute  to  further  research  on  how  collaborative  teamwork  can 
facilitate  better  understanding  and  comprehension  of  the  requirements  for  graduates  of 
computer  information  systems  graduate  programs. 

The  two  instructors  decided  to  monitor  graduate  students  in  the  computer 
information  systems  capstone  graduate  projects.  These  students  must  complete  a major 
software  program  development  project  in  lieu  of  a masters’  thesis,  and  they  must  do  so 
while  working  in  teams,  much  like  they  will  do  in  the  realm  of  commerce  after  they 
graduate. 

The  instructional  technology  instmctor  served  on  each  of  the  four  teams  over  the 
eighteen-month  period,  acting  as  a mentor  to  a capstone  group  each  semester,  and 
participating  in  the  dialogue  and  development  processes  of  the  project.  Both  instructors 
learned  that,  despite  the  programming  and  management  information  system  classes  taken 
by  the  students,  the  actual  integration  of  what  the  students’  studied  often  did  not  relate  to 
what  they  were  doing  in  these  “social”  development  team  projects.  The  instructors 
observed  that  the  computer  information  systems  students  (CIS)  often  entered  the 
collaborative  team  environment  with  a preconception  that  their  knowledge  would  be 
received  without  question,  as  a “top  down”  event.  Seldom  did  students  enter  the  group 
with  an  open-minded,  receptive  mind  set.  It  quickly  became  clear  to  the  students, 
however,  that  to  survive  the  group  and  emerge  with  a product  for  the  project,  a shift 
would  need  to  occur  in  the  minds  of  the  CIS  students  with  respect  to  actually  applying 
what  they  learned  to  the  social  context  of  software  development  teams.  At  the  end  of  each 
semester,  students  would  somberly  admit  that  there  was  much  more  to  working  in 
software  teams  than  simply  developing  software  in  the  protected  environment  of 
university  labs. 
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The  two  instructors  would  meet  several  times  a month  and  compare  notes  as  to 
how  the  students  were  progressing  and  whether  they  were  working  satisfactorily  with 
other  group  members.  In  one  instance,  over  a summer  semester,  the  group  actually  fell 
apart  due  to  the  inability  of  the  members  to  coordinate  their  schedules.  In  this  instance  all 
of  the  group  members’  efforts  were  fiiistrated  because  they  were  not  able  to  meet 
together  on  an  ongoing  basis.  As  a result  their  project  suffered  and  so  did  their  grades. 
The  instructional  technology  researcher  chronicled  the  discussions  and  decision-making 
processes  of  each  group,  then,  at  the  end  of  the  semester,  convened  with  the  CIS 
instructor  to  assess  what  worked  successfully  versus  what  could  be  done  better  with  the 
next  crop  of  students  the  next  semester. 

What  the  researchers  discovered  was  that:  first,  each  group  dynamic  was  truly 
unique  and  the  products  created  were  unique,  even  though  the  same  assignment  was 
given  to  a new  group  at  the  start  of  the  new  semester;  second,  the  more  flexible  and 
willing  the  CIS  students  were  to  work  with  other  team  members  often  determined  how 
much  was  accomplished  on  the  project  and  how  quickly  it  was  done;  third,  when 
collaboration  did  not  emanate  from  members  in  the  team  context,  less  work  on  the  project 
was  accomplished;  when  a CIS  student  had  a positive  attitude  and  was  willing  to  be 
flexible  about  discussing  how  the  programs  could  be  created,  it  seemed  that  she/he 
enjoyed  better  rapport  with  the  group  and  the  group  also  seemed  to  accomplish  more  on 
the  project. 

By  working  together  and  demonstrating  that  collaboration  worked  between  the 
two  instructors  from  different  departments,  the  students  had  an  example  of  how  people 
from  different  environments  can  come  together  and  work  toward  a common  goal.  This 
point  was  missed  at  times  by  the  students,  but  by  the  end  of  the  semester  when  they  were 
accountable  for  their  grades,  they  quickly  figured  out  how  to  work  collaboratively,  as 
best  they  could  under  the  circumstances.  The  two  instructors  also  presented  a united  front 
when  reviewing  the  projects  for  completeness  and  for  the  decision  of  the  students’  final 
grades.  The  students  emerged  from  the  experience  with  knowledge  gained  from  the 
collaborative  process,  both  in  their  collaborations  with  their  development  team,  and  with 
the  development  team  instructors. 

Software  engineering  programs  have  utilized  the  collaborative  team  approach  for 
years,  but  computer  information  systems  programs  are  now  beginning  to  understand  the 
importance  of  collaborative  modeling  for  their  students.  Collaboration  in  development 
teams  could  also  offer  insights  to  other  academic  disciplines,  like  Instructional 
Technology,  because  each  team  member  brings  a unique  perspective  as  well  as  expertise 
to  the  team  project. 

Future  studies  of  the  role  of  collaboration  and  team-building  in  postsecondary 
environments,  particularly  in  the  computer  science  and  instructional  technology  areas, 
could  greatly  contribute  to  the  body  of  knowledge  on  mentoring  and  collaboration  for 
computer  information  systems’  studies. 
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Today,  the  Internet  is  a part  of  a new  educational  culture  with  its  own  unique  characteristics.  Online  education  is 
attempting  to  fill  a niche  in  the  changing  the  nature  of  education  around  the  globe  specifically  in  the  area  of  higher 
education.  Consequently,  many  traditional  educators  are  either  considering  or  making  the  transition  to  the  online 
medium.  Educators  are  coming  under  increasing  pressure  to  adapt  their  teaching  to  accommodate  the  new 
technologies. 

This  study  focuses  on  a single,  but  arguably  the  most  important  area  - teaching  via  the  Internet.  Towards  this  focus, 
faculty  members  from  various  universities  and  departments  were  interviewed  to  look  at  the  many  different  ways  the 
Internet  is  being  used  for  teaching,  and  faculty  experiences  in  that  regard.  The  lessons  they  have  learned  over  the 
years  provide  valuable  insight  into  new,  more  effective  ways  of  teaching  students  in  the  online  environment.  The 
term  ’Online  teaching’  here  means  teaching  and  learning  that  takes  place  over  a computer  network  of  some  kind  (e.g. 
an  Intranet  or  the  Internet)  in  which  interaction  between  people  is  an  important  form  of  support  for  the  learning 
process  (Goodyear,  et.  al.  2001). 

This  study  is  an  investigation  of  the  perceptions  of  higher  education  instructors  about  teaching  courses  via  the 
Internet.  Qualitative  methodology  was  used  to  determine  issues  important  from  the  perspective  of  the  participants 
(Bogdan  & Biklen  1992).  Analysis  was  conducted  using  the  grounded  theory  methodology,  also  referred  to  as 
constant -comparative  method  (Glaser  & Strauss,  1967).  Through  this  study,  several  themes  emerged  - Time, 
Instructional  tools,  and  student-faculty  relations.  The  findings  and  possible  implications  in  teaching  will  be 
presented  at  the  Conference. 

References 

Bogdan,  R.  & Biklen,  S.  K.  (1992).  Qualitative  research  for  education:  An  introduction  to  theory  and  methods. 
Boston:  Allyn  and  Bacon. 

Collison,  G.,  Elbaum,  B.,  Haavind,  S.,  & Tinker,  R.  (2000).  Facilitating  online  learning:  Effective  strategies  for 
moderators.  Madison,  WI:  Atwood  Publishing. 

Cyrs,  T.  (1997).  Competence  in  teaching  at  a dis  tance.  New  Directions  for  Teaching  and  Learning,  77,  15-18. 

Drucker,  P.  (2000,  May).  Putting  more  now  in  the  Internet.  Forbes  Global,  May  1 5,  2000. 

Glaser,  B.G.,  & Strauss,  A.L.  (1967).  The  discovery  of  grounded  theory:  Strategies  for  qualitative  research. 

Chicago:  Aldine. 

Goodyear,  P.,  Salmon,  G.,  Spector,  M.,  Steeples,  C.,  & Tickner,  S.  (2001)  Competencies  for  online  teaching. 
Educational  Technology  Research  & Development  49  (1),  65-72. 

Kearsely,  G.  (2000).  Online  education:  Learning  and  teaching  in  cyberspace.  Belmont,  CA:  Wadsworth. 

Salmon,  G.  (2000).  E^moderating:  The  key  to  teaching  and  learning  online.  London:  Kogan  Page. 


703 


A Web-Based  Tandem  System  for  Language  Learning  in  South  Tirol 


Johann  Gamper 
Faculty  of  Computer  Science 
Free  University  of  Bolzano 
Bolzano,  Italy 
johann.gamper@unibz.it 


Judith  Knapp 

Information  & Communication  Technology 
European  Academy  of  Bolzano 
Bolzano,  Italy 
judith.knapp@eurac.edu 


Abstract:  In  this  paper  we  introduce  a Web-based  tandem  system  which  helps  to 
establish  language  learning  partnerships  between  people  who  prepare  for  the  exam  in 
bilingualism  in  South  Tyrol.  The  system  will  be  integrated  as  a module  into  the  ELDIT 
language  learning  system  which  already  includes  a small  text  corpus  and  a learner’s 
dictionary.  The  new  module  supports  the  formation  of  partnerships  and  the  exchange 
of  information  including  exercises  between  learning  partner  for  the  purpose  of  error 
correction  and  feedback.  Adaptation  technologies  are  applied  in  order  to  support 
autonomous  use  of  the  program. 


Motivation 

South  Tyrol  is  a bilingual  province  located  in  the  north-east  of  Italy  with  two  official  languages,  German  and 
Italian.  Due  to  a rather  strict  geographical  and  social  separation  of  the  two  ethnic  groups,  only  a few  people  con- 
sider themselves  truly  bilingual  [Cavagnoli  & Nardin,  1999].  Passing  the  so-called  examination  of  bilingualism, 
however,  is  a prerequisite  for  employment  in  the  public  sector. 

Learning  a language  via  tandem  means  that  two  learners  with  different  native  languages  try  to  learn  the  lan- 
guage of  their  partner  by  having  controlled  conversations  with  each  others.  Such  contacts  and  conversations  can 
also  occur  via  electronic  media  such  as  e-mail,  chat,  or  video  conferencing.  These  media  offer  new  possibilities 
to  overcome  problems  like  spatial  and  temporal  distances. 


The  eTandem  System 

To  overcome  some  of  the  above  mentioned  problems  and  to  facilitate  language  learing  in  South  Tyrol,  we 
are  currently  developing  a Web-based  tandem  system,  called  eTandem.  The  eTandem  system  will  be  inte- 
grated into  the  ELDIT  language  learning  system  which  already  contains  a learner’s  dictionary  and  a text  cor- 
pus [Abel  & Weber,  2000,  Gamper  & Knapp,  2002].  In  this  way,  eTandem  provides  access  to  the  preparation 
material  for  the  exam  in  bilingualism  and  provides  opportunities  to  establish  language  learning  partnerships 
between  the  German  and  Italian  population. 

The  eTandem  system  can  be  accessed  via  the  following  pages: 

• Advertisement  page:  This  page  serves  either  to  advertise  interest  in  a learning  partnership  or  to  contact 
somebody  else  who  already  announced  interest  in  a partnership.  Furthermore,  information  about  the 
preferred  date  of  the  exam  or  personal  learning  interests  can  be  indicated. 

• Text  page:  On  this  page  the  user  can  practice  a text.  Every  word  in  a text  unit  is  linked  to  the  corresponding 
ELDIT  dictionary  entry.  This  facilitates  to  check  unknown  words  or  to  lookup  for  words  which  the  learner 
may  need  to  answer  the  queries  in  the  target  language.  Each  text  contains  a couple  of  questions  which 
the  learner  has  to  answer,  producing  in  this  way  complete  sentences  in  the  target  language.  As  long  as  no 
partnership  is  established,  the  texts  can  be  saved  as  drafts  and  sent  later. 


704 


• Mail  page:  This  page  provides  a communication  forum  between  learning  partners.  After  working  through 
a text  unit,  the  learner  can  send  the  anwers  to  the  partner  for  correction.  Similar,  the  messages  from  the 
learning  partner  arrive  on  this  page.  Furthermore,  short  messages  such  as  comments  on  the  work  done  or 
organisational  aspects  can  be  exchanged  in  this  forum. 

• User  profile  page:  On  this  page  each  learner  can  inspect  and  change  the  parameter  values  set  in  the  user 
model.  Evaluations  of  adaptive  systems  revealed  that  it  is  extremely  important  that  the  user  has  control 
about  the  system.  Therefore,  we  want  to  make  the  modeling  and  adaptation  process  as  transparent  as 
possible. 

• Help  page:  This  page  eplains  the  exam  in  bilingualism  and  the  usage  of  the  program.  Depending  on  the 
learner’s  familiarity  with  the  use  of  computers  the  online  help  is  more  or  less  detailed.  Moreover,  for  a 
novice  a number  of  direct  links  to  the  corresponding  help  sections  are  provided. 

Unlike  other  tandem  systems,  eTandem  is  intended  to  be  used  fully  autonomously  via  the  WWW.  Therefore, 
a variety  of  users  will  access  it.  The  eTandem  system  adopts  adaptive  technologies  in  order  to  cope  with  the 
problem  of  user  diversity.  A user  model  stores  information  about  the  users  and  his/her  interaction  with  the 
system,  which  allows  the  system  to  adapt  to  the  individual  learner.  When  accessing  eTandem  for  the  first  time,  an 
introduction  page  will  be  shown  and  substituted  by  a link  later  on.  Advertisements  and  the  exam  level  are  adapted 
to  the  indidvidual  learner,  and  a “What  shall  I do  next?”  suggests  the  best  next  step  to  be  carried  out.  Adaptive 
features  are  also  included  in  the  other  modules  of  ELDIT  and  are  described  in  [Gamper  & Knapp,  2002]. 

Discussion 

Computer-assisted  language  learning  is  a widely  explored  research  field.  Language  learning  via  e-mail  conversa- 
tions has  been  explored  in  several  successful  projects.  Since  1994,  universities  and  other  educational  institutions 
from  many  different  countries  work  together  in  the  “International  Tandem  Network”  in  order  to  help  their  stu- 
dents to  learn  languages  via  tandem,  primarily  via  the  Internet  [Brammerts,  1994].  “Number2.com”  is  an  Internet 
site  which  provides  online  preparation  for  college  admission  and  placement  tests.  A coaching  system  is  included. 
Teachers  and  parents  can  register  as  coaches.  Their  job  is  to  provide  encouragement  and  to  help  students  to  set 
goals.  Coach  and  student  can  also  work  together  on  the  tutorials  and  practice  sessions  [Crespi  et  al.,  1995]. 

According  to  our  opinion,  the  eTandem  system  will  provide  a great  opportunity  for  the  population  in  South 
Tyrol  to  learn  and  improve  the  second  language.  As  computer-assisted  langauge  learning  has  been  proven  to 
be  motivating  for  langauge  learners,  especially  for  young  peolple,  this  could  also  be  a contribution  to  over- 
come geographical  and  social  separation  of  the  two  ethnic  groups.  The  eTandem  system  is  currently  under 
development  together  with  the  other  modules  of  ELDIT.  A preliminary  version  of  ELDIT  can  be  accessed  at 
http://www.eurac.edu/Eldit.  A systematic  evaluation  of  the  overall  system  and  in  particular  of  the  eTandem 
module  remains  future  work. 
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Abstract:  Based  on  the  research  of  [Stil  1998]  [Maurer  1992]  [Lennon  1994]  [Elliot  1995] 
[Colboum  1995]  [Reeves  1997]  [Lugner  1997],  the  application  of  an  expert  system  as  an 
interactive  multimedia  knowledge  module  with  explanation  features  named  Virtual  Tutor  (VT) 
was  developed.  For  students,  the  individual  dialogue  based  session  with  the  VT,  provides  the 
possibility  to  apply  the  knowledge  acquired  in  combination  with  indirect  assessment.  The  VT 
was  further  embedded  in  the  Web-based  on-line  learning  environment.  The  reported  research 
merges  qualities  of  an  expert  system  with  advantages  of  multimedia,  thus  creating  a variety  of 
innovative  ways  of  knowledge  mediation. 

Based  on  the  evaluation  results  the  conclusion  can  be  drawn  that  dialog  based  system  meets 
more  accurate  the  requirements  of  the  students  in  case  of  solving  problems.  Problem  solution 
can  be  found  easier  and  faster  than  using  conventional  literature.  The  possibility  of  accessing 
additional  information  related  to  the  subject  domain  and  explanation  on  request,  VT  is  an 
interesting  and  different  tool  for  knowledge  transfer. 


Introduction 

Higher  education  has  entered  a transition  from  the  Teaching  Paradigm  to  the  Learning  Paradigm. 
Emerging  from  this  process  are  powerful  new  teaching  styles  founded  on  principles  of  active-learning 
and  improved  insights  on  the  cognitive  development  of  learning.  As  outlined  in  [Buckley  1999], 
learning  environments  that  exploit  interactive  multimedia  are  of  special  interest.  The  educational 
potential  of  this  technology  closely  parallels  the  pedagogical  goals  of  the  Learning  Paradigm. 
According  to  [Buckley]  a simple  pedagogical  set  of  features  that  can  foster  transition  to  the  Learning 
Paradigm  is  as  follows:  (1.)  Interactivity  fosters  active  learning,  (2)  The  sensory -rich  nature  of 
technology  facilitates  the  engagement  of  additional  powerful  cognitive  processes,  and  (3)  Integration  of 
assesment  tools  into  the  environment  can  provide  students  with  feedback  and  encouragement  etc. 

"From  a pedagogic  point  of  view,  learning  requires  "a  deep  understanding  of  the  subject  content^* 
through  a cognitive  re- elaboration  of  the  information"  [Colboum  95].  Inspired  by  the  work  of 
Mark&Greer  1995,  Chan  1996,  Youngblut  1994,  Nunez  concludes  [Nunez  1999]  as  follows: 
"Knowledge -based  learning  environments  have  offered  significant  potential  for  fundamentally 
changing  the  educational  process.  " 

In  general,  classical  Web-based  tutoring  systems  rely  on  memory  and  do  not  take  into  account  the 
cognitive  information  treatment  process.  In  order  to  overcome  this  problem,  the  intelligent  tutoring 
systems  came  into  being,  as  described  in  [Han  1999].  The  use  of  multimedia  objects  in  educational 
systems  can  enhance  their  efficacy  to  a great  extent  in  facilitating  cognitive  skills  besides  other 
components  of  domain  competence  as  shown  in  research  by  Kinshuk  [Kinshuk  1999]. 


Idea  of  the  Virtual  Tutor 

The  aim  of  the  research  presented  in  this  paper  was  to  apply  an  expert  system  as  a knowledge 
module  within  the  on-line  learning  environment.  There  are  many  benefits  for  the  students  within  the 
on-line  learning  environment  independent  from  the  course  content  aid  style,  like  e.g.  tools  for 
asynchronous  and  synchronous  communication  and  collaboration,  search  facility  within  a background 
library,  a progress  indicator  monitoring  the  learning  success  etc.  The  application  of  ES  makes  it 
possible  to  use  different  knowledge  representation  and  explanation  approach.  The  individual  session 
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with  the  VT  provides  also  the  possibility  for  students  to  apply  the  knowledge  acquired  in  combination 
with  indirect  assessment. 

With  interactive  sessions  with  the  VT,  various  students’  activities  could  be  increased  as  follows: 
raising  the  learning  motivation,  research  work,  stimulating  the  creativity  by  carrying  out  analysis  and 
synthesis,  searching  for  solutions,  interdisciplinary  learning. 

VT  system  requirements  are  as  follows:  (1)  the  interaction  and  the  communication  should  be 
performed  by  a question -answer  dialog,  (2)  the  system  has  to  explain  why  a particular  question  is  asked 
(i.e.  why  a certain  fact  is  questioned)  and  the  influence  of  the  fact  on  the  problem  solution,  (3)  the  VT 
must  be  capable  of  explaining  reasons  for  the  suggestion  of  a solution  or  a particular  ranking  of 
possible  solutions,  (4)  descriptions  and  illustrations  of  items  concerning  the  problem  domain  have  to  be 
provided  by  hyperlinks  connecting  the  multimedia  infonnation  within  the  static  and  dynamic 
background  library,  (5)  the  system  has  to  run  at  least  under  Netscape  4.x  and  MSIE  5.x,  (6)  the  VT  has 
to  be  designed  and  implemented  considering  the  possibility  of  the  easy  change  of  the  problem  domain 
as  well  as  easy  enhancement  within  the  same  problem  domain. 


The  Concept  and  the  Architecture  of  the  VT  System 

Following  the  requirements  stated  so  far,  we  have  decided  to  combine  the  properties  of  expert 
systems  with  the  advantages  of  Web -based  information  transforming  and  multimedia  information 
structures.  Because  of  this,  as  well  as  following  also  the  ideas  of  platform  independence  and  usage 
without  installing  the  VT,  the  system  was  implemented  in  Java.  The  VT  is  built  on  JESS  (Java  Expert 
System  Shell),  a Java  implementation  based  on  Clips.  [JESS]  This  makes  it  possible  to  provide  the  VT 
system  as  a Java  applet  within  the  GENTLE-WBT  environment.  It  must  be  mentioned  that  the  entire 
system  (VT  system,  UI  and  the  JESS  system)  has  to  be  loaded  by  the  client.  To  reduce  network  loads,  a 
future  implementation  will  be  done  by  splitting  the  VT  into  a client-server  architecture. 

Within  the  following  sections,  the  system  and  the  working  principle  are  discussed. 


The  Working  Principle 

The  aims  of  the  VT  system  within  the  course  'TCnowledge  processing"  are:  (1)  To  provide 
Decision  support  of  using  proper  AI  techniques  related  to  a particular  problem.  The  first 
implementation  takes  into  account  expert  systems,  neuronal  nets,  fuzzy  logic  and  neuro  fuzzy.  (2)  An 
interactive  system  has  to  be  provided.  (3)  The  suggestion  has  to  be  explained  by  the  system.  (4) 
Additional  descriptions  and  illustrations  have  to  be  provided  by  a static  and  dynamic  background 
library  of  multimedia  information. 

To  meet  the  aims  stated  above,  the  VT  - in  generalexpert  systems  - has  to  comprise  three  types  of 
knowledge:  Problem  Domain  Knowledge  (PDK),  the  Description  Knowledge  (DK),  and  the  Subject 
Domain  Knowledge  (SDK).  Within  the  PDK  the  system  has  to  know  about  various  problem  types  that 
can  be  covered  by  the  four  AI  systems,  and  thus  to  provide  the  proper  questions  (collect  information 
about  facts)  for  the  user  interaction.  The  DK  is  used  to  describe  the  results  (suggestions  of  AI  systems) 
and  influences  of  facts  on  the  particular  problem.  The  SDK  is  responsfcle  for  reasoning  and  building 
suggestions.  Furthermore,  it  controls  the  sequence  of  the  questions  related  to  already  known  facts[Pivec 
1997]. 

The  procedure  of  the  working  principle  can  be  subdivided  into  three  tasks  (see  also  Figure  1):  (1) 
Dialog  between  the  user  and  the  VT  system.  The  student  has  to  answer  questions  about  the  problem 
domain.  (2)  Internal  representation  of  the  problem.  Related  to  the  problem  domain,  an  internal  image  of 
a particular  problem  is  built  by  a set  of  attributes.  For  the  first  implementation  the  following  properties 
are  used  to  describe  the  AI  problem  domain:  modularity,  extensibility,  mathematics  model,  expert 
knowledge,  training  sets,  explanation,  adaptive  system,  uncertainty,  black  box.  The  PDK  and  SDK 
components  control  the  sequence  of  questions  and  change  the  internal  representation.  Weightiness  of 
the  attributes  are  changed  (increase,  equal  or  decrease)  depending  on  the  questions  and  the  answers. 
That  means  that  the  properties  are  changed  with  respect  to  the  facts  questioned  and  the  answers  given 
by  the  students.  (3)  Suggestion  of  possible  solutions.  Based  on  the  internal  representation  of  the 
problem,  the  SDK  determines  a ranked  output  of  the  AI  techniques.  It  is  to  be  mentioned  that  it  is  easy 
to  extend  (e.g.  enlarge  the  subject  domain)  or  change  the  knowledge  (e.g.  to  use  the  system  for  another 
course  placed  in  another  subject  domain),  because  of  the  use  of  a knowledge -based  system  (expert 
system  JESS).  This  makes  it  possible  to  change  and  maintains  knowledge  without  changing  the  entire 
program. 
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The  Architecture 


The  architecture  of  the  VT  system  is  illustrated  in  Figure  2.  The  JESS  system  contains  three  types 
of  knowledge  (PDK,  DK,  SDK)  and  manages  the  internal  representation  of  the  problem  as  well  as 
processes  the  input  and  output  stream  of  information.  The  parser  within  the  output  stream  handles 
hyperlink  information  for  the  user  interface  (UI)  of  the  applet.  The  hyperlinks  are  references  to  the 
static  and  the  dynamic  background  library.  For  further  work,  it  is  planed  that  the  pre- knowledge  of  the 
students  is  taken  into  account.  This  will  make  it  possible  to  provide  background  information  related  to 
the  current  knowledge  level  of  each  particular  student.  A novice  in  AI  will  receive  basic  information, 
and  an  advanced  student  will  get  much  deeper  information.  The  UI  represents  the  dialog  between  the 
students  and  the  VT  system. 

As  already  stated,  the  VT  system  is  integrated  in  the  GENTLE-WBT  [GENTLE]  learning 
environment,  as  illustrated  in  Figure  3.  The  learning  environment  allows  compiling  courseware  (lecture 
notes,  exercises,  etc.)  and  provides  also  diverse  communication  features  (asynchronous  discussion 
forum,  chat,  etc).  The  decision  was  made  to  place  the  static  background  library  into  the  GENTLE-WBT 
system.  The  static  background  library  contains  descriptions,  definitions  and  illustrations  concerning 
items  of  the  Al  techniques  handled  by  the  VT  system.  The  information  is  prepared  in  multimedia 
modules  using  textual  information,  graphics,  video  files,  and  flash  presentations.  The  later  presentation 
form  is  used  for  an  interactive  information  transfer. 

Figure  3 also  shows  the  interaction  between  the  VT  system  and  the  lecture  notes.  For  the  first 
implementation  of  VT  the  existing  lecture  notes  of  “Knowledge  processing”  were  applied.  Hints  at 
proper  pieces  of  information  refer  to  the  VT  system.  Vice  versa,  explanations  by  the  VT  system  also 
provide  hyperlinks  to  subjects  of  Al  techniques  within  the  lecture  notes.  Students  are  enabled  to  work 
with  the  VT  system  and  can  get  information  or  re-read  topics  of  the  lecture  notes  to  get  familiar  with 
the  relation  and  influences  of  a particular  (given)  problem. 

The  third  part  of  referred  information  provided  by  the  VT  system  is  the  dynamic  background 
library.  A huge  amount  of  useful  information  is  available  on  the  Internet.  The  intention  is  to  collect 
trustworthy  information  from  university  institutes,  research  centres,  online  libraries,  company 
information,  etc.  related  to  the  subject  of  the  course.  One  of  the  advantages  of  including  such  a 
dynamic  background  library  is  that  the  students  are  supplied  with  up-to-date  information.  The  xFIND 
(extended  Framework  for  Information  Discovery)  system,  an  open  source  project  developed  also  at  the 
llCM,  perfectly  meets  the  requirements  for  such  a dynamic  background  library.  The  dynamic 
background  library  has  already  been  used  in  the  course  “Knowledge  processing”  for  more  than  a year. 
The  xFlND  system  is  a highly  scalable  and  distributed  system  for  the  management  of  online  resources 
that  consists  of  three  components,  the  Gatherer,  the  Indexer  and  the  Knowledge  Broker.  The  system 
manages  the  content  of  Internet  resources,  and  in  addition  a broad  spectrum  of  meta  information 
(xFlND  quality  metadata)  about  the  resources.  Thus,  e.g.  the  topics,  target  audience,  quality  level,  etc. 
of  Web  pages  or  entire  structures  of  Web  pages  can  be  managed  [xFlND].  The  VT  system  requests 
information  by  a somewhat  predefined  dynamically  compiled  search  query  related  to  the  topic  or 
subject  of  interest.  It  is  planed  that  the  dynamic  background  library  should  provide  different 
information  related  to  the  student  overall  performance.  Such  a feature  has  been  already  implemented  in 
the  xFlND  system. 


Evaluation  of  the  System  and  Conclusions 

The  evaluation  of  the  VT  was  carried  out  with  students,  as  a part  of  the  practical  work  within  the 
course  Knowledge  processing  at  the  Graz  University  of  Technology.  The  students  were  divided  into 
two  evaluation  groups:  one  group  worked  with  a printed  version  of  lecture  notes,  whereas  the  other 
group  used  VT  to  provide  the  solution  to  the  presented  problem.  The  group  that  worked  with  the  VT 
had  also  access  to  the  background  library.  The  evaluation  was  carried  out  in  parallel  sessions. 

The  evaluation  showed  that  the  application  of  the  VT  is  easy  and  the  on-line  help  is  sufficient. 
The  most  often  applied  system  features  were  as  follows:  (1)  - application  of  the  explanation 
component,  (2)  - inquiring  about  rules.  All  students  that  worked  with  the  VT  solved  the  task  correctly. 
Taking  into  the  consideration  the  students’  comments  outlined  later  in  this  chapter,  the  conclusion  can 
be  drawn  that  when  using  the  VT  students  have  the  possibility  to  interact  with  the  knowledge  in  a 
different  way  as  when  using  only  the  classical  lecture  notes.  This  makes  it  possible  to  provide  different 
points  of  view  of  a problem  domain  and  introduce  more  complex  ways  of  reflection. 
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Figure  1 : Working  principle  of  the  VT  concept  presented  on  the  example  of  the  AI  domain  of  the  course 
"Knowledge  Processing".  Students  have  to  answer  questions  about  facts,  which  describe  the  problem  within  the  A I 
domain.  The  answers  are  used  to  build  an  internal  knowledge  representation.  Based  on  the  internal  representation 
of  the  problem,  the  VT  ranks  suggestions  of  A I systems  to  be  used  for  a particular  problem. 


VIRTUAL  TUTOR  (VT) 


Figure  2:  The  architecture  of  the  VT  system.  The  expert  shell  JESS  manages  the  knowledge  (PDK,  DK,  SDK,  and 
IPR)  of  the  system.  The  parser  handles  the  hyperlink  management  referencing  information  from  the  static  and 
dynamic  background  library.  The  applet  represents  the  interface  to  the  user. 
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Figure  3 : System  integration.  The  VT  system  is  embedded  in  the  GENTLE-WBT  learning  environment.  It  is 
rendered  via  the  GENTLE-WBT  Web  gateway  as  a Java  applet. 

From  students  collected  comments  indicated  that  the  group  that  worked  with  the  lecture  notes  had 
time  problems.  They  expressed  that  within  the  time  given  it  was  very  difficult  to  find  various 
parameters  that  were  necessary  to  solve  the  task.  In  general,  the  comments  reflected  that  because  of  the 
time  limit,  the  knowledge  processing  was  insufficient  and  relatively  shallow. 

Based  on  the  evaluation  carried  out,  the  conclusions  can  be  drawn  as  follows.  Dialog  based  system 
meets  more  accurate  the  requirements  of  the  students  in  case  of  solving  problems.  VT  is  an  interesting 
and  different  way  for  knowledge  transfer  that  provides  help  by  problem  solving.  Problem  solution  can 
be  found  easier  and  faster  than  using  conventional  literature.  VT  provides  explanation  of  solutions 
suggested  and  makes  it  possible  to  access  additional  information  related  to  the  subject  domain.  Besides 
conventional  lecture,  speeches  and  lecture  notes,  a VT  -based  subject  domain  presentation  proved  to  be 
very  helpful.  To  improve  the  knowledge  transfer  a range  of  exercises  that  have  to  be  solved  by  the  VT 
should  be  provided  to  students. 
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Abstract;  This  paper  argues  for  changing  design  priorities  in  multimedia  based  e-leaming 
design.  Current  multimedia  research  is  media  oriented,  questioning  which  media  to  select  for 
presenting  information  in  general.  The  focus  on  media  in  this  kind  of  research  is  converging  at 
the  same  results:  dynamic  media  are  rarely  reported  more  effective  than  static  media.  An 
important  reason  for  the  weak  support  for  dynamic  media  could  be  that  it  often  is  implemented 
for  unsuitable  information  contents.  Multimedia  design  should  start  with  a thorough  analysis 
of  the  information  content.  We  tested  and  found  support  for  the  following  hypothesis  in  two 
experiments:  information  content  and  media  should  match  on  static  and  dynamic  descriptions. 


Introduction 

Information  presentation  with  multimedia  is  a challenge  compared  to  traditional  teaching  and  publishing 
methods.  New  information  technology  offers  easy  access  to  development  tools  for  dynamic  and  visual  ways  of 
presenting  information.  Consequently,  easy  access  to  guidelines  for  how  to  employ  dynamic  media  is  important. 
Successful  multimedia  design  does  not  result  from  the  employment  of  dynamic  media  as  such,  but  depends  on  a 
number  of  factors,  of  which  effective  media  combinations,  relationship  between  media  and  information  content, 
individual  characteristics  of  the  learner  and  implementation  and  design  of  the  presentations  must  be  considered. 
This  paper  addresses  particularly  the  employment  of  dynamic  media  (e.g.  video,  animations,  voice)  and  the  fit 
between  media  and  information  content.  Static  media  (e.g.  pictures,  graphics,  text)  has  a long  tradition,  hence 
recent  research  has  focused  on  new  dynamic  presentation  forms. 

We  hold  that  a central  part  of  information  presentation  is  to  first  analyse  the  information  and  then  select  the 
media  that  fit.  In  order  to  accomplish  this  we  need  descriptions  of  information,  which  are  objective  and  topic 
independent.  Objective  descriptions  must  also  be  reliable,  enabling  different  people  to  identify  the  same 
information  class.  Very  few  such  information  classifications  exist  for  the  e-leaming  context.  Merrill  (1983) 
suggests  four  useful  information  classes,  which  fit  our  criteria:  facts,  concepts,  principles  and  processes. 

Recent  multimedia  research  gives  remarkable  little  attention  to  the  information  content.  Current  multimedia 
research  is  media  oriented,  questioning  first  which  media  to  select  for  presenting  information  in  general,  and 
eventual  with  which  media  combinations.  Common  co-research  factors  are  various  individual  factors  like  ability, 
aptitude  and  experience,  as  well  as  design  and  didactical  aspects.  Experimental  research  in  this  context  often 
employ  similar  design;  information  is  presented  with  selected  media,  learning  performance  is  tested  and  a 
conclusion  related  to  success  of  the  tested  media  is  drawn.  Information  content  is  mostly  not  treated 
systematically.  A discussion  or  meta-description  of  the  presented  information  is  almost  entirely  lacking.  Various 
definitions  of  information  contents  are  employed,  which  makes  a systematic  evaluation  of  the  interaction  between 
media  and  information  content  from  the  different  studies  literally  impossible.  A review  analysing  25  studies 
revealed  that  the  tested  information  contents  or  learning  tasks  belonged  to  seven  different  meta-categories  (i.e. 
cognitive  skill,  short  term  recognition,  procedural  skill,  concept  understanding  or  learning,  rule  learning,  fact 
memorisation,  self  assessment),  of  which  all  studies  tested  different  example  tasks  (Park  & Hopkins,  1993). 

The  focus  on  media  in  this  kind  of  research  seems  to  have  reached  a boundary,  converging  at  the  same 
results:  few  studies  find  dynamic  media  effective,  mostly  dynamic  media  are  reported  equal  or  less  effective  than 
static  media.  Hence,  a new  approach  is  necessary  to  evaluate  how  to  employ  dynamic  media  effectively. 


Relevant  research 

We  suggest  starting  multimedia  design  with  a thorough  analysis  of  the  information  content.  The  applied  media 
should  reflect  the  information  characteristics.  Both  media  and  information  content  can  be  described  in  terms  of 


static  and  dynamic  characteristics.  The  static  and  dynamic  aspects  of  media  are  addressed  when  media  are 
described  as  modalities  of  communication  e.g.  as  visual,  auditory,  verbal  or  non-verbal  coded  information 
(Paechter  1996).  In  the  media  context,  static  vs.  dynamic  refers  to  whether  or  not  we  can  perceive  the  information 
with  a self-chosen  speed.  A representation  is  static  when  it  enables  us  to  decide  how  long  the  information  shall  be 
presented  (e.g.  text,  picture).  A representation  is  dynamic  when  the  information  flow  proceeds  automatically  (e.g. 
voice,  video).  Also  information  can  be  described  in  terms  of  static  and  dynamic  characteristics.  Static  information 
remains  stable  in  its  context  e.g.  states,  descriptions,  concepts,  facts.  Dynamic  information  is  not  stable  over  time, 
but  refers  to  variable  interactions  between  certain  variables  or  factors,  e.g.  processes  which  can  be  described  as  a 
sequence  of  steps  necessary  to  accomplish  some  goal,  solve  a particular  class  of  problems,  or  produce  a product. 

We  inferred  that  an  important  reason  for  the  weak  and  varying  support  for  dynamic  media  in  research  is  that 
dynamic  media  often  is  implemented  and  tested  for  information  contents  for  which  it  is  not  suited.  Although 
dynamic  media  is  often  appreciated  as  a “nice”  feature,  a disharmony  between  information  characteristics  and 
media  representation  may  hinder  optimal  learning.  Studies,  that  tested  various  dynamic  information  contents 
confirm  a bound  between  dynamic  information  and  dynamic  media.  The  tested  information  was  defined  as 
complex,  dynamic  and  spatial,  procedural  texts  (Large,  1994;  Large,  1996),  complex  structural,  functional,  and 
procedural  relationships  among  objects  and  events  (Park,  1992),  Newton’s  law  of  motion  (Rieber,  1990). 

We  tested  the  following  hypothesis  in  two  experiments  (Guttormsen  Schar,  in  review):  Information  content 
and  media  should  match  their  respective  static  and  dynamic  descriptions.  Our  experiments  also  addressed  the 
complex  relationship  between  information,  media  representation  and  test  of  knowledge.  Relevant  results  are: 

• There  is  a close  relation  between  information  content  and  knowledge  acquisition.  Test  of  dynamic 
knowledge  should  reflect  causal,  procedural  and  eventual  interactive  aspects  of  the  information.  Tests  of 
static  knowledge  should  reflect  the  stable  information  aspects. 

• it  is  difficult  tu  acquire  static  informatioii  enibedded  tn  a d)aianiic  context. 

• There  is  a close  relationship  between  information  presentation  and  knowledge  acquisition.  Visual  media 
supports  visual  knowledge.  Verbal  presentations  relate  to  verbal  knowledge. 


Conclusion 

Our  results  show  that  static  and  dynamic  information  qualities  should  guide  media  selection  on  an  initial  level.  A 
simple  key  to  multimedia  design  is  to  first  select  media  according  to  basic  static  and  dynamic  information 
qualities.  Further  differentiated  characteristics  of  the  information  content  should  guide  the  didactical  media 
design  and  set  the  information  in  an  appropriate  learning  context.  An  overview  over  multimedia  in  e-leaming  is 
given  in  Guttormsen  Schar  (2000). 
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Abstract;  An  user  does  not  achieve  optimum  performances  if  he  uses  pre-defmed 
interfaces  that  misfit  with  his  cognitive  abilities.  Our  goal  is  to  build  cognitive 
profiles  of  users  and  adaptation  techniques  of  HCI.  We  briefly  present  cognitive 
indicators  and  styles  that  are  used  in  our  filtering  process  of  the  output  medias.  We 
have  chosen  XML  and  ASP  technologies  to  implement  the  system. 


Introduction 

In  e-leaming  systems,  each  student  is  confronted  to  lessons  and  exercises  which  are  relevant  to  his/her 
needs  and  preferences  i.e.  educational  level,  domain  knowledge  (expert  or  novice)  but  which  do  not  take  into 
account  his/her  abilities  for  assimilation,  memorization,  etc.  The  aim  of  our  project  is  to  build  an  adaptation 
technique  of  multimedia  interfaces  where  cognitive  abilities  are  considered.  This  dynamic  process  tries  to 
estimate  what  combination  of  small  media  can  compose  the  most  suitable  document  for  each  user. 

The  context 

In  collaboration  with  SBT  company,  we  work  on  an  interactive  web  site  for  a supervised  cognitive 
training:  www.happvneuron.com  (Tarpin-Bemard  et  al,  2001).  During  each  training  session  (i.e.  each 
connection),  the  user  executes  a set  of  exercises  that  the  system  suggests.  Presented  into  a playful  and  cultural 
dimension,  the  exercises  vary  in  difficulty’s  level,  speed. . . in  order  to  train  the  user  (Habieb-Mammar  et  al, 
2001).  A database  stores  normalized  data  (means  and  standard  deviations)  for  each  variant  of  exercise  and 
family  of  population  distinguishing  gender,  level  of  education  and  age.  The  current  statistics  show  that  since  the 
web  site  has  been  opened  to  the  public,  on  may  2002,  the  number  of  performed  exercises  exceeds  750.000. 
Comparing  the  trainee’s  results  and  the  normalized  data  we  progressively  build  his/lier  cognitive  profile.  Tlianks 
to  it,  the  system  advises  the  elderly  user  in  the  choice  of  exercises. 

In  this  context,  we  built  an  evaluation  module  composed  of  ten  precise  exercises  that  allows  to  quickly 
build  a cognitive  profile.  Then,  this  profile,  which  is  quite  stable,  can  be  used  in  very  different  context.  Our  first 
purpose  is  to  elaborate  an  adaptive  multimedia  course  explaining  how  the  brain  is  working. 

Cognitive  styles 

Cognitive  styles  refer  to  a person’s  habitual,  prevalent,  or  preferred  mode  of  perceiving,  memorizing, 
learning,  judging,  decision-making,  problem-solving  (Dufresne,  1997).  Individual  differences  about  how  people 
carry  out  tasks  involving  these  functions  may  constitute  a style  if  they  appear  to  be:  pervasive,  which  means  that 
they  emerge  consistently  in  different  contexts,  independently  of  the  particular  features  of  situation;  or  stable, 
that  is,  they  are  always  die  same  at  different  times.  They  are  one  of  the  most  stable  user  characteristics  overtime. 
They  are  consistent  across  a variety  of  situations,  as  opposed  to  user  knowledge  or  experience  that  are  more 
specific  and  evolving.  Cognitive  styles  induce  persons  to  adopt  similar  attitudes  and  behaviors  in  a variety  of 
domains  they  concern.  They  are  important  in  determining  the  most  effective  interface  for  a particular  category  of 
user,  especially  in  the  formative  stages  of  an  interaction  (Fowler  et  al,  1985). 

Cognitive  user  profile 

The  cognitive  user  profile  is  composed  of  numeric  indicators  representing  cognitive  abilities  and  styles: 

- The  cognitive  indicators  are  dispatched  into  5 sectors:  memory,  attention,  executive  functions, 
language  and  visual  & spatial  capacities.  In  detail,  25  indicators  have  been  determined.  We  can  mention  several 
of  them  as  an  illustration:  cultural  memory,  recent  memory  (verbal,  visual  or  musical),  working  memory,  lexical 
spelling,  categorization,  comprehension,  planning,  reasoning,  form  recognition,  etc  ; 


713 


- The  style  indicators  reveal  some  cognitive  styles  such  as  field  dependence.  These  indicators  are 
determined  by  a combination  of  cognitive  indicators  or  through  the  comparison  of  the  results  obtained  in  several 
exercises.  For  example  to  determine  whether  the  subject  is  field  dependent  or  not,  we  use  an  indicator  which 
measures  the  difference  in  performance  after  running  the  same  exercise  with  two  different  images  (the  first  with 
significant  context  and  the  second  with  non-significant  context)  then  we  adjust  it  with  other  cognitive  indicators 
such  as  recent  memory  (verbal,  visual  or  musical)  and  comprehension. 

Adaptation  Process 

This  user  profile  enables  the  selection  of  the  outcomes  modalities.  The  multimedia  document  is  defined 
into  an  XML  document.  To  illustrate  the  adaptation  process,  here  is  an  example  of  a page  dealing  with  the 
following  subject : “The  main  parts  of  a language  : the  vocabulary  and  the  syntax”.  In  the  XML  document,  the 
page  contains  2 elements.  Each  one  could  be  presented  according  to  different  modalities: 


Figure  1:  an  XML  document  structure 


The  problem  is  to  find  the  “best”  combination  of  medias  according  to  the  willing  of  the  designer  and 
the  abilities  of  the  reader.  In  this  XML  structure,  the  possible  combinations  are:  (Mia  ; Mlb),  (Mia  ; M2b), 
(M2a,  M3a  ; Mlb),  etc.  Thus,  we  can  build  a matrix  whose  lines  corresponds  to  compatibility  criteria  (e.g. 
capacity  to  manage  of  multimodality,  respect  of  preferred  modality,  field-dependence,,,)  and  whose  rows  are  the 
possible  combinations  of  medias.  For  each  combination  and  each  criteria,  a compatibility  factor  is  calculated 
using  an  arithmetic  formula  that  combines  the  profile  and  the  characteristics  of  the  combination.  The 
combination  of  medias  that  obtains  die  biggest  sum  is  considered  as  the  most  compatible.  Hence  the  medias  are 
selected  by  enabling  relevant  tags  before  applying  the  stylesheet. 

Conclusion 

The  first  working  track  has  been  defined,  experimentations  and  validation  work  are  currently  in 
progress  especially  to  determine  the  compatibility  factors.  Our  aim  is  to  build  a reliable  user  profile  in  order  to 
exploit  it  not  only  in  our  site  but  in  other  coming  applications.  All  this  work  has  been  possible  by  adopting 
multidisciplinary  approach  (computer  sciences,  cognitive  psychology  and  education  sciences). 
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Abstract:  This  paper  focuses  on  an  online  video  based  correction  system  for  language 
learning.  The  prototype  system  using  the  proposed  model  supports  learning  between  a 
native  English  teacher  and  a non-native  learner  using  a videoconference  system.  It 
extends  the  videoconference  system  so  that  it  can  record  the  conversation  of  a learning 
scene.  If  a teacher  edits  the  video  to  include  explanations,  the  video  can  become  very 
useful  as  teaching  material.  However,  in  ordinal  video-editing  systems,  it  is  difficult  to 
show  edited  parts.  Therefore,  this  paper  proposes  a video  correction  system  and  focuses 
on  the  online  video-editing  model. 


Introduction 

With  fast  development  of  the  Internet,  people  around  the  world  have  more  chances  to 
communicate  directly  with  CMC  (Computer  Mediated  Communication)  tools,  such  as  e-mail,  chat  and 
bulletin  board  systems.  By  applying  the  CMC  tools  to  language  learning,  it  is  possible  for  learners  to 
communicate  with  native  speakers  in  foreign  languages.  In  fact,  many  approaches,  which  may  lead  to 
improved  exchanges  among  different  cultures  and  language  learning  with  the  CMC  tools,  have  been 
proposed.  Hanson  et  al.  (1998)  used  the  CMC  tools  for  improving  international  collaboration  and 
comprehension  among  university  learners  around  the  world.  The  experiment  showed  that  a learner's  ability 
to  comprehend  and  express  themselves  in  foreign  languages  was  highly  improved.  Tt  also  pointed  out  direct 
communication  with  native  speakers  was  very  important  for  foreign  language  learning.  An  approach  that 
emphasizes  foreign  language  study  through  communication  is  known  as  the  communicative  approach 
(Johnson  & Morrow,  1981).  In  this  approach,  grammatical  rules  and  vocabularies  are  of  secondary 
importance.  Recently,  the  communicative  approach  has  attracted  much  interest  in  CALL  Computer 
Assisted  Language  Learning). 

From  a technical  point  of  view,  the  high-speed  and  broadband  network  has  realized  rich,  smooth  and 
synchronous  communication  and  collaboration  using  not  only  text  but  also  audio  and  video  materials 
(LeeTieman  et  al,  2001).  For  example,  in  an  English  conversation  school  in  Japan,  Japanese  learners 
attempt  to  master  English  with  a distant  teacher  (native  speaker)  using  a videoconference  system.  The 
interactive  video  makes  learning  more  effective  (Dalton,  1990).  What  seems  to  be  lacking,  however,  is 
reuse  of  the  conversational  video  for  language  learning  by  reviewing  and  editing  the  video  of  the  past 
lesson. 

This  paper  proposes  an  online  video-editing  model  for  language  learning,  ^\hich  allows  the  teacher  and 
the  learner  to  revise  and  edit  their  conversational  video.  If  the  videoconference  system  records  their 
conversation  into  video  files,  the  teacher  can  edit  the  videos  using  a video-editing  tool.  However,  it  may  be 
very  difficult  for  the  learner  to  understand  the  teacher's  task  in  the  original  video  because  the  ordinal  tools 
create  quite  a different  video  from  the  original  one.  Therefore,  we  propose  a markup-based  video-editing 
system  called  Viclle  (^deo-based  Communicative  Language  Learning  Support  System)  through  the 
proposed  model,  which  provides  a similar  environment  to  a real  one  where  texts  are  corrected  using  red- 
pen  on  paper.  Using  this  system,  a learner  can  correct  their  conversational  video,  e.g.,  a teacher  inserts  a 
new  video  segment  and  annotations  with  correction  marks.  Using  this  system,  it  becomes  easier  to 
understand  the  teacher's  correction  comments.  The  features  of  Viclle  are  as  follows: 

(1)  A teacher  can  correct  a video  recorded  by  the  videoconference  system  in  Viclle. 
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(2)  The  system  uses  VCML  (Video-based  Correction  Markup  Language)  to  describe  the  contents  of  the 
video  correction. 

(3)  A learner  can  see  what  the  teacher  has  just  corrected.  This  is  based  on  WISIWYS  (What  I See  Is  What 
You  See). 

(4)  A learner  can  study  by  interacting  with  this  system  based  on  the  correction  marks. 

(5)  The  revised/explained  video  can  be  generated  from  an  original  video  and  a VCML  document. 

This  paper  is  organized  as  following.  First,  related  work  is  introduced.  After  that,  a video  correction  and 
an  online  video-editing  model  are  described.  In  the  next  section,  a VCML  is  used  to  describe  the  video 
correction  and  the  implementation  system  is  shown.  Finally,  concluding  remarks  are  given. 

Related  Work 

Recently,  wide  ranges  of  authoring  ystems  have  been  developed,  which  allow  both  text  and  media 
annotations.  Among  them,  a multimedia  annotation  environment  for  young  children  using  Java  has  been 
developed  (Bouras,  et  al.,  2000).  It  is  used  to  provide  children  with  the  opportunity  to  reflect  upon  and 
annotate  episodes  from  their  everyday  life.  This  system  can  lead  to  arguments  with  many  learners. 
However,  it  has  no  functions  to  learn  with  the  inserted  annotations. 

The  approach  used  in  the  project  of  ACTS082  called  DIANE  (Benz,  et  al,  1997)  recorded  an  arbitrary 
application  output  of  the  basic  content  of  a multimedia  document  and  to  annotate  it  with  all  kinds  of  media 
available  to  the  user.  This  system  defines  the  annotation  framework,  however,  the  target  for  usage  is  not 
defined  clearly.  Therefore,  it  can  cope  with  the  ordinal  video  but  cannot  deal  with  two  videos  recorded  by  a 
videoconference  system.  It  has  no  players  to  show  these  synchronously. 

The  classroom  2000  (Abowd,  et  al,  1996)  project  focused  on  teaching  material.  There  is  a hypothesis; 
tools  to  aid  in  the  capture  and  subsequent  access  of  classroom  information  will  enhance  both  the  learning 
and  teaching  experience.  Therefore,  the  teacher  can  make  teaching  materials  with  various  media  and  show 
it  to  the  learner.  The  learner  can  annotate  on  the  teaching  material.  However,  the  learner  cannot  study  with 
his/her  own  video. 

Lieberman  et  al  (1994)  have  developed  a system  to  use  video  as  a tool  for  procedural  knowledge 
representation.  However,  this  system  is  not  suited  to  language  learning  because  it  can  only  annotate  still 
pictures  extracted  from  the  original  video  by  drawing. 

Language  Learning  using  Video  Correction 

Video  Correction 

Conventionally,  texts  have  been  corrected  using  a red  pen  to  highlight  errors.  Thus,  rectified  contents  are 
made  clear.  According  to  such  a traditional  correction  manner,  an  online  markup-based  collaborative 
writing  support  system,  called  CoCoA  (Communicative  Correction  Assisting  System)  (Ogata  et  al,  1997) 
and  CoCoA-J  (CoCoA  for  Java)  (Ogata  et  al,  2000)  has  been  developed,  which  is  very  similar  to  the  real 
correction  environment  with  a pen  and  paper.  This  system  has  been  used  for  language  learning,  and  it  can 
automatically  generate  the  original  text  and  the  revised  one  from  the  corrected  text.  However,  it  cannot  deal 
with  multimedia  contents  such  as  video  and  audio.  Therefore,  it  cannot  correct  a video  like  a text,  exactly. 

In  order  to  rectify  a video,  one  should  focus  on  each  of  the  smallest  units.  A text  consists  of  many 
characters.  The  correction  of  a text  operates  on  a character.  Similarly,  a video  consists  of  many  frames.  It 
can  operate  like  a text  if  a frame  is  treated  like  a character.  [Figure  1]  shows  text  correction  and  video 
correction.  The  correction  of  a text  in  (A)  shows  deletion  and  insertion  for  characters.  Video  correction  in 
(B)  shows  them  for  frames. 


the  smallest  unit 


Figure  1 ; Comparison  between  correction  of  a text  and  video  correction. 
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Online  video-editing  model 


Several  editing  systems  aid  teachers  when  reviewing  and  correcting  learners’  drafts.  Farkas  & Poltrock 
(1995)  classified  an  online  text -editing  model  that  considered  four  models;  the  silent  editing  model,  the 
comment  model,  the  edit  trace  model,  and  the  traditional  markup  model.  In  the  silent  editing  model,  the 
teacher  overwrites  and  corrects  the  learner's  document.  In  the  comment  model,  annotations  are  inserted  in 
the  original  document.  In  the  edit  trace  model,  the  teacher  works  in  the  manner  of  an  author,  deleting, 
adding,  and  moving  text.  The  computer  can  compare  an  editor's  new  version  with  an  original  one.  It  allows 
learner  to  view  the  draft  that  contains  teacher's  changes.  Microsoft  Word  accepts  this  model.  The  traditional 
markup  model  employs  correction  marks  to  the  computer  screen,  for  example,  deletion,  insertion,  and  move. 
According  to  this  classification,  this  paper  proposes  an  online  video-editing  model  as  follows:  (See  [Figure 
21) 

(A)  The  silent  model:  After  correcting  the  wrong  parts  of  an  original  video  with  a general  video-editing  tool, 
the  teacher  creates  a new  video  file.  This  is  the  simplest  model,  however,  it  is  very  difficult  for  the  learner 
to  check  the  teacher's  work.  This  model  is  destructive  because  the  teacher  cannot  readily  recover  the 
original  video  once  he/she  has  made  a revised  video. 

(B)  The  comment  model:  This  model  employs  pop-up  notes,  temporary  footnotes,  hidden  text,  and  special 
symbols  placed  within  the  original  video.  A system  based  on  this  model  is  developed  with  SMIL.  It  often 
takes  a considerable  amount  of  time  for  teachers  and  learners  to  obtain  a revised  video  from  comments  on 
the  original  video. 

(C)  The  Mark-up  model:  This  adapts  the  traditional  marks  used  with  pen  and  paper  to  video  editing  on  the 
computer  screen.  The  symbols  are  both  familiar  and  intuitive  for  teachers  and  learners:  for  example, 
deletion,  insertion,  and  move.  For  instance,  MATE  (Hardock  et  al,  1993)  that  is  a text -based  collaborative 
writing  tool,  allows  teacher  to  use  both  digital  ink  and  voice  commands  toward  pen  and  voice  computing. 
In  this  model,  both  learners  and  teachers  can  interpret  the  teacher's  modification  with  markings  much  more 
readily  than  in  the  other  models.  Moreover,  they  can  easily  derive  both  the  original  video  and  the  revised 
one  from  a marked-up  video. 

Many  systems  employ  a traditional  mark-up  fliat  allows  multiple  users  to  mark-up  an  electronic 
document  as  if  they  were  marking  up  a printed  copy  of  the  document.  However,  no  studies  have  ever  tried 
to  adapt  a markup  model  into  a video-editing  system.  Therefore,  this  paper  proposes  a video  mark-up 
model  for  supporting  language  learning.  In  addition,  a correcting  system  has  also  been  developed. 


(A)  Silent  model 


(Q  Markup  model 
Correction 


Original  video 
(1) 

Figure  2 : Taxonomy  of  online  video  editing  models. 

Learning  process  using  Viclle 

In  order  to  learn  a video  through  video  connection,  the  video  to  be  revised  by  a teacher  is  needed. 
Then,  the  teacher  can  correct  it  and  the  learner  can  study  with  it.  The  learning  flow  shown  as  [Figure  3]  is 
as  follows. 

(1)  Recording  with  a videoconference  system:  A teacher  gives  lessons  in  speaking  a language.  The 
videoconference  system  in  Viclle  records  a video  of  both  the  teacher  and  the  learner.  The  learner  and  the 
teacher  share  the  files  via  network.  The  video  is  used  as  a basal  video  for  later  operations. 
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(2)  Correcting  video:  The  teacher  corrects  the  recorded  video  with  Viclle  Editor.  The  instructor  makes  the 
data  for  correction  and  rectifies  the  video  with  it.  The  teacher  sends  the  VCML  file  (made  by  Viclle - 
Editor)  to  the  learner  after  correcting.  The  VCML  file  shows  only  the  contents  of  the  correction  but  the 
video  is  shared  in  (1)  so  the  teacher  does  not  need  to  resend  the  original  video. 

(3)  Learning  by  correction:  The  learner  studies  using  the  corrected  video  with  Viclle  - Viewer.  According  to 
the  VCML  file  and  the  learner’s  process,  Viclle-Viewer  plays  the  contents.  When  learning,  the  student 
reviews  and  repeats  places  that  are  different  to  understand  on  the  corrected  video. 

(4)  Entering  questions:  If  the  learner  has  some  problems  and  questions,  s/he  can  describe  them.  In  addition, 
if  there  is  a question  from  the  teacher,  s/he  can  answer  it.  After  that,  the  Viclle-Viewer  generates  a VCML 
file  and  sends  it  to  the  teacher.  The  VCML  file  contains  the  learner's  question  and  feedback.  The  data 
relates  the  correction  to  the  original  VCML  with  a link. 

(5)  Grasping  learner's  feedback:  The  teacher  grasps  the  learner's  feedback  from  the  VCML  file  with  the 
Viclle -Editor  and  refers  to  that  work  for  the  next  time.  The  Vi  die -Editor  shows  the  feedback  for  each 
correction. 

The  above  is  repeated  when  learning.  The  learner  and  the  teacher  repeat  the  learning  process  and 

solve  the  learner's  problem.  As  a result,  the  learner  may  improve  his/her  communicative  skill. 


Correcting  the  videos  Learning  by  the  video  correction  Entering  questions 


Figure  3 : Learning  process  using  Viclle. 

VCML:  Markup  Language  for  Video  Correction 

Video-based  Correction  Markup  Language  (VCML)  is  proposed  to  describe  video  correction.  In 
addition,  it  can  represent  the  learner's  condition  after  studying.  This  markup  language  is  based  on  XML. 
XML  is  a markup  language  used  to  exchange  the  corrected  contents  on  network.  Therefore,  a teacher  and  a 
learner  can  exchange  the  correction  only  with  VCML  because  the  original  video  is  linked  by  VCML.  Some 
of  the  advantages  of  VCML  are  as  follows: 

(1)  It  does  not  edit  the  original  video. 

(2)  It  uses  the  correction  marks  shown  as  [Table  1 ]. 

(3)  It  becomes  a revised  video  by  applying  the  tags  of  revised  corrections. 

(4)  It  becomes  the  explained  video  by  applying  the  tags  of  explained  corrections. 

(5)  It  is  possible  to  represent  the  condition  of  the  learner's  studying. 

(6)  It  is  independent  of  the  kinds  of  computer  and  software. 

(7)  It  is  possible  to  send  and  receive  by  e-mail  because  of  the  text  format. 
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Correction 

Icon 

Contents 

Insert 

1 1 M 

Inserting  media  for  explanation 

Overlap 

nil 

Composing  comments 

Replace 

Replacing  with  other  media 

Delete 

D eleting  part 

Repeat 

Repeating  the  part 

Pause 

ipr 

Pausing  the  point 

Annotation 

:rpj: 

Inserting  media  as  comment 

Table  1 : VCML  tags  for  video  correction. 
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Implementation 

VicUe  has  been  developed  on  Windows  2000.  The  development  language  is  Java  SDK  1.3.1.  Java  Media 
Framework  (JMF)  2.1.1a  is  used  as  a framework  to  deal  with  media,  JAXP  (Java  API  for  Parsing)  is 
used  as  a XML  Parser,  RELAX  (Regular  Language  description  for  2^ML)  to  analyze  XML  documents,  and 
Relaxer  as  a parser  of  RELAX  are  also  used. 

A snapshot  of  Viclle-Editor  and  its  pop-up  windows  is  shown  in  [Figure  4].  In  the  (A)Correction 
Editor,  a teacher  can  edit  the  original  video  with  correction  marks.  The  videos  for  the  correction  and  the 
operation  buttons  are  shown  on  it.  When  the  teacher  wants  to  correct  the  original  video,  s/he  can  rectify  by 
choosing  the  correction  mark  on  the  (D)Correction  Palette.  The  correction  mark  is  inserted  by  clicking  the 
palette.  An  example  that  inserts  replace  correction  is  shown.  When  the  replace  button  is  clicked,  the 

(E) Correction  Dialog  appears.  Then,  the  user  selects  the  media  type;  text,  audio,  video,  picture  or  link.  In 
this  case,  the  user  selects  video  as  the  media  and  inputs  he  text  to  the  comment  box.  After  that,  the 
correction  of  replace  is  completed.  Similarly,  the  user  can  insert  other  corrections.  In  order  to  confirm  or 
select  the  correction  frame,  the  frames  of  the  original  video  are  shown  in  the  (B)Timeline.  The  display 
method  can  be  altered  by  the  (C)Timeline  Controller.  In  order  to  affirm  the  correction  condition,  the 

(F) Correction  Status  Bar  is  used.  The  inserted  corrections  are  shown  by  the  marks  on  the  bar.  In  order  to 
change  the  contents  for  playback,  the  (G)Play  Mode  is  used.  According  to  the  attribute  of  correction  marks, 
the  playback  contents  can  be  changed. 

Conclusions 


This  paper  proposed  an  online  video-based  correction  model  for  language  learning.  No  studies 
have  ever  tried  to  adapt  a markup  model  into  a video-editing  system.  The  prototype  called  Viclle  can 
correct  videos  recorded  by  the  videoconference  system.  In  addition,  VCML  is  proposed  to  exchange 
corrected  contents  and  learner's  condition.  A video  was  used  as  a tool  for  commentating  or  giving  lessons. 
However,  as  the  video  correction  uses  the  markup  model,  it  can  discover  information  required  by  a learner 
in  real  time.  Thus,  a video  through  video  correction  can  be  used  as  teaching  materials.  Although  Viclle  was 
used  for  language  learning  in  this  paper,  the  video  correction  is  independent  of  the  domain.  For  instance,  it 
can  use  sports  training  for  the  form;  acquire  gestures  in  different  cultures  like  that  between  Japanese  and 
Americans.  In  future  work,  a VCML  document  will  be  used  to  retrieve  video  databases  that  are  stored 
VCML  documents.  The  data  can  be  used  as  error  databases.  Another  point  under  consideration  is  the 
examination  of  a speech  recognition  system  to  help  the  correction  work  and  train  learner's  pronunciation  or 
intonation. 


M-m  1 t'MM 


Figure  4 : Snapshots  of  Viclle-Editor. 
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Abstract:  The  Open  University  of  Israel  (OUI)  is  a distance  education  institution  which  offers 
over  250  computer-mediated-courses  through  the  Internet.  All  OUI  students  must  pass  an 
English  reading  comprehension  exemption  exam  or  take  the  University’s  English  reading 
comprehension  courses.  Because  reading  instruction  differs  from  content  instruction,  different 
considerations  need  to  affect  how  best  to  utilize  the  computer  when  planning  CMC  delivered 
reading  courses.  The  distance  courses  in  English  reading  comprehension  are  transmitted  via  e- 
mail  rather  than  the  Internet.  The  purpose  of  this  paper  is  to  recommend  this  “primitive” 
option,  and  to  explain  why  it  suits  courses  of  this  kind.  There  is  a tendency  to  move  on  to 
ever-more  impressive  modem  technologies,  and  constantly  seek  new  ones.  To  avoid  falling  into 
the  trap  that  Gabi  Salomon  referred  to  as  the  “technological  tail  wiggling  the  educational  dogf’, 
we  decided  on  the  e-mail  option. 


The  Open  University  of  Israel  (OUI)  is  a distance  education  institution  which  caters  to  students  of  all 
ages  who  wish  to  earn  an  academic  degree,  in  Israel  or  abroad.  There  are  no  entrance  requirements,  but  students 
must  live  up  to  the  University’s  stringent  course  demands,  and  all  must  pass  an  English  reading  comprehension 
exemption  exam  or  take  the  University’s  English  reading  comprehension  courses.  Though  the  Open  University 
offers  over  250  computer-mediated-courses  through  the  Internet,  the  distance  courses  in  English  reading 
comprehension  are  transmitted  via  e-mail.  The  purpose  of  this  paper  is  to  recommend  this  “primitive”  option,  and 
to  explain  why  it  suits  courses  of  this  kind. 


The  OUI  Study  Method 

The  OUI’s  teaching  method  is  not  space-  or  time -dependent  as  it  is  not  based  on  a central  campus 
where  lecturers  and  students  gather,  or  on  an  established  and  uniform  schedule  [1].  Students  enroll  in  specific 
courses  rather  than  in  a faculty  or  department.  They  select  the  courses  in  their  program  of  study  from  a varied 
and  wide-ranging  selection.  The  individual  program  may  include  a diverse  array  of  disciplines  (which  in  other 
universities  belong  to  separate  departments  or  faculties),  or  may  be  based  on  a more  focused  disciplinary  format 
which  resembles,  to  some  extent,  single -discipline  or  dual-discipline  programs  of  study  in  other  institutions. 

Students  ’ rate  of  progress  is  not  measured  in  units  of  time  (years  or  semesters),  but  rather  in  the  number 
of  accumulated  credits.  The  individual  rate  of  progress  is  determined  by  the  students  themselves  - based  on  the 
amount  of  time  available  to  them.  The  only  constraining  time -frame  is  the  semester,  as  the  duration  of  most 
courses  is  one  semester  of  15  weeks,  and  the  students  are  required  to  meet  the  requirements  of  the  course  during 
the  semester  in  which  they  are  enrolled. 

The  OUI  offers  close  to  500  courses  based  on  scholarly  or  scientific  works,  consisting  of  one  or  more 
volumes  written  by  renowned  specialists  in  their  field  and  produced  especially  for  OUI  students.  The 
coursebooks  are  adapted  to  self-study:  they  are  usually  divided  into  units,  each  of  which  deals  with  a specific 
topic  to  be  studied  within  a fixed  period  of  time.  Guiding  questions  are  integrated  into  the  material,  as  are 
exercises  and  self-assessment  questions.  Each  course  includes  face-to-face  components:  group  tutorial 
sessions,  laboratory  work  or  study  excursions.  While  participation  in  these  sessions  is  usually  not  mandatory, 
it  is  highly 


[1]  The  description  of  the  Open  University  study  method  is  based  in  large  part  on  Zippy  Erlich,  Judith  Gal-Ezer,  and  David 
Lupo,  “Traditional  Distance  Education  vs.  Technology  Integrated  Distance  Education”  in  Intelligent  Internet  Based  Teaching 
and  Learning,  edited  by  R.  J.  Howlett  and  L.  C.  Jain  to  be  published  in  2002  by  Springer- Verlag. 

recommended,  as  the  sessions  contribute  considerably  to  clarification  of  the  study  material,  and  also  provide 
an  opportunity  to  interact  with  other  students  enrolled  in  the  course.  The  sessions  are  held  at  the  regional  study 
centers  throughout  the  country. 

Open  University  courses  have  traditionally  taken  advantage  of  existing  technologies  for  enrichment. 
Until  the  eaiiy  1990s,  lectures  by  experts  were  broadcast  daily  on  Israel  radio;  with  the  advent  of  cable  TV, 
television  broadcasts  which  accompany  many  OUI  courses  have  replaced  the  radio.  Segments  are  aired  several 
times  a day,  every  day  of  the  week.  With  the  rapid  development  of  information  and  communication  technologies 
in  the  1990s,  it  was  clear  that  the  integration  of  state-of-the-art  technology  into  OUI  courses  would  have  a 
significant  impact  on  distance  teaching  at  the  OUI. 


Technology  at  the  Open  University 

The  main  reason  for  integrating  technologies,  and  the  Internet  in  particular,  into  OUI  courses,  was  to 
provide  a virtual  campus  as  a substitute  for  a real  campus,  to  compensate  students  for  what  they  miss  when 
learning  at  a distance.  A second  objective  was  to  take  advantage  of  additional  educational  resources,  beyond 
what  the  university  provides.  These  include  all  public  domain  educational  resources  and  digital  library  services, 
which  enables  students  to  access  bibliographic  and  other  databanks,  electronic  journals,  computerized 
encyclopedias,  as  well  as  the  computerized  catalog  of  the  university  libraries  network  in  Israel,  from  their  homes. 
Another  aim  was  to  compensate  for  the  lack  of  face-to-face  tutorials  for  students  whose  nearest  study  center  is 
far  from  their  homes,  or  who  are  unable  to  attend  tutorial  sessions  for  various  personal  reasons,  by  providing 
virtual  tutorials  through  electronic  communication. 

The  Center  for  Information  Technology  in  Distance  Education  (^hoham)  was  established  in  1995  to 
help  the  Open  University  enter  world-wide  academic  activity  in  the  field  of  information  technology  in  teaching 
and  education.  The  major  focus  of  the  Center’s  activities  is  on  research  and  development  of  teaching  methods 
based  on  state-of-the-art  information  technologies  and  incorporating  them  into  OUI  courses. 

Telem,  a department  within  Shoham,  specializes  in  computer-mediated  communication  (CMC).  It  has 
developed  an  interactive  on-line  learning  environment  on  the  Internet,  which  is  part  of  the  teaching/leaming 
process  and  serves  both  students  and  faculty.  Today  close  to  250  CMC  courses  are  offered,  with  state-of-the-art 
technologies  incorporated  into  course  development  and  teaching  procedures. 

Interactive  learning:  The  computerized  teaching/leaming  environment  provides  for  one-on-one  and 
group  interaction  among  all  participants  in  the  course:  students,  instructors  course  coordinators  and  guest 
lecturers.  Every  CMC  course  has  its  own  HTML  and  Java-based  Web  site  that  provides  interactive  learning 
materials  through  electronic  asynchronous  communication  including  discussion  forums,  e-mail,  and  materials 
that  students  can  download.  The  sites  can  be  accessed  by  students  in  Israel  or  abroad  through  the  Internet.  All 
course  sites  are  graphically  and  functionally  similar,  and  linked  to  administrative  information  drawn  directly  from 
the  GUI’s  central  database:  the  course  schedule  and  description,  the  course  tutors  and  the  list  of  students.  The 
academic  aspect  of  the  sites  includes  Hebrew-based  applications  that  were  specially  developed  to  enable  the 
course  coordinator  to  easily  and  independently  update  the  data,  without  the  need  for  any  knowledge  of  Web 
programming.  With  this  platform  available  and  widely  used  at  the  OUI,  the  question  arises  as  to  why  the  English 
reading  comprehension  courses  do  not  take  advantage  of  it. 


English  Reading  Comprehension  Courses  at  the  Open  University 

Reading  comprehension  skills  in  English  are  necessary  for  all  students  for  whom  English  is  a foreign 
language.  Without  the  ability  to  read  in  English,  students  find  that  sources  needed  for  much  of  their  academic 
studies,  especially  in  advanced  courses,  are  not  accessible  to  them.  As  a result,  all  Israeli  universities  require  a 
minimum  level  of  reading  comprehension  in  English  and  students  cannot  be  awarded  a degree  without  passing 
an  exemption  exam  or  taking  English  reading  comprehension  courses.  Since  only  10%  of  students  entering  the 
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university  come  with  an  exemption  in  English,  the  reading  comprehension  courses  are  among  the  most  highly 
populated,  if  not  the  most  popular,  at  the  OUI. 

The  Open  University  offers  a series  of  courses  in  English  reading  comprehension  for  academic 
purposes  on  five  levels:  A (advanced);  B,  C (intermediate);  and  D,  E (beginning).  The  courses  are  offered  in  three 
study  formats:  intensive  (a  semester-long  course  comprising  14  weekly  face-to-face  meetings),  regular  (a 
semester-long  course  comprising  7-8  bi-weekly  face-to-face  meetings),  and  computer-mediated  (a  semester-long 
course  which  includes  1-3  face-to-face  meetings  and  9 e-mail  sessions,  of  which  7 are  compulsory).  Of  the  three 
formats,  only  the  latter  is  a distance  education  format. 

Most  university  courses  are,  by  nature,  content  courses.  In  contrast,  learning  a language  is  a process 
of  communication  and  goes  well  beyond  learning  lists  of  words  and  practicing  grammar  exercises.  Because  the 
classroom  is  a more  natural  environment  for  language  use  and  social  interaction,  language  courses  are 
traditionally  taught  in  a face-to-face  (f2f)  format.  If  we  consider  reading  as  a mode  of  language  use,  and  distance 
education  as  a mode  of  language  instruction,  distance  education  for  this  particular  mode  of  language  use  seems 
particularly  appropriate.  Indeed,  we  encounter  striking  parallels:  the  reader  (when  reading  in  “real  life”)  and  the 
learner  (when  learning  to  read)  both  function  in  isolation  when  interacting  with  a text.  In  both  cases,  discourse  is 
enacted  at  a distance,  a disassociated  first  person  (the  author  or  the  instructor)  is  actively  present,  and  no 
reciprocity  is  manifest,  within  the  interactive  context  [2].  One  could,  therefore,  argue  that  reading  instruction  and 
distance  education  are  particularly  well-suited.  It  would  therefore  seem  logical  to  use  the  available  platfomi  for 
teaching  reading  comprehension.  But  because  reading  instruction  differs  from  content  instruction,  different 
considerations  need  to  affect  how  best  to  utilize  the  computer  when  planning  CMC  delivered  reading  courses. 

The  Telem  course  sites  are  designed  for  content  instruction,  and  are  intended  to  afford  additional 
interaction  among  the  participants  and  to  provide  technology-based  learning  aids  that  increase  the  effectiveness 
of  and  enjoyment  from  the  learning  process.  Learning  materials  are  based  on  multimedia  and  hypertext,  and 
include  animations,  simulations,  multiple-choice  exams  with  immediate  feedback;  and  access  to  databases  on  the 
Internet.  These  aspects  are  less  relevant  when  transmitting  an  existing  reading  comprehension  course  via 
computer.  Traditionally,  reading  comprehension  programs  rely  entirely  on  pre -defined  and  pre -produced 
teaching/leaming  materials  in  which  the  selection  and  organization  of  the  reading  texts  and  the  tasks  are 
determined  by  the  syllabus-writer.  In  a f2f  teaching/leaming  context,  the  instructor,  in  selecting  what  to  teach, 
turns  the  abstract  syllabus  into  what  s/he  considers  to  be  the  most  appropriate  means  to  activate  the  learning 
process.  Thus  it  is  the  instructor  who  controls  and  promotes  the  learning.  In  a CMC  reading  course,  while  the 
instructor  is  responsible  for  dividing  the  syllabus  into  sessions,  arranging  deadlines  for  task 
completion/submission,  and  making  decisions  for  assessment  and  evaluation,  once  the  course  gets  started,  the 
participants  take  over  [3].  In  order  to  explain  the  choice  of  technology,  it  is  necessary  to  briefly  describe  the 
course. 


The  E-Mail  Format 

The  following  description  is  of  a B-level  e-mail  mediated  reading  course  which  has  been  implemented 
for  the  last  five  semesters  at  the  Open  University.  The  objective  of  the  course  is  for  students  to  reach  the  level  of 
proficiency  expected  in  the  parallel  f2f  B-level  course  (intensive  or  regular  formats).  The  students  sit  for  the  same 
mid-term  and  final  examinations  taken  by  students  registered  for  the  course  in  all  formats. 

Participants:  In  order  to  make  the  course  manageable  for  the  tutor,  the  number  of  participants  in  a 
group  is  limited  to  20. 

Materials:  The  teaching  materials,  which  are  pre -prepared  and  handed  out  to  the  students,  comprise  the 
following: 

• A collection  of  reading  passages  consisting  of  articles  selected  on  the  basis  of  topic  interest  and  level  of 
difficulty,  the  same  texts  used  by  the  rest  of  the  students. 

• Workbooks  dealing  with  nine  texts,  arranged  in  modules  according  to  test  organization  patterns 
(comparison/contrast,  description,  cause  and  result,  argument,  research)  including  approximately  20 
questions  to  guide  the  reading  process  and  relevant  teaching  points  (reading  strategies  and  linguistic 
features).  The  workbooks  are  similar,  but  not  identical  to  the  regular  materials. 
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• A “Resource  Centef*  which  brings  together  all  the  major  teaching  points  dealt  with  in  all  modules  as  a 
handy  reference. 


[2]  I am  grateful  to  Dr.  Esther  Klein-Wohl  of  the  Open  University  of  Israel  for  pointing  out  this  parallel. 

[3]  This  concept  was  proposed  by  Dr.  Anita  Pincas  of  the  Institute  of  Education,  University  of  London. 

Process:  The  initial  f2f  meeting  provides  the  students  with  the  opportunity  to  get  to  know  each  other,  to 
familiarize  themselves  with  the  structure  of  the  course  and  to  understand  what  their  responsibilities  are.  This 
meeting  serves  as  a “kick  off’  point  for  the  course  and  its  main  focus  is  to  deal  with  students’  queries  and 
technical  problems,  as  well  as  an  introduction  to  pre-reading  activities.  The  students  can  attend  two  additional 
f2f  meetings  by  joining  regular  groups  near  to  their  homes:  one  is  held  a week  before  the  midterm  examination, 
and  another  is  held  before  the  final  examination.  These  serve  to  prepare  the  students  for  the  exams  by  explaining 
the  format  of  the  exam  and  supplying  a practice  examination. 

The  rest  of  the  course  consists  of  asynchronous  e-mail  sessions.  Prior  to  each  e-mail  session,  students 
are  sent  a list  of  the  questions  which  they  are  assigned.  Each  student  is  assigned  8-9  of  the  20  questions  which 
relate  either  to  pre-reading  or  to  close  reading,  as  well  as  post-reading  questions  for  all  students.  The  date  of 
submission  is  mandated  in  advance  as  a specific  day  of  each  week  and  the  questions  are  sent  out  one  week  in 
advance.  The  weekly  tasks  demand  active  learner  engagement  rather  than  passive  participation  in  classroom 
sessions.  The  tutor’s  feedback,  which  provides  the  correct  answers  as  well  as  specific  comments  regarding 
language  features,  useful  reading  strategies  and  general  and  individual  feedback  relating  to  students  ’ cognitive 
and  metacognitive  awareness,  is  sent  out  to  all  students  within  three  or  four  days  of  the  deadline  for  submission, 
and  serves  as  additional  English-language  reading  material. 

Our  students  have  reacted  extremely  positively  to  the  course,  and  their  achievements  have  consistently 
been  as  good  as,  or  better  than  those  of  their  peers  in  f2f  groups.  Their  feedback  (provided  on  an  anonymous 
questionnaire  filled  out  by  all  students  at  the  end  of  each  course)  has  invariably  emphasized  the  one-on-one 
teaching  and  the  close  contact  they  enjoy  with  the  teacher,  thanks  to  the  e-mail  communication. 


Why  E-Mail? 

In  recent  years,  scholars  in  various  content  fields  have  been  grappling  with  the  question  of  appropriate 
formats  for  CMC.  Perhaps  surprisingly,  among  the  scholars  who  recommend  the  simple  e-mail  format,  rather  than 
what  Margaret  Anderson  (1997)  calls  “new  and  ‘glitzy’  technology”,  are  faculty  who  specialize  in  Engineering 
and  even  in  Computer  Information  Systems  [4].  This  indicates  that  the  choice  of  e-mail  is  not  necessarily  related 
to  students’  assumed  computer  skills,  nor  to  the  technological  options  available,  but  rather  to  the  specific  needs 
of  the  course  and  its  pedagogical  context. 

As  we  have  shown,  our  distance  education  reading  courses  also  do  not  utilize  all  the  multimedia 
aspects  of  what  can  today  be  called  “traditional  Internet  teaching”  as  exemplified  by  the  Telem  platform.  In  terms 
of  pedagogy,  our  reading  comprehension  students  are  given  the  reading  texts  on  paper,  and  not  through  the 
computer,  because  reading,  and  especially  reading  comprehension  skills,  are  dependent  in  large  part  on  back- 
and-forth  reading,  marking  the  text,  and  note-taking.  Reading  from  paper  is  still  considered  easier  than  reading 
from  a screen  [5].  We  also  need  to  consider  the  purpose  of  this  course.  Beyond  the  formal  requirement  of  English 
language  proficiency,  our  goal  is  to  enable  students  to  read  and  understand  the  required  course  readings  that 
they  are  assigned,  some  of  which  may  be  in  English.  These  readings  are  usually  provided  to  students  in  a 
printed  booklet.  Thus,  in  “real  life”  - for  their  studies  - students  will  be  reading  on  paper. 

In  terms  of  technical  aspects,  the  platform  utilized  by  Telem  (Opus)  makes  it  difficult  to  mix  English  and 
Hebrew  within  a single  message,  and  students  often  answer  reading  comprehension  questions  in  their  LI. 
Another  problem  relates  specifically  to  the  Telem  platform.  After  connecting  to  the  Internet,  students  need  to  go 
to  their  course  site,  enter  a password,  and,  often,  wait.  They  then  need  to  go  into  the  weekly  forum  to  find  the 
assignment,  print  it  out,  leave  the  computer  to  do  the  assignment,  and  then  repeat  the  process  to  post  their 
assignment.  Having  posted  their  assignment,  they  need  to  check  the  forum  to  see  when  the  feedback  is  posted. 
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and  repeat  the  process  to  get  the  feedback.  Thus  students  need  to  remember  to  go  into  the  course  site  at 
least 


[4]  Interesting  papers  in  this  context  are  by  Margaret  Anderson  (“An  Internet -Based  Asynchronous  Distance  Education 
Course”)  and  Donna  Ehrhart  (“Interaction  Through  Electronic  Mail”)  in  the  context  of  CIT  ’97,  Learning  with  Technologies, 
May  27-30,  1997  (Brockport,  SUNY)  and  Julie  Sharp’s  paper,  “E-teaching  Simply  with  E-mail”  presented  at  the  30th 
ASEE/IEEE  Frontiers  in  Education  conference,  October  18-21,  2000  (Kansas  City,  MO). 

[5]  See,  for  example,  Andrew  Dillon  (1992),  Reading  from  Paper  Versus  Screens:  A Critical  Review  of  the  Empirical 
Literature,  Ergonomics,  35  (10),  1297-1326  and  Designing  usable  electronic  text:  Ergonomic  aspects  of  human  information 
usage,  London:  Taylor  and  Francis,  1994. 

three  times  a week.  Moreover,  there  are  often  announcements  which  relate  to  administrative  aspects  of  the 
course  which  are  often  urgent  messages,  and  students  have  no  way  of  knowing  that  these  have  been  posted 
unless  they  go  to  the  course  site  daily. 

In  contrast,  when  the  course  is  transmitted  via  e-mail,  merely  by  opening  their  e-mail,  which  they  do 
anyway,  students  receive  assignments,  announcements  and  feedback.  They  can  post  their  answers  in  a 
combination  of  English  and  LI  within  the  same  e-mail  message,  or  send  them  as  attachments  in  Word  or  any 
other  word  processor.  Because  there  is  no  need  to  remember  to  check  the  site,  students  can  be  sure  that  they  are 
up-to-date  and  have  not  missed  anything.  The  simplicity  of  e-mail  works  in  both  directions:  If  the  student  has  a 
problem,  s/he  can  send  a “help”  message  to  the  instructor  by  e-mail,  and  be  assured  of  an  immediate  response. 
The  price  students  need  to  pay,  in  terms  of  time  on  task  (or,  more  precisely,  time  around  task)  and  dealing 
with  what  is  often  an  unwieldy  system,  is  too  high  for  a course  that  does  not  utilize  the  advantages  of  the  course 
sites  available  at  the  OUI.  To  avoid  falling  into  the  trap  that  Gabi  Salomon  referred  to  as  the  “technological  tail 
wiggling  the  educational  dog,”[6]  we  decided  (to  continue  with  the  dog  metaphor)  that  in  some  cases,  a mutt  is 
better  than  a certified  pure -bred,  even  if  less  elegant. 

Technology  in  distance  education  is  constantly  improving  and  becoming  more  user  friendly  and 
flexible.  There  is  a tendency  to  move  on  to  ever-more  impressive  modem  technologies,  and  constantly  seek  new 
options.  But  our  objective  is  to  teach,  and  we  have  to  be  careful  not  to  lose  the  baby  because  of  the  bathwater. 
Sometimes  the  simplest,  least  sophisticated  technology  makes  more  sense  than  state-of-the-art. 
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Abstract:  This  paper  describes  the  beginning  designs  for  a museum  education  expansion 

project.  Museums  have  not  only  been  at  the  forefront  of  protecting  artifacts,  they  have  been 
educational  initiators,  encouraging  humans  to  value  and  ieam  from  artifacts  through  exhibits  and 
educational  programs.  The  objective  of  this  project  is  to  advance  the  meaning  of  museum 
education  with  innovative  uses  of  communication  technologies.  Web-based  artifact  databases  and 
handheld  communication  devices  are  introduced  as  promising  vehicles  for  the  collaborative 
construction  of  modem  museums,  existing  beyond  the  walls  of  buildings  that  house  artifacts. 


Introduction 

Museum  educators,  like  many  people,  are  eager  to  experiment  with  the  latest  technology.  Soon  after 
museums  started  having  websites,  ‘virtual’  exhibits  began  to  appear.  The  virtual  exhibit  like  its  on-site  counterpart 
allows  a curator  or  web  designer  to  interpret  the  contents  of  the  museum  to  engage  and  educate  the  virtually  visiting 
public.  Also  like  their  on-site  counterparts,  virtual  exhibits  reach  their  audience  with  mixed  degrees  of  success. 

Beverly  Sheppard  of  the  Institute  for  Museum  and  Library  Services,  describing  the  strengths  of  msueums, 
observes  that  communities  and  families  trust  museums  as  resources,  that  they  act  as  stewards  for  collections  and 
ideas,  and  that  they  “offer  authentic,  first-hand  encounters  with  the  objects  and  information  they  collect”  (Sheppard, 
2001).  Trust,  stewardship,  and  authenticity  can  be  seen  as  central  qualities  of  museums,  qualities  as  important  to  the 
museum  learner  as  the  subject-specific  knowledge  presented  by  museums. 

Unfortunately,  web  based  exhibits  have  difficulty  expressing  these  very  qualities  of  museums.  The 
difficulty  of  knowing  what  to  trust  on-line  has  been  discussed  since  its  popularization  and  though  some  well-known 
museums  might  be  trusted  web  resources,  smaller  museums  do  not  necessarily  share  in  this  trust.  Web  presentation 
of  authenticity  also  becomes  complicated  because  of  the  many  problems  of  representing  physical  objects  in  a digital 
media.  Clearly,  interaction  with  an  artifact  in  a museum  is  an  “authentic”  experience  but  the  authenticity  of 
interacting  with  a virtual  artifact  is  more  problematic.  Similarly,  despite  some  efforts  to  show  stewardship  on-line  it 
is  not  clear  how  well  these  behind-the-scenes  exhibits  translate  to  an  appreciation  of  the  stewardship  conducted  in 
museums  or  the  question  of  each  individual’s  responsibility  as  stewards  to  objects  in  their  own  environment.* 

The  developing  tools  of  wireless  communication  offer  an  opportunity  to  address  these  qualities  of 
trustworthiness,  stewardship,  and  authenticity.  Instead  of  providing  virtual  access  to  distant  objects,  handheld 
computers  and  wireless  networks  can  bring  virtual  museums’  interpretive  powers  to  the  authentic  objects  as  they 
exist  in  our  surroundings.  These  technologies  can  be  used  to  help  visitors  organize  their  existing  knowledge  and  to 
deliver  the  expertise  of  the  museum  to  learners  studying  in  their  own  homes  and  communities.  Mobile  computing 
can  also  be  used  to  facilitate  truly  interactive  educational  experiences  in  and  out  of  the  museum.  Despite  advocates’ 
push  for  the  use  of  constructivist  techniques  in  museums,  only  a handful  of  authors  have  explored  the  connection  of 


* For  example,  see  the  “Conservation  Time”  section  of  “Tempus  Fugit:  Time  Flies”  « httn://www.nelson- 
atkins.org/tempusfugit/default2.htm)^  created  for  the  Nelson-Atkins  Museum  of  Art. 


constructivist  theory  to  museum  use  of  technology.^  While  web  exhibits  and  gallery  kiosks  provide  interactivity 
separately  from  the  artifacts,  mobile  technology  offers  an  opportunity  to  integrate  the  advantages  of  instructional 
computing  directly  into  the  gallery  visit  or  out-of-museum  activity. 

Goals 


In  this  project  we  redefine  the  virtual  exhibition  as  an  interactive  experience  of  visitors  with  real  objects 
using  mobile  communications  technology  to  deliver,  organize,  and  record  jointly  constructed  knowledge  (Hewitt  & 
Scardamalia,  1996).  Wireless  communications  through  their  flexibility  and  portability  offer  the  opportunity  to  make 
the  authentic  interaction  with  artifacts  part  of  a complex  situated  learning  experience  (Brown,  Collins,  & Duguid, 
1989).  We  will  unite  concepts  from  museum  education  and  educational  technology  with  the  strengths  of  museums  to 
improve  learning  in  exhibitions. 


Program  Design 

Development  of  the  project  will  involve  a test  school  and  museum  working  within  a single  exhibition.^ 
The  design  will  include  two  different  sections.  First,  a database  driven  website  developed  in  coordination  with  the 
museum  will  facilitate  collection  and  organization  of  content  including  text,  images,  and  other  media  Students 
planning  a visit  to  the  museum  will  use  the  site  as  a locus  for  self-directed  study  where  they  will  gather  information 
provided  by  the  museum,  linked  web  resources,  and  off-line  resources  based  on  their  own  research  agenda. 

The  second  module  delivers  information  gathered  by  the  students  to  a handheld  computer,  allowing  access 
to  their  existing  research.  Transmitters  within  the  museum  would  feed  the  computers  location  appropriate 
information  as  the  visitor  traverses  the  exhibit.  Additional  tools,  such  as  a digital  camera  to  allow  students  to 
photograph  details  of  artifacts  or  a color  wheel  to  record  student  observations  about  color  and  form,  will  make  the 
handheld  into  a data-gathering  device.  Information  gathered  during  the  exhibit  would  be  uploaded  to  the  central 
database  to  be  made  available  once  the  student  returns  to  school  for  collaborative  projects. 

Future  Directions 

The  structure  of  the  project  will  easily  adapt  to  use  in  any  exhibition  or  subject  area.  Further  development 
will  be  necessary  to  extend  the  project  for  home  use.  Development  of  the  communications  aspect  of  the  project  is 
central  to  the  project’s  vision  of  fostering  shared  meaning  development  between  visitors  and  institutions. 

Ultimately,  the  project  will  create  a series  of  tools  that  can  be  made  available  to  any  institution  interested  in 
using  mobile  connectivity  interactively  within  a museum.  The  tools  developed  in  this  project  would  help  museums 
to  teach  their  core  functionality.  These  qualities  of  museums:  reliable  (trustworthy)  research,  stewardship  over 
cultural  and  natural  objects,  authenticity  of  experience  describe  a manner  of  considering  the  world.  Traditionally 
museums  hope  to  teach  us  to  look  more  carefully  at  objects,  think  more  deeply  about  meaning,  and  approach  our 
surroundings  with  a sense  of  wonder.  By  helping  us  put  these  ideas  into  practice,  mobile  communication  technology 
helps  us  to  appreciate  more  fully  not  only  the  artifacts  in  museums  but  also  the  objects  in  our  lives. 
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The  Problem 

The  playing  field  in  our  nation’s  public  schools  is  not  level  and  East  Carolina  University  and 
Newport  Elernentaiy  are  changing  that  by  launching  a collaborative  project  to  ensure  that  students  in  one 
second  grade  class  have  the  technology  needed,  at  home  and  at  school,  to  master  expected  North  Carolina 
Standard  Course  of  Study  content.  The  main  research  question  is:  how  strong  is  the  correlation  between 
achievement  gaps  and  technology  gaps  in  a second  grade  class  in  Newport,  North  Carolina? 

The  literature  is  replete  with  information  about  the  “digital  divide.”  Government  reports  have 
measured  this  divide  for  the  past  half-decade.  But,  by  using  a conservative  definition,  it  reports  a 
technological  “stabilization.” 

The  Study 

Researchers  are  measuring  ameliorative  factors  designed  to  even  the  technological  playing  field  in 
the  classroom  and  the  home.  Each  student  has  been  given  a Hewlett-Packard  Jornada  Pocket  PC,  equipped 
with  electronic  reader,  software  for  word  processing,  spreadsheet  application,  email,  Internet  access 
capability,  a calculator  and  scheduling/tasking  packages  for  homework,  classwork,  and  communication 
with  parents. 

Each  is  being  quantitatively  tested  for  knowledge  of  the  technology  and  content  related  to  the 
North  Carolina  Standard  Course  of  Study.  Data  has  been  gathered  and  is  presently  being  analyzed. 
Preliminary  results  are  available  and  are  presented  in  the  next  section. 

Preliminary  Results 

Preliminary  results  indicate  a tendency  for  handheld  technology  and  the  appropriate  software  to 
improve  the  perfonnance  of  “at  risk”  students,  especially  in  the  areas  of  reading,  writing,  and  the  practical 
use  of  technology. 

The  digital  divide  is  muted  by  the  use  of  technology  that  students  can  take  home  with  them  and 
parents  are  generally  supportive  of  the  initiative. 

Future  Studies 

Quantitative  studies  are  needed  to  measure  more  specifically  the  impact  of  technology  on  students 
and  the  digital  divide.  Studies  focusing  on  subjects  such  as  reading  or  writing  only  might  shed  more  light 
on  what  works  and  what  doesn’t. 

Case  studies  with  a larger  N,  e.g.,  an  entire  district,  different  grade  levels,  etc.  would  bolster  the 
literature. 


7 28 


Evaluating  Online  Instruction:  Adapting  a Training  Model  to  E-Learning  in 

Higher  Education 


Karen  Hallett 
Director 

e-mail:  hallett@indiana.edu 

Christopher  Essex 
Distance  Education  Specialist 
e-mail:  cessex@indiana.edu 

Instructional  Consulting  Office 
School  of  Education 
Indiana  University 
United  States 


Abstract:  In  this  paper,  we  present  our  model  for  the  evaluation  of  postsecondary  online 
distance  education  courses  and  programs.  To  better  address  the  unique  nature  and 
audience  for  these  courses  and  programs,  and  the  related  institutional  needs  for  assessing 
their  success  or  failure,  we  focus  on  a model  from  corporate  training  that  provides  a 
comprehensive,  multi-level  evaluation  of  a distance  course  or  program.  This  model  is 
based  on  Kirkpatrick’s  (1975)  levels  of  evaluation.  We  provide  a general  description  of 
how  this  model  can  be  implemented  via  online  means  to  provide  the  assessment 
information  required  by  postsecondary  institutions. 


Background 

Over  the  past  decade,  online  instruction  has  been  become  a popular  delivery  option  in  many  institutions  of  higher 
education  (Green,  1997).  One  criticism  leveled  against  conventional  “student  evaluations  of  teaching”  suggests  that 
when  used  alone,  these  provide  limited  information  about  student  comfort  without  addressing  significant  factors 
such  as  student  learning  and  ability  to  transfer  that  learning  to  real-world  situations  (Trout,  2000).  Moreover,  many 
evaluation  systems  for  online  learning  emphasize  usability  and  reliability  of  the  technology  over  the  value  of  the 
instruction  to  the  students  (Anglin  & Morrison,  2000).  Finally,  the  unique  nature  of  B-leaming  suggests  that 
traditional  course  and  program  evaluation  may  be  particularly  deficient  in  providing  necessary  data  that  can  help 
drive  further  development  and  support  of  effective  online  instruction  (Clark,  2000). 

The  needs  driving  the  evaluation  of  B-leaming  in  postsecondary  education  coincide  with  the  classical  reasons  for 
evaluating  any  type  of  learning  experience  as  well  as  expand  on  thenr  As  with  any  evaluation  of  conventional 
instruction,  instructors,  administrators  and  other  stakeholders  in  online  learning  want  to  know  the  value  of  their 
course  and  program  offerings  in  terms  of  student  satisfaction  and  student  learning.  This  includes: 

• Student  satisfaction  with  the  learning  process  (interactions  with  course/program  materials,  instructor  and 
other  students) 

• Appropriate  level,  pace,  depth  and  breadth  of  instruction 

• Accomplishment  of  intended  instructional  objectives 

• Areas  in  need  of  further  instructional  and  pedagogical  development 

Although  most  university  evaluation  systems  do  not  address  transferability  of  learning  to  any  significant  degree, 
since  students  taking  distance  education  courses  are  often  out  in  the  workplace,  and  facing  real-world  demands  from 
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clients  and  supervisors,  it  is  especially  important  that  E-leaming  assessment  activities  address  whether  the  learning 
that  takes  place  is  useful  to  the  students.  Information  collected  about  this  process  might  include  whether: 

• The  content  of  the  learning  provides  a basis  of  knowledge  that  can  be  built  upon  in  later  instructional 
situations 

• The  learning  can  be  transferred  to  enhance  performance  in  real-world  settings 

In  addition  to  these  basic  assessment  needs  for  instructors,  other  stakeholders  in  the  online  learning  process 
(administrators,  technical  support,  designers,  etc.)  also  hope  to  measure  these  facts  about  the  technologies  used: 

• Performance  of  technologies  used 

• Usability  of  the  technologies 

• Reliability  of  the  technologies 

• Compatibility  of  the  technologies  with  each  other  and  existing  student  technologies 

Because  E-leaming  is  generally  a new  and  unproven  method  of  instructional  delivery  that  faces  bias  against  its 
efficacy,  cost  efficiency,  and  quality  from  those  in  traditional  academia  (Ashworth,  1996;  Cordes,  1998),  it  faces 
additional  institutional  demands  for  accountability,  such  as: 

• The  costs  and  benefits  of  installing  and  maintaining  the  technology  infrastructure  required  for  distance 
education. 

• Concerns  about  the  effect  of  distance  education  efforts  to  the  institutional  reputation 

• The  need  to  collect  data  to  assist  in  future  institutional  decision-making 

The  Significance  of  Kirkpatrick’s  Model  to  Online  Instruction 

In  general,  a rigorous  online  evaluation  system  should  measure  the  success  of  the  content,  process,  and  delivery  of 
online  instruction  in  terms  of  the  individual  student,  the  program  and/or  institution,  and/or  the  field  in  general.  It 
should  also  be  able  to  address  the  efficacy  of  its  chosen  pedagogy  and  technology.  This  paper  seeks  to  adapt  a 
model  for  the  evaluation  of  B-leaming  from  the  field  of  training  and  development  that  takes  into  account  a number 
of  different  aspects  of  the  online  learning  process.  This  four-level  model  of  evaluation,  developed  by  Don 
Kirkpatrick  (1975)  is  currently  utilized  by  67%  of  businesses  that  evaluate  their  training  (ASTD,  1997).  However, 
while  there  have  been  a few  efforts  to  apply  Kirkpatrick’s  levels  to  evaluate  distance  education  in  a higher  education 
setting  (Essex  & Cagiltay,  2001;  Hack  & Tarouco,  2000),  this  practice  is  still  far  from  commonplace. 

Once  adapted  to  higher  education  needs  and  perspectives,  Kirkpatrick’s  evaluation  model  can  be  used  to  address  the 
many  layers  of  questions  that  exist  about  E-leaming.  Specifically,  Kirkpatrick’s  model  consists  of  4 levels,  each  one 
more  extensive  in  its  purview  of  instructional  effectiveness  than  the  one  preceding  it  and  each  stage  relying  on 
achievements  gained  in  the  one  preceding  it  for  success.  Therefore,  it  is  a cumulative  model  that  can  be  used  to 
provide  in-depth  information  about  the  value  of  an  online  course  or  program.  Within  each  of  the  four  levels 
described  below,  a systematic  method  of  evaluation  is  integrated  into  the  process  that  specifies  the  goals,  objectives, 
criteria,  and  methods  of  assessment. 

The  Proposed  Evaluation  Model 
Level  1:  Student  Satisfaction 
Identify  Goals 

As  the  course  was  being  developed,  instructors  and  developers  should  have  identified  the  goals  associated  with 
attaining  student  satisfaction  with  the  course  or  program  in  terms  of  both  instructional  design  and  the  delivery 
technology.  These  goals  are  fairly  generalized,  such  as  making  sure  the  course  is  attractive  and  accessible  to 
students,  and  that  students  feel  comfortable  and  competent  with  the  course  requirements.  Ideally,  this  is  done  as  part 
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of  the  initial  course  design  process,  but  if  these  goals  were  not  previously  explicitly  identified,  the  evaluation  team 
will  need  to  work  with  the  course  designers  to  extract  them  before  the  evaluation  process  begins. 

Identify  Objectives 

Objectives  need  to  be  based  on  the  overall  goals  of  the  course,  which  are  transformed  into  objectives  through 
specific  and  concrete  definition.  The  developers  of  an  online  course  evaluation  should  measure  student  satisfaction 
based  on  the  objectives  regarding  the  instructional  materials,  the  instructional  strategies,  and  the  types  of  interactions 
provided  by  the  course.  In  terms  of  delivery  technology,  evaluation  measures  should  look  at  specific  objectives  in 
terms  of  the  ability  of  the  hardware,  software,  and  networking  to  meet  student  expectations  and  needs.  Again,  if 
these  objectives  were  not  previously  explicitly  identified,  the  evaluation  team  will  need  to  work  with  the  course 
designers  to  extract  them  before  the  evaluation  process  begins. 

Develop  Performance  Criteria 

In  the  design  phase,  developers  and  instructors  of  online  courses  need  to  determine  appropriate  criteria  for 
measuring  the  success  (student  satisfaction)  of  the  instruction,  and  these  criteria  need  to  be  based  on  the  course  goals 
and  objectives  previously  decided  upon.  Furthermore,  Clark  (2000)  points  out  that  it  is  important  to  separate 
instructional  technology  from  delivery  technology  when  evaluating  online  courses.  Some  examples  of  performance 
criteria  for  instructional  technology  are:  whether  the  course  met  student  goals;  if  the  level  of  instruction  was 
appropriate  to  the  student  audience;  if  the  pace  was  appropriate;  if  the  course  covered  an  appropriate  range  and 
depth  of  content;  and  if  the  student-student  and  student -instructor  interactions  in  the  course  were  considered  to  be 
helpful  by  students  to  the  learning  process.  Criteria  for  delivery  technology  might  include  whether  it  was 
compatible  with  student  equipment  and  existing  institutional  systems,  and  its  reliability,  usability,  and  speed. 

Specify  Data  Collection  Method 

Evaluation  at  Kirkpatrick  Level  1 can  be  done  on  either  a summative  (end-of-term)  or  formative  (midterm)  basis,  or 
both  The  most  common  method  of  collecting  Level  1 data  is  through  a survey  instrument,  usually  administered  as  a 
summative  student  course  evaluation  that  asks  student  opinions  on  whether  the  course  fulfilled  their  objectives. 
Document  analysis  is  also  a popular  method  of  evaluation  with  online  courses,  since  the  unique  nature  of  an  E- 
course  is  that  a record  of  all  discussions,  assignments  and  group  work  is  maintained.  Other  statistics,  such  aswhen, 
how  often,  and  how  long  students  access  various  aspects  of  the  course,  are  also  available  in  server  log  documents. 
There  are  also  options  to  collect  formative  data  that  can  be  used  during  the  course  of  the  instruction  to  guide  changes 
in  course  delivery  and  design.  Focus  groups  provide  students  a forum  in  which  to  build  on  each  other’s  impressions 
of  the  course,  and  these  can  be  implemented  in  secure  online  forums,  where  students  can  provide  open  and  honest 
feedback  that  is  later  summarized  anonymously  for  the  instructor..  Individual  student  interviews,  while  more  time  - 
consuming,  can  allow  for  deeper  expbration  of  student  attitudes  and  experiences  with  the  course.  Observations  of 
the  course  by  a faculty  colleague  or  instructional  consultant  (attained  by  allowing  them  access  to  the  course 
discussion  forums  and  other  documents  and  communication)  can  also  reveal  much  about  student  engagement,  levels 
of  interaction,  and  general  conduct  of  the  course. 

Level  2:  Student  Learning 

Identify  Goals  of  the  Instructional  Effort 

Cross  & Angelo  (1993)  identify  five  basic  areas  of  higher  education  from  which  instructional  goals  are  drawn.  They 
are:  higher  order  thinking  skills  (problem  solving,  analysis,  synthesis);basic  success  skills  (reading,  writing, 
listening,  speaking);  field  dependent  knowledge  and  skills  (methods,  materials,  tools);  academic  perspectives  and 
values  (objectivity,  rigor,  collaboration);and  personal  development  attitudes  and  skills  (responsibility,  leadership, 
commitment).  Theoretically,  the  student  learning  goals  for  an  online  course  do  not  differ  from  those  of  a traditional 
course.  However,  competency  with  the  technology  itself  always,  whether  implicitly  or  explicitly,  comprises  part  of 
the  online  instructional  process  and  the  basic  requirements  for  success  in  an  online  course. 
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Identify  Objectives  of  the  Instructional  Effort 

Some  objectives  related  to  higher  order  thinking  skills  might  be:  identifying  the  important  elements  of  a problem; 
understanding  the  relationship  between  the  elements;  and  determining  what  changes  in  which  elements  need  to  be 
made  to  solve  a problem.  Basic  success  skills  objectives  might  include:  understanding  what  is  important  and  what  is 
secondary  in  written  information;  articulating  ideas  in  written  format;  listening  to  others'  opinions  and  ideas;  and 
articulating  ideas  in  verbal  interactions  and  presentations.  Objectives  relating  to  field-dependent  knowledge  may 
encompass:  understanding  which  methods  are  appropriate  and  their  uses  in  the  field;  identifying  appropriate 
materials  and  their  characteristics;  and  selecting  the  best  tools  for  the  task(s)  and  knowing  how  to  utilize  them. 
Objectives  developed  under  academic  perspectives  and  values  might  be:  learning  how  to  approach  intellectual 
problems  with  objectivity;  understanding  the  thoroughness  required  for  rigor;  describing  and  defining  practices  that 
lead  to  teamwork  and  accomplishing  collaborative  goals.  Personal  development  attitudes  and  skills  objectives  might 
include:  responsibility  to  one’s  work  within  the  discipline;  leadership  in  the  field;  and  commitment  to  one’s  own 
values. 

Develop  Performance  Criteria 

Defining  performance  criteria  for  assessing  student  learning  is  normally  part  of  any  instructional  module  or  course, 
and  leads  the  evaluation  of  student  learning.  Specifically,  criteria  generally  fall  into  two  categories  of  student 
performance  (demonstrating  learning):  those  that  pertain  to  understanding  of  content;  and  those  that  pertain  to  the 
mastery  of  a process. 

Specify  or  Review  Existing  Assessment  Methods 

Except  for  evaluation  of  the  student’s  mastery  of  technology,  this  level  of  evaluation  may  borrow  heavily  from 
established  course  materials  and  evaluation  instruments;  e.g.,  examinations,  written  assignments,  projects,  and 
portfolios.  These  evaluations  for  the  E-course  can  then  be  analyzed  in  exactly  the  same  way  as  they  are  in 
traditional,  on-campus  courses,  such  as  providing  average  student  scores  and  grade  percentages  for  individual 
assignments  to  grade  point  averages  for  the  course.  Student  mastery  with  the  technology  can  be  tested  directly,  such 
as  by  giving  them  technology-related  tasks  to  perform,  or  inferred  by  their  success  with  the  assignments  in  general. 
If  grades  are  not  an  issue,  mastery  learning  techniques  will  provide  instructors  with  the  number  of  modules  or 
instruments  completed  at  a given  proficiency  level  (which  should  ideally  be  at  or  near  100%). 

Level  3:  Transfer  of  Learning 

Identify  Goals  of  Transfer  Learning 

The  goals  of  Level  3 evaluation  are  twofold.  First  of  all,  learning  in  an  academic  setting  is  often  considered  to  be 
sequential  and  progressive.  That  is,  basic  knowledge  and  understanding  must  be  acquired  before  students  may 
“advance”  to  higher  order  thinking  and  problem  solving.  This  is  the  philosophy  behind  offering  some  courses  in 
sequence,  and  requiring  that  the  preceding  course  be  successfully  completed  before  moving  on  to  the  next  level. 
Secondly,  the  goal  of  much  higher  education  is  that  student  learning  will  be  transferred  to  reaHife 
situations/problems  (such  as  internships,  student  teaching,  service  learning  projects,  professional  tasks,  etc.). 

Whether  E-leaming  focuses  on  the  professional  development  needs  of  those  already  in  the  field,  or  are  a part  of 
university  programs  in  professional  education,  successful  transfer  of  learning  is  usually  the  ultimate  goal  of  the 
educational  process.  Thus,  they  often  encompass  the  goals  of  both  pre -service  professional  training  and  the  further 
development  of  professionals  in  the  field.  Transfer  of  learning,  though,  is  often  difficult  to  evaluate,  since  it  must  be 
evaluated  after  the  learning  process  takes  place,  and  in  a setting  other  than  the  real  or  virtual  classroom.  Universities 
are  not  accustomed  to  tracking  student  learning  post  instruction,  except  for  programs  with  comprehensive  or 
qualilying  exams  or  summative  portfolios.  However,  since  transfer  of  learning  is  so  important,  any  rigorous 
assessment  program  must  include  this  aspect  in  its  design. 

Identify  objectives 
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The  objectives  of  transfer  learning  are  diverse  and  depend  heavily  on  the  field  of  study,  but  in  general  the  objectives 
will  address  students*  ability  to  access  specific  skills  and  relevant  knowledge  post  instruction,  and  their  knowledge 
about  when  and  how  to  use  these  skills  and  relevant  knowledge  for  real-world  problems  and  tasks. 

Develop  Performance  Criteria 

Specific  performance  criteria  for  transfer  learning  will  depend  heavily  upon  the  field  of  study  and/or  the  professional 
field  involved,  and  the  level  at  which  the  students  work  in  that  field.  Professional  fields  often  determine  the  criteria 
that  their  members  must  meet  through  granting  licenses  and  certification.  Academic  fields  typically  measure 
performance  through  instruction,  research  and  publication.  Subprofessional  performance  criteria  in  both  of  these 
fields  often  approximate  professional  tasks,  but  at  a lower  level  of  masteiy. 

Specify  Methods  to  Evaluate  Transfer  Learning 

In  both  academic  and  professional  settings,  the  transferability  of  learning  can  be  assessed  using  simulated,  authentic 
scenarios  or  case  studies  in  which  the  students’  problem-solving  skills  are  put  to  tes  t.  In  order  to  approach  real-life 
situations,  cases  must  be  complex,  ill -defined  (Savery  & Duffy,  1996)  and  require  the  use  of  higher-order  thinking 
skills  on  the  part  of  the  students.  To  further  authenticate  the  process,  when  the  task  would,  in  the  real-world,  be  a 
collaborative  effort,  students  can  be  required  to  work  in  teams  so  that  the  skills  of  communication,  negotiation,  and 
compromise  are  also  implicated  in  the  solution.  In  these  scenarios  or  cases,  roles  can  either  be  initially  defined  for 
the  students  or  can  evolve  naturally  from  the  process.  Another  option  for  assessing  transfer  learning,  if  the  students 
are  employed  or  interning  in  a real-world  professional/academic  environment,  is  that  the  students  may  be  observed 
in  real-life  situations  by  supervisors,  by  colleagues,  and  by  clients  (students).  Another  option  would  involve  the 
students  submitting  a video  or  audiotape  of  a job-related  performance  for  evaluation. 

Level  4:  Return  on  Investment  or  Business/Academic  Impact 

Identify  Goals 

Whereas  in  a corporate  training  program  the  return  on  investment  (ROI)  of  any  online  training  effort  is  measured  in 
terms  of  whether  or  not  the  cost  of  the  training  is  more  or  less  than  the  additional  profit  gained  through  performance 
improvement  based  on  the  training,  academic  programs  define  success  in  different  ways.  While  the  financial  aspect 
still  exists  for  the  academic  world,  because  most  students  hope  to  be  in  a better  financial  position  after  a course  of 
study  than  before,  there  may  be  other  indicators  of  success  for  students  that  they  count  into  their  calculations  of  ROI, 
such  as  increased  job  satisfaction,  lower  stress,  etc.  In  any  case,  deciding  whether  a course  “was  worth  it,”  is  often  a 
very  personal  kind  of  ROI  calculation,  and  difficult  to  quantify.  Looking  at  ROI  from  an  institutional  perspective, 
colleges  and  universities  often  look  to  intermediary  indices,  only  some  of  which  indirectly  refer  to  the  financial 
health  of  the  organization.  Specifically,  academic  institutions  and  programs  often  use  academic  reputation,  program 
rankings,  and  accreditation  status  as  indicators  of  success.  Although  the  reputation  is  somewhat  ephemeral,  high 
program  ranking  established  by  a reputable  organization  and  accreditation  through  appropriate  organizations  are 
attributes  that  institutions  of  higher  education  that  offer  E-leaming  are  not  willing  to  sacrifice.  Other  indicators  of 
program  success  are  the  success  of  graduates  of  online  programs  as  well  as  increasing  popularity  with  the  E-leaming 
program  for  students. 

Identify  Objectives 

Calculating  objectives  related  to  ROI  from  an  individual  student  perspective  can  be,  as  stated  above,  a very  personal 
process,  involving  an  estimation  of  how  much  the  learning  from  a given  course  or  program  improves  the  life  ofthe 
learner  in  terms  of  fulfilling  monetary,  career  and  personal  goals.  From  the  institutional  perspective,  objectives 
relating  to  academic  or  program  reputation  may  also  be  difficult  to  decide  upon,  since  much  of  reputation  is 
established  through  word-of-mouth  and  on  other  informal  bases.  School  and/or  program  rankings,  however, 
generally  occur  at  predictable  intervals  through  identifiable  evaluative  agencies.  Therefore,  objectives  can  be  set 
which  relate  to  these  specific  evaluative  events.  In  addit  ion,  objectives  can  be  set  related  to  a positive  change  in  the 
type  and  level  of  positions  attained  by  graduates  of  the  program  or  institution,  and/or  positive  changes  in 
institution/program  enrollment  rates. 
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Develop  Criteria  for  Positive  Changes 


Again,  students  will  determine  their  own  individual  criteria  for  assessing  the  ROI  relative  to  a given  course  or 
program  of  study.  The  threshold  for  which  the  student  decides  that  the  course  or  program  was  “worth  it”  may  be 
difficult  to  quantify,  but  most  students  will  have  an  implicit  sense  of  it.  From  the  institutional  perspective,  given  the 
imprecise  nature  of  measuring  a concept  such  as  institutional  reputation,  criteria  for  assessing  an  institution’s  or 
program’s  reputation  may  merely  focus  on  the  documentation  of  a stakeholder’s  opinions  regarding  positive 
changes.  Positions  attained  by  graduates  of  a program  can  be  evaluated  along  the  lines  of  perceived  status,  entry 
salaries,  and/or  numbers  hired  within  a certain  period  of  time.  Rankings  and  enrollment  rates,  however,  can  reveal 
quantitative  changes,  in  which  positive  changes  can  be  tracked  and  documented.  Care  must  be  taken  to  ensure  that 
perceived  positive  change  is  not  the  result  of  other,  external  factors  such  as  changes  in  the  nature  of  the  field, 
changes  in  professional  demands,  changes  in  the  availability  of  resources,  or  changes  in  other  policies  or  regulations. 

Develop  Evaluation  Methods 

Evaluating  student  assessment  of  ROI  may  simply  consist  of  asking  them,  in  a survey  or  interview  format,  whether 
they  regard  the  benefits  of  the  course  of  study  to  have  been  greater  than  the  associated  costs.  Since  the  objectives  of 
each  student  and  the  criteria  for  measuring  these  objectives  are  likely  to  be  so  idiosyncratic,  it  would  be  difficult  or 
perhaps  impossible  to  get  at  a more  objective  assessment.  From  the  institutional  side,  evaluation  methods  for 
determining  the  impact  of  an  online  program  on  an  institution  or  program  can  be  accomplished  through:  inspection 
of  published  ratings/rankings;  the  hiring  rates  of  graduates;  the  level  of  positions  obtained  by  the  graduates;  the 
accreditation  and  licensing  status  of  graduates;  the  accreditation  status  of  the  program,  and  enrollment  rates.  These 
measures  all  provide  data  that  can  be  used  to  indicate  program  success.  Data  may  also  be  collected  on  the  status  of 
graduates  through  surveys,  interviews,  and  focus  groups  with  them. 


Conclusion 

The  major  advantage  of  the  Kirkpatrick  model  of  evaluation  is  that  it  fosters  a thorough  assessment  of  the 
instructional  process  by  viewing  the  process  systematically,  through  four  different  lenses.  Most  college  and 
university  efforts  to  evaluate  E-leaming  are  not  done  systematically  and  fall  short  of  their  goals,  either  by 
emphasizing  student  satisfaction  for  the  process  over  the  actual  learning  achievements  or  lack  thereof,  by 
concentrating  on  positive  or  negative  aspects  of  the  technology  while  overlooking  the  learning  process  altogether, 
and/or  by  ignoring  whether  or  not  the  learning  will  transfer  to  the  real  world  environments  that  the  learner  will  face. 
The  Kirkpatrick  model  can  also  be  used  to  integrate  evaluation  on  the  local  scale  with  a more  systematic  and 
administrative  look  at  institutional  and  programmatic  influences.  This  new  evaluation  method  helps  put  into 
perspective  data  that  can  inform  future  university  decisions  about  their  technology  infrastructure,  the  professional 
development  of  faculty  involved  in  online  learning,  and  the  prospective  role  of  E-leaming  in  university  programs. 
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Abstract: 

The  WWW  has  developed  as  the  de  facto  standard  for  computer  based  learning. 
However,  as  a server-centered  approach  it  confines  readers  and  learners  to  passive  non- 
sequential reading.  Authoring  and  web-publishing  systems  aim  at  supporting  the  authors' 
design  process.  Consequently,  learners'  activities  are  confined  to  selecting  and  reading 
(downloading  documents)  with  almost  no  possibilities  to  structure  and  arrange  their 
learning  spaces  nor  do  that  in  a cooperative  manner. 

This  paper  presents  a learner- centered  - completely  web-based  - approach  through 
virtual  Imowledge  rooms.  Based  on  this  concept,  the  goal  of  the  presented  work  is  firstly 
to  develop  a theoretical  framework  to  explain  the  design  potentials  of  technology - 
supported  baming  processes  (distinguishing  individual  and  cooperative  primary  media 
functions).  Secondly,  a technical  framework  (cf.  www.open-steam.org)  should  be 
developed  allowing  us  to  study  different  technical  configurations  within  the  traditional 
university  setting.  Considering  the  systems  design  the  concept  of  virtual  knowledge 
rooms  is  to  combine  event-based  technology  of  virtual  worlds  with  the  classical 
document  management  functions  in  a client-server  framework.  Knowledge  rooms  and 
learning  materials  such  as  documents  or  multimedia  elements  are  represented  as  a fully 
object  oriented  model  of  objects,  attributes  and  access  rights.  We  do  not  focus  on 
interactive  systems  managing  individual  access  rights  to  knowledge  bases,  but  rather  on 
cooperative  management  and  structuring  of  distributed  knowledge  bases. 


Introduction 

Discussions  about  the  role  of  technology  in  teaching  and  learning  center  on  two  basic 
paradigms.  Firstly  on  hypermedia  systems  that  aim  to  support  individual  learning 
processes,  here  special  emphasis  is  being  placed  on  new  didactic  qualities,  attributed  to 
the  interactive  combination  of  different  media  types  such  as  text,  graphics,  audio,  video, 
etc.  The  second  paradigm  embodies  the  notion  of  "delivering  education"  that  is 
networking  technology  being  used  to  distribute  and  access  study  materials  as  well  as  to 
establish  communication  channels  between  students  and  teachers.  Although  the  idea  of 
utilizing  net  services  has  strong  collaborative  connotations,  the  main  argument  in  favor  of 
networking  is  the  temporal  and  spatial  independence  it  offers  and,  consequently,  the 
independence  from  close  collaboration  with  others. 

Now,  students  can  learn  individually  at  their  own  pace  and  at  a location  chosen  by 
themselves. 

However,  many  studies  evaluating  the  role  of  technology  in  learning  processes  have 
yielded  conflicting  results,  indicating  that  there  is  no  general,  clear-cut  connection 
between  the  effort  required  to  produce  high-quality  multimedia  educational  materials  and 
improvements  in  the  learning  process.  The  problem  is  to  attribute  certain  benefits  to  a 
single  variable  - say,  the  specific  technology  used.  Mostly,  the  results  are  a combination 
of  different  variables  such  as  didactic  style,  educational  strategy,  technology  deployment, 
appropriate  selection  of  content  and  the  personal  qualities  of  teachers  and  students,  (see 
[Hesse  1997]  and  [Keil-Slawik  et  al.  1996]).  Thus,  the  widely  accepted  idea  that  learning 
can  be  improved  by  the  individualization  of  learning  processes  using  hypermedia  and 
networking  technology  is  not  generally  borne  by  scientific  research. 
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Instead  of  using  technology  to  individualize  learning  processes,  we  have  chosen  the 
opposite  approach  - namely,  using  technology  to  support  cooperative  learning  processes 
within  the  framework  of  traditional  university  education.  Rather  than  trying  to  do  away 
with  the  need  for  physical  presence  in  the  learning  situation,  we  seek  to  support  social 
processes  in  which  students  and  teachers  meet  for  a specific  time  at  different  places. 
Apart  from  practical  considerations,  theoretical  research  reveals  that,  ultimately,  the 
social  embedding  of  learning  is  a crucial  factor  for  success. 

Furthermore,  our  approach  is  based  on  the  general  assumption  that  technology  can  only 
solve  technological  problems,  didactic  problems  requiring  didactic  solutions.  Hence, 
teaching  activities  cannot  be  replaced  by  technological  functions,  and  the  embedding  of 
technology  into  teaching  and  learning  activities  must  be  studied  carefully. 

Media  Functions 

Media  are  generally  viewed  in  terms  of  communication,  based  in  most  cases  on  the 
classical  transmitter/  receiver  or  producer/  consumer  model.  The  integration  of  different 
media  types  such  as  text,  image  and  sound  (multimedia)  in  combination  with  the  fusion 
of  transmission  channels  and  services  (Internet),  make  it  necessary  to  extend  our  concept 
of  media.  Media  are  no  longer  simply  means  of  communication;  they  are  - and  have 
always  been  - both  means  of  expression/  cognition  and  means  of  organization.  Without 
media  our  cultural  achievements  are  inconceivable.  Complex  social  processes  based  on 
the  division  of  labor  are  just  as  reliant  on  media  as  they  are  on  science  and  education. 
Media,  for  their  part,  require  technology  to  create  an  objective,  mostly  symbolic  world. 
To  determine  the  potential  and  possibilities  offered  by  new  media,  we  must  first  remind 
ourselves  of  what  constitutes,  in  technical  terms,  the  benefits  they  provide. 

Keil-Slawik  and  Selke  at  first  distinguish  three  classes  of  media  functions,  called 
primary,  secondary  and  tertiary  media  fimctions.  They  closely  examine  the  relevance  of 
technology  in  human  learning  processes.  Primary  media  functions  describe  fundamental 
functions  to  place  artifacts  in  the  perceptual  space  of  a person  (a  group  of  persons). 
Secondary  media  functions  extend  them  taking  the  learning  process  itself  into  account, 
thus  they  refer  to  applicable  didactic  models.  Tertiary  media  functions  consider  methods 
of  self  adapting  and  artificial  intelligence,  that  means,  now  the  medium  itself  is  subject 
matter  (see  [Keil-Slawik  & Selke  1998]  and  [Hampel  2001]). 

The  next  important  step  is  to  differentiate  between  primary  individual  media  functions 
and  primary  cooperative  media  functions.  Concerning  the  primary  cooperative  media 
functions  the  primary  individual  media  functions  such  as  Creating,  Deleting,  Arranging 
and  Linking  are  seen  in  the  context  of  the  cooperative  taming  process.  Consequently, 
the  primary  cooperative  media  functions  are  realized  through  the  options  of  Transmitting, 
Accessing  and  Synchronizing.  To  sum  up,  we  distinguish  four  primary  individual  media 
functions  and  three  primary  cooperative  media  functions  as  follows: 

Primary  individual  media  functions: 

Creating,  Deleting  - Media  serve  to  create  or  delete  a perceptual  space.  Concerning  the  first  dlowing 
conception  and  reality  to  be  correlated  by  action  and  the  relevant  conclusions  to  be  drawn.  Scientific 
instruments,  experimental  apparatus,  models  and  simulation  programs  are  just  as  much  instances  of  artifacts 
that  perform  this  function  as  are  symbolic  descriptions,  diagrams,  images,  formalisms  and  visualizations  of 
large  data  sets. 

Arranging  - To  attain  new  insights  it  is  necessary  to  correlate  different  documents.  Problem  solving  and 
learning  invariably  involve  identifying  differences  and  concurrence,  combining  different  types  of  descriptions 
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and  representations  or  weighing  statements  from  different  sources  against  one  another.  To  support  these 
processes,  the  artifacts  to  be  correlated  must  be  brought  into  the  field  of  perception,  simultaneously  F 
possible.  Here,  logical  connections  should  be  represented,  if  possible,  by  spatial  connections  as  well,  to 
enable  them  to  be  swiftly  identified  and  processed. 

Linking  - Arrangements  embodying  an  important  connection  should  be  preserved  after  the  act  of  arranging.  In 
this  way,  they  do  not  need  to  be  regenerated  at  a later  point  in  order  to  continue  working  with  them.  Suitable 
links,  for  example,  enable  users  - ideally  in  a single  step  - to  refer  to  all  documents  that  are  of  relevance  for 
the  respective  meaning  context. 


Primary  cooperative  media  functions 

Transfer  - The  transfer  of  media  between  learners.  Cultural  learning  achievements  are  social  processes. 
Reducing  learning  to  the  individual  processing  of  a document,  e.g.,  means  failing  to  realize  that  the  same 
document  has  already  been  used,  assessed  and  passed  on  in  a specific  social  teaching/leaming  situation.  The 
primary  cooperative  media  function  “Transfer’’  is  meant  to  describe  the  exchange  of  media  between  two 
persons  or  between  one  person  and  an  object,  which  arranges  an  exchange  to  a person. 

Accessing  - Giving  access  rights  to  shared  material,  cooperative  use  of  individual  primary  media  functions: 
With  the  media  function  “Transfer”  we  explicitly  address  one  person,  whereas  the  media  function  of 
Accessing  describes  the  access  as  a process  which  is  not  influenced  by  others,  such  as  the  creator.  An 
illustrative  example  provides  the  filing  of  a document  in  a virtual  room.  Depending  on  the  access  rights  to  the 
concerning  document  it  is  possible  to  access  it  by  “reading”  and  “writing”.  Another  example  is  the  use  of  a 
Shared  Whiteboard,  which  offers  the  possibility  to  access  materials  of  another  person  or  to  access  materials 
from  different  learning  places. 

Synchronizing  - Conveying  shared  views  on  shared  objects,  coupling,  awareness:  We  can  identify  the  media 
function  of  Synchronizing  in  physical  acts  of  updating  presentations.  On  the  one  hand  these  acts  are  about 
creating  shared  views  on  documents  and  the  report  about  changes  in  these  documents  and  on  the  other  hand 
the  awareness  of  cooperation  partners.  Primary  individual  media  functions  in  a cooperative  structure  require 
shared  views  of  the  partners  on  their  materials. 


The  rationalization  potential  of  new  media  lies  in  the  implementation  and  integration  of 
these  primary  media  functions,  concerning  the  more  effective  handling  of  the  physical 
artifacts  in  terms  of  the  mentioned  basic  areas  of  functionality.  Here,  the  new  media  offer 
a wealth  of  new  forms  for  the  effective  generating,  transferring,  giving  access  to, 
arranging  and  linking  semiotic  artifacts,  ranging  from  the  integration  of  different  types  of 
media  to  net-based  services  and  search  facilities. 

To  this  extent,  learning-supportive  infrastructures,  like  the  sTeam-system  make  use  of  all 
materials  at  any  location  where  learning  takes  place. 

Architecture 

The  developed  basic  architecture  combines  two  different  approaches.  On  the  one  hand 
the  development  of  document  based  CSCW/  CSCl^ Systems  (mostly  office  motivated) 
and  on  the  other  hand  the  event-based  architecture  of  MUDs  and  MOOs.  As  a result  we 
have  a flexible  structure  of  cooperative  virtual  knowledge  spaces  which  is  strongly 
related  to  the  WWW,  but  furthermore  develops  new  methods  of  managing  documents 
and  other  materials. 

Interactions  among  users  and  between  users  and  documents  are  realized  as  interactions 
between  cooperating  shared  objects.  Shared  objects  are  persistent  and  are  oriented 
towards  a room  structure  (room  metaphor).  Access  rights  control  the  instance  of  new 
objects  and  the  interaction  between  objects.  These  interactions  may  take  place  between 
objects  in  client  and  server. 

The  complex  design  of  the  basic  architecture  originates  in  the  need  for  a scalable  and 
easy  to  maintain  architecture  which  can  easily  be  suited  to  exiting  and  future  standards. 
Design  patterns  describe  recurring  patterns  in  an  object- orientated  software  engineering. 


The  designed  basic  architecture  is  subdivided  in  a sTeam-server,  an  external  Web  Server, 
a relational  database  and  several  clients. 

The  sTeam-server  coordinates  the  administration  of  users  and  groups,  provides  for  the 
room- structure  with  its  containing  objects,  organizes  the  communication  between  users 
and  realizes  the  S5mchronization  of  the  presented  contents  between  the  different  clients. 
The  Roxen  Web  Server  is  used  as  a gateway  to  access  different  Web  browsers  of  sTeam. 
The  relational  database  guarantees  the  persistence  of  the  various  objects  (see  [Hampel  & 
Keil-Slawik  2001]). 

Core  server 

The  sTeam-core  server  provides  for  the  necessary  mechanisms  which  guarantee  the 
reciprocal  perception  and  the  cooperative  use  of  knowledge  structures.  As  runtime 
environment,  the  open  source  LPC  interpreter  Pike  is  employed.  Aside  from  a good 
performance  Pike  is  equipped  with  a wide  code  base  and  a great  number  of  existing 
libraries.  The  native  Pike  libraries  are  supplemented  by  a connection  to  different  open 
source  libraries  which  are  implemented  in  the  programming  language  C for  performance 
reasons. 

The  concept  of  cooperative  knowledge  spaces  demands  a special  administration  of 
objects,  users  and  different  events.  Thus  the  chosen  architecture  follows  an  entirely 
object-oriented  design.  Objects  own  different  attributes  and  sometimes  in  the  case  of 
document  types  a content.  Interactions  between  objects  are  guaranteed  by  remote  method 
calls  or  events.  The  runtime  environment  is  composed  of  further  modules  for  the 
implementation  of  various  protocols.  NNTP,  POP3,  IRC,  LDAP  and  COAL  are  directly 
supported.  By  applying  NNTP  the  annotations  of  documents  are  presented  as  news.  POP3 
is  employed  to  integrate  news  that  are  transmitted  within  the  sTeam  platform  into  the 
usual  working  environment.  The  chat  channels  connected  to  rooms  are  released  by  the 
IRC  protocol.  Finally  LDAP  permits  the  access  to  the  user  administration  within  the 
learning  environment  of  the  institution  where  the  system  is  employed.  This  way  within  a 
certain  university  students  could  get  access  to  the  cooperative  learning  environment  by 
using  their  general  identification  and  password.  The  communication  between  server  and 
clients  is  guaranteed  by  the  implementation  of  a specific  communication  layer,  the  Client 
Object  Access  Layer  CAOL.  Currently  COAl^API  is  implemented  for  the  programming 
languages  Java,  C-i-i-  and  Pike.  The  COAL-/Java-API  allows  the  login/logout  of  a client 
and  the  subscription  and  the  reception  of  events  from  the  server,  furthermore  method 
calls  and  the  transfer  (up-,  down-load)  of  objects.  The  object  structure  of  the  server  is 
reproduced  in  an  analogue  way  on  the  client  side.  For  each  server  object  type  a 
corresponding  class  exists  in  the  COAL/Java-API.  Instances  of  these  classes  correspond 
to  real  objects  on  the  server  but  they  do  not  contain  data  and  contents  of  the  object.  A 
replication  of  objects  does  only  reveal  a part  of  the  viewed  or  needed  objects.  Thus 
through  the  generated  proxy-objects  server  objects  may  be  accessed  by  direct  method 
calls  (e.g.  in  the  case  of  attribute  changes). 

For  different  reasons  the  Web  server  is  separated  from  the  core  architecture.  We  employ 
the  open  source  Roxen  Web  server  (cfwww.roxen.com).  In  this  way  protocols  which  are 
implemented  within  Roxen  (e.g.  the  HTTP-protocol  coded  by  the  Secure  Socket  Layer 
(SSL)  can  be  immediately  used.  Beside  of  the  very  robust  implementation  of  the  HTTP 


and  the  FTP  protocols  the  gain  in  security  seems  to  be  an  important  aspect  for  the 
acceptance  of  the  system.  The  database  responds  to  the  generic  SQL  interface  of  the 
programming  language  Pike.  At  the  present  time  the  free  database  MySQL  is  accessed. 
Through  an  abstraction  layer,  referring  to  papers  from  Ambler  about  the  mapping  of 
object  structures  on  relational  databases,  (see  [Ambler  2000]),  the  entire  object 
administration  is  separated  from  the  database  logic.  Thus  the  independence  from  the 
really  employed  database  technology  is  guaranteed. 

Evaluation 

Designing  technology  only  based  on  hypothesis  calls  for  empirical  confirmation  in  order 
to  have  evidence  of  the  usability  of  the  own  results.  Therefore  we  carried  out  the  lecture 
“Grundlagen  der  Informatik  fur  Lehramter”  (“Basics  of  computer  science  for  prospective 
teachers”).  Our  evaluation  concentrates  on  the  use  of  virtual  knowledge  spaces  as  a 
structuring  and  learning  platform  for  cooperative  knowledge  management.  As  such  it 
provides  the  students  with  diverse  possibilities  to  wDrk  in  cooperation  with  others  as  well 
as  to  work  for  themselves.  Technically,  we  speak  of  a collection  of  documents  directly 
assigned  to  a synchronous  communication  channel.  It  is  possible  for  the  students  to  attach 
comments  to  all  of  the  documents.  The  ised  configuration  allows  the  students  to  insert 
documents,  to  annotate,  to  copy  and  to  create  new  links.  Furthermore  they  could  create 
new  rooms  and  link  already  existing  rooms  with  new  exits.  The  setting  for  the  mentioned 
lecture  is  characterized  by  three  groups  of  virtual  rooms.  These  are  private  areas  of  the 
students,  assigned  areas  for  the  different  learning  groups  and  one  central  area  containing 
all  of  the  course  materials. 

The  main  goal  of  this  setting  was  to  prompt  the  students  to  search  for  additional  material, 
which  could  be  presented  to  the  other  students.  Comments  and  annotations,  which  are 
connected  to  the  documents  should  help  the  students  discuss  the  content  of  the  course 
with  each  other  and  with  the  lecturer  throughout  the  semester.  Thus,  the  contents  of  the 
lecture  can  be  structured  in  an  according  way  and  new  insights  may  be  drawn  from  the 
different  personal  styles  of  structuring  information. 

Based  on  observations  during  the  first  weeks  the  students  rather  annotated  any  material, 
thus  a new  area  called  “Speakers  Comer”  was  created.  The  goal  was  to  focus  on  the 
possibilities  for  discussions  in  only  one  special  area.  Furthermore  two  students  were 
asked  weekly  to  select  five  slides  of  a presentation  which  in  their  opinion  represented 
main  aspects  of  the  lecture.  To  discuss  these  slides,  the  students  should  as  well  annotate 
why  they  had  especially  selected  the  chosen  ones.  Now  all  the  other  students  were  asked 
to  comment  on  the  selection,  to  present  further  arguments  or  to  show  other  possibilities. 
Solutions  to  the  weekly  exercises  were  given  as  annotations  to  the  respective  documents 
if  possible.  If  the  answers  were  more  extensive,  another  way  was  to  create  new 
documents  or  even  new  rooms.  It  was  very  interesting  to  see  that  the  students  agreed  on 
publishing  their  results.  Thus  it  was  easier  to  discuss  the  results  and  refer  to  other 
solutions.  As  a first  surprising  outcome  of  this  experiment,  it  was  observed  that  the 
possibilities  to  stmcture  the  learning  environment  newly  were  not  used  as  much  as  we 
thought  it  would.  Such  a lack  in  the  use  of  provided  technical  possibilities  can  be  traced 
back  to  several  aspects,  which  in  the  following  have  to  be  examined  more  closely. 

One  possible  hypothesis  is  that  the  method  of  creating  new  exits,  etc.  is  too  complicated 
(thesis  I).  Another  explanation  might  be  that  the  students  realize  only  slowly,  which  new 
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competencies  they  have  while  working  with  the  system  (thesis  II).  We  have  to  examine 
how  far  this  problem  is  a principal  one,  that  means,  how  far  hierarchical  structures  are 
internalized  so  that  the  students  do  not  deviate  from  them  (thesis  III).  One  point  to 
confirm  the  correctness  of  thesis  I is  the  observation,  that  students  only  used  aimotations 
where  they  were  forced  to  do  so. 

To  sum  up,  the  six  month  practical  use  may  be  assessed  positively.  The  concept  of 
cooperative  virtual  knowledge  spaces  has  mastered  its  first  practical  application  period. 

Conclusions 

Virtual  knowledge  spaces  are  a new  approach  towards  the  extension  of  traditional 
hypertext  systems  in  their  way  of  structuring  documents  with  event-based  cooperative 
approaches.  At  the  same  time,  cooperative  knowledge  spaces  create  new  design  conflicts 
which  have  to  be  handled  in  a systematical  and  well-founded  way  to  gain  from  their 
inherent  creative  potential.  The  main  goal  of  our  research  is  to  acquire  the  necessary 
knowledge  and  to  develop  a theoretical  foundation.  Therefore  we  need  a basic 
architecture,  which  allows  us  to  realize  a variety  of  virtual  knowledge  spaces  with  a small 
amount  of  time  and  energy.  During  the  last  years  we  have  succeeded  in  developing  such 
architecture.  Another  important  prerequisite  are  theoretical  concepts  which  allow  us  to 
distinguish  between  technical  and  non-technical  aspects  and  furthermore  to  examine  the 
correlation  between  them. 

To  experience  the  potentials  of  virtual  knowledge  spaces  the  system  has  been  practically 
tested  in  a traditional  learning  situation. 
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Abstract;  This  brief  contribution  describes  how  to  employ  a scripting  interface  for 
proper  embedding  of  highly  interactive  learning  objects  (Java  Applets)  into  Web- based 
courseware.  We  start  with  a short  overview  of  our  current  toolkit  of  reusable  software 
components  that  allow  the  development  of  a greater  pool  of  virtual  experiments.  We 
illustrate  how  we  overcome  their  isolation  by  scripting  from  other  learning  objects,  e.g. 
from  course  text,  illustrations,  or  exercises.  Further  we  discuss  how  to  build  up  a 
community  that  may  create  and  modify  scripts  online  by  using  online  wizards.  As  proof- 
of-concept  we  demonstrate  applications  in  the  field  of  Scientific  Visualization'. 


Introduction 

Process-oriented  learning  environments  enforce  active  self-learning  and  therefore  include  constructive 
concepts  that  are  embodied  as  interactive  learning  objects.  Today,  Java  ^plets  allow  for  intuitive,  bi- 
directional interaction  together  with  high-level  visualization  that  may  be  seamlessly  embedded  into  a Web- 
based  courseware.  In  1998  we  demonstrated  how  to  reduce  the  immense  costs  that  affords  the  development 
of  a large  pool  of  virtual  experiments  by  creating  a toolkit  of  reusable  software  components  (Klein  & 
Hanisch  1998).  Since  then,  many  other  projects  focused  on  developing  component  repositories  for  learning 
objects  (Laleuf  & Spalter  2001). 

Unfortunately,  actual  profit  of  resulting  courseware  is  reduced  by  poorly  implemented  interlinking  of 
learning  objects.  In  particular,  interactive  learning  objects  appear  isolated:  they  neither  can  be  modified 
sufficiently  (e.g.  by  choosing  parameters  or  enhancing  functionality)  nor  be  interlinked  properly  with  their 
context  (e.g.  by  synchronizing  with  a guided  tour).  We  therefore  supplied  our  virtual  experiments  with  a 
scripting  interface.  Scripting  enables  authors  and  learners  to  steer  interactive  learning  objects  by  textual 
commands  defined  within  other  learning  objects  (Christian  2000). 

To  enable  community  members  to  take  part  in  the  development  of  our  courseware,  we  set  up  online 
wizards.  Single  learning  objects  may  be  modified,  annotated,  or  rated.  We  deal  with  non-interactive  and 
interactive  elements,  e.g.  we  implemented  prototypes  that  enable  authors  to  modify  an  experiment’s  set  of 
scripts. 

Software  Components 

Our  component-based  architecture  consist  of  geometry  objects,  renderers,  constraints,  scene  graph 
components,  and  GUI  elements.  Renderers  encapsulate  an  object’s  visual  appearance.  The  may  hold 
arbitrary  properties  to  describe  their  visual  appearance  (e.g.  colors,  line  strength,  strokes).  Similar, 
constraints  separate  an  object’s  construction  (e.g.  orthogonal  lines)  and  update  dependencies  automatically. 
A plug-in  architecture  assures  extensibility.  The  object  itself  (point,  mesh,  etc.)  contains  only  primitive 
functionality  together  with  a transformation  cache.  Adapters  synchronize  objects  with  GUI  elements  (e.g.  a 
scalar  with  a slider)  automatically.  Using  a hierarchical  scene  graph  we  result  in  default  interaction 
behavior  and  visualization  for  all  our  experiments.  Scene  graph  nodes  contain  a geometry  object  together 
with  a matching  renderer  and  an  interaction  sub-tree  that  provides  the  desired  interaction  behavior  for  our 
geometry  object.  The  set  of  scene  graph  actions  (rendering,  picking,  dragging,  etc.)  is  extensible. 


[1]  http://www.gris.uni-tuebingen.de/projects/vis 


Scripting  Interface 


The  experiments’  base  component  is  equipped  with  a scripting  interface.  Scripts  may  import  user-defined 
classes,  instantiate  objects,  and  call  their  methods.  They  also  might  be  bound  to  GUI  elements.  Scripts  are 
defined  and  documented  by  programmers,  or,  using  online  wizards,  by  community  members.  A well- 
defined  state  machine  manages  all  data  input  steps  and  provides  facilities  for  authorization,  default  values, 
undo,  and  preview  (Figure  1).  After  verification  of  an  editorial  board,  the  given  input  is  integrated  into  the 
script  database.  This  entails  other  challenges.  How  to  ensure  bug-free  and  generally  applicable  scripts?  Or, 
by  what  degree  should  we  tolerate  scripting  if  participants  work  collaboratively?  Our  answer  is:  simply  do 
the  same  as  for  non-interactive  content.  Require  authors  to  test  their  scripts  at  preview  step.  Evaluate  and 
enhance  scripts  like  any  other  learning  object  - gather  access  statistics,  utilize  a rating  system,  and  equip 
scripts  with  forum,  help,  and  annotation  facilities. 


Applications 

Figure  1 illustrates  how  scripting  may  help  us  to  overcome  an  interactive  learning  object’s  isolation.  The 
current  course  text  describes  the  definition  and  basic  properties  of  a vector  field.  We  embed  scripts  to 
match  the  experiment’s  vector  field  to  the  illustrations  (which  show  a symmetric,  radial,  and  potential 
field).  The  scripts  will  insert  particles  into  our  scene  graph,  or  adapt  the  GUI  and  add  text  fields  for  a 
function  parser.  The  learner  may  continue  to  work  with  his  oqjeriment.  Other  chapters  contain  scripts  to 
exchange  renderers  for  popular  visualization  techniques  (arrow  plot,  colorization,  streamlines,  LIC). 
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Figure  1:  Online  wizards  for  scripting  virtual  experiments  (left  side).  Scripting  allows  for  adapting  an 
experiment’s  state  to  the  current  course  text  or  illustration  (right  side). 
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Abstract:  This  paper  is  a report  on  the  findings  obtained  from  evaluating  the  planning  and 
implementation  of  an  online  symposium  addressing  the  issues  of  continuing  professional  development. 
The  participants  included  educators,  researchers,  and  training  specialists  in  Canada.  The  symposium  was 
part  of  an  Office  of  Learning  Technologies  (OLT)  funded  project.  The  symposium  was  delivered  through 
a blended  learning  model  delivered  via  asynchronous  and  synchronous  technologies.  The  symposium’s 
goal  was  to  gather  and  disseminate  knowledge  regarding  the  process  of  implementing  learning 
technology  initiatives  in  the  workplace.  It  also  evaluated  the  use  of  a blended  learning  model  for 
professional  development.  Overall,  the  symposium  achieved  its  goals  using  a blended  learning  model, 
however,  the  synchronous  virtual  classroom  environment  worked  more  effectively  in  group  collaboration 
and  discussion  compared  to  the  asynchronous  learning  environment.  Recommendations  are  summarized 
for  planning  and  implementing  an  online  blended  learning  model. 


Introduction 

While  researchers  claim  that  the  majority  of  adult  learners  are  satisfied  with  learning  in  computer -mediated  contexts,  there  are  still  students 
that  do  not  appreciate  this  type  of  learning  or  find  it  extremely  challenging  because  of  the  lack  of  human  elements  like  visual/verbal  cues  in  Internet 
based  forums  (Holt,  Kleiber,  Swenson,  Rees,  and  Milton,  1998).  Therefore,  there  is  a constant  need  for  distance  educators  to  evaluate  the  experience 
of  adult  learners  in  the  context  of  interactive  computer  mediated  commiinication  (Sorg  and  McEihinney,  2000).  One  of  the  functions  of  this  project  is 
to  examine  the  effectiveness  of  using  a blended  learning  solution,  including  both  synchronous  and  asynchronous  delivery  tools  for  collaboration, 
interactivity,  and  dissemination  of  information. 

Several  researchers  and  practitioners  have  noted  the  benefits  and  disadvantages  of  using  synchronous  online  conferencing  for  instructional 
purposes.  Perceived  disadvantages  of  text -based  synchronous  online  communication  include  a lack  of  focus  and  reflection  in  students’  comments 
(Honeycutt,  2000).  Other  limitations  noted  in  the  literature  are  that  students  are  constantly  challenged  because  visual  and  verbal  cues  are  not  present 
during  the  text -based  discussion  (Sorg  and  McEihinney,  2000). 

However  these  disadvantages  seem  to  be  outweighed  by  the  several  benefits  associated  with  text -based  synchronous  online 
communication.  Such  benefits  include  increased  student  participation,  liberation  of  minorities,  and  facilitation  of  small  group  discussions  (Bump, 
1990).  Sorg  and  McEihinney  (2000)  concluded  that  by  using  text-based  synchronous  environments  students  gain  a sense  of  empowerment,  a feeling 
of  belonging  to  a group,  and  construct  their  own  knowledge. 

While  useful,  the  previous  studies  are  limited  to  synchronous  online  conferencing  that  is  text-based.  Recent  studies  have  started  examining 
the  effects  of  synchronous  online  learning  environments  or  virtual  classrooms  that  use  video  and  audio  (Voice -over- Internet  Protocol)  for 
communication.  In  a study  that  used  synchronous  online  instruction  with  one-way  video  and  two-way  audio,  Hofmann  (2000)  observed  that  the 
technologies  became  transparent  in  the  learning  process,  and  the  focus  became  the  content  rather  than  the  medium.  This  is  an  indicator  of  a successful 
technology  based  program.  According  to  Appelt  and  Mambrey  (1999)  synchronous  virtual  learning  environments  seemed  to  increase  the  quantity  and 
quality  of  interaction  among  students,  and  between  students  and  teachers,  subsequently  establishing  a network  of  communication  and  collaboration 
among  students  and  teachers.  The  importance  of  collaboration  in  learning  is  evident,  as  it  is  tied  to  knowledge  construction  (Jonassen,  1996). 

These  virtual  learning  environments  may  be  even  more  effective  if  a combination  of  synchronous  and  asynchronous  learning  is  utilized, 
which  is  referred  to  as  blended  learning.  Blended  learning  “focuses  on  optimizing  achievement  of  learning  objectives  by  applying  an  appropriate 
combination  of  learning  technologies  for  the  learners,  context,  and  skills  taught  (Singh  and  Reed,  2001).  Charr-Chellman  and  Duchastel  (2001)  claim 
that  the  ideal  online  course  may  include  web-based  textual  materials  such  as  study  guides,  discussion  forums,  both  synchronous  (reakime)  and 
asynchronous  (delayed),  e-mail,  and  voice/audio  communication,  which  could  be  audio/video  conferencing  over  the  Internet  or  traditional  telephony. 
These  elements  would  be  used  depending  on  the  context  and  content  of  instruction.  An  advantage  of  using  asynchronous  interchange  is  that  students 
can  participate  in  a flexible  manner  at  their  own  time  (Charr-Chellman  and  Duchastd,  2001)  for  thoughtful  analysis  and  critical  reflection 
(Bhattacharya,  1999).  On  the  other  hand,  the  benefits  of  using  synchronous  interchanges  include  “a  more  direct  sense  of  collegial  interaction, 
immediate  resolution  to  questions  posed,  and  possibly  a strong  contribution  to  team  building”  (Charr-Chellman  and  Duchastel,  2001,  p.7).  Blended 
learning  would  take  advantage  of  the  benefits  of  both  synchronous  and  asynchronous  instruction. 

The  Rensselaer  80/20  Model  (Lister,  Danchak,  Scalzo,  Jennings,  and  Wflson,  1999)  proposes  the  use  of  blended  learning.  Based  on  this 
model  Renneselaer  Polytechnic  Institute  created  undergraduate  courses  that  were  80%  asynchronous,  including  streamed  on-demand  video  and  audio 
or  other  resources  accessed  on  the  Web,  and  20%  %mchronous.  A technology  used  for  the  synchronous  component  was  an  interactive  distance  system 
called  LeamLinc,  which  emulates  “the  proven  techniques  of  interacti%'e  teaching  that  are  so  effecti%'e  in  the  face-to-face  classroom”  (Lister  et.  al, 

1999,  p.2).  Lister  et.  al  (1999)  conclude  that  while  the  asynchronous  component  is  invaluable  as  long  as  it  is  instructionally  sound,  the  synchronous 
interaction  is  an  integral  component  of  the  distance  learning  course,  which  helps  keep  students  on  track  with  course  deadlines,  allows  students  to 
receive  immediate  feedback,  builds  a team  and  community  spirit  among  students,  and  improves  retention  rates. 

Blended  learning  seems  to  improve  learning  effectiveness  and  extends  learning  to  a wider  audience  in  different  geographical  locations  and 
time  zones,  while  optimizing  development  cost,  time,  and  business  results  (Singh  and  Reed,  2001).  Research  completed  by  the  University  of 
Termessee  demonstrated  that  blended  learning  programs  can  be  completed  in  approximately  one  half  the  time  and  at  less  than  half  the  cost  of  using 
traditional  classroom  instruction  (Dean,  Stahl,  Sylwester,  Peat,  2001). 

The  Study 


Beyond  Learning  Technologies:  An  Online  Symposium  was  part  of  an  Office  of  Learning  Technologies  (OLT)-funded  research  project 
entitled  Evaluating  Learning  Technologies  Initiatives  (ELTI)  in  Continuing  Professional  Development.  The  Beyond  Learning  Technologies  online 
symposium  was  organized  and  hosted  by  the  TELUS  Centre  for  Professional  Development  and  the  Institute  of  Professional  Development  at  the 
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University  of  Alberta,  in  collaboration  with  the  ELTI  research  team.  Participants  in  this  symposium  included  educators  and  researchers  from  various 
universities  across  Canada  and  training  specialists  from  the  federal  government  and  private  industry. 

The  main  purpose  of  the  online  symposium  was  to  disseminate  the  findings  of  the  ELTI  project  associated  with  implementing  learning 
technologies  initiatives  and  generate  comments  and  feedback  from  the  participants  on  these  findings.  Since  innovative  learning  technologies 
initiatives  were  used  for  the  symposium,  including  both  synchronous  and  asynchronous  delivery  tools,  the  planning  and  implementation  of  these 
technologies  was  evaluated  and  recommendations  are  provided  for  professionals  in  the  field. 

The  symposium  took  place  completely  online  and  consisted  of  an  asynchronous  (delayed)  and  synchronous  (live,  real-time)  component. 

The  delivery  tool  used  for  the  synchronous  component  was  CentraOne.  This  is  a Web- enabled  virtual  classroom  environment  using  Voice- 
Over-Internet  Protocol  technology.  Each  of  the  synchronous  sessions  was  1 .5  hours  in  duration.  The  content  for  the  sessions  was  provided  by  the 
ELTI  research  team  and  was  modified  by  TELUS  Centre  eLearning  team  to  ensure  that  instructbnal  design  principles  were  followed  and  that  the 
content  was  appropriate  for  synchronous  online  delivery. 

For  the  asynchronous  part  of  the  symposium,  an  online  course  was  created  in  Web  Course  Tools  (WebCT),  which  contained  case  studies, 
executive  summaries,  links  to  resources,  a draft  of  the  final  report,  and  an  asynchronous  discussion  board. 

The  primary  methods  used  in  evaluating  the  Beyond  Learning  Technologies  online  symposium  involved  evaluation  surveys,  the  recordings 
from  the  synchronous  online  sessions,  and  the  archived  asynchronous  discussions. 

As  a secondary  method  the  tracking  system  from  within  WebCT  was  also  used  to  collect  information  on  which  participants  logged  on  to 
the  WebCT  course  and  how  many  pages  were  accessed  within  the  course. 

Findings 


Overall  the  symposium  seemed  to  be  successful  as  it  achieved  its  primary  goals,  which  were  to  disseminate  knowledge  about  the 
Evaluating  Learning  Technologies  Initiatives  in  the  Workplace  project  and  to  gather  feedback  from  participants  on  the  quest  ions  examined  in  the 
project.  It  also  achieved  its  second  goal,  which  was  to  implement  and  evaluate  the  use  of  a blended  delivery  model,  which  is  an  innovative  learning 
technology  in  itself,  where  by  both  synchronous  and  asynchronous  online  delivery  modes  were  implemented  for  this  symposium. 

Specifically,  the  participants  seemed  very  enthusiastic  about  the  synchronous  online  sessions.  Several  participants  noted  that  they  enjoyed 
the  interactivity  and  the  opportunity  to  have  a discussion  with  colleagues  in  different  geographical  locations,  using  the  Internet,  without  having  to 
travel.  Additional  positive  comments  were  made  related  to  the  facilitation  of  the  event,  ease  of  use,  effectiveness  of  the  tool,  and  the  collaborative 
human  interaction  amongst  the  participants.  Negative  comments  centered  on  microphone  volume  levels,  which  were  not  consistent  across  users. 

The  asynchronous  part  of  the  symposium,  a course  built  in  WebCT  did  not  seem  be  as  successful.  While  participants  used  it  to  access  the 
resources  associated  with  the  symposium  (case  studies,  executive  summaries,  links  etc.),  the  use  of  the  asynchronous  discussion  forums  were 
underutilized. 

Recommendations 

The  success  of  a blended  learning  model  using  Web-enabled  communication  technologies  depends  on  planning  and  implementing  key 
procedures  during  the  lifecycle  of  the  project. 

General  Instructional  Design  Recommendations 

• Identify  the  audience  (participant) 

• Identify  the  learning  obj  ecti ves  and  outcomes 

• Identify  discussion  topics  and  evaluation  methodology 

• Choose  the  appropriate  amount  of  time  needed  for  synchronous  and  asynchronous  instruction 

• Choose  the  appropriate  tools  within  each  learning  environment  to  best  convey  the  learning  concepts 

• Develop  electronic  curriculum  which  is  instructional ly  sound  and  suited  for  online  delivery 

• Develop  discussion  and  survey  questions  to  encourage  participants  to  interact  and  collaborate 


Recommendations  for  a Blended  Learning  Model  (including  synchronous  and  asynchronous  online  instruction): 

• Ensure  that  participants  meet  minimum  computer  hardware  requirements,  which  is  critical  for  remote  participant  connectivity 
and  for  reducing  participant  frustration. 

• E-mail  logon  credentials  to  the  remote  participants  as  well  as  timelines  and  dates. 

® Provide  an  orientat  ion  of  each  learning  environment  to  participants. 

• Provide  access  to  technical  support  for  the  participants  to  help  those  who  experience  difficulty  in  getting  properly  connected  to 
both  learning  environments,  during  the  events. 

• Encourage  participants  to  lo  g in  one  half-hour  prior  to  the  commencement  of  the  event.  This  facilitates  establishing  a comfort 
level  for  the  participants.  Microphone  and  speaker  volumes  can  be  adjusted  appropriately. 

• Complete  a dry-run  session  prior  to  the  actual  synchronous  live  event  to  ensure  the  presenters  are  well  prepared  and 
comfortable  with  the  Graphic  User  Interface. 

• Integrate  the  use  of  synchronous  and  asynchronous  components  of  the  course  so  they  are  interdependent  rather  than 
independent.  For  example,  answering  questions  in  the  asynchronous  component  successfully  will  require  that  you  have  attended 
the  synchronous  component  of  the  course. 

• Evaluate  the  participants’  learning  experience. 
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Tagging  the  didactic  functionality  of  learning  objects 

Per  Skafte  Hansen,  IMM,  DTU,  psh@imm.dtu.dk 
Stig  Brostroem,  LRQ  DTU,  sb@dtv.dk 

Introduction:  We  recently  undertook  to  implement  and  package  a number  of  short  presentations  on  a 
CD-ROM,  as  a supplement  to  other  material  for  a course  in  numerical  analysis.  The  items  vary  in  terms 
of  size,  natural  viewing  media  and  intended  use;  and  the  purpose  of  our  work,  other  than  the  didactical 
one,  is  to  monitor  the  resource  requirements  for  the  production  and  wrapping  of  such  material. 

This  seemingly  stra^htforward  task  brought  us  into  direct  confrontation  with  issues  concerned  with  the 
concept  of  Imning  objects  (http:/ / works  .opencontent.org/ writings2.pD  and  their  future  usability. 

Background:  The  Q>ROM  in  question  will  eventually  contain  fixed  texts  (surveys,  exercises  etc.), 
modifiable  texts  (a  report  layout,  the  framework  for  a do-it-yourself  manual,  etc.),  code  exampks,  anim- 
ations with  optional  voice-overs,  and  video-recorded  miniature  lectures.  To  make  the  whole  effort  use- 
ful and  at  least  to  some  extent  re-usable,  we  ventured  into  marking  up  (“tagging”)  each  item  as  a learn- 
ing object.  The  difficulties  encountered  in  this  process  hinge  on  die  fact  that  the  standards  (consult 
http://www.imsproject.org/ specifications  .html'l  for  learning  object  content  packaging,  whether  final- 
ized or  drafted,  are  biased  towards  bibliographical  usages  and  pay  insufficient  attention  to  the  needs  of 
the  author,  the  teacher  and  the  learner.  In  particular,  we  wanted  to  address,  as  equally  important: 

Subject  Within  the  subject  of  numerical  analysis  (our  present  example),  there  are  sub-subjects  such  as 
one- dimensional  interpolation,  with  sub-sub -subjects  such  as  the  proof  of  the  structure  of  the  error 
term  in  polynomial  interpolation,  and  the  “sub-  vs.  sub -sub ’-relations  are  difficult  to  state  in  the  tags 

Dickcticfimction:  An  interactive  self-assessment  is  in  quite  another  category  than  a video- recorded  proof 
of  a theorem,  yet  the  current  standards  do  not  allow  for  this  kind  of  distinction 

Gramkrity:  A whole  alternative  exam  for  a course  is  too  large  to  be  one  learning  objea,  a single  graph 
too  small,  an  exercise  in  programming  a grapt  drawing  function  suitable 

Note,  that  adequately  monitoring  granularity  is  the  responsibility  of  the  author,  not  of  the  tagging  tools; 
but  it  will  be  of  importance  for  what  follows. 

Recommendations:  There  is  a need  for  extending  the  type  dedarationto  have  it  describe  the  didactic 
fimctionoi  the  object.  A crude  approximation  to  what  is  meant  may  look  as  follows: 

Simple  factfs)  maybe  such  things  as  definitions,  measurements,  precise  names 
Rdatiors  mzyhe  rules,  formulas,  laws,  observable  causal  cormections 

Operations  maybe  experiments,  a^orithms,  typical  des^  manipulations,  exercises  and  self -tests 

Insists T02cy  he  explanations,  proofs,  rationales  of  (best)  practices 

Stimuli  of  imcfizitum^  such  open-ended  items  as  project  (exercise)  proposals 

(The  five  italicised  categories  used  are  part  of  a general  paradigm  currentiy  under  development  by  the 
first  author  and  should  only  be  considered  a starting  point  in  the  present  context). 

The  type  declaration  idea  might  in  fact  allow  for  composite  types,  as  in  procedural  programming.  The 
simplest  version,  a fixed-depth,  fixed -span  type  declaration  of  depth  D corresponds  to  the  addition  of 
D tags,  each  with  a pre-defined  table,  table  of  tables  etc.,  of  allowed  values,  after  die  above  manner. 
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Tlie  stn4CtU!^  'mlo  which  a user  (teacher)  will  fit  the  individual  component  cannot  and  should  not  be 
standardized:  the  whole  of  such  a structure  will  form  a course  or  the  equivalent,  and  — to  name  an 
example—  the  position  of  a learning  object  in  the  time-line  of  the  course  and  in  the  logical  flow  in  a 
final  exercise  maybe  very  different,  tqydoimvs,  hcttanup  composition  being  but  one  such  disparity. 

Yet,  engineering  disciplines  abound  with  components- in- connection,  as  exemplified  by  circuits  and 
networks,  process  plants,  in  faa  the  general  idea  of  dynamical  analogies  and  system  design.  — One  may 
safely  arg;ue  that  the  very  notion  of  a “component”  requires  a network  for  the  component  to  sit  in. 

Also,  one  must  avoid  the  situations  of  a learning  object  author  blindly  composing  what  he  believes 
someone  eke  may  need,  or  the  user  receiving  the  result  without  knowing  whether  or  not  it  is  of  any  use 
in  an  overall  arrangement  unknown  to  the  original  author.  We  therefore  most  strongly  recommend  that 
learning  objects  and  some  mechanism  for  their  assembly  be  considered  dual  and  of  equal  importance  in 
standardization  initiatives.  An  idea  of  what  we  have  in  mind  is  illustrated  in  the  following  diagrams: 


Numsriad  amfysis 

n\ 

Type:  insight/  theorem 

Interpolation 

Appl 

LuJ-U-" 

□ 

Subject:  interp,  polynomial  /uniqueness 

Integration  

■nito'y/Appi 

□/  / 

Pte-iequisites : Fund,  thm  of  algebra,... 

1-D  non-linear  eqs.  — 

■n>“'y/Appi 

Linear  eqs.  

■"“■y/Appi 

: : : 

□ 

Step  1:  assumption  of  two  interp.  polys. 

Non-linear  systems  

■"“■y/Appi 

/ 

ODEs  _ 

■^■VappI 

n-n 

Natural  follower:  thriL  of  existence 

Figure  text:  The  left  listing,  easily  recast  for  a graphical  user  interface,  gives  a choice  of  subjects  for  a course  in  elementary 
numerical  analysis,  one  topic  highlighted;  the  middle  diagram  indicates  the  learning  objects  as  components-in- a- network, 
one  selected;  and  the  right  table  the  inner  structure  of  this  specific  learning  object,  with  a few  tag  declarations  (in  bold). 

To  allow  for  composition  with,  and  use  of,  such  networks,  meta-tags  might  assist  by  providing  inform- 
ation concemii^  e.g.  the  pre-requisites  of  the  learning  object  at  hand.  Such  information  requires  stand- 
ardization pertaining  to  the  teaching  and  use  of  the  subject  items  and  may  thus  eventually  be  unaccept 
able  outside  the  realm  of  the  curriculum  of  a given  university.  Ako,  the  tagging  process,  already  time- 
consuming,  will  become  truly  demanding  of  intellectual  effort  whenever  a new  subject  area  k entered. 

Still,  learning  objects  are  to  be  “launched”  by  authors,  but  searched  for  and  employed  by  teachers  and 
consumed  by  students.  We  expect  to  see  authoring,  searching  and  viewii^  took  merge  in  the  near 
future  and  wkh  to  point  out  that  the  choices  of  identical  or  symmetric  display  and  operations  lie  with 
the  tool  designers,  not  with  the  designers  of  the  markup  standards.  But  as  we  envkage  a future  in 
which  learning  objects  are  aplenty  and  computer  networking  full-grown,  we  must  require  of  the 
learning  objects  themselves,  and  hence  of  their  tag  structures,  that  they  support  a specialized  search. 
Otherwise,  the  well  known  “app.  1,001,300  hits” -message  will  soon  preclude  the  use  and  hence  the 
benefits  of  learning  objects.  Tins  k why  questions  of  object  size  interweave  with  the  technical  ksues: 
you  can  only  find  what  you  are  lookbg  for  if  it  k suitably  tagged;  and  you  can  only  carry  the  burden  of 
“tagging”,  if  each  item  k of  an  appropriate  size.  Otherwke,  you  either  have  too  many  to  look  after,  or 
quite  few,  but  each  too  complex  to  describe  in  the  simple  manner  required  by  the  markup  procedure. 

Brief  conclusion:  While  the  ideas  and  concepts  behind  learning  objects  are  sound  and  natural, 
especially  to  engineering  educators,  aspects  of  their  des^  and  use  are  still  un-  or  underdeveloped. 
From  a components-  in-  a-  network  point  of  view,  the  most  important  ksues  are:  a didactically  based 
typing  of  the  learning  objects  themselves;  the  entire  desi^  superstructure  into  which  the  learning  objects 
must  be  fitted;  and  the  symmetry  cf  the  mtetfaces^  as  seen  by  each  pair  of  the  triad:  author,  teacher,  learner. 
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Exploring  Student  Performance  on  a Multimedia  Exam  Program  (V exam) 
the  relationship  between  item  formats  and  item  difficulty  levels 


Yung-Wei  Hao 
School  of  Education 
University  of  Texas -Austin,  USA 
hao@niail.utcxas.cdu 


Abstract:  This  research  study  aims  to  explore  the  influence  of  multimedia  and  text  formats  on 
student  performance  in  the  multimedia  exams.  The  focus  group  is  the  390  students  who  took  an 
online,  self-paced  undergraduate  course  in  Department  of  Anthropology  at  the  University  of  Texas 
at  Austin  in  fall  2001 . Vexam  was  the  assessment  tool  for  the  class.  The  test  items,  created  by 
Vexam,  were  multiple-choice  items  and  consisted  of  text,  movie  clips,  and/  or  pictures;  they  were 
categorized  in  the  two  groups,  multimedia  and  text  only.  The  research  question  is  whether  there  is  a 
significant  relationship  between  the  use  of  multimedia  and  text  only  formats  and  item  difficulty 
levels  in  the  exams.  An  exploratory  research  study  and  result  will  be  presented. 


Introduction 

With  information  technology  designs  becoming  more  user  friendly,  and  the  cost  of  multimedia  decreasing, 
multimedia  technology  provides  the  education  field  with  additional  degrees  of  freedom,  enabling  the 
exploration  of  alternatives  to  traditional  education  (Vogel  & Klassen,  2001).  In  order  to  relieve  pressure  on 
teaching  staff  caused  by  increasing  enrollments,  more  institutions  are  investigating  comp  uterized  testing  as  a 
tool  for  formal  assessment.  (Stephens,  Bull,  & Wade,  1998;  Zakrzewski  & Bull,  1998). 

The  computer  screen  offers  numerous  options  for  overcoming  the  disadvantages  of  uninteresting  and 
unvaried  format  (Byrnes,  Forehand,  Rice,  Garrison,  Griffin,  McFadden,  & Stepp-Boiling,  1991).  By 
incorporating  multimedia,  some  assessment  tasks  can  be  made  more  relevant.  Item  stimuli  can  be  made 
dynamic  by  having  video,  audio,  and  animation  (Bennett,  1999).  Multimedia  testing  environments  can  be  more 
authentic,  because  they  can  be  contextualized  with  multiple  media  elements,  and  the  text  can  be  combined  to 
present  the  tests  with  items  resembling  real-life  tasks. 

However,  simply  adding  an  illustration  next  to  a word  does  not  make  the  situation  better  (Gyselinck, 

1996).  Especially  in  testing,  whether  the  type  of  presentation  can  influence  student  performance,  it  is  not  yet 
known  and  needs  further  research.  More  investigation  is  needed  to  establish  whether  people  can  be  confused  or 
distracted  by  multimedia  when  the  items  are  easier.  The  purpose  of  this  study  is  to  gain  deeper  insight  into  the 
effects  of  use  of  multimedia  and  text  only  on  student  performance  in  the  multimedia  exam. 


The  Study 

The  study  focuses  on  whether  using  multimedia  can  help  people  answer  questions  correctly.  The  item 
difficulty  level  is  used  to  interpret  the  students’  performance.  The  item  difficulty  level  is  defined  as  the  number 
selecting  correct  answer  divided  by  the  total  number  taking  the  test  (Kubiszyn  & Borich,  1999).  By  analyzing 
the  difficulty  level  of  test  items,  the  study  investigated  whether  there  was  significant  relationship  between  item 
formats  and  item  difficulty  levels  (the  students’  performance).  The  research  questions  were  raised  for  this  study 
as  follows. 

• Is  there  a relationship  between  the  use  of  multimedia  and  text -only  and  the  item  difficulty  level  (the 
students’  performance)?  If  the  answer  is  yes,  what  relationship  is  it? 

The  390  students  took  the  online,  self-paced  course  and  took  the  multimedia  exams  weekly.  Vexam,  a 
multimedia  exam  program  and  not  a computer-adaptive  testing  program,  was  the  assessment  tool  for  the  class. 
The  585  test  items  were  retrieved  from  the  10  exams  students  took  during  the  semester.  The  item  formats 
consisted  of  text,  movie  clips,  and/  or  pictures.  For  those  items  whose  item  discrimination  index  is  less  than  0 
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were  taken  away.  Only  the  items  with  positive  discrimination  index  are  worth  analyzing.  Thereafter,  all  the  rest 
of  the  test  items  with  positive  discrimination  index  in  10  exams  were  categorized  by  their  item  difficulty  levels 
(from  1 to  9)  and  by  their  item  formats  (multimedia  and  text  only).  By  comparing  different  types  of  item 
formats  with  different  item  difficulty  levels  in  the  exams,  the  study  tested  whether  there  was  significant 
relationship  between  item  formats  and  item  difficulty  levels. 


Findings 

According  to  the  result  of  the  study,  item  formats  have  a significant  relationship  with  item  difficulty  levels. 
Namely,  different  item  formats  have  a significant  relationship  with  the  proportion  of  people  who  got  the  item 
correct.  On  the  other  hand,  there  is  no  significant  difference  between  the  multimedia  format  and  the  text  format 
in  each  item  difficulty  level. 


Conclusion 

The  results  of  the  study  showed  that  the  use  of  different  item  formats  had  a significant  relationship  with  the 
students’  performance.  Also,  in  each  item  difficulty  level  the  text  format  and  the  multimedia  format  do  not  have 
significant  differences.  This  is  a work- in-progress.  Further  research  will  investigate  the  reliability  of  the 
research  results  and  students’  behavior  and  perspectives  on  taking  the  multimedia  exams.  Educators  should  be 
careful  when  using  multimedia  elements  in  testing,  since  item  formats  have  a significant  influence  on  students’ 
performance.  There  still  lacks  a standard  methodology  for  developing  computerized  testing  (Drasgow  and 
Olson-Buchanan,  1999).  More  innovative  research  is  needed  to  explore  when  and  how  multimedia  can  cause 
the  effects,  which  can  influence  student  performance. 
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Abstract;  Theoretical  approaches  to  instructional  design  for  skill  development  in 
complex  domains  lags  behind  the  changing  nature  of  education  and  training  reflected  in 
the  application  of  contemporary  views  of  learning.  In  particular,  employing  novel 
pedagogical  approaches  for  advanced  skill  development  has  not  been  extensively 
investigated.  The  development  of  an  application  of  guided  simulation  incorporating 
reflection  tools  to  train  naval  officers  in  pilotage  has  been  undertaken  to  illustrate  this 
process.  The  skills  and  knowledge  which  are  part  of  the  conceptual  framework  that 
learners  need  to  acquire  to  be  pilots  have  been  analyzed  and  the  design  incorporates  a 
variety  of  instructional  strategies  that  are  well  suited  to  supporting  outcomes  driven 
learning. 


Introduction 

The  possibilities  afforded  by  new  multimedia  technology,  combined  with  contemporary  ideas  about 
learning,  have  opened  up  new  design  opportunities  for  educational  simulations.  In  particular,  the  use  of 
sophisticated  multimedia  environments  has  made  the  design  of  experiential  simulations,  in  which  the 
learner  plays  an  authentic  role  carrying  out  complex  tasks,  a much  more  tractable  design  task. 

However,  theoretical  approaches  to  instructional  design  for  skill  development  in  complex  domains  has 
lagged  behind  the  changing  nature  of  education  and  training  reflected  in  the  application  of  contemporary 
views  of  learning.  As  theories  of  learning  have  developed  and  educationists  have  gained  more  experience 
in  the  use  of  computer  based  technology,  there  has  been  a shift  of  emphasis  from  the  behaviourist 
paradigm,  and  through  what  Atkins  (1993)  has  called  the  weak  artificial  intelligence  approach,  to  a 
constructivist  view.  This  shift  of  emphasis  has  not  been  extensively  investigated  for  advanced  skill 
development  with  some  notable  exceptions  such  as  the  ADAPTIT  project  (Pass,  et  al,  1999).  This  view  of 
design  has  influenced  the  development  of  an  application  of  guided  simulation  incorporating  reflection  tools 
to  train  naval  officers  in  Pilotage.  The  skills  and  knowledge  which  are  part  of  the  conceptual  framework 
that  learners  need  to  acquire  to  be  pilots  have  been  analysed  and  the  design  incorporates  a variety  of 
instructional  strategies  that  are  well  suited  to  supporting  complex  cognitive  skill  development. 

Contemporary  Theories  of  Learning 

Many  writers  have  stressed  the  need  for  open-ended  exploratory  authentic  learning  environments  in 
which  learners  can  develop  personally  meaningful  and  transferable  knowledge  and  understanding.  The 
lead  provided  by  these  writers  has  resulted  in  the  proposing  of  guidelines  and  criteria  for  the  development 
of  software  based  on  a constructivist  view  of  learning  (eg.  Savery  and  Duffy  1995;  Duffy  and  Cunningham 
1996;  Grabinger  1996;  Squires  1996;  Hannafin  and  Land  1997;  Grabinger,  Dunlap  and  Duffield  1997). 
Harper,  Squires  and  McDougall  (2000)  have  proposed  a recurrent  theme  of  these  guidelines  is  that  learning 
should  be  authentic.  They  have  noted  that  a review  of  the  literature  points  to  three  seminal  concepts,  which 
originate  from  the  notion  of  authenticity:  credibility,  complexity  and  ownership. 

For  learners  to  feel  that  an  environment  offers  credible  opportunities  for  learning,  especially  when  skill 
development  is  paramount,  they  need  to  be  able  to  explore  the  behaviour  of  the  systems,  which  are  the 
focus  of  their  learning.  One  way  to  do  this  is  through  working  with  simulations  of  the  systems  to  be 
mastered.  The  environment  should  provide  the  learner  with  intrinsic  feedback,  which  represents  the  effects 
of  the  learner’s  action  on  the  system  and  mechanisms  to  experiment  with  ideas,  try  out  different  solutions 
to  problems  and  reflect  on  those  solutions. 

Grabinger  and  Dunlap  (1995)  emphasise  that  learners  should  be  presented  with  complex  environments 
that  represent  interesting  and  motivating  tasks,  rather  than  contrived  sterile  problems.  Only  in  complex. 
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rich  environments  will  learners  have  the  opportunity  to  construct  and  reconstruct  concepts  in  idiosyncratic 
and  personally  meaningful  ways.  However,  learners  may  need  help  in  coping  with  the  skills  to  be 
developed  in  complex  domains.  Strategies,  which  have  proved  useful  in  helping  learners,  include 
scaffolding  (Krajcik  et  al.  1998),  anchoring  (Cognition  and  Technology  Group  at  Vanderbilt  1990),  and 
problem-based  environments  (Tobin  and  Dawson  1992;  Grabinger,  Dunlap  and  Duffield  1997). 
Additionally,  metacognition,  in  which  learners  reflect  on  their  own  cognition  to  improve  their  learning,  has 
been  advocated  by  a number  of  writers,  eg.  Scardamaliaer  al  (1989)  and  Papert  (1980).  It  is  claimed  that 
by  a conscious  personal  appraisal  of  cognitive  processes,  an  individual  can  improve  his  or  her  capacity  to 
learn  (Hoban,  2000). 

The  Simulation  Paradigm 

Simulations  as  learning  environments  have  had  a long  history  of  use  in  education  and  training  and  have 
been  based  on  a variety  of  theoretical  views  of  learning.  Initial  claims  for  the  educational  benefits  of  using 
simulations  tended  to  emphasise  pragmatic  solutions  to  classroom  problems.  Processes,  which  take  a long, 
time,  eg.  population  growth  or  genetic  change,  or  which  happen  very  quickly,  eg.  changes  in  impulsive 
force  during  a collision,  are  possibilities  for  simulation.  Difficult,  dangerous  or  expensive  processes  are 
also  candidates  for  simulation,  eg.  experiments  with  radioactive  materials 

The  key  feature  of  an  educational  or  training  simulation  is  that  it  makes  use  of  a model  to  represent  a 
process,  event  or  phenomenon,  which  has  some  learning  significance.  The  learner  is  able  to  interact  with 
this  representation  and  the  simulation  provides  intrinsic  feedback  that  the  learner  can  interpret  as  the  basis 
for  further  interaction.  The  underlying  model  may  be  mathematical  leading  to  the  generation  of  numerical 
results,  rule -based  with  the  intention  of  providing  feedback  on  subjective  input,  or  even  context  -based  in 
that  the  learner  is  placed  in  a context  that  simulates  a real  situation. 

A broader  view  of  simulation  can  be  taken  if  an  ‘in  context’  environment  is  considered  as  a simulation. 
In  this  form,  the  simulation  is  designed  to  place  the  user  in  a professionally-relevant  context  where  they  can 
investigate  an  issue  and  solve  a problem,  without  necessarily  receiving  the  intrinsic  feedback  characteristic 
of  the  more  mathematically  or  rule -based  simulations.  While  working  with  such  models,  the  user  often 
comes  to  see  the  simulation  as  a ‘real  world’  in  its  own  right  (Crookall  et  al,  1987).  Gredler  has  described 
such  simulations  as  experiential  and  proposes  they  aim  to  establish  a particular  psychological  reality  and 
place  learners  in  defined  roles  within  the  reality.  He  describes  the  essential  components  of  an  experiential 
simulation  as  ‘(1)  a scenario  of  a complex  task  or  problem  that  unfolds  in  part  in  response  to  learner 
actions,  (2)  a serious  role  taken  by  the  learner  in  which  he  or  she  executes  the  responsibilities  of  the 
position,  (3)  multiple  plausible  paths  through  the  experience,  and  (4)  learner  control  of  decision  making’ 
(1996,  p.  523).  Role  playing  simulations  of  environmental  planning  are  classic  examples  of  experiential 
simulations.  Such  models  can  be  considered  as  representing  real-world  systems  as  either  an  “in-place-of  ’ 
or  a “bring-to-life”  format. 

Regardless  of  the  type  or  format  of  the  simulation,  the  overriding  purpose  for  simulating  systems 
remains  to  provide  a learning  environment  that  supports  the  learner  to  develop  mental  models  about  a 
process  or  the  interrelationships  of  variables.  Simulations  can  help  learners  achieve  these  objectives  by 
providing  a substitute  experience  for  those  processes  and  systems,  which  by  reason  of  cost,  scale,  time  or 
risk  would  not  normally  be  accessible. 

Complex  Cognitive  Skill  Development 

In  many  areas  of  training,  where  learners  are  to  achieve  a high  level  of  efficiency  in  complex  tasks, 
achievement  of  complex  cognitive  skills  is  a key  goal.  The  term  complex  cognitive  skills  can  be  thought  of 
as  complex,  in  the  sense  that  they  comprise  a set  of  constituent  skills  and  at  least  some  of  these  constituent 
skills  involve  conscious  processing  and  cognitive,  in  that  the  majority  of  constituent  skills  are  in  the 
cognitive  domain.  Merrienboer,  (1997)  has  developed  an  instructional  design  model,  the  4C/ID  model, 
which  has  influenced  the  design  process  which  lead  to  development  of  the  application  described  in  this 
paper.  The  model  is  based  on  the  view  that  in  development  of  complex  cognitive  skills,  learners  should 
practice  simple -to-complex  versions  of  either  the  whole  complex  cognitive  skill  or  meaningful  clusters  of 
constituent  skills  and  that  complex  cognitive  skills  have  both  automated  and  cognitive  components. 


Pilotage  Training 

The  skill  of  piloting  a ship  into  an  anchorage  is  a complex  cognitive  one,  which  involves  not  only  a 
constant  process  of  decision-making,  based  on  data  and  crew  input,  but  also  management  of  multiple  inputs 
in  real  time. 

The  training  of  navy  personnel  on  the  piloting  of  ships  has  had  a history  of  making  use  of  sophisticated 
simulators,  but  initial  training  has  tended  to  be  based  on  more  traditional  classroom  experiences.  With  an 
aim  of  developing  a more  efficient  training  process,  drawing  from  our  experience  and  research  with 
simulation,  a PC  based  simulation  has  been  developed,  based  on  contemporary  views  of  learning.  Our 
instructional  design  strategies  for  this  package  are  based  on  authentic  experiences,  a problem  solving 
context  and  discovery  (de  Jong  & van  Joolingen,  1998)  and  reflection  (Schon,  1983;  Schon,  1987). 

The  development  of  an  understanding  of  Pilotage  is  the  key  goal  of  this  package.  The  skills  and 
knowledge  which  are  part  of  the  conceptual  framework  that  learners  need  to  acquire  to  be  pilots  have  been 
analysed  and  incorporated  into  instructional  strategies  that  are  well  suited  to  supporting  outcomes  driven 
learning,  and  yet  place  learners  in  an  environment  which,  in  Gredler’s  terms,  can  be  considered  to  be 
experiential. 

The  package  makes  use  of  learner-interaction  in  a visual  context  that  is  consistent  with  being  a pilot  on  a 
combat  ship.  The  strategies  used  to  develop  the  high  levels  of  skill  necessary  consist  of: 

• Lesson  materials,  containing  support  elements  for  concept  development  and  review  as  well  as  a set  of 
reference  materials  commonly  used  when  planning  and  executing  pilotage 

• Guided  simulation  where  the  learner  makes  decisions  within  a simulated  environment  and  can  then 
consider  the  consequences  of  their  decision,  and 

• Use  of  constituent  skills  which  build  to  complex  cognitive  skill  sets  which  are  necessary  to  solve  the 
task 

• Reflection  where  the  learner  can  review  their  experience  and  reflect  on  their  success  (Figure  1). 


Lesson  Overview 

1 . Introduction  to  Pilotage 

1 .1  Types  of  Navigation 

t .2  Roles  and  Responsibilities 
t .3  Preparatory  Work  (Coastal) 

2.  Planning  (Visual  and  Blind) 

2.1  Preparatory  Work  (A) 

2.2  Preparatory  Wor*f-teV 

3.  Chart  & Prepare 

3.1  Fixing  the  Ship 

3.2  Danger  avoidani 

3.3  Other  factors  ; 

4.  Chart  & Prepare 

4.1  Fixing  the  Ship 

4.2  Altering  Course 

5.  Execution  of  Vi! 

5.1  Preparatory  WO 

5.2  Maintaining  the 

5.3  Altering  course 

5.4  Other  Factors  ; 

5.5  Dos  and  Don't s 

6.  Execution  of  Blii 

6.1  Preparatory  Wo 

6.2  Fixing  the  Ship 
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Figure  1 A view  of  the  introductory  tutorial  window 
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A set  of  lesson-style  tutorials  was  developed  to  expose  trainees  to  a logically  sequenced  set  of  pilotage 
theory  concepts.  Essential  ‘lessons’  are  available  for  trainees  to  use  to  increase  their  understanding  within 
this  area.  Where  the  lesson  concept  is  complex  or  difficult  it  is  supported  with  animated  sequences 
relationship  models  or  diagrams.  Each  lesson  also  contains  a review  module  where  the  key  concepts  of  the 
lesson  are  presented  for  review  using  a question  and  answer  approach  Once  an  initial  sub-set  of  the  tutorial 
concepts  has  been  worked  through  by  the  beginning  trainees  they  can  commence  the  simulation.  The 
content  remains  easily  accessible  when  a trainee  needs  to  review  or  check  a concept  relevant  to  the 
complex  skills  demonstrated  when  working  on  the  simulation  environment. 

The  Simulation  provides  a pseudo-realistic  environment  modelled  on  the  visual  cues,  pilotage  processes 
and  control  tools  found  on  a typical  naval  vessel.  The  experience  of  using  the  simulator  gives  the  trainee 
the  opportunity  to  plan,  observe,  and  make  decisions  that  directly  impact  upon  the  movement  of  the  vessel 
with  realistic  outcomes  such  as  being  able  to  maintain  a planned  course  to  varying  degrees  from 
successfully  reaching  your  destination  through  to  running  aground.  The  key  interface  ele  ments  of  the 
simulation  are:  A chart  that  details  a fictitious  island  group  with  the  information  needed  for  planning  a 
pilotage  track  through  the  area,  tools  to  ‘mark-up’  the  chart  for  a planned  pilotage  run,  data  such  as  depth 
soundings  and  navigational  aid  such  as  beacons,  landmarks  and  buoys.  In  visual  pilotage  mode,  a 3- 
dimensional  view  from  the  bridge  of  the  ship  is  derived  from  the  chart  as  well  as  incorporating  extra  data 
such  as  tidal  streams  and  currents  (Figure  2).  The  trainee  is  able  to  manoeuvre  the  ship  using  speed  and 
course  adjustment  controls,  monitor  the  ship's  progress  via  tools  such  as  the  depth  sounder,  wind-direction 
and  wind-strength  display,  stopwatch,  ship's  clock,  propeller  revolutions,  heading  and  speed  readouts.  As 
well  as  interacting  with  these  controls,  the  trainee  is  able  to  take  bearings  on  objects  within  the  3D  visual 
environment,  which  can  be  panned,  to  starboard  or  port.  This  enables  the  monitoring  of  the  ship's  progress 
relative  to  the  planned  pilotage  route. 


Figure  2 Visual  Pilotage  mode  showing  the  chart,  a 3D-engine  display  and  reflection  tool. 

The  simulation  can  also  be  run  in  a blind-pilotage  mode.  In  this  instance  instead  of  a 3-dimensional  view 
of  the  experience,  the  trainee  is  provided  with  a radar  view  typical  of  that  used  in  situations  of  limited  or  no 
visibility.  The  manoeuvring  controls  remain  constant  between  modes. 

Reporting  progress  to  the  ship's  commander  is  achieved  through  a set  of  interfaces  that  allows  the  trainee 
to  focus  on  single  or  multiple  facets  of  pilotage  such  as  monitoring  a headmark,  determining  gyro  error, 
fixing  the  position  of  the  ship  etc.  These  actions  are  recorded  in  the  ship's  log,  designed  to  support  learner 
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reflection.  The  simulation  can  be  operated  within  a practice  or  assessment  mode.  Within  the  practice  mode, 
prompts  triggered  by  the  actions  and  sequence  of  actions  of  the  trainee  are  provided  to  supply  intelligent 
feedback  on  their  progress.  In  this  mo  de  the  trainee  is  also  able  to  ’record’  and  ’playback’  their  pilotage  run 
from  a user-determined  point  in  time.  In  assessment  mode  the  trainee  enters  an  environment  absent  of  these 
support  and  review  mechanisms,  with  a complete  record  of  the  run  recorded  for  assessment  by  an 
instructor,  or  self-assessment  by  the  learner. 

The  reflective  process  is  facilitated  through  the  use  of  a number  of  key  features.  Whilst  the  trainee  is 
working  within  the  simulator,  practising  their  pilotage  skills,  their  current  actions  are  recorded  in  such  a 
manner  that  enables  them  to  review  their  past  actions  in  two  forms.  The  first  is  a list  version  of  the  key 
events  that  have  occurred  displayed  within  a floating  log  device  window.  The  second  is  a visual  ’re-run'  that 
reproduces  their  actions  just  as  they  occurred  and  displays  them  within  the  3D  visual  environment.  The 
provision  of  a ’snail-trail’  that  is  visible  within  the  chart  module  when  the  ship’s  movement  is  paused  or 
stopped  in  practise  mode  gives  a visual  trail  that  shows  the  history  of  the  ship's  movements  in  the  current 
simulation  run.  There  is  also  the  provision  to  ’save’  pilotage  chart  and  run  sequences.  These  are  small 
enough  to  be  easily  emailed  to  instructors  for  use  in  review  procedures.  A notes  device  which  allows 
trainees  to  collect  text,  images  or  animations  as  well  as  store  pilotage  run  files,  provides  a mechanism  to 
assist  them  in  making  sense  of  the  training  environment  through  establishing  and  storing  links  to  material 
of  interest  and  annotating  their  experiences  with  their  own  language.  The  combination  of  these  tools  of 
reflection  provides  a substantial  means  of  capturing,  reviewing  and  assessing  the  performance  of  the 
trainee. 

Conclusions  and  Research  Agenda 

The  package  offers  a wealth  of  research  opportunities,  particularly  in  investigation  of  the  use  of 
simulation  and  reflection  tools  to  develop  complex  cognitive  skills.  Initially  the  package  will  be  used  with 
the  Singapore  and  Australian  Navy  with  an  evaluation  process  informing  researchers  about  the 
effectiveness  of  the  approach.  An  initial  investigation  of  the  usefulness  of  the  package  will  be  undertaken 
through  user  testing  and  expert  review.  The  package  will  then  be  introduced  to  defence  personnel  in  both 
countries,  with  a comparison  of  the  traditional  classroom  outcomes  against  outcomes  when  using  the 
package.  Outcomes  measures  will  be  based  on  authentic  assessment  on  board  vessels  modelled  in  the 
package.  This  will  be  a starting  point  for  more  detailed  research  on  the  specific  tools  built  into  the  package. 
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Abstract: 

An  instructional  application  consists  of  a set  of  resources  and  activities  that  implement 
interacting,  interrelated  and  structured  experiences  oriented  towards  achieving  specific 
educational  objectives.  Computer-based  instructional  applications  have  to  be  faced  as  any  other 
development  activity  following  a well  defined  process.  With  this  purpose  some  design  methods 
for  computer-based  instructional  applications  have  been  proposed.  However  most  of  them  are 
focused  on  "on-line"  courseware  structures  which  are  quite  rigid  as  far  as  they  have  serious 
shortcomings  to  deal  with  courses  structured  in  multiple  ways.  Moreover,  these  methods  usually 
lack  a specific  mechanism  to  model  instructional  concepts  and  strategies.  This  work  proposes  a 
design  framework  to  develop  multi-structured  instructional  applications  combining  a didactic 
model  with  a software  engineering  approach  to  deal  with  educational  and  technical 
requirements.  The  underlying  model  extends  knowledge  structures,  such  as  those  involved  in 
the  Merrill’s  Transaction  Instruction  Theory,  adding  them  didactic  information.  It  also  considers 
the  functional  aspects  of  these  structures.  An  XML-based  notation  is  proposed  to  represent  such 
structures  and  their  management. 

1.  Introduction 

An  instructional  application  can  be  defined  as  a set  of  resources  and  activities  which  implement  interacting, 
interrelated,  structured  experiences  that  are  designed  to  achieve  specific  educational  objectives.  Instructional 
applications  are  usually  structured  by  means  of  static  patterns  which  are  based  on  a sequence  of  book-like 
electronic  pages.  They  are  designed  using  courseware  methodologies  that  focus  on  presentation  and  navigation 
issues  but  lack  a didactic  basis.  The  resultant  products  are  mostly  "pretty-printing"  on-line  courses  but  didactic 
aspects  are  hardly  considered.  The  current  work  proposes  a design  framework  which  is  independent  from 
computer-based  delivery  technologies  and  it  allows  the  representation  of  multiple  didactic  structures  from  an 
instructional  point  of  view.  Such  a framework  requires  a model  to  deal,  on  one  hand,  with  instructional  design 
topics  and,  on  the  other  hand,  with  the  design  of  computer-based  applications. 

Instructional  design  can  be  defined  as  the  discipline  that  connects  descriptive  theories  with  instructional  practice. 
Among  the  different  instructional  theories  proposed  in  the  literature,  we  will  assume  the  Instructional  Transaction 
Theory  (Merrill  1996)  since  it  makes  possible  to  figure  out  the  relationships  between  educational  and  technical 
components.  Merrill’s  mental  models  consist  of  two  major  components:  knowledge  structures  (schema)  and 
processes  for  using  this  knowledge  (mental  operations).  The  Merrill's  hypothesis  states  that  adequate  instruction 
would  require  multiple  types  of  knowledge  structures  to  be  identified  and  made  explicit  to  the  learner.  Thus,  the 
Instructional  Transaction  theory  provides  a powerful  tool  to  structure  the  knowledge  about  a given  topic  and  to 
define  the  procedures  for  accessing  it.  In  the  framework  proposed  in  this  paper,  Merril’s  model  is  extended  to  add 
didactic  information  to  knowledge  objects  and  structures.  A Didactic  model  is  built  using  the  previous  entities  as 
the  basis  and  assigning  them  attributes  whose  values  suggest  how  they  can  be  used  or  even  adapted  in  a specific 
learning  or  teaching  scenario. 

On  the  other  hand,  the  proposed  framework  deals  with  the  design  of  educational  computer-based  applications. 
Most  of  the  existing  methodologies  are  addressed  to  author  courseware  using  formats  such  as  HTML  documents. 
They  seem  adequate  for  presenting  and  accessing  information  in  a open  environment  like  the  Web  but  lack  a data 
structuring  capability.  Another  option  is  using  proprietary  multimedia  authoring  tools  like  Authorware'*'*^  or 
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Tooolbook'*''^  that  provide  a more  powerful  data  model  but  they  bound  the  reuse  and  exchange  of  content  module 
(Wiest,  2001).  In  this  sense,  an  important  effort  has  been  made  to  organize  and  to  manage  educational  data  using 
metadata  (Duval  2001).  However,  standard  metadata  proposals  mainly  aim  at  the  reuse  and  exchange  of  learning 
material  and  they  are  not  directly  involved  in  the  context  of  instructional  application  design.  In  the  current  work, 
notational  systems  based  on  metadata  are  used  to  represent  instructional  and  didactic  entities.  The  proposal 
consists  of  defining  an  XML-based  notation  which  specifies  the  Merrill's  Instructional  Transaction  Theory 
entities  and  their  extension  introduced  in  our  Didactic  model.  Such  notation  will  allow  an  instructor  to  define  his 
own  didactic  structures  which  can  be  built  from  a common  repository  of  instructional  objects. 

The  remainder  of  the  paper  is  organized  as  follows.  Second  section  revises  some  related  works.  Third  section 
introduces  the  design  framework  proposed  in  the  current  work.  Such  framework  is  based  on  a didactic  model 
whose  structural  components  are  presented  in  fourth  section.  Fifth  section  complements  this  model  with  its 
functional  view.  An  application  example  is  described  in  section  6.  Finally,  section  7 presents  some  remarking 
conclusions. 

2.  Related  work 


There  are  several  technology-based  educational  initiatives  that  propose  a separation  between  didactic  aspects  and 
content  related  issues.  In  this  context,  the  Multibook  project  ^teinacker  et  al.  1999)  considers  two  domain 
spaces:  the  Concept  Space  and  the  Media  Brick  Space.  The  first  one  contains  a network  of  knowledge  topics 
which  are  connected  via  semantic  relations.  The  second  one  contains  information  units  of  various  multimedia 
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formats.  Media  Brick  elements  are  linked  to 
“example’*,  “deepen”  or  “explain”  are  setup  between  these  elements.  A similar  approach  is  proposed  in  LMML 
(SuB  et  al  2000)  which  differentiates  Pedagogical  and  Instructional  properties  from  the  Module  and  Content 
objects.  However,  only  strategy  attributes  with  possible  constant  values  “beh”  (behaviouristic)  or  “con” 
(constructivistic)  can  be  specified.  Other  proposals  such  as  the  Targeteam  project  (Teege  2000)  or  the  Palo 
language  (Rodriguez  et  al  1999)  are  also  constrained  to  relations  or  mechanisms  such  as  motivation,  illustration, 
exercise  or  explanation  without  an  additional  didactic  value. 

In  metadata  contexts,  there  are  Educational  Modeling  Languages  (EML)  which  allow  the  specification  of  many 
kinds  of  educational  data.  These  data  can  be  implemented  using  notations  which  take  advantage  of  standard 
proposals  on  one  hand,  or  particular  and  specific  proposals  on  the  other  hand.  The  first  option  is  used  in  the 
Multibook  project  to  define  the  Media  Brick  elements  from  IEEE  Learning  Objects  or  the  Chameleon  project 
and  itsTeachML  (Wehner  2001)  notation  that  is  based  on  IMS  standards.  These  standard  formats  are  useful  for 
exchanging  them  in  different  learning  contexts  but  their  didactic  attributes  are  very  restrictive.  In  the  second 
category,  there  are  formats  such  as  LMML  that  defines  instructional  ContentObjects  which  contain  media  units 
such  as  tables,  lists,  images  or  text.  The  problem  is  that  these  units  are  strongly  coupled  with  instructional  objects 
and  it  prevents  to  assign  them  with  different  media  objects.  Similar  problems  are  found  on  other  EML  proposals 
such  as  Targeteam  contents,  EML  learning  objects  (Koper  2001)  or  Palo  elements. 


3.  Design  of  Instructional  Applications 


Figure  1 shows  the  global  architecture  in  the  proposed  instructional  application  design  framework.  The  upper 
level  deals  with  Instructional  Design  Theory  which  manages  entities  such  as  "knowledge  objects",  "knowledge 
structures"  and  "transaction  shells".  According  to  Merrill,  a knowledge  object  is  defined  as  “a  precise  way  to 
describe  the  content  to  be  taught”.  Knowledge  objects  can  be  combined  into  knowledge  structures.  Knowledge 
structures  are  external  representations  of  knowledge  that  are  parallel  with  mental  models  that  in  turn  are  internal 
(cognitive)  representations  of  models.  Transaction  shells  consist  of  rules  for  selecting  and  sequencing  knowledge 
objects.  The  entities  defined  in  the  Merrill's  model  are  the  basis  for  the  next  level  (Instructional  Design 
Modeling).  This  central  level  is  characterized  by  a Didactic  Model  whose  components  extend  the  previous 
entities  with  didactic  information.  They  are  divided  in  two  categories  that  represent  the  structural  and  functional 
model,  respectively.  Structural  model  is  composed  by  instructional  objects,  derived  from  knowledge  objects,  and 
didactic  structures  which  extend  knowledge  structures.  Functional  model  is  based  on  instructional  tasks  and 
learning  scenarios.  An  EML  notation  is  being  designed  to  specify  both  structural  and  functional  components. 
Next  section  will  describe  them  in  depth.  The  lower  two  levels  deal  with  the  computational  implementation  of 
instructional  applications.  The  first  one  is  based  on  a hypermedia  model  to  represent  the  instructional  entities  and 
their  relationships  in  an  formal  and  abstract  notation  ^uendia  et  al  2001).  The  second  level  is  related  to  the 
technology  involved  in  the  delivery  of  instructional  applications  using  an  e-leaming  environment. 
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Figure  1.  Instructional  Application  Design  Architecture. 


4.  Structural  model 

This  section  focuses  on  the  structural  components  of  the  Didactic  Model  proposed  in  this  paper.  The  simplest 
element  of  the  current  model  is  the  Instructional  Object  (10)  which  is  used  to  manage  Merrill’s  knowledge 
objects  from  a learning  point  of  view.  Instructional  Transaction  Theory  describes  knowledge  in  terms  of  three 
types  of  knowledge  objects:  entities,  activities  and  processes.  When  an  entity,  activity  or  process  is  addressed  in  a 
learning  context,  there  are  multiple  aspects  that  differ  depending  on  the  learning  conditions  around  the  knowledge 
object.  The  main  goal  of  10  elements  is  to  represent  such  aspects.  If  the  learning  target  is  an  entity,  lOs  such  as 
definitions,  statements  or  examples  can  be  used  to  describe  it.  Activity  knowledge  items  can  be  characterized 
using  lOs  such  as  exercises,  questions,  analogies,  hints  an  so  on,  and  finally,  processes  are  also  qualified  with 
simulations,  feedback  elements,  or  animations. 

lOs  are  not  only  isolated  information  units  such  as  definitions  or  exercises  but  they  are  also  concerned  with  the 
way  these  units  are  used  in  a learning  context.  For  instance,  an  entity  definition  is  assigned  with  a difficulty  level, 
a statement  declares  a property  value  assigning  it  a relevance  index  and  an  example  displays  the  entity  with  a 
specific  portrayal  (text,  audio,  image,  and  so  on).  Therefore,  lOs  must  consider  the  multiple  possibilities  for  a 
knowledge  object  to  be  learned  in  order  to  deal  with  different  learning  styles  and  needs. 

The  model  presented  in  this  paper  provides  a flexible  framework  to  incorporate  multiple  kinds  of  didactic 
information  which  can  be  added  to  lOs.  The  instructor  has  to  decide  the  different  difficulty  or  abstraction  levels 
assigned  to  each  object,  its  relevance  in  a certain  learning  context  or  the  portrayal  that  requires  a given  user 
profile.  This  design  process  is  highly  related  to  the  features  of  a subject  domain  and  it  becomes  a manual  and 
laborious  process.  Tools  as  metadata  notations  have  been  used  to  support  this  process. 

In  this  work,  there  is  a simple  XML  notation  proposal  and  the  definition  of  lOs  is  adapted  according  to  the 
subject  domain.  This  means  that  they  can  be  defined  on  top  of  other  entities  such  as  Merrill’s  knowledge  objects 
or  IEEE  learning  objects,  extending  them  with  didactic  elements  as  those  referred  in  Figure  2.  In  this  example,  a 
Description  10  is  used  to  teach  the  ’’magnetic  disk  structure”  (knowledge  object).  The  “ObjectDescription”  shows 
that  an  image  representation  has  been  selected  in  the  current  teaching  context.  Didactic  parameters  as  the 
portrayal  configuration  or  the  abstraction  type  indicate  how  this  image  is  displayed  and  the  level  of  abstraction  it 
represents.  EML  notations  usually  also  include  information  about  the  organization  of  the  educational  contents  in 
different  structures.  In  some  cases  like  the  Multibook  project  this  information  is  mixed  with  the  own  instructional 
objects.  In  our  model,  there  is  a strong  separation  between  both  types  of  information. 

Instructional  objects  are  organized  usmg Didactic  Structures  (DS)  which  can  be  managed  as  independent  entities. 
These  entities  are  addressed  to  capture  the  didactic  relationships  between  those  objects.  There  are  two  kinds  of 
didactic  relationships  in  the  current  model:  explicit  and  implicit.  Explicit  relationships  link  lOs  using  a specific 
action,  for  instance,  “an  example  illustrates  a concept  definition”  or  “a  question  evaluates  an  explanation”. 
Implicit  relationships  are  derived  from  the  way  knowledge  items  are  organized. 


4 “ 


59 


<ComponentValue  DidacticRef-' Image  reference"> 

<ObjectRef>Magnetic  Disk  Structure  reference  (storage  area).</ObjectRef> 

<lnstructlonalObjec1  ObjectID-’ 10-404"  ObjectName-' Description"  ObjectDescription-'BkIimensional  image"> 
<Portrayal  Dimension-' 2"  Background- false"  Position- 'center"/> 

<Abstraction  Type-'structural"  Depth- ’0"/> 

</!nstructionalObjec1> 

</ComponentValue> 

Figure  2.-  Instructional  object  example. 

In  this  case,  we  are  interested  in  knowledge  structures  coming  from  the  Instructional  Transaction  Theory  which 
are  used  in  the  current  work  to  model  DSs.  Merrill  mentions  different  types  of  knowledge  structures  such  as  lists, 
taxonomies,  dependencies,  algorithms  and  causal  nets.  A DS  can  be  built  on  top  of  one  or  more  knowledge 
structures.  For  instance,  the  description  of  a magnetic  disk  in  a computer  system  can  be  based  on  identifying  its 
components  and  assigning  them  a portrayal  configuration.  It  can  also  include  an  algorithm  representation  to  show 
the  access  to  a specific  component  (e.g.  a cylinder).  The  DS  connections  with  other  model  entities  are  represented 
on  Figure  3 It  shows  a simplified  UML  diagram  which  specifies  that  DSs  are  entities  aggregated  from 
instructional  objects  (10)  and  knowledge  structures  (KS).  Both  10  and  KS  entities  are  based  on  knowledge 
objects  (KO)  which  are  composed  by  elements  such  as  Identify  Portrayal  and  Properties.  DS  entities  also  extend 
the  basic  information  coming  from  10  and  KS,  using  didactic  attributes  such  as  the  portrayal  selection. 


Figure  3.-  Structural  Design  Model  entities. 


The  current  work  is  closer  to  the  IMSDL  proposal  l^ilberhom  & Gaede  1999)  which  defines  instructional 
strategies,  responsible  for  structuring  the  information  units  to  be  learned  but  independent  from  the  subject 
domain.  However,  these  structures  are  addressed  mainly  for  courseware.  Nevertheless,  we  do  not  know  any 
proposal  focused  on  generating  structure  templates  which  can  be  applied  in  specific  didactic  contexts  in  a 
reusable  and  modular  way.  We  are  developing  XML  Schemas  to  represent  a wide  range  of  didactic  structures 
from  basic  knowledge  structures. 

5.  Functional  model 

The  functional  model  describes  the  entities  that  allow  a user  (student  or  instructor)  to  interact  with  DSs.  DS 
entities  are  the  nexus  with  the  other  structural  elements.  Figure  4 shows  a simplified  UML  diagram  that 
represents  the  main  functional  entities  and  their  relationships  with  the  remainder  entities.  These  entities  are 
learning  scenarios  and  instructional  tasks,  respectively. 

Learning  scenarios  (LS)  are  defined  as  the  set  of  terms  and  conditions  that  characterize  the  user  learning.  Each 
individual  user  or  group  of  them  is  assigned  with  one  or  more  LSs.  Instructors  have  to  configure  the  LS  entities, 
assigning  features  such  as  learning  modes,  timing  schedules,  instructional  methods  and  learning  goals.  Each  LS 
aggregates  one  or  more  Instructional  Tasks  (IT).  They  are  defined  as  the  operations  a user  has  to  perform  to 
achieve  a specific  learning  goal.  IT  entities  model  the  functions  associated  to  the  didactic  structure  interface. 
They  can  access  to  one  or  more  didactic  structures  and  a certain  DS  can  be  attached  to  multiple  ITs. 
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Figure  4.-  Functional  Design  Model  entities. 


ITs  are  related  to  the  Merrill’s  concept  of  "transaction  shell"  which  consists  of  rules  for  selecting  and  sequencing 
knowledge  objects.  In  the  current  model,  IT  entities  do  not  access  directly  to  lOs  which  are  encapsulated  into  the 
didactic  structures.  This  feature  eases  the  design  process  because  the  instructor  deals  with  lOs  in  a more  abstract 
way.  For  instance,  in  a dependency  DS,  the  10  components  are  all  considered  as  chain  elements  without 
differentiating  them.  An  IT  like  Explain  specifies  the  navigational  operations  through  the  branches  of  the 
Dependency  structure.  A navigational  operation  can  consist  in  the  selection  of  a given  branch  and  it  could  depend 
on  learning  modes  such  as  “learning  by  examples”,  “by  doing”  or  by  "exploration  and  experimentation".  In  the 
first  case,  the  Explain  task  is  based  on  revising  Example  lOs  while  the  other  cases  involve  the  working  with 
Activity  lOs.  One  of  the  main  instructor  responsibilities  is  to  define  the  ITs  attached  to  each  DS.  This  definition  is 
also  a manual  and  laborious  task  and  an  EML  notation  is  being  developed  to  assist  the  instructor  in  this  process. 

6.  Application  example* 

In  this  section,  an  example  is  used  to  show  the  application  of  the  proposed  Didactic  model.  The  example  consists 
of  a Didactic  Guide  called  "Learning  XML  basics"  which  is  based  on  a Dependency  structure.  Figure  5 shows  a 
diagram  of  top-down  dependency.  The  root  node  represents  the  main  learning  goal  described  as  "Learning 
XML".  The  descendent  node  has  assigned  a Definition  10  which  intends  to  answer  the  "What  is  XML"  question. 
The  didactic  relationship  between  the  Goal  and  Definition  lOs  is  an  IsBasedOn  relationship  type  and  it  means 
that  the  knowledge  about  the  XML  notion  (current  node)  is  required  to  meet  the  goal  defined  in  the  previous 
node.  From  the  current  node,  there  are  several  branches  which  represent  extensions  (JsExtendedBy  relationships) 
of  the  current  definition  node.  These  branches  store  requirements  in  order  to  understand  the  previous  node  and 
each  one  has  assigned  a specific  competency  (difficulty)  level.  According  to  this  assignment,  a different  IT  can 
be  attached  to  each  branch.  These  competency  levels  are  based  on  the  learning  modes  cited  previously.  Learning 
by  examples  can  involve  identifying  instances  of  a concept,  e.g.  an  XML  document  (left  branch)  or  a DTD 
document  (central  branch).  Moreover,  each  XML  document  example  can  be  assigned  with  a certain  difficulty 
level  which  is  used  as  parameter  by  the  IT.  It  causes  that  the  structure  branches  can  be  navigated-accessed  in 
different  ways,  k Predict  IT  can  check  a specific  process,  e.g.  the  validation  of  an  input  XML  document  using  a 
DTD  syntax.  This  validation  process  is  based  on  2ir{  Activity  10  which  is  related  to  a previous  Example  object  by 
means  of  an  IsCheckedBy  didactic  relationship.  In  a higher  level  learning  mode,  the  Predict  task  could  include 
additional  activities  such  as  the  DTD  configuration. 

1.  Conclusions 

A design  framework  has  been  proposed  for  developing  instructional  applications  beyond  the  rigid  courseware 
structures  that  underlie  the  typical  Web-based  courses.  The  proposed  framework  is  based  on  a didactic  model 
which  provides  a gateway  between  instructional  theories  and  concepts,  and  hypermedia  and  Web  application 
engineering.  This  model  considers  two  issues:  a structural  view  based  on  instructional  objects  and  didactic 
structures,  and  a functional  view  that  manages  the  previous  entities  using  instructional  tasks  and  learning 
scenarios.  The  didactic  model  is  supported  by  an  XML-based  notation  which  eases  its  translation  to  computing 
environments. 
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Figure  5.-  Didactic  Structure  example. 


This  notation  has  been  applied  to  represent  taxonomy-like  didactic  structures.  It  provides  the  possibility  to 
organize  available  educational  resources  in  a way  closer  to  the  instructor  teaching  requirements.  We  have  also 
planned  to  extend  the  XML-based  specification  to  other  didactic  structures  based  on  dependencies  and 
algorithms.  Further  works  include  their  usage  in  specific  learning  contexts  such  as  Electric  and  Information 
Engineering  areas.  We  are  also  developing  a tool  to  allow  the  access  to  these  structures  in  a Web-based 
environment. 
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Videoconferencing:  A Tool  For  Collaboration  And  Professional  Development 
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Introduction 

With  the  expanding  infrastructure  for  telecommunications  in  the  world  today,  new  applications  for 
communications  are  emerging  in  society.  To  assure  success  in  the  workplace  of  tomorrow,  it  will  be 
necessary  for  students  to  have  collaboration  and  communication  skills  that  utilize  technology  tools 
(Schlechty,  1997).  As  a communicative  technology,  videoconferencing  offers  the  potential  to  provide 
students  relevant  skills,  connect  distant  or  remote  learners,  and  induce  change.  Teachers  and  administrators 
must  receive  training  in  the  design  and  application  of  appropriate  uses  of  videoconferencing  to  support 
effective,  efficient,  and  meaningful  learning.  Videoconferencing  has  been  used  in  higher  education  and 
businesses  for  several  years  and  now  is  spreading  to  K-12  environments.  “Educators  tend  to  use  new 
technologies  within  the  context  of  their  perceptions  of  instructional  strategies”  (Hayden,  1999). 

The  instructionist  teacher  views  the  characteristic  of  videoconferencing  that  allows  connection  to  a 
distant  expert,  as  an  opportunity  for  a remote  lecture.  On  the  other  hand,  the  constructivist  teacher  views  this 
same  characteristic  as  an  opportunity  for  students  to  ask  questions  and  clarify  meanings.  Educational  reform 
efforts  suggest  that  constructivist  teaching  strategies  and  learning  environments  should  be  explored  for 
student  activities  and  assessments  (Brooks  & Brooks,  1993;  Sandholtz,  Ringstaff  & Dwyer,  1997).  In  order 
to  prepare  teachers  and  schools  for  best  practices  using  videoconferencing  technology,  constructivist 
instructional  strategies  need  to  be  matched  with  new  technologies  (Hayden,  1999). 

This  paper  will  report  the  findings  from  a study  (Hayden,  1999)  that  investigated  the  “best  practices” 
for  videoconferencing.  The  paper  will  describe  the  strategies  that  were  applied  from  the  research  to  an 
educational  technology  professional  development  program  for  teachers  and  administrators  in  California. 

Research  on  Videoconferencing 


Description  of  study 

A study  completed  in  1999  (Hayden)  used  the  Delphi  process  in  order  to  facilitate  a panel  of  experts 
in  projecting  their  experiences  with  education,  technology,  and  videoconferencing  in  order  to  identify 
important  characteristics  and  critical  support  strategies  necessary  for  videoconferencing  in  constructivist  K- 
12  environments.  The  panel’s  recommendations  provide  a guide  for  technolo  gy  trainers  and  professional 
development  planners  in  implementing  effective  teacher  training  and  support.  As  one  participant  in  the 
Delphi  study  stated: 

I have  been  more  conscious  about  what  I’ve  organized  for  teachers  (VC  project  development)  and  in 
teacher  trainings.  I have  stressed  more  active  participation  among  the  students.  This  has  been  very 
helpful  for  me;  I have  referred  to  the  information/results  from  this  survey  several  times  in  trainings. 
Teachers’  ears  seem  to  perk  up  when  you  have  data  to  sipport  your  philosophies  and  different 
examples  to  give  (Hayden,  1999). 

Characteristics  of  Videoconferencing 

Twenty  characteristics  of  videoconferencing  that  support  constructivist  learning  emerged  in  the  study 
completed  by  Hayden  (1999).  These  twenty  characteristics  are  categorized  into  four  themes  in  Table  1:  (a) 
connections,  (b)  questioning,  (c)  learning,  and  (d)  interaction.  Within  this  list  of  twenty  characteristics,  ten 
characteristics  of  videoconferencing  were  rated  higher  than  the  other  ten.  By  organizing  the  top  ten 
characteristics  by  theme  area,  learning  experiences  using  videoconferencing  to  support  constructivist 
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learning  could  be  planned.  Table  1 shows  how  a videoconferencing  project  can  incorporate  all  ten  of  the  top 
videoconferencing  characteristics. 


Hieme 

Characteristic 

Scenario: 

Connections 

Synchronous  connections  between 
students  and  primary  sources  such  as 
experts  and  remote  locations; 
involving  multiple  sites  in  activities. 

Students  are  assigned  an  authentic 
task  that  offers  oppottunities  for 
planning  connections  outside  the 
classroom  with  experts  and 

Questioning 

Students  develop  and  ask  questions; 
they  are  in  charge  of  their  learning. 

primary  sources  such  as  museums. 
The  students  plan,  develop  and  ask 

Learning 

Students  present  to  remote  partners 
using  audio  and  video  for 
communication. 

questions  during  a 
videoconference  in  order  to  collect 
information  and  gain  an 

Interaction 

Students  work  in  groups  on  authentic 
activities  with  remote  sites.  This 
involves  videoconferencing  with 
remote  sites  and  use  of  an  online 
suite  of  tools  to  support 
videoconferencing  activities. 

understanding  of  key  concepts. 
The  students  use  a suite  of  online 
tools  as  follow-up  to  the  initial 
interaction  and  then  present  their 
findings  “live”  to  remote  partners 
concluding  their  research. 

Table  1:  Planning  An  Activity  Using  The  Top  Ten  Characteristics  Of  Videoconferencing  that  Support  Constructivist 
Learning  Environments  (Hayden,  1 999) 

Talking  head.  The  theme  of  remote  lecture,  sometimes  referred  to  as  “talking  head”,  has  been  a 
common  format  for  videoconferencing  sessions.  New  technologies  are  usually  implemented  first  in  the  old 
paradigm.  In  the  case  of  videoconferencing,  the  classroom  lecture  can  be  viewed  by  remote  learners  at 
different  locations.  The  problem  with  the  “talking  head”  model  is  that  participants  are  often  not  engaged 
because  they  are  not  physically  connected  with  the  presenter  by  being  in  the  same  room.  It  is  difficult  for  the 
presenter  to  see  all  of  the  remote  participants  and  he/she  may  not  be  aware  that  viewers  are  losing  interest  in 
the  session.  The  linking  of  videoconferencing  and  constructivist  strategies  tend  to  avoid  the  “talking  head” 
model  and  participants  become  more  engaged  as  they  actively  participate  in  the  session. 

Shared  electronic  documents.  One  way  of  engaging  participants  in  a videoconference  is  to  display 
shared  documents  and  images  to  reinforce  concepts  and  address  multiple  learning  modalities.  The 
characteristic  of  “students  creating  a shared  electronic  document  using  synchronous  connections”  was  ranked 
19^^  in  the  twenty-item  list  of  videoconferencing  characteristics.  One  suggestion  made  by  an  expert  in  the 
Dephi  study  was  for  “more  accounting  for  the  various  media  that  are  used  in  VC,  i.e.  elmo,  video  tape, 
computer,  whiteboard,  etc.  All  of  these  can  enhance  the  potential  for  constructivist  uses  of  VC.” 

Support  Strategies 

The  three-round  Delphi  process  identified  critical  strategies  that  support  videoconferencing  in 
constructivist  environments.  The  top  ten  strategies  for  supporting  videoconferencing  are  categorized  into  six 
themes:  (a)  people,  (b)  access,  (c)  hardware  and  software  resources,  (d)  materials,  (e)  staff  development,  and 
(f)  cost/budget  (see  Table  2). 
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Abstract:  The  need  to  convert  our  stand-alone  computer-based  learning  modules  to  a form  that  would  be 
independent  of  any  single  location  or  brand  of  computer  became  clear  to  us  several  years  ago.  We  were  also 
interested  in  improving  the  availability  of  our  software.  The  growth  and  standardization  of  the  World  Wide 
Web  provided  us  with  a format  offering  color,  graphics,  animation,  and  hypertext.  Distribution  problems 
would  be  eased,  and  the  distance  education  possibilities  were  also  an  advantage  of  this  approach.  This 
presentation  focuses  on  bow  the  initial  choice  of  an  interactive,  hypertext  format  was  central  to  our 
repurposing  effort. 


Education  as  the  first  motive 

Bonfils  Blood  Center’s  Transfusion  Medicine  Education  Program  was  created  to  address  the  lack  of 
familiarity  of  many  medical  practitioners  with  transfusion  issues  (Simon,  1985).  We  wanted  to  provide  medical 
students  with  application-level  experience  of  practical  procedures  or  principles  (clinical  correlates)  involving 
topics  already  identified  as  important  in  the  field.  Cases  were  chosen  that  involved  many  of  those  procedures  or 
principles.  These  cases  were  grouped  into  thematically  related  modules,  which  were  part  of  the  regular  course 
of  medical  studies.  Given  the  extensive  medical  curriculum  and  the  already  busy  schedule  of  the  medical 
student,  we  felt  a learner-centered,  computer-based  approach  would  be  best.  We  have  developed  over  twenty 
cases,  grouped  into  four  modules,  in  our  first  dozen  years. 

Active  Learning 

The  instructional  advisors  to  this  program,  from  the  University  of  Colorado  at  Denver's  School  of 
Education  and  University  of  Colorado  School  of  Medicine,  felt  strongly  that  a learner-centered,  active  learning 
design  would  be  most  appropriate  (Jonassen,  Ambruso,  Olesen,  1992).  We  had  to  provide  experiences  leading 
to  competency  in  a complex,  ill-structured  domain.  We  hoped  to  avoid  the  usual  over-simplified  presentation 
that  often  leads  to  "inert  knowledge"  (Whitehead,  1929).  We  decided  on  an  approach  combining  elements  of 
cognitive  flexibility  theory,  case-based  learning,  and  instructional  simulation  (Heath,  Ambruso  1996). 

Hypertext 

The  obvious  form  for  our  educational  experiences  was  hypertext  (linked,  interactive,  randonvaccess 
information).  However,  by  the  time  we  started  working  on  our  fourth  set  of  instructional  cases,  it  was  obvious 
that  the  Internet  itself  had  far  outpaced  the  neglected  HyperCard  product  as  an  implementation  of  the  hypertext 
concept.  We  decided  to  try  to  replicate  our  first  module  as  a series  of  HTML  documents  (Internet  "pages").  The 
result  proved  very  popular  with  students  and  faculty.  Internet-based  versions  of  two  other  modules  were 
produced,  and  a third  conversion  is  underway  using  DHTML  and  other  advanced  web  technologies. 

Opportunities  for  Updating,  Refining 

When  a case  or  module  is  rewritten  as  World  Wide  Web  pages,  it  can  also  be  revised.  The  new  version 
can  reflect  the  latest  advances  in  both  the  field  of  transfusion  medicine  and  (“below  the  hood")  in  the  newly 
available  capabilities  of  the  Internet. 
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Adding  Color 

The  use  of  color  in  the  standalone  CBT  versions  was  limited.  No  need  was  seen  for  color  when  these 
programs  were  originally  written.  Furthermore,  the  colorization  schemes  that  became  available  for  HyperCard 
were  awkward  and  prone  to  technical  difficulties.  In  the  Internet  environment,  color  has  always  been  an 
inherent  part  of  all  visual  elements,  at  least  in  the  Netscape  and  Explorer  programs . We  hope  that  the  presence 
of  color  and  other  appropriate  graphic  design  elements  may  serve  to  enhance  the  learner’s  experience. 

Client-side  Data 

One  challenge  of  any  Internet-based  instructional  system  is  the  lack  of  an  obvious  way  to  store 
information  for  later  use.  Security  issues  require  browser  makers  to  prevent  any  Internet  page  from  gaining 
direct  access  to  the  computer’s  permanent  storage.  The  ‘cookie’  method  allows  limited  creation  of  small 
amounts  of  data  on  a computer,  but  many  people  turn  this  feature  off. 

Global  Variables 

Instead  of  using  local  files  for  learner  data  that  our  programs  may  need,  we  store  information  in 
variables  associated  with  the  opening  page,  which  remains  open  but  hidden  once  the  current  page  is  opened  in 
front  of  it.  Alternatively,  we  use  a 'frame'  portion  of  the  visible  window  to  store  information  used  by  subsequent 
'frame'  portions  of  the  same  window. 

For  example,  the  students  enter  their  names,  which  appear  at  various  points  in  the  program  and  on  the 
completion  report  that  is  sent  to  us  automatically  when  the  student  completes  the  program.  Another  example  is 
the  laboratory  test  orders  placed  at  one  point  in  the  program,  which  must  be  acknowledged,  and  the  results 
given,  at  a later  point  in  the  program.  Both  the  student  name  and  their  orders  must  be  stored  in  such  a way  as  to 
be  available  to  the  routines  that  send  us  their  report  or  that  show  them  their  earlier  decisions  or  test  results. 

Data  Management 

Combining  JavaScript  variables,  DOM  element  addressing,  and  Cascading  Style  Sheet  controls,  we 
have  created  a client-side  database  management  system  that  would  not  have  been  possible  with  the  technology 
available  when  this  program  began. 

Conversion  Process 

We  considered  the  following  factors  when  we  decided  to  convert  our  CBT  to  web-based  instruction: 

Does  my  CBT  need  to  be  converted  to  web-based  instruction? 

• Hypertext  is  suited  to  Internet 

• Cross-platform  operability  (need  to  use  any  computer) 

• Availability  of  graphic  elements  (color,  animation) 

• Ease  of  distribution,  maintenance 
What  challenges  face  such  a conversion? 

• Different  results  on  different  platforms,  browsers,  versions . . . 

• New  capabilities  not  evenly  implemented  across  browsers 

• An  Internet  application  does  not  have  full  disk  access 
What  are  some  possibilities 

• Client-side  data  management 

• Animation 

• Wide  distribution 
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Over  the  past  decade,  there  have  been  many  changes  in  the  tools  we  use  to  design,  the  ways 
information  can  be  represented  and  the  underpinning  theories  which  drive  educational 
experiences.  This  presentation  will  focus  on  several  examples  of  software  design  that  have 
been  pedagogically  successful  and  have  demonstrated  what  is  possible  in  software  design 
and  online  learning.  Contrasts  will  be  made  with  some  examples  of  the  current  push  into  c- 
leaming  and  how  best  to  structure  learning  environments  to  ensure  student  participation  and 
high  quality  learning  outcomes  especially  when  students  come  from  differing  backgrounds 
and  cultural  traditions. 


“A  poor  tradesman  blames  his  tools” 

Over  the  past  decade  there  have  been  several  major  uevelopmerits  which  have  helped  the  growth  of  interactive 
multimedia  and  more  recently  the  concept  of  e-leaming.  The  task  for  designers  has  been  challenging  and  this 
paper  reviews  several  projects,  many  of  which  broke  new  ground  and  won  international  awards.  Along  the  way 
the  reality  of  creating  innovative  products  which  represent  good  practice  and  seek  to  model  modem  educational 
principles  has  been  driven  by  several  factors  not  the  least  being  the  tools  which  have  been  used  and  the  visual 
representations  of  the  knowledge  domain  of  each  product.  This  paper  will  describe  several  projects  and  critique 
their  design  ideas  and  their  execution. 

Significant  efforts  have  been  made  to  develop  and  implement  alternative  frameworks  for  learning  often  based  on 
a class  of  theories  collectively  referred  to  as  constructivism.  Fundamentally,  constructivism  asserts  that  we  learn 
through  a continual  process  of  constructing,  interpreting,  and  modifying  our  own  representations  of  reality  based 
on  our  own  experiences.  Indeed  many  books  enumerate  a long  list  of  ideas  about  how  these  principles  might  be 
applied  to  the  design  of  learning  environments,  but  how  to  place  the  ideas  strategically  into  the  learning 
experience  is  often  omitted  (see  for  example,  Khan,  2001;  and  Mills,  Lawless  and  Merrill,  2001).  Often  the 
advice  is  very  broad  and  covers  all  aspects  of  pedagogical  design  from  methods  to  integrate  new  technologies  to 
potential  assessment  strategies.  The  integration  of  technologies,  which  may  allow  the  representation  of  ideas  in 
many  different  media  forms,  provide  opportunities  for  the  designer  or  instructor  to  customis  e instruction  and 
place  learners  in  open-ended,  student-centred,  rich  tasks. 

This  paper  explores  what  has  been  effective  and  what,  while  superficially  might  be  seen  as  an  effective  product 
or  set  of  ideas,  might  not  have  been  as  successful  as  first  expected.  It  will  generalise  some  of  the  lessons  that 
might  be  drawn  from  the  decade  of  effort  to  ensure  that  the  learning  environments: 

1 . fostered  judgement  and  learner  responsibility. 

2.  supported  critical  inquiry  and  creative  approaches  to  problem-solving 

3.  created  engagement  through  the  effective  combination  of  learning  task,  visual  representation  and 
authentic  assessment  of  the  product  goals. 

Principles  assumptions  and  quality 

Like  past  revolutions  in  education,  e-leaming  will  go  the  way  of  previous  technologies  unless  there  are  changes 
to  the  design  framework  used  as  the  starting  point.  Savery  & Duffy  (1995)  described  four  principles  that  should 
be  applied  to  modem  technology-based  learning  environments  based  on  constructivist  views.  These  were: 

1.  Learning  is  an  active  and  engaged  process.  “Learners  are  actively  engaged  in  working  at  tasks  and 

activities  that  are  authentic  to  the  environment  in  which  they  would  be  used.”  (Savery  & Duffy,  1995, 
p.37). 
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2.  Learning  is  a process  of  constructing  knowledge.  Learners  need  structures  and  challenges  from  which  to 

develop  their  understanding  of  ideas  and  of  the  world. 

3.  Learners  function  at  a metacognitive  level.  Learning  is  focused  on  thinking  skills  rather  than  working 

on  the  “right  answer  the  teacher  wants”.  Students  generate  their  own  strategies  for  defining  the 
problem  and  working  out  a solution.  Student  can  gain  wisdom  through  reflection. 

4.  Learning  involves  “social  negotiation”.  Students  are  able  to  challenge  their  thoughts,  beliefs,  perceptions 

and  existing  knowledge  by  collaborating  with  other  students  thus  assisting  their  cognitive 
development  process. 

In  a recent  paper,  David  Boud  and  Mike  Prosser  (2002)  have  also  attempted  to  specify  the  characteristics  of  high 
quality  learning  outcomes.  They  suggest  that  the  four  major  areas  of  concentration  in  a high-quality  learning 
environment  should  be: 

1.  How  do  learning  activities  support  learner  engagement?  (This  reasons  for  the  learner  wishing  to 
become  involved  with  the  learning  tasks  and  the  way  the  tasks  require  them  to  reflect  or  employ  their 
previous  interests  and  understandings) 

2.  How  does  this  learning  activity  acknowledge  the  learning  context?  (In  the  case  of  e-leaming  there  are 
unique  characteristics  such  as  the  unique  position  of  the  learner  in  a real  context  and  how  the 
assessment  matches  the  real  world  skills  that  are  seen  as  easily  transferred  from  the  learning  context  to 
professional  practice) 

3.  How  does  the  learning  activity  seek  to  challenge  learners?  (Novices  need  supportive  structures, 
experts  require  information  to  fill  in  the  missing  blanks  in  an  existing  knowledge  structure,  too  much 
ambiguity  can  turn  a student  away,  too  little  and  they  become  bored.  Students  might  need  support  to 
extend  the  information  provided  as  part  of  a problem-solving  scenario) 

4.  How  does  the  learning  activity  provide  practice?  (As  with  most  effective  learning  contexts  the  matches 
between  assessment,  learning  tasks,  and  the  transfer  tasks  might  align  and  model  performance.  To 
ensure  that  it  occurs,  the  feedback  must  support  the  ongoing  development  of  the  learning) 

In  addition,  in  the  e-learning  context  the  choices  of  technology  infrastructure  and  its  deployment  are  crucial  to 
support  the  effective  learning  outcomes.  So  the  above  lists  describe  the  goals  for  constructivist  design  and  high 
quality  outcomes,  but  the  choice  of  tools  and  the  range  of  pedagogical  options  that  the  tools  themselves  either 
constrain  or  facilitate  will  also  contribute  to  the  learning  outcomes.  David  Jonassen  (2000)  has  provided  recent 
guidance  about  the  importance  of  the  design  of  learning  tasks  by  suggesting  a range  of  problem  types  that  vary  in 
the  degree  of  structure  and  the  linkage  they  have  to  authentic  real  world  tasks.  Providing  structure  and  support 
for  the  more  ill -structured  task  is  the  real  challenge  for  the  designer  working  in  a constructivist  framework. 

Design  intentions  for  digital  media 

With  an  understanding  of  the  shortcomings  of  much  of  the  commercially  generated  available  learning  packages, 
a combination  of  ideas  taken  from  constructivist  learning  environments,  situated  learning  and  problem-based 
learning  in  rich  information  landscapes  can  be  used  to  form  the  basis  for  effective  design.  Hedberg  et  al  (1994) 
proposed  that  learning  outcomes  in  digital  environments  depend  on  starting  points  such  as;  the  learning 
environment;  the  learner's  view  of  the  purpose  of  the  task;  and  the  motivation  of  the  learner. 

The  process  of  learning  involves  the  construction  of  meanings  by  the  learner  from  what  is  said,  demonstrated  or 
experienced.  Thus  the  role  of  the  teacher  is  one  of  facilitating  the  development  of  understanding  by  selecting 
appropriate  experiences  and  then  allowing  students  to  reflect  on  these  experiences.  Often  constructivist  learning 
situations  suddenly  throw  students  on  their  own  management  resources  and  many  fend  poorly  in  the  high 
cognitive  complexity  of  the  learning  environment.  Cognitive  support  tools  and  the  explicit  acknowledgment  of 
the  double  agenda  of  metacognitive  self-management  and  learning  can  help.  The  scaffolding  and  coaching  of  the 
cognitive  apprenticeship  model  offers  another  solution,  a strategy  which  many  design  teams  have  explored  with 
a great  deal  of  success. 

Several  multimedia  design  models  have  been  developed  which  illustrate  the  combination  of  complex  learning 
environments  and  which  also  give  students  their  own  real  control  over  their  learning  environment.  If  one  of  the 
primary  goals  of  e-leaming  is  to  stimulate  active  involvement,  then  educators  and  instructional  designers  need  to 
better  understand  the  symbiosis  between  the  visual  interface  and  the  design  of  the  learning  tasks  in  promoting 
and  sustaining  learner  engagement.  Engaged  learners  are  intrinsically  motivated  to  perform.  They  direct  their 
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efforts  to  understanding  the  tasks  and  challenges  in  a learning  context;  and  they  strive  to  construct  knowledge 
and  derive  meaning  from  their  prior  experience  and  available  resources.  Well  designed  visual  interfaces  can  help 
stimulate  learner  engagement  or,  conversely,  disengage  learners  if  they  are  poorly  designed.  Poor  design  can 
place  high  cognitive  demands  upon  the  learner  that  can  reduce  interest  and  divert  attention  away  from  the 
primary  learning  tasks.  While  not  described  in  detail  here,  the  combination  of  visual  clarity  of  knowledge 
representation  and  manipulation  and  the  sensitivity  to  outcomes  of  the  learning  task  is  which  create  challenge 
and  engagement  is  a measure  of  the  design  success  (Metros  and  Hedberg,  2002). 

Instructional  designers  continue  to  wrestle  with  the  challenge  of  affording  a new  generation  of  visually  savvy 
learners  with  engaging  online  experiences.  In  her  seminal  book  Computers  as  Theatre,  Laurel  (1993)  suggested 
ways  to  use  the  notion  of  theatre,  not  simply  as  a metaphor,  but  as  a way  to  conceptualize  human-computer 
interactions.  Laurel  defines  this  type  of  engagement  as,  “what  happens  when  we  are  able  to  give  ourselves  over 
to  a representational  action,  comfortably  and  ambiguously.  We  gain  a plethora  of  new  possibilities  for  action  and 
a kind  of  emotional  guarantee*’  (pi  15).  What  Laurel  is  referring  to  isF/ow  State,  Csikszentmihalyi  (1996)  coined 
the  term  to  describe  the  state  of  total  engagement.  Users  attain  Flow  State  when  they  have  no  conscious 
awareness  of  the  passage  of  time.  Flow  State  occurs  when  users  enjoy  a sense  of  playfulness,  a feeling  of  being 
in  control,  a period  of  concentration  when  attention  is  highly  focused,  an  interlude  of  enjoyment  of  an  activity 
for  its  own  sake,  a distorted  sense  of  time,  and  a rewarding  match  between  the  challenge  at  hand  and  one's 
personal  skills.  The  emphasis  on  motivating  tasks  situated  within  well-designed  and  engaging  interactions 
provides  the  instructional  designer  greater  surety  that  the  final  experience  will  be  effective. 

Creating  a tramework  for  review 

Step  1:  Information  Design  and  Project  Space  Deflnition 

At  the  beginning  of  any  project  it  is  necessary  to  compile  information  on  learners’  needs  and  describe  the 
parameters  of  the  project  space.  The  purpose  of  this  initial  stage  is  to  begin  a holistic  structuring  of  the 
information  and  to  model  it  so  that  it  will  eventually  form  the  basis  of  an  organizing  visual  metaphor.  Isolating 
the  key  attributes  within  a learning  experience  is  not  trivial.  Three  questions  need  to  be  answered  by  the  client 
and  designers. 

1 . What  is  the  topic  (content)  of  this  project? 

2.  Who  are  the  intended  users  of  this  knowledge  domain?  The  content  might  be  the  same  for 
different  groups,  but  learners  may  want  to  “view,  use  or  manipulate”  the  content  in  different 
ways. 

3.  Why  is  this  project  (course)  being  undertaken  or  developed?  The  client’s  original  stated  objective 
almost  always  needs  revisions  to  better  identify  and  describe  the  underlying  purpose. 

Step  2;  Interaction  Design 

Effective  interaction  design  that  matches  users’  cognitive  expectations  ensures  they  are  motivated  and  engaged. 
Simply  the  use  of  interactive  technologies  does  not  necessarily  ensure  that  meaningful  interactions  will  occur, 
rather  The  challenge  is  to  create  interactions  that  are  easily  manipulated  at  the  users’  technology  skill  level. 
Norman  (1988)  provides  guidelines  for  constructing  interactions. 

1.  Visibility:  The  user  can  tell  the  state  of  the  device  and  the  alternatives  for  action  through  observation. 

2.  An  effective  conceptual  model:  There  should  be  consistency  in  how  program  functions  “work” 
leading  to  a coherent  conceptual  user  model. 

3.  Effective  mapping:  There  are  clear  relationships  between  actions  and  results,  controls  and  their 
effects,  and  between  system  state  and  what  is  visible. 

4.  Feedback  : There  must  be  continuous  feedback  about  the  results  of  actions.  Novice  designers  often 
fail  to  realize  that  almost  every  action  creates  some  perturbation.  For  feedback  to  be  effective, 
designers  should  employ  a variety  of  feedback  that  link  to  specific  learning  outcomes. 

In  addition  to  Norman’s  guidelines,  there  are  alternative  techniques  for  reducing  the  cognitive  load  on  a learner’s 
working  memory  to  enhance  interaction  design. 

1 . Use  visual  conventions  borrowed  from  the  real  world, 

2.  Apply  consistent  visual  metaphors, 

3.  Recognize  the  role  of  the  learner  as  actor.  The  user  is  participating  in  a dialogue  that  is  unfolding, 
often  in  real-time  (Laurel,  1993;  Hedberg  & Sims,  2001). 
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In  several  products,  we  provide  access  to  the  data  in  the  same  way  as  individuals  would  access,  manipulate  and 
explore  resources  in  the  real  world.  While  there  still  must  be  supportive  scaffolds  and  structures,  the  choices  the 
learners  make  under  this  model  are  similar  to  those  they  make  as  “experts”  in  a knowledge  domain.  The  only 
difference  is  that  the  tasks  have  supportive  elements  that  describe  decisions  in  context  (Lave  & Wenger,  1991). 
Visual  metaphors  and  information  structures  must  provide  information-rich  presentations,  especially  when  the 
structures  are  extensible  and  can  be  unfolded,  as  the  learners  need  more  or  less  support  and  scaffolding  to 
complete  their  chosen  learning  tasks. 

Step  3:  Presentation  and  Interface  Design 

The  third  step  transforms  the  design  concepts  derived  in  the  first  two  steps  into  a visual  presentation  structure, 
which  represents  concepts,  conveying  order,  classifying  information,  clarifying  meaning,  directing  focus, 
stimulating  interest,  facilitating  interactions,  confirming  choices,  supporting  recall,  directing  navigation,  creating 
ambience  and  otherwise  engaging  the  learner.  In  many  products,  we  chose  visual  metaphors  of  commonly-used 
places  to  serve  as  the  context  in  which  decisions  about  the  learning  tasks  take  place.  Thus  we  used  a field 
research  centre  for  the  ecological  set  of  problems,  a classroom  with  small  groups  placed  around  it  to  model 
groupwork  for  teaching  K-2  mathematics. 

Issues  that  have  influenced  the  journey 

While  the  above  design  intentions  and  process  has  been  following  to  a greater  or  lesser  extent,  the  following  key 
aspects  that  have  influenced  the  “judged  effectiveness”  of  each  project  and  have  formed  the  basis  of  comparison: 

• Technological  advances  — major  advances  in  the  past  ten  years  have  made  a difference.  Especially 
the  advent  of  QuickTime  in  late  1991,  the  widespread  availability  of  World  Wide  Web,  and  the  use 
of  browsers  to  drive  cross  platform  products  from  1994. 

• Visual  Design  — mirroring  the  technology  developments,  several  graphical  design  tools  have 
improved  and  developed  in  the  decade.  Photoshop  enables  visual  design  overlays  with  their  “layers” 

• Design  of  Learning  Tasks — the  visual  representation  and  the  tools  for  constructing  environments 
have  been  instrumental  in  achieving  the  growth  in  complex  learning  tasks  and  how  they  have  been 
implemented  in  software. 

• Authentic  assessment  options — as  digital  media  became  easier  to  integrate  within  products,  the 
possibilities  of  presenting  authentic  environments  where  the  assessment  tasks  were  realistic  and 
authentic  became  greater. 

• Clarity  of  team  goals  and  ease  of  operation  — there  are  several  options  about  how  a team  functions, 
as  projects  grew  bigger  the  size  of  the  team  also  increased  but  the  relationships  between  client  and 
designers,  and  the  others  contributing  to  the  final  product  can  have  a critical  bearing  on  the  clarity  of 
the  representation  of  the  knowledge  space  and  the  symbiotic  relationship  between  the  learning  task 
and  the  visual  representation  of  the  task  through  the  interface. 

• Degree  of  control  of  design  team  versus  client  demands  for  particular  outcomes.  The  client  can 
support  options  to  ensure  new  and  creative  implementation  of  their  ideas.  If  this  occurs  then  the 
project  can  push  the  boundaries,  create  new  representations  and  more  challenging  tasks. 

• Implementation  support  in  the  field  — a project  may  have  the  best  of  intentions  but  it  may  not  have 
support  which  makes  it  viable  to  generate  high  use  by  the  intended  audience.  Also  the  increasing 
combination  of  CD-ROMs  and  web  sites  designed  to  maximise  the  feedback  and  interactivity  can 
ensure  that  the  products  have  the  equivalent  of  “after  sales  service”. 

Examples  of  effective  Symbiosis 

The  complete  list  of  examples  is  provided  at  http://www.emlab.uow.edu.au/.  However  simple  illustrations  of  the 
review  can  be  drawn  from  the  CD  entitled  “723  Count  with  mef  which  illustrates  the  application  of  the  model 
(Figure  1).  The  CD  introduces  basic  mathematical  concepts  to  K-2  teachers  and  shows  them  how  they  might  use 
an  innovative  instructional  strategy  to  group  students  and  introduce  basic  mathematical  concepts.  If  a 
comparison  is  made  between  the  two  different  approaches  to  a similar  task  (Figures  1 and  2),  it  can  be  seen  that 
the  poor  representation  of  Figure  2 (and  the  fact  that  it  varies  with  cursor  movement)  can  be  very  difficult  for  an 
audience  which  does  not  hold  sophisticated  interface  conventions.  The  comparison  can  be  made  over  a range  of 
factors  which  are  outlined  in  the  next  section.  (Other  examples  will  be  demonstrated  in  the  presentation.  These 
will  be  based  on  several  products,  including:  Stagestruck,  Exploring  the  Nardoo,  aspire^  Pilotage  and  Count  me 
in  too.) 
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Figure  1,  Modelling  the  information  and  the 
message  in  ^123  Count  with  me^’  through  a spatial 
metaphor  familiar  to  the  intended  audience 


Figure  2.  An  alternative  implementation  of  similar 
ideas  without  solid  links  to  the  environment  in 
which  they  are  to  be  employed. 


Ensuring  successful  outcomes 

There  are  many  factors  that  might  be  judged  as  creating  successes  and  when  they  are  lacking  producing  less 
successful  products.  However  in  this  paper,  I will  concentrate  on  a few: 

Implementing  the  design  concept.  An  idea  at  the  concept  stage  may  not  necessarily  translate  into  a good  visual 
representation  as  the  project  develops.  For  instance,  in  developing  materials  to  teach  teachers  how  to  implement 
a math  program  then  the  visual  representation  of  those  ideas  has  to  correspond  with  the  original  concepts  and 
intentions.  Comparing  the  two  figures  (1  and  2)  above  which  illustrate  two  forms  of  the  same  idea.  Figure  1 was 
judged  more  comprehensible  by  the  client.  While  in  Figure  1,  the  major  focus  was  to  introduce  mathematical 
understandings  to  an  audience  of  remotely  located  teachers,  for  Figure  2,  the  emphasis  was  on  explaining  the 
nature  of  the  mathematical  framework  and  the  visual  elements  became  more  focussed  on  decoration  rather  than 
functionality. 

Representing  information  so  that  it  can  be  accessed  and  manipulated.  In  the  same  example,  in  providing  a list  of 
options  the  use  of  the  directory  metaphor  has  been  employed.  In  the  early  stage  of  design  of  123  count  with  me^ 
we  identified  three  organizing  areas  in  which  the  related  elements  of  information  could  be  clustered  and  which 
provide  meaningful  structures  for  a teacher  using  a classroom  metaphor. 

Implementing  constructive  and  problem-based  activities.  In  many  learning  environments,  learners  can 
manipulate  the  elements  which  form  the  basis  of  their  construction.  Again  in  the  above  example,  several 
capabilities  have  been  provided  which  simplify  some  of  the  teacher/user  tasks.  There  is  a subtlety  and  a strategy 
to  the  design  of  activities  which  will  lead  to  active  use  and  engagement.  The  choice  of  problem  (Jonassen,  2000) 
and  the  subtle  development  of  activities  that  move  from  social  to  task  focus  suggest  the  direction  of  challenge  for 
the  designer.  Jonassen  (2000)  has  demonstrated  how  his  initial  theory  of  problem  design  can  be  transited  into 
challenging  tasks.  Often  this  will  mean  that  initially  task  a designed  as  series  of  discreet  assignments,  a rethink 
can  create  a series  of  linked  tasks  and  challenges,  and  when  these  are  woven  into  an  ongoing  scenario  in  which 
the  student  builds  their  skills  as  the  go  through  the  activities  sequence  then  powerful  learning  outcomes  can  be 
produced. 


Conclusion 

Effective  e-leaming  environments  build  upon  solid  foundations  of  learner  needs  and  outcomes,  metaphorically 
crafted  cognitive  processes,  and  instructional  strategies  communicated  through  effective  visual  interfaces  that 
intrigue,  challenge  and  engage  the  learner.  Crafting  of  such  projects  requires  an  approach  in  which  the 
instructional  designer,  technologist,  graphic  designer,  educator  and  student  collaborate  to  ensure  that  the  end 
result  is  usable,  functional  and  visually  communicative  and  attractive.  This  can  be  accomplished  by  employing 
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reasonable  yet  innovative  conventions;  such  as  organizing  visual  metaphors  that  scaffold  access  to  the 
underpinning  knowledge;  choosing  tasks  that  support  the  desired  level  of  challenge  and  engagement.  As 
interactive  technologies  become  the  staple  communication  vehicle  for  virtual  worlds,  effective  interactive 
instructional  design  based  on  constructivist  principles  will  ensure  that  the  learner’s  focus  on  the  learning  task 
rather  than  simply  operating  the  software.  The  choice  of  authoring  tools  can  also  greatly  affect  the  way 
information  may  be  structured  and  manipulated.  Popular  authoring  tools  may  actually  reduce  the  interactivity 
and  information  representations  that  are  possible  (Hedberg  & Sims,  2001).  Achieving  success  means  that  there  is 
coherence  between  task,  its  visual  representation  and  the  project  functionality.  So  the  choice  of  tool,  the  selection 
of  task  and  the  importance  of  an  engaging  learning  experience  need  to  be  undertaken  with  insight.  And  for  e- 
learning,  while  some  tools  might  facilitate  the  physical  process  of  putting  educational  materials  online  and 
creating  learner  access,  they  may  also  never  address  the  educational  outcomes  for  high  quality  learning. 
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Summary  of  key  projects  and  their  focus 

123  - Count  me  in  Too  Credentialing  Package  (in  production)  Client:  NSW  Department  of  Education  and 
Training,  Curriculum  Support  Directorate. 

Following  the  success  of  the  earlier  package,  this  package  is  being  designed  to  assess  the  performance  of  those 
teachers  who  are  seeking  accreditation  as  “experts”  in  the  use  and  application  of  the  123  K-2  mathematics  program.  It 
consists  of  a CD-ROM  and  links  with  a central  site  that  will  mark  each  authentic  assessment. 

123  - Count  with  Me  (2002)  Client:  NSW  Department  of  Education  and  Training,  Curriculum  Directorate. 

The  primary  focus  of  the  ‘123  Count  With  Me’  CD-ROM  project  is  to  provide  professional  development  to  primary 
school  teachers  in  small,  remote  schools  who  do  not  have  easy  access  to  district  consultancy  support.  The  project  is 
based  on  the  content  of  the  Curriculum  program  “Count  Me  in  Too”,  which  is  a numeracy  program  aimed  at  K-2. 

This  numeracy  program  increases  teachers'  understandings  about  how  children  can  be  assisted  to  progress  through  the 
early  components  of  the  Mathematics  syllabus  in  relation  to  Number. 

K-6  Creative  Arts  Syllabus  Professional  Development  (2002).  Client:  NSW  Department  of  Education  and 
Training,  Curriculum  Directorate 

The  aim  of  this  project  is  to  provide  a tool  for  teachers  that  helps  them  to  implement  the  Creative  Arts  K-6  syllabus. 
The  CD-ROM  illustrates  and  explicates  the  syllabus  and  CSD  resources,  demonstrating  the  syllabus  in  practice. 
Practice  and  methodology  is  shown  through  a range  of  visual  and  text  modes.  The  CD-ROM  allows  teachers  to 
explore  all  aspects  of  the  syllabus,  in  a way  that  suits  them  as  adult  learners.  The  structure  and  content  contained 
within  the  resource  will  support  investigation  by  individual  teachers  working  alone  or  in  groups  working  under  the 
guidance  of  a curriculum  officer. 

Pilotage  Courseware  (2001).  Client:  IMPART  Corporation,  RAN  and  SAF 

The  Royal  Singapore  Navy  and  Royal  Australian  Navy  requested  the  submission  of  a creative  solution  to  increase  the 
level  of  success  that  trainee  Officers  of  the  Watch  currently  enjoy  and  the  level  of  access  they  have  to  a flexible, 
practical  training  environment.  Trainee  Officers  are  provided  with  the  basic  background  theory  of  pilotage,  and  a 
means  of  practicing  the  application  of  this  knowledge  and  developing  associated  skills.  This  occurs  in  a simulated 
‘real  life’  environment  that  is  consistent  with  being  a pilot  on  a combat  ship.  In  effect  this  provides  an  effective 
training  environment  whilst  avoiding  the  usual  risks  and  costs  associated  with  conducting  such  practice  in  the  real 
world. 

aspire  —Olympic  Resource  Kit  for  Australian  Schools  (1999).  Clients:  SOCOG;  AOC;  Collaborators:  Dept  of 
Education  and  Training;  Curriculum  Corporation 

aspire,  an  educational  kit  developed  for  the  2000  Olynpic  Games  comprises,  a CD-ROM,  booklet,  posters  and  video 
and  was  distributed  to  schools  Australia  wide  in  1999.  The  aim  of  the  kit  is  to  foster  enthusiastic  engagement  in  the 
Olympic  experience  and  a belief  in  the  Olympic  spirit.  It  sought  to  raise  intaest  in  the  Sydney  2000  Olympic  Games 
and  promote  the  richness  and  cultural  diversity  of  Australian  life  and  national  identity.  The  target  audience  was  wide 
K-12. 

StageStruck(1998).  Collaborators:  The  National  Institute  of  Dramatic  Art,  The  Australian  Ballet,  Opera 
Australia  and  The  Sydney  Opera  House 

StageStruck  - was  developed  with  the  assistance  of  the  Federal  Government’s  Australia  on  CD  Program.  StageStruck 
was  designed  for  students  to  experience  the  world  of  the  performing  arts  as  well  as  showcasing  the  diversity  of 
Australian  Performers  and  Perfomiing  Arts  Companies.  StageStruck  provides  a highly  interactive  environment  where 
students  are  able  to  explore  each  of  the  elements  associated  with  designing  and  directing  a performance  whilst 
creating  their  own.  Awards  have  been  numerous  and  include:  In  1998,  EMMA  Gold  Award-  best  overall;  BAFTA 
Interactive  Treatment  award  in  the  United  Kingdom.  In  1999,  New  York  Festivals  International  Interactive 
Multimedia  competition  - Gold  Medal  in  the  Education  section;  AIMIA  Award  - Best  Arts/Cultural  Title  or  Site  and 
in  2000  it  won  the  Milia  d'Or  Awards  for  the  Education  & Training  Category 
Exploring  the  Nardoo  (1996).  Client:  Department  of  Land  and  Water  Conservation,  New  South  Wales 

This  is  a program  based  on  an  inland  water  catchment,  which  can  be  investigated  through  four  time  zones  and  four 
locations  in  each  time  zone.  Each  time  zone  has  a particular  theme  that  transposes  the  four  locations  and  each 
location  will  have  an  embedded  investigation  in  the  form  of  a scenario.  Users  can  move  readily  from  location  to 
location  in  each  time  zone  and  from  zone  to  zone.  Students  will  be  able  to  interrogate  data  and  personalise  results. 
The  user  investigation  is  supported  by  help  facilities  and  guides  as  well  as  simulations  and  games  designed  to  support 
solutions  to  problems.  In  1?  ? ? it  won  an  EMMA  Award  (European  Multi-Media  Association)  and  one  of  best  five 
new  educational  products  for  1997  by  the  American  Software  Publishers  Association. 

Investigating  Lake  I luka  (1993).  Collaborator:  Apple  Computer  Australia 

A simulated  lake  environment  on  CD  ROM.  Students  can  investigate  various  ecosystems  of  the  lake,  make  physical, 
chemical  and  biological  measurements,  collect  information  about  individual  animal  and  plant  species  in  and  around 
the  lake,  and  see  and  hear  news  reports,  expert  views  and  local  opinions  on  a variety  of  issues  relating  to  the  lake  and 
its  ecology.  Inquiry  and  problem-solving  techniques  have  been  embedded  in  the  package  through  case  studies  of 
ecological  scenarios  presented  to  the  user  via  media  reports  of  problems  posed  directly  to  the  user. 
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This  paper  describes  the  evaluation  of  high  quality  learning  designs  which  are  being 
selected  for  possible  redevelopment  in  a National  Project  funded  by  the  Australian 
University  Teaching  Committee  (AUTC).  The  project  focuses  on  ‘‘Information  and 
Communication  Technologies  (ICTs)  and  Their  Role  in  Flexible  Learning”  and  is 
evaluating  over  50  projects  with  a view  to  developing  a range  of  software  tools, 
templates  and/or  guidelines  based  on  those  that  are  de  emed  to  be  effective  ICT  - 
based  learning  projects.  The  approach  is  unique  in  that  it  tries  to  pinpoint  the  key 
attributes  of  ICT-based  projects  that  make  them  suitable  for  application  in  other 
contexts  and  in  other  knowledge  domains. 


Introduction 

There  tends  to  be  general  consensus  among  experts  that  the  forms  of  learning  environments  most  effective  for 
meaningful  learning  in  higher  education  are  those  that  are  based  on  the  contemporary  theories  of  learning  which 
support  knowledge  construction  through  learner -centred  settings  (e.g.  Duffy  & Cunningham,  1996;  Bostock, 
1998).  These  perspectives  about  learning  are  challenging  conventional  teaching  approaches.  For  example, 
Cunningham  et  al.  (1998)  state: 

The  growing  acceptance  of  new  educational  philosophies  and  practices,  such  as  constructivism 
and  action  learning  during  the  1980s,  have  challenged  the  valence  of  the  didactic 
lecture/tutorial/textbook  model  common  in  higher  education,  promoted  the  notions  of  the 
academic  role  as  ‘a  guide  on  the  side*  rather  than  ‘the  sage  on  stage’,  and  conceived  of  the  student 
role  as  one  of  independent  self  -directed  learner,  (p.  25) 

The  growing  awareness  of  effective  and  meaningful  teaching  and  learning  plus  the  recent  developments  in  ICT 
has  led  to  synergies  emerging  between  the  use  of  ICT  and  the  adoption  of  powerful  learning  strategies.  The  Web 
is  one  technology  that  shows  particular  promise  for  supporting  meaningful  learning  through  its  remarkable 
functionality,  support  for  flexible  delivery  modes  and  capacity  to  link  and  connect  those  involved  in  the  learning 
process  (e.g.  Levin,  1999).  The  possibilities  exist  for  rich  learning  based  on  this  technology,  but  for  the  most 
part,  pedagogically  sound  and  exciting  Web  courseware  tools  have  yet  to  be  developed  to  take  advantage  of  such 
opportunities. 

One  of  the  key  issues  is  that  the  pace  of  change  of  emerging  Web  technologies  is  so  rapid  that  pedagogical 
models  may  be  needed  to  help  create  Web  tools  from  a learner-centred  perspective.  Salomon  (1998)  has 
supported  this  concern  and  has  noted  that  for  the  first  time  in  history,  technologies  are  outpacing  pedagogical 
and  psychological  rationale.  However,  a body  of  literature  is  starting  to  report  on  innovative  tools,  with  strong 
pedagogical  underpinning.  Bonk  (1998)  has  reported  on  interactive  tools  for  on-line  portfolio  feedback,  profile 
commenting,  and  Web  link  rating.  Oliver  and  McLoughlin  (1999)  are  building  tools  for  on-line  debate, 
reflection,  concept  mapping  and  student  surveying  and  discussion.  Wills,  Ip  & Bunnett  (2000)  are  building 
engines  for  online  role  plays.  As  a way  of  describing  the  range  of  options  that  might  contribute  to  a learning 
design  in  such  contexts  Figure  1 shows  the  combination  of  elements  that  might  be  considered  in  such  an 
endeavour. 

In  this  project,  the  terms  “learning  designs”,  “high  quality  learning  experiences”,  and  “flexible  learning”  are 
defined  as  follows:  Learning  designs:  refer  to  a variety  of  designs  that  support  student  learning  experiences. 
Learning  designs  may  be  at  the  level  of  a whole  subject,  subject  component  or  learning  resource.  High  quality 
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learning  experiences : refer  to  experiences  resulting  from  an  environment,  which  encourages  students  to  seek 
understanding  rather  than  memorisation  (only  for  the  purposes  of  assessment),  and  which  encourage  the 
development  of  lifelong  learning  skills.  Flexible  learning,  refers  to  an  educational  approach  that  meets  the 
diverse  needs  of  students.  The  project  is  to  focus  on  how  ICT  can  be  used  to  design  flexible  opportunities  for 
students. 
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Figure  1:  Elements  of  learning  designs  for  online  learning 


Current  settings  hold  fewer  impediments  to  ICT  uptake  than  have  been  present  in  the  past.  Universities  within 
Australia  have  moved  swiftly  in  recent  years  to  develop  the  necessary  infrastructure  to  support  ICT  as  a delivery 
medium  and  most  universities  now  boast  a solid  ICT  infrastructure  aimed  at  supporting  teaching  and  learning 
programs.  The  uptake  of  ICT  as  a delivery  medium  has  been  supported  by  professional  development  programs 
and  activities  aiming  to  develop  the  ICT  literacy  of  staff  plus  clearinghouses  and  Web  sites  for  dissemination  of 
information  about  ICT  in  teaching.  Funding  has  been  applied  by  government  sources  to  support  the  development 
of  university  teaching  and  learning  and  many  organisations  now  exist  that  support  and  promote  quality  teaching 
as  a scholarly  pursuit.  Among  the  major  impediments  that  still  stand  are  the  lack  of  quality  teaching  and  learning 
models  and  appropriate  instructional  material  and  software  for  teachers  to  apply. 

This  project  aims  to  provide  some  relief  to  these  impediments  by  identifying  and  creating  quality  resources  for 
generic  and  mainstream  application  and  by  providing  appropriate  support  and  resources  that  will  guide  and 
encourage  their  use.  Projects  of  this  type  should  result  in  a coming  decade  that  witnesses  a growth  in 
pedagogically  based  learning  technologies. 


Aims  of  this  project 

The  aim  of  this  project  is  to  assist  university  instructors  to  create  high  quality  flexible  learning  experiences  for 
students  by  providing  a range  of  generic  resources/tools/templates/guidelines  that  draw  upon  successful  flexible 
learning  projects  that  utilise  ICT  and  which  may  be  generalised  beyond  the  scope  of  the  individual  project. 
Successfijl  ICT -based  learning  projects  are  those  that  facilitate  high  quality  learning  experiences  for  students.  A 
study  conducted  by  Alexander  and  McKenzie  (1998)  highlighted  that  one  contributing  factor  towards  a 
successful  learning  outcome  for  an  ICT-based  learning  project  was  the  learning  design  employed.  Thus,  this 
project  has  followed  the  process: 

1.  Identification  of  a range  of  learning  designs  that  have  been  demonstrated  to  contribute  to  high 
quality  learning  experiences  and  which  can  be  applied  generically; 
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2. 


Design  and  subsequent  development  of  a series  of  re-usable  software,  templates  and/or  guidelines 
for  the  learning  designs  previously  identified;  and 
3.  Dissemination  of  good  practice  for  the  use  of  or  implementation  of  the  software,  templates  and/or 

guidelines  in  new  contexts. 

Crucial  to  the  success  of  this  project  was  the  development  of  an  evaluation  instrument  referred  to  by  the  project 
as  an  Evaluation  and  Redevelopment  Framework  with  a twofold  purpose: 

1.  To  facilitate  the  identification  of  learning  designs  that  foster  high  quality  learning  experiences;  and 

2.  to  provide  a mechanism  to  determine  whether  such  learning  activity  designs  have  the  potential  for 
re-development  in  a more  generic  form. 


Project  structure 

The  project  began  in  November  2000  and  is  structured  against  four  milestones: 

1.  Milestone  One  (May  2001):  Development  of  the  Evaluation  and  Redevelopment  Framework 

2.  Milestone  Two  (November  2001):  Identification  and  documentation  of  learning  designs  that  foster 
high  quality  learning  experiences  and  that  have  the  potential  for  redevelopment  in  a more  generic 
form. 

3.  Milestone  Three  (June  2002):  Development  of  a selected  number  of  learning  designs  in  a generic 
form  to  at  least  prototype  stage. 

4.  Milestone  Four  (December  2002):  Completion  of  the  developiuent  of  learning  designs  in  a more 
generic  form  and  finalisation  of  a web  site  that  will  store  the  project’s  developed  resources. 


Development  of  the  Evaluation  and  Redevelopment  Framework 

Characterising  High  Quality  Learning 

A major  project  activity  has  been  the  critique  of  what  constitutes  “high  quality  learning”.  Professor  Boud  and 
Associate  Professor  Prosser  were  commissioned,  as  two  leading  thinkers  about  learning  in  higher  education  in 
Australia,  to  develop  a paper  on  high  quality  learning.  Their  ideas  together  with  feedback  from  the  project  team 
led  to  the  development  of  a set  of  “Key  Principles  for  High  Quality  Student  Learning  in  Higher  Education— 
from  a Learning  Perspective”  (Boud  & Prosser,  2001).  The  key  principles  describe  four  main  characteristics  that 
underpin  high  quality  learning  in  t he  higher  education  context.  The  principles  are  elaborated  through  a series  of 
questions  that  provide  a lens  through  which  learning  environments  can  be  explored.  The  four  principles  are 
holistic  in  that  they  incorporate  both  learning  outcomes  and  learning  processes  and  are  based  upon  an 
experience- based  leame^centred  view  of  learning.  The  four  principles  are  outlined  below  in  the  form  of 
descriptions  of  high  quality  learning  activities. 

High  quality  learning  activities: 

1.  Engage  learners  through: 

■ Building  on  their  learning  intents  generally  and  their  particular  expectations  of  the  activity  in  question; 

■ Acknowledging  and  taking  account  of  their  prior  experience,  both  their  knowledge  and  experience  of 
situations  which  might  impinge  on  the  present  ones; 

■ Mobilising  their  will  and  desire  and  developing  some  kind  of  emotional  engagement  with  the  task  in 
hand; 

■ Providing  them  with  a sense  of  agency  with  respect  to  the  activity  or  significant  parts  of  it;  and 

■ Recognising  that  learning  is  a social  act  and  involves  other  learners  for  at  least  part  of  the  activity. 

Z Acknowledge  context  through: 

■ Involvement  with  problems  in  context; 

■ Recognising  the  context  of  the  learner  (who  may  see  themselves  as  decontexualised); 

■ Maintaining  an  awareness  of  the  cultural  assumptions  and  stereotyping  which  may  be  incorporated  in 
the  context; 

■ Situating  learning  tasks  within  disciplinary  or  professional  or  practical  knowledge  as  appropriate; 

■ Taking  account  of  the  site  of  application  of  what  is  to  be  learned  (this  poses  different  challenges  when 
the  learner  is  currently  engaged  in  the  site  of  application  and  when  they  are  not); 

■ Appreciating  the  knowledge  demands  on  students  and  equipping  them  to  deal  with  them;  and 

■ Ensuring  that  there  is  a clear  alignment  between  the  activities  in  whidi  students  will  be  engaged  and  the 
ways  in  which  they  will  be  assessed. 
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3.  Challenge  learners  through: 

■ Prompting  them  to  seek  and  discern  variation  in  the  knowledge  and  experiences  in  which  they  are 
involved; 

■ Questioning  the  assumptions  they  bring  to  the  activity  and  the  assumptions  they  develop  through  it; 

■ Encouraging  them  to  see  what  is  provided  as  a means  to  wider  ends  and  go  beyond  what  is  provided; 
and 

■ Creating  situations  in  which  they  are  required  to  take  responsibility  for  their  own  learning  and  to  shape 
the  activity  to  their  own  ends. 

4 Involve  practice  through: 

■ Demonstrating  what  has  been  learned  for  themselves  and  for  others; 

■ Gaining  feedback  at  strategic  points  in  learning,  but  also  recognising  that  finding  ways  of  gaining 
feedback  for  one  self  other  than  that  provided  is  also  important; 

■ Reflecting  on  and  making  sense  of  their  experiences.  Continuous  exposure  to  new  activities  without 
integration  and  consolidation  within  the  learner’s  framework  is  not  conducive  to  good  learning;  and 

■ Developing  confidence  in  performance  from  practice. 


Developing  the  evaluation  instrumentation 

Whilst  the  above  principles  formed  the  basis  of  the  evaluation  framework,  the  following  issues  were  raised  by 
the  reviewers  that  they  should  be  incorporated  into  the  instrument: 

■ How  technology  is  embedded  in  a learning  design  and  how  its  use  supports  or  hinders  the  learning 
experience. 

■ The  issues  of  scalability,  transferability,  and  technology  affordances. 

■ To  determine  suitability  of  redevelopment  of  a learning  design,  the  evaluation  should  provide  a 
mechanism  to  glean  the  critical  design  features  from  a learning  design  and  consider  how  these  design 
features  could  be  implemented  in  a more  generic  form. 

■ To  place  the  review  framework  within  a staged  process  which  might  inform  the  project  through  a series 
of  critical  decision  points. 

The  first  complete  version  of  the  evaluation  instrument  was  devised  by  the  Core  Team  and  Research  Team  after 
the  first  workshop.  This  version  was  formatively  evaluated  in  the  second  workshop  (s  cheduled  one  month  after 
the  first).  The  Research  Team  also  examined  existing  evaluative  instruments  to  determine  whether  these  could 
inform  and/or  be  incorporated  into  the  project’s  evaluation  framework. 

Since  the  second  workshop  (held  at  the  end  of  April  2001)  the  evaluation  framework  has  undergone  further 
review  and  formative  evaluation.  Feedback  from  the  Project  Review  Panel  and  International  Reference  Group 
has  been  considered  and  via  discussions  with  the  Core  Team,  Research  Team  and  Steering  Committee,  a revised 
version  of  the  evaluation  framework  has  been  developed.  A challenge  for  the  project  has  been  how  to  elucidate 
the  key  and/or  unique  elements  of  the  learning  design  that  enable  the  facilitation  of  a high  quality  learning 
experience  for  studerts.  The  strategy  thus  adopted  is  to  request  a description  of  the  learning  design  by  the 
designer(s)  in  a contextualised  form  in  terms  of  the  following: 

■ The  learning  activities  (and  their  sequence)  that  students  are  required  to  do. 

■ The  resources  that  are  required  to  support  the  activities. 

■ The  support  mechanisms  that  characterise  the  learning  design,  eg.,  role  of  the  instructor,  establishment 
of  collaborative  teams,  etc. 

In  addition,  all  resources  utilised  by  the  students  along  with  any  evaluation  data  or  findings  have  also  been 
submitted.  The  evaluation  review  framework  was  implemented  in  two  phases.  The  purpose,  process  and 
outcome  for  each  phase  are  outlined  in  table  1. 


Table  1:  Evaluation  Redevelopment  Framework  Implementation 

Phase 

Purpose 

Process 

Outcome 

Phase  1 

• Identify  and 
describe  the 
learning  design. 

• Assess  the  data 
sources 

Completion  of  two  instruments: 

1.  Learning  Design  Submission  Form — to  be 
completed  by  the  designer^ s)  of  the  learning 
design. 

2.  Learning  Design  Assessment  Form — to  be 

• Detailed  description  of  the 
learning  design  from  the 
designer(s). 

• Decision  whether  to  proceed 
to  Phase  2.  (If  to  proceed  to 
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Table  1:  Evaluation  Redevelopment  Framework  Implementation 

Phase 

Purpose 

Process 

Outcome 

provided  and 
determine 
whether  to 
proceed  to 
Phase  2. 

completed  by  the  Project  Manager  on  receipt 
of  the  completed  Learning  Design 
Submission  Form 

Phase  2,  details  about 
Evaluation  Team  specified.) 

Phase  2 

Evaluation  of  the 
learning  design  in 
terms  of: 

• Its  potential  to 
facilitate  high 
quality  learning 
experiences  for 
students. 

• Its  suitability 
for 

redevelopment 
in  a more 
geno-ic  form. 

Completion  of  one  instrument:  Learning 
Design  Evaluation  Form . 

The  instrument  comprises  eight  questions: 

• Questions  1 to  4 address  the  potential  of  the 
learning  design  to  foster  high  quality 
learning. 

• Question  5 addresses  how  the  technologies 
employed  facilitate  the  learning  design. 

• Questions  6 and  7 are  designed  to  elucidate 
the  key  and/or  unique  elements  of  the 
learning  design. 

• Question  8 requires  a judgement  to  be  made 
about  whether  the  learning  design  is  suitable 
for  redevelopment  i n a more  generic  form. 

The  instrument  is  to  be  completed  individually 
by  two  evaluators.  The  evaluators  are  to  reach 
consensus  and  submit  one  completed 
Evaluation  Form  . 

• Judgement  of  the  potential  of 
the  learning  design  to  foster 
high  quality  learning. 

• Generic  description  of  the 
learning  design. 

• Judgement  of  the  learning 
design’s  suitability  for 
redevelopment  in  a more 
generic  form. 

The  current  stage  of  the  project 

The  current  stage  of  the  project  is  applying  the  framework  to  a number  of  Learning  Design  exemplars.  The 
outcome  from  this  activity  is  intended  to  provide: 

1.  Documentation  of  Learning  Designs  identified  as  having  potential  for  redevelopment  in  a more 
generic  form;  and 

2.  A formative  evaluation  of  the  framework  and  its  operationalisation  to  a level  of  “robustness” 
deemed  adequate  by  the  project  team. 

In  this  stage  we  have: 

■ Identified  over  50  potential  ICT -based  learning  exemplars  for  examination.  28examples  will  undergo 
full  evaluation.  Some  strategies  employed  to  compile  the  list  of  exemplars  included:  nominations  made 
from  the  project  team,  review  of  past  CUTSD  (Committee  for  University  Teaching  and  Staff 
Development)  projects;  and  a review  of  relevant  literature  sources. 

■ Established  approximately  30  Evaluation  Teams.  Evaluation  Teams  comprise  pairs  of  national  and 
international  experts  in  the  use  of  information  and  communication  technologies  for  teaching  and 
learning  in  Higher  Education.  Nominations  have  been  made  by  the  Project  Core  Team  and  by 
participants  who  attended  a national  flexible  learning  conference  in  July  2001. 

From  the  feedback  received  it  appears  that  many  of  the  learning  designs  the  core  team  identified  for  evaluation 
are  suitable  for  redevelopment  but  the  expense  may  not  warrant  it.  Preliminary  analysis  of  the  evaluations 
returned  to  date  raises  the  following  questions  for  the  core  team  to  consider: 

■ Do  we  have  a large  enough  range  of  types  of  learning  designs? 

■ Is  the  evaluation  framework  adequate  to  enable  reviewers’  analysis  of  the  generic  attributes  of  any 
learning  des  ign? 

■ Are  reviewers  capable  of  distinguishing  between  the  task  of  software  evaluation  and  this  task  of 
evaluating  the  underpinning  learning  design? 

■ Are  reviewers  evaluating  a particular  implementation  of  a learning  design  for  redevelopment  or  are  they 
taking  into  account  other  implementations  of  that  particular  learning  design? 

■ Do  we  have  an  adequate  definition  of  learning  design  and  has  it  been  communicated  to  the  evaluators? 

■ Is  the  evaluation  process  able  to  cover  levels  of  granularity  in  learning  designs?  Should  the  tasks, 
resources  and  supports  in  the  learning  design  also  be  evaluated? 

■ Can  learning  designs  really  be  context  -free? 
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■ Will  the  essence  of  a learning  design  always  be  translatable  into  a software  product  or  would  it  instead 
be  better  to  write  guidelines  for  good  design? 

In  order  to  ensure  the  range  of  learning  designs  reviewed  is  not  limited  in  representation  of  a broad  range  of 
pedagogical  approaches,  the  project  team  explored  the  development  of  a Learning  Design  classification 
framework.  An  analysis  of  the  learning  design  exemplars  collected  to  produce  a grounded  learning  design 
categorisation  plus  a review  of  categorisations  of  learning  designs  in  the  literature  will  be  presented  at  the 
conference.  The  project  Web  site  aims  to  inform  people  of  the  progress  and  provide  access  to  the  resources  and 
materials  as  they  have  been  developed,  (http://www.learningdesigns.uow.edu.au) 
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Abstract:  This  paper  provides  an  overview  of  Student  Modeling  techniques  that  have  been 
employed  in  Intelligent  Language  Tutoring  Systems  (ILTSs)  over  the  past  decade.  We  further 
discuss  the  Student  Model  of  our  ESL  Tutor,  an  ILTS  for  English  as  a Second  Language.  The 
Student  Model  is  based  on  student  subject  matter  performance  and  provides  feedback  and 
remedial  exercises  suited  to  learner  expertise.  We  further  report  on  a study  in  which  we 
determined  the  extent  to  which  our  Student  Model  addresses  the  need  for  an  individualized 
language  learning  environment. 


1.  Introduction 

individualized  language  instruction  has  long  been  recognized  as  a significant  advantage  of  Computer-Assisted 
Language  Learning  (CALL)  over  more  traditional  workbook  tasks.  A "one  size  fits  all"  approach  is  not 
appropriate  for  a learning  environment.  Students  learn  at  their  own  pace  and  often,  work  for  their  own  purposes. 
Learners  also  vary  with  respect  to  prior  language  experience,  aptitude,  and/or  learning  styles  and  strategies. 
According  to  the  Individual  Differences  Theory  as  described  by  Oxford  (1995),  if  learners  learn  differently, 
then  they  likely  benefit  from  individualized  instruction. 

Despite  the  need  for  an  individualized  learning  environment.  Student  Modeling  has  not  been  a strong 
focus  of  CALL.  One  likely  reason  is  that  in  order  for  a computer  program  to  adapt  itself  to  different  learner 
needs,  the  system  needs  a dynamic  model  of  the  strengths  and  weaknesses  of  the  learner  (McCalla  & Greer, 
1992).  Even  when  it  comes  to  Intelligent  Language  Tutoring  Systems  (ILTSs),  only  a few  have  employed 
Student  Models  to  individualize  the  learning  environment.  According  to  Holland  & Kaplan  (1995),  this  is  likely 
due  to  the  challenging  task  of  representing  the  domain  knowledge  itself,  the  module  which  contains  facts  and 
information  about  the  language  being  taught.  If  the  grammar  is  not  accurate  and  complete,  even  a precise 
Student  Model  cannot  compensate.  For  instance,  Holland  (1994)  states  that  a system  which  does  not  detect 
ambiguous  errors  accurately  will  obscure  a Student  Model. 

There  are  a number  of  modeling  techniques  that  can  be  implemented  in  a computer  program.  The 
system  can  model  subject  matter  performance,  students'  learning  strategies  and/or  cognitive  styles.  ILTSs 
primarily  model  subject  matter  performance,  that  is,  students’  surface  errors.  While  such  a Student  Model  might 
not  be  complete,  it  assists  in  individualizing  the  language  learning  process  and  “is  sufficient  to  model  the 
student  to  the  level  of  detail  necessary  for  the  teaching  decisions  we  are  able  to  make”  (Elsom-Cook,  1993: 
238). 

In  this  paper,  we  describe  the  Student  Model  of  the  ESL  Tutor,  our  Web-based  ILTS  for  English  as  a 
Second  Language  (ESL).  The  ESL  Tutor  analyzes  sentences  from  the  student  and  detects  grammatical  and  other 
errors.  The  feedback  modules  of  the  system  correlate  the  detailed  output  of  the  linguistic  analysis  with  an  error- 
specific  feedback  message.  The  Student  Model  is  based  on  student  subject  matter  performance.  It  provides 
feedback  and  remediation  suited  to  learner  expertise. 

In  the  following  section,  we  provide  examples  of  CALL  systems  that  employ  Student  Models  and 
discuss  their  distinct  emphasis.  In  section  3,  we  describe  the  architecture  of  the  Student  Model  of  the  ESL  Tutor. 
Section  4 reports  on  a study  in  which  we  determined  the  extent  to  which  our  Student  Model  addresses  the  need 
for  an  individualized  language  learning  environment.  Concluding  comments  can  be  found  in  section  5. 


2.  Student  Modeling  and  Intelligent  Language  Tutoring  Systems 

In  analyzing  Student  Models,  McCalla  (1992)  makes  a distinction  between  implicit  and  explicit  Student 
Modeling  which  is  particularly  useful  in  classifying  the  Student  Models  in  ILTSs. 
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An  implicit  Student  Model  is  static,  in  the  sense  that  the  Student  Model  is  reflected  in  the  design 
decisions  inherent  to  the  system  and  derived  from  a designer’s  point  of  view.  For  instance,  in  an  ILTS  the  native 
language  of  the  learner  can  be  encoded  as  a bug  model  that  includes  frequently  made  errors  and  ultimately 
diagnoses  them. 

In  contrast,  an  explicit  Student  Model  is  dynamic.  It  is  a representation  of  the  learner  which  is  used  to 
drive  instructional  decisions.  For  ILTSs,  for  instance,  the  Student  Model  can  assist  in  guiding  the  student 
through  remedial  exercises  or  it  can  adjust  instructional  feedback  suited  to  the  level  of  the  learner.  In  either 
case,  the  decisions  are  based  on  the  previous  performance  history  of  the  learner.  The  following  discussion  will 
provide  examples  of  ILTSs  which  have  implemented  implicit  and  explicit  Student  Models. 

2.1  Implicit  Student  Models 

Implicit  Student  Modeling  has  been  applied  to  ILTSs  to  diagnose  errors.  For  example,  in  Catt  & Hirst’s  (1990) 
system  Scripsi  the  native  language  of  the  student  represents  the  learner  model.  It  is  used  to  model  the  learner’s 
interlanguage.  With  regard  to  Student  Modeling,  the  pitfall  of  such  an  implementation  is  that  it  is  a static 
conception.  The  system’s  view  of  the  learner  cannot  change  across  interactions  with  the  system.  It  has  no 
impact  on  instructional  decisions  and  provides  only  a gross  individualization  of  the  learning  process  when 
ideally,  a Student  Model  is  dynamic  (Holt  et  al.,  1 994). 

In  a more  individualized  example.  Bull  (1994)  developed  a system  that  teaches  clitic  pronoun 
placement  in  European  Portuguese.  The  Student  Model  is  based  on  the  system’s  and  the  student’s  belief 
measures,  language  learning  strategies,  and  language  awareness. 

The  system’s  belief  measure  is  comprised  of  the  proportion  of  incorrect/correct  uses  of  the  rule;  the 
students  provide  the  data  for  the  student’s  belief  measure,  being  required  to  state  their  confidence  in  their 
answer  when  entering  sentences.  Learners  also  identify  their  preferred  learning  strategies  when  using  the 
program.  According  to  Bull  (1994),  language  awareness  is  achieved  by  allowing  the  student  access  to  all 
information  held  in  the  system.  The  information,  however,  is  not  used  to  drive  the  instructional  process.  A 
number  of  studies  have  also  shown  that  students  tend  to  not  take  advantage  of  the  option  to  access  additional 
information.  For  example,  Cobb  c&  Stevens  (1996)  found  that  in  their  reading  program  learners'  use  of  self- 
accessible  help  was  virtually  non-existent,  in  spite  of  their  previously  having  tried  it  in  a practice  session,  and 
also  having  doubled  their  reading  performance  as  compared  to  either  a no  help  or  dictionary  help  option  in  the 
practice  session. 

2.2.  Explicit  Student  Models 

In  developing  an  explicit  Student  Model  one  typically  starts  by  making  some  initial  assumptions  based  on 
pretests  or  stereotypical  postulations  about  the  learner.  For  example,  initially  every  student  could  be  assessed  as 
an  intermediate.  During  the  instructional  process,  the  Student  Model  adjusts  to  student’s  behaviour  moving  to  a 
novice  or  expert  profile,  as  appropriate.  This  technique  is  used  in  explicit  Student  Models  to  make  instructional 
decisions. 

Explicit  Student  Modeling  has  been  used  in  a number  of  ILTSs,  primarily  in  the  form  of  tracking. 
Tracking  can  be  as  simple  as  calculating  percentages  of  correct  answers  or  more  sophisticatedly,  identifying 
particular  errors  which  occurred  in  the  student’s  input.  The  information  is  then  used  to  alter  the  instructional 
process,  either  in  the  form  of  further  language  tasks  or  feedback. 

Explicit  Student  Modeling  is  found  in  the  system  The  Fawlty  Article  Tutor  (Kurup,  Greer  & McCalla, 
1992)  which  teaches  correct  article  use  in  English.  The  system  presents  the  student  with  scenarios  whereby  the 
student  must  select  the  correct  article  form  and  the  appropriate  rule.  The  tutor  keeps  an  error  count  and  selects 
the  scenarios  on  the  basis  of  the  performance  of  the  student;  thus  the  path  through  the  program  is  individualized 
by  altering  the  instructional  process  according  to  prior  performance  of  the  student. 

Bailin  (1988,  1990)  in  his  system  Verbcon/Diagnosis  also  employs  the  tracking  method.  Diagnosis 
provides  practice  in  using  English  verb  forms  in  written  texts.  All  verbs  are  presented  in  their  infinitival  form 
challenging  the  student  to  provide  the  appropriate  verb  form.  The  system  tracks  the  most  frequent  error 
occurrence  and  the  context  in  which  the  error  occurred.  The  information  is  used  to  provide  informative 
feedback  based  on  contrasting  correct  and  ungrammatical  uses  of  tenses.  In  addition.  Diagnosis  suggests 
exercises  to  help  with  the  remediation  process. 

In  the  following  section  we  describe  the  ESL  Tutor  and  discuss  the  modeling  technique  used. 


3.  The  ESL  Tutor 


The  goal  of  the  ILTS  we  have  developed  for  ESL  is  to  provide  meaningful  and  interactive  vocabulary  and 
grammar  practice  for  second  language  learners.  The  ESL  Tutor  analyzes  sentences  from  the  student  and  detects 
grammatical  and  other  errors.  The  feedback  modules  of  the  system  correlate  the  detailed  output  of  the  linguistic 
analysis  with  an  error-specific  feedback  message. 


Hello  Bill! 

Build  a short  sintence  using  all  of  ths  following  vords. 

Exercise  h of  5 


my  / grendperents  / horn*  / tomorrow  / errivo  (prosontjeontlnuous)  / , 


My  gr«ndp«r*ntf  Is  arriving  horn*  tomorrow, 

Chook 

The  yerb  IS  ia  '3rd  singulAr'  but  that  la  iiicorrect  here. 

Answor 

NoKt  » 

Figure  1:  System  Feedback  for  the  Intermediate  Learner 


In  the  ESL  Tutor^  feedback  is  individualized  through  an  adaptive,  explicit  Student  Model,  which 
monitors  a user’s  performance  over  time  across  different  grammatical  constructs.  This  record  of  strengths  and 
weaknesses  is  used  to  tailor  feedback  messages  to  learner  expertise  within  a framework  of  guided  discovery 
learning:  a beginner  student  will  receive  the  most  explicit  feedback  while  the  instructional  messages  for  the 
expert  will  merely  hint  at  the  error.  The  feedback  aimed  at  the  beginner  will  also  contain  less  technical 
terminology  than  that  for  the  intermediate  and  expert.  For  example.  Figure  (1)  shows  a feedback  message  for  an 
intermediate  student,  which  indicates  incorrect  subject-verb  agreement. 

In  contrast  to  Figure  (1),  the  feedback  message  for  the  beginner  learner  will  provide  less  linguistic 
terminology  and  state  that  The  verb  IS  is  not  correct  here.  For  the  advanced  learner,  the  feedback  will  provide 
less  of  a clue  and  simply  display  There  is  an  agreement  error  in  this  sentence. 

In  the  following  section  we  discuss  the  technique  employed  in  the  Student  Model  of  the  ESL  Tutor. 

3.1.  Hie  Student  Model  of  the  ESL  Tutor 

The  Student  Model  of  the  ESL  Tutor  dynamically  evolves  based  on  the  student's  performance.  The  information 
in  the  model  is  used  for  two  main  functions:  modulation  of  instructional  feedback  and  assessment  and 
remediation. 

The  Student  Model  keeps  track  of  an  individual  student’s  performance  on  a variety  of  grammar  skills; 
from  subject- verb  agreement  to  passives  to  count/mass  nouns.  A student  has  a score  for  each  grammar  skill. 
This  score  ranges  from  0 - n,  where  we  have  set  n to  30.  The  score  increases  when  the  student  provides 
evidence  of  a successful  use  of  that  grammar  skill,  and  decreases  when  the  student  provides  evidence  of  an 
unsuccessful  use  of  that  grammar  skill.  The  amount  by  which  a student’s  score  increases  or  decreases  can  vary 
depending  on  the  current  value  of  the  score.  Initially,  we  set  all  scores  to  an  intermediate  level. 

For  the  purposes  of  modulating  instructional  feedback,  we  identify  3 categories  of  scores.  Scores  from 
0-10  are  assigned  to  the  novice  category,  11-20  to  the  intermediate  category,  and  21-30  to  the  expert  category. 
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When  a student  makes  an  error  on  a particular  grammar  skill,  the  message  they  receive  depends  on  their  score 
for  that  skill.  If  they  are  ranked  as  novice,  they  will  receive  a more  informative  message  than  if  they  are  ranked 
as  an  expert.  Since  the  score  for  each  grammar  skill  is  independent  of  the  score  for  the  other  grammar  skills,  a 
student  may  be  expert  at  subject- verb  agreement,  but  novice  at  forming  the  passive  - and  receive  the  appropriate 
message. 

The  score  information  is  also  used  for  a variety  of  remediation  and  assessment  tasks.  By  comparing  the 
Student  Model  at  the  beginning  and  end  of  a session,  we  can  provide  a summary  of  the  mistakes  that  a student 
made  during  that  session.  In  our  current  system,  these  are  summarized  into  general  categories  such  as  "Verb 
Tenses",  "Pronouns",  etc.  These  groups  are  set  by  means  of  a parameter  file.  Similarly,  we  can  also  identify  the 
grammar  skills  where  the  student  was  correct  and  provide  a "positive"  of  what  the  student  did  right.  At  present 
we  show  a list  of  the  errors  at  the  end  of  each  exercise  set. 

Further,  one  can  also  examine  the  Student  Model  overall  and  identify  the  current  strengths  and 
weaknesses  of  the  student.  We  identify  the  strengths  of  a student  as  the  five  highest  scoring  grammar  skills  that 
have  a score  greater  than  1 5 (half  of  the  total  scale).  We  identify  the  weaknesses  of  a student  as  the  5 lowest 
scoring  grammar  skills  that  have  a score  less  than  15.  Students  can  access  this  information. 

Finally,  the  Student  Model  information  can  also  be  used  to  provide  exercises  to  the  student  which 
focus  on  their  areas  of  weaknesses.  Instead  of  repeating  the  same  exercise  which  the  student  made  the  mistake 
on,  the  ESL  Tutor  has  the  capacity  to  identify  examples  which  require  the  same  grammar  skill.  This  avoids  the 
problem  of  the  student  rote  learning  the  solution  to  a particular  example,  without  actually  learning  the  general 
solution.  We  have  not  yet  implemented  this  functionality  in  the  ESL  Tutor. 


4.  Evaluation 

The  Student  Model  of  the  ESL  Tutor  is  based  on  our  German  system  (Heift  & Nicholson,  2001)  which  has  been 
tested  extensively.  In  one  of  the  studies,  we  determined  the  extent  to  which  the  Student  Model  addresses  the 
need  for  an  individualized  language  learning  environment.  33  students  participated  in  the  study  and  a total  of 
1352  sentences  were  considered  for  analysis. 

When  analyzing  the  data  with  respect  to  individualized  instruction,  we  were  interested  in  the  types  of 
errors  that  occurred  during  practice  and  their  distribution  with  respect  to  the  three  learner  levels:  beginner, 
intermediate  and  advanced. 

The  error  break-down  in  Table  (1)  shows  that  students  were  most  often  at  the  intermediate  level,  which 
is  not  surprising  since  each  student  is  initially  placed  at  the  intermediate  level.  Nonetheless,  approximately  one 
third,  or  30%,  of  the  time,  students  either  required  more  elaborate  feedback  suited  to  the  beginner  learner,  or,  in 
the  case  of  the  advanced  learner,  less  detailed  feedback  was  sufficient  to  correct  the  errors.  Moreover,  and 
although  not  illustrated  in  Table  (1),  ten  students  or  30.3%  of  all  participants  received  remedial  exercises  for  at 
least  one  of  the  six  chapters. 


Beginner 

Intermediate 

Advanced 

Total 

% 

Direct  Objects  (gender,  number,  case) 

64 

226 

1 

291 

21.5% 

Subject-Verb  Agreement  (person,  number) 

27 

188 

63 

278 

20.6% 

Prepositional  Phrases:  Dative  (gender,  number,  case) 

48 

185 

1 

234 

17.3% 

Indirect  Objects  (gender,  number,  case) 

42 

97 

7 

146 

10.8% 

Subjects  (gender,  number,  case) 

3 

82 

43 

128 

9.5% 

Missing  Words 

17 

37 

12 

66 

4.9% 

Prepositional  Phrases:  Two-way  (gender,  number,  case) 

21 

47 

68 

5.0% 

Prepositional  Phrases:  Accusative  (gender,  number,  case) 

15 

39 

54 

4.0% 

Extra  Words 

11 

19 

11 

41 

3.0% 

Word  Order 

10 

16 

10 

36 

2.7% 

Auxiliaries  {to  havews.  to  be) 

1 

6 

7 

0.5% 

Verb  complements  (infinitive  vs.  past  participle) 

1 

2 

3 

0.2% 

260(19.2%) 

944  (69.8  %) 

148(11%) 

1352 

100% 

Table  1 : Break-down  of  Grammar  Errors 


7S3 


The  data  further  indicate  that  most  errors  occurred  with  direct  objects  (21.5%)  and  subject-verb 
agreement  (20.6%).  However,  these  were  the  most  frequent  constructions  contained  in  the  120  exercises  of  this 
study.  For  instance,  only  chapters  5 and  6 (40  exercises  in  total)  focus  on  the  present  perfect  and  modals.  These 
constructions  are  not  contained  in  any  of  the  previous  chapters,  thus  there  is  less  opportunity  for  errors  with 
these  grammar  topics  than,  for  example,  subject- verb  agreement. 

It  is  interesting,  however,  to  consider  the  number  of  grammar  errors  made  by  each  learner  level.  From 
a learning  perspective,  the  data  in  Table  (1)  indicate  three  distinct  groups: 

1.  those  grammar  topics  where  the  error  distribution  for  the  beginner  and  advanced  levels  is  fairly 
balanced  (missing  words,  extra  words,  word  order,  auxiliaries,  verb  complements), 

2.  those  grammar  points  where  students  are  far  more  often  at  the  beginner  than  the  advanced  level 
(direct  and  indirect  objects,  accusative,  dative,  two-way  prepositions),  and 

3.  those  grammatical  constructions  where  students  are  far  more  often  at  the  advanced  than  the 
beginner  level  (subject-verb  agreement,  subjects). 

The  data  of  our  Student  Model  also  allow  us  to  gain  insight  into  students'  performance  on  a particular 
grammar  skill  over  time.  For  example.  Figure  (2)  illustrates  the  performance  on  direct  objects  by  four  students 
who  were  randomly  selected  from  our  data  set.  The  x-axis  displays  the  five  chapters  that  contain  direct  object 
constructions  and  the  y-axis  shows  the  scores  which  correspond  to  the  three  learner  levels.  The  graphs  indicate 
that  one  of  the  students  stayed  at  the  intermediate  level  throughout  practice.  In  contrast,  the  remaining  three 
students  shifted  from  the  intermediate  to  the  advanced  level.  The  data  confirm  that  while  there  is  variation 
across  learners  each  student  also  changes  performance  levels  as  s/he  progresses  through  the  course. 


Fi^ire  (2):  Pciformukcc  om  Direct 

Objects 


The  data  support  the  need  for  an  individualized  system  which  makes  subtle  distinctions  between 
learners  and  error  types.  Our  Student  Model  has  a number  of  advantages.  It  takes  into  account  students’  past 
performance,  and  by  adjusting  the  score  value  to  be  incremented  or  decremented,  it  is  adaptable  to  a particular 
grammatical  constraint  in  an  exercise  or  the  pedagogy  of  a particular  instructor.  For  example,  a language 
instructor  might  rate  some  errors  more  salient  than  others  in  a given  exercise.  In  such  an  instance,  the 
increment/decrement  of  some  grammar  constraints  can  be  tuned  to  change  their  sensitivity. 

The  main  strength  of  our  Student  Model,  however,  is  that  a single  erroneous  message  will  not 
drastically  change  the  overall  assessment  of  the  student.  The  Student  Model  indicates  precisely  which 
grammatical  violations  occurred,  allowing  for  a fine-grained  assessment  of  student  competency.  In 
consequence,  a student  can  be  at  a different  level  for  each  given  grammar  constraint  reflecting  her  performance 
of  each  particular  grammatical  skill.  This  subtlety  of  evaluation  is  desirable  in  a language  teaching  environment 
because  as  the  student  progresses  through  a language  course  a single  measure  is  not  sufficient  to  capture  the 
knowledge  attained  and  to  distinguish  among  learners.  The  Student  Model  aids  in  directing  each  student  toward 
error-specific  and  individualized  remediation. 


5.  Conclusion 


In  this  paper  we  provided  examples  of  Student  Models  that  have  been  employed  in  CALL  over  the  past  decade. 
We  introduced  our  ESL  Tutor,  an  ILTS  that  provides  error-specific  and  individualized  feedback.  The  Student 
Model  of  the  ESL  Tutor  is  based  on  learner  performance  history  and  makes  system  decisions  accordingly. 

A study  in  which  we  evaluated  the  extent  to  which  our  Student  Model  addresses  the  need  for  an 
individualized  language  learning  environment  emphasizes  the  importance  of  an  adaptive  language  learning 
system  that  considers  user  diversity.  Approximately  one  third,  or  30%,  of  the  time,  students  either  required 
more  elaborate  feedback  suited  to  the  beginner  learner,  or,  in  the  case  of  the  advanced  learner,  less  detailed 
feedback  was  sufficient  to  correct  the  errors.  The  data  further  confirm  that  while  there  is  variation  across 
learners  individual  students  also  change  performance  levels  as  they  progress  through  a course. 

Our  study  also  provided  some  interesting  insights  into  the  error  typology  of  different  learner  levels. 
Due  to  the  constrained  environment  of  the  exercises  of  our  system  where  students  select  from  a given  pool  of 
vocabulary  and  grammatical  structures,  errors  in  omission,  insertion  and  word  order  were  less  frequent  than 
other  grammar  errors.  Fewer  errors  occurred  overall  and  thus  the  error  distribution  with  respect  to  beginner  and 
advanced  levels  was  fairly  balanced.  We  are  currently  establishing  a similar  error  typology  for  our  ESL  Tutor. 
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Abstract.  This  article  describes  the  New  Learning  Environments  in  the  Work  of  the  Teacher  (NET) 
devebpment  project  in  the  Province  of  Lapland  for  university-level  teaching  methods  NET  is 
organized  by  the  Centre  for  Media  Pedagogy  and  funded  by  the  European  Union’s  European  Social 
Fund  (ESF).  Its  objective  is  to  improve  and  support  the  use  of  information  and  communications 
technology  (ICT)  in  the  work  of  university-level  teachers  and  the  teaching  support  staff  The  article 
presents  the  content,  some  assessment  principles,  and  the  effectiveness  of  the  NET  study  programme 
(15  cr.)  developed  during  the  project.  The  study  programme  will  be  realized  twice  in  2(X)1  -2002,  The 
research  into  effectiveness  of  the  study  programme  will  examine  the  methodological  and  ontological 
commitments  of  empowerment-evaluation.  Self-direction  will  be  examined  later  in  relation  to 
meaningful  learning,  methods  of  guidance  and  to  the  more  extensive,  regional  effectiveness  of 
training. 


1.  Introduction 

New  Learning  Environments  in  the  Work  of  the  Teacher  (NET)  is  a development  project  for  university- 
level  teaching  methods  in  the  Province  of  Lapland,  It  is  organized  by  the  Centre  for  Media  Pedagogy  at  the 
University  of  Lapland  and  realized  through  funding  by  the  European  Union’s  European  Social  Fund  (see 
Ruokamo,  Syijakari  & Karppinen.)  The  project  is  the  result  of  cooperation  between  university-level  educational 
institutions  and  the  Provincial  Government  of  Lapland  and  it  is  being  realized  in  the  years  2000-2002.  The  goal 
of  the  project  is  to: 

1.  support  the  professional  development  of  teaching  and  the  teaching  support  staff  at 
the  university-level  using  client -oriented  multi-disciplinary  courses  by  focussing  on 
the  use  of  information  and  communicat  ion  technology , 

Z develop  the  content  and  methods  of  training  and  development  services  in  order  to 
serve  working  life,  and 

3.  stabilize  the  cooperation  of  the  institutions  of  higher  education  in  the  development 
of  the  area. 

The  15-credit  (NET)  study  programme,  which  corresponds  to  the  first  objective  of  the  project,  supports 
the  preparedness  of  university-level  teachers  and  teaching  support  staff  to  benefit  from  information  and 
communications  technology  in  their  work.  Its  purpose  is  also  to  increase  awareness  of  open  learning 
environments,  their  related  perspectives  on  learning  and  pedagogical  trends.  The  study  programme  is  organized 
twice  in  the  years  2001-2002  and  the  research  of  effectiveness  is  closely  related  to  both  courses.  In  this  article, 
empowerment -evaluation  is  taken  as  a perspective  of  the  research  of  effectiveness.  Its  fimctionality  as  a method 
of  assessing  effectiveness  will  be  examined  in  relation  to  both  the  course  students  and  the  objective  of  the 
project.  Empowerment -evaluation  s related  closely  to  the  concept  of  self’direction.  The  aim  is  to  consider  the 
development  of  the  participants’  self-direction  during  the  study  programme  and  later  to  examine  whether  self- 
direction  can  have  a wider  regional  effectiveness  and  how  assessment  by  itself  can  be  effective. 


2.  New  Learning  Environments  and  Sel^Direction 


Learning  environments  are  comprehensive  environments  of  operation  that  are  formed  from  many 
different  factors:  environment,  students,  teachers,  different  perspectives  of  learning  and  forms  of  action,  sources 
of  learning,  and  the  tools  and  objectives  in  using  them  (http://www.uta.fi/tvt/verkkotutor/oppvmp.htm1.  Thus,  a 
learning  environment  is  much  more  than  a plysical  space.  New  learning  environments  are  often  connected  to  the 
I use  of  developed  technology  in  teaching.  However,  new  technology  can  also  be  used  in  a traditional  learning 
environment.  Challenges  and  obligations,  which  in  themselves  are  not  bonded  to  new  technology  any  more  than 
I they  are  to  older  technology,  are  often  connected  to  a new  learning  environment.  An  attempt  is  made  in  new 
learning  environments  to  create  natural  forms  of  working  in  which  a student  can  direct  events  in  the  direction  of 
his  or  her  personal  objectives  and  assessment.  Often,  a virtual  learning  environment,  an  open  learning 
environment,  and  a network- based  learning  environment  are  used  as  a parallel  or  replaceable  concept  of  the  new 
learning  environments  (Manninen  & Pesonen  1997).  With  the  concept  of  an  open  learning  environment,  student 
centricity  and  flexibility  form  the  basis  of  the  learning  experience.  Openness  can  mean,  for  example, 
emphasizing  the  goal-orientation  of  the  individual  and  individual  self-direction.  (Kivimaki  2000.) 

Working  in  a network  forces  a student  to  be  active  and  to  make  choices,  in  other  words  to  be  self- 
directed.  A student  also  influences  how  he  or  she  works  in  the  environment,  at  what  speed,  and  at  what  depth  he 
or  she  wants  to  become  familiar  with  certain  contents  on  his  or  her  computer  (Kivimaki  2000).  Self-direction  and 
autonomy  is  not  only  a question  of  technical  control;  rather,  it  is  also  a question  of  the  student’s  internal 
expansion  of  knowledge.  Instead  of  self-direction.  Garland  (1994)  uses  the  concept  of  personal  control.  The 
personal  control  of  learning  requires  that  a student  aspires  toward  study  in  a critical  manner  and,  for  example, 
aims  when  studying  to  separate  essential  knowledge  from  the  immaterial  or  believable  theory  from  the 
uribelievable.  Often,  this  type  of  activity  docs  not  happen  automatically,  nor  do  the  skills  of  an  adult  student 
always  give  the  readiness  to  act  in  a self-directed  manner.  The  role  of  a tutor  becomes  emphasized  in  studies 
within  new  learning  environments.  Lehtinen  and  Jokinen  (1999)  describe  the  change  in  the  roles  between  a guide 
I and  a student  towards  that  of  self-direction  as  presented  in  Fiuure  1. 


Student 

Guide 

dependent 

authority 

develops  critically  in  order  to  assess  one’s  life  situation  and  to 
consider  one’s  learning  needs  / beating  opposition 

interested 

motivator 

sets  individual  goals  and  selects  the  suitable  means  to  conclude  them 
as  well  as  considers  learning  strategies 

committed 

aid 

develops  the  skills  of  group  work,  the  responsibility  for  learning  in  a 
group  and  as  individuals,  commonly  agreed  criteria  and  learning 
contracts 

self- 

directed 

delegator 

provides  space,  the  interaction  of  students  and  the  surrounding 
world,  e.g.  follow-up  study,  mentor  activities,  expert  and  supporter 

Figure  1.  The  roles  of  students  and  guides 


3.  Empowerment  in  Order  to  Strengthen  Self-Direction  and  Effectiveness 

The  effectiveness  of  training  can  be  studied  from  many  perspectives.  In  addition,  its  focus  and  the  role  of 
the  researcher  influence  the  formation  of  the  research  process.  These  dimensions  are  outlined  in  Figure  2. 
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Figure  2,  The  focus  of  evaluation  and  the  role  of  the  researcher  (Lindquist  1999,  see  also  Albaek  1996) 

The  methodological  and  ontological  commitments  of  assessment  research  are  defined  according  to  the 
focus  of  assessment,  the  role  of  the  researcher,  and  the  nature  of  the  knowledge.  The  point  of  emphasis  for 
assessment  can  be  either  in  the  result  or  in  the  process.  The  role  of  the  researcher  can  change  from  the  external 
observation  of  a process  to  that  of  an  internally  operating  therapist.  Moreover,  the  nature  of  knowledge  can 
change  from  one  objective  reality  to  the  relative  relativism  of  subjective  knowledge  and  to  the  pluralism  of 
knowledge.  In  the  development  project  New  Learning  Environments  in  the  Work  of  the  Teacher  for  university- 
level  teaching  methods,  project  planners  work  in  the  role  of  both  tutor  and  researcher.  Thus,  the  position  of  the 
researcher  suits  the  role  of  empowerment/confirmer.  An  attempt  is  made  through  process-formed  assessment  to 
develop  the  training  project  and  the  study  programme.  The  central  task  of  process  evaluation  is  how  well  the 
material  collected  during  the  process  is  used  in  guiding  the  project  and  in  specifying  its  objectives  (Seppanen- 
Jarvela  1999).  The  challenge  is  to  be  able  clearly  to  bring  forth  the  informative  material  obtained  through 
experience,  its  basic  observations  and  findings.  It  will  help  the  work  of  development,  learning,  and  reflectivity 
and  thus  improvement.  The  core  of  empowerment  is  participation  and  participating.  Conquest  includes  the  two 
related  sides  of  empowerment:  change  in  the  individual* s self-determination,  in  other  words  in  the  ego,  and 
participation  through  changes  taking  place  in  the  environment.  It  strengthens  the  functionality  of  the  subject. 
(Antikainen  1996,253-254.) 

The  meaning  of  the  concept  of  empowerment  in  relation  to  professional  adult  education  differs  slightly 
from  the  above.  A study  programme  should  provide  its  participants  with  such  tools,  that  in  their  work  (and  life) 
they  are  able  to  activate,  motivate,  and  solve  their  problems  by  themselves.  The  afore -mentioned  claim  is  based 
on  the  humanistic  concept  of  a person,  which  believes,  among  other  things,  in  the  following  strengths  and 
opportunities  for  individual  development. 

l.The  subject  principle:  a human  is  a conscious  being  that  can  affect  his  or  her 
thinking  and  action;  he  or  she  is  not  a passive  object  in  which  information  can  be 
accumulated. 

2 The  development  principle:  a human  is  not  naturally  a subject;  rather,  he  cr  she 
develops  gradually  as  an  increasingly  knowledgeable  actor  and  thinker. 

3.  The  action  principle:  development  takes  place  through  human  action. 

4.  The  societal  principle:  development  always  takes  place  in  some  societal  and 
cultural  environment.  (Toiskallio  1994,  20-22.) 

Therefore,  empowerment-evaluation  offers  the  opportunity  for  the  work  of  development;  in  addition,  it 
notes  the  relationship  between  the  individual  and  the  environment  through  the  activity  of  an  individual* s 
development  of  activation  and  self-direction  (see  Heikkila  2001). 

4.  New  Learning  Environments  in  the  Work  of  the  Teacher-  the  Realization  and  Content  of  Training 

The  NET  training  provides  the  preparedness  to  benefit  from  new  learning  environments  and  to  renew 
one*s  pedagogical  practices  and  skills  in  information  and  communications  technology  (ICT).  As  part  of  their 


practice  during  the  course,  the  participants  develop  network-based  training  modules,  wel> based  learning 
materials,  and  the  beneficial  tools  for  building  knowledge  in  the  learning  environment,  the  use  of  which  is 
assessed  during  the  course. 

The  NET  study  programme  is  intended  for  university-level  teachers  and  teaching  support  staff  in  the 
Province  of  Lapland.  It  has  130  participants  from  the  University  of  Lapland,  the  Rovaniemi  and  Kemi-Tomio 
Polytechnics,  and  class  teacher  students  specializing  in  media  education. 

Training  includes  contact  teaching  in  Rovaniemi,  video- conferencing  to  distant  localities,  and 
collaborative  working.  It  takes  advantage  of  environments  based  on  information  networks.  In  addition,  support 
training  is  organised  to  further  the  development  and  realization  of  network  courses  and  web-based  learning 
materials. 

The  training  offered  in  the  New  Learning  Environments  in  the  Work  of  the  Teacher  (NET)  project 
provides  a 15 -credit  study  programme  organized  twice  during  2001  and  2002.  The  NET  1 course  was  and  is 
being  realized  from  January  2001  - February  2002  and  NET  11  from  November  2001  - December  2002. 
[http://www.urova.fi/home/mpk/lCoulutus.htm1  The  training  includes  the  following  courses: 

1.  Orientation  to  Studies, 

2 Learning  Environments  and  Network  Pedagogy  (3  cr.), 

3.  The  Planning  and  Realization  of  Network-Based  Studies  (4  cr.), 

4.  Copyright,  Data  Privacy,  and  Data  Protection  (1  cr.), 

5.  Information  and  Communication  Technology  (ICT)  in  Learning  (1  cr.), 

6.  Interaction  in  a Network  (3  cr.),  and 

7.  Assessing  the  Learning  Process  and  Learning  Environments  (3  cr.). 

The  study  programmeis  in  accordance  with  the  study  requirements  at  the  Faculty  of  Education  at  the  University 
of  Lapland. 

The  first  NET  course  began  in  January  2001  with  the  participation  of  74  university- level  teachers  and 
teaching  siqjport  staff  from  the  University  of  Lapland,  the  Rovaniemi  Polytechnic,  and  the  Kemi-Tomio 
Polytechnic.  This  first  course  group  acted  as  the  focus  for  the  experiment  and  development  of  the  study 
programme.  Its  contents  were  mainly  considered  satisfxtory;  however,  some  parts  were  seen  as  problematic.  ' 
The  lack  of  clear  and  objective-oriented  tutoring  was  one  of  the  more  significant  problems.  Insufficient  attention 
was  paid  to  the  relationship  of  interaction  among  the  student  group  and  to  benefiting  from  group  strength  and 
shared  expertise.  These  factors  can  in  turn  affect  the  preparation  of  the  final  work  on  time  and  the  realization  of 
the  objectives  of  the  course.  The  NET  1 course  came  to  an  end  in  February  2002. 

The  second  NET  course  began  in  November  2001.  The  first  experience  gained  in  the  course  was  that  at 
the  beginning,  insufficient  attention  was  paid  to  group  strength  (face-to-face  presentation),  to  taking  into 
possession  and  working  in  the  learning  environment,  and  to  rules.  Moreover,  time  was  used  separately  for 
guiding  personal  curricula  and  the  principles  of  goal-oriented  learning.  The  Sonera  experience  Training 
environment  was  used  as  the  network  environment  for  NET  1.  Discendum  Optima  was  chosen  as  the  network- 
based  environment  for  NET  IL  The  Discendum  Optima  environment  enables  better  non-hierarchic  and 
interactive  activity  in  a network  than  the  Sonera  experience  Training  environment  does. 
[http://www.discendum.com/]  Of  the  training  contents,  thematic  discussion  channels  are  built  into  the 
environment,  where  the  aim  is  to  promote  learning,  shared  expertise  and  experience.  The  concept  of  shared 
expertise  refers  to  a process  during  which  several  people  share  intelligent  resources  related  to  knowledge,  plans, 
and  objectives  in  order  to  achieve  something  that  an  individual  person  is  unable  to  realize  on  his  or  her  own 
(Brown  et  al.  1993;  Oatley  1990).  The  students  in  NET  11  will  graduate  by  the  end  of  2002,  when  information 
about  the  achievement  of  their  goals  will  have  been  obtained. 


5.  Conclusions 

This  article  has  examined  the  New  Learning  Environments  in  the  Work  of  the  Teacher  {NET) 
development  project  in  the  Province  of  Lapland  for  university-level  teaching  methods  - its  contents,  objectives, 
and  some  assessment  principles.  The  goal  has  been  to  present  a model  based  on  empowerment -evaluation,  which 
is  one  means  of  performing  effective  and  planned  assessment.  Post-assessment  is  insufficient;  rather,  pre- 
assessment and  guided  assessment  are  also  needed.  As  a central  concept,  self-direction  rises  to  the  fore  in 
empowerment- evaluation,  which  increases  the  meaningfiilness  of  learning  and  the  effectiveness  of  training.  Self- 
direction  is  often  related  to  network-based  education,  to  new  and  open  learning  environments,  in  that  it  is  in 
these  that  study  demands  some  form  of  self  direction  from  a student.  Its  internalization  is  not,  however,  self- 
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evident  for  the  adult  student;  rather,  its  development  requires  the  development  of  guiding  processes.  On  the  one 
hand,  the  learning  occurring  in  new  learning  environments  in  itself  can  encourage  self-direction;  on  the  other 
hand,  simply  being  along  on  a network  course  does  not  automatically  bring  about  self-direction.  This  research 
will  examine  self-direction  in  relation  to  methods  of  guidance  and  in  relation  to  achieving  the  objectives  of  the 
project  by  answering  the  following  questions:  1)  Does  NET  strengthen  and  confirm  the  self-direction  of  the 
participants  in  the  study  programme?  2)  What  were  the  significance  and  procedures  of  guidance  in  the  process? 
3)  Can  sdf-direction  have  a more  extensive,  regional  influence? 

Based  on  the  assessment  of  the  NET  course,  an  advanced  study  programme  will  be  developed  that, 
instead  of  being  based  on  teaching  methods  related  to  new  learning  environments,  will  provide  more  systematic 
information  on  teaching  methods  related  to  new  learning  environments  and  to  the  opportunities  offered  by  digital 
media  in  open  learning  environments.  In  addition,  the  pedagogical  models  and  principles  occurring  within 
network-based  education  are  developed  and  assessed  in  the  national  HelLa  project,  which  is  a cooperative 
project  of  the  Universities  of  Lapland  and  Helsinki  (see  Ruokamo,  Telia,  Vahtivuori,  Tuovinen  & Tissari),  We 
are  able  to  utilize  the  principles  of  assessment  developed  by  the  HelLa  project  in  analysing  results  of  this  study 
programme,  and  will  present  results  of  effectiveness  research  when  both  NET  study  program  mes  have  ended. 
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Abstract:  In  order  to  take  a part  in  a particular  Web-based  training  session  learners  need  to 
work  with  a number  of  tools  reflecting  a particular  training  strategy.  By  operating  these  tools 
learners  access  different  training  objects  contained  in  that  training  session.  Such  training 
objects  contain  relevant  information  for  their  current  training  task.  Nowadays,  modern  WBT 
systems  support  dozens  of  different,  sometimes  rather  complex  tools  and  provide  access  to 
thousands  of  training  objects.  In  this  paper  we  analyze  the  problems  of  user  interface  that  can 
become  a rather  complex  one  in  training  sessions  conducted  in  such  systems.  In  order  to 
overcome  such  problems  we  provide  a simple  general  user  interface  solution  for  these  WBT 
systems.  Further  we  provide  an  evaluation  of  the  responses  from  users  we  gathered  in 
applying  that  solution  in  a number  of  Web-based  training  sessions.  This  evaluation  showed  us 
that  there  is  much  more  potential  in  our  solution  than  we  believed  at  the  first  resulting  in  the 
evolution  of  our  user  interface  solution  to  a simple  knowledge  delivery  tool.  Such  tool  might 
be  used  to  conduct  training  sessions  that  contain  always  the  most  up-to-date  and  the  most 
relevant  training  objects  residing  in  the  system. 


1.  Introduction 

Technically,  WBT  systems  (Gaines  et  al.,  1997)  consist  of  a large  repository  of  training  objects  and  a 
number  of  tools  to  manipulate  these  training  objects.  Usually,  these  tools  support  operations  such  as 
creating,  deleting,  accessing,  or  updating  of  training  objects  (Dietinger  et  al.,  1997). 

According  to  the  variety  of  types  of  training  objects  supported  by  a particular  WBT  system,  or  in  other 
words  according  to  the  number  of  tools  and  functionality  that  such  tools  support  we  may  distinguish 
between  (Helic  et  al.,  2000): 

Standard  WBT  systems 
Advanced  WBT  systems. 

Standard  WBT  systems  support  only  basic  training  objects,  such  as  learning  unit,  learning  course  or 
discussion  forum.  Tools  provided  by  such  systems  include  authoring  tools,  publishing  tools,  navigation 
tools,  simple  search  tools,  etc.  Usually,  such  tools  are  rather  primitive  with  a simple  basic  functionality  and 
a comprehensive  user  interface.  On  the  other  hand,  advanced  WBT  systems  support  a wide  range  of 
different  training  objects.  For  instance,  these  training  objects  include  learning  goals,  mentoring  sessions, 
brainstorming  sessions,  knowledge  cards,  knowledge  profiles,  etc.  Structurally  these  objects  may  be  rather 
complex.  For  instance,  a knowledge  card  is  an  instance  of  so-called  semantic  network,  which  is  a rather 
complex  structure.  Obviously,  tools  provided  by  advanced  WBT  systems  are  by  far,  more  numerous. 


provide  a richer  functionality  and  more  complex  than  tools  provided  by  standard  WBT  systems. 
Consequently,  these  tools  have  a rather  complex  user  interface. 

From  the  users'  point  of  view  WBT  systems  offer  possibilities  to  take  part  in  different  training 
sessions.  Usually,  a training  session  is  considered  to  consist  of  a so-called  training  strategy  and  a number  of 
training  objects  (Helic  et  al.,  2001a).  Training  strategy  reflects  a particular  way  of  working  through  the 
subset  of  training  objects  to  achieve  a particular  training  goal  (Helic  et  al.,  2001b).  Actually,  a training 
strategy  is  a collection  of  tools  combined  in  a certain  way  and  reflecting  a particular  training  methodology. 
For  instance,  consider  the  training  strategy  known  as  Web-based  learning.  This  training  strategy  is  a 
collection  of  tools  that  provide  functionality  needed  to  access  and  navigate  through  different  learning  units 
and  learning  courses  (Andrews  et  al.,  1995).  Web-based  learning  might  be  considered  as  the  basic  training 
strategy  supported  by  all  WBT  systems.  Another  example  of  a training  strategy  would  be  so  called  Web- 
based  tutoring.  This  strategy  is  a rather  advanced  training  strategy  supported  by  an  advanced  WBT  system 
called  WBT-Master.  Web-based  tutoring  prescribes  working  with  a set  of  special  WBT-Master  tools.  These 
tools  allow  users  to  navigate  through  a sequence  of  so-called  learning  actions  and  training  objects 
associated  to  such  learning  actions  in  order  to  achieve  a particular  learning  goal. 

Obviously,  advanced  training  strategies  supported  by  means  of  advanced  WBT  systems  might  consist 
of  a rather  large  number  of  tools  (Helic  et  al.,  2001a;  Helic  et  al.,  2001b).  Considering  that  such  tools  have 
rather  complex  user  interface  themselves  any  collection  of  these  tools  would  have  a rather  complex  user 
interface  as  well.  Thus,  users  of  advanced  WBT  systems  might  be  confronted  with  a serious  problem  of  a 
complex,  inscrutable  user  interface.  Also  taking  into  account  that  more  advanced  training  strategies  might 
include  training  objects  of  many  different  t^^pes,  thus  resulting  in  an  increased  number  of  different  tools 
needed  to  implement  such  a training  strategy,  the  user  interface  problem  is  becoming  even  larger. 
Evidently,  if  we  want  to  be  able  to  conduct  training  session  by  means  of  advanced  training  strategies  we 
need  to  solve  the  increasing  user  interface  problem  on  a larger  scale.  Hence,  not  only  do  we  need  to 
simplify  the  user  interface  of  a particular  tool  but  we  need  also  a more  general  user  interface  solution  that  is 
able  to  reflect  all  peculiarities  of  any  training  strategy  and  provide  a single  access  point  to  each  of  tools 
needed  to  implement  a particular  training  strategy.  Apparently,  such  solution  needs  to  be  highly 
configurable,  customizable  user  interface  solution. 

WBT-Master  provides  such  general  user  interface  solution  in  the  form  of  so  called  Personal  Desktop. 


2.  Personal  Desktop 

Conceptually,  a personal  desktop  is  just  a set  of  folders  containing  references  to  designated  training  objects 
and  WBT-Master  tools.  Tools  from  a particular  folder  reflect  a particular  training  strategy,  whereas  the 
training  objects  from  that  folder  contain  information  relevant  for  achieving  a certain  training  goal. 
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Figure  1 : Content  of  a personal  desktop  folder 


The  main  idea  behind  the  concept  of  a personal  desktop  might  be  seen  as  the  following.  An 
advanced  user,  say  an  author  or  a tutor  manages  a particular  training  session  with  a group  of  learners. 


He/she  creates  a personal  desktop  folder  containing  different  WBT-Master  tools  and  a number  of  training 
objects  residing  on  the  server.  Once  when  the  personal  desktop  folder  has  been  created  he  may  share  that 
folder  with  the  learners'  group,  thus  allowing  them  to  participate  in  his/her  training  session.  Obviously,  by 
accessing  the  created  personal  desktop  folder  learners  have  access  to  designated  training  objects  by  means 
of  the  implemented  strategy.  However,  learners  are  not  any  more  confronted  with  arbitrary  tools  or  training 
objects  provided  by  the  system.  Rather  they  are  supposed  to  access  only  few  relevant  training  objects  and 
that  by  means  of  tools  reflecting  the  desired  training  strategy. 
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Figure  2:  working  with  training  objects  from  a personal  desktop  folder 


This  concept  greatly  facilitates  the  fact  that  a typical  user  works  with  just  a few  tools  and  training 
objects  offered  by  the  system.  In  other  words,  this  special  customization  mechanism  is  used  to  adjust  the 
rich  system  functionality  to  personal  needs  of  a particular  learner  or  a group  of  learners.  Hence,  a tutor  or 
an  author  may  decide  on  preferable  user  interface  for  such  group  of  learners  and  on  training  objects,  which 
are  needed  to  accomplish  a particular  training  task.  Another  important  aspect  of  the  personal  desktop 
concept  is  collaboration  facilities,  which  are  provided  by  so-called  shared  folders  and  internal  messaging 
system.  For  example,  a learner  group  may  share  a certain  folder  to  put  all  contributions  of  the  group 
members  into  it.  In  this  way,  the  contributions  may  be  easily  accessed  by  group  members  and  discussed  by 
attaching  messages  to  such  contributions. 


Figure  3:  Exchanging  messages  with  a personal  desktop  folder 
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3.  Working  with  Personal  Desktop 

Generally,  first  experiments  with  personal  desktop  show  a great  acceptance  of  this  concept  by  users. 

Authors  very  much  appreciated  a possibility  to  define  all  requisites  of  a particular  training  session, 
i.e.,  all  tools  that  are  needed  to  implement  a particular  training  strategy,  as  well  as  all  relevant  training 
objects  as  members  of  a folder.  In  this  way  they  were  not  supposed  to  know  all  peculiarities  of  the  system 
in  order  to  combine  the  system  tools  into  a coherent  training  strategy.  For  instance,  they  were  not  supposed 
to  know  how  to  attach  a discussion  forum  to  a learning  course  to  provide  learners  with  a board  to  discuss 
topics  from  that  learning  course,  but  rather  they  just  put  that  learning  course  and  a discussion  forum  into  a 
personal  desktop  folder  and  the  above-mentioned  relationship  between  these  objects  was  automatically 
established  by  the  system. 

On  the  other  hand,  tutors  liked  the  messaging  functionality  of  a personal  desktop  folder  the  most. 
This  functionality  allowed  them  to  keep  the  communication  between  members  of  a particular  learner  group 
within  the  scope  of  a particular  training  session.  Also,  possibility  to  reuse  (for  instance,  in  the  form  of 
FAQ)  the  results  of  such  communication  was  quite  well  accepted  by  tutors. 

Finally,  learners  appreciated  the  simplicity  of  user  interface  as  offered  by  such  training  sessions 
very  much.  Not  any  more  did  they  need  to  find  their  way  through  dozens  of  system  tools  and  large  number 
of  training  objects.  Rather  they  just  worked  with  a few  tools  and  relevant  training  objects  combined  into  a 
simple  navigable  list. 

However,  there  exist  some  disadvantages  of  this  concept  as  well.  The  most  important  shortcoming 
of  such  an  approach  can  be  stated  as  follows.  A particular  training  strategy,  i.e.,  a particular  collection  of 
tools  that  implements  that  training  strategy  might  be  seen  as  a rather  static  entity.  That  means  that  a 
particular  training  strategy  is  likely  not  to  change  in  the  course  of  the  time.  For  instance,  consider  the 
above-mentioned  Web-based  learning  training  strategy.  This  strategy  consists  always  from  say  a number  of 
learning  courses  and  a discussion  forum  attached  to  these  courses.  What  really  changes  in  training  sessions 
is  training  objects.  Actually,  training  objects  might  be  seen  as  a very  dynamic  entity.  Not  only  that  in 
different  training  sessions  training  objects  are  completely  different  but  also  in  one  and  the  same  training 
session  training  objects  are  likely  to  change.  For  instance,  some  training  objects  might  become  obsolete, 
newly  updated  versions  of  training  objects  might  be  created,  etc. 

These  results  led  us  to  the  conclusion  that  we  need  to  extend  the  concept  of  personal  desktop  and 
redesign  it  in  a way  that  it  becomes  robust  to  such  changes.  Thus,  we  decided  to  incorporate  a mechanism 
that  would  be  able  to  automatically  select  training  objects  that  are  the  most  relevant  and  the  most  up-to-date 
training  objects.  Only  such  training  objects  should  become  members  of  a particular  personal  desktop  folder. 
Obviously,  such  mechanism  must  be  able  to  "reason"  about  training  objects  and  decide  whether  a particular 
training  object  is  relevant  to  a particular  training  session,  i.e.,  does  it  match  certain  criteria  posed  by  that 
training  session. 


4.  Personal  Desktop  as  knowledge  delivery  tool 

WBT-Master  supports  semantic  data  structures,  which  can  be  used  to  create  different  semantic  overviews 
of  training  objects  residing  in  the  system  (Helic  et  al.,  2001a).  These  structures  include  so-called 
knowledge  cards  and  knowledge  domains. 

A knowledge  card  is  a description  of  particular  concept  (i.e.  semantic  entity).  For  example,  a 
semantic  entity  "Database  technology"  may  be  seen  as  a knowledge  card.  Practically  speaking,  each 
knowledge  card  may  provide  access  to  a number  of  associated  training  objects.  For  example,  a course  on 
"Relational  Data  Model"  may  be  associated  with  the  knowledge  card  "Relational  Data  Model",  some  other 
training  objects  may  be  associated  with  the  same  knowledge  card.  Knowledge  cards  may  be  interrelated 
into  a semantic  network  using  different  types  of  relationships:  "is  a part  of,  "is  a kind  of,  "synonym  for", 
etc.  For  example,  the  knowledge  card  "Relational  Data  Model"  may  be  related  as  "is  a part  of  to  the 
knowledge  card  "Database  Technology".  The  knowledge  card  "World  Wide  Web"  may  be  related  as  "is  a 
kind  of  to  the  knowledge  card  "Hypermedia  Systems".  The  knowledge  card  "Web  Base  Training"  may  be 
related  as  "is  a synonym  for"  to  the  knowledge  card  "Computer  Supported  Collaborative  Learning". 

The  infer  mechanism  essentially  utilizes  the  other  important  property  of  the  semantic  network  - a 
possibility  to  infer  training  objects  using  semantic  relationships.  Whenever  users  access  a knowledge  card. 


the  system  automatically  infers  all  training  objects,  which  are  associated  with  this  particular  knowledge 
card  and  with  knowledge  cards  related  to  this  one.  This  mechanism  greatly  facilitates  the  initial  access  to 
the  most  relevant  training  objects. 

On  the  other  hand,  a knowledge  domain  is  a collection  of  training  objects,  which  are  structured 
using  a predefined  template  called  the  knowledge  domain  schema.  A knowledge  domain  schema  may  be 
seen  as  a definition  of  semantic  categories  and  all  possible  semantic  relationships  between  them.  Thus,  a 
knowledge  domain  contains  training  objects,  which  might  be  seen  as  instances  of  a particular  semantic 
category  interrelated  by  means  of  semantic  relationships  with  training  objects  that  are  instances  of  related 
semantic  categories.  Basically,  we  may  use  this  mechanism  to  express  facts  such  as:  the  document  (an 
instance  of  say  "Module”  category)  is  related  to  the  document  ”A”  (an  instance  of  say  "Author"  category) 
by  means  of  the  "Author  Modules"  semantic  relationship.  Similarly,  the  document  "C"  is  related  to  the 
document  "B"  (an  instance  of  say  "Project")  by  the  means  of  the  "Project  Modules"  semantic  relationship, 
etc. 

Now,  users  may  browse  and  search  knowledge  domains  by  means  of  the  terms  defined  by  the 
knowledge  domain  schema.  For  instance,  users  accessing  an  instance  of  the  "Author"  category 
automatically  get  a link  to  an  index  of  all  instances  of  related  categories,  i.e.,  a link  o all  "Author  Modules" 
appears  on  the  screen.  Similarly,  knowledge  domain  might  be  used  to  execute  semantic  queries,  i.e.,  to 
search  for  all  "Modules"  that  are  related  to  a certain  "Project"  training  object,  etc. 

The  extension  of  the  personal  desktop  concept  treats  knowledge  cards  and  knowledge  domains  as 
training  objects  that  might  be  added  to  a personal  desktop  folder. 
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Figure  4:  Adding  a knowledge  card  to  a personal  desktop  folder 


By  adding  a knowledge  card  as  a training  object  in  a personal  desktop  folder  we  achieve  the 
following.  Whenever  users  access  a knowledge  card  from  a personal  desktop  folder  the  system 
automatically  infers  all  related  and  to  that  particular  concept  relevant  training  objects. 
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Figure  5:  Accessing  a knowledge  card  from  a personal  desktop  folder 
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If  a new  training  object  is  associated  with  this  knowledge  card,  the  system  automatically  captures 
that  change  by  inferring  the  newly  added  training  object  the  next  time  we  access  the  personal  desktop 
folder.  Thus,  this  mechanism  is  a rather  robust  one  to  changes  in  the  underlying  repository  of  training 
objects.  On  the  other  hand,  by  adding  a knowledge  domain  to  a personal  desktop  folder  users  are  provided 
with  possibility  of  browsing  semantic  overviews  of  training  objects  residing  in  the  system.  Again,  if  a new 
training  object  is  added  to  such  an  overview,  the  system  automatically  updates  the  overview  the  next  time 
when  we  access  it.  Thus,  changes  in  the  system's  repository  of  training  objects  are  captured  again.  Hence, 
by  incorporating  these  two  simple  knowledge-processing  mechanisms  into  the  concept  of  personal  desktop 
we  were  able  to  enhance  personal  desktop  from  a general  user  interface  solution  to  a simple  knowledge 
delivery  tool. 


5.  Conclusion 

Generally,  first  experiments  with  the  Personal  Desktop  system  demonstrate  a rather  good  functionality  and 
acceptance  by  users.  Learners  like  the  situation  where  they  are  not  confronted  with  too  much  system  tools 
or  irrelevant  training  objects  but  rather  with  just  few  important  tools  and  training  objects.  On  the  other 
hand,  tutors  and  authors  very  much  appreciate  the  fact  that  they  were  able  to  completely  control  their 
training  sessions  and  especially  corresponding  training  strategy.  They  can  be  sure  that  their  learners  are 
confronted  with  only  those  tools  that  are  the  essential  part  of  the  implemented  training  strategy.  Moreover, 
the  extension  of  the  concept  of  a simple  user  interface  tool  to  a knowledge  delivery  tool  leads  to  the 
conclusion  that  learners  are  not  only  confronted  with  the  relevant  system  tools  but  also  with  the  most 
relevant  and  up-to-date  training  objects  that  are  needed  for  a successful  training  sessions. 
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Abstract:  The  study  examines  a dominant  aspect  of  youth  culture:  playing  video 
computer  games.  The  study  is  interested  in  the  recreational  video  game  as  a means  of 
informal  cognitive  education.  Centered  within  an  information  processing  theory  and 
introspection  methodological  framework,  the  study  investigates  via  stimulated  recall 
methods  the  thinking  skills  and  strategies  used  by  two  teenagers  when  playing  an 
action-adventure  video  computer  game.  The  players  used  a wide  range  of  cognitive 
processes  that  are  valued  in  schools  such  as:  metacognition,  predction,  justification, 
inductive  and  deductive  reasoning,  strategy  planning,  and  parallel  processing. 


Surveys  (eg.,  Australian  Bureau  Statistics,  2000;  DigiPlay.org.uk,  2001;  Hall,  2000)  have  replica 
findings  revealing  that,  although  video  games  are  predominantly  a male  youth  pastime  (approx,  80%  boys  9-14 
years),  female  youth  are  increasingly  playing  (approx.  60%  girls  11-14  years);  both  play  daily  for  periods 
ranging  from  30  minutes  to  five  and  a half  hours.  Recreational  video  games  are  one  of  the  most  dynamic 
influential  forces  shaping  youth  culture. 

There  is  a remarkably  limited  research  literature  concerning  the  cognitive  effects  of  video  game  playing 
(eg.,  the  special  edition.  Journal  of  Applied  Developmental  Psychology^  15,  1994;  Camioni,  Ercolani, 
Perrucchini,  & Greenfield,  1990;  Pillay,  Brownlee,  & Wilss,  1999).  This  is  perhaps  because  recreational  video 
games  are  popularly  seen  as  trivial  without  educational  worth  and  something  from  which  educators  must 
rescue  children  and  distance  themselves.  Yet  cognitive  processes  are  broader  than  those  taught  and  tested  in 
school.  As  part  of  our  informal  education,  playing  games  has  been  identified  as  an  important  facilitator  of 
cognitive  development.  This  study  is  interested  in  the  recreational  video  game  as  a means  of  informal  cognitive 
education.  The  exploration  of  the  cognitive  processes  utilised  by  teenagers  while  developing  video  game 
literacy  will  be  of  theoretical  and  practical  interest  to  teachers,  researchers,  and  the  interactive  multimedia 
industry.  Overall,  the  project  aimed  to  understand  one  key  question:  Do  recreational  video  games,  a dominant 
aspect  of  teenage  culture,  provide  a context  that  encourage  valued  cognitive  processes?  To  this  end,  the  study 
investigated  (identified,  categorised,  and  analysed)  the  thinking  skills,  processes,  and  strategies  utilised  by 
young  teenagers  when  playing  recreational  (as  opposed  to  school-based  educational)  video  games. 

This  was  an  empirical  qualitative  study  based  in  an  information  processing  theory,  mediating  processes 
paradigm,  and  introspection  methodological  framework.  This  paper  reports  data  from  two  13  year  old 
volunteers,  one  male  and  one  female  (self-identified  as  Tony  and  Eyore,  respectively),  who  had  been  playing 
for  over  two  years.  Final  Fantasy  IX,  an  adventure/action  game,  was  chosen  by  the  students  and  approved  by 
their  parents  as  it  contains  some  low  level  violence  but  no  “blood  and  gore”.  The  data  were  collected  through 
videotaped  individual  stimulated  recall  interviews  at  the  beginning  of  the  game  and  up  to  the  same  screen  in 
the  game.  The  stimulated  recall  technique  is  designed  to  probe  the  players’  reasons  behind  their  actions  and 
interpretations  of  the  results  of  the  action  as  well  as  to  reveal  the  range  and  types  of  thinking  skills,  strategies, 
and  processes  utilised  within  the  context  of  their  use.  Hence,  these  interviews  occurred  when  playing  the  game 
which,  together,  took  approximately  one  hour.  Participants  were  not  allowed  to  read  the  instruction  help 
booklet  before  commencing  the  game. 


Thinking  skills 

Eyore 

Tony 

Total 

Strategies 

Eyore 

Tony 

Total 

Predicting 

20 

30 

50 

Trial  and  error 

9 

7 

16 

Evaluating 

14 

24 

38 

Equipping 

5 

7 

12 

Justifying 

14 

8 

22 

Exploring 

4 

5 

9 

Affecting 

5 

14 

19 

Induction 

4 

4 

8 

Comparing 

17 

1 

18 

Deduction 

3 

5 

8 

Metacognising 

7 

10 

17 

Talking  to  characters 

4 

3 

7 

737 


Confirming 

4 

10 

14 

Double  checking 

2 

2 

4 

Generating 

4 

9 

13 

Putting  values  on  hold 

0 

3 

3 

Reflecting 

6 

7 

13 

Attacking 

1 

1 

2 

Judging 

1 

12 

13 

Using  potions 

2 

0 

2 

Translating 

7 

5 

12 

Watching  movies 

1 

1 

2 

Linking 

8 

4 

12 

Naming  characters 

2 no 

1 yes  1 no 

1 yes  2 no 

Anticipating 

1 

6 

7 

Categorising 

2 

5 

7 

Reading  instructions 

usually 

no 

Analysing 

0 

3 

3 

Imaging 

0 

2 

2 

Applying 

0 

1 

1 

Recalling 

0 

1 

1 

Total 

118 

147 

265 

37 

40 

77 

Table  1:  Thinking  Skills  and  Strategies  when  Playing  Video  Games 

Eighteen  types  of  thinking  skills  were  utilised  during  the  students’  first  encounter  with  the  game.  Higher 
order  thinking  skills  were  utilised  by  both  students,  for  example:  predicting  what  may  happen  and 
hypothesising  “if  ...  then  ...”  consequences  of  actions  and  events;  evaluating  the  game,  graphics,  characters, 
and  their  own  gameplay;  justifying,  rationalising,  and  explaining  their  actions  and  what  was  happening  in  the 
game;  and  metacognising  their  awareness  of  what  they  knew  and  did  not  understand  and  suggesting  ways  to 
troubleshoot  their  lack  of  comprehension.  Some  types  of  thinking  skills  were  unevenly  engaged  in  by  the 
students:  comparing  this  game  with  other  games;  confirming  the  accuracy  of  their  strategies,  predictions,  and 
ideas;  judging,  particularly  issues  of  morality  in  the  game  and  expressing  thoughts  about  their  affective  feelings 
such  as  frustration  (more  so  Eyore)  and  empathy  for  what  the  characters  were  experiencing  (more  so  Tony). 

Inductive  and  deductive  reasoning  were  utilised  a pleasing  number  of  times.  However,  trial  and  error,  a 
legitimate  problem  solving  approach  though  not  the  most  efficient  (Martinez,  1998),  was  the  major  strategy 
because  they  were  “learning  how  this  game  worked”  (Tony’s  Interview).  Nor  was  it  surprising  that  equipping 
their  characters  for  fighting  came  a close  second  during  the  short  session. 

In  terms  of  overall  processes,  the  players  had  to  work  out  how  the  symbols,  icons,  images,  and  control 
buttons  acted  individually  and  in  unison.  They  had  to  attend  simultaneously  and  selectively  to  a number  of 
different  pieces  and  types  of  information  displayed  on  various  parts  of  one  screen  and  from  one  screen  to  the 
next;  that  is,  they  had  to  further  develop  their  skills  of  parallel  processing  (Greenfield,  et  al.,  1994).  They  had 
to  figure  out  the  rules  of  the  game.  Indeed,  what  the  data  shows  is  that  they  were  engaged  in  a process  of 
transforming  and  manipulating  their  mental  models  of  previous  games  and  gameplaying  with  this  game  and  its 
quite  different  gameplay. 

Even  with  such  a short  playing  time  as  this  research  episode,  the  data  reveal  that  playing  Final  Fantasy 
IX  provided  a valuable  informal  educative  experience.  They  engaged  in  cognitive  skills,  strategies,  and 
processes  that  are  valued  in  schools.  They  demonstrated  that  playing  this  game  involved  complex  cognitive 
processes. 
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Abstract;  The  purpose  of  this  brief  paper  is  to  invite  additional  project  partners  into  our 
research.  We  believe  that  an  international,  rather  than  country  specific,  survey  of  the 
various  issues  involved  in  playing  recreational  video/computer  games  will  help 
characterize  this  important  aspect  of  youth  culture  on  a global  basis.  It  will  also  help 
inform  parents,  teachers,  governments,  the  media,  and  video/computer  game  designers, 
including  educational  and  edutainment  instructional  computer  game  designers. 


Recreational  video/computer  games  are  a significant  cultural  artifact  of  youth  culture  and  are  a multi- 
billion dollar  industry.  “Video/computer  games”,  is  a generic  term  encompassing  four  types  of  games:  (a) 
video  games  that  use  a CD  or  cassette  console  with  attached  control  devices  connected  to  a TV  monitor;  (b) 
computer  games  that  are  played  on  the  computer  using  a CD-ROM  or  the  Internet;  (c)  “gameboys”  that  are 
small  hand-held  rectangular  devices  that  have  the  display  screen  in  the  top  half  and  the  control  buttons  in  the 
lower  half;  and  (d)  arcade  games  that  are  played  in  public  shops  or  arcades.  The  pervasiveness  of  these 
cultural  play  artifacts  with  their  “scintillating  pulsating  surfaces  that  are  dynamically  seductive”,  invite 
exploratory  play  wh  ile  shaping  the  users’  lives  (Turkle,  1997).  This  pervasiveness  has  led  to  a corresponding 
rise  in  the  emotional  debate  regarding  the  consequences  of  video/computer  game  playing. 

There  have  been  a number  of  surveys  establishing  children’s,  teenagers’,  and  young  adults’  gender 
usage  patterns,  game  preferences,  and/or  opinions  on  issues  of  violence  and  addiction  (e.g.  Australian  Bureau 
of  Statistics,  2000;  Centre  for  Research  on  Innovation  and  Competition,  2002;  Downes,  2000;  Drotner,  2001; 
Media  Analysis  Laboratory,  1998;  Griffiths,  1997;  & Walsh,  2001).  However,  these  surveys  and  studies  have 
limitations  that  our  international  multi-focussed  survey  would  help  overcome.  For  instance: 

■ previous  surveys,  except  for  Drotner’s  European  survey  based  on  1998  data,  have  been  country 
specific; 

■ they  sometimes  target  different  age  groups,  making  international  comparison  incomplete; 

■ they  do  not  target  all  types  of  recreational  video  games:  computer  CD-ROM  games,  Internet 
games,  arcade  games,  and“gameboy”  games; 

a many  o f their  questions  are  not  the  same,  making  international  comparison  incomplete; 

■ they  usually  do  not  target  identity -status  issues; 

■ they  do  not  target  the  participants’  perceptions  of  educational  versus  recreational  video/computer 
games;  and 

" they  do  not  target  English  literacy,  video/computer  game  genre  literacy,  and  identity  issues  with 
respect  to  the  Web  and  print  computer  gaming  magazines. 

We  have  commenced  research  that  involves  a comparison  of  various  issues  concerning  video/computer 
game  playing  by  13  and  14  year  old  youth  in  Australia,  Israel,  England,  and,  in  a minor  way,  China.  This  age- 
group  was  chosen  because  it  has  been  identified  in  some  surveys  to  be  the  age  when  teenagers  lose  interest  in 
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video/computer  game  playing  and  this  age  group  is  consequently  often  ignored  in  the  wider  literature.  (Yet  our 
survey  reveals  that  the  13/14  year  olds  all  play  one  or  more  types  of  video/computer  games.)  The  research 
comprises  a 28  item  questionnaire  that  was  designed  to  help  identify  various  aspects  of  computer/video 
gaming  for  this  age  group.  The  questionnaire  obtains  gender,  age,  and  ethnicity  profile  data,.  While  some 
items  contain  multiple  parts  with  fixed  choices  within  those  parts,  others  (including  a drawing)  are  open-ended 
thereby  allowing  the  students  to  delineate  the  issues  relevant  to  them.  (Surveys  usually  opt  for  closed 
categorical- mode  entries  for  ease  of  coding  and  correlation.) 

The  broad  issues  targeted  in  our  survey  concerning  video  games,  computer  CD-ROM  games,  computer 
Internet  games,  arcade  games,  and  “gameboy”  games  include: 

■ usage  patterns  (what  types  are  played,  and  for  how  long,  when,  where  and  with  whom  with  each 
game  type);  use  of  Internet  game  playing  is  particularly  targeted; 

■ reasons  for  playing  each  type  they  play; 

■ reasons  why  they  quit  a particular  game  playing  session; 

■ role  of  video/computer  games  in  their  recreational  lives; 

■ game  genres  played  and  preferred,  and  reasons  why; 

■ identity-status  construction  and  positioning  (of  themselves,  of  and  by  their  peers,  by  the  games, 
and  by  the  gaming  print  media);  this  area  is  also  linked  to  gender  and  violent/non-violent  issues; 

■ perceptions  of  effects  of  games  on  themselves  and  others; 

■ concerns  associated  with  video/computer  games; 

■ characteristics  that  make  a good  video/computer  game; 

■ video/computer  game  literacy; 

■ comparison  of  educational  vs  recreational  computer  games.  (Our  data  highlights  the  relevance  of 
identity-status  factors  in  this  comparison.) 


At  present,  we  are  processing  some  584  completed  questionnaires  from  the  participating  countries. 
Categories  and  correlations  have  been  delineated  and  a database  is  being  constructed.  We  expect  this  analysis 
to  enable  us  to  obtain  a global  characterization  of  this  growing  digital  gaming  culture,  and  to  shed  light  on  its 
role  in  the  youths’  lives  internationally. 


Literature  References 

Australian  Bureau  of  Statistics  (2000)  ABS  Catalogue  No,490L0  Children’s  participation  in  cultural  and  leisure  activities, 
2000.  Canberra:  ABS  [Online:  http://www.abs. gov.au/1 

Centre  for  Research  on  Innovation  and  Competition  and  Department  of  Psychology  (2002),  Digiplay  Initiative. 

Manchester:  University  of  Manchester  and  University  of  Central  Lancashire,  f Online  http://digiplav.org,  ikl 
Downes,  T.  (2000).  An  Australian  study:  The  computer  as  a "playable  tool."  Paper  presented  at  American  Education 
Research  Association  Conference:  Creating  knowledge  in  the  21^*  century:  Insights  from  multiple  perspectives, 
New  Orleans,  24-28  April. 

Drotner,  Kirsten  (2001).  Medier  for  Fremtiden,  Copenhagen:  Host  og  Son 

Griffiths,  M.  (1997).  Computer  game  playing  in  early  adolescence.  Youth  & Society;  29(2),  223-37, 

Hall,  W.  (2000).  Demographics:  The  kids  are  all  right.  Are  today’s  Japanese  children  - the  next  Internet  generation  - going 
to  hell  in  a handbasket?  Tokyo:  Japan.Inc, 

[Online  http://www.iapaninc.net/mag/comp/2000/03/mar00  reports  is.htmll 
http://game-research.com/statistics.html) 

Media  Analysis  Laboratory.  (1998).  Video  game  culture:  Leisure  and  plav  preferences  of  B.C.  teens  Vancouver:  Simon 
Fraser  University  [Online  http://www.media-awareness.ca/eng/issues/violence/resource/reports/vgames.htm1 
Turkle,  S.  (1997).  Seeing  Through  Computers:  Education  in  a Culture  of  Simulation,  The  American  Prospect.  3UMarch- 
April),  76-82.  [Also  Online  http://epn.org/prospect/3 1 /3 1 turkfs.htmll. 

Walsh,  D.  (2001),  Sixth  annual  video  and  computer  game  report  card  National  Institute  on  the  Media  and  the  Family, 
[Online:  http://www.mediafamilv.org/research/vgrc/20Ql  - 1 .shtmll 


300 


Students  Composing  Music  in  a Cognitive  Apprenticeship 
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Abstract:  Music  Uniting  Schools,  Youth,  and  Communities  through  Information  Technologies 
(MUSYC-IT)  is  an  innovative  multidisciplinary  program  offered  to  Grade  S8  students  in 
Schools  of  Distance  Education  and  small  rural  schools.  Students  conduct  an  oral  history 
interview,  reinterpret  this  as  a lyric,  compose  a melody  to  that  lyric  using  computer  software  in 
a cognitive  apprenticeship  model  through  mentoring  by  experts  via  the  Internet.  The  paper 
analyses  764  emails  to  identify  the  issues  involved.  The  findings  reveal  four  major  themes: 
information  technologies;  intersystemic  development;  implementation  and  delivery;  and 
student  learning  and  outcomes.  Technological  gliches,  variability  in  mentors,  commitment  of 
participants,  maintenance  of  the  status  quo  were  major  issues.  Most  students  completed  their 
song  and  presented  it  publicly;  principals,  parents,  and  students  request  it  be  continued. 


In  Australia,  there  is  unequal  access  to  music  studies  for  rural  and  remote  students.  Where  offered, 
music  is  limited  as  it  is  theoretical  or  extra-curricular.  Music  Uniting  Schools,  Youth,  and  Communities 
through  Information  Technologies  (MUSYC-IT)  is  significant  in  that  it  addresses  rural  youth  identity,  place, 
computer  technologies,  and  music  in  a way  not  previously  approached  in  Australian  curriculum  or,  we  believe, 
internationally.  Compare  the  face  to  face  PACERS  Small  Schools  Cooperative  which  employs  a professional 
musician  (\nderson,  1999)  and  the  Harvard’s  SongSmith  classroom  based  program  (Walters,  Meyaard,  & 
Scripp,  1990).  The  e- mentoring  approach  adopted  by  Canada’s  Composers  in  Electronic  Residence 
(http://www.edu.vorku.ca/CIERmain.html)  and  the  USA’s  Vermont  MIDI  Project  (http://www.vtmidi.org)  are 
both  extension  programs  supplementing  classroom  teaching  (also  see:  MacKenzie,  1998;  Ruippo,  1999), 

MUSYC-IT  is  the  music  program  for  our  students,  whether  learning  in  their  own  homes  through  the 
Schools  of  Distance  Education  (SDE)  or  in  a classroom  at  a small  rural  school.  Students  of  various  ages  (10*13 
years),  grade  levels  (5-8),  and  abilities  participated  individually  and  collaboratively,  contributing  at  their  own 
level  within  a cognitive  apprenticeship  model  (Brown,  Collins,  & Duguid,  1989).  The  students  conducted  oral 
history  interviews  with  community  people  thereby  validating  local  identity  and  community  within  an 
increasingly  globalised  world.  Students  then  reinterpreted  the  content  of  the  interview  as  a lyric,  composed  a 
melody  using  PrintMusic  software,  and  were  coached  during  the  process  through  guided  scaffolding  by  expert 
mentors,  particularly  the  nationally  acclaimed  composer  and  lyricist  mentors,  via  the  Internet  (cf  Schlager, 
Poirier,  Beans,  1996).  The  students  were  learning  something  of  what  is  involved  in  being  a professional 
historian,  lyricist,  and  composer. 

The  three  year  project  is  an  agent  for  educational  change.  This  study  is  part  of  a larger  research  project 
that  attempts  to  identify  social  and  educational  issues  concerning  intersystemic  development,  delivery, 
implementation,  and  outcome  factors  involving  a computer-mediated  interdisciplinary  curriculum  in  SDE  and 
small  rural  classroom  context.  As  the  research  literature  reveals,  information  technology  innovation  in  schools 
is  difficult  with  variable  implementation  success  (eg.,  Riley,  2000;  Rogers,  1995).  The  first  year  or  phase  of 
this  project  was  no  exception.  The  specific  aim  of  the  paper  is  to  identify  the  systemic  issues  concerning  the 
implementation  and  delivery  factors  in  the  first  year  with  respect  to  the  online  email  communications  between 
the  participants  of  the  program. 


Methodology 

The  procedure  employed  in  this  analysis  utilized  an  interpretive  perspective  (Erickson,  1986).  This 
interpretative  approach  advocates  the  portrayal  of  the  idiosyncratic  and  the  particular  as  legitimate  in 
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themselves  (Walker,  1986;  Wignall,  1998).  Pseudonyms  or  role  descriptions  are  used  throughout  this  report. 
Ethical  dilemmas  associated  with  the  use  of  confidential  information  were  addressed  by  providing  drafts  to 
participants  who  could  be  identified  in  the  text.  Modifications  and  suggestions  were  incorporated  at  their 
request.  The  setting  of  the  study  involved  a number  of  different  sites.  James  Cook  University  was  the  location 
of  the  two  university  project  directors  and  researchers.  Two  types  cf  school  sites  were  involved:  One  was  the 
four  SDE  whose  students  are  on  grazing  properties  isolated  from  regular  schools  or  travelling  (one  of 
MUSYC-IT  students  was  in  England  with  her  family).  Each  of  the  students  enrolled  through  the  SDE  had  their 
own  site  that  was,  in  fact,  their  own  home.  The  second  school  type  was  a rural  multi-grade  classroom,  half  of 
whose  students  undertook  the  program  under  the  guidance  of  a classroom  teacher.  Finally,  both  the  lyricist  and 
composer  mentored  from  their  homes  as  occasionally  did  the  project  directors.  The  participants  involved  the 
following: 

Research  and  development  project  directors  (1  male;  1 female)  conceptualized  and  coordinated  the  program, 
designed  and  developed  course  materials  and  the  Website,  mentored,  wrote  grant  applications,  and  oversaw 
the  data  collection  by  two  research  assistants. 

Students:  One  self-selected  cohort  involved  1 1 Grade  6-8,  SDE  students  (3  male  and  8 female)  who  had  some 
rudimentary  knowledge  of  music  and  worked  under  the  guidance  of  a home  tutor,  usually  their  mother.  The 
second  cohort  involved  13  (8  male  and  5 female)  Grade  5-7  students  selected  by  the  teaching-principal  on  the 
basis  of  their  ability  to  work  unsupervised. 

Mentors:  Four  subject  specific  mentors  (3  males;  1 female)  provided  expertise  to  the  students;  and  one  parent 
acted  as  the  IT  troubleshooting  mentor. 

Home  Tutors  & Teachers:  12  home  tutors  (one  had  2 children  enrolled)  and  1 teaching  principal  for  Term  3 
and  1 classroom  teacher  for  Term  4 with  the  rural  multi- aged/grade  cohort.  Most  home  tutors  and  both 
teachers  had  no  or  little  music  knowledge;  and 

Administrators:  Two  SDE  female  coordinators  (one  left  at  the  beginning  of  Term  3).  Theirs  was  an 
administrative  role  that  did  not  involve  mentoring  and  was  extra  to  any  regular  teaching  duties;  2 principals. 


Data  Results  and  Analysis 

Specifically,  this  paper  reports  an  analysis  of  the  email  documentation  connected  with  the  overall 
project.  There  were  764  emails  between  the  participants  during  2001  (additional  ones  were  inadvertently 
deleted  or  not  sent  to  the  project  directors).  The  data  were  firstly  categorized  according  to  the  source  of  the 
documentation.  The  constant  comparative  method  provided  a theoretical  construct  from  which  to  operate 
(Dye,  Schatz,  Rosenberg,  & Coleman,  2000;  Glaser  & Strauss,  1967).  Each  category  was  coded  with  a set  of 
interpretive  codes  that  emerged  from  the  data  and  from  discussions  between  the  researchers.  These  coding 
categories  were  combined  into  main  themes  about  the  processes  and  issues  involved  in  the  implementation  of 
the  curriculum.  This  text  is  an  interpretive  representation  of  those  themes  and  aims  to  describe  and 
characterize  the  factors  involved  in  the  implementation  of  the  online  curriculum. 

The  intensive  and  exhaustive  analysis  of  the  data  yielded  many  significant  insights  into  the  processes 
and  issues  involved  in  the  development,  delivery,  and  implementation  of  the  online  curriculum.  Four  major 
themes  presented  in  Table  1 elaborate  these  insights  in  an  attempt  to  qualitatively  describe  what  the 
participants  were  talking  about  as  the  program  was  implemented.  Evaluative  conclusions  about  the 
implementation  of  the  program  are  drawn  and  implications  for  future  implementation  are  discussed. 


Information 

Technologies 

Intersystemic 

Development 

Implementation,  Delivery 
& Research 

Student  Learning  & 
Outcomes 

Internet  Issues: 

■ Email 

■ Address 
changes 

■ Connection 
problems 

■ Size 
limitations 

■ Web  Site 

■ Firewalls 

Planning : 

■ Scope 

■ Cohort  size 

■ Curriculum 

■ Core  or  elective 

■ Funding 

■ Publicity 

■ Enrolment 
procedure 

■ Communication 

Administrative  Duties; 

■ Enrolments 

■ Mentors’  role 

■ Music  camp 

■ Organisation 

■ Implementing 

■ Timetables 

■ General  information 

■ Distribution  materials 

■ Distribution  fliers 

Student-Mentor 

Interaction 

■ Organising  & 
reorganising  student 
groups 

■ Feedback 

■ from  mentors  to 
individuals  & general; 

■ from  students:  erratic 

■ Difficulty  meeting 
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■ Security 
issues 

■ Development 

■ Delays 

■ Uploading 

■ Content 

methods 

■ Roles  stakeholders 

■ Letters  of  support 

■ Engaging  mentors 

■ Meetings: 

■ Formal  & informal 

■ Teleconference 

■ Face  to  face 

deadlines 

■ Delays  communicating 
between  mentors  & rural 
school  students 

■ Mostly  positive 

Software: 

Grant  Applications : 

Information  & Feedback 

Collaboration 

■ Selecting 

■ Internal  - research 

■ Reading 

■ Confusion  interviewing 

■ Ordering  issues 

■ External 

■ Coordinating  SDEs 

in  pairs/groups  at  a 

■ Distribution 

■ Philanthropic 

“ Fliers 

distance 

issues 

■ Industry 

■ Calendar;  assessment 

■ Cancelled  & late  SDE 

■ Testing 

■ Government 

■ Teaching/leaming  online 

enrolments  meant  group 

platforms 

■ Ascertain 

methods 

reorganisation 

■ Installation 

guidelines 

■ Working  collaboratively 

■ Difficult  for  SDE 

issues 

■ Writing 

■ Website 

students  as  not  common 

■ Plug-in  issues 

■ Liaison  JCU  R&D 

■ Liaison  SDE 

practice 

“ Rural  school:  changing 
grouping  for  activities 
increased  load  for 
mentors 

Hardware: 

Media: 

Resources: 

Learning  Outcomes: 

■ Platform 

■ Liaison  JCU  unit 

“ Music  tutorials  & 

■ Oral  history:  completed 

compatibility 

■ Writing  drafts 

worksheets 

■ Lyric:  all  but  3 students 

■ Crashes 

■ Interviews;  photo 

■ Oral  history  themes. 

“ Melodies:  8 completed; 

■ Speakers 

shoots 

questions  & protocols 

others  extended  to  2002 

malfunction 

“ Web  site 

“ Increased  IT 

■ iMovie 

■ iMovies 

competencies 

■ Camera 

Research: 

“ Collaboration:  variable 

■ Storage 

■ Appointing  RA’s 

■ Developing  instruments 

■ Data  collection 

■ Transcription  audiotapes 

■ Analysis  & writing 
papers 

■ Archival: 

“ Publication  songs; 

student  comments 
■ iMovie  music  camp 

■ Public  presentation 
songs 

Table  l:Major  Themes  from  Email  Analysis 


Theme  1:  Information  Technologies 

This  theme  deals  with  the  issues  and  processes  involved  in  getting  a the  IT  operational  for  the  students 
enrolled  in  the  program,  as  summarised  in  Table  1:  Internet  issues,  software,  and  hardware. 

Handling  of  the  Internet  issues  was  lengthy.  There  were  issues  such  as  the  schools’  and  university’s 
firewalls,  Web  site,  Internet  access,  and  security  that  needed  quick  resolution.  There  was  an  underestimation  of 
the  amount  of  time  required  for  these  issues  to  be  addressed  by  the  different  stakeholders  with  messages 
becoming  increasingly  more  urgent.  The  failure  of  the  Website  to  be  up  and  running  at  this  stage  caused  much 
frustration  for  both  students  and  administrators  alike.  The  delay  meant  that  multiple  mailings  of  files  and 
attachments  were  necessary.  Once  the  Website  was  working  students  were  able  to  access  the  site  at  their  own 
convenience. 

The  initial  stages  of  the  course  involved  correspondence  that  reflected  further  fhistrations  concerning 
uncertainty  in  the  accuracy  of  email  addresses  because  of  changed  email  accounts,  the  rural  school’s  firewall, 
and  the  families  of  distance  education  students  having  two  email  accounts  to  handle  various  sized  email  files. 
In  one  case,  these  email  problems,  exacerbated  by  low  bandwidth  technology,  resulted  in  two  students 
withdrawing  within  the  first  3 weeks  of  MUSYC  -IT. 


A significant  issue  involved  the  selection,  ordering,  and  distribution  of  the  relevant  music  software 
required  for  the  program.  After  much  discussion  about,  and  trialing  of,  the  most  appropriate  software  to  be 
used,  there  was  a problem  with  the  supply  of  this  particular  product.  Decisions  for  alternative  software  had  to 
be  quickly  made  and  actioned.  Unfortunately,  once  the  software  arrived,  installation  of  the  PrintMusic  two  CD 
pack  - the  original  program  on  one  CD  with  the  upgrade  on  the  other  - was  variable  as  some  worked  and  some 
did  not.  Conveniently  for  MUSYC-IT,  a parent  from  the  rural  school  volunteered  to  be  the  troubleshooter  for 
software,  email,  downloading  plug-ins,  and  hardware  problems.  Utilisation  of  community  resources  cannot  be 
overlooked  as  an  invaluable  factor  when  dealing  with  IT  and  school  change. 


Theme  2:  Intersystemic  Development 

This  theme  involved  the  intersystemic  issues  of  planning,  grant  applications,  and  media  publicity,  as 
summarised  in  Table  1.  Crucial  issues  involved  the  scope  and  cohort  size  of  the  project. 

To  help  avoid  a top-down  imposition  of  MUSYC-IT  on  teachers  and  students,  negotiation  and 
consultation  with  the  SDE,  parents,  and  students  commenced  in  December  2000  for  implementation  from  July 
through  to  mid -December  2001,  with  a student  involvement  of  ip  to  three  hours  per  week.  Initially,  it  was 
thought  that  20-30  students  from  Grades  7-10  would  enrol  in  MUSYC-IT  as  a core  interdisciplinary  subject. 
This  did  not  eventuate.  Some  reasons  had  to  do  with  the  maintenance  of  stasis  and  custodianship  of  the  status 
quo  (Southcott,  2000),  as,  when  it  came  to  the  crunch,  there  was  opposition  to  changing  the  SDE  segmented 
curriculum.  Other  reasons  had  to  do  with  the  initial  SDE  coordinator’s  workload  aggravated  by  being  new  to 
distance  education  and  the  level  of  appointment,  and  being  a self-acclaimed  neophyte  with  technology. 
Another  possible  reason  is  that  there  were  no  obvious  status  or  career  rewards,  incentives,  or  recognition  for 
teachers  to  support  innovate  change  through  MUSYC-IT  (Ely,  1999). 

An  alternative  to  a core  subject  was  implemented.  MUSYC-IT  was  offered  as  an  elective  with  students 
exempted  from  some  activities  and  assessment  to  ensure  that  the  students’  total  workload  was  not  increased. 
Lack  of  clear  communication  about  the  latter  was  late  and  inadequate,  helping  to  cause  small  SDE  enrolments. 
In  fact,  this  negotiated  and  agreed  upon  strategy  was  not  operationalised  for  two  Grade  8 students  in  one  SDE 
district,  and  it  was  the  reason  for  their  failure  to  complete,  and  hence  postpone  till  2002,  the  lyric  and  melody. 
As  it  eventuated,  having  11  SDE  students  geographically  dispersed  throughout  half  of  Queensland  with 
different  bandwidth  Internet  access  and  one  in  England  and  13  in  a rural  classroom  proved  a worthwhile 
“pilot”  cohort  for  ascertaining  the  viability  of  MUSYC-IT  under  both  conditions. 

The  various  issues  and  balance  of  power  involved  with  this  area  of  intersystemic  innovation  in  schools 
intrigued  the  project  leaders.  The  literature  helped  us  understand  that  the  principal  was  adopting  the  role 
approved  by  Full  an  (1992)  and  supported  by  Glickman  (1991),  arguing  that  too  much  store  is  placed  in  the 
leader  as  solution  compared  to  the  leader  as  enabler  of  solutions. 

Funding  applications  for  both  research  and  development,  particularly  the  latter,  commenced  well  before 
the  setting  up  of  the  project,  and  still  continue.  Collaboration  with  one  of  the  administrator  participants,  the 
principal  of  the  coordinating  SDE,  was  pivotal  in  acquiring  funding  for  the  composer-educator  mentor,  the 
purchase  of  the  PrintMusic  software,  and  the  music  camp.  Without  this  grant,  the  project  would  have  been 
postponed.  The  assistance  of  the  lyricist-mentor  helped  obtain  an  Arts  Queensland  grant  for  his  involvement  in 
MUSYCIT.  The  project  leaders  underestimated  the  time  taken  in  submitting  numerous  grant  applications,  five 
of  which  were  successful. 

In  requesting  assistance  from  industry  partners,  the  research  project  was  hoping  to  offset  costs  involved 
in  purchasing  the  necessary  equipment  to  run  the  course.  Requests  for  hardware  support  were  not  successful 
because  industry  were  hoping  to  reach  wider  audiences  and  receive  more  in  return  for  their  support  than 
simply  recognition.  Telstra  CountryWide’s  support  for  free  Internet  access  for  30  accounts,  a designated 
secure  Website  and  a Website  domain  name  for  six  months  demonstrated  a significant  level  of  support.  It  was 
especially  welcomed  by  the  families  involved  in  MUSYC-IT. 

Besides  developing  their  own  curriculum  resources,  the  project  directors  became  clients  for  four  final 
year  B.Ed.  students  in  a core  IT  course:  the  students  created  and  acted  in  three  i Movies  that  formed  part  of 
their  course  assessment  and  that  will  be  utilised  as  resources  for  next  year’s  MUSYC-IT  intake.  Three  of  the 
students  also  critiqued  the  music  curriculum  tutorials  and  worksheets  and  suggested  extension  material  using 
PrintMusic  software.  This  unforseen  outcome  of  MUSYC  -IT  capitalised  on  widening  the  scope  of  IT  diffusion 
to  JCU  graduating  students. 
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Theme  3:  Implementation,  Delivery,  & Research 

This  theme  describes  the  processes  and  issues  involved  in  operationalising  the  program  and  conducting 
research  (see  Table  1). 

The  project  directors  committed  an  enormous  amount  of  time  to  the  project  in  terms  of  receiving 
information,  attending  formal  and  informal  implementation  meetings  both  at  the  university  and  at  the 
coordinating  SDE  (2  hour  drive)  and  rural  school  (1  hour  drive),  devising  enrolment  applications  both  in  hard 
copy  and  an  email  version,  distributing  these  via  the  SDE  coordinator,  processing  enrolments,  inducting  the 
mentors,  organising  and  running  the  music  camp,  providing  feedback,  troubleshooting,  and  conducting 
research.  Email  correspondence  between  project  leaders  was  heavy  during  periods  when  either  one  of  the 
project  leaders  was  away.  Email  correspondence  was  open,  frequent,  prompt,  and  with  the  intent  of  keeping 
the  leaders  up  to  date  with  current  developments  pertaining  to  the  whole  project.  The  time  commitment  during 
the  early  part  of  Term  3 of  the  coordinator  of  the  SDE  to  help  solve  the  technical  and  enrolment  gliches  that 
delayed  the  start  of  the  program  by  a week  was  a positive  factor  in  the  implementation  of  the  project. 

The  project  leaders  also  attempted  to  organize  a teleconference  in  the  initial  stages  of  the  program . This 
required  that  a range  of  times  for  all  participants  be  offered  and  then  narrowed  to  a common  time. 
Unfortunately,  because  of  some  unforeseen  circumstances,  the  teleconference  did  not  eventuate.  The  purpose 
was  that  students,  home-tutors,  teachers,  SDE  coordinators,  the  rural  school  principal,  and  mentors  would  hear 
each  other’s  voices,  troubleshoot  any  major  problems,  and  feel  a greater  sense  of  collegiality  in  this  new 
venture  of  teaching  and  learning  music.  From  the  em.ail  exchanges,  this  sense  of  mllegiality  between  students 
did  not  eventuate  for  all  participants.  Perhaps  a teleconference  would  have  helped. 


Theme  4:  Student  Learning  and  Outcomes 

This  theme  is  concerned  with  the  learning  outcomes  experienced  by  the  students  themselves.  It  involves 
the  quality  of  interactions  between  students  and  mentors,  their  responses  to  the  various  tasks  of  the  program, 
and  the  degree  of  satisfaction  with  their  final  product.  The  types  of  issues  and  processes  discussed  include 
working  in  pairs/groups,  how  to  conduct  interviews,  and  providing  feedback  to  mentors. 

Once  the  IT  problems  were  overcome  and  the  students  were  able  to  work  through  their  worksheets  with 
the  music-educator  mentor’s  guidance,  there  was  a much  more  positive  note  to  the  email  correspondences: 
“Whilst  I am  writing  I just  want  to  say  how  much  I am  enjoying  doing  this  course.  Sometimes  I find  some  of 
the  words  a little  hard  to  understand  and  it  has  been  a little  bit  complicated  with  the  few  problems  we’ve  had. 
Now,  though,  I think  it’s  great  to  see  the  music  on  the  computer  and  I look  forward  to  doing  activities  in  the 
future”  (Email  correspondence  27/08/2001  from  A to  music  curriculum  mentor). 

Students  appeared  eager  to  conduct  interviews  and  were  able  to  identify  elements  about  their 
interviewee’s  story.  However  there  was  some  confusion  about  how  the  SDE  students  were  to  work  online  in 
pairs  for  this  activity.  Group  work  for  SDE  students  appears  to  be  a challenge  and,  although  suggestions  on 
how  this  activity  could  be  carried  out  were  provided,  they  were  obviously  inadequate  for  some  students. 

The  lyricist- mentor  undertook  a very  prominent  mentoring  role  and  his  concern  for  student  progress 
was  obvious.  Rather  than  sit  back  and  wait  for  student  outcomes,  he  often  was  proactive  in  ensuring  that 
communications  were  working  and  that  students  were  making  adequate  progress  on  their  lyrics.  Some  of  the 
email  messages  from  the  lyricist  mentor,  rather  than  contain  attachments,  contained  the  information  for 
students  in  the  body  of  the  message.  Sometimes  the  messages  were  up  to  five  pages  long.  Depending  on  the 
preferences  of  the  mail  programs  used  to  receive  mail,  these  messages  may  have  had  untidy  formatting. 
Whether  these  email  postings  were  somewhat  overwhelming  for  students  is  something  that  will  be  explored 
further  in  the  participant  interviews.  Students  were  appreciative  of  his  mentoring;  for  instance:  “Please  find 
attached  The  Value  of  Time2.  All  of  your  suggestions  were  wonderful-  they  made  me  look  more  deeper  [sic] 
into  what  I was  trying  to  acconplish.  I’m  a lot  happier  with  the  song  now  that  you’ve  helped  and  I agree  -it’s 
worth  changing  the  things  that  don’t  quite  fit,  even  if  you  really  like  them  ...  You’ve  been  brilliant  and  I really 
am  grateful  for  the  easy  ways  you  have  explained  things  to  me”  (Email  29/09/2001  from  S to  lyricist  mentor 
cc.  to  the  project  directors).  As  the  students  were  nearing  the  completion  of  their  lyrics,  there  was  evidence 
that  they  had  established  clear  and  effective  lines  of  communication  with  their  lyricist- mentor. 

Unfortunately  just  as  the  melody  composition  stage  of  the  project  was  commencing,  the  composer- 
mentor  withdrew  from  the  program  and  a replacement  mentor  had  to  be  found.  The  new  composer-educator 
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appeared  to  find  mentoring  via  email  more  problematic  than  did  the  lyricist  mentor.  He  was  also  not  as  prompt 
with  replies  as  students  would  have  liked.  This  figured  in  the  non-completion  of  some  of  the  students 
melodies.  It  also  suggests  that  the  project  directors  should  have  provided  formal  induction  into  online 
mentoring. 

There  were  very  few  correspondences  from  rural  school  students.  This  comparative  dearth  of  email 
correspondence  from  the  students,  who  all  had  their  own  email  accounts,  reflects  the  fact  that  the  teacher-led 
class  spent  time  face-to-face  jointly  brainstorming  the  various  mentors’  emails,  activity  tasks,  composing  the 
lyric,  and  working  with  PrintMusic.  There  were  fewer  problems  with  hardware  and  software  at  this  site 
because  the  IT  mentor-troubleshooter  was  a parent.  Overall,  this  seemed  to  result  in  fewer  frustrations  or  need 
to  email  the  mentors  quite  so  frequently  as  did  the  students  who  were  studying  in  an  individualized  setting. 

The  music  camp  and  presentation  of  their  songs  was  a success.  Both  principals,  the  mentors,  and  most 
parents  and  students  asked  that  MUSYC-IT  be  offered  at  both  first  and  second  year  levels  in  2002.  As  one 
Grade  6 student  summarised:  “Cliches  in  the  program  are  only  to  be  expected  as  it  is  new  and  innovative. 
These  will  be  overcome  next  year.  MUSYC-IT  helps  you  grow  as  a student.  I really  enjoyed  it.” 


Implications 

■ It  is  not  only  technical  resources  that  count  in  technological  diffusion  and  school  change  but  also  the  time 
and  energy  demanded  of  people  to  plan,  share,  observe,  and  take  action  (Boyd,  1992). 

■ More  leaiup  time  is  necessary  to  avoid  technical  gliches  interfering  with  the  mentoring  and  learning, 
and  thereby  lessening  frustrations. 

■ Maintaining  a designated  IT  troubleshooter  is  important;  the  project  directors  thought  they  had  it  covered 
inhouse  at  both  JCU  and  the  SDE.  Not  surprisingly,  hindsight  proved  that  such  IT  personnel  prioritised 
their  commitments  to  their  organisation  before  assisting  a project. 

■ Mentoring  online  needs  induction  workshops  for  some  to  be  the  expert  efficiently  and  effectively. 

■ Developing  effective  guidance  for  collaboration  online  needs  implementation. 

■ Obviously  having  a designated  project  coordinator  would  ensure  a smoother  running  of  MUSYC  -IT. 
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Abstract;  This  paper  describes  a hypertext  system  to  support  novice  Java  programmers 
to  improve  their  understanding  of  software  design  issues.  The  system  provides  the  usual 
hypertext  support  such  as  cross-referencing  and  formatted  display  of  source  code.  In 
addition  it  implements  a number  of  object-oriented  metrics  and  it  detects  and  remediates 
typical  misconceptions  of  novice  programmers. 


Introduction 

For  many  years  we  have  been  involved  in  teaching  a first  programming  course  to  students  of 
physics,  mathematics  and  informatics.  Three  years  ago  we  have  switched  from  Pascal  to  Java  as  a first 
programming  language. 

Our  experience  thus  far  has  shown  that  despite  the  fact  that  Java  is  relatively  clean  conceptually 
(at  least  compared  to  C or  C++),  many  students  struggle  to  learn  the  syntax  and  concepts  of  Java  and  gain 
enough  experience  with  programming  to  be  able  to  reason  about  program  design.  An  important  aspect  of 
the  course  are  individual  programming  assignments,  in  which  students  have  two  to  three  weeks  to  design 
and  implement  a program  which  is  then  discussed  individually  with  a teaching  assistant.  These  discussions 
are  invaluable  for  detecting  and  remediating  the  student’s  misconceptions  about  programs  and  program 
design  practice.  The  following  is  a typical  conversation  that  might  take  place  between  a teaching  assistant 
and  a student: 

TA:  What  are  these  instance  variables  i and  J in  the  class  Matrix? 

Student:  Those  are  the  loop  index  variables  I use  for  various  methods,  like  the  ones  for  adding  and 
multiplying  matrices. 

TA:  But  why  are  those  declared  as  instance  variables?  Surely  i and  J are  not  actual  properties  of  a 
matrix. 

Student:  Yes.  I originally  had  declared  those  variables  locally  in  each  method,  but  I figured  that  I could 
make  the  program  shorter  and  simpler  if  I moved  the  declarations  to  the  object's  scope. 

TA:  Isn't  that  dangerous?  It  is  quite  difficult  for  me  to  check  whether  these  methods  will  work  correctly. 
And  suppose  one  method  called  the  other  and  both  used  the  same  index  variable,  or  that  multiple 
threads  called  these  methods  simultaneously.  That  would  mix  things  up. 

Student:  Yes,  I suppose  so. 

This  conversation  reveals  a number  of  implicit  assumptions  and  misconceptions  that  the  student 
has  about  what  it  means  to  design  classes  and  programs.  The  student  moved  the  declarations  to  the  object 
level  because  he  is  aware  that  simplicity  and  elegance  of  programs  is  highly  valued  in  program  design 
culture.  But  this  particular  simplification  is  rooted  in  a misguided  notion  that  simplicity  is  achieved  by 
having  less  code. 

From  the  student's  point  of  view  the  program  is  perfectly  understandable,  and  the  experience  of 
having  the  teaching  assistant  tell  him  that  it  is  not  is  quite  sobering.  Students  generally  do  not  think  about 
the  possibility  of  others  reading  and  using  their  code,  and  what  that  implies  for  the  way  their  program 
should  be  designed.  Their  only  immediate  concern  is  (1)  that  their  program  compiles,  and  (2)  that  it  works. 
Their  thought  processes  are  organized  to  achieve  those  goals,  in  that  order.  Many  of  these  misconceptions 
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are  situated  at  a meta-cognitive  level  (Schoenfeld,  1992).  They  are  not  learned  explicitly  from  books  and 
lecturing,  but  are  acquired  implicitly  by  participating  in  a community  of  design  practice. 

We  do  not  believe  that  direct  interaction  sessions  can  be  replaced  by  technology  alone.  But  what 
we  can  do  is  make  them  more  effective  by  removing  the  most  common  misconceptions  earlier  in  the 
process  so  that  the  interactive  sessions  can  focus  on  other,  more  important  issues.  Our  aim  is  therefore  to 
design  a tool,  which  will  help  students  detect  such  common  misconceptions  and  advise  them  on  how  to 
improve  their  work  while  doing  it.  To  motivate  students  to  actually  use  it,  it  must  be  conceived  as  a 
generally  helpful  programming  tool  rather  than  a teaching  tool.  Students  are  not  aware  of  their 
misconceptions  and  therefore  will  not  be  inclined  to  actively  look  for  them.  For  this  reason,  we  will  embed 
it  in  a more  general  hypertext  documentation  tool. 


Design  Approach 

To  design  the  system,  we  have  followed  the  following  approach: 

1.  Create  a feature  model  (see  e.g.  (Czamecki  & Eisenecker,  2000))  of  the  features  found  in  the 
many  similar  tools  that  exist  on  the  market  today.  These  include  tools  such  as  LINT,  object- 
oriented  software  metrics  (Chidamber  & Kemerer,  1 994),  and  HTML  cross-referencing  tools 
such  as  javadoc*  andjavasrc^. 

2.  Use  XML  to  represent  Java  programs  at  different  levels  of  detail.  For  Java  interfaces  and  code 
we  selected  JavaML  (Badros,  2000)  and  for  Java  byte-codc  wc  have  developed  our  own  XML 
syntax  called  BCML  (byte  code  mark-up  language). 

3.  All  other  configuration  and  state  information  is  also  maintained  in  the  form  of  XML.  The 
syntax  of  this  XML  reflects  the  feature  model  developed  earlier. 

4.  XSLT  (XML  transformation  language  (Clark,  1999))  is  used  for  laying  out  source  code, 
calculating  metrics  and  detecting  misconceptions. 

By  studying  the  programs  that  students  had  submitted  in  previous  assignments,  we  were  able  to 
identify  a number  of  misconceptions  (currently  some  20)  that  we  believe  cause  students  to  write  badly 
designed  code.  For  each  of  these  misconceptions  we  then  looked  for  symptoms  (and  sometimes  counter- 
indications) in  code.  For  symptoms  that  can  feasibly  be  detected  automatically,  we  then  select  the 
representation  (currently  JavaML  and  BCML)  which  is  best  suited  for  detecting  it.  Finally,  an  XSLT 
template  is  written  that  matches  a particular  symptom  and  generates  a suggestion  for  improving  the  design. 
Table  1 shows  one  example  of  such  a misconception. 


Misconception:  The  importance  of  encapsulation 

Symptoms 

Public  instance  variables 

Direct  use  of  public  instance  variables  of  another  class 

Counter-Indications 

The  class  with  public  instance  variables  is  instantiated 
and  used  only  by  a single  other  class 

Measure 

The  number  of  classes  (excluding  subclasses)  that 
access  or  update  its  member  variables  a superclass 
or  nested  class  of  that  class 

Table  1:  An  example  misconception. 


The  suggestions  for  improvement  generated  are  general  natural  language  statements.  It  is  not  our 
intention  to  attempt  to  improve  code  automatically.  Focus  of  the  project  is  not  on  formal  specification  or 
verification  techniques,  but  rather  on  a heuristic  approach. 


'http://java.sun.com/j2se/javadoc/ 

^ http://home.austin.rr.com/kjohnston/javasrc.htm 


System  Architecture 


To  keep  a system  like  this  maintainable  and  extensible  we  have  decided  to  take  a very  strict 
attitude  towards  simplicity  and  architectural  integrity. 

• The  architecture  should  not  depend  on  the  language  used  (Java)  nor  on  its  specific  XML 
representation  (JavaML).  The  use  of  XML  and  stylesheets  allow  some  form  of  independence 
from  programming  languages  and  representations. 

• The  system  architecture  should  be  independent  of  the  specific  insights  about  misconceptions. 
It  should  be  straightforward  to  add  support  for  new  misconceptions  and  to  add  new  measures 
to  detect  them. 

• The  number  of  available  views  (e.g.  class,  package,  inheritance  diagram,  index,  ...)  and  their 
structure  should  be  completely  configurable,  so  that  developers  can  compose  and  add  new 
views  without  modifying  existing  code  and  users  can  select  a view  dynamically. 

These  principles  lead  to  a surprisingly  simple  but  powerful  architectural  design.  Figure  1 shows  a sketch  of 
the  system's  architecture.  When  a request  is  received  by  the  servlet,  it  loads  a configuration  file  containing  a 
description  of  the  structure  of  each  view.  This  conf  ig.xml  file  contains  parameters  that  are  placeholders 
for  information  provided  by  the  request,  such  as  the  URL,  session  parameters  and  HTTP  cookies.  The 
system  then  processes  the  resulting  XML  file  with  the  master  style  sheet  which  in  turn  will  call  upon  a 
parser  to  generate  XML  from  Java  source  code  or  byte  codes  and  applies  the  appropriate  templates  on  the 
resulting  XML. 
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Figure  1 : Sketch  of  the  system  architecture  and  component  interactions. 

The  resulting  architecture  is  very  simple  in  that  the  servlet  is  not  even  aware  that  it  reads  Java.  It 
only  deals  only  with  HTTP,  and  simply  calls  the  XSLT  engine  to  process  the  configuration  file.  All  of  the 
particulars  of  locating  Java  files,  parsing  them  and  transforming  them  are  local  to  the  stylesheets. 
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Current  Status  and  Future  Work 


The  current  implementation  provides  an  interface  that  is  very  similar  to  Sun’s  Javadoc.  It  provides 
package,  class,  usage,  tree,  and  index  pages.  The  difference  is  that  the  class  view  provides  the  ability  to 
view  source  code  for  each  method  as  well,  with  a feature  to  show  the  type  and  declaration  of  each  variable 
that  is  referenced  (illustrated  in  Figure  2),  and  one  to  highlight  each  usage  of  a particular  variable. 


Figure  2:  The  source  code  view  of  a class.  This  view  has  a number  of  useful  features,  such 
as:  syntax  highlighting,  the  ability  to  expand  and  collapse  code  sections,  cross-links  from 
variable  references  to  their  declaration.  When  the  cursor  is  moved  over  a variable  declaration, 
all  its  uses  are  highlighted  and  when  the  cursor  is  moved  over  a variable  reference,  its 
declared  type  is  shown.  Note  that  this  class  shows  three  misconceptions:  there  is  a public 
variable,  this  variable  is  used  as  a control  variable  for  the  for-loop,  and  the  control  variable  is 
updated  in  the  body  of  the  for-loop. 


In  addition,  a ‘metrics’  and  ‘misconceptions’  view  are  provided.  The  metrics  page  displays  a 
number  of  standard  object-oriented  metrics  as  defined  by  (Chidamber  & Kemerer,  1994).  The  other  shows 
a number  of  misconceptions,  as  is  illustrated  in  Figure  3. 
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Figure  3:  The  misconceptions  view,  showing  how  the  system  has  detected  four  possible 
problems  with  the  code  of  Figure  2.  It  has  detected  that  there  is  a public  variable  which  could 
be  made  private;  the  loop  control  variable  is  an  instance  variable,  and  it  is  updated  inside  the 
body  of  the  loop.  The  last  section  indicates  that  the  argument  to  the  ‘main’  method  is  never 
used  (although  in  this  case,  the  argument  is  required  for  another  reason). 


Currently  we  are  developing  measures  for  detecting  more  complex  misconceptions  specific  to 
object-oriented  design.  The  system  has  also  been  extended  to  that  students  can  easily  use  the  service 
remotely.  This  requires  that  their  Java  files  can  automatically  be  transferred  to  the  server  for  processing.  To 
allow  this,  the  student  currently  has  to  run  a small  server  program  that  allows  the  server  to  access  files  from 
his  or  her  class  path. 

We  intend  to  make  the  system  available  to  students  as  a tool  for  doing  preliminary  self-evaluation 
of  their  assignments  before  handing  them  in. 
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Abstract: 

17  volunteers  were  administered  an  online  drawing  environment  where  they  generated  images  as  a response  to  musical 
intervals,  then  tested  on  their  recall  of  these  musical  intervals  while  using  their  images  as  cues.  A paired  t-test 
comparing  pre  to  post  treatment  showed  a significant  effect  in  the  online  treatment  (t  ( 16  ) ~ 2.759,  - .014  ).  A 

scries  of  multiple  regressions  also  yielded  a significant  model  ( £ ( 4,  12  ) = 4.284,  £ = .022)  with  - .588.  The 
findings  are  explained  with  respect  to  dual  coding  theory  (Clark  & Paivio,  1991 ; Paivio,  1971 , 1990)  and  the  generative 
theory  of  reading  comprehension  (Wittrock,  1 990). 

When  examining  research  into  web-based  learning  environments,  the  most  noticeable  omissions  occur  in  music  education. 
One  of  the  largest  obstacles  to  designing  effective  online  music  applications  has  been  hardware,  i.e.,  computers  that  were  being  used 
for  instruction  were  not  capable  of  effectively  reproducing  authentic  sound.  Though  this  has  changed  over  the  last  six  years,  there  is 
little  research  on  how  to  effectively  use  computers  to  teach  music.  In  attempting  to  design  instructional  software  that  seeks  to  teach 
any  aspect  of  music  (performance,  composition,  history,  or  theory),  it  is  imperative  to  understand  how  learners  process  music  and  how 
this  can  inform  effective  design  of  musical  educational  media. 

This  pilot  study  is  investigating  some  possible  connections  between  aural,  visual,  and  verbal  information  in  musical  ear 
training  tasks  and  more  specifically,  the  learning  of  musical  intervals  (distance  between  two  notes).  By  making  a connection  between 
these  modalities,  a more  effective  method  for  designing  music  education  software  may  be  possible. 

Theoretical  Foundation 

The  research  involving  educational  implications  of  musical  auditory  processing  with  respect  to  images  is  scarce.  Many 
studies  look  at  the  relationships  between  verbal  and  visual  information  (Mayer,  1997,  1999;  Mayer  & Sims,  1994;  Moreno  & Mayer, 
2000)  or  between  verbal  and  nonverbal  information  (Clark  & Paivio,  1991).  Datteri  (2000)  investigated  the  influences  of  music  on  the 
perception  of  apparent  motion  in  stick  figures  and  action  terms,  and  through  Gestalt  audiovisual  principles  concluded  there  may  be  a 
deeper  semantic  system  connecting  the  visual  and  auditory  modalities. 

Clark,  Stamm,  Sussman,  and  Weitz  (1974)  examined  recall  of  sounds  that  could  be  labeled  versus  those  that  could  not  be 
easily  labeled.  They  concluded  that  dual  coding  theory  (Clark  & Paivio,  1991;  Paivio,  1971,  1990)  could  be  expanded  to  auditory 
information  in  the  form  of  sounds  as  well.  Additionally  Lipscomb  and  Kendall  (1994)  examined  the  interaction  between  music 
soundtracks  and  film,  suggesting  that  a model  of  motion  picture  perception  must  focus  on  the  ''source  of  meaning  within  both  the 
auditory  and  visual  modalities”  (p.  88)  and  is  therefore  “individual-specific”  (p.  91).  In  another  study  that  examined  the  effects  of 
verbal  labels  on  music  recognition,  Hiraoka  and  Umemoto  (1981)  used  19*  and  20*  Century  classical  excerpts  to  examine  how  verbal 
labels  affect  the  memory'  recall  of  musical  passages.  They  concluded  that  the  subjects  who  made  their  own  imagery  seemed  to  use  it 
as  a retrieval  cue  that  resulted  in  higher  recognition  scores. 

Thus,  there  may  be  a dual  coding  process  in  learning  that  exists  within  the  auditory  modality,  and  that  by  having  a user 
generate  meaning  or  images,  a stronger  connection  can  be  made.  Wittrock ’s  generative  theory  of  reading  comprehension  (1990) 
discusses  the  importance  of  generative  meaning  through  reading.  Could  this  process  be  similar  to  verbal  information  with  regards  to 
language? 

There  are  several  well-established  theories  about  processing  music  with  respect  to  language.  The  assumption  that  music  is 
processed  in  the  same  way  as  language  is  far  from  clear,  and  it  actually  appears  that  there  are  two  different  systems  that  exist  within 
the  auditory  modality,  one  for  verbal  information  and  one  for  auditory  information  (Sloboda  , 1985). 

Even  if  there  are  two  different  systems  that  exist,  perhaps  there  are  similarities  in  the  way  information  is  processed. 
Therefore,  if  a user  were  able  to  generate  an  image  based  on  a music  interval,  then  a stronger  connection  may  be  made  between  the 
visual  and  aural  modalities  that  might  result  in  the  user  learning  the  intervals  more  effectively. 


Methods 

Research  Questions:  Based  on  the  previous  suppositions,  a number  of  research  questions  arises: 

1 . What  are  the  effects  of  user-generated  images  on  interval  recognition  tasks? 

2.  What  is  the  relationship  between  user-generated  images  for  interval  recognition  and  a)  prior  music  knowledge,  b) 
prior  drawing  knowledge,  c)  music  interest  and  d)  drawing  interest? 
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Sample:  The  sample  consisted  of  17  graduate  students.  While  most  had  a limited  musical  background  there  was  a mixed 
background  in  graphic  design. 

Materials  and  Procedures:  The  experiment  involved  a web-based  drawing  program  developed  in  Macromedia’s  Flash  5.0. 
The  participants  were  able  to  generate  images  based  on  musical  intervals  that  were  saved  in  a database  and  later  used  to  test  their 
interval  recognition.  Participants  were  given  pre  and  post-tests.  Other  variables  that  were  explored  were  prior  music  and  drawing 
experience,  as  well  as  attitudes  towards  music  and  drawing  through  online  questionnaires. 


Results 


In  order  to  answer  the  first  question  a paired  t-test  was  conducted.  This  showed  a significant  increase  in  scores  from  the  pre 
to  post-test  (t  (16)  = 2.759,  p_  = .014).  Further  analysis  was  performed  to  determine  if  a particular  interval  was  recognized  more  often 
than  the  other  intervals.  The  pre  and  post-test  scores  from  each  of  the  three  intervals  used  in  the  study  (Major  3"*,  Tritone,  and  Perfect 
4*^)  were  then  analyzed  using  paired-t  tests.  Significant  differences  were  observed  between  the  pre  and  post-tests  only  for  the  Perfect 
4*^  (/  (16)  = 2.634,  p_  = .018).  In  order  to  answer  the  second  question  a multiple  regression  was  conducted  using  responses  from  the 
questionnaire  to  rate  prior  music  and  drawing  knowledge  and  interest  as  the  independent  variables,  and  their  answers  from  the  post- 
test as  the  dependent  measure.  This  overall  model  was  found  to  be  significant  ( F ( 4,  12)  = 4.284,  p = .022)  with  = .588.  The  only 
variable  that  contributed  significantly  to  the  model  was  the  music  background  score  (£  ~ .574,  ^=.039). 

Conclusions 

There  are  several  possible  conclusions  that  can  be  drawn  from  these  results.  Since  the  Perfect  4*^  was  the  last  interval 
presented  in  the  study,  it  makes  sense  that  it  was  identified  significantly  more  then  the  other  intervals.  The  results  from  the  multiple 
regression  indicate  that  having  a music  background  may  be  more  important  than  having  a background  or  interest  in  drawing.  In  any 
case,  the  pilot  study  shows  that  the  web  can  be  used  as  an  effective  multimedia  learning  aid.  In  terms  of  dual  coding  theory,  the 
connections  between  an  abstract  user-generated  picture,  abstract  auditory  information,  and  abstract  verbal  information  may  have 
increased  the  cognitive  load  experienced  by  some  learners.  The  participants  were  still  able  to  significantly  improve  their  skills  despite 
this,  so  perhaps  the  act  of  “generating”  was  still  effective  though  not  conscious. 

The  results  from  this  study  may  not  necessarily  be  attributed  to  the  act  of  drawing  as  normal  music  ear-training  involves  the 
same  type  of  rote  memorization.  The  fact  that  only  the  last  interval  was  significantly  recalled  enforces  this.  Since  the  music  labels 
were  a problem  for  some  subjects,  further  research  should  be  conducted  using  those  students  who  have  a stronger  background  in 
music,  as  well  as  including  interface  changes  that  might  help  lower  the  extraneous  cognitive  load  of  the  treatment. 
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Abstract:  This  paper  presents  an  overview  of  Fixed  which  is  a Web-based  Adaptive 
Educational  System.  The  design  of  Fixed  is  based  on  the  use  of  annotated  notional 
networks  to  represent  the  user’s  point  of  view  on  the  domain  model  in  order  to  provide 
an  interactive  help  based  on  re-using  experience. 

We  first  explain  how  the  annotated  notional  network  representation,  together  with 
several  annotation  systems,  provides  help  to  learners  and  to  teachers.  Then  we  introduce 
the  automatic  re-use  of  experiences  which  constitutes  the  other  keystone  of  Fixed. 

Keywords : Web-based  Adaptive  Educational  Systems,  Constructivism,  Adaptive 
Hypermedia,  Case-Based  Reasoning. 


1.  Introduction 

Internet  success,  as  well  as  the  Software  companies’  interest  for  the  ’’educational  market”  pushed  universities 
towards  the  web-based  teaching.  This  unavoidable  evolution  modifies  the  teacher's  role  who  must  help  learners  to 
build  their  knowledge  rather  than  being  a "knowledge  transmitter". 

Currently,  teachers  do  not  have  powerful  tools  helping  them  in  using  this  new  workspace.  Most  online  courses  are 
only  numerical  transcriptions  of  paper  versions,  and  commercial  training  environments  on  the  Web  (WebCT 
[Href2],  LeamingSpace  [HreO])  counterfeit  traditional  modes  of  teaching.  This  way  of  teaching,  inspired  by  the 
programmed  instruction  [Ski58],  is  unsuited  to  the  learner’s  needs  [Laf96]. 


2.  Related  works 

To  compensate  the  physical  absence  of  the  teacher  and  to  decrease  the  hypermedia's  induced  difficulties  (cognitive 
overload  [Rou98],  confusion  [Cas96]),  research  systems  such  as  "ELM-ART  II"  [Web97]  or  "AHA!"  [DebOl]  try  to 
adapt  teaching  material  (navigation  in  the  course  and  presentation  of  documents)  to  the  learner,  facilitating  his  task. 
Like  these  systems.  Fixed*  provides  the  learner  with  some  help  thanks  to  Adaptive  Hypermedia  [BruOl]. 

In  the  previously  quoted  systems,  the  teacher  describes  a course  which  will  be  transmitted  in  a form  adapted  to  the 
learner.  The  knowledge  thus  belongs  to  the  teacher  and  the  system  is  only  a media.  In  a close  way  of  the  Nestor 
navigator  [Ekl99],  Fixed  is  based  on  a constructivist  model  [Fhi95]  where  the  learner  builds  his  knowledge 
representation  actively  by  annotating  and  choosing  documents. 

Fixed  also  differs  from  the  others  Adaptive  Hypermedia  systems  by  re-using  the  system’s  experience.  By  system’s 
experience  we  point  out  elements  (traces)  of  training  that  the  system  records  during  its  use.  These  elements,  added  to 
a basic  user  model,  will  constitute  the  skeleton  of  the  cases  used  in  a Case-Based  Reasoning  (CBR)  engine 
[Aam94].  The  integration  of  the  CBR  paradigm  aims  to  help  the  adaptation  process  and  to  help  the  building  of  the 
domain  model. 


* Frojet  d’lnt^gration  de  I’eXperience  en  Enseignement  a Distance 
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3.  How  Fixed  helps 


Among  all  the  hypermedia  adaptation  techniques  pointed  out  by  Peter  Brusilovsky  [Bru96],  we  chose  to  focus  our 
search  on  curriculum  sequencing  and  adaptive  navigation  (3.1.)  because  it  is  on  these  points  that  we  think  that  the 
projections  in  terms  of  assistance  to  training  are  the  most  significant.  However  Pixed’s  documents  have  annotation 
skills  (3.3.)  which  allow  the  system  to  adapt  the  document  presentation  to  the  learner  (3.2.). 


3.1.  Help  learners  to  find  their  path 

Pixed  proposes  a path  in  the  hyperspace  of  the  course  adapted  to  the  learner  model.  This  model  fills  a part  of  the 
PAPI  specifications  [PapOO],  and  integrates  an  ANN  (annotated  notional  network)  [Her98].  Figure  1 presents  an 
example  of  ANN.  This  type  of  network  can  be  resume  as  an  impoverished  form  of  semantic  network  where  the 
relations  are  strongly  related  to  didactic  skills. 


— ► precedence  relation 

witli  threshold  “ n 


sufficienty  relation 

witli  threshold  = nj  and  contribution  =*  n» 


Concretely  Pixed  uses  ANN  to  propose  3 helping  level  : 

• The  linear  path  mode  : a document  sequence  generated  from  the  learner  model  [Her99]  is  provided  to 
learner.  The  system  thus  chooses  notions,  the  presentation  sequence  of  these  notions  and  relevant 
documents.  This  mode  is  the  closest  to  traditional  teaching  where  the  curriculum  of  the  course  is  decided  in 
advance  by  the  teacher  as  well  as  the  list  of  the  associated  documents. 

• The  assisted  mode  : the  learner  gets  a documents  sequence  (with  crossings  possibilities)  as  well  as  a map 
of  the  corresponding  hyperspace.  This  map  only  represents  the  notions  semantically  close  to  the  notions 
located  on  the  path  proposed  in  the  preceding  mode.  The  learner  can  consult  documents  explaining  notions 
in  direct  relationship  with  those  necessary  for  understanding  the  goal  notion  (according  to  the  model). 

• The  free  path  mode  : in  this  mode,  an  hyperspace  map  is  the  only  navigation  means.  The  learner  is  free  to 
navigate  among  all  the  course  notions,  and  for  each  notion  he  can  choose  among  the  associated  documents. 
However,  the  system  offers  him  some  advice  (icons  colors)  on  the  estimated  relevance  of  the  notions 
according  to  his  goal,  like  on  the  relevance  of  each  document  compared  to  the  selected  notion. 

Thus  the  learner  can  choose  to  be  more  or  less  helped  by  switching  his  navigation  mode. 

3.2.  Adapt  the  content  to  the  learner 

There  are  two  ways  of  contents  adaptation  in  Pixed  : 

• When  for  a given  notion  there  are  several  relevant  documents,  Pixed  can  select  which  one  to  present  to  the 
learner.  For  that,  Pixed  takes  account  of  the  recommendations  of  the  teachers  (annotations),  of  the  learner 
model  and  of  the  system  experience. 

• When  an  HTML  document  is  selected,  Pixed  carries  out  a second  adaptation  by  masking  some  parts  of  the 
document.  For  that,  Pixed  takes  into  account  the  teacher's  annotations  and  of  the  learner  model.  The 
masked  parts  can  however  be  consulted  by  learner  with  a click  on  the  context  beacons  which  announce 
more  information. 

One  of  Pixed's  originality  is  that  the  adaptive  presentation  is  not  only  based  on  the  learner  model  but  is  also  a 
function  of  the  system's  experience  (4.)  and  of  the  document  annotations  (3.3.).  Thus  teachers  and  learners  take  part 
in  the  adaptation  process. 


3.3.  Offer  annotation  tools  to  users 


In  Fixed,  learners  and  teacher  can  annotate  any  whole  document  on  the  server,  or  just  a parts  of  HTML  documents. 
This  in  order  to  guide  the  adaptation  but  also  to  provide  a tool  allowing  teachers  to  communicate  between  them  and 
with  learners.  Thus  we  hope  to  stimulate  a dialogue  around  the  domain’s  ANN. 

3.3.1.  Annotating  a whole  document 

Teachers  can  annotate  a whole  document  in  two  ways  : by  attaching  it  to  a notion  and  by  attaching  a textual 
description  to  it.  The  interest  note  can  be  used  by  the  system  when  it  has  to  make  a document  choice  or  simply  to 
make  it  available.  The  textual  annotation  being  intended  for  a learner  wishing  to  choose  a document  among  those 
that  can  be  consulted,  or  to  teachers  who  wish  to  re-use  a document  of  this  course  in  another  course. 


3.3.2.  Annotating  a part  of  a document 


To  annotate  a document’s  part,  teachers  and  learners  have  three  tools  : 

• The  first  enables  them  to  colour  parts  of  the  text  while  specifying  for  whom  (beginner,  intermediary, 
expert)  these  parts  are  useful.  These  annotations  will  allow  the  system  to  mask  parts  which  seem  unsuited 
according  to  the  learner's  model. 

• The  second  tool  allows  an  attachment  of  parts  of  a document  to  notions.  Teachers  and  learners  can  specify 
the  explained  notions  and  the  necessary  notions  in  order  to  understand  these  parts.  The  goal  of  this 
annotation  is  to  guide  the  availability  and  the  choice  of  documents. 

• The  last  tool  enables  them  to  attach  textual  annotations  inside  the  document.  They  can  highlight  the  part 
which  they  want  to  annotate  with  a mouse  click,  then  a contextual  dialog  box  appears  with  annotations 
possibilities  (figure  2).  These  annotations  will  be  added  to  the  document  in  the  shape  of  small  icons  "notes" 
in  the  text  of  the  displayed  document. 

All  these  annotations  can  be  kept  private  or  shared  with  other  users. 
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millimetres,  ce  qui  n'est  pas  instantan^.  H faut  done  disposer  d'une 
zone  de  stockage  intermediaire  pour  decoupler  le  temps 
d'integration  du  temps  de  transfert  des  donn^es.  C'est  surla 
definition  de  cette  zone  de  stockage  que  deux  technologies 
s'opposent : 


13 


figure  2 ; While  reading  an  HTML  document,  the  learner 
opens  a contextual  dialog  box  to  annotate  a word 
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3.4.  Help  teachers  to  build  a domain  model 


Fixed  separates  structure  and  contents  of  the  course  while  proposing  a much  greater  freedom  in  the  edition  of  this 
structure.  The  hyper-textual  structure  does  not  physically  exist,  it  is  generated  from  an  ANN  built  by  teachers.  Each 
teacher  has  his  own  representation  which  he  can  share  with  the  other  users.  As  course  documents  don't  have  hyper- 
textual  links  between  them,  they  are  added  and  described  by  the  teacher  (3.5.).  One  way  of  describing  is  to  link  them 
with  notions.  Fixed  provides  a graphical  interface  allowing  teachers  to  edit  and  publish  their  ANN. 

The  creation  and  the  maintenance  of  this  model  constitute  an  additional  task  for  teachers,  but  we  think  that  this  extra 
work  is  beneficial  in  the  long  run  : the  domain  model  is  represented  explicitly  by  what  will  support  the  sharing  and 
the  re-use  of  modelings  and  documents  between  teachers. 

3.5.  Facilating  the  addition  of  interactive  documents 

Teachers  can  deposit  and  remove  documents  in  the  format  of  their  choice  with  a description  (author,  date...).  They 
can  specify  which  main  notions  are  explained  in  a document  and  can  also  give  a list  of  notions  necessary  in  order  to 
understand  this  document.  The  system  provides  a help  to  teachers  by  pre-setting  the  necessary  notions  for 
understanding  according  to  the  domain's  ANN.  Teachers  remain  free  to  follow  these  recommendations  by  adding  or 
removing  notions  of  this  list. 

This  document  availability  method  can  seem  a constraint  for  teachers,  but  it  allows  a rapid  exploitation  of  these 
documents.  However  Fixed  leaves  the  possibility  to  the  teachers  of  adding  documents  to  the  course  without 
connecting  them  to  an  ANN.  The  document  will  thus  not  be  usable  in  the  state,  but  it  will  become  so  as  soon  as  a 
teacher  annotates  it. 

Fixed  also  offers  a tool  for  the  interactive  creation  of  multiple-choice  question  set.  Teachers  can  set  them  on  the 
server  like  other  documents,  and  the  system  can  use  them  in  the  evaluation  of  a pupil's  knowledge. 

4.  Integration  of  the  system’s  experience 

The  second  keystone  of  Fixed  is  the  re-use  of  the  system’s  experience.  In  this  part,  we’ll  try  to  explain  what  we  call 
experience  and  how  it  can  improve  help  for  users. 

4.1.  What  experience  are  we  re-using  ? 

The  experience  of  the  system  is  composed  of  the  training-episode's  trace,  that  the  system  records  during  its  use, 
stored  as  cases.  A case  is  a triplet  made  up  of  an  initial  learner  model,  of  his  trace  of  navigation  and  of  the  final  state 
of  his  model. 

4.2.  Why  reusing  the  experience  ? 

4.2.1.  The  experience  improve  adaptive  navigation 

The  use  of  the  experience  for  the  navigation  assistance  such  as  we  conceive  it  can  be  summarized  as  follows  : the 
system  creates  a case  from  new  learning,  then  it  searches  in  the  base  for  similar  cases.  This  similarity  (Tversky  type 
[Tve77])  is  function  of  the  particular  distances  between  identical  notions,  of  the  annotations  and  of  the  final  goal- 
notion. 

The  system  extract  the  path  from  the  adapted  case  obtained,  then  it  eventually  increases  this  path  and  annotates  it 
according  to  the  annotations  of  learners  which  were  used  to  work  out  the  similar  cases.  This  adapted  course  is 
proposed  to  the  learner  which  is  evaluated  at  the  end  of  the  course.  The  system  then  learns  this  new  case  by 
memorizing  it. 

At  the  end  of  the  training.  Fixed  also  preserves  the  re-use  trace  (elaborate  case  + similar  cases  + stored  case).  This 
trace  will  increase  one  second  base  which  allow  the  system  to  refine  its  measurements  of  similarities. 


4.2.2.  The  experience  improve  adaptive  presentation 

When  the  system  must  choose  between  several  documents,  it  integrates  the  experience  in  the  evaluation  of  the 
relevance  of  each  document.  Thus  a document  which  will  have  been  consulted  much  in  similar  cases  of  training 
succeeds  and  little  in  cases  of  failed  training  will  be  favored. 

When  an  HTML  document  is  selected,  Fixed  can  unmask  the  parts  of  the  document  that  were  selected  in  similar 
cases  of  training  successes.  These  cases  can  also  be  used  to  add  some  annotations. 

4.2.3.  The  experience  helps  the  maintenance  of  the  ANN  and  of  the  document  database 

The  first  use  of  the  system  experience  is  to  inform  teachers  about  the  use  of  their  course  as  an  improved  log  file.  So 
they  are  helped  in  maintaining  the  document  database. 

Modifications  of  relations  weightings  can  also  be  proposed  on  a statistical  principle  : if  a badly  known  notion  seems 
more  penalizing  to  reach  the  goal,  the  system  proposes  an  increase  in  the  threshold  of  the  relation  of  associated 
precedence.  The  experience  constitutes  a significant  help  here,  because  it  is  very  difficult  for  a teacher  to  estimate 
the  thresholds  and  contributions  of  the  relations. 


Fixed  helps  teachers  by  proposing  changes  of  their  model  for  a better  explanation  of  learners'  successes  or  learners’ 
failures.  For  example,  if  a notion  announced  by  a teacher  as  preceding  to  a training  is  never  consulted  by  learners 
without  harming  their  training.  Fixed  'will  propose  to  teachers  to  call  into  Question  this  relationship. 


By  observing  the  successes  of  learners  having  followed  the  courses  proposed  by  the  system.  Fixed  will  be  able  to 
propose  to  the  teacher  some  new  links  which  he  would  not  have  provided  in  his  model. 


figure  3;  The  use  of  the  experience  in  Fixed 


5.  Discussion 

New  methods  of  teaching  must  come  with  new  educational  technologies.  Fixed,  by  separating  structure  and  contents 
of  the  course,  improves  the  hypertext  use  and  also  explore  a new  way  of  teaching. 

We  believe  that  this  course  creation  methodology  is  adapted  to  training  hypermedia  environments  and  that  a shared 
experience  makes  the  users  feel  helped  in  their  tasks. 

The  actual  stage  of  our  research  tasks  is  a phase  of  a test  with  a great  number  of  users  in  order  to  validate  our 
approach  on  a greater  scale. 
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Abstract:  Our  activities  in  co-teaching  an  engineering  design  course  across  the  Atlantic,  i.e., 
at  Carnegie  Mellon  University  (CMU),  USA  and  at  Delft  University  of  Technology  (DUT), 
the  Netherlands,  at  the  same  time,  required  the  use  of  information  and  communication  tools 
for  communication  and  collaboration  purposes  between  students  and  between  instructors  and 
students.  In  this  paper  we  analyze  the  overseas  com  munication  and  collaboration  processes 
among  students  and  instmctors,  and  their  implications  for  learning.  We  have  used  a 
theoretical  framework  for  'collaborative  learning'  and  for  'stimulating  active  participation'  for 
analyzing  our  observations  and  for  translating  our  results  to  a broader  theoretical  framework. 
In  practice,  it  meant  that  we  experimented  among  other  variables  with  group  compositions 
and  with  insiruclur  role  descriptions.  We  concluded  that  many  of  the  techniques  mentioned  in 
literature  did  enhance  collaboration  and  learning  between  students,  but  that  intense 
communication  with  overseas  instructors  is  still  a major  stumbling  block. 


1 Introduction 

The  availability  of  various  information  and  communication  technology  tools  (ICT)  has  enticed  maiy 
experiments  that  are  aimed  at  evaluating  the  use  of  ICT  tools  in  university  courses.  Our  activities  in  co  -teaching 
an  engineering  design  course  across  the  Atlantic,  i.e.,  at  Carnegie  Mellon  University  (CMU),  USA  and  at  Delft 
University  of  Technology  (DUT),  the  Netherlands,  at  the  same  time,  also  required  the  use  of  ICT  tools  for 
communication  and  collaboration  purposes  between  students  and  between  instructors  and  students. 

In  earlier  papers  (Herder  et  aly  2002;  Subrahmanian  et  al,  2001)  we  have  reported  on  the  use  and 
effectiveness  of  tools  like  video  conferencing  and  video  taped  lectures  in  the  course.  In  this  year's  edition  of  the 
course  we  have  focused  on  communication  between  students  and  instructors  and  on  the  role(s)  of  the  instructors 
on  both  sides  of  the  Atlantic.  The  experiments  in  this  course  are  part  of  a large  project  executed  at  DUT,  in 
which  various  instructors  research  the  functional  use  of  ICT  in  their  courses  (Brakels  et  al,  2002). 

The  main  objective  of  the  course  "Engineering  Design  Problem  Formulation"  is  to  teach  the  process  of 
engineering  design  problem  formulation  and  the  role  of  mathematical  and  other  modeling  techniques  to  aid  in 
that  formulation.  The  pedagogical  objective  is  to  get  the  students  to  understand  that  articulating  the  right 
problem,  based  on  a process  of  negotiation  and  clear  representation  of  the  problem  among  the  stakeholders  and 
the  design  team,  is  more  important  than  being  able  to  solve  perfectly  the  wrong  problem  (Subrahmanian  et  al, 
2001).  The  reason  for  involving  students  from  the  USA  as  well  as  from  the  Netherlands  was  that  by  doing  so,  we 
were  able  to  use  a "constructivist  approach"  to  learning:  showing  the  students  that  knowledge  is  not  developed  in 
isolation  but  within  a social  and  cultural  context  (Brown  and  Duguid,  2000;  Fetherston,  2001). 

First,  we  ask  groups  of  students  to  formulate  a number  of  small  design  problems,  to  become  familiar  with 
the  use  of  the  basic  framework  of  goals,  tests,  design  spaces  and  starting  points  and  with  the  use  of  mathematics 
as  a language  for  articulating  and  unambiguously  representing  a problem.  The  groups  are  mixed  groups  in  the 
sense  that  they  include  students  from  DUT  as  well  as  from  CMU.  Once  they  are  reasonably  familiar  with  this 
way  to  recast  problems,  we  step  up  to  larger  group-based  assignments.  They  are  asked  to  read  and  report  on 
books  from  a list  we  provide  in  the  area  of  engineering  and  design.  The  students  were  to  analyze  the  design 
exemplars  in  the  books  and  characterize  them  by  using  the  four-part  description  we  present  to  aid  in  formulating 
design  problems.  The  final,  large  assignments  include  design  problems,  such  as  design  of  a transportation  system 
for  Pittsburgh  and  the  design  of  a water/sewage  system  for  a new  housing  development  near  Amsterdam. 

In  this  papa*  we  analyze  the  communication  and  collaboration  processes  among  students  and  instructors. 
The  next  section  will  describe  the  theoretical  framework  that  we  will  use  to  analyze  our  results.  During  the 
course,  we  have  experimented  among  other  variables  with  group  compositions  and  with  instructor  role 
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descriptions,  which  will  be  described  in  the  next  section.  The  results  describe  our  experiences  with  on-line 
participation  and  on-line  learning,  based  upon  a survey  we  held  among  the  students  after  the  course  and  based 
upon  on  our  observations  during  the  course. 


2 Research  Model  and  Approach 

2.1  Introduction  to  the  Course  as  an  Electronic  Conference 

The  course  described  in  this  paper  required  on-line  communication  because  of  distance  (USA  and 
Europe)  and  time  difference  (six  hours)  between  the  two  groups  of  students  and  instructors.  Based  upon  our 
experiences  from  running  last  year's  course  (Herder  etal:,  2002,  Subrahmanian  etal,  2001),  in  which  our  efforts 
focused  on  the  technicalities  of  the  use  of  ICT  tools  in  teaching,  we  decided  this  year  to  focus  on  the 
collaboration  and  communication  aspects  between  the  groups  and  instructors.  In  particular,  we  ran  the  different 
elements  of  the  course  in  different  ways. 

The  course  required  students  to  work  on  formulation  problems  together  by  discussing  the  problems, 
posing  questions  to  each  other,  critiquing  and  commenting  each  other's  work,  sharing  the  workload  and 
decreasing  the  project's  run  time  by  'leap-frogging',  i.e.,  half  of  the  group  works  while  the  other  half  of  the  group 
is  out-of- office  - the  second  half  continues  the  work  when  the  first  half  is  out-of-office,  etc.  All  communications, 
reports,  comments  and  the  like  were  to  be  posted  in  the  web-based  document  management  system  LIRE'  («-dim 
group,  200 U so  that  all  information  would  be  retrievable  for  anyone  at  any  time.  This  type  of  on-line 
collaboration  and  interaction  is  often  referred  to  as  a computer  mediated  conference  (CMC). 

Salmon  (2000)  developed  a five-stage  model  for  on-line  learning  and  moderation  of  electronic 
conferences,  which  we  will  use  in  this  paper  as  our  research  framework.  The  consecutive  stages  defined  by 
Salmon  are:  (1)  Access  and  Motivation,  (2)  Socialization,  (3)  Information  Exchange,  (4)  Knowledge  Con- 
struction, and  (5)  Development  and  Reflection. 

For  each  of  these  steps  it  is  important  to  focus  on  the  technical  issues,  the  mutual  learning  issues  and  the 
E-moderating  issues.  The  technical  issues  have  been  described  in  our  papers  mentioned  earlier,  and  we  refer  to 
those  papers  for  the  conclusions.  This  paper  focusses  on  the  other  two  issues  in  the  five  stage  model,  i.e.,  the 
learning  issues  (student  - student  interaction)  and  the  E-moderating  issues  (student  - instructor  interaction).  Steps 
three  and  four  from  the  five  stage  model  will  be  worked  out  in  detail  in  this  paper.  Table  1 shows  some  brief 
previous  results  and  implementations  for  the  steps  and  issues  that  are  not  being  discussed  elaborately  in  this 
paper:  the  technical  issues  for  all  steps  are  summarized  in  the  second  column,  and  the  learning  and  e-moderating 
issues  are  summarized  in  the  two  righ  hand  side  columns  for  steps  one,  two  and  five. 


Table  1.  Application  of  the  five  stages  (Salmon,  2000)  to  the  course  (bold  typeset  is  discussed  in  this  paper). 


Technical* 

Learning 

E-moderating 

1.  Access  & Motivation 

Communication  tools 
Video  conference 
Web  Based  Document 
Management  System 
Video  clips 

Personalities  and 
group  formation* 

On-line  course  material 
User  instructions* 

2.  Socialization 

3.  Information  Exchange 

Collaborative 

learning 

Stimulating  active 
particpation 

4.  Knowledge  Construction 

5.  Development  & Reflection 

Evaluation  & Survey " 

1)  See,  for  example,  Herder  etal  (2002) 

2)  Executed  in  a face-to-face  meeting 


Salmon  (2000)  provides  some  advice  relevant  to  each  of  the  five  stages.  Other  authors,  however,  provided 
more  in-depth  advice  on  how  to  stimulate  collaborative  learning  and  active  participation.  In  particular,  we  have 
used  the  techniques  proposed  by  Hiltz  (1994)  to  stimulate  active  participation  and  collaborative  learning.  These 
techniques  are  briefly  summarized  in  the  next  two  sections. 
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2.2  Collaborative  Learning 


The  ultimate  goal  of  taking  a course  is  to  gain  knowledge  or  learn  new  skills.  Particularly  in  a course 
where  group  work  is  key,  collaborative  learning  is  probably  dominant  over  traditional  one-to-many  learning 
from  instructors.  In  order  to  get  students  to  (collaboratively)  learn  the  skills  and  knowledge  they  are  required  to 
learn,  it  is  imperative  that  some  techniques  are  being  applied  that  support  and  encourage  collaborative  learning 
and  student  to  student  interaction.  We  have  turned  to  Hiltz  (1994)  for  a number  of  concrete  collaborative 
learning  formats  and  assignments.  The  ones  that  we  used  in  the  course  are:  students  as  teachers,  writing  groups, 
groip  or  team  projects,  and  team  building. 


2.3  Stimulating  Active  Participation 

Any  conference  or  discussion  requires  a moderator  who  is  able  to  stimulate  participants  to  actively 
participate  in  the  discussions.  For  computer  mediated  conferences  the  issue  is  even  more  significant,  as  the 
moderator  and  participants  are  not  in  direct  contact  with  each  other.  Hiltz  (1994)  suggests  a number  of 
techniques  that  may  stimulate  participants  to  engage  into  electronic  conferences,  some  of  which  we  have  applied 
in  parts  of  our  course.  The  techniques  used  in  the  course  in  the  instructor-student  communications  are  "require 
regular  participation,"  "instructors  present  conflicting  opinions,"  "responding  and  weaving  by  instructors," 
"explicitly  request  responses,"  and  "role  playhg." 


2.4  Research  Variables  and  Approach 

Our  experiences  in  running  the  course  last  year  indicated  that  we  needed  to  provide  the  students  from  the 
outset  with  some  basic  instructions  about  how  to  communicate  effectively  with  people  at  a distance  and  with  a 
time  lag.  Instead  of  letting  each  group  'invent  the  communication  wheel'  all  over  again,  we  pointed  them  toward 
the  various  opportunities  besides  emailing,  such  as  chatting,  telephoning  or  even  video-conferencing. 


Table  2.  Application  of  techniques  in  the  course. 


Assignment  1 

Assignment  2 

Assignment  3 

Collaborative  learning 

students  as  teachers 

X 

writing  groups 

X 

X 

X 

group  projects 

X 

X 

X 

team  building 

X 

X 

Active  participation 

require  participation 

X 

X 

X 

conflicting  opinions 

X 

responding  / weaving 

X 

X 

X 

request  responses 

X 

role  playing 

X 

Table  2 shows  the  techniques  we  applied  in  the  various  stages  of  the  course.  For  the  first  small 
assignment  in  the  course,  we  dropped  the  students  in  at  the  deep  end  of  the  pool  with  respect  to  getting  the 
overseas  cooperation  to  work.  We  provided  them  with  basic  technical  support  and  with  conventional  instructor 
support.  As  the  course  progressed,  instructions  and  structure  became  increasingly  unambiguous  by  applying 
more  and  more  techniques  for  collaborative  learning  (student -student  communication)  and  active  participation 
(student -instructor  communication). 

The  following  sections  describe  our  specific  implementations  and  experiences  with  the  various 
techniques.  The  impacts  on  the  learning  and  collaboration  process  have  been  measured  by  distributing  a survey 
after  the  course  and  by  evaluating  the  course  with  the  students  in  a face-to-face  meeting.  Our  personal 
observations  were  used  with  caution  as  a third  source  of  information. 
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3 Collaborative  Learning 

The  course  was  taught  simultaneously  at  both  locations  by  two  instructors  at  each  side  during  fall  2001. 
We  were  able  to  re-use  the  schedule  and  material  (lectures,  handouts)  that  we  developed  in  fall  2000  when  we 
ran  the  first  experiment  of  this  course.  A total  number  of  24  students  participated  in  the  course  (6  at  DUT  and  1 8 
at  CMU).  The  students  were  instructed  about  the  basics 
on  cross  Atlantic  cooperation  in  the  first  class, 
comprising  communication  tools  and  etiquette.  The 
survey  indicated  that  the  methods  for  communication 
were  reasonably  clear  at  the  start  of  the  course,  as 
indicated  in  figure  1.  It  shows  the  number  of  students 
that  agreed  or  disagreed  on  the  statements  in  the 
survey,  using  scores  on  a 1 (disagree)  - 5 (agree)  scale. 

Cross-Atlantic  collaboration  was  initiated  for 
the  first  assignment  by  doing  group  work  - groups 
based  on  personality  tests  of  the  students  (Meyers- 
Briggs  test,  2001)  that  spanned  the  two  participating 
universities  - and  by  forming  writing  groups ^ i.e., 
letting  the  students  antique  each  other's  intermediary 
work  and  reports.  Some  students  indicated  that  they 
felt  disconnected  from  the  students  at  the  other  side  of 
the  Atlantic  because  they  only  knew  each  other  frc 
electronic  communications.  Other  students  divided  the  group  work  such  that  they  had  to  communicate  with  each 
other  as  little  as  possible.  Only  towards  the  deadline  of  the  first  assignment,  the  students  reformed  into  writing 
groups  to  critique  each  other's  work. 

In  order  to  enhance  the  communication  and  collaboratioi  in  the  groups  we  asked  the  students  to  form  the 
groups  for  the  second  assignment  themselves,  resulting  in  better  team  building  (Hiltz,  1994).  All  groups 
managed  to  cooperate  more  intensely  on  the  book  reports,  and  in  most  cases  the  reports  were  reviewed  and 
critiqued  by  all  group  members.  For  the  third  assignment  the  groups  were  reshuffled  again  and  we  explicitly 
asked  the  students  to  present  new  or  complex  aspects  of  their  work  on  the  assignment  in  class  to  the  other 
students.  In  addition,  they  were  as  ked  to  pose  questions  to  the  other  students.  The  students  became  teachers, 
which  further  enhanced  collaborative  learning  in  and  between  the  groups. 


The  use  of  communication  tools 
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Figure  1.  Communication  tools  in  the  course. 


Working  with  the  local  group 
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Figure  2.  The  usefulness,  intensity  and  efficiency  of  Figure  3.  Students'  opinions  on  international 
local  collaboration  as  compared  to  collaboration  for  future  courses, 

international  collaboration. 


The  students  were  asked  to  rate  the  usefulness,  intensity  and  efficiency  of  the  overseas  collaboration 
compared  to  the  collaboration  with  local  group  members.  As  expected  from  theory  and  observations,  the  results 
in  figure  2 indeed  show  that  local  collaboration  was  more  useful,  intense  and  efficient.  Figure  3 shows  that, 
despite  the  difficulties  of  international  collaboration  shown  in  figure  2,  the  students  would  still  rather  intensify 
the  international  collaboration  than  cancel  it. 

The  students  were  found  to  spend  an  average  of  2,5  hours  per  week  in  emailing  with  each  other,  in 
addition  to  over  1 hour  per  week  of  on-line  chatting  and  over  1 hour  per  week  of  using  LIRE'.  The  telephone  and 
the  video  clips  available  were  used  only  a couple  of  minutes  per  week  on  average. 


4 Stimulating  Active  Participation 

The  first  student  assignment  comprised  two  small  formulation  problems.  The  instructors  were  given  no 
specific  roles  other  than  to  quickly  answer  questions  and  to  teach  a number  of  regular  classes.  The  group 
assignments  were  mandatory  and  any  results  were  to  be  posted  into  LIRE',  thereby  requiring  participation. 
Students  tended  to  consult  their  local  instructor  primarily  and  left  the  coordination  of  instructors'  advice  and 
responses  to  the  instructors.  Topics  that  appeared  to  be  relevant  for  all  the  students  were  discussed  or 
communicated  plenary  by  the  instructors  to  all  students  {responding  and  weaving). 

For  the  third  assignment  we  decided  to  boost  active  participation  by  applying  a number  of  extra 
techniques.  The  instructors  were  given  specific  roles  with  particular  objectives  that  would  conflict  with  other 
instructors'  objectives.  More  specifically,  the  students  were  the  project  contractors,  one  instructor  was  given  the 
role  of  project  commissioner  for  the  design  problem,  while  the  second  instructor  overseas  played  the  role  of  one 
of  the  stakeholders  with  possibly  conflicting  objectives.  For  example,  we  added  an  environmental  lobbyist  as  a 
stakeholder  to  the  design  of  a disposal  method  for  an  oil  rig  (commissioned  by  a fictitious  oil  company).  Finally, 
we  required  the  students  to  send  weekly  progress  reports  to  the  overseas  instructors.  By  explicitly  requesting 
responses,  we  expected  that  instructor- student  interaction  and  active  participation  would  be  enhanced  even 
further.  However,  the  students  never  managed  to  send  a progress  report  to  any  instructor,  indicating  that  our 
response  request  did  not  fit  their  sp  ecific  activities  or  needs.  We  did  observe  that  more  frequent  contacts 
occurred  between  instructors  (local  as  well  as  overseas)  and  students  in  the  final  assignment  compared  to  the  first 
two  assignments. 


International  instructor-student  communication 
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Figure  4,  The  possibility,  necessity  and  preference 
with  respect  to  communication  with  the 
international  instructor. 


The  role  of  the  Instructors 


Figure  5,  Preference  of  students  on  the  roles  of 
instructors  in  the  large  assignment. 


The  results  shown  in  figures  4 and  5,  confirmed  our  hypothesis  of  international  instructor-student 
communication:  although  the  international  communication  is  necessary  and  feasible,  the  students  still  prefer  to 
communicate  with  a local  instructor.  This  confirms  results  obtained  by  Walkington  and  Maroulis  (2001),  who 
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found  that  social  presence  and  meaningful  relationships  between  instructors  and  students  are  crucial  for  sound 
pedagogy,  but  are  very  hard  to  achieve  in  CMC  activities.  Although  the  role  playing  technique  may  have  had  the 
effect  we  aimed  at,  i.e.,  more  intense  communications  between  the  two  sides  of  the  Atlantic,  the  opinion  of  the 
students  about  the  role  playing  in  the  final  assignment  is  indistinct,  rendering  that  technique  debatable  for 
application  in  a cross  Atlantic  design  course. 


5 Conclusions 

We  have  used  the  framework  developed  by  Salmon  (2000)  to  identify  two  important  areas  for  improving 
the  cross  Atlantic  design  course,  taught  for  the  second  year  in  an  international  setting.  A number  of  techniques 
for  these  two  areas,  i.e.,  stimulating  active  participation  and  collaborative  learning,  were  taken  from  Hiltz  (1994) 
and  were  gradually  incorporated  into  the  course. 

We  found  that  the  techniques  for  collaborative  learning  in  a CMC  environment  contributed  to  a smoother 
learning  process,  although  they  did  not  take  away  fully  the  stumbling  blocks  for  international  cooperation. 
Students  indicated  that  an  intensification  of  international  collaboration  was  desirable,  if  and  only  if  this  would  be 
supported  by  appropriate  communication  tools,  aimed  at  the  contents  of  the  work,  but  also  on  the  sodal 
interaction  between  the  students. 

Techniques  that  we  used  to  enhance  active  participation,  mainly  techniques  applied  by  instructors,  were 
found  to  be  successful,  except  for  the  role  playing  technique.  Our  rationale  for  assigning  roles  to  instructors  was 
that  we  expected  it  to  form  an  incentive  for  boosting  communication  between  students  and  instructors  from 
overseas.  Students,  however,  preferred  to  talk  to  their  "own"  instructors,  after  which  they  relayed  the  messages 
to  their  overseas  colleague  students  if  necessary.  Communication  between  the  students  was  nevertheless 
increased  which  may  have  enhanced  collaborative  learning. 

General  comments  from  the  students  after  the  course  have  been  encouraging.  Most  students  indicated  that 
they  found  the  international  course  an  interesting  experiment.  We  should,  however,  increase  their  access  to 
communication  means  that  allow  more  social  interaction  than  email  and  CMC  alone.  Many  of  the  techniques  we 
only  gradually  incorporated  into  the  course  this  year,  will  be  used  from  the  beginning  of  the  course  next  year. 
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Abstract;  Group  techniques  and  methods  show  great  promise  with  respect  to  the  learn  effect,  in  Computer 
Supported  Collaborative  Learning  (CSCL).  These  should  result  from  the  perspective  of  cognitive  elaboration,  the 
construction  of  shared  knowledge  and  the  possibility  of  multiple  perspectives.  However,  the  previous  results  of 
this  research  were  not  very  extensive.  Therefore  we  started  a jigsaw  in  CSCL  in  the  context  of  the  federal 
flagship  project,  “Virtual  University  of  Applied  Sciences  for  Technology,  Computer  Sciences  and  Business 
Administration”  (VFH).  During  the  course  of  ‘Environment  oriented  Management”  (EoM)  we  tested  the 
different  assumptions  and  suppositions  regarding  the  effect  of  the  jigsaw  method.  In  the  case  study  we  focused 
the  evaluation  on  the  degree  of  collaboration  and  the  influence  of  the  group  method.  The  result  was  that  jigsaw 
proved  to  be  an  efficient  method  to  intensify  collaboration  in  CSCL.  To  archive  this  certain  conditions  have  to  be 
consider.  Aspects  like  group  cohesion,  individual  competence,  and  awareness,  have  a significant  influence  on  the 
result  of  the  jigsaw. 

1 Introduction 

Teamwork  is  strongly  associated  with  high  degree  of  effectiveness  and  efficiency.  The  apparent  evidence  toward 
the  superiority  of  cooperative  vs.  individual  work  is  also  transferred  to  the  area  of  learning.  It  ist  widely  accepted 
that,  '"'^cooperative  learning  is  supporting  the  success  in  learning  in  general'  [PfWeOO,  pp.I40].  As  plausible  as 
this  statement  appears,  you  can  also  state  that:  “you  can  decide  neither  theoretically  nor  empirically  which  social 
form  of  learning  and  working  is  better"  [ReMa99,  p.4].  Both  negative  and  positive  effects  within  the  area  of 
Computer  Supported  Collaborative  Learning  (CSCL)  can  be  proven  empirically  in  a relatively  effective  manner. 

A way  to  stress  the  positive  aspects  and  to  minimize  the  negative  ones  is  to  use  group  work  techniques. 

In  the  context  of  the  course  “Environment  oriented  Management”  (EoM),  which  was  conceived  and  produced  for 
use  within  the  federal  flagship  project  "Virtual  University  of  Applied  Sciences”  (VFH),  a group  work  technique 
called  jigsaw  was  implemented. 

The  cooperative  task  to  complete  was  structured  using  this  method.  A jigsaw  uses  a redundant  group  structure: 
main  groups  and  expert  groups.  The  group  task  can  only  be  completed  with  the  knowledge  acquired  in  the  exp  ert 
groups,  and  thus  with  the  intensive  cooperation  of  all  team  members. 


2 CSCL  - arguments  and  implications 

As  positive  impacts  and  arguments  speaking  for  CSCL,  three  positions  are  taken  into  consideration.  These 
positions  are  particularly  scrutinized  for  their  feasibility  and  their  implications  regarding  the  use  of  the  jigsaw 
method 


2.1  Cognitive  Elaboration 


The  possibility  of  cognitive  elaboration  in  cooperative  settings  is  valued  positively  and  used  as  a main  argument 
for  the  necessity  of  CSCL  in  the  psychology  of  learning.  In  general  one  can  assume  that  new  information  can  be 
linked  with  the  existing  schemes  only  after  it  has  been  restructed  an  elaborated.  One  of  the  most  effective 
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processes  of  elaboration  is  the  explanation  of  the  learning  matters  from  one’s  own  point  of  view.  Therefore 
teaching  is  the  most  effective  way  of  learning.  This  fact  is  conceptualised  as  the  perspective  of  cognitive 
elaboration  [e.  g.  Slav93]. 

But  the  explanation  of  the  learning  matter  relates  only  slightly,  or  has  no  impact  at  all  on  the  learning  success  of 
the  recipient  [Webb9 1].  The  advantages  lie  rather  clearly  on  the  person  who  gives  the  explanations. 

Thus,  a structure  needs  to  be  created  which  makes  explanation  possible  for  all  group  members  - independently 
from  group  status  and  individual  competence.  Within  this  context  the  main  starting  point  is  the  creation  of  the 
task.  If  designed  as  a jigsaw,  one  can  assume  that  explanatory  processes,  independent  from  the  status,  are  to  be 
stimulated. 

2.2  Multiple  contexts  and  perspectives 

Relevant  for  CSCL  are  the  demands  on  the  learning  settings  and  the  task  formulated  by  instructional  research. 
Significant  matters  include:  authentic  and  situation -based  learning  settings,  social  and  multiple  contexts  and 
multiple  perspectives. 


Within  the  framework  of  multiple  perspectives,  a learning  situation  is  created  so  that  the  learner  possesses  the 
various  possible  perspectives  of  the  task.  During  cooperating  the  group  members  are  supposed  take  the 
perception  of  the  other  group  members.  Thus,  alternatives  can  be  identified  and  rated.  As  one  can  see,  the 
learning  goal  is  much  more  than  just  the  acquisition  of  factual  knowledge. 

Multiple  perspectives  should  exist  in  cooperative  settings,  however  a contention  with  divergent  positions  is  not 
inevitably  necessary.  Group  members  can,  for  example,  ignore  existing  diverging  opinions.  Dealing  with 
multiple  perspectives  should  thus  be  anchored  within  the  learning  task  as  well.  The  jigsaw  should  be  an  effective 
way  not  only  to  make  multiple  perspectives  possible,  but  also  to  present  it  as  necessary  for  solving  the  learning 
task. 

2.3  Construction  of  shared  knowledge 

A further  reason  for  the  use  of  CSCL  is  propagated  in  the  situated  learning  approach.  Within  this  context 
learning  is  conceived  as  an  active,  self  co-ordinated,  situated  and  constructive  process. 

Additionally,  learning  is  primarily  considered  as  shared  knowledge  construction  [e.g.  DeDu99].  Cognition  and 
the  acquisition  of  knowledge  are  socially  divided  activities  and  thus  refered  to  the  use  of  co-operative  learning 
scenarios  like  CSCL. 

Indeed,  the  social  context  is  necessary,  however,  it  is  an  insufficient  basis  for  the  construction  of  shared 
knowledge.  Intense  interaction  and  the  construction  of  shared  knowledge  must  be  specifically  promoted.  Using 
the  jigsaw  could  be  one  possibility  for  that  purpose. 

3 Jigsaw  Process 

Jigsaw  is  one  method  which  makes  the  interdependence  of  group  members  possible,  promotes  interaction  and 
cognitive  elaboration,  takes  into  consideration  the  principle  of  the  multiple  perspectives  and  contexts  as  well  as 
the  construction  of  common  knowledge.  In  the  following  the  course  EoM  will  be  used  as  an  example  (see  Fig.  I) 
to  present  the  different  phases  of  the  jigsaw  process  [Clar85,  Clar94]. 

After  the  introduction  of  the  topic,  the  learning  materials  were  divided  into  sections  (phase  1)  and  the  learners 
were  divided  into  interest  groups,  which  independently  compiled  a section  of  the  material  (phase  2).  Thereby 
different  parts  of  the  material  are  covered  by  different  interest  groups.  In  the  module  EoM,  the  theme  ecological 
tax  was  processed  by  three  different  interest  groups,  which  each  had  another  emphasis  as  well  as  a divergent 
view  on  the  problem.  The  basic  positions  were  sketched  up  in  advance. 

The  first  task  was  to  identify  with  the  position  alleged  for  the  group  and  to  collect  arguments  and  pieces  of 
information  to  support  it.  From  these  considerations,  according  to  the  theme  of  ecological  tax  as  well  as  from 
the  view  of  the  respective  group,  a position  paper  should  be  prepared,  in  which  the  point  of  view  of  the  group  is 
written  up  along  with  background  information. 
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To  produce  multiple  perspectives  the  participants  should  likewise  consider  how  the  other  groups  could  argue 
their  perspectives  and  include  this  in  their  discussions.  An  essential  aspect  was  the  construction  of  shared 
knowledge  through  intense  interaction  within  the  group.  Additionally,  cognitive  elaboration  was  necessary  in 
those  areas  were  the  focus  was  on  bringing  together  the  divergent  opinions  for  the  respective  position  paper. 

In  the  third  phase  new  groups  were  formed  based  on  the  chance  principle.  The  scenario  was  an  expert  conference 
initiated  by  the  government.  At  these  „meetings“  in  which  the  different  points  of  view  on  ecological  tax  were 
discussed,  agents  of  all  interest  groups  participated.  For  that  purpose  the  participants  had  to  acquaint  themselves 
with  the  position  papers  of  the  other  interest  groups.  After  that  the  pros  and  cons  of  the  individual  positions  were 
analyzed  and  discussed.  At  the  end  of  the  discussion,  each  group  had  to  find  a consensus  and  outline  a course  of 
action  for  the  future.  In  a written  summary  the  consensus  found,  as  well  as  the  course  of  action  along  with  the 
underlying  arguments  were  to  be  presented. 


The  third  phase  particulary  intensively  included  the  three  implications  mentioned  above.  Here,  taking  multiple 
perspectives  was  imperative.  To  prepare  a common  consensus  paper,  the  participants  had  to  take  the  other 
positions  and  try  to  understand  their  argumentation.  Here  the  cognitive  elaboration  became  more  important  than 
in  phase  2.  The  participants  had  to  explicitly  explain  the  arguments  for  their  positions  in  the  groups.  The  main 
point  is  that  the  possibility  to  give  an  explanation  is  not  bound  to  the  status  and/or  to  the  competence  of  the 
participants.  To  achieve  an  adequate  result,  all  participants  had  to  take  in  turns  both  the  role  of  the  teacher  and 
the  role  of  the  learner.  All  these  processes  required  intense  interaction  and  led  to  the  construction  of  common 
knowledge. 


The  fourth  phase  included  the  evaluation  and  reflection  of  the  results.  For  that  purpose  the  consensus  papers  of 
the  „expert  meetings"  were  made  accessible  to  all  course  participants.  The  participants  were  to  read  the 
consensus  papers  of  the  other  expert  groups  and  to  give  their  opinion  to  the  produced  results.  Additionally,  they 
were  asked  to  reflect  on  the  group  work  itself,  as  well  as  on  the  jointly  compiled  results. 


phase  I:  introduction  & group  forming 

group  1:  „contractors  from  the  area  of  carrier” 
position:  "immediate  withdrawal  of  the  ecological  tax." 

group  2:  „representatives  from  the  government" 
position:  "ecological  tax  is  necessary" 

group  3:  „ecologists" 

position:  "the  ecological  tax  is  not  sufficient” 


A 

A A 
□ 

□□ 

O 

oo 


phase  II:  work  in  the  interest  groups 

task:  „lobby  work" 


O □ 

OO  □□ 


A 

A A 


(interest  groups) 


goal:  creating  a paper  in  order  to  reason  and  explain  the  respective  position 


phase  ni:  work  in  the  expert  groups 

task:  acquainting  with  the  other  positions  and  gaining  a consensus 


□ □ 

AO  AO 


□ 

AO 


(expert  groups) 


goal:  creating  a consensus  paper 


phase  IV:  rating  and  reflection 


Fig.  1 : Phases  in  the  process  of  the  group  task 
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4 Evaluation 

The  empirical  foundation  of  cooperative  learning  methods  is  largely  fragmentary  in  Germany.  Systematic 
comparison  investigations  to  the  employment  possibilities  and  effectiveness  controls  are  not  widely  available 
[GiesOl].  Still  more  deficits  can  be  seen  in  the  research  situation  with  respect  to  the  employment  of  the  jigsaw 
method  in  the  area  of  CSCL.  According  to  Friedrich,  a.o.  [FrHe99,  p.l26]  „ the  suitability  of  this  (...)  can  first 
of  all  only  be  assumed*^ . 

Inasmuch  as  the  module  „EoM“  has  been  introduced  in  a small  scale  up  to  now,  first  locally  in  the  pilot  mode  of 
the  VFH,  the  evaluation  refers  more  to  the  current  status.  The  primary  goal  of  the  formative  evaluation  was  to 
draw  qualitative  statements,  which  refer  concretely  to  improvement  possibilities.  The  main  question  was  how 
deep  the  degree  of  collaboration  was  and  how  the  degree  of  collaboration  was  influenced  by  the  group  method. 

4.1  Results 

Thus,  the  main  question  was  whether  the  group  cooperation  can  be  considered  as  sufficient.  The  majority  of  the 
participants  (53%)  answered  this  question  negatively  and  a lower  value  was  reached.  Therefore  it  seems  that  the 
expectations  for  the  concept  were  not  realized.  On  the  other  hand,  there  were  also  explicit  positive  statements 
like;  Jhe  group  dynamic  inspired  md"  and  the  "'Shared  responsibility  for  the  resulF.  There  were  seen  as  an 
advantage.  It  was  also  mentioned,  that  the,  ,preation  of  the  consensus  paper  as  a conclusion  [...]  made  a lot  of 
furi\  Considering  all  these  facts  the  concept  of  the  division  of  work  was  rated  positively.  This  is  supported  by 
statements  like  „the  result  crystallizes  out  by  putting  all  these  different  pieces  together.  Everybody  helped  to  put 
the  pieces  of  the  puzzle  together  and  the  picture  got  its  shape  from  various  views" . This  statement  emphasized 
the  main  features  of  the  jigsaw  method  like  cognitive  elaboration  and  construction  of  shared  knowledge. 

How  can  this  contradiction  be  explained?  First  of  all,  the  result  regarding  the  cooperation  in  the  groups  must  be 
seen  in  a relative  terms.  Here  different  aspects  play  a role.  One  the  one  hand,  the  degree  of  collaboration  in 
virtual  groups  is  considered  to  be  relatively  low.  Lack  of  cooperation  is  also  dependent  on  the  deficits  in  the 
computer  mediated  communication.  The  evaluation  of  the  other  CSCL  projects  within  the  framework  of  the 
VFH  pilot  phase  produced  results  that  generally  stated  that  the  cooperation  was  perceived  by  the  majority,  in  part 
or  even  by  the  entire  group,  as  not  close  enough.  Thus  the  value  in  the  EoM  module  is  comparatively  high. 

Basically,  there  was  a lack  of  awareness,  i.e.  of  the  mutual  perception  and  based  on  this  of  the  group  cohesion. 
The  production  of  awareness,  tersely  defined  as  „knowing  what  is  going  on“  [Ends95,  p.  36]  is  a problem  that 
has  social  effects  and  also  refers  to  the  responsibilities  of  the  members  and  groups.  In  pure  virtual  group  work, 
basic  social  information  is  often  missing.  This  information  usually  gained  in  the  presence  phase  at  the  beginning 
of  the  course  which  has  at  least  the  implicit  goal  of  getting  to  know  one  another.  Providing  this  information 
likewise  on  a homepage  or  a „wanted  poster“  still  remains  not  to  be  an  optimum  solution.  For  example  there 
was  the  reference:  ,/  would  like  to  know  more  precisely  who  the  other  persons  are.  But  / don  t like  this  to  do 
through  homepages.  I surely  won  t like  to  present  myself  completely  in  the  www." 

After  all,  it  remains  difficult  to  find  optimal  solutions  to  quickly  create  a confident  atmosphere  and  intense  group 
cohesion  that  is  purely  computer  based.  One  aspect  are  the  students  references  that  the  lacking  social  cohesion 
can  possibly  be  met  in  advance  with  a,  “ virtual  getting-to-know-gamd\  Another  possibility  to  make  getting  to 
loiow  each  other  informally  possible  would  be  "by  providing  different  chats  about  different  themes  at  the 
beginning  of  the  course  "".  But  all  together  it  can  be  noted  that  the  students  of  the  pilot  phase  of  the  VFH  did  not 
or  just  hardly  knew  each  other  prior  to  the  group  work.  Thae  were  no  informal  contacts  like  the  ones  they 
developed  during  a longer  regular  course  of  study  at  a virtual  professional  school.  This  aspect  was  also 
emphasised  by  the  participants:  ""One  should  get  the  chance  to  get  to  know  the  coursemates  better  (thus  on-line 
over  a longer  time  period).  To  my  mind  the  social  aspect  plays  (even  under  the  criticism  of  the  " loneliness**  in 
front  of  the  monitor)  a substantial  rold\  However  here  the  opinions  differ.  The  statement  that  one  of  the  central 
advantages  of  CSCL  is  that:  ""Due  to  the  lacking  social  contacts  one  does  not  become  distracted  from  the  actual 
group  work*"  shows  that  the  expectations  for  group  work  are  quite  different.  Thus  it  remains  to  see  whether  and 
how  the  problems  appears  in  the  regular  teaching  of  the  VFH. 

A further  complementary  aspect  that  led  to  a partial  negative  view  of  the  co-operation,  was  the  lack  of  time. 
Processes,  for  example  the  discussion  about  the  common  knowledge  background  - the  so-called  “grounding” 
[ClBr93]  - which  contribute  to  a smooth  co-operation,  require  a larger  time  framework  especially  for  virtual 
groups  where  they  take  place  through  computer  mediated  communication  (cmc).  The  lacking  grounding  was  also 
stated  paticularly  regarding  the  cmc:  “/  think  that  several  clear  formulations  (e.g.  7 do  not  consider  this  as 
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practicable')  mil  be  taken  seriously  through  email  or  they  will  even  be  taken  as  provocative.  The  personal 
verbal  expression,  which  in  Emails  is  usually  transferred  more  casually  than  it  whould  be  the  case  in  formal 
letters,  is  experienced  as  ‘stronger'.  Clear  statements  are  often  experienced  as  provocative  and  responsed 
arrcordingly.  Thus,  unfortunately,  a front  quickly  develops'\ 

The  time  aspect  is  especially  important  for  groups,  because  the  phase  of  optimal  performances,  begins  relatively 
late.  Accordingly,  the  ,^hort  time  frame  provided'  and  the  Jime  pressurd"  were  mentioned  as  difficult  aspects. 

In  conclusion  one  can  say  that,  "in  this  virtual  learning  atmosphere  - although  there  is  a set  of  communication 
aids  available,  - it  is  quite  more  difficult  to  find  a consensus  than  in  a real  course  where  you  can  meet  personally 
for  one  or  two  hours'".  The  intended  processes  of  negotiation  requires  intensive  interaction,  therefore  they  need 
an  adequate  time  framework. 

Another  aspect  that  showed  up  rather  indirectly  was  the  partial  overtax  of  the  participants  with  self-controlled 
learning.  The  participants  have  considered  the  autonomy,  provided  while  dealing  with  the  task,  rather  to  be  a 
lack  of  guidance  than  a chance  for  self-controlled  learning.  Therefore  the  lack  of  "concrete  clues"  was  criticized. 
The  need  for  strong  guidance  becomes  clear  in  the  following  statement:  "Everyone  was  working  on  the  first 
draft- deleting  paragraphs,  adding  new  notes  with  different  coburs  and  asking  and/or  answering  questions 
within  the  same  document.  Thus  the  document  became  very  meaningful,  however  it  was  no  longer  clearly 
arranged.  I would  have  liked  it  better  if  there  had  been  a concrete  rule  for  everybody  which  states  that,  firs  t of 
all,  everyone  has  to  provide  his/  her  own  version/  approach  in  which  the  results  of  his/  her  own  internet- research 
are  presented".  Here  the  adjustments  to  the  learning  process  as  before  appeared  to  be  as  a structured  and  rather 
receptive  process.  Unfortunately,  the  self-controlled  learning  forms  are  usually  less  accepted  than  it  would  be 
didactically  desirable.  In  particular  it  was  shown  that  the  positive  effects  of  the  jigsaw,  in  this  concrete  case  of 
the  shared  knowledge  construction,  were  existent  but  were  also  not  always  perceived  as  helpful. 

4.2  Conclusion 

All  in  all  the  result  is  an  ambivalent  picture.  The  jigsaw  method  is  not  only  feasible  on-line  but  also  as  a 
meaningful  learning  method.  The  advantages  such  as  perspective  change,  intensive  interaction  with  items  of 
cognitive  elaboration  and  co-construction  of  knowledge  were  clearly  recognizable.  These  advantages  were 
mainly  assumed  as  positive  by  the  students. 

A principal  requirement  for  successful  CSCL  is  accurate  scheduling  and  precise  formulation  of  the  task.  In 
addition  it  is  also  useful  to  offer  enough  material  and  sources  for  background  information. 

Another  condition  for  the  effective  realization  of  the  jigsaw  is  the  consideration  of  the  basic  conditions.  Like 
each  good  learning  method  the  jigsaw  is  not  successful  by  itself.  Three  basic  conditions  played  a substantial  role 
in  the  case  study. 

One  focal  point  were  the  group  cohesion  and  the  social  awareness  as  central  items  of  the  procedure.  These  can 
be  threated  in  different  ways.  On  the  one  hand,  one  could  execute  a presence  phase  before  the  CSCL  that  helps 
to  overcome  the  communication  barriers  and  thus  relieve  the  group  identification. 

On  the  other  hand,  with  purely  virtual  group  work  it  should  be  considered  how  implemented  awareness 
supporting  tools  can  facilitate  the  social,  spontaneous  and  informal  interaction  and  thus  improve  the  cohesion  in 
the  group. 

In  addition  to  form  groups  and  to  perform  the  relatively  complex  task  a larger  time  framework  is  required  for 
communication,  co-operation  and  co-ordination. 

A final  aspect,  which  should  be  considered  likewise  more  specifically  for  the  realisation  of  jigsaw,  includes  the 
individual  capabilities  and  competences  of  the  learners.  Experience  and  knowledge  of  the  students  taking  courses 
in  further  education  are  very  heterogeneous.  Collaborative  learning  also  covers  processes  of  self-  controlled  and 
self-  organized  learning.  These  abilities  are  differently  distributed.  There  are  different  learning  types  for  which 
CSCL  is  well  suited  however  theie  are  other  for  which  is  less  suited.  Finally  it  can  be  considered  to  implement  a 
graded  tutoring  and  structuring  quasi  “on  demand”. 

Therefore  the  jigsaw  is  a very  good  method  to  initiate  positive  processes  of  collaborative  learning  in  the  CSCL. 
This  also  showed  up  in  the  case  study.  As  it  is  the  case  for  all  complex  learning  forms,  the  organization  and 
counterbalancing  of  the  multiple  basic  conditions  and  factors,  remain  an  iterative  process  of  optimization.  This 
offers  further  opportunities  for  empirical  research,  which  may  concern  the  concrete  learning  success  needed. 
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Abstract 

The  New  Four  R’s:  Reading,  Writing,  Arithmetic  and  the  Internet -Teaching  Teachers  to  develop  Web-Based  Units  of 
Instruction  is  a case  study  examining  an  online  secondary  methods  of  instruction  course  for  post  baccalaureates  which 
requires  students  to  develop  web-based  unit  plans.  The  authors  posit  that  in  order  for  teachers  to  effectively  teach  with 
technology  it  must  be  integrated  into  their  teacher  preparation  course  work  by  modeling  effective  technology 
integration  into  their  teaching  and  assignments. 

Introduction 

Educators  will  be  charged  with  preparing  students  to  be  successful  citizens  and  members  of  a society  that  is 
rapidly  being  transformed  by  technology.  All  of  the  major  fields  in  the  world  of  work  such  as  engineering, 
medicine,  transportation,  manufacturing,  and  entertainment,  the  government  and  the  military  have 
embraced  technology  in  order  to  stay  current — so  must  education  (The  Milken  Foundation,  2001).  Yet 
education  preparation  programs  are  being  charged  with  not  adequately  preparing  preservice  teachers  with 
adequate  technological  and  pedagogical  skills  to  prepare  students  to  usefully  implement  technology. 

A study  of  teacher  education  programs  initiated  by  the  Milken  Exchange  on  Educational  Technology  and 
carried  out  by  ISTE  suggested  that  “these  programs  should  increase  teachers’  exposure  to  appropriate 
technology  if  they  are  to  aptly  prepare  them  for  today’s  classrooms”(Moursund,  1998). As  a professor,  in  a 
web-based  distance  education  program,  I work  with  students  who  have  degrees  and  are  seeking 
certification  in  teacher  education  through  distance  education.  These  nontraditional  students  are  teaching  on 
emergency  certification  or  working  in  other  jobs  preparing  to  transition  into  teaching.  Within  these  same 
classes,  I also  worked  with  students  who  have  varying  degrees  of  experience  as  teachers  and  are  pursuing 
graduate  degrees  in  education. 

I chose  as  our  text  of  study  How  People  Learn:  Brain,  Mind,  Experience,  and  School  (Brans ford,  et  al 
2000),  which  is  a compendium  of  research  on  people  learn  and  subsequently  how  educators  can  teach  to 
build  upon  what  then  know  about  how  people  learn.  Bransford  et  al  posit  that  there  are  four  critical 
ingredients  to  learning  environments,  and  those  are  that  they  must  be:  learner-centered,  knowledge- 
centered,  assessment-centered,  and  community-centered. 

As  1 approached  the  design  of  Advanced  Methods  of  Instruction,  one  of  the  foundation  courses  in  teacher 
certification,  I looked  at  the  elements  traditionally  taught  as  part  of  the  course:  how  to  develop  a syllabus, 
how  to  develop  goals  how  to  write  objectives,  how  to  plan  units  of  study,  and  how  to  write  daily  lesson 
plans.  I asked  myself  how  I could  take  these  elements  and  apply  them  pragmatically  to  a twenty-first 
century  classroom?  I took  as  a basic  element  of  a twenty-first  century  classroom,  access  and  connection  to 
the  Internet,  and  asked  my  students  to  develop  web-based  unit  plans  which  would  incorporate  the  four 
elements  of  an  optimal  learning  environment.  It  was  important  to  be  that  these  be  value  added — that  there 
be  some  purpose  for  them  to  be  web-based  which  would  enhance  the  learning  environment. 

Methodology 

I chose  to  examine  the  following  sources  for  data:  my  syllabus,  online  lessons,  student  responses  on 
threaded  message  boards  and  student  e-mail  which  referenced  this  assignment.  In  addition  we  examined  the 
web-based  unit  plans.  And  finally  we  analyzed  student  assessments  of  their  own  work  to  ascertain  how  they 
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saw  themselves  fulfilling  the  requirements  of  the  assignment.  Then  we  triangulated  the  data  by  comparing 
the  results  of  the  analysis  with  each  other  and  the  two  students  from  the  case  studies.  The  questions  that 
guided  my  inquiry  were:  How  did  I as  an  educator  of  pre  and  in  service  teachers  use  technology  to  model 
the  four  elements  of  an  optimal  learning  environment-learner,  knowledge,  assessment,  and  community 
centeredness — while  charging  the  students  with  the  assignment  of  integrating  technology  into  their  own 
curriculum  design?  Did  the  web-based  student  unit  plans  use  technology  to  incorporate  the  four  elements  of 
optimal  learning?  Were  the  web-based  student  unit  plans  value  added? 

The  research  design  for  this  study  is  an  action  research  project  which  I examine  two  case  studies  chosen 
Four  different  kinds  of  data  were  analyzed  (document  review,  threaded  message  boards,  o-mails,  web- 
based  unit  plans  and  interviews).  This  writing  is  an  intermediate  look  at  the  study.. 

Data  Analysis  and  Discussion 

A detailed  discussion  of  the  data  analysis  can  be  obtained  my  e-mailing  the  author  of  this  paper. 
Conclusions  and  Implications: 

As  a result  of  this  study,  I found  that  I was  able  to  integrate  traditional  elements  of  the  unit  planning  into  a 
technology -enhanced  curriculum  after  modeling  technology  integration  in  my  own  lessons  with  students, 
and  through  requiring  students  to  participate  in  technology-enhanced  lessons,  as  well  as  charging  them  to 
create  a technology-enhanced  lesson  plan.  I found  several  student  developed  “model”  technology- 
enhanced  unit  plans,  but  also,  had  nany  that  fell  short  of  the  mark,  by  just  putting  a unit  plan  which 
required  little  or  no  technology  on  the  Internet.  I chose  two  case  studies  from  the  “model”  unit  plans  to 
indicate  what  optimally  could  be  done.  The  only  conclusion  I can  draw  as  to  why  students  did  not  utilize 
technology  is  hypothetical  at  this  point  and  not  data  driven.  In  speaking  informally  with  students,  I found  it 
seems  that  students  could  not  conceptualize  that  classrooms  they  teach  in  would  have  access  to  basic 
technology  such  as  the  Internet,  or  would  have  computer  in  the  classroom  or  nearby  accessible  for  student 
use.  And  in  my  own  thinking,  I think  that  students  without  a lot  of  schema  for  lesson  planning,  and/or  with 
limited  technology  backgrounds  could  do  just  enough  to  accomplish  a traditional  lesson  plan  posted  on 
their  web  page.  I also  found  that  while  I felt  I explicitly  taught  and  modeled  with  technology  integration 
the  four  elements  of  learning — student-centered,  knowledge-centered,  assessment-centered,  and 
community-centered  and  required  students  to  have  that  focus  in  their  lessons,  many  fell  short  of  that  mark 
and  relied,  again  on  traditional  lesson  planning  without  the  four  elements  of  learning  and  minimal 
technology  integration. 

When  a professor  in  a teacher  preparation  course  models  the  integration  of  technology  as  well  as  requires 
students  to  integrate  technology  into  their  course  work,  these  preservice  or  inservice  teachers  can  do  this 
successfully.  But  in  order  to  make  technology  an  integral  part  of  curriculum,  it  mist  be  modeled  in  teacher 
education  classes..  Based  on  the  findings  of  this  study,  1 suggest  that  traditional  components  of  curriculum, 
unit,  and  lesson  design  can  be  incorporated  in  a technology-enriched  assignment.  Enhancing  curriculum 
with  technology  does  not  have  to  be  the  responsibility  of  the  technology  educators.  Rather,  it  can  and  in 
some  cases  should  become  the  responsibility  of  teacher  education  professors  and  curriculum. 

References 

Bransford,  J.  Brown,  A.  and  R.  Cocking.  (2000)  How  People  Learn:  Brain,  Mind,  Experience, 
and  School.  National  Academy  Press:  Washington,  D.C. 

Moursan,  D.  & T.  Belefeldt.  (1999)  Will  Teachers  be  Preapared  to  Teach  in  a Digital  Age.  The 
International  Society  for  Technology  in  Education:  Eugene,  Oregon  and  Milken  Exchange  on  Education 
Technology:  Santa  Monica:  California. 

The  Milken  Foundation  (2001).  Information  Technology  Underused  in  Teacher  Education. 
http://www.mff.org/edtech/article.taf?  fiinction=detail&Content  uid  1 = 131. 


834 


The  Design  and  Development  of  an  Electronic  Performance  Support 
System  for  University  Professors  in  the  Field  of  Biomedical  Science  and 

Biotechnology  in  Taiwan 


Li-An  Ho 

Department  of  Educational  Technology 
Tamkang  University 
Taiwan,  R.O.C. 
lianho@mail.tku.edu.tw 


Abstract;  This  project  plans  to  design  and  develop  an  electronic  aid  in  collaboration  with  a 
national  university  (NU)  in  Taiwan.  Using  the  concept  of  knowledge  management  and 
computer-mediated  learning  theories  and  strategies,  this  electronic  ystem  takes  the  form  of 
electronic  performance  support  system,  which  aims  to  provide  university  professors  a 
framework  for  1)  effective  administration  of  data  involving  teaching,  2)  better  organization  and 
management  of  the  subject  matter  knowledge  and  information,  3)  successful  adoption  and 
application  of  technological  means  to  transform  a traditional  classroom  teaching  method,  4) 
opening  multiple  channels  of  communication  between  teachers  and  students,  and  5)  amplifying 
students’  motivation  of  learning. 


Background  Introduction 

As  society  moves  into  the  information  age,  changes  must  be  made  in  the  educational  process  to  ensure 
that  students  will  have  the  skills  they  need  in  the  changing  workplace.  Soon  after  Electronic  Performance 
Support  Systems  (EPSS)  gained  recognition  in  the  corporate  setting,  many  educational  professionals  and 
researchers  began  exploring  the  potential  advantages  of  adopting  computer  technologies  in  different  learning 
environments.  As  Scales  (1994)  points  out,  there  are  two  trends  in  the  educational  reform  movement  that 
prepare  students  to  keep  up  with  the  changes  in  the  society.  They  are  integrating  electronic  technology  in  the 
classroom  and  using  EPSS  as  a tool  for  promoting  training  and  support  in  education  and  in  the  workplace. 

Thus  EPSS  gradually  becomes  visible  and  applicable  for  instructional  purposes  is  an  inevitable  trend. 
However,  I found  that  it  has  not  been  recognized  and  correctly  implemented  at  most  universities  in  Taiwan. 
This  project  sought  potential  partnership  in  universities  where  an  EPSS  could  help  professors  utilize  existing 
resources  and  improve  instructional  effects  for  greater  teaching  and  learning  outcome. 


Project  Description 

This  new  system  aims  to  provide  a framework  to  facilitate  both  teaching  and  learning.  This  electronic 
framework  is  presented  through  an  electronic  performance  support  system  (EPSS).  The  system  is  expected  to 
raise  the  recognition  of  the  use  of  EPSS  in  educational  settings,  and  to  introduce  a new  perspective  of  teaching, 
learning,  and  instructions.  The  goal  of  this  project  is  to  design  and  develop  an  electronic  aid  in  collaboration 
with  professors  at  the  department  of  Applied  Chemistry  at  National  University  (UN,  pseudonym)  in  Taiwan.  By 
adopting  the  concept  of  knowledge  management  and  those  theories  and  strategies  of  computer-mediated 
learning,  this  new  EPSS  will  provide  professors  effective  tools  and  means  to:  1)  effectively  administrate  data 
involving  teaching,  2)  effectively  organize  and  manage  subject  matter  knowledge  or  content  information  for 
anticipated  learning  outcome,  3)  successfully  adopt  and  effectively  employ  technological  means  to  transform  the 
traditional  classroom  teaching/leaming  format  to  a new  computer-facilitated  instructional  environment,  4) 
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increase  the  number  of  communication  cannels  between  teachers  and  students,  and  5)  amplify  students' 
motivation  of  learning  by  using  electronic  means. 

This  EPSS  has  been  constructed  based  on  the  needs  of  the  professors  and  students  at  the  Department 
of  Applied  Chemistry  atNU.  The  new  system  has  been  integrated  with  functions,  information  and  knowledge 
structure,  and  instructional  activities  to  supplement  their  regular  teaching  and  learning  routines.  The  new 
system  was  expected  to  incorporate  a Web  function  so  that  the  distance  and  scheduling  barriers,  which  often 
occur  in  the  context,  could  be  overcome  between  professors  and  students. 

The  new  system  incorporates  two  sub-systems:  one  for  the  professors  and  the  other  for  the  students 
(Fig.  1:  system  functional  framework).  Both  professors  and  students  will  need  authorizations  to  log  onto  their 
designated  menu  pages.  For  professors,  the  Recourse  Center,  so  far,  has  three  functions:  a posting  board  of 
the  departmental  information,  a grade  template  for  student  data,  and  a public  forum  for  daily  conversations. 
The  Student  Menu  page  contains  seven  components.  They  are:  1)  a news  board  for  posting  most  recent 
course  update,  2)  a course  syllabus  containing  course  information  and  schedule,  3)  problem-based  learning 
section  allowing  professors  to  post  questions  or  project  requirement  for  stimulating  students'  thinking,  4)  course 
material  based  on  course  calendar,  5)  a discussion  forum  promoting  out-of-class  communications  among  all 
parties,  6)  online  tests,  and  7)  a system  functional  descriptions  served  as  system’s  online  help.  Up  to  date  this 
new  EPSS  is  still  under  construction.  The  system  framework  might  be  changed  based  on  users’  feedback 
generated  through  the  iterative  design-evaluation-analysis  cycle. 


Figure  1 : System  Functional  Framework. 
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Abstract:  The  emerging  National  Science  Math  Engineering  and  Technology  (SMET) 
Education  Digital  Library,  which  is  being  constructed  by  a variety  of  educational  institutions, 
is  an  "online  network  of  learning  environments  and  resources  for  SMET  education  at  all 
levels"  (NSF,  2000).  It  is  envisioned  as  a premier  portal  to  a rich  array  of  current  and  future 
high-quality  educational  content  and  services,  and  a forum  where  resource  users  may  become 
resource  providers.  Technical,  social,  and  economic  issues  involved  in  developing  the  library 
are  reviewed,  with  emphasis  on  implications  for  teacher  education  and  instructional  support. 


Introduction 

The  National  Science,  Mathematics,  Engineering,  and  Technology  Education  Digital  Library  (NSDL) 
is  an  educational  vision  with  initial  funding  from  the  National  Science  Foundation  (NSF) —a  vision  that  is 
different  from  any  existing  library  today.  Projects  led  by  universities,  K-12  entities,  professional  associations, 
government  agencies,  non-profit  organizations,  and  corporations  are  part  of  the  Library’s  realization.  The 
NSDL  will  be  the  culmination  of  over  a decade  of  research  and  collaboration  among  librarians,  technologists, 
and  educators  in  the  public  and  private  sectors  (Wattenberg,  1998). 

Recommendations  for  a national  digital  library  for  science,  mathematics,  engineering,  and  technology 
education  (SMETE)  have  focused  on  the  need  to  define  and  interconnect  user  communities,  establish 
collections  and  services,  and  develop  tools  that  focus  on  learning  and  teaching  needs. 

The  Internet  Paradox:  Is  More  Less? 

High-quality  content  can  be  difficult  and  time  consuming  to  locate  on  the  Internet  despite  the  increase 
in  sites  that  attempt  to  provide  searchable,  usable  materials  for  teachers  and  students.  Digital  library 
development  must  go  beyond  simply  providing  selected  links  on  specific  topics.  Critical  barriers  for  digital 
libraries  today  are  as  much  human  issues  as  technological  ones.  Even  when  good  collections  are  developed, 
getting  them  known  and  used  remains  problematic  in  this  age  of  Internet  information  overload  (Zia,  2001). 

The  NSDL  has  the  potential  to  encourage  and  support  inquiry-driven,  collaborative  learning  while 
incorporating  distributed  architecture  to  provide  stability,  reliability,  and  smooth  interoperability.  The  NSDL 
has  the  possibility  of  allowing  pre-service  and  in-service  teachers  to  bring  innovative  tools  and  practices  into 
their  classroom  in  a supported  and  effective  manner. 

To  some  degree,  the  Library’s  debut  is  premised  on  the  concept  that  given  the  right  funding  and 
organizational  structure,  technical  solutions  are  either  in  existence  but  not  yet  well  placed  for  optimal  use  and 
interoperability  or  can  be  developed  within  known  frameworks  in  a relatively  short  timeframe. 
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A Virtual  Library  Vision 


A prime  consideration  in  building  digital  libraries  is  meeting  users’  unique  needs.  Therefore,  the 
NSDL  will  have  to  provide  value-added  services  that  enhance  the  user  experience;  users  must  have  a 
compelling  reason  to  visit  the  Library  rather  than  to  use  public  search  engines. 

The  NSDL  intends  to  support  pre-service  and  practicing  teachers  through  its  commitment  to  assuring 
resource  quality,  supporting  contextual  learning,  fostering  critical  literacy  skills,  acknowledging  access 
inequities,  and  advancing  scientific  understanding  with  many  resources  and  services  beyond  traditional  books 
and  journals,  including  non-textual  sources,  lesson  plans,  curriculum  guidelines,  and  tutorials. 

While  NSDL’s  focus  is  on  SMET  education  content,  services,  infrastructure,  and  community,  the  tools 
and  techniques  that  are  developed  and  tested  in  this  program  will  apply  to  other  virtual  library  development. 
The  NSF  effort  is  seen  as  a prototype  that  can  lead  to  further  deployment  in  fields.  In  addition,  NSF  is 
developing  international  linkages  that  have  the  potential  to  create  global  resources  for  SMET  education. 

Progress  and  Achievements 

The  NSDL  efforts  began  with  projects  focused  on  development  and  deployment.  Projects  are  grouped 
into  four  tracks:  Core  Integration,  Collections,  Services,  and  Targeted  Research.  Universities  and  professional 
societies  lead  most  efforts.  All  projects  represent  more  than  one  institution  and  many  have  K-12  linkages. 
Project  abstracts  and  links  to  project  sites  can  be  found  at  http://www.nsdl.nsf.gov/. 

Key  organizational  concepts  that  guide  NSDL  progress  are  its  distributed  nature,  the  need  for  active 
governance  that  involves  all  stakeholders,  a variable  collections  structure  that  includes  those  following 
prescribed  requirements,  harvested  through  targeted  selection,  and  gathered  with  smart  agents.  The  NSDL  also 
prioritizes  the  development  of  a technical  infrastructure  that  will  support  multiple  portals  for  entry,  and  the  use 
of  multiple  metadata  standards  to  meet  the  diverse  needs  of  the  different  disciplines  and  users. 

Future  Challenges  and  Considerations 

Traditional  roles  for  librarians  and  teachers  are  blurring,  and  the  skills  of  each  are  expanding. 
Certainly,  then,  there  are  implications  for  pre-service  preparation  and  continued  professional  development.  To 
maximize  its  effective,  the  NSDL  will  need  to  go  beyond  an  awareness  campaign. 

New  learning  resources  bring  opportunities  to  reuse  and  repurpose  digital  learning  objects.  New  media 
types,  such  as  digitized  audio  and  video  collections  and  data  sets,  are  also  becoming  available.  Recognizing  the 
opportunities  in  this  area,  NSF  and  the  Institute  for  Museum  and  Library  Services  (IMLS),  have  begun  to 
explore  ways  in  which  library  and  museum  holdings  may  contribute  to  new  curricular  materials  (Zia,  2001). 

Finally,  evaluation  in  its  many  forms  will  be  increasingly  important  as  the  NSDL  continues  to  grow. 
The  NSF  intends  for  the  NSDL  to  be  a community-building  resource  for  students  and  teachers;  significant 
involvement  of  the  country’s  teachers  and  librarians  will  be  vital  to  achieving  a strong  program  and  a thriving 
community. 
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Abstract  This  paper  presents  the  structure  of  a system  that  supports  interactively  teaching 
on  multimedia  material.  The  system  has  many  advantages  with  respect  to  e xisting  systems: 
it  is  not  proprietary,  and  allows  portable  data  on  many  operating  systems.  It  supports 
communication  over  networks,  both  synchronous  and  asynchronous.  It  allows  students  to 
deposit  information  and  comments  into  the  system  itself.  It  has  strong  support  for 
reusability  of  the  teaching  material.  It  also  permits  navigation  in  the  material  at  the 
metalevel  of  topic  maps.  A critical  review  on  what  has  been  done  and  what  still  has  to  be 
done  is  also  carried  out. 

1.  Introduction 

The  development  of  Computer  Based  Learning  systems  has  a long  history,  which  dates 
back  to  the  late  fifties  [Cro59].  At  first,  such  systems  were  command- line  or  menu  based,  and 
allowed  limited  interaction  to  the  user.  Users  were  allowed  to  read  some  text  material,  and 
automatic  tests  could  be  performed  on  the  progress  of  learning.  Such  systems  were  very 
limited  in  scope  and  had  little  advantage  with  respect  to  traditionally  learning  based  on  books. 

With  the  development  of  graphical  user  interfaces  (Mac,  Windows,  Amiga,. . .)  and  the  rise 
of  authoring  tools  like  hypercard  [Goo87],  the  presentation  of  learning  material  got  more 
complex,  allowing  the  insertion  of  pictures,  animations,  video  and  audio  elements  and 
interactive  elements  to  the  teaching  material.  The  more  variety  in  the  possibilities  of  the 
systems,  the  more  complex  the  system  became,  and  therefore  the  more  expensive  to  develop. 
Today,  the  wide  availability  of  internet  connection  and  multimedia  capable  computers  has 
brought  into  computer  based  teaching  software  yet  another  new  level  of  complexity.  Both 
teachers  and  pupils  can  work  on  the  material  independently  from  location,  in  a synchronous  or 
asynchronous  way,  and  can  communicate  online  whenever  problems  arise.  Not  only,  but  the 
distribution  of  content  to  a wider  audience  and  the  possibility  of  distributed  real  time 
collaboration  raise  new  questions,  both  from  the  computer  science  point  of  view,  due  to  the 
necessary  management  of  large  amounts  of  text  and  multimedia  material,  as  well  as  for  the 
effectiveness  of  inter- human  communication  over  networks.  Moreover,  the  amount  of 
information  present  on  the  network  makes  it  difficult  to  organize,  maintain  and  search  through 
the  material. 

Together  with  the  University  of  Applied  Sciences  in  Bielefeld,  the  University  of  Bielefeld, 
the  School  of  Film  and  Television  (HFF)  Konrad  Wolf  in  Potsdam  and  the  School  of  Arts  in 
Berlin  (HdK),  the  Bauhaus  -University  Weimar  has  decided  to  offer  a study  course  in  Media 
enrolling  online  students.  Moreover,  also  “regular”  students  of  all  five  institutions  should  be 
allowed  to  use  the  material  for  their  study  courses. 

The  main  goal  of  the  project  is  to  develop  a system,  a framework  for  teaching  and  for 
content  material.  The  material  is  stored  in  the  framework  itself,  permitting  students  of  the 
faculties  involved  to  learn  material  at  the  undergraduate  level  using  the  computer.  As  a first 
step,  traditional  physical  presence  based  teaching  will  be  expanded  by  allowing  students  to 
use  the  system.  Once  completed,  the  system  will  support  the  use  of  computers  in  three 
different  scenarios: 

• By  a tutor  in  a lecture,  live  in  the  classroom 


• By  a student,  synchronously  or  asynchronously,  at  home  and  complementary  to 
presence  based  teaching 

• By  a student  in  distant  education,  as  sole  connection  means  to  the  curriculum 

It  is  well  known  that  one  third  of  the  knowledge  built  during  University  studies  is  derived 
from  exchange  of  information  among  classmates.  The  system  allows  therefore  students  to 
make  annotations  on  texts,  to  integrate  and  develop  new  material,  and  provides  control 
mechanisms  for  allowing  such  student  generated  content  to  be  fed  back  into  the  teaching 
system  itself 

To  facilitate  searching  in  the  material,  and  to  support  alternative  navigation  paths  through 
the  material,  a meta  layer  based  on  topic  maps  [Bie99]  is  laid  above  the  basic  contents.  The 
user  will  be  able  to  navigate  through  the  abstract  mesh  of  the  material  freely,  almost  following 
its  quest  for  information.  Extended  usage  of  topic  maps  is  still  at  its  infancy,  and  looks  a 
promising  technology  since  it  allows  a flexibility  which  is  barely  available  when  crawling 
through  the  material  itself 

The  Bauhaus-University  Weimar  has  already  developed  interactive  courseware  for  Civil 
Engineering,  [HofSchOl,  HofSchOO].  The  new  system  bases  on  the  strengths  cf  the  previous 
system  called  Mambo,  which  was  a test  bed  for  the  use  of  multimedia  in  distant  education. 
Focus  is  now  set  not  only  on  reusability  of  content,  which  improves  the  cost  factor,  but  more 
on  user  centred  design  of  the  framework,  especially  when  it  comes  to  Computer  supported 
collaborative  working  (CSCW).  Also  new  are  the  integration  of  topic  maps,  CSCW  and 
communication  elements  through  the  system. 

2.  Description  of  the  System 

2.1  Main  systems  features 

Based  on  an  overview  of  state  of  the  art  learning  platforms  in  the  second  half  of  2000 
[Com01,Com01a],  no  XML-based,  cross  media  publishing  capable  system  was  then  available, 
which  would  have  allowed  to  add  the  features  needed  for  the  goals  listed  in  the  last  section. 

As  a result,  it  was  decided  to  implement  such  a system  around  existing,  commercial  content 
management  systems,  allowing  to  concentrate  on  the  features  missed  and  at  the  same  time  aim 
for  professional  features  like  scalability  and  security  (locking,  roll-back,  and  so  on).  The  use 
and  application  of  open  standards  wherever  possible  permits  content  portability  to  other 
systems. 

Figure  1 shows  a schematic  structure  of  the  systems  architecture  and  functionality.  The 
system  is  built  around  two  commercial  software  systems  residing  on  a server  (centre  of  Fig.  1) 
and  extended  using  self  developed  or  adapted  software. 

Next  we  describe  the  elements  and  their  functionality  of  the  system  with  the  help  of  three 
use  cases  - content  creator,  teacher  and  student. 

2.2  Content  creator  role 

The  left  side  of  Figure  1 shows  the  authoring  clients,  where  content  creators  (authors  and 
media  producers)  access  the  content  stored  and  managed  on  the  servers.  The  check  in/out 
process  is  simple  - the  users  check  out  text  or  binaries  using  an  editor  or  a media  client,  which 
locks  access  to  these  elements  for  other  users. 
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Figure  1 : Scheme  of  the  frameworks  functionality 


These  clients  are  connected  via  TCP/IP  and  authorized  through  an  LDAP  system  to  access 
the  Server,  on  which  two  management  systems  are  responsible  for  the  content.  The  Content 
Management  System  (CMS)  used  is  “Sorman  CMS”[Sor01].  Its  role  is  to  take  care  of  all 
XML-based  content  (text,  tables,  equations,  and  the  structure  of  the  content).  The  second 
system  used  is  an  Asset  Management  System  (AMS),  which  is  used  to  store  raw  data  (binary 
data,  such  as  pictures,  video  sequences,  animations  and  audio  data).  We  chose  to  use 
“Cumulus  5 Enterprise  Edition”  by  Canto  [CanOl]  for  this  task. 

Content  is  created  either  by  teachers  with  experience  in  the  production  of  multimedia 
material  or  by  professionals,  which  work  as  contractors  for  the  teachers.  Together,  they 
develop  the  media  units  to  be  implemented,  thus  meeting  both  didactical  and  technical 
requirements. 

All  primary  content  has  to  be  created  adhering  to  existing  standards  and  well-documented 
file  formats  as  far  as  possible.  For  text  and  structuring  this  means  XML,  for  the  binaries 
proprietary  formats  are  avoided  as  much  as  possible.  This  is  supported  by  the  need  of  client 
side  operating  system  independency  — many  of  the  proprietary  file  formats  cannot  be  viewed 
or  played  on  machines  using  Linux  or  Solaris  as  operating  system. 

2.3  Teacher  role 


In  case  of  a new  course  to  be  created,  a teacher  usually  does  most  of  the  writing  and  mark- 
up of  text  himself  This  is  different  for  multimedia  production,  where  expert  media 
professionals  have  to  support  the  teacher  both  by  suggesting  the  media  which  are  going  to  be 
used  for  the  purpose  as  well  as  by  checking  the  compatibility  of  the  teacher’s  desires  with  the 
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specifications  concerning  usability  and  reusability.  This  includes  applied  technologies  (file 
formats  etc.)  as  well  as  font  size,  colour  schemes,  line  widths  etc.  ensuring  usability  with 
different  output  devices. 

As  more  and  more  material  is  stored  in  the  system,  it  will  become  possible  to  reuse  text  or 
media  elements  from  other  users.  We  believe  that  this  is  the  only  economically  feasible  way 
of  handling  the  use  of  computers  in  education,  since  otherwise  the  production  cost  of  quality 
material  is  too  high.  All  material  set  in  the  system  gets  semi-automatically  indexed  during 
insertion.  Metadata  is  then  added  to  the  data  inserted.  Thus,  the  topic  map  of  the  new  material 
can  be  generated  in  an  easy  way. 

2.4  Student  role 

Basically,  student  work  is  divided  into  three  different  fields:  learning  with  course  material, 
communication  and  creation  of  new  content. 

The  learning  with  course  materials  is  done  via  a client  computer,  from  where  the  student 
logs  into  the  system  and  participates  in  courses  he  has  signed  up  for.  The  planned  structure  of 
the  courses  will  make  it  possible  to  have  the  student  pass  simple  tests  before  he  can  access 
further  material,  if  desired  by  the  teacher.  Additionally,  obligatory  participation  in  chats, 
discussions  or  simple  tests  can  be  added  to  the  curriculum  of  a course. 

In  cases  where  people  are  performing  goal  oriented  searches  in  the  system  and  are  not 
following  the  course,  topic  maps  can  be  used  for  navigation.  Topic  maps  are  abstract  meshes 
of  “entities”  which  one  might  think  of  as  terms,  concepts  or  objects,  and  meaningful 
connections  between  them.  These  topic  maps  do  not  contain  learning  material  themselves,  but 
can  link  to  them  using  “occurrence”  links  pointing  towards  the  object.  One  can  imagine  topic 
maps  to  be  a meta  layer  above  the  course  material.  Once  a student  has  problems 
understanding  the  meaning  of  the  courseware  he  is  following,  he  is  allowed  to  leave  the 
course  structure  “upwards”  to  the  topic  map  and  reorient  himself  there  by  locating  his  position 
within  the  map.  He  might  then  go  to  another  connected  topic  and  from  there  back  to  the 
course  material  to  continue  “grazing”  on  the  material. 

Communication  is  needed  at  different  levels,  depending  on  the  teaching  methods  being 
used.  Tutoring  via  e-mail  is  provided  throughout  all  courses,  as  well  as  discussion  forums  and 
chat  lines,  and  videoconferencing. 

Also,  means  are  provided  for  inserting  the  new  material  generated  by  communication  into 
the  system.  Students  are  also  encouraged  to  insert  homework,  or  seminar  work,  or  comments 
into  the  system  itself  The  first  purpose  of  this  practice  is  to  allow  the  student  to  present  his 
results  in  front  of  others,  to  discuss  them  and  finally  to  improve  his  knowledge  and  share  it 
with  others.  Some  of  the  student  material  can  be  used  by  the  teachers  themselves,  once  it  has 
been  screened  by  an  expert  ensuring  its  quality.  This  allows  a more  rapid  growth  of  the 
material  than  it  would  be  possible  when  done  only  by  the  teachers. 

2.5  Integrating  commercial  software 

The  two  commercial  systems  employed  are  responsible  for  managing  text,  binary  data,  and 
topic  maps.  They  provide  the  following  advantages: 

• Both  systems  come  with  built  in  mechanisms  protecting  users  from  inadvertently 
deleting  data  by  providing  so  called  “roll  back”  functionality,  which,  like  an  undo 
function,  allows  to  go  back  to  an  earlier  state  of  the  work.  Furthermore,  both  systems 
proved  to  scale  to  a larger  number  of  concurrent  users  working  on  content  at  the  same 
time,  which  is  important  for  daily  use  in  a University. 
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• Both  systems  are  accessible  on  a programming  level  to  adapt  them  to  additional  needs. 
Existing  APIs  provide  functionality  which  can  be  used  to  “glue”  other,  self-developed 
programs  to  them. 

• Experience  [HofSchOO]  has  shown  that  two  main  workflows  can  be  differentiated,  one 
concerning  text  and  ordering  of  content,  the  other  concerning  the  creation  of  binaries 
like  pictures,  animations,  films,  or  interactive  programs.  The  separation  of  these 
workflows  is  sometimes  difficult,  as  most  authors  are  not  used  to  outsource  the 
production  of  binaries,  but  can  be  supported  by  dedicated  management  systems. 
Especially  with  binaries,  it  is  necessary  to  have  good  support  for  describing,  searching 
and  viewing  to  reduce  production  time  and  cost. 

These  systems  serve  as  a query-able  backend  and  repository  for  the  data  generated  by  the 
teachers  and  the  students. 

3.  Conclusions  and  future  work 

We  have  presented  a system  structure  for  a system  that  supports  teaching  interactively  with 
multimedia  material.  The  system  has  many  advantages  with  respect  to  existing  systems:  it  is 
not  proprietary,  and  allows  portable  data  on  many  operating  systems.  It  supports 
communication  over  networks,  both  synchronous  and  asynchronous.  It  allows  students  to 
store  information  and  comments  into  the  system  itself.  It  has  strong  support  for  reusability  of 
the  teaching  material.  It  also  permits  navigation  in  the  material  at  the  meta  level  of  topic 
maps. 

The  current  implementation  state  is  the  following:  the  CMS  and  AMS  are  running  and 
provide  the  basic  functionality  of  the  system.  The  development  of  a first  prototype  for  the 
underlying  multimedia  data  types  has  been  finished.  The  first  document  type  definition  has 
been  drafted.  The  first  integrated  prototype  will  undergo  a first  evaluation  at  the  end  of 
February  2002.  Courseware  is  being  developed  while  regular  courses  are  taking  place:  an 
analysis  of  the  teaching  methods  in  the  different  course  types  is  being  done. 

It  proved  a real  challenge  to  develop  a common  language  between  extremes  such  as 
engineering,  the  humanities  (specifically  cultural  studies)  and  fine  arts.  Studying  media 
involves  competences  from  the  arts,  from  the  humanities,  from  economy  and  from 
technology.  Especially  when  it  comes  to  teaching  methods  and  ways  to  formalize  different 
approaches,  often  different  worlds  collide.  Obviously,  people  from  such  different  fields  of 
work  have  quite  different  understandings  of  science,  work,  and  especially  teaching.  Such 
differences  pop  up  at  each  step  in  the  conception  phase,  and  it  is  sometimes  a big  challenge  to 
find  a common  ground. 

This  explains  why  it  is  far  less  easy  to  modularise  learning  material  as  it  has  been  thought 
from  an  engineering  background.  Complex  topics  of  cultural  studies  are  usually  taught  in 
seminars,  and  base  on  a strong  personal  interaction  between  the  teacher  and  the  pupil. 
Reproduchg  this  situation  in  a virtual  e -learning  environment  is  a challenge,  both  from  the 
technical  as  well  as  from  the  communication  point  of  view. 

Also  non  trivial  is  the  task  of  having  teachers  producing  material  which  is  usable  by  the 
system.  The  material  itself  is  often  not  clearly  modularised,  nor  is  it  tagged  with  the  name  of 
one  or  more  concepts,  so  that  the  creation  of  the  topic  maps  has  often  to  be  done  by  hand 
through  indexing  the  material  delivered,  and  modules  are  inconsistent  within  the  system.  This 
could  be  solved  through  issuing  guidelines  for  the  creation  of  the  material. 

Publication  of  content  stored  in  the  management  systems  is  from  a technical  point  of  view 
not  as  difficult  as  it  has  been  a few  years  ago.  We  plan  to  connect  a cocoon-engine  [ApaOl]  to 
the  content  management  system,  which  will  do  rule  based  conversions  suitable  for  both  screen 
and  paper  publication.  Further  publication  to  other  output  devices  should  be  no  problem,  as 
long  as  suitable  media  for  the  device  is  available  and  marked  up  accordingly. 
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The  question  of  user  management  is  one  which  seems  more  technical  than  anything  else. 
We  hope  to  find  this  assiunption  confirmed  as  we  develop  the  part  of  the  system  responsible 
for  that.  We  will  look  in  rather  fine  detail  into  the  problems  arising  from  questions  like  user- 
defined  annotations  to  text,  as  we  believe  to  find  some  technical  surprises  there. 

The  navigation  using  topic  maps  has  been  mentioned  above.  A first  prototype  is  being 
developed  allowing  for  3D -interaction  with  topic  maps,  to  allow  both  users  and  creators  of 
topic  maps  better  interaction  with  the  system  in  three-dimensional  virtual  spaces. 

Support  of  specifications  is  needed  when  describing  the  content  in  metadata  and 
grouping  content  to  learning  material.  Currently  it  is  evaluated  in  how  far  full  support  of  specs 
like  EML,  IMS,  LOM  and  SCORM  is  feasible  and  useful,  or  if  such  a specification  should  be 
adapted  to  specific  needs  like  CanCore  did  [CanOla]  - especially  as  at  the  time  of  this  writing 
the  project  group  is  trying  to  develop  a first,  prototype,  dociunent  type  definition. 

In  the  field  of  communication  many  things  like  e-mail,  chat,  newsgroups,  blackboards 
and  the  like  have  been  in  use  for  quite  some  time.  An  overview  of  current,  mainly  open  source 
solutions  will  be  evaluated  for  incorporation  into  the  system.  Other  means  like  whiteboard 
sharing  and  video  conferencing  will  be  looked  at  at  a later  stage  of  the  project. 
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Abstract:  This  short  paper  discusses  the  unique  ways  in  which  educational  media 
technology  can  help  meet  the  instructional  needs  of  ESL  (English  as  a Second  Language) 
students  in  the  regular  content  area  classrooms.  One  of  the  most  promising  pedagogical 
approaches  for  working  with  ESL  students  in  the  regular  classroom  is  called  Sheltered 
Content  Instruction,  but  finding  the  appropriate  materials  for  the  regular  classroom 
teacher  to  use  in  reaching  the  goals  of  this  approach  can  be  difficult.  However,  with 
some  simple  blending  of  the  Internet  and  media  technologies  already  in  place  in  most 
content  area  classrooms,  this  challenge  can  be  reduced.  This  paper  discusses  the 
theoretical  potential  of  using  Internet  and  media  technology  to  meet  the  goals  of 
Sheltered  Content  Instruction  and  demonstrates  the  practical  viability  of  this  merger  by 
briefly  describing  a step-by-step  construction  of  a Sheltered  Content  Instruction 
Electronic  Learning  ModiJe. 


Background:  ESL  Sheltered  Content  Instruction 

Currently  there  is  a rapid  rise  in  the  numbers  of  ESL  students  in  American  schools.  The  ESL  research 
community  is  working  to  develop  teaching  methods  that  can  assist  in  the  difficult  task  of  teaching  content 
area  academic  knowledge  to  students  having  limited  English  proficiency.  One  of  the  most  promising 
pedagogical  approaches  for  working  with  ESL  students  in  the  regular  classroom  is  called  Sheltered  Content 
Instruction.  The  focus  of  Sheltered  Content  Instruction  is  to  teach  academic,  or  content  area  subject  matter 
to  ESL  students  using  more  comprehensible  or  “sheltered**  English  in  the  spoken  and  written  forms 
(Eschevarria  & Graves,  1998).  This  involves  consistently  providing  context  clues  such  as  gestures  and 
visual  aids,  along  with  reading-level  appropriate  learning  materials,  and,  ideally,  other  interactive  stimuli  to 
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guide  ESL  students  through  their  content  learning.  This  type  of  approach  is  designed  to  be  utilized  in  the 
regular  classroom  by  content  area  teachers  who  have  received  some  training  in  teaching  ESL  students,  and 
who  have  the  resources  available  to  provide  the  sort  of  scaffolding  needed  to  meet  the  goals  of  Sheltered 
Content  Instruction.  Finding  the  appropriate  materials  for  the  regular  classroom  teacher  to  use  in  reaching 
the  goals  of  this  approach  can  be  difficult.  However,  with  some  simple  blending  of  the  Internet  and  media 
technologies  already  in  place  in  most  content  area  classrooms,  this  challenge  can  be  reduced. 


Technology  Use 

Technology  use  in  ESL  instruction  is  developing  (c.f.  Feyten,  et  al.,  2002).  The  fluidity  of  student  learning 
needs  from  one  ESL  learner  to  another  should  make  the  use  of  computer  technology  to  target  learning 
activities  to  specific  learning  needs  a common  solution.  However,  in  conducting  a search  for  web  sites,  it 
becomes  evident  that  the  Internet  does  not  typically  have  technology  applications  recommended  on  specific 
Sheltered  Content  Instruction  sites.  In  addition,  sites  that  present  detailed  information  about  Sheltered 
Content  Instruction  do  not  have  recommended  sites  that  teachers  could  use.  Consequently,  the  opportunity 
to  integrate  technology  into  the  ESL  classroom  in  a Sheltered  Content  Instruction  context  poses  certain 
obstacles.  The  focus  of  this  paper  is  to  explain  how  it  is  possible  to  blend  the  concepts  of  Sheltered  Content 
Instruction  with  readily  available  technology  so  that  content  area  teachers  with  ESL  students  can  develop 
appropriate  and  interesting  learning  materials  that  accomplish  intended  ESL  student  learning  outcomes. 
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specifically,  aligning  web-based  content  area  reading  materials  with  student  readability  levels 
technique  that  teachers  need  to  be  able  to  do  as  quickly  as  possible.  Copying  text  and  putting  it  in  a Word 
document  allows  a teacher  to  quickly  assess  the  readability  level.  Once  the  text  is  in  a Word  document,  it 
can  be  chunked  into  shorter  narratives  and  organized  with  multimedia.  Readability  can  then  be  adjusted  by 
using  synonyms  to  make  text  and  passages  more  available  by  lowering  or  raising  the  language  to  a 
student’s  reading  level.  The  shorter  blocks  of  text  also  can  be  presented  through  text-to-speech  software 
and  allow  students  to  focus  on  the  content  without  tripping  over  words  that  take  so  long  to  decode  that 
reading  comprehension  is  diminished.  Once  there  is  familiarity  with  these  approaches,  a more  ambitious 
use  of  technology  is  the  development  of  specific  learning  modules  on  the  web,  or  with  a CD,  or  using  a 
PDA  that  beams  students’  work  from  one  appliance  to  another  for  review  and  comment.  These 
technologies  present  the  information  to  be  learned  through  a student  or  teacher  presentation  that  is  age 
appropriate,  motivating,  and  educationally  sound. 


Sheltered  Content  Instruction  Electronic  Learning  Module  (SCIELM) 


A sample  SCIELM  could  be  developed  by  a regular  content  area  teacher  using  the  following  steps:  1) 
identify  the  content  reading  assignment  that  the  majority  of  the  class  will  complete;  2)  search  the  Internet 
for  similar  content  material  online  that  is  written  at  a more  appropriate  reading  level  for  the  ESL  student(s); 
3)  copy  and  paste  the  text  into  a Word  document  and  run  grammar  check  to  determine  the  reading  level  of 
the  text;  4)  substitute  words  and  phrases  as  necessary  for  further  alignment  with  ESL  student  abilities;  5) 
add  graphics  or  other  multimedia  as  available;  6)  assign  readings.  Additionally,  this  sort  of  SCIELM  could 
very  effectively  be  developed  within  a Virtual  Learning  Circle  environment  (Thomas  & Hofmeister,  2002) 
which  would  allow  for  continued  asynchronous  academic  interaction  regarding  the  textual  material. 
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Web-Based  Surveys:  A Comparison  to  Paper-Based  Surveys. 

The  use  of  web-based  surveys  in  educational  research  is  growing  in  popularity  every  year  (Solomon,  2001). 
And  as  familiarity  with  technology  increases,  survey  methodologist  and  researchers  alike  will  find  developing  and 
implementing  web-based  surveys  can  help  to  preserve  valuable  resources  as  well  as  speed  the  collection  and 
analysis  of  data.  When  compared  to  paper  based  surveys,  serious  measurement  issues  have  overshadowed  the 
benefits  that  web-based  surveys  can  bring  to  the  researcher  (Dillman,  2001).  This  disparity  between  paper  based 
and  web-based  survey  methodologies  have  created  the  need  for  further  study  into  the  effect  that  web-based  surveys 
might  have  on  data  and  its  subsequent  evaluation.  Web-based  surveys  are  a relatively  new  area  of  research.  Kiesler 
& Sproull  (1986)  published  one  of  the  first  experimental  surveys  on  the  differences  between  email  surveys  and 
postal  mail  surveys.  With  the  exception  of  this  article  and  a few  others,  the  sheer  paucity  of  research  into  the 
reliability  and  validity  issues  of  web-based  surveys  is  troubling.  This  creates  measurement  issues  because  we 
currently  do  not  know  what  impact  the  design  of  the  web-based  survey  will  have  on  data,  nor  do  we  know  whether 
the  answers  respondents  supply  on  web-based  surveys  will  differ  significantly  from  those  provided  via  a paper- 
based  survey.  It  is  clear  that  web-based  research  is  a new  and  untested  tool,  with  further  research  needed,  especially 
in  web-based  sur/ey  design  and  implementation  as  well  as  a closer  comparison  of  web-based  and  paper  based 
responses  (Dillman,  2001). 

The  purpose  of  this  study  is  to  determine  whether  the  serious  measurement  issues  of  web-based  surveys  can 
be  overcome  through  proper  implementation  and  design,  and  whether  the  web-based  survey  will  yield  equivalent 
results  to  those  obtained  from  a paper  based  survey.  In  order  to  examine  these  questions,  researchers  will  present 
results  from  the  administration  of  two  like  -content  survey  forms  (paper  and  pencil  and  web-based)  that  were 
administered  in  a quazi-experimental  design  to  over  300  participants  of  a teacher  training  session.  This  sample 
provides  unique  opportunities  into  determining  relative  validity  and  reliability  of  the  web-based  survey  by  non; 
randomly  assigning  teacher  to  control  and  experimental  groups  based  on  previous  web  experience  so  that  there  is 
equal  representation  of  web  abilities  in  both  groups  (Roberts  & Onwuegbuzie,  2000). 

Preliminary  results  from  analyzed  data  indicate  the  following  responses  to  the  above  research  questions. 
First,  web-based  surveys  were  more  likely  to  provide  complete  results  by  improving  examinee  fidelity  (e.g., 
accidentally  missing  an  item,  skipping  blocks  of  questions  or  pages,  forcing  responses  to  all  items).  Second, 
because  of  completed  teacher  response,  reliability  estimates  tended  to  be  higher  since  alpha  is  negatively  affected  by 
persons  randomly  missing  items  (Roberts  & Onwuegbuzie,  2000).  Third,  the  ease  of  transfer  of  data  from  an 
already  established  database  to  a statistical  analysis  program  in  the  web-based  survey  ensures  that  there  is  no  error 
when  researchers  are  transferring  data  from  paper  to  the  computer  (as  in  the  paper  and  pencil  version).  This  can 
prove  an  important  issue  in  both  quantitative  and  qualitative  research.  Fourth,  as  with  the  previous  point,  data  entry 
time  is  dramatically  reduced  for  researchers.  And  finally,  the  combination  of  the  above  four  points  in  favor  of  web- 
based  surveys  is  advanced  by  the  fact  that  web-based  survey  lost  no  reliability  or  validity  in  the  transfer  from  paper 
and  pencil  form  (as  will  be  shown  by  the  data). 
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Abstract:  This  paper  reports  an  ongoing  Ph.D.  program  conducted  at  Department  of 
Didactics  and  Educational  Technology,  University  of  Aveiro,  Portugal.  The  program 
undertakes  to  improve  English  grammar  teaching  and  learning  within  the  field  of  the  English 
modal  verbs  at  univeraty  level  employing  Didaktos  (Didactic  Instructional  Design  for  the 
Acquisition  of  Knowledge  and  Transfer  to  Other  Situations)  (Moreira  et  al.  2001),  a prototype 
shell  designed  on  the  basis  of  Cognitive  Flexibility  Theory  (Spiro  et  al.  1987;  1988;  1989; 
1991a;  1991b;  Spiro  & Jehng  1990). 


The  paper  describes  an  ongoing  study  of  university  students  learning  English  modal  verbs  in  a 
hypermedia  learning  environment,  Didaktos  at  Department  of  Didactics  and  Educational  Technology,  University 
of  Aveiro,  Portugal.  The  hypermedia  shell  is  designed  following  the  theoretical  orientations  of  Cognitive 
Flexibility  Theory,  a constructivist  theory  of  learning  and  instruction  in  complex  and  ill-structured  domains  at 
advanced  stages  of  learning. 

Spiro  and  his  collaborators  contend  that  learning  and  instruction  in  complex  and  ill -structured  domains 
require  multiple  representations,  multiple  explanations,  multiple  analogies  and  multiple  dimensions  of  analysis. 
The  central  claim  of  the  theory  is  the  metaphor  of  a criss-crossed  conceptual  landscape  expounded  by  Ludwig 
Wittgenstein  in  Philosophical  Investigations  (Wittgenstein  1997).  The  same  materials  should  be  revisited  at 
different  times,  from  different  points  of  view,  in  different  contexts,  for  different  purposes,  in  rearranged 
instructional  sequences  so  as  to  master  complexity  in  understanding  and  achieve  knowledge  transfer  in  new 
situations.  Moreover,  the  theory  provides  a solution  to  the  widely  reported  problem  of  disorientation  in  hypertext 
systems  as  the  user  is  never  nx)re  than  one  link  away  from  the  focus  of  instruction. 

The  principles  of  Cognitive  Flexibility  Theory  are  best  implemented  in  hypermedia  systems  because  non- 
linear links  can  be  established  to  allow  for  multiple  dimensions  of  knowledge,  for  multiple  interconnections 
across  knowledge  components  and  so  on.  In  accordance  with  the  guidelines  of  Cognitive  Flexibility  Theory, 
Didaktos  offers  teachers  the  possibility  to  present  any  domain  of  knowledge  through  cases  divided  into  chunk- 
size  scenes  in  different  media.  The  scenes  are  codified  with  multiple  perspectives  together  with  information  on 
description  and  context  of  each  scene  and  thematic  commentaries.  Furthermore,  teachers  can  build  special 
sequences  traversing  among  the  scenes  so  that  users*  attention  can  be  called  to  differences  in  superficially  similar 
scenes  and  similarities  between  apparently  different  scenes.  Students  can  view  the  cases  one  by  one,  consulting 
the  description  and  context  of  each  scene,  getting  access  to  the  thematic  commentaries;  look  for  scenes  in  which 
particular  themes  are  relevant  and  follow  the  pre-defined  special  sequences. 

English  modal  verbs  are  selected  as  the  domain  under  study  as  they  pose  difficulty  to  learners  irrespective 
of  their  mother  languages  and  noted  by  their  irregularity  and  complexity  embracing  various  conceptual  domains. 
The  problem  is  aggravated  by  the  traditional  out-of-context,  oversimplified  and  fragmentary  teaching  methods. 
Modal  verbs  will  be  presented  in  authentic  examples  of  use  in  diversified  genres  such  as  poetry,  speech,  play, 
fiction,  song,  movie,  news  broadcasts  and  so  on.  Cases  will  be  analysed  from  distinct  but  complementary 
perspectives:  normative  grammar,  systemic  functional  grammar,  communicative  grammar,  semantics  and 
relevance  theory.  Besides  sequences  will  be  drawn  among  the  scenes  to  show  the  relationship  between  modal 
verbs  and  time  reference,  modal  verbs  and  interpersonal  relationships,  modal  verbs  and  speech  acts  as  well  as 
modal  verbs  in  direct/indirect  speech,  and  modal  verbs  in  formal/informal  use. 

University  students  learning  English  as  a foreign  language  will  participate  in  the  experiment  assigned  to 
three  groups:  the  control  group,  the  experimental  group  (consisting  of  the  passive  sub-group  and  the  active  sub- 
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group).  We  propose  implementing  two  different  orientations  of  the  same  learning  hypertext  with  contrasting 
manners  of  navigation:  organising  contents  in  a linear,  traditional  way  and  in  a flexible,  non-linear  way.  The 
control  group  will  study  the  cases  in  a linear,  traditional  way  while  the  two  experimental  groups  will  be  engaged 
in  a flexible,  non-linear  environment.  The  control  group  will  look  at  “view  cases  sequentially”  and  “look  for 
specific  themes”  while  the  experimental  groups  will  also  have  access  to  “get  special  sequences”.  The  difference 
between  the  two  experimental  sub-groups  is  that  subjects  of  the  active  one  will  be  given  more  autonomy  in 
interacting  with  the  hypertext.  Instead  of  being  offered  the  thematic  commentaries  and  special  sequences  as  what 
the  passive  sub-group  will  be  exposed  to,  they  get  the  opportunity  to  provide  answers  by  themselves  before 
checking  their  answers  with  the  information  in  the  hypertext. 

It  is  expected  that  the  study  will  yield  the  following  results: 

- Subjects  of  the  three  groups  will  make  similar  performance  in  post-tests  that  assess  factual  knowledge. 

- Subjects  of  the  experimental  groups  will  perform  better  than  those  of  the  control  group  in  post-tests  that  assess 
procedural  knowledge. 

- Subjects  of  the  experimental  groups  will  obtain  better  results  on  the  post-tests  than  on  the  pre-tests  assessing 
procedural  knowledge  in  comparison  to  those  of  the  control  group. 

- Subjects  of  the  active  sub-group  will  obtain  better  scores  than  those  of  the  passive  sub-group  in  post-test 
assessing  procedural  knowledge. 

- Subjects  assigned  to  the  active  sub-group  will  hold  a more  positive  attitude  towards  the  hypertext  and  the 
experience  than  those  assigned  to  the  passive  sub-group. 

- Subjects’  epistemological  learning  beliefs  and  preferences  will  change  after  the  treatment. 

- The  epistemological  learning  beliefs  and  preferences  differ  in  gender. 
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Abstract:  The  paper  reports  the  effects  of  animated  characters  in  multimedia  presentations  on 
comprehension  and  attention.  The  results  showed  that  the  lively  character  did  not  influence 
learner's  comprehension  of  presented  material;  nevertheless,  they  showed  the  significant 
differences  of  the  effect  of  lively  character  in  recall  performance.  In  addition,  because  pedagogical 
benefits  of  different  learning  environments  tend  to  be  significant  among  individuals,  this  paper 
revealed  that  introverts  had  significantly  greater  recall  performance  than  did  extroverts. 


Intruductiun 

Drawing  attention  and  active  engagement  are  pedagogical  techniques  for  designing  effective  multimedia 
presentation.  The  value  of  directing  attention  is  based  on  cognitive  information  processing  theories  which  hold  that 
attended  information  consumes  more  cognitive  resources  and  is  more  thoroughly  processed  than  unattended 
information.  Research  reports  that  anthropormphic  computer  interfaces  have  more  capabilities  for  drawing  users' 
attention  and  engaging  them  in  active  tasks  than  nonanthromopormphic  interfaces  (Dehn,  & van  Mulken,  2000). 
Therefore,  human-like  characters  could  be  more  effective  presenters  or  guiders  in  multimedia  presentations  where 
comprehension  and  recall  of  materials  need  to  be  enhanced. 

The  focus  of  this  study  is  to  examine  the  effects  of  anthropomorphics  on  comprehension  and  attention  in 
multimedia  presentations  of  technical  content.  With  respect  to  (King,  & Ohya,  1996;  Laurel,  1997),  in  this  study  a 
lively  character  was  defined  as  a human-like  representation,  which  is  capable  of  displaying  emotional  expressions, 
while  a lifeless  character  was  an  object  or  a part  of  human  form,  which  was  not  able  to  express  emotions.  The  study 
measures  the  degree  of  attentiveness  attributed  to  animated  characters  by  their  effects  on  incidental  learning, 
measured  as  the  accuracy  of  recall  performance  for  attended  visual  messages  and  the  number  of  errors  in  recall 
performance  for  unattended  visual  messages.  This  study  also  examined  the  effect  of  animated  characters  as  a 
function  of  the  participant’s  degree  of  extroversion. 


Manipulation 

In  this  study  the  PowerPoint  slides  used  in  an  actual  computer  graphics  course  were  used.  The  script  for 
each  slide  comprised  of  both  the  instructor’s  explanation  and  a sequence  of  locations  to  which  attention  should  be 
directed.  The  manipulation  of  the  characters  accounted  for  three  conditions  as  screen  examples  in  Figure  1. 
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Figure  1 : Screenshot  examples  of  Pointing  Behaviors 
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An  analysis  of  variance  showed  that  there  were  no  significant  differences  of  comprehension  among  three 
conditions.  For  attention,  there  were  significant  differences  of  recall  involving  the  pointed  objects  (F(2,27)=4.40, 
p<0.022),  while  there  were  no  differences  of  recall  involving  objects  that  were  not  explicitly  pointed.  The  post-hoc 
analysis  resulted  that  the  subjects  in  the  lively  character  condition  had  better  recall  involving  the  pointed  objects 
than  the  subjects  in  the  lifeless  character  condition. 
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With  respect  to  the  degree  of  extroversion,  the  results  of  t-test  showed  that  there  were  no  significant 
differences  on  the  comprehension  between  the  low  and  high  extroversion  groups.  However,  the  results  showed  that 
the  subjects  in  the  low  group  had  better  recall  involving  objects  explicitly  pointed  out  than  the  subjects  in  the  high 
group  (t6=2.298,  p<0.025). 

The  results  correspond  to  the  previous  studies  (Andre,  Rist,  & Muller,  1998)  in  which  the  personified 
characters  did  not  contribute  to  the  students’  comprehension  of  technical  materials.  Nevertheless,  the  lively 
character  was  better  able  to  draw  student’s  attention  to  the  messages  that  were  being  presented  than  the  lifeless 
character.  Therefore,  although  lively  animated  characters  may  not  help  learners  digest  presented  materials,  they  can 
be  an  effective  peripheral  learning  tool  for  directing  attention.  This  study  also  found  that  introverted  learners  had 
better  recall  involving  objects  in  the  presentation  than  extroverted  learners.  This  may  result  from  difficulty  that 
extroverted  learners  find  in  maintaining  concentration  in  non-interactive  environments  since  they  prefer  non- 
structured  materials  with  lots  of  collaborative  activities  and  interactivity.  Introverts,  on  the  other  hand,  benefited 
from  the  structured  material  and  quiet  learning  environment  suggests  that  personality  should  be  taken  into  account 
when  designing  multimedia  presentation. 
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Abstract.  SEA,  the  Mexican  at  distance  system  of  Secondary  Education  for  Adults  is 
characterised  by  being  an  educational  program  on  basic  education  whose  aim  is  to  fulfil  needs 
and  characteristics  of  the  working  young  and  adult  population  who  has  not  finished  nor 
studied  secondary  school.  This  educational  proposal  is  based  on  the  use  of  certain  printed 
materials,  TV  programs  and  Saturday  tutorship  bearing  in  mind  the  adult’s  learning 
achievement  processes.  Furthermore,  its  aim  is  to  acknowledge  that  the  learner  is  an  active 
being,  capable  of  developing  study  skills  to  achieve  autonomous  learning.  In  this  paper  we 
emphasise  on  Saturday  tutorship  on  numeracy  as  an  observant  instrument  which  has  produced 
information  on  the  system’s  efficiency.  Especially  regarding  the  tutors  requirements  in  order 
to  promote  learning  autonomy. 


Introduction 

An  Open  System  on  Basic  Mathematical  Education  in  Mexico 

On  the  21**  of  May  2001  it  was  published  in  the  Official  Federation  Diary  the  295  Agreement  about  the 
Secondary  School  at  Distance  for  Adults.  This  document  states  that:  “The  general  aim  of  the  Secondary 
Education  at  Distance  for  Adults  (SEA)  is  to  offer  a model  on  Education  based  on  needs  and  special 
characteristics  of  the  adult  population  who  remained  behind  regarding  further  education.  In  this,  it  is  expected  to 
help  adults  develop  knowledge,  skills,  attitudes  and  useful  values  to  face  distinct  problems,  as  well  as  getting 
access  to  higher  educational  levels.”  This  system  provides  learners  with  two  learning  guides  of  mathematics 
{Square  Accounts,  SEP,  1998,  for  beginners  and  Figures,  Problems  and  Solutions,  SEP,  2000,  for  advanced 
level),  based  on  calculus  and  problem  solving,  to  lead  and  help  them  along  eight  months  of  instruction.  In 
addition,  and  with  the  purpose  of  backing  up  learning  activities,  36  support  and  18  tutorial  TV  programs  are 
transmitted.  Finally,  there  are  Saturday  tutorships  which  provide  further  support  to  those  learners  whose  doubts 
arose  when  studying. 

From  the  pedagogical  point  of  view,  the  main  objective  is  to  acknowledge  adults’  experience  and  needs, 
developing  knowledge  from  adult  life  contexts,  to  study  contents  meaningful  to  life  and  continual  study.  Also,  to 
improve  studying  methods  to  become  an  independent  learner  and  last,  to  set  up  a way  of  assessing  according  to 
the  approach. 


Problems  Coming  up  within  the  System  and  Research  Methodology 

Adult’s  autonomy  for  studying 

Here,  adult  education  is  essentially  basic  education  addressed  to  people  older  than  15.  Adult  education 
arises  as  a result  of  the  educational  system's  inefficiency,  and  has  specific  connotations,  socially  as  much  as 
political  and  economical  (Schmelkes  y Kalman,  1994). 

One  of  the  fundamental  principles  is  to  acknowledge  the  adult  person  as  an  active  being  in  his  or  her 
learning  process.  Adult  education  should  fulfill  different  requirements  from  those  pertaining  to  common 
education,  due  to  the  fact  that  it  is  focused,  in  great  extent,  to  working  people  who  have  certain  responsibilities 
distinctive  to  adults.  Thus,  education  should  not  absorb  learner’s  full  time  and  effort.  On  the  other  hand,  adult’s 
interests  are  different  from  those  of  children,  as  most  adults  expect  to  apply  or  get  immediate  benefit  from  their 
studies  commitment. 
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Numeracy  through  problem  solving 

Both  texts  of  calculus  and  problem  solving  are  organized  in  units  whose  content  is  shown  from 
problematic  situations  and  immersed  in  contexts  assumed  to  be  close  to  adult  life.  Taken  as  a syllabus  axis,  for 
instance,  trading,  health,  etc.  But  without  neglecting  formal  mathematical  knowledge  and  consequential 
generalizations.  Under  these  pedagogical  principles,  the  aim  is  in  fact,  that  students  develop  mathematical  skills 
and  learn  mathematics  by  building  them  up,  identifying,  selecting  and  using  strategies  adopted  by 
mathematicians. 

Due  to  the  fact  that  some  teachers  do  not  know  how  to  make  up  mathematics  and  doubt  their  own 
mathematical  skills  (Santos,  1997),  some  questions  arise:  How  does  the  SEA -tutor  face  the  concrete  curriculum 
proposal  in  his  hands?  How  to  achieve  that  (or  what  learning  actions  carry  out  for  this  purpose)  the  tutor  in  this 
educational  system  could  build  up  a wide  criterion  to  be  able  to  decide,  and  formulate  the  kind  of  answers  he 
will  be  giving  to  his  or  her  students? 

Saturday  tutorships:  Realm  of  interaction  and  meaning  negotiation 

In  fact,  acknowledgement,  meaning  build  up  or  productions  are  being  developed  during  the  interaction 
process  itself  (Hoyos,  1998).  Meaning  assignation  is  rather  given  from  a joint  process  in  which  the  teacher 
thinks  of  the  student’s  work  in  order  to  state  a suitable  question  or  proposal  to  allow  him  elicit  different 
meanings.  Then,  with  the  purpose  of  identifying  the  main  components  intervening  in  this  system’s  tutorship,  we 
carried  out  an  exploratory  study  (Amador,  2000)  with  one  of  the  SEA  tutors  trainer  and  three  workers*.  The 
following  results  were  obtained:  A)  In  spite  of  tutor’s  efforts  to  regain  the  learner’s  previous  experiences  and 
knowledge,  he  is  not  always  successful.  Usually,  because  the  tutor  forgets  to  retrieve  the  idea  adults  brought  up 
on  this  subject  and  focus  on  the  procedure;  B)  Each  participant  made  up  a different  interpretation  of  the  guide- 
text.  Some  times,  such  interpretations  may  be  taken  as  wrong  or  not  conclusive;  C)  Tutorship  sessions  were  a 
negotiation  realm  for  interpretations  arose  from  the  text  and  for  helping  the  adult  learner  in  his  or  her  ongoing 
learning. 

As  a means  of  conclusion:  A Prospective  on  research  and  re-education  at  distance  for  SEA  tutors 

It  has  been  detected  that  tutors  try  to  cover  in  one  session  the  learning  students  should  develop  in  five 
sessions  of  independent  study.  May  be  an  important  number  of  SEA  students  are  not  developing  their  skills  for 
independent  study.  Therefore,  their  study  activities  are  sometimes  restricted  to  tutorship  session.  How  often  is 
this  practice  repeated?  Which  constitute  the  possible  causes  of  this  phenomenon?  In  what  way  do  this  affect 
mathematics  learning?  What  obstacles  hinder  SEA  students  from  developing  skills  for  achieving  independent 
study?  One  of  our  research  hypothesis  is  that  technology,  communication  and  telematics  will  provide  new 
perspectives  regarding  the  open  and  at  distance  education  system  discussed  in  this  paper.  These  new  resources 
may  be  advantageous  allies  to  endure  initial  practices  of  mathematical  communication,  at  least  between  re- 
educators  and  tutors.  It  may  be  feasible  to  reproduce  in  the  SEA,  during  the  Saturday  tutorship,  communicative 
practices  that  could  produce  achievements  in  learners’  autonomy. 
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‘ The  three  workers  have  daily  jobs,  are  older  than  forty  and  finished  elementary  education.  They  were  a locksmith,  a 
secretary  and  a customers  flow  supervisor  in  a dining  room.  In  a whole,  three  tutorial  sessions  were  carried  out  under  the 
SEA  system,  on  the  subject  of  decimal  numbers;  all  of  them  were  videotaped,  transcribed  and  analysed. 
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Abstract 

This  article  presents  the  development  and  evaluation  of  a web-based  lesson  with  multiple  representations  that  is 
developed  to  cultivate  situated  learning.  The  quaswxperimental  method  along  with  semi  -structured  interviews  was 
used  to  investigate  the  effects  of  a web-based  lesson  on  science  learning  at  the  senior  high  school  level.  Three 
classes  of  second-year  students  from  two  senior  high  schools  in  Taipei  were  selected  as  the  participants  for  this  study. 
The  total  number  of  participants  was  1 10  including  49  males  and  61  females.  The  statistical  results  indicated  that  (a) 
student  conceptual  progress  almost  reached  the  significant  level  (t=l  .98,  p<0.051)  before  and  after  the  experiment, 
(b)  there  were  significant  differences  between  the  conceptual  progress  of  male  and  female  students  on  their 
conceptual  progress  before  and  after  the  experiment  (F=l  1.48,  p<0.001),  (c)  computer  logs  also  showed  that  less 
male  students  participated  in  the  online  discussion  than  female  students  did,  and  that  (d)  most  students  had  positive 
opinions  about  this  web-based  lesson.  The  qualitative  data  analysis  indicated  that  some  students  thought  that  the 
web-based  lesson,  named  Lesson  Rainbow,  provided  a daily -life  situation  could  promote  their  motivation  on 
learning  and  help  them  integrate  knowledge. 

Keywords:  Web-Based  Learning,  Situated  Learning,  Hypermedia,  Science  Learning 


Introduction 

Many  researchers  have  suggested  that  computer  hypermedia  and  networking  technology  are  effective  tools  to 
simulate  realistic  situations  when  a realistic  situation  can  not  be  provided  in  a traditional  classroom  (Winn,  1993; 
Hay,  1996;  McLellan,  1994;  The  Cognition  and  Technology  Group  at  Vanderbilt,  1997;  Harley,  1993).  If  learning 
activities  can  make  a connection  between  the  real  situation  and  its  underlying  theory,  it  is  realistic  to  learners  (Moor 
et  al.,  1994).  If  computer  multimedia  can  simulate  realistic  situation  in  a meaningful  way,  it  can  make  learners  to 
immerse  in  and  to  feel  it  realistic.  This  study  attempts  to  develop  a web-based  lesson  according  to  the  theory  of 
situated  learning  and  to  examine  how  the  Internet  supports  student  situated  learning. 

Internet  technology  can  not  only  integrate  the  advantages  of  traditional  C AI  into  instruction  but  also  provide  a 
variety  of  learning  environments  from  self-directed  learning  to  individual  learning,  one-to-one  interactive  learning, 
group  learning,  and  situational  learning  (Mason,  1995).  With  the  Internet,  students  are  able  to  work  with  current  data 
that  are  much  more  up-to-date,  and  authentic  than  the  material  in  textbooks.  Online  resources  can  help  students 
make  connections  between  their  schoolwork  and  the  concerns  of  people  in  the  real  world.  In  these  ways  new 
technologies  can  make  learning  and  curriculum  more  generative  (Wiske,  2000).  Many  researches  have  shown  that 
web-based  instructions  produced  positive  effects  on  student  learning  motivation,  scientific  attitude  and  learning 
efficiency  (Hsu  & Thomas,  in  press;  Krajcik,  2000;  Edelson,  2001;  Hoadley  & Linn,  2000).  Most  of  these  researches 
concluded  that  the  positive  effects  came  from  careful  designs  on  web  pages,  learning  materials  and  web 
managements.  The  benefits  of  web-based  instructions  are  not  reaped  automatically  but  only  come  as  a result  of 
careful  planning. 

In  this  study,  we  chose  the  theory  of  situated  learning  as  the  theoretical  base  for  the  development  of  a 
web-based  lesson,  named  “Lesson  Rainbow”.  It  is  because  most  teaching  contents  and  processes  in  school  activities 
do  not  start  fromreaHife  situations.  This  produces  students  who  can  not  apply  their  learning  to  solve  problems  in 
real  life  or  work  related  situations.  Educational  reform  in  many  countries  focused  primarily  on  lifelong  learning  and 
transfer  of  what  students  learn  to  real-life  situations  (Ministry  of  Education  in  Taiwan,  1999;  National  Science 
Education  Standards,  1996).  Situated  learning  emphases  that  learning  occurs  in  real  situations  and  the  construction 
of  knowledge  is  in  the  continuous  interaction  between  humans  and  situations  (Brown  et  al.,  1989;  Lave  & Wenger, 
1991;  McLellan,  1996).  This  leads  students  to  gain  synthetic  knowledge  instead  of  inert  knowledge.  Three  realistic 
situations  can  be  used  in  class:  (1)  taking  students  to  the  real  workplace;  (2)  immersing  students  in  an  authentic  or 
similar  situation;  (3)  providing  students  an  anchoring  context  (McLellan,  1996).  This  study  used  flash  animations  to 
immerse  students  in  an  authentic  situation,  a trip  to  the  northeast  in  Taiwan. 

Goldenberg  (1995)  claimed  that  we  can  not  expect  to  understand  how  students  learn  if  we  look  only  at  the 
student’s  facility  with  one  representation  or  student’s  handling  of  each  representation  in  isolation.  It  is  limited  to 
understanding  students’  understanding  when  research  is  preformed  only  in  a single  -representation  environment 
because  it  is  hard  to  tell  whether  a particular  observation  is  or  is  not  merely  an  artifact  of  that  representation.  In  a 
multiple  representation  environment,  the  researcher  can  easily  discriminate  between  surface  confusions  about  any 
one  representation  and  true  misconstructions  of  the  concept.  Multi-representational  simulations  embedded  in  a 
situated  learning  environment  provide  scaffoldings  and  context  for  students  to  explore  the  simulations.  Some 
multi-representational  simulations  are  designed  in  Lesson  Rainbow.  Through  the  Internet,  we  record  student  actions 
in  detail  so  that  we  can  describe  a web-based  environment  that  instantiates  this  approach,  and  we  analyze  formative 
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evaluation  data  to  test  the  hypothesized  mechanisms  of  learning.  This  study  attempts  to  investigate  how  multiple 
representations  support  students  with  different  learning  preference  on  the  construction  of  their  scientific  concepts, 
whether  students  with  different  learning  character  tics  prefer  difference  functions  design  in  the  situated  learning 
environment,  and  how  students  benefit  from  multi-representation  simulations  in  a situated  learning  environment. 

Method 

This  research  project  employed  the  quasi-experimental  method  along  with  semi-structured  interviews  to  investigate 
the  effects  of  a web-based  lesson  on  science  learning  at  the  senior  high  school  level. 

Participants 

The  participants  for  this  study  were  selected  from  two  senior  high  schools  (called  as  School  A and  School  B) 
in  Taipei.  They  were  second-year  students  and  their  majors  were  social  sciences.  There  were  four  classes  who 
enrolled  in  Earth  Science  class  at  School  A and  two  of  them  were  randomly  chosen  as  our  participants  including 
forty-four  males  and  forty-two  females.  One  of  two  classes  who  enrolled  in  Earth  Science  class  at  school  B was 
selected  randomly  as  the  sample  including  16  males  and  30  females.  Totally,  the  participants  were  132.  After 
excluding  invalid  data,  the  number  of  the  valid  participants  was  1 10  (49  males  and  61  females). 

Instrumentation 

In  addition  to  the  web-based  lesson  (called  as  Lesson  Rainbow),  there  were  three  other  instruments,  a test,  a 
questionnaire  and  a follow-up  interview,  that  were  developed  to  collect  data  on  student  conceptual  progress  and  their 
opinions  about  the  web-based  lesson.  Experts  validated  all  instructions  used  in  this  study.  The  characteristics  of  each 
instrument  are  shown  as  follows: 

Lesson  Rainbow:  McLellan  (1996)  suggested  there  are  eight  key  components  included  in  situated  learning:  stories, 
reflection,  cognitive  apprenticeship,  collaboration,  coaching,  multiple  practices,  articulation  of  learning  skills  and 
technology.  Five  of  these  were  designed  in  Lesson  Rainbow  except  for  cognitive  apprenticeship,  coaching,  and 
articulation  of  learning  skills . How  did  we  apply  situated  learning  theory  to  Lesson  Rainbow?  The  learning  activities 
are  designed  with  story -based  animations  to  display  situations,  discipline-integrated  themes  to  integrate  the  concepts 
of  mathematics,  physics,  biology  and  earth  sciences,  problem-based  assessments,  social  interaction  to  empower 
knowledge  construction,  and  implicit  hints  to  scaffold  student  learning.  Associated  with  the  design  of  animated 
tutorials(  Berger  et  al.,  1994) , we  provide  some  analytic  or  problem-based  questions  which  can  guide  students  apply, 
analyze  and  synthesize  data  or  information  shown  in  the  Lesson  Rainbow.  Animated  tutorials  aim  to  increase  new 
concepts  into  students’  schema  so  the  computer  program  provide  only  guiding  feedback  but  not  correct  answers.  The 
learning  tools  provided  in  Lesson  Rainbow  include  an  electronic  notebook,  an  equation  editor,  a discussion  board, 
and  an  electronic  map.  The  mapping  between  these  components  and  Lesson  Rainbow  is  shown  as  below  (see  Table 
1): 


Table  1 : Mapping  between  the  Components  of  Situated  Learning  and  the  Designs  of  Lesson  Rainbow 


Component 

The  mapping  design  of  Lesson  Rainbow 

Stories 

(Story -based  animations) 

The  animation  design  displays  realistic  situations  about  a story 
about  a trip  to  the  northeast  coast  in  Taiwan. 

Collaboration 
(Discussion  board) 

The  function  of  asynchronous  online  discussion  provides 
students  with  an  opportunity  to  form  a virtual  learning 
community  for  collaborative  learning. 

Multiple  practices 
(Animated  tutorials) 

The  formative  tests  following  after  each  unit  in  the  story  provide 
students  multiple  practices. 

Reflection 

(An  electronic  notebook) 

The  electronic  notebook  that  was  designed  as  a learning  tool 
helps  students  reflect  and  take  notes  on  what  they  are  learning. 

Technology 

(Internet  technology  and 
multi-representational  simulations) 

Hypermedia  and  network  technology  display  learning  materials 
to  students. 

Test:  A test  was  conducted  to  detect  student  understanding  of  the  concepts  related  to  rainbows,  humidity,  and 
condensation.  There  were  23  items  on  the  test,  which  were  validated  and  was  examined  for  reliability  (The 
Cronbacha=  0.76)  before  the  formal  experiment. 

Questionnaire:  A questionnaire  with  34  Likert -type  items  was  used  to  conduct  student  opinions  about  the  design  of 
this  web-based  lesson.  Four  dimensions  in  the  questionnaire  solicited  opinions  on  interface  design,  situation  design, 
the  design  of  learning  tools  and  the  overall  design  of  Lesson  Rainbow.  The  Cronbachaof  the  questionnaire  was 
reported  as  0.87. 
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The  follow-up  interviews:  Semi -structured  interviews  were  conducted  to  investigate  student  understanding  of  the 
relative  concepts  and  to  collected  their  opinions  about  the  lesson.  Twenty  students  (8  males  and  12  female  s)  were 
selected  for  interviews  based  on  specific  purposes.  Each  student  was  interviewed  for  20-30  minutes. 

Procedure 

This  study  included  three  stages: (1)  The  preparation  stage  (01/2000~1 1/2000):  In  this  stage,  the  major  work 
was  to  develop  Lesson  Rainbow  and  the  instruments.  After  a pilot  study,  the  instruments  were  validated  and  revised 
for  the  experimental  stage.  (2)  The  experimental  stage  (12/2000~  1/2001):  A pretest  on  the  concepts  was  conducted 
in  the  week  before  the  experiment.  In  the  experiment,  students  received  a training  session  for  one  hour  and 
completed  Lesson  Rainbow  in  two  hours.  The  week  after  the  experiment,  a posttest  on  the  concepts  and  the 
questionnaire  were  administrated.  The  selected  students  received  follow-up  interviews  a few  weeks  later  because  we 
needed  time  to  analyze  the  data  and  determine  typical  cases  for  the  interviews.  (3)  The  data  analysis  stage 
(1/2001^5/2001):  Data  analysis  and  concluding  remarks  were  the  major  Jobs  in  this  stage. 

Data  Analysis 

The  data  were  analyzed  in  several  ways.  In  order  to  examine  if  there  was  a significant  difference  between  before  and 
after  treatments,  the  paired  t-test  was  used  to  compare  the  pretest  and  posttest  scores  on  concepts.  A mixed  design  of 
repeated  ANOVA  was  used  to  test  the  hypotheses  that  stated  male  and  female  students  had  a significant  difference 
before  and  after  the  experiment.  We  used  descriptive  statistics  for  the  analysis  of  each  item  in  the  questionnaire  and 
used  Chi  square  to  test  if  there  was  a significant  difference  among  the  four  dimensions  (the  opinions  on  the  interface 
designs,  the  design  of  situations,  the  design  of  learning  tools  and  the  overall  design  of  Lesson  Rainbow. ) in  the 
questionnaire.  The  collected  data  was  analyzed  using  SPSS  (Statistical  Package  for  Social  Science,  version  7.0).  The 
assumptions  of  normal  distribution  and  homogeneity  of  variance  for  dependent  variables  were  tested  before 
applying  statistic  methods,  t test  and  ANOVA.  If  the  dependent  variable  did  not  display  a normal  distribution,  the 
significant  level  was  cut  to  0.04  in  order  to  reduce  the  Type  1 error  (Stevens,  1996).  Qualitative  data  were  coded  and 
summarized  to  show  a deep  understanding  of  student  learning  processes  and  perspectives  using  the  web-based 
lesson.  Lesson  Rainbow. 

Results  and  Discussion 

It  was  hypothesized  that  there  was  a significant  difference  between  student  pre  and  post  concept  tests.  For  Lesson 
Rainbow,  the  mean  for  the  pretest  was  16.7  with  a standard  deviation  of  2.32.  In  contrast,  the  posttest  mean  was  17.4 
with  a standard  deviation  of  2.45.  The  result  of  repeated  t-test  showed  thatthere  was  nearly  a significant  difference 
(at  a 0.04  significant  level  because  of  the  abnormally -distributed  scores  of  tests)  between  student  pretest  and 
posttests  on  concepts  (t=  1.98,  p<0.051;  The  data  is  shown  in  Table  2). 

The  pretest  mean  was  16.6  for  male  students  and  that  for  female  students  was  17.0.  In  contrast,  the  posttest  mean  for 
male  students  was  16.5  and  that  for  female  was  18.1  (The  data  is  shown  in  Table  4).  The  result  of  mixed  design  of 
2X2  ANOVA  showed  that  there  was  a significant  difference  between  male  and  female  student  performances  on  the 
pretest  and  posttest  (F=l  1 .48,  p<0.001 ; The  data  is  shown  in  Table  5).  Few  male  students  said  that  the  animations  in 
Lesson  Rainbow  were  not  as  attractive  as  online  games  in  interviews.  Their  low  attention  to  the  learning  materials 
leaded  to  their  low  performances  on  the  posttest.  Computer  logs  also  showed  that  less  male  students  participated  in 
the  online  discussion  than  female  students  did.  This  may  reduce  male  students  to  construct  their  knowledge  when 
they  used  Lesson  Rainbow.  These  two  reasons  made  male  students  could  not  learn  better  than  female  students. 


Table  2:  Summary  Table  for  Paired-t  test 


N=110 

Mean 

S.D. 

t Value 

P 

Pretest 

16.7 

2.32 

1.98 

0.051 

Posttest 

17.4 

2.45 

p<  0.04 


Table  3:  Descriptive  Statistics 


Male(  N= 
Mean 

49) 

S.D. 

Female  ( N= 
Mean 

61) 

S.D. 

Pretest 

16.6 

2.22 

17.0 

2.39 

Posttest 

16.5 

2.55 

18.1 

2.14 

Table  4 : Summary  Table  for  Two  Way  ANOVA 

Sources 

df 

SS  MS 

F 

Sig 

Before  &After(  A) 

1 

17.01 

17.01 

3.22 

0.075 

Gender(  Q 

1 

63.11 

63.11 

11.48 

0.001 

Interaction(  A*  C) 

1 

15.79 

15.79 

2.99 

0.087 

ResiduaK  A*  S) 

108 

570.20 

5.28 

ResiduaU  C*  S) 

108 

593.60 

5.50 

p<  0.04;  p<  0.01 


The  data  from  interviewing  the  20  students  showed  that  most  students  thought  the  web-based  lesson  was  more 
interesting  than  textbooks  because  of  the  animations  and  interactions.  The  online  discussions  made  students  feel 
more  involved  in  the  learning  activities.  Five  of  the  twenty  students  suggested  that  the  teacher  should  participate  in 
the  online  discussion  with  them  because  sometimes  they  did  not  know  how  to  solve  problems  without  the  teacher’s 
scaffolding.  A student  said  in  his  interview:  “ If  the  teacher  could  participate  in  the  online  discussion,  it  would 
increase  the  interactions  and  cleared  confusions.  I would  learn  better  if  the  teacher  involved  in  the  online 
discussion.”  Different  students  have  different  needs  in  a web-based  learning  environment.  Few  of  them  need 
teachers’  assistances  because  they  are  lack  of  skills  on  communication  and  reflective  thinking.  It  is  advised  that  the 
well-trained  teachers  involve  in  online  communications  with  students  in  order  to  help  weak  students  overcome  their 
obstacles. 

Conclusions 

This  study  investigated  the  effect  of  a web-based  lesson  (Lesson  Rainbow)  developed  to  cultivate  situated  learning. 

A realistic  situation  served  as  a bridge  to  connect  student  daily-life  experiences  and  constructing  knowledge.  Most 
students  had  positive  opinions  about  Lesson  Rainbow.  From  the  interviews,  some  students  said  that  the  animations 
for  the  simulated  authentic  situations  could  promote  their  learning  motivation  and  immerse  them  in  an  interesting 
context  for  meaningful  learning. 

Networked  technologies  supported  collaborative  work  in  which  the  students  combined  components  or  worked 
together  to  make  successive  drafts  (Wiske,  2000).  Social  learning  theory  emphasized  the  value  of  dialogue  and 
collaboration  in  helping  students  to  develop  and  articulate  their  understanding.  In  order  to  reach  effective 
cooperation,  students  needed  to  share  ideas,  adventure  and  argue  with  others  so  that  they  could  come  to  reasonable 
interpretations  of  the  subjects  they  were  studying  (Blumfnfeld,  et  al.,  1997).  Students  can  compare  the  varied 
views  of  a topic  and  enhance  cohesive  understanding  of  science  from  a well -designed  online  asynchronous 
discussion.  In  this  study.  Lesson  Rainbow  succeeded  in  helping  students  to  integrate  knowledge  by  evoking 
discussions  to  search  for  answers  to  the  questions  following  realistic  situations.  When  students  communicate  with 
others,  they  retrieve  their  pre-knowledge  and  reconstruct  concepts.  Meaningful  learning  occurs  when  students 
interact  with  others  or  environments  (Savery  & Duffy,  1995).  Therefore,  online  asynchronous  discussion  designed 
for  a situated  learning  environment  can  promote  knowledge  integration. 
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Abstract 

The  purpose  of  this  study  is  to  compare  the  effects  of  single  versus  multiple  metaphors  in 
computer  expert  and  novice  users’  change  of  mental  models  as  well  as  information  search 
performance.  Related  issues,  such  as  mental  models,  differences  between  novice  and  expert 
users,  and  effects  of  single  versus  multiple  metaphors  will  be  reviewed  first  in  this  paper, 
followed  by  the  discussions  of  methodology  and  expected  results.  The  research  findings  will 
suggest  useful  guidelines  in  designing  metaphorical  interfaces  in  the  area  of  human-computer 
interaction. 

Introduction 

People  often  use  metaphors  to  build  bridges  from  the  known  to  the  unknown.  In  the  area  of 
humaivcomputer  interaction,  metaphors  have  been  used  by  practitioners  to  design  computer 
interfaces  to  facilitate  user  learning  since  1981.  As  to  their  effects  in  facilitating  user  learning, 
studies  show  that  metaphors  may  better  help  novices  in  learning  computer  programs,  because 
they  have  less  complete  mental  models  compared  to  the  mental  models  of  experts.  However,  the 
effects  of  metaphors  on  experts  may  be  questionable. 

Metaphor  (conceptual  model)  vs.  mental  models 

The  use  of  conceptual  models  (or  metaphors)  to  help  users  construct  their  mental  models 
has  been  advocated  by  several  researchers.  A mental  model  is  an  internal  model  that  an 
individual  uses  to  facilitate  interaction  with  the  environment,  with  other  individuals,  or  with 
technology  (Norman,  1983).  When  people  interact  with  a new  system,  they  formulate  mental 
models  of  that  system  and  continue  to  modify  their  mental  models  to  get  a workable  result. 

Eberts  (1994)  notes  that  designers  apply  their  conceptual  model  to  the  design  of  interface, 
and  this  model  is  conveyed  to  users  through  display  representation.  In  order  to  understand  the 
operations  of  the  system,  users  need  to  develop  mental  models  of  the  computer  tasks.  The 
mental  models  of  a computer  user  will  continuously  change  with  the  increase  of  one’s  computer 
experience.  Metaphor  is  a type  of  conceptual  model.  By  explaining  the  workings  of  the  system, 
good  conceptual  models  help  users  to  form  more  comprehensive  mental  models  of  the  system. 
There  is  a close  relationship  existing  between  the  provision  of  metaphors  and  the  formation  of 
more  comprehensive  mental  models,  which  results  in  better  performance  as  users  carry  out 
computer  tasks.  However,  this  relation  has  not  been  well  examined  by  researchers. 

Expert  / novice  users 

Expert  and  novice  users  differ  in  the  degree  of  the  integration  of  their  knowledge,  but  even 
experts  may  still  possess  fragmentary  mental  models;  that  is,  their  "mental  models  continue  to 
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be  specified  by  both  assimilated  and  fragmented  components"  (Sebrechts,  Marsh,  and 
Furstenburg  1990,  p376).  Because  novice  users  possess  less  computer  knowledge  and  their 
mental  models  are  more  fragmented  than  expert  users,  the  use  of  metaphors  will  be  most  useful 
for  novices  who  are  learning  new  computer  systems.  In  the  present  study,  the  researchers  will 
also  compare  the  effects  of  metaphorical  interfaces  on  novice  and  expert’s  information  search 
performances  as  well  as  the  chance  of  their  mental  models. 

Single  metaphor  vs.  multiple  metaphor 

According  to  Centner’s  (1997a,  1997b)  structure  mapping  theory,  the  major  advantage  of  a 
metaphor  lies  in  the  relationship  of  objects  with  the  metaphor.  The  unified  structure  of  an 
integral  metaphor  allows  users  to  see  the  interrelationships  between  elements  in  order  to  identify 
each  element’s  function  On  the  other  hand,  mismatches  happen  in  situations  in  which  the 
attributes  and  relations  of  a single  metaphor  can  not  be  perfectly  correlated  with  the  attributes 
and  relations  of  the  target  domain.  This  is  especially  true  when  the  target  domain  is  so  complex 
that  no  individual  model  can  fully  explain  anticipated  behavior.  In  this  case,  the  use  of  multiple 
metaphors  to  design  interfaces  may  be  a solution  (William,  Hollan,  and  Stevens,  1983).  A study 
(Hsu,  2000)  compared  the  effects  of  single  metaphor  and  multiple  metaphors  in  designing 
interfaces,  the  findings  show  that  interface  cues  from  multiple  metaphors  helped  subjects  to  find 
a greater  number  of  accurate  answers  in  a shorter  time.  However,  little  was  known  about  what 
change  happened  in  subjects’  brain . It  may  be  a better  way  to  use  the  thiirk  aloud  method  to 
explore  subjects'  changes  of  mental  models  of  the  system. 

Methodologyand  expected  findings 

The  independent  variables  in  the  study  are  the  three  interfaces  with  different  degrees  of 
structural  cues  derived  from  single  or  multiple  metaphors.  The  dependent  variables  are  the 
subjects'  search  performances  in  terms  of  the  accuracy  of  their  search  results,  time  spent  on  each 
task,  navigational  paths  chosen  for  completing  each  task,  and  their  satisfaction.  Another  variable 
to  evaluate  is  their  mental  model.  Interview  and  questionnaire  will  be  used  to  collect  data 
regarding  the  change  of  subjects’  mental  models.  A total  of  90  college  students  will  be  used  as 
the  subjects  to  perform  some  information  search  tasks.  Some  of  the  subjects  will  be  asked  to 
perform  think  aloud  tasks  to  further  identify  their  change  of  mental  models. 

The  researchers  are  collecting  experimental  data  and  the  analysis  will  be  completed  in  June 
2002.  It  is  expected  that  the  research  findings  will  suggest  useful  guidelines  in  designing 
metaphorical  interfaces  for  users  with  different  degrees  of  computer  expertise. 
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Abstract:  The  purpose  of  this  study  is  to  determine  and  compare  faculty  perceptions  regarding  the 
use  of  WebCT  in  selected  public  universities  in  the  Midwest  on  United  States,  Specifically,  the 
study  examined  the  demographic  characteristics  of  faculty  who  deliver  courses  using  WebCT  and 
barriers  encountered  in  their  use  WebCT.  The  Web-based  survey  was  sent  to  300  faculty 
members  who  use  WebCT  to  deliver  courses  in  the  selected  Midwestern  universities.  The  data 
were  analyzed  using  descriptive  and  inferential  statistics.  Primary  conclusions  to  emerge  from 
the  study  include  the  following.  (1)  The  majority  of  university  faculty  who  use  WebCT  to  deliver 
courses  in  the  Midwestern  United  States  are  female,  (2)  Most  WebCT  courses  are  offered  both 
on  and  off  campus.  (3)  The  majority  of  faculty  believe  WebCT  is  moderately  ease  to  learn  and 
use.  (4)  Lack  of  time  and  heavy  instructional  load  are  the  greatest  barriers  for  faculty  using 
WebCT  to  deliver  courses. 


Introduction 

As  technological  innovations  continue  to  change  and  expand,  it  becomes  increasingly  necessary  to 
support  faculty  in  adopting  innovations  to  keep  up  with  the  educational  demands  of  the  21st  century. 
These  demands  include  the  changing  learning  environment,  computer-aided  instruction,  Web-based 
course  management,  and  distance  education  Web- based  Instruction  (WBI)  is  defined  as  an  innovative 
approach  for  delivering  instruction  to  a remote  audience  using  the  World  Wide  Web  as  the  instructional 
delivery  system  (Khan,  1997).  Currently,  WBI  is  growing  faster  than  any  other  instructional  technology 
(Hill  & Raven,  2000).  i has  been  used  not  only  in  exclusive  distance  programs  but  also  in  supporting 
face-to-face  traditional  classroom  instruction.  According  to  research  conducted  by  Mclsaac  & 
Gunawardena  (1996)  distance  learning  is  the  fastest  growing  form  of  education  and  there  will  be  a 
continued  expansion  of  Web- based  technologies  in  higher  education.  Although  new  technology  is 
creating  new  educational  challenges  and  opportunities,  it  also  creates  new  problems.  Moreover,  WBI  is 
primarily  a mechanism  for  delivering  instructions,  it  cannot  resolve  problems  created  by  inadequate  or 
poor  teaching.  As  the  Web  will  be  increasingly  used  as  a teaching  and  learning  medium,  it  is  important 
to  consider  the  emerging  roles  of  instructors  and  learning  environments  (Shotsberger,  1997). 
Consequently,  as  expanding  and  improving  instructor  usage  is  the  key  to  enhancingthe  success  of  WBI, 
increasing  attention  has  been  focused  on  faculty  assistance  and  student needs.  This  present  study 
investigated  faculty  perspectives  of  delivering  courses  using  WebCT  in  Web-based  Instruction.  The 
purpose  of  this  study  is  to  determine  and  compare  faculty  perceptions  regarding  the  use  of  WebCT  in 
selected  public  universities  in  the  Midwest  United  States.  Specifically,  the  study  examined  the 
demographic  characteristics  of  faculty  who  delivered  courses  using  WebCT  and  any  barriers  encountered 
in  their  use  of  WebCT.  The  following  questions  were  considered  in  guiding  this  study: 

1.  What  are  the  demographic  characteristics,  in  terms  of  age,  gender,  tenure  status,  rank, 
academic  teaching  areas,  level  of  training,  and  location  of  course  delivery,  of  faculty  who  utilize 
courses  using  WebCT? 

2.  What  is  the  extent  of  barriers  encountered  when  faculty  deliver  courses  using  WebCT? 


The  Study 

The  population  for  this  study  consisted  of  all  faculty  members  who  use  WebCT  for  delivering 
courses  in  four  selected  public  universities  located  in  the  Midwest  on  United  States.  The  Web-based 
survey  was  sent  to  300  faculty  members  who  use  WebCT  to  deliver  courses  at  selected  universities.  Out 
of  the  106  surveys  returned,  52.8%  were  in  the  36-50  age  bracket.  Regarding  faculty  status,  most 
faculty  were  in  the  tenured  group.  In  addition,  more  assistant  professors  (32.1%)  participated  in  the 
study  than  professors  (21.7%),  associate  professors  (20.8%),  and  instructors  (20.8%).  The  majority  of 
faculty  in  the  study  were  in  the  School  of  Education  (38.1%)  and  College  of  Arts  and  Sciences  (30.5%). 
Regarding  the  level  of  skill  as  a WebCT  user,  a large  group  of  faculty  (49.1%)  rated  themselves  at  the 


intermediate  level.  Nevertheless,  more  faculty  participating  in  the  study  were  delivering  WebCT  both 
on  and  off  campus  (54.3%)  than  off  campus  exclusively  (6.7%). 

Lack  of  time  (time  consuming)  (M.  = 4.25)  and  heavy  instructional  load  (M  = 3.50)  are  the  greatest 
barriers  for  faculty  using  WebCT  to  deliver  courses.  Other  barriers  reported  by  faculty  are  lack  of 
technical  skills,  lack  of  incentives,  lack  of  technical  support,  lack  of  technical  training  resources.  The 
least  important  barriers  faculty  indicated  for  delivering  courses  usingWebCT  were  class  size  too  large  (M 
= 2.59)  and  lack  of  personal  interests  (J^  = 2.67). 


Conclusion 

The  majority  of  faculty  responded  that  WebCT  is  moderately  easy  to  learn  and  use;  however,  lack 
of  time  and  heavy  teaching  loads  are  the  greatest  barriers  and  weakness  associated  with  delivering 
courses  using  WebCT.  For  many  years,  a systems  approach  has  been  adopted  as  essential  to  the 
successful  practice  of  instructional  design  among  industries  and  education  for  many  years.  When  a 
system  approach  is  applied  to  Web-based  instruction,  departments,  administrators,  teaching  assistants, 
instructional  designers,  technical  supports,  students,  and  training  programs  are  integrated.  As  a matter 
of  fact,  education  is  one  of  the  few  areas  where  division  of  labor  or  specialization  is  still  not  practiced  to 
any  extent  (Kearsley  & Moore,  1996).  Individual  teachers  who  develop  and  deliver  their  own  courses 
try  to  be  effective  curriculum  designers,  evaluators,  motivators,  group  discussions  facilitators,  and 
content  experts  as  well.  Kearsley  & Moore  (1996)  claimed  this  is  a huge  waste  of  human  resources. 
Williams  & Peters  (1997)  also  reported  that  the  current  atmosphere  in  major  research  universities  is  still 
competitive,  not  collaborative. 

Time  and  resources  are  inevitable  obstacles  to  developing  good  WBI.  Neither  the  teacher  nor  the 
technology  alone  can  make  WBI  work.  One  solution  would  be  for  faculty  members  with  similar  courses 
to  collaborate  and  share  their  ecpertise  and  resources.  To  establish  an  instructional  team  involving 
faculty  peers,  TAs,  student  technology  fellows,  and  technical  support  persons  could  help  by  sharingthe 
workload  and  designing  the  program  to  accommodate  both  courses. 

WBI  will  benefit  from  supportive  faculty  input  resulting  in  technological  advancements.  In  spite 
of  providing  fundamental  technical  skills,  there  is  also  a need  to  regularly  provide  and  update  faculty 
knowledge  about  the  technological  developments  and  strategies  that  impact  teaching  and  learning  that 
will  foster  collaboration  by  faculty  with  common  curricular  interests. 
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Abstract:  The  ability  to  easily  find,  edit  and  re-use  content  adds  significant  value  to  that  content.  When 
the  content  consists  of  complex,  multimedia  objects,  both  the  difficulty  of  implementing  such 
capabilities  and  the  added-value  are  multiplied.  A distributed  archive  of  SMIL  presentations  was  built 
and  indexed  using  tools,  developed  by  the  authors,  which  enable  digitized  videos  of  lectures  to  be 
automatically  synchronized  with  their  corresponding  PowerPoint  slides.  In  addition  a full-text  and 
keyword  search  interface  was  developed.  As  the  archive  grew,  it  became  clear  that  the  ability  to  edit, 
update  or  customize  existing  presentations  by  deleting,  adding  or  replacing  specific  slides  without  re- 
filming the  entire  lecture  was  extremely  desirable.  This  paper  describes  the  application  that  was  built  to 
satisfy  this  need  and  the  changes  necessary  to  the  original  metadata  schema  to  ensure  that  editions 
could  be  easily  tracked  and  new  versions  of  presentations  easily  managed. 


1 Introdnction 


The  use  of  digital,  synchronised,  multimedia  presentations  in  education  is  increasing  and  as  the  volume  of  presentations  available 
grows,  so  does  the  need  for  tools  to  facilitate  the  re-use  and  management  of  these  presentations.  Presentation  editing  tools,  which 
permit  updates,  additions  and  concatenations  of  existing  presentations,  whilst  maintaining  synchronization  and  updating  metadata, 
can  improve  the  quality  of  the  content  and  reduce  the  time  involved  in  production  and  indexing.  Simple  functionality  such  as  cut, 
copy,  paste  and  delete  would  enable  educators  to  re-use  existing  presentation  segments,  update  presentations  with  more  recent  or 
topical  material  and  customize  presentations  for  particular  audiences,  without  having  to  re-film  the  entire  lecture  each  time. 

This  work  is  part  of  an  ongoing  project  [1]  in  which  a set  of  tools  were  developed  to  automate  the  synchronisation  of  digitized  lecture 
videos  with  their  corresponding  PowerPoint  slides.  These  tools,  described  briefly  in  the  next  section,  enable  the  cost-cffcctivc, 
streamlined  production  of  SMIL  presentations  and  associated  metadata  for  archival  and  seardiing.  Such  online  presentations  provide 
an  extremely  useful  educational  resource,  particularly  in  a distance  learning  environment.  However,  as  the  size  of  the  presentation 
archive  grew,  so  did  the  need  to  edit,  update  or  customize  existing  presentations  by  deleting,  adding  or  replacing  specific  slides 
without  re-filming.  The  economic  benefits  associated  with  such  re-use  are  obvious.  This  paper  describes  the  PresentationEditor 
application  which  was  developed  to  satisfy  this  need. 

In  developing  an  editor  for  the  synchronised  multimedia  presentations  a number  of  requirements  were  identified.  Firstly  the  editor 
should  maintain  the  synchronization  of  the  video  and  PowerPoint  slides,  automatically,  without  need  for  user  input.  Secondly  the 
metadata  created  for  the  presentation  must  be  updated  to  reflect  the  changes  made,  who  made  them  and  when  they  were  made.  The 
system  also  needs  to  record  details  of  the  composition  of  new  ‘virtual’  presentations  (i.e.,  presentations  that  are  composed  of  edited 
segments  of  previous  presentations,  but  which  have  never  been  presented  in  reality)  to  enable  accurate,  fine-grained  (full-text  or 
keyword)  searching  across  the  presentations.  Finally  it  is  important  that  the  editor  is  easy  to  use  and  lightweight.  No  knowledge  of 
SMIL  or  the  underlying  structure  of  the  preseiitation  should  be  required  in  order  to  use  the  application. 

In  the  next  section  we  describe  previous  related  work  and  how  the  work  described  here  differs  from  and  builds  on  top  of  that. 


2 Previons  Work/Background 


DSTC’s  SMIL  Presentation  Project  [1]  developed  a suite  of  tools  to  cost-effectively  create  and  index  synchronized  PowerPointA^ideo 
presentations  for  deployment  either  on  the  Internet,  CD-ROM  or  a LAN.  The  PresentationLogger  records  the  time  that  each 
PowerPoint  slide  is  displayed.  The  Presentationindexer  collects  the  metadata  about  the  pres-entation  in  a semi-automated  fashion  and 
creates  an  XML  file  containing  a complete  description  of  the  presentation,  its  contents  and  structure.  The 
Presen tationBuilder/Archiver  creates  the  finished  SMIL  presentation  and  stores  the  metadata  as  a separate  stand-alone  XML  file  or 
inserts  the  metadata  into  a database.  An  Internet  search  and  browse  interface  [2]  has  also  been  built  and  can  search  either  flat  XML 
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flies  or  utilise  the  database  for  more  efficient  searching.  Figure  1 shows  the  system  components  and  architecture.  In  this  paper  we 
describe  the  PresentationEditor  component,  at  the  bot-tom  of  Figure  1 . 


Synchronized  Multimedia  Integration  Language  (SMIL  1.0)  [3]  is  a W3C  Recommendation  designed  for  choreographing  web-based 
multimedia  presentations  which  combine  audio,  video,  text  and  graphics  in  real-time.  It  uses  a simple  XML-based  mark-up  language, 
similar  to  HTML,  which  enables  an  author  to  describe  the  temporal  behaviour  of  a multimedia  presentation,  associate  hyperlinks  with 
media  objects  and  describe  the  layout  of  the  presentation  on  a screen.  Since  this  work  was  started,  the  W3C  SYMM  (Synchronised 
Multimedia)  Working  Group  has  released  the  SMIL  2.0  Recommendation  [4]  which  extends  the  functionality  contained  in  SMIL  1 .0 
by  enabling  interactivity  in  multimedia  presentations,  advancing  the  timing  model  and  improving  the  accessibility  features.  In  the 
work  described  here  we  have  used  SMIL  1.0  to  layout  the  presentations  spatially  and  to  temporally  synchronize  the  video  and 
PowerPoint  slides.  Possibilities  for  improvement  exist  as  SMIL  2.0  is  more  widely  deployed  and  more  SMIL  2.0  tools/players 
become  available. 

SMIL  has  the  following  advantages  to  offer: 

• Dynamic  generation  - similar  to  HTML  this  makes  for  easy,  cheap  alterations  to  presentations  as  opposed  to  complex  and 
intensive  digital  editing 

• Platform  independence  - as  a W3C  recommendation,  SMIL  is  not  a proprietary  technology  and  hence  does  not  tie  the 
implementation  to  particular  platforms  or  programming  languages; 

• Network  and  client  adaptability  - SMIL  provides  a switch  tag  which  can  be  ised  to  c^^namically  choose  the  most 
appropriate  media  object  to  stream,  depending  on  client  display  capabilities  or  connection  speed  e.g.,  use  audio  instead  of 
video  for  low  bandwidths; 

• Ready  availability  of  SMIL  players  - there  are  nine  SMIL  1 .0  players  available  covering  a wide  range  of  platforms  [5].  The 
most  popular  of  these  are  Apple's  QuickTime  4.1 , Microsoft's  Internet  Explorer  5.5  Browser  and  RealNetworks'  RealPlayer 
8. 

Figure  2 illustrates  the  Replay  interface  for  browsing  and  viewing  SMIL  presenUtions. 
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A number  of  commercial  and  research  tools  are  available  for  multimedia  and  SMIL  editing.  These  include  RealPresenter  [6], 
RealSlideshow  [7],  Fluition  [8]  and  GRINS  [9].  RealPresenter  and  RealSlideshow  are  specifically  for  generating  multimedia 
presentations  while  Fluition  and  GRINS  are  general  SMIL  editing  tools.  While  the  Real  tools  provide  good  support  for  generating 
basic  multimedia  presentations,  they  lack  the  metadata  required  for  fine-grained  searching  and  archiving  of  multimedia  presentations 
or  for  recording  changes  or  tracking  versions.  In  addition,  the  presentation  interfaces  are  somewhat  restricted  and  the  Slideshow  tool, 
which  has  perhaps  the  most  user-friendly  interface,  is  limited  to  only  providing  audio  with  the  slides.  GRINS  and  Fluition  are  both 
very  capable  SMIL  editors  but  assume  a level  of  technical  knowledge  far  above  that  which  could  reasonably  be  expected  in  this  case. 
They  also  do  not  provide  any  support  for  complex  and  fine-grained  or  editing  event  metadata. 

Hence  the  major  goal  of  the  work  described  in  this  paper  was  to  develop  a tool  that  provides  a simple,  intuitive,  graphical  user 
interface  which  supports  presentation  editing  and  a metadata  model  to  enable  both  fine-grained  searching  and  re-use  tracking.  The 
next  section  describes  the  system  design  and  functionality. 


3 System  Description 

The  PresentationEditor  is  a Visual  Basic  application  that  exploits  the  temporal  information  and  media  links  contained  within  a 
presentation’s  metadata  (XML)  file  to  divide  the  presentation  into  manageable  self-contained  segments  which  can  be  manipulated  to 
implement  easy  editing.  Each  segment  consists  of  a video  component  (video  source,  start  and  end  times)  and  its  associated  metadata 
(pointer  to  corresponding  PowerPoint  slide,  keywords,  text  etc.),  expressed  as  an  XML  file.  Using  a graphical  user  interface,  these 
segments  can  be  cut,  paste,  reordered  or  moved  within  and  between  presentations  to  create  new  virtual  presentations.  Once  the 
rearrangement  of  the  presentation  is  complete,  the  editor  generates  the  new  RealMedia  video  and  the  new  SMIL  file  which  represents 
a synchronized  presentation.  The  PresentationEditor  also  includes  search  facilities  to  help  the  user  find  relevant  segments  for 
insertion  into  the  new  presentation  and  previewing  facilities  for  viewing  draft  presentations  during  editing. 


Figure  3 illustrates  a typical  editing  process  in  which  the  user  creates  a new  virtual  presentation  by  combining  parts  of  existing 
presentations  with  new  segments  that  have  been  filmed  to  replace  outdated  sections  of  the  original  presentation. 
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Figure  3:  Steps  in  the  creation  of  a virtual  presentation 


The  PresentationEditor  incorporates  and  integrates  the  following  functionality: 

• Search  tools  to  find  and  retrieve  relevant  presentation  segments; 

• Ability  to  cut,  copy  and  paste  of  segments  both  between  and  within  presentations; 

• Re-ordering  of  slide/segments  through  drag-and-drop; 

• Direct  updating  of  metadata  for  the  presentation; 

• Automatic  adjustments  to  indexing  and  temporal  synchronization  information  resulting  from  post -production  editing  of  the 
video; 

• Updating  and  correction  to  segment  contents; 

• Previewing  of  draft  presentations  during  editing; 

• Saving  of  updated  metadata  XML  files; 

• Re-construction  of  edited  video  files  (RealMedia)  and  creation  of  new  presentations  (SMIL). 


Figure  4 illustrates  the  graphical  user  interface  to  the  PresentationEditor.  The  interface  includes  a video  replay  window,  a PowerPoint 
slide  window  and  an  editing  panel.  In  the  example  in  Figure  4:  extra  slides  have  been  added  to  the  original  presentation  (“6:  Metadata 
Standards”  and  “5:  Metadata  Requirements”);  the  original  slide  6 has  been  deleted  and  the  order  of  the  slides  in  the  original  presentation 
have  been  rearranged  using  drag-and-drop  tools  in  the  bottom  editing  panel. 


Figure  4:  PresentationEditor  Interface 


Figure  5:  Editing  event  details  form 


4 Changes  to  the  Metadata  Model 

A metadata  model  was  developed  for  the  original  system,  which  enabled  both  coarse  and  fine-grained  discovery  of  complete  presentations 
and  individual  slide-segments  respectively.  The  resource  discovery  of  complete  presentations  was  supported  through  metadata  such  as: 
presentation  location,  lecturer,  department,  institution,  description.  The  search  and  retrieval  of  segments  was  possible  through  either  fiill- 
text  or  keyword  searches  over  the  textiial  content  of  each  PowerPoint  slide.  The  segment -level  metadata  consisted  of:  source,  keywords, 
text,  start  and  end  time,  duration,  related  materials.  The  XML  Schema  corresponding  to  the  complete  metadata  model  for  the  original 
system  is  available  at  [10]. 

The  development  of  the  PresentationEditor  demanded  certain  changes  to  the  original  metadata  model.  The  recording  of  additional 
metadata  associated  with  “editing  events”  was  required  to  ensure  effective  tracking  of  re-use  and  version  control.  This  additional  metadata 
included: 

• details  of  the  person  who  performed  the  editing  (the  editor’s  Name  and  Email); 

• when  and  why  they  did  this  (Date.Modified,  Purpose); 

• the  version  number  of  the  new  presentation; 
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the  source  of  new  segments  and  their  associated  media  objects. 


Figure  5 shows  the  form  displayed  during  the  creation  process  for  the  new  virtual  presentation,  to  gather  this  additional  metadata.  The 
Date.Modified  field  is  updated  automatically  by  the  application.  The  user  is  given  the  option  to  ignore  this  metadata  update  if  they  feel 
that  there  have  been  insufficient  changes  to  warrant  the  presentation  being  defined  as  a completely  new  version  or  new  presentation. 

In  addition,  certain  metadata  associated  with  the  original  base  presentation  may  no  longer  be  applicable  (e.g.,  presentation  location)  and 
may  need  to  be  deleted  or  modified.  Changes  to  the  “segment”  metadata  were  also  required.  A “segment”  is  no  longer  purely  a video 
component  but  is  now  a presentation  component  - containing  both  slide  and  video  components  which  require  two  sets  of  associated 
temporal  data.  The  slide  component  consists  of  a pointer  to  a Powa-Point  presentation,  the  slide  number  within  that  presentation,  an 
image  location  and  the  temporal  location  of  the  segment  within  the  presentation.  This  timing  information  will  change  if  the  segment  is 
relocated  within  the  presentation.  The  video  component  consists  of  a source  file,  Sart  and  end  times  and  duration.  This  temporal 
information  identifies  the  section  of  video  within  the  source  video  that  should  be  displayed  with  the  segment  and  does  not  change  if  the 
segment  is  moved.  These  alterations  support  the  concept  of  a segment  object  that  can  be  independently  moved  within  and  between 
presentations  and  therefore  provides  critical  support  for  effective  editing. 

The  changes  to  the  metadata  model  described  here  demanded  alterations  to  the  Presentationindexer  application  and  the  legacy  metadata 
files  in  the  existing  archive.  Because  the  metadata  is  expressed  as  XML  files,  these  modifications  were  relatively  simple  to  implement. 
Once  the  Presentationindexer  was  updated,  the  legacy  metadata  XML  files  were  rapidly  regenerated.  Alternatively  the  metadata  files 
could  have  been  easily  regenerated  through  mechanisms  such  as  XSLT  or  a simple  script.  The  XML  Schema  corresponding  to  the  new 
metadata  model  is  available  at  [1 1]. 


5 Problems  and  Limitations 

Whilst  the  segmentation  approach  has  a number  of  advantages  in  this  context,  it  may  not  be  suitable  for  all  lecturing  styles.  The  indexing 
and  synchronization  of  our  SMIL  presentations  is  based  on  the  assumption  that  the  video  can  be  precisely  segmented  according  to  slide 
changes  and  that  within  each  segment,  the  lecturer  only  refers  to  the  currently  displayed  slide.  However  some  presenters  may  refer  to 
slides  other  than  the  currently -displayed  one  or  begin  an  introduction  to  the  next  slide  whilst  the  previous  slide  is  still  displayed.  Such 
situations  can  lead  to  incongruities  between  slides  and  video  content.  Irrelevant  and  distracting  information  may  be  present  or  valuable 
and  important  information  could  be  misplaced. 

A lack  of  continuity  between  contiguous  video  segments  recorded  at  different  times  or  at  different  places,  is  an  obvious  problem  that  may 
occur  within  “virtual”  presentations  and  distract  viewers  from  the  content.  A presenter  may  be  wearing  different  clothes,  or  have  a 
different  hairstyle  between  contiguous  slides.  Background  and  lighting  may  vary  widely  within  a single  virtual  presentation.  The  extent  to 
which  this  variation  disturbs  viewers  and  detracts  from  the  learning  experience  needs  to  be  assessed. 

Originally  the  presentation  incorporates  a single  continuous  video  but  after  editing  and  the  rearrangement  of  presentation  segments,  the 
video  consists  of  a collection  of  separate  video  segments  in  a particular  order.  The  segmentation  and  reorganisation  of  the  video 
components  can  be  time-  and  resource-intensive  and  result  in  decreased  video  quality  and  reduced  playback  performance.  Consequently 
the  PresentationEditor  provides  two  alternative  options  for  generating  the  virtual  presentation.  The  first  option  involves  dividing  and  then 
recombining  the  video  to  create  a single  digital  video  object  which  plays  in  parallel  with  the  synchronised  slide  images  in  exactly  the 
same  manner  as  the  original  presentations.  The  second  approach  involves  the  creation  of  a SMIL  file  which  defines  the  start  and  end  times 
of  a sequence  of  video  sections  • giving  the  illusion  of  a continuous  video  file  playing  in  parallel  with  the  slide  images.  Both  solutions 
offer  advantages  and  disadvantages.  Choosing  to  divide  and  recombine  the  video  is  extremely  time  consuming,  CPU-intensive  and 
restricted  by  a number  of  factors  including  video  frame  sizes.  It  also  requires  a large  amount  of  storage  space  and  the  quality  of  the  final 
video  is  reduced.  Never-the-less,  the  playback  quality  is  reasonable  with  no  loading  or  seek  problems  and  there  is  a single  video  file 
associated  with  the  presentation.  Using  SMIL  timing  to  artificially  rearrange  the  video  file  results  in  greatly  reduced  playback  quality.  The 
player  must  load  the  video  file  to  the  requested  start  time  at  the  beginning  of  each  presentation  segment  which  can  cause  disruptions  in  the 
synchronisation.  However  this  second  approach  is  cheaper,  faster,  more  flexible  and  less  reliant  on  external  factors  such  as  video  size  or 
quality  and,  since  the  video  file  itself  is  not  changed,  the  video  quality  remains  the  same. 

The  PresentationEditor  operates  primarily  through  the  manipulation  of  flat  XML  files,  either  SMIL  files  or  metadata  XML  files.  The 
application  does  not  support  the  direct  editing  of  digital  media  objects  such  as  the  video  files  or  PowerPoint  slides.  While  this 
ftinctionality  may  be  useful,  it  can  also  be  complex  and  resource  intensive  and  maintaining  synchronization  would  be  problematic.  One  of 
the  advantages  of  the  SMIL  Presentation  Tools  is  that  they  rely  solely  on  manipulation  of  textual  descriptions  and  thus  avoid  expensive 
and  complex  digital  video  editing. 


6 Conclusions  and  Future  Work 
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6.1  Conclusions 


The  PresentationEditor  tool  described  here  has  the  potential  to  increase  the  value  of  multimedia  presentation  content  and  enable  more 
efficioit  and  cost-effective  production  of  distance  learning  resources  by  maximizing  re-use,  providing  more  effective  metadata  and 
facilitating  ease  of  management.  It  provides  educators  with  greater  flexibility  in  the  development,  maintenance  and  management  of  a 
digital  multimedia  presentation  archive  - allowing  them  to  improve,  update  and  customize  digital  multimedia  presentations  for  specific 
audiences  and  to  provide  a richer,  more  flexible  means  of  learning,  with  a minimum  of  time,  effort  or  cost. 


6.2  Future  Work 

Our  immediate  objective  is  to  deploy  this  application  within  a university  department  and  to  carry  out  user  trials  to  obtain  feedback  from 
lecturers/content  creators  and  students  on  the  user  interface  and  the  application’s  usefulness  as  a tool  for  building  and  re-using  effective 
online  learning  resources. 

Future  improvements  in  multimedia  players  and  streaming  technologies  are  expected  to  result  in  improvements  to  the  SMIL  presentation 
development  tools,  the  replay  interface  and  the  PresentationEditor.  For  example,  RealNetworks  have  recently  released  the  RealONE 
player  [12],  which  includes  the  ability  to  display  related  hyperlinks  and  references.  Upgrading  the  system  to  SMIL  2.0,  which  provides 
greater  interactivity  and  embedded  metadata  [13],  should  also  result  in  further  changes  and  improvements. 

Finally,  an  area  of  research  requiring  more  work  is  that  of  customized  dynamic  SMIL  generation.  In  a manner  similar  to  that  of  third 
generation  HTML  web  pages  [14],  it  may  be  possible  to  combine  semantically -related  mixed  media  objects  to  dynamically  generate 
knowledge-enhanced  SMIL  presentations,  which  have  been  customized  for  particular  users  or  a given  set  of  constraints. 
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Abstract:  This  paper  introduces  and  analyzes  problems  and  issues  in  the  design  and  use  of  cognitive  tools 
in  open,  user-centered  learning  environments,  introduces  a classification  scheme  for  tool  functions,  and 
showcases  several  tools  in  a current  educational  hypermedia  research  and  development  effort. 
Implications  for  future  research  and  development  in  the  design  and  use  of  cognitive  tools  in  hypermedia 
learning  environments  are  addressed. 


Background 

Many  hypermedia  applications  support  open-ended,  user-centered,  interactive  constmctivistic  learning 
environments  (Hannafm  & Land,  2000).  In  hypermedia,  multimedia  resources  are  linked  dynamically  in  web-like  fom. 
Hyper-links  emphasize  expanding  versus  constraining  the  user*s  access  to  multimedia  resources,  thereby  supporting 
a myriad  of  potential  uses  and  learners.  Hypermedia  systems  “open  up”  knowledge  domains  and  support  learners  in 
exploring  issues  of  unique  interest  and  need  (Hannafm  & Land,  1997).  Hypermedia  systems  can  also  facilitate 
construction  via  student-centered  exploration,  manipulation,  and  inquiry  (Allen  & Hoffman,  1991;  Oliver  & Hannafm, 
2000). 

Despite  considerable  interest  and  potential,  however,  “open”  hypermedia  systems  present  unique 
challenges  to  both  designers  and  users.  Due  to  the  absence  of  explicit  pedagogical  support  typical  in  direct 
instruction,  users  can  become  disoriented  and  overtaxed  cognitively  in  open  hypermedia  learning  systems  (Roselli, 
1991).  A principal  cause  of  disorientation  and  cognitive  overload  is  the  quantity  of  simultaneous,  and  largely 
undifferentiated,  multimedia  resources  available.  The  learner  is  responsible  for  accessing,  selecting,  organizing,  and 
analyzing  information  according  to  his  or  her  unique  needs  (Jonassen  & Grabinger,  1 990;  Land  & Hannafm,  2000), 
but  the  systems  provide  little  guidance.  Learners  are  often  ill-equipped  cognitively  to  navigate  vast  multimedia 
networks  and  interpret  their  meanings. 

The  shift  in  pedagogical  control  to  the  individual  involves  providing  more  than  resources  of  increasingly 
powerful  multimedia  images;  it  involves  the  deployment  of  well-reasoned  strategic  uses  of  available  multimedia 
resources.  The  shift  requires  more  than  procedural  assistance;  it  requires  tools  that  scaffold  the  open-learning 
processes  of  diverse  learners  for  varied  purposes.  The  purpose  of  this  paper  is  to  describe  and  illustrate  the 
functions  of  cognitive  tools  in  hypermedia  learning  environments. 


Cognitive  Tools:  Functions  and  Examples 

Cognitive  tools  include  both  mental  and  computational  devices  that  support,  guide,  and  extend  the 
cognitive  processes  of  learners.  They  can  amplify  cognitive  functioning  and  facilitate  the  creation  of  personal 
knowledge  (Pea,  1985;  Salomon,  Perkins,  & Globerson,  1991).  Within  hypermedia  environments,  cognitive  tools 
enable  learners  to  regulate  the  amount,  sequence,  and  flow  of  available  resources,  make  metacognitive  judgments 
while  navigating,  and  evaluate  relevance  (Park  & Hannafm,  1993).  Cognitive  tools  may  also  assist  users  in  locating 
key  information,  recording  or  modifying  available  resources,  connecting  available  resources,  generating  and  linking 
personally  relevant  ideas  with  existing  multimedia  resources,  and  creating  individual  pathways  that  link  the  various 
multimedia  resources  contained  in  the  system. 

Tools  are  not  inherently  cognitive  independent  of  the  manner  in  which  they  are  used.  Their  functions  vary 
based  upon  the  manner  and  context  of  their  use  (Jonassen  & Carr,  2000).  In  some  instances,  a given  tool  might  be 
used  to  collect  information  (e.g.,  text,  illustrations)  from  a hypermedia  learning  environment;  in  others,  the  same  tool 
might  be  used  to  organize  relationships  among  various  resources. 


Information-Seeking  Tools 

Information-seeking  tools  are  especially  important  for  open-ended  hypermedia  learning  environments. 
Information-seeking  tools  extend  the  learners’  ability  to  identify  the  availability  of  key  information,  locate  it,  and 
differentiate  the  nature  of  the  information.  Information-seeking  includes  recognizing  and  interpreting  the  problem, 
establishing  a plan  for  searching,  conducting  the  search,  evaluating  the  results,  and  if  necessary,  repeating  the 
process  iteratively.  Metacognition  research  suggests  that  the  ability  to  reflect  on  actions  while  searching  is  critical  to 
success  during  learning  in  open-ended  environments  (Duffy  & Knuth,  1990).  To  enable  learners  to  establish  a plan 
of  search,  conduct  the  search,  and  evaluate  the  results,  cognitive  tools  should  help  learners  to  monitor  their 
information-seeking  activities. 

In  practice,  information-seeking  activities  are  influenced  by  many  learner  attributes:  prior  domain  knowledge, 
motivation,  cognitive  styles,  familiarity  with  learning  environments,  and  gender.  Cognitive  tools,  therefore,  support 
three  interrelated  seeking  functions:  locating,  viewing,  and  retrieving  relevant  information. 


Information-Presentation  Tools 

Information-presentation  tools  support  learners  as  they  attempt  to  understand  the  information  they 
encounter.  Such  tools  allow  learners  to  access,  then  represent,  information  in  varied  ways.  In  open-ended 
hypermedia  learning  systems,  users  frequently  encounter  too  much  information.  Consequently,  they  are  often  unable 
to  identify  or  establish  relationships  among  information.  Thus,  information-presentation  tools  also  need  to  help  users 
to  manage  cognitive  load  (Oren,  1990). 

Effective  information-presentation  tools  extend  three  basic  capabilities:  1)  the  ability  to  select  relevant 
attributes  and  details  while  ignoring  the  irrelevant;  2)  the  ability  to  select  information  that  enhances  interpretation; 
and  3)  the  ability  to  provide  alternative  representations.  The  same  information  can  be  represented  in  a variety  of 
forms,  such  as  verbal  statements,  mathematical  expressions,  voice  narration,  tables,  and  illustrations.  Appropriately 
represented  information  helps  learners  to  act  on  their  beliefs,  construct  higher-order  relationships,  and  establish 
conceptual  associations. 


Knowledge-Organization  Tools 

Effective  knowledge-organization  tools  enable  learners  to  manipulate  representations  and  relationships, 
promoting  unique  interpretations  of,  and  relationships  between,  the  information  encountered.  When  learners  attempt 
to  organize  vast  amount  of  new  information,  they  tend  to  oversimplify;  the  scope  and  complexity  is  often  difficult  to 
grasp.  Accordingly,  while  some  details  may  become  well-organized,  important  relationships  are  often  organized 
inappropriately  or  incompletely.  Knowledge-organization  tools,  which  allow  learners  to  organize  knowledge  from 
various  perspectives  and  dimensions,  enable  learners  to  establish  key  relationships  among  to-be-leamed  concepts. 

Perhaps  the  most  important  function  of  knowledge-organization  tools  is  the  ability  to  tentatively  structure 
(or  restructure)  information  as  it  is  encountered.  This  enables  learners  to  construct  working  models  of  the  domain 
under  study.  Knowledge-organization  tools  allow  learners  to  manipulate  information  from  various  perspectives, 
helping  them  to  organize  concepts  multi-dimensionally.  To  lessen  cognitive  load,  knowledge-organization  tools  help 
learners  to  simplify  organization  and  reduce  unnecessaiy  task  complexity. 


Knowledge-Integration  Tools 

Knowledge-integration  tools  support  the  connecting  new  with  existing  knowledge.  One  way  to  facilitate 
integration  is  to  elaborate  and  upgrade  one’s  mental  model.  Cognitive  tools  can  facilitate  conceptual  understanding 
by  supporting  the  testing  of  presumed  relationships  between  newly  organized  knowledge  and  existing  knowledge. 
Salomon  (1993)  suggested  that  computer  tools  may  be  especially  useful  in  executing  lower-level,  tedious 
computational  and  graphic  operations,  allowing  to  the  learner  to  focus  on  hypothesis  generation. 

Some  cognitive  activities  may  seem  beyond  the  reach  of  users  because  assistance  is  unavailable  to  help 
establish  connections  between  new  information  and  existing  knowledge.  Knowledge-integration  tools  can  help  to 
bridge  this  gap.  For  example,  Sherlock  /,  a computer-based  environment  for  avionics  troubleshooting,  supports 
hypothesis  testing  by  providing  multiple  paths  for  technicians  to  explore  and  coaching  the  technicians  as  they  test 
their  hypotheses  (Lajoie,  1993).  The  hierarchical  structure  of  the  problem  is  modeled  using  a menu  interface.  Learners 
can  constrain  or  expand  problem  parameters  systematically  as  they  test  the  limits  of  alternative  hypotheses. 


Knowledge-Generation  Tools 
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The  creation  of  unique  learning  artifacts  is  an  important  component  in  constructionist  views  of  learning. 
Knowledge-generation  tools  help  users  to  manipulate  and  generate  unique  interpretations  and  to  represent  newly 
generated  knowledge  flexibly  and  meaningfully.  Allowing  learners  to  represent  newly  generated  knowledge,  using 
different  perspectives  and  modalities,  is  essential  to  deep  understanding.  Handy,  a hypermedia  editing  tool,  enables 
users  to  construct  scenes  and  present  them  via  computer  (Nix,  1990).  The  user  can  generate  and  re-order  scenes 
according  to  individual  goals,  interests,  and  needs.  The  scene  can  integrate  source  media  such  as  videodisc, 
audiotape,  synthesized  voice,  digitized  voice,  animation,  and  graphics. 


Cognitive  Tools  in  Practice:  The  Human  Body 

The  Human  Body  is  an  interactive  CD-ROM  based  on  the  TV  series.  The  Universe  Within  Human  Body.  The 
Human  Body  was  designed  and  developed  by  instructional  designers,  TV  producers,  subject  matter  experts, 
teachers,  and  multimedia  developers.  This  open-ended  hypermedia  environment  contains  approximately  1,000 
individual  multimedia -enhanced  database  screens  featuring  computer  graphics,  digital  video,  sound,  and  text  that 
support  learner-centered,  construct! vistic  learning  in  introductoiy  anatomy  and  physiology  (liyoshi  & Kikue,  1995, 
1996). 

A variety  of  information-seeking  tools  are  provided  such  as  an  alphabetical  index,  keyword  search, 
hypertext  and  hyperpicture  links,  and  concept  maps.  These  tools  enable  learners  to  locate  information  they  need 
from  various  perspectives;  using  their  prior  knowledge,  conceptual  relationship  among  text  and  pictorial  information, 
and  domain  structure.  For  example,  using  the  Structure  Map  tool  illustrated  in  Figure.  1,  the  learner  can  determine  the 
availability  of  a concept  using  a hierarchically -structured  map  of  each  human  body  system.  All  components  are 
displayed  using  both  visual  images  and  their  corresponding  names.  The  related  information  for  each  topic  can  be 
accessed  by  clicking  the  image  or  tenn  of  each  concept.  This  tool  provides  a topical  overview  of  related  terms  and 
concepts,  as  well  as  a means  for  accessing  any  or  all  of  the  terms  and  concepts. 

Tools  are  also  provided  to  support  information  selection  and  embedding  into  presentations.  The  Bookmark 
tool  shown  in  Figure.  2 allows  the  learner  to  “flag”  key  information  they  encounter,  then  subsequently  access  that 
information  quickly  and  accurately.  The  learner  can  place  a bookmark  on  any  information  screen,  and  generate  a list 
of  all  selected  screens.  In  effect,  the  cognitive  load  associated  with  ongoing  review  is  managed  by  selecting  screens 
for  subsequent  review. 


Figure  1 : Structure  Map  Figure  2 : Bookmark 

As  previously  noted,  tools  are  not  inherently  cognitive;  the  context  and  nature  of  their  use  determine 
whether  or  not  the  tool  augments,  extends,  or  enhances  the  cognitive  processes  of  users,  and  the  manner  in  which 
such  processes  have  been  influenced.  Stated  differently,  the  same  computer  tool  can  support  multiple  cognitive 
processes.  For  instance,  the  previously  described  Structure  Map  presents  a detailed  concept  map  of  each  system 
also  helps  the  learner  to  construct  and  elaborate  the  relationship  between  and  among  the  systems  and  their 
components.  Likewise,  the  Bookmark  also  enables  learners  to  traverse  selected  information  screens  in  user- 
customized  ways. 

The  Text  Memo  tool,  shown  in  Figure.  3,  also  supports  information  organization.  It  allows  the  user  to 
elaborate  or  annotate  information  contained  in  any  information  screen.  When  a learner  writes  a memo,  the 
corresponding  screen  is  automatically  bookmarked.  This  electronic  memo  can  then  be  linked  to  other  information 
within  the  system.  The  tool  aids  not  only  in  information  organization,  but  also  supports  the  development  of  user- 
specified,  customized  links. 

To  integrate  newly  organized  information  into  pre-existed  knowledge,  metacognitive  tools  are  provided. 
Path  Tracker  plays  back  all  the  information  screens  the  learner  went  through  since  beginning  study.  This  helps 


871 


3EST  COPY  AVAIMBtt 


learners  to  monitor  what  they  have  learned  for  better  integration  of  their  knowledge.  Another  example  is  Reviewer 
(Figure.  4),  which  allows  learners  to  monitor  the  information  screens  they  examine.  The  ratio  of  the  number  of  screens 
examined  to  the  total  screens  in  a particular  component  or  a system  is  displayed.  The  tool  is  useful  to  identify  the 
distribution  of  information  they  have  looked  through  the  system. 


Figure  3 : Text  Memo  Figure  4:  Reviewer 

In  addition,  some  tools  ad  learners  in  both  integrating  their  knowledge  as  well  as  in  generating  new 
knowledge.  The  Presentation  Maker  enables  learners  to  create  individual  collections,  combining  information 
screens  and  their  personal  comments.  As  shown  in  Figure.  5,  users  can  organize  bookmarked  and/or  annotated 
resources  via  the  Text  Memo  tool  to  indicate  how  the  body  reacted  to  create  a personal  affliction  (e.g.,  heart  attack  of 
a family  member).  System  contents  are  both  modified  according  to  unique  experiences  and  integrated  with  the 
experiences  of  each  individual  user.  In  the  presentation  mode  illustrated  in  Figure.  6,  each  screen  can  be  displayed 
according  to  the  user’s  specifications. 


Figure  5:  Presentation  Maker  (Authoring  Mode)  Figure  6:  Presentation  Maker  (Presentation  Mode) 


Implications 


Hypermedia  systems  provide  open-ended  environments  for  learner-centered,  constructivistic  learning. 
However,  learners’  cognitive  resources  are  often  overtaxed  when  exploring  these  vast  information  networks,  limiting 
their  ability  to  use  the  systems  effectively.  Although  tools  have  supported  cognitive  processes  in  directed  learning 
contexts,  research  is  needed  to  better  understand /low  they  support  processing  in  open-ended  learning 
environments.  Several  significant  research  issues  remain  related  to  the  design  and  use  of  cognitive  tools. 
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Facilitation  of  Tool  Use 


Simply  providing  cognitive  tools  with  open-ended  hypermedia  systems  ensures  neither  usage  nor  success. 
We  need  to  better  understand  how  to  facilitate  the  use  of  multiple  tools;  we  also  need  to  learn  more  about  how 
creative  users  actually  employ  the  tools  we  provide.  Providing  a variety  of  tools  may  be  important  to  accommodate 
learners  with  different  prior  Imowledge  and  tool-use  skills.  While  it  is  evident  that  well-designed  and  implemented 
tools  can  facilitate,  it  is  also  apparent  that  tool  use  can  hinder  learning  and  performance.  The  cognitive  load 
associated  with  tool  use  can  actually  increase  rather  than  lessen  the  demands  of  the  learning  task.  Individuals  must 
often  invest  considerable  cognitive  resources  learning  how  to  use  a tool;  this  problem  becomes  magnified  as  the 
number  and  variety  of  tool  features  increase.  It  is  important  to  determine  how  tool  use  can  be  facilitated — both  prior 
to  as  well  as  during  use — to  support  user-centered  learning. 


Domain-Free  Tools  Versus  Domain-Specialized  Tools 

The  rapid  growth  in  open  hypermedia  systems,  such  as  the  World  Wide  Web,  suggests  a paramount  need 
for  generalizable  cognitive  tools.  However,  such  tools  have  rarely  been  developed  much  less  validated.  Increased 
attention  to  the  design  of  generic  cognitive  tools,  rooted  in  theory  and  research  on  human  cognition  and  open-ended 
learning  rather  than  particularized  domain  nuances,  is  needed.  It  may  be  possible  to  provide  “meta  tools”  that  help 
learners  to  construct  and  customize  tools  needed  for  a content  domain,  that  is,  to  select  and  adapt  from  a suite  of 
tools  that  are  well-known  and  broadly  applicable.  While  the  widespread  interest  in  tools  that  are  uniquely  crafted  to 
support  learning  in  particular  domains  has  proven  the  viability  and  utility  of  cognitive  tools,  research  on  scalable 
tools  is  needed. 


Evaluation  of  Tool  Use 

In  order  to  examine  how  cognitive  tools  are  used  to  support  learners  with  open-ended  hypermedia  learning 
environments,  close  evaluation  of  actual  tool  use  is  critical.  Several  significant  questions  need  to  be  addressed. 

(1)  Are  tools  used  as  initially  intended? 

(2)  Do  patterns  of  cognitive  tools  utilization  exist? 

(3)  How  do  individuals  use  multiple  cognitive  tools  in  their  learning? 

The  proposed  research  is  of  significant  potential  consequence.  Few  researchers  have  immersed  themselves 
in  issues  of  design,  and  few  designers  are  attuned  to  available  research  and  theory.  Tremendous  interest  has  been 
generated  in  open  learning  environments,  but  little  research  and  theory  is  available  to  guide  or  support  the  interest. 
We  have  taken  large  steps,  but  we  may  not  be  making  needed  progress.  Neither  the  research  nor  the  applied 
community  has  been,  or  will  likely  be  able,  to  singularly  advance  the  state  of  the  art.  Greater  convergence  of  interest 
and  expertise  between  theoreticians  and  practitioners  is  needed. 
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Abstract:  This  paper  analyzes  problems  and  issues  in  making  knowledge  of  teaching 
and  learning  visible  and  sharable  online.  It  also  describes  some  of  the  ongoing  research 
and  development  efforts  at  the  Carnegie  Foundation’s  Knowledge  Media  Laboratory  that 
advance  this  work  through  the  use  of  emerging  technologies.  Implications  for  future 
research  and  development  are  also  addressed. 


Background  and  Rationale 

The  Carnegie  Knowledge  Media  Laboratory  (KML)  uses  some  of  the  newest  technologies  to 
challenge  one  of  the  oldest  problems  in  education — making  the  knowledge  of  teaching  and  learning  visible 
and  sharable  through  the  use  of  multimedia  and  the  Internet.  Beyond  increasing  the  quantity  or  availability 
of  materials  for  teachers  to  use,  the  KML  hopes  to  support  the  development  of  a qualitatively  different 
approach  to  professional  development:  one  which  views  faculty  as  critical  producers  of  knowledge  and  the 
Internet  as  a new  medium  through  which  they  can  represent  and  exchange  aspects  of  their  practice  and 
knowledge  that  often  cannot  be  captured  in  conventional  written  texts. 

In  order  to  accomplish  this  mission,  the  work  of  KML  addresses  three  critical  issues.  First,  merely 
documenting  what  goes  on  in  a classroom  or  course  does  not  communicate  what  teachers  have  learned  or 
the  ideas  and  expertise  they  have  developed.  For  example,  if  one  were  to  film  every  hour  of  class  time  and 
put  every  document  related  to  a class  online,  it  would  be  extremely  difficult  to  make  sense  of  all  that 
material  or  to  figure  out  why  the  teacher  had  designed  the  course  as  they  did  or  what  they  had  learned  in  the 
process.  It  would  be  like  looking  at  all  the  field  notes  that  an  anthropologist  collected  or  all  the  data  from 
an  experiment  without  any  of  the  rationale,  context,  or  analysis  that  the  anthropologist  or  scientist  provides 
in  a book  or  document.  Second,  even  if  many  teachers  do  succeed  in  developing  and  producing  books, 
videos,  websites  or  other  products  that  reflect  the  knowledge  implicit  in  their  teaching,  there  are  few  means 
for  helping  other  educators  or  the  wider  public  to  find  and  take  advantage  of  those  that  may  be  most  useful 
for  them.  Third,  even  if  faculty  members  can  find  a wide  range  of  examples  of  the  scholarship  of  teaching 
relatively  easily,  there  is  no  guarantee  that  they  will  choose  to  do  so.  It  may  take  considerable  time  before 
people  discover  how  to  take  advantage  of  and  build  on  the  knowledge  of  teaching  and  learning  that  can  be 
represented  in  multimedia  and  online  materials. 

Over  the  past  three  years,  the  KML  has  pursued  a set  of  activities  which  address  these  problems. 

In  particular,  since  few  models  or  examples  exist  that  can  give  faculty  a chance  to  imagine  how  they  can 
use  the  online  environment  to  create  new  and  more  powerful  representations  of  their  practice,  we  have 
worked  with  accomplished  faculty  in  both  K-12  and  higher  education  to  create  a small  number  of 
multimedia  websites  (Figure  1 & Figure  2). 
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Figurel:  Website  Example  1 


Figure  2:  Website  Example  2 


These  sites  model  how  teachers  can  organize  carefully  selected  materials,  images,  videos,  and 
reflections  that  make  the  ideas  and  expertise  that  go  into  their  teaching  public  and  available  for  others  to 
examine  and  build  upon  (Cambridge,  et  al.,  2001).  In  order  to  make  it  easier  for  faculty  to  create  these 
kinds  of  sites  and  CD-ROMs,  DVDs,  or  videos  in  which  they  articulate  their  ideas  and  examine  their 
practice,  we  have  also  developed  a small  set  of  resources  and  formed  an  online  gallery  and  exhibition  space 
for  the  display  of  their  multimedia  work.  In  addition,  we  have  established  a virtual  workspace  that  now 
supports  the  review  and  exchange  of  work  by  over  two  hundred  scholars  who  are  participating  in  the 
Carnegie  Academy  for  the  Scholarship  of  Teaching  and  Learning  (CASTL), 

At  the  KML,  we  are  currently  working  with  the  groups  and  individuals  affiliated  with  the 
Foundation’s  programs  to  expand  this  work  by: 

• Inventing  new  means  for  faculty  to  develop  and  represent  the  knowledge  needed  to  design  powerful 
learning  experiences. 

• Constructing  the  tools  and  resources  that  faculty  can  use  to  develop  and  represent  their  knowledge, 

• Fostering  the  exchange  and  use  of  that  knowledge  in  more  effective  educational  programs. 

Ultimately,  we  believe  that  these  activities  will  help  to  ensure  that  the  tremendous  energy  and 
resources  that  are  being  invested  in  putting  courses  and  resources  for  both  teachers  and  students  online 
actually  lead  to  significant  improvements  in  teaching,  learning,  and  professional  development. 

In  education,  with  the  development  of  the  Internet  and  digital  media  in  particular,  the  latest  wave 
of  changes  in  technology  are  making  their  way  into  the  lives  of  teachers  and  students.  While  equity  of 
access  remains  a constant  concern,  one  can  imagine  a time  in  the  not  too  distant  future  when  students  and 
teachers  in  many  different  communities  can  have  computers  and  Internet  hookups  and  communicate  and 
exchange  information  and  ideas  with  their  peers;  when  they  can  use  digital  cameras  and  video  to  conduct 
research  and  record  their  work  to  present  to  others;  and  when  they  can  design  and  take  advantage  of  a wide 
range  of  online  classes  for  their  personal,  academic  and  professional  development.  But  how  will  they  take 
advantage  of  the  opportunities  to  use  these  new  technologies  to  share  ideas  that  can  improve  teaching  and 
learning? 

The  impact  of  new  technologies  on  the  educational  system  depends  on  a host  of  factors  outside  the 
control  of  a single  organization  or  initiative.  At  the  same  time,  the  Carnegie  Knowledge  Media  Laboratory 
is  poised  to  carry  out  the  kind  of  research  and  development  that  can  demonstrate  how  new  technologies  can 
transform  key  aspects  of  teachers’  work  and  professional  development.  Through  strategic  initiatives  and 
the  national  reach  of  the  Carnegie  Foundation,  the  KML  can  help  educators  to  build  the  understanding, 
resources,  and  support  needed  to  use  emerging  technologies  in  new  ways  to  transform  the  development  and 
dissemination  of  knowledge  among  teachers  at  all  levels. 

Making  Teaching  Visible 

Already  teachers  can  get  access  to  lesson  plans,  standards,  syllabi  and  other  teaching  resources 
through  the  Internet.  In  some  cases,  they  can  see  the  work  of  students  in  other  schools  and  classrooms  by 
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visiting  recently  created  school  websites.  Nevertheless,  while  the  possibility  to  place  many  of  the  artifacts 
and  resources  of  teaching  online  seemed  particularly  promising  initially,  in  our  work  at  the  KML  we  have 
seen  that  simply  making  a large  number  of  teaching-related  materials  available  through  the  Internet  does 
not  provide  easy  access  to  the  ideas  and  expertise  that  went  into  the  development  of  those  documents. 

For  example,  the  Internet  is  often  used  primarily  to  present  the  same  kinds  of  lesson  plans  or 
syllabi  that  one  can  receive  in  a paper  format.  Furthermore,  although  some  sites  link  these  lesson  plans 
with  related  resources,  student  work,  or,  increasingly  for  K- 1 2 work,  lists  of  standards,  relatively  few 
integrate  sites  these  materials  with  the  reflections  and  examinations  by  faculty  that  can  make  visible  the 
thought  that  went  into  the  design  of  courses  or  lessons  or  what  the  faculty  member  learned  in  the  process. 
As  a result,  many  critical  aspects  of  teaching  and  learning  - classroom  discussions  and  interactions, 
relationships  among  students  and  between  teachers  and  students  — remain  invisible  and  unexamined  and  the 
culture  and  character  of  classroom  activity  remain  undocumented.  Under  these  circumstances,  viewers 
cannot  easily  assess  the  value  of  many  of  the  materials  that  exist  online  or  learn  much  about  how  to  build 
them  into  their  own  teaching  practice. 

In  contrast,  the  websites  that  the  KML  has  helped  to  develop  over  the  past  three  years  explore  new 
ways  of  integrating  teachers’  reflections  and  conclusions  about  their  teaching  with  extensive  collections  of 
their  materials  - including  syllabi,  assignments,  student  work,  rubrics,  images  from  the  classroom,  and 
videos  of  classroom  discussions  and  presentations  (liyoshi.  Hatch,  and  Pointer,  2000).  Over  the  next  five 
years,  we  seek  to  expand  this  initiative  on  “making  teaching  visible”  to  develop  new  and  even  more 
succinct  and  accessible  formats  for  organizing  and  displaying  teachers’  knowledge  and  to  explore  the 
possibilities  for  creating  school  websites  that  capture  and  convey  the  ideas  and  practices  of  particularly 
effective  programs  and  institutions  in  K-12  and  higher  education. 


Building  Teachers’  Knowledge 

In  addition  to  the  growth  of  online  teaching  materials,  numerous  campuses  and  organizations  are 
at  work  creating  online  courses  and  web-based  learning  platforms.  Already,  many  universities  have  “virtual 
offerings”  - for  students,  alumni  and  others  - and  organizations  like  the  “virtual  high  school”  are  creating 
online  classes  that  give  younger  students  across  the  country  access  to  faculty  and  subjects  that  might  not 
ordinarily  be  available  to  them.  Web  CT,  Blackboard,  and  other  companies  are  establishing  online 
platforms  that  foster  communication  and  the  sharing  of  resources  among  students  and  faculty.  But  so  far 
these  courses  and  platforms  are  used  primarily  for  the  conventional  delivery  of  information  not  the 
production  of  knowledge  or  the  generation  of  new  ideas.  In  order  to  create  “knowledge-building  systems” 
in  which  many  faculty  can  share  their  ideas  and  build  on  the  work  of  others,  they  need  tools  and  resources 
that  even  those  without  extensive  technical  expertise  can  use  to  make  their  teaching  visible. 

The  KML  is  currently  developing  a variety  of  tools  and  online  portfolio  templates  that  can  be  built 
into  online  learning  platforms  (Figure  3 & Figure  4).  In  conjunction  with  articles,  presentations  and  online 
guides  already  in  development,  these  tools  and  templates  will  provide  faculty  with  the  means  to  collect  and 
organize  their  teaching  materials  and  make  what  they  have  learned  available  to  others.  Furthermore,  we 
will  work  with  those  who  are  developing  these  platforms  to  take  advantage  of  the  possibilities  that  these 
kinds  of  tools  and  others  can  offer  to  faculty.  By  prototyping  tools  and  temp  lates  in  this  way,  the  KML  will 
increase  the  chances  that  faculty  across  the  country  have  relatively  easy  access  both  to  the  materials  and 
expertise  of  their  colleagues. 
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Figure  3:  Tool  Example  Figure  4:  Portfolio  Template  Example 


Creating  a Professional  Knowledge-Base 

Along  with  the  growth  in  the  forums  and  platforms  for  sharing  their  examinations  of  classroom 
practice  online,  faculty  at  many  levels  are  also  encountering  more  online  opportunities  to  discuss  one 
another’s  work.  Many  schools,  departments,  disciplines,  and  professional  organizations  are  already 
sponsoring  listservs,  discussion  forums,  “virtual  chats,”  and  online  conferences.  However,  our  experiences 
at  the  KML  and  our  work  with  others  in  the  field  suggest  that  these  resources  remain  underused;  they 
remain  the  province  of  those  who  are  most  interested  in  technology,  and  they  do  not  necessarily  lead  to  the 
level  of  critique  and  discussion  that  can  promote  significant  and  widespread  improvements  in  teaching 
practice. 

In  our  own  work  at  the  KML,  we  have  created  a “gallery  for  the  scholarship  of  teaching  and 
learning”  (http://www.camegiefoundation.org/KML/)  in  which  faculty  can  view  some  of  the  multimedia 
examples  that  we  have  developed  in  conjunction  with  CASTL  faculty  (Figure  5).  Furthermore,  we  have 
pioneered  the  development  of  a virtual  workspace  that  can  support  the  review  and  exchange  of  many 
aspects  of  teaching  including  videos,  images,  student  work,  assignments,  reflections,  and  articles  and 
presentations  (Figure  6).  This  workspace  has  grown  into  a rich  resource  with  contributions  from  over  200 
faculties  from  the  K-12  and  higher  education  programs  of  the  Carnegie  Academy  of  the  Scholarship  of 
Teaching  and  Learning  (liyoshi,  2000).  Yet,  despite  the  possibilities  of  the  virtual  environment,  the  faculty 
who  view  these  materials  often  respond  in  the  same  way  that  they  do  to  materials  they  find  in  journals  or 
hear  about  in  workshops  or  presentations.  While  they  refer  to  and  draw  on  these  materials,  they  seem  to  be 
more  likely  to  use  this  material  “offline”  either  in  the  development  of  their  own  work  or  in  face-to-face 
discussions  or  private  email  exchanges. 


Figure  5:  KML  Gallery  Figure  6 : Carnegie  Workspace 
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In  order  to  better  understand  how  to  stimulate  and  support  online  opportunities  for 
communication,  exchange  and  peer  review,  in  the  next  year  we  plan  to  examine  the  most  effective  sites  and 
contexts  for  the  exchange  and  development  of  knowledge  of  teaching.  To  follow-up  on  that  research,  in 
conjunction  with  our  partners  in  other  Carnegie  programs,  we  plan  to  establish  a small  number  of  web 
forums,  galleries  and  exhibition  spaces  that  create  a context  to  display  representations  of  classroom 
practice,  and  more  significantly,  foster  the  critical  examination  and  use  of  those  representations.  For 
example,  as  part  of  a major  initiative  on  learning  in  the  liberal  arts  carried  out  by  Carnegie  and  the  Hewlett 
Foundation,  we  expect  to  work  with  selected  liberal  arts  colleges  to  create  websites  that  capture  their 
approach  to  learning  and  provide  the  materials  and  information  that  can  be  used  to  assess  that  approach. 
Ultimately,  we  expect  that  these  sites  could  be  accessed  through  a web-based  gallery  that  facilitates 
comparisons  and  fosters  exchanges  across  colleges.  In  this  manner,  we  hope  to  contribute  to  the 
development  of  a professional  knowledge-base  that  extends  beyond  the  individuals  and  across  institutions. 


Imagining  a Future  that  Demands  High  Quality  Teaching 


Together,  these  major  initiatives  of  the  KML  are  designed  to  contribute  to  a future  in  which  many 
faculty  members  can  be  involved  in  very  different  kinds  of  activities  than  they  engage  in  today: 


We  imagine  that  faculty  will  begin  planning  learning  activities  by  looking  not  only  at  related 
syllabi  and  lesson  plans  but  also  by  examining  the  analyses,  ideas  and  knowledge  that  have  gone 
into  the  teaching  of  their  colleagues.  In  the  process,  they  will  also  be  able  to  access  student  work 
and  faculty  comments  and  evaluations  so  that  they  can  develop  their  own  capacity  to  judge  their 
students’  learning  and  compare  their  students’  performances  to  others. 


During  their  courses,  faculty  will  put  many  of  their  assignments  and  resources  online,  and  students 
will  be  able  to  submit  some  or  all  of  their  work  in  an  electronic  form.  As  a result,  as  the  course 
proceeds  or  afterwards,  with  relatively  little  effort,  faculty  will  be  able  to  create  web  pages  or 
“electronic  portfolios”  that  illustrate  significant  findings  and  analyze  key  artifacts  and  resources. 

After  the  completion  of  a course,  should  they  wish  to,  faculty  could  contribute  their  web  pages  or 
“portfolios”  to  archives  and  forums  organized  by  their  school,  discipline,  or  professional 
organizations.  These  archives  and  forums  could  provide  a platform  for  deep  and  rich  discussions 
of  the  teaching  taking  place  in  many  levels  across  the  country.  In  turn,  these  models  of  teachers’ 
work  and  the  related  discussions  could  inspire  others  to  make  changes  in  their  own  practice  and 
document  the  results. 


While  these  activities  focus  directly  on  using  new  media  and  emerging  technologies  to  develop  a 
high  quality  teaching  force  and  create  an  accessible  professional  knowledgebase,  they  also  create 
opportunities  for  many  people  to  see  what  really  goes  on  in  classrooms.  By  making  teaching  public  online, 
for  the  first  time,  administrators  and  policymakers  will  be  able  to  see  what  really  goes  on  inside  schools  and 
classrooms.  Should  faculty  or  schools  and  colleges  wish  to,  they  will  have  the  materials  and  the  means  to 
help  parents  and  other  members  of  their  school  communities  to  see  beyond  grades  and  test  scores,  instead, 
they  will  get  a glimpse  of  the  kinds  of  classes  and  work  in  which  students  are  involved.  As  some  schools 
and  colleges  make  the  teaching  that  goes  on  inside  them  public,  students  and  their  parents  may  well  begin 
to  expect  to  be  able  to  view  the  teaching  that  goes  on  in  others  and  to  demand  the  kind  of  high  quality 
instruction  they  can  see  in  some  of  the  most  advanced  schools  and  institutions. 

It  is  these  kinds  of  demands  to  see,  analyze,  and  understand  teachers’  work  that  require  changes  in 
priorities.  Those  changes  in  priorities,  in  turn,  can  create  changes  in  the  ways  time  is  spent.  With  new 
demands  to  see  what  is  really  going  on  in  classrooms,  K-12  teachers  may  be  able  to  shift  some  of  their  time 
so  they  can  engage  in  the  kind  of  reflection  and  professional  development  they  need  to  teach  effectively 
and  efficiently.  Similarly,  this  kind  of  demand  can  begin  to  influence  the  formal  and  informal  reward 
systems  and  provide  some  new  incentives  for  faculty  in  higher  education  to  focus  on  their  teaching  and 
give  teaching  the  attention  it  deserves. 
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Abstract:  The  purpose  of  this  research  is  to  establish  a community  model  in  a large 
number  of  classrooms.  “Rice  Club”  is  a Web  community  to  promote  collaborative 
learning  with  a TV  program.  Teachers  and  students  exchange  their  opinion  on  the  TV 
program  and  relevant  practices  on  the  Web.  The  result  of  log  analysis  and  interviews  to 
the  teachers  show  that  the  choice  of  communication  tools  is  conducted  by  intentions  of 
the  teachers  and  their  curriculum. 


Introduction 

NHK  educational  channel  has  been  broadcasting  “Okome,”  rice  in  Japanese,  as  a series  of  TV  program 
for  elementary  school’s  students  since  April  2001.  They  learn  about  rice  from  various  aspects,  such  as 
cultivation,  gastronomic  culture,  and  circulation  of  rice.  Inagaki  et  al.  (2001)  developed  “Rice  Club” 
which  works  on  the  Web  for  inter-classroom  collaborative  learning.  It  has  four  types  of  asynchronous 
chatting  board  (BBS).  Each  BBS  plays  a different  role  for  collaboration  (Table.  1).  Kurokami  et  al.  (1999) 
pointed  out  that  it  is  necessary  for  inter-classroom  collaboration  to  have  both  personal  communication 
and  classroom  communication.  As  the  solution,  the  four  types  of  BBS  in  Rice  Club  is  designed  to  cover 
both  levels  of  communication. 

Method 

For  encouraging  to  build  the  Web  community,  this  project  started  with  13  invited  classrooms  on  April 
2001.  The  coordinators  explained  how  to  use  BBS  as  occasion  demands.  By  the  end  of  March,  65 
classrooms  participated  and  10,258  messages  are  currently  on  the  BBS.  We  focused  on  the  longitudinal 
change  of  the  numbers  of  opinion  on  each  BBS.  It  will  reflect  the  qualitative  shift  of  communication  in 
classrooms.  In  addition,  to  grasp  the  purpose  of  teachers'  usage  of  BBS,  interviews  to  13  teachers 
including  9 invited  classroom  teachers  and  4 volunteer  classroom  teachers  are  conducted. 


Result 

From  the  trends  of  messages  and  the  BBS  (Fig.  1),  the  quality  of  the  Web  community  shifted  from 
training  with  non-friendly  communication  in  all  participants  to  intensive  collaboration  in  the  specific 
classrooms.  In  the  beginning,  they  used  Open-ended  Type  boards  for  a training  to  send  topics  triggered 
by  the  TV  program  and  to  find  out  suitable  classrooms  for  collaboration.  By  September,  about  a half-year 


Discussions  among 
the  students 

Open-ended  Type 

General  discussion  about  the  TV  program 
Various  messages  from  all  participating  students 

Specific  Type 

In-depth  discussion  about  a specific  theme. 
Host  school  decide  the  members  and  topics 

Presentations  of  the 
Classroom 

Diary  Type 

Activities  of  the  classrooms  to  text  and  pictures. 
Other  school  can  send  comments  . 

Web  Page  Type 

Activities  of  the  classrooms  to  Web  pages. 
Other  school  can  send  comments 

Table  1.  Types  of  BBS 


Messages 


Diary 


Classrooms 
70 

60 


40 

30 

20 

10 

0 


2001 


2002 


Figure  1.  Number  of  participated  classrooms  and  Messages 
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Figure  2.  Messages  by  BBS  type  in  three  different  collaborations 


passed  of  the  grade,  Specific  Type  boards  and  Diary  Type  boards  became  to  be  used  much  more.  The 
students  had  well  consciousness  for  the  partner  and  grew  up  messages  about  the  focused  theme  in  the 
Specific  Type  boards.  On  the  other,  Diary  Type  boards  tend  to  use  for  wide  open  communication  among 
various  classrooms. 

From  the  result  of  interviews,  three  types  of  collaboration  are  extracted;  Open  Collaboration,  In-depth 
Collaboration,  and  Mixed  Collaboration  (Fig.  2).  These  differences  are  caused  by  an  influence  of 
teacher’s  attitude  toward  their  practice.  In  open  collaboration,  studentes  made  short  term  collaborations 
with  any  other  students  from  any  classrooms.  The  teachers  let  their  students  to  use  Open-ended  type  and 
Diary  type  boards.  On  the  contrary,  the  teachers  who  expected  In-depth  collaboration,  had  a tendency  to 
fix  the  classroom  to  exchange  opinions  and  narrow  the  argument  down  on  a specific  theme.  The  theme 
was  decided  through  the  negotiation  on  Specific  Type  board.  Mixed  Collaboration  was  used  in  order  to 
the  both  purpose  of  the  above  collaborations. 


Conclusion 

From  the  log  analysis,  it  is  showed  that  the  quality  of  community  gradually  changed  from  non-friendly 
communication  in  all  participants  to  well-consciousness  collaboration  with  a specific  partner.  In  addition, 
it  is  clear  from  the  interviews  that  there  were  three  types  of  collaboration  by  the  purpose:  Open 
Collaboration,  In-depth  Collaboration,  and  Mixed  Collaboration.  In  a conclusion,  intentions  of  teachers 
were  clearly  concerned  for  collaborative  learning.  That  is,  curriculum  of  teachers  affects  process  of 
students'  collaboration.  Therefore,  differences  in  types  of  BBS  are  well  understood  by  teachers  and 
students  and  used  as  we  designed.  To  promote  collaboration  on  the  BBS  more,  we  should  strengthen  the 
feature  of  each  BBS. 
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ABSTRACT:  Alberta  recently  mandated  an  ICT  Program  of  Studies  that  requires  the  infusion  of 
digital  technologies  into  core  curricula  and  across  all  grades.  Effective  implementation  of  ICT 
demands  that  all  teachers  create  inquiry -based,  technology  enabled  learning  environments  for 
children.  The  design  of  digital  and  media  rich  learning  experiences  calls  for  a dramatic  shift  in 
teacher  preparation.  Student  teachers  must  routinely  encounter  the  effective  infusion  of  technology  in 
all  aspects  of  on-campus  learning  as  well  as  in  classroom  placements  in  schools.  This  paper  describes 
the  design,  implementation  and  evaluation  of  an  undergraduate  seminar  offered  to  student  teachers  in 
the  fourth  semester  of  a two-year  degree  program  at  the  University  of  Calgary.  Outlined  are  ways  in 
which  fundamental  issues  in  technology  integration  were  addressed,  how  successes  were  achieved, 
and  how  discoveries  from  the  first  two  seminars  direct  on-going  modifications. 
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In  September  2000,  teachers  in  Alberta,  Canada  began  a three-year  implementation  process  for  an  ICT  Program  of 
Studies  (Alberta  Learning,  2000).  This  innovative  curriculum,  demanding  the  effective  infusion  of  technology  for 
communicating,  inquiring,  problem-solving  and  decision-making  in  core  curricula,  raises  important  questions  about 
what  it  means  for  students  to  think  and  learn  with  the  full  range  of  digital  technologies  that  are  so  much  a part  of 
today’s  changed — and  changing — world. 

Educational  technology  research  (Clifford  & Friesen,  2001a;  diSessa,  2000;  Goldman-Segall,  1998;  Jacobsen  & 
Goldman,  2001;  Papert,  2000,  1980)  and  the  experience  of  the  Galileo  Educational  Network  (http://www.galileo.org) 
paint  a similar  picture  of  what  thinking  and  learning  need  to  look  like  in  a knowledge  era.  We  no  longer  live  in  a 
world  in  which  information  is  scarce.  The  old  certainties  of  a world  defined  by  four  classroom  walls  and  impermeable 
boundaries  have  disappeared  forever,  replaced  by  global  interdependencies  and  complex  systems  that  require 
flexibility,  responsiveness,  and  imagination.  Overwhelmed  by  information  from  a wealth  of  sources,  students 
desperately  need  the  skills  to  create  new  knowledge,  not  just  consume  the  old.  Problems  do  not  come  neatly  packaged, 
defined-in-advance,  and  amenable  to  the  rote  application  of  familiar  strategies. 

Our  society  can  no  longer  afford  to  think  of  engaged  learning,  nimbleness,  creativity  and  commitment  to  action  as 
educational  embellishments  to  “the  basics”.  Multiple  and  conflicting  perspectives  are  not  problems  to  be  fixed, 
ignored  or  eliminated;  they  are  the  way  the  world  works.  Our  human  survival  depends  on  an  ability  to  learn  new 
things,  imagine  creative  possibilities  and  design  useful  solutions  in  deeply  ambiguous  and  confusing  situations.  It 
depends  on  our  ability  to  teach  our  children  how  to  do  this,  too.  And  thus,  questions  of  how  to  prepare  a new 
generation  of  teachers,  many  of  whom  have  been  schooled  in  old  ways  despite  their  relative  youth,  are  increasingly 
pressing  in  their  urgency. 

NEW  WAYS  OF  LEARNING  DEMAND  NEW  WAYS  OF  TEACHING 

Today’s  classrooms  do  not  look  much  different  than  they  did  20  years  ago  when  schools  began  to  invest  heavily  in 
technology.  While  recognizing  that  there  are  pockets  of  genuine  innovation  in  classrooms,  schools,  and  universities 
across  North  America,  we  feel  confident  in  making  a few  generalizations  about  the  current  state  of  affairs.  First,  while 
many  school  and  university  students  are  using  technology  in  their  personal  lives  in  a wide  variety  of  ways,  they  are 
not  using  computers  very  extensively  in  classrooms  in  order  to  learn  effectively  in  a variety  of  subject  areas.  The  gap 
between  presence  and  use  of  technology  is  particularly  wide  in  high  school  (Cuban,  Kirkpatrick  & Peck,  2001).  In 
fact,  there  is  a growing  “digital  divide”  between  what  students  actually  do  with  technology  at  home  and  what  they  are 
allowed  to  do  in  school.  A growing  number  of  students  routinely  expect  their  school  computers  to  be  out  of  date, 
connectivity  to  be  slow,  networks  to  be  unstable,  and  their  teachers’  knowledge  and  confidence  about  technology  to  be 
significantly  less  than  their  own. 
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A second  general  trend  is  that  many  classroom  teachers  and  faculty  members  in  teacher  preparation  programs  lack 
confidence  in  their  own  ability  to  think  broadly  with  technology.  Conventional  models  of  professional  development, 
like  workshops  and  courses,  have  not  been  particularly  successful  in  helping  educators  find  ways  to  integrate 
technology  for  learning.  Related  to  this  second  trend  is  our  third  observation  that  even  education  faculty  and  teachers 
who  feel  confident  about  their  own  ability  to  use  computers  for  professional  tasks  often  feel  uncertain  about  how  to 
use  technology  in  their  teaching.  Almost  by  default,  visions  for  the  use  of  technology  for  teaching  and  learning  are 
often  created  by  IT  specialists  who  are  not  educators.  Network  designs  and  levels  of  student  access  are  often 
determined  according  to  what  is  standard,  easy  to  monitor  and  maintain,  rather  than  according  to  what  is  educationally 
sound.  Dominant  curriculum  models  and  assessment  agendas  tend  to  emphasize  course  delivery  and  information- 
transfer  rather  than  knowledge  creation  and  designing  something  new.  While  there  are  thousands  of  examples  of 
digital  media  objects  and  teacher-created  units  and  lessons  that  claim  a meaningful  technology  component,  there  are 
far  fewer  authentic  images  of  the  effective  and  imaginative  use  of  technology  to  create  new  learning  experiences. 

Teachers  and  leaders  in  the  schools  and  school  districts  are  looking  to  new  teachers  to  shore  up  the  gap  between 
technology  presence  and  use.  However,  our  fourth  observation  is  that  the  current  generation  of  student  teachers 
simply  do  not  routinely  infuse  technology  in  their  own  learning  and  teaching,  and  thus,  too  few  of  them  graduate  with 
the  skills  and  experiences  that  are  needed  to  transform  today’s  classrooms.  People  entering  the  profession  today  are  as 
unlikely  to  have  experienced  technology-rich,  constructivist  learning  environments  as  their  more  experienced 
colleagues  in  schools.  There  is  a great  deal  of  talk  about  constructivism  on  North  American  campuses,  but  very  few 
actual  examples  either  on  campuses  or  in  schools  of  how  to  live,  learn  and  lead  in  these  ways. 


TEACHER  PREPARATION  AND  TECHNOLOGY  INTEGRATION 

A shift  in  thinking  is  required  for  teacher  preparation  that  is  similar  to  that  needed  in  professional  development  for 
classroom  teachers  (Clifford  & Friesen,  2001b;  Jacobsen,  2002,  2001).  It  is  simply  not  good  enough  to  teach  the  next 
generation  of  teachers  in  ways  we  were  taught.  Student  teachers  must  routinely  encounter  the  effective  infusion  of 
technology  in  the  normal  course  of  their  learning  at  the  university  and  in  their  practicum  placements  in  schools.  The 
three  of  us  have  co-taught  students  teachers  about  integrating  technology  into  their  learning  and  teaching  for  two 
years.  Our  planning  for  this  special  topics  seminar  on  integrating  technology  across  the  curriculum  was  guided  by  our 
vision  of  engaged  learning  and  educational  reform  (Clifford  & Friesen,  2001b,  1998,  1993;  Jacobsen,  2002,  2001), 
and  our  commitment  to  address  the  kinds  of  concerns  that  we  outlined  at  the  beginning  of  this  paper.  Our  seminar 
was  not  about  technology;  it  was  about  teaching  and  thinkingwith  technology.  Moving  well  beyond  skills  acquisition 
or  a focus  on  software  applications,  we  instead  created  a context  of  use  within  which  student  teachers  learned  by 
designing  learning  opportunities  for  real  children  in  real  classrooms. 

Opportunities  were  created  for  student  teachers  to  learn  in  just  the  ways  they  will  be  called  upon  to  teach  children 
(Clifford,  Friesen  & Jacobsen  1998).  We  drew  on  what  we  know  about  good  professional  development  practice:  (1) 
technology  is  best  learned  just-in-time,  instead  of  just-in-case,  (2)  planning,  designing,  implementing  and  evaluating 
are  best  done  in  collaboration  with  others,  (3)  learning  must  be  situated  in  authentic,  challenging  and  multidisciplinary 
tasks,  (4)  a culture  of  inquiry  around  technology  for  learning  supports  risk-taking  and  knowledge  creation,  and  (5) 
teachers  need  intentional  and  meaningful  opportunities  to  reflect  on  professional  development  and  growth. 

Technology  Is  Best  Learned  Just-In-Time,  Not  Just-In-Case 

In  our  seminar,  student  teachers  took  advantage  of  new  technology-enabled  learning  spaces  in  the  Faculty  of 
Education.  Our  learning  environment  leveraged  the  ubiquitous  and  unfettered  access  to  technology  in  the  seminar 
space  (i.e.,  16  networked  workstations)  and  also  in  the  larger  public  learning  spaces  (i.e.,  additional  workstations, 
multimedia  development  suites,  scanners,  digital  cameras,  CD  burners,  and  so  on).  The  fluid  access  to  technology 
tools,  Internet  access  and  to  each  other  enabled  students  to  gather  around  a workstation  or  deskspace  to  collaborate, 
and  to  move  out  into  public  spaces  as  needed.  The  permeable  environment  permitted  flexible  arrangements  and 
grouping,  and  also  provided  ready  access  to  other  experts  (i.e.,  Faculty  support  staff).  Technology  skill  was  developed 
in  the  context  of  developing  a web-based  portfolio,  completing  a group-selected  focused  technology  task,  constructing 
an  integrated,  multidisciplinary  unit  of  studies,  and  carrying  out  an  independent  inquiry  project  in  the  field. 


“OA,  easy  for  Leonardo** 

Our  first  task  seemed  simple  enough;  students  were  required  to  engage  with  the  program  of  studies,  and  publish  a 
reflective  response  on  a web  site  they  themselves  had  to  design  and  maintain.  Predictably,  it  sent  many  into  a tailspin 
(not  unlike  that  experienced  by  seasoned  classroom  teachers  and  faculty  members  when  they  attempt  a new 
technology  project).  Some  students  cried,  a few  complained,  and  many  were  worried  about  a perceived  mismatch 
between  their  present  ability  and  our  expectations.  Some  students  had  created  web  pages  before,  and  set  to  designing 
their  web  site  at  once.  Others  worked  with  us  to  learn  how  use  the  template  and  how  to  upload  their  finished  sites  to 
the  university  server.  Each  time  a new  site  went  up,  there  were  whoops  and  squeals  and  broad  grins  of  pure  delight. 
Students  felt  the  special  rush  of  seeing  their  own  work  on  the  web.  They  were  like  parents  with  a newborn  - even 
though  some  of  the  initial  sites  were  a little  plain  and  a bit  wrinkled,  they  were  beautiful  to  their  creators. 

Several  of  our  students  talked  about  "being  inspired  into  a culture  of  use"  which  is  different  in  kind  than  the 
application  focus  of  many  preservice  teacher  courses  on  computers.  The  pervasive  assumption  of  most  courses 
available  to  student  teachers  is  that  everyone  needs  to  learn  the  same  technology  skills  at  the  same  time  before  they 
can  do  anything  meaningful  with  them.  Different  assumptions  permit  new  possibilities  to  emerge.  We  concentrated  on 
just-in-time,  not  just-in-case,  instruction  with  technology  applications.  We  introduced  meaningful,  challenging  and 
multidisciplinary  tasks  that  posed  complex  and  meaningful  learning  problems,  and  that  enabled  a host  of  possible 
solutions.  In  the  context  of  these  tasks,  the  three  of  us  coached  and  guided  individuals  and  groups  of  students  to  design 
creative  solutions,  and  to  acquire  the  skills  and  competencies  they  needed  to  solve  their  problem  in  the  way  they 
wanted  to  approach  the  task,  and  respected  their  individual  starting  points  and  needs. 

Collaborative  Planning,  Design,  Implementation  And  Evaluation 

All  of  our  approaches  to  using  and  learning  technology  in  the  seminar  were  in  service  of  actual  design  tasks,  not  in 
service  of  learning  technology  for  its  own  sake.  We  put  student  teachers  in  the  position  of  the  children  they  are  going 
to  teach.  In  collaborative  groups,  students  completed  one  of  eleven  tasks  initially  designed  for  kindergarten  to  grade 
eleven  students.  The  tasks  required  little  prior  experience  with  technology  in  order  to  get  started.  However,  because  of 
the  nature  of  the  tasks,  there  was  no  upper  limit  on  the  sophistication  of  technology  use  that  was  possible.  Thus,  there 
were  steep  learning  curves  for  all  students  no  matter  what  their  starting  point.  Everybody  got  to  sweat  the  same,  all 
students  experienced  the  value  and  necessity  of  working  in  teams  to  build  on  the  strengths  and  diversity  in  the  group, 
and  everyone  began  with  the  experience  of  actually  designing  and  constructing  a solution  to  an  interesting  problem. 

Situating  Learning  In  Authentic,  Challenging  And  Multidisciplinary  Tasks 

Planning  for  engaged  student  learning  (NCREL,  2000)  and  technology  integration  requires  an  applied  understanding 
of  project,  instructional  and  task  design.  To  move  beyond  the  "add  on"  approach  of  using  PowerPoint™  or  word 
processing  as  ends  in  themselves,  teachers  need  to  think  and  plan  carefully  about  how  to  infuse  technology  in  teaching 
and  learning.  Working  in  small  groups,  students  collaborated  on  the  design  and  development  of  an  Integrated  Unit  of 
Study  for  authentic  and  meaningful  integration  of  technology  into  one  or  more  core  curricular  area. 

A major  requirement  of  the  fourth  semester  in  the  University  of  Calgary  program  is  that  students  engage  in  an 
independent  inquiry  project  that  is  curriculum  and  practice  oriented.  Ideally,  these  are  field-oriented  projects  at  a 
school  or  Community  Work  Place  site  that  enable  student  teachers  to  work  side-by-side  with  classroom  teachers  and 
students  to  study  essential  questions  to  do  with  teaching  practice.  Students  engage  in  critical  inquiry  in  a systematic 
and  intentional  manner,  contribute  to  ongoing  efforts  to  improve  teaching  and  learning  at  the  field  site,  and 
demonstrate  the  understandings  and  skills  acquired  throughout  the  first  three  semesters  of  the  Master  of  Teaching 
program.  There  is  an  expectation  that  the  students’  work  will  leave  a legacy  in  the  field.  Inquiry  projects  were 
designed  to  immerse  student  teachers  in  challenging  and  multidisciplinary  work  that  made  a difference  in  the  present. 
One  of  our  students.  Heather,  learned  how  to  construct  and  program  Lego®  robots  alongside  grade  one  and  two 
students  and  their  teachers  in  an  elementary  school.  As  the  opportunity  to  work  with  robotics  progressed  through  each 
grade.  Heather  became  part  of  the  coaching  team,  working  with  teachers  and  students  right  up  to  grade  five.  As  her 
own  expertise  grew,  the  school  came  to  depend  on  Heather’s  contribution,  her  insights  and  her  creative  solutions.  She 
focused  her  inquiry  project  on  documenting  and  interpreting  her  experiences  with  young  children  and  robotics.  The 
result  of  the  inquiry  project  is  not  simply  a research  report;  instead,  the  inquiry  process  fosters  closer  connections 
between  campus  and  field  experiences,  observations  and  learning. 


Creating  A Culture  Of  Inquiry  Around  Technology  For  Learning 

One  of  the  most  powerful  aspects  of  our  special  topics  seminar  was  the  intentional  placement  of  students  in  enriched 
field  settings  for  their  inquiry  projects.  About  one  third  of  the  class  worked  in  schools  in  which  the  Galileo 
Educational  Network  provided  on-site  support  to  teachers  to  design  new  learning  experiences  via  effective  technology 
infusion.  In  most  teacher  preparation  programs,  field  placements  are  more  generic:  students  are  matched  to  schools 
and  partner  teachers  according  to  grade  and  subject  specialty.  This  approach  to  field  placements  builds  on  the  notion 
that  the  main  goal  is  to  make  sure  the  numbers  work  given  certain  broad  category  matches.  It  assumes  that,  in  essence, 
schools  and  classrooms  are  convenient  places  for  student  teachers  to  “practice**.  Teacher  associations,  that  vehemently 
argue  that  all  teachers  are  equally  excellent  in  exactly  the  same  way,  present  an  obstacle;  to  them,  targeted  placements 
smack  of  elitism.  Our  experience  counters  this  view;  we  did  not  target  “good**  or  “better**  schools  and  teachers. 
Instead,  we  asked  ourselves:  what  energies  can  we  leverage  by  placing  student  teachers  committed  to  developing  their 
ability  to  infuse  technology  into  classrooms  with  experienced  teachers  who  are  also  intentionally  pursuing  their  own 
professional  growth  in  this  area?  Given  a four-month  semester,  we  knew  that  we  could  maximize  the  impact  of 
students*  on-campus  work  by  placing  them  in  field  placements  where  the  infusion  of  technology  was  also  a priority. 

Reflection  On  Professional  Development  And  Growth 

All  certified  Alberta  teachers  are  required  to  complete  an  annual  professional  growth  plan  that  includes  goals, 
strategies  and  evaluation.  Teachers  are  expected  to  consider  the  Teacher  Quality  Standard  (Alberta  Learning,  1996), 
the  school  division’s  goals  and  plans,  and  their  own  school's  improvement  plan  when  developing  professional  growth 
plans.  While  plans  of  such  detail  were  not  appropriate  in  a special  topics  seminar,  we  did  reinforce  the  experience  of 
our  students  throughout  the  whole  MT  program:  the  importance  of  cultivating  reflective  and  thoughtful  habits  of  mind 
about  professional  practice.  To  that  end,  our  students  prepared  a professional  growth  plan  throughout  the  seminar  that 
included  three  self-assessments  that  were  published  as  part  of  a web-based,  electronic  portfolio. 


WHY  DID  THIS  SEMINAR  WORK? 

The  most  significant  indicator  that  our  seminar  was  successful  is  the  high  caliber  of  the  students'  scholarship.  The 
quality  of  their  work,  their  thinking  and  their  reflection  was  exemplary.  Each  student  was  able  to  meet  seminar 
requirements  for  curriculum  design,  for  planning  and  carrying  out  substantive  inquiry,  for  reflection  on  professional 
growth,  and  for  the  acquisition  of  technology  skills.  We  prepared  a CD  ROM  of  all  the  units  of  study  that  students 
created,  and  the  focused  tasks  on  which  they  had  worked.  From  our  perspectives  both  in  staff  development  and  in 
graduate  level  teaching,  we  have  no  hesitation  in  saying  that  each  of  our  students  left  the  class  better  prepared  to 
infuse  technology  in  their  own  classrooms  than  many  experienced  teachers.  Many  tackled  inquiry  projects  at  a level 
of  complexity  that  approached  performance  expectations  for  graduate  work.  It  was  hard  work.  For  most,  there  was  a 
huge  learning  curve,  coupled  with  a determination  to  dig  in  and  learn  what  was  needed  in  service  of  important  ideas. 
That  is,  the  culture  of  inquiry  we  talked  about  creating  in  classrooms  became  a living  part  of  the  seminar,  itself 

Building  on  Diversity  - We  believe  the  exceptional  degree  of  student  success  and  engagement  was  an  outcome  of 
deliberate  design  and  instructional  decisions  that  were  informed  by  our  knowledge  of  the  current  state  of  affairs  in 
schools  and  on  campus.  First,  the  space  in  the  seminar  was  open  enough  for  all  students  to  define  a place  for 
themselves.  We  designed  the  seminar  so  that  it  required  a wide  range  and  diversity  of  experience,  ideas  and  projects. 
From  the  first  day,  our  students  learned  that  it  did  not  matter  what  grade  or  subject  they  were  preparing  to  teach.  We 
were  not  concerned  whether  they  had  extensive  experience  with  technology  or  none  at  all.  It  was  all  right  to  prefer 
different  platforms,  different  software,  and  alternative  approaches  to  tasks  than  the  ones  we  suggested.  That  is,  we 
structured  exactly  the  situation  we  wanted  tomorrow  *s  teachers  to  create  for  their  own  students.  Students  came  to  see 
that  diversity  was  not  a problem  to  be  overcome,  but  an  essential  resource  on  which  we  all  can  draw  when  asked  to 
complete  complex  and  demanding  tasks  together. 

Insisting  on  a Pedagogical  Focus  - Second,  all  aspects  of  the  seminar  were  centered  on  pedagogical  issues  rather  than 
technology  issues.  There  are  two  apparently  contradictory  consequences  of  a strong  initial  focus  on  technology  skills 
acquisition  in  the  common  kinds  of  workshops  and  courses  designed  for  those  who  are  new  to  technology.  The  first 
consequence  is  that  participants  often  seize  on  one  or  two  of  the  applications  they  first  learn  to  use,  assuming  that  now 
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the  job  of  covering  technology  is  taken  care  of.  This  enthusiasm  is  apparent  in  how  often  teachers  introduced  to 
planning  a technology  enriched  experience  for  students  start  with  statements  like  this,  '*!  was  thinking  of  letting  kids 
do  Power  Point™  reports  on  an  animal  they  choose...”  Jamie  McKenzie  (2000)  uses  the  felicitous  phrase  ’’power 
pointlessness"  to  describe  the  careless  adoption  of  an  otherwise  effective  presentation  tool  as  if  it  were  the  be-all  and 
end-all  of  technology  use  in  the  classroom.  The  second  consequence  of  an  initial  focus  on  skills  acquisition  is  that 
such  a focus  feeds  the  growing  sense  of  panic  that  sets  in  when  many  of  us  squarely  face  a harsh,  but  often  unspoken 
reality  about  technology:  no  mere  mortal  can  keep  up  with  the  innovations.  No  one  in  a classroom  is  going  to  win  the 
race  against  new  hardware,  new  applications,  and  new  capabilities.  There  is  always  a new  version,  a new  digital 
device  or  a new  idea  coming  down  the  pipe.  Considering  the  range  of  applications  now  available  to  students  and 
teachers,  and  the  rate  at  which  new  versions  are  introduced,  it  is  easy  to  feel  overwhelmed  even  by  finding  a starting 
place. 

When  student  teachers  can  be  convinced  to  give  up  the  idea  that  they  need  to  know  this  application,  and  this 
application,  and  this  application,  and  accept  that  they  will  never  know  everything  about  every  piece  of  software,  they 
undergo  a transformation.  Many  suddenly  feel  liberated.  They  feel  a burden  lifted  when  we  say  to  them,  "Look,  there 
is  no  way  you  will  ever  be  faster,  more  fluent,  or  more  knowledgeable  about  the  technology  that  is  out  there  than  all  of 
your  students.  And  you  know  what-that's  okay.  It  actually  gives  you  a whole  lot  more  room  to  get  important  things 
happening.  There  are  some  things  that  the  kids  will  always  do  better  than  you.  Let  them.  And  there  are  important 
things  that  they  need  to  be  taught.  Your  job  as  a teacher  is  to  design  rich  and  meaningful  learning  experiences.  They 
don't  know  how  to  do  that.  You  have  to  figure  out  what  these  applications  are  good  for.  The  kids  will  figure  out  how 
to  drive  them.” 

It  is  often  the  ideas  that  accompany  technology  integration  that  intimidate  teachers  and  university  faculty  alike. 
Teachers  sometimes  ask  us,  ”If  I start  letting  the  kids  use  all  this  technology,  what’s  happened  to  my  role  as  a 
teacher?"  Of  course,  what  they  are  really  asking  is  what  will  happen  to  me?  Do  I still  have  anything  of  value  to 
offer?  Confident  technology  users  ourselves,  we  know  that  deep  understandings  of  the  character  of  inquiry -based 
learning  and  knowledge  construction  have  never  been  more  important  than  they  are  in  digitally  rich  environments 
(Clifford  & Friesen,  2001a;  Jacobsen  & Goldman,  2001).  It  is  entirely  possible  to  do  foolish  things  with  powerful 
tools  just  because  they  are  there.  Our  challenge  was  to  help  our  student  teachers  develop  fluency  with  teaching  and 
learning  with  technology,  not  just  with  technology,  itself.  A critical  awareness  of  issues  to  do  with  technology 
integration  was  revealed  when  one  of  our  students  wrote  to  us  about  her  reaction  to  the  ICT  program  of  studies  and 
our  talk  about  teaching  and  learning  as  the  primary  focus  of  the  seminar.  “I  thought”,  she  told  us,  “that  only  people 
who  were  against  technology  ever  raised  questions  about  ethics  and  values  and  what  is  worth  doing.  I was  amazed  to 
find  it  in  the  ICT  program  of  studies,  and  to  hear  the  three  of  you  say  critical  things,  too”. 

Team  Teaching  - The  three  of  us  worked  collaboratively  with  one  another  and  with  the  support  staff  who  were 
available  to  assist  our  students.  The  seminar  was  enacted  through  the  genuine  collaboration  and  diverse  range  of  skills 
that  are  required  to  get  complex  things  done.  We  depended  on  our  own  collaborative  efforts,  and  we  valued  and 
encouraged  opportunities  for  students  to  access  one  another’s  expertise.  This  collaboration  extended  to  the  field 
placements,  and  the  students  contributed  to  and  benefited  from  the  diversity  among  teaching  staff  as  well.  In  a world 
in  which  we  know  we  must  prepare  students  to  work  effectively  in  teams  both  to  define  and  to  solve  problems  in 
ambiguous  situations,  teacher  isolation  is  a terrible  problem.  The  culture  of  schooling  makes  it  very  difficult  for 
teachers  to  form  strong  work  teams.  As  part  of  helping  student  teachers  both  experience  and  understand  the  power  of 
colleagial  support,  we  required  students  to  become  interdependent.  There  was  no  way  that  any  student  could  meet  the 
demands  of  this  seminar  alone.  They  needed  each  other,  they  sought  out  one  another,  and  they  had  to  negotiate  all  the 
ordinary  troubles  of  working  together  as  part  of  their  learning. 


CONCLUSIONS 

A deep  commitment  to  the  principles  that  pervade  the  entire  teacher  preparation  program  at  the  University  of  Calgary 
guided  the  creation  of  our  special  topics  learning  environment.  We  opened  spaces  in  which  student  teachers  used  ICT 
fluently  for  personal  productivity  in  the  creation  and  maintenance  of  scholarly  work.  Student  teachers  developed  an 
understanding  of  Alberta  Learning’s  (2000)  ICT  Program  of  Studies,  and  discussed  implications  for  learning  and 
teaching  in  their  discipline/grade  level  and  for  their  own  professional  growth  plans.  Students  were  articulate  in 
describing  the  ways  that  technology  had  influenced  their  own  learning  and  in  describing  the  ways  they  had  seen 


technology  play  a role  in  others'  teaching  and  learning.  In  groups,  and  as  individuals,  students  wrote,  communicated, 
made  decisions,  and  conducted  inquiry  smoothly  and  effectively  using  technological  aids  if  and  when  these  digital 
tools  contributed  to  those  processes.  Students  intelligently  questioned  uses  of  ICT  and  were  appropriately  skeptical 
about  naive  enthusiasms  and  overly  simple  solutions.  In  the  context  of  focused  tasks  and  integrated  units  of  study  that 
they  designed  and  created,  students  discussed  the  strengths  and  weaknesses  of  ICT  in  a wide  range  of  applications. 

In  the  course  of  this  special  topics  seminar,  students  moved  beyond  being  mere  proponents  of  ICT  usage,  or  already- 
hardened  skeptics,  and  developed  further  as  thoughtful  professionals  who  choose  and  design  tools  appropriate  for  the 
tasks  they  needed  to  accomplish.  Students  developed  an  informed  personal  position  on  technology  integration  in 
education  and  articulated  and  defended  that  position  with  each  other.  In  the  school  placements,  student  teachers 
encouraged  children  both  to  actively  question  the  place  of  ICT  in  their  learning  and  to  make  responsible  use  of  ICT  in 
their  own  work.  In  the  context  of  their  own  creative  work,  student  teachers  understood  and  developed  ethical 
dispositions  and  practices  in  relation  to  the  uses  of  technology  in  the  classroom.  Finally,  our  student  teachers 
developed  a capacity  for  critical  inquiry  in  the  applications  of  technology  in  learning  and  teaching,  an  essential 
disposition  given  the  rapid  pace  at  which  information  technologies  are  transforming  our  world. 
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ABSTRACT:  The  Galileo  Educational  Network  is  an  innovative  educational  reform  initiative  that 
brings  learning  to  learners.  Expert  teachers  work  alongside  teachers  and  students  in  schools  to 
create  new  images  of  engaged  learning,  technology  integration  and  professional  development.  This 
case  study  is  based  on  the  nine  schools  involved  with  Galileo  in  2000/2001 . Field  visits  and 
interviews  (n=l  13)  with  Galileo  staff,  school  administrators,  teachers  and  students  yielded  rich 
information  about  the  initiative.  Indicators  of  success  include:  1)  demand  for  Galileo  Network’s 
services,  2)  corporate  investment  in  Intelligence  Online  (10),  3)  satisfaction  among  stakeholders  at 
each  level  of  the  initiative,  4)  evidence  of  engaged  student  learning,  and  5)  evidence  of  transformed 
teaching  practice.  Essential  conditions  for  the  sustainability  and  growth  of  the  Galileo  Network 
initiative  are  described.  Factors  that  both  contributed  to  and  hindered  the  success  of  the  initiative 
are  discussed. 


The  Galileo  Educational  Network  Association  (GENA)  is  a continuous  professional  development  and  research 
organization  in  Alberta,  Canada,  that  focuses  on  the  fundamental  changes  to  teaching  and  learning  required  by 
digital  media  and  technology.  GENA's  expert  teachers  work  in  schools  alongside  teachers  and  students  to  create  new 
images  of  engaged  learning,  technology  integration  and  professional  development.  GENA  works  on-line  to  develop 
and  share  examples  of  innovative  and  high  quality  student  work  and  sustains  face-to-face  initiatives  via  its  website 
[http://www.galileo.org]  and  Intelligence  Online  (lO)  professional  development  service.  The  professional 
development  and  leadership  approaches  of  the  Galileo  Network  are  firmly  grounded  in  current  educational  research, 
and  Galileo  Network  members  contribute  to  new  knowledge  by  actively  publishing  and  disseminating  results  of 
their  efforts  (Clifford  & Friesen,  2001a,  2001b,  1998,  1993;  Clifford,  Friesen  & Jacobsen,  1998;  Jacobsen  2001). 
GENA  has  developed  a number  of  strategic  alliances  with  private,  corporate,  community  and  government 
organizations  to  generate  innovative  approaches  and  strategies  for  professional  development.  GENA  is  a charitable 
organization  that  began  operations  in  1 999  in  office  space  provided  by  the  Faculty  of  Education  at  the  University  of 
Calgary.  The  Galileo  Network  operates  autonomously  from  any  one  school  district  or  division  as  it  pursues  a 
province-wide  educational  reform  agenda.  The  majority  of  GENA’s  funding,  both  cash  and  in-kind,  comes  from  a 
variety  of  government,  private,  corporate  and  institutional  organizations.  Each  school  division  and/or  individual 
school  funds  their  participation  with  the  Galileo  Educational  Network  differently. 

Classroom  teachers  need  ongoing  support  and  professional  development  for  the  effective  implementation  of 
technology  for  meaningful  student  learning.  Conventional  approaches  to  professional  development  for  technology 
integration,  usually  short-term  skill -based  workshops  and  seminars  about  working  the  computer,  do  not  transform 
pedagogy.  What  makes  the  Galileo  Educational  Network  approach  innovative  is  that  teachers  are  provided  with  time 
during  the  school  day  to  collaborate,  to  participate  in  professional  conversations  about  practice,  and  to  work  with  a 
trusted  mentor  who  supports  and  extends  their  technology  integration  efforts.  The  professional  development 
relationship  is  guided  by  essential  principles  of  engaged  student  learning  and  high  technology  performance,  and  is 
responsive  to  the  individual  teacher's  current  practice  and  needs.  Therefore,  the  professional  development  is  more 
closely  aligned  with  mentorship  than  with  coaching  from  the  side.  Far  from  being  a set  model,  the  GENA  approach 
is  generative,  flexible  and  responsive  to  the  individual  teacher's  needs  and  ideas  for  his  or  her  own  learning.  Galileo 
teachers  start  from  where  classroom  teachers  are  in  their  development  and  beliefs.  The  individual  starting  point  for 
each  teacher  is  respected  in  much  the  same  way  that  teachers  respond  to  the  diverse  needs  and  capabilities  of  their 
students.  Different  questions  and  needs  are  honored,  and  projects  that  arise  from  the  relationship  come  from  the 
teacher's  current  practice,  beliefs  and  ideas  about  teaching,  technology  and  student  learning. 


Transformational  Professional  Development 


It  is  important  to  pause  here,  and  provide  operational  definitions  of  key  terms.  A decade  ago,  Leithwood  (1992) 
described  a move  towards  transformational  leadership,  which  he  defined  as  school  leaders  and  classroom  teachers 
seeking  meaning  together  as  a community,  and  supporting  individuals  as  they  seek  meaning  in  their  professional 
lives.  The  Galileo  Network’s  approach  to  transformational  professional  development  extends  this  tradition  by 
building  mutually  benefic  ial  relationships  at  all  levels  of  a school  system  at  once  with  the  goal  of  empowering 
teachers  to  be  innovative  curriculum  designers,  and  to  free  teachers  to  author  rich  learning  experiences  with  their 
students.  Terms  like  empowerment  and  freedom  are  chosen  deliberately  to  convey  the  vision  of  enfranchisement  and 
democracy  that  is  at  the  foundation  of  GENA's  work  in  schools.  The  Galileo  approach  is  not  just  about  enabling,  or 
making  possible,  although  this  is  an  important  and  powerful  part  of  their  work  with  educators  and  educational 
leaders.  Galileo’s  approach  to  transformational  professional  development  aims  to  empower  teachers  and  school 
leaders  to  be  learners  themselves  again,  and  to  form  supportive  communities  that  are  committed  to  staff  and 
individual  growth.  Galileo  teachers  work  alongside  teachers  to  help  them  develop  as  learners  themselves  so  that  they 
can  translate  this  new  freedom  of  inquiry  into  the  design,  implementation  and  evaluation  of  meaningful,  authentic 
and  challenging  learning  experiences  for  children. 

Research  Context 

The  present  evaluation  of  the  second  year  of  the  Galileo  Network's  professional  development  initiative  builds  and 
extends  upon  a study  conducted  at  three  elementary  schools  involved  with  the  Galileo  Educational  Network  in 
1999/2000,  its  first  year  of  operations  (Jacobsen,  2001).  A goal  of  the  present  investigation  was  to  expand  upon 
findings  from  three  schools  by  investigating  further  the  relationship  between  teacher  and  student  perceptions  about 
classroom  events,  the  role  of  the  Galileo  Network  in  schools,  and  the  duty  of  leadership  in  supporting  and  extending 
professional  development  initiatives  during  the  second  year  of  operations.  The  study  was  essentially  guided  by  two 
overall  research  objectives:  (1)  Evaluate  the  impact  of  effective  technology  integration  on  engaged  student  learning, 
and  (2)  Evaluate  the  impact  of  the  Galileo  Educational  Network  on  teaching  practice  and  transformed  learning 
environments  by  evaluating  the  sustainability  of  these  professional  development  initiatives. 

Case  study  research  methods  (Merriam,  1998;  Stake,  1995)  were  employed  to  identify  appropriate  sources  of  data, 
and  gather  information.  Field  visits  were  made  to  eight  schools  to  observe  daily  classroom  events  and  instruction, 
and  to  conduct  in-depth  interviews.  Over  one  hundred  interviews  (n=l  13)  were  conducted  with  Galileo  Network 
staff,  school  administrators  and  technical  personnel,  classroom  teachers  and  students.  Observations  of  whole  class 
dynamics,  small  group  collaborative  work  and  individual  student  engagement  in  learning  were  made,  and 
observation  notes  recorded.  The  26  indicators  of  engaged  learning  and  22  indicators  of  high  technology 
performance  developed  by  the  North  Central  Regional  Educational  Laboratory  (2000,  1995)  provided  a conceptual 
framework  through  which  interview  and  observational  data  was  understood,  analyzed  and  discussed. 

This  case  study  research  was  rarely  a linear,  step-by-step  process  that  proceeded  logically  from  data  collection  to 
analysis.  Instead,  “data  collection  and  analysis  is  a simultaneous  activity”  (Merriam,  1998,  p.  151).  Analysis  began 
by  framing  the  present  study  in  the  context  of  first-year  findings,  and  continued  as  both  an  iterative  and  recursive 
process  with  each  site  visit  and  interview.  The  triangulation  of  data  and  information  from  site  visits,  interviews  and 
observations  enabled  researchers  to  analyze  and  report  on  themes,  trends  and  understandings  of  the  group  as  well  as 
honor  individual  experiences  and  observations  in  a rich,  thick  description  of  the  case.  Interview  data  was  evaluated 
using:  (1)  content  analysis  (Merriam,  1998)  of  themes  and  recurring  patterns  of  meaning,  (2)  constant  comparative 
method  (developed  by  Glaser  & Strauss,  1967,  described  in  Merriam,  1998)  to  identify  emergent  themes  and 
categories,  and  (3)  narrative  analysis  (Merriam,  1998)  to  study  experience  via  the  stories  that  people  tell  and 
remember.  Field  notes,  observation  and  reflection  on  site  visits,  photographs  of  learning  environments  provided  rich 
data  for  content  analysis,  and  also  categorical  aggregation  and  direct  interpretation  (Stake,  1 995).  Aggregation  of 
instances  that  suggested  trends  and  shared  understandings  of  the  group,  along  with  direct  interpretation  of  an 
individual’s  stance,  contributed  to  the  development  of  themes. 

DISCUSSION 

Initial  findings  that  have  emerged  from  this  case  study  of  Galileo  Network’s  second  year  of  operations  are 
summarized  as  follows:  (1)  sustainability  of  the  initiative  requires  a long-term  relationship  be  established  and 


maintained  between  school  teachers  and  Galileo  teachers,  (2)  sustainability  requires  that  teacher  leaders  who  can 
champion  the  initiative  be  identified  and  cultivated;  (3)  the  role  of  district  and  school-based  leaders  is  vital  to  the 
effective  establishment  and  cultivation  of  a inquiry -based  learning  environment  in  which  teachers  were  prepared  to 
take  risks,  and  (4)  indicators  of  success  were  focused  on  what  the  children  could  do  differently,  and  how  technology 
provided  opportunities  for  learning  that  were  previously  unavailable.  The  context  for  these  findings  will  be 
discussed  in  subsequent  sections  of  the  paper. 

Transforming  Pedagogy 

Both  philosophical  and  pedagogical  barriers  to  innovation  exist  when  teachers  shift  from  information-transmission 
to  designing  technology-enabled,  constructivist  learning  environments.  When  teaching  roles  change  from  content 
expert  to  designer  of  technology-rich,  inquiry-based  projects,  some  parents  and  educational  leaders  get  nervous.  A 
supportive  and  continuous  relationship  with  Galileo  Network  teachers  over  an  extended  period  of  time  is  perceived 
by  classroom  teachers  and  school  administrators  to  be  an  essential  condition  for  achieving  transformed  teaching 
practices  with  technology.  The  Galileo  Network  recognized  early  on  that  in  order  to  transform  classroom  teaching, 
they  had  to  work  at  all  levels  of  the  educational  system  at  once.  To  that  end,  the  Galileo  Network  has  formed 
strategic  alliances  with  the  provincial  ministry  of  education,  superintendents  of  schools,  district  technology  and 
curriculum  leaders,  school-based  administration,  school  teachers,  parents  and  students  to  support  the  growth  of 
innovative  practices  in  schools.  Educators  can  often  be  under  intense  pressure  from  the  public,  the  ministry,  the 
school  board,  their  administrators,  and  even  from  themselves,  to  change  practices  and  processes  overnight. 

However,  the  pressure  to  change  quickly  is  an  unrealistic  expectation,  and  Galileo  teachers  help  classroom  teachers 
to  set  realistic  and  achievable  goals  for  inquiry -based  learning  and  technology  integration. 

Creating  Supportive  Environments  for  Risk-Taking  and  Knowledge  Creation 

When  asked  to  describe  the  nature  of  their  on-site  relationships  with  each  teacher  or  group  of  teachers,  co-founders 
Dr.  Pat  Clifford  and  Dr.  Sharon  Friesen  explained  that  the  work  with  teachers 

. . . really  showed  us  very  clearly  that  there  is  no  model  for  doing  this. . . it  "s  not  models  that  make  a difference,  it  *s 
ideas  that  make  a difference  and  it's  relationships  that  make  a difference.  So  it's  in  hearing  what  people  want  to 
do.  It's  just  like  teaching  in  a classroom  where  you  work  literally  from  where  people  are  ...  because  people  can 't 
be  in  any  other  place  than  where  they  are.  You  can 't  wish  them  to  be  somewhere  else.  They  are  where  they  are 
from  the  start.  And  so  you  listen  to  that.,  where's  your  opening?  And  then  around  the  opening  they  provide  you, 
the  job  of  the  expert  then,  we  think,  is  to  say  what  could  we  put  in  place  for  this  person,  given  who  they  are,  what 
they  want  to  accomplish,  and  what  they're  working  with?  So  it  looks  different,  as  it  ought  to,  for  each  person. 

We  called  it  being  responsive  and.,  it's  actually  to  say  if  you  're  going  to  have  people  work  with  kids  in  this  way, 
you  have  to  provide  them,  as  teachers,  a learning  experience  that's  like  that. 

The  intent  of  the  Galileo  Network  initiative  is  to  model  transformed  approaches  to  leadership,  teaching  and  learning 
that  inspire  and  foster  inquiry  in  learners  of  all  ages.  Pat  and  Sharon  believe  that. .. 

...the  work  we  do  here,  it's  not  without  huge  impact  and  it's  not  about  individuals.  It's  far  bigger  than  any 
individual.  You  are  equal  to  get  involved  in  it.  You  have  to  let  it  go.  It's  the  gift  you  give  to  people.  But  they  pick  it 
up  in  ways  that  you  don't  anticipate.  It's  far  bigger  than  any  person.  So  we  may  work  with  the  people,  but  what  we 
put  them  in  touch  with  is  a part  of  themselves  that,  I think,  they're  often  apt  to  give  up  when  they  enter  the 
classroom.  And  once  they  get  a hold  of  it,  they  don 't  want  to  let  go  because  they  see  the  impact.  It's  right  in  front 
of  their  face.  You  see  what's  happening  to  those  kids. 

An  ongoing  and  sustained  relationship  with  a Galileo  Network  teacher  provided  some  teachers  with  a safe  and 
caring  environment  for  exploring  new  approaches  to  teaching,  and  the  motivation  to  continue  when  faced  with 
questions  and  challenges  from  parents  and  other  colleagues.  For  many  teachers,  the  fact  that  Galileo  teachers  had 
been  where  they  wanted  to  go  was  an  important  motivator  for  getting  involved,  and  taking  the  necessary  risks  to 
change  their  teaching  practice.  Whether  or  not  changes  to  practice  are  sustainable  has  a great  deal  to  do  with  the 
ongoing  nature  of  the  relationship  between  teachers  and  the  Galileo  Network.  Teaching  practices  are  developed  over 
years  of  experience  with  children,  and  years  of  observations  in  classrooms  - hence,  they  are  rarely  transformed 
quickly.  For  a teacher  to  transform  practices  that  have  become  comfortable,  to  abandon  methods  that  have  enabled 


her/him  to  achieve  some  form  of  results  with  children,  the  teacher  needs  ongoing  support  and  professional  dialogue 
with  other  teachers  who  understand  her/his  concerns,  and  can  provide  images  of  how  it  can  work  with  children.  The 
teacher  has  to  trust  that  the  changes  being  made  offer  some  relative  advantage  over  what  is  being  done  right  now. 

Inquiry  Based  Learning  With  Technology 

What  is  the  nature  of  change  that  occurs  when  meaningful  partnerships  are  formed  between  learners  and  digital 
media?  Teachers  who  designed  and  implemented  inquiry  projects  emphasized  the  importance  of  student  questions 
versus  teacher  questions.  One  of  the  desired  learning  goals  for  teachers  was  to  promote  student’s  meaningful  and 
personal  connections  to  the  topic  of  study.  Instead  of  designing  integrated  units  or  students’  projects  with  pre- 
determined outcomes,  inquiry  projects  were  designed  in  such  a way  that  students’  ideas,  questions  and  prior 
knowledge  became  an  important  contribution,  and  an  assessment  strategy  was  developed  that  accommodated 
multiple  outcomes.  Students  made  choices  about  questions  that  they  would  pursue  in  the  context  of  the  inquiry 
project,  and  formed  collaborative  groups  based  on  shared  interests  rather  than  teacher-formed  groups.  Learning 
tasks  were  big  and  complex  enough  that  students  needed  to  rely  on  each  other  to  gather  the  information  needed  to 
solve  problems  and  generate  solutions.  Students  shared  their  ideas  and  knowledge  with  their  peers,  not  just  with  the 
teacher.  Students  were  encouraged  to  use  a variety  of  means  to  demonstrate  their  understanding  of  concepts  and 
problems,  from  textual  explanations,  concept  maps,  and  drawings  to  dramatizations,  multimedia  presentations, 
functional  robots  and  web  sites. 

Teachers  rarely  used  whole  class  instruction  as  a method  of  delivering  the  curriculum.  In  response  to  the  student’s 
emerging  learning  needs,  the  teacher’s  role  became  one  of  scholarly  mentor  and  guide  as  she  met  with  groups  of 
students  for  discussion  about  specific  problems,  and  with  individual  students  who  needed  assistance.  Within  the 
context  of  a well -structured  and  challenging  inquiry  project,  and  with  readily  available  guidance  and  instruction  as 
they  needed  it  from  the  teacher,  students  worked  collaboratively  with  their  peers  to  design  creative,  innovative  and 
diverse  projects.  Instead  of  there  being  one  best  solution  to  a complex  problem  posed  by  the  teacher,  students  were 
given  the  license  to  design  alternate  and  unique  interpretations  and  projects,  and  inquire  into  topics  and  questions 
that  reflected  their  own  interests,  knowledge  and  understanding. 

Different  groups  of  students  required  different  technology  tools  to  create  their  designs  and  construct  their  projects. 
Technology  skills  were  taught  just-in-time  in  service  of  learning  tasks  and  goals,  rather  than  taught  to  the  whole 
class  at  once.  One  group  might  need  a spreadsheet  program  to  model  their  data,  while  another  group  may  choose  to 
represent  their  understanding  of  relationships  between  concepts  using  a semantic  mapping  tool.  To  the  extent 
possible,  given  the  design  and  resources  of  the  school,  computers  were  at  hand  wherever  and  whenever  the  students 
needed  them,  rather  than  walking  students  down  the  hall  to  a computer  lab  for  a booked  amount  of  time  each  day. 
Novel  solutions  were  created  in  schools  where  there  were  only  2-  4 workstations  available  in  the  classroom.  For 
example,  at  one  elementary  school,  teachers  negotiated  arrangements  whereby  students  could  use  computers 
distributed  throughout  the  school  and  in  other  classrooms  as  needed  for  their  projects.  A culture  of  inquiry  was 
created  in  this  particular  school  such  that  students  were  trusted  to  work  where  they  needed  to  be  for  their  inquiry 
tasks  and  projects,  and  appropriate  behavior  was  continuously  demonstrated. 

Among  educators  in  Galileo  schools,  there  was  a monumental  shift  in  the  perception  of  the  role  of  technology  in 
teaching  and  learning.  Through  professional  dialogue,  mentoring  and  working  alongside  teachers  with  their  students, 
the  Galileo  Network  has  facilitated  a changed  understanding  of  the  intent  and  fundamental  goals  of  Alberta 
Learning’s  ICT  Program  of  Studies,  from  an  initial  perception  of  an  emphasis  on  technology,  to  an  understanding  of 
the  emphasis  on  higher  order  thinking,  inquiry,  communicating,  problem  solving,  and  decision  making.  Teachers 
clearly  regarded  Galileo’s  vision  and  methods  as  a focus  on  teaching  and  learning,  which  was  a shift  from  their 
initial  perception  that  GENA  was  all  about  technology. 

Developing  A Culture  of  Inquiry  for  Teachers 

The  Galileo  Network  worked  with  people  at  all  levels  of  the  educational  organization  in  order  to  help  build  a culture 
that  supports  and  expects  reformed  ways  of  teaching  and  learning  with  ICT.  A social  and  political  culture  of  reform 
and  innovation,  and  the  expectation  that  people  would  be  charting  new  territory,  helped  to  support  teachers  in 
making  changes  to  their  practice.  An  integral  component  of  the  culture  of  expectation  in  each  of  these  schools  was 
the  support  for  change  provided  by  regular  access  to  onsite  support  and  expertise,  and  the  time  to  make  optimal  use 
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of  the  onsite  professional  development.  Teachers  felt  energized  and  excited  about  the  changes  to  their  practice,  and 
were  convinced  by  evidence  of  student  ownership,  joy  and  success  in  their  learning.  There  were  many  reasons  why 
schools  and  teachers  first  became  involved,  and  then  were  highly  motivated  to  continue,  with  the  Galileo  Network; 
(1)  The  Galileo  Network  teachers  have  worked  with  children  on  innovative  and  successful  technology-enhanced 
projects,  and  share  many  examples  of  high-quality  scholarship  achieved  by  students;  (2)  The  reputations  of  Galileo 
Network  staff  as  experts  and  innovators,  and  the  personal  relationship  networks  that  these  experienced  educators  and 
managers  have  established  in  the  Alberta  education  system,  attract  educators,  and  provide  a basis  on  which  to  build 
trusting  and  productive  relationships;  (3)  The  push  in  Alberta  to  implement  the  ICT  Program  of  Studies  provides  an 
important  source  of  motivation;  the  Galileo  Network  bridges  a professional  development  gap  in  the  province  with 
regard  to  achieving  the  expected  ICT  learning  outcomes  with  students;  (4)  Teachers’  desire  for  continuous 
professional  development,  and  support  while  implementing  a new  program  of  studies,  makes  them  hghly  value  the 
opportunity  to  rediscover  the  learner  and  scholar  within,  and  cherish  the  opportunity  to  work  with  dynamic, 
approachable  and  trusted  Galileo  colleagues;  (5)  The  satisfaction  of  being  involved  with  an  innovative  organization, 
and  working  side-by-side  with  energetic,  enthusiastic  and  experienced  Galileo  Network  teachers;  (6)  After 
participating  in  single  or  multiple  projects  with  Galileo  Network  staff,  the  satisfaction  of  achieving  higher  results 
with  students  is  a motivating  factor  for  continued  participation. 

Indicators  of  Success 

Ihs  first  indicator  of  success  is  the  continued  demand  for  Galileo  Educational  Network’s  services.  There  is  a great 
demand  from  schools  all  over  the  province  to  get  involved  in  the  Galileo  Educational  Network.  In  it’s  third  year  of 
operation,  the  Galileo  Network  initiative  is  growing  from  the  current  9 schools  in  five  school  divisions  to  working  in 
1 8 schools  in  7 different  school  districts  in  2001/2002.  A second  indicator  used  to  evaluate  the  success  of  the 
initiative  is  corporate  investment  in  co-developing  an  online  professional  development  service.  Intelligence  Online 
(10),  to  extend  and  grow  the  Galileo  Network’s  capacity  to  provide  support  to  teachers.  A third  indicator  used  to 
assess  whether  the  Galileo  Network  has  been  successful  in  achieving  their  objectives  is  stakeholder  satisfaction. 
Overall,  it  can  be  reported  with  confidence  that  there  were  very  high  levels  of  satisfaction  among  stakeholders  at 
every  level  of  the  Galileo  Network  initiative.  Conversations  with  Galileo  Network  staff  and  school  administrators 
yielded  positive  indications  that  key  school  board  personnel  are  not  only  satisfied,  but  are  actively  developing 
strategies  to  spread  the  Galileo  Network  initiative  to  other  schools  in  their  division.  A fourth  indicator  used  to  assess 
whether  the  Galileo  Network  has  been  successful  in  achieving  their  objectives  is  evidence  of  engaged  student 
learning.  Instead  of  designing  integrated  units  or  students  tasks  that  had  pre -determined  outcomes,  teachers 
designed  projects  in  such  a way  that  students  ideas,  questions  and  prior  knowledge  become  an  important 
contribution,  and  developed  assessment  strategies  that  responded  to  multiple  ways  of  being  right.  Experienced 
teachers  became  convinced  of  the  value  of  an  inquiry -based  approach  to  technology  integration  when  they  observed 
the  enthusiasm,  interest  and  independence  of  their  students.  Teachers  and  school  administrators  commented  on  how 
learning  was  enhanced  for  students  of  diverse  abilities,  and  that  the  quality  of  work  achieved  by  many  students  went 
well  beyond  their  expectations.  A fifth  indicator  used  to  assess  whether  the  Galileo  Network  has  been  successful  in 
achieving  their  objectives  is  evidence  of  transformed  teaching  practice.  The  professional  development  support 
provided  by  the  Galileo  Network  is  much  less  about  technology  integration,  per  se,  and  is  instead  focused  more  on 
inquiry  into  fundamental  teaching  and  learning  issues.  Teachers  appreciated  having  ongoing  contact  with  a Galileo 
teaching  colleague  who  has  lived  experience  with  integrating  technology  with  children  and  can  assist  in 
pedagogical,  project  and  technology  skill  development.  Teachers  critically  examined  their  practice  and  determine 
next  steps  for  pushing  their  practice  forward  with  the  onsite  support  of  the  Galileo  Network. 

CONCLUSIONS 

Sustaining  learning  environments  that  enable  technology -infused,  inquiry -based  approaches  to  teaching  require  that 
a number  of  essential  conditions  in  the  school  be  maintained,  of  which  this  is  a non-exhaustive  list;  (1)  Supportive 
Leadership;  (2)  A Learning,  Risk-Taking  Culture  Among  Staff;  (3)  A Colleague,  From  Within  Or  Without,  To  Walk 
This  Road  With  You;  (4)  Ubiquitous  Access  To  Reliable  Technology;  (5)  Time  For  Professional  Dialogue  And 
Connections;  And  (6)  School  Board  And  Parent  Support.  A number  of  essential  conditions  for  the  sustainability  of 
the  Galileo  Network  initiative  within  a school  emerged  from  conversations  with  school  leaders,  classroom  teachers 
and  Galileo  staff.  Among  these  were;  (1)  Securing  Sustainable  Sources  of  Funding,  (2)  Building  On-Site  Capacity 
And  Leadership,  (3)  Diffusion  Of  The  Mentorship  Relationships,  and  (4)  Designing  Learning  Communities  to  Resist 
the  Urge  To  Turn  Back. 
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Key  Factors  that  contributed  to  the  success  of  the  Galileo  Educational  Network  initiative  during  the  2000/2001  can 
be  summarized  as  follows:  (i)  the  vision,  commitment  and  credibility  (i.e.,  reputation)  of  the  founding  members  of 
the  Galileo  Educational  Network;  (ii)  the  effective  enculturation  and  professional  development  of  new  staff  who  join 
the  Galileo  Educational  Network  (i.e.,  cultivation  of  a learning  organization);  (iii)  an  organization-wide  vision  and 
mandate  for  transformational  leadership  and  educational  reform  in  the  area  of  technology  integration  for  teaching 
and  learning;  (iv)  authentic  examp  les  and  images  of  best  practice  and  engaged  learning  to  share  with  school 
teachers;  (v)  active  involvement  of  Galileo  staff  in  teaching,  coaching  and  mentoring  within  the  classroom  context; 
(vi)  open  and  clear  communication  with,  and  support  from,  district  and  school  administration;  (vii)  willingness  and 
preparedness  of  school  staff  members  to  examine  practice  and  to  try  new  approaches  to  teaching  and  learning;  (viii) 
maintenance  and  growth  of  a research  agenda,  and  a mutually  beneficial  relationship  with  Alberta  universities  and 
research  partners;  (ix)  the  simultaneous  development  and  cultivation  of  mutually  beneficial  relationships  with  key 
personnel  at  the  school  board/district  level,  the  school  level,  and  with  corporate,  public  and  private  partners;  (x) 
strategic  leverage  of  existing  district  structures  and  personnel;  (xi)  widespread  satisfaction  of  participant 
stakeholders  at  all  levels  of  the  initiative;  (xii)  a professional  development  model  that  is  responsive  to  the  needs  and 
differences  within  and  between  school  districts  and  schools;  (xiii)  ability  to  be  innovative,  flexible  and  responsive  to 
stakeholders  because  the  Galileo  Network  operates  as  an  autonomous  organization  that  is  not  subject  to  the  political 
or  bureaucratic  constraints  of  any  one  educational,  corporate,  public  or  private  partner;  (xiv)  building  and  extending 
upon  innovative  ideas,  creative  solutions  and  information  on  latest  trends  through  strategic  alliances  with  critical  and 
political  friends,  corporations,  public  and  private  organizations  within  and  beyond  education;  (xv)  a focus  on  school- 
wide initiatives  that  permit  buy-in  as  teachers  feel  able  and  willing  to  participate. 
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Student  Perspectives  of  Workload  Pacing  in  International  Online  Graduate  Education: 

A Case  Study 

Diane  P.  Janes 

University  of  British  Columbia 

The  Context:  In  1997  the  Distance  Education  and  Technology  unit  of  Continuing  Studies  at  the 
University  of  British  Columbia  (UBC),  Canada,  in  partnership  with  the  Virtual  University  of  the  Monterrey 
Institute  of  Technology  (ITESM),  Mexico  launched  a five  course  post-graduate  Certificate  in  Technology- 
based  Distributed  Learning  (TBDL‘).  (For  details  on  the  partnership  see  Bates  and  Escamilla  de  los  Santos, 
1997).  The  first  course  in  the  certificate  was  offered  in  September  of  that  year  and  by  2001  students 
participating  came  from  over  27  countries  with  over  400  students  taking  the  courses  with  UBC  and  more 
than  1500  students  with  our  ITESM  partners.  The  keystone  to  the  design  of  each  of  the  courses  is 
collaborative  online  learning  and  constructivist  interactivity. 

This  paper/presentation  is  a case  study  of  one  of  these  courses  (Research  and  Evaluation  Issues  inTBDL). 
Using  a quasi-Stake  Responsive  Model  and  participant-observation  for  data-gathering  this  paper  will 
review  the  issues  raised  by  the  students,  while  the  presentation  will  review  the  lessons  learned  and 
recommendations  for  future  online  collaboration  and  research. 

Issues  Raised  by  Students. ..Workload:  In  this  course,  one  of  the  first  issues  raised  by  the  students 
involved  the  concept  of  heavy  workload.  All  of  the  learners  were  adult  (ranging  in  age  from  early  20s  to 
early  60s)  with  family,  community  and  work  responsibilities,  in  addition  to  their  learning.  Given  the 
graduate-level  nature  of  the  course,  a blend  of  reading,  reflection,  online  posting/discussion  and 
collaborative  group  work  formed  the  basis  for  interaction  and  eventual  assessment,  if  the  learners  were 
taking  the  course  for  credit  (There  were  opportunities  to  take  the  course  as  an  audit  student). 

As  one  student  noted  early  on  in  the  course:  “. . .I've  been  wondering  all  weekend  - as  1 write  assignment  2 - 
how  am  1 going  to  have  the  time  to  read  all  the  postings  that  are  going  to  arrive  by  Monday?. . .”  [Student  J, 
online  posting,  January  2001] 

Another  issue  of  concern  surrounded  the  group  assignments.  Collaborative  assignments  required  students 
to  work  together  online  through  time  zones,  language  barriers  and  different  experiences.  The  group  work 
culminated  in  a joint  paper  or  project  submission  and  students  expressed  concerns  about:  1 . the  need  to 
build  trust  within  the  group  as  each  member  was  unknown  to  the  other  and  2.  how  to  focus  the  group.  As  a 
student  asked  for  help,  she  questioned  “..  .Should  1 look  for  people  with  shared  interests  or  who  have 
different  backgrounds  to  complement  my  lack  of  expertise  in  certain  areas?”  [Student  C,  online  posting, 
January  2001] 

The  third  issue  of  workload  does  appear,  as  well,  to  be  more  an  issue  of  groups  in  general  rather  than 
specific  to  online  group  work.  As  one  student  noted,  “[The]  Last  time  I worked  with  a group,  was  on  a 
project  proposal.  I ended  up  writing  the  whole  thing  because  the  other  people  let  me  down. . ..”  [Student  M, 
online  posting,  January  200 1 ] 

Working  with  Different  First  Languages:  While  this  course  was  taught  in  English  and  all  course 
requirements  were  in  English,  the  two  partners,  UBC  and  ITESM,  found  a great  deal  of  interest  on  the  part 
of  the  international  community.  As  part  of  the  strength  of  the  course,  many  learners  with  English  as  their 
second  or  third  language  joined  the  international  collaborative  online  group  work.  Students  who  were 
experienced  in  working  online  with  ESL  (EngBsh  as  a Second  Language)  colleagues,  offered  this  advice: 
“When  you  have  ESL  issues  you  also  need  to  allow  time  to  sort  out  what  the  non-native  speaker  of  English 
is  trying  to  say,  unless  their  English  is  very  good  (as  was  the  case  with  one  of  the  Mexican  students).  As  the 
only  native  English  speaker,  1. . .volunteered  to  be  the  editor.  However,  I found  that  1 often  wasn't  certain 
what  they  were  trying  to  say  due  to  the  second  language  issues.  When  I edited,  1 needed  time  to  check 
back... to  make  sure  I hadn't  changed... intended  meaning,  which  turned  out  to  be  the  case  a few  times...” 
[Student  Q,  online  posting,  January  2001] 


“Getting  your  feet  wet”  vs.  “lurking”:  One  of  the  areas  that  instructional  developers,  online  course 
designers,  and  instructors  struggle  with  is  how  one  might  offer  the  learner  the  ‘safest’  way  to  enter  the 
course  and  the  learning.  What  we  found  over  the  five  courses  that  make  up  the  certificate,  is  that  each 
student  has  his  or  her  own  way  of  coping  and  that  often  we  cannot  predict  which,  of  the  choices  available  to 
them,  they  might  choose.  The  key  is  to  offer  options  rather  than  one  option.  In  this  course,  one  student 
suggested,  after  finding  the  beginning  introduction  to  the  new  concepts  a bit  overwhelming,  that  “.  ..It 
might  be  worth  creating  a dynamic  FAQ  page  in  this  course,  which  captures  and  presents  these. . . [general 
and  process]  questions  and  responses. . .”  [Student  D,  online  posting,  January  2001] 

Another  student  acknowledged  his  suggestion  but  offered  this  rationale  for  letting  these  general  and  process 
questions  and  responses  be  encouraged  early  on  in  a course:  “ ...  I am  just  thinking  out  loud  here  so  I could 
be  way  off  base...  Perhaps  the  confusion  provided  opportunities  for  people  to  "get  their  feet  wet"  by  asking 
relatively  safe  information  and  clarification  questions  rather  than  jumping  in  with  their  own  opinions.  A 
way  of  hanging  out  in  the  "student  union"  developing  the  social  context  needed  for  dialogical  teaching...” 
[Student  N,  online  posting,  January  2001] 

This  opinion,  was  followed  up  by  another  participant  who  noted  that  “. . .There's  another  point  as  well, 
though,  which  is  that  the  learning  styles  of  some  (e.g.  moi)  contain  less  of  the  hanging  out  at  the  Student 
Union,  discussing  and  arriving  at  a collective  conclusion  and  more  of  the  researching,  cogitating  and 
reflecting  before  we  feel  informed  enough  to  venture  an  opinion.  I think  a rough  parallel  could  be  made 
between  those  who  learn  and  retain  knowledge  through  mainly  oral  means  and  those  who  need  to  see 
something  written  to  absorb  it.”  [Student  Z,  online  posting,  January  200 1 ] 

Conclusion:  In  many  circumstances,  pacing  the  workload  is  both  physical  and  psychological  for  the 
instructor  and  the  learner.  It  is  important  to  address  the  reality  of  the  online  learning  as  well  as  its 
perception  by  the  new  online  learner.  It  has  been  the  experience  of  this  online  instructor  that  once 
comfortable  many  initial  workload  issues  become  more  familiar  and  less  stressful.  The  instructors  and  the 
learners  shared  a number  of  lessons.  The  accompanying  presentation  will  review  these  lessons  and  identify 
areas  that  need  to  be  addressed  by  those  new  and  experienced  in  the  field  of  online  teaching  and  learning. 
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Abstract:  A diverse  range  of  instructional  design  and  facilitation  strategies  are  emerging 
as  alternatives  to  content-focused  courseware  designed  for  self-paced  independent 
learning.  Improvisation  through  open  platform  design  is  one  such  emergence.  My 
colleague,  Dr  Sivasailam  Thiagarajan  and  I have  been  using  improvisation  as  a fast, 
cheap  and  effective  strategy  to  adapt,  re-use  and  continually  improve  over  20 
instructional  templates  we  call  email  learning  games.  Improvisation  allows  us  to  play 
with  the  rules,  not  by  the  rules. 


Introduction 

Since  1998,  my  colleague  Dr  Sivasailam  Thiagarajan  and  I have  designed  and  facilitated  email 
learning  games  with  over  2,500  people  worldwide  (Jasinski,  M & Thiagarajan,  S,  2000a,  Jasinski,  M.  & 
Thiagarajan,  S,  2000b).  An  email  learning  game  is  a reusable  text -based  template.  It  provides  a pre- 
structured  process  for  guiding  a group  of  participants  through  a series  of  tasks.  These  tasks  are  designed  to 
progressively  generate  and  process  content  around  a salient  issue.  A game  is  played  in  several  rounds  and 
through  a facilitated  process,  participants  engage  in  activities  like  brainstorming,  analyzing,  evaluating, 
synthesizing,  predicting  and  persuading  each  other.  While  individual  games  have  a specific  purpose  and 
outcome,  they  all  encourage  the  construction  and  sharing  of  new  knowledge,  understanding,  perspectives, 
and  insights. 


Open  platform  design  and  the  emergence  of  improvisation 

Our  work  with  email  learning  games  has  been  accelerated  by  our  practice  of  open  platform  design, 
where  we  provide  a growing  community  of  colleagues  with  access  to  our  templates  to  use,  improve, 
customize  and  return.  We  have  found  open  platform  design  to  be  powerful  in  rapidly  improving  design  and 
facilitation  processes  for  these  games  and  for  confirming  their  usability  in  a range  of  contexts.  Through 
this  iterative  and  dynamic  process,  what  has  also  emerged  is  the  concept  and  practice  of  improvisation  as 
an  instructional  design  framework.  Improvisation  is  a mix  of  ad-hockery  and  know-how. 

Improvisation  involves  reworking  precomposed  material  and  designs  in  relation  to  unanticipated  ideas  conceived, 
shaped  and  transformed  under  the  special  conditions  of performance,  thereby  adding  unique  features  to  every  creation 
(Berliner,  1994  p,24J), 

Like  improvisation  in  jazz,  instructional  design  using  improvisation  is  a disciplined  craft  that 
works  creatively  within  a planning  framework,  yet  can  take  advantage  of  the  unexpected.  In  short: 
Improvisation  does  not  involve  the  complete  discarding  of  planning,  but  a change  in  how  planning  is  done  and  in  how 
the  plan  is  viewed  (Isenberg,  1987,  p,  92), 

Improvisation  as  an  instructional  design  concept  exploits  and  works  with  contradictions  and 
paradox:  stable  and  unstable,  structured  and  flexible,  predictable  and  spontaneous,  planned  and  landom, 
content  and  process. 
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Improvisation:  dimensions  of  complexity 

Weick  (1998)  offers  a particularly  useful  insight  that  is  both  conceptually  and  practically  relevant 
to  the  context  of  improvisation  as  an  instruction  design  framework.  As  well  as  an  all-or-nothing  process, 
Weick  suggests  improvisation  can  also  be  incremental  with  varying  dimensions  or  levels  of  complexity. 
This  staged  approach  XQqmxQS... increased  demand  on  imagination  and  concentration  (p.  544). 

These  stages  range  from  interpretation  (taking  minor  liberties  with  the  original),  embellishment 
(reworking  parts  of  the  original  without  changing  it  drastically)  variation  (new  aspects  are  inserted  but  align 
with  the  original)  to  improvisation  (transforming  the  original  to  create  a drastically  different  version). 

This  supports  the  experience  we  have  had  with  coaching  practitioners  to  adopt  and  adapt  our  email 
learning  games.  The  emphasis  on  experiencing,  experimenting  and  incremental  development,  lends  itself  to 
work -based  methodologies  that  support  ‘designing-by-doing’. 

Design  jamming:  playing  with  the  rules  not  by  the  rules 

The  construction  of  email  learning  game  templates  and  their  subsequent  facilitation,  involves  what 
we  call  ‘design  jamming’,  a concept  adopted  from  jazz  improvisation  and  applied  to  the  instructional 
design  context.  Based  on  improvisation,  design  jamming  is  a dynamic  and  iterative  co-design  process  of 
constructing,  performing  and  adjusting  instructional  designs  as  a result  of  close  contact  with  the  learning 
audience  and  a community  of  colleagues.  The  essence  of  design  jamming  is  to  play  with  the  rules,  not  by 
the  rules. 

Conclusion 

Improvisation  provides  an  alternative  instructional  design  strategy  for  the  fast  changing  environment  of 
the  online  world  for  the  following  reasons: 

■ Small  and  incremental  changes  can  be  effected  quickly,  allowing  fast  responses  to  user 
requirements. 

■ Improvisation  provides  high  levels  of  feedback  and  closer  contact  between  teacher  and  learner. 

■ Design  jamming  involves  co-designing  with  a learning  audience  and  a community  of  colleagues 
allowing  for  rapid  improvement  of  both  design  and  facilitation  processes. 

Improvisation  through  open  platform  design  is  a complex  game  simply  played.  The  key  to 
improvisation  is  to  play  with  the  rules,  not  by  the  rules  — or  to  create  new  ones. 
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Navigating  the  Information  Space  of  a PC  Computer-Based  Aviation 
Fiight  Training  Device:  an  Expert  Evaiuation  of  Information  Search  and  Retrievai 

Jeffrey  S.  Foirest,  James  L.  Simmons,  and  Steven  D.  Zink 

The  Department  of  Transportation’s  Federal  Aviation  Administration  (FAA)  administers 
aviation  flight  training  and  pilot  certification  within  the  United  States.  Beginning  in  1997,  the 
FAA  began  permitting  the  use  of  PC-based  Aviation  Training  Devices  (called  PCATD’s)  for 
flight  training.  Research  has  provided  information  evaluating  PCATD  interface  design  and 
usability,  the  transfer  of  learning  associated  with  various  PCATD  related  training  procedures, 
and  pilots’  attitudes  concerning  the  PCATD  concept.  Most  of  the  current  research  that  has  been 
applied  to  the  PCATD  system  has  focused  on  assessing  simulation  fidelities  and  their  influence 
upon  training  effectiveness. 

However,  recent  concern  has  focused  on  problems  associated  with  a “multidimensional 
information  space,”  since  a PCATD  system  is  analogous  to  the  computer  based  interface  or 
“glass  cockpit.”  PC-based  flight  displays  have  introduced  a new  concern  regarding  human 
factors  identified  as  “navigating  through  an  information  space.”  These  systems  must  balance  the 
considerations  of  determining  what  information  should  be  provided  with  properly  designed 
features  that  enable  successful  operational  control  by  the  user. 

The  particular  PCATD  used  in  this  study,  a commercially  available  Jeppesen  Model  FS- 
200 AC,  operates  in  essentially  two  modes:  (1)  a full-screen  “glass  simulator”  mode,  showing  a 
view  from  the  pilot’s  seat  of  the  full  aircraft  instrument  panel,  including  a modest  amount  of 
ground  features  such  as  runways,  and  a “map- screen”  mode,  essentially  showing  representations 
of  aviation  charts.  Thus,  three-dimensional  information  is  simulated  within  a two  dimensional 
electronic  environment. 
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The  goal  of  this  study  was  to  identify  factors  affecting  the  user’s  ability  to  navigate  and 
find  information  identified  as  necessary  for  the  enhancement  of  situational  awareness  while 
using  the  PCATD’s  map-screen.  Five  expert  flight  and  ground  instmctors  evaluated  PCATD 
system.  These  experts  were  given  a basic  orientation  to  the  PCATD  hardware  and  software; 
specifically,  they  were  shown  how  to  “toggle”  back-and- forth  between  the  two  main  modes  of 
presentation.  They  were  then  asked  to  fly  a series  of  “precision  instrument  approaches”  to 
various  runways  contained  in  the  database.  Expert  evaluators  were  not  given  specific 
information  about  either  the  precision  instrument  approaches  (e.g.,  altitudes,  headings,  radio 
frequencies,  etc.)  or  related  data  contained  within  various  presentations  in  the  map-screens. 

Thus,  they  needed  to  successfully  formulate  and  execute  searches  using  the  map- screen  for 
precise  information  needed  to  successfully  complete  each  task. 

Although  the  relatively  small  number  of  expert  evaluators  to  this  study  precluded  the 
usage  of  inferential  or  parametric  statistical  analysis,  the  experts  were  able  to  describe  relevant 
issues  of  situational  awareness,  information  searching,  and  knowledge  are  generally  qualitative 
in  nature  by  the  use  of  open-ended  questioning. 

Overall,  the  experts  reported  that  it  was  moderately  difficult  to  locate  and  identify  tools 
inherent  to  the  map-screen  that  would  help  in  formulating,  searching,  and  retrieving  information. 
Considerable  exploration  of  icons,  menus,  and  menu  commands  was  needed  to  find  pertinent 
information.  The  experts  suggested  the  need  for  “finding  aids,”  such  as  intermediary  help  menus 
or  screens.  The  “zoom  control”  was  particularly  problematic,  since  it  frequently  contributed  to 
the  loss  of  situational  awareness  of  the  instrument  approach  being  flown.  The  experts  generally 
thought  a “pop-up”  quick-search  function,  ideally  one  that  would  allow  for  the  use  of  a specific 
search  term,  would  be  the  most  helpful. 
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Abstract.  This  article  deals  with  the  research  of  the  realization  interaction  in  the  Finnish 
Virtual  University’s  network-based  studies  of  cognitive  science  in  the  autumn  2001.  In  this 
research,  we  examine  the  independent  study  of  cognitive  science  which  occurs  through  a 
network  and  the  face-to-face  study  that  takes  place  in  cooperation  in  a group  and  the  study 
that  occurs  in  WebCT  groups.  The  research  seeks  answers  to  the  kind  of  interaction  that 
promotes  network- based  studies  and  what  causes  a lack  of  interaction.  The  data  has  been 
collected  by  questionnaires  and  observation.  The  results  indicated  that  students  considered 
interaction  with  the  teacher  and  other  students  more  binding  than  self-directed  study.  The 
expectations  and  biases  of  students  did  not  always  correspond  to  the  objectives  of  studies  and 
thus  have  prevented  functional  interaction  in  a network. 


1.  Introduction 


The  objective  of  the  study  is  to  examine  the  realization  of  interaction  in  cognitive  science  network  studies 
in  the  Finnish  Virtual  University.  Good  interaction  with  both  the  teacher  and  other  students  motivates  and  binds 
students  to  a greater  degree  of  study  (Richardson  2001).  Interaction  occurring  through  a network  with  the  teacher 
and  with  other  students  is  important  from  the  perspective  of  learning,  even  if  it  is  seen  as  far  more  laborious  than 
studying  face -to-face  (Heikkinen  2000). 

Interaction  in  a network  is  created  by  the  relationship  of  interaction  between  a student,  a teacher, 
technology,  and  the  others  who  are  part  of  the  process  (Ruokamo  & Pohjolainen  1999).  In  network  studies, 
interaction  between  the  teacher  and  the  student  is  limited,  neither  does  it  always  occur  in  real  time,  and  thus  it 
presents  its  own  challenge  to  successful  study  (Paakkola  1992). 

Cognitive  science  can  be  studied  in  a self-directed  manner,  under  the  direction  of  the  teacher,  or  with 
other  students  using  a WebCT  as  well  as  on  the  Internet  through  contact  teaching  by  studying  in  groups 
[http://www.virtuaaliyliopisto.fi/osahankkeet/connet/webct.htm]. 

In  studying  cognitive  science,  we  utilize  problem-based  learning  and  exercises  are  performed  face-to-face 
as  group  work  (Hakkarainen,  Lonka  & Lipponen  1999).  In  the  study,  we  examine  the  realization  of  interaction  in 
network  studies,  self-directed  study,  as  well  as  the  adaptability  of  group  work  face-to-face  exercises  as  part  of 
network -based  studies,  and  the  types  of  problems  that  arise  in  interactive  study  through  a network,  and  what  the 
lack  of  interaction  causes. 


2.  Background 

The  Finnish  Virtual  University  Project  [http://www.virtuaaliyliopisto.fi/English/index.php]  has  been 
established  based  on  the  information  strategy  of  the  Ministry  of  Education  (for  the  years  2000-2004).  Its  concept 
is  a joint  virtual  university  in  Finland  consisting  of  several  institutions  for  advanced  learning,  businesses,  and 
research  institutions  that  provides  internationally  high  quality  competitive  training  services.  (Ministry  of 
Education  1999). 

The  study  of  cognitive  science  in  a network,  in  other  words  the  Connet  Project,  has  been  planned  since 
1999  [http://www.virtuaaliyliopisto.fi/osahankkeet/connet/].  The  project  includes  students  from  eight  different 
universities:  Helsinki,  Tampere,  Turku,  Joensuu,  Oulu,  Lapland,  Jyvaskyla,  and  the  Helsinki  University  of 
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Technology.  The  Connet  Project  was  coordinated  by  the  University  of  Helsinki  during  the  autumn  semester 
2001;  with  respect  to  the  University  of  Lapland,  it  was  coordinated  by  the  Centre  for  Media  Pedagogy 
(Ruokamo,  Syijakari  & Karppinen  2002).  Basic  studies  in  cognitive  science  began  at  all  the  universities 
participating  in  the  project  in  the  autumn  200 1 . Seventy -eight  students  had  registered  for  the  common  e -mail  list. 
Following  their  basic  sUidies,  the  students  will  have  the  opportunity  to  conduct  their  cognitive  science  studies  as 
network  studies.  Each  university  is  responsible  for  a defined  credit  unit  and  studies  will  consist  of  modules 
chosen  by  the  student. 

Teaching  consists  of  study  periods  in  cognitive  science,  data  processing,  psychology,  labour  science, 
pedagogy,  media  education,  cognitive  technology,  and  the  philosophy  of  technology.  Teaching  methods  use  the 
possibilities  brought  about  by  new  technology  that  utilizes  problenvbased  project-oriented  method. 

In  network  studies , learning  is  a process  and  a social  event  that  needs  for  its  support  feedback  and  support 
from  teachers  as  well  as  interactive  study  with  the  other  students  (Richardson  2001).  Network  students  are 
generally  highly  motivated,  even  if  the  lack  of  social  communication,  interaction,  and  unclear  expectations  affect 
studying  (Bonk  200 1 ). 

Interaction  skills  are  important  cooperative  skills,  but  group-centred  skills  are  broader  in  nature.  The 
cooperative  skills  of  an  individual  come  into  play  only  when  in  a community  where  the  social  activity  that 
rewards  them  is  organized.  It  is  perceived  that  in  learning,  there  are  still  too  many  individually  centred  learning 
models  that  prevent  interaction  and  forms  of  communal  activity.  The  individual  needs  an  encouraging  and 
socially  respected  operating  environment.  (Kuusinen  2001 .) 


3.  1 he  Objective  and  Research  Questions  of  the  Study 

The  study  examines  the  realization  of  interaction  in  cognitive  science  network  studies.  Network 
interaction  is  created  from  different  interactive  contexts.  The  forms  of  interaction  are  the  interaction  between 
students,  teacher  and  student,  student  and  technology,  and  between  the  actors  (Ruokamo  & Pohjolainen  1999). 
The  study  examines  the  type  of  interaction  that  promotes  network  learning  in  the  study  of  cognitive  science  and 
what  causes  the  lack  of  interaction.  A focus  for  examination  is  also  self-directed  study,  inter-student  interaction 
in  WebCT,  and  the  face-to-face  study  that  takes  place  in  groups  with  other  course  members.  This  study  seeks 
answers  to  how  self-directed  study  is  located  within  the  context  of  interaction.  In  multiform  teaching,  there  is  a 
reduction  in  teacher  leadership;  different  forms  of  interaction,  the  purpose  of  which  would  be  to  support  study, 
replace  interaction  between  student  and  teacher  (Paakkola  1992).  Moreover,  the  study  examines  whether  face-to- 
face  group  study  best  supports  a student* s motivation  and  learning  in  network  studies,  and  whether  it  binds  the 
student  more  firmly  to  his  or  her  studies. 

The  culture  of  interaction  in  a network  environment  is  a broad  area  and  thus  it  places  its  own  challenges 
on  the  work  of  research  and  its  analysis.  Research  into  network  studies  has  increased  in  recent  times,  but  most  of 
the  research  has  focused  on  analysing  information  produced  for  the  network  and  the  nature  of  student 
participation.  The  nature  of  a network -based  social  information  structure  and  the  characteristics  of  its  interaction 
process  have  still  to  be  studied  whereas,  in  addition  to  the  network  environment,  the  entire  interaction  context 
with  its  discussions  must  receive  attention.  Learning  is  perceived  as  a communal  process  and  interaction  and, 
from  the  perspective  of  learning,  it  is  created  outside  of  the  network  environment.  (Salo,  Hurme  & Jarvela  2001.) 


4.  Research  Method 

The  study  is  limited  to  the  study  periods  realized  during  basic  studies  in  cognitive  science  in  the  autumn 
semester  of  2001.  The  interaction  in  a network  linked  to  study  is  a multkctor  phenomenon.  The  phenomenon 
consists  of  students,  teachers,  the  network  tools  of  the  learning  environment,  and  other  essential  actors 
(Ruokamo  & Pohjolainen  1999).  In  order  to  gain  relevant  information  and  new  perspectives,  interaction  should 
be  examined  both  quantitatively  and  qualitatively. 

The  data  was  collected  by  questionnaires  and  observation.  The  first  questionnaire  was  performed  using  an 
Internet  form  and  through  e-mail  before  the  actual  course  began.  The  questionnaire  was  sent  to  the  78  students 
that  had  registered  for  the  common  cognitive  science  students’  e-mail  list.  The  questionnaire  measured  the 
students’  biases,  expectations,  and  thoughts  on  the  realization  of  interaction  in  the  study  of  cognitive  science. 

A second  questionnaire  was  performed  at  the  end  of  the  autumn  semester  courses.  It  was  sent  to  the  78 
students  and  to  43  students  that  had  performed  their  practice  work.  The  second  questionnaire  focussed  on  the 
realization  of  interaction  in  the  network-based  study  of  cognitive  science  and  the  use  of  WebCT  tools.  In 
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addition,  the  same  questionnaire  included  questions  directed  at  the  students  on  the  Project  Working  and  Creative 
Planning  and  Communication  and  Cognition  courses.  The  students  on  the  Project  Working  and  Creative 
Planning  course  studied  in  groups  using  the  WebCT  and  the  students  on  the  Communication  and  Cognition 
course  studied  in  face-to-face  groups. 

The  perspective  of  teachers  on  interaction  has  been  studied  in  a questionnaire  directed  at  those  teachers 
who  gave  their  permission  for  the  form  to  be  sent.  The  aim  has  been  to  resolve  how  they  think  they  have 
succeeded  in  creating  interaction  with  the  students,  how  group  work  has  been  articulated  for  network -based 
studies,  and  how  they  have  experienced  teaching  through  a network. 

The  project  researcher  has  participated  in  cognitive  science  planning  meetings  as  a participatory  observer 
since  the  spring  of  2001  and  she  has  collected  data  using  the  teachers’  and  assistants’  common  email  list. 
Moreover,  the  researcher  has  participated  as  a student -observer  for  the  video  lectures  on  the  Communication  and 
Cognition  course. 

The  Likert  scale  theorems  of  the  questionnaires  have  been  processed  by  quantitative  methods  using  the 
SPSS  statistics  application.  The  participatory  observations  of  the  researcher  as  well  as  the  data  collected  using 
open  questionnaires  has  been  analysed  using  qualitative  methods. 


5.  Results  and  Conclusions  of  the  Study 

Eleven  students  (15.4  %)  returned  the  first  questionnaire  and  16  students  (13.2  %)  returned  the  second 
questionnaire.  The  students  encountered  technical  problems  in  returning  both  the  Internet -based  questionnaire 
and  the  one  sent  by  e-mail  in  Word  format.  Twenty-seven  forms  were  used  in  the  study.  There  were  more 
returned,  but  these  were  either  empty  or  insufficiently  completed  or  the  student  had  not  completed  his  or  her 
network-based  studies  into  cognitive  science  even  though  his  or  her  name  appeared  on  the  e-mail  list. 

When  examining  self-directed  study  in  a network  in  relation  to  other  network  interaction,  students  saw 
interaction  with  teachers  and  other  students  as  more  binding  that  self-directed  study.  Network  interaction  was 
seen  as  more  binding  in  relation  to  time  and  place  than  self-directed  study  was.  Part  of  the  courses  had  to  be 
completed  as  video  lectures  and  study  diaries.  The  defined  times  of  video  lectures  and  their  related  study  diaries 
to  a degree  prevented  graduate  students  and  those  students  at  work  from  completing  the  courses  in  question. 
Network-based  study  is  generally  seen  as  study  independent  of  time  and  place  and  it  is  also  marketed  as  such.  In 
this  relation,  the  pre  -expectations  of  the  students  for  study  independent  of  time  and  place  were  not  realized. 

In  the  pre  -expectations  of  the  students  before  courses  began,  it  was  apparent  that  teachers  did  not  have  a 
clear  picture  of  the  possibilities  provided  by  network-based  teaching.  The  students  in  different  localities 
considered  important  the  help  and  guidance  of  a local  guide  and  responsible  person,  the  possibility  to  discuss 
courses,  as  well  as  the  opportunity  for  technical  support.  Interaction  with  the  teacher,  especially  in  the  initial 
phase  of  the  courses,  acts  as  a motivating  factor  for  a course. 

The  interaction  between  the  students  in  the  study  has  been  examined  in  the  face-to-face  group  work  on 
the  Communication  and  Cognition  course  and  in  the  WebCT  group  work  on  the  Project  Working  and  Creative 
Planning  course.  Face-to-face  group  work  was  conceived  as  being  part  of  network-based  studies.  It  was  seen  to 
make  handling  matters  easier  and  students  thus  gained  extra  motivation  for  other  network-based  study.  Typical 
problems  related  to  the  performance  of  group  work,  such  as  the  heterogeneity  of  the  group  and  the  uneven 
distribution  of  responsibility,  were  seen  in  the  face4o-face  group  work.  The  groups  included  students  from 
different  localities,  so  dstances  and  timetables  prevented  and  slowed  study.  The  students  did  not  consider 
slowing  or  preventing  studies  as  the  intention  of  network-based  studies. 

Based  on  the  open  questions  in  the  qualitative  examination  of  face-to-face  practice,  contact  teaching  arose 
as  the  best  form  of  guidance,  next  was  video-conferencing  and  in  addition  to  these,  e-mail.  The  students  thought 
all  forms  of  teaching  supported  one  another,  when  they  were  well  organized. 

In  the  students’  ^=11)  pre -expectations,  it  was  apparent  that  almost  half  the  students  (45,5  %)  to  a 
degree  believed  that  WebCT  supported  interaction  between  a student  and  other  students.  Almost  one -third  of  the 
students  (27.3  %)  did  not  have  the  same  or  a different  opinion;  less  than  one-fiflh  (18.2  %)  was  somewhat  of  a 
different  opinion  and  only  one  student  was  of  a different  opinion.  Over  one-third  of  the  students  had  pre- 
expectations  that  WebCT  would  to  a degree  support  the  interaction  between  the  student  and  teacher.  Only  one 
student  has  the  same  opinion.  Less  than  one-fifth  (18.2  %)  of  the  students  did  not  have  the  same  or  a different 
opinion,  similarly  were  to  a degree  of  a different  opinion  and  the  same  number  were  of  a different  opinion  (18.2 
%). 

The  group  on  the  Project  Working  and  Creative  Planning  course  was  formed  though  WebCT  discussions 
field.  More  real-time  discussion  would  have  been  needed  at  the  beginning  of  studies  and  there  would  have  been  a 
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need  for  functional  group  working.  There  was  limited  experimenting  with  WebCT  in  group  work,  and  it  was  not 
found  to  support  the  group’s  joint  writing  process. 

The  students  (N=16)  evaluated  the  best  interactive  situations  on  a scale  of  4-10.  The  best  interactive 
situations  among  the  students  were  created:  1.  in  practice  work  (mean  = 9),  2.  with  e-mail  (mean  = &3),  3.  in 
contact  teaching  (mean  = 8.2)  and  4.  in  WcbCT  discussions  (mean  = 7.6).  Other  interactive  situations  were 
realized  using  WebCT  communication  and  group  tools,  the  mean  of  which  varied  between  5.8- 6.5. 

The  students  (N=16)  chose  the  best  interactive  tools  on  a scale  of  4— 10.  The  best  interactive  tools  were:  1. 
some  e-mail  other  than  WebCT  (mean  = 8.5),  2.  video -conferencing  (mean  = 7.8),  3.  contact  teaching  (mean  = 
7.6),  4.  in  WebCT  discussions  (mean  = 7),  5.  WebCT  e-mail  (mean  = 6.6),  6.  the  WebCT  notice  board  (mean  = 
6.3),  7.  telephone  discussions  (mean  = 6),  and  8.  WebCT  real-time  discussion  (mean  = 5.7). 

The  fluent  and  versatile  use  of  a learning  environment  appears  to  increase  the  entire  interaction  with  the 
student,  teacher,  and  other  students.  Those  coming  on  a network -based  course  for  the  first  time  need  guidance 
and  instructions  on  the  use  of  the  learning  environment  and  its  tools.  The  use  of  WebCT  in  the  study  of  cognitive 
science  could  include  credits  during  the  orientation  phase  to  studies.  In  other  respects,  the  biases  of  the  students 
to  computers,  the  use  of  e-mail  and  information  searches  were  good  in  relation  to  their  own  skills. 

The  interaction  between  actors  was  observed  using  the  common  email  list.  Email  functions  as  a 
common  information  channel  between  teachers,  designers,  and  assistants.  A common  email  list  provides 
support  for  the  actors  to  agree  on  plans,  timetable,  and  meetings. 

The  objectives  of  teaching  and  the  expectations  of  students  do  not  always  correspond  to  each  other.  A 
factor  affecting  interaction  is  the  cultural  background  to  the  organization  arranging  teaching.  The  results  of  the 
research  support  the  importance  of  guiding  teachers  and  other  students  in  network-based  studies.  Face-to-face 
study  should  be  a part  of  network-based  studies.  However,  self-directed  study  in  a network  should  be  respected 
and  various  opportunities  should  be  provided  on  courses  for  interactive  contexts. 

The  research  has  examined  interaction  from  various  interaction  contexts  as  well  as  noted  self-directed 
study  and  how  it  is  located  within  the  context  of  interaction.  The  research  has  become  extensive  and  it  has 
provided  a lot  of  information  and  additional  material  for  those  organizations  arranging  teaching  and  for  all  those 
interested  in  interaction  in  network-based  study  and  teaching.  The  information  and  additional  material  from  the 
research  can  benefit  educational  purposes,  the  development  of  learning  environments,  and  follow-up  research 
both  in  Finland  and  internationally. 

This  study  examined  different  face-to-face  group  work  and  the  group  work  that  takes  place  in  a learning 
environment.  The  researcher’s  personal  interest  has  grown  in  studying  face-to-face  group  work  and  the 
integration  of  group  work  into  a network-based  learning  environment  aid  thus  into  the  appearance  of  the 
observed  problems  as  well  as  into  the  development  of  the  fluent  and  versatile  opportunities  for  network -based 
group  work. 
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Abstract:  This  paper  describes  our  experiences  with  sequence  learning  activities  for  intelligent  3- 
D practice  environments.  An  intelligent  practice  environment  offers  learning  by  doing  and 
learning  by  discovery  in  a realistic  practice  situation.  A Knowledge-Based-System  (KBS) 
improves  learning  opportunities  with  dynamic  advice  and  feedback.  An  expert  system  monitors 
user  interactions  to  provide  dynamic  advice  for  the  expected  sequence  of  user  activities  and 
updates  objects  in  the  3-D  world  for  the  current  state  in  the  sequence.  A prototype  implementation 
is  described  for  intelligent  3-dimensional  (3-D)  practice  environments  developed  using  a web 
browser,  the  Virtual  Reality  Modeling  Language  (VRML),  JAVA,  and  an  expert  system.  The 
VRML  enabled  web  browser  provides  an  interactive  3-D  world  where  the  user  can  learn  and 
practice  sequential  tasks. 


Overview 

This  paper  focuses  on  our  experiences  integrating  3-D  practice  environments  with  a sequence  learning  strategy 
managed  by  an  expert  system.  Sequences  are  a foundation  for  learning  complex  skills  and  high-level  problem 
solving.  An  example  sequence  learning  approach  was  developed  to  illustrate  the  capabilities  of  intelligent  3-D 
practice  environments  for  mechanical  skills  training.  This  is  an  on-going  research  activity  whose  long-term 
success  is  largely  determined  by  the  maturity  of  the  technology  needed  to  implement  the  intelligent  3-D  practice 
environments.  The  ability  to  implement  a few  working  prototypes  has  been  proven  by  integrating  the 
technologies  described  in  this  paper.  Our  goals  are: 

• Understand  the  capabilities  required  to  ensure  users  can  learn  by  doing  and  discovery 

• Identify  and  investigate  useful  learning  strategies,  such  as  sequence  learning,  that  take  advantage  of  the 
exploratory  learning  process  offered  by  practice  environments 

• Develop  a basic  framework  or  architecture  that  supports  re-using  basic  components 

• Devise  ways  to  separate  practice  session  content  from  the  basic  re-usable  framework. 

Cost  is  always  an  issue  and  affordability  is  a major  objective.  Finding  ways  to  make  practice  environments  an 
affordable  solution  is  a critical  success  factor  for  adoption  of  the  technology.  There  are  two  major  cost 
considerations  for  this  web  technology,  first  we  must  be  able  to  afford  to  develop  the  practice  environments  and 
second  we  must  be  able  to  afford  to  deploy  the  content  to  the  users.  One  way  of  reducing  development  cost  is 
to  develop  modular  components  that  can  be  reused.  There  is  a high  reuse  potential  for  the  components  we  have 
developed.  The  expert  system  with  its  JAVA  applet  and  VRML  interface  applet  are  reusable.  The  rule  base  for 
processing  sequence  learning  procedural  steps  and  goals  is  reusable.  The  sequence  learning  fact  structure  is 
reusable.  The  specific  facts  for  the  procedures  and  the  VRML  worlds  are  content  specific  and  have  to  be 
developed  as  lesson  content.  Techniques  for  deploying  content  to  the  user  will  evolve  as  the  technologies 
mature. 

Sequence  Learning  Strategies 

The  objective  of  intelligent  3-D  practice  environments  is  to  provide  a highly  interactive,  realistic  learning 
experience  where  the  user  learns  by  doing  and  by  discovery.  Sequence  learning  is  an  important  component  of 
learning  complex  skills  and  advanced  problem-solving  skills.  Sequence  learning  strategies  can  provide  practice 
and  discovery  activities  for  the  user.  To  develop  advanced  skills,  the  user  needs  to  practice  solving  a wide 
range  of  problems  and  handling  different  situations.  A highly  interactive  practice  environment  encourages  the 


'906 


user  to  explore  complex  relationships  and  increases  the  development  of  advanced  skills  through  self- motivated 
discovery  and  problem  solving  activities.  Sequence  learning  features  of  a practice  environment  are: 

1.  Monitoring  and  controlling  the  situation  in  the  3-D  world 

2.  Recognizing  goals  accomplished  by  the  user 

3.  Using  facts  and  rules  to  recognize  correct  sequences  of  actions  by  the  user 

4.  Offering  advice  and  feedback  for  correct  and  incorrect  sequences  of  user  actions 

5.  Generating  actions  for  the  objects  in  the  3-D  world  based  on  the  current  sequence  location 

6.  Dynamically  controlling  the  behavior  of  objects  in  the  3 -D  world  based  on  user  actions. 


Practice  Environment  Development  Process 


There  are  two  main  phases  in  our  current  prototype  development  process.  First,  the  technology  must  be 
developed  and  integrated  to  create  the  desired  capabilities  of  the  practice  environment.  Our  research  activities 
have  shown  that  it  is  critical  for  the  technology  to  be  sufficiently  mature  and  robust  to  support  the 
implementation  of  the  desired  practice  environment  capabilities  and  learning  strategies.  Second,  after  a practice 
environment  has  been  developed  and  tested,  learning  content  must  be  developed  for  the  practice  sessions. 


Developing  an  intelligent  3-D  practice  environment  requires  software  developers,  instructional  designers,  3-D 
graphics  experts,  subject  matter  experts,  and  rule  base  programmers.  The  initial  development  of  an  intelligent 
practice  environment  requires  a technical  team  that  can  implement  the  practice  environment  with  the  desired 
learning  strategics.  A robust  practice  environment  can  be  de’/eloped  with  re-usable  software  components, 
expert  system  rule  bases,  and  expert  system  fact  bases.  Once  a re-usable  practice  environment  has  been 
developed,  the  focus  shifts  to  learning  content  development. 


Learning  content  development  for  an  intelligent  3-D  practice  environment  involves  three  critical  activities  in 
addition  to  the  traditional  subject  matter  expertise  required  for  creating  a good  learning  session.  For  a sequence 
learning  practice  session,  the  subject  matter  expert  first  decomposes  the  tasks  into  interactive  actions  and 
sequences  of  actions  that  can  be  performed  by  the  user.  The  behavior  of  objects  in  the  practice  environment  is 
identified  for  each  user  action  and  accumulated  sequence  of  user  actions.  Second,  3-D  worlds  must  be  created 
and  populated  with  3-D  objects  that  support  the  required  user  interactions,  controls,  and  object  behaviors. 

Based  on  the  desired  practice  and  exploratory  activities,  sensors  must  be  added  to  the  3-D  objects  to  control 
their  behavior  and  trigger  actions,  to  manage  user-controlled  movement  and  placement  of  objects,  to  monitor 
user  interactions  with  the  objects,  to  monitor  the  user’s  position,  and  to  control  the  state  of  objects.  Third, 
knowledge  engineering  is  required  to  define  the  facts  and  develop  the  rules  necessary  to  support  the  learning 
and  practice  activities  performed  by  the  user  in  the  3-D  world. 


Example  Prototype  Architecture 

Technology  and  especially  the  maturity  of  current  technologies  are  critical  to  the  development  of  intelligent  3-D 
practice  environments.  For  the  intelligent  3-D  practice  environment  illustrated  in  this  paper,  we  used  the 
VRML  External  Authoring  Interface  (EAI)  to  integrate  the  Jess  expert  system  with  VRML  practice 
environments.  Jess  (Copyright  1998  E.  J.  Friedman-Hill  and  the  Sandia  Corporation)  [Friedman-Hill  1998]  is 
an  expert  system  shell  written  in  JAVA.  The  source  code  for  Jess  is  available  for  integration  with  a practice 
environment.  Jess  is  also  distributed  with  sample  applets  and  console  applications.  We  modified  the  Jess 
expert  system  interface  to  include  buttons  for  feedback  and  advice.  Figure  1 shows  the  configuration  of  existing 
products  and  developed  software  integrated  to  create  an  intelligent  3-D  practice  environment. 

A simple  knowledge  base  manages  the  sequence  of  practice  activities,  feedback,  and  advice  for  the  practice 
environment.  A set  of  facts  defines  the  practice  activities,  the  sequence  of  activities,  advice  for  each  activity 
the  user  can  accomplish  at  the  current  time,  and  feedback  when  an  activity  is  successfully  accomplished.  The 
knowledge  base  contains  a set  of  rules  that  manage  the  user's  practice  activities  and  the  behavior  of  objects  in 
the  practice  environment  by  determining  which  activities  are  valid  at  the  current  point  in  time  and  updating  the 
3-D  world  based  on  the  current  situation.  Table  1 provides  information  about  each  of  the  interfaces  in  the 
implementation. 
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5 


VRML  Nodes 


Knowledge-Based-System 
(Jess  Expert  System) 


Facts  for 
Knowledge  Base 

Rules  for 
Knowledge  Base 

Persistent  data  for 
multiple  sessions 


Figure  1:  Prototype  Implementation  of  an  Intelligent  3-D  Practice  Environment 


Interface 

Number 

Interface 

Origin 

Interface 

Destination 

Purpose  of  Interface 

1 

Rule  Base 
Fact  Base 

Expert  System 

Load  learning  strategy  rules  and  initial  set  of  facts  into 
the  expert  system  knowledge  base. 

2 

Persistent  Data 
Store 

Expert  System 

Load  current  data  for  facts  and  other  state  information 
stored  from  last  session 

3 

3-D  scene  User 
interactions 

3-D  Nodes  and 
Sensors 

Access  information  about  user  interactions  in  the 
practice  environment 

4 

3-D  Nodes  and 
Sensors 

Java  EAI  event 
handler 

Pass  user  interaction  information  and  other  state 
information  to  the  Java  EAI  event  handler  for  analysis 

5 

Java  EAI  event 
handler 

Expert  System 

Update  knowledge  base  and  assert  new  facts  based  on 
user  interactions  and  state  of  the  3-D  objects 

6 

Expert  System 

Persistent  Data  Store 

Store  facts  and  state  information  to  be  available  in 
another  session. 

7 

Expert  System 

Java  EAI  event 
handler 

Update  state  data,  feedback,  and  advice  in  the  practice 
environment 

8 

Java  EAI  event 
handler 

3-D  Nodes  and 
Sensors 

Pass  updated  data  from  the  expert  system  and  Java  EAI 
event  handler  to  the  practice  environment 

9 

3-D  Nodes  and 
Sensors 

3-D  Scene 

Update  the  scene  with  expert  system  and  Java  EAI  event 
handler  data. 

Table  1:  Interfaces  in  the  Intelligent  Practice  Environment 


Mechanical  Skills  Example 

An  example  is  the  best  way  to  illustrate  the  intelligent  3-D  practice  environment  concepts.  This  exa  mple  was 
developed  to  teach  maintenance  technicians  how  perform  shaft  alignment  tasks.  Figure  2 shows  a horizontal 
parallel  misalignment  correction  practice  environment.  The  expert  system  advice  and  feedback  is  displayed  at 
the  top  of  the  screen.  The  user  receives  feedback  from  his  actions  in  the  expert  system  display.  The  expert 
system  determines  which  activities  the  user  can  perform  at  the  current  time  in  the  3-D  scene.  State  information 
from  the  expert  system  is  used  to  activate  sensors,  animations,  and  user  selections  in  the  3-D  scene.  The  expert 
system  display  has  two  buttons  that  give  the  user  access  to  advice  and  feedback  at  any  time.  The  user  requests 
advice  with  the  advice  button.  The  user  automatically  receives  feedback  when  he  accomplishes  one  of  the 
active  tasks.  The  effectiveness  of  the  advice  and  feedback  is  dependent  on  the  quality  of  the  knowledge 
captured  in  the  knowledge  base. 
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Intelligent  31>  Practice  Environments 


You  have  completed  a goal  in  step  5 3 

BoU  9 o'clock  back  correctly  Tightened 

VRML  action  completed  goal  tightenboltSf  in  step  5 
You  have  completed  a goal  in  step  5 
Bolt  9 o'clock  front  correctly  Tightened 

VRM  L action  completed  goal  tightenbolt3f  in  step  5 
You  have  completed  a goal  in  step  5 
Bolt  3 o’clock  front  correctly  Tightened 

Feedback:  Step  5 completed  I 

<Fact-99>  J 

Jess>  Advice:  Mount  dial  indicator  on  back  motor  bolt  on  9 o'clock  side 

Advice^  IZIZZIIIZZI  If  Feedback"] 


Figure  2:  Horizontal  Parallel  Correction  Practice  Session 

The  practice  session  in  Figure  2 illustrates  problem  solving  for  a set  of  sequential  activities  that  the  user  will 
accomplish  in  the  3-D  practice  environment.  An  example  of  the  fact  data  for  this  sequence  learning  problem  is 
shown  in  Table  2.  The  initial  fact  data  is  loaded  into  the  knowledge  base  when  the  practice  environment  is 
started  and  the  facts  are  updated  as  the  user  works  in  the  practice  environment.  The  data  in  Table  2 reflects  the 
current  fact  set  for  the  situation  shown  in  Figure  2 where  the  user  has  completed  steps  1 through  5 and  is  ready 
to  perform  step  6 in  the  sequence. 

The  rules  in  the  rule  base  interact  with  the  3 -D  practice  environment  to  activate  objects  that  the  user  must  work 
with  to  accomplish  goals  in  the  procedure.  These  objects  include  the  dial  indicator,  the  reset  or  zero  knob  on 
the  dial  indicator,  the  motor,  and  the  motor  mounting  bolts.  For  step  6,  the  only  active  component  in  the  3-D 
practice  environment  is  the  dial  indicator  that  needs  to  be  mounted  on  the  9:00  side  of  the  motor  near  the  back 
mounting  bolt.  There  were  3 active  components  in  step  5 where  the  user  was  required  to  tighten  3 bolts  in  an 
undefined  order. 

The  current  rules  implemented  in  the  sequence  learning  rule  base  have  the  following  behavior: 

1.  All  goals  for  the  current  step  are  activated  and  can  be  accomplished  by  the  user. 
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2.  All  goals  in  a step  must  be  accomplished  before  activating  the  goals  in  the  next  step. 

3.  The  user  can  accomplish  the  active  goals  in  the  current  step  in  any  order. 

4.  When  requesting  advice,  advice  for  any  of  the  active  goals  can  be  given  to  the  user. 

5.  Feedback  is  automatically  given  to  the  user  whenever  the  user  accomplishes  a goal  or  a step 

6.  The  user  can  request  feedback  identifying  their  current  situation  at  any  time. 


Step 

Goal 

Next 

step 

Advice  for  next  action  in 
sequence 

Feedback  for  correct 
action  in  sequence 

Active 

Completed 

1 

Mount 

dial 

indicator 

2 

Mount  dial  indicator  near 
front  motor  bolt  on  9 
o’clock  side 

Dial  indicator  correctly 
mounted  on  front  motor 
bolt  on  9 o’clock  side 

No 

Yes 

2 

Zero  dial 
indicator 

3 

Zero  the  dial  indicator  on 
front  motor  bolt  on  9 
o’clock  side 

Dial  indicator  correctly 
zeroed  on  front  motor  bolt 
on  9 o’clock  side 

No 

Yes 

3 

Loosen 
bolt  3f 

4 

Loosen  3 o’clock  front 
mounting  bolt 

Bolt  3 o’clock  front 
correctly  loosened 

No 

Yes 

3 

Loosen 
bolt  9f 

4 

Loosen  9 o’clock  front 
mounting  bolt 

Bolt  9 o’clock  front 
correctly  loosened 

No 

Yes 

3 

Loosen 
bolt  9b 

4 

Loosen  9 o’clock  back 
mounting  bolt 

Bolt  9 o’clock  back 
correctly  loosened 

No 

Yes 

4 

Move 

Motor 

5 

Push  front  of  motor  on  9 
o’clock  side 

Motor  9 o’clock  front 
correctly  pushed 

No 

Yes 

5 

Tighten 
bolt  3f 

6 

Tighten  3 o’clock  front 
mounting  bolt 

Bolt  3 o’clock  front 
correctly  Tightened 

No 

Yes 

5 

Tighten 
bolt  9f 

6 

Tighten  9 o’clock  front 
mounting  bolt 

Bolt  9 o’clock  front 
correctly  Tightened 

No 

Yes 

5 

Tighten 
bolt  9b 

6 

Tighten  9 o’clock  back 
mounting  bolt 

Bolt  9 o’clock  back 
correctly  Tightened 

No 

Yes 

6 

Mount 

dial 

indicator 

7 

Mount  dial  indicator  on 
front  motor  bolt  on  9 
o’clock  side 

Dial  indicator  correctly 
mounted  on  front  motor 
bolt  on  9 o’clock  side 

Yes 

No 

7 

Zero  dal 
indicator 

8 

Zero  the  dial  indicator  on 
front  motor  bolt  on  9 
o’clock  side 

Dial  indicator  correctly 
zeroed  on  front  motor  bolt 
on  9 o’clock  side 

No 

No 

Table  2:  Facts  for  a Sequential  Activity 


Related  Research 

There  are  several  research  areas  that  should  continue  to  have  a positive  impact  on  our  ability  to  add  intelligence 
to  3D  practice  environments  to  make  them  more  effective  training  tools.  3D  worlds  offer  a rich  environment 
for  visualization  and  expression  of  ideas.  Researchers  have  already  begun  to  populate  and  inhabit  3D  worlds  so 
they  can  gain  knowledge  about  the  unique  challenges  and  potential  of  3D  learning  environments.  The 
interaction  between  man  and  machine  is  often  viewed  as  a dialog.  An  effective  practice  environment  needs  to 
be  able  to  participate  in  a meaningful  dialog  with  the  user(s)  in  a problem  solving  session.  Two  important  areas 
of  research  aimed  at  improving  the  ability  for  man  and  machine  to  communicate  are  mixed-initiative 
interactions  [Hearst  1999]  and  intelligent  dialog  systems  [McRoy,  Ali,  Restificar,  and  Songsak  1999]. 
Researchers  in  these  areas  are  developing  new  techniques  to  improve  communications  between  man  and 
machine  through  more  meaningful  and  flexible  interaction  strategies.  Collaboration  among  real  and  simulated 
users  in  multi-user  worlds  is  another  hot  research  area.  Pedagogical  agents  [Lester,  Towns,  and  FitzGerald 
1999]  are  being  created  to  act  as  teachers  and  tutors  in  future  3-D  learning  environments.  Other  researchers  are 
simulating  the  actions  of  peers  for  team  problem  solving  activities.  Dynamic,  real-time  explanations  [Callaway, 
Daniel,  and  Lester  1999]  should  increase  learning  effectiveness  for  future  generations  of  practice  environments. 

Summary 

This  project  is  an  on-going  research  activity.  We  have  not  conducted  any  large-scale  user  evaluations  or  trials. 
However,  we  are  able  to  provide  the  following  assessment  of  the  current  state  of  the  technologies  used  for  our 
experiments.  The  mechanical  skills  training  example  demonstrates  the  integration  of  existing  technologies  with 
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a minimum  amount  of  software  development.  With  the  objective  of  making  3-D  practice  environments  an 

affordable  training  solution,  reuse  was  a prime  consideration  in  the  design  of  the  implementation. 

1.  The  Java  applet  encapsulating  the  Jess  expert  system  is  reusable  and  is  not  training  content  dependent. 

2.  The  Java  EAI  event  handler  applet  is  reusable.  A table  linking  the  Jess  fact  base  with  the  VRML  nodes  and 
sensors  in  the  practice  environment  is  content  specific  and  must  be  created  for  each  practice  session. 

3.  The  rules  developed  for  the  sequential  activity  knowledge  base  are  totally  reusable.  These  rules  provide  the 
logic  for  managing  the  user  activities  and  VRML  objects  for  a sequential  procedure  practice  environment. 
The  rules  also  provide  the  logic  needed  to  determine  the  advice  and  feedback  requested  by  the  user  with  the 
Jess  applet  interface  buttons. 

4.  The  fact  structure  for  the  knowledge  base  is  reusable  for  other  sequential  procedure  practice  environments. 
The  facts  are  content  specific  and  must  be  created  for  each  practice  session. 

5.  Reusing  3-D  Computer  Aided  Design  (CAD)  objects  adds  realism  to  the  practice  environments.  Reuse  is 
never  easy.  The  original  CAD  model  designer’s  objectives  are  very  different  than  those  of  a training 
practice  environment  designer.  Inevitably,  the  objects  must  be  edited.  A good  approach  is  to  create  a 
hierarchical  scene  with  each  object  of  interest  in  a separate  VRML  file.  The  3-D  scene  is  created  as  a 
composite  of  all  of  the  individual  VRML  objects.  The  user's  interactions  are  easier  to  monitor  and  analyze 
with  a hierarchical  scene  composition 
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Abstract.  This  project  is  an  extension  of  ongoing  efforts  to  develop  Immersive  Virtual 
Environments  (IVEs)  for  education  at  North  Dakota  State  University.  This  IVE  recreates 
a Native  American  village  and  the  adjacent  fur  trading  post  as  they  existed  when  they 
were  inhabited  in  1854  and  also  as  the  area  appeared  while  being  excavated  by 
archeologists  in  1954.  One  version  will  be  used  by  archeology  students  who  will  learn 
the  methods  and  reasoning  used  by  archeologists;  another  version  will  be  used  by 
creative  writing  students  at  NDSU  and  at  Fort  Berthold  Community  College  as  they  write 
historical  fiction  about  the  area.  This  presentation  will  focus  mostly  on  the  version  being 
developed  for  use  by  creative  writing  students. 


For  the  past  several  years,  the  NDSU  World  Wide  Web  Committee  has  been  engaged  in  the 
development  of  Immersive  Virtual  Environments  (IVEs)  for  education.  These  virtual  environments,  which 
focus  on  science  education,  especially  understanding  scientific  method  and  reasoning,  include  The  Geology 
Explorer  and  The  Virtual  Cell  (Slator  et  al.,  1999).  A new  development  of  these  efforts  is  an  IVE  that  will 
be  used  by  archeology  students  and  creative  writing  students;  information  about  this  project  is  available  at 
http://vcell.ndsu.nodak.edu/fish-hook/.  One  version  will  have  a 3-D  fly -through,  graphically  rich  interface 
and  a teaching  system  for  archeology.  Much  like  the  other  IVEs  created  at  NDSU,  archeology  students  will 
use  this  version  to  learn  scientific  methods  and  reasoning  by  doing  what  an  archeologist  would  do  in  the 
simulated  environment.  The  other  version,  which  shares  the  same  basic  content,  is  a largely  text -based 
MOO;  this  version  will  be  used  by  creative  writing  students  at  both  NDSU  and  at  the  Fort  Berthold 
Community  College  in  New  Town,  North  Dakota,  who  will  write  historical  fiction  based  on  their 
interactions  with  it.  In  both  cases,  the  place  is  an  actual  Native  American  village.  Like -a -Fishhook,  and  the 
adjacent  fur  trading  post.  Fort  Berthold,  an  area  in  western  North  Dakota  that  was  inhabited  during  the 
1800s.  Located  on  the  Fort  Berthold  Reservation,  home  of  the  Three  Affiliate  Tribes  (MHA  Nation)^  this 
area  will  be  represented  as  it  existed  in  1954,  during  the  excavation  process,  and  in  1854,  when  it  was 
inhabited  it  was  inhabited  by  Mandans,  Hidatasas,  and  various  European  Americans  associated  with  the  fur 
trade.  Visitors  will  be  able  to  "time  travel"  between  the  two  time  periods. 

Like-a-Fishhook  village  was  established  by  Mandans  and  Hidatsas  around  1844.  These  two  tribes 
had  previously  lived  in  separate  villages,  but  decimated  by  a smallpox  epidemic  in  1837,  the  survivors 
banded  together  to  build  a new  village  of  earth  lodges  on  the  Missouri  River  upstream  from  their  old 
villages.  In  addition  to  hunting  and  gathering,  the  Mandans  and  Hidatsas  cultivated  com,  beans,  squash, 
and  tobacco  along  the  river  bottoms,  and  they  continued  their  traditional  trading  with  other  tribes  and  with 
Europeans  in  the  area.  A fur  trading  company  soon  established  Fort  Berthhold,  and  fur  trade  was  important 
to  the  economy  of  the  area  for  the  length  of  its  existence.  The  Arikaras  joined  the  Mandans  and  Hidatsas  in 
1862.  In  the  mid -1880s,  the  United  States  government  pressured  the  residents  of  Like-a-Fishhook  village 
to  relocate  to  individual  allotments  on  the  reservation,  and  the  village  was  abandoned.  In  1947,  despite 
protests  by  the  Three  Affiliated  Tribes,  construction  of  Garrison  Dam  began.  Between  1950  and  1954, 
salvage  archeological  excavations  were  conducted  by  the  North  Dakota  State  Historical  Society  under 
contracts  with  the  National  Park  Service  and  by  the  River  Basin  Surveys  of  the  Smithsonian  Institution.  By 
late  summer  of  1954,  the  rising  waters  of  Lake  Sakakawea  flooded  the  site  of  Life-a-Fishhook  village  as 
well  as  the  best  farmland  on  the  reservation  (Smith,  1 972).  The  Like  -a-Fishhook  village  and  Fort  Berthold 
area  was  chosen  for  this  project  for  several  reasons.  In  the  1800s,  this  village,  the  last  earth-lodge 
settlement  on  the  northern  plains,  was  occupied  by  three  separate  and  distinct  tribes  that  maintained  their 
cultural  identities.  In  addition,  European  Americans  and  people  of  mixed  blood  were  associated  with  the 
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village  since  shortly  after  it  was  first  established.  In  the  1900s,  the  building  of  Garrison  Dam,  the  struggle 
by  archeologists  to  excavate  the  area  and  save  the  cultural  record  before  the  area  was  flooded,  and  the 
inundation  of  this  site  and  other  tribal  lands  were  important  events  for  the  Three  Affiliated  Tribes  and  for 
the  state  as  a whole.  The  history  of  the  area  will  provide  a rich  source  of  material  for  writers. 

Although  historical  fiction  should  accurately  depict  other  times  and  places,  it  frequently  does  not. 
The  Like-a-Fishhook/Fort  Berthold  MOO  will  serve  as  a text  that  provides  accurate,  detailed  information 
about  this  place.  Visiting  the  MOO  will  simulate  a field  trip  to  a location  and  to  times  that  would  not 
otherwise  be  possible.  The  MOO  interface,  which  presents  authentic  details  through  textual  descriptions, 
interactions,  and  a limited  number  of  carefully  chosen  images,  is  designed  to  stimulate  the  imagination  of 
the  students.  As  Ann  Bertoff  points  out,  “...looking,  seeing,  turns  on  the  mind.... students  of  composition 
and  creative  writing  who  do  a lot  of  looking  will  learn  that  perspective  and  context  are  essential  to 
interpretation.  In  short,  they  will  learn  habits  of  mind  essential  to  critical  and  creative  thinking”  (Bertoff, 
1981,  p.  37). 

The  connection  between  print  fiction  and  recreational  MOOs  and  MUDs  is  interesting.  Some 
role-playing,  recreational  MUDs  have  been  inspired  by  works  of  fiction,  especially  by  those  of  J.R.R. 
Tolkien,  Anne  McCaffrey,  and  Frank  Herbert;  on  the  other  hand,  participation  in  MUDs  inspires  some 
mudders  to  write  conventional  albeit  not  particularly  meritorious  fiction  based  on  the  authors’  experiences 
in  MUDs.  Just  as  many  students  are  motivated  to  learn  science  by  environments  that  incorporate  game- 
like  strategies,  we  believe  that  students  will  be  creatively  inspired  by  immersion  in  a MOO  environment. 
Because  a MOO  is  a social  environment  and  because  this  MOO  will  be  visited  by  students  from  varied 
ethnic  backgrounds,  including  descendants  of  the  inhabitants  of  the  village,  this  experience  will  introduce 
them  to  the  challenges  and  concerns  of  depicting  a multicultural  society.  Further,  visitors  who  interact  with 
each  other  as  well  as  with  various  bots  in  the  MOO  will  experience  a sort  of  virtual  improvisation  that  will 
help  them  to  develop  characters  and  to  write  dialogue.  Creative  writing  students  will  have  opportunities  to 
write  short  narratives  or  poems  about  objects  in  the  MOO;  after  a review  process,  excellent  examples  of 
these  short  writings  will  be  attached  to  the  respective  objects  in  the  MOO  via  an  object  called  a “story 
point.”  These  story  points  can  be  read  by  other  visitors  to  the  MOO.  In  this  way,  writing  created  as  a result 
of  interaction  with  the  MOO,  some  of  which  will  be  displayed  in  the  MOO  and  some  of  which  will  exist  as 
conventional  print  media,  will  not  only  be  creatively  enhanced  by  the  precision  of  the  setting,  it  will  also 
flow  from  the  students'  immersion  in  virtual  time  and  space  and  their  struggles  with  the  cultural  issues  they 
find  there. 

The  creation  of  this  MOO,  which  represents  a real  space  in  two  historical  time  periods,  presents 
several  challenges.  Extensive  historical,  ethnographic,  and  archeological  records  were  used  to  create  the 
virtual  depiction.  However,  how  much  prior  knowledge  of  the  subject  potential  users  might  have  as  well  as 
cultural  perspectives  of  language,  time,  and  space  must  be  considered.  The  number  and  type  of  visual 
representations  to  include  in  a largely  text -based  MOO  is  also  a consideration.  The  engagement  of  the 
students  who  visit  the  MOO  and  then  write  about  it  will  be  determined  by  the  use  of  surveys  and 
interviews.  The  narratives  they  write  as  a result  of  participation  in  the  MOO  will  be  assessed  for  accuracy 
and  creativity.  Students'  understanding  of  the  concerns  related  to  writing  historical  fiction  and  their  ability 
to  explain  the  decisions  they  made  when  writing  historical  narratives  will  be  judged  by  reflective  essays 
they  will  write  after  completing  the  process. 
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Abstract;  This  communication  presents  an  educational  and  collaborative  forum  : DIACOM 
(Interactive  Case  Based  Discussions  for  Medical  Training).  DIACOM  offers  a distance, 
continuous  training  activity,  dedicated  to  physicians.  Its  principle  is  based  on  the  sharing, 
confrontation  and  discussion  about  clinical  cases.  It  can  be  considered  as  a more  structured  and 
collaborative  approach  of  the  discussion  forums  already  in  existence  on  the  Internet.  Its  first 
function  is  to  allow  physicians  to  describe  cases.  It  permit  also  to  put  them  in  touch  with  other 
practitioners,  whose  cases  share  common  subject  and  important  point  of  view  differences.  This 
operation  is  performed  by  a pairing  process  between  described  cases.  This  paper  aims  to  introduce 
a description  of  the  DIACOM  interface  module  that  makes  collaboration  and  interactions  between 
peers,  effective  and  formative. 


Introduction 

Internet  discussion  forums  constitute  a means  of  communication  used  to  enable  the  sharing  of  experiences,  in 
distance  education.  In  medicine,  they  are  particularly  interesting.  But  the  themes  discussed  on  these  forums  are  of  a 
general,  theoretical  or  technical  nature  and  rarely  focused  on  the  practical  content  of  courses  (Peraya  99). 
Moreover,  many  users  prefer  to  add  their  own  contribution  rather  than  to  try  to  make  contact  with  someone  who 
has  previously  related  a similar  experience. 

DIACOM  forum  (Interactive  Case  Based  Discussions  for  Medical  Training)  offers  a distance,  continuous  training 
activity,  dedicated  to  physicians.  Its  principle  is  based  on  the  sharing,  confrontation  and  discussion  about  clinical 
cases.  In  fact,  its  main  function  is  to  allow  physicians  to  describe  cases.  It  permit  also  to  put  them  in  touch  with 
other  practitioners,  whose  cases  share  common  subject  and  important  point  of  view  differences.  Thus,  the 
DIACOM  forum  can  be  considered  as  a more  structured  and  collaborative  approach  to  the  discussion  forums.  This 
operation  is  performed  by  a pairing  process  between  described  cases. 

The  DIACOM  forum  structure  is  composed  of  three  modules.  These  modules  are  described  in  th  second  section. 
Before,  in  the  first  part  of  this  paper,  we  present  the  medical  context  of  this  research  work.  Then  we  conclude 
throuch  the  interface  module,  by  describing  the  collaborative  sight  of  the  forum. 
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A Medical  Context 


In  medicine,  learners  have  frequently  to  study  practical  experiences,  called  clinical  cases.  These  cases  studied 
individually,  or  by  groups,  stem  from  real  practices  and  described  by  physicians  concerning  patients.  This  kinf  of 
learning  is  also  called  Problem  Based  Learning  (PBL)  (Barrows,  Tamblyn,  80).  In  the  framework  of  the 
Continuing  Medical  Education  (CME),  the  case  based  learning  approach  is  also  performed.  Internet  has  an 
abundance  of  websites  that  offer  interactive  clinical  cases  intended  for  physicians  to  train  themselves  (Interact  - 
http).  However,  the  more  experienced  a physician  is,  the  less  he  seems  to  be  content  with  a simple  study  of  clinical 
cases.  In  fact,  he  would  prefer  to  discuss  about  these  cases  with  his  peers.  This  is  why,  aware  of  this  need  of 
collaboration  between  practitioners  on  training  tasks,  continuing  education  associations  still  organise  regular 
meetings.  During  theses  meetings,  real  cases  are  raised  and  discussed  by  participants.  Firstly,  these  meetings  allow 
practitioners  to  improve  their  own  knowledge  on  a subject.  Secondly,  these  discussions  enable  also  to  test 
physicians  skills  and  to  discuss  about  diagnosis  or  therapeutic  management  protocols  that  occur  in  their  daily 
practice.  Nevertheless,  practitioners  have  less  time  at  their  disposal  to  actively  and  regularly  participate  to  these 
meetings.  Therefore,  it  could  be  interesting  to  allow  physicians  share  and  discuss  their  views,  through  the 
development  of  a distance  learning  system  based  on  individual  experiences. 

Some  kind  of  existing  systems  have  a similar  objective.  These  are  Computer  Supported  Collaborative  Learning 
systems  (CSCL)  (Dillenbourg  and  al.,  96).  In  fact,  their  general  functioning  principle  consist  in  making  the 
learners  learning  in  small  groups,  solving  common  problems  and  interacting  each  others.  In  the  medical  domain, 
constructing  CSCL  environments  is  very  interesting  for  PBL  (Koschman  and  al.,  95)  (Mendelez  and  al.,  99).  Some 
CSCL  works  aims  at  perpetuate  the  training,  mainly  by  the  way  of  discussions  without  considering  any  problem- 
solving angle  for  the  system.  For  example,  CONNECT  (Baker  and  al.,  99)  concerns  epistemic  interactions  between 
learners  in  a CSCL  environment,  dedicated  to  sciences  learning.  In  CONNECT,  the  system  analyse  learners 
contributions  and,  thus,  evaluate  their  similarities,  in  a textual  analysis.  The  DIACOM  principle  is  similar  from 
the  CONNECT  paradigm.  In  fact,  the  learning  method  of  DIACOM  is  also  based  on  interactions  between  peers. 
Nevertheless,  DIACOM  is  dedicated  to  medical  training  and  doesn’t  focalise  on  textual  contribution  but  on 
structured  clinical  cases.  Moreover,  CONNECT  is  designed  for  children  education  organized  by  a teacher. 
DIACOM  is  then  designed  for  CME.  Physicians  have  to  learn  from  each  other,  by  confronting  their  knowledge  and 
their  approaches  of  a problem.  So,  they  train  themselves,  and  above  all  autonomously,  without  any  tutor  or 
moderator. 

The  DIACOM  approach  is  especially  relevant  when  it  comes  to  training  practitioners  in  new  medical  practices  or 
in  domains,  for  which  there  is  no  established  guidelines.  So,  as  to  raise  discussions  of  clinical  cases  it  is  necessary 
to  attribute  to  the  forum,  a specific  medical  topic.  It  is  preferable  for  the  forum  to  belong  to  the  domains  of  medical 
research,  and  of  technical,  diagnosis  or  therapeutic  innovations.  In  fact,  in  this  kind  of  domain,  physicians  have 
their  individual  methods  of  practice.  Consequently  they  could  express  their  opinions  and  demonstrate  a willingness 
to  engage  in  discussions  and  to  participate  in  training  about  the  subject.  So,  the  pairings  and  debates  resulting  from 
these  topics  become  all  the  more  interesting,  efficient  and  valuable  for  the  learners.  The  pediatric  pain 
management  domain  corresponds  to  these  criteria.  As  patients  in  this  field  are  very  young,  consultations  or  clinical 
examinations  may  result  in  difficulties  and  complications  that  are  uncommon  in  other  medical  fields.  Moreover, 
the  consensual  guidelines  for  pain  management  are  still  quite  rare  in  the  pediatrics  domain.  Some  reference  books 
composed  by  scientific  papers,  show  how  much  the  pediatrician  community  is  interested  in  the  subject  (Ecoffey  and 
Murat,  99).  Thus,  the  cases  described  in  the  forum  can  illustrate  different  solutions  and  strategies  in  solving 
problems.  These  are  mainly  based  on  a pediatrician’s  individual  experiences  of  child  pain  management. 

The  following  section  deals  with  the  three  modules  that  make  up  DIACOM  forum  architecture. 


DIACOM  Architecture 

DIACOM  forum  structure  is  composed  of  three  modules  : the  data  module  that  aims  at  making  a list  of  various 
data’s  models  used  by  the  system  ; the  pairing  module  which  is  charged  of  the  processing  realized  by  the  system  to 
manage  pairing  between  cases  ; and,  the  interface  module  that  manages  interactions  between  users  and  the  system. 
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The  data  module 


DIACOM  forum  uses  a lot  of  domain  dependant  knowledge  and  data,  in  order  to  help  authors  during  their  case 
description.  These  data  aims  also  at  peforming  the  pairing  comparison  of  cases.  These  data  are  managed  by  the 
data  module  of  the  DIACOM  forum.  This  module  is  composed  of  a generic  model,  a specific  model  and  a 
generated  model. 

Firstly,  the  generic  model  allows  to  generate  a modelization  of  the  case  structure.  In  DIACOM  forum,  a clinical 
case  is  considered  as  a sequence  of  steps  named  scenes.  Each  of  theses  scenes  represent  a collection  of  elementary 
concepts  called  entities  and  describing  the  patient  features.  In  order  to  get  through  to  the  following  scene,  a scene 
must  contain  a conclusive  action.  This  action  represents  the  decision  he  has  taken  or  he  would  take,  in  front  of  this 
situation.  It  contains  also  a modeling  of  the  solving  problem  strategies  described  by  physicians  in  cases. 
Consequently,  the  generic  model  is  composed  of  a modeling  of  cases,  of  entites  and  action,  and  of  entities  and 
action  types.  It  is  independent  from  the  pediatric  pain  management  domaine. 

Secondly,  the  specific  model  contains  all  the  domain  dependant  data  and  makes  up  the  generic  model  instanciation 
in  the  pediatric  pain  management  domain.  With  this  specific  model,  we  collect  all  the  entities  and  action  types, 
defined  for  the  pediatric  pain  management.  This  model  contains  also  the  complete  corpus  of  cases  described  by 
physicians  on  the  domain,  and  the  solving  problem  strategies  associated  to  each  case. 

The  generated  model  aims  at  storing  information  issued  from  the  pairing.  In  this  way,  this  model  maintains  a case 
categorization,  and  the  whole  information  about  paired  cases.  It  is  charged  of  and  also  the  storage  of  discussions 
resulting  from  the  pairing  and  processed  by  physicians. 


The  Pairing  Module 

The  pairing  module  aims  at  managing  the  processing  realized  on  cases  in  sort  to  constitute  relevant  couples  of 
cases  for  following  discusions.  These  couples  of  cases  are  those  which  present  similarities  in  accordance  with  sure 
criteria  but  mostly  important  differences  in  accordance  with  other  ones. 

Once  a physician  enters  a new  case,  the  underlying  system  performs  a pairing  process  based  on  three  domain 
dependent  criteria.  The  first  criterion  named  the  theme  represents  the  pathology  illustrated  in  the  case.  The  second 
criterion  is  called  the  object  of  the  case  and  identify  the  case  objective  next  to  the  pain  (evaluating  pain  or  treating 
pain).  At  last,  the  third  criterion  is  the  solving  problem  strategy  proposed  in  the  case  by  the  physician.  The  aim 
here  is  to  introduce  people  who  deal  with  the  same  problems  but  who  have  different  approaches  in  solving  them. 
So,  as  well  as  being  paired  through  their  similarities,  the  cases  are  also  identified  by  the  differences  in  opinions 
expressed  by  the  physicians.  The  importance  in  identifying  these  differences  lies  in  the  fact  that,  if  people  are  in 
accordance  with  each  other  they  will  not  benefit  from  the  forum  and  as  a consequence  the  discussion  becomes 
useless.  The  principle  of  favoring  different  opinions  is  presented  in  (Dillenbourg  and  Schneider,  95).  It  is 
cobnsidered  as  a fundamental  element  in  generating  beneficial  discussions  in  a collaborative  learning  group. 
Consequently,  a socio-cognitive  conflict  (Doise  and  Mugny,  84)  could  arise  and  provide  discussions  and  debates 
that  are  richer  and  more  beneficial  in  their  content. 

This  pairing  module  is  composed  by  a general  algorithm  that  manages  progress  of  the  processing  chain.  This 
algorithm  executes  mainly  two  sub-algorithms.  The  first  one  is  named  criteria  extracting  algorithm.  It  allows  to 
extract  the  pairing  criteria  from  cases  (the  theme,  the  object  and  the  strategy).  These  criteria  permit  to  put  the  cases 
together  by  categories  in  the  generated  model  category  level.  Next,  the  second  sub-algorithm  is  the  pairing 
algorithm.  This  algorithm  makes  use  of  the  last  extracted  criteria  in  order  to  compare  them  with  those  of  other 
cases  already  stored  in  the  category  level.  Thus,  the  collection  of  cases  that  deal  with  the  same  object  is  then 
extracted.  The  pairing  module  select  cases  with  a similar  theme  but  a distant  strategy,  then  those  with  a distant 
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theme  but  a similar  strategy,  and  finally  those  with  distant  theme  and  strategy.  This  retrieval  of  relevant  cases 
generate  three  types  of  paired  couples.  The  pairing  can  be  considered  as  an  uprising  of  an  identified  problem  by 
analyzing  the  two  cases.  Once  relevant  case  have  been  extracted,  the  information  concerning  the  pairing  and  the 
reasons  of  these  pairing  is  stored  in  the  generated  model  pairing  level.  This  information  is  then  transmitted  to 
cases’  authors  through  the  interface  module.  Discussions  can  now  take  place.  In  a pedagogic  point  of  view,  the 
discussion  is  benefic,  because  physicians  learn  from  the  cases  of  each  of  them,  but  also  from  their  own 
explanation’s  tentative  (self-explanation  effect  of  Chi  (Chi  and  al.  98)). 


The  Interface  Module 

The  interface  module  proposes  two  user  interface.  The  author  interface,  DIACOM-IA  (DIACOM  Interface  for 
Authors)  allows  a physician  to  simply  describe  his  experience.  This  interface  can  be  considered  as  an  author 
interface  because  it  aims  at  « allowing  authors,  non  programmers,  to  describe  and  to  make  visible  their 
knowledge  » (Murray,  99).  By  the  way,  an  author,  must  be  able  to  apprehend  the  description  of  a new  case  in  the 
base,  in  accordance  with  a specific  process.  The  current  author  interface  version,  is  developed  in  Java.  Moreover, 
because  a case  contains  several  composition  levels,  we  have  chosen  to  present  it  to  the  user,  in  form  of  an 
« arborescence  ».  At  each  level  of  this  arborescence,  the  user  has  got  the  possibility  to  develop  the  lower  level  with 
a mouse  click.  At  the  top  of  the  tree,  we  can  see  the  name  of  the  case  accompanied  with  a small  description.  At  the 
lower  level,  we  can  see  the  ordered  scene  list  making  up  the  case.  Under  each  scene  appear  entities  names,  and  this 
of  the  action  that  allow  to  go  through  the  next  scene.  In  another  way,  to  manage  the  user  acts  we  have  chosen  to 
offer  contextual  menus  that  appear  with  a right  mouse  click.  On  each  hierarchy  level,  the  presented  menu  is  then 
different.  For  example,  at  the  case  level,  the  contextual  menu  proposes  to  modify  information  about  the  case,  to  add 
a new  scene  to  this  case,  or  to  delete  the  last  scene  of  the  case.  Nevertheless,  at  the  scene  level,  it  enable  to  add  a 
new  entity  or  a new  action  in  the  clicked  scene. 

The  discussion  interface,  DIACOM-ID  (DIACOM  Interface  for  Discussion),  constitutes  the  part  of  the  interface 
module  that  enables  the  management  of  discussions  between  learners.  Obviously,  discussions  proposed  on  the 
forum  are  asynchronous.  In  fact,  the  aim  of  the  DIACOM  forum  is  to  propose  to  physicians  a tool,  easy  to  use  and 
without  any  time  and  place  constraint.  Moreover,  discussions  have  to  be  serious  and  authorized  only  to  regular 
physicians.  Consequently,  the  access  to  the  DACOM-ID  web  site  content  begins  with  an  identification  phase  in 
order  to  authorize  discussions  and  case  base  consultation  only  to  previous  registered  physicians.  In  the  first  step,  an 
authorized  physician  access  to  a customized  page  in  which  he  can  inquire  pairing  about  his  own  cases.  Then  he 
can  link  to  the  paired  case  description  or  directly  to  the  related  discussion  page,  corresponding  to  a pairing.  When 
the  physician  click  on  one  of  these  discussions,  the  totality  of  the  discussion  is  displayed.  On  another  way,  the 
customized  page  give  him  an  access  to  the  case  page  that  present  the  whole  cases  stored  in  the  system,  and  to  the 
discussion  page,  that  allows  to  link  to  the  whole  current  discussions  of  the  DIACOM  forum. 

These  two  interfaces  are  at  the  center  of  the  collaboration  and  interaction  management  of  the  DIACOM  system. 


Collaborative  Learning  Through  the  Interface  Module 

The  DIACOM  forum  interface  module  is  a part  of  the  system  through  which  collaborative  pedagogical  interactions 
can  be  effectives. 

With  the  DIACOM-IA  interface,  a physician  begins  to  formulate,  formalize  and  structure  his  expertise.  He  take  the 
first  plunge  in  order  to  integrate  a real  group  discussion,  efficient  for  his  continuing  education  and  knowledge 
improvement.  In  fact,  each  collaborative  learning  could  only  be  efficient  if  each  participant  has  performed  a 
preliminary  individual  reflection  about  the  subject  that  must  be  discussed  and  debated.  It  is  the  only  way  to  obtain 
interactions  and  an  exchanges  of  each  individual  thought  (D’Halluin,  01). 

Moreover,  DICAOM-ID  aims  at  stimulate  these  interactions  between  peers  by  presenting  paired  cases  to  physicians 
authors  of  clinical  cases.  For  the  moment,  it  is  for  example  the  matter  of  saying  to  the  practitioner:  “Your  case  has 
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been  paired  wit  Dr  X*s  case.  His  case  deals  with  similar  subject  as  yours,  but  it  doesn’t  present  the  same  problem 
solving  approach.  You  have  a real  interest  to  interact  with  him  through  a discussion.  We  encourage  you  to  begin 
the  discussion  with  him,  starting  with  a presentation  of  your  opinion  about  his  own  case”.  Couples  of  paired  cases 
generates,  on  their  own,  a discussion  subject  on  which  two  learners  doesn’t  agree.  So  make  these  persons 
interacting,  through  asynchronous  point  of  views  debates  on  the  subject,  is  beneficial  to  the  pedagogy  in  the 
system.  Two  persons  is  the  minimal  group  number  required  in  order  to  obtain  beneficial  interaction  in  terms  of 
knowledge  transfer  and  enriching.  This  principle  of  creating  couple  of  cooperative  learners  is  near  from  the  dyads 
constitution  of  Quignard  and  Baker  in  (Quignard  and  Baker  99). 

Obviously,  we  have  seen  in  the  DIACOM-ID  description  that  the  discussion  access  is  not  saved  for  paired  case 
authors.  In  fact,  it’s  turned  out  that  the  discussions  activated  by  pairing  have  interest  for  each  physician  became 
aware  of  the  question  raised.  Thus,  the  discussion  interface  allows  each  participant  to  participate  to  the  whole 
discussions  currently  developed  in  the  forum.  By  the  way,  a physician  have  not  to  submit  a case  to  interact  with 
other  physicians  about  interesting  subjects.  It  could  be  noticed  that,  for  a people  who  has  not  submitted  any  case, 
the  access  to  discussions  is  not  easier  than  in  a traditional  discussion  forum.  Even  if  group  discussions  are  possible, 
the  choice  of  these  extern  participants  is  not  funded  on  a point  of  view  difference  as  it  is  done  when  people  are 
connected  according  to  the  pairing  process.  Moreover,  these  participants,  didn’t  previously  have  the  possibility  to 
describe  a clinical  case  based  on  the  current  discussion  subject.  So,  they  didn’t  make  the  thought  process  to 
structure  and  formalize  their  expertise  on  this  specific  subject.  But,  interactions  generated  by  these  external  people 
can  be  a plus  for  the  whole  collaborative  learning  group  then  constituted.  In  fact,  they  can  re- incite  and  re-motivate 
interactions  by  introducing  new  questions  about  the  discussion  subject. 

The  following  paragraph  conclude  on  the  actual  state  and  on  the  evolution  perspectives  of  this  research  work. 


Conclusion 

A first  experimentation  phase  has  been  carried  out  on  the  pediatric  pain  management.  These  experiences  have 
allowed  to  collect  an  experimental  corpus  in  order  to  constitute  the  specific  model  of  the  forum.  Our  current  works 
are  firstly  centered  on  the  strategies  modelization  and  abstractions,  from  the  experimental  corpus.  Once  the  author 
interface  would  have  been  validated  by  experts,  a second  experimental  corpus  will  be  able  to  be  collected.  This 
corpus  will  support  our  conclusions  on  the  strategies  modelization  and  by  extension  the  pairing  module.  A 
complete  prototype  of  the  DIACOM  forum  will  then  be  finalized.  This  prototype  will  allow  us  to  test  and  validate 
the  complementarity  of  the  two  DIACOM  interface  in  the  process  of  collaborative  learning,  based  on  interactions 
between  peers. 
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Abstract:  RCOS.java  is  a simulated,  web-based  operating  system  that  provides  a 

platform  on  which  to  build  constructivist -based  approaches  to  teaching  operating 
systems.  This  p^er  briefly  describes  the  design,  capabilities  and  availability  of 
RCOS.java 


Introduction 


A course  covering  the  algorithms,  concepts  and  theories  behind  the  design,  construction  and 
operation  of  operating  systems  is  a recommended  core  component  for  courses  in  computer  science 
(Denning,  1989).  For  a number  of  reasons  students  find  operating  systems  related  topics,  like  many 
computer  science  topics,  difficult  to  learn.  In  an  attempt  to  address  these  problems  and  based  on  previous 
experience  RCOS.java  has  been  developed.  RCOS.java  aims  to  provide  a platform  on  which  students  can 
actively  experiment  with  the  algorithms,  data  structures  and  services  of  an  operating  system.  RCOS.java 
combines  the  capabilities  of  Java  and  the  World-Wide  Web  to  implement  a single  system  that  includes 
features  of  previous  operating  systems  courseware  including  animation,  concurrency  simulation  and  toy 
operating  systems. 


RCOS.java  Components 

RCOS.java  acts  as  a simulated  operating  system  within  a Web  browser.  Students  can  drill  down 
and  peer  underneath  the  user  interface  and  interact  with  graphical  animations  of  the  data  structures  and 
algorithms  used  to  implement  RCOS.java's  simulated  operating  system  and  hardware.  RCOS.java  is 
implemented  as  a Java  application  consisting  of  250  classes  and  over  100,000  lines  of  source  code  broken 
up  into  the  following  five  major  categories:  hardware,  operating  system,  animators,  message  passing  and 
compiler. 

The  lowest  levels  of  RCOS.java  are  a collection  simulated  computer  hardware  including  a CPU, 
disk  drives,  RAM  and  terminals.  The  operating  system  is  based  on  a message  passing  micro-kernel 
architecture.  The  micro -kernel  handles  interrupts,  context  switching  and  generates  messages  that  are 
passed  onto  other  operating  system  components  that  perform  higher-level  tasks  such  as  memory 
management,  inter-process  communication,  process  and  disk  scheduling. 

The  Animators  provide  the  visual  representations  of  RCOS.java’s  internal  operations.  The 
Animators  can  also  provide  interface  elements  that  allow  students  to  modify  the  operation  of  RCOS.java  by 
choosing  between  different  parameters  and  algorithms.  The  animator  system  also  includes  a recorder  that 
can  store  a RCOS.java  session  for  subsequent  playback. 

The  entire  RCOS.java  system  is  based  on  a message  based  architecture.  This  architecture  allows 
the  replacement  of  individual  components  including  hardware,  operating  system  and  animator  with  little  or 
no  influence  on  other  components.  The  message  passing  architecture  also  enables  the  inplementation  of 
the  recorder. 
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RCOS.java  will  execute  programs  written  by  students.  Students  write  these  programs  using  a 
programming  language  similar  to  C/C++.  The  RCOS.java  compiler  will  compile  these  programs  into 
RCOS.java  executables.  The  RCOS.java  C/C++  dialect  includes  simple  support  for  inter-process 
communication  primitives  including  semaphores  and  shared  memory. 


Use  of  RCOS.java 

RCOS.java  is  designed  to  enable  and  encourage  active  manipulation  and  experimentation  by 
students.  There  are  four  main  methods,  which  can  be  used  including:  visual  investigation  and 
experimentation,  creating  presentations,  writing  user  programs,  and  modifying  the  internal  operations  of 
RCOS.java. 

The  animators  provide  visual  representation  of  what  is  going  on  under  tfie  RCOS.java  hood. 
Students  can  use  these  to  observe  RCOS.java's  operation  under  different  scenarios  to  gain  a better 
understanding  of  how  the  algorithms  and  data  structures  work.  A number  of  the  animators  also  allow  the 
student  to  modify  the  operation  of  the  underlying  operating  system  and  hardware  further  enabling 
investigation  of  performance  under  alternate  scenarios. 

Using  the  RCOS.java  compiler  students  can  create  and  execute  their  own  RCOS.java  programs. 
The  C/C++  dialect  includes  support  for  services,  such  as  semaphores,  which  students  are  generally  not 
familiar.  By  combining  their  programs  with  the  visual  representation  provided  by  the  animators  students 
can  further  investigate  and  experiment. 

The  recorder  allows  students  (and  teachers)  to  create  stand-alone  RCOS.java  sessions  for  future 
playback.  This  allows  the  creation  of  animations  designed  to  demonstrate  important  concepts. 

Students  can  also  modify  the  internal  operation  of  RCOS.java  by  modifying  its  source  code.  This 
allows  students  to  implement  alternate  algorithms,  data  structures,  animators  and  simulated  hardware.  Due 
to  RCOS.java  being  implemented  with  a message  passing  architecture  it  is  possible  for  students  to 
concentrate  solely  on  the  component  they  are  modifying. 


Current  Status  and  Future  Work 

RCOS.java  has  been  under  development  since  1996.  Developing  a computer-aided  learning 
package  as  complex  as  RCOS.java  has  been  a time  consuming  process  fraught  with  many  difficulties.  The 
long  development  cycle  has  itself  created  a nun±>er  of  problems  including  project  students  graduating, 
changing  teaching  responsibilities,  and  changes  in  the  Java  platform. 

If  the  potential  advantages  of  RCOS.java  are  to  be  realized  integration  into  the  curriculum  is 
necessary.  RCOS.java  is  currently  a usable  tool  and  yet  there  is  still  a significant  development  effort 
required  to  develop  the  appropriate  learning  activities  and  associated  support  material  to  enable  its  use  in  a 
course.  Current  work  on  RCOS.java  is  concentrating  on  this  process. 

RCOS.java  has  been  released  under  an  open  source  license  and  the  system  itself  and  the  associated 
resources  are  available  on  the  web  from  http://www.rcosjava.org  /.  Any  interest  in  the  use  or  development 
of  RCOS.java  is  more  than  welcome. 


Conclusions 

RCOS.java  is  a simulated  operating  system  that  aims  to  enable  the  use  of  constructivist  based 
learning  activities  in  operating  systems  education  by  providing  a concrete  platform  to  enable  visualization 
and  experimentation. 
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Abstract:  A Course  Barometer  is  a method  for  addressing  the  loss  of  informal  feedback 
in  a distance  education  setting.  Originally  proposed  and  used  at  the  University  of 
Trollhattan  Uddevella  this  paper  describes  how  the  idea  of  a course  barometer  has  been 
adopted  by  Central  Queensland  University.  The  paper  suggests  connections  between 
anonymity,  observable  change  and  the  level  of  student  contributions.  Usage  statistics, 
staff  and  student  feedback,  and  stories  are  used  to  identify  trends,  draw  conclusions  and 
make  suggestions  for  further  improvement. 


Introduction 


Collis  and  Oliver  (1999)  report  in  their  analysis  of  the  papers  submitted  to  EdMedia'99  that  the 
majority  of  papers  report  on  prototype  development  and  evaluation  with  few  ideas  going  beyond  this  stage. 
One  of  those  papers  (Svennsson,  Andersson,  Gadd,  Johansson,  1999)  described  the  initial  development  and 
evaluation  of  the  course  barometer.  A course  barometer  is  a web-based  application  designed  to  provide 
students  a simple,  anonymous  method  for  providing  informal  feedback  about  their  feelings  toward  a course 
while  it  is  being  taught.  This  paper  reflects  on  the  use  of  the  course  barometer  idea  within  the  Faculty  of 
Informatics  and  Communication  (Infocom)  at  Central  Queensland  University  (CQU). 

The  absence  of  informal,  in-term  feedback  is  a major  problem  within  Infocom  since  most  courses 
are  taught  with  a large  number  of  staff  via  both  print-based  distance  education  and  in  a face-to-face  mode  at 
13  sites  spread  through  out  Australia,  Fiji  and  South-East  Asia.  The  course  barometer  concept  within 
Infocom  was  initially  intended  for  a course  taught  by  the  author  in  the  second  half  of  1999.  Since  then  the 
potential  and  obvious  need  for  in -term  feedback  from  students  has  led  to  the  course  barometers 
contributions  being  received  in  67  different  course  offerings. 

The  paper  starts  by  offering  a brief  overview  of  the  course  barometer  concept  followed  by  a 
description  of  the  data  sources  used  to  examine  Infocom's  barometer  experience.  Two  stories  are  then 
provided  which  show  how  a course  barometer  has  made  a positive  difference  for  both  staff  and  students. 
The  paper  then  examines  the  connection  between  implementation  choices,  anonymity,  observable  change, 
and  the  level  of  student  contributions  to  the  course  barometers. 


Barometer  Design  and  Implementation 

There  have  been  three  versions  of  the  course  barometer  concept.  Two  versions,  alpha  and  beta, 
have  been  implemented  at  the  University  of  Trollhattan  (Svensson  and  Sorenson,  2001)  and  the  Infocom 
version  described  here.  All  three  barometers  provide  a mechanism  by  which  a student  can  indicate  their 
current  mood  and  provide  a small  textual  comment  about  a course.  All  three  provide  a statistics  page 
where  staff  and  students  can  view  the  atmosphere  of  the  course.  Minor  differences  between  the  barometers 
include  changes  in  the  interface  design,  moods  that  can  be  expressed,  administrative  mechanisms,  and  the 
implementation  environment. 

A more  significant  difference  between  the  Trollhattan  barometers  and  the  Infocom  barometer  is 
the  number  of  times  per  week  students  could  make  contributions.  At  Trollhattan  students  could  make  as 
many  contributions  as  they  wanted,  whenever  they  wanted.  In  an  attempt  to  increase  the  value  placed  on 
barometer  contributions  the  Infocom  barometer  restricted  students  to  one  contribution  per  week  per  course. 
The  intent  was  to  encourage  students  to  be  more  thoughtful  in  their  contributions.  This  limitation  also 
prevents  the  possibility  of  the  barometer  taking  on  an  unbalanced  view  due  to  large  numbers  of  repeat 


contributions  from  a small  number  of  students.  Lowder  and  Hagan  (1999),  writing  about  another  web- 
based  anonymous  feedback  system,  report  on  a student  using  the  ability  to  make  multiple  postings  as  a 
nuisance  technique. 

To  implement  this  restriction  it  was  necessary  to  be  able  to  identify  which  student  was  making  a 
contribution.  This  violates  the  aim  of  providing  an  anonymous  environment  where  students  need  not  fear 
repercussions  for  making  negative  comments.  Recognizing  the  negative  impact  this  may  have  upon  student 
participation  the  irrplementation  of  the  password  protection  system  aimed  to  make  it  as  difficult  as  possible 
to  identify  which  student  made  a specific  contribution.  Additionally,  teaching  staff  are  never  provided  with 
any  access  to  information  connecting  students  to  contributions  and  an  effort  was  made  to  communicate  this 
information  to  students.  A positive  side  effect  of  the  password  requirement  was  to  address  another 
problem.  Infocom  course  websites  generally  are  not  password  protected.  In  the  past  this  has  led  to  non- 
CQU  students  misusing  facilities  that  allow  for  visitor  contributions. 

The  alpha  and  beta  barometers  at  Trollhattan  were  used  over  a four-week  period  in  a single  course 
during  two  different  terms.  There  were  a total  of  213  submissions  for  the  alpha  and  308  for  the  beta 
barometer  (Svennson  and  Sorenson,  2002).  The  Infocom  barometers  have  been  used  for  the  entire  12 
weeks  of  5 different  terms  and  contributions  received  for  67  different  course  offerings.  For  all  but  the  last 
term  the  Infocom  course  barometers  were  only  included  at  the  request  of  the  course  coordinator.  In  the  last 
term  of  2001  the  course  barometer  was  made  a standard  part  of  the  default  web  site  for  all  Infocom  courses. 
In  this  term  there  was  a total  of  around  100  courses  with  default  web  sites.  Barometer  contributions  were 
only  received  in  46  of  these  courses. 


Data  Sources 

Information  about  student  contributions  to  the  Infocom  course  barometer  is  stored  in  two  database 
tables.  One  table  stores  the  course,  week,  feeling  (good,  bad  or  indifferent)  and  any  comments.  The  other 
table  stores  the  student  number,  course,  and  week.  It  is  this  second  table  that  is  used  to  prevent  students 
making  more  than  one  comment  per  course,  per  week.  The  data  in  these  tables  has  been  used  to  generate 
statistics  about  students  and  their  contributions.  The  student  comments  on  the  Trolhattan  alpha-barometer 
underwent  content  analysis.  Due  to  time  and  resource  limitations  such  analysis  has  not  yet  been  performed 
on  the  Infocom  data. 

Logs  from  the  Infocom  website  have  been  archived  since  early  1997.  These  logs  record  the  date, 
time  and  web  page  of  every  visit  to  the  Infocom  website.  The  web  server  logs  have  been  used  to  compare 
the  number  of  visitors  to  course  barometers  with  the  number  of  people  making  contributions. 

To  gather  more  information  about  student  feeling  toward  the  course  barometers  a 15-question 
survey  was  placed  on  the  Web  toward  the  end  of  2001.  The  survey  contained  8 likert  style  questions  and  7 
short  answer  questions.  An  email  message  announcing  the  survey  was  sent  to  all  students  enrolled  in 
Infocom  courses  during  the  second  half  of  the  year.  CQU's  student  record  system  contained  email 
addresses  for  948  of  these  students.  350  of  those  email  addresses  suffered  from  a number  of  problems  that 
prevented  delivery.  A week  after  the  first  email  initial  results  were  released  on  the  web  and  another  email 
sent  informing  students  and  reminding  them  about  the  survey. 

61  responses  to  the  survey  were  received  giving  a response  rate  of  10.2%.  70%  of  the  students 
who  responded  to  the  survey  knew  about  the  course  barometer.  58%  of  respondents  agreed  or  strongly 
agreed  with  the  statement  "The  course  barometer  is  a useful  feature".  While  responses  to  the  survey  are  a 
useful  guide  to  student  feeling  there  is  some  bias  in  the  sample.  The  type  of  students  who  had  correctly 
entered  an  email  address  into  CQU's  student  records  system  and  completed  an  online  survey  during  the 
final  stages  of  a term  are  more  likely  to  make  use  of  an  online  resource  like  the  course  barometers. 

An  email  message  about  the  course  barometers  was  distributed  on  the  Infocom  staff  mailing  list 
early  in  the  second  half  of  2001.  Of  the  60+  course  coordinators  within  Infocom,  many  of  whom  had 
previously  never  used  or  known  of  the  course  barometer  facility,  there  were  6 replies.  In  some  cases  there 
was  on-going  discussion  between  the  author  and  individual  staff 
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Statistics,  Trends  and  Potential  Reasons 


Over  three  years  of  use  1188  student  contributions  were  made  via  the  Infocom  course  barometers. 
The  average  length  of  a comment  on  the  Infocom  barometers  was  1 20  characters.  Over  the  5 terms  of  year 
the  average  number  of  contributions  per  course  offering  was  less  than  1 8.  However,  three  of  these  terms 
had  significantly  higher  levels  of  contributions.  The  three  busy  terms  had  an  average  of  40  contributions 
per  course  offering.  While  the  two  slow  terms  had  an  average  of  7 contributions  per  course  offering.  The 
defining  difference  between  these  two  groups  of  terms  appears  to  have  been  the  motivation  and 
involvement  of  the  course  coordinators  in  encouraging  and  responding  to  barometer  contributions. 

In  the  last  main  term  for  2001  the  course  barometers  were,  for  the  first  time,  a compulsory  part  of 
every  Infocom  course  website.  However,  staff  were  not  required  to  encourage  students  to  contribute  to  the 
course  barometers.  In  this  term  448  contributions  in  46  different  course  offerings  were  made  for  an 
average  of  9.7  per  course.  Only  16  of  the  100+  courses  in  this  term  had  greater  10  contributions. 

29.5%  of  the  Trollhattan  alpha-barometer  submissions  included  comments  (Svennson,  et  al,  1999) 
while  71.7%  of  the  Infocom  submissions  included  comments.  Trollhattan  contributions  showed  overall 
student  feelings  of  57%  bad,  10%  indifferent,  and  33%  good.  The  Infocom  breakdown  was  29%  bad,  20% 
indifferent  and  51%  good  with  the  1188  student  contributions  being  made  by  538  different  students.  The 
most  prolific  student  made  50  contributions  over  6 courses  during  two  terms  in  2001 . 

If  you  accept  that  the  one  comment,  per  week,  per  course  increased  the  effort  placed  into  the 
infocom  barometer  contributions  this  may  help  explain  why  the  number  of  coiUfibutions  with  comments  at 
Infocom  is  almost  the  inverse  of  that  at  Trollhattan,  29.5%  to  71 .7%.  It  may  also  explain  the  switch  in 
reported  feelings  from  33%  good  at  Trollhattan  to  51%  good  within  Infocom.  Especially  given  the 
observation  that  both  the  Trolhattan  and  Infocom  barometers  have  been  used  primarily  in  courses  with  staff 
interested  in  improving  the  students'  experience. 

The  number  of  contributions  per  week  for  almost  all  courses  follow  a common  trend.  There  is  an 
initial  spike  early  in  the  term  when  the  course  barometer  is  first  discovered.  Followed  by  a gradual  drop  off 
until  some  minimum  level  is  found  and  maintained  until  the  last  week  or  so  of  term.  At  this  stage  there  is 
usually  a small  increase  in  contributions  due  to  the  end  of  term.  Any  exceptions  to  this  trend  were  caused 
by  events  specific  to  the  individual  course  that  encouraged  students  to  contribute. 


Two  Stories 

While  the  overall  level  of  contributions  to  the  Infocom  course  barometers  has  been  limited  there 
have  been  a number  of  specific  incidents  where  they  have  played  an  important  role  in  improving  the 
experience  of  both  students  and  staff.  The  first  story  takes  place  during  the  first  term  in  which  the  Infocom 
course  barometers  were  used.  In  the  second  week  of  the  term  a serious  student  complaint  was  made  about 
a relatively  new  staff  member.  The  complaint  was  very  critical  of  the  staff  member's  lecture  style.  The 
week  after  the  complaint  was  raised  the  number  of  comments  on  the  course's  barometer  was  over  four  times 
the  average.  Almost  all  contributions  directly  addressed  the  topic  of  the  initial  complaint  and  all  were  very 
positive.  For  example. 

After  hearing  XXXX  talk  about  the  student  who  pretty  much  is  jeopardizing  his  job,  I feel  really  bad  for 
him.  He  is  a GREAT  lecturer  and  the  way  he  has  summed  up  Programming  A in  just  a matter  of  weeks 
I understand  the  concepts  a lot  better. 

The  second  story  comes  from  the  last  main  term  in  2001  and  demonstrates  how  the  course 
barometer  was  able  to  contribute  toward  an  improvement  in  the  student's  experience.  Many  of  Infocom's 
courses  are  delivered  via  video-conferencing  between  four  regional  campuses.  Such  presentations  are 
usually  made  as  traditional  lectures  originating  from  CQU's  Rockhampton  campus  where  most  course 
coordinators  are  located.  In  2001  a new  lecturer  located  on  the  Bundaberg  campus  was  given  responsibility 
for  a course  that  had  to  be  delivered  by  video-conferencing.  During  the  first  three  weeks  the  course 
barometer  shows  an  increasingly  negative  feeling  from  students  about  the  course.  Most  of  the  comments 
related  to  the  quality  of  the  lecture  delivery.  It  should  be  noted  that  contributions  in  these  early  weeks 
involved  less  than  1 0%  of  the  students  attending  these  lectures. 

Lecturers  voice  fades,  notably  at  the  end  of  sentences.  There  were  technology  problems  as  well,  but 
lecturing  techniques  need  to  be  improved.... 
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72%  of  the  comments  in  those  first  three  weeks  came  from  students  at  the  Rockhampton  campus  (11% 
Bundaberg,  16%  Mackay).  Many  of  the  Rockhampton  students  would  have  been  on  the  receiving  end  of  a 
videoconference  delivered  lecture  for  the  first  time.  The  difference  in  viewpoints  between  campuses 
became  more  obvious  during  the  following  weeks. 

You  people  in  Rockhampton  are  pathetic.  You  complain  that  the  lecturer  was  late.  I know  for  a fact 
that  he  was  not 

Well  I think  the  students  in  bundy  should  think  before  they  speak  or  in  this  case  write 

Often  problems  such  as  this  would  have  continued  throughout  the  term  with  little  or  no  change. 
However,  in  this  case  the  problems  raised  were  addressed  in  a variety  of  ways  and  from  week  4 onwards 
the  overall  student  feeling  on  the  barometer  was  positive. 

The  lecture  was  far  different  than  last  week,  the  voice  of  the  lecturer  was  much  better 


Why  don’t  students  contribute? 

In  the  period  July  1999  to  November  2001  there  were  an  average  of  just  less  than  18  barometer 
contributions  per  course  offering.  This  raises  concern  about  why  more  students  are  not  making 
contributions  to  course  barometers.  This  section  draws  on  data  from  web  logs,  student  barometer 
comments  and  responses  to  the  student  survey  to  examine  this  issue  further. 

Of  the  students  surveyed  who  knew  about  the  course  barometer  7%  never  read  the  contributions, 
21%  read  them  once,  36%  every  now  and  then,  17%  once  every  couple  of  weeks,  and  19%  once  a week. 

Of  the  students  surveyed  who  read  contributions  57%  never  made  a contribution,  34%  made  a comment 
every  now  and  then.  Only  9%  of  students  surveyed  made  a contribution  every  week.  Using  web  logs  it  is 
possible  to  determine  that  during  the  three  busy  terms  6%  of  the  people  looking  at  a course  barometer  made 
a comment.  During  the  last  term  of  2001,  where  all  Infocom  courses  had  barometers,  only  1.5%  of  visitors 
made  a contribution. 

When  asked  why  they  didn't  make  a contribution  the  most  popular  student  responses  included: 
having  no  complaints  or  strong  opinions  to  contribute,  having  a preference  for  using  another  medium,  and 
the  belief  that  there  would  be  no  change  if  a contribution  was  made.  Other  reasons  given  included  a lack 
of  other  comments,  didn't  want  to  repeat  an  existing  comment  and  having  no  time. 

When  asked  why  they  made  contributions  the  most  common  reasons  included:  the  importance  of 
giving  feedback  in  order  for  change  to  happen,  having  strong  feelings  about  an  issue  and  supporting  a 
lecturer  against  negative  comments.  Other  reasons  included  agreeing  with  a comment,  disagreeing  with  a 
comment  and  because  the  lecturer  asked. 

69%  of  survey  respondents  agreed  or  strongly  agreed  with  the  statement,  “If  visible  changes  were 
made  because  of  comments  to  the  course  barometer  I would  be  more  likely  to  add  comments  to  the  course 
barometer”.  Student  comments  to  various  survey  questions  include 

Because  the  lecturer  *really  * wanted  feedback  - asked  every  week.  I though  it  was  worth  the  trouble 
to  help  improve  the  course  for  the  future 

Does  it  get  back  to  the  Dean  or  Head  of  school?  It’s  no  use  going  to  the  lecturer.  If  the  barometer  will 
serve  to  fix  slackness  then  yes  it  will  be  useful 

It  is  Just  another  mouthpiece  for  the  University  not  to  take  into  consideration.  It  is  the  same  as  the 
evaluations.  None  of  them  are  ever  taken  into  consideration. 

Any  changes  that  were  made  came  to  late  for  the  majority  of  the  class.  In  fact,  when  the  changes  were 
made  they  advantaged  the  few  and  disadvantaged  the  majority  of  the  class. 

I don't  think  I saw  any  changes.  I did  however  notice  in  one  nameless  subject  that  I did  this  semester, 
that  at  the  start  there  were  some  very  negative  comments  about  it  (that  I totally  agreed  with)  but  when 
nothing  changed  the  use  of  the  barometer  just  died 

The  lack  of  anonymity  was  not  given  by  any  student  as  a reason  for  not  contributing.  However,  a 
barometer  contribution  did  show  that  at  least  one  student  had  a problem  with  the  apparent  lack  of 
anonymity. 

I suppose  you  think  that  if  no-one  submits  a comment,  then  everyone  must  be  going  ok.  Not!  Just 
maybe,  its  the  fact  that  the  submission  doesn't  appear  to  be  anonymous  when  your  asked  for  your 
student  details  to  submit  your  comments.  I have  lots  of  concerns  and  would  like  to  pass  comments  to 
the  course  barometer,  but  am  concerned  that  its  the  messenger  that  will  be  shot  and  not  the  message 
addressed. 

Other  students  expressed  concern  about  anonymity  when  answering  the  survey  question  “What 
didn't  you  like  about  the  course  barometer?”. 
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While  realising  the  need  to  prevent  malicious  access,  it  was  disconcerting  to  need  username  and 
password  to  access  an  anonymous  site. 

When  you  submit  you  need  to  logon.  This  gives  the  feeling  that  who  is  responding  is  being  recorded 
instead  of  it  being  anonymous. 

Some  students  appear  to  have  believed  that  the  course  barometer  was  anonymous.  Responses  to 
the  survey  question  “What  did  you  like  about  the  course  barometer?”  included 

students  can  feel  safe  in  that  it  is  anonymous,  so  they  won't  be  attacked  for  their  opinion. 

Allows  student  who  do  not  wish  to  be  named  to  provide  any  kind  of  feedback  without  fear  of  it  being 
held  against  them  come  marking  time. 

Some  students,  usually  when  supporting  a lecturer  against  negative  contributions,  defeated  the 
supposed  anonymity  of  the  barometer  contribution  by  including  their  name,  for  example. 

I do  not  feel  the  need  to  make  my  comments  anonymously,  so  here  goes Linda. 

Though  this  practice  was  given  other  interpretations  by  other  students 
....It  is  amazing  how  some  students  say  they  don't  need  to  be  anonymous  about  their  comments  and  then 
piss  in  the  lecturers  pocket  and  attach  their  name 

Some  students  perceived  that  the  anonymity  provided  by  the  course  barometer  was  actually  a 
negative  attribute. 

I would  approach  the  lecturer/Co-ordinator  directly.  I believe  that  the  barometer  restricts/reduces 

open  communication  with  the  source  of  the  problem 

Often  I noticed  other  students  use  anonymity  to  feel  OK  about  not  bothering  to  constmet  appropriate 
arguments  and  blurt  out  unconstructive  criticism. 

Other  students  suggested  that  knowledge  of  the  contributors  identity  should  be  used 
Force  student  ID,  student  number  would  suffice,  so  that  course  organisers  can  follow  up  true  complaints 
Very  good  thing,  maybe  every  time  somebody  votes,  they  get  an  entry  into  a prizepool  to  win 
something,  maybe  a textbook  refund  or  something  simple  like  that. 

While  the  apparent  lack  of  anonymity  was  a problem  for  some  students  it  appears  it  is  not  as 
important  a factor  as  the  lack  of  observable  change.  Students  do  not  wish  to  give  feedback  if  there  is  no 
visible  response. 


Staff  Impact  and  Views 

Only  a small  number  of  motivated  staff  have  made  significant  use  of  the  course  barometers.  As  a 
result  all  of  these  staff  see  the  potential  benefits  that  the  barometer  can  provide. 

..the  Course  Barometer  is  an  excellent  tool,  and  staying  "in  the  green"  all  term  is  a good  challenge  for 
teaching  staff.  At  a glance,  you  can  see  how  some  students  are  going.  And  because  there  is  no  way  for 
teaching  staff  to  see  who  made  which  comment,  students  can  say  how  they  are  going  without  fear  of 
recriminations. 

However,  they  have  also  experienced  a number  of  problems  including:  unconstructive  feedback, 
inability  to  he^  students  with  problems  directly,  and  frustration  caused  by  negative  feedback  when  trying 
their  best. 

Because  the  tool  keeps  students  anonymous,  the  feedback  tends  to  attract  occasional  students  who  want 
to  have  a little  rant  or  hissy  fit  without  any  chance  of  people  finding  out  who  they  were.  So,  instead  of 
these  students  seeking  help  or  doing  something  really  useful,  they  vent  their  frustrations.  We  need  to 
reach  these  students  and  help  them,  but  instead  they  hide  behind  anonymous  feedback 

So,  maybe  this  student  is  looking  at  the  slides  for  the  wrong  subject,  or  simply  cannot  put  two  and 

two  together,  or  else  has  some  severe  fundamental  misunderstandings,  or  is  on  another  planet,  or 
something.  In  any  case,  I cannot  find  out  who  they  are  or  contact  them  or  help  them,  and  this  is  a real 
shame. 

Comments  made  on  the  feedback  barometer  can  be  incredibly  frustrating  because  the  students  who 
make  them  can  sometimes  need  urgent  help  or  assistance,  but  they  have  chosen  the  barometer  to  make 
their  conrmients,  and  this  keeps  them  anonymous,  so  we  have  no  chance  to  find  out  who  they  are  or  help 
them.  This  is  incredibly  frustrating.  I want  to  help  these  students,  but  my  hands  are  completely  tied  - 
unless  they  also  decide  to  email  me  or  contact  me  in  some  other  w^  - which  sadly,  does  not  seem  to 
happen.  These  students  seem  to  replace  email  with  the  feedback  barometer,  so  they  seem  to  use  one 
instead  of  the  other. 
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Conclusions 


A course  barometer  is  a method  for  addressing  the  loss  of  informal  feedback  in  a distance 
education  setting.  While  the  use  of  course  barometers  within  Infocom  has  been  done  in  an  ad  hoc  manner 
with  limited  contributions  by  students  there  have  been  a number  of  situations  where  its  presence  has  helped 
students,  staff  and  the  organization. 

The  small  number  of  contributions  on  the  Infocom  barometers  can  be  explained  by  a number  of 
factors.  It  appears  that  a combination  of  staff  promotion  of  the  barometers  and  observable  change  in 
response  to  student  feedback  are  amongst  the  most  important  factors  for  promoting  student  contributions. 

It  appears  that  the  lack  of  anonymity  provided  by  the  Infocom  implementation  of  course  barometers,  while 
an  issue  for  some,  is  not  of  prime  concern  and  some  students  and  staff  actually  see  it  as  a negative  influence 
over  the  quality  of  contributions. 

In  the  future  tools  such  as  the  course  barometer  will  be  of  increasing  importance  to  Infocom. 
Recent  moves  to  increase  the  importance  of  quality  assurance  within  the  Australian  University  environment 
is  one  factor.  Infocom's  increasingly  complex  mix  of  distributed  campuses  and  distance  education  also 
increase  the  importance  of  having  a simple,  heavily  used  student  feedback  mechanism. 

As  a result  the  Infocom  barometer  concept  is  likely  to  have  an  expanded  role  with  a number  of 
planned  changes  to  improve  the  level  of  student  contributions.  In  particular,  it  is  likely  that  responsibility 
for  promoting  student  contributions  and  ensuring  changes  are  seen  to  happen  will  be  given  to  a position 
seen  as  being  independent  and  separate  from  teaching  staff  Another  likely  change  is  the  combination  of 
the  course  barometer  idea  with  the  anonymous  threaded  discussion  functionality  described  in  Lowder  and 
Hagan  (1999).  This  would  allow  discussions  and  responses  to  develop  around  a particular  student 
contribution.  This  would  provide  a place  where  change  can  be  seen  and  also  encourage  further  learning 
(Lowder  and  Hagan,  1999)  and  facilitate  informal  student  interaction. 

The  work  described  here  is  still  in  the  stage  of  early  investigation.  More  research  is  required  to 
further  test  and  clarify  some  the  observations  and  conclusions  that  have  been  made.  As  the  use  of  the 
course  barometer  concept  expands  it  will  become  increasingly  important  to  investigate  how  the  course 
barometer  fits  in  with  and  relates  to  other  mediums  used  to  support  teaching  and  learning.  However,  course 
barometers  can  be  a positive  and  useful  tool. 
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Abstract 

Many  tertiary  institutions  in  Australia  provide  support  to  develop  online  teaching  and  learning 
resources,  an  environment  characterised  by  demands  from  students  for  quality  face-to-face 
and  distance  education,  staff  concern  over  workloads,  institutional  budgeting  constraints  and 
an  imperative  to  use  management  systems.  There  also  remains  a legitimate  focus  on  using 
online  learning  to  facilitate  new  learning  strategies  within  a complex  social  setting.  This  paper 
presents  an  extended  instructional  design  model  in  which  the  development  cycle  for  online 
teaching  and  learning  materials  uses  a scaffolding  strategy  in  order  to  cater  for  learner- centred 
activities  and  to  maximisc  scarce  dcv'cloper  and  sademic  resources.  The  model  also 
integrates  accepted  phases  of  the  instructional  development  process  to  provide  guidelines  for 
the  disposition  of  staff  and  to  more  accurately  reflect  the  creation  of  resources  as  learning 
design  rather  than  instructional  design. 


Introduction 

Teaching  and  learning  in  tertiary  education  has  shifted  over  the  past  two  decades  to  an  environment  where 
technology  is  a significant  component  of  the  overall  infrastructure  and  the  skills  and  credentials  of  both  teachers 
and  learners  are  crucial  to  enable  them  to  work  effectively  with  collaborative,  online  activities.  While  many 
teachers  have  embraced  these  new  environments  and  take  responsibility  for  the  development  and  delivery  of 
resources,  many  other  academic  staff  rely  on  central  support  units  to  provide  expertise  in  both  curriculum  design 
and  strategies  for  online  teaching  and  learning.  It  is  this  latter  group  of  people  to  whom  this  paper  is  specifically 
directed,  although  the  concepts  will  also  have  ramifications  for  all  online  development. 

As  leaders  of  support  units  in  two  Australian  universities,  one  Faculty-based  the  other  University-wide,  we  find 
that  academic  staff  often  have  too  little  time  or  too  few  skills  to  maximise  the  benefits  of  online  learning.  At  the 
same  time  our  institutions,  like  many  others,  are  emphasising  the  role  of  enterprise-based  Learning  Management 
Systems.  Within  this  environment,  our  roles  involve  the  design  and  development  of  online  teaching  and  learning 
resources  within  tight  timeframes  and  institutional  constraints,  which  often  force  the  units  to  be  in  a ‘responsive 
or  reactionary  mode’  without  proper  and  significant  long  term  planning.  This  can  result  in  Just-in-time  delivery 
packages  rather  than  more  preferable  long-term  rollouts. 

This  short  paper  speculates  on  extending  the  way  in  which  instructional  design  strategies  are  applied  to 
educational  resource  development  such  that  that  production  efficiency  can  be  increased  and  the  ongoing 
maintenance  of  online  environments  enabled.  While  instructional  design  and  development  processes  integrate 
current  good  practice,  the  proposed  variations  from  existing  models  are  based  on  an  extended  approach  to  the 
development  process  conceptualised  in  three  discrete  phases  and  the  integration  of  professional  development 
scaffolding  to  effectively  align  online  teaching  resources  with  learner  needs  and  expectations.  In  essence,  the 
model  articulated  provides  a means  to  enhance  the  production  environment  for  online  materials  while 
maintaining  or  even  increasing  quality  by  conceptualising  the  design  and  delivery  environment  within  the 
iterative  and  rapid  prototyping  methods  available  through  contemporary  development  systems. 
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Three  Phase  Design  and  Scaffolding 


The  general  instructional  design  model  (for  example,  Morrison,  Ross  & Kemp,  2001)  typically  prescribes  the 
creation  of  resources,  their  implementation  and  delivery  that  is  then  followed  by  evaluation  and  improvement. 
Our  enhancement  proposes  the  initial  creation  of  a fully  functional  prototype,  which  is  then  used  for  delivery, 
with  the  evaluation  and  improvement  activities  being  integrated  with  scaffolding  (support)  for  the  teacher  and 
learners  to  provide  a dynamic  teaching  and  learning  environment  in  which  resources  or  strategies  can  be 
developed  or  modified  during  the  actual  delivery  stage.  The  need  for  scaffolding  has  largely  arisen  because  of 
the  rapid  implementation  of  learning  management  systems,  the  increased  used  of  online  teaching  and  learning 
and  the  evolution  of  learner-centred  educational  paradigms  (Herrington  & Oliver,  2001). 


Integral  to  this  process  is  the  notion  of  iterative  development  or  successive  approximations,  with  initial 
prototypes  being  built  to  test  the  water  before  completion  of  the  entire  course.  In  the  first  iteration  learning 
environments  are  generally  created  to  provide  functional  delivery  with  the  necessary  componentry  for  effective 
online  teaching  and  learning.  This  can  include  the  outputs  of  a preliminary  needs  analysis  of  the  learning 
environment  and  resources  that  are  scaled  to  fit  the  proposed  teaching  and  learning  context.  However  with  the 
second  and  subsequent  iterations,  development  can  be  enhanced  with  each  generational  change.  In  addition,  the 
model  is  based  on  a team  approach,  bringing  together  the  three  main  elements  of  course  development  in  a more 
lateral  manner.  No  longer  is  process  driving  the  development,  but  the  project  itself  (i.e.  the  course)  is  dictating 
the  make  up  of  the  teams  (a  cross  section  of  skills  from  educational  design  and  production)  in  a much  more 
targeted  and  effective  manner.  These  teams  ideally  stay  formed  for  the  duration  of  the  project,  potentially  over  a 
number  of  semesters,  with  communication  and  collaboration  between  academic  staff  and  developers  a key  focus. 


Phase  1:  Function 


Phase  2:  Enhancement 


Phase  3:  Maintenance 


PRE-DELIVERY 


* Create  a functional 
prototype  of  the  teaching 
and  learning  environment 
for  delivery 

* Provide  Initial  professional 
development  (scaffolding) 


INITIAL  DELIVERY 
(Semester  1) 


Edj/catlonal  DesIqKier 


ONGOING  DELIVERY 
(Semester  2) 


* Work  with  teacher  to  identify 
outcomes  of  functional 
design. 

Where  appropriate,  enhance 
delivery  environment  and/or 
scaffold  online  teaching  and 
learning  behaviours 


'Developer\ 


Edkfcatlonal  PesIgKier 


Modify  (add  or  remove) 
resources,  activities  or 
strategies  based  on 
evaluative  feedback 


Figure  1:  Three-Phase  Design  & Scaffolding 
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The  model  therefore  reinforces  both  the  team-based  approach  to  the  design  and  provision  of  resources  as  well  as 
an  iterative  development  process.  One  of  the  essential  aspects  of  the  model  is  the  specification  of  baselines  in 
levels  that  correspond  to  these  iterations  - the  first  relating  to  course  functional  and  essential  elements,  the 
second  to  multimedia  enhancement  or  interactivity  and  the  third  to  ongoing  maintenance.  These  iterations  are 
identified  within  the  strategy  as  three  scheduled  phases  of  development  that  integrate  both  a methodological 
approach  to  unit  development,  scaffolding  and  quality  controls  and  assurance,  as  illustrated  in  Figure  1. 

The  triangles  indicate  the  relative  efforts  of  the  critical  members  of  the  project  team  at  each  phase  of  the  process, 
based  on  the  influence  model  (Sims,  1997),  which  articulated  the  period  at  which  factors  had  specific  influence 
over  the  project.  The  allocation  of  resources  to  enable  this  process  involves  establishing  “unit  teams”  whose 
commitment  will  vary  according  to  the  position  of  the  unit  in  the  development  cycle,  with  expertise  based  on  the 
varying  requirements  of  the  course.  More  importantly,  within  the  context  of  our  work  environment,  it  is  the 
allocation  of  resources  for  the  duration  of  the  project  life  that  differentiates  the  model,  as  detailed  in  the 
following  description  of  the  phases. 


Phase  1:  Prototype  Development 


The  aim  of  this  phase  is  to  design  and  create  a functional  teaching  and  learning  online  environment  that  will 
meet  all  learning  outcomes  as  well  as  faculty  teaching  and  learning  strategies.  The  first  phase  therefore  becomes 
easier  or  simpler  than  more  traditional  models  of  instructional  design,  as  it  is  functional,  and  production  does  not 
try  to  complete  a final  package  at  the  first  attempt  - the  process  can  therefore  be  likened  to  enabling  a “dress 
rehearsal”  for  both  teacher  and  leaner.  The  process  also  involves  specifying  the  core  items  for  this  phase,  such  as 
specific  teaching  resources  (e.g.  unit  guides,  study  guides,  readings),  their  mode  of  access  (e.g.  print,  online)  and 
the  essential  educational  strategies  (e.g.  experiential,  situated,  learner-centred).  In  this  way  the  academic  who  has 
minimal  experience  with  online  teaching  and  learning  environments  has  an  relatively  easy  introduction  to  the 
environment  while  knowing  that  ongoing  support  will  enable  the  generational  development  of  that  environment. 


An  equally  important  aspect  of  this  phase  is  the  allocation  of  team  members  and  their  specific  role  within  the 
project,  which  can  be  articulated  in  terms  of: 


• The  Support  Team:  Providing  the  Educational  Designer  (responsible  for  educational  advice  and 
curriculum  design),  the  Interactive  Architect  (responsible  for  ensuring  the  online  interactions  and 
communications  are  consistent  with  the  design)  and  Information  Analyst  (responsible  for 
ensuring  all  required  learning  resources  and  objects  are  available).  In  addition.  Project 
Management  support  will  be  required 

• The  Faculty  Team:  Allocating  the  Content  Specialist(s),  who  are  responsible  for  ensuring  all 
necessary  content  is  defined  and  that  all  learning  outcomes,  learning  activities  and  assessment 
tasks  are  defined.  In  addition,  a commitment  to  the  schedule  and  baselines/guidelines  is  critical. 


In  addition,  an  Online  Developer,  Network  Specialist  and  Technical  Specialist  will  both  advise  and  be  advised 
on  required  and/or  appropriate  learning  environments.  As  an  extension  to  the  triangular  concept  indicated  in 
Figure  1,  the  detailed  representation  of  influence  (see  Sims,  1997)  is  elaborated  in  Figure  2 for  each  of  the 
perceived  roles,  where  the  apex  of  the  triangle  or  polygon  represents  the  phase  in  the  development  cycle  at  which 
that  team  member  will  have  most  influence.  In  this  illustration,  the  different  skills  are  also  aligned  with  a 
particular  unit  - Support  representing  the  central  unit  within  the  institution  that  provides  educational  advice  and 
development  services.  Faculty  representing  the  knowledge  based  to  be  provided  from  the  teaching  unit  and  IT 
representing  the  potential  need  for  highly  specialised  network  and  programming  expertise.  In  addition,  these 
teams  will  also  link  across  the  various  phases  as  the  courseware  assembly  process  progresses . 

Another  aspect  of  the  concept  of  influence  is  that  members  of  the  development  team  are  understood  to  have 
potential  levels  of  influence  at  any  stage  of  the  development  and  delivery  process,  although  that  influence  will  be 
affected  by  the  current  status  of  the  project.  For  example  the  Interactive  Architect,  who  has  the  main 
responsibility  (influence)  for  creating  the  design  specifications,  may  also  be  active  in  the  quality  review  of  the 
project  as  it  nears  completion.  An  important  concept  underpinning  this  model  is  that,  like  actors  in  a play,  the 
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team  members  all  have  roles  to  play  and  particular  scenes  or  acts  within  that  process  will  require  their 
leadership.  But  they  can  also  have  smaller,  but  by  no  means  unimportant,  roles  throughout  the  whole 
development,  delivery  and  maintenance  cycle. 


Resource 

Phase  1 Phase  2 Phase  3 

Educational  Designer  (Support) 

Interactive  Architect  (Support) 

Online  Developer  (Support) 

Information  Analyst  (Support) 

Content  Specialist  (Faculty) 

Network  Specialist  (Support  or  IT) 

Technical  Specialist  (Faculty) 

Figure  2:  Influence  of  Team  Members  During  Project  Life-Cycle 


Phase  2:  Evaluating  and  Enhancing  Delivery 

The  second  phase  is  conceptualised  to  take  place  during  the  delivery  of  the  unit,  with  feedback  from  both 
teachers  and  learners  used  to  modify  and  enhance  the  delivery  environment.  This  may  include  the  introduction  of 
content  items  and  enhancement  of  teacherdeamer,  leamerxontent  or  leamerdeamer  interaction  conditions  (cf 
Sims,  Dobbs  & Hand,  2001).  It  is  also  an  opportunity  for  teachers  to  work  in  a scaffolded  environment  to 
maximise  the  effectiveness  of  online  environments,  where  the  efforts  of  both  teacher  and  learner  can  be 
evaluated  and  the  delivery  environment  enhanced  on  the  basis  of  that  evaluation.  This  process  also  allows  for 
clearer  scheduling  of  resources  and  consequently  planning,  production  and  workflow  processes. 

This  phase  will  require  a team-based  approach  to  delivery  combining,  where  appropriate,  both  academic  and 
technical  staff  in  two  discrete  components.  The  first  requires  more  technically-oriented  teams  to  “shadow”  the 
delivery  of  the  unit  materials  defined  and  created  in  Phase  1 to  both  assess  their  efficacy  as  well  as  integrate 
additional  content,  interactive  learning  objects  and  collaborative  activities.  The  second  includes  the  provision  of 
targeted  professional  development  or  scaffolding  on  an  “as  required”  basis  for  all  participants  in  the  learning 
process.  Overall,  this  phase  emphasises  generational  changes  with  an  increased  emphasis  on  the  production 
(completion)  of  resources,  with  the  students  or  learners  having  the  role  of  research  or  evaluation  assistants.  There 
is  less  emphasis  on  handover,  and  more  emphasis  on  duty  of  care  through  the  availability  of  sustainable  course 
materials  and  teaching  resources. 


Phase  3:  Maintenance 

Following  completion  of  the  course  of  study,  additional  modifications  and  enhancements  are  prescribed  and 
implemented  for  subsequent  delivery.  The  unit  would  then  continue  in  “maintenance  mode”,  involving  ongoing 
support  and  training,  until  it  undergoes  a more  formal  review.  Again,  the  important  concept  underpinning  this 
model  is  that  the  original  functional  system  developed  will  always  be  subject  to  change  and  that  any 
development  environment  must  cater  for  resources  to  be  available  for  the  duration  of  the  life-time  of  a course  (or 
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unit)  of  study.  Within  tertiary  institutions  this  can  be  as  long  as  five  years,  the  time  between  a unit’s  conception 
and  its  formal  review  for  reaccreditation.  However,  the  sustainability  of  the  course  is  catered  for  by  the  continual 
process  of  gathering  and  incorporating  evaluation  data. 

The  success  factors  will  depend  not  only  on  the  concept  being  accepted  but  also  for  academic  staff,  students  and 
the  development  team  to  reconceptualise  their  roles  in  the  design  and  delivery  of  online  educational  resources. 
For  teachers  there  is  the  option  to  collaborate  with  an  online  development  expert  while  delivering  the  course  to 
implement  modifications  based  on  student  feedback;  for  learners  there  is  the  opportunity  to  contribute  to  both  the 
content  base  and  the  educational  strategies 

In  Figure  3 following,  a sample  model  is  provided  to  demonstrate  how  the  model  may  be  implemented  over  a 
three-year  cycle,  with  the  assumption  that  units  of  study  are  delivered  on  a semester  basis. 


2002 

2003 

2004 

SI 

S2 

SI 

S2 

SI 

S2 

prepare 
materials  for 
S2/02  units 

deiiver  S2  units 

deliver  SI  units 

deliver  S2  units 

deliver  SI 
units 

deliver  S2  units 

shadow  and 
enhance  S2 
units 

shadow  and 
enhance  SI 
units 

shadow  and 
enhance  S2 
units 

shadow  and 
enhance  SI 
units 

shadow  and 
enhance  S2 
units 

prepare 
materials  for 
Sl/03  units 

prepare 
materials  for 
S2/03  units 

prepare 
materials  for 
Sl/04  units 

prepare 
materials  for 
S2/04  units 

evaluate  s2/02 
units 

evaluate  sl/03 
units 

evaluate  s2/03 
units 

evaluate  sl/04 
units 

Figure  3:  Implementing  the  Plan 


Conclusion 

Higher  education  in  Australia  is  changing  and  to  meet  these  changes  and  challenges  innovative  models  for 
academic  support  are  required.  The  model  proposed  in  this  paper  articulates  an  enhancement  to  traditional 
instructional  design  processes  where  specific  aspects  of  development  and  delivery  are  viewed  in  parallel  rather 
than  in  sequence.  Instead  of  a development  team  watching  delivery  of  resources  remotely,  it  is  proposed  that, 
where  feasible,  members  of  the  development  team  actively  participate  with  both  teachers  and  learners  in  the 
delivery  process.  In  this  way  support  or  scaffolding  in  the  form  of  professional  development  can  be  provided  on 
an  as  requiredhzsx^  while  technical  specialists  can  implement  modifications  to  both  content  and  pedagogy. 

The  value  of  this  model  can  therefore  be  realised  through  the  innovative  ways  in  which  it  conceives  the 
development  process  as  develop  baseline  — implement/evaluate/develop  — maintain/evaluate  rather  than  the  more 
traditional  process  of  design  — develop  — implement  — evaluate.  The  model  provides  an  holistic  framework 
consisting  of  long-term  development  teams,  course  templates,  design  and  delivery  standards  and  specified 
delivery  platforms.  The  development  of  course  materials  is  therefore  not  a short-term  production  process  but  a 
long-term  collaborative  process  by  all. 

Based  on  this  analysis  we  believe  the  benefits  to  teaching  and  learning  in  higher  education  will  include  the 
following: 

• it  can  be  a try  it  and  see  approach,  where  the  first  phase  enables  strategies  to  ‘test  the  water’  so 
initial  budgets  aren’t  blown  out  in  expensive  experiments,  as  has  been  evidenced  in  many 
multimedia  projects; 
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• funds  can  be  allocated  across  more  projects  for  a longer  period  of  time,  such  as  towards  second 
iteration  enhancements  that  really  target  the  learners  and  are  appropriate  to  the  learning 
environment;  and 

• course  design  in  this  development  model  includes  both  teacher  and  learner  feedback  and  is 
enhanced  incrementally  to  match  learner  needs. 
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Abstract:  For  the  last  few  years,  we  have  been  developing  an  online/in-house  teaching 
environment  in  Web-design  and  multimedia  animation.  At  this  stage,  the  outcome  is  a hybrid 
system,  known  as  HyWeb  (Hybrid- Web).  It  is  the  result  of  combining  online  delivery  with 
traditional  teaching.  The  experience  has  led  to  the  belief  that  a holistic  approach  to  course 
design  and  delivery  is  not  only  practical  but  also  essential.  As  certain  features  are  required 
they  are  added  to  enhance  the  quality  and  efficiency  of  the  system,  making  it  a dynamic  and 
evolving  entity.  This  paper  discusses  the  approach  used  for  course  development  within  the 
Multimedia  degree.  School  of  Information  Technology  at  Griffith  University,  Gold  Coast 
campus. 


1.  IntroduCtiGn 

At  Griffith  University  Gold  Coast,  several  faculty  members  of  the  School  of  IT  are  working  together  to  create  a 
holistic  and  evolving  teaching  environment  aimed  at  meeting  the  needs  of  their  students.  Two  important 
components  of  good  online  instruction  are:  methods  to  cope  with  the  limited  bandwidth  of  today,  and  the 
inclusion  of  the  human  element.  The  problem  of  limited  bandwidth  is  diminishing  with  time,  but  the  “human 
element”  still  needs  to  be  considered.  Communication,  involvement  and  connectivity  can  be  addressed  by  a 
more  flexible  approach  to  online  education. 

2.  Online  Delivery  System 

HyWeb  was  developed  at  Griffith  University,  Gold  Coast  and  implementation  has  been  ongoing  since  1999. 
The  system  focuses  on  delivery  and  can  be  applied  to  any  field  of  study.  There  are  two  major  limitations: 

• Bandwidth  and  Data  Transfer  Rates  - with  the  increasing  amount  of  Internet  traffic,  it  is  difficult  to 
guarantee  adequate  data  transfer  speeds,  site  availability  and  consistency.  One  of  the  major  problems 
reported  by  students  using  the  system  has  been  the  slow  data  transfer  rate  (Jones,  et  al.^  2000).  High 
bandwidth  options,  such  as  broadband,  are  still  expensive  and  not  practical  for  many  students. 

• Communication  and  the  “human  element”  - another  limitation  reported  by  students  is  the  reduced  face-to- 
face  contact  (Jones,  et  al.,  2000).  Student  feedback  has  shown  that  they  prefer  some  face-to-face  contact 
during  an  online -delivered  course.  At  Griffith  University  Gold  Coast,  we  have  endeavoured  to  maintain  a 
face-to-face  component  within  the  online  system,  using  a combination  of  traditional-style  lectures,  tutorials, 
lecturer/tutor  consultation  and  online  delivery.  The  one-hour  consultations,  available  weekdays,  provide  the 
opportunity  for  students  to  seek  assistance  and  have  contact  with  the  lecturer/tutors.  The  students  have  used 
this  resource  extensively,  especially  before  assignments  and  examinations  are  due. 


3.  HyWeb  - The  Evolving  System 

HyWeb  is  still  evolving.  As  needs  arise,  these  needs  are  met  by  system  growth.  Communication  is  an  essential 
component  and  has  been  maintained  in  the  HyWeb  model  via  several  media:  email,  online  forum,  online  notice 
board  and  face-to-face  (Jones,  etaL,  2001).  ^ introducing  a wireless  component,  additional  communication 
via  WAP-based  email.  Internet-based  forwarded  email  and  SMS  (short  message  service)  can  be  achieved. 
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3,1  Wireless  System  Component 


To  provide  more  options  for  students,  wireless  technology  is  being  incorporated  within  the  system.  It  will  be 
available  through  either  a Web-enabled  mobile  phone  and/or  a handheld  device  such  as  a PDA.  Students  can 
also  have  limited  access  to  the  wireless  system  using  a non-WAP  mobile  phone,  by  receiving  SMS 
notifications  and  email  through  various  Internet-based  free,  email  forwarding  services. 

Along  with  its  versatility  and  adaptability,  wireless  technology  has  certain  limitations,  which  include:  mainly 
text-based  information;  small  screen  size;  high  running  costs;  no  keyboard  or  mouse  (although  a plug-in 
keyboard  is  now  available  for  some  models);  and  lack  of  storage  space.  Wireless  communication  bandwidth  is 
low  (transfer  rates  are  currently  around  9600  bps),  but  WAP  was  designed  to  cope  with  low  bandwidth  and 
intermittent  coverage.  There  are  many  positive  aspects  and  promising  features  of  wireless  technology,  such  as: 
portability  and  mobility;  flexibility;  convenience;  remote  access;  and  user-friendly  devices  (Jo,  et  al.y  2001). 

The  idea  of  mobile  innovations  and  a wireless  university  campus  is  not  new.  Minnesota  State  University, 
Mankato  (MSU)  introduced  a wireless  system  in  mid-2000.  Mobile  handsets  were  provided  and  the  students 
were  expected  to  perform  many  of  their  daily  tasks  using  wireless  services  (Virtanen,  et  aL,  2000).  Mobile 
innovations  were  also  introduced  at  Twente  University  in  Holland,  where  the  mobile  phone  is  now  considered 
to  become  an  important  tool  for  learning  (Dorsey,  2001).  Wireless  technology  in  education  is  also  referred  to 
as  wireless  e-leaming,  and  can  offer  many  benefits.  Grimes  (2000)  states  that,  “If  wireless  communication  is 
still  in  its  infancy,  wireless  e-leaming  is  in  an  embryonic  stage”.  Convenience  is  a key  element  of  this 
technology,  allowing  anytime/anyplace  access.  Using  wireless  technology  within  the  established  delivery 
system,  students  will  be  able  to:  check  assignment  due  dates  and  exam  timetables;  send  email  to  the  lecturer; 
access  a web-lecture  using  a PDA;  check  the  online  notice-board;  and  join  an  online  forum  or  chat  session. 


4.  Conclusion 

The  HyWeb  system  has  undergone  many  changes  over  the  last  few  years,  the  most  recent  being  the 
introduction  of  wireless  technology.  This  is  expected  to  have  a positive  effect  on  the  students  and  their 
perception  of  the  course.  Handheld  and  wireless  devices  are  used  extensively  in  today’s  society  and  the 
concept  of  using  such  technology  is  both  exciting  and  inspiring.  HyWeb  is  constantly  evolving  with  new 
technology  being  integrated  into  the  system  as  it  emerges.  HyWeb  enables  easy  modification  of  the  system  and 
allows  for  growth  and  flexibility.  By  constant  reviews  and  modifications,  ideas,  inspiration  and  motivation  are 
maintained  and  those  involved  strive  to  develop  the  system  into  the  more  fully  expanded  and  improved 
structure.  This  is  an  ongoing  project  aiming  at  an  effective  and  efficient  dynamic  learning  environment,  which 
can  demonstrate  increased  flexibility,  portability  and  convenience. 
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Abstract:  In  terms  of  education  technology  the  Irish  education  system  is  a late 
developer.  In  1997  a government  led  initative,  “Schools  IT  2000”  was  launched  to 
computerize  the  nation's  schools.  This  qualitative  research  study  documents  the 
experience  of  five  schools  involved  in  pilothg  a communications  and  collaborative 
project  called  “Wired  for  Learning”  that  was  sponsored  by  IBM  and  the  Department 
of  Education  and  Science.  The  data  and  the  findings  discussed  pertain  to  the  first 
year  of  the  project's  introduction  in  the  participating  schools.  A follow-up  study  will 
be  completed  on  the  project's  overall  progress  and  development  when  the  three-year 

project  period  comes  to  an  end  in  2002 . 


Introduction 

In  1997  the  Irish  Government  launched  a major  drive  to  computerize  Ireland’s  schools  when  it 
established  the  National  Center  for  Technology  in  Education  (NCTE)  to  implement  its  “Schools  IT 
2000”  policy  initiative.  The  Schools  Integration  Project  (SIP),  administered  by  the  NCTE,  was  a key 
strand  of  this  initiative.  Its  remit  was  to  explore  creative  ways  in  which  ICT  could  be  successfully 
integrated  into  the  existing  education  system  at  primary  and  post  primary  levels.  To  achieve  this,  SIP 
invited  schools  to  establish  best  practice  models  across  a wide  variety  of  school  settings  including, 
special  needs,  curricular  and  technical  support.  One  of  the  SIP  projects,  known  as  “Wired  for 
Learning”  is  the  focus  of  this  research. 

Wired  for  Learning  (WFL)  is  a secure  communications  and  collaboration  tool  designed  to  develop  the 
use  of  ICT  for  communications  and  collaboration  purposes  for  the  entire  school  community  i.e. 
teachers,  pupils,  parents  and  mentors.  It  was  developed  by  IBM  as  part  of  its  global  ‘Reinventing 
Education’  programme  that  began  in  1994  in  the  U.S.  In  1998  IBM  Ireland  and  the  Department  of 
Education  and  Science  announced  a special  education  partnership  and  a $1  million  investment  in 
Ireland’s  school  system  to  bring  the  WFL  platform  to  a number  of  Irish  schools  under  the  SIP 
umbrella.  Its  core  stated  objectives  were  (a)  To  empower  teachers  with  the  capability  to  share  resources 
and  knowledge  within  and  across  primary  and  post-primary  schools  and  (b)  To  facilitate 
communication  and  collaboration  between  schools,  parents  and  the  wider  community. 

There  were  five  schools  in  total  in  the  WFL  project,  3 primary  schools  and  2 post -primary  schools  at 
two  sites.  Site  one  comprised  three  schools  located  in  a disadvantaged  suburban  community  in  Dublin. 
Site  two,  was  located  in  a more  affluent  semi-urban  area  in  the  far  south  of  the  country.  As  part  of  the 
project  all  schools  were  equipped  with  a state  of  the  art  networked  computer  lab. 

Access  to  the  WFL  environment  is  facilitated  via  a standard  web  browser,  a registered  user  ID  and 
password.  The  environment  is  made  up  of  a suite  of  tools  or  applications  that  support  communication, 
collaboration  and  learning.  Communication  tools  include  secure  email  for  all  pupils,  teachers,  parents 
and  mentors  and  a bulletin  board  interface  which  supports  conferencing  between  parents,  teachers  and 
students.  Collaboration  tools  facilitate  student  team  projects,  a private  area  where  teachers  can  work 
together  and  the  involvement  of  both  local  and  global  mentors.  A suite  of  related  databases  known  as 
the  Instructional  Planner  support  the  learning  process.  A demo  version  of  WFL  is  available  for 
downloading  at  www.wiredforleaming.net/. 
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The  Research  Context  and  Research  Questions 


The  research  was  carried  out  by  a full  time  PhD  Research  Fellow  located  at  Dublin  City  University 
using  a case  study  methodology.  Research  tools  included;  (a)  Classroom  observations  of  how  ICT  was 
being  used  to  support  teaching  and  learning;  (b)  Attendance  at  project  strategy  meetings  and  staff 
training  courses  and  (c)  In-depth  interviews  with  key  personnel  i.e.  I.T.  coordinators,  school  principals 
and  teachers  (n=37).  The  key  three  research  questions  focused  on  (1)  teachers’  perceptions  of  the 
innovation;  (2)  its  usefulness  as  a communications  and  collaboration  tool  and  (3)  the  sustainability  and 
scalability  of  the  innovation. 

Teachers’  views  on  the  innovation  were  mixed.  Many  however  questioned  its  usefulness  and  relevance 
to  their  classroom  teaching  and  practice.  The  Instructional  Planner  section  of  WFL  was  particularly 
criticised  h this  regard.  The  main  reason  being  that  (a)  it  forced  them  to  plan  out  lessons  in  a very 
structured  way  which  was  not  how  they  traditionally  planned  their  lessons;  (b)  the  system  was  too 
cumbersome;  (c)  the  process  was  too  time  consuming  and  (d)  there  was  a deep  suspicion  that  it  was  an 
attempt  to  bring  in  greater  teacher  accountability  through  the  back  door.  In  fact  opposition  to  the 
Instructional  Planner  was  so  strong  that  most  schools  had  effectively  abandoned  its  use  as  a lesson- 
planning tool  by  the  time  the  interviews  were  conducted.  Instead  the  IT  coordinators  and  principals 
were  concentrating  their  efforts  on  encouraging  staff  to  use  other  aspects  of  the  innovation. 

In  general  most  teachers  reported  that  WFL  had  given  them  the  opportunity  to  collaborate  more  with 
each  other.  Teachers  reported  how  training  opportunities  to  learn  the  software  as  a group,  in  addition  to 
the  use  of  teacher  release  time  which  gave  them  the  opportunity  to  plan  how  to  use  the  system  and  to 
input  lessons  and  resources  into  WFL,  all  helped  this  process.  As  one  teacher  phrased  it  - “the  project 
has  given  us  a chance  to  work  together  as  adults”.  The  facility  afforded  by  the  system  to  create  a 
repository  of  resources  for  each  year  group  was  cited  as  a particularly  welcome  feature  of  the  system 
and  evidence  of  greater  transparency,  co-operation  and  collaboration  among  staff  members.  One  school 
reported  how  its  use  of  the  system  for  encouraging  staff  members  to  electronically  communicate 
agenda  items  for  staff  meetings  had  increased  teacher  involvement  and  contributions. 

Opinions  on  its  usefulness  as  a communications  tool  were  less  positive.  Many  could  not  see  the  point 
of  sending  another  member  of  staff  an  email  when  they  could  talk  to  them  in  the  staff  room  during 
break  time  or  in  the  corridor.  Email  communications  with  parents  was  viewed  in  an  even  more 
problematic  light.  Teachers  expressed  a fear  that  emailing  parents  could  potentially  lead  to  a situation 
where  too  much  teacher  time  would  be  required  to  service  parent  emails , particularly  those  of  a 
demanding  parent.  The  schools  in  the  disadvantaged  area  also  felt  that  emailing  was  impractical 
because  very  few  of  their  students  had  PC’s  at  home  anyhow.  Consequently  none  of  the  schools  in  this 
cluster  group  had  yet  put  procedures  in  place  to  involve  the  parents  in  the  project. 

A number  of  reservations  were  expressed  at  both  sites  in  relation  to  the  sustainability  and  scalability  of 
the  innovation.  Some  of  the  main  reasons  given  were  (a)  the  expense  involved  in  purchasing  the  WFL 
system  which  the  pilot  project  sites  did  not  have  to  incur;  (b)  a feeling  that  there  were  other  systems 
available  on  the  open  market  which  could  do  the  same  thing  in  a more  efficient  and  less  cumbersome 
manner  and  (c)  a belief  that  the  WFL  was  culturally  more  suited  to  the  American  education  system 
than  to  the  Irish  system.  There  was  also  a strong  feeling  among  participating  teachers  that  teaching 
colleagues  in  other  schools  would  not  be  prepared  to  invest  the  same  level  of  time  and  effort  as  they 
had.  In  addition  many  were  of  the  view  that  any  attempt  to  roll  out  the  project  nationwide  would  be 
opposed  by  the  teachers  unions.  It  was  also  difficult  to  see  how  other  schools  would  be  prepared  to  run 
with  such  a system  if  they  were  not  going  to  receive  outside  technical  assistance  and  support,  on  a par 
with  that  provided  to  the  two  start-up  sites. 


All  in  all  the  early  evidence  seemed  to  indicate  that  schools  were  struggling  to  integrate  WFL 
successfully  into  the  school  environment  and  that  it  ws  unlikely  to  be  a success  if  rolled  out  to  other 
schools.  However,  as  the  project  still  had  another  year  to  run,  the  researcher  is  of  the  view  that  it  is 
probably  too  early  to  make  any  substantive  claims  about  the  innovation’s  long-term  sustainability  and 
scalability. 


Conclusion 
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1.  Introduction 

Integrating  encyclopedias  with  the  Internet  is  changing  the  way  K-12  students  study  with  encyclopedias  from  looking  at  an 
encyclopedia  to  making  one  with  all  students  and  teachers  participating  and  contributing.  From  this  point  of  view,  we  have 
developed  an  Internet-based  collaborative  encyclopedia  platform  called  CyberPedia  for  K-1 2 education. 

It  has  three  main  features.  (l)Easy  uploading:  Anyone  can  contribute  to  the  encyclopedia  by  uploading  his/her  knowledge 
or  learning  results  with  multimedia  contents  via  the  Internet.  (2)Easy  Searching:  Anyone  can  easily  search  for  them.  (3)Easy 
Authoring:  A nyone  can  make  a custom  encyclopedia  on  a browser  without  having  a database  of  technical  knowledge. 

2.  Basic  Design 

CyberPedia  supposes  three  types  ofusers:  a student,  a teachei;  and  a site  manager  who  is  an  expert  or  educator  supervis  ing 
designs  and  contents  of  CyberPedia  as  well  as  being  a database  manager. 


First,  a student  explores  and  studies  existing  data  in  CyberPedia,  which  mean  initial  contents  that  site  managers  prepared 
beforehand.  Then,  he/she  chooses  an  attractive  topic  and  goes  out  in  the  field  with  a digital  camera  or  a video  camera  to 
further  investigate  it  independently.  After  returning  to  the  school,  the  student  completes  observations  in  a template  and 
submits  them  to  CyberPedia.  Finally,  the  student  broadenshis/her  knowledge  by  comparing  it  to  other's  reports.  CyberPedia 
offers  students  the  opportunity  to  explain  what  they  gathered  so  that  it  has  some  meaning  for  others. 


The  teacher’s  main  tasks  are  to  register  user  information  beforehand  and  to  manage  data  uploaded  by  students.  CyberPedia 
enables  the  teacher  to  return  comments  or  advice  on  each  student's  work  and  to  make  it  public  or  private  on  t he  Internet.  The 
default  isprivate. 

(3)  Site  manager 

The  site  manager  designs  CyberPedia  itself  according  to  a learning  theme.  The  design  of  an  encyclopedia  necessarily 
differs  according  to  its  contents  or  the  author’s  expertise.  Therefore,  CyberPedia  offers  the  site  manager  authoring  functions 
to  decide  what  search  conditions  are,  what  object  data  categories  are,  and  so  on.  Moreover,  the  site  manager  can  change  the 
web  site  visually  and  insert  the  initial  data  into  it.  In  short,  CyberPedia  is  a platform  for  making  custom  encyclopedias . It  can 
also  provide  students  with  multiple  different  encyclopedias  at  the  same  time. 

3.  Implementation 

We  used  Linux  as  the  operating  system,  Apache  as  the  Web  server,  and  PostgreSQL  as  the  database  because  this  software  is 
all  free  and  stable.  The  programming  language  is  Java  servlet  and  JSP. 

Figure  1 shows  a screen-shot  of  the  search  window  of  the  microorganism  guide  housed  in  Sendai  Science  Museum  and 
used  by  junior  high  school  students  in  Sendai.  The  left  frame  is  for  specifying  characteristics  of  the  search  conditions.  Each 
entry  in  the  encyclopedia  (which  we  call  an  object)  is  linked  with  relevant  characteristics.  The  middle  one  lists  object  names 
of  the  search  results . Objects  have  the  same  data  categories  to  classify  data  according  to  the  contents.  When  one  selects  an 
object  and  a data  category,  its  contents  are  shown  in  the  right  frame. 

Needless  to  say,  fixing  the  format  reduces  the  flexibility  of  the  encyclopedia's  design  to  some  degree.  However,  we 
considered  simplicity  and  productivity  to  be  more  important  than  flexibility.  This  enables  site  managers  to  make  a custom 
encyclopedia  by  editing  characteristics,  objects,  and  data  categories  and  to  design  it  by  inserting  icons  and  changing  colors 


(\)  Student 


(2)  Teacher 


Fig.  1 Search  window.  Fig.  2 Upload  window. 

on  a browser.  They  can  do  almost  all  of  the  operations  to  make  an  encyclopedia  just  by  selecting  from  menus. 

Figure  2 shows  a screen-shot  of  the  upload  window  This  one  page  is  a template  for  students  to  submit  observations.  They 
have  only  to  input  or  select  some  items  such  as  a title,  object,  data  categoty,  and  comments.  When  the  submit  button  is 
pressed,  all  information  is  automatically  accumulated  in  CyberPedia.  Contents  can  be  uploaded  in  various  formats,  such  as 
JPEG,  MPEGl,  and  Quicktime  movies. 

4.  Discussion  about  the  applications  using  CyberPedia 

The  strengths  of  CyberPedia  are  to  teach  science  as  a process,  to  provide  a good  opportunity  for  students  to  express  their 
thoughts,  and  to  facilitate  collaboration  among  distant  students  and  experts.  In  some  applications  using  CyberPedia  (Kura 
2001),  the  students  tackled  the  learning  theme  independently.  From  the  results  of  questionnaires,  we  found  that  all  students 
were  interested  in  uploading  their  findings  and  in  debating  with  peers  using  CyberPedia.  Not  only  exploring  the  initial 
contents  but  also  observing  the  neighboring  natural  environment  and  uploading  observations  to  CyberPedia  were  good 
opportunities  for  students  to  think  deeply  about  science  as  a real  natural  phenomenon.  Simply  working  in  the  CyberPedia 
environment  challenged  both  teachers  and  students  to  change  their  teaching  and  learning  methods. 

However,  the  experiments  revealed  a lack  of  functions.  For  example,  teachers  said  that  the  page  uploading  format  did  not 
challenge  students  to  develop  their  own  means  of  organizing  information.  Of  course,  it  depends  on  students  ’ skills,  but  we 
should  offer  alternative  functions  supporting  drawing  tools  on  tlie  Web  to  develop  the  students  ’ creativity  (Sugiyama  2001), 
Besides  that,  we  will  also  think  about  team-building  activities  and  reflective  writing  tools  in  future. 

CyberPedia  is  really  a simple  structure  and  has  a great  potential  to  various  applications  such  as  digital  museums  and 
community  platforms . We  think  the  most  promising  area  is  student  portfolios.  CyberPedia  offers  the  potential  for  individual 
students  to  develop  portfolios  that  they  might  develop  over  a lifetime  of  learning, 

5.  Conclusions 

We  described  the  concept  and  development  of  a new  encyclopedia  platform  called  CyberPedia,  It  is  a web-based 
application;  therefore  it  can  provide  an  opportunity  for  distant  students  and  teachers  to  make  an  encyclopedia  in 
collaboration.  This  is  very  effective  pedagogically,  because  uploaded  contents  include  the  position,  date,  and  time  of  the 
observation,  and  students  all  over  the  country  can  study  and  investigate  a specific  theme,  for  example,  the  movement  of  the 
cherry-blossom  front  in  spring  or  the  distribution  of  growing  trees,  collaboratively. 
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Abstract:  This  paper  deals  with  a specific  aspect  of  the  joint  project  "Wissenswerkstatt 
Rechensysteme"  (WWR,  Knowledge  Factory  for  Computing  Systems)  of  twelve  German 
universities.  It  describes  how  to  integrate  external  simulators  and  experiments  into  courseware 
based  on  an  XML  teaching  and  learning  environment.  The  approach  is  based  on  wrapping 
experiments  as  Web  Services.  It  provides  an  indirect  access  to  the  experiments  through  the 
central  university  learning  system.  We  are  convinced  that  learning  by  experiencing  the 
phenomena  of  computer  systems  in  simulation  or/and  in  real  working  systems  (is  very  helpful 
for)  helps  a computer  science  student  to  comprehend  and  to  get  acquainted  with  the  more 
complicated  topics  in  this  field. 


Introduction 

In  the  ongoing  joint  project  "Wissenswerkstatt  Rechensysteme"  [1]  (WWR,  Knowledge  Factory  for  Computer 
Systems)  (Lucke,  2002),  a broad  set  of  multimedia  teaching  and  learning  modules  are  being  developed  in  the 
field  of  computer  engineering.  Each  module  can  be  offered  at  three  levels,  such  as  basic,  advanced,  and  expert  as 
per  the  target  group  (teachers,  learners)  in  following  output  formats  (HTML  for  Internet-based  learning  material, 
PDF  for  printed  manuscripts,  PPT  for  slide  presentations).  In  order  to  achieve  such  a flexibility,  modules  are 
prepared  in  the  Extensible  Markup  Language  (XML)  (Bradley,  2000),  maintained  in  a repository  and  organized 
using  metadata  based  on  the  IEEE  Learning  Objects  Metadata  (LOM)  standard  (IEEE,2000), 
(Monthienvichienchai,  2001).  Based  on  learning  objectives,  teachers  select  the  modules  as  per  the  needs  of  their 
students  and  offer  these  as  a course.  As  a result,  the  materials  required  for  a course  are  generated  as  per  the 
request  in  the  desired  formats  from/by  the  content  repository  using  XSL  (Extensible  Style  sheet  Language) 
(W3C,  2001)  processing  techniques. 

The  WWW -based  online  course  material  so  generated  is  described  below  in  greater  detail.  For  students,  such 
courses  mainly  consist  of  pieces  of  HTML  text,  diagrams,  pictures  and  other  embedded  multimedia  documents. 
But  computing  systems  and  especially  operating  systems  are  not  directly  perceivable  by  human  beings!  A 
student  cannot  see,  listen,  touch  and  smell  the  running  processes,  the  events  happening  etc.  The  most 
complicated  are  the  practical  exercises.  In  order  to  really  comprehend  and  understand,  how  the  system  works, 
probably  the  best  way  is  to  experience  a running  system  with  all  its  effects  and  side  effects,  its  dynamics,  the 
occurring  events  and  exceptional  situations,  the  relations  "if...  then  ..."  etc.  (Gershon,200 1 ).  As  a consequence,  a 
much  higher  degree  of  interactivity  is  required  compared  to  traditional  course  materials.  This  means 
incorporating  simulators  and  interacting  in  real-time  with  external  real-life  systems  and  experiments.  This  type 
of  environment  was  set  up  and  used  with  great  success  for  practical  exercises  about  operating  systems  based  on  a 
more  traditional  technical  platform  and  hand-coded  experiment  interfaces  (Kalfa,  1997). 

This  paper  deals  with  the  higher  degree  of  interactivity  required  and  proposes  a method  for  solving  the  technical 
problems  arising  from  the  external  applications  put  on  the  Internet  as  if  they  were  locally  available  to  the 
student.  The  university  has  to  keep  the  control  on  the  interaction  of  a large  number  of  students  during  such 
prearranged  experiments.  Chapter  2 discusses  the  problem  of  interactivity  in  more  detail.  Chapter  3 provides 


[1]  This  work  is  supported  by  the  German  Ministry  of  Education  and  Research  (BMBF). 
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insight  into  the  architecture  developed,  while  chapter  4 discusses  the  initial  examples  of  the  application  of  this 
architecture. 


The  Problem  of  Interactivity 

Web-based  interactive  course  materials  are  written  in  HTML.  Such  courses  run  locally  on  the  student’s  computer 
and  use  a standard  browser  as  the  user's  interface.  HTML  links  are  used  for  referring  to  the  local  and  external 
Web  documents.  Interactivity  is  typically  based  on  buttons,  animated  GIFs,  image  maps  and  animations.  Here, 
animation  is  defined  as  a time -based  manipulation  of  some  target  elements  visualizing  some  (e.g.  technical) 
process.  Animations  are  considered  to  be  running  locally  as  a part  of  the  interactive  course.  They  are  well -suited 
for  illustrating  specific  aspects  of  the  behavior  of  the  complex  processes.  In  this  sense,  an  animation  is  typically 
based  on  an  abstract  model  of  the  real-world,  and  it  is  completely  specified  at  design  time  concerning  its  time 
behavior.  Thus,  an  animation  can  not  be  considered  as  a substitute  for  a real-life  experiment.  Today,  for 
example,  Macromedia  Flash  (Macromedia,  2001)  is  a well  accepted  commercial  solution  for  defining  animations 
in  the  Web  environment.  It  allows  link-based  navigation  to  other  elements  of  the  course  and  requires  a browser 
plug-in  for  playing  the  so-called  Flash  movies.  In  future,  open  W3C  standards  for  animation  like  SMIL  (W3C, 
2001a,  2002b)  may  become  important  as  well,  when  they  are  directly  supported  by  browsers. 

Compared  to  animations  running  locally,  interfacing  of  external  simulators  and  real-life  experiments  is  more 
complicated.  The  principal  architectural  requirements  are  illustrated  in  figure  1.  The  protocol  "user-experiment" 
demonstrates  the  dynamic  interaction.  The  user  can  change  the  parameter  of  the  simulation  as  well  as  the  kind  of 
measurements  and  the  monitoring  time  interval,  on  the  fly.  The  minimum  requirements  are  a control  window,  a 
visualization  window,  and  a measurements  window. 


Figure  1:  Requirements  of  /for  the  architecture  (full  lines  designate  the  data  flow,  and  the  dashed  lines  the 
control  flow) 

A simulation  program  or  a running  experiment  typically  requires  some  control  input  and  will  produce 
measurement  data  and  other  output  information.  Measurements  are  feasible  in  a simulation  model  or  can  be 
implemented  by  monitoring  a real-life  experiment.  In  the  second  case,  an  instrumentation  of  the  system  under 
measurement  is  necessary.  Operating  systems,  such  as  UNIX/LINUX  as  well  as  Microsoft  Windows  offer 
services  for  information  about  the  parameters  specific  to  the  running  system.  Other  may  be  added  to  operating 
systems,  if  the  source  code  of  the  operating  system  is  given,  i.e.  LINUX  and  now  Windows,  as  well. 

The  integration  problem  is  even  harder,  when  a "tight"  integration  is  required,  i.e.  when  the  e^eriment  or  the 
simulator  output  affects  the  course  materials  presented.  In  this  case,  a different  paragraph  of  the  text  may  be 
presented,  some  alternative  link  in  the  presentation  may  be  followed  etc.  Such  cases  require  a dynamic 
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rearrangement  of  the  couise  material  presented.  Besides,  the  teacher  may  be  interested  in  providing  all  his 
students  access  to  the  supplied  experiments  and  in  monitoring  the  students*  progress. 


Architectural  Approach 

This  chapter  describes  the  current  approach  to  solve  the  problem  of  interactivity  as  outlined  in  the  last  chapter. 
The  architecture  developed  is  illustrated  in  figure  2.  It  distinguishes  between  the  interactive  course  and  the 
external  simulators  or  experiments.  The  interactive  course  may  be  running  on  the  learner's  computer,  perhaps  at 
home,  and  on  the  learning  system  server  running  at  the  university  site.  The  external  simulators  or  experiments 
may  be  running  somewhere,  usually  also  at  the  university  site.  In  some  cases,  it  may  be  desirable  to  access 
these.on  the  learner's  computer.  In  other  cases,  a complex  and  expensive  experiment  setup  may  need  to  be 
provided  centrally  for  several  universities  over  the  Internet.  Principally,  interaction  between  the  learner  and  the 
experiment  takes  place  indirectly  through  the  learning  system  at  the  university.  This  ensures  full  supervision, 
control  and  feedback.  It  also  keeps  account  of  the  learner's  actions  and  controls  his  access  to  the  experiments 
offered  as  well  as  synchronizes  these. 


Figure  2 : System  Architecture 

A general  approach  for  interacting  with  a remote  program  component  is  called  the  Remote  Procedure  Call  (RPC) 
paradigm,  considering  the  request/response  type  of  interaction  between  a client  and  a server  . Classical  RPC 
systems,  like  SunRPC  or  DCE/Microsoft  RPC  are  based  on  a binary  encoding  scheme  for  parameters  and  results. 
This  binary  communication  scheme  is  quite  inadequate  for  the  WWR  learning  modules,  programmed  as  XML 
documents.  Thus,  a new  approach  is  needed,  which  allows  /provides  for  a seamless  integration  of  the 
simulator/experiment  results  into  the  final  HTML  pages  visible  to  the  learner.  Especially,  this  approach  should 
be  as  generic  as  possible  to  cover  all  types  of  external  components  for  all  the  different  WWR  modules  under 
development  as  Boolean  logic,  circuit  synthesis,  processors,  caches  andsomething  like  this. 

Our  approach  is  based  on  XML  documents.  We  currently  envisage  to  use  the  so-called  Web  Service  paradigm 
based  on  the  Simple  Object  Access  Protocol  (SOAP)  with  attachments  (W3C  2001c)  for  interaction  between  the 
learning  system  and  an  external  experiment.  SOAP  uses  XML  documents  for  encoding  input  and  output 
parameters  and  forwards  requests  using  the  HTTP  protocol.  The  Web  Services  Description  Language  (WSDL) 
(W3Cd)  is  another  XML-based  notation.  This  describes  services  as  a set  of  endpoints  operating  on  messages. 
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These  may  contain  either  document-  or  procedure -oriented  information.  As  a result,  existing  simulators  or  real 
experiments  can  be  wrapped  in  order  to  transform  them  into  a Web  service.  A big  advantage  is  that  SOAP 
wrappers  can  be  often  generated  automatically,  if  an  appropriate  interface  description  is  available. 

Thus,  the  following  interaction  pattern  results  therefrom:  The  student  is  able  to  work  most  of  the  time  locally  on 
the  course  . Only  when  dynamic  information  is  required  for  the  external  experiment  or  simulator,  the  browser 
will  contact  the  learning  system  at  the  university  site  (1).  Beside  other  components,  which  arc  not  important  for 
the  discussion  here,  the  learning  system  consists  of  a Web  server  with  an  associated  servlet  engine,  an  XML 
parser  and  a style  sheet  processor.  Based  on  the  student's  input,  a servlet  will  generate  a SOAP  request  for  the 
simulator/experiment  Web  service  (2).  When  the  response  comes  back  as  an  XML  document  (3),  it  will  be 
parsed  and  merged  with  the  stored  XML  fragments.  These  are  generated  at  course  creation  time  by  the  included 
modules  and  processed  by  a stored  style  sheet.  Its  output  is  the  final  HTML  page  (4).  This  goes  back  to  the 
student's  browser  (5).  If  a real-time  measurement  data  stream  from  the  experiment  to  the  learner  has  to  be 
supported  as  well,  the  approach  described  here  can  open  a private  communication  channel  between  the 
experiment  and  a signed  applet  running  on  the  student's  browser,  after  this  applet  has  been  loaded  from  the 
university  learning  server  (dashed  lines). 

This  approach  fulfils  all  requirements  related  to  the  dynamic  generation  of  HTML  pages  and  also  has  some  other 
advantages:  it  keeps  the  student's  environment  simple.  It  just  consists  of  his  browser  displaying  standard  HTML 
pages.  The  burden  is  put  on  the  central  learning  system  server  at  the  university  site  which  generates  the  more 
complicated  dynamic  HTML  pages.  This  central  server  is  able  to  uniformly  interact  with  all  simulators  or  real- 
life  experiments  used  in  all  active  courses.  Furthermore,  the  learning  system  server  is  able  to  collect  information 
about  the  students'. access  to  the  external  simulators/experiments.  This  information  may  be  useful  for  other 
components  of  the  learning  environment. 


Examples 


Currently  two  modules  are  being  prepared  for  evaluating  the  approach  described  in  the  previous  chapter.  The 
“source  code”  of  the  modules  are  stored  in  an  XML-based  repository.  From  this  repository,  different  kinds  of 
modules  of  the  same  subject  can  be  generated: 

pictures  and  interactive  animation  of  phenomena  supporting  real  lectures 

textbooks  with  pictures  and  interactive  animation  for  self-study  and  for  complementing  real  lectures  (s. 
Figure  3) 

local  an^or  in  a client/server  architecture  usable  experiments  (simulation  or  on  a real  system). 


Figure  3:  Snapshot  of  a “textbook-like”  module 


IGOW  fflMMBK 


943 


By  means  of  the  first  module  ’’Processor  Scheduling”  the  student  can  measure  the  arrival  rate,  the  service  rate  as 
well  as  the  waiting  time,  the  actual  queue  length  and  utilization  in  a simulation  experiment  for  processor 
scheduling.  Thereafter,  he  is  able  to  compare  the  simulation  results  with  the  theoretical  ones  according  to  the 
model. 


Figure  4:  Snapshot  of  the  visualization  in  an  “experiment”  module 

The  second  module  presents  ’’Inter-Process  Communication  (IPC)",  another  topic  of  a classical  lecture  on 
operating  systems.  Here,  the  learning  module  interacts  with  real  inter  process  communication  examples  being 
carried  out  on  a remote  Linux  machine  or  a remote  Windows  NT  system.  In  this  way,  the  student  experiences  the 
classical  UNIX  IPC  primitives  like  pipes  and  message  queues  and  recognizes  differences  to  similar  NT 
abstractions. 


Conclusions 

We  are  convinced  that  learning  by  experiencing  the  phenomena  of  computer  systems  in  simulation  or/and  in  real 
working  systems  is  very  helpful  for  a computer  science  student  to  comprehend  and  to  get  acquainted  with  the 
more  complicated  topics  in  this  field.  Based  on  this  assumption,  we  presented  an  architecture  for  integrating  the 
external  simulators  and  real-life  experiments  seamlessly  into  the  course  materials.  These  are  being  currently 
developed  for  the  XML  teaching  and  learning  environment  in  the  joint  project  "Wissenswerkstatt 
Rechensysteme"  (WWR,  Knowledge  Factory  for  Computing  Systems). 
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Abstract;  The  purpose  of  this  research  is  to  evaluate  the  new  interface  designs  of  the 
digital  video  clips  for  K-6  from  the  data,  questionnaires,  and  interviews.  The  interface 
was  changed  on  the  following  two  points  by  the  result  of  the  previous  research  (Kamei 
et  al,  2001)^^^:  1)  Add  up  a function  to  stop  a key  scene  of  a clip  for  teaching,  2)  Add 
up  a function  to  connect  the  related  clips.  These  two  changes  clearly  meet  teachers’ 
needs  and  help  their  utilization  in  the  classroom. 


Introduction 


The  Japanese  government  promises  that  every  classroom  will  have  accessed  to  the  Internet  and  have 
two  PCs  and  a projector  by  2005.  From  the  promise,  digital  video  clips,  one  of  the  digital  materials, 
will  have  been  much  more  developed  near  future 

On  our  previous  research,  we  grasped  the  tendency  of  the  teachers’  utilization  and  problems  when 
teachers  use  the  clips.  In  this  time,  we  fixed  the  interface  for  viewing  the  clip  by  the  above  findings. 

-|jj 


Research  and  Development 


Clips  Background 

The  clips  and  the  interface  were  customized  for  elementary 
schools  (K-6)  in  Japan.  It  was  assumed  that  teachers  would 
show  the  clips  for  their  students.  The  subject  is  Japanese  History. 
The  image  on  the  clips,  30-90  seconds,  were  selected  from  NHK 
(Japanese  public  broadcaster)  archives,  and  re-edited. 

Change:  Add  up  a function  to  stop  a key  scene  of  a clip 
for  teaching 

It  was  clear  that  teachers  often  try  to  make  a stop  in  the  middle 
of  clips,  but  they  struggled.  Therefore,  we  put  small  pictures 
which  shows  the  position  where  the  key  scene  is  (Fig.  2). 


2"**  Change:  Add  up  a function  to  connect  the  related  clips 
It  was  clear  that  teachers  combined  the  clips  in  the  different 
units.  Although  they  wanted  to  use  such  way,  it  was  very  hard 
because  each  clip  was  isolated  by  the  era  categories.  From  that, 
our  solution  to  this  problem  is  to  put  and  link  a small  picture 
which  is  related  (Fig.  3). 


Methodology 

Figure  3:  2”*^  Change 

25  teachers  were  chosen  and  72  clips  were  prepared  for  this  research,  April- July,  2001. 
Questionnaires  and  interviews  were  made  in  order  to  evaluate  the  changes.  22  of  the  25  teachers 
were  replied  our  questionnaire  after  using  the  clips.  The  interviews  were  made  to  12  of  the  22 
teachers. 
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Results  and  Implements 


l“ change 

In  the  first  investigation,  we  compared  with  two 
groups,  a group  of  key  scenes  with  small  pictures 
and  without  small  pictures  by  the  number  of  the 
teachers  who  stop  and  use  the  clips.  As  a result, 
the  clips  with  small  pictures  are  more  stopped  by 
teachers,  comparing  with  the  clips  without  small 
pictures  (^4.75,  ^<0.01  )(Tab.  1). 

From  the  interviews,  there  are  two  types  of 
reactions:  one  is  positive  and  the  other  is  average.  The  reason  of  the  positive  reaction  is  that  it  is  easy 
to  show  clips  quickly.  Tn  addition,  the  teachers  who  are  novice  for  teaching  could  easily  recognize 
the  key  scene  where  many  teachers  think  it  is  important  for  teaching. 

The  reason  of  the  average  reaction  is  mostly  from  the  teachers  who  have  much  experienced  for 
teaching  because  they  did  not  care  the  key  scene. 

From  these  results,  we  concluded  that  the  first  change  was  efficient  for  the  teachers  because  there  is 
no  negative  opinion. 


Mean 

With  small  pictures(43  scenes) 

6.95 

3.92 

Without  small  pictures(293  scenes) 

4.03 

2.50 

Welch's  t-test  (N-22,  |=4.75,£<0.01) 
Table  1.  Scene  stopped 


Clip  A 

2002  (N=22) 

95.5% 

100.0% 

2001  (N=27) 

77.8% 

- 

2"**  change 

Table  2 shows  the  ratio  of  the  pair  of  clips  (A  and  B) 
with  the  new  interface.  The  utilization  of  the  clip  A in 
2002,  which  is  put  and  linked  the  small  pictures,  is 
higher  than  Clip  A in  2001 . 

From  the  interviews,  all  teachers  evaluated  that  the 
second  change  were  efficient.  The  main  reason  is  that  it 
is  easy  to  show  clips  quickly  because  they  did  not  need  to  go  back  to  the  top  page  of  the  unit  one  after 
another. 

From  these  results,  the  second  change,  put  and  link  the  small  pictures  on  the  same  page,  was  efficient  for 
the  teachers. 


Table  2.  Utilization  of  a pair  of  clips 


Conclusion 

It  is  concluded  that  the  new  interfaces,  which  put  small  pictures  under  control  bar,  would  be  helpful 
for  a teacher  to  select  a key  scene.  Teachers  immediately  could  show  the  scene  that  they  want  the 
students  to  watch.  The  second  change,  put  and  link  the  small  pictures  on  the  same  page,  are  also 
efficient  for  the  teachers. 

Further  research  would  be  needed  especially  on  the  following  three  points:!)  Needs  to  conduct 
experimental  study.  2)  Needs  to  collect  more  data  about  the  teacher’s  utilization  of  the  clips. 3)  Needs 
to  develop  the  system  to  reflect  the  data  from  the  teachers’  past  usage. 
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Abstract:  The  research  focus  of  this  study  is  how  to  use  computers  to  support 
and  improve  dyslexic  students’  reading  and  writing  performance.  More  than 
10%  of  the  population  is  estimated  to  experience  substantial  difficulties.  We 
propose  to  offer  personalised  text  formatting  according  to  individual  profiles 
when  reading  screen  texts,  and  preliminary  studies  indicate  improved  reading 
performance  among  poor  readers.  We  also  propose  to  use  speech  synthesis  in 
order  to  give  auditive  support  to  the  reading  process.  Improved  reading  is  of 
importance  both  to  the  students’  motivation  and  the  overall  learning  outcome.  It 
is  important  that  the  increasing  use  of  computers  in  primary  schools  does  not 
result  in  just  another  arena  for  shortcoming  for  dyslectic  students. 

Introduction 

Reading  consists  of  two  components;  decoding  and  comprehension.  Decoding  refers  to  the  technical  skill 
that  enables  the  reader  to  transform  printed  letters  into  recognised  words  as  the  basis  for  activation  of 
meaning.  Comprehension  is  a cognitive  process  that  allows  the  reader  to  interpret  the  text,  which  includes 
thinking  processes  on  higher  levels.  This  basic  concept  of  reading  must  be  completed  with  an  additional 
component;  motivation.  Motivation  is  crucial  in  all  learning  tasks. 

Some  research  shows  that  as  many  as  15--20  % of  the  students  in  primary  and  secondary  schools  have 
problems  in  meeting  the  required  level  of  literacy  in  today’s  society  (Catts  and  Kahmi  1999,  Solheim  and 
Tonncscn  1999).  Other  studies,  however,  show  a substantially  lower  prevalence  of  reading  problems. 

At  the  onset  of  the  21^^  century  the  computer  is  increasingly  being  adopted  as  a relevant  tool  (and  medium) 
for  reading  and  writing.  Both  in  Norway  and  internationally,  the  number  of  computers  per  student  has 
increased  over  the  last  few  years  as  well  as  the  time  spent  working  with  them.  It  is  of  great  importance, 
however,  to  realise  that  the  introduction  of  computer  technology  in  schools  implies  both  a risk  and  a 
possibility.  If  the  special  needs  of  students  with  literacy  problems  are  not  recognised  and  taken  into 
account,  the  use  of  computers  in  classroom  based  project  groups  may  well  represent  an  additional  hurdle 
instead  of  a compensatory  asset. 

In  our  work  we  have  developed  a prototype  to  support  reading  on  screen  for  persons  with  reading 
disabilities.  The  basic  functionalities  offered  in  the  first  version  are: 
personalised  text  formatting; 

auditive  support  using  speech  synthesis  (single  words,  sentences  and  paragraphs). 

The  target  group  is  students  (primary  and  secondary  school),  but  the  software  may  be  useful  to  adults  with 
reading  difficulies  as  well.  In  a future  version  functionality  supporting  writing  and  project  work  will  be 
added.  Our  study  has  a twofold  focus:  a)  utilisation  of  the  computer  to  help  students  with  reading  and 
writing  difficulties  to  decode  and  comprehend,  and  b)  application  of  technology  in  such  a way  that  these 
children  are  integrated  in  the  computer-based  activities  in  the  classroom. 
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Reading:  support  for  decoding,  comprehension  and  motivation 

Decoding  support:  From  research  conducted  over  the  past  20  years,  we  know  that  many  factors  influence 
legibility  of  texts  on  computer  screens,  see  (Kamstrup  et  al.,  2002  and  Harrell  1999).  Computer  texts  may 
be  formatted  according  to  individual  reading  levels  and  personal  preferences,  a possibility  the  printed  book 
does  not  offer.  Our  prototype  supports  adaptation  of  font  type,  font  size,  line  length,  line  spacing,  word 
spacing,  type  colour  and  background  colour  (see  Kamstrup  et  al.,  2002).  Each  student  can  create  her  own 
individual  profile,  and  subsequently  all  texts  that  the  student  chooses  to  read  will  be  formatted  according  to 
her  established  profile. 

Comprehension  support:  Several  researchers  have  recommended  the  use  of  speech  synthesis  to  support 
decoding  and  comprehension  in  children  with  reading  difficulties  (Wise  1997,  Lewin  2000).  Wise  et  al. 
(1997)  found  that  speech  synthesis  substantially  improved  the  disabled  reader’s  reading  performance,  and 
this  has  recently  been  confirmed  by  Lewin  (2000).  The  prototype  offers  a “read-it-loud-to-me” 
functionality  through  speech  synthesis.  The  student  can  choose  to  listen  to  words,  sentences  or  paragraphs 
according  to  the  individual  needs.  The  words  are  highlighted  while  being  read. 

Motivation:  Speech  synthesis  supports  comprehension,  and  provides  for  the  poor  reader  a necessary 
distance  to  the  laborious  task  of  technical  decoding  and  as  such  strengthens  the  communicative  aspect  of 
literacy.  This  may  make  the  reading  task  more  enjoyable,  stimulate  further  reading,  enhance  a sense  of 
mastery,  and  trigger  more  motivation. 

Results  and  future  research 

We  have  conducted  preliminary  studies  that  indicate  both  improved  reading  performance  (Haugen  2001) 
for  poor  readers  and  increased  motivation  for  reading  when  using  our  prototype.  Further  and  more  in  depth 
studies  are  necessary  to  show  the  real  effect  of  reading  individually  adapted  texts  on  the  computer  screen  as 
compared  to  traditional  paper-based  reading,  and  experimental  designs  that  can  show  this  effect  must  be 
developed.  Such  studies  will  be  carried  out  in  the  context  of  this  research  project  over  the  next  two  years. 

Conclusion 

Knowing  that  children  with  reading  and  writing  difficulties  are  motivated  by  certain  uses  of  computer 
technology,  this  must  be  exploited  for  the  best  of  struggling  readers.  Computers  are  an  integral  part  of  the 
daily  life  of  many  children,  and  we  must  ensure  that  the  process  of  introducing  computers  in  the  classrooms 
will  be  an  asset  for  all  students.  Several  researchers  have  shown  improved  reading  either  by  the  means  of 
screen  design  and  typography  or  by  speech  synthesis.  In  our  prototype  we  have  integrated  these  two  options 
to  make  an  even  better  support  for  dyslexic  children. 
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Abstract:  The  purpose  of  this  research  is  to  develop  a collaborative  architectural  design  system. 
Design  collaboration  requires  an  extensive  use  of  communication  methods  as  well  as  the 
participation  of  various  experts  from  different  domains.  Such  facts  address  several  issues  when  the 
Internet  and  digital  media  are  able  to  create  a completely  new  work  environment.  The  building 
design  process  was  studied,  and  possible  modes  of  design  collaboration  were  defined.  A prototype 
system  is  being  developed  in  accordance  with  the  defined  collaboration  model.  The  system 
integrates  a set  of  communication  tools  and  web-based  design  media.  Such  media  include  a 
synchronous  multi-user  Web  CAD  tool,  a schematic  3D  design  tool,  and  a graphic  whiteboard.  A 
project  database  was  designed  in  order  to  coordinate  the  project -wide  communication  which 
elaborates  technologies  such  as  web-based  data  access.  In  order  to  find  out  the  effectiveness  of 
the  system,  a usability  test  was  performed  both  in  quantitative  and  qualitative  manner.  The  research 
will  contribute  to  the  development  of  world- wide  design  and  construction  collaboration  through  the 
Internet,  which  is  becoming  a mainstream  building  process  model. 


Introduction 

In  architectural  design  team  works  are  often  required,  and  thus  collaboration  from  a number  of  participants  of 
various  specializations  is  needed  to  complete  a project.  For  collaboration  the  design  process  should  be  formalized, 
and  communication  methods  should  be  established.  Since  Internet  and  multimedia  play  an  important  role  in  current 
conputer  applications,  it  is  desirable  to  develop  a collaboration  system  which  manages  and  integrates  design  data 
and  project  data  by  various  communication  tools  and  digital  media  [11].  Collaboration  based  on  multimedia 
communication  though  World  Wide  Web  enables  designers  to  share  both  synchronous  and  asynchronous  information 
on  database,  and  thus  to  manage  design  project  effectively.  This  is  due  to  enhancement  of  operating  system  tool 
libraries  and  programming  API’s  which  make  Web-based  programs  efficiently  share  3 dimensional  design  data  and 
work  space  among  multi-users.  These  kinds  of  technological  opportunities  and  communication  needs  in  design 
process  naturally  lead  to  the  development  of  real  time  design  information  sharing  system,  virtual  design  studio  for 
collaboration. 

Recently  efforts  on  the  development  of  collaborative  systems  have  been  made,  but  they  focused  on  asynchronous 
text -based  document  management  instead  of  synchronous  information  sharing.  For  a collaborative  system  to  share 


950 


design  work  space,  it  is  required  not  only  to  overcome  technical  barriers  but  also  to  understand  the  design  process. 
The  purpose  of  this  study  is  to  develop  a collaborative  design  system  which  let  designers  generate  design  objects 
and  discuss  them  synchronously  through  the  Internet.  For  the  system  a number  of  different  types  of  graphic  tools  are 
introduced  according  to  design  development  stages;  graphical  whiteboard,  Space  Builder,  Space  Planner  and 
WebCAD.  The  system  also  supports  asynchronous  communication  means  like  web-mail,  project  bulletin  board, 
chatting  board  and  video  conferencing. 

For  effective  collaboration  in  architectural  design  following  aspects  should  be  considered.  First,  visual 
communication  by  graphics  is  more  prevalent  than  by  text  or  dialogue.  Second,  design  data  should  be  stored  in  a 
database  along  with  media  objects  as  their  container  and  project  management  schedules  [7],  Third,  collaborative 
working  environment  should  be  differentiated  and  be  supported  accordingly.  Fourth,  such  design  related  information 
as  building  regulations  should  be  linked  to  generated  design  data  for  design  evaluation. 


Conceptual  model  and  system  components 

In  order  to  develop  the  system  the  communication  model  of  the  design  process  was  investigated,  and  then  tools  are 
defined  for  model  components.  The  system  environment  is  designed  to  integrate  these  tools  (Figure  1),  while 
multicasting  technology  is  adopted  for  synchronous  communication  (Figure  2).  The  system  has  project  database  for 
design  data  at  the  center,  and  design  data  are  reported  by  project  work  schedule. 


Design  and  communication  tools 

To  support  design  and  communication  activities  through  the  design  process  Web-based  functional  tools  are 
implemented.  The  design  process  consists  of  primitive  sketch,  conceptual  modeling,  schematic  modeling  and  detail 
construction  modeling.  For  this  study  three  early  stages  are  taken  into  consideration  to  cover  design  studio 
curriculum  at  colleges.  Light-weighted  graphic  programs  which  have  limited  graphic  functions  but  enable  designers 
to  share  work  space  in  real  time  is  very  important  for  synchronous  communication  during  early  design  stages. 
Following  tools  are  developed  to  share  work  space  by  Internet  for  multi-users,. 

Early  sketch  tool  : Graphic  whiteboard 

At  an  early  design  stage  designers  need  to  communicate  with  a simple  sketch  tool.  Graphic  whiteboard  is  similar  to 
existing  electronic  whiteboard  augmented  with  graphics  functions.  However,  sketch  data  are  stored  for  reuse  at  later 
stages. 


msir  copf  0MJMX 
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Figure  4 Mass  builder 


Mass  design  and  study  tool : Mass  Builder 

For  a conceptual  model  3 dimensional  solid  design  objects  (building  mass)  are  represented  and  displayed  in  real 
time  to  project  participants. 

Space  configuring  tool : Space  Planner 

After  a building  mass  is  studied,  division  of  the  mass  vertically  and  horizontally  for  interior  space  configuration  is 
required.  Space  planner  is  developed  as  a separate  module. 


Figure  5 Space  planner 


Figure  6 WebCAD 


Schematic  design  and  drawing  tool : WebCAD 

At  the  schematic  design  stage  spaces  are  defined  elaborately  with  precise  dimensions  and  antidotes  for  building 
elements.  For  this  purpose  a Web-based  CAD  program  is  developed. 

Design  Information  Center  and  Project  scheduler 

To  evaluate  a proposed  design,  related  information  is  retrieved  from  the  database  through  Web  page  interface. 
Building  codes,  construction  and  engineering  specifications  and  similar  project  design  data  are  primary  data  sets 
which  will  be  growing  richer  as  users  contribute  relevant  information.  When  users  participate  in  a project  session, 
the  project  scheduler  will  provide  work  management  space  of  a calendar  type  interface. 
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Figure  8 Project  schedule 


Notable  features  of  virtual  design  system 


Virtual  design  system  provides  an  integrated  collaboration  design  environment. 

Ccllahoraticn  design  cnvironnicnt  rcc^uircs  a nunihc. 


V/A  11< 


vv/irvwu 


application  softvv^arcs  v/liicli  enable 
multiple  users  at  different  locations  to  share  ideas,  design  information  and  data  both  synchronously  and 
asynchronously.  The  system  supports  this  kind  of  work  environment. 


The  system  has  a number  of  rich  multimedia  databases. 

Architectural  design  information  is  documented  in  tow  primary  formats;  text  and  geometric  objects.  Project  database, 
case  database  and  scheduling  database  are  constructed  with  various  multimedia  formats  such  as  graphics,  images, 
animated  data,  video  and  voice  data.  In  an  era  of  multimedia  generation,  it  would  be  quite  natural  to  store,  search 
and  use  multimedia  data  for  design  projects. 


The  system  has  convenient  visual  communication  tools. 

General  purpose  communication  tools  such  as  web  mail,  text  and  voice  chatting  are  included  in  the  system. 
Furthermore,  synchronous  whiteboard  for  sharing  ideas  and  data  is  embedded  in  the  system.  Therefore,  synchronous 
/ asynchronous  and  text/multimedia  communications  are  supported  in  the  system. 

The  system  has  outstanding  web-based  architectural  design  tools. 

In  architectural  design,  visual  objects  are  used  for  effectiveness  and  efficiency  in  communicating  with  other 
members  who  are  involved  in  the  same  project.  At  the  early  stage  of  design,  visual  objects  are  sketched  images.  As 
the  design  proceeds,  they  become  geometric  objects  that  have  mathematical  properties.  To  be  more  precise,  physical 
object  specification  annotations  are  added  to  the  objects.  For  the  system,  web-based  visual  tools  for  sketching,  2D 
drawing  and  3D  modeling  are  implanted. 


The  system  has  near-perfect  work  schedule  management  tools. 

Design  project  management  involves  precise  scheduling  of  design  process  and  workers  in  various  areas  such  as 
construction,  electrical  field,  etc.  Therefore,  the  system  provides  work  calendars  for  keeping  track  of  work  progress. 
That  could  be  used  as  checking  points. 


Usability  test 

Usability  testing  has  been  performed  to  evaluate  the  efficiency  and  effectiveness  of  the  system.  In  order  to  increase 
the  test  authenticity,  three  real  architectural  design  projects  - museum,  office  and  house  design-  were  assigned  to 
three  different  groups.  Each  group  consists  of  three  designers.  The  entire  design  process  of  three  teams  has  been 
recorded  using  videotapes.  A survey  and  interviews  were  also  conducted  to  investigate  the  perception  of  the 
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participants  toward  the  function  of  the  system. 


On  the  basis  of  videotape  data  analysis,  the  design  process  of  each  team  has  been  identified  and  integrated  as  the 
following  steps: 

First,  after  the  log-in  process,  the  designers  opened  the  whiteboard  and  chatting  board  to  communicate  their  initial 
ideas  synchronously.  They  used  the  picture  files  of  the  ground  and  surrounding  environment  to  review  the 
possibilities.  Secondly,  the  designers  used  the  Info -center  to  search  for  the  similar  drawings  and  related  regulations. 
Thirdly,  they  started  to  Mass  Builder  to  generate  3D  mass  and  communicate  through  the  chatting  board.  Finally,  they 
used  the  WebCAD  to  draw  the  2D  drawing  and  share  their  comments  using  bulletin  board.  Designers  were  hardly 
used  the  Space  Planner  to  avoid  the  risk  of  spending  too  much  time  to  build  the  3D  simulation. 

The  results  of  the  survey  and  interviews  identified  the  most  used  and  comfortable  functions  and  identified  the  most 
needed  modification  of  the  system.  Whiteboard  and  Mass  Builder  were  the  two  most  used  functions.  Info -center. 
Mass  Builder  and  whiteboard  are  the  most  comfortable  functions  perceived  by  the  designers. 

Designers  asked  to  modify  the  following  four  functions  the  most: 

First,  designers  asked  to  integrate  the  drawing  and  writing  functions  of  the  whiteboard.  Second,  designers  asked  to 
integrate  the  whiteboard  and  Mass  Builder  in  order  to  share  their  ideas  effectively  as  well  as  efficiently.  Third, 
designers  wanted  to  have  their  own  team  bulletin  board  and  team  Info -center  for  better  communication  and 
ownership.  Finally,  and  mostly  designers  required  to  have  data  transfer.  For  example,  data  generated  by  whiteboard 
should  transmit  to  the  Mass  Builder  for  the  foundation  to  avoid  the  repeated  work. 

Other  than  functional  requirement,  the  role  of  team  leader  or  facilitator  is  a major  variable  to  the  success  factor  in 
collaboration.  Therefore,  the  training  or  assisting  function  a team  leader  or  project  manager  should  be  included  in 
the  system. 


Conclusions  and  future  works 

Using  the  Internet  as  a communication  method  arises  security  problem  of  the  system.  This  problem  is  not  solely 
subject  to  technical  solution,  but  multitudinously  subject  to  communication  models  during  the  design  process. 

Virtual  design  system  also  needs  extended  works  in  following  areas.  First,  graphic  data  file  format  should  be 
standardized  for  the  data  exchange  with  existing  systems.  Second,  mark-up  methods  should  be  developed  to  search 
and  manage  semantic  values  of  design  objects.  Third,  formalization  of  the  design  process  should  be  elaborated  to 
establish  project  schedule  by  stages. 
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Abstract:  This  paper  describes  development  of  the  Media  Literacy  Tool  (MLT),  a web-based  media  literacy  and  media  effects 
research  and  learning  tool.  Media  messages  have  permeated  our  world.  Humans  today,  in  many  cultures,  are  bombarded  with 
communications  from  advertising  television,  video  games,  etc.  Many  of  these  messages  are  created  with  specific  intentions  to  evoke 
particular  responses.  Most  of  us  are  not  equipped  to  critically  receive  these  signals,  and  many  of  us  fall  victim  to  effects  of  media 
exposure.  MLT  is  designed  to  address  the  poverty  in  media  research  and  education.  As  a tool  for  supporting  collaborative  content 
research,  critical  observation  and  emotional  awareness,  MLT  will  facilitate  media  effects,  literacy  and  resiliency  research. 

Rationale 

Television,  video  games  and  music  have  been  cited  as  a causal  factor  in  increased  levels  of  violent  behavior  in  many 
studies  including  National  Institute  of  Mental  Health,  1982;  Murray,  1980;  Rehman  & Reilly,  1985.  Heavy  viewing  of  television  has 
been  tied  to  increased  fear  about  real  world  personal  safety  (Signorielli,  1991).  Videogames  have  been  found  to  play  a role  in  the 
development  of  unhealthy  self-image  among  girls  (Children  Now,  2000).  Positwe  effects  of  viewing  television  have  also  been 
documented  through  the  demonstration  of  increased  prosocial  behavior  (Fairchild  1984)  and  the  development  of  academic  skills 
(Rice,  Huston,  Truglio  & Wright,  1990). 

Media  educates  both  intentionally  and  unintentionally.  Yet  not  enough  research  is  conducted  to  document  the  contents  of 
media  messages  and  examine  the  effects  of  media  exposure  and  possibilities  of  media  education.  Only  a few  countries,  such  as 
Canada  and  England,  have  recognized  the  need  for  formal  media  education  at  the  pre-college  level. 

Many  public  interest  groups  and  concerned  individuals  have  recognized  the  need  to  have  a media  savvy  society.  Groups 
like  Action  for  Children’s  Television,  The  Center  for  Media  Education,  Alliance  for  a Media  Literate  America,  National  Institute  on 
Media  and  the  Family,  and  Citizens  for  Media  Literacy  were  formed  to  have  greater  political  and  programmatic  influence  over  the 
production,  consumption  and  monitoring  of  media. 

The  Media  Literacy  movement  is  desperate  for  consistent  broad  research  and  data  collection.  In  only  a handful  of 
published  studies  have  researchers  taken  a sustained  look  at  a breadth  of  elements  and  their  relationships.  Research  has  focused  on 
analyzing  a specific  aspect  of  a particular  medium  during  a brief  time  period.  Quantification  and  deconstruction  of  various  media  has 
never  been  done  on  a large  scale  for  a sustained  time  period. 

Media  educator  Rene  Hobbs  emphasizes  the  skills  of  reading  and  writing  in  relation  to  media  literacy.  In  addition  to  data 
compilation,  MLT  focuses  on  the  reading  of  media.  Through  the  act  of  critical  observation,  participants  will  become  more  cognizant 
of  what  they  are  viewing.  Becoming  more  observant  and  critical  of  media  messages  perhaps  makes  one  more  resilient  to  media 
exposure.  As  Anderson  and  Ploghoft  (1993)  suggest,  awareness  individuals  build  allow  for  better  comparison  and  contrast  with 
every  day  life.  Pilot  research  will  explore  the  potential  of  critical  observation  in  viewing  and  its  influence  in  developing  media 
resiliency. 

Program  Design 

MLT  will  collect,  compile  and  organize  information  about  the  content  and  structure  of  various  media,  including  television, 
music,  movies,  magazines,  Internet,  videogames  and  advertisements.  The  databases  will  be  accessible  through  the  web,  enabling  a 
wide  range  of  individuals  to  access  and  contribute  to  the  data  and  fields,  and  perform  analyses  of  variables  across  different  media. 
Example  variable  sets  include  character  (such  as  species,  gender,  race,  age,  sexual  orientation,  occupation,  behavior  [acts  of 
kindness]),  setting  (indoor/outdoor,  view  of  sky),  time  (day/night,  sequence  of  events,  period,  season),  formal  features  (#cuts  per 


956 


second,  dissolves).  The  goal  is  to  compile  the  most  comprehensive  database,  tracking  all  messages  transmitted  through  various 
media,  while  fostering  more  critical  observation  of  television  content  and  greater  consciousness  of  emotional  reactions  in  media 
viewing  experiences . 

The  original  concept  for  MLT  arose  in  development  of ‘Television  and  Development  of  Youth”  online  (Kaplan,  Espinet, 
Mouza,  & Van  Nest,  1999)  and  was  discussed  in  a paper  on  tools  for  online  collaboration  (Mouza,  Kaplan,  & Espinet,  2000).  Rough 
drafts  of  MLT  were  piloted  by  the  Fall  1999  and  Fall  2001  classes.  Pooled  data  collected  during  individual  television  viewing  was 
informative,  documenting  ethnic,  gender  and  behavioral  trends  in  television  programming,  and  serving  as  acommimal  resource  in 
discussion  and  research.  Students  reported  that  observation  activity  led  to  greater  awareness  of  media  messages. 

MLT  will  include  several  interfaces  for  various  age  levels  including:  children  age  2-  5;  age  6-11;  adolescents  from  age  12- 
17;  and  adults  age  18  and  up.  Each  interface  will  be  designed  for  age  appropriateness,  informed  consent  will  differ,  and  certain 
variables  will  not  be  accessible  to  all  audiences.  Most  fields  will  be  common  to  all  users. 

MLT  encourages  community  data  collection.  One  pool  of  “data”  will  be  submitted  by  trained  coders.  Example  media  will 
be  viewed,  analyzed  and  discussed  in  order  to  achieve  the  highest  degree  of  inter-coder  reliability.  A second  pool  of  data  will  be 
contributed  by  participants  who  access  the  MLT  via  the  web.  A personal  profile  will  include  personal  characteristics  fields,  and  track 
submitted  data  (with  permission). 

Participants  will  respond  to  sets  of  fields  about  media  under  observation.  Fields  will  be  divided  into  sections  based  on 
characters,  action,  setting,  and  formal  features.  Databases  will  allow  for  comparison  and  compilation  of  fields.  Participants  will  be 
able  to  customize  the  interface  and  variables  sets  for  personal  observation  or  for  facilitation  of  use  by  a target  group,  curriculum  or 
research  study.  A media  diary  will  be  available  for  participants  to  indicate  what  shows,  songs,  etc.,  they  have  experienced  on  a 
particular  day,  and  thoughts  and  feelings  they  inspire.  A lesson  archive  will  provide  frameworks  around  which  to  design,  submit  and 
conduct  lessons  involving  viewing,  observing  and  analyzing  data.  Printable  forms  will  be  available  for  offline  work. 

The  project  team  plans  to  connect  with  several  other  database  projects  currently  under  development  at  Columbia 
University.  One  tool  under  development  by  Kaplan  and  Hasher,  the  Ecology  Observer,  will  facilitate  observation,  data  collection  and 
analysis  of  ecological  elements  in  real  spaces,  as  in  the  comer  of  120^  and  Broadway,  over  space  and  time.  The  combined  use  of 
both  tools  will  allow  for  comparisons  of  the  real  world  to  the  media  world. 
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Abstract.  GAIA  is  a microworld-based  learning  environment  aiming  to  provide  means  for 
exploratory  learning  to  secondary  school  students  in  the  fields  of  physics,  mathematics  and 
geosciences.  This  paper  presents  Gilbert,  one  of  GAIA^s  microworlds  aiming  to  familiarize  students 
with  contemporary  geosciences. 


Introduction 


Unlike  in  other  fields  of  studies,  such  as  physics  and  mathematics  (Thornton  and  Sokolo,  1990),  most  of 


oxro«1oV\1«> 


f/al  ot»/^  t/%  Th  o-rfli  tp  » /y**»  ^ *>* 

LJUltVVUlv/  1 V/lutV/U  tU  kll  Ovrll  W1  ij  klAKAXWO,  &0  WXCllwl  Ci/  X » » ML/ 


oriented,  where  students  are  presented  with  problems  to  solve  and  they  receive  feedback  about  the  accuracy  of 
their  responses,  or  iuior\2\-based  oriented,  following  the  archetypal  form  of  text,  graphics  or  multimedia 
information  associated  with  relevant  self-evaluation  tests.  However,  such  environments  (Kali  et  al  1997)  do 
not  strongly  facilitate  students  towards  constructing  their  own  knowledge  through  open-ended  experimentation 
(Jonassen,  2000).  On  the  other  hand,  microworld -based  educational  software  can  provide  the  means  for 
exploratory  learning,  by  providing  the  environment  for  students  to  leam-by-doing,  rather  than  just  watch  or 
listen  to  a description  of  how  something  operates.  Briefly,  a microworld  is  a computer-assisted  problem 
exploration  and  experimentation  space  in  which  one  can  explore  and  construct.  Typically,  they  are  constrained 
versions  of  reality  that  enable  learners  to  manipulate  variables  and  experiments  within  parameters  of  some 
system.  Furthermore,  the  most  advanced  micro  world -based  educational  software  provides  the  means  to 
educators  and  learners  to  construct  their  own  exploration  spaces.  This  is  this  the  case  of  GAIA  which  allows 
learners  to  represent  their  own  thinking  in  the  ways  that  they  explore,  manipulate  and  experiment  with  the 
environment. 


- Component«- 
based  Software 
Process 


Figure  1.  Overview  of  the  GAIA  Project  Design  and  Development  Process  (Papageorgiou  et  al,  2001) 
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GA/A  is  a microworld-based  exploratory  learning  environment  in  the  field  of  geosciences  education, 
physics  and  mathematics.  This  is  accomplished  through  constrained  simulations  of  real-world  phenomena  in 
which  students  can  navigate,  manipulate  or  even  create  objects,  and  experiment  with  their  effect  on  each  other. 
In  the  current  version  of  GA/A,  seven  microworlds  -named  after  great  scientists  - are  designed  and  developed  to 
provide  the  environment  for  replicating  the  essential  functionalities  needed  to  explore  phenomena  about  the 
Earth  and  the  planets,  regarding  their  interior,  atmosphere,  morphology,  magnetic  field,  their  orbits  in  space  as 
well  as  their  astronomical  environment.  Figure  1 gives  an  overview  of  the  GA/A  project  design  and 
development  process  (Papageorgiou  eta/,  2001).  The  development  of  GA/A  is  based  on  the  component-based 
educational  authoring  environment  “E-Slate”  (Birbilis  G.  et  al,  2000),  which  provides  a workbench  for  creating 
highly  dynamic  software  with  rich  functionality  (Roschelle  et  al,  1999). 

The  Gilbert  microworld,  presented  in  this  paper,  deals  with  Earth’s  magnetic  field,  within  the  frame  of  the 
school  curriculum.  This  microworld  aims  to  the  comprehension  of  Orth’s  magnetic  field  concept  and  how 
contemporary  science  studies  and  explores  this  field.  The  Earth’s  magnetic  field  is  one  of  the  most  attractive 
concepts  to  students’  imagination,  but  its  teaching  becomes  difficult  as  it  is  hard  for  them  to  resolve  its  vector 
within  the  3D  space  in  relation  to  geographical  coordinates,  and  to  precisely  comprehend  its  existence  as  they 
do  for  instance  with  the  concept  of  temperature.  The  Gilbert  microworld  contributes  to  all  the  aforementioned 
and  familiarizes  students  with  the  existence  and  nature  of  Earth’s  magnetic  field. 


Earth’s  magnetic  field  study  through  GAIA  software 

The  Earth’s  magnetic  field  can  be  directly  noticed  by  the  students  with  the  use  of  a magnetic  needle  and 
that  is  the  reason  why  they  must  be  familiarized  with  the  compass  use  in  school  laboratory.  This  is  usually  done 
through  the  measurement  of  the  field’s  direction.  Although  laboratory  measurement  is  useful,  it  limits  students 
to  identify  and  measure  earth’s  magnetic  field,  only  at  the  place  they  are. 

The  GA/A  interactive  software  presented  in  this  paper  allows  students  to  virtually  travel  everywhere  on 
Earth,  and  measure  earth’s  magnetic  field  elements  (Figure  2).  Students  have  also  the  possibility,  besides  the 
compass,  to  use  a total  magnetic  fie  Id  magnetometer.  The  selection  of  the  virtual  site  for  the  experiment  is 
accomplished  either  on  a 3D  rotating  globe  (see  Figure  2:  top  left  window)  or  on  a plane  geophysical  map.  The 
altitude  where  the  measurement  is  taken  place  is  set  in  another  window  (see  Figure  2:  top  right  window),  by 
pointing  the  mouse  on  a magnetometer  sketch  on  the  screen,  or  alternatively  by  regulating  a properly  designed 
“slider”  (see  Figure  2:  bottom  right  window).  The  perspective  of  earth’s  projection  is  done  in  such  a way,  as  to 
show  the  magnetometer’s  sketch  on  the  screen.  In  another  window  (see  Figure  2:  bottom  left  window)  students 
can  observe  and  use  three  instruments:  a horizontally  positioned  compass  that  measures  declination,  a compass 
placed  in  the  magnetic  meridian  plane  that  measures  magnetic  inclination  and  a total  field  magnetometer. 

In  parallel  to  the  virtual  measuring  of  the  field,  students  can  also  observe  the  resolved  components  of  the 
magnetic  field  vector  at  any  point  around  the  Earth  (see  Figure  2:  top  middle  window).  In  this  resolution  the 
quantities  of  field’s  intensity,  declination  and  inclination  are  clearly  noticeable.  The  students  have  also  the 
possibility  to  visualize  and  resolve  the  vector  within  the  Cartesian  system  coordinates.  In  addition,  they  can 
rotate  the  projection  of  the  vector  representation  for  more  convenient  observation  of  the  relative  angles  and 
vectors. 

At  a higher  knowledge  level,  students  can  also  change  the  value  of  the  measurement  date  parameter, 
provided  that  they  have  been  introduced  in  the  field’s  variation  with  time.  Of  course,  it  is  assumed  that  the 
students  have  also  learned  some  basic  notions  about  the  origin  of  the  Earth’s  magnetic  field.  Measurement 
values  are  simulated  to  real  ones  as  they  based  on  the  IGRF  (International  Geomagnetic  Reference  Model) 
geomagnetic  field  model  of  International  Association  of  Geomagnetism  and  Aeronomy  (lAGA  1995).  This 
model  is  designed  to  approximately  calculate  Earth’s  magnetic  field  from  Earth’s  interior  upwards  into  space 
without  taking  into  account  external  magnetic  field  sources.  It  is  based  on  the  usual  expansion  of  the  spherical 
harmonics  of  the  potential  in  geocentric  coordinates  (Campbell,  1997;  Lowrie,  1997)  and  describes  the  field’s 
variation  within  time.  At  regular  times,  new  spherical  harmonics  coefficients  supplement  it  in  order  to  renew 
and  fix  its  anticipation  in  the  future.  In  view  of  the  educational  purposes  for  which  the  model  was  used,  the 
magnetic  field  values  had  no  higher  accuracy  than  that  of  the  order  of  1 |i?. 
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Figure  2.  In  a special  configured  environment,  students  can  select 
whichever  place  of  the  world  they  like  on  a 3D  rotating  globe  or  on  a 
plane  map,  and  to  measure  the  magnetic  field’s  elements  at  a certain 
height,  meanwhile  they  can  see  the  field’s  vector  analysis. 


Figure  3.  A window  of  Gilbert 
micro  world  that  presents  Earth’s 
magnetic  field  main  dipole 
component.  Students  experiment  with 
placing  and  moving  a number  of 
compasses  and  finally  become 
accustomed  to  the  lines  of  force 
notion. 


The  dipole  component  of  the  Earth’s  magnetic  field,  which  consists  the  95%  of  the  total  field,  is  presented 
in  another  window  of  the  Gilbert  microworld  (Figure  3).  In  that  window,  students  can  test  the  field’s  existence 
around  the  whole  planet  Earth  by  a number  of  compasses  that  can  freely  move  into  space.  In  this  environment, 
students  can  easily  learn  the  lines  of  force  as  well  as  to  magnetic  poles  natural  position.  Field’s  lines  of  force 
may  be  displayed  at  student’s  choice. 

Displaying  the  three  elements  of  the  magnetic  field  (intensity,  declination,  inclination)  on  maps  (Figure  4), 
it  helps  students  to  realize  that  these  three  elements  coexist  in  every  point  and  are  modified  at  the  same  time. 
For  instance,  students  can  observe  the  magnetic  poles  in  the  three  maps,  to  compare  their  observations  and  to 
realize  that  in  all  of  these  three  plots  the  poles  have  characteristic  features.  Students  also  realize  that  in  many 
places  on  earth,  compass  points  out  to  a direction  deviating  considerably  from  the  North.  The  coincident  display 
of  these  three  maps  reveals  the  value  of  the  “multiple  representations”  as  well  as  the  “direct  manipulation”  of 
the  objects  (Teodoro,  1990;  Shama  and  Layman  1997). 

The  introduction  of  examples  from  an  Applied  Geophysics  geomagnetic  survey  was  one  of  the  essential 
innovations  of  GAIA  in  the  teaching  of  the  magnetic  field  in  secondary  education  level.  Beyond  the 
comprehension  of  notions  as  the  induced  magnetization,  the  addition  of  the  induced  field  vector  to  the  Earth’s 
field  and  the  magnetic  permeability  of  the  materials,  the  microworld  further  aimed  to  present  a simplified 
example  of  a practical  application  of  Earth’s  magnetic  field  measurements,  such  as  the  detection  of  mineral 
deposits  and  the  antiquities. 

In  the  environment  shown  in  Figure  5,  students  experiment  with  a buried  plate  of  square  or  rectangular 
section.  The  plate  is  considered  infinitely  elongated  in  order  to  simplify  the  problem  from  the  existence  of  edge 
anomalies.  The  plate  is  oriented  B-W  while  the  magnetometer  is  moving  and  measuring  vertically  to  the  plate, 
to  N-S  (Figure  6).  The  plate’s  magnetization  is  considered  only  induced  and  its  material  can  be  selected  out  of  a 
large  variety  of  geological  rocks  with  different  values  of  magnetic  permeability.  For  each  rock,  typical  values  of 
permeability  and  susceptibility  are  given,  in  companion  with  the  range  of  these  values  in  the  real  state. 
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Figure  4.  The  coincident  display  of  the  magnetic  maps 
facilitates  the  observation  and  modification  of  all  Earth’s 
magnetic  field  elements  at  the  same  time. 


Figure  5.  In  the  magnetic  survey  environment, 
students  can  expertise  with  the  buried  plate 
detection  by  the  implementation  of  a 
magnetometer.  They  can  also  select  the  material  of 
the  plate,  and  the  location  where  the  experiment 
takes  place.  Here  the  experiment  takes  place  with  a 
magnetite  plate  of  square  section,  at  an  area  close 
to  the  North  Pole. 


Although  the  virtual  experiment  can  take  place  everywhere  on  Earth,  we  suggest  the  choice  of  three  places 
with  characteristic  inclination  (?)  values.  These  are;  the  North  magnetic  Pole  (?  = 90\  the  magnetic  equator  (?  = 
0)  and  an  intermediate  place  like  North  Africa  with  inclination  equal  to  45  \ Anomalies  observed  in 
measurements  of  the  total  field  above  the  buried  plate  in  these  three  areas  are  advisable  for  the  induced 
magnetization  presentation  to  the  high  school  students. 


Figure  6.  The  orientation  of  the  buried  plate  is  E-W 
while  this  of  the  survey  traverse  is  N-S 


Figure  7.  Earth’s  magnetic  field  anomaly  caused  by 
the  induced  magnetization  of  the  square  section  buried 
plate  in  the  North  magnetic  Pole. 


3B§T  COPY  ATAIMBILK 


961 


At  the  North  magnetic  Pole  the  Earth’s  magnetic  field  vector  is  perpendicular  to  the  plane  of 
measurements.  Since  the  particle  magnets  of  a paramagnetic  material  are  polarized  according  to  the  field  (a 
diamagnetic  material  causes  the  opposite  effect)  the  induced  magnetization  will  be  also  perpendicular  to  the 
plane  of  measurements.  In  the  case  of  paramagnetic  materials,  which  are  the  most  common  in  nature,  the  plate’s 
induced  magnetization  is  shown  in  Figure  7.  The  lines  of  force  of  the  induced  magnetization  field  of  the  body 
are  symmetrical  relative  to  the  vertical  axis  that  passes  through  the  centre  of  the  body’s  section.  The  values  of 
the  measurements  are  increased  at  the  points  where  the  body’s  field  has  the  same  direction  with  Earth’s  field, 
while  are  reduced  where  the  two  vectors  are  opposed.  At  the  points  where  body’s  field  is  horizontal  (like  on 
body’s  sides),  no  essential  variation  is  resulted. 

In  magnetic  equator  (?  - 0^)  Earth’s  magnetic  field  vector  is  horizontal.  Following  the  aforementioned 
technique,  it  is  easy  to  find  out  that  the  diagram  of  the  measurements  is  also  symmetric  to  the  body  but  inversed 
in  sign  in  relation  to  the  one  of  the  North  magnetic  Pole  (Figure  8).  In  body’s  centre,  the  lines  of  force  of  the 
body’s  induced  magnetization  are  opposed  to  the  Earth’s  magnetic  field  and  thus  the  field’s  anomaly  is 
negative.  In  the  case  that  inclination  is  45^,  the  lines  of  force  are  symmetrical  to  body’s  diagonal  (Figure  9). 
The  magnetic  field’s  anomaly  has  an  asymmetrical  pattern  and  therefore  its  interpretation  regarding  body’s 
position  becomes  more  complicated.  The  point  where  Earth’s  field  vector  is  vertical  to  that  of  dynamic  lines  is 
just  above  the  body,  and  there  we  observe  that  field’s  anomaly  is  practically  zero.  With  exception  to  the 
magnetic  poles  and  equator  the  diagram  is  asymmetric  at  all  the  other  positions  (More  details  in  Dobrin  and 
Savit,  1988). 

By  the  final  stage  of  the  microworld  Gilbert  students  can  comprehend  the  notion  of  induced  magnetization 
and  how  it  is  depending  on  the  magnetic  properties  of  materials.  From  the  example  of  the  magnetic  survey  the 
students  are  introduced  to  the  field  of  Applied  Geophysics  and  its  contribution  to  the  detection  of  oil  deposits, 
minerals,  antiquities,  etc.  Contemporary  science  can  provide  students  with  many  such  examples,  possibly 
exciting  their  interest  for  acquiring  new  knowledge. 


Figure  8.  Earth’s  magnetic  field  anomaly  caused  by  Figure  9.  Earth’s  magnetic  field  anomaly  caused  by 
the  induced  magnetization  of  the  square  section  buried  the  induced  magnetization  of  the  square  section  buried 
plate  in  the  magnetic  Equator.  plate  in  position  with  magnetic  inclination  equal  to  45° 

(e.g.  North  Africa) 
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Conclusions 


The  GAIA  software  was  designed  and  developed  to  allow  to  operate  a virtual  educational  laboratory 
addressed  in  secondary  school  students.  Students  may  comprehend  basic  notions  of  Earth’s  magnetic  field  and 
become  accustomed  to  them  as  well  as  with  other  more  fundamental  notions  of  magnetic  field  theory,  such  as 
induced  magnetization.  A number  of  activities  based  on  Gilbert  microworld  have  been  already  created. 
Students,  working  in  teams  under  teacher’s  guidance,  can  enhance  their  knowledge  on  Earth’s  magnetic  field  by 
experimenting  with  Gilbert.  Initial  evaluation  of  students  performance  indicate  a significant  advantage  in  their 
understanding  on  basic  Geoscience  concepts  when  using  GAIA  as  compared  to  their  traditional  school  teaching 
methods. 
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Abstract:  The  Internet  remains  as  a wonder  for  the  century  and  its  growth  is  phenomenon. 
According  to  a recent  survey  the  online  population  is  now  about  500  million  globally  and  this  trend 
continues,  it  should  reach  700  million  by  the  end  of  2002.  This  exponential  growth  of  the  Internet  has 
given  rise  to  several  security,  privacy  and  ethical  concerns.  There  are  laws  governing  those  issues  at 
several  countries,  but  hard  to  apply  it  due  to  the  rapid  change  of  the  technology  and  also  security 
breach.  Internet  commerce  or  electronic  commerce  (e- commerce)  poses  constant  threats  to  privacy 
and  security.  The  web  has  become  a playground  for  lawbreakers.  The  aim  of  this  article  is  to  give  a 
snapshot  of  the  current  status  of  the  Internet  and  also  how  it  is  creating  a nightmare  for  governments 
to  find  a way  to  safeguard  both  consumers  and  providers  of  information  from  possible  misuse.  The 
author  argues  that  the  use  of  the  Internet  must  be  controlled  with  proper  legislation  to  minimize  its 
negative  impact  on  our  society.  There  should  be  an  international  law  and  or  a 3'^'*  party  monitoring 
authority  so  that  proper  protection  could  be  offered  to  the  users  of  this  ever-expanding  technology. 


Introduction 

The  rapid  development  of  information  technology  and  the  Internet  have  dramatically  increased  the  quantity  of 
information  available  in  digital  form.  This  has  resulted  in  a proliferation  of  uses  of  personal  information.  Some  of  these 
have  major  implications  for  the  privacy  of  individuals.  The  World  Wide  Web  (WWW  or  the  Web)  has  created  a totally 
new  global  business  culture  and  environment.  The  new  way  of  doing  business  across  the  globe  is  called  electronic 
commerce  (e -commerce)  or  online  or  Internet  commerce  or  business  by  computers  and  networks.  So  the  Internet  will 
create  a huge  market  in  cyberspace  and  carry  valuable  information  to  a large  number  of  people  worldwide.  This  will 
give  rise  to  a global  knowledge  based  economy  or  information  economy  (Karmakar,  2001a  & 2001b).  The  emerging 
electronically  networked-based  information  economy  will  affect  how  we  are  governed  and  how  we  live  (Kobrin  1998). 
The  growth  of  the  information  economy  also  means  that  there  are  new  threats  of  security,  privacy,  ethics  and  other  types 
of  online  harassment  such  as  fraud  and  deception. 

The  Internet  Statistics  in  a Nutshell 

The  convergence  of  information  and  communication  technology  (ICT)  is  transforming  most  aspects  of  business  and 
consumer  activities.  The  growth  of  the  Internet  is  dramatic.  According  to  a recent  survey  by  Nua  , Internet  population 
worldwide  as  of  August  2001  was  513.41  million.  The  same  survey  gives  the  estimated  number  of  online  population 
around  the  world  [Table  1]. 


Region 

Number  (million) 

Africa 

4.15 

Asia/Pacific 

143.99 

Europe 

154.63 

Middle  East 

4.65 

Canada  & USA 

180.68 

Latin  America 

25.33 

Table  1:  The  number  of  online  population  around  the  world 


* http://www_nua.ie/surveys/how_many_online/ 
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Electronic  Commerce:  A Challenge 


Electronic  commerce  will  significantly  increase  online  business  across  countries.  It  will  help  the  delivery  of  wide 
range  of  goods  and  services  in  a cost-effective  way.  With  e-commerce  set  to  top  SUSIOOO  billion  at  the  end  of  2002,  the 
potential  for  making  a fortune  by  exploiting  security  breaches  is  huge.  A recent  survey  of  Fortune  500  companies 
indicated  that  62  per  cent  of  firms  have  suffered  computer  break-ins  during  the  past  year.  Identity  theft  - the  use  of 
another  person’s  credit  card  number  and  personal  details  - is  on  the  rise,  and  threatens  the  viability  of  electronic  trading 
(Besserglik,  2000).  As  more  and  more  business  goes  online,  confidential  files  are  increasingly  exposed  to  the  risk  of 
infiltration. 

Cyber  crime  in  the  new  millennium 

Companies  worldwide  have  lost  $3  trillion  from  cyber  crime.  Requests  for  assistance  from  the  Australian 
Securities  and  Investment  Commission  have  increased  from  eight  to  200  in  the  past  two  years  (Cant,  2002).  Research 
shows  companies  fail  to  report  cyber  crime  for  fear  of  negative  publicity.  A survey  by  the  US  department  of  Defence 
Cyber  Crime  centre  showed  36  percent  of  companies  reported  such  crimes.  In  one  of  the  biggest  attempted  cyber-scams 
to  date,  a thief  stole  300,000  credit  card  number  from  Internet  music  company  CD  Universe  in  December  1999  and 
posted  them  on  a website  after  it  refused  to  pay  a SUSlOO,  000  ransom.  From  January  1 to  December  31,  2001,  federal 
ID  Theft  Data  Clearinghouse  received  86,168  reports  of  identity  theft  from  across  the  US,  with  the  District  of  Columbia 
and  California  being  the  main  hot  spots.  Other  uses  for  stolen  information  involved  employment  related  fraud  (9  per 
cent)  and  govemiiient  benefits  iraud  (6  per  cent)(Deanne,  2000).  Network  security  is  not  generally  very  strong,  and  web 
and  web  server  vulnerabilities  are  the  main  source  of  credit  card  information  breaches. 

Ethical  Issues  in  Cyberspace 

Ethics  is  at  the  heart  of  social  and  political  debates  about  the  Internet.  Ethics  is  the  study  of  principles  that 
individuals  and  organizations  can  use  to  determine  right  and  wrong  courses  of  action  (Laudon  & Traver,  2002). 
Companies  using  Web  sites  to  conduct  electronic  commerce  should  adhere  to  the  same  ethical  standards  that  other 
businesses  follow.  Ethical  considerations  are  important  in  determining  advertising  policy  on  the  Web.  Shea  (Shea,  1994) 
identified  10  Core  Rules  of  Netiquette: 


i. 

Remember  the  Human 

vi. 

Share  expert  knowledge 

ii. 

Adhere  to  the  same  standards  of  behaviour 
online  that  you  follow  in  real  life 

vii. 

Help  keep  flame  wars  under  control 

iii. 

Know  where  you  are  in  cyberspace 

viii. 

Respect  other  people’s  privacy 

iv. 

Respect  other  people’s  time  and  bandwidth 

ix. 

Don’t  abuse  your  power 

V. 

Make  yourself  look  good  online 

X. 

Be  forgiving  of  other  people’s  mistake 

Issues  on  Security  and  Privacy 

Security  and  privacy  are  interrelated.  Information  should  be  used  only  for  the  purpose  it  is  collected.  Many 
companies  are  using  the  Internet  with  caution  because  of  concern  about  network  and  transaction  security.  Similarly 
many  customers  do  not  feel  secured  to  making  payments  over  the  Internet.  As  the  information  concerning  credit  card 
numbers  or  other  confidential  records  traverses  on  the  Internet,  there  is  not  yet  a reliable  method  of  preventing  third 
parties  from  accessing  this  confidential  information.  A third  party  unlawful  intervention  arises  when  a hacker  or  some 
other  source  has  the  potential  to  interrupt  data  or  network  resources.  This  may  bring  to  a company  heavy  economic  loss, 
which  may  be  in  the  form  of  destruction,  disclosure,  modification  of  data  and  other  form  of  abuses.  In  order  to  allay 
consumer  fear  and  protect  the  confidential  data  on  public  networks,  companies  are  starting  to  pay  greater  attention  to 
transaction  privacy,  authentication,  and  anonymity. 

The  Internet  with  no  legal  boundaries  raises  concerns  about  personal  privacy.  The  security  and  privacy  issues  are 
very  serious  as  long  as  hackers  continue  to  hack  on  the  Net.  Personal  information  and  privacy  of  communications  must 
be  protected  without  any  question.  Business  and  consumer  concerns  about  security  are  legitimate  irrespective  of 
countries  we  live.  To  minimise  risks,  the  protocols  and  infrastructure  for  secure  transmission  of  information  are  in  place 
or  developing. 

Security,  Privacy  in  Cyberspace:  Legal  Framework 
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Real  space  means  our  physical  environment  consisting  of  temporal  and  geographic  boundaries,  where  as 
cyberspace  is  defined  the  realm  of  digital  transmission  not  limited  by  geography.  Privacy  online  is  a legal  issue  and 
there  is  difficulty  of  applying  traditional  law  to  the  Internet.  The  creation  and  use  of  knowledge  or  information  are  key 
economic  activities  all  over  the  world.  Sometimes  information  does  harm.  It  ruins  reputations,  exposes  personal  secrets, 
inflicts  emotional  injury,  and  misleads  people  into  mistaken  purchases  and  investments.  The  law  must  determine  who 
bears  the  risk  of  loss  from  such  harm-not  only  originators  and  victims,  but  also  among  originators,  victims,  and  all  the 
intermediaries  who  handle  injurious  information  (Perrritt,  Jr.  1996). 

Commercialization  of  the  Internet  has  created  a difficult  task  to  establish  an  international  legal  framework  to 
maintain  security  and  privacy  in  cyberspace.  There  should  be  technological  means  to  secure  data  on  the  net  and  also  a 
legal  safeguard  to  protect  individuals  from  the  possible  misuse  of  the  medium.  Electronic  commerce  presents 
fundamental  challenge  to  the  law.  The  suitable  law  should  tackle  the  threat  of  security  and  privacy  when  we  do  business 
online  in  the  digital  economy  (Karmakar  et  al.,  2001). 

Network  security  will  help  to  protect  privacy.  Legal  systems  must  be  adaptive  to  the  rapid  changes  of  technology. 
Existing  legal  frameworks  are  outdated  for  protecting  privacy  (Loudon  1 996).  Until  the  legal  environment  of  privacy 
regulation  becomes  clearer,  electronic  commerce  sites  should  be  conservative  in  their  collection  and  use  of  customer 
data.  Mark  Van  Name  and  Bill  Catchings,  writing  in  PC  Week  in  1998,  outlines  four  principles  for  handling  customer 
data  that  provide  a good  online  for  Web  site  administrators.  These  principles  include  (Schneider  and  Perry,  2000): 

• Use  the  data  collected  to  provide  improved  customer  service, 

• Do  not  share  customer  data  with  others  outside  your  company  without  the  customer’s  permission, 

• Tell  customers  what  data  you  are  collecting  and  what  you  are  doing  with  it, 

• Give  custoincrs  the  right  to  have  you  delete  any  of  the  data  you  have  collected  about  them. 


International  Dilemmas 

Surveys  have  found  that  privacy,  piracy,  and  pornography  are  the  most  serious  concerns  of  Internet  users.  The 
European  Union  has  established  strict  privacy  laws:  “all  members  countries  of  the  European  Union  must  restrict  the 
export  of  personal  data  to  countries  that  do  not  provide  privacy  protection  that  is  up  to  European  standards,  at  least  from 
October  1998  for  those  that  have  not  already  done  so,  and  the  restrictions  are  much  tougher  than  in  previous  European 
laws.”  The  European  Union’s  privacy  Directive  of  1995  is  the  most  significant  international  statement  of  information 
privacy  principles  since  the  early  1980s  [Greenleaf  1998].  The  Organisation  for  Economic  Cooperation  and 
Development’s  Guidelines  on  the  Protection  of  Privacy  and  Transborder  Flows  of  Personal  Data  ^ attempt  to  balance 
the  protection  of  privacy  and  individual  liberties  and  the  advancement  of  free  flows  of  personal  data  through  eight 
privacy  principles:  Collection  Limitation,  Data  Quality,  Purpose  Specification,  Use  Limitation,  Security 
Safeguards,  Openness,  Individual  Participation  and  Accountability  which,  if  observed,  are  supposed  to  guarantee  a 
free  flow  of  personal  information  from  other  OECD  countries[Greenleaf  1998].  All  25-member  countries  of  the  OECD 
have  adopted  the  Guidelines  [Tucker  1988].  Most  European  countries  have  passed  laws  for  the  public  and  private 
sectors  based  on  the  OECD  principles.  New  Zealand,  Hong  Kong  and  Taiwan  also  have  privacy  laws  that  apply  to  both 
public  and  private  sectors.  Other  countries  like  Australia^  have  implemented  them  in  part  only. 

The  Council  of  the  OECD  has  adopted  Guidelines  for  a Cryptography  Policy  in  March  1997^^,  setting  out  criteria  for 
encryption  of  computerised  information  for  governments  to  adopt  and  for  businesses,  individuals  and  law  enforcement 
officials  to  apply  in  safeguarding  electronic  transactions,  communications  and  data  storage. 

The  U.S.  Government’s  Framework  for  Global  Electronic  Commerce  calls  for  governing  commercial  transactions 
on  the  Internet  ‘consistent  principles  across  state,  national,  and  international  borders  that  lead  to  predicable  results 
regardless  of  the  jurisdiction  in  which  a particular  buyer  or  seller  resides’  (Clinton  & Gore  1 997).  Europe  an -American 
disagreement  on  encryption  and  piracy  may  lead  to  a constant  problem  to  introduce  a uniform  safeguard  (Kobrin  1998). 
An  increase  in  the  use  of  tools  such  as  E-mail,  data  bases  and  computer  networks  has  Canadian  companies  generally 
feeling  more  exposed  when  it  comes  to  the  security  of  internal  communications  and  proprietaiy  information  (Church 
1 998).  Northern  Telecom,  based  in  Toronto,  Canada  has  programs  to  ensure  that  its  employees  realize  how  important  it 
is  to  safeguard  sensitive  information.  It  also  conducts  physical  and  electronic  audits  to  ensure  that  important  papers 
aren’t  left  on  desks. 


^ http://euroDa.eu.int/comm/dg  1 5/en/media/dataprot/pri v.htm 

^ The  1 1 Information  Privacy  Principles  in  the  Privacy  Act  1988(Cth)  are  intended  to  implement  the  OECD’s  8 principles  insofar  as 
personal  information  held  by  Commonwealth  public  sector  agencies  are  concerned.  Australia  has  still  failed  to  comply  with  the 
guidelines  for  thirteen  years  after  announcing  its  adherence  in  1 984. 

^ http://www.oecd.org/dsti/sti/it/secur/index.htm 
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The  global  Internet  governance  is  a big  issue.  Under  the  present  system,  an  Australian  company  in  dispute  with  a 
multinational  company  over  a domain  might  have  to  fight  that  battle  in  a US  court.  There  is  now  a genuine  concern  that 
Australian  companies  are  going  to  be  faced  with  having  to  go  US  courts  to  have  their  Internet  disputes  resolved  (Riley 
1998).  The  law  should  respond  to  business  concern  in  the  information  economy.  The  existing  law  cannot  cope  with  the 
transborder  data  flow.  Technology  and  business  are  changing  very  rapidly,  but  law  is  somewhat  static.  Many  business 
firms  are  very  adaptive  to  rapidly  changing  global  environment  due  to  the  information  revolution  (Johnston  et  al.  1997). 

Privacy  & Australia 

The  Commonwealth  Privacy  Act  1988^  lays  down  strict  privacy  safeguards  which  Commonwealth  (federal)  and 
ACT  government  agencies  must  observe  when  collecting,  storing,  using  and  disclosing  personal  information.  The  Act 
also  gives  individuals  access  and  correction  rights  in  relation  to  their  own  personal  information.  The  Act  applies  to  the 
wider  community  (including  the  private  sector  and  state  and  local  governments)  only  in  relation  to  specific  categories  of 
information:  tax  file  number  information  and  consumer  credit  information. 

Privacy  issues  arise  in  a wide  range  of  areas  and  circumstances.  Privacy  legislation  deals  mainly  with  information 
privacy  - the  handling  of  personal  information.  Other  privacy  issues  such  as  video  surveillance,  telephone  interception 
or  'bugging',  and  other  laws  may  cover  physical  intrusion  into  private  spaces. 

In  December  2000,  the  Privacy  Amendment  (Private  Sector)  Act  2000  (the  Amendment  Act)  was  passed  by  federal 
Parliament.  The  private  enterprises  became  subject  to  privacy  laws  on  December  21,  2001.  The  application  of  the 
Privacy  Act  is  triggered  by  the  financial  size  of  the  organization  and  the  nature  and  use  of  the  data  it  collects  or 
processes.  It  is  also  subject  to  some  ad  hoc  exceptions  and  differing  start  dates.  The  National  Privacy  Principles  (NPPs) 
in  the  Privacy  Act  set  out  how  private  sector  organisations  should  collect,  use,  keep  secure  and  disclose  personal 
information.  The  principles  give  individuals  a right  to  know  what  information  an  organisation  holds  about  them  and  a 
right  to  correct  that  information  if  it  is  wrong. 

The  surveys  showed  that  Australians  regard  privacy  as  a closely  held  and  highly  personal  value.  People  look  for 
signals  that  an  organisation  will  manage  their  personal  information  well,  for  example,  59%  said  they  would  trust  an 
organisation  more  if  that  organisation  gave  them  control  over  how  their  information  was  to  be  used,  55%  said  that 
organisations  with  privacy  policies  would  be  more  likely  to  gain  their  trust.  Various  government  bodies  have  been 
involved  in  projects  and  activities  designed  to  encourage  the  uptake  of  E-commerce  and  Electronic  Service  Delivery. 

Among  critics  of  the  private  sector  privacy  regime  has  been  the  European  Commission  (EC),  which  commented 
unfavorably.  In  essence,  the  EC  said  it  did  not  regard  Australia’s  privacy  regime  as  sufficiently  protective  on  about  nine 
grounds.  All  in  all,  as  noted  at  the  time  of  is  passage  into  law,  the  application  criteria  is  open  to  criticism  for  being 
arbitrary  and  the  wording  of  the  provisions  makes  it  less  than  crystal  clear  how,  or  even  if,  it  is  to  apply  in  given  cases, 
especially,  to  small  businesses  (Minahan,  2002). 

Australia  should  be  prepared  to  keep  pace  with  technological  innovation  and  join  the  new  global  business 
environment  if  it  is  to  remain  competitive  internationally.  Australian  Business  law  in  the  current  form  cannot  cope  with 
the  contemporary  business  operations.  The  technological  change  is  revolutionary,  but  the  change  in  the  legal  system  is 
evolutionary.  There  should  be  a number  of  changes  in  Australian  legal  systems  to  accommodate  electronic  commerce. 
Under  Australian  law,  if  someone  steals  a credit  card  it  is  not  an  offence  until  the  card  is  used.  But  in  the  US  it  is  against 
the  law  to  misuse  an  identity  with  the  intention  of  committing  a crime. 

Concluding  Remarks 

We  have  introduced  debate  concerning  Internet  regulation.  The  U.S.A  is  the  key  player  to  bring  about  any  effective 
change.  There  is  a need  to  keep  clean  our  online  information  environment  through  generally  accepted  international  laws 
or  other  regulatory  mechanisms.  The  Internet  is  global,  multi-jurisdictional  structure;  national  legal  systems  will  not 
control  the  uses  and  abuses  of  Internet  use.  The  continuing  growth  of  the  Internet  has  seen  a corresponding  growth  in 
concern  about  online  ethical  privacy  and  security.  Surveys  continue  to  show  that  users  are  concerned  about  the 
collection,  security,  use  and  disclosure  of  information  about  them  on  the  Net. 


^ http://www.privacy.gov.au 
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ABSTRACT 

Curricular  goals  that  traditionally  focused  on  the  delivery  of  the  teaching  rather  than  the  outcomes  of 
the  learning  are  in  a state  of  major  transition.  Implementation  of  academic  standards  requires  the 
application  of  content  and  not  merely  its  delivery.  As  a result,  colleges  of  education  must  prepare  a 
new  genre  of  teacher  leaders  equipped  to  align  students’  classroom  experiences  with  the  changing 
realities  of  performance  learning  in  an  information  era.  This  presentation  focuses  on  preparing 
teachers  who  can  link  electronic  information  resources  with  engaging  learning  experiences.  The  work 
is  grounded  in  the  beliefs  of  cognitive  theory  and  applied  practice.  It  presents  educational  media  as  a 
support  structure  in  the  curricular  architecture  of  teaching  and  learning  rather  than  as  an  ancillary 
element.  It  specifically  focuses  on  preparing  educators  to  motivate  their  students  through 
comparative,  investigative,  and  collaborative  problem  solving. 


1.  INTRODUCTION 

Words  such  as  excellence,  quality,  equity,  and  opportunity  dominate  the 
rhetoric  of  school  reform.  But  behind  the  words  is  a serious  confrontation  to 
rethink  how  we  organize  learning  opportunities.  Possibly  nothing  has  more 
dramatically  challenged  curriculum  construction  than  the  accessibility  and 
availability  of  electronic  resources. 

Much  of  my  work  over  the  last  decade  has  addressed  educational  change  in 
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de-Sovietizing  environments  in  East  Central  Europe.  In  that  transitional  time  I 
observed  teachers,  administrators,  and  policy  makers  struggling  with  what  they 
called  the  ‘tuimel  at  the  end  of  the  light.’  In  this  frequently  used  metaphor,  the 
light  symbolized  political  changes  that  externally  were  cause  for  celebration.  The 
tunnel  symbolized  the  internal  challenges  and  the  new  mismatch  of  learning  and 
life.  For  although  simultaneous  political,  economic,  and  educational  transitions 
created  new  opportunities,  they  also  created  new  challenges  regarding  the 
organization  of  schooling  and  the  acquisition  of  information.  Presently  my  work 
environment  is  an  American  college  of  education.  But  although  the  geography  of 
my  attention  has  shifted,  the  circumstances  that  influence  new  alignments  between 
school  and  society  is  marked  by  similar  uncertainty  regarding  the  realignment  of 
learning  experiences  in  a digital  era. 

2.  GOALS  OF  TECHNOLOGICAL  IMPROVEMENT:  AN  ISSUE  OF  USE 

Policy  makers  in  many  nations  portray  social  transitions  to  an  information 
era  and  school  improvement  as  associated  concerns.  As  a result,  schools  on  a 
global  basis  are  in  the  midst  of  technology  related  refoniis  and  their  associated 
challenges.  Linda  Darling-Hammond  (1997,  p.2)  aptly  captures  the  cultural 
climate  of  this  transitional  time: 

“Never  before  has  the  success,  perhaps  even  the  survival,  of  nations 

and  people  been  so  tightly  tied  to  their  ability  to  learn. 

Consequently,  our  future  depends  now,  as  never  before  on  our 

ability  to  teach.” 

The  teachers  at  the  core  of  these  changes  are  realizing,  however,  that  successful 
past  practices  may  be  incongruent  or  even  obsolete  in  electronically  rich 
environments.  If  teachers  are  indeed  “navigators”  (Bmer,  1994)  into  the  future, 
then  the  altered  fit  of  technology,  teaching,  and  learning  presents  a contemporary 
navigational  challenge. 

The  scope  of  technologies  that  have  historically  impacted  education  is  vast. 
This  work  concentrates  only  on  one  technology  that  has  dramatically  altered 
available  resource  information  in  the  past  decade,  the  Internet.  Although  the 
Internet  provides  a powerful  platform  for  accessing  information,  the  educational 
impact  of  that  body  of  information  is  dependent  on  its  use. 

3.  DILEMMAS  REGARDING  TECHNOLOGY  INVESTMENT 

At  both  national  and  community  levels,  increasing  expenditures  are  being 
appropriated  to  enhance  student  learning  through  technology.  But  unless  such 
efforts  give  adequate  attention  to  the  changing  alignment  between  resources  and 
applied  knowledge,  such  expenditures  are  questionable  (Dede,  1997). 

American  studies  indicate  that  over  seventy-five  percent  of  the  general 
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public  believe  that  technology  improves  both  teaching  and  learning  (Trotter, 

1 997).  But  teachers  who  have  Internet  access  do  not  necessarily  change  their 
pedagogical  strategies  (Johnston,  1997).  And  student  learning  does  not  increase 
in  relationship  to  time  spent  with  electronic  resources,  but  rather  in  the  manner  in 
which  new  information  is  linked  to  problem  solving  (ETS,  1998).  If  we 
contemplate  similar  relationships  for  networked  learning  without  attention  to  how 
teachers  can  most  effectively  use  this  resource  as  a means  to  better  thinking,  access 
to  vast  information  banks  may  have  limited  impact  on  student  achievement.  So 
what  should  we  do  differently?  Papert  (1998)  contends  that  we  must  stop  trying 
to  merely  improve  current  practices,  that  we  must  fundamentally  change  how  we 
organize  learning  opportunities  to  incorporate  technological  tools  rather  than 
teaching  about  them. 

4.  CHALLENGES  TO  EFFECTIVE  TEACHING  AND  LEARNING 

Studies  indicate  that  while  most  teachers  have  incorporated  technology’s 
storage  capabilities,  they  have  not  fully  capitalized  on  its  capacity  to  motivate 
knowledge  construction  and  to  facilitate  problem-solvmg  (Nicaisc  and  Barnes, 
1996;  Perkins,  1992a).  These  two  elements,  constructing  new  knowledge  and 
engaging  students  in  the  application  of  information  to  problem  solving,  are  the 
focus  of  the  following  examples  that  share  the  use  of  Internet  resources  in  a 
manner  that  is  interesting,  sustaining,  and  rewarding  for  both  student  and  teacher. 

5.  SAMPLE  EXPERIENCES  BASED  ON  ELECTRONIC  RESOURCES 

Because  of  the  dire  statistics  about  American  teachers’  knowledge  of 
geography,  these  examples  grew  from  a core  assignment  requiring  teachers  to 
construct  new  knowledge  about  a global  region  about  which  they  previously  knew 
little  or  nothing.  The  follow-up  assignment  involved  translating  their  new  learning 
as  adults  into  problem  related  experiences  for  students.  Several  examples  follow. 

In  an  electronic  trip  to  France,  one  educator  used  her  own  new  knowledge 
about  art  as  a means  to  motivate  the  construction  of  historic  and  cultural 
knowledge  that  incorporated  a multitude  of  Internet  museum  sites.  Through 
focusing  on  a selection  of  chosen  works  of  Impressionist  art,  she  asked  her 
students  to  hypothesize  about  the  cultural  climate  of  that  era.  Students  themselves 
were  then  required  to  explore  additional  sites  to  gather  more  information  through 
which  they  could  support  or  negate  ideas  they  had  formulated.  An  extension 
opportunity  encouraged  students  to  further  examine  the  life  and  work  of  an  artist 
of  their  choice  and  compare  it  with  the  same  art  forms  on  other  continents  during 
that  time  frame. 

An  electronic  field  trip  to  Poland  enabled  another  teacher  to  inspire  his 
students  to  create  comparative  journals  for  an  American  teen  living  in  the 
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Appalachian  Mountains  and  a Polish  teen  in  the  Carpathian  Mountains.  Resources 
reprinted  from  the  Internet  ensured  that  students  would  have  accurate  data  for  this 
literacy-based  activity  that  integrated  written  communication  skills  with  geological 
and  cultural  information  sources. 

Another  prospective  educator  used  her  electronic  trip  to  Switzerland  as  the 
foundation  for  an  iimovative  health  ixnit  that  examined  the  effects  of  altitude  on  the 
human  body.  An  electronic  field  trip  to  Kenya  led  to  a lesson  that  asked 
students  to  first  use  Internet  resources  to  explore  actual  habitats  and  then  use  data 
to  assess  whether  commimity  proposals  for  a new  wildlife  sanctuary  were 
scientifically  grounded.  A less  traditional  journey  involved  applying  Internet 
learning  from  the  NASA  site  to  design  a playground  in  space  addressing  the 
particular  problem  solving  challenges  in  building  a playground  in  zero  gravity. 

A final  case  exemplifies  how  teachers  can  use  Internet  resources  to 
motivate  the  synergy  that  results  when  students  assume  not  only  greater 
responsibility  for  their  individual  learning,  but  also  contribute  in  a manner  in  which 
the  whole  class  benefits  from  the  combined  efforts  of  its  members.  Their  teacher 
used  Internet  resources  to  explore  the  themes  of  economic  production, 
distribution,  and  consumption  and  to  apply  them  in  environmental  relations  within 
ecosystems.  Students  discovered  how  bacterium  in  geysers  contributes  to  medical 
research  on  Lyme  disease  and  AIDS,  and  they  evaluated  conflicting  interests 
introduced  by  mining  and  the  economic  needs  of  communities  surrounding 
protected  environmental  areas.  As  a follow  up  on  the  economic  effects  of  oil 
spills,  students  used  the  Internet  to  share  findings  about  how  negative 
circumstances  such  as  environmental  disasters  for  some  populations  can  create 
opportunity  for  other  groups,  such  as  new  financial  markets  in  environmental 
clean-up. 

Traditionally,  when  teachers  sent  students  to  the  library  for  independent 
research,  most  came  back  with  similar  information  from  the  same  basic  sources. 
The  richness  of  electronic  resources  provides  opportunity  for  more  expansive  and 
encompassing  learning  opportunities  for  students  if  their  teachers  rethink  the 
traditional  construction  of  teaching  and  learning  experiences. 


6.  FINDINGS  AND  CONCLUSIONS 

Although  no  generalizations  regarding  variances  in  academic  achievement 
can  be  drawn  from  these  examples  implemented  in  different  school  settings,  one 
recurrent  comment  from  other  educators  who  observed  and  supervised  the 
redesigned  use  of  resources  was  how  they  enabled  students  to  target  particular 
standards  for  the  social  studies  with  greater  clarity.  Of  ten  particular  national 
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standards  for  the  social  studies,  four  performance  areas  were  noted  as  being 
apparent  to  students.  These  were:  1)  global  connections,  2)  time,  continuity,  and 
change,  3)  people,  places,  and  environments,  and  4)  production,  distribution,  and 
consumption. 

The  maimer  in  which  teachers  align  technology  with  curricular  goals  is 
largely  dependent  on  the  beliefs  and  experiences  established  in  teacher  preparation 
programs.  If  graduates  of  universities  and  colleges  are  to  be  effective  navigators 
for  their  students,  then  they  must  become  comfortable  and  competent  in 
transforming  large  networks  of  mere  information  into  significant  problem  solving 
experiences. 

The  process  of  redesigning  the  learning  process  in  any  nation  must  begin 
with  the  adult  learners,  the  teachers.  In  order  for  educational  change  to  be 
institutionalized  it  must  be  part  of  the  teachers’  foundational  competencies 
(Hargreaves,  1992,  Greenfield,  1995,  Fullan,  1993).  Just  as  successful  businesses 
need  workers  who  are  better  thinkers,  so  too  do  educational  enterprises  need  both 
learners  and  leaders  who  use  technology  to  bccomic  better  thinkers  (Crouter  and 
Manke,  1994).  These  projects  reflect  the  juxtaposition  of  electronic  information 
and  the  thinking  process.  They  use  computers  for  what  they  do  best,  storage  and 
organization,  and  the  human  mind  for  what  it  does  best,  judging,  interpreting, 
analyzing,  synthesizing,  and  constructing  meaning  (Jonassen,  1995).  They  present 
ideas  that  foster  connections  between  information  and  thinking  rather  than 
fragmented  and  isolated  approaches  that  mark  failed  educational  changes  (Fullan, 
1996). 


Although  the  long-term  impact  of  cyberspace  learning  on  student 
achievement  is  yet  to  be  determined,  the  integration  of  Internet  resources  gives 
teachers  the  opportunity  to  significantly  alter  curricular  construction. 

While  the  Internet  has  the  potential  to  bring  the  world  into  the  classroom,  quality 
teaching  results  from  quality  thinking.  Teachers  who  successfully  navigate 
journeys  into  more  electronic  environments  are  those  whose  old  way  of  thinking 
become  transformed  and  who  see  themselves  as  both  learners  and  leaders,  as 
change  agents,  and  as  individuals  who  can  consistently  add  value  to  their 
profession. 
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Abstract:  This  paper  presents  a review  of  the  Virtual  Factory  Teaching  (VFTS)  project,  a 
web-based,  multi-media  collaborative  learning  network.  The  system  allows  students, 
working  alone  or  in  teams,  to  build  factories,  forecast  demand  for  products,  plan 
production,  establish  release  rules  for  new  work  into  the  factory,  and  set  scheduling  rules 
for  workstations.  Included  in  the  paper  are  a system  description  and  list  of  project  tasks. 
The  evaluation  component  involving  three  different  campuses  is  described  and  the  results 
of  three-years  of  analyses  are  presented,  including  demographic  descriptions,  self- 
assessment  results,  performance  results,  attitudinal  responses,  and  usability.  In  addition, 
instructor  observations  and  course  project  output  is  also  examined  as  components  of  the 
VFTS  evaluation  effort.  It  is  believed  through  the  analysis  that  the  VFTS  is  an  excellent 
instructional  method  to  teach  students  the  integration  of  the  different  modules  in  operations 
planning. 


Background 

To  address  the  manufacturing  educational  needs  of  new  engineers,  a web-based,  multi-media 
collaborative  learning  network,  referred  to  as  a Virtual  Factory  Teaching  System  (VFTS),  was  developed 
under  an  National  Science  Foundation  (NSF)  grant  (Dessouky,  et.  al  1998;  Dessouky,  et.  al.  2001; 
Kazlauskas,  et.  al.  2000;  Kazlauskas,  2001).  This  tool  is  currently  being  used  by  engineering  students  from 
the  University  of  Southern  California,  San  Jose  State  University,  the  University  of  Virginia,  and  most 
recently  North  Carolina  Agricultural  and  Technical  State  University.  The  overall  aims  of  the  project  are: 

• To  provide,  disseminate,  and  evaluate  a manufacturing  education  pedagogical  tool  that 
promotes  student  understanding  of  complex  factory  dynamics. 

• To  improve  student  skills  in  communication,  persuasion,  negotiation,  and  management,  as 
well  as  in  the  technical  arenas  of  production  planning,  forecasting,  simulation,  scheduling, 
and  integration. 

• To  provide  a forum  in  which  engineering  and  business  school  students  can  participate  in 
virtual  teams  that  cut  across  universities. 

The  research  plan  of  the  project  is  aimed  at  exploring  the  interface  between  virtual  factories, 
engineering  education,  intelligent  agents,  and  the  Internet  for  new  ways  of  teaching  modem  manufacturing 
problems,  practices,  theory,  and  techniques  to  engineering  and  business  undergraduate  students.  In 
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addition,  it  is  aimed  at  examining  its  potential  for  use  as  an  information  vehicle  on  the  topic  of 
manufacturing  for  K-12  students.  Various  research  questions  are  addressed,  such  as:  how  students  perform 
when  using  new  technology-enhanced  modes  of  learning;  what  are  the  effects  on  attitudes;  how  intelligent 
agents  might  assume  tutoring  and  participative  roles;  and  how  team  performance,  hampered  by 
geographical  separation,  might  be  enhanced  via  advanced  communication  technologies. 

The  project  timetable  consisted  of  the  following:  a Baseline  period  to  provide  an  understanding  of 
the  Industrial  Engineering  courses  (with  scheduling  content)  using  traditional  instructional  methodologies, 
i.e.  without  the  VFTS;  the  use  of  the  VFTS  to  provide  an  understanding  of  the  Industrial  Engineering 
courses  (with  scheduling  content)  offered  at  the  various  institutions;  then  the  use  of  an  expanded  version  of 
the  VFTS  at  the  same  institutions  with  pedagogical  agents  and  use  of  virtual  teaming  included.  The 
following  presents  an  overview  of  the  features  of  the  VFTS;  a description  of  the  evaluation  approaches;  and 
three-year  project  results. 


System  Description 

The  architecture  of  the  VFTS  was  kept  simple  and  modular.  There  are  three  layers  in  the  design: 
AweSim  Server,  VFTS  Java  Server  and  Clients.  Clients,  which  are  students  in  our  case,  use  standard 
WWW  browsers  like  Microsoft  Internet  Explorer,  to  connect  to  the  VFTS  Java  Server  using  its  Web  Page. 
Most  of  the  communication  between  the  clients  and  the  server  takes  place  using  Java  applets.  The  Java- 
Server  functions  as  a mediator  between  the  AweSim  factory  servers  and  the  clients.  The  Awesim  Server  is 
responsible  for  factory  knowledge  and  simulation.  The  layers  interface  using  a message  protocol  set  up  to 
minimize  bandwidth  requirements.  Other  components  of  the  VFTS  include  the  use  of  Ptolemy,  a graphing 
package,  and  Drasys,  an  Operations  Research  (OR)  package. 

The  system  allows  students,  working  alone  or  in  teams,  to  build  factories,  forecast  demand  for 
products,  plan  production,  establish  release  rules  for  new  work  into  the  factory,  and  set  scheduling  rules  for 
workstations.  They  can  run  simulations  where  an  animated  panel  displays  jobs  progressing  through  their 
factory,  with  queue  counts,  finished  goods  counts,  graphs,  and  reporting  functions  all  available.  Students 
access  via  the  VFTS,  using  computers  in  university  computer  labs,  or  in  their  dorms  or  homes,  a virtual 
representation  of  the  factory.  The  professor  posts  assignments  related  to  this  factory  over  the  Internet, 
“unfreezing*’  parameters  as  necessary  so  students  may  experiment  without  redefining  the  entire  factory. 
Students  observe  the  effects  of  their  decisions,  and  student  teams  assume  factory  roles  to  solve  problems;  if 
they  reach  an  impasse,  intelligent  agents  provide  guidance.  Selective  information  may  be  given  based  on 
student  roles;  for  example,  the  production  supervisor  may  have  equipment  information,  the  engineer  new 
technology  information,  etc.  Students  sort  out  strategies  and  can  discuss  options  via  e-mail  and  electronic 
chat  rooms.  Since  this  course  is  a common  one  found  in  many  universities,  collaboration  among 
universities  is  feasible.  Faculty  members  virtually  “team  teach”  the  course.  Intelligent  agents  are 
incorporated  into  the  VFTS  to  monitor  student  progress  and  provide  immediate  feedback. 

The  latest  version  of  the  VFTS  software  includes  support  for  user  account  management,  adding 
security  to  allow  students  within  a group  to  share  data  while  preventing  access  to  it  from  students  outside 
the  group,  a more  easy  to  use  interface,  instrumentation  so  that  the  software  will  gather  data  on  student 
usage  patterns,  on-line  help,  on-line  documentation  and  an  on-line  tutorial  to  help  for  students  and  faculty 
learn  how  to  use  the  VFTS,  and  an  introductory  homework  exercise  to  help  students  learn  how  to  integrate 
the  VFTS  into  the  course.  A project  that  uses  the  VFTS  was  developed  to  complement  the  students’ 
classroom  learning  and  was  integrated  into  the  courses  at  the  participating  universities.  The  latest  version 
also  includes  a pedagogical  agent  that  monitors  students'  use  of  the  VFTS  and  provide  guidance.  This 
required  integrating  the  pedagogical  agent  software  (ALI)  into  the  VFTS,  adding  the  ability  for  ALI  to 
maintain  a persistent  model  of  each  student's  knowledge  across  sessions,  extending  All  to  include  more 
sophisticated  explanation  capabilities,  adding  VFTS-specific  knowledge  to  allow  ALI  to  understand  the 
instructional  objectives  of  the  VFTS  and  provide  appropriate  support  to  students,  and  instrumenting  ALI  to 
maintain  a log  of  interactions  with  students  to  aid  our  evaluation. 

The  current  version  VFTS  is  available  at  http://vfts.isi.edu. 


Project  Tasks 

The  VFTS  project  included  a large  set  of  tasks  involving  design,  development,  usage,  evaluation, 
and  dissemination.  The  various  tasks  associated  with  the  VFTS  project  are  as  follows: 

■ Define  instructional  objectives  and  complete  evaluation  design. 

■ Develop  and/or  acquire  evaluation  instruments. 

■ Solicit  feedback  on  the  instructional  objectives,  evaluation  design,  and  VFTS  use  from  each 
university. 

■ Gather  evaluation  data  for  control  group  (engineering  classes  without  the  VFTS). 

■ Analyze  evaluation  results  for  control  groups. 

■ Complete  instrumentation  of  the  software  to  support  evaluation 

■ Teach  engineering  classes  with  the  VFTS  and  gather  evaluation  data. 

■ Analyze  evaluation  results  for  experimental  group  and  compare  to  control  group. 

■ Use  the  evaluation  results  to  revise  and  refine  the  VFTS  and  its  use  in  the  engineering 
curricula,  its  instrumentation,  and  the  pedagogical  agents. 

■ Teach  engineering  classes  with  the  VFTS,  including  the  use  of  virtual  teams  that  span 
multiple  universities,  and  gather  evaluation  data  for  this  final  experimental  group. 

■ Analyze  summative  evaluation  results. 

■ Make  final  revisions  to  the  VFTS  software  based  on  evaluation  results. 


Evaluation 

In  the  preceding  list  of  project  tasks,  it  should  be  noted  that  considerable  emphasis  is  placed  on  the 
design  and  implementation  of  an  extensive,  multi-university  evaluation,  which  is  a central  component  of 
the  VFTS  project.  The  initial  evaluation  efforts  were  used  a)  to  check  to  make  sure  each  project  step  was 
implemented  according  to  plan  and  that  milestones  are  being  met;  and  b)  to  consider  project  modification. 
This  evaluation,  formative  in  nature,  assisted  in  modifying  the  design  and  development  of  the  evaluation,  as 
well  as  of  the  overall  design  of  the  web-based  VFTS.  For  example,  a Computer  Competencies  Multiple 
Domains  instrument  that  was  used  initially  to  determine  the  level  of  entering  competency  in  such  areas  as 
spreadsheets  and  statistical  packages  was  dropped.  It  was  determine  through  initial  surveys  that  the  learners 
were  homogeneous  in  terms  of  a high  level  of  computer  competencies.  Through  the  evaluation  process,  it 
was  decided  to  gather  more  data  on  such  issues  on  the  ease  of  use  and  operation  of  the  web-based  VFTS, 
and  to  modify  the  actual  design  by  changing  the  opening  screen,  and  by  including  more  tutorial  help. 

An  emphasis  of  this  project  is  to  evaluate  learner  outcomes,  attitudes,  and  learning.  This  effort 
includes  the  use  of  various  instruments  and  examination  of  student  work. 

The  following  are  the  instruments  that  are  presently  being  used  in  evaluation: 

• Student  Pre -course  and  Post-course  Survey  instruments  which  are  used  to  gather  data  on 
student  demographics  and  student  self-rated  assessment  of  entering/post-course  knowledge  of 
operations  scheduling,  such  as  the  skills,  knowledge,  and  abilities  in  the  use  of  gantt  charts 
and  regression  and  time  series  models . 

• Operations  Scheduling  Pre-test/Post-test  instruments  which  are  used  measure  course  content 
before  and  after  instruction  and  use  of  the  VFTS. 

• Affective-Level/Attitudes  Instrument  (VFTS  Participant  Opinion  Survey)  Pre-test/Post-test 
instrument  which  are  used  to  measure  reactions  to  the  use  of  the  VFTS,  such  as  reactions  to 
use  of  simulations,  working  collaboratively,  and  relevance  to  future  career. 

• Usability  Evaluation  instrument  which  is  used  to  gather  data  on  such  issues  on  the  ease  of  use 
and  operation  of  the  web-based  VFTS. 


The  VFTS  Studies 

Various  studies  were  conducted  in  the  VFTS  Project.  The  first  of  these  was  the  Baseline 
evaluation  conducted  at  the  various  universities,  to  provide  an  understanding  of  the  Industrial  Engineering 
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courses  (with  scheduling  content)  using  traditional  instructional  methodologies.  The  second  of  these  was  an 
investigation  of  the  use  of  the  VFTS  at  the  same  universities,  to  provide  an  understanding  of  the  Industrial 
Engineering  courses  (with  scheduling  content).  A second  usage  of  the  VFTS  occurred,  an  enhanced  version 
with  teaming  and  pedagogical  agency.  The  following  represents  the  findings  from  these  investigations  (at 
three  universities  to-date). 


Demographics 

For  the  most  part  the  demographic  analyses  showed  similarities  between  both  the  Baseline  and 
VFTS  groups  at  the  various  universities,  and  the  demographics  appear  to  parallel  those  that  are  found  in 
most  engineering  programs.  Most  students  were  in  the  21-22  age  category,  with  the  age  distributions  at 
most  universities  typical  of  a ‘traditional’  undergraduate  student.  However,  students  at  one  institution  had 
an  older  mean  and  the  age  range  was  quite  large.  At  all  institutions,  students  were  mostly  seniors.  The 
majority  of  students  at  all  three  campuses  were  male,  as  reflects  the  common  engineering  enrollment 
pattern.  There  was  no  significant  difference  on  the  gender  variable  between  the  groups.  The  primary 
language  was  English  but  many  other  languages  were  represented,  with  50%  or  more  students  at  two 
institutions  having  a language  other  than  English  as  their  first  language. 


Self-Assessment 

Instruments  gathered  pre  and  post  data  on  a student’s  interest  in  the  course  and  on  their  self- 
assessment  of  knowledge  of  course  content,  such  as  whether  they  could  apply  forecasting  methods  to  new 
problems. 

Students  have  a moderate  interest  in  the  course.  One  can  support  the  general  statement  that 
students  know  little,  if  anything,  about  the  content  of  the  course  but  think  that  it  will  help  them  in  a more 
general  way.  At  two  institutions,  students  indicated  that  they  did  see  the  course  assisting  in  developing 
industrial  engineering  fundamentals.  Pre  and  post-test  measures  of  self-assessment  indicate  significant 
gains  in  most  areas.  In  particular,  the  overall  gains  in  self-assessment  in  the  second  year  offerings  of  the 
VFTS  at  two  universities  were  significant.  The  self-report  assessment  provides  evidence  that  the  students 
feel  as  if  they  have  learned  the  subject  in  the  course. 


Performance 

As  a general  statement,  it  seemed  desirable  to  determine  if  there  are  significant  differences 
between  and  among  the  participant  groups.  Representative  analyses  reflect:  performance  at  each  institution 
in  both  the  Baseline  and  VFTS  options;  and  Baseline  to  VFTS  Comparisons.  In  examining  Baseline  and 
VFTS  groups  at  the  various  institutions,  analysis  indicated  overall  gains  in  learning  in  each  offering  of  the 
course  between  pre-test  and  post-test,  and  in  specific  content  areas  of  scheduling.  There  appears  to  be  a 
significant  difference  among  the  schools  on  the  pretest  measure,  with  one  institution  scoring  significantly 
higher  than  the  others.  Comparison  between  the  Baseline  and  VFTS  performance  at  each  institution 
indicated  a difference  on  selected  pre-test  items.  Correlations  were  calculated  for  the  pretest  instruments. 
Baseline  to  VFTS  groups.  Although  there  were  some  correlations  between  age  and  certain  items  and  among 
certain  items,  it  appears  that  these  results  were  of  no  significance  in  this  particular  study.  There  were  no 
significant  difference  between  the  Baseline  and  the  VFTS  groups  on  the  final  exam  except  for  a few 
selected  items.  As  noted,  the  VFTS  groups  did  not  perform  as  well  on  the  pretest  as  the  Baseline  groups, 
yet  scored  approximately  the  same  on  the  final  examination.  This  could  imply  that  the  less  strong  content- 
knowledge  groups,  using  the  VFTS,  performed  equivalent  to  the  stronger  content-knowledge  groups.  The 
post-test  scores  at  one  institution  are  significantly  higher  than  at  the  other  universities.  We  remark  that  at 
only  this  university  was  the  post-test  a component  of  the  student’s  final  grade.  In  fact,  it  was  treated  as  the 
final  examination  so  in  this  case  the  students  took  it  much  more  seriously  where  they  most  likely  studied 
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for  the  examination.  A comparison  of  the  post-test  scores  between  the  baseline  and  VFTS  groups  at  this 
university  show  a significant  gain  for  the  VFTS  group,  although  for  a smaller  class  size. 


Attitudes 

Instruments  gathered  pre  and  post  data  on  a student’s  opinions,  on  such  scales  as  usefulness, 
interest,  importance,  difficulty,  and  confidence,  for  example  whether  students  thought  learning  to  use 
computer  simulations  to  work  on  class  assignments  would  be  useful  to  their  future  career.  Results  of  the 
analysis  indicated  that  there  was  no  significant  change  in  the  attitudes  of  the  students,  even  with  the  use  of 
the  VFTS. 


Usability 

In  regard  to  the  use  of  the  VFTS,  most  students  seem  to  be  using  Internet  Explorer  on  a Windows 
98  platform.  Student  comments  on  the  usability  of  the  software  isolated  several  problems  that  were 
resolved  in  the  updated  version  of  the  VFTS.  The  various  modules  well-received;  for  the  most  part  the 
online  documentation  and  tutorials  not  used  or  students  commented  that  they  were  of  limited  value; 
students  were  not  positive  towards  the  use  of  the  VFTS  teaming  aspect  of  the  course.  However,  effort  is 
being  made  to  modify  the  teaming  aspect  of  the  course  with  the  assertion  that  this  may  impact  usability 
findings  on  this  topic. 


Other  Results 

Input  was  solicited  from  the  instructors  of  the  courses  at  the  various  universities.  Their  comments 
regarding  the  use  of  the  VFTS  were  positive.  They  noted,  in  particular,  that  students  using  the  VFTS  had  a 
more  realistic  course  project  than  their  counterparts  in  the  Baseline  classes. 

The  NSF  funding  for  this  project  has  supported  five  graduate  students  at  the  University  of 
Southern  California.  These  students  have  gained  valuable  experience  in  Web-based  software  development, 
user  interface  principles,  pedagogical  agent  technology,  educational  materiak  development,  and 
educational  evaluation.  Perhaps  the  most  valuable  training  for  all  the  students  is  the  close  multidisciplinary 
collaboration  between  computer  scientists,  engineering  educators,  and  educational  researchers  and 
educators.  These  students  will  have  a much  broader  perspective  on  research  than  typical  graduate  students 
that  work  closely  with  others  only  in  their  own  field. 


Conclusion 

The  VFTS  was  instrumental  in  facilitating  students'  understanding  of  the  integration  between  the 
models  for  forecasting,  production  planning,  material  planning,  inventory  planning,  and  scheduling. 

It  was  demonstrated  through  an  analysis  of  project  reports  that  students  had  a better  understanding 
of  the  integration  between  the  different  modules.  This  was  illustrated  by  the  fact  that  students  clearly  spent 
a good  part  of  their  time,  with  the  VFTS,  in  testing  many  different  scenarios,  and  students  were  able  to  see 
the  impact  of  changing  a parameter  on  the  output.  It  is  suggested  that  another  major  benefit  of  the  VFTS  is 
that  a more  complex  system  can  be  modeled  than  otherwise.  Thus,  it  was  seen  that  students  had  a more 
realistic  project  than  their  counterparts  in  the  Baseline  groups.  To  summarize,  it  is  believed  through  the 
analysis  that  the  VFTS  is  an  excellent  instructional  method  to  teach  students  the  integration  of  the  different 
modules  in  operations  planning. 
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Faculty  Development  in  Technology:  Outcomes  of  a Project  Based 

Team  Approach 

Mario  Kelly,  Hunter  College  of  the  City  University  of  New  York,  USA 


This  paper  is  a report  on  the  implementation  of  a grant  funded  project  designed  to  provide  faculty 
development  in  the  effective  integration  of  technology  into  the  teacher  education  programs  at  a large, 
public,  urban  university  in  the  Eastern  United  States.  The  principal  immediate  goal  of  the  project  was  for 
the  faculty  to  acquire  specific  basic  knowledge  and  skills  regarding  computer  hardware  and  software.  The 
principal  medium-range  goal  was  that  faculty  would  develop  technology  related  products  or  projects  that 
would  result  in  more  effective  and/or  efficient  teaching  and  learning  of  a unit  of  a teacher  education  course. 
A long  range-goal  was  the  modification  of  the  teacher  education  curriculum  to  reflect  greater  integration  of 
technology  and  to  meet  technology  standards  of  various  professional  organizations  and  government 
agencies.  This  paper  will  describe  the  faculty,  the  intervention  strategy  utilized  with  the  faculty,  and 
finally,  the  types  of  data  that  have  been  obtained,  and  on  the  basis  of  this  data,  the  degree  to  which  each  of 
the  principal  goals  has  been  achieved  to  date. 


The  Faculty 

The  subjects  were  17  faculty  members  who  teach  pre-service  teachers  in  an  urban,  public 
university  program,  and  4 public  school  teachers  who  serve  as  master  or  mentor  teachers  to  pre -service 
teacher  candidates  during  the  required  semester  of  student  teaching  in  a public  school.  At  the  start  of  the 
project  a modified  version  of  the  School  Hardware  Technology  Survey  (U.S.  Department  of  Education, 
1998)  was  administered  to  the  population  of  faculty  from  which  the  sample  was  drawn.  The  results  of  the 
survey,  which  faculty  completed  anonymously,  were  supplemented  with  data  from  faculty  focus  groups, 
and  information  from  the  college’s  administration  regarding  funds  allocated  for  the  purchase  of  equipment. 
The  data  revealed  that  faculty  all  had  basic  computers  with  Internet  access  and  printers  in  their  offices. 
However,  at  the  time  the  most  common  uses  of  technology  reported  were  word  processing  and  accessing  e- 
mail.  Only  1 of  the  17  in  the  sample  was  teaching  a course  that  specifically  included  computers  and  was 
taught  in  a computer  laboratory.  Most  faculty  members  could  not  identify  the  specific  configuration  of 
their  office  computer  or  report  the  specific  software  it  contained. 


The  Intervention 

The  main  focus  of  the  intervention  was  the  provision  of  staff  development  activities  to  enable 
faculty  to  integrate  technology  into  coursework.  Development  activities  followed  a constructivist  model  in 
which  each  faculty  meniier  was  responsible  for  developing  a technology  product  to  improve  the 
effectiveness  and  efficiency  of  teaching  and  learning  one  unit  of  a course  they  were  currently  teaching. 
Intervention  activities  unfolded  in  4 phases  over  the  course  of  one  academic  year. 

Phase  1:  Faculty  attended  an  intensive  2-day,  hands-on  staff  development  workshop  on 
PowerPoint  Basics,  Advanced  PowerPoint,  Internet  Searching  Strategies,  and  Inspiration  (semantic 
mapping/outlining  software)  conducted  by  national  experts  in  the  integration  of  technology  into  teaching. 
The  workshop  included  application  components  that  allowed  faculty  and  staff  to  create  an  actual  “mini” 
product  by  the  end  of  the  second  day. 

Phase  2:  Faculty  worked  on  their  individual  projects  to  develop  technology  products  for  use  in  a 
course.  While  the  projects  were  individual,  following  a constructivist  approach,  the  faculty  was  grouped 
into  supportive  teams,  to  share  ideas,  assist  each  other,  and  perhaps  most  importantly,  have  a forum  for 
discussion. 

Faculty  development  teams : Five  (5)  teams,  each  consisting  of  2-5  members  were  established,  in 
each  case  with  the  mutual  consent  of  all  team  members.  The  focus  of  4 teams,  and  hence  the  projects  of 
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team  members,  was  a specific  content  area.  Each  of  these  teams  was  named  according  to  the  respective 
content  area:  Educational  Research,  Science  Instruction,  Language  & Literacy,  and  Social  Studies.  The  5 
team  was  Interdisciplinary,  and  included  special  education,  health  education,  and  art  education.  On  each  of 
the  teams,  with  the  exception  of  the  Science  Team,  one  of  the  4 technology-skilled  public  school  master 
teachers  served  as  a consultant  to  the  faculty.  As  consultants  the  master  teachers  did  not  have  to  produce  a 
technology  product.  Instead,  the  consultant’s  role  was  to  assist  with  design  and  implementation  issues,  and 
in  particular,  provide  feedback  regarding  the  real-life  classroom  viability  of  the  technology  products  being 
developed.  Also  consistent  with  the  constructivist  tradition,  faculty  members  were  given  a set  of  guidelines 
for  completing  the  technology  projects.  The  guidelines  required  that  faculty  demonstrate  application, 
analysis,  synthesis  and  evaluation  on  Bloom’s  Taxonomy  for  the  cognitive  domain. 

Phase  3:  Faculty  incorporated  their  technology  products  into  the  course. 

Phase  4:  Faculty  evaluated  the  effectiveness  of  the  technology  products. 

As  an  incentive  for  successful  completion  of  all  phases  of  the  project  faculty  are  released  from 
teaching  I course  a subsequent  semester. 


The  Outcomes 

Phase  1;  The  benefits  of  staff  development  activities  were  almost  immediately  noticeable. 

• Most  of  the  faculty  had  not  used  PowerPoint  and  were  not  even  aware  of  the  existence  of  Inspiration. 
By  the  end  of  the  workshop  each  person  had  produced  a PowerPoint  presentation  for  one  lesson  of  a 
current  course,  some  also  incorporating  a search  strategy  or  an  Inspiration  concept  map. 

• Most  faculty  members  used  the  PowerPoint  and  Inspiration  software  in  classes  and  presentations  in  the 
weeks  immediately  following  the  workshop. 

• 16  of  the  17  faculty  had  never  utilized  the  computer  laboratory,  where  the  workshop  was  conducted. 
They  learned  about  the  networked  computers  and  how  the  network  functions. 

• 1 4 of  the  1 7 faculty  have  since  independently  taken  an  additional  workshop  offered  at  the  college,  to 
learn  Blackboard,  a Web-based  course  management  system. 


Phase  2:  The  teams  varied  considerably  in  the  degree  to  which  they  completed  Phase  2.  See  Table  I 
below. 


TABLE  1 


Team 

Num.  of  Phase  2 Phase  3 
Members  Completed 

Phase  4 
Completed 

Completed 

Educational 

Research 

4 2 

2 

0 

Inter 

Discinlinarv 

4 3 

2 

0 

Language 
& Literacv 

5 4 

3 

0 
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Science 2 2 2 0 

Social 

Studies 2 2 2 I 


On  the  basis  of  the  number  of  members  who  completed  Phase  2 and  Phase  3,  the  Science  and  the 
Social  Studies  teams,  those  with  the  fewest  number  of  members,  were  clearly  the  most  successful. 

However,  the  numbers  do  not  reflect  another  aspect  of  the  teams’  success,  namely  the  degree  to  which  team 
members  were  engaged  in  the  process  of  not  only  developing  their  projects,  but  also  in  the  process  of 
acquiring  technology  knowledge  and  skills.  By  this  measure  the  Language  & Literacy  group  was  clearly 
the  most  successful,  as  indicated  by  responses  to  formal  interview  questions,  email  communications  among 
group  team  members  (teams  were  asked  to  send  a confirmation  copy  of  team  emails  to  the  project  director), 
and  number  of  queries  for  assistance  with  hardware,  software,  or  expertise.  The  Language  & Literacy 
Team  communicated  with  the  greatest  frequency  and  engaged  in  the  greatest  number  of  technology  skill 
building  activities.  They  met  and/or  exchanged  emails  almost  weekly,  and  placed  the  greatest  and  most 
consistent  demand  on  the  project  staff  for  assistance.  The  difference  among  the  teams  is  an  issue  worthy  of 
further  analysis  which  due  to  space  limitations  is  not  possible  here. 

Phase  3:  As  Table  1 reveals,  faculty  who  completed  the  design  of  their  projects  were  almost 
certain  to  implement  the  projects  in  a course  the  following  semester.  The  single  exception  to  this  rule  was 
found  in  the  Social  Studies  Team  and  was  due  to  the  fact  that  the  faculty  member  did  not  teach  the  course 
for  which  the  project  was  designed. 

Phase  4:  As  Table  1 reveals,  only  1 of  the  faculty  members  has  completed  the  evaluation  of  the 
implementation  of  the  project.  However,  the  numbers  do  not  adequately  represent  state  of  completion  of 
the  projects.  In  actuality  all  faculty  who  completed  Phase  3 did  gather  obtain  data  to  evaluate  the 
effectiveness  of  the  implementation.  However,  until  a formal  written  analysis  of  the  data  is  provided, 

Phase  4 is  not  considered  completed. 


DISCUSSION 

In  summary,  technology  faculty  development  activities  for  faculty  who  teach  pre -service  teachers 
were  described.  These  activities,  aimed  at  increasing  the  faculty’s  level  of  technology  proficiency  and  its 
ability  to  integrate  technology  into  teacher  education  courses  were  based  on  constructivist  principles. 
Among  the  constructivist  elements  were  hands-on  workshops,  project-based  learning,  and  learning  teams  of 
mutually  supportive  members.  The  development  activities  were  divided  into  4 phases  and  the  success  of 
the  members  of  each  team  at  each  phase  was  compared  and  analyzed.  The  role  of  the  incentive  provided  to 
faculty  for  successful  completion  of  the  project  and  a detailed  analysis  of  the  various  dimensions  of  the 
teams  that  may  have  fostered  or  inhibited  project  completion  are  areas  worthy  of  further  study. 
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Brownfield  Action:  An  Integrated  Environmental  Science  Simulation  Experience  for  Undergraduates 
By  Ryan  Kelsey,  Columbia  University  Center  for  New  Media  Teaching  and  Learning 

Abstract:  This  paper  presents  the  results  of  three  years  of  development,  implementation,  and  evaluation  of  a cd- 
rom/web  hybrid  simulation  known  as  Brownfield  Action  for  an  introductory  environmental  science  course  at  an 
independent  college  for  women  located  in  a large  city  in  the  northeastern  USA.  Brownfield  Action  is  not  intended 
as  a cost-cutting  measure,  a time-saver,  or  a replacement  for  other  material.  It  is  a method  of  integrating  what  was 
formerly  disparate  labs  and  lectures  into  a seamless  learning  experience  that  improves  student  learning  and 
motivates  students  to  critically  consider  the  importance  of  all  the  issues  involved  in  the  system  of  human  impact  on 
the  environment.  Brownfield  Action  gives  students  a skill  set  for  conducting  scientific  work  and  the  perspective  that 
science  is  connected  to  one’s  everyday  experience. 

Background:  Science  instructors  and  students  often  struggle  to  link  the  material  in  lecture  and  laboratory. 
Laboratories  are  often  taught  by  different  personnel,  and  must  fit  into  constraints  of  equipment  and  time,  as  well  as 
safety  and  level  of  difficulty.  A large  survey  course  often  covers  dozens  of  topics,  but  a student  typically 
experiences  no  more  than  fifteen  laboratory  periods  in  a given  semester.  Life  science  lectures  require  rapid  note 
taking  and  passive  concentration  as  verbal  information  is  streamed  at  students  for  one  to  two  hours.  Laboratory 
sessions  typically  require  two  to  three  hours  of  recipe-following  and  occasional  problem  solving.  These  two 
elements  of  the  course  operate  in  isolation  because  there  is  little  time  in  lecture  to  discuss  experimental  technique  (if 
the  lecturer  even  knows  that  information),  and  a laboratory  is  not  typically  set  up  for  someone  to  give  verbal 
instruction  as  occurs  in  the  lecture  hall.  Typical  science  students  spend  most  of  their  time  memorizing  verbal 
information  for  lecture  examinations,  and  memorizing  techniques,  anatomy,  and  taxonomies  for  laboratory  practical 
examinations.  Rarely  is  information  from  one  relevant  to  the  other  because  the  lecture  focuses  on  concepts  and  the 
laboratory  is  intended  to  teach  technique. 

In  response  to  this  disconnect  between  laboratory  and  lecture,  many  science  curricula  incorporate  problem  sets  as  a 
third  element  in  an  attempt  to  provide  some  unification.  Unfortunately,  the  typical  drill-and-practice  nature  of  the 
problems  backfires  as  it  encourages  students  to  follow  a similar  surface  study  approach  that  primarily  consists  of 
pattern  matching.  This  approach  prevents  students  from  tying  lectures  to  laboratories  as  the  problem  sets  become 
their  own  separate  entity  within  the  course.  Some  schools  even  create  separate  class  sessions  monitored  by  teaching 
assistants  to  accommodate  this  reality,  but  these  sessions  only  reinforce  the  separateness  of  the  problem  sets.  Upon 
completion  of  the  semester,  students  of  these  science  courses  often  feel  they  have  taken  two  or  three  courses.  They 
fail  to  get  a sense  of  the  content  as  a whole,  and  they  never  actively  engage  the  content  at  more  than  a surface  level. 
One  might  even  propose  that  in  order  to  succeed  in  these  types  of  courses  as  they  are  designed,  it  is  in  the  student’s 
best  interest  to  remain  at  the  surface  with  his  or  her  engagement. 

Related  Work:  Several  studies  in  recent  years  have  led  to  the  development  of  design  principles  for  the  construction 
of  computer-based  learning  environments  such  as  simulations,  many  of  which  can  be  found  in  Brent  Wilson’s 
compilation  Constructivist  Learning  Environments:  Case  Studies  in  Instructional  Design  (1996).  Peter  Honebein’s 
“Seven  Goals  for  the  Design  of  Constructivist  Learning  Environments”  outlines  a set  of  instructional  design 
principles.  He  then  evaluates  two  projects  based  on  those  principles  (Wilson  pi  1-25).  John  B.  Black  and  Robert  O. 
McClintock,  similarly,  provide  an  “Interpretation  Construction  Approach  to  Constructivist  Design,”  a seven-point 
system  that  they  apply  to  three  projects  created  under  the  Dalton  Technology  Plan  (Wilson  p26-3 1 ).  John  R.  Savery 
and  Thomas  M.  Duffy  borrow  from  Barrows’  model  for  case-study  teaching  and  learning  in  order  to  create  a set  of 
principles  for  problem-based  learning  (Wilson  pi  35- 148).  More  specific  to  simulations,  Margaret  Gredler  examines 
one  approach  for  the  design  and  evaluation  of  simulations  in  Designing  and  Evaluating  Games  and  Simulations:  A 
Process  Approach  (1993).  Her  work  involves  an  interwoven  three-part  design  of  role,  task,  and  environment. 

The  closest  work  to  the  experience  of  the  Brownfield  Action  design  and  development  process  is  found  in  Goodrum, 
Dorsey,  and  Sch wen’s  work  on  defining  and  designing  Enriched  Learning  and  Information  Environments  (ELIEs) 
(Educational  Technology  November  1 993).  In  this  paper,  they  describe  how  their  experience  with  designing 
educational  environments  led  them  away  from  entirely  technology-based  and  theory-based  definitions  and  more 
towards  what  they  define  as  a socio-technical  definition  that  focuses  on  the  people  involved  and  the  specific  work 
they  are  asked  to  perform.  They  perceive  that  the  latest  learning  theory  and  the  latest  technology  does  not 
necessarily  lead  to  innovation.  Instead  they  claim  that  all  innovations  are  situated  within  a context  of  people  trying 
to  accomplish  work  in  a particular  environment,  and  a well-designed  teaching  and  learning  tool  should  support  that 


work.  They  go  on  to  describe  their  design  work  as  a series  of  relationship  building  with  users,  rapid  prototyping 
with  mock-ups  and  a focus  on  the  tasks  that  users  needed  to  perform. 

Methods:  The  Columbia  Center  for  New  Media  Teaching  and  Learning  (CCNMTL)  developed  a cd  rom/web 
hybrid  simulation  from  1999-2001  known  as  Brownfield  Action.  It  was  implemented  at  Barnard  College,  a women’s 
college  affiliate  of  Columbia  University.  The  project  demonstrates  how  an  undergraduate  Environmental  Science 
course  can  directly  integrate  its  lecture  and  laboratory  components  through  the  use  of  a simulation.  In  a semester- 
long  course,  students  solve  a complex  problem  that  requires  the  application  of  knowledge  and  skills  learned  in 
lecture  and  in  the  laboratory  setting.  Student  performance  and  engagement  is  improved  while  developing  real-world 
problem  solving  skills,  and  students  experience  environmental  science  as  a highly  integrated  field.  Rather  than 
reserving  higher  level  thinking  skills  such  as  analysis  and  synthesis  for  smaller,  more  advanced  courses.  Brownfield 
Action  allows  introductory  students  to  see  environmental  science  as  an  integrated  and  dynamic  part  of  society,  rather 
than  a series  of  abstract  concepts  and  recipe-driven  techniques. 

Context  of  Brownfield  Action:  Brownfield  Action  is  a simulation  that  provides  a learning  environment  for 
applying  many  of  the  major  concepts  of  environmental  science.  Students  operate  in  a virtual  town  serving  as 
consultants  to  a real  estate  developer  who  wants  to  avoid  purchasing  potentially  contaminated  land.  Students  must 
become  actively  involved  in  the  lecture  and  laboratory  in  order  to  succeed.  They  must  learn  to  explore  and  to 
discover  a path  to  a solution  to  the  problems  they  encounter  to  reach  a valid  conclusion.  Pairs  of  students  form 
environmental  consulting  companies  to  investigate  a hypothetical  abandoned  factory  site  in  a small  town.  A mall 
developer  who  wishes  to  purchase  the  factory  site  contracts  with  each  two-student  company  to  conduct  an 
investigation.  Students  collect  socio-historical  and  scientific  data,  construct  maps  of  the  site’s  basic  geology, 
topography,  and  any  contamination  they  discover  while  operating  within  a given  simulated  budget.  Their 
investigation  culminates  in  a report  to  the  mall  developer  that  summarizes  their  results  and  gives  recommendations 
about  how  to  proceed. 

Socio-Historica]  Component: 


Figure  1:  the  site  map  interface  for  Brownfield  Action 


To  complete  the  maps  and  report,  students  must  first  gather  a site  history  from  the  town’s  resources  using  the  Site 
Map  (Figures  1 and  2).  Through  the  simulation,  students  visit  government  offices,  businesses,  and  residences  to 
conduct  interviews  with  town’s  officials  and  citizens.  They  also  obtain  and  analyze  simulated  public  documents. 
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Figure  2:  the  municipal  directory  where  students  visit  government  offices 


Scientific  Testing  Component;  Using  the  information  obtained  from  the  town’s  history,  students  then  conduct  a 
series  of  environmental  tests  to  determine  the  presence,  extent,  and  probable  cause  of  any  contamination  (Figure  3). 
Some  tests,  such  as  soil  permeability,  are  conducted  as  traditional  laboratory  exercises  placed  within  the  context  of 
the  simulation.  Other  tests,  such  as  well  monitoring,  are  conducted  virtually  using  the  computer.  Over  two  million 
data  points  are  available  for  collection  over  the  64,000  square  foot  virtual  site  map,  including  bedrock  and  water 
table  data  as  well  as  contamination  concentrations  at  depths  of  over  150fr.  This  vast  quantity  of  data,  both  through 
the  site  history  and  the  environmental  testing,  allows  for  an  infinite  number  of  strategies  for  testing  and  a unique 
data  set  for  every  student  company.  Successful  students  work  within  the  given  operating  budget  and  clearly  identify 
the  cause  and  extent  of  the  contaminated  areas  on  the  site. 


COPY  AYAIMBLIE 

^B6 


_ _ 

''|i.  til  f I 

If  W mfl  dn  Hi ' 

'.ii  ill  jdiJHMIi  ■HBlHi  i"i| 


CW.’ 

4teui 

i^ntinfo 


«M^  t /itjd  i»«l  fir  )nw) 


«.  iMMA  • 

1;  OhMTMi ««  ' tell 

«*•««  rnm>tmm>mmm  ptW^0(iitt9$ 


t;-«  K T!  Mi«:> 


Figure  3:  the  testing  interface 


Brownfield  Action  is  not  intended  as  a cost-cutting  measure,  a time-saver,  or  a replacement  for  other  material.  It  is  a 
method  of  integrating  what  was  formerly  disparate  labs  and  lectures  into  a seamless  learning  experience  that 
improves  student  learning  and  motivates  students  to  critically  consider  the  importance  of  all  the  issues  involved  in 
the  system  of  human  impact  on  the  environment.  Brownfield  Action  gives  students  a skill  set  for  conducting 
scientific  work  and  the  perspective  that  science  is  connected  to  one’s  everyday  experience. 


Course  Objectives:  The  primary  faculty  member  and  subject  matter  expert  identified  the  following  objectives: 


After  experiencing  Brownfield  Action,  students  will  be  able  to  explain  how  to  approach  and  solve  a scientific 
problem  by: 

• describing  the  strategy  used  to  discover  contamination  sites  in  Brownfield  Action; 

• identifying  and  explaining  the  outcomes  of  environmental  tests  they  conduct  and  related  information, 
making  recommendations  and  being  aware  of  the  consequences  of  their  decisions; 

• drawing  inferences  from  data  about  structures  that  contribute  to  environmental  contamination; 


and  students  will: 


• read  articles  on  ecology  with  different  understanding,  interest,  and  personal  commitment; 

• appreciate  that  real  world  decision-making  about  ecology  involves  ambiguity  rather  than  certainty. 

Evaluation:  To  assess  the  effectiveness  of  Brownfield  Action  (BfA2.0)  in  meeting  its  objectives  in  Fall  2000, 
students  were  surveyed  at  the  beginning  of  the  simulation  to  obtain  an  indication  of  their  perceptions  of  the  levels  of 
knowledge  and  skill  targeted  by  the  course  with  which  they  started.  The  laboratory  directors  prepared  a daily 
implementation  log  providing  detailed  documentation  of  the  BfA2.0  experience  in  the  labs.  The  evaluators  observed 
five  lectures  and  six  of  the  three-hour  labs  to  further  their  understanding  of  the  instructional  setting  within  which  the 
Brownfield  Action  simulation  was  conducted.  At  the  end  of  the  course,  students  rated  Brownfield  Action  on  how 
well  it  had  contributed  to  the  objectives  of  the  course  and  other  matters,  and  all  students  participated  in  hour-long 
focus  groups  to  provide  in-depth  responses  to  questions  not  amenable  to  discovery  through  written  survey 
questionnaire.  The  evaluators  conducted  hour-long,  post-course  interviews  with  the  lead  designer  of  the  simulation; 
the  primary  faculty  member  and  subject  matter  expert,  and  the  laboratory  directors.  Finally,  a cross  section  of  the 
papers  prepared  by  BfA2.0  students  in  the  fall,  2000  was  reviewed  and  compared  with  student  papers  from  previous 
years. 
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Data  Sources:  The  entire  course  of  one  hundred  and  twelve  female  students  were  surveyed  using  two 
questionnaires  (one  at  the  beginning  of  the  semester,  one  at  the  end)  and  a series  of  focus  groups  at  the  end  of  the 
term  (one  for  each  of  the  eight  lab  sections). 

In  the  survey  questionnaire  and  focus  groups  we  investigated  students’ 

• perceptions  of  pre-  and  post-test  levels  of  their  knowledge; 

• perceptions  of  the  success  of  BfA2.0  in  meeting  its  objectives; 

• perceptions  of  the  contributions  of  various  components  of  BfA2.0  in  assisting  them  to  solve  the  overall 
problem  addressed  by  BfA2.0,  and  how  well  BfA2.0,  the  lectures  and  the  lab  succeeded  in  directing  them 
to  focus  on  the  major  problem  addressed  by  BfA2.0. 

• overall  evaluation  of  BfA2.0,  positive  and  less  positive; 

• recommendations  for  technical  improvements  in  BfA2.0; 

• perceptions  of  the  most  valuable  parts  of  BfA2.0; 

• perceptions  of  what  could  be  done  to  improve  BfA2.0. 

The  evaluation  team  also  examined  students’  final  reports  in  accordance  with  the  learning  objectives  outlined  by  the 
primary  faculty  member  as  well  as  the  daily  laboratory  implementation  log  prepared  by  the  laboratory  directors. 

Summary  of  Results: 

■ Students  learned  more  and  in  greater  depth  using  Brownfield  Action  than  in  previous  years  without  it. 

■ Student  work  looked  more  authentic  and  professional,  closer  to  what  would  be  expected  of  a professional 
performing  these  tasks. 

■ Students  appreciated  how  the  simulation  contributed  to  their  understanding  of  the  material. 

■ Brownfield  Action  was  successful  in  (1)  enhancing  the  scientific  literacy  of  students,  (2)  facilitating  their 
construction  of  new  meaning  based  upon  what  they  saw  and  experienced  and  (3)  enhancing  practices 
leading  to  increased  student  learning. 

■ Students’  abilities  to  construct  new  meaning  based  upon  what  they  saw  and  experienced  were  expanded. 

■ Students  found  the  content  useful  and  the  simulation  a good  way  to  learn  it. 

■ More  staff  development  is  necessary  to  foster  the  teaching  strategies  needed  for  a true  discovery  process 
and  to  build  comfort  with  technology. 

■ Minor  technical  glitches  created  levels  of  frustration  that  interfered  with  its  ability  to  meet  its  educational 
potential.  These  glitches  notwithstanding,  the  students  gave  high  ratings  to  other  features  of  the 
technology.  In  particular,  eight  out  often  students  noted  the  ease  with  which  it  was  possible  to  move 
through  information  in  each  of  the  sections  of  the  simulation. 

Simulations:  For  as  long  as  there  has  been  computing,  many  have  tried  to  simulate  aspects  of  reality  with 
computing  power,  but  the  reality  is  that  all  that  has  been  built  to  date  would  be  better  called  simulators  rather  than 
simulations.  Simulators  model  real-world  processes  with  mathematical  equations  and  variables  that  can  be 
manipulated  so  that  changes  can  be  observed  and  analyzed.  Simulators  have  inputs  and  outputs  and  variables  that 
can  be  manipulated.  The  bulk  of  the  software  for  Brownfield  is  no  different  upon  close  inspection.  It  is  a model  of  a 
town  with  a mathematically  driven  contamination  event.  Students  input  instructions  through  the  map  in  order  to 
receive  the  outputs. 

A simulator  only  becomes  a simulation  when  it  is  given  a narrative,  and  participants  are  given  roles  to  fulfill,  a 
problem  to  solve  within  a given  context,  and  tools  to  help  them  solve  the  problem.  In  Brownfield  Action,  the 
components  of  the  simulation  are  in  the  classroom  environment  supported  by  the  technology  and  in  technology 
itself  The  course  curriculum  provides  the  problem  to  be  solved  and  the  roles  to  be  played  in  the  form  of  an 
assignment.  The  context  for  the  problem  is  provided  in  the  course  lectures.  Tools  are  provided  within  the  software 
in  the  form  of  testing  tools,  in  the  laboratory  in  the  form  of  soil  samples  and  mapping  techniques,  and  in  the  lectures 
and  reading  material  in  the  form  of  mathematical  formulas  and  models  of  other  investigations. 

Brownfield  Action  could  be  run  in  a classroom  without  any  technology  at  all,  and  in  fact  it  was  done  on  a smaller 
scale  for  many  years  using  an  elaborate  note  card  scheme.  The  computer  software  supports  the  expansion  of  the 
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simulation  created  in  the  classroom,  giving  it  a visual  space  with  the  map,  a more  efficient  process  for  collecting 
data  in  the  form  of  better  tools,  a means  for  communication  with  email  and  links  to  the  course  materials,  and  more 
definition  and  depth  (literally  and  figuratively)  to  the  environment  that  is  to  be  examined  in  the  form  of  a complex 
three-dimensional  contamination  simulator  and  a cast  of  characters  that  creates  the  socio-historical  narrative.  The 
software  itself  does  not  teach.  Instead  it  invites  the  user  to  engage  in  the  problem.  To  foster  engagement,  one  needs 
motivation  and  a safe  space  to  take  risks,  and  that  is  where  the  classroom,  the  curriculum,  and  the  instructors  come 
in  to  play. 

Simulations  can  be  powerful  teaching  and  learning  environments  because  learning  to  operate  within  a simulation  is 
good  training  for  operating  within  the  real  world.  Everyone  has  roles  to  play  in  the  real  world  (probably  more  than 
one),  and  there  are  problems,  conflicts,  and  obstacles  to  be  encountered  and  overcome  in  each  of  those  roles.  The 
method  for  overcoming  these  problems  is  in  communication  and  experimentation,  which  includes  hypothesizing, 
collecting  data,  analyzing,  synthesizing,  and  applying  previous  discoveries  with  other  role  players  using  the  tools 
that  are  available.  Solving  these  problems  in  a simulation  is  no  different.  The  major  difference  is  that  simulations 
provide  a simplified  set  of  conditions  within  a contained  space  for  students  to  operate  in.  A good  simulation  still  has 
ample  noise  that  must  be  sorted  through  in  order  to  find  the  important,  or  signal  information  that  will  lead  them  to  a 
solution,  but  students  can  operate  with  confidence  that  the  signal  exists.  Simulations  also  provide  a safe 
environment,  away  from  the  consequences  of  the  real  world:  the  money  involved  isn’t  real,  one’s  job  isn’t  at  stake, 
etc.  Students  can  essentially  practice  living  in  the  real  world  by  immersing  themselves  in  the  problem  to  be  solved 
in  the  simulation. 
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Making  the  Transition  From  Print:  Integrating  Concept  Mapping  and 
Online  Communication  With  Traditional  Distance  Education  Materials 


Abstract:  The  move  from  traditional  paper-based  distance  education  subject  materials  to  the 
use  of  information  and  communication  technologies  (ICT)  has  increased  the  ways  in  which 
students  can  engage  with  their  lecturers,  peers  and  the  unit  materials.  In  this  paper,  strategies 
for  enhancing  print -based  learning  resources  are  discussed.  These  include  concept  mapping 
by  which  the  conceptual  structure  of  a unit  can  be  articulated,  and  online  communication  that 
can  enhance  the  interactions  between  students  and  teachers.  A review  of  student  responses  to 
concept  mapping  indicates  a general  acceptance  of  the  tool,  with  comparisons  of  assessment 
grades  between  successive  years  of  students  suggesting  an  improvement  for  students  who 
were  ejq^osed  to  concept  mapping  in  conjunction  with  online  communication.  The  study 
highlighted  several  curriculum  issues  that  need  further  work. 


The  problem  faced  by  any  university ...  is  how  to  structure  itself  so  that  its  central  academic  activity 
is  facilitated,  not  undermined,  by  technological  developments  (Laurillard  & Margetson,  1997,  p,  4), 


Monash  University  is  a large  institution  with  campuses  or  affiliates  in  Australia,  Malaysia,  South  Africa,  Italy 
and  the  United  Kingdom.  The  use  of  ICT  is  increasing  at  Monash  because  of  changes  in  higher  education 
attributed  (but  not  limited)  to,  funding,  diverse  student  backgrounds  (Australian  Vice  Chancellors’  Committee, 
1996),  changing  student  expectations  and  lifestyles,  new  developments  in  technology,  and  opportunities  for 
improved  approaches  to  learning  and  teaching. 

In  1994,  Monash  University  was  asked  to  provide  coursework  for  the  Australian  Department  of  Foreign  Affairs 
and  Trade  (DFAT)  in  Canberra.  At  that  time,  students  were  either  existing  or  newly  appointed  officers  of 
DFAT,  as  well  as  diplomats  from  developing  countries  throughout  Africa,  Asia  and  the  South  Pacific.  As  a 
consequence  of  this  endeavour,  the  Monash  University  Diplomacy  and  Trade  Program  began.  The  program  is 
run  by  the  Department  of  Management  in  the  Faculty  of  Business  and  Economics  of  Monash  University  and  is 
designed  specifically  for  aspiring,  trainee  or  existing  diplomats.  Successful  completion  of  the  course  results  in 
the  conferring  of  a Graduate  Diploma  or  Master  degree. 

For  the  cohort  under  review  in  this  paper,  students  generally  range  in  age  from  24  to  40  years  of  age. 
Approximately  half  are  Australian,  with  the  other  half  being  international  students.  In  almost  all  cases,  academic 
backgrounds  and  vocational  experiences  are  dominated  by  socio-political  training.  As  a consequence,  there  is  a 
minimal  understanding  of  the  more  numerate  sciences,  of  which  economics  is  one.  At  the  time  of  the  review,  the 
numbers  in  each  off-campus  class  are  small  (approximately  six),  though  there  are  approximately  40  students  in 
each  concurrent  on-campus  class.  Thus,  this  unit  is  economically  viable,  especially  as  the  fees  are  high. 

Given  the  importance  of  economics  in  a global  trade  context,  the  two  economics  units.  Principles  of 
Microeconomics  and  Macroeconomics,  as  well  as  the  Economics  of  Trade  and  Investment,  are  core  compulsory 
subjects.  This  paper  discusses  the  learning  design  of  the  unit,  Principles  of  Microeconomics  and 
Microeconomics.  The  unit  incorporates  all  of  the  key  issues  raised  in  a full  first  year  course  of  undergraduate 
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The  Context 


economics,  with  the  added  emphasis  of  trade  protection  and  microeconomic  reform  (Reiman,  1999)  The 
challenge  is  to  teach  these  fundamentals  in  one  semester  so  that  students  are  conversant  with  the  policy- 
orientated  approach  of  their  next  unit.  Economics  of  Trade  and  Investment,  A second  challenge  involved  the 
physical  location  of  the  off-campus  students  where  there  are  issues  of  access  to  the  proposed  online 
environment  (for  example,  one  student  travelled  very  frequently  and  logged  into  the  online  discussions  from  a 
variety  of  countries  during  the  course  of  the  semester). 

The  team  who  developed  the  unit  materials  reached  consensus  on  a view  of  learning  based  upon  constructivist 
principles,  in  which  knowledge  is  constructed  individually  and  socially  co-constructed  by  learners  based  on 
their  interactions  and  experiences  with  the  world  (Biggs,  1999;  Jonassen  & Rohrer-Murphy,  1999;  Kennedy  & 
McNaught,  1997a)  Integrating  computer-mediated  communication  (CMC)  to  support  a more  student-centred 
learning  environment  was  a key  component  of  the  design  of  the  unit  materials.  The  incorporation  of  online 
discussion  groups  has  been  shown  to  foster  collaborative  learning  (Agostino,  Lefoe,  & Hedberg,  1997),  improve 
flexibility  in  teaching  and  learning  (Freeman,  1997)  and  support  problem-based  learning  (Oliver  & Omari, 
1999), 

In  the  past,  the  study  materials  for  the  off-campus  cohort  were  entirely  print-based.  These  materials  were 
redeveloped  to  be  used  in  conjunction  with  concept  maps,  and  a number  of  online  discussion  groups.  The  on- 
campus  students  had  a traditional  set  of  lectures  and  tutorials;  the  concepts  maps  were  given  to  the  students  and 
used  by  the  lecturer  during  classes. 


The  Concept  Maps  and  Online  Discussion  Groups 

Concept  mapping  has  a history  of  extensive  use  for  improving  student  learning  and  metacognition.  It  has  been 
used  in  domains  as  diverse  as  chemistry  (Cullen,  1990;  Ross  & Munby,  1990);  biology  (Jegede,  Alaiyemola,  & 
Okebukola,  1990);  physics (Pankratius,  1990);  microbiology  (Barenholz  & Tamir,  1992);  hypermedia 
development  (Kennedy,  1995;  Kennedy  & Taylor,  1994),  and  the  design  of  computer-based  learning  tools 
(Kennedy  & McNaught,  1 997b), 

The  concept  maps  were  developed  using  software  Inspiration  (Helfgott,  Shankland,  Stafford,  & Samson, 
1997),  In  Figure  1,  the  master  concept  map  is  shown.  Very  positive  feedback  was  received  from  a number  of 
students  on  its  usefulness  (discussed  later).  The  master  concept  map  was  further  broken  down  into  sections  that 
addressed  a single  week.  In  Figure  2,  the  concepts  in  italics  are  from  pass  and  future  weeks,  while  the  concepts 
related  to  week  1 1 are  in  normal  style  print 

Online  discussion  groups  were  linked  to  the  content  of  the  print-based  materials.  Five  discussion  groups  were 
created.  These  were  announcements  (read-only  by  students),  general  discussion  (for  ongoing  discussions  during 
the  semester),  and  three  groups  associated  with  specific  assessment  tasks.  The  three  online  assessable  discussion 
groups  were  designed  to: 

■ help  the  students*  written  communication  skills; 

■ develop  their  ability  to  structure  an  economic  argument; 

■ develop  their  understanding  of  economic  concepts; 

■ allow  students  to  share  ideas  with  peers,  wherever  they  may  be;  and 

■ develop  online  collaborative  skills. 

Students ’marks  for  these  discussion  groups  contributed  only  1 5%  to  the  final  grade.  The  overall  assessment 
strategies  were  not  markedly  changed  and  were  similar  for  both  off-campus  and  on -campus  groups. 


Development  Issues  for  the  Unit 

In  the  frequently  perceived  headlong  rush  to  put  all  course  content  online,  academics  and  students  are  expected 
to  engage  with  new  ways  of  thinking  and  working,  while  simultaneously  developing  new  skills.  In  the  transition 
between  more  traditional  approaches  to  curriculum  development  and  design,  older  technologies  are  often 
disregarded  as  being  irrelevant  and  outmoded.  The  second  author  of  this  paper  was  an  experienced  lecturer  in 


face-to-face  situations,  but  new  to  the  use  of  concept  maps  as  a holistic  organisational  learning  tool,  and  the  use 
and  management  of  asynchronous  online  discussion  groups. 
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Figure  1:  The  major  concept  map  for  the  course 

The  development  of  these  unit  materials  represented  a departure  from  his  previous  experience.  The  first  author 
is  an  educational  designer,  experienced  in  the  use  of  both  learning  tools.  The  development  of  the  unit  materials 
involved  building  effective  electronic  communication  methods  (fax,  email  and  telephone)  between  the  two 
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authors,  as  author  one  was  based  in  Melbourne,  while  author  two  was  in  Canberra,  a one -hour  air  flight  away. 
The  development  of  the  concept  maps,  in  particular,  required  extensive  use  of  the  fax,  particularly  as  hand 
written  comments  on  the  drafts  of  Figure  1 were  crucial  to  the  final  outcome. 


prompting 


Microeconomic  1 
reform  I 


prompting 


Figure  2:  The  individual  concept  map  for  Week  12. 

The  development  of  the  online  communication  groups  was  less  problematic,  as  Monash  University  had  recently 
developed  a web-based  virtual  doorway  (the  MyMonash  Portal)  to  support  students’  administrative  and 
educational  needs  (Kennedy,  Webster,  Benson,  James,  & Bailey,  In  press).  Issues  of  managing  the  online 
discussion  groups  were  addressed  in  the  development  of  the  print -based  materials,  with  particular  focus  on  the 
assessment  linked  to  the  participation  in  three  of  the  discussion  groups,  and  access  to  appropriate  readings 
(Berge,  1995;  Mason,  1997;  Salmon,  2000). 

Evaluation  of  students’  perceptions  of  the  concept  maps 

This  small  study  was  with  a class  taught  in  1999,  with  the  evaluation  data  being  collected  early  in  2000  after  the 
completion  of  the  unit.  A questionnaire  that  focused  on  the  integration  of  the  concept  maps  in  the  written 
materials  was  given  to  both  the  off-campus  and  on -campus  students.  The  responses  to  selected  questions  are 
shown  below  in  Table  1 . None  of  the  students  had  any  prior  experience  with  concept  mapping.  There  were  three 
off-campus  students  and  all  replied;  there  were  48  on -campus  students,  but  only  eleven  students  responded  to 
the  questionnaire.  As  with  all  groups  of  students,  there  are  differing  levels  of  interest  in  the  unit  matter,  and 
diversity  of  learning  styles,  such  as  whether  students  were  ‘visual’  in  their  cognitive  approaches.  Few  of  the  on- 
campus  students  used  the  concept  maps;  they  cited  the  complexity  of  the  maps  and  their  lack  of  time  for  the 
reasons.  The  off-campus  students  appeared  to  use  the  maps  more  but  these  questionnaire  results  suggest  that 
they  found  the  concept  maps  more  useful  with  revision  rather  than  as  a primary  learning  tool. 


Q1 : Did  you  find  that  the  major  concept  map  was  informative?  Please  explain  your  answer. 

Off-campus 

On-campus 

• The  Concept  Map  was  useful  for  seeing  how 
all  of  the  sectors  of  the  course  interrelated.  It 
was  informative  although  I didn't  use  it  or 
retain  the  information  in  it  when  I was 
enquiring  what  was  up  next. 

• It  was  to  some  extent  as  it  visualised  the  big 
picture  of  economics  and  where  each  concept 
fitted  in.  However,  I only  ever  used  it  for 
reference  between  topics  to  see  how  things 
interacted. 

• Yes.  It  briefly  explains  the  basic  concepts  and  guides  us  as  to  the 
differences  (and  similarities)  between  macroeconomic  and 
microeconomics  throughout  the  semester. 

• By  the  end  of  the  course  I found  it  useful  to  assist  in  reviewing  the 
topics  studied  and  to  connect  them  together  in  context. 

• It  was  useful  to  look  at  once  or  twice,  but  I have  to  admit  I didn’t  really 
use  it. 

• No  - hard  to  follow  and  confusing. 

• I found  it  useful  to  review  prior  to  tests,  in  terms  of  ‘putting  it  all 
together’. 

Q2:  Do  you  believe  that  the  major  concept  map  assisted  you  in  your  overall  process  of  learning  in  this  unit? 

Off-campus 

On-campus 

• Yes,  but  not  until  later,  when  I had  the  earlier 
ideas  in  my  mind 

• It  was  a good  diagrammatical  representation  of 
the  course’s  concepts,  and  helped  me  to  know 
where  my  knowledge  was  lacking,  or  where  I 
needed  to  review  my  understanding 

• Helped  with  an  overview  of  the  topics  when  reviewing,  and  to 
understand  how  they  connected. 

• Didn’t  use  it. 

• When  I first  started  economics,  I was  confused  because  I had  to  learn 
so  many  things  in  a short  time.  The  concept  map  guided  me  in  finding 
the  right  approach  to  economics. 

• There  was  too  much  to  learn  within  a short  space  of  time. 
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03:  Did  you  attempt  to  construct,  as  the  study  guide  suggested,  your  own  concept  maps  of  your  understanding  of  economics? 

Off-campus 

On-campus 

• Because  I tend  not  to  be  a very  visual  person, 
they  do  little  to  expand  my  learning  or 
understanding  of  a topic.  Lots  of  other  people 
might  benefit  greatly  from  them  though 

• Had  no  time  and  the  existing  concept  maps 
were  good  enough  for  revision  etc. 

• Maybe  not  as  detailed  as  the  attached  concept  map,  but  it  helps 
simplify  relationships. 

• Didn’t  occur  to  me! 

• It  doesn’t  help  me. 

• No  time. 

• It  never  occurred  to  me.  I would  probably  do  it  in  the  future  given 
time. 

• Yes,  specific  issues  only:  not  ‘the  big  picture’, 

• Since  we  had  little  time  for  Economics  1,  to  understand  and  learn 
economics  in  a systematic  way  is  very  crucial.  Your  map  helped  me  a 
great  deal  in  understanding  the  subject, 

• Yes,  for  microeconomics  test. 

Table  1:  Sample  questionnaire  responses  for  both  groups  of  1999  students 

It  may  be  that  the  lack  of  familiarity  and  their  inability  (mainlya  lack  of  time)  to  develop  their  own  concept 
maps  were  problematic.  The  development  team  has  learnt  that  an  innovation  in  learning  design,  such  as  the  use 
of  concept  maps,  must  be  carefully  budgeted  for  in  terms  of  students’  time. Students  may  appreciate  these  maps, 
but  the  time  to  become  familiar  with  them,  and  to  explore  their  potential,  must  be  allocated  carefully.  The 
question  of  whether,  to  what  degree,  and  how  innovations  such  as  concept  mapping  and  online  discussion 
groups  should  be  assessed  has  not  been  resolved;  assessment  is  likely  to  become  one  of  the  main  curriculum 
issues  in  the  next  few  years.  The  use  of  the  discussion  groups  was  not  evaluated  by  specific  questionnaires,  or 
requests  for  specific  feedback.  However,  all  online  assessment  tasks  were  completed  extremely  well,  with 
students  able  to  articulate  a deep  understanding  of  specific  concepts. 

Unit  Learning  Outcomes 

With  concept  mapping  and  online  discussions  providing  a focussed  approach  to  the  learning  of  economics,  it 
may  be  anticipated  that  students  might  learn  more  about  the  unit.  This  added  learning  may  result  in  higher 
marks.  These  results  should  be  seen  as  anecdotal  rather  than  definitive,  if  only  due  to  the  small  sample  size.  In 
this  regard,  it  should  be  noted  that  the  nature  and  content  of  the  course  work  in  the  two  economics  units  taught 
in  the  Diplomacy  and  Trade  program  remained  relatively  unchanged  over  a two  -year  period  under  review  in  this 
paper.  Specifically,  assessment  details  were  available  for  the  year  prior  to  the  introduction  of  concept  mapping 
in  Economics  1,  as  well  as  in  the  year  during  which  Economics  1 concept  mapping  was  provided  as  an 
additional  educational  tool.  As  noted  earlier,  the  major  assignments  for  the  two  groups,  off-campus  and  on- 
campus,  were  similar.  Note  also  that  the  lecturer  was  the  same  for  all  cohorts.As  a consequence,  a comparison 
was  made  between  assessment  scores  gained  bystudents  who  had  been  exposed  to  concept  mapping  and  online 
discussion,  as  well  as  scores  gained  by  students  who  had  not  seen  either.  As  can  be  seen  in  Table  2,  scores  for 
individual  tests,  and  the  total,  were  higher,  on  average,  for  the  innovation  cohort.  An  overall  advantage  of  16.1 
per  cent  is  noted  between  successive  years. 


Assessment 

Pre-concept  mapping 

Post  concept  mapping 

Change  (%) 

First  test 

62.6 

76.4 

22.0 

Second  test 

60.6 

66.1 

9.1 

Third  test 

65.3 

82.9 

26.9 

Overall  result 

64.2 

74.6 

16.1 

Table  2:  Sequential  first  semester  results:  Before  and  after  concept  mapping 

Conclusions  and  Summary 

The  design  and  development  of  the  unit  materials  involved  the  establishment  of  trust,  understanding  and  good 
communication  between  the  content  expert  and  the  educational  designer.  The  current  structure  of  the  unit  is 
being  used  by  the  Faculty  of  Business  Economics  as  an  example  for  other  academics  who  wish  to  start  the 
transition  from  more  traditional  distance  education  materials  to  more  engaging,  interactive  ways  of  developing 
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student  knowledge.  The  book  (paper-based  unit  guide)  still  has  its  place,  but  integration  of  online 
communication  enables  students,  who  are  often  isolated  from  their  colleagues,  to  engage  with  their  peers, 
receive  feedback  from  the  lecturer  and  complete  assessment  tasks  that  are  more  relevant  to  the  desired  learning 
outcomes.  Whilst  there  are  potential  confounding  factors  at  work  when  making  absolute  comparisons  between 
two  groups  of  students  across  years,  the  anecdotal  inference  is  that  there  is  a benefit  to  students  who  have  been 
exposed  to  an  holistic  view  of  the  unit  by  the  use  of  concept  mapping  as  an  advance  organiser  and  revision  tool, 
and  had  the  opportunity  to  engage  in  asynchronous  online  discussions. 
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Abstract;  This  paper  examines  a multimedia  learning  design  which  utilises  a 
virtual  dental  clinic  and  a fictitious  eight  year  old  patient  - "Matthew",  to  situate 
students  within  an  authentic  learning  environment.  A case  is  presented  which 
requires  the  student  to  examine  clinical  information  about  "Matthew"  in  the  form 
of  patient  history,  clinical  slides,  radiographs  and  expert  information  from 
teachers,  psychologists  and  an  endocrinologist.  The  dental  student  creates  a 
legitimate  treatment  plan  for  "Matthew"  by  analysing  information  typically 
obtained  in  a real  patient  encounter,  A situatedleaming  design  was  adopted  as  it 
provided  a means  of  engaging  the  dental  student  with  a legitimate  case  of  a child 
with  diabetes,  A virtual  "Matthew"  was  created  because  it  has  become 
increasingly  difficult  to  access  suitable  patients,  especially  child  patients,  for 
dental  students.  Consequently  dental  students  may  not  be  adequately  experienced 
in  treating  paediatric  patients  with  medical  conditions  such  as  diabetes.  This 
scenario  aims  to  improve  dental  clinical  management  through  case  management 
of  medically  compromised  dental  patients  - in  this  case  a child  with  diabetes. 


Authentic  Learning  Environments 

Authentic  learning  experiences  are  "those  which  are  problem-or  case-based,  that  immerse  the  learner  in 
the  situation  requiring  him  or  her  to  acquire  skills  or  knowledge  in  order  to  solve  the  problem  or 
manipulate  the  situation"  (Jonassen,  Mayes,  & McAleese,  1992,  p,  235).  "Authentic  tasks  enable  students 
to  immerse  themselves  in  the  culture  of  the  academic  domain,  much  like  an  apprentice"  (Young,  1993,  p, 
43),  Interest  in  environments  that  immerse  students  in  authentic  learning  experiences,  where  the  meaning 
of  knowledge  and  skills  are  realistically  embedded,  has  been  longstanding  (Dewey,  1938;  Piaget,  1952), 
More  recently,  attempts  to  enhance  cognition  in  authentic  learning-performing  tasks  have  become 
widespread  (e,g,.  Brown,  Collins,  &Duguid,  1989;  Cognition  and  Technology  Group  at  Vanderbuilt, 
1991,  1992,  1993),  The  need  for  realistic  experience  and  real- world  problem  solving  is  also  advocated, 
Ramsden  (1987)  suggests  that  university  graduates  are  not  obtaining  deep  conceptual  knowledge  that  will 
allow  them  to  think  like  experts  in  their  discipline.  We  contend  that  a key  role  of  a university  education  is 
to  produce  health  professionals  who  are  capable  of  solving  complex  and  ill -structured  problems  and  have 
the  ability  to  reflect  on  their  professional  practice.  This  practitioner  is  capable  of  reflecting  on  the 
practical  demands  of  the  real  world,  continually  monitoring  their  own  professional  thinking  for  the 
ultimate  aim  of  solving  related  or  new  professional  problems.  However,  there  is  a continuum  of 
knowledge  acquisition  that  leads  to  a fully-fledged  expert  in  professional  practice.  Jonassen,  Mayes,  and 
McAleese  (1992)  suggest  a continuum  of  introductory,  advanced  and  expert  learning  phases.  The 
introductory  phase  is  typified  by  initial  knowledge  acquisition  by  the  novice  learner  which  involves 
learning  about  well -structured  content  domains.  The  advanced  phase  deals  with  ill -structured  domains. 


"During  this  phase,  learners  acquire  more  advanced  knowledge  in  order  to  solve  more  complex,  (bmain- 
or  context-dependent  problems"  (Jonassen,  Mayes  & McAleese,  1992,  p.  231).  Expertise  is  the  third 
phase  in  the  continuum.  Experts  appear  to  be  able  to  uniquely  represent  problems,  which  facilitates  the 
solving  of  problems  in  an  efficient  and  effective  manner.  The  role  of  providing  authentic  learning 
experiences  in  the  university  setting  is  an  attempt  to  immerse  the  learner  in  the  advanced  phase  in  order 
to  move  them  toward  the  development  of  expertise. 

Authentic  learning  contexts  like  the  virtual  dental  clinic  may  have  a number  of  advantages  over  more 
decontextualised  teaching  and  learning  settings.  The  authentic  nature  of  the  technology -enhanced, 
student -centred  learning  environment  may  create  a context  within  which  knowledge  is  anchored  in 
authentic  contexts.  An  effective  learning  environment  enables  learners  to  use  its  resources  and  tools  to 
process  more  deeply  and  extend  thinking  (Jonassen,  1996;  Jonassen  & Reeves,  1996;  Kozma,  1987). 
Harper,  Squires  and  McDougall  (2000)  suggest  that  "only  in  complex,  rich  environments  will  learners 
have  the  opportunity  to  construct  and  reconstruct  concepts  in  idiosyncratic  and  personally  meaningful 
ways"  (p.  1 1 8). 


Learning  Design 

Teaching  dental  students  to  treat  patients  safely  and  efficiently  presents  many  challenges.  The  declining 
availability  of  suitable  patients  attending  the  teaching  clinics  of  the  Royal  Dental  Hospital  of  Melbourne 
in  association  with  the  University  of  Melbourne  prevents  students  from  developing  a range  of  experiences 
in  total  patient  management.  A disadvantage  of  traditional  preclinical  laboratory  teaching  is  that  students 
are  not  able  to  integrate  theoretical  and  practical  skills.  Consequently  there  are  concerns  that  dental 
students  are  not  competent  in  combining  preventive  and  restorative  management  philosophies  while 
integrating  diagnosis  and  treatment  planning  (Suvinen,  Messer,  Franco,  1998).  Multimedia  case 
simulations  were  considered  to  be  a viable  alternative  as  they  replicate  the  dental  clinic  without  requiring 
'live'  patients.  The  diabetes  module  provides  an  opportunity  to  develop  and  consolidate  the  concept  of 
integrated  patient  care.  In  particular,  the  learning  outcomes  for  the  modules  focus  on:  (1)  competence  in 
history,  examination,  diagnosis  and  treatment  planning  skills,  (2)  integration  of  theoretical  and  practical 
aspects  of  total  patient  management,  (3)  understanding  of  the  relevance  of  the  medical  condition  to  dental 
health  and  dental  treatment  with  attention  to  precautions  when  delivering  treatment,  and  (4)  recognition 
of  the  significance  of  medical  conditions  in  treatment  planning  for  their  dental  condition. 

In  order  to  create  a realistic  learning  experience  for  the  student  dentist  it  was  essential  that  we  immersed 
the  student  in  an  authentic  case.  We  utilised  principles  of  situated  learning  which  suggests  that  there 
should  be  "an  active  relationship  between  an  agent  and  the  environment,  and  learning  must  take  place 
during  the  time  the  student  is  actively  engaged  in  with  a complex,  realistic  instructional  context"  (Young, 
1993,  p.  45).  We  utilised  the  following  principles  of  authenticity  to  guide  our  creation  of  the  diabetes 
scenario.  These  include: 

• real-life  problem-solving 

• ill-structured  complex  goals 

• opportunity  for  the  detection  of  relevant  versus  irrelevant  information 

• active/generative  engagement  in  finding  and  defining  problems 

• involvement  of  student's  beliefs  and  values 

• opportunity  to  engage  in  collaborative  interpersonal  activities  (Young,  1993,  p.  45).  Likewise 
Herrington  and  Oliver  (1995)  suggest  nine  critical  characteristics  of  situated  learning  which  focus  on  the 
interactive  multimedia  program,  implementation  and  the  learner. 

The  virtual  dental  clinic  provides  an  authentic  learning  environment  in  which  the  trainee  dentist  can 
examine  information  patient  records,  clinical  slides,  radiographs  and  expert  reports)  about  a paediatric 
patient  with  diabetes.  This  approach  attempts  to  immerse  the  learner  within  a typical  clinical  scenario  and 
then  asks  them  to  create  a relevant  treatment  plan.  The  virtual  case  attempts  to  bridge  the  gap  between 
novice  and  expert  by  asking  the  trainee  dentist  to  begin  thinking  and  acting  like  an  expert  dentist. 

The  first  step  in  our  design  process  was  to  select  a situation  suitable  for  authentic  situated  learning. 
Previou  sly  we  had  created  two  other  modules  on  Congenital  Heart  Disease  and  Down  syndrome  in  which 
scenarios  were  used  to  engage  the  student  with  the  relevant  content.  A module  on  diabetes  created  unique 
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challenges  for  the  design  team.  A virtual  paediatric  patient  with  diabetes  was  created  in  order  to  address 
the  difficulty  of  obtaining  relevant  patient  photographs.  In  our  interactions  we  wanted  to  increase  the 
level  of  student  engagement  with  the  content  and  embarked  on  contextualising  the  scenario  content  within 
a virtual  dental  clinic.  We  were  also  careful  not  to  overwhelm  the  dental  student  with  an  overly  complex 
case.  "While  complexity  may  be  necessary  to  provide  authentic  learning  environments,  too  much 
complexity  can  make  learners  feel  insecure  and  lose  track  of  learning  objectives"  (Harper,  Squires  & 
McDougall,  2000,  p.  127).  This  setting  would  allow  the  embedding  of  authentic  activities  such  as  the 
examination  of  radiographs  and  patient  information  within  a realistic  clinical  situation.  We  began  by 
photogr^hing  the  dental  clinic  and  then  creating  the  virtual  dental  clinic  and  waiting  room.  Figure  1 
illustrates  the  virtual  dental  clinic  with  "Matthew",  the  virtual  patient. 
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Figure  1:  Virtual  dental  clinic  and  drop-down  menu  which  allows  access 
to  clinical  information  about  "Matthew" 


Secondly,  we  provided  multiple  entry  points  into  the  clinical  information.  This  allowed  the  user  to 
explore  the  clinic  and  obtain  the  necessary  clinical  information  about  Matthew.  We  wanted  the  student  to 
"criss-cross  the  landscape  of  knowledge"  in  order  to  obtain  information  that  would  enable  them  to 
complete  the  relevant  treatment  plan  (Young,  1993,  p.  46).  The  user  can  navigate  around  the  clinic  and 
find  information  in  two  ways.  By  travelling  around  the  clinic  they  will  find  ’hotspots'  which  provide 
clinical  information.  A site  map  also  provides  clinical  information  enabling  the  student  to  access  patient 
records  (patient  information,  medical  history,  height  and  weight,  dental  history  and  social  history),  seven 
clinical  sfides,  three  radiographs,  and  expert  information  from  a teacher,  psychologist  and 
endocrinologist.  By  providing  the  necessary  scaffolding  for  the  novice  learner  we  attempted  to  move  the 
learner  toward  expert  case  management. 
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The  Case 

Presenting  Complaint: 

"Matthew"  is  8 years  old.  He  presented  with  sensitive  teeth,  the  left  side  being  worse  than  the  right.  His  mother 
mentioned  that  he  will  not  brush  his  teeth  because  they  hurt.  She  is  concerned  that  he  has  not  been  eating  well  during 

the  past  few  weeks. 

Dental  History: 

"Matthew’s"  last  dental  visit  was  when  he  was  5 years  old.  The  school  dental  service  performed  a checkup  and  no 
treatment  was  required  at  that  stage.  Matthew  brushes  his  own  teeth  once  a day  and  does  not  floss  his  teeth.  He  has 

fair  oral  hygiene. 

Medical  History: 

This  young  boy  was  diagnosed  with  insulin  dependant  diabetes  at  14  months  of  age.  He  has  2 injections:  before 
breakfast  and  dinner.  The  injections  consist  of  a combination  of  short -acting  and  intermediate  insulin.  "Matthew’ s" 
parents  give  the  injections.  "Matthew"  has  a high  carbohydrate  diet  with  frequent  intakes  throughout  the  day. 
"Matthew"  has  regular  medical  checkups  with  his  paediatric  endocrinologist  every  three  months.  His  diabetes  is  well 

monitored  and  stable.  There  have  been  no  complications. 

Height  and  weight  measurements: 

Height:  130  cm  (75*^  percentile);  Weight:  27  kg  (50  *^-75^  percentile) 

Social  History: 

"Matthew"  was  bom  in  Victoria  and  lives  with  his  parents  and  two  older  brothers.  He  attends  a primary  sdiool  close 
to  home.  He  has  tennis  lessons  every  Sunday  morning,  swimming  lessons  on  Thursday  afternoons  and  last  year  he 
started  rugby  on  Saturday  mornings.  "Matthew"  has  cello  lessons  on  Tuesday  after  school.  "Matthew"  belongs  to  the 

local  cub  pack  and  they  meet  every  Monday  evening. 

Teacher 

Due  to  his  checkups  at  the  hospital,  "Matthew"  is  absent  from  school  for  a significant  number  of  days.  He  is  in  an 
age-appropriate  level  at  school  and  has  remedial  teaching  to  keep  him  at  the  same  level  as  his  class.  The  other 
students  are  aware  of  "Matthew’s"  medical  condition  and  they  are  supportive  of  him,  especially  when  he  misses  a day 
of  school  to  go  for  his  checkups.  He  has  many  friends  in  his  class.  "Matthew"  does  not  need  his  insulin  injections 

during  school  hours.  In  case  of  emergencies,  "Matthew's"  teachers  will  contact  his  parents  for  instructions. 

Endocrinologist 

"Matthew"  has  been  a diabetic  for  7 years.  He  attends  for  regular  checkups  every  3 months  to  assess  his  blood 
glucose  control  and  to  decide  if  fliere  is  any  need  for  adjustments  in  medication.  He  adapted  well  to  a modified  diet 
and  daily  administration  of  medication  by  his  parents.  There  was  a period  of  time  when  "Matthew"  experienced  some 
frustration  regarding  his  lifestyle.  His  parents  were  supportive  through  this  episode.  If  any  treatment  under  general 
anaesthesia  is  required,  it  should  be  performed  in  a major  hospital  so  that  the  endocrinologist  can  monitor  "Matthew" 
and  adjust  his  glucose  level.  "Matthew"  should  be  admitted  the  day  prior  to  the  procedure  for  blood  glucose 
monitoring.  He  will  be  able  to  return  home  on  the  same  day  as  the  treatment  if  his  glucose  level  is  stable.  It  is 

important  that  "Matthew"  has  not  had  any  recent  episodes  of  hypoglycemia. 

Psychologist 

A year  ago  "Matthew"  went  through  a difficult  stage.  He  was  angry  that  he  needed  his  insulin  injections  twice  a day. 
During  medication  times,  he  would  refuse  the  injections  and  his  parents  would  spend  significant  time  convincing 
"Matthew"  to  have  his  medication.  At  birthday  parties  "Matthew"  was  upset  that  he  could  not  eat  the  whole  bag  of 
sweets  like  other  children.  I consulted  with  "Matthew"  and  his  parents  over  a number  of  sessions  and  these  issues 
were  resolved.  


Table  1:  Case  information  that  can  be  accessed  in  the  virtual  dental  clinic 

Our  third  strategy  provided  an  expert-supported  learning  environment.  Lab  sessions  were  scheduled  in 
which  students  interacted  with  the  module  and  were  supported  by  specialist  paediatric  dentists.  "From  the 
perspective  of  situated  cognition,  the  teacher's  role  should  be  to  'tune  the  attention'  of  students  to  the 
important  aspects  of  the  situation  or  problem-solving  activity,  specifically  those  attributes  that  are 
invariant  across  a range  of  similar  problems  and  therefore  will  transfer  to  many  novel  situations"  (Young, 
1993,  p.  47).  Educators  also  assisted  the  dental  academics  in  the  tutorial  setting  by  facilitating  their 
efficient  use  of  the  technology  -enhanced  environment.  "Instructional  designs  for  situated  learning  must 
not  only  provide  scaffolding  for  students:  they  must  also  provide  scaffolding  for  teachers"  (Young,  1993, 
p.  51).  The  advantage  of  expert  support  is  that  case  information  could  be  clarified,  explained  and 
elaborated  by  the  specialist.  The  use  of  specialist  paediatric  dentists  was  a central  focus  of  our  approach 
as  they  were  able  to  support  learning  and  monitor  student  engagement  with  the  module.  By  providing  this 
support  it  meant  that  we  did  not  need  to  overly  simplify  the  authentic  task  (Barab  & Duffy,  2000). 

Young  (1993)  suggests  that  traditional  forms  of  assessment  in  situated  learning  may  prove  to  be 
inadequate.  Authentic  learning  tasks  must  be  assessed  using  methods  that  best  align  with  the  task.  Our 
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fourth  strategy  was  to  align  activities/assessment  with  the  creation  of  the  authentic  learning  situation.  We 
matched  the  task  and  assessment  by  asking  students  to  develop  a treatment  plan  using  the  protocols 
utilised  in  a traditional  clinic.  In  order  to  simulate  the  use  of  this  chart,  an  electronic  dental  chart  was 
created  to  allow  users  to  create  treatment  plans  within  the  multimedia  module.  The  electronic  dental  chart 
allows  the  trainee  dentist  to  select  a category  of  dental  services  and  allocate  a specific  treatment  for 
individual  teeth.  This  chart  also  allows  the  user  to  complete  cases  and  submit  information  to  obtain  expert 
feedback.  The  expert  feedback  provides  precise  information  about  the  treatment  protocol  for  each  tooth. 
The  student  can  compare  their  treatment  plan  to  the  expert  and  then  re-enter  the  virtual  clinic  to  rectify 
any  inaccuracies  and  misconceptions.  This  concept  of  authentic  assessment  tasks  is  essential  in  the 
instructional  design  as  "assessment  should  be  a seamless,  continuous  part  of  the  activity  (a 
leaming/assessment  situation)"  (Young,  1993,  p.  48). 
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Figure  2:  Electronic  treatment  plan  utilised  for  the  diabetes  clinical  case 

Our  fifth  strategy  was  to  provide  detailed  feedback  to  the  student  in  relation  to  the  case.  At  the  conclusion 
of  the  case  and  after  reinforcement  activities  the  student  is  pro\ided  with  comprehensive  patient  feedback 
in  relation  to  "Matthew’s"  case.  The  following  information  provides  an  example  of  this  detailed  feedback 
about  the  case.  "The  patient  with  diabetes  should  be  placed  on  a frequent  recall  maintenance  program 
after  dental  treatment  is  completed  until  his  susceptibility  to  recurrent  oral  disease  has  been  established. 
Regular  topical  fluoride  applications  may  be  considered  at  home  and  professionally.  Sealants  can  be 
placed  as  the  permanent  teeth  erupt.  A mouthguard  for  sports  activities  will  need  regular  replacement 
with  development  of  the  dentition.  Patients  and  parents  should  be  aware  of  the  importance  of  oral  health 
in  relation  to  the  diabetic  condition.  The  dentist  should  take  time  to  inform  them  of  the  potential  for 
increased  susceptibility  to  oral  infection  and  periodontal  disease  in  the  inadequately  controlled  diabetic 
patient.  The  well-controlled  diabetic  patient  will  have  good  oral  health  especially  if  the  child  maintains 
good  home  oral  care  and  continues  recall  compliance". 


Conclusion 

Our  next  step  in  the  process  is  to  create  a number  of  scenarios  that  may  foster  transfer  to  similiar  cases 
and  to  more  ill-defined  cases.  This  is  important  as  "one  of  the  major  concerns  about  situating  instruction 
in  specific  contexts  is  that  students’  understanding  and  application  of  these  concepts  will  stay  welded  to 
the  context"  (Cognition  and  Technology  Group  at  Vanderbuilt,  1993,  p.  52).  In  order  to  address  this  issue 
we  intend  to  construct  new  dental  scenarios  that  are  direct^  analogous  to  the  diabetes  case.  This  is 
considered  to  be  'transfer  to  analogous  problems'.  The  virtual  dental  clinic  can  be  reconfigured  with 
different  information  and  different  case  parameters.  Individual  treatment  plans  can  be  developed  in 
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conjunction  with  specialist  paediatric  dentists  utilising  other  cases  of  importance  to  the  learner.  Numerous 
cases  can  be  completed  allowing  the  trainee  dentist  to  become  familiar  with  expert  dentist  strategies  and 
treatment  plans.  The  overall  aim  of  designing  authentic  learning  tasks  is  to  improve  dental  health  care. 
Once  we  have  achieved  transfer  to  real-contexts  the  individual  dentist  should  continue  to  adapt  to 
complex  real  -world  cases. 
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Abstract:  The  Clinicians  Health  Channel  (CHC)  is  a resource -based  portal 
website  providing  online  access  to  information  sources  that  assist  Victorian 
clinicians  in  decision  making,  research  and  education.  Resources  include  an 
electronic  health  library  with  citation  databases  and  full  text  journals,  links  to 
clinical  reference  materials  and  guidelines  & protocols.  An  inter-disciplinary, 
collaborative  team  from  the  University  of  Melbourne  is  undertaking  an 
evaluation  to  determine  how  successfully  the  introduction  of  the  CHC  has  met 
its  original  aims  specifically  in  relation  to  the  development,  implementation 
and  impact  on  changing  behaviour  and  increasing  usage  of  evidence-based 
health  care.  The  addition  of  online  communities  of  practice  may  assist  the 
sharing  of  practitioner  knowledge  and  may  complement  the  resource -based 
nature  of  the  channel. 


Rationale 

Clinical  decisions  about  options  for  a patient’s  care  should  be  based  on  both  individual  clinical 
expertise  and  best  available  external  clinical  evidence  (Sackett,  Rosenberg,  Gray,  Hayes  & 
Richardson,  1996).  However,  it  has  been  estimated  that  for  the  medical  profession  to  keep  up  to 
date  with  advances  in  drugs,  treatment  and  technologies  alternatives,  they  would  need  to  read  19 
articles  per  day,  365  days  of  the  year  (Evidence-based  working  group,  1992).  Information 
resources  made  available  to  clinicians  ia  telecommunication  are  increasingly  being  advocated 
because  it  is  believed  that  they  can  provide  better  access  to  and  use  of  the  evidence  to  allow  the 
clinician  to  keep  up  to  date  and  enhance  the  quality  of  care  (Lindberg,  Siegel,  Rapp,  Wallingford 
& Wilson,  1993;  Elson  & Connelly,  1995). 

However,  these  resources  have  had  limited  use  in  direct  patient  care  settings,  meet  only  a small 
part  of  clinicians'  information  needs,  and  have  not  had  significant  impact  on  clinical  practice. 
While  some  may  disagree  wth  the  need  to  prove  benefit  to  advocate  the  use  of  information 
resources,  most  will  agree  with  the  need  for  further  research  into  the  content  and  delivery  methods 
as  well  as  the  social  and  behavioural  impacts  of  such  resources  on  clinicians,  their  patients,  work 
practices  and  organisations  (Gorman,  Ash  & Wykoff,  1994;  Covell,  Uman  & Manning,  1985; 
NHS  Centre  for  Reviews  and  Dissemination,  1999;  Hersch  & Hickam,  1998).  Our  evaluation  is 
thus  concerned  with  the  use  of  evidence-based  information  in  the  context  of  clinical  practice,  and 
with  the  use  of  a web-based  portal  to  facilitate  evidence  based  practice. 
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The  introduction  of  the  Clinicians  Health  Channel  (CHC)  is  a four-year  project  which  aims  to 
provide  equity  of  access  to  critical  clinical  knowledge  resources  for  clinicians  in  the  public  health 
care  sector  throughout  Victoria.  The  CHC  nittp://www.clinicians.vic.gov.au/l  is  a portal  web-site 
providing  online  access  to  current,  accurate  and  reliable  sources  of  information  to  help  inform  and 
assist  clinicians  in  decision-making,  research  and  education. 


Evaluation  Approach 

Our  approach  to  the  evaluation  is  notable  in  three  key  respects.  Firstly,  the  evaluation  is 
conceptualized  as  the  evaluation  of  strategic  change  to  work-practices,  not  simply  as  the 
evaluation  of  a web-site.  Secondly,  and  as  a consequence  of  this  broad  conceptualisation,  the  team 
is  multidisciplinary  - made  up  of  people  with  experience  and  expertise  in  medicine,  multimedia 
design,  educational  evaluation,  socio-technical  relations,  workplace  ethnography,  and  qualitative 
and  quantitative  research  methods.  Thirdly,  the  approach  is  one  that  intends  to  be  formative  as 
well  as  summative,  and  takes  the  CHC  to  be  an  emergent  system  rather  than  stable  or  fixed. 

The  evaluation  is  an  integral  part  of  improving  the  quality  and  usefulness  of  the  CHC.  The 
University  of  Melbourne  team  has  adopted  a collaborative  approach  to  research  and  evaluation  of 
the  CHC,  and  the  wide  range  of  disciplines  represented  on  the  team  is  a major  strength.  The 
evaluation  focuses  on  issues  related  to  professional  clinical  practice  (specifically,  an  anticipated 
increase  in  the  practice  of  evidence-based  health  care),  and  on  issues  related  to  technological 
mediation  (specifically,  the  use  of  the  web  to  achieve  this  objective).  These  two  intertwined 
research  areas  require  an  analysis  through  three  dimensions  - social,  behavioural  and  technical  - 
and  these  constitute  our  conceptual  framework  for  the  evaluation  of  the  CHC. 

The  social  dimension  focuses  on  the  regional,  institut  ional,  professional  and  education  context  in 
which  clinical  practice  occurs.  It  acknowledges  the  fact  that  the  CHC  is  situated  in  particular 
social  spaces  and  is  attentive  to  those  spaces.  It  addresses  and  evaluates  the  aim  of  the  CHC  to 
provide  equity  of  access  to  critical  clinical  knowledge  resources  for  clinicians  in  the  public  health 
care  sector  throughout  Victoria.  The  evaluation  questions  will  include  the  "who,  when,  where"  as 
well  as  impacts  of  the  CHC  on  accessibility  of  resources.  The  behavioural  and  professional 
dimension  focuses  on  the  extent  and  character  of  the  intersection  or  overlap  of  the  CHC  program 
and  evidence  based  practice.  It  addresses  and  evaluates  the  aim  of  the  CHC  to  support  integration 
of  evidence  based  practice  into  the  health  care  system.  The  evaluation  will  address  the  use  of  the 
CHC  by  clinicians  and  whether  it  met  their  information  needs.  We  will  identify  knowledge 
resources,  awareness  of  available  resources,  ability  to  access  these,  current  usage  patterns  and 
factors  influencing  utilisation.  The  evaluation  will  also  address  any  impact  on  decision  making  by 
measuring  clinician  change  and  examining  the  barriers/opportunities,  any  relevant  forces  at  work, 
and  strategies  to  positively  influence  behaviour.  The  technical  dimension  addresses  and  evaluates 
the  aim  of  the  CHC  to  facilitate  electronic  dissemination  of  these  resources  and  other  clinically 
relevant  information,  so  that  the  information  can  be  accessed  whenever  and  wherever  required. 
The  evaluation  will  examine  the  useability  of  the  system  in  terms  of  functionality,  frequency  of 
use,  purpose  of  use,  user  satisfaction,  user  friendliness,  time  required  to  locate  information, 
searching  utility,  success  or  failure  of  searches  for  relevant  information,  flexibility,  and 
customisation.  Training  and  support  will  also  be  examined  here  as  we  consider  it  an  integral 
component  of  the  CHC  program.  Evaluation  areas  will  include  teaching  and  learning  needs, 
critical  appraisal  skills,  evaluation  of  teaching  and  learning,  best  practice  teaching  methods, 
training  models,  and  the  development  of  a tool  kit  for  trainers. 


Questionnaires 

The  first  stage  of  the  baseline  evaluation  comprised  a paper  based  questionnaire  survey  (online). 
This  survey  was  provided  to  both  users  and  non- users  of  the  CHC  across  Victoria.  We  focussed  on 
both  users  and  non-users  of  the  CHC  as  we  wanted  to  begin  with  how  clinicians  make  decisions 
before  investigating  how  clinicians  use  electronic  sources  of  information  for  making  decisions. 
Over  600  surveys  were  distributed  across  Victoria  to  114  hospitals.  233  surveys  were  returned. 
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153  female  and  75  males  responded  to  the  survey.  The  average  number  of  years  respondents 
worked  in  the  health  care  profession  was  18.65  years.  Fifty-five  allied  health,  49  medical  doctors, 
102  nurses  and  26  others  (managers,  administrators,  etc)  responded  to  the  survey.  A detailed 
examination  of  the  survey  results  will  not  be  examined  in  this  paper.  However  we  will  examine  a 
question  on  'how  clinicians  use  information  resources  to  make  a clinical  decision'. 
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Table  1;  Information  resources  utilised  by  clinicians 
(doctors,  nurses,  allied  health)  to  make  a clinical  decision 

The  most  frequent  resources  used  to: 

• guide  current  treatments  included:  paper-based  clinical  guidelines  and  protocols, 

workshops/seminars,  journals  and  consultation  with  colleagues. 

• assist  with  diagnosis  included:  consultation  with  colleagues,  textbooks,  journals  and 
academic-based  websites. 

• check  patterns  of  illness  included:  textbooks,  journals,  consultation  with  colleagues  and 
internet  search  engines. 

• confirm  my  actions  included:  consultation  with  colleagues,  paper -based  clinical  guidelines 
and  protocols,  workshop/seminars  and  textbooks. 

• refresh  my  knowledge  included:  workshop/seminars,  conferences,  textbooks,  journals. 
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• access  local  information  included:  staff  meetings,  local  services  directory,  consultation  with 
colleagues  and  workshops/seminars. 

• for  background  research  included:  journals,  academic  based  websites,  internet  search  engines 
and  computer-based  journals. 

As  seen  from  Table  1 and  the  above  analysis,  clinicians  carefully  choose  information  resources 
appropriate  for  the  function.  The  most  frequent  resource  utilised  to  'assist  with  diagnosis'  and 
'confirm  my  actions'  involved  consultation  with  colleagues.  Consultatio  n is  a high  priority  in  six 
out  of  seven  clinical  decision  making  activities.  Electronic  resources  appear  to  be  extensively  used 
for  background  research  as  opposed  to  obtaining  point  of  care  information.  This  emphasis  on  peer 
consultation  for  making  decisbns  is  supported  by  Young  and  Ward,  (2001)  who  suggested  that 
"the  most  frequently  nominated  source  of  information  participants  actually  would  turn  to  in  a time 
of  clinical  uncertainty  were  specialists  colleagues  (74%),  textbooks  (36%),  other  GPs  (16%)  and 
the  Internet  (12%)".  (p.  207).  This  is  understandable  where  colleagues  are  available  for 
consultation  in  large  metropolitan  hospitals.  However,  a major  part  of  our  evaluation  is  to  examine 
how  resources  can  be  accessed  more  effectively  by  rural  clinicians  throughout  the  state  of 
Victoria.  This  finding  may  suggest  implications  for  the  redesign  of  the  CHC  website  in  order  to 
provide  equity  of  access  to  expert  colleagues  in  rural  areas.  Presently  the  site  acts  as  a resource - 
based  portal  whereas  isolated  clinicians  may  require  access  to  colleagues  either  in  person  or 
through  an  online  community. 


Focus  Groups 

In  addition  to  the  surveys,  eight  focus  groups  have  been  completed  at  six  sites  chosen  to  represent 
a range  of  size,  rurality  and  level  of  enthusiasm  and  readiness  (infrastructure  and  organizational 
processes)  for  the  CHC.  The  focus  groups  were  conducted  as  open-ended  interviews  on  targeted  or 
focussed  issues  surrounding  the  introduction  of  computer-based  resources  to  Victorian  public 
hospitals.  There  were  a number  of  factors,  which  were  indicated  to  influence  clinical  decision- 
making. These  included:  routines,  traditions,  commonsense,  doctor  preferences,  patient  needs, 
cost,  availability  of  medical  service,  number  of  staff  available  and  time  available.  As  a result, 
decision-making  was  seen  as  situational,  depending  on  the  information  available  at  the  time. 

Focus  group  participants  used  computers  to  access  patient  records,  email,  results  and  local 
protocols.  Less  commonly  they  were  also  used  to  access  clinical  information.  Issues  of  time, 
training  and  perceived  clinical  need  for  information  would  decide  whether  clinicians  would 
become  regular  users.  Participants  wanted  more  efficient  computers  and  more  portable  computers, 
so  that  they  could  be  utilised  at  the  bedside.  Speed  of  computers  was  a common  concern  for  the 
participants.  The  design  of  online  resources  utilised  by  busy  clinicians  in  emergency  departments 
or  clinics  needs  to  optimise  time  and  minimise  tiers  of  information.  ^Any  web  site  [requires] 
minimum  key  strokes  — like  to  get  the  information  quickly.  So  even  if  our  computers  are  slow  or 
fast,  if  you  \e  just  got  the  function  keys  to  get  into  certain  things  rather  than  accessing  all  the  sub 
menus  -short  cuts'.  Ease  of  use  was  an  important  aspect  of  the  computer  environment.  Clinicians 
want  computers  and  resources  that  are  both  useful  and  user-friendly.  However,  lack  of  resources, 
training  and  time  prevented  clinicians  from  making  best  use  of  computers. 

Interviews 

Interviews  were  structured  around  the  themes  of  the  study  - in  this  case  information  needs, 
resources  use  and  computer  experience.  The  project  interviews  were  semi-structured,  with 
questions  asked  around  the  themes  and  opportunity  for  interviewees  to  discuss  other  related  issues. 
Interviews  examined  participants  detailed  personal  accounts.  Twenty-four  interviews  have  been 
undertaken  and  are  currently  being  analysed.  In  the  preliminary  analysis  of  interviews  there 
appears  to  be  a number  of  consistent  themes.  In  particular  there  were  numerous  comments  on  the 
applicability  of  databases  for  research  as  opposed  to  point  of  care  information.  For  instance  some 
databases  provide  background  research  whereas  others  provide  specific  point  of  care  information. 
It  is  also  a consistent  theme  that  clinicians  desire  a ‘readers  digest’  of  pertinent  information  and 
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that  access  to  suitable  computers  would  assist  CHC  usage.  It  is  also  suggested  that  informal 
training  and  support  is  more  appropriate  then  formal  large  group  sessions.  Understandably, 
isolated  rural  clinicians  also  desire  access  to  other  professional  colleagues  or  people  resources. 
The  use  of  a 24  hour  help  desk  could  provide  assistance  to  the  isolated  clinician. 

Design  Principles  for  the  CHC  Website 

In  completing  the  evaluation  for  stage  1 of  the  project  it  was  essential  to  focus  on  how  clinicians 
use  all  types  of  resources  to  make  a decision  - not  just  web-based  resources.  In  the  second  stage  of 
the  project  we  are  utilising  a variety  of  methodologies  including:  focus  groups,  interviews,  online 
questionnaires,  direct  observation  and  log  analysis  in  an  attempt  to  improve  the  CHC  portal 
website.  Part  of  our  discussions  as  an  evaluation  team  focus  on  redesign  of  the  website.  These 
redesign  principles  are  based  on  multimedia  and  online  experts'  evaluation  of  the  existing  website 
and  an  analysis  of  the  surveys,  focus  groups  and  interviews  to  date.  By  accounting  for  these  user 
viewpoints  we  recommend  a number  of  principles  that  may  improve  clinical  use  of  the  CHC. 

1.  The  current  website  needs  to  be  restructured  conceptually  in  order  to  improve  navigation, 
useability  and  access  to  the  resources.  Time  is  a precious  commodity  in  the  health  care  setting 
as  suggested  by  the  following  quote./  want  to  get  in,  get  it,  go  to  the  bottom  line  and  get  out  — 
I don*  t want  to  be  in  there  for  half  an  hour.  The  use  of  a site  map,  clearer  layout  and  the 
application  of  instructional  design  principles  will  assist  this  process. 

2.  Further  instructional  design  on  the  CHC  website  should  reduce  the  time  required  to  access 
informdion  and  make  searching  for  relevant  information  more  intuitive.  The  application  of 
these  principles  should  provide  a seamless  entry  into  using  online  resources  for  clinical 
practice.  One  participant  commented  that  clinicians  need  predigested  information  on  specific 
topics  that  you  encounter  frequently  in  clinical  practice,  you  want  a readers  digest  version, 
you  want  to  know  what  to  do  and  you  want  to  know  quickly.  For  unusual  cases  you  want  to  be 
able  to  identify  relevant  literature  and  examine  it  yourself  Frequently  asked  questions 
(FAQs)  may  also  assist  novice  users  and  their  utilisation  of  resources  best  suited  to  clinical 
practice.  The  use  of  FAQs  may  provide  guidelines  for  "newcomers  to  move  toward  full 
participation  in  the  sociocultural  practices  of  a community"  (Lave  & Wenger,  1991,  p.  29).  It 
is  envisioned  that  these  FAQs  would  be  continually  updated  with  user  experiences  and  hints. 

3.  Training  is  a major  aspect  of  encouraging  utilisation  of  the  CHC  for  clinical  practice.  Face-to- 
face  training  is  currently  being  undertaken  by  the  Department  of  Human  Services.  However 
there  needs  to  be  a variety  of  training  strategies.  / think  a lot  of  the  time  people  haven  V got  the 
time  to  get  to  the  education  sessions,  it  needs  to  be  a little  bit  more  user  friendly  for  people 
who  just  want  to  get  on  and  go  in.  Another  project  on  the  CHC  is  developing  online  tutorials 
to  assist  clinicians  use  of  the  channel.  The  aim  of  these  tutorials  is  to  teach  users  evidence- 
based  practice  and  critical  appraisal  of  research  resources.  By  utilising  this  approach  the  user 
will  learn  how  to  use  the  databases  and  search  mechanisms  and  also  efficiently  navigate 
around  the  CHC.  Training  was  also  seen  as  a critical  issue  with  participants  asking  for  local, 
individualised  training  at  their  worksites  as  well  as  more  help  on  the  CHC  site  itself. 
Suggestions  included  context  -specific  help,  on-line  tutorials  and  just-in-time  training. 

4.  The  final  concept  focuses  on  assisting  clinicians  in  remote  areas  to  interact  with  colleagues. 
Currently  the  CHC  is  a web-based  portal  which  allows  users  to  access  resources  similiar  to  a 
library  catalogue.  Stuckey,  Lockyer  & Hedberg,  (2001)  outline  a model  for  evaluating  the 
characteristics  of  portals,  networks,  interest  groups  and  online  communities.  They  suggest  that 
portals  are  resource-based  entities  where  consumers  may  be  passive,  have  varied  membership 
with  no  ties  between  members  and  allow  users  to  link  to  resources  and  indexed  sites  that  are 
database  driven.  In  this  model  there  are  few  interactions  with  clinical  colleagues  over  the 
internet.  Participants  in  both  focus  groups  and  interviews  have  suggested  that  communication 
with  other  health  professionals  could  be  a major  benefit  of  the  CHC.  / suppose  if  there 's  one 
thing  that  could  be  added  to  an  online  server  it  would  be  posting  questions.  Where  a 
professional  dilemma,  a diagnostic  dilemma  — so  if  you  are  remote  and  you  are  removed  from 
colleagues  and  you  can  *t  just  go  to  the  residents  quarters  and  discuss  it  with  the  registrar  - 
you  are  able  to  put  it  online  so  you  can  discuss  it  with  a hematology  registrar  within  that 
system.  That’s  what  they're  doing  in  physiotherapy  - the  chat  room  is  fantastic.  You  can  say 
I've  got  this  really  difficult  patient  and  get  everyone  from  around  the  world.  The  ideal 
community  "requires  member  participation  and  contribution,  ownership,  quality  support  and 
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facilitation,  shared  direction,  goals  and  projects"  (Stuckey,  Lockyer,  Hedberg,  2001,  p.  1). 
Within  this  ideal  community  collaboration  between  colleagues  for  the  purposes  of  improving 
clinical  practice  requires  both  experts  and  novices  "to  communicate,  contribute  to  and  initiate 
ideas  and  joint  projects"  (p,  6),  Fostering  an  online  community  requires  sustained 
committment  from  developers  and  members,  multi-dimensional  communicat  ion,  shared  focus 
of  attention  on  a real  problem  and  members  and  facilitators  who  can  collaborate,  mentor  and 
develop  solutions  together  in  a clinical  setting  (Stuckey,  Lockyer  & Hedberg,  2001). 

Conclusion 

Overall  this  paper  has  outlined  our  rationale,  approach,  evaluation  strategies  and  findings  from 
selected  aspects  of  the  questionnaires,  focus  groups  and  interviews.  Our  evaluation  website 
rhttp://www.medfac. unimelb.edu.au/health  channel/)  continues  to  update  our  latest  findings  in 
relation  to  the  three-year  evaluation.  The  questionnaires,  focus  groups  and  interviews  provide  an 
insight  into  clinical  decision  making  and  how  clinicians  use  resources  in  evidence-based  practice. 
In  our  evaluation  of  the  website  portal  we  have  accounted  for  clinical  perspectives  and  have 
suggested  a number  of  key  principles  for  the  redesign  and  reorienting  of  the  portal  so  that  it 
embodies  some  online  community  principles  that  complement  the  resource-based  characteristics 
of  the  website.  In  addition  the  website  should  provide  recommendations  to  professional  groups 
regarding  information  needs  and  recommendations  to  novice  and  experienced  clinicians.  The  use 
of  web-based  clinical  scenarios  will  also  significantly  reinforce  navigation  and  use  of  the  resources 
on  the  Clinicians  Health  Channel,  Ultimately  that’s  what  a good  single  site  is  about.  Where  it  can 
offer  you  two  or  three  different  opinions,  so  that  you  can  be  reasonably  sure  that  the  decision 
you  ’re  making  is  based  upon  current  practice  and  evidence.  Whatever  is  on  there,  there  should  be 
at  least  one  or  two  different  approaches  for  a problem,  I don’t  trust  any  one  person  or  any  one 
thing;  1 need  to  have  several  approaches  before  1 agree  to  anything. 
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Abstract:  In  the  case  of  executive  learning,  the  imperative  is  to  create  new  collective  knowledge,  the  fuel  of  personal  and 
organizational  effectiveness.  It  is  also  to  improve  personal  and  organizational  effectiveness,  the  fuel  of  new  collective 
knowledge.  We  discuss  this  conundrum  and  how  it  is  addressed:  a collaborative  approach  between  participants,  who  relate 
the  group’s  discussion  to  their  own  organization’s  challenges,  and  vice  versa.  However,  if  ongoing  collaboration  is  desired, 
virtual  collaboration  is  a necessity  given  current  organizational  realities.  We  consider  the  impact  on  instructors  and 
participants,  and  then  show  how  the  use  of  a relatively  simple  web-based  discussion  forum  effectively  enables  this  approach. 


The  Executive  Context  and  the  Collaborative  Approach 

The  profile  of  an  executive  education  participant  that  we  consider  here  is  based  on  our  institute’s 
experience  of  conducting  programs  for  over  5000  executives  every  year.  These  executives  are  drawn  from  a 
worldwide  geographic  base,  and  also  come  from  a wide  range  of  companies,  industries  and  functions.  A typical 
participant  is  40+  years  old,  and  has  an  average  of  1 5 years’  management  experience.  For  these  executives,  the 
essence  of  management  is  the  capability  to  make  decisions  and  act  wisely  in  a wide  range  of  complex  situations, 
especially  those  that  require  judgment,  discretion,  and  risk-taking  (Squires,  2001 ).  Decisions  are  often  made 
under  time  pressure,  in  an  environment  of  constant  change  and  pressure  for  quick  results,  and  with  incomplete 
information  and  analysis.  Such  situations  typically  are  found  in  the  fields  of  leadership,  strategy,  and  others  that 
require  adaptive  decision-making  as  more  information  emerges  or  conditions  change  (Mintzberg,  1996). 

Given  this  profile,  “an  [ideal]  executive  learning  experience  exposes  one  to  tensions,  interactions  and 
discussions,  which  in  turn  result  in  action,  experimentation  and  implementation,  with  the  ultimate  purpose  being 
to  improve  one’s  personal  effectiveness”  (Lorange  & Gilbert,  2001).  Yet  personal  effectiveness  is  only  one  part 
of  the  equation.  In  organizational  life,  we  cannot  discuss  and  act  in  isolation.  Views  on  topics  such  as 
leadership  are  beginning  to  recognize  to  a much  greater  extent  the  collaborative  element  involved  - for  example, 
“all  kinds  of  different  people  [contribute]  critical  leadership  in  initiating  and  sustaining  and  diffusing  significant 
organizational  innovations”  (Senge,  2001). 

Through  their  interactions,  executive  learning  participants  construct  a shared  understanding,  ora 
collective  knowledge  of  how  things  are  represented  (Brown  & Duguid,  1991).  The  benefits  of  creating  new 
collective  knowledge  are  significant:  collective  knowledge  is  the  most  strategically  significant  advantage  that 
executive  managers  seek  (Spender,  1996). 

Not  Guides:  Instructors  are  placers,  spacers,  and  provocative  challengers 

Once  settled  on  a collaborative  approach,  the  traditional  role  of  an  instructor  is  redefined.  Tradition 
says  instructors  are  responsible  for  delivering  content,  presentations  and  lectures  to  participants  who  absorb  and 
apply  - acting  as  “sponges”.  Being  a star  instructor  who  delights  and  entertains  with  energetic  delivery  of  well- 
packaged  insights  produces  instant  gratification.  These  insights  fade,  however,  when  the  realization  sets  in  that 
the  “lessons  learned”  are  not  truly  situated  in  the  context  of  the  participant.  The  instructor  has  made  all  of  the 
choices:  what  is  important,  for  whom,  when,  and  why.  It  is  not  the  creation  of  meaning,  but  the  transfer  of 
content,  from  one  to  many,  in  a relationship  devoid  of  significance. 

For  the  delivery  of  lasting  impact,  participants  need  to  not  only  take  center  stage,  but  to  create  the 
experience  themselves.  So  the  instructor  structures  the  time  and  space  in  which  the  dialogue  and  action  occurs. 
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A complementary  role  is  the  highly  skilled  and  sometimes  intuitive  process  of  framing  the  discussions.  The 
highly  skilled  instructor’s  role  is  to  provoke,  probe,  challenge,  surface  latent  curiosity,  encourage  divergent 
thinking,  facilitate  synthesis,  encourage  adoption  of  a systemic  view,  make  sure  learning  is  linked  to  action,  and 
ensure  reflection  is  built  into  the  process  (Kolb,  1984;  Lorange  et  al.,  2001). 

The  Participant’s  Realities  and  Role:  Going  Virtual  - A Fait  Accompli 

Participants  need  the  opportunity  to  gather  information  from  different  sources  to  craft  arguments,  create 
and  explore  viewpoints,  investigate  alternate  courses  of  action,  consider  complex  patterns  of  interaction  within 
and  beyond  their  own  organizations,  and  observe  and  reflect  on  the  effects  of  applied  action  learning  over  time. 
These  longer,  more  complex,  more  uncertain  paths  tend  to  favor  longer  collaboration  timeframes.  However, 
collaborators  are  rarely  co-located  all  of  the  time,  given  today’s  global  organization  structures  and  division  of 
responsibilities.  Virtual  collaboration  allows  participants  to  carry  on  the  construction  and  development  of  shared 
understanding  in  multiple  contexts  and  over  the  longer  term  of  project  work.  Participants  construct  new  content, 
as  a by-product  of  collaboratively  exploring,  making  connections,  hypotheses,  experiments,  reflecting,  building  a 
new  shared  understanding  and  synthesizing  and  embodying  new  collective  knowledge. 

Virtual  Collaboration  Example:  International  Political  Economy  in  a Business  Context 

The  assignment  is  to  draw  lessons  from  history  regarding  the  impact  of  cultural,  structural  and  systemic 
factors  and  natural  advantages  on  the  competitiveness  of  a region,  and  find  parallels  for  business,  strategy  and 
competition.  Participants  have  a common  basis  to  begin  their  discussions  due  to  the  assigned  reading  of  a 
contemporary  survey  work  on  this  topic.  In  addition,  prior  to  this  assignment,  participants  have  already 
collaborated  face-to-face  in  an  exploration  of  the  political  and  economic  dynamics  of  a growing  city.  This 
increases  their  personal  and  group  fluency  with  the  topic,  and  becomes  both  a basis  for  further  investigation  and 
a shared  lens  through  which  participants  view  the  current  assignment’s  readings  and  discussions. 

The  initial  task  is  to  read  the  survey  work  and  post  key  ideas  in  a web-based  discussion  forum,  along 
with  explanations  of  the  parallels  participants  find  for  business.  Then,  they  read  each  other’s  comments  and 
begin  to  virtually  debate,  share,  and  build  further  on  each  other’s  ideas.  During  the  process,  a particularly 
striking  passage  or  example  from  the  reading  sometimes  becomes  a shared  metaphor,  and  therefore  a shorthand 
expression  of  a broader  business  issue.  For  example,  the  metaphor  of  using  instruments  to  navigate  at  sea,  a key 
development  in  the  exploration  and  exploitation  of  new  worlds,  could  serve  as  a metaphor  for  navigating 
business  performance  in  the  modem  company.  Future  reference  to  an  in-depth  discussion  and  its  associated 
dilemmas,  talking  points,  issues  of  resolution  and  confusion,  can  be  raised  by  reference  to  this  metaphor.  The 
most  powerful  metaphors  stay  with  the  participants  into  their  field  expeditions  assignments  and  even  beyond. 
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Abstract 

Virtual  learning  environments  (VLEs)  are  often  perceived  by  education  establishments  as  an  opportunity  to  widen  access  without 
traditional  overheads.  An  integral  part  of  most  VLEs  is  asynchronous  computer  conferencing  and  on-line  moderators  must  help 
students  migrate  quickly  to  the  new  virtual  environment  to  minimize  learning  disruption.  This  paper  focuses  on  21  new  on-line 
moderators  and  reports  their  changing  perceptions  re  their  role  and  concerns,  from  their  first  appointment  through  to  the  end  of  the 
delivery  of  their  first  on-line  course.  The  findings  suggest  that  it  is  only  after  socialization  occurs  that  information  can  be  exchanged 
and  lead  to  knowledge  construction.  These  are  supported  by,  and  extend,  prior  research  by  Z Berge,  R Mason,  M Paulsen  and  G 
Salmon  and  are  reinforced  by  empirical  work  with  a further  19  new  on-line  moderators. 

1.  INTRODUCTION 

Virtual  Learning  Environments  (VLE)  are  increasingly  being  used  in  education  because  they  offer  a potential  solution  to  current 
academic  problems  of  increasing  student  numbers  within  a climate  of  reduced  funding.  Whether  they  provide  the  solution  is 
debatable  however,  it  is  only  by  recognising  and  fully  utilising  their  potential  that  maximum  effectiveness  can  be  achieved  [11]  [16]. 
Computer  conferencing  is  an  essential  part  of  VLEs  [4].  It  promises  the  ability  for  dispersed  students  to  communicate  with  each 
other  and  experience  the  pedagogical  techniques  of  collaborative  learning  and  support.  When  learning  moves  on-line,  students  and 
tutors  do  not  automatically  adapt  to  the  new  medium  [3]  and  for  some  people  it  can  even  provoke  strong  negative  reactions  [1]. 
Therefore,  a cultural  shift  needs  to  take  place  and  we  “...must  learn  how  to  cultivate  communication  in  a largely  asynchronous 
environment.”  [5].  This  will  engender  confidence  in  the  new  environment,  for  all  participants,  and  enable  effective  learning  to  take 
place. 

This  paper  follows  40  distance  learning  tutors  during  their  first  experiences  as  on-line  conference  moderators  within  an  on-line 
course.  It  investigates  how  they  adapt  to  the  role  of  moderator  and  focuses  on  what  they  perceive  as  their  role  and  reports  on  their 
concerns.  Of  these,  21  tutors  provided  the  initial  focus  and  further  empirical  work  with  19  tutors,  plus  previous  research  in  this  area, 
reinforce  those  findings.  Qualitative  data  is  also  drawn  from  students  studying  on  their  courses  [7].  The  results,  in  conjunction  with 
an  analysis  of  their  concerns  will  enable  us  to  improve  the  support  and  training  we  offer  new  on-line  moderators. 

2.  PRIOR  RESEARCH 

Research  into  the  role  of  an  on-line  conference  moderator  began  in  the  mid  1980s  and  wzis  articulated  largely  from  a group  of  very 
experienced  moderators.  Robin  Mason’s  seminal  1991  paper  “Moderating  Educational  Computer  Conferencing”[12]  organised  the 
skills  required  for  effective  moderation  from  several  researchers  [Kerr  1986,  Davie  and  Palmer  1985,  Brochet  1985.  Feenberg  1986, 
Hiltz  1988,  McCreary  1990.  Davie  1989]  into  three  key  roles:  Intellectual,  Social  and  Organisational.  Researchers  worldwide  cite 
this  classification  in  their  work  perge  1995  (America)  [3],  Paulsen  1995  (Norway)[14],  Salmon  1998  (UK)  [17],  Green  1998 
(Canada)  [8],  Murphy  & Collins  1998  (America)  [13],  Pearson  1999  (Australia)  [15]). 
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Further  research  in  1995  by  Zane  Berge  and  Morten  Paulsen  built  on  this  work  and  related  it  to  their  native  lands  ie  America  and 
Norway.  Their  results  supported  and  reinforced  Mason’s  paper  with  one  additional  role  being  added  by  Berge,  ie  technical.  She  felt 

it  was  important  that 

Table  1:  On-line  moderator  roles  moderator  should 

help  those  new  to 

conferencing  to  overcome  the  initial  technical  issues  associated  with  going  online  and  becoming  familiar  with  new  software. 


CLASSIFICATION  OF  THE  ROLE  OF  THE  ON-LINE  MODERATOR 


Task 

Researcher 

Task 

Researcher 

Intellectual  Role 

Facilitate 

Mason  1990 

Present  conflicting  opinions 

Paulsen  1995 

Summarise 

Mason  1990 

Respond  to  students 
comments  and  weave  them 
together 

Paulsen  1995 

Support/Shape/Guide 

Discussions 

Mason  1990 

Ask  questions 

Paulsen  1995 

Make  material  relevant  by 
developing  questions  and 
assignments  that  relate  to 
student  experiences  and 
current  affairs 

Paulsen  1995 

Don't  reply  on  off-line 
materials 

Paulsen  1995 

Explain 

Paulsen  1995 

Social  Role 

Create  context  conducive  to 
thought 

Berge  1995 

Provide  a good  social 
environment 

Mason  1990 

Deter  negativity 

Berge  1995 

Encourage  good  netiquette 

Mason  1990 

Encourage  contributions 

Berge  1995 

Provide  positive  feedback  on 
student  input 

Mason  1990 

Mediate 

Berge  1995 

Reward  positive  contributions 

Mason  1990 

Encourage  equal 
participation 

Berge  1995 

Praise  and  model  positive 
discussant  behaviour 

Mason  1990 

Be  sensitive  to  participants 
needs 

Mason  1990 

Discourage  inappropriate 
material 

Paulsen  1995 

Organisational  Role 

Set  the  Agenda:  declare 
discussion  objectives,  state 
the  timetable,  state  the 
decision-making  norms, 
provide  strong  leadership 
and  direction. 

Berge  1995 

Refer  inappropriate  digressions 
to  another  conference 

Paulsen  1995 

Synchronise  and  resynchronise 

Paulsen  1995 

Move  misplaced  content 

Paulsen  1995 

Give  a decisive  end  to  each 
discussion 

Paulsen  1995 

Technical  Role 

Make  participants  comfortable  with  the  system 

Berge  1995 

Although  it  is  ten 
years  since 

Mason’s  original 
classification  of  an 
on-line 
moderator’s  role 
and  five  years 
since  Berge  and 
Paulsen’s  work, 
the  roles  and  tasks 
identified  are  still 
cited  today.  The 
intellectual  role  is 
sometimes 
referred  to  as 
pedagogical  and 
the  organisation 
role,  managerial. 
The  tasks  therein 
are  the  same  and 
the  original  names 
have  been  used 
here  for 

consistency.  To 
provide  a frame  of 
reference  for  the 
research  a matrix 
depicting  the  roles 
and  tasks  of  on- 
line moderators 
identified  by  these 
researchers  was 
compiled  and  is 
shown  in  Table  1 . 
Although  some  of 
the  tasks  listed  can 
be  said  to  be 
subset  of  others, 
for  example  a 
facilitator  does 
support  and  guide 
discussions,  no 
attempt  has  been 
made  to  change 
the  level  of 


granularity  to  enable  a more  consistent  matrix  to  be  compiled. 


Gilly  Salmon’s  action  research  into  on-line  moderation  introduces  a five  layer  model  of  teaching  and  learning  on-line  through 
computer  mediated  communication  which  she  invites  us  to  explore  and  contextualise  into  our  own  discipline.  The  five  layers  are: 
access  and  motivation;  online  socialization;  information  exchange,  knowledge  construction  and  development.  She  summarises  her 
model  thus:  “ Stage  One:  Individual  access  and  the  ability  of  participants  to  use  CMC  are  essential  prerequisites  for  conference 

participation.  Stage  two  involves  individual  participants  establishing  their  online  identities  and  then  finding  others  with  whom  to 
interact.  At  stage  three,  participants  give  information  relevant  to  the  course  and  each  other.  Up  to  and  including  stage  three,  a form 
of  co-operation  occurs,  ie  support  for  each  person’s  goals.  At  stage  four,  course-related  group  discussions  occur  and  the  interaction 
becomes  more  collaborative.  At  stage  five,  participants  look  for  more  benefits  from  the  system  to  help  them  achieve  personal  goals, 
explore  how  to  integrate  CMC  into  other  forms  of  learning  and  reflect  on  the  learning  process.”  [17] 
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3.  BACKGROUND 


1.  The  Course 

The  Open  University  in  the  UK  piloted  a pioneering  on-line  course  during  1999  entitled  ‘You,  Your  Computer  and  the  Net’.  It  is  a 
first  year  degree  module  and  it  aims  include: 

1.  introduce  students  to  computers  and  the  internet; 

2.  give  students  direct  experience  of  working  at  a distance  by  themselves  and  as  a group; 

3.  develop  the  new  skills  needed  for  studying  and  communicating  using  online  media; 

The  course  is  web  based  and  runs  from  the  middle  of  February  to  the  beginning  of  October.  The  main  study  material  is  delivered  via 
the  World  Wide  Web.  The  asynchronous  text  based  conference  facility  is  central  to  the  delivery  model  with  the  primary  computer 
conference  being  the  tutor  group  ie  the  tutor  and  their  students  (15-20  members).  Moderated  by  the  tutor,  it  is  the  main  area  for 
course  activities,  discussions,  and  general  chat.  All  tutor  groups  are  within  one  of  the  13  Open  University  regions  and  this  research  is 
based  on  one  of  those  regions. 

2.  The  Students 

All  Open  University  courses  use  distance  learning  and  students  enrol  on  that  premise.  They  are  part-time,  usually  mature  and  have 
other  commitments  ie  work  and/or  family. 

3.  The  Tutors 

Student  demand  for  the  2000  presentation  of  the  course  resulted  in  a large  number  of  tutors  being  recruited.  Two  key  attributes  the 
interviewers  were  looking  for  was  either  a neutral  or  positive  attitude  towards  the  delivery  of  a course  on-line  plus  the  required 
underpinning  skills  and  knowledge  necessary  for  the  academic  content  of  the  course.  The  40  tutors  who  participated  were  over  30 
years  of  age,  predominantly  male  (76%)  and  had  all  tutored  a distance  learning  course  before.  61%  were  in  the  41-50  age  bracket 
and  almost  half  are  teachers/lecturers.  None  had  used  asynchronous  conferencing  as  an  integral  part  of  a learning  programme. 


4.  METHODOLOGY 

To  support  new  tutors  there  was  a regional  briefing  day  where  each  tutor  met  their  mentor  who  they  could  go  to  for  advice  during 
their  first  year.  Each  mentor  had  tutored  on  the  pilot  year  of  the  course.  In  order  to  track  the  new  tutors  changes  in  perceptions  about 
their  role  as  an  on-line  moderators  a number  of  questions  were  administered  at  significant  points  throughout  the  course.  These  were 
at  the  beginning  of  the  briefing  day  (November),  before  the  course  started  (end  January),  mid  course  (end  June)  and  at  end  of  the 
course  (October).  The  questions  used  to  ascertain  their  perceptions  and  concerns  were: 

1 . What  are  your  perceptions  of  the  role  of  an  on-line  moderator? 

2.  What  concerns  do  you  have  about  the  role  of  an  on-line  moderator? 

They  were  open  ended  because  although  responses  are  complex  to  analyse,  if  one  offered  them  a list  of  tasks  from  the  role/task 
matrix  (Table  1)  it  could  stop  innovative  thoughts  and  preclude  new  tasks  being  identified.  The  responses  for  the  question  1 were 
compared  to  the  moderator  role/task  matrix  and  any  similarities/difFerences  noted.  A list  detailed  their  concerns  was  also  produced. 

During  the  regional  briefing  day  in  November  new  tutors  were  put  into  groups  and  participated  in  a 30  minute  session  to  experience 
the  medium  of  text  based  narrow-band  communication.  No  questionnaire  was  issued  at  this  stage  to  allow  any  immediate  changes  in 
perception  to  dissipate.  Between  their  regional  briefing  day  and  the  start  of  the  course  an  informal  group  conference  was  used  to 
enable  them  to  practice  their  skills.  At  the  end  of  January,  before  the  course  started,  tutors  completed  a second  questionnaire  without 
any  reference  to  the  first.  It  comprised  their  name  and  the  two  key  questions  given  above.  The  third  questionnaire  was  issued  mid 
course  and  the  final  one  just  after  the  course  was  completed.  The  participants  had  no  reference  to  their  previous  responses. 
Completed  questionnaires  were  analyzed  to  identify  what,  if  any,  changes  had  occurred.  The  research  was  then  extended  to  cover  a 
further  19  new  on-line  moderators  to  reinforce,  or  otherwise,  the  original  findings. 

5.  RESULTS 

Tutor  responses  to  the  question  “What  are  your  perceptions  of  the  role  of  an  on-line  moderator?”,  were  examined  and  compared  to 
the  on-line  moderator  role/task  matrix  (Table  1).  It  was  interesting  to  note  that  only  two  tasks  mentioned  by  tutors  did  not  map 
directly  to  one  of  these  tasks,  ie  housekeeping  and  control.  The  statistical  data  gathered  from  the  tutors  provided  evidence  that  as  the 
course  evolved  so  did  their  perception  of  their  role  (Figures  1 and  2). 
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line  moderators  perception  of  their  role  at  four  key  stages. 


Figure  1:  21  new  on- 


Initially  they  were  concerned  primarily  with  intellectual  issues  however,  after  a short  submersion  in  an  asynchronous  environment 
there  was  a significant  switch  to  social  issues.  It  remained  their  focus  two  months  later  just  before  the  start  of  the  course  and  suggests 
that  they  became  more  concerned  with  the  issue  of  ensuring  that  students  adapt  to  the  new  environment.  This  concurs  with  one  of  the 
concerns  identified  for  Internet  based  learning  ie  loss  of  social  interaction.[5]  [10].  As  the  course  progressed  the  perceptions  of  21  of 
the  tutors  implied  that  the  importance  of  social  issues  decreased  and  by  mid-course  the  balance  between  intellectual  and  social  tasks 
equalized  (Figure  1).  The  extended  research  with  a further  19  tutors  reinforced  this  pattern  (Figure  2). 

The  end  of  course  questionnaire  was  completed  by  all  40  tutors.  It  suggests  that  whereas  the  number  of  intellectual  tasks  almost 
returns  to  the  original  level  when  they  were  appointed  to  the  role  of  an  online  moderator,  the  social  tasks  cited  decreased.  The 
following  results  from  the  four  roles  are  drawn  from  the  input  of  all  40  tutors  and  feedback  from  their  students. 
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Figure  2:  Extended  research:  A further  19  new  on-line  moderators 
perception  of  their  role  at  three  key  stages 


1.  Intellectual  Role 

Four  of  the  tasks  under  the  intellectual  role  wCTe  mentioned  (Table  1)  ie  explainer,  facilitator,  summarizer  and  to  support/shape/guide 
discussions  with  the  last  task  representing  over  50%  of  the  replies.  All  of  the  remaining  tasks  except  one  in  the  on-line  moderator 
role/task  matrix  could  be  considered  as  a subset  of  these  four.  The  exception  was  Paulsen’s  “don’t  reply  to  off-line  materials”. 

2.  Social  Role 

Their  initial  thoughts  were  that  it  was  important  to  encourage  contributions  for  everyone  and,  when  required,  they  should  mediate. 
After  experiencing  asynchronous  conferencing  the  number  of  tutors  citing  these  tasks  increased  significantly.  Providing  a good 
social  environment,  deter  negativity,  discourage  inappropriate  material,  and  to  create  context  conducive  to  thought,  were  also 


mentioned.  Fewer  tasks  were  identified  mid  course  and  by  the  end  of  the  course  only  four  tasks  were  cited:  encourage  contributions, 
ensure  equal  participation,  deter  negativity  and  mediation.  It  was  interesting  to  note  that  tutors  did  not  mention;  be  sensitive  to 
participants  needs,  praise  and  model  positive  discussant  behaviour,  provide  positive  feedback  on  students’  input  and  to  reward 
positive  contributions. 

3.  Organisational  Role 

No  organisational  tasks  were  mentioned,  including;  to  provide  the  framework  and  rules  within  which  the  conference  will  operate 
[12].  This  encompasses  the  activities,  schedule  and  the  requirements  of  the  course.  Perhaps  this  is  because  all  of  the  module  material 
is  provided  on  the  course  web  site.  Housekeeping  was  mentioned,  but  does  not  appear  on  the  role/task  matrix  in  Table  1 . It  has  not 
therefore  been  shown  in  Figure  2 but  should  be  added  upon  update  of  the  role/task  matrix. 

4.  Technical  Role 

Only  one  on-line  moderator  cited  this.  The  Open  University  offers  a comprehensive  computer  support  service  for  tutors  and  students. 
However,  students  still  raised  technical  issues  [7]  as  an  area  for  concern  and  expected  that  the  on-line  moderator  should  solve  them. 

5.  Areas  of  concern 

Time  became  of  increasing  concern  to  50%  of  all  tutors  by  the  end  of  the  course.  It  took  a lot  longer  than  they  had  anticipated  to 
assimilate  messages  and  the  responses  took  longer  to  compose  because  they  were  aware  that  the  written  word  could  be 
misinterpreted.  As  one  student  said,  “Many  people  have  problems  not  being  able  to  articulate  easily  on-line”.  They  stated  that  tutors 
must  have  superior  written  skills  as  body  language  and  voice  intonation  do  not  apply  [7]. 

Lack  of  experience  of  what  the  role  of  an  on-line  moderator  was  also  a key  area  for  concern.  Although  it  decreased  by  the  end  of  the 
course,  the  post -course  student  survey  showed  that  some  of  the  students  were  also  unaware  of  what  the  role  of  an  on-line  moderator 
is.  This  is  an  area  for  future  research. 

Table  2:  Concerns  raised  by  40  tutors  as  they  adapted  to  their  role  as  an  on-line  moderator. 
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NS 


The  results 
suggest  that 
tutors  new  to 
moderating 
text  based 
online 
computer 
conferences 
come  with  the 
preconception 
that  their 
primary  role  is 
intellectual. 
However,  even 
a short 

exposure  to  the 
medium  raised 
many  new 
social  issues. 
These  were 
still  the 

overwhelming 
issues  two 
months  later 
just  before  the 

start  of  the  course.  As  the  course  progressed  and  online  socialization  occurred,  social  and  intellectual  issues  became  of  equal 
importance.  At  the  end  of  the  course  social  issues  had  decreased  to  below  their  initial  level.  The  issue  of  gleaming  information  only 
from  text  without  any  visual  and/or  oral  clues  remained  an  issue,  primarily  from  the  students*  viewpoint.  Some  students  also 
expected  technical  help  from  online  moderators  whereas  the  moderators  did  not  perceive  this  to  be  part  of  their  role.  There  is  a need 
to  match  the  expectations  of  the  online  moderator  and  the  students  to  avoid  discontentment. 

Stages  two,  three  and  four  of  Gilly  Salmon’s  five  layer  model  of  teaching  and  learning  online  through  CMC  [17]  ie  that  online 
socialization  must  take  place  before  information  can  be  exchanged  and  lead  to  knowledge  construction,  are  also  supported..  No  data 
was  available  from  this  research  to  comment  on  stages  one  (access  and  motivation),  or  five  (development)  of  her  model. 


Concerns  of  40  Tutors. 

Initial  Thoughts 

Pre  Course 

End  Course 

Category 

How  to  control  multithreading 

0 

2 

0 

Intellectual 

Lack  of  visual  clues 

1 

4 

0 

Social 

Serve  student  needs 

3 

5 

2 

Social 

How  to  encourage  participation 

8 

5 

3 

Social 

Becoming  too  involved 

2 

5 

0 

Social 

When  to  intervene 

7 

6 

0 

Social 

Censorship 

4 

3 

0 

Social 

Impose  own  views 

1 

5 

0 

Social 

Technical  problems 

1 

1 

0 

Technical 

Time 

5 

11 

21 

New 

Cost 

2 

8 

6 

New 

Lack  of  experience  of  what  the  role  is 

12 

16 

5 

New 

Students  not  be  aware  of  moderator  role 

4 

7 

0 

New 

Not  knowing  all  the  answers 

0 

1 

0 

New 

Total 

50 

79 

37 

Two  key  areas  for  on-line  moderator  concern  were:  time  and  lack  of  experience.  Time  became  increasingly  important  as  the  course 
progressed.  It  took  far  longer  than  they  had  anticipated  to  compose  new  messages.  This  may  partly  answer  some  students  concerns 
about  feedback  time.  Lack  of  experience  decreased  as  the  course  progressed  however  their  knowledge  is  only  based  on  experiential 
learning  from  one  group  of  students  on  one  course  and  from  dialogue  with  colleagues. 

Training  and  support  for  new  on-line  moderators  must  include  on-line  conferencing  as  an  integral  part  of  the  process  in  order  to 
cultivate  the  ability  to  confidently  communicate  by  text.  This  will  enable  them  to  realize  the  importance  of  socialization  before 
pedagogical  issues  can  be  fiilly  addressed, 
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Abstract;  Developers  of  computer-based  tools  face  a dual  challenge  when  trying  to  understand 
users’  mental  models,  because  users  are  simultaneously  building  mental  models  of  the  tool  and 
of  the  world  represented  by  the  tool.  Understanding  users’  mental  models  of  complex  computer- 
based  tools,  such  as  the  4-D  modeling  tool  described  in  this  article,  can  help  inform  the  design 
and  development  of  those  tools.  For  this  study,  usability  tests  using  the  teaching  method  were 
conducted  to  determine  the  method’s  usefulness  for  determining  the  mental  models  of  both 
advanced  and  novice  users.  The  results  of  the  tests  revealed  sufficient  information  about  the 
differentiation  between  users’  mental  models  to  inform  further  development  of  the  software. 


Introduction 

The  Virtual  Solar  System  (VSS)  is  a powerful  virtual  reality  (VR)  modeling  tool  in  which  students  create 
four-dimensional  (4-D)  models  of  the  solar  system  to  investigate  astronomical  phenomena  (Hay  & Barab, 
1998).  Students  construct  and  test  their  models  to  answer  inquiry -oriented  questions,  using  their  models  as  a 
source  of  data  for  their  answers.  The  4-D  environment  allows  students  to  navigate  through  their  model  in  all 
three  physical  dimensions  as  well  as  backward  and  forward  in  time  so  as  to  look  at  astronomical  phenomena 
from  multiple  viewpoints.  For  example,  students  are  able  to  watch  solar  and  lunar  eclipses  from  the  viewpoint 
of  the  Sun,  the  Earth,  or  the  Moon,  or  anywhere  else  they  choose.  This  tool  affords  students  the  ability  to 
explore  the  solar  system  from  perspectives  that  were  previously  unattainable  through  other  media.  However, 
this  new  environment  raises  new  questions:  How  do  students  get  their  bearings  in  this  new  world?  Can  they 
understand  the  tools  well  enough  to  explore  this  world?  Are  they  able  to  differentiate  between  their  mental 
models  of  the  solar  system  and  their  mental  models  of  the  tool  and  its  interface?  If  so,  how  do  they  incorporate 
this  new  modeled  reality  into  their  existing  mental  models  of  the  solar  system? 

Developers  of  computer-based  tools  face  a dual  challenge  when  trying  to  understand  users’  mental  models, 
because  users  are  simultaneously  building  mental  models  of  the  tool  and  mental  models  of  the  concepts 
represented  by  the  tool.  Understanding  users’  mental  models  of  complex  computer-based  tools  like  the  VSS 
4-D  modeling  tool  can  help  inform  the  design  and  development  of  such  tools.  One  potential  method  of 
exploring  users*  mental  models  is  through  usability  testing.  Although  the  term  “usability  testing”  is  often  used 
in  its  most  restrictive  sense,  that  is,  to  find  and  document  usability  problems  in  software  and  then  to  make 
recommendations  of  how  to  fix  these  problems,  our  definition  of,  and  purposes  for,  usability  testing  are  much 
broader.  We  believe  that  the  data  uncovered  using  certain  usability  testing  methods  is  suitable  for  determining 
novice  and  advanced  users’  mental  models  when  they  are  immersed  within  complex  computer  environments. 


ERIC 


Theoretical  Frame  work 


The  VSS  project  (http://lpsl.coe.uga.edu/vss/index.html)  is  based  on  two  educational  theories: 
constructionism  and  inquiry.  Constructionist  learning  environments  require  students  to  create  artifacts  as  an 
essential  component  of  personal  knowledge  construction  (Papert,  1993).  Inquiry -oriented  instruction 
recommends  that  student's  individual  interests  should  guide  their  learning  activities  (Welch,  Klopfer, 
Aikenhead,  & Robinson,  1981).  Their  natural  curiosity  into  exploring  phenomena  leads  to  a cycle  of 
questioning,  designing  investigations,  seeking  solutions,  and  then  asking  new  questions.  Student  curiosities  are 
satisfied  when  they  have  constructed  mental  frameworks  that  adequately  explain  their  curiosities  (Haury,  1993). 

Models  are  powerful  ways  of  understanding  that  are  integral  to  scientists'  research.  The  act  of  model 
building  allows  learners  to  engage  in  a design  process  (Jackson,  Stratford,  Krajcik,  & Soloway,  1994;  Lehrer, 
Horvath,  & Schauble,  1994),  which  begins  with  a set  of  tentatively  accepted  theories  that  evolve  into  coherent 
understandings  as  represented  in  their  models  (Roth,  1996).  Model  building  forces  learners  to  make  clear  their 
understandings  of  phenomena,  validate  their  resultant  model,  and  iterate  the  process  to  develop  increasingly 
richer  conceptions  (Confrey  & Doerr,  1994).  When  students  use  computer  tools  to  work  in  real  and  virtual  (or 
modeled)  scenarios,  they  develop  richer,  more  complex  mental  models  of  the  content  being  studied,  but  they 
must  also  develop  mental  models  of  the  software  environment  itself  in  order  to  facilitate  their  movement 
between  the  two  realities.  The  software  programs  used  in  modeling  are  examples  of  cognitive  tools. 

Cognitive  tools  refer  to  technologies,  tangible  or  intangible,  that  enhance  cognitive  powers  during  thinking, 
problem  solving,  and  learning  (Jonassen,  1996).  One  goal  of  cognitive  tools  is  to  assist  students  to  follow 
mental  models  similar  to  experts'  cognitive  processes  to  find  and  solve  problems,  set  achievable  goals,  test 
hypotheses,  and  make  their  own  discoveries  (Collins,  1991).  As  visualization,  simulation,  and  modeling 
become  more  important  for  mathematicians  and  scientists,  the  use  of  similar  technologies  for  learning 
mathematics  and  science  has  also  increased  in  importance  (Roschelle,  Kaput,  & Stroup,  2000).  For  cognitive 
tools  to  emulate  the  professional  tools  that  scientists  use,  the  developers  of  these  tools  must  be  cognizant  of  the 
differences  in  mental  models  between  novice  and  advanced  users. 

People  try  to  understand  the  external  world  by  constructing  internal  representations  of  it  in  their  minds;  we 
call  these  representations  mental  models  (Johnson-Laird,  1989).  These  models  are  assumed  to  have  structures 
that  reflect  our  perception  and  interpretation  of  what  the  world  contains,  how  those  pieces  are  related,  how  it 
works,  and  why  it  works  the  way  it  does  (Bibby,  1992;  Carroll  & Olsen,  1988;  Jih  & Reeves,  1992;  Kearney  & 
Kaplan,  1997).  Thus,  the  process  of  building  mental  models  becomes  an  important  issue  in  balancing  between 
what  exists  in  the  world  and  what  the  knower  understands. 

Users  of  computer-based  cognitive  tools  build  two  kinds  of  mental  models:  a mental  model  of  the  tool  and 
a mental  model  of  the  world  represented  by  the  tool  (Payne,  1992).  Learning  about  the  tool  and  gaining  a clear 
mental  model  of  it  takes  place  throughout  the  user's  interactions,  but  it  is  more  pronounced  in  early  stages  of 
learning  about  the  tool.  Building  mental  models  of  the  tool  requires  users  to  become  aware  of  what  external 
knowledge  they  need  to  acquire  and  how  to  approach  the  provided  functionalities  (O’Malley  & Draper,  1992). 
The  next  developmental  stage  requires  users  to  construct  a mental  model  of  the  situations  and  actions  of  the 
world  that  is  represented  by  the  system  (Bower  & Morrow,  1990).  Their  previous  mental  models  affect  the 
decoding  of  the  meanings  and  relationships,  depending  on  the  level  of  proximity  of  the  user’s  language  or 
interface  to  that  of  the  system  (Calvin,  1996). 

Novice  and  expert  users’  mental  models  are  different  in  two  respects:  sharing  tasks  with  the  tool  and 
reasoning  through  the  represented  world.  Novices  usually  do  not  have  enough  knowledge  about  what  the 
system  can  do  to  decide  what  needs  to  be  internalized  by  the  user  and  what  the  tool  can  do  to  fill  in  the  gaps 
(Kelly,  1995).  This  lack  of  knowledge  might  lead  novice  users  to  spend  too  much  time  on  tedious  tasks  that  the 
system  could  take  care  of  Novices  build  and  process  their  mental  models  by  focusing  on  visible  objects  that 
simulate  processes  in  real  time,  whereas  experts  are  able  to  construct  mental  models  of  abstract  relations  and 
properties  (Johnson-Laird,  1989).  The  greatest  challenge  of  researching  mental  models  comes  from  our  inability 
to  directly  observe  users’  mental  models  (Kelly,  1995).  Additionally,  researchers  have  no  choice  but  to  rely  on 
their  own  mental  models  to  conceptualize  users'  mental  models,  which  may  affect  their  interpretation  of 
research  results  (Rouse  & Morris,  1986;  Rutherford  & Wilson,  1992). 
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Usability  Testing 


Usability  is  generally  thought  of  as  how  easy  it  is  to  learn  and  use  a product  (Nielsen,  1994).  The  level  of 
usability  for  a product  can  be  tested  through  a variety  of  methods.  Quantitative  methods  typically  include 
measurements  of  the  length  of  time  users  take  to  complete  specific  tasks.  Qualitative  methods  are  more 
descriptive  in  nature  and  lend  themselves  to  holistic  overviews  of  usability  problems.  One  of  the  most 
commonly  used  methods  is  the  think-aloud  method,  where  a single  user  interacts  with  a product  and  talks  aloud 
about  what  he  is  doing  and  thinking  during  the  test.  This  method,  although  highly  effective  in  revealing  the 
thoughts  of  the  user,  seems  unnatural  for  many  users.  A closely  related  method,  the  teaching  method,  pairs  an 
advanced  user  with  a novice  user.  The  advanced  user  guides  the  novice  through  a number  of  structured  tasks, 
and  both  users  are  instructed  to  talk  aloud  to  each  other  throughout  the  test  about  what  they  are  doing  and 
thinking.  The  tester  can  analyze  both  the  advanced  and  the  novice  users’  mental  models  of  the  system  (Nielsen, 
1994).  The  method  also  reduces  the  likelihood  that  the  users  will  try  to  interact  with  the  testers,  which  can 
detract  from  the  validity  of  the  data.  We  believe  the  interplay  that  occurs  between  the  advanced  and  novice 
users  as  they  think  aloud  in  the  teaching  method  can  give  us  insight  into  the  mental  models  of  the  users. 


The  Study 

Our  goal  for  this  study  was  to  determine: 

1.  What  are  the  differences  between  novice  and  advanced  users’  mental  models  of  VSS  software? 

2.  Does  the  teaching  method  of  usability  testing  provide  sufficient  information  to  reveal  novice  and 
advanced  users’  mental  models  of  the  software,  and  the  concepts  represented  by  this  software? 

The  teaching  method  of  usability  testing  pairs  an  advanced  and  novice  user  together  for  each  usability  test. 
The  advanced  users  for  this  study  were  selected  from  the  students  currently  enrolled  in  the  introductory 
astronomy  lab  course  that  one  of  the  researchers  was  teaching.  The  three  students  who  agreed  to  participate  in 
this  study  were  among  the  most  competent  users  of  the  VSS  by  the  end  of  the  course.  Additionally,  they  all 
received  an  A in  the  course,  indicating  a strong  understanding  of  the  astronomy  concepts. 

Three  novice  participants  were  selected  from  among  graduate  students  working  in  the  Learning  and 
Performance  Support  Laboratory  (LPSL)  at  the  University  of  Georgia  who  had  no  familiarity  with  the  VSS. 
They  all  had  low  to  average  computer  experience,  and  no  background  in  astronomy. 


Data  Collection  and  Analysis 

Each  pair  of  users  was  given  a list  of  four  common  tasks  to  perform  using  the  VSS.  The  tasks  ranged  from 
the  most  basic  (e.g.,  constructing  a model  of  a basic  Sun-Earth-only  solar  system)  to  moderately  complex  tasks 
(e.g.,  investigating  the  eccentricity  of  planetary  orbits  and  the  timing  of  planetary  orbits).  Each  of  the  advanced 
users  had  completed  these  tasks  at  least  once  during  the  astronomy  lab  course.  Users  were  discouraged  from 
asking  the  testers  questions  about  the  software  or  the  test  itself  during  the  test. 

Data  consisted  of  audio  and  video  recordings  of  the  users’  interactions  with  the  software,  video  recording 
of  the  users’  computer  screen,  researcher  entries  into  usability  data-logging  software  and  researcher  field  notes. 
The  data  logs  and  field  notes  were  analyzed  for  instances  of  user  talk  or  action  that  revealed  evidence  of  the 
user’s  mental  model  of  either  the  tool  or  the  astronomy  content.  Each  instance  was  further  coded  to  indicate 
whether  it  related  to  the  mental  model  of  a novice  user  or  an  advanced  user,  and  whether  that  instance  was  a 
representation  of  the  modeling  environment  (tool)  or  of  the  astronomy  content  (real  world).  The  findings  from 
each  individual  were  then  grouped  together  by  common  themes. 


Results  and  Discussion 

We  found  that  users  have  difficulties  in  building  mental  models  of  the  tool  and  the  represented  world.  Four 
major  themes  emerged  from  the  data:  (a)  conflict  with  existing  interface  mental  models,  (b)  difficulties  in 
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building  and  integrating  mental  models  of  interface  functionality,  (c)  difficulties  in  building  and  integrating 
mental  models  of  space,  and  (d)  difficulties  in  building  and  integrating  mental  models  of  space  and  time. 

Conflict  with  existing  interface  mental  models:  Novice  users  found  aspects  of  the  VSS  interface  to  be 
confusing,  particularly  the  menus  and  shortcut  keys,  which  are  not  consistent  with  Windows  software 
conventions.  This  did  not  match  the  novice  users’  existing  mental  models  of  how  an  interface  is  laid  out  and 
how  it  should  work.  In  tests  1 and  2,  even  though  the  advanced  users  spent  considerable  time  explaining  the 
functions  of  keys  and  menus  in  the  program,  novice  users  were  still  confused.  The  VSS  also  relies  on  keyboards 
commands  for  navigating  through  the  space  and  selecting  planets  and  stars  in  the  space.  Novice  users  expected 
to  use  the  mouse  as  their  primary  navigation  tool  and  often  clicked  on  the  screen  to  navigate  through  space  or  to 
edit  parameters  of  planets  and  stars. 

All  of  the  advanced  users  were  able  to  articulate  clearly  the  functionality  of  the  interface  and  were  able  to 
accomplish  tasks  efficiently  and  effectively.  This  highlights  how  users  can  modify  their  existing  mental  model 
of  an  interface  to  adapt  to  different  interfaces.  The  control  needs  of  a 4-D  environment  differ  so  greatly  from, 
and  are  so  much  more  demanding  than,  a normal  2-D  Windows  program  that  additional  documentation  and  help 
features  need  to  be  incorporated  into  the  software  to  help  new  users  accelerate  their  learning  and  understanding 
of  the  controls  (see  Figure  1 ). 


Figure  1.  VSS  Basic  Screen  Controls 

Difficulties  in  building  and  integrating  mental  models  of  interface  functionality:  The  main  phenomenon  of 
interest  is  generally  located  in  the  center  of  the  VSS  screen,  but  there  is  also  ancillary  information  in  each 
comer  of  the  screen  at  all  times.  Novice  users  were  unsure  of  which  aspect  of  the  interface  to  attend  to  first  and 
which  elements  of  the  screen  were  the  most  important.  A task  such  as  timing  the  orbital  period  of  a planet 
required  them  to  use  both  hands  on  the  keyboard  while  simultaneous^  watching  three  different  portions  of  the 
screen. 

Advanced  users  built  their  mental  models  to  accommodate  these  demands.  On  many  occasions,  however, 
they  were  using  the  tools  without  fully  understanding  the  concept  behind  it  (e.g.,  negative  number  for  rotation 
rate,  increase  or  decrease  in  speed  by  a factor  of  ten).  Both  novice  and  advanced  users  suggested  that  a more 
simplified  method  of  timing  objects,  such  as  a built-in  stopwatch,  would  allow  them  to  conceptually  understand 
the  elements  involved  in  determining  time  in  space  without  distracting  their  focus  from  watching  the 
phenomenon  occur. 

Difficulties  in  building  and  integrating  mental  models  of  space:  All  users  had  difficulty  comprehending  the 
vastness  of  space  that  is  represented  in  the  software,  the  relatively  small  size  of  objects,  and  the  challenges  of 
navigating  to  other  locations.  For  example,  when  a student  creates  an  Earth  and  then  observes  it  from  the  Sun, 
the  Earth  appears  as  little  as  a pixel  or  two  of  light  on  the  screen.  Novice  users  expected  to  see  larger  objects, 
similar  in  size  to  traditional  textbook  drawings  of  solar  system  objects.  Their  current  mental  models  of  the 
worlds  represented  by  the  software  did  not  include  the  concept  that  most  of  space  is,  in  fact,  space. 

The  vastness  of  space  created  another  problem  for  novice  users  when  they  tried  to  travel  between  planets. 
The  A and  Z keys  controlled  the  forward  and  backward  movement  of  the  user’s  viewpoint.  All  of  the  novices 
consistently  overshot  their  target  by  accelerating  to  a high  speed,  then  not  decelerating  rapidly  enough  as  they 
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approached  another  planet.  Advanced  users  were  consistently  proficient  at  maneuvering  their  viewpoint  by 
using  the  keyboard  keys. 

Difficulties  in  building  and  integrating  mental  models  of  space  and  time:  Time  is  the  fourth  dimension  of 
the  VSS.  The  difficulties  in  this  section  arise  from  the  naive  mental  models  of  both  novice  and  advanced  users 
that  do  not  incorporate  time  as  a key  element  of  the  modeling  environment.  VSS  has  a functionality  called 
waypoints  that  allows  users  to  create  customized  views  of  the  solar  system  by  attaching  a viewpoint  to  any 
object  and  looking  towards  any  other  object.  Novice  and  advanced  users’  mental  models  of  a waypoint  were 
based  on  an  incorrect  or  naive  idea  of  returning  to  a specific  location  at  any  time,  with  the  expectation  that  the 
view  would  be  the  same.  Even  though  we  do  not  expect  to  see  stars  at  the  same  spot  looking  into  a telescope 
pointing  to  a certain  part  of  the  sky  when  we  return  after  a while,  the  novice  users  could  not  integrate  the  time 
factor  into  their  mental  model  at  first. 

The  passage  of  time  is  an  integral  feature  of  solar  system  modeling.  A view  from  any  one  point  at  any  one 
instant  will  never  be  the  same  as  that  same  view  at  a different  instant.  Novice  users  also  expected  that  VSS 
would  have  an  “undo”  feature  of  some  sort  that  would  enable  them  to  return  to  a prior  position  (and  time)  with 
the  click  of  the  mouse.  Time  in  many  ways  is  the  most  important  dimension  in  this  environment,  because 
without  it  the  solar  system  model  would  not  run,  that  is,  all  of  the  planetary  movement  that  occurs  in  the  model 
does  so  because  of  the  passage  of  time.  More  thought  should  be  given  to  how  the  user  is  informed  of  time  on 
the  screen,  and  how  it  is  controlled  by  the  user. 


Conclusions 

The  teaching  method  of  usability  testing  revealed  information  about  the  advanced  and  novice  users’  mental 
models  of  the  software  interface  and  of  the  astronomy  content.  Recommendations  from  this  form  of  usability 
testing  can  help  the  developers  make  changes  to  the  interface  that  would  not  have  been  revealed  through  an 
expert  review,  heuristic  evaluation,  or  other  forms  of  usability  testing  that  focus  primarily  on  finding  errors  with 
a program.  If  the  interface  is  too  challenging  (non-intuitive  or  counter-intuitive),  users  may  be  developing  a 
more  complex  mental  model  of  the  tool  but  not  of  the  world  represented  by  that  tool.  An  ideal  interface  would 
not  require  significant  mental  model  shifts  on  the  part  of  novice  users,  allowing  them  to  focus  on  the  use  of  the 
tool  for  learning  the  new  content. 

Further  research  should  be  conducted  on  the  users’  understanding  of  the  spatial  and  temporal  world 
represented  by  the  VSS.  The  time  aspect  (the  fourth  dimension)  of  the  VSS  is  an  important  issue,  because  it 
enables  the  users  to  understand  the  motion  of  planetary  bodies  (by  allowing  the  3-D  spatial  movement  to 
occur).  The  spatial  dimension  is  also  critical  considering  that  the  user  in  a computer-based  3-D  (or  4D) 
environment  is  forced  to  interact  with  this  environment  through  a 2-D  interface.  More  research  can  uncover  the 
tension  that  must  exist  between  the  mental  models  of  a 2-D  interface  and  the  mental  models  of  a 3-D  (or  4-D) 
environment. 
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Abstract 

This  study  is  on  the  design  and  implementation  of  an  educational  dialogue  system  to  support  e-Leaming. 
The  learning  domain  to  apply  the  dialogue  system  used  the  subject  of  the  geometry.  The  knowledge  in 
dialogue-based  system  for  learning  the  geometry  was  created  and  represented  by  XML-based  AIML. 
Implemented  system  in  this  study  can  understand  the  student’s  context  of  the  dialogue.  Moreover  the 
dialogue  system  can  answer  the  student’s  question  by  referring  and  saving  the  previous  knowledge  while 
having  a conversation  with  a student.  To  refer  and  save  a student’s  state  of  knowledge,  we  used  an  overlay 
student  model  in  this  system.  Finally,  we  evaluated  an  educational  dialogue  system  to  test  the  efficiency  of 
the  designed  and  implemented  system  with  geometry  learning. 


1.  Introduction 

The  existing  web-based  learning  systems  have  provided  the  learner  the  contents  with  a simple  feedback. 
The  learner  interacts  to  the  system  through  an  interface  screen  simply.  These  methods  do  not  support  an 
intelligent  and  efficient  learning.  The  interaction  between  the  learner  and  the  learning  system  advances  on 
automation  by  an  intelligent  dialogue  agent. 

Established  intelligent  dialogue  systems  were  used  the  Natural  Language  Processing  system  based  on 
the  Expert  systems.  Because  these  systems  interact  with  a guided  question  and  restricted  answer  in  system, 
the  system  and  the  learner  do  not  carry  on  a free  dialogue  concerning  the  related  leaming[8].  Therefore,  we 
design  and  implement  a natural  language-based  learning  system  using  an  intelligent  dialogue  agent.  In  order 
to  implement  the  baming  system,  we  used  an  engine  that  is  a so-called  Tutor-bot  using  the  Pattern 
Matching(PM)  method.  We  selected  the  learning  domain  about  the  geometry  learning  and  extracted  the 
knowledge  base  of  the  Plane  Euclidean  geometry. 

Chapter  2 would  introduce  the  basic  concept  behind  the  PM  method,  and  introduce  the  readers  to 
AIML.  Chapter  3 discusses  about  the  design  of  an  intelligent  dialogue  e-leaming  system.  Chapter  4 talks 
about  the  study  an  implementation  with  the  use  of  tags.  A sample  code  from  geo.aiml  is  included  in  this 
chapter  to  allow  the  reader  to  fully  understand  the  customization  process.  Then  this  study  ends  with  Chapter 
5,  which  talks  about  the  future  implementation  of  dialogue-based  learning  system. 


2.  Dialogues  System  for  Tutoring 

Pattern  Matching  and  AIML 

The  Pattern  Matching(PM)  method  is  usually  driven  by  fairly  simple  mechanisms.  There  is  several  in  an 
existence.  The  first  one  was  ELIZA  created  by  Joseph  Weizenbaum[  13],  which  mimicked  a psychotherapist. 
It  made  quite  a name  for  itself  because  it  succeeded  in  fooling  some  people  it  was  a person.  For  the  most  part, 
they  are  implemented  by  rules  that  map  simple  pattern  matches  on  the  left-hand  side  to  canned  responses  on 
right-hand  side.  Alice  is  a chatter-bot  developed  by  Dr.  Richard  S.  Wallace[12].  Alice  is  an  acronym  for 
Artificial  Linguistic  Internet  Computer  Entity.  It's  programmed  using  an  XML  DTD  called  AIML(Artificial 
Intelligence  Markup  Language).  When  the  user  enters  statements  at  a prompt  on  WebPages,  Alice  fits  that 
statement  to  the  most  'specific'  category  it  can  find,  and  returns  the  response  associated  with  that  category.  It 
is  capable  of  understanding  natural  language  syntax.  However,  the  natural  language  is  extremely  complex.  A 
person  is  presumed  to  have  implicit  or  meta-knowledge  about  the  world  when  having  conversation.  In  human- 
to-human  conversations,  implicit  and  meta-knowledge  is  always  used.  This  in  someway  poses  as  a huge 
challenge  when  customizing  the  Tutor-bot  for  e-leaming  system. 

The  Tutorbot  was  implemented  by  AIML(Artificial  Intelligence  Markup  Language),  which  is  a non- 
standard evolving  markup  language  to  create  the  chat  robots.  The  primary  design  feature  of  AIML  is 
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minimalism.  Compared  with  other  chat  robot  languages,  AIML  is  perhaps  the  simplest.  The  pattern  matching 
language  is  very  simple,  for  example  permitting  only  one  wild-card  (’*')  match  character  per  pattern.  AIML  is 
an  XML  language,  implying  that  it  obeys  certain  grammatical  meta-rules.  The  choice  of  XML  syntax  permits 
integration  with  other  tools  such  as  XML  editors.  Another  motivation  for  XML  is  its  familiar  look  and  feel, 
especially  to  people  with  HTML  experience.  An  AIML  tutor  agent  begins  and  ends  with  the  <alice>  and 

</alice>  tags  respectively.  Typical  Alice  category  tags  have  two  component  tags. 

-A  pattern,  eg:  <pattem>  I HA  VE  A * <pattem> 

-A  template,  eg:  <template>  Where  did  you  get  it?  <template> 

The  pattern  matching  used  by  Tutor-bot  starts  with  analyzing  the  words  in  the  phrase  entered  by  the 
leamert9].  These  words  are  in  natural  language,  separated  by  spaces  and  converted  to  uppercase  to  enable 
case  independent  matching.  If  a matching  category  is  found,  it  is  possible  that  the  category  is  a recursive 
category.  If  this  is  the  case,  the  target  phrase  will  be  altered  and  evaluated  again  by  the  pattern  matcher.  This 

adds  the  wildcard  symbols  . . . and  ' ' to  the  pattern  language.  Many  AIML  patterns  start  with  the  same  words: 
{WHAT  IS  A CIRCLE,.,,  WHAT  IS  A ...  . WHAT  IS.,.,  WHATOVAL..,,  WHAT,,.} 

All  these  patterns  overlap  in  one  or  more  words.  This  fact  suggests  the  use  of  a tree  to  store  the  patterns 
in.  Alice  uses  a rooted,  directed  tree.  Each  node  in  the  tree  contains  a hash  table,  which  is  used  to  store  its 
successor  nodes.  This  way,  the  pattern  matching  time  is  almost  constant  independent  of  the  number  of 
categories.  It  only  depends  on  the  length  of  the  sentence;  the  longer  the  sentence,  the  longer  the  time  to  find  a 
matching  pattern.  Yet,  this  relation  is  linear.  All  patterns  are  stored  in  alphabetical  order. 

Related  Works 

Numerous  experiments  have  been  performed  with  natural  language  interfaces.  Several  domains  have 
been  tested  with  varying  success.  Here,  a few  projects  are  discussed  to  determine  which  domains  are  suitable 
and  which  are  not.  Biermann  [3]  conducted  an  experiment  to  test  the  feasibility  of  natural  language  for 
programming  environments.  Subjects  had  to  solve  linear  equations  and  calculate  averages  on  a natural 
language  system,  which  allowed  them  to  manipulate  tables  and  matrices.  About  80%  of  the  users^  sentences 
were  processed  correctly.  These  results  were  compared  with  a similar  experiment  using  a formal 
programming  language  called  PL/C.  Even  though  the  subjects  who  used  the  natural  language  system  used  up 
to  50%  more  characters,  they  completed  the  problems  using  less  time  than  the  subjects  who  used  the  PL/C. 
This  result  can  be  explained  in  two  ways:  either  natural  language  is  easier  to  use  and  understand,  or  it  is  more 
verbose.  As  discussed  before,  verbosity  is  not  a mayor  problem  and  it  must  be  said  PL/C  is  more  suitable  for 
these  kinds  of  problems  than  natural  language  because  of  the  tables  and  matrices.  If  a domain  had  been 
chosen  that  was  not  obviously  better  suited  for  a mathematical  notation,  shorter  lengths  of  the  natural 
languages  sentences  should  be  expected. 

Hauptman  and  Green  [6]  compared  three  interface  styles;  a command  line  language,  a natural  language 
and  a menu-based  interface.  Each  subject  was  given  three  hand-sketched  diagrams  of  graphs  that  they  were 
supposed  to  reproduce  using  the  interface.  It  appeared  that  the  natural  language  mode  was  not  more  difficult 
than  any  of  the  other  two.  It  must  be  said  though  that  this  task  as  well  as  the  previous  one  are  good  examples 
of  tasks  that  are  inappropriate  for  natural  language  interaction.  Domains  involving  musical  notations,  algebra 
and  graphical  content  are  other  unsuitable  domains.  Kelley  [7]  investigated  the  use  of  natural  language  for 
retrieving  information  from  a calendar.  He  found  that  between  86%  and  90%  of  the  questions  asked  by  the 
users  were  answered  correctly.  This  result  may  look  promising,  but  it  must  be  noted  that  the  domain  is  very 
simple  and  fairly  limited. 

3.  Intelligent  Dialogue  Learning  System 

Overview  of  Dialogue-based  Tutoring  System 

Figure  1 shows  the  overview  of  the  intelligent  dialogue-based  e-leaming  system  architecture.  Dialogue- 
based  Tutoring  System  is  composed  of  3 parts,  learner  interface  by  dialogue  system,  intelligent  dialogue 
tutoring  engine  and  intelligent  tutoring  system  with  learner  base.  The  interface  module  is  the  part  that  learner 
proceed  learning  through  the  web.  Dialogue  module  is  embedded  in  learning  content  for  communication 
between  system  and  learner.  The  dialogue  module  consists  of  client  and  server.  Moreover,  this  module  is  the 
core  of  this  study.  Server  of  this  module  contains  the  pattern  matching  engine  and  the  knowledge  base. 
Learner's  knowledge  level  has  diagnosed  through  learner  profiles  and  dialogue  among  learning.  The 
knowledge  base  saves  domain  knowledge  on  geometry  information.  The  intelligent  tutoring  module  is  typical 
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Intelligent  Tutoring  System.  ITS  is  consisted  of  student  module,  tutor  module,  and  expert  module.  The 
student  model  was  designed  the  overlay  model,  bug  model,  hybrid  model[5]. 


Dialogue  system  Interface 

1 input  sentence  | 

Intelligent  Tutoring  System 


Figure  1.  Overview  of  Dialogue-based  Tutoring  System 


If  learner  inputs  question,  Tutorbot  makes  an  analysis  context  through  preprocessing.  At  this  time,  a 
learner’s  knowledge  and  a mistake  are  stored  in  learner  database.  For  reasoning  a whether  last  question  is 
related  former  questions,  the  pattern  matching  engine  extracts  stored  former  contexts  at  temporary  space. 
Passing  through  pattern  matching  processing,  dialogue-based  system  understands  learner’s  question  and 
shows  a suitable  reply  to  learner.  Finally,  the  system  stores  the  result  in  learner  database  and  repeats. 

The  reason  of  former  question  among  the  rules  of  the  pattern  matching  uses  recursive  method  and 
temporary  space  tag.  This  reasoning  method  is  necessary  for  constructing  student  model.  AIML(Artificial 
Intelligence  Markup  language)  contains  a simple  yet  powerful  XML  markup  tag  called  <srai>.  The  <srai> 
tag  is  the  symbolic  reduction  tag.  This  allows  minimalism.  <srai>X</srai>  is  simple:  The  <srai>  tag  always 
appears  in  the  response  template,  but  the  tutorbot  treats  X just  like  an  input  to  the  tutoibot.  The  tutoibot  scans 
through  its  memory  and  finds  the  best  response  for  X.  The  only  tricky  part  is,  the  response  to  X may  itself 
contain  more  <srai>  tags. The  best  way  to  understand  the  recursive  action  of  the  AIML  <srai>  tag  is  by 
example. 

Student:  You  may  say  that  again  Tutorbot 

Tutorbot:  Once  more?  "that" 

The  Tutoibot  has  no  specific  response  to  the  pattern  ’’You  may  say  that  again  Tutorbot.”  Instead,  the 
Tutorbot  builds  its  response  to  the  learner  input  in  four  steps.  This  simple  sentence  activated  a sequence  of 
four  categories  linked  by  <srai>  tags.  The  tutorbot  constructed  the  reply  ’’Once  more?  ’’that”  recursively  as 
each  sub  sentence  triggered  the  next  matching  pattern.  <srai>  allows  the  translation  of  Natural  Language  to 
keywords.  On  the  other  hand,  the  tag  ’’that”  in  Tutorbot  refers  to  whatever  the  tutoibot  said  before  a user 
input.  Conceptually  the  choice  of ’’that”  comes  from  the  observation  of  the  role  of  the  word  ’’that”  in  dialogue 
fragments  like: 

Tutorbot:  Today  is  yesterday. 

Student:  That  makes  no  sense. 

Tutorbot:  The  answer  is  3.14159 

Student:  That  is  cool. 

In  AIML  the  syntax  <that>...</that>  permits  an  optional  ’’ThatPattem”  to  match  the  tutoibot’s  ’’that” 
expression. 


Representation  of  Geometry  learning  knowledge  for  AIML  Modeling 

We  should  need  that  analyzes  the  scheme  of  geometry  knowledge  for  performing  geometry  learning. 
Because  the  dialogues  on  the  knowledge  of  the  geometry  learning  are  to  teach  knowledge  of  geometry 
domain,  the  domain  knowledge  of  geometry  is  represented  with  a definite  rule  or  context.  Geometry  object  is 
core  that  is  diagram  information.  The  diagram  information  is  divided  the  quantity  and  quality  information. 
The  quantity  information  contains  presented  independence  information  in  learning  spaces  3 an  absolute 
coordinates,  number  of  pixel,  an  inclined  degrees  etc.  The  quality  information  represents  the  ontology  of 
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geometry  objects  like  dot,  line,  triangle,  circle,  as  well  as  the  proposition  alike  a connected  relation  among 
each  geometry  objects,  and  various  knowledge  for  the  form  of  a proposition.  Such  information  gets  to  be  main 
content  for  constructing  learning  and  presents  problem  to  learner  as  well  as  a tutor  system.  Consequently,  this 
diagram  information  determines  an  efficiency  of  dialogue-based  tutoring  system. 

The  quality  information  of  geometry  mainly  is  divided  the  structural  information  and  the  geometric 
information  as  followed  figure  2.  The  structural  information  represents  knowledge  about  relation  among  the 
objects  and  the  features  of  geometry  object.  The  geometric  information  is  an  added  proposition  on  the  basic 
structure.  This  information  is  reasoned  by  the  geometric  or  algebraic  knowledge,  ffficient  dialogue-based 
tutoring  system  needs  suitable  knowledge  representation  about  the  quality  information  of  geometry.  Main 
knowledge  constructed  with  AIML  is  represented  by  structural  information;  on  the  other  hand,  the  used 
knowledge  by  answer  of  dialogue  is  presented  by  the  geometric  information. 


Structural  information:  The  line  AB  is  composed  of  the  dot  A and  dot  B 

The  line  FD  is  composed  of  the  dot  F and  dotD 
The  line  BC  is  composed  of  the  dot  B and  dot  C 
The  dot  B locates  on  the  line  AE  . . . 

Geometry  information : L ines  AE  and  FD  are  parallel 

The  distance  between  the  dot  and  the  dot  Cis  7m 
A Iternate  interior  angles  ABC  and  BCD  is  same  . . . 

Figure  2.  Classification  of  Geometry  information 

Student  Mode)  and  Dialogue  Representation 

The  geometric  knowledge  is  constructed  variously  by  the  learner’s  knowledge  state  and  learning  process 
for  using  intelligent  dialogue.  Intelligent  dialogue  method  is  different  complexity  by  who  take  the  hegemony 
of  dialogue.  The  hegemony  of  dialogue  classifies  by  a case  of  system-initiative,  case  of  learner- initiative  and 
hybrid  case.  A case  of  system- initiative,  system  asks  to  a learner  a question  and  a learner  answer  mostly.  This 
method  is  same  the  one  of  an  expert  system  and  system  reason  the  learner’s  knowledge  in  the  course  of  a 
conversation. 

On  the  other  hand,  a case  of  learner-initiative  doesn’t  use  planed  dialogue  but  system  replies  the  most 
suitable  answer  by  learner’s  question.  Because  this  method  doesn’t  exist  a constructed  knowledge  base 
previously,  the  number  of  answer  is  increase  exponentially.  As  a result,  system  processes  dialogue  with 
restriction  the  band-with  of  learner’s  question. 

Hybrid  case,  if  the  learner  has  a mistake  or  does  not  understand  the  learning  content,  the  dialogue  method 
executes  with  a system-initiative  mode,  if  learner  has  understanding  or  tries  review  learning,  learner-initiative 
drives.  A leamef  s knowledge  through  dialogue  is  stored  by  student  model  and  diagnoses  a learner’  level.  The 
system-initiated  method  uses  overlay  model  and  fixes  up  a learner’s  knowledge  and  the  pre-made  dialogue 
path.  While  the  learner-initiated  method  uses  bug  model  to  find  learner’s  mistakes.  Hybrid  method  uses  a bug 
library  model  namely  machine  learning  for  storing  new  bug. 

AIML  Design  for  Geometry  Learning 

Tutorbot  is  able  to  understand  the  flow  of  conversations  by  capturing  the  questions  and  going  into  a 
particular  topic.  <topic>  allows  Tutorbot  to  prefer  responses  that  deal  with  the  topic  currently  being 
discussed.  This  creates  topical  conversation,  yet  still  has  the  ability  to  move  from  one  subject  to  another.  For 
example,  if  the  questions  were  asked  in  relation  to  Plane  Euclidean  Geometry,  the  file  Geo.aiml  is  that  will  be 
put  into  use. 

Example:  <Student>  Hi! 

<Tutorbot>  Hi!  there  [general  category  ] 

<Student>  Can  you  please  tell  me  more  about  a quadrangle  ? 

<Tutorbot>  Lets  move  on  the  geometry  section  [ geometry  category  ] 

<Student>  How  many  lines  do  a quadrangle  have? 

<Tutorbot>  Four. 

<Student>  How  many  dots  do  a quadrangle  have? 

<Tutorbot>  Four,  too. 
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<Student>  What  is  an  angle? 

<Tutorbot>  That  is  the  degree  of  an  angle  formed  where  two  lines  cross  each  other, 

<Student>  Thanks  you, 

<Tutorbot>  You  ’re  welcome.  [ general  category  J 

Tutorbot  is  capable  of  throwing  in  random  responses  along  with  the  proper  response.  This  in  a way  is 
like  a freebie  guide  that  may  assist  students.  This  type  of  random  responses  tied  with  logical  responses  is 
feasible  in  this  type  of  a setting.  This  gives  some  form  of  personality  to  Tutorbot.  It  would  be  like  dialogue 
with  an  intelligent  tutor  and  from  past  experience,  an  intelligent  tutor  always  throws  in  additional  hints  to 
assist  students  when  making  decisions  about  their  future. 

<category> 

<pattern  >CIRCLES</pattem  > <template> 

<random> 

<li>You  will  have  Jim  in  geometry  learning.  </li> 

<li>Circle  is  very  interesting  figure,  </li> 

<li>What  is  difference  a circle  and  an  oval?  </li> 

<li>Ask  me  anything  you  want  about  circle  </li> 

<li>Did  you  know  that  circles’  features  for  Geometry?  </li> 

</random> 

</template> 

</category> 


4.  Implementation  and  Experience 

Implementation  of  intelligent  dialogue  system 

Figure  3 shows  the  prototype  of  a dialogue-based  e-leaming  system.  This  system  was  designed  to  learn 
the  plane  Euclidean  geometry  learning  with  intelligent  dialogue  method.  A learner  can  learn  geometry 
learning  through  screen  interface  and  can  ask  the  toturbot  a question  by  using  dialogue  interface.  For  reducing 
the  bandwidth  of  learner’s  question,  the  context  of  question  is  limited  in  related  learning  contents . 
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Figure  3.  Screenshot  of  intelligent  dialogue-based  e-leaming  system 


The  dialogue-based  engine  used  JDK  1.3  program  and  dialogue  server  executes  by  servlet.  When  server 
drives  by  servlet,  the  dialogue  engine  loads  category  items  of  geo.aiml  file  in  memory.  Moreover,  server 
creates  log  file  for  grasping  the  learner’s  information.  Right  in  figure  3 shows  server  screenshot  of  Tutorbot. 
H/W  by  using  research  is  Windows  2000server.  If  a Tutorbot  server  executes,  dialogue  port  is  used  the  2001 
port  with  HTTP  and  all  learner  can  communicate  with  system.  Tutor-bot  server  in  figure  3 can  see  that 
geo.aiml  was  loaded  in  memory  by  driving  server 


Sample  of  Conversation  with  Tutor-bot 

The  category  on  geometry  learning  was  made  with  23000-domain  knowledge  and  knowledge  was 
stored  in  geo.aiml  file.  A learner  can  communicate  with  a special  knowledge  in  geo.aiml  file  and  a general 
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knowledge  in  general.aiml  file  The  following  Figure  4 was  recorded  between  Tutorbot  and  a learner.  A 
learner  asks  a system  a question  about  the  quadrangle  and  we  can  see  that  a tutorbot  replies  a suitable  answer. 
When  learner  uses  an  approximated  question  or  an  illegal  grammar,  a tutorbot  introduces  to  change  a 
direction  of  dialogue. 


<Student>  Hi  t 
<Tutort»ot>  Hi  I ttiere. 

<Student>  Can  you  Please  tell  me  more  about  a 
Quadrangle  ? 

<Tutorbot>Lets  move  on  the  geometry  section, 
<Student>  How  many  lines  do  a quadrangle  have? 
<Tutorbot>Four, 

<Student>  How  many  dots  do  a quadrangle  have? 
<Tutorbot>  Four.  too. 

<Student>  What  is  an  angle? 

<Tutorbot>That  is  the  degree  ot  an  angle  formed  where 
two  lines  cross  each  other, 

<Student>Thanks  you, 


You  say.  ’P" 


■■[teSlir]  J_j 


Figure  4.  Conversation  with  Tutor-bot 


5.  Conclusion  and  future  work 


Dialogue  systems  are  becoming  very  common  in  many  web  sites  today.  They  serve  to  give  personality  and 
intelligent  information  to  web  sites.  Specially,  a dialogue  system  approves  a merit  in  o-leaming  system. 
Typical  technique  of  intelligent  tutoring  system  is  intelligent  dialogue  based  student  model.  We  discussed  on 
dialogue  technique  for  such  intelligent  tutoring  system  in  this  paper.  Moreover  we  extracted  the  domain 
knowledge  of  geometry  learning  and  geometry  analyzed  structural  information  and  geometric  information. 
This  information  and  knowledge  could  use  effectual  for  intelligent  dialogue  system.  We  hope  for  developing 
intelligent  dialogue-based  tutoring  system  by  our  study. 

Futures  works  need  that  study  on  temporal  reasoning  method  by  changing  a learner’s  knowledge  as  well 
as  research  on  virtual  interface  by  using  3D  and  virtual  reality.  Furthermore,  for  improving  the  accuracy  of 
dialogue,  we  will  study  on  the  reasoning  method  of  an  ambiguity  and  uncertain  knowledge. 
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Abstract:  The  need  of  authoring  tool  insp  ired  the  creation  of  VRCapture,  which  is  a module 
in  the  Configurable,  Incremental  and  Re-structurable  Contributive  Learning  Environments 
(CIRCLE)  and  is  capable  of  providing  a tool  to  capture  the  authoring  teacher’s  expertise.  To 
ensure  the  expandability,  object-oriented  analysis  and  design  methodology  is  employed,  and 
reusability  guides  the  development  process.  VRCapture  with  its  environment  CIRCLES 
assemble  a computer-based  learning  environment  that  overcomes  many  traditional  CAI 
systems’  deficiencies  such  as,  high  development/modification  cost,  non-modularity/  non- 
reusability, and  commonly  low  acceptances  of  end-user  teachers. 


Introduction 

The  primary  function  of  the  teaching  process  is  to  achieve  the  transfer  of  knowledge  from  someone 
who  is  richer  in  terms  of  these  commodities  (knowledge)  to  those  who  own  less.  In  order  to  secure  the 
knowledge  transfer  process,  some  channels  (or  mediums)  of  communication  must  link  the  parties  involved. 
Possible  channels  include: 

1 . schools/  learning  and  training  institutions 

2.  home 

3.  work  places 

4.  social  and  leisure  centres 

Channel  (1)  and  (2)  are  very  important  in  our  early  life,  whereas  channels  (2)  to  (4)  become  of 
increasing  value  in  our  later  life.  Barker  & Yeates  (1985)  noted  that  “Home  and  work  environments  are  two  of 
the  most  effective  ways  of  disseminating  educational  information”.  Computers  have  been  increasingly  used  to 
facilitate  or  even  used  as  the  communication  channel  due  to  the  difficulties  for  a human  teacher  to  fulfil  all 
requirements,  which  Barker  & Yeates  (1985)  argued  as  the  necessities  of  a good  teacher: 

1 . An  agent  for  the  dissemination  of  information  and  knowledge. 

2.  A motivator  for  the  development  of  learning  skills. 

3.  A means  of  measuring  the  effectiveness  of  1 , and  2. 

4.  Adaptable  and  patient. 

5.  Able  to  modify  teaching  strategies  to  the  needs  of  individual  students. 

6.  Always  available  when  needed. 


Computer  Assisted  Learning  (CAL)  System 

High  availability  is  the  most  obvious  contrast  between  a computer  assisted  learning  system  and  a real 
human  teacher.  And  if  intelligence  is  built  into  the  learning  systems,  they  can  even  provide  an  adaptable 
learning  environment,  which  can  be  tailored  to  the  student’s  individual  needs.  For  example,  CAPIT  (Mayo  et  al., 
2000)  is  an  intelligent  system  that  teaches  10-11  years  old  school  children  the  capitalisation  and  punctuation 
rules  in  English  grammar.  It  can  provide  selectively  appropriate  levels  (brief  to  detailed)  of  feedback  to  a 
student,  when  he/she  makes  a mistake,  based  on  the  individual  student’s  performance. 

As  explained  in  the  ‘Situated  Learning’  section  later,  expertise  is  achieved  by  continually  repetitive 
practices.  Computers  are  excellent  tools  for  doing  repetitive  tasks  (as  they  will  never  complain!)  such  as  to 
teach  a child  how  to  punctuate  English  sentences  or  to  provide  exercises  on  an  accounting  topic.  Teachers’  time 
can  be  spent  more  on  the  creativity  or  strategies  of  the  curriculum  design  rather  than  on  the  repetitive  tasks  that 
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many  of  computerised  systems  are  aimed  for.  'Computers  can  make  good  teachers” y said  Barker  & Yeates 
(1985),  whereas  our  interpretation  is  that  ^‘computers  make  it  easier  for  a teacher  to  be  a good  one”. 


Issues  in  Computer  Assisted  Learning  Systems 

The  CAL  systems  are  generally  built  by  computer  programmers  who  may  have  only  little  or  no 
understanding  of  the  theories  of  education.  As  a result,  those  systems  created  could  be  excellent  in  terms  of 
program  design,  computer  resource  utilisation  and  performance  optimisation,  but  useless  in  real  classrooms. 

When  a traditional  CAL  system  is  built,  the  author  has  to  specify  its  screen  design,  instructional 
content,  and  instructional  flow.  Any  change  has  to  be  preformed  explicitly  in  terms  of  the  design  layout, 
sequences  of  instructions  and  the  content  materials.  In  other  words,  the  system  is  non-modular,  difficult  to  reuse 
and  any  modification  can  be  very  time  consuming  and  costly. 

Another  issue  for  traditional  CAL  systems  is  that  the  systems  have  to  be  accepted  by  the  teachers  who 
are  actually  part  of  the  end  users.  The  NDH  (Not  Developed  Here)  ^ndrome  psychologically  creates  the 
hindrance  to  the  acceptance  and  leads  to  a mistrust  from  teachers  to  the  system.  It  is  already  hard  to  develop  a 
system  that  could  suit  two  mathematical  teachers’  flavour  let  alone  a system  that  could  be  accepted 
internationally. 

Next  in  this  paper,  we  shall  talk  about  how  VRCapture  and  the  Configurable,  Incremental  and  Re- 
structurable  Contributive  Learning  Environments  (CIRCLE)  address  those  problems  mentioned  above.  We  shall  then 
discuss  how  object-orientation  facilitates  the  achievement  of  reusability  and  modularity  in  the  system  and  how 
the  employment  of  situated  learning  theory  provides  the  benefit  of  more  complete  and  accurate  knowledge 
acquisition.  In  the  last  two  sections,  we  shall  describe  the  VRCapture  system  architecture  and  future 
possibilities. 


Contribution  from  VRCapture 

By  using  the  VRCapture,  the  authoring  task  is  shifted  from  the  traditional  programmers  of  traditional 
CAL  to  the  teachers  who  use  the  material  and  know  the  students.  The  teachers  create  what  they  want  the 
students  to  learn  from  the  system  by  themselves.  Thus,  the  teachers  would  feel  more  in  control  of  the  teaching 
process  and  naturally  they  are  motivated  to  encourage  the  students  to  use  ‘their’  systems.  In  addition,  the 
knowledge  repository  can  be  improved  directly  based  on  the  teachers’  teaching  experiences  and  students’ 
feedback. 

The  knowledge  or  the  rules  are  stored  separately  in  CIRCLE  (see  figure  1).  Any  modification  in  the 
knowledge  repository  immediately  brings  its  effect  to  the  system  in  front  of  the  students.  While  the  knowledge 
is  not  hard-coded  into  the  program,  it  is  an  inspectable  repository  and  can  be  queried.  Future  expansion  could 
possibly  introduce  an  additional  module  that  could  make  the  knowledge  repository  runnable.  By  runnable,  we 
could  imagine  a portrayal  of  a system  that  behaves  like  an  expert  system  and  answering  queries  by  induction  or 
reduction  of  the  rules  stored  in  the  repository.  Furthermore,  the  knowledge  repository  and  the  program  code  can 
be  either  maintained  or  even  reused  for  different  subject  areas  independently. 

Teachers  or  instructors  do  not  need  to  worry  about  the  design  of  the  screen  layouts  and  the 
construction  of  instructional  sequence.  They  only  need  to  input  appropriate  parameters  by  which  the  system  will 
determine  how  to  layout  and  sequence  each  topic  when  students  come  to  use  them.  The  simple -to-use  and  easy- 
to-leam  characteristics  provide  a friendly  environment  for  the  teachers  who  have  less  experiences,  or  who  do 
not  wish  to  spend  too  much  time  on  the  meta-leaming  of  the  system. 


VRCapture  in  a Larger  System 

Inside  the  CIRCLE  (see  figure  1),  VRCapture  is  responsible  for  providing  an  authoring  tool  to  capture 
the  teachers’  expertise  into  the  knowledge  repository.  The  knowledge  repository  contains  all  the  rules  and 
relationships  of  domain  concepts.  The  data  inside  the  knowledge  repository  is  then  fed  to  the  Intelligent  Tutor 
on  the  demand  of  students.  The  main  purpose  of  Intelligent  Tutor  is  for  the  students  to  utilise  those  knowledge 
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stored  in  the  repository.  It  provides  a user-friendly  and  adaptive  interface  from  that  the  students  can  learn,  do 
exercises,  and  test  them-selves. 


Benefit  of  CIRCLES  Architecture 

Each  of  the  CIRCLES*  components  can  be  developed  independently  by  providing  their  interface 
specifications  to  the  developers  of  the  other  components.  Its  modular  structure  enhances  greatly  its 
expandability  and  maintainability.  By  this  modular  architecture,  new  modules  or  functions  can  be  just  plugged 
in,  or  existing  modules  can  be  re-structured  or  disposed  without  causing  any  vital  effect  to  the  overall  system. 


circles; 

Users 


Teacher  Student 


Figure  1 -The  CIRCLES  Architecture 


The  users  of  CIRCLES  can  also  enjoy  the  benefits  from  its  light-weightiness.  The  students  only  need 
the  part  of  the  system  to  perform  their  tasks,  so  do  the  teachers.  No  matter  how  much  the  system  will  expand  in 
the  future,  the  users  can  still  use  the  version  they  have  processed  or  just  require  a small  incremental  patch. 

Openness  is  a highly  valued  criteria  when  the  underlying  technology  is  selected.  The  underlying  Java 
technology  ensures  the  system  is  portable  over  platforms.  The  portable  and  concise  characteristics  of  CIRCLES 
components  open  to  us  the  possibility  and  ambition  for  migrate  CIRCLES  on-line  and  hopefully,  in  the  near 
future,  more  distant  learner  can  be  benefited  from  it. 


Object-Orientation  and  Reuse 

There  are  primary  two  divisions  of  reuse  in  terms  of  the  development  style;  Development- for- Reuse, 
and  Development-with-Reuse.  Development- for- Reuse,  by  Hallsteinsen  and  Pad’s  definition,  “w  the  process  of 
developing  and  preparing  components  such  that  they  can  be  reused  in  future  applications  and  contexts” 
Therefore,  during  the  design  or  implementation  stage  the  developer  understands  that  the  artefacts  under 
development  will  be  reused.  Whereas,  Development-with-Reuse  is  the  construction  of  software  by  utilising 
existing  reusable  components.  It  is  an  activity  that  involves  the  analysing,  organizing,  cataloguing,  and 
evaluating  of  software  artefacts  which  can  be  placed  in  reuse  library. 

VRCapture  adopts  the  Development-For-Reuse  guideline  and  use  00  techniques  to  achieve  least 
coupling  of  objects  and  maximum  cohesion  by  clearly  and  logically  separating  different  concepts  into  different 
implementation  classes.  Now  let’s  look  at  how  the  Object-Orientation  helps  the  reusability  by  its  two 
distinguishing  concepts:  encapsulation,  and  inheritance. 

Encapsulation  provides  independence  to  an  object,  which  can  be  deployed  with  other  objects  to  form 
a functional  component.  Object  independence  is  an  important  characteristic  in  terms  of  reusability.  Object- 
Orientation  facilitates  high  cohesion,  since  every  object  stands  for  an  explicit  concept  in  the  problem  domain 
whereas,  low  coupling  and  high  modularity  are  the  ‘should-be’  of  Object-Orientation.  Encapsulation  also 
provides  a mean  to  achieve  information  hiding  by  which  the  details  that  is  likely  to  change  over  the  time  are 
concealed.  Very  possibly  the  components  that  communicate  to  hardware  need  a replacement  when  the  system’s 
hardware  is  upgraded  or  the  system  migrates  from  one  platform  to  another.  Database  selection  can  be  replaced 
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just  by  modifying  the  ‘DBManager’  (see  figure  2)  in  VRCapture.  Only  those  that  are  unlikely  to  change  should 
be  selected  as  the  external  interfaces  of  components. 

Class  inheritance  defines  class  relationships  by  reusing  the  methods  and  data  defined  in  another  class. 
By  doing  this,  redundancy  is  removed  and  tested  code  can  be  reused  to  achieve  both  quality  control  and  cost 
saving. 


Situated  Learning 

In  the  VRCapture  system,  the  main  focus  is  to  store  the  teacher’s  knowledge  electronically,  so  the 
knowledge  can  be  accessed  without  the  limitation  of  time  and,  may  be  in  the  future,  even  without  the  limitation 
of  spaces,  if  a web-based  interface  is  set  up  for  students.  The  system  aims  at  capturing  the  domain’s  knowledge 
effectively,  and  this  aim  is  achieved  by  not  applying  the  theory  of  situated  learning  to  the  students  but  to  the 
teachers.  We  will  first  look  at  the  general  theory  of  situated  learning  and  then  we  will  talk  about  the  application 
of  the  theory  in  the  VRCapture. 

Situated  learning,  also  called  problem-based  learning,  proposed  that  the  contextual  attribute  of 
knowledge  makes  it  not  effective  to  teach  conceptual  knowledge  without  defining  its  application  situations. 
“Rea/  world  learning  has  a backbone  of  problem  solving,  production  of  work-authentic  products,  and 
investigation  and  research,  in  which  all  knowledge,  processes  and  techniques  connect  and  are  used*'  (Glasgow, 
1997). 

What  Glasgow  (1997)  also  describes  in  a traditional  school  nowadays,  is  that  the  teachers  hope  that  the 
curriculum  design  and  their  teaching  style  will  meet  the  need  of  the  majority  of  their  students,  but  in  fact  “Mwc/i 
of  today  *s  curriculum  is  based  on  teacher  *s  past  experience  in  schools,  input  from  textbook  manufactures, 
discipline  frameworks,  standards,  and  information  from  peers". 

However,  learning  in  a real-world  situation  is  usually  contextual,  continual,  and  repetitive.  For  learning 
to  be  able  to  be  packed  into  the  long-term  memory,  continual  repetition  is  a method  being  employed 
traditionally  and  it  proves  itself  successfully  (Shou,  1998). 

For  learning  to  being  contextual,  we  are  able  to  link  pieces  of  conceptual  knowledge  with  real  life 
situations,  which  in  term  helps  the  validation  or  incurs  more  inspirations  and  associations  on  the  conceptual 
knowledge.  Additionally,  real  world  learning  usually  happens  while  there  is  a need  - a problem  we  encounter  in 
the  workplace  or  a question  we  want  to  solve. 

It  is  more  difficult  initially  for  the  students  to  grasp  the  big  picture  of  how  to  apply  the  given 
conceptual  knowledge.  Brown  (1989)  gave  an  analogical  links  of  knowledge  to  tools,  ‘W  should  abandon  any 
notion  that  they  (idea  and  concepts)  are  abstract,  self-contained  entities.  It  may  be  more  useful  to  consider 
conceptual  knowledge  as,  in  some  way,  similar  to  a set  of  tools" . Knowing  the  description  of  the  tool  or  the 
functions  of  this  tool  doesn’t  make  a person  expert  or  even  capable  of  using  this  tool.  To  master  a tool,  the 
apprentice  (student)  observes,  mimics  what  the  master  (teacher)  does,  and  then  repeats,  and  explore.  The 
process  to  master  a conceptual  knowledge  can  be  similar  as  well. 


Application  of  situated  learning  in  the  VRCapture 

Situated  learning  is  applied  in  this  project  in  the  creation  of  the  knowledge  that  is  inputted  by  the 
teachers  and  can  be  stored  in  the  repository.  Experienced  teachers,  who  are  in  charge  of  inputting  the 
relationships  of  the  domain  concepts,  can  more  completely  and  more  precisely  identify  the  relationships 
between  the  concepts.  However,  for  someone  who  has  less  experience  on  both  the  domain  practice  and  the 
system,  he/she  may  need  some  assistance  on  inputting  what  he/she  knows  about  the  domain  concepts.  A 
question  is  therefore  asked,  “How  can  we  facilitate  the  knowledge  capturing  process  and  make  sure  the 
knowledge  is  completely  and  correctly  inputted,  as  far  as  in  the  teacher’s  personal  understanding”? 

An  optional  mechanism  is  created,  by  which  the  teacher  can  refine  the  stored  knowledge  by  doing 
exercises.  This  is  optional  for  more  adaptability  of  power  users.  This  mechanism  situates  the  teacher  in  the 
student’s  point  of  view  in  order  to  provide  a different  perspective,  which  is  aimed  to  reverse  the  tradition  of 
teacher-centred  education  to  a student-centred  one.  Questions  are  posed  to  the  teacher  randomly  and  he/she 
answers  the  questions  like  every  student  would  do.  By  doing  the  exercise,  the  teacher  has  to  examine  what 
he/she  has  inputted  into  the  database,  and  what  he/she  will  do  as  a student  in  response  to  those  questions.  The 
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teacher  is  actually  learning  the  student’s  learning  process,  and  hopefully  the  resulted  knowledge  capture  can  be 
more  robust  and  more  suitable  for  deploying. 

Additional  knowledge  can  be  found  by  comparing  the  teacher’s  answer  to  the  existing  knowledge  in 
database.  Mis -concepts  or  conflicts  in  existing  captured  relationship  can  also  be  identified  and  corrected 
accordingly.  Though  this  is  an  optional  step  but  it  ensures  the  completeness  of  the  captured  knowledge  before 
the  students  actually  start  to  work  on  these  knowledge.  Therefore,  even  for  experienced  users,  this  mechanism  is 
still  highly  recommended. 


Local  Network 


Figure  2:  VRCapture  system  architecture 


The  VRCapture  system  is  basically  a two-layered  system  (see  figure  2).  It  uses  a database  as  the 
knowledge  repository  to  store  the  captured  relationships.  A database  interface  called  ‘DBManager’  is  built  to 
interface  the  transactions  with  knowledge  repository.  The  implementation  details  which  could  change  over  time 
or  requirements  are  all  placed  in  ‘DBManager’.  Thus,  the  rest  of  the  program  can  be  database  independent.  This 
is  the  implementation  of  information  hiding  mentioned  above.  Figure  3 shows  an  interface  for  formulating  and 
recording  relationships/rules. 


Development  Process 

The  development  started  by  trying  to  understand  the  problem  or  the  requirement  statements.  The  initial 
general  requirement  statement  was  expanded  into  a list  of  sub  statements.  Confirmation  of  those  sub  statements 
with  the  supervisor  also  confirms  my  understanding  of  the  problem. 
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In  the  next  step,  the  selection  of  methodology  was  performed.  Object-oriented  methodology  was 
selected  due  to  the  following  reasons: 

• The  implementation  tool,  Java,  is  itself  object-oriented. 

• Object-oriented  system  encourages  explorative  development  by  which  a object  can  be  created, 
experimented  and  then  kept  or  discarded. 

• Object-oriented  system  has  more  flexibility  to  expand. 

• Object-oriented  system  provides  an  easier  maintenance  and  debugging  environment. 

• Object-oriented  development  facilitates  reusability. 

In  the  analysis  phase,  use  cases  are  used  to  document  and  identify  required  functions.  Use  cases 
provide  insights  of  how  the  system  interacts  with  the  outside  world  and  they  can  be  collected  into  the  user 
manual  for  a step-by-step  guidance.  The  creation  of  class  diagrams  provides  a direct  mapping  from  design 
artefacts  to  program  code.  Each  class  in  class  diagram  represents  an  implementation  class  in  the  VRCapture. 
The  links  between  classes  may  be  association,  relationship  or  mapping  (see  Odell  and  Martin,  1998  for  more 
details  about  object  relationships). 

Duplication  check  is  the  mechanism  to  check  whether  two  relationship  in  conflict; 

(x  + y = 3)  and  (x  + y = 5)  are  in  conflict.  When  system  detects  any  potential  conflict,  it  will  rise  a diabg  asking 
the  teacher  whether  this  conflict  is  intentionally  allowed  or  not. 


Conclusion  and  Future  Development 

The  need  of  authoring  tools  for  computer  assisted  learning  system  has  inspired  the  creation  of 
VRCapture.  CIRCLES  serve  as  architecture  that  d ifferent  tools  can  be  integrated  to  perform  intelligent  tutoring. 
However,  in  the  near  future,  we  would  like  to  overcome  some  of  the  limitations  of  CIRCLES  to  expand  the 
usable  coverage  to  a wider  range. 

The  current  system  has  already  the  support  for  use  in  a local  network.  The  CIRCLES  system  can  also 
be  expanded  to  embrace  the  popularity  of  Internet.  Through  the  Internet,  the  system  can  be  within  reach  of  a 
wider  range  of  audience.  The  underlying  Java  technology  makes  it  very  easy  to  have  an  interface  using  Internet 
browsers.  Browser  interface  modules  can  be  created  and  plugged  in  to  current  architecture. 

The  current  system  can  only  takes  in  rules  that  have  three  elements  (e.g.  elementl  + element2  = 
elements)  and  only  certain  operations  This  limitation  can  be  overcome  by  Qmip\oy\n%  operation  stack 

and  operator  procedure  parsing  (Weiss,  1998),  which  could  interpret  formulae  with  no  length  limitation  and 
can  identify  operation  precedence.  And  with  additional  operations,  the  system  can  also  be  used  for  complex 
mathematical  formulae  or  other  numerical  disciplines.  Finally,  the  question  of  whether  computerised  system 
such  as  CIRCLES  can  be  used  for  other  non-numerical  subjects  like  philosophy  or  psychology  has  to  be  left 
open  and  requires  more  future  corroboration  and  investigation. 
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Abstract:  This  paper  presents  the  research  and  development  of  knowledge  management 
system  for  Internet  based  intellectual  assets  to  leverage  the  learning  process  for  individual 
learners.  The  literature  review  reveals  that  corporate  solutions  of  knowledge  management  are 
not  directly  applicable  due  to  the  lack  of  adaptivity  needed  to  provide  individualized  feedback 
to  the  learner,  and  their  localized  nature  that  does  not  directly  deal  with  authentication  and 
verification  of  distributed  information.  Architecture  has  therefore  been  formulated  for  the 
knowledge  managpment  system  that  specifically  supports  web-based  learning. 


1.  Introduction 

Strategic  concern  for  survival  in  the  global  information  technology  and  information  systems  disciplines 
calls  for  constant  upgrade  of  knowledge  especially  in  computing  related  areas.  The  primary  reason  for  this  is 
the  ever-changing  technology,  which  compels  employees  to  be  up-to-date  in  their  knowledge.  The  trend  of 
increasing  Internet  based  learning  is  evident  from  the  increasing  number  of  universities  providing  facilities  to 
finish  degrees  and  diplomas  through  Internet.  In  many  instances  it  becomes  pertinent,  as  otherwise  it  would 
mean  loss  of  job.  The  implication  is  working  population  trying  to  upgrade  their  knowledge  through  Internet 
based  learning. 


2.  Problems  with  the  existing  online  education  systems 

Online  education  systems  provide  a great  solution  for  working  community.  But  they  do  not  come  with 
unmixed  blessing.  One  of  the  primary  considerations  or  constraint  for  both  student  and  teachers  is  the  time 
element.  This  necessitates  a means  by  which  the  accurate  and  relevant  knowledge  can  be  accessed  efficiently 
by  the  students  without  wasting  much  time.  On  the  other  hand,  teachers  should  be  able  to  save  their  precious 
time  by  reusability  of  knowledge.  This  warrants  the  use  of  a knowledge  management  system. 

In  a large  distant  learning  class,  it  is  very  often  not  possible  to  answer  individual  queries  due  to  low 
students  to  teacher  ratio.  It  also  lacks  some  important  benefits  of  traditional  classroom  setting  such  as  instructor- 
student  and  student -student  interaction,  collaborative  work  among  peers,  and  so  on.  Although  these  factors  can 
be  translated  into  the  Internet  based  learning  by  use  of  newsgroups  or  mailing  lists  and  use  of  collaborative 
technology  such  as  discussion  forums,  but  there  are  several  limitations.  The  first  problem  is  the  hesitation 
within  students  to  post  questions  in  a public  arena  such  as  discussion  forum.  Instead,  they  tend  to  post  their 
queries  to  individual  instructors  asking  for  individual  attention.  The  second  problem  is  the  information  overload 
in  the  collaborative  environment  over  time,  and  it  becomes  extremely  difficult  to  mine  the  knowledge.  The 
information  exchange  in  such  environments  takes  place  in  a haphazard  manner  due  to  different  temporal  and 
spatial  conditions  of  individual  students.  This  results  in  non-synchronization  and  repetition  of  information 
leading  to  even  more  tedious  process  of  knowledge  mining. 

Another  inherent  problem  for  students  who  wants  to  have  an  in-depth  knowledge  on  any  subject 
through  Internet  is  the  overwhelming  amount  of  information  available  on  the  Internet  from  all  over  the  world. 
To  quote  Koniger  (Koniger  et  al.,  1995)  in  the  article  on  ’’Drowning  in  information  but  thirsty  for  knowledge”, 
''Information  is  only  valuable  to  the  extent  that  it  is  structured.  Because  of  a lack  of  structure  in  the  creation, 
distribution  and  reception,  information  often  does  not  arrive  where  it  is  needed  and  therefore,  useless^\  This 
disconcert  becomes  a serious  barrier  to  potential  learners.  There  are  no  formal  mechanisms  available  to  filter  the 
information  for  the  quality  and  authenticity  verification.  Added  to  this  is  that  the  information  is  not  adapted  to 
individual  learners  and  therefore  learning  does  not  take  any  consideration  of  hdividual  student  attributes. 
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Another  problem  associated  with  gathering  knowledge  from  the  Internet  is  that  the  available  information  is  not 
structured  and  not  available  in  a uniform  format.  In  other  words,  the  available  information  is  in  heterogeneous 
formats  varying  even  within  a single  source.  This  makes  it  extremely  difficult  to  search  and  retrieve  the 
information,  posing  one  of  the  biggest  challenges  in  making  flexible  and  robust  knowledge  management 
systems. 


3.  Solution 

In  this  background,  the  use  of  knowledge  management  in  Internet  based  learning  can  provide  an 
efficient  solution  for  learners.  The  benefits  of  using  knowledge  management  system  in  internet  based  learning 
would  include  increased  efficiency  in  knowledge  solicitation,  avoid  wasting  time  on  repetitive  queries,  better 
use  of  resources  etc. 

In  the  following  section  we  will  look  at  the  applicability  of  knowledge  management  systems  to  Internet 
based  learning.  The  next  section  will  look  at  the  methodology  to  integrate  knowledge  management  and  Internet 
based  systems  to  alleviate  the  problems  common  to  Internet  based  learning.  This  is  aimed  to  cater  for  both  basic 
and  advanced  learners.  The  following  section  will  look  at  the  prototype  showing  various  processes  involved  in 
the  system.  The  next  section  will  look  at  the  benefits  of  the  prototype  system,  followed  by  the  conclusion. 


4.  Link  between  Internet  based  learning  and  knowledge  management 

The  central  theme  of  knowledge  management  perceived  by  many  experts  in  the  field  is  that  it  is  an 
integrated  and  systematic  process  of  acquiring,  eliciting,  organizing,  representing  and  retrieval  of  information 
asset.  The  objective  of  the  knowledge  management  in  Internet  based  learning  is  to  generate  value  in  terms  of 
knowledge  from  intelligent  capital  to  enable  faster  and  efficient  learning.  According  to  Liebowitz  (1999), 
knowledge  management  deals  with  the  conceptualization,  review,  consolidation,  and  action  phase.  Knowledge 
management  in  Internet  based  learning  is  about  connecting  learners  with  learners,  and  teachers  and  learners 
with  information  and  knowledge  (Corrall,  1999).  It  is  about  getting  the  right  knowledge  to  the  right  person  at 
the  right  time. 

The  phenomenon  of  overload  of  information  to  learners  results  in  the  learners’  inability  to  cope  with 
the  processing  of  increased  amount  of  available  information.  Internet  based  knowledge  management  systems 
can  prove  to  be  efficient  solution  to  manage  the  problems  associated  with  Internet  based  learning.  They  can 
improve  the  productivity  through  achieving  reusability  and  enhancing  educational  services  to  the  learners.  Also, 
immediate  clarification  of  doubts  of  the  learners  through  on-line  knowledge  management  system  can  enable 
them  to  correct  the  mistakes  without  waiting  for  the  teachers  or  tutors  to  respond  to  learners.  Features  specific 
to  learning  institutions  are  knowledge  intensive  nature  and  extensive  reusability  of  learning  materials.  These 
reasons  call  for  the  use  of  knowledge  management  system  in  learning.  Synergy  of  Internet  based  learning  and 
knowledge  management  systems  would  be  ideal  for  saving  time,  avoiding  reinventing  wheel,  increasing 
efficiency  and  so  on. 


5.  Methodology 

This  paper  proposes  the  use  of  knowledge  management  systems  for  both  basic  and  advance  learners. 
Basic  learners  are  those  who  would  like  to  just  finish  the  course  without  any  reference  to  any  additional 
material.  The  advance  learners  are  those  who  would  like  to  enhance  their  knowledge  in  a given  area  beyond  the 
requirements  of  the  course  curriculum. 


Part  1:  Catering  for  basic  needs  of  the  learner  matching  the  curriculum 

Catering  for  the  basic  learners  by  giving  immediate  and  relevant  access  to  the  knowledge  is  a multi- 
step  process.  The  first  step  is  the  collection  of  all  basic  material  namely  concepts,  exercises,  definitions, 
examples  etc.  and  store  them  in  the  knowledge  base.  Additional  material  can  be  collected  from  various  sources 
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like  the  previous  years  archives  (if  available)  of  all  questions  including  frequently  asked  questions  and  answers 
for  them  from  various  sources  like  newsgroups,  discussion  forums  and  mailing  list  etc.  This  archive  can  be  used 
to  compile  a knowledge  base  and  updated  based  on  the  new  questions  from  the  current  students.  Student  profile 
parameters  are  also  collected  including  extent  of  learning  required,  existing  level  of  expertise,  key  interest,  etc. 
The  second  step  is  to  organize  and  transcribe  the  collected  information  in  the  knowledge  base.  The  third  step  is 
to  create  a context -based  search  technology,  which  would  enable  the  system  for  easy  retrieval  and  to  provide 
quick  and  accurate  information  to  the  learner.  The  search  mechanism  should  allow  the  students  to  search  by 
topic  or  index  or  by  keyword  as  in  common  software  help  systems.  The  fourth  step  is  the  integration  of  user 
profile  and  knowledge  base  to  give  the  best  possible  contextual  resources  related  the  search  query.  The  fifth  and 
final  step  is  to  create  a user-friendly  and  intuitive  interface  for  students  to  explore  the  knowledge  content  and 
ask  queries. 


Part  2:  Catering  for  the  advanced  learner 

Catering  to  advance  learners  requires  quick  acquisition,  integration  and  presentation  of  relevant  and 
correct  knowledge  from  within  the  knowledge  base  and  from  websites  distributed  on  the  Internet  and  related  to 
the  context  of  the  learner.  The  major  concern  in  this  process  is  the  explosive  growth  of  information,  which  lacks 
any  quality  checks,  and  integration  of  such  information  to  make  it  useful  knowledge  in  the  context  of  learner’s 
learning  process.  This  requires  meticulous  evaluation  of  material  available  on  the  Internet  and  compiling  and 
organizing  the  material  to  suit  individual  learning.  Links  to  such  evaluated  material  is  maintained  in  the 
knowledge  base  for  future  use.  This  directory  of  these  links  is  updated  on  a frequent  basis.  The  rest  of  the  steps 
involved  in  this  part  of  the  system  are  very  similar  to  the  first  part  and  include  information  collection, 
evaluation  for  quality  and  authenticity,  organization,  creating  proper  retrieval  mechanisms,  integration  of  user 
profile  and  knowledge  base  and  finally  creation  of  user  interface. 

Figure  1 explains  the  processes  involved  in  knowledge  management  systems  in  Internet  based  learning. 


Figure  1:  Knowledge  Management  Systems'  Architecture 

The  results  provided  from  the  knowledge  management  system  to  any  specific  query  are  precise  and 
concise.  In  other  words  students  do  not  have  to  filter  through  abundance  of  information.  Instead  the  knowledge 
management  system  does  the  Job  of  sifting  through  the  available  information  and  gives  qjpropriate  and  precise 
query  results  based  on  user  profile. 


6.  Prototype 

Based  on  the  architecture,  a prototype  system  is  being  developed  which  aims  to  guide  the  learners  a 
guided  discovery  learning  process.  The  guidance  is  individualized,  based  on  the  behavioural  attributes  of  the 
learner.  The  components  of  the  prototype  are  knowledge  base,  user  profile  database,  knowledge  engine, 
inference  engine  and  interface.  Knowledge  base  contains  locally  available  information,  references  to  Internet 
based  distributed  information,  and  cache  of  frequently  accessed  information.  User  profile  database  refers  to  the 
attributes  of  the  learner  gathered  through  user  modeling  mechanisms.  Knowledge  engine  helps  in  manipulating 
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the  acquired  information  and  effective  indexing  and  storage.  Inference  engine  enables  retrieval  of  knowledge  as 
per  individual  learner’s  attributes.  Interface  provide  adaptive  representation  of  the  knowledge  to  the  learner. 

Figure  2 summarizes  the  sequence  of  actions  as  a result  of  the  student’s  query  and  the  mechanism  used 
in  the  processes.  Student's  question  on  the  interface  triggers  the  following  set  of  activities.  Student  identification 
is  sent  to  the  learner  profile  database  and  the  details  are  integrated  with  the  student  query.  The  modified  query  is 
sent  to  the  inference  engine.  The  engine  attempts  to  retrieve  the  information  from  the  local  knowledge  database. 
If  the  context  related  information  is  not  available,  it  fetches  it  from  distributed  Internet  websites.  The  newly 
acquired  information  is  then  integrated  with  the  existing  information  and  converted  into  contextual  knowledge 
before  presenting  it  on  the  interface  to  the  student. 


Figure  2:  Student  query  and  knowledge  retrieval  process 
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7.  Conclusion 


Knowledge  management  systems  combined  with  htemet  based  learning  have  a lot  to  offer  to  the 
teaching  and  learning  community.  They  can  save  the  precious  time  for  both  teachers  and  students  through 
immediate  retrieval  of  knowledge  and  information.  This  paper  described  the  rationale  behind  the  development 
of  such  a system  and  provided  the  details  of  functionality  being  developed  in  a prototype. 

Even  if  the  course  material  is  structured  very  well,  there  is  still  a value  addition  to  a system  like  this  for 
a simple  reason,  that  in  distant  education,  students  learning  takes  place  in  different  points  in  time  and  such  a 
system  can  provide  right  kind  of  material  at  right  time,  while  still  taking  advantage  of  the  information  available 
on  the  Internet,  and  individualized  it  to  a particular  learner’s  needs.  As  the  Internet  grows,  the  strategic  impact 
of  knowledge  management  in  Internet  based  system  will  be  greatly  felt.  There  is  bright,  exciting  future  for  this 
area  of  research  and  knowledge  management  will  play  an  important  role  in  its  evolution  and  integration. 
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Abstract:  This  paper  presents  the  application  of  “Bee-gent”  mobile  agents  technology  in 
designing  web-based  learning  environments  that  provide  adaptivity  to  the  students  by  having 
two  separate  student  models:  Individual  student  model  for  each  student;  and  Group  student 
model  for  generalizing  the  attributes  of  a group  of  students.  This  combination  enables 
adaptation  in  web-based  learning  environments  at  different  granularity  and  in  different 
environments,  such  as  on-line  mode,  off-line  mode  or  unreliable  network  connection. 


Introduction 

In  recent  years,  educational  information  on  the  web  has  increased  exponentially,  and  web-based 
learning  is  currently  an  important  research  and  development  area.  An  effective  web-based  learning  environment 
should  interact  with  the  students  and  adapt  to  the  needs  of  individual  students.  Despite  previous  research  in  this 
area,  these  systems  have  not  yet  exploited  their  full  potential  in  facilitating  adaptivity  to  the  individual  students. 
This  paper  presents  the  application  of  “Bee-gent”  mobile  agents  technology  in  designing  web-based  learning 
environments  that  provide  adaptivity  to  the  students  by  having  two  separate  student  models:  Individual  student 
model  for  each  student;  and  Group  student  model  for  generalizing  the  attributes  of  a group  of  students. 


Design  of  Two-fold  Student  Modelling 

Kinshuk  et  al.  (1999)  pointed  out  that  dynamic  adaptation  requires  consideration  of  following  criteria: 

(a)  adaptation  with  respect  to  current  domain  competence  level  of  the  learner; 

(b)  suitability  with  respect  to  domain  content; 

(c)  adaptation  with  respect  to  the  context  in  which  the  information  is  being  presented. 

Based  on  the  above  criteria  and  the  nature  of  web-based  environment,  we  have  developed  a system, 

using  mobile  agents  technology.  The  system  uses  two  separate  student  models: 

1.  Individual  student  model,  which  serves  an  individual  student  and  contains  the  detailed  information  about 
particular  student’s  domain  competence  level,  preferences,  interaction  information  and  other  relevant 
details.  A partial  individual  student  model  resides  on  the  central  server  and  another  partial  individual 
student  model  on  the  user’s  machine  (local  server). 

2.  Group  student  model,  which  generalises  various  attributes  over  a number  of  students  and  attempts  to 
classify  students  in  various  categories  (for  various  attributes),  resides  on  the  host  base. 


Design  with  Mobile  Agent 

In  high-level  view,  a mobile  agent  interacts  with  client  side  inference  engine  to  pick  up  the  data,  which 
in  turn  relies  on  individual  student  model  at  the  client  side.  Then  mobile  agent  moves  to  host  (or  server)  side.  At 
host  side,  mobile  agent  performs  all  the  processes  needed,  such  as  updating  the  partial  individual  student  model 
based  on  summary  of  client  side  student  model  brought  by  mobile  agent,  and  interacting  with  group  student 
model  to  be  updated  if  required.  After  mobile  agent  finishes  all  the  tasks  at  the  host  side,  it  gathers  all 
information  it  needs,  and  returns  to  the  client  side.  Then  it  updates  the  client  side  individual  student  model.  In 
addition,  mobile  agent  approach  can  work  in  intermittent  connectivity  between  client  and  host  because  mobile 
agent  can  be  dispatched  and  it  can  work  autonomously  even  if  the  sender  is  not  available  any  more. 

The  Bee-gent  framework  httD://www2.toshiba.co.  ip/bee  gent/index. htm^i  is  used  to  implement  two 
types  of  mobile  agents  in  the  system  (figure  1):  agent  wrappers  and  mediation  agents. 

• Agent  Wrappers  are  used  to  agentify  existing  applications. 
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Mediation  Agents  support  inter-application  co-ordination  by  moving  from  the  site  of  an  application  to 
another  where  they  interact  with  remote  agent  wrappers. 


Scrrer 


Cbent 


Figure  1:  Architecture  of  adaptation  mechanism  using  Bee-gent 


Beneflts  of  the  Approach 

The  main  benefits  of  the  combination  of  mobile  agents  and  two -fold  student  modeling  are  as  follows: 

A)  Mobile  agents  can  be  used  to  pre-fetch  the  domain  content  that  will  be  requested  shortly  by  the  student. 
Thus  agents  technology  avoids  unnecessary  networking  delays,  cope  the  bandwidth  limitation  and  adapt  the 
representations  to  students,  based  on  the  student  performance. 

B)  The  distributed  nature  of  mobile  agents  allows  sharing  of  educational  resources  on  heterogeneous  systems 
and  platforms. 

C)  Mobile  agents’  migrate-and-disconnect  style  of  operation  has  lot  of  appeal  for  mobile  phones,  PDAs  and 
other  mobile  devices  that  are  now  commonly  used  by  the  on-the-road  learners. 


Conclusion 

In  conclusion,  the  mechanism  of  the  two -fold  student  modeling  and  using  mobile  agents  technology 
provides  an  attractive  alternative  to  implement  and  improve  web-based  learning  environments. 


References 

Kinshuk,  Oppermann  R.,  Patel  A.  & Kashihara  A.  (1999).  Multiple  Representation  Approach  in 
Multimedia  based  Intelligent  Educational  Systems.  Artificial  Intelligence  in  Education  (Eds.  S.  P.  Lajoie  & M. 
Vivet),  lOS  Press,  Amsterdam,  259-266. 


1040 


COFt  WAIIABK 


Computer- Assisted  Instruction  and  Multimedia  for  Students  with  Learning  Disabilities  at  the  Open  University  of  Israel 


Joel  Klemes,  Alit  Epstein  and  Tamar  llovitch 
The  Open  University  of  Israel,  Tel  Aviv,  Israel 
E-mail:  joelkI@oumail.openu.ac.il 


Abstract 

The  purpose  of  the  project  is  to  examine  whether  digitized  speech>text  synchronicity  combined  with  other  computerized  assistive  features 
supports  students  with  learning  disabilities  who  are  enrolled  in  distance -learning  programs  in  institutions  of  higher  education.  The  technology 
enables  students  to  follow  a text  that  is  presented  on  a computer  screen  as  they  listen  to  it  being  read  aloud  In  addition,  for  instance,  users  can 
navigate  within  the  text,  reread  and  listen  again  to  material  in  topics  that  are  difficult  to  understand  and  add  written  and  voice  notes.  The 
preliminary  results  of  a controlled  study  on  the  benefits  of  this  technology  for  a group  of  distance-learning  students  with  learning  disabilities  in 
the  Open  University  of  Israel  will  be  presented. 

Introduction 

Learning  disabilities  are  disturbances  in  the  learning  process  that  are  caused  by  deficits  in  the  functioning  of  the  central  nervous  system.  These 
disabilities  can  create  discrepancies  between  learners'  academic  achievement  and  their  intellectual  and  cognitive  capabilities  (Bigler,  Lajiness- 
O'Neill  & Howes,  1998).  During  the  second  half  of  the  twentieth  century,  particularly  the  1990s,  special  attention  was  devoted  to  research  and 
treatment  efforts  surrounding  learning  disabilities.  A body  of  evidence  accumulated  indicating  that  assistive  technologies,  primarily  in  the  areas 
of  computer  hardware  and  software,  can  help  students  overcome  various  kinds  and  degrees  of  dyslexia  and  attention  deficiencies  and  join  the 
mainstream  educational  system  (Raskind  & Higgins,  1998;  Lewis,  1998;  Lewis,  2000).  Indeed,  according  to  a report  published  in  1992,  the 
proportion  of  students  with  learning  disabilities  enrolled  in  institutions  of  higher  education  increased  more  than  the  proportion  of  students  with 
other  disabilities  (American  Council  on  Education,  1992). 

However,  research  in  this  area  has  been  limited  in  scope.  For  example,  most  investigations  have  been  conducted  on  students  in  kindergarten 
through  twelfth  grade,  as  opposed  to  adults  in  either  fice  to  face  or  distance-learning  environments.  Many  questions  remain  unanswered:  Which 
assistive  technology  should  be  used  with  each  kind  of  learning  disability?  Does  the  use  of  assistive  technologies  in  higher  education  affect  the 
number  of  students  who  complete  their  studies  and  earn  a degree,  particularly  in  distance-learning  programs?  Which  technology  best  suits  each 
field  of  knowledge? 

One  reason  that  such  issues  are  still  unsolved  is  that  the  assistive  technologies  used  by  students  with  learning  disabilities  are  produced  primarily 
by  commercial  companies  for  the  general  population.  In  countries  where  English  is  not  the  native  language,  such  as  Israel,  the  situation  is  even 
more  unclear,  since  most  of  the  commercial  programs  in  use  were  developed  for  English-speaking  students.  In  the  project  described  here,  we 
examine  whether  a learning  environment  that  provides  assistive  technologies — including  the  synchronization  of  digitized  speech  and  text  and 
additional  features,  might  enhance  learning  in  students  with  learning  disabilities  who  are  enrolled  in  a distance-learning  program  at  the  Open 
University  of  Israel  (OUI). 

Students  with  Learning  Disabilities  at  the  Open  University  of  Israel 

The  number  of  students  with  learning  disabilities  in  institutions  of  higher  education  is  smaller  than  the  number  of  people  with  learning  disabilities 
in  the  general  population.  For  example,  the  percentage  of  students  with  learning  disabilities  at  the  OUI  is  only  2%,  whereas  in  the  total  population 
it  is  4%.  Also,  we  might  assume  that  students  who  are  enrolled  in  a distance -learning  course  face  more  difficulties  than  students  with  learning 
disabilities  in  universities  where  instruction  is  mainly  face  to  face.  At  the  OUI,  the  total  number  of  students  with  learning  disabilities  who  are 
enrolled  each  semester  reaches  about  1,000,  and  the  predominant  syndromes  are  attention  deficit  and  attention  deficit  hyperactivity  disorders 
(ADD/H),  dyslexia,  and  dyscalculation.  The  OUI's  Center  for  Students  with  Learning  Disabilities  (CSLD)  provides  the  students  with  special 
examination  conditions,  individual  instruction  by  trained  tutors,  and  counseling.  However,  until  now,  the  OUI  has  not  used  any  assistive 
technologies  to  support  these  students  in  their  academic  studies. 

Description  of  the  Project 

We  developed  a CD-ROM  that  contains  speech  synchronization  for  the  text  of  a complete  chapter  of  a textbook  as  well  as  other  digitized  features 
that  are  intended  to  help  students  with  learning  disabilities.  The  CD-ROM  enables  the  user  to  carry  out  the  following  tasks: 

• Read  text  that  is  displayed  on  the  computer  screen  and  simultaneously  listen  to  it  being  read  aloud 

• Read  and  listen  again  to  a selfselected  audio  reading  of  paragraphs  that  were  difficult  to  understand  the  first  time 

• Navigate  bet  ween  topics  without  having  to  reread  or  to  listen  again  to  parts  of  the  text  that  are  irrelevant  to  the 
understanding  of  a specific  subject 

• Jump  to  a glossary  with  hyperlinks 

• Use  different  colors  to  highlight  paragraphs  that  are  essential  to  an  understanding  of  the  topic,  that  need  emphasizing,  or 
that  require  a repeat  reading 

• Write  notes  and  add  them  to  a text  while  reading  and  listening  to  it 

• Use  a microphone  to  add  voice  notes  to  a text  while  reading  and  listening  to  it 

• Compile  the  highlighted  paragraphs  and  the  notes  in  a separate  file  for  further  studying  support 

• Alter  the  way  in  which  text  is  displayed  on  the  screen;  for  example,  change  the  size  of  the  font  or  the  amount  of  space 
between  the  lines 

The  advantage  of  speech-text  synchronicity  for  school  students  with  learning  disabilities  has  already  been  described  in  the  literature  (Hebert  & 
Murdock,  1994;  Wise  & Olson,  1995).  The  other  digitized  elements  and  functions  were  implement^  upon  the  recommendation  of  specialists 
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OUl,  who  have  had  considerable  experience  tutoring  students  at  the  OUl  Center  for  Students  with  Learning  Disabilities.  We  used  the  Acrobat 
Writer®  software  to  create  a learning  environment  that  combines  digitized  speech  and  text  and  to  provide  the  functionality  described  above.  This 
developer  software  was  convenient  for  our  purposes  because,  unlike  other  software  that  has  been  developed  to  present  electronic  text,  it  enabled 
us  to  combine  Hebrew  text  with  digitized  speech  in  the  learning  environment  and  to  include  features  that  are  not  present  in  the  standard,  widely 
used  Acrobat  Reader®  and  other  software.  To  gain  access  to  all  the  desired  functions  in  the  CDROM,  the  students  installed  the  same  software  on 
their  home  computers. 

The  Benefits  of  the  Project 

The  potential  advantages  of  speech-text  synchronicity  have  been  investigated  with  dyslexic  college  students  and  others  (Hebert  & Murdock, 

1994;  Elkind,  Black  & Murray,  1996).  The  results  of  the  college  students'  research  showed  that  the  assistive  technology  improved  the  reading  rate 
and  the  understanding  of  most  of  the  students  who  participated  in  the  study.  However,  other  studies  suggest  that  speech-text  synchronicity  is 
beneficial  only  for  students  with  severe  disabilities,  whereas  for  students  with  mild  disorders  the  technology  actually  presents  a disturbance 
(Raskind  & Higgins,  1998).  To  the  best  of  our  knowledge,  the  influence  of  an  assistive  technology  that  includes  all  the  computerized  features 
described  here  has  not  been  explored  with  adult  distance-learning  students.  This  pilot  project  will  enable  us  to  ascertain  the  kind  of  disorder  that 
this  technology  best  supports  in  a large  group  of  students  with  various  learning  disabilities. 

The  Test  Group  and  the  Research 

Each  semester,  between  200  and  300  students  enroll  in  an  introductory  psychology  course  offered  through  distance -learning  at  the  OUl,  and  of 
them,  10  to  15  usually  request  support  from  the  CSLD.  These  students  receive  all  of  the  course  materials  in  print  and  have  to  fulfill  the  same 
requirements  as  their  classmates.  For  our  project,  we  have  provided  the  students  who  requested  aid  from  the  CSLD  with  a digitized,  speech - 
synchronized  version  of  one  chapter  of  the  course's  print  textbook  (a  chapter  on  memory  that  is  50  pages  long). 

Through  personal  interviews,  focus  groups,  and  questionnaires,  we  are  looking  at  the  following  issues: 

• Does  the  synchronicity  of  digitized  speech  and  text  support  the  learning  process  of  distance -learning  students  with  learning 
disabilities  and  in  what  way? 

• Did  the  technology  help  the  students  complete  the  assignments  related  to  the  chapter  in  question? 

• Is  there  a difference  between  the  students'  grades  on  the  assignments  for  the  chapter  in  question  and  their  grades  on  the 
assignments  for  the  other  chapters? 

• Did  the  digitized  speech-text  synchronicity  affect  the  time  that  the  students  invested  in  learning? 

• Which  of  the  digitized  features  of  the  CDROM  that  are  described  above  helped  the  most  in  the  learning  process? 

• Did  the  speech -text  synchronicity,  along  with  the  additional  features,  affect  the  students'  need  for  a tutor? 

The  control  group  consists  of  students  with  learning  disabilities  who  were  assigned  the  same  chapter  in  the  same  course  during  a different 
semester;  these  students  were  provided  with  the  print  version  of  the  text  without  the  digitized  technology.  The  two  groups  of  students'  attitudes, 
grades,  and  achievements  related  to  the  chapter  will  be  compared,  as  well  as  their  success  at  answering  the  questions  related  to  the  chapter  on 
the  final  exam. 

The  research  is  being  conducted  over  two  semesters.  At  the  conference  we  will  demonstrate  an  English  version  of  the  CD-ROM  with  the 
assistive  technology  and  present  the  preliminary  results  regarding  the  effects  of  this  approach  for  distance-learning  students  with  learning 
disabilities. 
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Abstract:  At  the  center  of  hypertext  is  the  link,  an  active  connection  from  part  of  one  document  to  another 
document  (or  part  of  that  document).  In  the  early  days  of  the  World  Wide  Web,  it  was  easy  for  readers  to  identify 
the  links  on  the  page:  the  unvisited  links  appeared  in  blue  and  underlined,  the  visited  links  appeared  in  red  or 
purple  and  underlined.  With  the  advent  of  advanced  formatting  techniques,  such  as  cascading  style  sheets,  links 
appear  in  different  ways  on  different  pages.  These  changes  may  make  pages  appear  more  appealing,  but  they  also 
make  it  harder  for  readers  to  engage  in  a key  hypertext  reading  activity,  identifying  and  selecting  the  most  relevant 
links.  At  the  same  time,  even  easily  identifiable  links  provide  readers  with  very  little  accompanying  information 
as  to  why  the  link  is  there  and  where  it  leads. 

In  this  paper,  we  describe  ongoing  research  in  providing  readers  with  additional  access  to  links  and 
information  about  those  links.  We  revisit  in  more  detail  the  need  to  provide  links  in  a consistent  and  clear  manner, 
consider  ways  to  make  links  more  useful  to  readers  without  affecting  the  design  of  a page,  suggest  techniques  for 
adding  information  to  links,  and  describe  a tool  we  have  built  that  permits  readers  to  obtain  a summary  of  the  links 
available  on  each  page  they  visit. 


1 Introduction 

Where  can  I go  from  here?  That  question  forms  the  center  of  hypertextual  reading.  Unlike  in  traditional  linear 
writing,  in  which  readers  simply  follow  the  narrative  from  beginning  to  end,  in  hypertext  readers  regularly  look 
for  links  and  consider  whether  or  not  to  follow  them.  To  permit  readers  to  select  the  links  that  are  most 
applicable,  a hypertext  system  should  provide  four  key  attributes  for  links,  summarized  in  Table  1 . 


Attribute 

Summary 

Identifiability 

The  links  on  a page  must  be  easily  identifiable. 

Link  Text 

The  source  of  the  link  must  be  obvious  to  readers. 

Link  Type 

The  purpose  of  the  link  should  be  clear. 

Link  Destination 

The  destination  of  the  each  link  should  be  easy  to  determine. 

Table  1:  Key  Attributes  of  Hypertext  Links 

1.  Identifiability:  The  links  on  a page  must  be  easily  identifiable.  That  is,  readers  should  be  able  to 
quickly  tell  what  on  a page  is  a link  and  what  on  a page  is  not  a link.  If  readers  cannot  easily 
identify  the  links  on  the  page,  they  will  not  follow  the  links.  While  some  experimental 
hyperfictions  may  make  the  problem  of  finding  the  links  part  of  the  reading  experience,  most 
hypertexts  should  reveal,  rather  than  obscure  the  links. 

2.  Link  Text:  The  source  of  the  link  should  be  obvious.  Most  links  do  not  go  from  page  to  page. 
Rather,  they  go  from  part  of  a page  to  another  page  (or  part  of  another  page).  In  a page  with  many 
links,  readers  should  be  able  to  clearly  identify  which  part  of  the  page  the  link  is  associated  with. 
Typically,  the  source  is  a word,  phrase,  or  image  (the  “anchor  text”  in  HTML). 

3.  Link  Type:  The  purpose  of  each  link  should  be  clear.  A reader  should  know  why  the  link  is 
there.  Does  it  provide  a definition  of  the  linked  text?  Some  examples?  A related  page?  A counter- 
argument to  an  argument  on  the  page?  The  next  page  in  a sequence?  Without  some  additional 
information,  readers  are  unlikely  to  know  what  the  link  is  there  for.  It  is  this  aspect  of  links  that  is 
perhaps  least  well  supported  on  the  Web. 

4.  Link  Destination:  The  destination  of  each  link  should  be  easy  to  determine.  Where  does  the 
link  go?  Is  it  elsewhere  on  the  page,  on  the  same  site,  or  on  another  site?  What  kind  of  page  does  it 
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link  to?  While  many  browsers  will  put  a URL  in  the  status  bar,  a URL  only  says  a little  about  the 
destination. 

These  attributes  are  particularly  desirable  for  educational  hypertexts,  since  we  put  links  in  pages  so  that 
students  can  explore  further  as  their  desires  or  needs  for  further  information  guide  them.  By  making  links  easily 
identifiable,  we  let  students  find  information  more  quickly.  By  telling  them  why  we  put  the  link  there,  we  help 
them  decide  whether  it’s  a kind  of  link  they  need  to  follow.  And,  by  telling  them  what  kind  of  information  they 
will  receive,  we  help  them  decide  whether  or  not  that  information  is  appropriate  before  they  spend  their  time 
following  and  exploring  a link.  Note  that  the  last  two  goals,  while  similar,  are  different.  For  example,  we  might 
provide  a link  for  students  who  do  not  understand  a short  reading  and  need  more  information  (that  is,  the 
purpose  of  the  link).  But  a “need  more  information”  link  might  lead  to  another  reading,  to  a problem  set,  or 
even  to  a set  of  dictionary  entries.  By  providing  both  kinds  of  information,  we  further  guide  the  student. 

Unfortunately,  as  we  suggest  in  Section  2,  Web  pages  are  often  incomplete  in  their  support  of  these  four 
key  aspects.  Designers  are  making  links  harder  to  identify.  Page  authors  too  often  use  “Click  here”  as  their 
source  text.  Browsers  provide  little  support  for  link  types.  And  the  primary  information  most  readers  get  on  the 
destination  of  a link  is  little  more  than  the  URL  for  the  link.  Even  if  the  designer  of  an  educational  hypertext  is 
careful  to  clarify  links,  choose  good  link  text,  type  links,  and  provide  information  on  destinations,  students  may 
not  be  able  to  access  all  this  information  and,  in  any  case,  lose  these  benefits  as  soon  as  they  leave  the  site. 

So,  what  can  be  done?  In  this  paper,  we  propose  a partial  solution.  We  have  developed  a prototype  system 
that  provides  a link  summary  at  the  end  of  each  page  that  readers  view,  no  matter  where  on  the  Web  those 
pages  are  located.  For  each  link  on  the  page,  the  summary  includes  the  source  text,  the  full  URL  of  the 
destination,  the  link  type  (if  available),  and  additional  information  about  the  link  or  destination  (if  available).  In 
the  near  future,  we  expect  to  add  summary  information  about  the  destination  page  (its  title,  its  size,  etc.). 
Because  the  links  appear  at  the  end  of  the  page,  they  are  easy  to  identify.  In  pages  that  provide  the  appropriate 
accompanying  information,  the  type  of  the  link  and  basic  information  on  the  destination  are  available.  Hence, 
the  summary  helps  meet  most  of  the  criteria  required  for  the  links  without  affecting  the  design  of  the  page. 

In  Section  2 of  this  paper,  we  revisit  the  current  status  of  links  on  the  World  Wide  Web  and  describe  some 
deficiencies  of  current  practices.  In  Section  3,  we  discuss  reasons  to  provide  link  summaries  on  pages.  In 
Section  4,  we  suggest  key  aspects  that  any  link  summary  system  should  provide.  In  Section  5,  we  describe  the 
architecture  of  our  link  summary  system.  In  Section  6,  we  revisit  the  need  to  summarize  and  clarify  links. 
Finally,  in  Section  7,  we  describe  planned  updates  to  our  system. 


2 Links  and  the  World  Wide  Web 

The  World  Wide  Web  (Berners-Lee  et  al.  1994)  is  the  leading  hypertext  system.  How  good  is  its  support  for 
links?  Unfortunately,  the  answer  is  “it  depends”,  particularly  on  the  page  designer.  In  the  early  days  of  the  Web, 
readers  found  it  easy  to  identify  links:  Blue  underlined  text  represented  unvisited  links,  red  or  purple  underlined 
text  represented  recently  visited  links.  However,  as  the  Web  has  evolved  to  provide  authors  with  more  control 
over  the  appearance  of  the  page,  each  page  designer  has  chosen  new  ways  to  show  links.  Readers  can  no  longer 
easily  identify  the  links  on  many  pages  by  simply  scanning  for  blue,  red,  and  purple  underlined  texts. 

Jakob  Nielsen,  one  of  the  leaders  in  Web  design,  has  emphasized  the  need  for  easily  identifiable  links  in  a 
number  of  his  Alertbox  columns.  He  first  warned  of  the  problem  in  1996. 

Links  to  pages  that  have  not  been  seen  by  the  user  are  blue;  links  to  previously  seen  pages  are 
purple  or  red.  Don't  mess  with  these  colors  since  the  ability  to  understand  what  links  have 
been  followed  is  one  of  the  few  navigational  aides  that  is  standard  in  most  web  browsers. 
Consistency  is  key  to  teaching  users  what  the  link  colors  mean.  (Nielsen  1996,  Item  8). 

Nielsen  returned  to  this  issue  in  a 1999  column.  As  new  designs  were  becoming  standard  for  Web  sites,  he 
repeated  his  warning.  He  also  noted  the  importance  of  the  underline  as  a way  of  identifying  links. 

[Non-standard  link  colors  continue]  to  be  a problem  since  users  rely  on  the  link  colors  to 
understand  what  parts  of  the  site  they  have  visited.  I often  see  users  bounce  repeatedly  among 
a small  set  of  pages,  not  knowing  that  they  are  going  back  to  the  same  page  again  and  again. 

(Also,  because  non-standard  link  colors  are  unpleasantly  frequent,  users  are  now  getting 
confused  by  any  underlining  of  text  that  is  not  a link.)  (Nielsen  1 999) 

Designers  can,  of  course,  continue  to  use  these  standards,  but  fewer  and  fewer  seem  to.  Readers  must  now 
spend  more  effort  determining  whether  a word  or  phrase  on  a page  serves  as  a link,  often  by  considering  the 
placement  of  a word  or  by  waving  the  mouse  over  the  word  to  see  if  anything  happens. 
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So,  the  clarity  and  text  of  links  were  originally  quite  good  on  the  Web,  but  have  been  weakened  by  the 
advent  of  careless  design.  What  about  the  two  other  desired  characteristics  of  links:  their  types  and  their 
destinations?  Once  again,  the  answer  is  “it  depends”.  In  part,  it  depends  on  the  page  author.  In  particular,  the 
current  HTML  specification  (Raggett  et  al.,  1999)  provides  at  least  four  attributes  for  every  links:  title,  rel 
(relationship),  rev  (reverse  relationship,  from  linked  page  to  current  page),  and  class.  The  title  is  a generic 
form  of  additional  information  that  may  be  displayed  by  the  browser.  It  can  provide  information  about  the 
destination.  The  relation  and  reverse  relation  tags  allow  authors  to  specify  a link  type.  Unfortunately,  the 
standard  link  types  (Raggett  et  al.  1999,  Section  6.12)  are  relatively  basic,  reflecting  simple  relationships  like 
“table  of  contents”  or  “next  element  of  a sequence”  rather  than  more  complex  elements  like  those  suggested  in 
(Trigg  1983),  such  as  “abstraction”,  “refutation”,  “continuation”,  “data”.  Particularly  for  educational  texts, 
which  will  need  types  like  “example”,  “exercises”,  “simplification”,  the  basic  set  does  not  suffice.  Finally,  the 
class  attribute  exists  primarily  for  formatting,  but  could  also  be  used  to  indicate  type. 

Even  more  importantly,  support  for  types  and  destinations  depends  not  only  on  author,  but  also  on 
browsers.  For  example,  not  all  browsers  support  the  important  title  attribute  (e.g.,  Netscape  4.7  does  not)  and 
those  that  do  often  require  readers  to  perform  specific  actions  (e.g.,  pause  the  cursor  for  at  least  a few  seconds) 
to  obtain  the  title  text.  Few  browsers  seem  to  do  anything  useful  with  rel  and  rev  attributes.  And,  while, 
many  browsers  support  the  class  attribute,  they  do  so  for  formatting  rather  than  logical  typing. 

So,  what  should  page  authors  do?  They  can  eschew  complicated  site  designs  that  obscure  the  links.  They 
can  choose  careful  texts  for  their  links.  They  can  use  the  title,  rel,  and  rev  attributes.  But  they  still  need  to 
hope  that  browsers  will  support  all  that  they  do  and  that  their  readers  will  choose  to  use  those  browsers.  Is  that 
enough?  We  think  not.  By  providing  link  summaries  for  all  pages,  we  can  support  careful  authors  without 
affecting  their  careful  designs. 


3 Why  Summarize  Links? 

In  the  introduction,  we  suggested  that  a link  summary  system  provides  one  key  characteristic  that  one  hopes  for 
in  a hypertext  system:  it  makes  the  links  clear  (they  all  appear  the  same  way  on  every  page  and  in  the  same 
place  on  every  page).  Since  many  pages  obscure  their  links,  we  hope  that  provides  reason  enough  for  link 
summaries.  However,  the  link  summary  system  provides  additional  characteristics  that  readers  benefit  from:  it 
automatically  provides  information  about  the  destination  of  the  link  and,  if  the  page  author  has  included  that 
information,  can  include  the  link  type  and  other  comments. 

But  there  are  still  other  reasons  to  summarize  links  at  the  end  of  a page.  One  of  the  benefits  we  first 
observed  when  using  the  system  is  that  it  aids  readers  in  dealing  with  mistyped  links  in  Web  pages.  Since  the 
URL  appears  in  the  summary,  it  is  possible  to  look  at  the  URL  and  identify  potentially  incorrect  parts  of  the 
URL.  While  this  is  a minor  benefit,  it  is  one  that  many  users  have  found  useful. 

More  importantly,  a link  summary  system  can  permit  readers  to  organize  links.  For  example,  on  a page  that 
presents  an  argument,  one  might  want  to  follow  all  the  links  to  counter-arguments.  By  arranging  the  link 
summary  by  link  type,  readers  can  quickly  find  all  the  counter-arguments.  Similarly,  a student  reading  a long 
reference  piece  might  want  to  be  able  to  quickly  identify  all  the  examples.  Both  cases  require  that  authors 
classify  their  links,  but  we  hope  that  as  the  types  of  links  are  made  available  through  tools  like  ours,  authors 
will  be  more  inclined  to  classify  their  links.  A reader  might  also  arrange  links  by  dbstination  (e.g.,  to  see 
whether  there  is  a site  or  page  that  is  linked  to  particularly  often). 

Should  these  summarized  links  replace  the  links  on  the  page?  Certainly  not.  Many  links  are  best  understood 
within  a broader  context.  The  link  summary  system  simply  provides  a way  for  readers  to  get  additional 
information  on  the  links  available  in  the  page. 
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4 Design  of  a Link  Summary  System 


Given  the  benefits  derived  from  summarizing  the  links  in  a page  at  the  end  of  a page,  one  would  hope  to  see  a 
number  of  systems  designed  to  provide  such  summaries.  In  considering  such  systems,  we  should  look  for 
systems  that  meet  a number  of  key  goals  that  are  summarized  in  Table  2. 


Goal 

Summary 

Universality 

Works  with  any  page  on  the  Web. 

Browser  Independence 

Works  with  any  reasonable  browser. 

Design  Preservation 

Maintains  the  original  design  of  the  page  as  much  as  possible. 

Author  Support 

Takes  advantage  of  information  the  author  provides. 

Customizability 

Allows  readers  to  set  appropriate  preferences. 

Table  2:  Primaiy  Goals  of  a Link  Summary  System 

1.  Universality.  The  link  summary  system  should  allow  readers  to  obtain  summaries  of  any  page  on  the 
World  Wide  Web  without  requiring  modifications  to  that  page.  While  we  could  hope  that  authors  would 
provide  their  own  summaries  or  use  server-side  software  to  provide  summaries,  not  all  authors  will  do  so.  A 
client-side  system  that  works  with  any  reasonable  Web  page  provides  a more  universal  system.  Since  a number 
of  pages  do  not  meet  HTML  standards,  the  system  should  do  its  best  even  with  incorrect  HTML. 

2.  Browser  Independence.  The  link  summary  system  should  work  with  any  browser,  past,  present,  or 
future.  Since  different  readers  clearly  prefer  to  use  different  browsers,  the  system  should  support  as  many 
browsers  as  possible.  This  requirement  suggests  that  a browser  plugin  is  an  inappropriate  implementation,  since 
each  browser  will  require  a different  kind  of  plugin. 

3.  Design  Preservation.  While  many  of  the  problems  described  earlier  are  caused  by  designers  who  ignore 
standards  for  link  appearance,  it  is  also  true  that  many  good  designs  may  require  links  to  appear  differently. 
Hence,  the  link  summary  system  should  not  modify  the  original  page,  except  to  add  the  links  at  the  end  (or  in  a 
separate  window). 

4.  Author  Support  Qven  that  the  HTML  standard  provides  a number  of  attributes  for  tags  that  can 
provide  link  type  and  destination  information,  the  system  should  support  those  standards.  Hence,  it  should  take 
link  types  from  the  rel,  rev,  and  class  attributes  and  additional  information  from  the  title  attribute. 

5.  Customizability.  As  the  previous  section  suggests,  different  readers  will  wish  to  organize  their  links  in 
different  ways.  The  system  should  give  readers  appropriate  control  over  the  link  summaries.  Aspects  of 
summaries  that  readers  may  wish  to  control  include:  the  order  of  links  within  the  summary  (do  they  appear  in 
the  same  order  as  on  the  page,  or  ordered  by  some  other  characteristics),  the  information  associated  with  each 
link  (just  the  source  text  and  URL,  or  with  additional  information),  and  even  how  they  appear  (font,  etc.). 


5 Technical  Details 

We  have  developed  a prototype  system  that  supports  the  design  goals  suggested  in  the  previous  section.  To 
provide  a relatively  universal  system  that  is  independent  of  browser,  we  rely  on  the  Web  Raveler  architecture 
(Kensler  and  Rebelsky  2000).  Web  Raveler  is  a collection  of  systems  that  mediate  the  conversation  between 
browsers  and  servers.  The  classic  implementation  of  Web  Raveler  is  as  a proxy  server.  Proxy  servers  receive 
each  request  and  response  that  is  sent  between  client  and  server.  Proxy  servers  are  used  for  a variety  of  reasons, 
including  caching  of  pages,  limiting  access  to  certain  pages,  and  logging  information  about  Web  use. 

Web  Raveler  provides  an  infrastructure  in  which  authors  can  write  small  “plugins”  (to  Web  Raveler)  that 
receive  each  page  before  it  goes  to  the  browser  and  may  modify  the  page  as  they  choose.  The  Link  Summary 
plugin  in  our  prototype  system  quickly  scans  the  page  for  links,  extracts  information,  and  adds  a table  of  links  at 
the  end  of  the  page. 

Web  Raveler  also  provides  additional  facilities  useful  for  the  link  summary  system.  In  particular,  it 
provides  account  information  which  permits  us  to  store  and  access  user  preferences  on  the  appearance  of  the 
link  summary.  In  addition,  the  Web  Raveler  team  is  currently  developing  a Web  cache  that  will  make  it  easy  for 
us  to  quickly  obtain  the  titles  associated  with  many  URLs.  While  it  would  possible  to  obtain  the  title  of  a Web 
page  by  sending  an  HTTP  request,  if  there  are  many  links  on  a page,  it  would  be  overly  time  consuming  to  send 
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an  HTTP  request  for  each  of  them.  The  Web  Raveler  cache  would  provide  quicker  access  and  user  preferences 
could  determine  what  the  system  does  if  the  URL  is  not  in  the  cache. 

Our  prototype  system  does  a fairly  simple  search  for  links  on  a page.  In  particular,  it  looks  for  all  the 
anchor  (A)  tags  in  the  page.  While  this  strategy  works  for  most  pages,  it  does  not  work  for  pages  in  which  the 
links  are  generated  on  the  client  by  Javascript  or  other  scripting  language.  We  are  currently  considering 
mechanisms  for  extracting  such  links  and  debating  the  tradeoffs  (for  example,  interpreting  a script  will  take 
additional  time,  making  the  system  slower). 


6 Conclusions 

While  links  make  hypertext  hypertext,  they  are  increasingly  difficult  to  identify  in  many  modem  Web  designs. 
In  addition,  the  HTML  standards  that  permit  richer  links  are  insufficiently  supported  by  browsers.  In  this  paper, 
we  have  suggested  that  a relatively  simple  mechanism,  a link  summary  automatically  generated  for  each  page 
viewed,  can  increase  the  usability  of  modem  Web  pages  and  better  support  both  readers  and  authors. 


7 Future  Work 

As  we  suggest  above,  we  hope  to  extend  the  system  for  provide  additional  information  about  the  destination  of 
each  link.  In  its  simplest  form,  this  additional  information  can  include  the  title  of  the  page  and  the  size  of  the 
destination  page.  However,  it  might  also  be  useful  to  provide  additional  information  about  the  destination,  such 
as  the  author  (if  available)  and  even  a summary. 

Our  prototype  system  was  built  as  a basic  proof  of  concept.  In  the  near  future  we  plan  to  begin  some  more 
careful  user  testing  to  see  whether  readers  take  advantage  of  link  summaries  and  whether  there  are  ways  to 
make  them  more  useful.  We  also  hope  to  investigate  what  aspects  of  link  summaries  readers  want  to  customize. 

Because  our  link  summary  system  appears  to  modify  pages,  we  are  also  working  to  consider  the  intellectual 
property  ramifications  of  the  system.  We  are  hoping  to  build  on  the  work  of  the  Web  Raveler  team  in  this 
direction. 
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Abstract:  This  brief  paper  will  report  the  process  and  results  of  a website  for  multicultural 
education  being  developed  by  international  graduate  students  from  MU.  The  resources  on  this 
web  site  have  important  implications  for  educational  technologists  who  wish  to  make 
instruction  more  responsive  and  effective  for  learners  with  diverse  backgrounds.  The 
web  site  includes  four  components:  a discussion  forum,  opportunities  to  meet  Korean 
children  and  read  their  stories,  literacy  lesson  plans  using  tri-lingual  multimedia  stories, 
and  research  papers. 

Multicultural  Education  and  the  Internet 

As  the  United  States  continues  to  become  more  ethnically  diverse,  the  composition  of  the  U.S. 
classroom  is  changing.  Teachers  are  encountering  students  from  other  cultures  whose  worldviews,  motivational 
levels,  and  learning  styles  are  substantially  different  from  those  of  most  teachers  and  native-born  students. 
Teachers  should  be  prepared  to  meet  the  educational  needs  of  ethnically  and  culturally  diverse  students  (Gay, 
1997).  Experts  in  the  field  of  multicultural  education  recommend  specific  competencies  that  will  empower 
teachers  to  create  a learning  environment  that  encourages  and  supports  the  academic  achievement  of  students 
from  diverse  racial  and  ethnic  groups.  In  addition  to  developing  positive  attitudes  and  behaviors  toward 
culturally  different  children,  teachers  are  expected  to  be  able  to: 

■ Understand  the  learner  characteristics  that  children  from  different  cultural  backgrounds  bring  to  the 
teaching/leaming  situation  which  may  affect  the  quality  of  learning;  and 

■ Create,  select,  and  use  appropriate  instructional  strategies,  pedagogical  techniques,  and  materials  to 
accommodate  the  learner  characteristics  (Sheffield,  1997). 

As  educational  technologists,  we  believe  that  these  expectations  can  be  reached  through  the  application 
of  educational  technology.  The  Internet  can  be  an  ideal  technological  tool  for  multicultural  education.  In 
addition  to  the  enormous  amount  of  information  available  on  the  Internet,  the  Internet  has  the  potential  to 
network  and  interact  with  people  (Gorski,  1996).  The  Internet  can  support  several  educational  ideals  consistent 
with  multicultural  education:  Inclusive  education.  Collaborative  education,  and  Interactive  education  (Gorski, 
1996) 

Designing  a Website  for  Multicultural  Education 

The  web  site  being  developed  is  an  example  that  models  multicultural  ideals.  It  is  a sample  of 
interactive,  inclusive,  and  collaborative  media  for  multicultural  education.  The  web  site  consist  of  four  parts: 
discussion  forum,  meet  the  children,  literacy  lesson  plans,  and  research  reviews. 

Discussion  Forum 

This  web  site  offers  a discussion  forum,  for  students  to  email  teachers  or  other  students  to  ask  for  more 
information  or  clarification  on  a particular  topic.  A vast  of  network  of  teachers  and  students  around  the  world 
and  across  myriad  cultures  could  share  questions  and  exchange  resources  on  a given  topic.  According  to 
Gamsey  and  Garton  (1992),  the  discussion  forum,  as  a form  of  CMC  (Corrputer-Mediated  Communication), 
solves  the  constraints  of  time  and  space  to  those  geographically  dispersed  organizations  that  are  looking 
forward  to  communicating  with  each  other.  It  also  allows  teachers  to  share  resources  and  discuss  their  teaching 
successes  and  failure,  which  is  an  excellent  model  of  collaborative  media  for  multicultural  education.  This  is  a 
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sample  of  the  interactive  learning  environment  made  possible  by  the  Web.  Another  use  of  the  discussion  forum 
is  to  provide  an  authentic  use  of  language  through  for  communication.  Some  educational  theorists  suggest  that 
authentic  tasks  are  very  important  for  students,  as  they  enable  students  to  learn  meaningfully  and  purposefully 
(Brown,  Collins  & Duguid,  1989). 

Meet  the  Children 

Inclusivity  is  another  important  notion  in  multicultural  curriculum  transformation  (Gorski,  1996).  If 
education  is  truly  to  be  student-centered,  then  the  experiences,  perspectives,  and  ideas  of  the  students 
themselves  must  be  brought  to  the  fore.  To  be  “included”  in  the  educational  process,  students  must  be  given 
opportunities  to  have  their  voices  heard  and  to  hear  each  other’s  voices,  which  are  prerequisite  to  student- 
student  interaction  and  cooperative  learning.  The  Internet  can  facilitate  this  kind  of  Inclusivity  by  encouraging 
teachers  and  students  to  participate  in  virtual  global  communities  with  other  teachers  and  students  who  may  be 
either  very  culturally  similar  or  different.  This  web  site  offers  the  opportunity  to  meet  Korean  children  who 
want  to  share  their  experience  and  writing  with  other  students  around  the  world.  Writing  samples  by  the 
children  are  included. 

Literacy  Lesson  Plans 

The  web  site  describes  an  innovative  multimedia  story-writing  project  with  elementary  children  in  an 
ESL  classroom.  The  children  wrote  and  illustrated  multicultural  stories  and  then  worked  with  international 
graduate  students  to  transform  the  stories  into  interactive  multimedia  stories  in  three  languages.  A sample  story 
is  included  and  a variety  of  lesson  plans  for  literacy  development  in  ESL  are  provided. 

Research  Reviews 

The  web  has  the  potential  to  greatly  expand  the  breadth  and  depth  of  collaborative  possibilities  of 
educators  (Gorski,  1996).  Five  research  papers  are  being  provided:  multicultural  education,  globalization, 
literacy  development,  technology-writing  and  publishing  on  the  web,  and  TESOL.  In  this  web  site,  research 
findings  will  be  compiled  to  share  resources  on  multicultural  education  and  tips  for  teachers  on  how  to  make  a 
learning  environment  effective  based  on  research. 

Importance  of  the  Work 

The  resources  on  this  web  site  have  important  implications  for  educational  technologists  who  wish  to 
make  instruction  more  responsive  and  effective  for  learners  with  diverse  backgrounds.  The  web  site  should  have 
positive  implications  for  training  pre -service  and  in-service  teachers  in  multiculturalism  and  provide  a resource 
site  for  teachers  and  children. 
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Abstract:  In  this  p^er  we  describe  how  learners  need  awareness  information  about  co-leamers  in 
computer-supported  collaborative  learning  systems.  Awareness  is  essential  for  working 
collaboratively  and  for  building  a learning  community.  If  the  learner  doesn’t  know  anything  about 
his/hers  co-leamers,  there  can’t  be  any  community,  which  means  that  there  can’t  be  any  real 
collaboration  either.  In  this  paper  we  first  describe  a theoretical  background  for  our  research  after 
which  we  introduce  one  possible  solution  to  solve  the  demands  for  awareness  in  a Web-based 
conferencing  systems.  We  also  present  our  initial  implementation  of  awareness  support  and  report 
experiences  from  a course  using  the  system. 


Introduction 

In  many  cases  Web-based  learning  environments  are  used  asynchronously,  which  obviously  gives  freedom  of  time 
and  place  for  learners.  But  when  a learner  logs  into  the  system,  it  is  very  hard  for  him/her  to  know  if  there  is 
someone  else  present  at  the  same  time  or  even  ever  have  been.  Most  of  these  asynchronously  used  Web-based 
learning  environments  include  some  kind  of  discussion  forum.  In  these  discussions  a learner  can  find  co-leamers 
who  have  made  comments  and  make  conclusions  that  he/she  is  not  alone,  but  especially  co-leamers  who  are  only 
reading  comments  are  quite  invisible.  Thus  learners  come  visible  only  through  their  contributions,  which  can  lead  to 
wrong  conclusions  about  the  state  of  the  discussion.  Ongoing  discussion  might  even  look  like  a dead  discussion 
because  learners  are  only  reading.  That  is  why  it  is  reasonable  to  give  information  about  their  visits  to  other  learners. 
This  information  can  enhance  learner’s  awareness  of  co-leamers  and  their  interactions,  which  are  essential  especially 
for  collaborative  learning  (Dillenbourg  1999).  It  even  can  be  argued  that  otherwise  there  will  not  be  any  chances  to 
collaborate.  This  awareness  can  inform  learner  that  co-leamers  are  interested  in  each  other’s  contributions  and  can  be 
in  important  role  in  community  building.  A learner  must  receive  information,  which  could  include  who  the  other 
learners  are,  what  they  have  done  and  what  they  are  doing  now.  If  learners  are  aware  of  who  are  the  ones  to 
collaborate  with  then  it  is  possible  to  start  building  a community  for  learning.  In  this  paper  we  consider  how  the 
technology  can  support  collaborative  learning  by  providing  awareness  information  (see  also  Jermann  et  al.  2001), 

First  in  this  paper  we  describe  a theoretical  background  for  the  research  and  after  that  we  introduce  one 
possible  solution  to  solve  these  demands  for  awareness  in  Web-based  conferencing  systems.  We  present  our 
implementation  and  report  experiences  from  our  experiment.  Finally  we  make  conclusions  of  the  work  we  have 
done. 

Collaborative  learning  and  community  building 

Learners’  work  and  learning  in  online  learning  environments  can  be  seen  at  its  best  as  collaborative  learning. 
Learners  are  working  together  and  building  new  knowledge  by  reflecting  their  thoughts  with  others.  However, 
collaborative  learning  is  a difficult  concept  in  practice.  According  to  Dillenbourg  (1999),  the  words  ‘collaborative 
learning’  describe  a situation  where  particular  forms  of  interaction  among  people  are  expected  to  occur.  This 
interaction  can  trigger  specific  learning  mechanisms  and  finally  lead  the  participants  to  learn. 

Dillenbourg  (1999)  classifies  four  categories  in  which  there  are  different  ways  to  increase  the  probability  that 
some  type  of  collaborative  interactions  will  occur.  These  categories  are: 
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• to  set  up  initial  conditions, 

• to  over-specify  the  ’collaboration’  contract  with  a scenario  based  on  roles, 

• to  scaffold  productive  interactions  by  encompassing  interaction  rules  in  the  medium,  and 

• to  monitor  and  regulate  the  interactions. 

With  reference  to  this  categorization  our  work  emphasizes  the  fourth  category.  Also,  as  an  alternative  to  the 
tools  for  the  teacher  to  monitor  the  learners,  we  provide  tools  for  the  learners  themselves  for  self-regulation  of  their 
interactions.  These  tools  offer  information  about  community’s  members  and  their  interactions,  which  can  be  used  by 
the  learner  to  maintain  the  awareness  of  the  state  of  the  community.  In  collaborative  processes  this  information  about 
other  learners  could  really  help.  Especially  in  asynchronous  learning  environments  this  kind  of  awareness 
information  would  be  helpful  for  interacting  with  other  learners. 

Because  learners  are,  in  most  cases,  inside  the  environment  at  different  times  this  awareness  information  would 
help  to  keep  discussions  alive.  A learner  can  check  from  awareness  information  if  other  learners  have  been  inside  the 
system  and  read  their  comments.  In  this  way  a learner  can  have  more  clearly  expectations  when  the  discussion  is 
going  to  continue. 

For  working  collaboratively  is  also  necessary  to  build  some  kind  of  community.  Or  more  clearly,  if  group  of 
people  are  working,  or  in  this  case  learning,  collaboratively  there  is  a community.  That’s  why,  because  communities 
are  everywhere  (Wenger  1998).  It’s  important  to  realize  that  these  communities  don’t  have  to  agree  in  issues,  which 
they  are  dealing.  Arguing,  as  well  as  discussions,  agreeing  and  so  on,  is  common  for  the  communities. 

Informal  information  about  others  is  also  good  for  community  building.  It  supports  especially  the  social  aspect 
of  the  community.  Nardi  et  al.  (1998)  were  surprised  how  well  a chat  facility  supported  community  building.  It  was 
especially  its  informal  and  social  features  that  are  valuable  for  community  building. 

Companion  Awareness 

Like  collaborative  learning,  awareness  is  also  a notion,  which  is  hard  to  define  precisely.  It  has  lot  of  different 
aspects  and  it  can  be  classified  by  many  different  ways  (Liechti  2000).  Definition  of  the  awareness  is  depending  of 
the  context  of  the  use  and  also  of  the  characteristics  of  the  system.  In  this  paper  we  are  exploring  new  opportunities, 
which  the  awareness  can  provide  for  asynchronous  discussion  forums. 

Gutwin,  Stark  & Greenberg  (1995)  have  defined  a framework  of  awareness.  In  this  framework  they 
distinguish  social,  task  concept  and  workspace  awareness.  Especially  they  focused  on  workspace  awareness,  which  is 
essential  for  students  to  learn  and  work  together  effectively.  Workspace  awareness  is  information,  which  describes 
who  is  participating  to  the  work,  where  are  they,  what  have  they  done  and  what  are  they  doing.  Their  focus  was  on 
the  synchronous  learning  environments  but  their  conclusions  about  awareness  are  valid  also  in  the  asynchronous 
environments. 

Jermann  et  al.  (2001)  have  defined  another  framework,  which  describes  capabilities  of  tools  to  support 
collaborative  interaction.  In  this  framework  they  set  awareness  features  in  learners  locus  of  processing.  This  means 
that  awareness  support  gives  information  to  the  learner,  but  it  depends  on  the  learner  how  the  information  is 
interpreted  or  understood.  Because  of  that  awareness  features  should  be  quite  simple  and  the  given  information 
should  be  easy  to  understand. 

Awareness  information  can  help  collaborative  learning  especially  in  the  community  building.  When  a 
learner  knows  which  other  persons  belong  to  the  group,  he/she  can  have  a feeling  of  belonging  to  the  community. 
This  belonging  strengthens  the  community  and  makes  interactions  easier.  The  benefits  of  awareness  are  more 
obvious  in  synchronous  systems,  but  we  believe  that  these  benefits  are  important  in  asynchronous  learning,  too.  Our 
focus  is  to  support  learners’  collaborative  work  with  awareness  of  their  co-leamers.  In  asynchronous  collaborative 
learning  systems  this  awareness  information  should  include  the  following: 

• information  which  helps  to  identify  co-leamers, 

• information  which  supports  social  aspects  of  the  community,  and 

• information  which  shows  the  usage  of  the  system. 

We  use  the  notion,  companion  awareness,  to  describe  this  kind  of  awareness  information. 

Dyn3W  - Learning  environment 

In  our  courses  at  the  University  of  Tampere  we  are  using  Dyn3W  learning  environment.  It  is  a product  of  project 
called  CoWoGLe  (Conferencing  on  the  Web  for  Group  learning),  which  have  been  going  since  1996.  (Hietala  et  al. 
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1997).  The  system  has  been  in  use  on  several  courses  and  more  than  1000  students  have  been  using  it  over  the  years. 

The  main  part  of  our  Dyn3W  system  is  a discussion  forum.  The  discussions  are  threaded  (figure  1)  and 
because  the  system  is  using  frames,  it  is  possible  write  a comment  and  read  old  comments  concurrently.  The  system 
keeps  record  of  each  user’s  usage.  It  also  counts  comments.  Students  are  divided  in  groups  and  they  working  in  these 
groups  with  same  assignment. 

New  awareness  features 

In  our  system  the  learners  are  working  together  in  small  groups  and  they  have  to  make  a decision,  which  they  present 
to  the  other  groups.  Without  any  awareness  about  the  other  group  members  this  group  work  is  difficult,  especially 
the  decision-making  is  hard.  That’s  why  we  implemented  awareness  support  for  learners.  These  new  awareness 
features  include  information  about  who  group  members  are,  what  they  have  done  and  when  they  have  been  inside  the 
system.  For  learners  it  is  also  possible  to  give  an  informal  description  of  themselves.  This  was  designed  for 
supporting  the  informal  aspect  of  communication.  It  is  also  one  way  to  support  group’s  community  building,  which 
is  important  in  learning  relationships  (Wenger  1998). 
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Figure  1 : View  from  small  group  working  area 


A view  from  the  small  group’s  working  area  is  shown  in  figure  1.  In  the  left  frame  discussion  tree  is  shown. 
Above  that  there  is  the  button  bar,  which  includes  a button  for  the  awareness  information.  This  awareness 
information  is  shown  in  the  right  frame,  which  normally  is  used  for  reading  comments  from  the  discussion  and 
writing  new  comments  to  the  discussion. 

In  our  implementation  the  learner  can  see  him/herself  as  one  of  the  members  on  the  list  of  the  group. 
Another  way  in  the  implementation  could  be  that  he/she  is  removed  form  lists  and  information  is  only  about  other 
group  members.  In  this  case  it  is  more  difficult  to  learner  compare  him/herself  to  others.  Showing  the  current  user 
also  in  those  group  listings  makes  it  clearer  that  the  user  is  a member  of  the  group.  He/she  can  see  that  he/she  is  in 
the  same  position  with  others.  Also  comparing  oneself  to  others  comes  very  easy  (on  right  in  figure  2). 
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Figure  2:  Views  of  awareness  features.  Presence  information  (on  left)  and  number  of  posts  (on  right) 


We  hope  that  this  would  encourage  learners  to  work  harder.  Because  they  can  easily  see  how  much  work 
other  group  members  have  done,  they  might  want  to  do  likewise.  Of  course  this  can  also  lead  learner  to  be  lazy, 
because  he/she  is  not  only  one.  This  problem  could  be  solved  with  forced  requirements,  which  seem  to  be  good  way 
to  get  discussion  going  (Sorenssen  & Takle  2001 ). 

The  awareness  features  included  also  information  about  group  members’  presence  in  the  system.  This  is 
implemented  by  telling  which  time  each  group  member  has  logged  in  and  out  of  the  system  (on  left  in  figure  2).  It 
could  be  implemented  also  so  that  system  just  tells  who  is  inside  and  who  is  outside  the  system.  But  we  wanted  to 
show  learners  more  detailed  information  about  each  group  member’s  visits  to  the  system. 

Experiment  and  results 

Dyn3W-  system  with  the  new  features  was  in  use  on  Computer-Aided  Instruction  course  at  Spring  2001.  The  course 
had  74  participants.  In  this  course,  which  includes  also  lectures,  exercises  and  two  group  projects,  the  assignment  for 
working  in  Dyn3W-  system  was  to  collaboratively  produce  a short  description  for  a good  online  course.  For  that 
students  were  randomly  divided  into  eight  groups.  Four  of  these  groups  had  the  awareness  support  and  four  did  not 
have  this  support  in  the  conferencing  system.  This  means  there  were  36  learners  with  support  and  38  without. 
Because  of  that,  we  can  find  out  different  groups’  usage  of  the  system  and  compare  if  there  is  any  difference  e.g.  in 
discussion  activity.  Participants  used  the  system  for  a quite  short  period,  which  lasted  only  4 weeks. 

These  new  awareness  features  weren’t  introduced  or  explained  to  the  learners  during  lectures  or  exercises. 
We  did  this  because  we  wanted  to  find  out  if  the  users  could  find  these  tools  by  themselves  and  how  they  would  be 
using  those  tools.  In  this  way  we  can  explore  if  learners  find  ways  to  use  them  and  how  many  times  they  are  using 
them.  Introducing  these  tools  just  for  part  of  participants  would  also  be  quite  difficult.  Introduction  would  also  lead 
learners  to  use  tools  in  a particular  way  that  the  teacher  is  thinking  to  use  it. 

As  we  can  see  in  Table  1,  the  new  awareness  did  not  encourage  our  students  to  produce  more  comments 
than  in  the  groups  without  the  features  (6.8  < 8.2).  However,  if  we  look  a little  bit  closer  to  the  data,  we  can  find  an 
interesting  issue.  It  was  compulsory  for  all  participants  to  produce  at  least  one  comment  (new  course  suggestion)  in 
the  group.  If  we  consider  those  who  produced  more  than  only  this  one  compulsory  comment,  we  find  that  there  were 
more  students  who  stayed  and  participated  in  the  discussions  in  the  awareness  support  groups  than  in  those  without 
the  support  (29  > 28).  In  groups  without  the  awareness  features  26,3  % of  learners  made  just  that  required  comment, 
but  in  groups  with  the  awareness  features  the  same  percentage  was  only  19,4.  This  gives  a weak  evidence  for  the 
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usefulness  of  the  awareness  tools.  Learners  with  the  features  also  had  more  sessions  than  learners  without  the 
features. 


The  awareness  features 

With 

Without 

Participants 

36 

38 

Sessions  by  average 

16,5 

14,7 

Comments  by  average 

6,8 

8,2 

Participants  working  more  than  compulsory 

29 

28 

Table  1:  Comparing  the  two  learner  groups 


Most  of  the  learners  used  the  new  awareness  features  at  least  once  (Table  2).  This  means  that  most  of  the 
learners  found  these  features  easily.  Most  of  them  were  using  all  of  the  features.  Many  learners  liked  to  read  other 
learners  selMescriptions  but  they  didn’t  give  a description  of  themselves.  As  we  see  in  Table  1,  learners  using  the 
awareness  features  had  16,5  sessions  by  average  and  they  used  awareness  support  during  4 different  sessions  (Table 
2). 


Used  the  awareness  features  at  least  once 

33  learners 

Read  other  users  descriptions  of  themselves 

25 

Wrote  description  of  themselves 

5 

Were  interested  in  the  ’’presence”  information 

29 

Were  interested  in  the  number  of  posts 

29 

Sessions  where  using  awareness  by  average 

4 sessions 

Table  2:  Usage  of  the  awareness  features 


Conclusions 

According  to  the  preliminary  results  we  got  from  experiment  it’s  quite  fair  to  say  that  learners  were  at  least  very 
interested  in  getting  companion  awareness  information.  The  fact  that  learners  did  use  awareness  features  quite  often 
but  not  in  every  session,  supports  that  conclusion. 

Participants  took  part  in  the  discussions  as  much  as  it  has  been  common  in  this  course  in  earlier  years.  So 
the  awareness  features  didn’t  have  effect  for  participation  in  way  we  were  expecting.  Duration  of  the  online  exercise 
might  also  have  effects.  Four  weeks  period  is  quite  short,  which  makes  the  learners  focusing  to  the  task  and  leaving 
social  aspects  to  background.  On  our  course  learners  could  also  meet  other  learners  at  the  lectures  twice  a week,  on 
the  campus  site  and  so  on,  which  probably  effected  also  to  the  need  of  awareness  information.  The  awareness 
features  can  be  more  valuable  for  an  online  course  where  learners  are  physically  distributed. 

Typically  on  every  course,  traditional  or  online,  there  are  learners  that  don’t  want  to  do  more  than  they  have 
to.  We  got  weak  evidence  that  with  the  awareness  support  for  the  learners  we  can  downsize  the  number  of  this  kind 
of  behavior  of  learners.  Somehow  the  awareness  features  seemed  to  get  more  learners  to  be  involved  to  the  work  of 
community.  This  might  mean  that  learners  are  more  motivated  for  the  course  and  therefore  are  working  harder, 
which  would  hopefully  lead  to  the  better  grades  and  learning. 

We  also  found  out  that  learners  are  interested  in  all  kind  of  awareness  information,  but  they  are  not  ready  to 
take  extra  effort  for  making  others  aware  about  themselves.  This  can  be  seen  from  the  interest  of  other  learners’ 
descriptions.  Somehow  it’s  surprising  that  the  learners  didn’t  make  a self-description  after  they  have  read  other 
learners  self-descriptions.  Nardi  et  al.  (1998)  have  made  same  kind  of  observation  with  personal  Web  pages  in  an 
online  course. 

Future  work 

In  the  near  future  we  are  going  to  rearrange  the  experiment  we  have  discussed  in  this  paper.  Especially  we  want  to 
get  more  evidence  for  the  finding  that  those  learners  with  the  awareness  features  were  more  active  than  was 
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compulsory.  We  are  also  going  to  make  some  improvements  to  our  system.  We  are  going  to  highlight  buttons  for 
learner  self-descriptions  after  the  description  have  been  done.  We  hope  that  this  way  the  learners  will  not  get  tired 
with  this  feature,  because  now  they  can  know  from  the  button  if  their  co-leamers  have  made  their  self-description. 
We  are  also  considering  ways  to  encourage  learners  to  writing  their  own  descriptions. 

It  is  also  possible  to  give  learners  more  or  even  better  awareness  information.  One  possible  way  to  do  that 
would  be  information  about  the  reading  times  and  the  readers  of  each  posting  (along  the  lines  of  the  BSCW  system 
(Appelt  & Mambrey  1999)).  This  means  that  learners  should  be  able  to  see  who  have  read  their  postings.  So  if 
learner  makes  a new  posting,  he/she  can  check  later  who  has  read  it  or  hasn’t  read  it.  In  this  way  learner  can  be  more 
aware  about  ongoing  discussion.  In  this  case  questions  about  privacy  have  to  be  considered  too.  Is  it  fair  to  the 
learners  to  provide  so  detailed  information  about  their  usage  of  the  system  for  their  co-leamers?  If  it  is  done  openly, 
it  might  also  encourage  learners  to  use  the  system  more  and  hopefully  in  the  meantime  learn  more  in  collaboration 
with  others.  We  are  planning  to  include  these  awareness  features  into  our  discussion  forum. 

In  our  experiment  there  were  also  peer  groups  without  awareness  support.  If  we  in  the  future  compare  more 
carefully  these  groups  with  those  with  awareness  support,  we  can  maybe  find  out  more  detailed  knowledge  of 
usefulness  of  supporting  awareness.  A problem  with  that  kind  of  analysis  is  the  short  time  of  usage  of  Dyn3W 
system  on  that  specific  course.  The  awareness  features  are  probably  more  useful  in  longer  lasting  usage  of  online 
learning  environment.  One  possible  way  to  get  analysis  little  bit  further  is  to  evaluate  the  quality  of  different  groups’ 
proposals  and  find  out  if  there  is  any  difference  and  what  might  be  the  reasons  for  them.  This  is  one  of  the  issues  that 
we  are  planning  to  focus  in  our  future  research. 
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Abstract:  The  design  rationale  for  the  development  of  an  exemplary,  flexible  online  course  for 
making  accessible  online  courses  is  described.  We  developed  the  course  by  adopting  the  roles  of 
content  expert  and  instructional  designer.  The  course  has  a dual  purpose;  the  assessment  for 
postgraduate  students  provides  a guide  to  learning  for  academic  staff  development.  The  design 
and  development  process  included  the  formulation  of  learning  outcomes,  assessment  criteria  and 
strategies,  learning  methods,  and  the  alignment  of  these.  The  learning  strategies  included  a 
variation  of  the  cognitive  apprenticeship  model  that  was  refined  following  feedback  from 
prototype  face-to-face  workshops.  Learner-centred  design  is  fundamental  and  includes:  high 
contrast  text  and  background;  no  unnecessary  graphics  or  icons;  tips  for  people  using  assistive 
technology;  avoiding  the  use  of  PDF;  direct  links  to  other  course  areas;  text  transcript  for  online 
videos;  and  easy  access  to  resources  for  learning  activities. 


Project  Initiation  and  Rationale 

The  production  of  guidelines  for  making  online  courses  accessible  (Pearson  and  Koppi,  2001),  lead  to 
considerations  of  developing  an  online  course  for  making  accessible  online  courses.  Because  the  authors  are  at 
different  universities,  in  different  fields  with  different  ‘clients’,  the  nature  of  the  course  had  to  be  negotiated. 
Pearson  (at  the  University  of  Teesside,  UK)  teaches  postgraduate  students  and  the  desired  accessibility  module 
would  be  part  of  an  MSc  in  Multimedia  Design.  Koppi  (at  the  University  of  New  South  Wales,  Australia)  is 
involved  with  academic  staff  development  and  the  course  would  have  to  help  teaching  staff  to  produce  their  own 
accessible  online  courses  (using  WebCT  at  UNSW). 

The  project  development  proposal  had  to  include  a rationale  stating  that  the  intended  course  would  have  the  dual 
purpose  of  an  MSc  component  and  a staff  development  purpose.  This  was  to  prove  problematic,  particularly 
where  assessment  was  concerned.  The  project  proposal  was  based  on  constructivist  learning  principles  and  was 
structured  to  include  learning  activities,  dialogue  and  collaboration,  and  student  support.  We  decided  to  adopt  the 
roles  of  content  expert  (Pearson)  and  instructional  designer  (Koppi)  to  help  us  include  the  essential  perspectives  in 
courseware  development.  The  structure  of  this  paper  describing  the  design  and  development  of  the  course  is  based 
on  an  instructional  design  plan. 

We  realised  that  the  course  itself  would  have  to  be  an  exemplar  of  accessible  course  design.  We  would  have  to 
ensure  that  it  would  be  accessible  to  people  with  disabilities.  We  intended  to  employ  the  services  of  an 
experienced  student  who  is  blind,  and  is  a regular  Internet  user,  to  assist  us  with  identifying  the  capabilities  of 
assistive  technologies,  and  with  checking  the  accessibility  of  the  course.  We  also  intended  to  use  the  checking 
tools  provided  by  Bobby  (CAST,  2001),  DreamW  eaver  4,  and  our  own  Guidelines  (Pearson  and  Koppi,  2001)  to 
check  for  accessibility. 
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Formulation  of  Learning  Outcomes 

Following  instructional  design  principles  (Biggs,  1999a  and  b),  the  aims,  objectives  and  learning  outcomes  were 
formulated  first,  and  the  learning  outcomes,  which  are  concerned  with  what  the  student  will  be  able  do,  are  given 
as  follows. 

“On  successful  completion  of  this  module  the  student  will  be  able  to: 

1.  Discuss  the  issues  relevant  to  access  for  people  with  disabilities  to  online  learning. 

2.  Demonstrate  skills  in  the  use  of  relevant  guidelines  and  accessibility  checking  mechanisms. 

3.  Describe  the  use  and  application  of  assistive  technologies. 

4.  Describe  the  needs  of  the  learner  in  the  design  of  accessible  online  courseware. 

5.  Demonstrate  skills  in  the  design  and  development  of  accessible  and  inclusive  online  courseware. 

6.  Analyse  barriers  to  accessibility  in  existing  web  sites  and  online  courses.” 

Assessment  Criteria 

The  assessment  criteria  are  concerned  with  measuring  how  well  the  students  have  achieved  the  learning  outcomes; 
they  were  formulated  for  each  learning  activity  designed  to  produce  the  desired  learning  outcomes.  For  example, 
for  the  learning  outcome  concerned  with  the  task  investigating  the  guidelines  based  on  those  developed  by  the 
Web  Accessibility  Initiative  (WAI),  part  of  the  World  Wide  Web  Consortium  (W3C),  the  assessment  criteria 
together  with  relevant  information  are  given  as : 

“If  you  are  being  assessed  for  this  course,  your  portfolio  contribution  for  this  activity  will  be  assessed  according  to 
the  following  criteria.  If  you  are  carrying  out  this  task  out  of  interest  or  for  your  personal  development,  the  criteria 
can  be  used  as  a checklist  for  your  own  learning. 

1.  The  extent  to  which  you  have  considered  the  purpose  and  the  extent  of  the  activities  of  W3C  and  WAI. 

2.  Your  analysis  if  the  applicability  of  W3C  guidelines  for  accessible  web  site  design  to  the  average  non- 
technical academic  developer. 

3.  Evidence  of  research  and  understanding  of  the  nature  of  other  guidelines. 

4.  Quality  of  the  overall  presentation  and  coherence  of  the  posting  and  additional  links. 

5.  The  quality  and  coherence  of  your  contribution  to  the  discussion  of  this  topic. 

Marked  out  of  1 00%  with  20%  for  each  element.” 

As  indicated  above,  this  online  course  has  a dual  purpose  (part  of  a postgraduate  degree  and  for  staff 
development),  and  so  the  assessment  criteria  can  be  used  in  different  ways.  The  criteria  can  be  used  for  summative 
assessment  (portfolio)  in  the  case  of  the  students,  and  as  a learning  guide  in  the  case  of  teaching  staff  learning 
about  accessibility.  The  students  can  also  use  them  as  a learning  guide  because  the  assessment  criteria  are 
provided  with  each  learning  task. 


Assessment  Strategy 

Types  of  Assessment 

The  assessment  strategy  is  concerned  with  the  methods  of  assessment  that  are  best  suited  to  the  students 
demonstrating  that  they  have  achieved  the  learning  outcomes.  There  are  many  ways  of  assessing  learning  and 
McLoughlin  and  Luca  (2001)  note  that  there  are  three  types  of  assessment: 

• Cognitive  : thinking,  knowledge,  application  and  understanding  of  principles,  concepts 

• Performance:  demonstration  of  skills  and  abilities,  complex  task  performance 

• Portfolios:  evidence  of  complete  student  record,  tasks,  achievement,  examples  of  work  etc. 

Of  course,  these  three  types  are  not  exclusive,  e.g.,  a portfolio  would  include  evidence  of  cognitive  and 
performance  attainment.  For  academic  staff  undergoing  staff  development  in  inclusive  courseware  design,  and  for 
postgraduate  students  learning  about  courseware  design,  the  most  appropriate  types  of  assessment  would  be 
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performance  and  portfolios  since  we  would  want  to  see  practical  examples  of  work  as  evidence  of  learning  and 
application  of  inclusive  and  accessible  principles. 

The  portfolio  is  used  as  half  of  the  assessment  and  is  described  as  follows. 

“You  should  develop  a portfolio  by  posting  to  the  Student  Presentations  area  or  your  personal  topic  area  in  the 
discussion  forum,  to  demonstrate  understanding  of  issues  related  to  disability,  including: 

1.  The  Disability  Discrimination  Acts  and  their  effect  on  education  and  online  learning. 

2.  Review  of  current  guidelines  and  their  usefulness  for  academic  developers. 

3.  Review  of  assistive  technologies. 

4.  Accessibility  checking  tools,  their  use  and  application,  reviews  of  good/bad  examples  of  websites. 

5.  Design  methods  and  tools  for  creating  accessible  documents  and  courses. 

This  assessment  will  take  place  throughout  the  module  and  will  be  directly  linked  to  the  activities  you  undertake 
through  the  course.” 

The  other  half  of  the  assessment  is  concerned  with  the  students  or  staff  either  (a)  creating  or  modifying  an 
accessible  and  inclusive  course  based  on  their  own  subject  expertise,  or  (b)  redesigning  a given  course  (especially 
prepared  from  common  practices)  which  does  not  meet  accessibility  requirements.  Participants  have  the  option  of 
working  in  groups  or  as  individuals  in  carrying  out  the  tasks. 


Authentic  Assessment 

In  addition  to  the  learning  outcomes,  assessment  criteria  and  assessment  strategies  being  aligned,  the  assessment 
tasks  should  also  be  authentic.  Herrington  and  Herrington  (1998)  in  their  review  of  authentic  assessment  provide 
descriptors  such  as:  situated,  practical,  realistic,  performance-based,  real-world,  and  ill -structured.  The  use  of  real- 
world  learning  environments  enables  the  same  activity  to  be  used  for  learning  and  assessment  (Herrington  and 
Oliver,  2000).  For  example,  the  assessment  of  how  well  teachers  are  able  to  design  and  develop  an  accessible 
online  environment  is  their  production  of  an  accessible  online  course  or  their  redesign  of  an  inaccessible  course, 
as  described  above. 

In  the  real  world,  work  is  often  collaborative  in  nature  and  authentic  tasks  should  include  collaborative  activities 
wherever  possible  (as  in  this  course).  If  well  designed,  collaborative  work  can  enhance  the  learning  experience 
and  the  social  negotiation  that  promotes  higher  order  thinking  (Herrington  and  Oliver,  1999).  In  the  course, 
discussion  topics  are  used  to  facilitate  collaboration  and  the  presentation  of  alternative  viewpoints. 


Learning  and  Teaching  Strategy 

The  learning  and  teaching  strategy  is  concerned  with  the  methods  will  best  help  students  achieve  the  learning 
outcomes.  Having  formulated  the  learning  outcomes  and  assessment  strategy  and  criteria,  it  follows  that  the 
learning  tasks  should  be  considered  and  that  they  should  all  be  in  alignment.  This  section  presents  a rationale  for 
the  learning  tasks  employed. 


Cognitive  apprenticeship  model 

The  cognitive  apprenticeship  approach  (Brandt  et  aL,  1993),  when  coupled  with  participation  in  the  community  of 
practice  (Lave  and  Wenger,  1991)  and  authentic  problem-based  learning  (Savery  and  Duffy,  1995;  Grabinger  and 
Dunlap,  2000),  can  provide  an  authentic  situated  learning  experience  that  bridges  the  gap  between  abstract  theory 
and  effective  practice  (Herrington  and  Oliver,  2000).  With  respect  to  designing  accessible  learning  environments, 
the  intention  is  for  the  teachers  to  apply  the  theory  in  their  everyday  practice  and  not  to  treat  it  only  in  the  abstract. 
Learning,  online  or  otherwise,  can  be  facilitated  by  the  use  of  scaffolding  (support)  in  a social  constructivist 
setting  (Roehler  and  Cantlon,  1997). 

We  decided  to  adapt  the  five  stages  of  the  cognitive  apprenticeship  model  (Brandt  et  al.^  1993)  as  follows. 
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Phase  I We  elected  to  present  the  expert  perspective  through  an  interview  (Koppi  interviewed  Pearson)  which 
was  filmed  and  converted  to  streaming  video.  The  five  major  issues  (as  identified  in  the  learning  outcomes)  and 
expert  considerations  were  discussed  (Pearson  and  Koppi,  2002).  The  video  sets  the  scene  for  the  five  learning 
tasks  corresponding  to  the  first  five  learning  outcomes. 

Phase  2 Working  in  groups,  ftie  participants  carry  out  the  learning  activities  and  discuss  their  findings  and 
conclusions. 

Phase  3 As  an  authentic  activity,  the  participants  apply  what  they  have  learned  to  the  design  and  construction  of 
their  own  online  learning  environment  (e.g.,  using  WebCT)  or  to  the  redesign  of  an  existing  problematic  one 
already  prepared  by  us. 

Phase  4 The  learners  continue  developing  their  websites  in  their  own  time  and  check  that  their  sites  conform  to 
acceptable  standards  (e.g.  by  using  the  Bobby  and  DreamWeaver  software).  Postgraduate  students  carry  out  the 
assessment  already  detailed  - redesign  of  inaccessible  course  or  design/re-design  a proposal  for  their  own  course 
or  web  site. 

Phase  5 The  learners  may  each  reflect  on  what  they  have  learned  and  describe  general  principles  for  developing 
accessible  online  learning  environments.  Students  illustrate  their  learning  by  creating  a portfolio  of  their 
accessible  designs  and  implementations  that  contribute  to  their  assessment.  Academic  staff  could  organise  and 
facilitate  a workshop  in  their  own  department  or  school. 

Face-to-Face  Workshop  Prototypes 

We  developed  face-to-face  workshops  on  accessibility  for  academic  staff,  with  the  intention  of  using  the 
workshop  tasks  as  prototypes  for  the  online  tasks.  The  workshops  enabled  us  to  see  how  the  learning  tasks 
worked,  how  long  they  took,  what  the  problems  were,  and  to  obtain  feedback  from  the  participants.  We  modelled 
the  workshop  along  the  lines  of  the  online  course:  orientation  introduction,  activities,  reporting  back,  with  support 
from  the  workshop  facilitator.  We  were  in  effect  doing  evaluation  of  the  intended  online  tasks  in  advance. 

The  results  of  the  workshop  experience  enabled  us  to  refine  activities  for  the  online  course,  mainly  by  adding 
more  support  and  tasks  to  provide  a background  and  orientation  to  the  issues  of  learner-centred  design.  We  also 
realised  how  much  more  valuable  an  online  course  can  be  than  an  ephemeral  face-to-face  workshop  which 
disappears  without  visible  trace.  The  online  course  can  be  revisited  and  re-examined  after  further  baming  or 
application  of  learning  has  occurred.  Fleeting  ideas  in  a workshop,  too  soon  gone  because  of  the  pace,  can  be 
explored  in  the  online  course.  The  advantages  of  face-to-face  though  cannot  be  denied  - the  time  commitment  is 
made  and  colleagues  are  there  on  hand  for  immediate  discussion,  however  brief.  It  seems  that  commitment  to  an 
online  course  can  be  problematic  for  busy  people  because  it  is  too  easy  not  to  set  the  time  aside  (Forsyth,  2001). 

We  also  utilised  the  services  of  Darren  Fittler,  a law  student  who  is  blind,  and  an  experienced  Internet  user.  This 
proved  to  be  a highly  engaging  time  for  the  workshop  participants.  Darren  was  a first-time  user  of  WebCT  and 
went  through  his  allotted  tasks  (of  which  he  had  no  prior  information),  speaking  his  thoughts  out  loud,  and  that 
gave  a powerful  message  with  respect  to  learner-centred  design.  We  felt  we  had  to  capture  that  and  incorporate  it 
into  the  online  course.  He  was  later  videoed  going  through  it  all  again  in  a studio  setting.  If  we  had  not  had  the 
experience  of  the  face-to-face  workshop  with  Darren  we  would  probably  not  have  thought  of  including  him 
online. 


Multipurpose  Course  Design 

The  issue  of  the  dual  purpose  of  the  online  course  (part  of  a postgraduate  degree,  and  staff  development),  and  its 
effect  on  the  inclusion  of  assessment  strategies,  was  resolved  essentially  by  giving  the  assessment  criteria  different 
purposes.  For  academic  staff,  doing  the  course  out  of  interest  or  personal  development,  the  assessment  criteria  can 
be  used  simply  as  a guide  to  learning.  This  obviated  having  to  develop  two  separate  courses,  one  with  and  one 
without  assessment.  Extra  instructions  and  clarification  had  to  be  repeated  for  every  activity  page  on  the  website 
because  the  course  can  be  dipped  into  at  any  place  by  staff  and  being  confronted  with  unexpected  assessment  may 
prove  a barrier  to  engagement. 


Design  Features  of  the  Online  Course 


Since  leamer-centred  design  is  a core  concern  of  the  online  course,  some  of  those  design  features  are  noted  here. 

• The  Home  page  has  text  and  icon  (with  alt-tags)  links  to  all  the  important  elements  of  the  course. 

• High  contrast  text  and  background  are  used  throughout,  and  no  unnecessary  graphics  or  icons  are  used. 

• On  the  Home  page  there  are  tips  for  accessibility,  e.g.,  for  people  using  a screen  reader  to  hide  the  left 
navigation  bar  which  adds  unnecessary  complexity  to  the  page. 

• The  tip  also  includes  a link  to  downloading  the  Adobe  Acrobat  Reader  5 to  enable  suitably  formatted  PDF 
documents  to  be  read  by  a screen  reader. 

• The  use  of  PDF  has  been  avoided  wherever  possible,  or  alternative  formats  have  been  provided. 

• The  Welcome  page  provides  tutor  contact  details,  describes  the  course,  its  rationale,  the  activities,  assessment 
portfolio,  and  invites  constant  feedback/evaluation  of  the  course. 

• The  aims,  objectives  and  learning  outcomes  are  visible  from  the  Home  page. 

• A link  to  other  parts  of  the  course  is  provided  wherever  they  are  mentioned,  and  that  link  opens  in  a new 
window  to  enable  easy  return  (closing  the  window). 

• Links  are  provided  directly  to  any  particular  discussion  topic  rather  than  Just  to  the  discussion  area  which 
would  require  further  searching  to  find  the  specific  relevant  topic. 

• A schedule  is  provided  of  tasks,  their  content,  deliverables  (particularly  if  the  course  is  assessable)  and  an 
indication  of  how  much  time  should  be  allocated  to  each  task.  This  time  allocation  is  a suggestion  only  in  that 
personal  interests  and  different  learning  styles  will  result  in  different  times  being  spent  on  the  tasks. 

• An  orientation  activity  is  provided  for  easing  new  online  learners  into  the  environment,  and  to  enable  course 
participants  to  meet  each  other  and  to  comment  on  each  other’s  interests.  Practice  is  also  provided  in 
uploading  a file  to  the  student  presentation  area,  with  tutor  contact  details  immediately  to  hand  in  case  of 
difficulties. 

• Each  of  the  activities  is  developed  as  self-contained  (to  minimise  searching  other  documents  for  relevant 
information)  with  introduction,  task,  reporting,  discussing,  assessment  criteria  (for  those  being  assessed),  and 
resources  being  provided. 

• A direct  link  to  assessment  is  also  provided  on  the  Home  page. 

• A link  to  resources,  including  links  to  relevant  free  software  downloads,  is  available  from  the  Home  page. 

• The  videos  include  either  subtitles  or  a text  transcript. 
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i.  Introduction 

There  is  a growing  tendency  to  strive  for  integration  of  individual  and  social 
dimensions  in  the  new  learning  environment  settings  where  technology  is  used  as  a tool 
for  organising  learning.  Computer  Support  for  Collaborative  Learning  (CSCL)  is  an 
emerging  area  of  research  when  using  Web-based  learning  environments  in  education. 

It  represents  a departure  from  previous  work  in  instructional  technology  where 
learning  was  seen  as  an  individual  phenomenon.  Many  current  learning  theories 
connected  to  CSCL  embrace  a view  of  learning  as  a largely  social  phenomenon.  Social 
interaction  is  seen  to  be  a paramount  site  for  the  development  and  practice  of 
cognition.  One  might  expect  the  group  to  perform  better  than  individuals  on  various 
challenging  tasks  like  learning,  knowledge  creation,  creativity,  and  problem  solving. 
(Solomon  1993;  Levine  et.al.  1993;  Scardamalia  & Bereiter  1994;  Kirschner  & 
Whitson  1997).  Collaborative  learning  can  here  be  defined  as  a process  through  which 
a group  of  learners  construct  knowledge  for  its  members  and  for  itself  as  a learning 
community.  It  is  typical  of  collaborative  learning  that  learners  have  an  inherent  and 
spontaneous  motive  for  collaborate,  that  learners  determine  and  plan  their  own 
learning  process,  and  learning  contents  are  tailored  in  that  learning  community. 
Learning  is  co-operative  and  based  on  interaction  with  others,  and  it  is  evaluated  on  a 
self-evaluation  basis,  (see  Korhonen  2001.)  Learners  thus  have  the  possibility  to  study 
and  apply  issues  closely  related  to  their  life  situations  and  contexts  flexibly  (cf  Brown, 
Collins  & Duguid  1989;  Lave  1988)  Structured  and  formal  teaching  and  learning  is 
replaced  with  a learning  environment  constructed  by  group  of  learners,  in  where  they 
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are  able  to  set  their  learning  goals  collectively  and  share  their  responsibility  to  follow 
the  learning  paths  to  meet  these  goals. 

There  are  some  couraging  research  findings  of  group  discussions  or  problem-solving 
activities  in  collaborative  learning  settings  which  give  a support  on  the  use  of 
asynchronous  group  discussions  in  CSCL  environment.  Benefits  of  group  discussion 
actions  brought  to  learning  are  e.g.  increased  motivation,  deeper  understanding  of  the 
concepts  and  an  increased  willingness  to  tackle  difficult  questions  that  learners  cannot 
answer  alone  (Weir  1992).  Participants  who  solved  problems  through  computer 
supported  group  discussion  were  more  satisfied  with  the  learning  process  and  virtual 
groups  tend  to  be  more  task  oriented  than  face-to-face  groups.  Computer  conferencing 
is  giving  flexibility  to  the  communication.  (Jonassen  & II  Kwon  2001.)  Group 
discussions  brought  in  a case-based  learning  environment  showed  also  that  students 
who  worked  in  groups  performed  significantly  better  that  those  who  worked  alone  on 
the  analysis  and  alternatives  tasks.  Participants  who  worked  in  groups  liked  their 
method  also  significantly  better  than  those  who  worked  alone  and  felt  they  have 
learned  more  working  in  a group.  (Flynn  & Klein  2001).  The  primary  disadvantage 
that  has  been  identified  in  recent  research  is  the  lack  of  nonverbal  cues  and  the  fact  that 
in  a virtual  group  discussion  it  is  more  difficult  to  reach  group  concensus.  Some  of  the 
students  might  complain  also  about  the  extra  time  and  effort  needed  for  written 
argumentation,  (see  Jonassen  & Kwon  2001.)  Computer-mediated  communication 
appears  to  support  cognitive  development  of  participants  in  many  ways  but  educators 
need  more  information  how  to  foster  student  student  development  in  general,  and  to 
enhance  teaching  and  learning  in  particular  in  CSCL  environments. 

Central  to  the  implementation  of  group  discussion  settings  mentioned  above  is  the 
organization  of  learning.  There  is  some  evidence  that  the  more  students  are  coached 
and  instructed  in  their  systematic  and  reflective  use  of  computer  supported 
communication,  the  more  varied  and  extensice  the  collaboration  will  be  (Admiraal 
et.al.  1997).  The  strength  of  group  discussion  setting  in  the  collaborative  learning 
environment  lies  in  its  members’  diversity,  with  each  member  should  contribute  his  or 
her  knowledge  and  competence  into  the  collective  action  (Sharan  & Sharan  1994; 
Korhonen  2001).  The  purpose  of  this  presentation  is  to  analyse  two  different  kind  of 
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cases  where  the  discussion  groups  have  been  brought  into  CSCL  setting.  The  analysis 
and  interpretations  of  the  discussion  groups  are  intended  to  imply  how  group 
investigation  based  or  on  the  other  hand  freely  organized  discussion  group  influence  of 
group  interaction  in  CSCL  environment. 

2,  Two  Cases  under  Examination 

The  group  discussion  cases  evaluated  here  are  carried  through  a WebCT-based 
network  learning  environment.  The  CSCL  environment  is  serving  as  a basis  for  sharing 
material  and  information,  discussions  and  publishing  written  documents  in  Web.  In 
these  cases  learning  is  not  only  virtual;  it  also  involves  contact  teaching  as  orientation 
or  group  work  under  a teacher’s  supervision. 

In  the  first  case  the  use  of  the  Web  discussion  group  is  based  on  certain  co-operative 
group  method  brought  into  CSCL  environment.  The  group  method  chosen  is  the 
Group  Investigation  method  developed  by  Sharan  & Sharan  (1992;  1994).  Group 
discussion  is  based  upon  research-like  learning  process  and  consensus  building  through 
cooperation  by  group  members.  The  group  of  students  attending  this  integrated  Web- 
based  course  is  consisting  of  28  Finnish  education  science  and  adult  education  students 
in  University  of  Tampere.  The  Group  Investigation  period  in  Web-based  learning 
environment  is  bringing  an  interesting  addition  to  the  traditional  seminar  work,  which 
is  carried  out  in  face-to-face  meetings.  The  character  of  Group  Investigation  lies  in  the 
integration  of  four  features:  investigation,  interaction,  interpretation,  and  intrinsic 
motivation.  Investigating  in  groups  calls  for  students  to  use  their  interpersonal 
communication  and  planning  skills,  reflect  and  integrate  their  findings  collectively  and 
present  their  findings  to  others  as  well.  (Sharan  & Sharan  1992).  These  basic  co- 
operative features  of  the  group  method  are  combined  in  the  six  stages  of  the  model 
(Sharan  & Sharan  1994): 

Stage  1:  Students  determine  subtopics  and  organize  into  research  groups. 

Stage  2:  Groups  plan  their  investigations. 

Stage  3:  Groups  carry  out  their  investigations. 

Stage  4:  Groups  plan  their  presentations. 

Stage  5:  Groups  make  their  presentations. 
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Stage  6:  Teacher  and  students  evaluate  their  projects. 

Group  Investigation  method  suits  for  small  group  of  learners,  who  have 
complementary  skills  and  have  a common  learning  goal  to  reach  for.  The  goal  of  the 
Group  Investigation  is  to  combine  social  interaction  and  communal  problem  solving  to 
a research  like  process  (Vahtivuori,  Wager  & Passi  1999).  This  method  is  quite  simple 
and  straightforward  for  implementation,  so  it  doesn’t  require  a long  careful  orientation. 
This  kind  of  instructional  co-operative  method  seems  to  work  particularly  well  for 
Higher  Education  students  of  theoretical  science  study  modules. 

In  the  second  case  learning  also  involves  contact  teaching  as  orientation  and  small 
group  work  under  a mentor’s  supervision.  However,  the  network  discussion  group  is 
organized  much  more  freely.  The  organization  of  groups  is  based  on  certain  theoretical 
discussion  themes.  The  discussion  groups  are  working  as  knowledge  construction 
areas  where  the  goal  of  the  discussion  is  to  share  the  knowledge  with  others  and  to 
reflect  the  findings.  The  group  of  students  attending  this  network-based  courses  is 
consisting  of  22  adult  nursing  science  students  in  Open  University  in  Institute  of 
Extension  Studies  in  Tampere.  The  requirements  in  this  network  courses  is  to  do 
several  written  assignments:  like  analysis  of  network  discussion  themes  and  write  a 
written  home  assignment.  The  role  of  the  virtual  discussion  forum  is  to  be  a basis  for 
all  the  written  assignments  in  the  study  period. 

The  both  of  the  research  groups  construct  a personal  learning  diary.  These  authentic 
learning  diaries  of  the  learners  are  composing  the  central  research  data.  So  the 
situations  are  analyzed  qualitatively.  The  main  research  question  in  this  case  is  to  try  to 
find  through  these  discussion  group  cases  some  evidence  on  how  group’s  movement 
toward  reaching  a conclusion  is  carried  out  in  different  kind  of  group  discussion 
settings.  Is  the  group  interaction  and  learners’  experiences  of  interpersonal  interactions 
different  when  there  is  certain  cooperational  method  utilized  and  is  the  quality  of 
outcome  better  or  worse  in  that  case? 

The  analysis  of  the  research  material  is  carried  out  during  the  winter  2001  -2002  and 
will  be  potentially  reported  in  the  coming  ED-MEDIA  2002  conference  in  June,  2002. 
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Abstract.  The  development  of  web-based  courses  for  virtual  universities  very  often  starts  from  a paper-based  tradition  of 
teaching  and  learning.  Web-based  learning  environments,  however,  have  an  added-value  by  the  potential  of  hypermedia, 
multimedia,  interactivity,  cooperation  and  collaboration,  mediated  by  digital  media  with  different  features  distinguishing 
contexts  for  presentation  of  content.  Currently,  there  exists  no  adequate  model  covering  media  as  distinguishing  contextual 
aspects  for  web- based  learning  environments.  This  paper  presents  a framework  to  deal  with  media  features  as  varying 
contexts  during  the  development  of  learning  environments. 


1 Introduction 

The  production  of  web-based  courses  often  start  from  a paper-based  tradition  of  teaching  and  learning  with  content  authors 
writing  a kind  of  book,  which  is  subsequently  the  basis  for  a continuous  media  production  process  considering  media  features 
as  distinguishing  contexts.  Therefore,  the  production  of  web-based  learning  material  does  not  only  mean  the  transfer  of  and 
adaptation  from  one  medium  to  another  one,  but  also  an  integration  into  an  adequate  media  production  process,  where  media 

features  change  contexts  for  design.  As  we  see  the  development  of  web-based  courses  as  equivalent  in  many  aspects  to 

software  development  processes  we  refer  to  ISO  13407  as  a reference  model  for  user  centered  design  processes,  which  focus 
on  understanding  user  contexts.  The  importance  of  contexts  have  already  been  recognized  in  many  other  areas  (see  for 
example  Suchman  1987).  This  paper  is  based  on  experience  of  the  authors  in  the  production  of  web-based  courses,  in  the 
design  of  user-adequate  learning  spaces  and  in  the  support  of  the  design  process.  The  IMIS  is  involved  in  a national  project 
called  “Distance  Education  in  Medical  Computer  Science”  (started  in  January  1999)  which  aims  at  providing  a complete  course 
of  studies  for  the  specialization  of  students  in  medical  computer  science  offered  at  a virtual  university  (Hagen,  Germany) 
(Kritzenberger/Hartwig/Herczeg  2001;  Kritzenberger/Herczeg  2001). 

2 Media  Contexts  in  a User-Centered  Production  Process 

The  stages  of  media  production  (writing  a book,  transmission  of  the  book  into  hypertext,  hypertext  functions,  multimedia 
course,  interactive  computer-based  training,  cooperative  learning)  provide  different  contexts  for  thinking  about  media  design 
for  learning  environments  integrated  in  one  process  model  of  media  production  (figure  1),  covering  media  features  in  cyclic 
sub-processes  (similar  to  the  spiral  model  for  the  user-centered  software  development  processes  as  specified  in  ISO  13407). 
Furthermore,  in  the  development  of  web-based  courses  we  have  to  distinguish  several  problem  areas  for  the  media 
development  process:  Domain  knowledge  selection,  teaching  model,  media  features,  interactivity,  communication  and 
collaboration.  Most  of  the  requirements  come  from  the  overriding  demands  of  the  teaching  model.  It  is  the  main  frame  for 
specification  of  hypermedia,  and  has  a secondary  effect  on  framing  the  multimedia  use  and  interactivity.  For  the  pedagogic 
conception  of  interaction  the  main  focus  has  to  be  on  the  learner's  cognitive  operations  and  information  processing 
procedures.  The  decision  is  also  the  embedding  of  hypermedia,  media  use  and  interactivity  in  the  overall  learning 
environment.  These  requirements  have  to  be  made  more  concrete  in  the  media  design  and  is  subject  to  discussion  between 
content  authors,  media  designers,  pedagogic  experts  and  several  other  specialists.  That  is,  the  case  is  not  to  enhance  the 
hypertext  with  time -based  multimedia  like  audio,  video  or  animations  etc..  The  critical  aspect  of  multimedia  design  is  to  use  it 
for  a more  precise  presentation  of  facts  wherever  it  seems  to  be  adequate,  e.g.  as  a more  concrete  way  of  explaining  processes, 
which  otherwise  would  be  too  complicated  to  be  explained  properly  and  therefore  could  otherwise  hardly  be  understood  by 
the  learners.  For  example,  simulations  can  help  to  demonstrate  how  time -based  actions  follow  each  others  in  a complex  process 
to  be  explained  in  the  subject  domain.  In  this  stage  of  development  it  is  helpful  to  analyze  the  kinds  of  knowledge  involved  in 
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order  to  deduce  consequences  for  presentation.  Furthermore,  there  is  a larger  organizational  context  of  teaching  and  learning, 
where  communication  and  collaboration  environments  are  applied.  We  have  also  to  understand  the  use  of  the  multimedia  and 
interactive  learning  environments  in  this  larger  context  of  communication  and  collaboration  processes  of  the  virtual  university 
or  other  institutions  where  the  web-course  will  be  offered. 


Figure  1 : Contexts  for  Media  Production  Processes 


Conclusions 

The  model  proposed  in  figure  1 is  capable  to  deal  with  different  media  features  separately  and  reflect  them  as  contexts  of 
earning.  The  model  also  shows  an  integration  of  the  interdependence  of  human-centered  design  activities  as  specified  in  ISO 
13407.  For  further  work,  this  model  has  to  be  refined  as  a generic  model  for  dealing  with  media  contexts  in  the  development  of 
web-based  learning  environments  in  order  to  improve  their  development  process  and  by  this  also  their  usability. 
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Abstract.  Learning  is  an  active,  constructive  and  collaborative  process,  where  people  construct  knowledge  from  their 
experiences  in  the  world.  People  construct  new  knowledge  with  particular  effectiveness  when  they  engage  in  constructing 
personally  meaningful  products,  that  is  meaningful  to  themselves  or  to  others  around  them.  The  construction  of  knowledge 
from  experiences  in  the  world  seems  to  be  especially  important  in  childhood  learning,  as  children  need  to  learn  through  their 
senses  and  through  physical  activity.  Unfortunately,  for  school  children  there  are  few  learning  situations  where  senses  and 
physical  activity  is  involved.  This  is  partly  due  to  the  nature  of  the  concepts  to  be  learned  and  partly  to  the  lack  of 
manipulative  learning  material.  This  contribution  introduces  a mixed  reality  environment  as  a collaborative  and  constructive 
learning  space  for  elementary  school  children.  During  recent  teaching  experiments  in  an  dementary  school  the  children 
created  their  own  mixed  reality  environment  which  served  from  a scientific  point  of  view  for  a semiotic-based  understanding 
for  the  creation  and  usage  of  physical  and  digital  media  bringing  together  arts  and  computer  science. 


1 Introduction 

Education  in  the  media  age  means  to  develop  media  competence  and  a deeper  understanding  of  media  characteristics  starting  from 
kindergarten  age  to  high  school  education.  The  learning  goal  is  to  understand  the  design  as  well  as  the  use  of  the  media  and  also  to 
understand  the  essence  and  nature  of  specific  media. 

Starting  from  these  learning  goals  the  authors  are  involved  in  a project  called  ArtDeCom  ("Theory  and  Practice  of  Integrating  Education  and 
Training  in  Arts  and  Computer  Science").  We  try  bringing  together  education  in  the  two  usually  separated  disciplines  of  arts  and  computer 
science  in  school  on  several  levels  of  education.  Pupils  should  be  given  an  opportunity  to  explore  the  relationships  of  the  two  disciplines 
with  adequate  tools  and  learning  goals  for  their  respective  age  and  learner  level.  The  project  is  funded  by  the  German  “Bund-Lander 
Commission  for  Educational  Planning  and  Research  Promotion”  (BLK)  within  the  general  funding  program  "Culture  in  the  Media  Age".  It 
aims  at  investigating  teaching  material  and  situation  for  a curriculum  definition  for  integrating  education  in  arts  and  computer  science  on  all 
educational  levels.  The  IMIS  is  cooperating  with  the  Muthesius  Academy  of  Arts,  Design  and  Architecture  and  with  the  Institute  of  Art 
History  of  the  Christian-Albrechts-University  in  Kiel  (Germany)  to  develop  curricular  elements  for  an  integrated  education  of  arts  and 
computer  science. 

This  situation  is  the  background  for  the  integrated  lessons  of  arts  and  computer  science  in  an  elementary  school  in  Luebeck  (Germany).  The 
teaching  attempt  is  in  the  3^^^  class  level  of  an  elementary  school  for  children  with  an  age  of  8-9.  During  several  lessons  the  children  work  in 
design  projects  developing  their  own  mixed  reality  environment,  within  which  they  later  perform  an  interactive  musical  revue.  The  children 
themselves  develop  the  scenes  of  the  mixed  reality  environment  working  in  groups  and  using  adequate  software  tools  as  well  as  real  world 
materials.  Collaborative  and  constructive  learning  is  practiced  permanently  during  the  design  process  as  well  as  during  interaction  with  the 
hybrid  environments  of  the  mixed  reality  scenery. 

A key  focus  of  our  research  refers  to  cross-settings  of  ways  of  thinking  in  arts  and  in  computer  science.  We  think  that  mixed  reality 
environments  are  a perfect  learning  space  that  enables  children  to  explore  concepts  from  both  disciplines  in  an  intuitive  way. 

2 Collaborative  and  Constructive  Learning 

Constructive  learning  theory  addresses  learning  as  an  active  process,  in  which  people  actively  construct  knowledge  from  their  experiences  in 
the  world.  People  construct  new  knowledge  with  particular  effectiveness  when  they  engage  in  constructing  personally  meaningful  products, 
which  are  meaningful  to  themselves  or  to  others  around  them.  Furthermore,  it  is  important  that  the  learning  environment  is  authentic  and 
situated  in  a real-life  situation.  Learners  must  get  an  opportunity  to  build  multiple  contexts  and  perspectives  in  a social  context.  Numerous 
examples  have  been  given  in  academic  context. 


1072 


The  construction  of  knowledge  from  experiences  in  the  world  seems  to  be  especially  important  in  childhood  learning,  as  children  need  to 
learn  through  their  senses  and  through  physical  activity.  Todays  kindergartens  and  playgrounds  are  full  of  physical  objects  and  physical 
activity.  But  as  children  move  on  through  elementary  school  and  into  secondary  school,  they  encounter  fewer  learning  situations  where 
senses  and  physical  activity  is  involved.  This  is  partly  due  to  the  nature  of  the  concepts  to  be  learned.  They  are  very  difficult  to  explore 
with  the  senses  when  manipulative  learning  material  is  not  available.  One  reason  is  that  many  abstract  concepts  seem  to  be  very  difficult  (if 
not  impossible)  to  explore  with  physical  media.  For  example,  traditional  physical  media  generally  do  not  support  children  understanding  the 
behavior  of  dynamic  systems  or  how  patterns  arise  through  dynamic  interactions  among  component  parts.  Such  concepts  are  typically 
taught  through  more  formal  methods,  involving  abstract  mathematical  formalisms.  Unfortunately,  many  students  have  severe  problems  with 
this  approach,  and  thus  never  develop  a deep  understanding  of  these  concepts  (Resnick  1998). 

With  this  background  in  mind  Resnick  (Resnick  1998)  has  created  a new  generation  of  computationally  enhanced  manipulative  materials, 
called  “digital  manipulatives",  developed  at  the  MIT  Media  Lab.  They  expand  the  range  of  concepts  that  children  (and  adults)  can  explore 
through  direct  manipulation  of  physical  objects.  They  aim  to  enable  children  to  continue  to  learn  concepts  with  “kindergarten  approach” 
even  as  they  grow  older.  As  children  build  and  experiment  with  these  manipulative  materials,  they  form  mental  models  and  develop  deeper 
understanding  of  the  concepts  they  enact  with.  Children  continue  to  learn  new  concepts  with  a “kindergarten  approach”.  Resnick  assumes 
that  children  learn  with  digital  manipulatives  concepts  that  were  previously  considered  “too  advanced”  for  them. 

There  are  several  other  approaches  with  physical  and  tangible  (graspable  and  touchable  objects)  interaction,  for  example  MIT's  KidsRoom 
(Bobick  et  al.  2000),  Triangles  and  ’strings’  (Gorbet  et  al.  1998),  Curlybot  (Frei  et  al.  1999);  StoryMat  (Ryokai/Cassel  1999)  in  order  to 
enhance  collaboration  among  learners  and  enable  constructive  learning  experiences  for  children. 

Furthermore,  several  design  projects  with  children  have  shown  positive  effects  on  learning  and  deep  understanding  (Alborzi  et  al.  2000; 
Druin/Perlin  1994;  Stanton  et  al.  2001).  In  these  projects  children  create  external  artifacts,  like  animate  stories,  video  games,  kinetic 
sculptures,  models,  simulations  and  so  on,  which  they  share  and  discuss  with  others.  These  artifacts  provide  rich  opportunities  for  learning. 
As  children  are  involved  as  active  participants,  they  have  a greater  sense  of  control  over  the  learning  process.  As  they  design  artifacts  in 
group  work,  they  experience  pluralistic  thinking,  multiple  strategies  and  solutions.  By  the  way  they  need  to  think  about  how  other  people 
will  understand  and  use  their  constructions.  Furthermore,  design  projects  are  mostly  interdisciplinary  and  therefore  bring  together  concepts 
from  different  disciplines. 

3 Mixed  Reality  Environments  as  Learning  Spaces 

In  our  approach  to  constructive  learning  we  use  a mixed  reality  environment  as  a collaborative  and  constructive  learning  space  for  children. 
As  we  have  discussed  in  the  previous  chapter,  in  early  elementary  classrooms  children  need  manipulative  material  to  initiate  learning 
processes,  as  they  allow  exploration  with  the  senses  and  therefore  enable  physical  experience  for  the  learner.  They  allow  children  to  explore 
concepts  by  interaction  and  with  their  senses  and  therefore  allow  to  build  and  to  correct  mental  models  and  a deeper  understanding  of  the 
underlying  domain. 

In  mixed  reality  environments  real  world  and  virtual  world  objects  are  presented  together  on  a single  display.  Mixed  Reality  techniques  have 
proven  valuable  in  single  user  applications.  For  example  in  other  contexts  single  user  mixed  reality  interfaces  have  been  developed  for 
computer  aided  instruction  (Feiner/Mcintyre/Seligmann  1993)  and  for  medical  visualization  (Bajura/Fuchs/Ohbuchi  1992).  These 
applications  have  shown  that  mixed  reality  interfaces  can  enable  a person  to  interact  with  the  real  world  in  ways,  which  were  never  possible 
before.  However,  there  has  been  less  research  on  collaborative  applications  (Billinghurst/Kato  1999)  or  on  interaction  of  children  with  mixed 
reality  environments  (www.animax.de)  or  even  multi-user  mixed  reality  environments  and  their  use  as  learning  spaces. 

3,1  Design  of  Mixed  Reality  Environments 

The  mixed  reality  environments  used  in  our  lessons  are  designed  by  the  children  themselves  in  collaborative  design  projects.  In  the  beginning 
the  class  is  divided  into  g'oups,  where  each  group  works  out  a project  development  plan.  It  includes  a story,  the  material  to  be  used,  the 
function  of  the  computer  and  its  connections  and  extensions  to  physical  space.  Afterwards  there  is  a phase  of  creating,  collecting  and 
arranging  objects  for  the  mixed  reality  environment.  These  objects  can  either  be  of  digital  or  of  physical  nature.  Various  materials  for  scenery 
creation  are  used,  like  wires  and  papier-mache  (see  figure  1).  During  the  development  process  the  pupils  are  acquainted  with  the  hardware 
and  software  to  be  used. 

Children  collaboratively  create  their  own  mixed  reality  environment.  During  this  working  process  the  children  get  involved  as  active 
participants  creating  something  meaningful  to  themselves  and  to  others. 
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Figure  1 : Children  are  creating  the  objects  for  the  mixed  reality  scene 

Furthermore,  during  the  working  process  the  children  do  not  only  create  an  animation  for  the  digital  world,  but  they  also  create  the  digital 
world.  In  our  project  they  simply  use  a LEGO  Cam  for  that  task  and  LEGO  MindStorms  Vision  Command  to  program  the  interactive 
sequences  of  certain  picture  recognition  sequences  (see  figure  2). 


Figure  2:  Children  producing  an  animation  with  the  LEGO  Cam 
3»2  Scenario:  The  "World  of  Dragons"  music  revue  in  a mixed  reality  environment 

Figure  3 shows  a sketch  of  the  collaborative  mixed  reality  environment,  which  was  designed  (as  described  in  Chapter  3.1)  by  the  children  for 
a music  revue  "World  of  Dragons".  It  consists  of  an  animation,  which  shows  a wild  landscape  with  mountains,  trees,  a watercourse,  an 
active  volcano,  a cave  and  several  moving  dragons.  On  the  stage  within  the  physical  world,  there  are  several  trees  which  have  been  made 
from  the  same  material  and  look  exactly  like  the  trees  in  the  animation.  Between  the  projection  wall  and  the  video  animation  the  children 
move  and  dance  on  the  stage.  They  wear  differently  colored  dragon  costumes  and  dance  to  the  music  of  Sergey  Prokofjew  ("Romeo  and 
Juliet",  "The  Montagues  and  Capulets").  The  rhythm  of  the  music  as  well  as  the  narrative  structures  of  the  animation  impose  a certain 
timing  of  the  acoustic  and  body  expression. 
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Figure  3:  Sketch  of  the  collaborative  mixed  reality  environment  for  the  music  revue 

By  their  movements  and  by  the  colors  of  their  costumes  the  children  activate  different  sounds  in  the  mixed  reality  environment.  There  are 
also  other  sounds  coming  from  objects  in  the  mixed  reality  environment,  like  the  blowing  of  the  wind,  the  rolling  of  thunder,  the  volcanic 
eruption,  the  sound  of  an  avalanche,  the  shouting  of  dragons,  as  well  as  the  recurring  sound  of  a stumbling  little  dragon. 

3.3  How  children  learn  from  mixed  reality  environments 

The  learning  experience  of  the  children  can  be  interpreted  on  several  levels.  On  a concrete  level  of  interpretation,  children  learn  about 
concepts  and  methods  in  both  disciplines,  in  arts  and  in  computer  science  (see  figure  4). 


theoretic  models 


Computer 
Science 

iconic  programming 
with  interface  to 
physical  world 


Software: 


LEGO  MindStorms 
Robotics  Invention  System 
Vision  Command 


physical  world 


Figure  4:  Connecting  methods  of  arts  and  of  computer  science 


In  an  active  hybrid  environment  the  activities  of  the  children  ground  in  a physical  world.  They  are  engaged  in  the  manual  development  of  the 
"landscape".  They  create  and  realize  the  objects,  which  are  used  for  the  animation.  They  act  and  dance  on  the  stage  and  interact  with  the 
animation  by  their  movements  and  they  influence  the  behavior  of  the  computer  program  with  signs  like  the  color  of  their  costumes.  With 
iconic  programming  the  learners  bridge  the  gap  between  the  physical  world  with  its  multiple  meanings  and  interpretations  and  the  abstract 
world  of  models  and  algorithms  in  computer  science.  Furthermore,  an  adequate  tool  is  needed  to  realize  programming  experience  for 
children.  For  our  purposes  we  found  LEGO  MindStorms,  Robotic  Invention  System  and  Vision  Command  fitting  perfectly  as  a 
programming  environment  for  the  purposes  of  our  project. 

The  most  interesting  aspect  of  children's  learning  experiences  in  mixed  reality  environments  is  of  semiotic  nature  and  can  be  explained  with 
the  "teridentity  relation"  (x=y=z)  of  Charles  Sanders  Peirce. 
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Figure  5:  Teridentity  relation  for  the  interpretation  of  learning  experiences  in  mixed  reality  environments  (acccording  to 

Peirce) 

Whereas  in  traditional  ways  of  teaching  only  separated  areas  are  interpreted  from  a semiotic  point  of  view,  in  our  form  of  transdisciplinary 
teaching  we  try  to  relate  all  possible  signs  to  each  other.  It  does  not  matter,  whether  these  signs  are  of  iconic  nature,  or  if  they  are  indices, 
symbols,  individual  events  and  states,  or  habits  of  predictive  / relational,  propositional  or  argumentative  nature. 

If  we  interpret  the  learning  experiences  of  the  children  in  a more  systematic  and  semiotic  way,  we  get  the  following  situation  and 
interpretations.  According  to  Peirce  (HartshomeAVeiss/Burks  1997)  there  are  10  possible  kinds  of  signs,  where  Peirce  endeavored  to 
construct  a theory  of  all  possible  natural  and  conventional  signs,  whether  simple  or  complex.  This  theory  is  helpful  to  explain  how  different 
levels  of  interpretation  and  learning  experience  are  related  to  each  other.  We  do  not  want  to  classify  the  elements  of  the  transdisciplinary 
learning  environment  according  to  these  10  classes  of  Pierce,  because  this  conclusion  would  neglect  a lot  and  shift  the  focus  to  the  theory. 
Instead  we  would  like  to  focus  to  very  important  practical  aspects: 

■ experience  in  design  and  creation  of  the  narrative  story;  the  dragons  mirror  the  fascination  and  the  anger  of  the  world  of  the  dragons  as 
well  as  the  kind  of  relationships  between  the  children  and  their  appraisals 

■ experience  of  the  relationship  to  the  body,  to  corporeality  and  being  touched  by  others,  especially  children  of  the  other  sex 

■ experience  of  elements  of  mimic  and  dancing  expressions  (which  exists  independently  of  verbal  expressions) 

■ empathy  in  music  and  rhythm,  with  its  attributes  like  loud  and  soft,  fast  and  slow,  pretty  and  grave 

■ experience  of  the  quality  of  material  during  the  creation  of  the  landscape  (which  is  either  light  or  heavy,  hard  or  soft,  dry  or  slippery) 
and  examination  of  the  shaping  of  the  three  dimensions  of  the  room 

■ experience  of  the  pictorial  nature  and  modelling  process  during  the  creation  of  the  digital  animation,  when  single  picture  sequences  are 
constructed  Irom  gradually  changing  the  real  world  models 

■ identification  of  characteristic  sounds  and  their  integration  into  the  narrative  and  expressive  dance  story  of  the  music  revue 

■ experience  in  abstract  information  processing  for  conception  and  development  of  the  picture  recognition  program  with  LEGO-RIS 
Vision  Command 

■ reflection  on  the  different  relationships  of  boys  and  girls  to  dancing  as  well  as  to  computer  technology 

■ experience  on  how  attribute  of  objects  in  the  real  world  (like  colour,  movement,  position  in  the  real  world  space)  cause  reactions  in  the 
virtual  world,  that  is  on  the  states  and  effects  of  the  computer  program 


4 Conclusions 

In  our  teaching  attempt  we  found  that  mixed  reality  environments  are  a very  promising  learning  space  for  a constructive  and  collaborative 
learning  of  children.  The  mixed  reality  environment  meets  all  requirements  for  a constructive  learning  space  for  children,  like  learning  through 
senses  and  physical  activity  using  digital  manipulatives. 

In  designing  and  in  working  with  a mixed  reality  environment,  the  children  construct  a configuration  of  the  world  instead  of  a describing 
knowledge.  They  do  this  by  individually  creating  and  constructing  the  world  and  by  subsequently  freely  interpreting  their  self-created 
world.  During  the  performance  of  the  music  revue  the  children  experience  a singular  event.  However,  for  every  event  numerous 
interpretations  are  possible.  By  interpreting  the  world  the  children  do  not  create  an  analytical  but  an  intuitive  understanding.  The  children 
are  engaged  in  the  world  of  constructing  the  objects,  in  the  world  of  programming  and  in  the  world  of  behavior.  They  write  programs  and 
they  control  the  computer  program  with  their  behavior  (dancing,  movements,  colors  of  their  costumes  etc.).  By  this  the  children  become 
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themselves  variables  in  the  computer  program  when  acting  in  the  world  of  the  musical  revue.  They  act  themselves  as  signs  with  different 
states  and  effects  on  each  other  as  well  as  on  the  elements  of  the  real  and  the  virtual  world. 

Further  research  will  explore  the  use  of  other  computer-based  learning  environments  as  tools  and  environments  for  trans-disciplinary 
teaching  and  understanding  in  the  field  of  arts  and  computer  sciences.  The  research  will  be  done  on  different  levels  of  education  and  age. 
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SCHEDULING  AND  MONITORING  DYNAMIC  LEARNING  OBJECTS  ON  THE  WEB 


JJ.  KUISMA,  J.A.  WATSON 
NTP  Information  Solutions  Ltd,  University  of  Huddersfield 


1.  ABSTRACT 

Next  generation  web  Learning  Management  Systems  (LMS)  will  deliver  courses  built  from  small  re- 
usable learning  objects  and  use  systems  to  tailor  curricula  so  that  the  learning  needs  of  each  individual 
student  can  be  met  in  an  automated  and  cost  effective  way.  Web  based  Intelligent  Tutoring  Systems 
(ITS)  aimed  at  compensating  for  the  absence  of  a human  tutor  have  to  date  mostly  concentrated  on 
providing  assistance  in  a particular  subject  domain  and  not  focussed  on  the  problem  of  utilising  content 
represented  as  learning  objects  across  multple  subject  domains.  This  paper  describes  the  creation  of  a 
non -dictatorial  tutoring  system  that  employs  a ‘learning  navigator’  to  help  trainees  visualise  the  best 
path  through  assignable  units  extracted  from  a learning  repository.  The  system  takes  a unique  approach 
to  deliver  personalised  leamingby  taking  advantage  of  the  SCORM  specification  and  storing  content  as 
Sharable  Content  Objects  (SCOs),  and  therefore  enabling  the  use  of  existing  and  third  party  content. 

2 PARTNERS 

This  paper  discusses  a collaborative  project  between  NTP  Information  Solutions  (NTPIS)  and  the 
University  of  Huddersfield  that  has  been  set  up  to  investigate  different  ways  to  enhance  web  based 
learning  environments.  NTP  Group  is  one  of  the  largest  multi-occupational  training  providers  in  the 
UK  Serving  25,000  trainees  throughout  the  UK.  Together  with  the  four  other  companies  in  the  NTP 
Group,  NTPIS  delivers  a range  of  management,  training  and  consultancy  services  for  UK  enterprises. 
A key  part  of  the  company’s  strategy  is  to  provide  e-training  solutions  directly  to  the  work  place  across 
diverse  sectors.  At  NTPIS  creative  designers  use  authoring  tools  such  as  Trivantis  Lectora  and 
Macromedia  Flash  to  produce  learning  applications  in  the  form  of  Sharable  Content  Objects  (SCOs) 
that  are  then  delivered  as  courses  through  a Learning  Management  System  (LMS).  This  method 
follows  the  SCORM  standard  that  facilitates  interoperability  of  learning  content  between  systems.  The 
University  of  Huddersfield  has  a track  history  of  research  into  the  monitoring  and  scheduling  of 
Computer  Based  Training  (CBT)  applications  delivered  over  the  Internet  and  delivers  several  of  its 
MSc  courses  using  the  Blacld>oard  LMS.  Both  organisations  are  aware  of  the  low  completion  rates 
associated  with  e- Learning  and  are  seeking  ways  to  combat  this  by  enhancing  LMSs. 

3.  OVERALL  VISION  AND  GOALS 

NTPIS  wish  to  develop  a total  Intemet/intranet  training  solution  for  their  customers.  This  includes 
developing  the  next  generation  of  their  LMS  to  include  tutoring  and  guidance.  Their  vision  is  to 
provide  a one-stop  shop  that  will  help  their  customers  to  grow,  by  the  use  of  improved  management 
information  systems  and  by  investing  to  it’s  most  valuable  capital;  the  employees  and  their  knowledge. 
The  next  generation  LMS  will  not  just  provide  a training  delivery  platform  but  also  act  as  the  central 
point  of  access  for  getting  help  and  advice.  An  agent  will  be  integrated  into  the  LMS  that  will  contain  a 
level  of  intelligence  to  answer  trainee’s  questions  and  be  able  to  direct  trainees  to  the  resources  that 
they  need.  It  \s^l  monitor  activities  and  determine  common  learning  and  browsing  patterns  and  suggest 
the  best  approach  for  each  individual  user  based  upon  their  profile  . 

This  p^er  focuses  on  the  problem  of  delivering  personalised  training  for  the  individuals  in  a manner 
that  matches  the  requirements  of  industry.  It  describes  a system  that  uses  a generic  domain  knowledge 
model  and  reusable-learning  objects  to  provide  cost-effective  way  to  provide  individualised  learning 
experiences.  The  learning  objects  are  authored  to  comply  with  the  Sharable  Content  Object  Reference 
Model  (SCORM)  specification  to  provide  the  interoperability  across  the  LMS  and  content  vendors.  The 
system  extends  the  SCORM  by  mapping  the  Sharable  Content  Objects  (SCO)  to  the  learning  outcomes 
in  the  system’s  knowledge  model.  Small  reusable  SCOs  are  published  in  the  data  repository  and 
combined  dynamically,  at  runtime.  The  system  provides  guidance  to  trainees  by  suggesting  appropriate 
learning  paths  through  the  content  and  gives  information  about  the  relevancy  of  SCOs  depending  on 
the  profile  of  the  trainee  and  their  individual  circumstances. 
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4. 


BACKGROUND 


The  use  of  e-learning  has  increased  enormously  during  the  last  few  years  and  even  the  most  modest 
estimations  predict  huge  growth  in  the  future.  The  main  reasons  for  this  have  been  based  on  the  Martini 
principle  of  delivering  training  anytime  and  anywhere,  giving  cost-savings  on  the  reduced  offeite  time, 
travel  and  accommodation  expenses  and  trainer  costs.  However,  a disadvantage  of  this  approach  is  that 
e-leaming  can  often  lead  to  an  isolated  experience  and  not  deliver  sufficient  tutoring  or  guidance.  This 
factor  has  often  been  linked  to  poor  completion  rates  for  online  courses.  In  a traditional  classroom 
scenario  training  tends  to  be  pitched  to  the  average  level  of  the  people  attending  the  class.  This  level 
may  not  suit  all  and  training  individually  can,  sometimes,  provide  an  advantage  if  training  is  tailored  to 
an  individual’s  specific  needs.  CBT  and  e-leaming  provides  direct  access  to  each  individual  being 
trained  and  should,  in  theory,  be  superior  to  the  broadcasting  methodology  used  in  the  traditional 
classroom  if  mechanisms  can  be  put  in  place  to  provide  an  individualised  experience.  However,  the 
benefits  of  this  have  not  yet  been  exploited.  Regardless  of  the  fact  that  the  training  is  done  individually 
and  at  any  time  the  trainee  prefers,  the  delivered  training  content  is  still  static  for  the  trainee.  The  same 
limitation  applies  to  the  majority  of  hypermedia,  as  web  content  is  often  the  same  static  information 
presented  to  each  reader  (Brusilovsky  2001).  Academic  research  has  delivered  some  pedagogically 
valid  systems  that  pitch  the  level  of  training  into  the  level  of  the  trainee  but  they  have  been  bespoke 
applications  for  a certain  knowledge  domain  (Mullier  et  al.  2001)  Also,  the  industry  has  developed 
several  generic  domain  user  modelling  systems  (Kobsa  2001)  but  nothing  focussed  purely  on  training 
and  education.  NTPIS  require  a system  that  can  tailor  content  extracted  from  a repository  of  re-usable 
SCOs  to  a large  diverse  audience.  To  provide  a cost  effective  solution  for  the  enterprise  the  content  has 
to  be  authored  for  the  industry  standards  so  that  it  can  be  reused  and  interchanged  across  the  platforms. 

The  Advanced  Distance  Learning  (ADL)  is  an  initiative  from  the  US  Department  of  Defense  (DoD), 
which  is  aimed  for  promoting  the  co-operation,  between  government,  academia  and  business  to 
develop  o-leaming  standardisation.  This  initiative  has  resulted  in  actions  that  have  delivered  the 
Sharable  Content  Object  Reference  Model  (SCORM),  which  is  currently  on  version  1.2.  The  SCORM 
defines  a web-based  learning  “Aggregation  ModeF’  and  “Run-time  Environment”  (RTE)  for 
learning  objects.  The  aggregation  model  specifies  how  to  aggregate  the  resources  into  the  structured 
learning  content.  The  RTE  defines  the  common  language  and  data  model  for  the  communication 
between  the  LMS  and  the  content.  This  enables  the  true  interoperability  between  the  content  and  LMSs 
developed  by  the  different  vendors.  SCORM  has  been  the  specification  that  has  brought  together  pieces 
of  the  learning  specifications  made  by  other  organisations,  like  Aviation  Industry  CBT  Committee 
(AlCC),  Alliance  of  Remote  Instructional  Authoring  & Distribution  Networks  for  Europe  (ARIADNE) 
and  the  Institute  of  Electrical  Electronics  Engineering  (IEEE)  Learning  Technology  Standards 
Committee  (LTSC).  NTPIS  has  recognised  the  advantages  offered  by  following  official  open  standards, 
and  developed  it’s  infrastructure  to  implement  the  SCORM  1.2  specification.  The  ADL  initiative 
envisions  use  of  the  training  content  to  be  developed  as  small  reusable  Sharable  Content  Objects 
(SCO),  which  could  be  stored  in  a repository  used  in  various  different  courses.  The  first  versions  of  the 
SCORM  specification  have  already  been  well  adopted  by  the  industry  and  implemented  in  most  of  the 
leading  LMSs.  ADL  takes  the  vision  of  interoperability  and  re -usability  further,  in  it’s  long  range 
vision;  once  the  SCO  are  commonly  available  they  can  be  assemble  in  the  real  time,  on  demand  and 
then  delivered  to  learners.  (ADL  SCORM  2001) . 

This  research  has  developed  the  system  that  implements  dynamic  scheduling  by  associating  SCOs  with 
the  data  of  its  learning  outcomes  and  by  developing  the  dynamical  routing  engine  that  can  suggest  to 
the  trainee  the  different  learning  paths  that  can  be  taken  to  achieve  the  same  learning  objectives. 

5.  SYSTEM  ARCHITECTURE 

NTPIS  have  developed  a web  based  LMS  that  acts  as  the  infrastructure  for  delivering,  managing  and 
monitoring  training  in  the  work  place.  The  system  contains  various  interfaces  that  allow  course  authors, 
tutors,  managers  and  trainees  to  create  a customisable  training  portal  that  displays  the  user  with  the 
menus  and  menu-items  based  on  the  given  rights.  When  trainees  access  this  system  they  are  presented 
with  a menu  showing  the  courses  that  they  are  enrolled  on,  where  they  are  up  to  and  what  they  need  to 
do  next.  Content  is  developed  using  authoring  tools,  such  as  Trivantis  Lectora  and  Macromedia  Flash, 
in  the  form  of  SCOs  and  submitted  into  a learning  repository  using  the  Lesson  Management 
Interface . A training  manager  uses  the  Course  Management  Interface  to  create  courses  by  defining 
paths  between  the  SCOs  that  are  present  in  the  repository.  Work  completed  so  far  has  focussed  on 
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extending  these  interfaces  and  including  a interface;  the  Learning  Navigator  Interface.  The 
objective  of  the  Learning  Navigator  is  to  provide  a graphical  way  to  visualise  the  contents  of  the 
learning  repository  and  the  structure  of  the  course  the  student  is  following.  The  interface  allows  the 
trainee  to  not  only  see  the  SCOs  that  form  part  of  the  course  that  they  are  enrolled  on,  but  also  any 
other  relevant  SCOs.  The  trainee  can  also  view  several  different  paths  through  to  SCOs  ranging  from 
how  the  course  was  originally  authored  to  what  the  system  considers  is  the  best  path  for  the  user  to  take 
in  their  particular  situation, 


5.1 


THE  Lesson  management  interface 


The  lesson  management  interface  (Fig,  1)  is  an 
access  point  to  the  learning  object  repository  and 
provides  the  tools  necessary  for  editing  and 
registering  SCOs  into  the  system.  The  SCORM 
Aggregation  Model  defines  the  basic  SCO 
information,  such  as  name,  description  and 
keywords.  We  have  extended  this  information  by 
assigning  the  SCO  with  weighted  learning 
outcomes  (skills).  These  learning  outcomes  are 
predefined  in  the  system  and  have  been  related  to 
each  other  in  a hierarchy,  which  consists  of  skills 
categories  in  a tree  type  structure . 


5.2 


THE  COURSE  MANAGEMENT  INTERFACE 


The  course  management  interface  (Fig.  2)  is  used 
to  create  courses  from  SCOs  in  the  repository. 
Courses  are  assigned  with  learning  outcomes  that 
are  expected  to  be  achieved  by  taking  a course. 
Like  the  skills  associated  into  the  SCOs,  these 
outcomes  are  mapped  to  the  same  skill  hierarchy. 
Course  tutor  uses  his  expertise  to  use  learning 
repository  to  combine  linear  series  of  SCOs  that 
fulfil  these  learning  outcomes..  This  forms  a basic 
course  structure  and  is  called  a default  Learning 
Path. 


r#»f 


Figure  1:  The  lesson  management  interface 


Figure  2:  The  course  management  Interface 


5.3  LEARNING  NAVIGATOR 

The  Learning  Navigator  (Fig.  3)  is  an  interface  that 
is  used  by  the  trainee  when  proceeding  through  the 
courseware.  It  presents  the  trainee  with  the  course 
structure  that  consists  of  Stage  SCOs  and 
Proximate  SCOs.  The  stage  SCO  is  a learning 
object  that  has  been  authored  to  be  part  of  the 
course.  The  Proximate  SCOs  are  dynamically 
pulled  out  from  the  learning  object  repository  and 
recommended  for  the  trainee,  based  on  the  skill 
relations  that  are  ass  igned  to  the  Stage  SCOs  and 
what  is  the  trainee’s  knowledge  on  the  those  skills. 
In  addition,  it  presents  the  Learning  Path 
recommendations  made  by  the  Routing  Engine. 


Examples  of  skills  of  the  SCOs  in  the  authored  path  (Stage  SCOs) 


Examples  of  skills  on  the  related  to  the  SCOs  in  the  authored  path  (Proximate  SCOs) 


Figure  3:  The  Learning  Navigator 


The  example  demonstrated  above  in  figure  3 is  part  of  a course  on  ''Protecting  individuals  form  abusive 
behaviour'".  There  are  three  stage  SCOs  , from  which  each  has  been  grouped  together  with  the 
additional  Proximate  SCOs.  If  the  path  viewed  had  been  the  default  path  (original  path  authored  by  the 
training  manager)  the  path  would  be  a straight  line.  However,  the  path  deviates  from  this  and  shows  a 
path  that  has  been  chosen  by  the  Routing  Engine.  The  Learning  Navigator  is  also  a trainee’s  tool  to  get 
all  the  necessary  information  about  the  course  structure  and  it’s  different  entities.  This  information  can 
be  divided  in  to  two  parts,  the  visual  data  and  the  text -based  data.  The  text -based  data  is  mainly 
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displayed  in  roll-over  popup  windows  and  contains  detailed  information  about  the  SCOs,  Skills,  and 
course  outcomes  (e.g.  Name,  description  etc).  The  visual  data  can  be  listed  as: 

• Stage  SCOs 

• Proximate  SCOs 

• Course  learning  outcomes 

• Proximate  SCO  relations  to  Stage  SCO 

• Relevancy  of  the  SCO  (Which  depends  on  the:) 

o Trainee  level  of  the  SCOs  (Skills  associated  for  the  SCOs) 
o SCO  relevancy  towards  learning  outcomes  (Mapped  using  skills  hierarchy), 
o SCO  relevancy  towards  the  Stage  SCO(Mapped  using  skills  hierarchy) 

• Different  Paths  with  the  order  of  the  SCOs 

• Current  course  position 


5.4  ROUTING  ENGINE 


The  decisions  made  by  the  Routing  Bigine  follow  loosely  the  framework  used  by  Karagianidis  et  al 
(2001a)  . This  defines  the  layered  structure  for  the  evaluation  for  adaptive  and  personalised  learning 
services,  which  uses  two  distinct  high  level  processes;  Interaction  Assessment  and  Adaptation  Decision 
Making.  The  Interaction  Assessment  phase  makes  the  high-level  conclusions  from  the  learner’s 
interactions  during  their  learning.  These  conclusions  are  analysed  by  the  Adaptation  Decision  Making 
component,  which  personalises  the  content  for  the  trainee.  The  NTP  Routing  Engine  (Fig.  4)  applies 
this  framework  in  the  manner  that  complies  with  the  requirements  of  the  SCORM  specification.  The 
Interaction  assessment  has  been  implemented  in  the  SCO  Performance  Assessment  (SPA)  module. 
This  component  makes  conclusions  based  on  the  communication  interchange  using  the  SCORM  RTE, 
rather  than  monitoring  lowlevel  events  such  as  keystrokes  and  mouse  clicks.  Conclusions  such  as  ""The 
trainee  Mike’  has  achieved  47%  level  on  the  "Protecting  individuals  form  abuse'  -skill\  are  based  on 
trainee’s  prior  knowledge  and  analysis  of  SCORM  data  model  items  lesson_^tatus^  score  and 
interactions  collection.  The  benefit  of  this  approach  is  that  the  learning  content  can  be  authored  to  the 
SCORM  standard  and  all  objects  in  the  repository,  whatever  their  subject  domain,  can  be  scheduled  by 
the  system.  Some  limited  extensions  to  the  SCORM  standard  had  to  be  made  so  that  SCOs  could  be 
linked  to  specific  skills.  Karagianidis’  Adaptation  Decision  Making  component  is  implemented  in  the 
Pathway  Generator  (PG)  module.  This  module  passes  the  information  about  the  Stage  SCOs, 
Proximate  SCOs  and  Pathways  to  the  Learning  Navigator.  The  possible  paths  could  include: 


• Original 

• Generic 

• Fastest 

• Suggested 

• Popular 


One  that  was  authored 

With  SCOs  that  will  generate  widest  range  of  knowledge 
Based  on  time  taken  by  the  previous  uses 

Contains  SCOs  that  best  fit  the  users  performance  and  entry  profile 
Based  on  what  paths  have  previous  users  selected 


The  PG  recommends  new  paths  by  assessing  how  relevant  SCOs  are  to  the  learning  outcomes  of  the 
course  and  the  Stage  SCOs.  These  relevancies  are  counted  by  taking  the  sum  of  the  linked  skill 
elements,  which  are  counted  by  dividing  the  amount  of  the  skill  association  percentage  {A)  by  the 
trainee’s  knowledge  level  on  the  skill  (K).  Each  type  of  elements  has  a factor  0 that  can  be  used  to 
adjust  the  importance  of  the  different  type  of  relationships.  Following  formula  is  used  to  count  the 
Proximate  SCO  relevancies.  The  Stage  SCO  relevancies  ignore  the  last  sum,  which  represents  the 
Proximate  SCO  skills  that  are  linked  to  a specific  Stage  SCO . 


allskills 

A ^ 

r ^ps 

j.  ^ 

L i 

linkedtogoals 

^ 1 

f/^1 

K"  K 

linked  tost  ages  cos 

S 1 

(f 

J IsS  -wr 

^ -I ^ -p 

^all  skills  ^linked  to  goals  ^ linked  tostages  cos 

lOBl 


Figure  4:  The  high  level  architecture  of  the  Routing  Engine 
5.5  Implementation  architecture 


The  system  has  been  implemented  using  a client  side  Java  Applet  as  the  SCORM  API  and  the  Learning 
Navigator,  providing  the  graphical  user  interface  and  interactivity  required  by  the  Learning  Navigator. 
The  functionality  of  the  Routing  Engine  and  all  database  access  was  left  on  the  server  side  Java  Servlet, 
implementing  a scalable  distributed  client-server  framework  (figure  5).  Even  though  there  are  other 
technologies  to  implement  the  required  system  (Sherman  2000),  this  framework  was  chosen  because  of 
the  followingbenefits: 

• Java  Applet  can  be  used  to  implement  high  quality  interactive  graphical  interface 

• Faster  response  times  compared  to  http -CGI  architecture 

• Logical  functionality  mainly  in  the  server  side  minimises  the  download  size  of  applet, 

• Distributed  architecture  provides  better  scalability  and  maintainability 


Figure  5:  The  implementation  architecture 

6.  VALIDATION 


The  proposed  system  will  be  validated  and  analysis  will  be  made  whether  the  pedagogical  benefits 
match  the  estimations  and  if  the  logic  of  the  system  is  suggesting  better  paths  for  the  user.  The 
intention  is  to  follow  the  same  validation  process  as  Karagiannidis,  Sampson  and  Brusilovsky  (2001b). 
Validation  will  focus  on  three  areas 

• Overall  success  towards  increased  learning  i.e.  comparison  with  and  without  the  Learning 
Navigator  facility. 

• Whether  the  SPA  module  can  provide  enough  information  to  the  pathway  generator  such  that 
useful  decisions  can  be  made. 

• Success  of  Proximate  SCO  and  pathway  recommendations  of  the  Pathway  generator 

The  granularity  of  the  adaptation  is  dependent  on  the  size  of  learning  objects  and  the  effect  of  this  will 
need  to  be  examined  In  addition,  we  will  need  to  measure  how  much  users  experience  is  affected  when 
presented  with  SCOs  that  contain  various  different  navigation  panels  and  graphical  user  interfaces.  This 
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can  be  avoided  by  the  use  of  standard  template,  but  when  SCOs  from  various  sources  are  used,  the 
outcome  might  be  disturbing,  ADL  envisions  including  the  templates  to  the  scope  of  future  versions  of 
SCORM, 


7. 


CONCLUSION 


This  paper  has  discussed  how  a LMS  can  be  extended  to  provide  individualised  tuition.  There  is  little 
doubt  that  a relevant  individualised  experience  should  lead  to  an  enhanced  training  experience.  E 
learning  is  an  ideal  vehicle  for  deployment  of  individualised  training  because  it  provides  direct  access 
to  each  individual.  We  have  demonstrated  how  courses  can  be  represented  as  small  discrete  re-usable 
objects,  stored  in  a learning  repository,  retrieved,  and  allocated  dynamically  at  runtime.  The  e -Learning 
industry  requires  a cost  effective  solution  of  using  existing/3rd  party  content  and  standard  authoring 
tools  to  develop  content.  In  addition,  e- learning  specifications,  such  as  SCORM,  should  be  used  to 
provide  interoperability  between  systems.  At  its  current  version  SCORM  requires  slight  extension  to 
provide  mechanism  for  delivering  individualised  training.  Mapping  the  SCOs  into  a LMS  skills 
architecture  enables  the  PG  to  recommend  learning  objects  outside  the  original  linear  course  structure 
on  the  basis  of  their  relevancy.  Still,  the  analysis  of  the  users  interactions  inside  a SCO  does  not  reflect 
towards  a specific  skill,  but  towards  all  skills  associated  in  the  SCO.  The  continued  proliferation  of  e- 
leaming  solutions  and  the  increasing  number  of  trainees  using  LMSs  means  that  the  functions,  which 
this  project  has  provided  could  play  an  essential  role  in  increasing  learning  effectiveness  in  the 
workplace 

We  have  demonstrated  how  courses  can  be  represented  as  small  discrete  re-usable  SCOs,  stored  in  a 
learning  repository,  retrieved,  and  allocated  dynamically  at  runtime. 
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Writing-4- Sharing — Web-based  Writing  Support  System 


Kung,  ShiaoChuan  & Liang,  Ting-Wen 
Wenzao  Ursuline  College  of  Languages 
National  Sun  Yat-Sen  University 
Kaohsiung,  Taiwan 


Introduction 

This  paper  describes  a web  - based  application  that  facilitates  the  meeting  of  pedagogical  goals  of 
process-oriented  communicative  writing.  It  supports  the  teaching  strategies  used  in  an  EFL  (English  as  a 
Foreign  Language)  writing  class  and  allows  the  construction  of  electronic  portfolios.  The  system  allows 
uploading  and  exchange  of  electronic  files  through  the  World  Wide  Web  and  performs  administrative 
tasks.  It  makes  possible  the  smooth  completion  of  collaborative  writing,  process  writing  and  peer 
reviewing  activities,  all  essential  components  of  the  curriculum  of  this  writing  course. 

Rationale 

The  “Writing-4- Sharing”  system  was  developed  in  response  to  a dire  need  to  manage  the  sizeable 
numbers  of  papers  in  large  writing  classes  efficiently.  One  of  the  difficulties  encountered  by  teachers  of 
writing  in  Asia  is  the  large  number  of  students  in  a class.  It  is  not  uncommon  for  an  EFL  teacher  to  be 
facing  classes  of  30  to  50  students.  While  collaborative  writing,  drafting  and  revising,  and  peer 
reviewing  are  techniques  that  are  frequently  used  in  writing  classes,  they  require  that  the  teacher  keep 
track  of  all  the  papers  and  all  the  changing  team  members  of  each  assignment,  thus  burdening  the 
teacher  with  menial  record-keeping  tasks.  A system  that  can  track  assignments,  assign  grades  to  all  the 
members  of  a team  and  publish  the  students’  work  in  an  electronic  space  where  all  the  students  can  view 
each  other’s  work  and  the  corrections  made  by  the  teacher  can  support  the  teaching  strategies  mentioned 
and  reduce  the  number  of  administrative  tasks  that  the  teacher  needs  to  perform. 

Description  of  the  System 

“Writing- 4- Sharing”  was  created  at  the  Wenzao  Ursuline  College  of  Languages  in  Taiwan  and  is 
currently  being  used  in  a freshman  level  writing  class  of  English  majors.  The  system  allows  students  to 
submit  drafts  and  completed  assignments  through  the  World  Wide  Web  and  the  teacher  to  download 
them,  point  out  the  locations  of  the  errors  and  send  the  corrections  back  also  by  uploading  the  papers 
onto  the  Web.  This  cycle  can  be  repeated  as  many  times  as  drafts  are  required.  The  teacher  can  therefore 
electronically  monitor  individual  or  group  writing  processes  from  the  brainstorming  phase  to  the  final 
draft.  The  system  also  enables  students  to  see  each  other’s  papers,  thus  facilitating  collaborative  writing 
and  peer  editing  as  well  as  viewing  of  the  comments  offered  by  the  teacher  to  all  the  papers.  In  addition, 
the  program  tracks  the  papers  submitted  by  every  student  and  keeps  record  of  all  the  grades. 


After  logging  in,  students  are  shown  a personalized  screen  from  which  they  can  choose  to  view 
announcements,  participate  in  discussions,  view  their  portfolios,  consult  online  resources,  upload  or 
download  assignments,  or  check  feedback  given  by  a peer  reviewer  or  the  teacher. 


The  teacher*s  interface  allows  her  to  post  announcements  or  course  materials.  She  can  also  start 
discussion  threads  or  recommend  valuable  online  resources. 


The  program  allows  the  teacher  to  see  groupings  of  papers  submitted  by  student  name  or  by  project 
name.  She  can  download  the  papers  and  use  features  of  Microsoft  Word  such  as  underlining, 
commenting,  or  coloring  to  mark  places  in  it  where  correction  is  needed.  When  the  errors  have  been 
marked,  she  can  upload  the  papers  back  onto  the  Web  next  to  the  original  paper.  The  system 
automatically  renames  each  file  when  it  is  uploaded  or  downloaded  according  to  a numerical  scheme  so 
that  there  are  no  problems  with  multiple  copies  of  the  same  file  name. 


“Writing- 4- Sharing”  also  allows  the  teacher  to  assign  students  to  work  individually  or  as  a team.  She  can 
assigi  students  to  different  groups  so  that  students  can  collaborate  on  different  projects  with  different 
partners.  When  a grade  is  given,  it  gets  posted  to  all  the  members  of  the  team.  The  teacher  can  also 
assign  students  to  peer  review  each  other’s  papers  or  to  view  model  papers  without  making  additional 
paper  copies. 
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Using  Technology  and  Constructivism  in  Business  Law: 
First  Year  Lessons 


Richard  G.  Kunkel,  J.D. 
Department  of  Legal  Studies  in  Business 
University  of  St,  Thomas,  St.  Paul,  Minnesota 
United  States 
rgkunkel@stthomas.edu 


Abstract:  Converting  to  new  teaching  methods  incorporating  technology  and  constructivist  principles 
can  be  a daunting  challenge  for  professors  and  students  alike.  This  paper  will  present  the  author’s 
experiences  in  applying  technology  and  constructivism  to  teach  Legal  Studies  in  Business.  Students 
worked  on  realistic  business  problems  that  required  them  to  conduct  online  legal  research  in  an 
assigned  U.S.  state.  The  students  used  WebCT  to  post  research  results  online;  summarize  and  discuss 
the  law,  and  collaborate  with  fellow  students  to  reach  solutions.  The  conversion  to  these  new  methods 
introduced  several  new  layers  of  complexity  to  the  teaching  and  learning  process,  including  use  of  legal 
research  techniques,  bulletins  boards,  and  new  ways  of  evaluating  student  performance.  Several 
strategies  have  been  developed  for  easing  the  transition  for  new  users  of  these  technologies,  and  for 
enhancing  the  author’s  second  year  of  teaching  with  this  method. 


Constructivist  learning  principles  call  for  teaching  methods  that  are  problem-based,  realistic,  ill-structured,  and  invite 
consideration  of  solutions  from  multiple  perspectives  (Dunlap  1999).  The  emergence  of  substantial  legal  information 
online  has  created  new  opportunities  for  problem-based  learning  about  real-life  situations  by  using  the  Internet  research 
tools  available  in  the  real  world. 

The  most  distinct  feature  of  the  Legal  Studies  in  Business  environment  is  that  applicable  legal  rules  vary  widely  from 
one  jurisdiction  to  another  (unlike  laws  of  physics,  chemistry  or  other  disciplines).  Thus,  business  law  provides  a unique 
context  for  using  multiple  perspectives  to  engage  in  problenvbased  learning. 

To  capitalize  on  this  opportunity,  the  author  assigned  each  student  to  a specific  U.S.  state,  and  required  the  students  to 
conduct  online  legal  research  in  the  assigned  state  to  solve  a realistic  business  problem.  The  students  also  were 
assigned  to  a group  of  four  fellow  students.  Using  a WebCT  bulletin  board,  the  group  was  required  to  collaborate  to 
reach  a group  solution  the  business  problem,  even  though  the  legal  rules  varied  in  the  students’  assigned  states.  This 
process  required  students  to  analyze  and  synthesize  the  rules  of  law,  and  to  deal  with  the  complexity  that  is  a part  of  the 
real  world  legal  environment.  Making  the  ransition  to  this  teaching  method  presented  several  challenges  for  students 
and  the  instructor. 


Complexity. 

This  method  introduced  a much  higher  degree  of  complexity  for  students  compared  to  traditional  lecture  environments. 
Students  were  challenged  to  learn  how  to  use  the  Internet  tools  for  research,  how  to  use  the  Lexis  -Nexis  legal  research 
database,  how  find  legal  information  online,  how  to  read  and  analyze  cases  competently,  how  to  reconcile  conflicting 
rules  of  law  in  multiple  states,  how  to  use  WebCT,  and  how  to  collaborate  effectively  online.  For  many  students, 
dealing  with  the  added  complexity  in  one  or  more  of  these  areas  was  a difficult  obstacle  that  interfered  with  learning. 
During  this  inaugural  course,  insufficient  support  materials  were  provided  to  prepare  students  for  this  adjustment  and  to 
lessen  complexity.  In  the  future,  significant  explanatory  materials  (e.g.  “How  to  Read  a Case”,  “How  to  Handle 
Information  in  Lexis -Nexis”)  must  be  provided. 


Ambiguity. 
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In  our  educational  culture  students  expect  and  prefer  specific  information  about  course  assignments  and  their  weight  in 
grading  from  the  opening  days  of  the  course.  In  preparing  the  first  attempt  of  this  new  teaching  method,  it  was  difficult 
to  predict  student  performance  on  problem-based  learning,  and  thus  difficult  to  state  a specific  number  or  problems  or 
their  relative  weight.  Two  choices  were  possible.  Be  specific  and  lose  flexibility  to  make  adjustments  as  needed  based 
on  student  performance,  or  be  ambiguous,  thereby  maximizing  flexibility  but  increasing  student  uncertainty. 

In  this  first  effort,  the  number  and  weight  of  the  assignments  was  left  ambiguous.  Students  were  asked  to  trust  the 
instructor’s  judgment  that  the  number  of  assignments  would  be  reasonable  based  on  their  performance  and  that  the 
relative  weight  of  assignments  would  be  adjusted  at  the  end  of  the  semester  as  fairness  dictated.  Many  students 
struggled  with  this  ambiguity  regarding  their  workload  and  grading  to  a surprising  degree.  For  some  students  it 
interfered  with  course  performance. 

To  reduce  ambiguity,  instructors  should  be  as  specific  as  possible  while  retaining  some  flexibility.  One  approach  would 
be  to  split  the  syllabus  into  two  parts:  a first  part  with  specific  numbers  of  assignments  and  grading,  and  a second  part 
that  will  be  pursued  and  adjusted  if  student  performance  permits. 


Clarity  of  Instructions. 

Constructivist  principles  hold  that  problems  should  be  ill -structured  and  that  learners’  have  a high  degree  of  autonomy. 
Accordingly,  few  restrictions  were  placed  on  how  students  organized  the  work  of  their  groups  and  the  form  their 
responses  should  take.  Instructions  were  general  and  often  given  verbally  in  class. 

Students  and  groups  that  struggled  in  their  performance  often  cited  the  “lack  of  instructions”  or  “misunderstanding”  of 
instructions  as  a reason  for  their  poor  performance.  Students  expect  specifics  on  what,  where,  when  and  how  to 
perform,  and  several  groups  had  difficulty  exercising  their  autonomy  in  this  area.  Confronted  with  the  challenge  of 
problem-based  learning,  some  groups  appeared  to  use  lack  of  specific  instructions  as  a “cop  out”  for  poor  performance. 
Specific  and  clear  written  instructions  should  be  given  early  in  the  course  to  set  patterns  for  performance,  and  learner 
autonomy  can  be  increased  as  the  semester  progresses. 


Workload  and  Feedback. 

This  approach  requires  the  instructor  to  assess  a large  volume  of  messages  and  legal  information  posted  in  WebCT.  The 
method  requires  good  methods  for  evaluating  student  posting,  keeping  adequate  records  and  posting  feedback  to 
students.  Disciplined,  consistent  attention  to  the  WebCT  bulletin  board  is  essential.  Students  found  it  difficult  to  move 
forward  on  further  assignments  until  sufficient  feedback  was  received.  A lackof  timely  feedback  was  a strong  source 
of  student  frustration.  Consistent  and  timely  assessment  is  an  essential  component  of  the  method. 
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Interface  Design,  Functionality  and  Course  Evaluation  Quality  Assurance  for  AGenetal  Education  Course  at  the  University  of 

California.  Davis 

Maria  Loma  A.  Kunnath,  Ed.D.^ 


Abstract 

This  paper  discussed  how  a Mediawork^  online  course  was  checked  for  quality  assurance  focusing  on  interface  design,  features, 
functionalities  and  performance.  Recommendations  to  improve  QA  practice  concluded  the  paper. 

Interface  Design  Features  with  Implications  to  Learning 

Moby  course  development^  uses  a familiar  interface.  The  main  LOGIN  screen  contains  an  embedded  watermark  of  the  UCDAVIS  logo 
as  a main  background  and  two  boxes  where  users  input  their  username  and  password.  The  Main  Page  contains  two  main  frames  which 
divides  the  course’s  contents  and  the  Home  Page  Menu.  The  entire  course  is  organized  with  the  Lecture  Menu,  My  Personal  Information, 
E-mail  Your  TA,  Review  My  Notes,  Course  Management,  Paper  Interactions,  Administration  and  Acknowledgment  information  on  the 
left  side.  The  right  side  merely  contained  directions  to  click  on  course  links.  The  Lecture  Menu  contained  all  24  lecture  modules 
supplemented  with  information  about  the  course  organization  on  the  right.  The  screen  frame  is  divided  into  four  movable  frame  sections. 
The  upper  left  section  is  the  graphics  page.  The  lower  left  is  the  lecture  page  where  all  lecture  files  are  displayed.  The  upper  right 
contains  the  module  number  wMe  the  lower  left  contains  the  lecture  titles,  sublectures,  slides  and  graphic  tides. 

The  Moby  course  generator  templates  allows  designing  course  content  in  a variety  of  combinations.  For  Anthropology  15,  content  was 
presented  using  a combination  of  highly  text- based  lecture  presentation  with  simultaneous  audio  version  of  the  text*-  While  this  choice  of 
mixing  media  method  worked  well  for  handicapped  users,  this  arrangement  is  found  (Hannafin  and  Hooper  1989)  to  place  a huge 
cognitive  load  on  the  learner  who  have  to  simultaneously  process  two  modalities  e.g.  sight  and  auditory,  for  the  same  information.  Still 
another  feature  worth  mentioning  is  the  way  information  were  grouped  together  in  a screen  by  screen  instance.  The  typical  interface  has  a 
whole  screen  filled  with  a dense  amount  of  text  forcing  users  to  read  all  lectures  from  the  screen.  A dense  screen  forces  users  to  stare  at 
the  screen  for  long  periods  of  time  (Aspillaga  !996)  and  puts  a big  load  on  the  user’s  eyes.  This  is  a feature  that  could  be  simplified  if  main 
points  were  bulleted.  This  could  be  complemented  with  an  option  for  downloading  and  printing  a PDF  version  of  the  same  lectures 
eventually  allowing  users  to  read  from  printed  text.  The  typical  Moby  generated  course  navigation^  options  for  the  hypertext  menu  have 
been  placed  on  the  right  side.  Interface  design  research  fTuUis  1997)  on  hypertext  branching  recommends  left  side  positioning  of 
navigation  branching  for  ease,  consistency  and  predicatability.  Screen  by  screen  viewing  for  each  lecture  modules  revealed  a need  for  the 
eyes  to  move  around  the  various  information-filled  frames  detracting  the  user  further  from  making  a coherent  sense  of  all  information 
(Verdi  1985).  Even  if  the  basic  usability  design  requirements  were  not  met  by  the  course,  the  design  choice  was  consistent  until  the  last 
screen.  The  navigation  options  for  all  lectures  were  placed  in  the  same  predetermined  design  location  adhering  to  user- interface  design 
principles  of  consistencyand  predictability  fTuUis  1997).  By  using  the  text  with  audio  combination  to  present  lecture  content,  the  course 
design  fulfilled  the  W3G  requirement  for  accommodating  various  users  including  people  with  handicap. 

Quality  Assurance  for  Anthropology  15 

QA  Testing  of  various  linkages  in  the  course  modules  required  clicking  on  all  available  active  links  from  course  lectures  to  all  graphics 
files.  This  was  a highly  important  part  of  the  QA  testing  and  was  considered  of  utmost  priority  over  features  testing.  The  graphic  file 
must  match  the  link  When  a file  mismatch  occurred  the  problem  was  immediately  alerted  to  the  person  assigned  and  was  resolved 
immediately.  Features  corrections  was  also  ranked  in  order  of  importance  from  highest  to  lowest  priority.  An  unresolved  issue  of 
highest  priority  would  be  one  which  will  alter  the  course  organization,  the  course  content  and  the  message  being  conveyed  such  as 
teaching  about  gorillas  and  the  graphics  shows  no  relation  to  gorillas,  e.g,  picture  of  an  urban  landscape. 


' Doctor  of  Curriculum  and  Instruction,  Instructional  Systems  UCF  and  Research  Analyst,  University  of  California  Davis  lET  Mediaworks, 

^ Mediaworks  is  the  Information  and  Educational  Technology  arm  of  UCDavis 

^ Moby  Course  Development  is  developed  in  three  server  environments,  the  development  server,  the  test  server  and  the  production  server.  The 
Development  Server  has  a machine  name  ofhalfmoon.ucdavis.edu.  This  server  takes  care  of  the  code  design,  development  and  graphic  design  in 
Windows  2000/IIS  syston  and  web  server.  This  server  allows  limited  user- defined  security  groups  (EduTech  and  Mellon  Team)  with  the  write  access 
limited  to  these  same  two  groups  and  the  Administrators.  The  Test  Server  has  a machine  name  of  cloudybay.ucdavis.edu.  It  is  a Windo  ws  2000/IIS 
system.  This  is  where  all  data  files  are  loaded  and  updated  during  the  testing  phase.  The  Production  Server  named  halfmoon.ucdavis.edu  is  housed  at 
and  managed  by  the  UCDavis  Data  Center.  It  is  a machine  with  a UNIX  box  with  Solaris  7.0/Apache  serving  as  the  system  and  web  server.  The 
procedure  sequence  starts  with  the  issues  log.  This  log  becomes  the  source  of  allocating  tasks  among  members  of  the  development  team.  The  team 
leader  then  assigned  tasks  based  on  this  log.  Existing  code  and  business  specification  are  reviewed  followed  by  a verbal  or  written  business 
specification  on  the  design  requirements.  The  developer  developed  the  code  then  tested  it  either  on  the  test  server  or  on  a self-created  personal  Test 
File  folder.  Through  e-mail  communication,  business  specifications  were  constantly  verified  to  make  sure  the  code  worked  and  passed  the  test.  After 
passing  the  code  test,  the  code  was  then  moved  to  the  test  server.  Once  moved  the  code  developer  was  notified.  The  programmer  then  informed  the 
team  leader  by  e-mail,  that  the  code  is  working.  If  more  work  was  required  the  programmer  would  likewise  be  notified. 

More  suited  to  people  with  a auditory  and  visual  handicap. 

^ Hypertext  linkages  within  Lecture  Modules  are  located  on  the  right  side  of  the  4 frame  lecture  page  screen 

® World  Wide  Web  Consortium  is  a consortium  that  develops,  recommends  and  standardizes  and  disseminates  information  related  to  web 
technologies. 
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Anthropology  15  is  one  of  the  purely  online  courses  developed  at  the  University  of  California,  Davis.  Towards  the  encF  of  the  Winter 
Quarter  of  2001,  project  development  had  already  progressed  significantly  to  the  point  of  completion.  Dr.  Alexander  Harcourt,  the 
course’s  instructor  worked  with  Mediaworks  to  produce  the  course  from  conception  to  final  production.  Prior  to  my  involvement  with 
the  project,  several  types  of  communications  in  the  form  of  meetings,  e-mail  and  telephone  conversations  had  transpired  between  the 
Instructor  and  the  Course  Development  Team  to  convey  how  the  project  progressed.  However,  a summary  doci^nting  all  requests  was 
unavailable  as  to  allow  the  development  team  any  form  of  quality  testing  at  various  points  in  the  project  cycle.  Still  a member  of  the 
development  team  this  author  was  given  the  task  of  conducting  the  Quality  Assurance  Testing  for  the  course.  With  little  time  to  waste  at 
such  short  notice,  copies  of  e- mail  communications  between  the  Professor  and  the  development  team  had  to  be  requested  and  turned 
over  to  this  author  for  summary  analysis^  and  provided  the  basis  for  modifying  features  and  functionalities  in  question.  As  QA  tester  this 
author^s  role  focused  on  documenting  all  requests  for  changes  by  Dr.  Harcourt  and  thereafter  performing  the  QA  task  of  ensuring  all 
change  requests  were  incorporated  in  the  course.  Requests  made  prior  to  October  1 were  already  documented  and  incorporated.  My 
responsibility  was  to  document  change  requests  made  after  that  date.  Assignment  of  my  responsibility  for  this  course  was  not  made  clear 
and  tended  to  vary  with  every  meeting.  In  a seemingly  hierarchical  and  seniority  driven  team,  offering  advice  or  providing  comments  of 
any  nature  especidly  from  a new  addition  to  the  group  was  not  positively  received.  I therefore,  had  to  wait  for  all  forwarded  e-mail 
messages  a week  before  the  scheduled  release  of  the  course.  The  e-mail  communications  were  the  only  means  of  creating  the  summary  of 
requests.  With  a pressure  for  time,  this  summary  could  only  be  done  in  a handwritten  form  in  an  excel  spreadsheet.  With  the  summary 
ready,  copies  were  provided  to  members  of  the  team  tasked  with  QA  testing.  One  member  worked  with  problems  that  were  mainly 
features  in  nature.  Another  member  worked  with  the  application  related  to  ColdFusion  templates.  Another  member  made  sure  that  all 
linkages  worked.  All  of  these  tasks  were  frantically  completed  before  the  CD  version  of  the  course  was  due  for  release  on  December  17. 

With  the  summary  as  a guide,  QA  testing  proceeded.  Lacking  a well-  defined  QA  procedure  in  place,  each  member  of  the  team  agreed 
which  aspect  of  the  course  required  more  attention  with  functionality  having  the  highest  priority  over  feature  or  application.  Due  to  the 
intense  QA  testing  activities  the  course  instructor  was  notified  that  no  additional  request  for  more  changes  were  to  be  made  after  a cert^ 
date.  This  was  done  essentially  to  smoothen  the  flow  of  QA  testing  tasks  among  team  members  who  prioritized  among  themselves  which 
request  must  be  completed  over  others  and,  whether  the  change  was  necessary  considering  the  time  pressure. 

Conclusion 

There  was  a major  drawback  to  doing  quality  assurance  testing  for  the  online  course  within  a very  limited  time.  First,  the  quality  of  the 
product  is  not  assured.  Technical  glitches  may  still  lie  undetected  and  can  pose  serious  problems  to  the  course.  Second,  depending  on  the 
severity  of  the  problem,  the  course  may  require  redesign  and  redevelopment  which  can  cause  severe  drain  on  the  development  team  s 
resources  in  terms  of  time,  human,  and  equipment.  In  addition  there  is  also  the  added  pressure  imposed  on  the  morale  of  the  entire 
members  of  the  team  who  have  to  go  through  the  whole  project  cycle.  Quality  assurance  in  an  online  course  delivery  is  a lot  more 
complex  than  traditional  format  because  rules  are  not  fixed  and  clear-cut  and  standards  are  evolving.  The  basis  of  comparison  for  both 
formats  are  totally  different.  In  the  traditional  method  we  have  a human  imparting  the  kno^dedge.  In  a computer- mediated  course  the 
human  is  replaced  by  a machine.  While  course  content  remain  unchanged  the  criteria  for  evaluating  one  against  the  other  can  be 
approached  in  various  combinations  and  in  many  different  ways. 
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Abstract:  According  to  the  four  steps  of  problem  solving  proposed  by  Marshall  in  1995, 
a knowledge  structure  is  designed  based  on  the  Concept  Map  and  the  Concept  Schema, 
which  is  so-called  Knowledge  Map,  Knowledge  Hierarchy  derived  from  Hierarchy  Map 
of  Concept  Map  can  be  used  to  indicate  the  relationships  among  concepts,  whereas 
Concept  Schema  represents  the  information  of  the  corresponding  concept.  After  the 
Knowledge  Map  is  well  analyzed  and  designed,  three  functions  for  manipulating 
knowledge  in  problem-solving  systems,  such  as  matching  operation,  locating  operation 
and  loading  operation  are  considered.  And  finally  an  intelligent  agent  within  Knowledge 
Map  to  accomplish  problem-solving  steps  is  implemented  to  providing  assistances  for 
students  in  distance  learning  environment. 


Introduction 

After  the  invention  of  computer,  some  interactive  programs,  called  CAI  Systems  (Computer- 
Assisted  Instruction  Systems),  are  developed  for  help  students  learning.  There  are  many  previous 
researches  from  building  a proper  problem-solving  system  for  students  to  solving  a common  problem  by 
using  computer  system  (Mayer  1992),  such  as  four  steps  for  solving  mathematic  problems  (Polya  1965), 
seven  stages  of  problem  solving  for  science  experiments  (AUP  86),  and  so  on.  Another  interesting  research 
issue  in  CAI  Systems  is  Intelligent  Agent.  Some  Intelligent  Agents  are  used  for  helping  students  doing 
science  experiments  in  V.E.E,  (Virtual  Experiment  Environment)  (Huang  et  al.  1999;  Kuo  et  al.  2001);  the 
others  are  used  to  diagnose  students'  learning  status  in  problem  solving  systems  (Chang  et  al.  1999;  Heh 
1999;  Chang  et  al.  2000).  In  this  paper,  an  Intelligent  Agent  is  designed  with  a kind  of  knowledge  structure. 
Knowledge  Map,  to  help  student  solving  problems. 

Using  Knowledge  Map  for  Solving  Problems  in  Intelligent  Agent 

This  paper  proposes  a new  knowledge  structure  used  for  problem  solving  called  Knowledge  Map. 
The  idea  of  Knowledge  Map  comes  from  Concept  Map  and  Concept  Schema.  An  example  of  Knowledge 
Map  for  partial  domain  of  Physics  is  shown  in  (Fig.  1).  The  gray  blocks  are  the  original  Concept  Map  and 
are  drawn  as  a Concept  Hierarchy,  which  comes  from  the  idea  ofhierarchicalCo/iccp/Mct/?.  For  example, 
the  concept,  "Motion  with  Constant  Velocity,”  is  one  type  of  “1 -Dimensional  Motion.”  The 
correspondence  of  white  block  to  the  gray  one  is  the  Concept  Schema,  which  comes  from  the  definition  of 
schema  in  (Gagne  et  al.  1993)  for  presenting  natural  categories,  events  and  text  of  a concept.  Take  “Motion 
with  Constant  Acceleration”  for  example,  it  has  its  own  definition  “acceleration  is  constant”  and  the 
physics  formula  “rule  Vt  = vq  + at”. 

After  constructing  the  Knowledge  Map,  some  operations  for  manipulating  knowledge  in  problem 
solving  systems  have  to  be  designed  in  an  Intelligent  Agent:  7.  Search-Operation’,  2.  Locate-Operation: 
finding  what  concepts  should  be  accessed;  3.  Load-Operation:  loading  contents  in  concept  schema  into  the 
short-term  memory.  To  use  the  Knowledge  Map  for  Solving  Problem,  this  paper  follows  four  problem- 
solving steps  extended  by  four  types  of  knowledge  schemas  (Marshall  1 995;  Cheng  et  al.  2001).  In  (Fig.  2), 
a problem  loaded  into  a Problem  Solving  system.  This  problem  is  processed  through  four  steps: 

1.  problem  identification:  loading  problem  into  problem  solving  system  and  using  knowledge 

map  to  consirvici  Knowledge  Object  Schema  (KOS). 
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2.  problem  elaboration:  elaborating  the  KOS. 

3.  problem  planning:  translating  KOS  to  Problem-Planning  Schema  (PPS)  and  using  knowledge  map 

to  do  problem  solving  plan. 

4.  problem  execution:  using  PPS  to  get  the  answer  and  produce  results. 

All  these  steps  get  knowledge  from  knowledge  base  in  long-term  memory  and  produce  different  schemas 
stored  in  short-term  memory. 


Fig.  1 Knowledge  Structure  - Knowledge  Map 

Conclusion 


Fig.  2 Flow  Chart  of  Problem  Solving  Agent 


This  paper  proposes  a new  knowledge  structure  for  problem  solving  called  Knowledge  Map 
constructed  by  Concept  Hierarchy  and  Concept  Schema.  Concept  Hierarchy  indicates  the  relations 
between  concepts  and  Concept  Schema  stores  concept-related  information.  This  paper  also  shows  the  four 
steps  of  problem-solving  and  designs  three  operations  for  manipulating  knowledge  on  Knowledge  Map. 
With  these  operations,  Knowledge  Object  Schema  and  Problem-Planning  Schema  are  created  to  complete 
the  whole  problem  solving  process  and  help  Problem  Executor  to  generate  results.  The  feature  work  of  this 
paper  is  to  change  Knowledge  Map  to  reticular  one,  increase  (or  insert)  knowledge  automatically,  and 
construct  Knowledge  Map  from  natural  language. 
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Abstract:  Digital  terrestrial  broadcasting  is  going  to  be  introduced  into  Japan  and  there  may  be 
some  possibility  that  it  can  be  used  for  educational  purposes  in  schools.  We  conducted  an 
experiment  to  try  DTB  collaborative  learning  in  three  elementary  schools.  Students  in  these  schools 
watched  TV  as  the  main  content  of  DTB  and  then  received  information  from  other  schools  from 
data  pages.  Interviews  of  teachers  and  a questionnaire  to  the  children  revealed  that  it  is  interesting 
to  use  DTB  as  a learning  tool  but  there  are  still  some  problems  to  be  solved. 

Digital  terrestrial  broadcasting(DTB)  is  going  to  be  introduced  into  Japan  and  there  may  be  some  possibility  that  it 
can  be  used  for  educational  purposes  in  schools.  We  tried  to  explore  the  possibility  of  using  DTB  in  schools  using 
them  as  a tool  of  collaborative  learning  in  three  classrooms. 

1.  Purposes  and  Framework  of  the  Experiment 

The  purposes  of  this  study  was  to  develop  a method  to  design  multimedia  contents  on  data  broadcasting,  to  try  the 
interactive  function  of  DTB,  and  to  make  a list  of  necessary  conditions  as  to  learning  using  DTB.  We  chose  three 
elementary  schools  for  our  experiment;  the  Attached  Elementary  School  to  the  Faculty  of  Education  of  Kanazawa 
University,  Meisei  Elementary  School,  and  Takahama  Elementary  School.  Each  school  has  an  excellent  IT  teacher 
who  leads  other  teachers.  Attached  elementary  school  is  in  the  suburbs  of  Kanazawa  city.  Meisei  elementary  school 
is  in  the  middle  of  Kanazawa  city,  and  Takahama  Elementary  School  is  in  a country  area  near  Kanazawa  city.  Every 
school  has  a river  near  by  and  has  an  integrated  curriculum  for  environmental  learning  in  the  sixth  grade. 

2.  Outline  of  Implementation 

Students  in  each  school  watch  shows  as  the  main  content.  The  shows  are  from  a series  of  TV  programs  named 
only  one  earth  f which  deals  with  various  environmental  issues  for  fifth  and  sixth  graders  in  Japan.  Collaborative 
learning  with  this  series  had  already  been  tried  and  the  results  were  analyzed(Kurokami,  1998,  2000).  We  were 
allowed  by  NHK  to  use  this  series  for  our  project.  We  picked  up  two  shows  about  “water”  because  topics  of  learning 
in  the  project  schools  are  about  water;  river,  water  pollution,  water  insects,  water  usage,  and  so  on. 

After  watching  TV,  students  learn  in  depth  about  issues  in  the  show  deeply  for  their  area.  Sometimes  they  go  to  a 
field  or  a river  outside  of  school  and  investigate  the  number  of  insects,  the  extent  of  water  pollution  and  the  way  to 
clean  up  water.  Afterwards,  they  make  a multimedia  data  broadcast  to  be  aired  with  the  next  main  show. 

The  curriculum  for  environmental  learning  started  at  the  beginning  of  sixth  year.  In  the  spring  term,  each  class 
learned  about  environmental  issues  independently.  After  the  summer  vacation,  they  started  learning  about  the 
environment  collaboratively.  We  conducted  DTB  learning  three  times  during  this  period.  The  main  show  of  the  first 
DTB  was  Water  and  Human  Life*'  and  the  content  of  the  data  broadcast  was  a self-introduction.  They  identified 
relationships  between  various  phenomenon  and  human  activities  and  watched  the  self-introductions  from  the  other 
schools.  The  self-introductions  contain  photographs  of  schools,  students  and  teachers,  and  texts  about  their  learning 
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activities.  The  second  DTB  was  on  Oct.  26,  2000.  Topics  of  the  main  broadcast  were  about  the  ways  to  clean  up 
water.  Children  of  the  Attached  Elementary  School  visited  Sai  river  and  filmed  their  activities.  From  this,  they 
edited  17  short  movies  out  of  their  original  digital  videotapes.  Meisei  Elementary  School  contributed  two  short 
movies  to  the  data  broadcast.  These  movies  became  the  main  resource  of  the  digital  broadcast.  The  last  broadcast 
for  the  children  was  to  exchange  comments  with  each  other  by  way  of  a data-broadcast  using  up-link  to  the 
broadcasting  center. 

3.  Result  and  Conclusions. 

We  conducted  formal  interviews  with  teachers  three  times  during  the  experiment  and  conducted  a questionnaire  to 
the  children  after  the  period  of  the  project.  Answers  from  the  questionnaire  were  as  follows: 

-It  was  very  interesting  to  find  out  about  the  activities  at  the  other  schools. 

-I  would  like  to  know  more  about  the  other  schools. 

-Watching  the  shows  and  the  data  was  very  interesting. 

From  this,  we  can  think  that  most  students  were  interested  in  the  collaboration  with  DTB. 

On  the  other  hand,  teachers  said  that  it  was  very  difficult  to  control  the  stream  of  learning.  One  of  the  reasons  is  the 
delay  in  seeing  the  data.  Data  sent  from  each  school  is  connected  to  the  show  of  previous  broadcast.  Therefore 
children  have  to  watch  the  next  show  first  before  they  can  see  this  data,  which  is  downloaded  to  each  school  behind 
the  main  show.  It  might  be  a little  confusing  for  children.  The  other  problems  cited  by  teachers  are  as  follows: 

-Lack  of  data  cache:  The  system  we  used  had  not  cache  for  the  main  content  at  all.  We  could  not  stop  the  TV  nor  re- 
watch it.  Also  we  could  not  also  cache  content  of  the  data.  Each  show  was  15  minutes  long  and  watching  data  from 
the  other  schools  took  more  than  30  minutes.  A lesson  in  Japan  is  45  minutes.  Therefore,  there  was  not  enough  time 
to  think  nor  discuss  topics  Irom  the  show  or  the  data.  It  is  necessary  to  add  a cache  device  to  the  system. 

-Difficulty  of  BML  editor:  Children  can  write  up  their  data  using  Broadcast  Markup  Language(BML)  editor,  but 
operation  of  the  BML  editor  is  not  easy  to  understand.  Sometimes  using  the  BML  editor  was  confusing  for  some 
students  who  has  knowledge  of  HTML.  BML  editor  has  to  be  come  more  user  friendly. 

-Screen  operation:  The  most  confusing  thing  for  children  was  that  a mouse  could  not  be  used  for  the  data  broadcast. 
It  is  designed  to  be  manipulated  by  remote  control.  Therefore  children  needed  to  be  familiar  with  the  screen  and  the 
movement  of  DTB. 

-Time  for  editing:  Children  have  to  film  and  take  pictures  of  their  learning  activities.  Then  they  have  to  integrate 
these  materials  into  a series  of  document.  However,  it  is  time  consuming  for  the  children  to  edit  their  own  film  and 
pictures  into  a multimedia  broadcast  of  DTB.  It  was  educational  for  children  to  carry  out  this  activity,  but  it  took  too 
much  time.  A more  flexible  curriculum  and  time  schedule  are  necessary. 

-Limited  size  of  movie  file:  In  the  regulations  about  DTB,  each  component  on  a data  page  cannot  be  over  1 MB.  As 
the  size  of  some  movies  from  schools  was  more  than  the  limitation,  we  ourselves  put  the  big  movies  on  the  HDD  of 
each  television  set  and  linked  them  to  the  data  pages.  Therefore,  we  need  to  increase  the  limitation  of  bytes.  School 
children  are  increasingly  making  digital  movie  at  school  as  a learning  activity,  so  they  need  to  be  able  to  present  all 
of  their  data. 

-Privacy  policy:  People  who  could  watch  the  DTB  were  strictly  limited  in  this  experiment,  because  there  were  only 
three  sets  on  which  to  watch  it  in  the  Hokuriku  area.  Therefore,  we  could  publish  private  information  in  the 
broadcasts.  When  this  is  launched  on  a full-scale,  we  would  have  to  hide  this  information.  The  problem  of  privacy 
policy  will  occur  at  that  time.  If  we  use  DTB  for  collaborative  learning,  we  would  need  communication  channel  to 
exchange  private  information. 

We  think  there  is  a huge  possibility  for  using  DTB  as  a learning  tool  in  the  future.  These  results  will  be  useful  for 
our  next  experiment. 
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Abstract:  Through  the  Preparing  Tomorrow’s  Teachers  to  Use  Technology  (PT3)  project,  online 
distance  courses  in  College  of  Education  at  New  Mexico  State  University  (NMSU)  have  become  an 
essential  component  of  the  Teacher  Education  Program.  Pre-service  teachers  have  been  engaged  in 
the  process  of  learning  with  and  about  technology.  The  main  purpose  of  this  study  is  to  describe 
and  analyze  College  of  Education  students’  beliefs,  expectations,  and  attitudes  toward  online 
learning  This  is  a qualitative  ongoing  case  study.  Through  purposive  sampling  techniques,  twenty- 
one  students  from  six  different  online  graduate  courses  in  the  College  of  Education  were  selected  as 
participants.  Videotaped  interviews  have  been  conducted  to  explore  and  evaluate  students’ 
reflections  toward  online  learning  in  its  natural  environment.  This  study  is  exploratory  in  order  to 
allow  insights  to  emerge  from  a recursive  data  analysis  process.  The  variables  in  the  site  of  the 
research  are  highly  complex  and  extensive.  The  research  data  is  very  context  dependent  and  needs 
to  be  collected  in  its  natural  environment  without  controls  and  manipulations. 


Overview  of  the  Study 

Over  the  past  few  years,  online  distance  education  has  been  explosively  popular  with  College  of 
Education  students  at  New  Mexico  State  University  (NMSU).  The  Preparing  Tomorrow’s  Teachers  to  Use 
Technology  (PT3)  project  has  been  helping  faculty  integrate  online  learning  environments  into  their 
courses.  As  a result,  the  College  is  dealing  with  the  enormous  growth  of  the  electronic  learning 
environments,  which  has  encouraged  the  University  to  open  its  door  to  learn  and  teach  in  virtual  milieus 
where  students  and  professors  can  communicate  with  each  other  electronically.  With  the  rapid  growth  and 
increasing  accessibility  of  the  Web,  many  professors  are  now  offering  various  types  of  online-based 
courses,  ranging  from  partially  online  to  completely  online,  to  their  students.  Email,  electronic 
conferencing,  synchronous  and/or  asynchronous  chat  and  electronic  bulletin  boards  have  been  used  to 
enhance  many  classes  in  the  College  of  Education.  With  the  use  of  virtual  learning  milieus  as  a supplement 
to  on-campus  classes  as  well  as  in  online  classes,  there  is  no  doubt  that  the  traditional  classrooms  are  no 
longer  bound  by  time  and  space  in  the  College  of  Education. 

When  used  appropriately  online  distance  education  can  provide  increasing  numbers  of  convenient 
and  relatively  inexpensive  learning  and  studying  environments;  it  also  has  the  potential  to  support  real- 
world  experiences  to  not  only  faculty  but  also  students  at  NMSU.  The  College  of  Education  has  invested 
heavily  in  WebCT  as  a virtual  learning  environment.  Since  WebCT  has  provided  new  opportunities  to  use 
the  Web  as  a mediumfor  instruction,  many  courses  and  programs  in  the  College  of  Education  have  been 
rapidly  changing  to  keep  pace  with  the  use  of  WebCT. 

WebCT  is  a unique  educational  delivery  system.  The  use  of  WebCT,  however,  as  a delivery 
medium  in  the  academic  setting  in  the  College  has  started  to  change  students’  roles  and  learning  strategies. 
Unlike  traditional  education,  WebCT  as  an  instructional  tool  has  started  to  make  College  of  Education 
students  take  responsibility  for  their  own  learning  at  their  own  pace  in  their  own  space.  Learning  with 


WebCT  has  also  put  students  at  the  center  of  their  learning  process.  Professors  are  embracing  WebCT  into 
their  curriculum  both  for  online  instruction  and  to  supplement  traditional  education  by  incorporating  new 
ways  of  accessing,  delivering,  exchanging  and  sharing  information  with  their  students  whose  beliefs, 
expectations  and  attitudes  toward  online  learning  have  become  an  important  tool  for  addressing  problems, 
which  may  arise  in  implementing  online  tools. 


Technology  Integration  into  Learning  Environment 

WebCT  by  itself  is  just  an  electronic  data  source  for  the  students.  Like  the  use  of  any  other 
educational  media,  designing  WebCT  as  an  instructional  tool  must  be  based  on  instructional  theories, 
design  models,  and  strategies.  Since  information  can  be  sent  though  WebCT  quickly,  students  may  need  to 
carefully  redesign  and  revolutionize  their  new  roles  around  online  learning,  methods,  techniques  and 
strategies  in  interactive  and  collaborative  virtual  learning  activities. 

This  study  has  taken  a grounded  theory  approach  to  allow  the  researchers  to  explore  and  discover 
the  students’  reflections  toward  online  teaching  for  two  reasons.  First,  increasingly  academic  institutions 
are  beginning  to  offer  graduate  level  teacher  education  courses  online.  Secondly,  online  learning 
opportunities  and  students’  beliefs,  expectations  and  attitudes  toward  online  learning  are  relatively  new 
phenomena.  Students’  reflections,  however,  toward  online  learning  need  to  be  more  clearly  defined, 
examined  and/or  analyzed  to  integrate  and  implement  this  technology  into  the  curriculum  efficiently. 
However,  there  is  a lack  of  theoretical  or  empirical  research  on  this  topic  (Wills  & Becht  1997,  Mendels 
1998,  Valenta  et  al  2001). 

The  main  purpose  of  this  study  is  to  examine  College  of  Education  students*  beliefs,  expectations 
and  attitudes  toward  online  learning  in  higher  education  settings.  Before  entering  the  study,  the  researchers 
tend  to  analyze  the  research  data  inductively  rather  than  to  prove  or  disprove  a hypothesis.  The  main  focus 
in  this  study  is:  1)  to  investigate  and  understand  the  insider’s  views  toward  teaching  online  courses  in 
different  subject  areas,  and  2)  to  expound  on  this  study  participants’  perspectives  and  interpretations  rather 
than  researcher  imposed  categories. 


Research  Design 

Participants  and  Setting 

The  purpose  of  this  study  is  to  investigate  the  students’  beliefs,  expectations  and  attitudes  toward  a 
particular  phenomenon  of  online  learning  in  different  subject  area  experiences.  During  the  2001-2002 
school  year,  research  was  conducted  within  the  College  of  Education  at  New  Mexico  State  University,  a 
medium-sized  research  university  in  the  Southwest.  The  College  of  Education  has  been  offering  classes 
online  (in  various  degrees,  i.e.  enhanced  or  completely  online)  approximately  four  years. 

The  students  in  the  College  of  Education  were  chosen  for  three  major  reasons:  First,  many 
members  of  the  current  students  have  been  strongly  interested  in  using  the  web  as  an  instructional  tool  to 
make  possible  communication  between  regular  class  sessions  (Baptiste  & Kurubacak  2002).  Each  week 
students  would  post  their  assignments  and  any  questions,  concerns,  or  ideas  online.  The  students  could 
read,  receive,  post,  exchange  and/or  share  information  on  the  discussion  topics  of  each  week  before  and 
after  class.  Sometimes  students  would  email  the  course  professor’s  account  directly,  when  they  had 
particular  questions,  suggestions  and  /or  comments  any  time.  Second,  the  most  of  course  curriculums  in  the 
College  of  Education  have  included  both  computer-based  learning  activities  into  classroom  and  web-based 
learning  activities,  WebCT  has  been  used  for  posting  the  students’  messages  and  papers  via  email  and 
electronic  bulletin  board.  Finally,  the  majority  of  students  in  this  study  have  been  considering  the  idea  of 
taking  courses  completely  online  in  the  near  future  and  some  of  them  are  already  taking  courses  online. 

Twenty-one  students  from  six  different  online  classes  in  Teacher  Education  Program  at  the 
College  of  Education  were  identified  for  the  interviews  according  to  their  willingness  to  participate  the 
study.  Four  of  these  twenty-one  students  were  male  and  seventeen  of  the  participants  were  female  in  this 
research. 
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Data  Collection 


This  study  utilized  qualitative  data  to  provide  detailed  information  to  the  researchers  for  analysis. 
Video  and  audio  type  taped  interviews  have  been  the  instruments  in  this  study.  Therefore,  the  open-ended 
interview  questions  are  designed  to  collect  and  analyze  data  on  the  students*  beliefs,  expectations  and 
attitudes  toward  online  learning  in  different  subject  areas.  Each  interview’s  questions  were  developed  and 
modified  according  to  investigating  the  focus  of  this  study. 

The  researchers  allowed  the  participants  from  the  six  different  online  courses  in  Teacher 
Education  Program,  as  groups,  to  set  the  interview  dates  and  locations  themselves.  Face-to-face  interviews 
were  held  with  all  six  groups  in  the  College  of  Education  building.  Each  interview  lasted  approximately  45 
minutes  in  length.  Although  all  interviews  were  video  and  audio  recorded,  the  researchers  wrote  notes  that 
summarized  the  major  points  of  each  session. 

Researchers  have  been  following  a careful  data  management  process  to  ensure  high-quality 
accessible  data  and  documentation  of  data  collected.  For  these  reasons,  the  researchers  regularly  recorded 
and  systematically  stored  qualitative  data  on  computers  and  videocassettes.  The  data  has  been  indexed  for 
easy  and  consistent  retrieval. 

Data  Analysis 

The  analysis  of  ^le  students’  beliefs,  expectations  and  attitudes  toward  online  learning  on  the 
different  subject  areas  is  ongoing  process,  that  began  at  the  end  of  the  Fall  2001  semester  and  will  continue 
through  the  final  written  report.  The  data  analysis  process  in  this  study  is  analytic  and  recursive  to  inform 
further  decisions  on  data  being  collected.  It  also  is  restructured,  flexible  and  open  to  the  discussions  with 
the  stakeholders  and  reviews  of  related  literature. 

During  the  data  analysis  procedure,  the  researchers  briefly  followed  these  steps  given  in  a logical 
order:  1)  transcribed  each  video  and  audio  type  cassettes,  and  2)  identified  patterns  and  themes.  After 
transcribing  the  cassettes  and  identifying  patterns  and  themes,  the  researchers  triangulated  the  qualitative 
data  and  reported  the  results  in  descriptive  and  narrative  form  together. 


Discussions  and  Interpretations 

Content  analysis  of  this  study  indicated  that  students  have  both  positive  and  negative  beliefs, 
expectations  and  attitudes  toward  online  learning.  Among  the  students’  positive  reflections  in  this  study 
were  that  online  courses  could  provide  not  only  enhanced  flexibility,  but  also  student  convenience  and 
access  to  updated  course  information.  Among  some  of  the  negative  reflections  that  students  reported  about 
their  experiences  with  online  learning  was  decreased  human  interaction  and  collaboration  as  well  as  the 
increase  in  students’  course  work.  The  lack  of  technical  support  and  knowledge  about  technology  seems  to 
be  major  problems  for  the  students  in  this  online  learning  environment. 

Positive  Students’  Beliefs,  Expectations  and  Attitudes  toward  Online  Learning 

The  participants  in  this  study  learned  best  when  they  developed  positive  attitudes  toward  virtual 
learning  environments.  These  positive  attitudes  motivated  them  to  solve  a problem,  accomplish  a task,  set 
specific  goals,  work  with  others  and  informational  resources  during  the  learning  process.  However,  it  took 
time  for  these  students  to  adopt  a positive  attitude  towards  online  learning.  The  twenty-one  interviewees  in 
this  study  could  eliminate  their  possible  biases  toward  online  learning  when  the  instructors  were  aware  of 
which  WebCT  methodologies  and  techniques  affected  their  beliefs,  expectations,  and  attitudes. 

Therefore,  the  structure  of  any  online  course  must  involve  determining  the  overall  instructional 
approach,  the  theoretical  and  educational  basis  for  that  approach,  potential  strategies,  methods,  and 
instructional  activities  (Bannan  & Milheim  1997).  The  participants  in  this  study  especially  emphasized  that 
they  enjoyed  working  within  the  virtual  environment  because  of  enhanced  flexibility,  convenient  course 
scheduling,  and  round-the-clock  access  to  course  related  information. 
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Flexibility  and  Convenience 


The  participants  in  this  study  were  working  full  time  or  part  time  while  earning  their  master  or 
doctorate  degrees  in  the  College  of  Education.  The  majority  of  students  had  to  commute  at  least  one  hour  to 
the  campus  everyday.  What  was  most  important  to  students  in  this  study  was  that  online  distance  education 
not  only  provided  flexible  time  management,  but  also  allowed  them  to  save  on  travel  time. 

All  of  the  interviewed  students  in  this  study  stated  that  they  could  work  anywhere  at  their  own 
pace.  Monica  pointed  out,  ''[It*]  comfortable  for  me  to  study  at  home,  especially  after  the  children  go  to  the 
beds... but  you  have  to  discipline  yourself  otherwise  you  can  easily  focus  on  doing  housework  rather  than 
doing  your  assignments  in  your  cozy  study  place''  However,  the  participants  in  this  study  were  also  aware 
that  it  required  high  self-discipline  and  self-confidence.  Robert  stated,  “/77  ^ould  like  to  study  at  home, 
however,  you  have  to  firmly  arrange  your  schedule...  to  take  the  class  online  is  different  from. ..to  attend 
the  class  on  the  campus. ..Also  you  can  raise  your  voice  online  to  ask  a help  when  you  need.  It's  not  easy 
every  time. ..sometimes  I feel  I am  the  goofiest  student  in  the  class  [laughing]...  WebCT-based  course 
requires  self  discipline.  It  is  easy  to  be  frustrated  and  drop  out.  " Robert  was  not  the  only  participant  who 
felt  that  the  online  learning  environment  was  different  from  face-to-face  class  settings.  Nineteen  of  the 
other  participants  also  agreed  with  Robert’s  viewpoint  on  virtual  learning  milieus;  “students  roles  and 
responsibilities  in  online  classes  were  not  the  same  as  their  positions  and  tasks  in  a traditional  face-to-face 
course”. 

Access  the  Course  Online 

Twenty-one  interviewees  highlighted  that  they  could  find  useful  information  about  the  course  and 
course  work  in  their  virtual  class.  These  participants  also  pointed  out  that  they  regularly  accessed  WebCT 
to  get  online  course  information  throughout  the  semester.  All  participants  in  this  study  indicated  that  they 
enjoyed  reading  about  the  course  and  assignments.  One  of  them,  Juanita,  made  her  statement  on  this  issue: 
‘fyou]  can  read  the  instructor  *s  messages  before  coming  to  class...  I had  a hard  copy  syllabus,  but  I always 
check  the  online  one,  if  she  [the  instructor]  changes  the  schedule  or  announce  new  activities....  I can  learn 
new  information  very  promptly,  it  is  helpful''  Juanita  had  significant  previous  experience  taking  WebCT 
classes  and  did  not  lack  self-confidence.  She  regularly  accessed  the  course  web  page  from  work  because 
her  work  place  provided  a faster  modem. 

Of  twenty-one  participants,  the  majority  pointed  out  that  they  would  like  to  read  the  assignments 
posted  by  the  other  students  in  their  classes.  John  stated:  "[I]  always  read  the  posted 

assignments... especially  before  the  class...!*  am  amazed. ..the  people  focus  on  different  details  on  the 
assignments... sometimes  I have  never  ever  thought  these  details."  John  also  highlighted  that  *‘'[it]  was  very 
difficult  to  express  himself  or  herself  sufficiently...  like  at  the  face-toface  class."  However,  the  majority  of 
the  interviewees  in  this  study  reflected  their  anxiety  about  the  online  assignments  and  other  posting 
messages.  Tim  expressed  the  following  statement:  "[it*s]  kind  of  weird;  because  everybody  can  read 
you... not  only  the  instructor."  The  participants  in  this  study  felt  that  they  were  more  motivated  to  work 
hard  on  their  assignments,  because  the  professor  and  other  students  would  be  reading  them. 

Negative  Students’  Beliefs,  Expectations  and  Attitudes  toward  Online  Learning 

There  were  three  main  barriers  for  the  students  in  this  study  to  learn  with  online  environment:  1) 
technological  problems,  2)  lack  of  technical  support,  and  3)  lack  of  knowledge  about  technology.  The 
students  in  this  study  sometimes  had  difficulties  accessing  the  online  course.  John  stated:  "[It*s]  frustrating 
to  have  problems,  it*s  frustrating  not  to  find  the  solution,  and  not  to  fix  it."  The  lack  of  accessibility  to  the 
online  course  created  negative  reflections  toward  virtual  learning  milieus. 

This  study  indicates  that  students  are  more  successful  online  when  they  are  trained  on  online 
learning  and  have  regular  technical  support.  Students  must  be  encouraged  and  motivated  to  take  full 
advantage  of  online  learning  milieus. 

Limited  Interaction  and  Collaboration  Online 

Interviewed  students  indicated  that  they  did  not  have  enough  information  about  online  learning 
and  their  changing  roles  in  electronic  learning  settings.  Kim  stated,  "[I]  always  miss  the  class  online,  thus  it 


is  very  hard  for  me  to  catch  up...  At  the  beginning  of  the  semester,  I had  no  idea  about  how  I could  access 
the  course  WebCT  site,  it  took  nearly  over  three  weeks  to  learn,  it  is  really  heart  breaking^  Thus,  they  did 
not  successfully  nor  actively  interact  with  other  students,  faculty,  and  online  educational  resources.  Also, 
the  majority  of  the  participants  in  this  study  stated  that  they  needed  to  learn  the  skills  of  online 
communication.  John  pointed  out,  '"[Maybe]  another  form  of  communication  is  needed  online.  WebCT 
courses  require  high-level  self  discipline  as  well.  You  have  to  logon  regularly  the  course.'* 

There  were  many  comments  that  specifically  addressed  the  lack  of  collaboration  in  online 
learning.  Tim  expressed  his  feelings  about  online  collaboration,  "[Online]  communication  is  not  like  to 
communicate  somebody  else  in  person.  WebCT  is  not  for  me  to  talk  with  other.  / always  prefer  to  interact 
my  friends  face-to-face.  It's  warm  and  comfortable.  You  cannot  get  feedback  promptly  online.  You  have  to 
always  be  careful  whether  somebody  else  answers  you  or  not."  When  the  interviewed  students 
communicated  with  each  other  on  WebCT,  the  students  were  not  able  to  reflect  on  their  experiences  and  fix 
their  learning  problems.  The  participants  in  this  study,  however,  indicated  that  a majority  of  students  in 
College  of  Education  when  lacking  knowledge  and  experience  with  online  environments  tend  to  generate  a 
lack  of  confidence  when  communicating  online.  The  learning  strategies,  methods,  and  activities  in  online 
learning  setting  are  different  from  face-to-face  courses.  Uncertainty  creates  curiosity,  anxiety,  and  conflict 
in  these  students  who  have  low  comfort  levels  with  using  online  technology.  As  a result,  the  students  were 
less  eager  to  participate  in  online  learning.  While  virtual  learning  settings  might  be  helpful  in  achieving 
vital  educational  goals,  faculty  members  suggest  that  face-to-face  communications  are  important  to 
promote  effective  social  interactions  among  students  and  between  students  and  faculty. 

The  majority  of  the  interviewed  students  preferred  face-to-face  class  interactions.  They  also 
pointed  out  that  their  age,  gender,  personality,  and  cultural  backgrounds  highly  affected  their  participation 
and  contributions  in  various  educational  settings.  Juanita  strongly  stated,  "[I]  am  better  at  face-to-face 
communication  in  person  than  at  communicating  online.  / would  like  to  observe  the  mimics,  gestures  and 
body  languages  when  talking  with  someone.  / have  to  see  the  eyes  when  / am  communicating  with 
her/him.”  Like  Juanita,  John  (another  participant)  made  few  comments  to  the  online  course.  He 
acknowledged  that  the  variety  cultural  backgrounds  of  the  students  could  support  effective  communication, 
but  the  face-to-face  communication  was  more  valuable  and  helpful  than  online  interaction. 

In  addition,  the  online  learning,  communicating,  and  writing  skills  of  students  could  be  major 
barriers  to  interactive  online  learning.  Kim  highlighted  that  "[special]  I learning  skills  are  needed  for 
taking  a WebCT  class... for  example  writing  is  not  easy  for  me...  I don  7 enjoy  typing  the  keyboard  to  do  my 
assignments...  It  takes  too  much  time  for  me  to  type,  and  it  is  not  fun...  It  is  easy  for  me  to  speak  in  person/' 
Like  Kim,  Tim  stated,  "[Reading]  on  screen  is  very  difficult...!  have  to  printout  each  screen  to  get  the  real 
his/her  point  of  view”  Juanita,  like  Kim,  also  expressed  similar  feelings  about  studying  online,  "[I]  am  one 
of  the  slower,  and  it  takes  all  my  time  to  write  something  for  the  class. ..I  don't  like  it.”  When  an  online 
learning  environment  has  cross-cultural  communication  and  uses  a systematic  approach  to  deliver  content, 
online  students  can  connect  theory  and  practice  as  well  as  have  more  in  depth  reflection  in  the  construction 
ofknowledge. 

Overwhelmed  Course  Work 

The  majority  of  the  participants  in  this  study  felt  more  time  was  spent  on  the  WebCT  class  than  in 
regular  classroom.  Jane  loudly  expressed  her  feelings  with  the  statement,  "[I]  did  have  to  work  very  hard 
for  the  online  class... I strongly  feel  online  courses  required  more  work.”  All  of  students  in  this  study 
pointed  out  that  they  often  times  felt  overloaded  with  information  in  their  online  classes  and  that  virtual 
learning  environments  required  more  work.  The  majority  of  the  participants  in  this  study,  like  Jane, 
expressed  their  feelings  of  being  overwhelmed  by  not  only  the  quality  but  also  the  quantity  of  contributions 
of  the  instructor  and  other  students.  Kim  pointed  out,  "[I]  feel  I cannot  write  and  think  at  a high  level  like 
the  others  in  the  class... I think  it's  depend  on  the  course  content,  but  I lost  my  concentration  on  the 
coursework...”  One  of  the  most  challenging  aspects  in  implementing  an  online  learning  in  higher  education 
can  be  the  structure  and  instructional  strategies  of  the  course  content. 
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Conclusions 


The  body  of  the  research  literature  dealing  with  students’  beliefs,  expectations  and  attitudes 
toward  virtual  learning  milieus  is  small,  but  growing  (Valenta  et  al.  2001).  Online  learning  in  higher 
education  is  a recent  phenomenon,  which  requires  more  research  as  well.  The  use  of  virtual  learning 
environments  is  gradually  shifting,  so  it  is  necessary  to  keep  pace  with  improvement  on  the  learning 
process.  The  use  of  online  communication  in  academic  settings  is  increasing  rapidly  and  has  become  a vital 
issue  studied.  This  study  shows  that  the  key  points  of  online  learning  are  to  offer  online  learning  techniques 
and  to  provide  reliable  and  regular  technical  support.  In  addition,  encouraging  and  motivating  students  to 
take  advantages  of  virtual  learning  as  an  educational  tool  is  also  a necessary  measure  to  insure  effective 
utilization  of  the  online  learning  milieu. 

An  essential  step  in  integrating  online  learning  environments  productively  in  higher  education  is 
starting  with  an  explicit  definition  of  the  educational  role  of  online  learning  and  understanding  the  changing 
roles  of  students  in  virtual  milieus.  Integrating  online  learning  into  learning  can  lead  the  College  of 
Education  to  adapt  changing  roles,  needs,  interests,  and  concerns.  Students  are,  doubtlessly,  one  of  the  most 
important  components  in  online  teaching.  On  the  other  hand,  the  roles  of  students  have  been  changing 
dramatically  in  virtual  learning  milieus;  their  needs,  concerns  and  roles  vary  substantially  from  face-to-face 
educational  settings.  The  major  problem  is  that  many  students  do  not  understand  their  responsibility  to  be 
active  contributors  of  learning  in  the  virtual  milieu  and  that  the  professor’s  role  is  to  be  a facilitator  and  a 
mentor.  Therefore,  not  only  must  students  be  trained  how  to  instigate  and  contribute  as  online  participants 
on  WebCT,  but  they  must  also  acquire  good  online  learning  skills  and  strategies  before  taking  a WebCT 
based  course. 

To  create  a successful  collaborative  online  learning  environment,  group  or  cooperative  learning 
must  be  emphasized  among  students,  and  between  students  and  professors.  Virtual  collaboration,  which 
covers  active  participation  and  interaction,  are  essential  experiences  that  students  should  have.  Thus, 
WebCT  must  organize  and  structure  online  interaction  among  students,  professors,  experts  from  the 
outside,  and/or  global  online  resources  with  fewer  time  and  space  limitations.  Online  learning  must  also 
encourage  and  engage  students  to  work  together  during  learning  activities.  Conversely,  promoting  and 
maximizing  group  work  and  collaborative  learning  in  a virtual  environment  is  not  easy.  It  is  also  difficult  to 
identify  social  presence,  authentic  tasks,  and  cognitive  strategies  such  as  taking  ownership  of  a task  whose 
cognitive  challenges  are  consistent  with  the  cognitive  demands  of  the  design  environment.  This 
environment  must  support  knowledge  construction  and  idea  sharing.  This  virtual  learning  setting  must  also 
enable  students  to  continue  work  on  their  own  tasks. 

The  overall  teaching  atmosphere  in  College  of  Education  is  highly  academic  and  professional. 
Nevertheless,  lack  of  online  learning  experiences  and  computing  skills  affect  students’  beliefs,  expectations 
and  attitudes  toward  online  learning.  The  major  challenge  facing  students  is  learning  how  to  restructure 
their  learning  styles  (through  the  use  of  online  learning  for  better  utilization  of  current  resources)  and  how 
to  implement  improvements  in  online  learning  milieus.  The  delivery  of  courses  via  virtual  learning 
environments  requires  training  students  in  ways  that  maximize  and  allow  them  to  further  adapt  their 
educational  strategies.  Thus,  it  is  an  essential  issue  to  provide  excellent  training  and  technical  support  for 
higher  education  students. 
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Abstract:  In  this  paper,  we  describe  the  development  of  an  adaptive  multimedia  learning  environment 
that  utilises  multimedia  presentation  techniques  in  its  interface  while  still  providing  Internet 
connectivity  for  management  and  delivery  purposes.  The  system  supports  the  WWW  as  its  addressing 
space  but  uses  the  local  client  areas  tostore  media  items  expensive  in  terms  of  delivery  time.  Learning 
objects  that  provide  frameworks  for  tasks  and  other  summative  assessment  activities  are  stored  on  a 
server  and  delivered  when  required.  The  system  supports  link  annotations  in  its  adaptivity  and 
employs  an  overlay  student  model  with  stereotyping  when  accessing  the  course  content. 

Keywords:  Educational  multimedia.  Adaptive  Systems,  Dynamic  course  generation.  Student  profiles, 

1.  Introduction 


Large  classes  of  students  with  different  cultural  and  academic  needs,  coupled  with  the  large  range  of 
individual  approaches  needed  to  deal  with  some  of  student  learning  issues  raised  above,  have  created 
problems  for  institutions  catering  for  large  numbers  of  students,  A direction  that  a number  of  academic 
institutions  have  pursued  is  to  place  course  materials  in  an  on-line  format.  These  materials  can  either  replace 
pre-existing  lectures/tutorials,  or  supplement  it.  Needless  to  say,  a great  variety  of  educational  materials  exist 
for  possible  reuse  within  the  public  domain  and  are  available  via  their  Universal  Resource  Locator  (URL), 

As  has  been  pointed  out  [17,  18],  the  World  Wide  Web  (WWW)  provides  a vehicle  for  the  development  of  a 
learning  environment  with  teaching  structured  towards  the  development  of  lifelong  learning  skills,  and 
catering  for  the  expectations  and  learning  styles  of  students  with  a wide  range  of  backgrounds.  Early 
Australian  examples  of  the  use  of  the  WWW  in  IT  education  abound,  Boalch  [3]provides  an  examination  of 
the  use  of  the  WWW  as  a support  medium  for  the  delivery  of  a first  year  unit  in  Information  Systems  at 
Curtin  University,  He  provides  an  evaluation  of  site  utilisation  and  user  feedback  in  the  case  where  subject 
information  and  course  details  were  provided  on  the  WWW  for  students. 

The  Eklunds  [8]  examine  the  use  of  the  WWW  to  supplement  traditional  IT  teaching.  They  provide  case 
studies  of  two  examples  of  the  re-structuring  of  traditional  forms  of  IT  course  for  Web-delivery,  Jones  [1  l]of 
Central  Queensland  University  gives  details  of  case  study  involving  the  design,  presentation  and  evaluation 
of  an  undergraduate  unit  in  Systems  Administration  taught  completely  via  the  WWW  to  on -campus  and 
distance  students. 

However  these  and  many  newer  systems  that  have  been  developed,  rely  on  the  low-level  concept  of 
interactivity  (derived  from  a distance  education  paradigm)  as  the  relationship  between  an  individual  student 
and  text,  and  fail  to  use  the  technological  interactivity,  which  is  available.  Common  online  learning 
environments  often  fail  to  maximise  the  potential  of  current  multimedia  resources.  While  the  value  of  HTML 
pages  and  threaded  discussions  is  acknowledged,  they  do  not  display  the  ability  to  adapt  teaching  material  for 
the  needs  of  individual  students. 

Multimedia  objects  within  learning  environments,  provide  a possible  enhancement  to  the  presentation  of  our 
on-line  materials,  Ricketts  [16]  describe  the  successful  use  of  a hybrid  CDAVWW  presentation  of  course 
materials  delivered  in  distributed  mode  where  multimedia  elements  were  provided  on  CD,  The  Herringtons 
[9]  describe  the  benefits  of  multimedia  within  authentic  assessment  used  in  a teacher  pre-service  course, 
Looi  et  al  [13]  describe  a collaborative  WWW  based  system  that  provides  for  on-line  communication  and 
collaboration  in  the  creation  of  multimedia  artefacts  for  the  WWW, 
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Further,  Jonassen  et  al  [10]  take  the  position  that  the  true  worth  of  multimedia  and  hypermedia  might  be 
obtained  through  the  learner  constructing  knowledge  via  the  use  technology,  rather  than  as  a mode  of 
delivery.  Although  the  authors  agree  that  the  creation  of  multimedia  and  hypermedia  artefacts  is  a powerful 
mechanism  for  individual  learning,  we  do  not  discount  the  learning  benefits  attributed  to  the  construction  of 
different  media  views  of  content;  we  would  argue  that  more  individual  learning  styles  could  be 
accommodated. 

Our  view  then  of  a learning  environment  is  a domain  populated  with  instructional  items,  presented  as  either 
multimedia  or  hypermedia  objects  with  some  addressing  mechanism.  This  address  might  be  a URL,  or  a 
directory  path  to  a locally  stored  object.  We  use  the  term  object  to  conform  to  the  IEEE  Learning  Technology 
Standards  Committee  specification  [7]  for  learning  objects  and  further,  agree  with  Wiley *s  [20]  interpretation 
as  ‘any  digital  resource  that  can  be  reused  to  support  learning*. 

2.  Adaptive  Teaching 


Various  researchers  who  have  experimented  with  adaptive  teaching  on  the  WWW  have  used  techniques  and 
principles  derived  from  Intelligent  Tutoring  Systems  (ITS)  and  particularly  Anderson’s  rule  based  cognitive 
modelling  [ 1 ].  Others  have  used  Adaptive  Navigation  Support  [4]  to  provide  adaptive  navigation  through 
hypertext  pages  and  thus  developed  adaptive  textbooks  for  the  tutoring,  particularly,  of  software  applications 
[4]. 

ITS  and  hypertext/hypermedia  architectures  provide  different  means  of  access  to  the  educational  materials 
contained  within  the  knowledge  domain.  Hypermedia  systems  employ  referential  links  that  enable  uses  to 
typically  follow  non-sequential  paths;  organisational  links  (eg  hierarchical)  do  provide  some  structuring  that 
maps  the  expert’s  view  of  the  domain.  The  pedagogical  module  of  the  ITS,  provides  information  about  the 
teaching  process;  this  is  non-existent  in  unstructured  hypermedia  systems  and  the  user  is  generally  free  to 
explore  the  domain. 

Adaptive  systems  couple  the  course  model  with  a student  model  to  provide  adaptive  navigation  support.  An 
adaptive  system  that  makes  available  specific  content  (adaptive  presentation)  [4]  provides  students  with  a 
format  of  the  new  knowledge  and  skills  appropriate  to  the  students’  level  of  understanding.  Alternately,  the 
system  might  provide  links  to  appropriate  content  (link  level  adaption)  [4].  Some  systems  also  employ 
stereotyping  within  their  student  modelling  structure.  This  appears  to  be  an  appropriate  technique  where 
distinct  student  groups  with  similar  characteristics  can  be  identified.  However,  any  system  based  on 
stereotypes  and  preferences  should  be  observable  and  thus  editable  by  the  user  [12]. 

This  paper  presents  techniques  that  are  being  developed  to  provide  adaptivity  within  an  introductory 
multimedia  course',  the  aim  of  which  is  to  provide  the  foundational  knowledge  and  skills  required  to  create 
and  utilise  a range  of  media  items  within  multimedia  presentations.  The  students  undertaking  the  course 
come  with  a diverse  range  of  knowledge  and  skills.  The  prototype  developed  is  referred  to  as  an  Adaptive 
Multimedia  Learning  Environment.  (AMLE). 

3.  Implementation  of  AMLE 

AMLE  is  essentially  a collection  of  multimedia  and  hypermedia  learning  objects  that  are  tagged  with  and 
address  or  path  that  enables  the  system  to  locate  it.  Learning  objects,  irrespective  of  their  form  and 
functionality,  can  then  exist  on  remote  servers,  or  indeed  as  local  files.  Our  architecture  is  based  upon  a 
concept  which  has  associated  documents  similar  to  [15].  Concepts  can  vary  in  presentation  and  can  have 
hypertext/hypermedia  links  embedded  within  them;  concepts  are  further  group  ed  into  sessions. 


a.  AMLE’s  Architecture 

Two  major  decisions  were  undertaken  in  the  early  development  of  the  prototype;  firstly,  to  ensure  that  our 
items  were  consistent,  we  decided  to  use  Adobe  Acrobat  pdf  files  as  the  default  format;  every  concept  has  a 
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version  in  this  format.  Associative  links  within  these  files  can  address  the  WWW  in  total,  as  well  as  the  local 
file  space.  A player  called  a concept  viewer  to  display  concepts  in  this  format,  has  been  created. 


The  second  decision  taken  was  to  keep  media  rich  items  on  the  local  file  system.  The  rationale  for  this  was 
that  the  course  offered  initially  would  be  administered  each  semester  and  the  concepts  would  be  static  over 
this  period;  the  time  taken  to  deliver  these  materials  would  be  greatly  reduced.  However,  these  still  could  be 
accessed  through  their  URL  if  they  existed  on  a server. 

We  have  created  an  event  driven  interface  (client)  with  Macromedia  Director  that  provides  hypermedia 
functionality,  and  a range  of  concept  viewers  to  support  different  media  formats.  The  main  interface  of  the 
client  connects  to  a remote  server  and  accesses  data  defining  both  the  course  domain  and  the  student  profiling 
information.  Thus,  media  files  that  constitute  concepts  and  related  activities  are  stored  locally  but 
administered  over  the  WWW. 

This  WWW  connectivity  then  provides  the  facility  to  monitor  student  usage  and  register  assessment  items. 
Lecturers/tutors  interact  with  the  course  and  student  model  through  a standard  WWW  browser.  Other 
learning  objects  that  document  assessment  tasks  and/or  other  course  information  are  stored  on  the  server  side 
and  accessed  through  either  a WWW  browser,  or  the  main  interface. 

We  have  developed  this  multimedia  learning  system  using  Macromedia  Director,  ColdFusion,  HTML  and 
CGI  Perl  script  that  allow  rapid  prototyping;  our  main  interface  exists  as  a run  time  version  while  some  tools 
and  other  applications  can  exist  as  source  code.  This  is  of  significance  in  the  initial  course  on  offer  because 
students  can  create  multimedia  artefacts  that  can  be  inserted  directly  into  their  workspace. 


The  main  interface  provided  for  students  to  access  the  learning  materials  can  be  classified  as  a hybrid 
browser.  It  takes  advantage  of  the  multimedia  development  environment  and  enables  a range  of  concept 
players  to  be  used  depending  on  content  availability  and  student  preference.  These  range  from  a concept 
being  displayed  as  text  formatted  with  HTML  tags,  to  a concept  being  displayed  as  an  acrobat  pdf  file, 
containing  both  text  ual  and  graphical  information.  Video  and  audio  formats  can  be  utilised,  along  with 
appropriate  animations.  A range  of  other  appropriate  tools  is  provided  in  the  interface  to  enhance  the  learning 
environment.  For  example,  on-line  tutorial  groups  can  be  established  with  the  ability  to  discuss  particular 
content-based  issues. 

The  main  interface  is  a windows  environment  that  initially  provides  authenticated  logging  on/off  facilities. 
Once  identified  and  registered,  the  user  is  provided  with  the  current  sessio  n,  or  the  pre-test  for  that  session  if 
that  is  the  first  time  that  the  student  has  accessed  that  session.  From  there,  individual  concepts  can  be 
accessed  through  a session  player.  Sessions  are  groupings  of  concepts  and  tasks  that  the  domain  expert  has 
predefined  for  the  course.  This  would  correspond  to  the  content  provided  in  a weekly  lecture(s)  and 
associated  tutorial/practical  activities 

It  has  been  argued  that  link  annotation  [6]  provides  the  user  with  extra  information  about  the  content 
available,  enabling  a more  informed  decision.  Annotations  are  employed  within  the  session  viewer  to  indicate 
the  content  that  a mouse  click  will  provide.  Where  the  system  can  infer  that  the  student  knows  the  concept  or 
can  perform  the  task,  the  session  viewer  is  annotated  accordingly.  Links  are  not  hidden  so  that  the  student  is 
still  able  to  follow  it  for  revisional  purposes. 


Our  course  model  is  organised  hierarchically  into  courses,  sessions,  concepts  and  documents.  Sessions  have 
been  defined  from  concepts;  sessions  can  have  any  number  of  concepts  but  a concept  can  only  appear  once 
within  any  session  of  a course.  A concept  consists  of  many  documents;  a document  can  be  the  content  of  a 
concept,  its  objective  or  some  practical  activity. 


b.  Interface  details 


c.  Domain  knowiedge 


Our  implementation  is  supportive  of  other  possible  domains  of  knowledge;  the  delivery  system  then  is 
content  independent.  Tools  for  both  the  generation  and  subsequent  use  of  the  system  have  been  investigated 
and  an  on-line  database  supporting  both  the  student  and  course  models,  has  been  established.  The  placement 
of  content  into  small  modules,  provides  a mechanism  for  the  flexible  reuse.  The  course  metadata  is  stored  on 
a remote  server  along  with  the  student  profile  information.  We  utilise  an  overlay  model  [5]and  allow  WWW 
access  to  this  information. 

To  accommodate  a range  of  students  at  various  levels  in  their  programmes,  we  pretest  at  the  session  level. 
Another  alternative  considered  and  still  to  be  investigated,  might  be  to  pretest  at  the  course  level  and  infer 
knowledge  from  the  results.  For  the  practical  components,  students  might  be  able  to  demonstrate  the 
possession  of  some  particular  skills;  this  then  needs  to  be  handled  manually  and/or  electronically. 

The  content  of  the  delivery  system  is  relatively  static;  that  is,  the  teaching  components  that  are  delivered 
through  a range  of  players  accommodating  text,  pdf  modules,  video,  audio,  animations,  etc.  and  any  other 
media  format  that  might  become  available  in  the  future,  are  stored  at  the  client  side.  The  components  of  the 
delivery  system  involving  the  course  structure  and  assessment  pieces,  student  profiles  and  assessment  results, 
data  of  those  on-line,  are  stored  on  the  server  side. 

Assessment  at  the  concept  level  is  based  on  a competency  model;  an  evaluation  is  made  of  a student’s 
understanding  of  a concept  through  questioning.  Mastery  of  these  randomly  selected  questions  is  reflected  in 
the  student  profile  that  drives  the  adaptivity  within  the  system.  Where  a concept  is  a skill  or  task,  students 
may  be  asked  to  do  something  and  have  it  registered  manually;  for  example,  the  concept  might  be  the 
scanning  of  an  image.  The  concept  might  be  demonstrated  and  the  student  asked  to  perform  the  task; 
completion  of  this  task  is  manually  registered.  We  distribute  weekly  tasks  and  longer  summative 
assignments  etc.  through  a server  when  required. 

d.  Instructional  design 

The  hypertext/hypermedia  learning  environment  constructed  allows  the  instructor  to  employ  a range  of 
instruct  ional  methodologies.  Constructivist  principles  underpin  the  environment  and  are  supported;  students 
create  multimedia  artefacts  and  place  them  into  their  workspace  (or  on  the  WWW),  search  out  content,  and 
satisfy  authentic  tasks.  These  in  total,  provide  the  macro  level  scaffolding  [2]  that  allow  the  students  to  use 
particular  content  in  different,  albeit  overlapping,  contexts. 

We  are  developing  tools  (search  engine,  glossary,  concept  map)  to  enable  the  user  to  search  and  access 
concepts  directly  as  required.  As  students  become  more  familiar  with  the  environment,  the  instructional 
strategies  may  vary;  for  example,  more  problem  based  learning  activities  could  be  introduced.  These 
activities  might  relate  to  the  construction  of  multimedia  artefacts,  the  acquisition  of  some  understanding,  or 
the  establishment  of  a reasoned  point  of  view  with  regards  some  ethical  issue  like  copyright. 

Student/student  and  tutor/student  interactions  are  being  established  via  online  discussion  groups  which  have 
the  potential  to  support  collaborative  learning.  This  could  be  tutorial  discussion  groups  with  a number  of  their 
peers  currently  logged  on  addressing  some  weekly  task.  The  number  participating  in  these  discussion  groups 
is  4 by  default;  however,  this  is  a system  parameter  and  can  be  changed.  Students  are  initially  allocated  on  a 
first  in,  first  allocated  basis  but  any  part  of  the  student  profile  could  be  used  for  this  purpose 


An  on-line  instructional  management  system  has  been  constructed  to  establish  courses  based  on  concepts  and 
sessions.  An  ‘Administration  Tool’,  enables  course  coordinators  to  create  particular  courses  and  direct 
content  at  particular  groups.  We  have  the  potential  to  administer  concurrent  versions  of  a course  directed  at 
particular  groups. 

Lecturers  and  tutors  have  WWW  access  to  our  student  model.  Lectures  can  add/delete  a tutorial  group,  add  a 
new  student,  search  for  existing  student,  and  update  student’s  detail  such  as  tutorial  scores,  assignmait  scores 
and  exam  scores.  Tutors  have  access  to  update  and  search  facilities  of  students’  tutorial  and  assignment 


e.  Student  and  course  administration 


details. 


AMLE  uses  and  maintains  student  profiles  that  contain  a summary  of  the  student’s  competencies  and  other 
preference  information.  The  coupling  of  the  course  model  with  the  student  profile  has  the  potential  to  allow 
students  to  proceed  at  their  own  rate  through  the  learning  material.  Further,  the  management  system  enables 
tutors  to  enter  marks  electronically  and  subsequently  manage  the  assessment  components  of  the  course. 
Reporting  mechanisms  then  cope  with  student,  tutor  and  course  co-ordinator  requirements. 


4.  Current  and  Future  Work 

Our  major  concern  has  been  to  provide  adaptivity  and  interactivity  for  our  students  since  we  recognise  that 
our  students’  backgrounds  and  cultures  contain  a wide  range  of  learning  styles  and  expectations.  We  have 
used  the  on-line  management  components  of  the  proposed  system  to  maintain  the  assessment  profiles  of 
current  students.  However,  the  web-based  materials  that  have  been  used  thus  far  have  not  provided  us  with 
the  granularity  at  the  concept  level  to  enable  adaptive  technologies.  To  this  end,  we  have  created  a range  of 
prototypes  with  varying  interfaces  to  access  the  course  model,  and  a number  of  players  to  display  the  content. 
These  will  be  trialled  with  students  in  the  near  future, 

A range  of  tools  including  tutorial  chat  facilities,  notebooks  and  email,  are  provided  in  the  interface  for  the 
users  to  take  advantage  of  Further  work  will  involve  the  pre-testing  of  students  and  the  full  utilisation  of 
student  preference  data  collected. 

The  range  a questions  and  question  types  that  are  randomly  presented  and  marked  by  the  system  are  a topic 
of  current  investigation.  Results  from  these  form  the  basis  of  the  student's  competency  model  and  the 
subsequent  adaptivity.  Tutors  are  able  to  enter  marks  directly  into  the  student  competency  profile  for  tasks 
where  the  competency  for  a concept  can  be  demonstrated.  The  studaifs  competency  profile  is  further 
enhanced  by  an  assessment  profile  that  includes  summative  activities  that  could  extend  over  a number  of 
sessions.  Both  profiles  are  available  to  the  students  through  a WWW  browser  interface  that  displays  their 
current  competency  and  assessment  states, 

A number  of  instructional  methodologies  and  educational  activities  underpinned  by  a constructivist 
philosophy  are  being  explored.  So  too  are  the  different  tools  that  will  be  available  within  the  interface;  for 
example,  an  online  helpdesk  that  simulates  a tutorial  helpdesk  situation  where  students  pose  questions  to 
tutors  who  are  currently  on  line 

5.  Conclusion 

We  are  developing  components  of  a multimedia  learning  environment  that  we  feel  have  the  potential  to 
enhance  on-line  learning.  The  system  presents  course  materials  as  concepts  and  adapts  to  prior  knowledge 
through  the  use  of  link  annotation.  We  have  also  introduced  stereotypes  into  the  overlay  student  model  and 
have  coupled  this  to  different  content  presentations.  Students  can  also  specify  the  type  of  media  presentation 
they  prefer  to  use  within  the  concept  viewer.  A range  of  other  tools  like  search  engines,  indexes,  helpdesk, 
tutorial  groupings  etc.  will  allow  the  lecturer  to  employ  a range  of  instructional  methodologies  to  satisfy  the 
requirement  of  the  course. 

The  dynamic  components  of  the  learning  environment  i.e.  assessment,  course  structure,  and  other 
instructional  artefacts,  are  stored  on  a remote  server  and  are  accessed  through  the  WWW.  Tutors  have 
convenient  on-line  access  to  student  groupings  to  enable  marks  entry. 

We  utilise  two  forms  of  assessment;  namely  formative  assessment  that  includes  the  mastery  (or  competency) 
of  particular  concepts  or  skills  conducted  generally  at  a weekly  level,  and  summative  as  sessment  that 
necessitates  the  use  a range  of  knowledge/skills  in  its  satisfactory  completion.  These  are  recorded  within  the 
overlay  model  of  the  student  profile.  Student  preferences  and  settings,  along  with  marks  etc  that  make  up  the 
assessment  profile  of  the  students  model,  are  available  to  the  student  for  perusal  and  alteration. 

6.  References 

1.  Anderson,  J.,  The  architecture  of  cognition.  1983,  Cambridge:  MA. 

2.  Bannan-Ritland,  B.,  N.  Dabbagh,  and  K.  Murphy, Learning  Object  Systems  as  Constructivist  Learning 
Environments:  Related  Assumptions,  Theories  and  Applications , in  The  Instructional  Use  of  Learning 


1105 


Objects.,  D.A.  Wiley,  Editor.  2001,  Association  for  Educational  Communications  and  Technology: 
Bloomington,  IN. 

3.  Boalch,  G.  WWW  as  an  Educational  Support  Medium:  An  Australian  Case  Study,  in  AusWeb96.  1996. 
Gold  Coast,  Australia. 

4.  Brusilovsky,  P.,  Methods  and  Techniques  of  Adaptive  Hypermedia.  User  Modeling  and  User-Adapted 
Interaction,  1996.  Vol  6(2-3):  p.  87-129. 

5.  Brusilovsky,  P.  Adaptive  Hypermedia:  From  Intelligent  Tutoring  Systems  to  Web^ Based  Education  in 
Intelligent  Tutoring  Systems.  2000.  Montreal,  Canada:  Springer- Verlag. 

6.  Brusilovsky,  P.,  J.  Eklund,  and  E.  Schwarz,  Web-based  Education  for  all:  a tool  for  development 
adaptive  courseware.  Computer  Networks  and  ISDN  Systems,  1998.  30:  p.  291  -300. 

7.  Committee,  I.L.,  Learning  Objects  Metadata  Draft  v4. 1.  2000. 

8.  Eklund,  J.  and  P.  Eklund.  Integrating  the  Web  and  the  teaching  of  technology:  Cases  across  two 
universities,  in  AusWeb96.  1996.  Gold  Coast,  Australia. 

9.  Herrington,  J.  and  R.  Oliver,  Using  situated  learning  and  multimedia  to  investigate  higher-order 
thinking.  Journal  of  Interactive  Learning  Research,  1999.  10(1):  p.  3-24. 

10.  Jonassen,  D.,  K.L.  Peck,  and  B.G.  Wilson,  Learning  with  Technology:  A Constructivist  Perspective. 

1999,  Upper  Saddle,  NJ:  Merrill,  Prentice  Hall. 

1 1.  Jones,  D.  Solving  some  problems  of  University  Education:  A Case  Study,  in  AusWeb96.  1996.  Gold 
Coast,  Australia. 

12.  Kay,  J.  Stereotypes,  Student  Models  and  Scrutability.  in  ITS2000.  2000.  Montreal,  Canada:  Springer- 
Verlag. 

13.  Looi,  C.K.  and  D.  Ang,  A multimedia -enhanced  collaborative  learning  environment.  Jouamal  of 
Computer  Assisted  Learning,  2000.  16:  p.  2-13. 

14.  Murray,  T.,  C.  Condit,  and  E.  Haugsjaa.  A Preliminary  Framework  for  Concept-Based  Adaptive 
Hypermedia  in  ITS-98  workshop  on  WWW- Based  Tutoring.  1998. 

15.  Pilar  da  Silvar,  D.,  R.  Van  Dum,  and  H.  Olivie.  Concepts  and  documents  for  adaptive  educational 
hypermedia:  a model  and  a prototype,  in  2nd  Workshop  on  Adaptive  Hypertext  and  Hypermedia, 
HYPERTEXT’98.  1998.  Pittsburgh,  USA. 

16.  Ricketts,  J.,  et  al..  Multi-media  - Asynchronous  distributed  education.  Social  science  computer  review, 

2000.  18(2):  p.  132-146. 

1 7.  Slay,  J.  Using  the  World  Wide  Web  to  Create  Foundations  for  Lifelong  Learning  - An  Australian 
Perspective  in  Teleteaching,  IFIP  World  Computer  Congress98.  1998.  Vienna. 

1 8.  Slay,  J.  Using  the  WWW  to  Create  an  Effective  Cross-Cultural  Learning  Environment,  in  Hong  Kong 
Web  Symposium  98.  1998.  Hong  Kong  University. 

19.  UNIS  A,  Graduate  Qualities -a  brief  guide  to  assessing  students  for  Graduate  Qualities,  UNISA 
Learning  Connection. 

20.  Wiley,  D.A.,  II,  Connecting  learning  objects  to  instructional  design  theory:  A definition,  a metaphor, 
and  a taxaonomy,  in  The  Instructional  Use  of  Learning  Objects.,  D.A.  Wiley,  Editor.  2001,  Association 
for  Educational  Communications  and  Technology:  Bloomington,  IN. 


1106 


The  eLearning  Concept  at  Helsinki  University  of  Technology,  Finland: 
Working  with  the  7 -Step  Approach  on  the  eLearning  Platform 


Inker!  Laaksonen 

Helsinki  University  of  Technology 
Finland 

inkeridaaksonen@hut.fl,  tiina.front@uta.fl 
Tiina  Front 

Helsinki  University  of  Technology 
Finland 

tiina.front@uta.fi 


Abstract:  The  E-Education  and  Research  Unit  (www.cs.hut.fi/itlab)  has  provided  multiformat 
eLearning  courses  since  spring  2000.  The  eLearning  platforms  underpinning  these  programs 
are  WebCT  (www.webct.com)  and  Edulink  (www.edusolutions.fi).  In  autumn  2001  the 
Education  Unit  launched  its  first  fully  Internet  based  courses  using  the  investigative  learning 
theory  as  a background  theory  and  applying  the  well-known  7- Step  Model  of  the  Problem  - 
based  learning  The  7- Steps  Model  was  supplemented  with  orientation  module  which 
included  personal  assignments  such  as  setting  own  learning  objectives  and  selecting  the 
methods  to  reach  them.  Mediamaisteri  LTD’s  (www.mediamaisteri.com)  eEvaluation  tool 
was  selected  for  self-evaluation.  Two  studies  are  in  process  concerning  these  courses:  adults’ 
comprehension  of  ICT  as  a tool  of  developing  and  creating  their  own  expertise  are  examined 
as  well  as  the  development  of  critical  thinking  as  the  manifestation  of  the  students’ 
epistemological  autonomy. 


Working  with  the  7-Step  Approach  on  the  eLearning  Platform 

The  E-Education  and  Research  Unit  {www,cs,hut,fi/itlab)  has  provided  multiformat  eLearning  courses 
since  spring  2000.  The  eLearning  platforms  underpinning  these  programs  are  WebCT  {www,webct,com)  and 
Edulink  {wfWfWf,edusolutionsfi),  In  autumn  2001  the  Education  Unit  launched  its  first  fully  Internet-based  courses 
using  the  investigative  learning  theory  as  a background  theory  and  applying  the  well-known  7-Step  Model  of  the 
Problem-based  learning.  The  focus  of  the  7-Step  Model  is  in  rational,  individual  problem  solving  and 
information  gathering  which  proceeds  step  by  step.  The  steps  of  the  model  are  clarification  of  terms  associated 
with  the  case,  determining  the  problem,  analyzing  through  brainstorming,  constructing  an  explanatory  model  to 
describe  the  phenomenon,  formulating  learning  objectives,  information  gathering  by  self  study  and  explication 
and  evaluation  of  knowledge  (Schmidt  1983.)  The  method  trains  to  solve  problems  in  groups,  problems  of 
which  one  have  no  previous  knowledge  or  information. 

The  eLearning  courses  lasted  from  August  to  December  2001,  accounting  for  2 or  3 credits.  Some  60 
students  completed  these  courses.  The  7-Steps  Model  was  supplemented  with  orientation  module  which  included 
personal  assignments  such  as  setting  own  learning  objectives  and  selecting  the  methods  to  reach  them. 
Mediamaisteri  Ltd’s  {yvww,mediamaisteri,cord)  eEvaluation  tool  was  selected  for  self-evaluation. 

The  groups  organized  themselves  independently,  with  each  student  joining  a group  that  had  targets 
closely  matching  his/her  own.  After  an  orientation  session,  each  group  was  given  a case-study  and  the  general 
discussion  forum  provided  by  the  Edulink  platform  was  closed.  In  line  with  the  7-Step  Model,  the  groups  elected 
a chairman  and  a secretary  at  the  very  beginning.  The  teacher’s  role  was  to  support  the  students  and  the  group  as 
and  when  required.  Group  work  proceeded  independently  and  the  tutors  played  their  planned  guiding  role. 
Technical  support  and  the  tutors  of  the  groups  monitored  the  step-by-step  progress  and  the  completion  of  the 
individual  learning  tasks  while  also  providing  related  guidance  and  feedback. 

The  Studies  and  the  Preliminary  Results 

The  main  issue  addressed  by  Mrs.  Inker!  Laaksonen’s  Master’s  Thesis  is  to  clarify  adults* 
comprehension  of  Information  and  Communication  Technology  as  a tool  of  developing  and  creating  their  own 
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expertise.  Another  issue  that  the  thesis  addresses  is  how  adult  students  perceive  their  own  expertise  and  whether 
the  learning  culture  changes  as  it  moves  towards  eLearning.  The  target  group  is  adult  learners  who  are  studying 
Master’  Degree  in  Computer  Science  while  being  employed  on  full-time  bases.  The  qualitative  basis,  and  the 
basis  for  the  research  methodology  of  the  thesis,  is  existential  phenomenology.  The  research  method  applied  is 
Colaizzi’s  method  of  phenomenological  psychology  (Lehtovaara  1995;  Valle  & Hailing  1989,  41-59). 

The  method  selected  for  gathering  material  for  the  thesis  was  written  stories.  The  preliminary  results  of 
the  thesis  indicate  that  group  work  on  the  Internet  was  p^ceived  to  be  easier  than  conventional  face-to-face 
group  work,  and  that  shy  or  quiet  students  could  make  themselves  heard  more  easily  over  the  Internet: 
''Discussion  forums  of  eLearning  platform  have  helped  me  take  other  people's  opinions  into  account  ”,  "Group 
work  is  much  freer  over  the  Internet  than  in  faceAoface  group^\  On  the  other  hand,  the  eLearning  culture  is 
taking  its  first  steps,  and  this  is  reflected  in  the  stories.  ICT  plays  a large  role  for  some  of  the  writers:  "ICT  has 
played  a very  large  role  in  developing  my  expertise'',  "Without  doubt  the  Internet  is  a natural  place  for  most 
people  to  develop  themselves,  seek  information  and  keep  in  touch".  Some  writers  miss  social  interaction:  "All  of 
us  want  to  talk  with  other  people  about  other  topics  besides  work  or  school  every  now  and  then,  but  on  the 
Internet  that  kind  of  small-talk  is  impossible  and  therefore  getting  to  know  someone  else  is  impossible  in  that 
environment'.  The  ease  of  searching  for  and  finding  information  on  the  Internet  in  the  course  of  developing 
one’s  expertise  was  underscored  in  the  stories,  although  critical  inspection  of  the  authenticity  of  the  information 
was  regarded  as  a problem:  "I  think  that  faced  with  today 's  information  overload,  filtering  out  correct  and  useful 
information  from  incorrect  and  useless  information  has  become  a key  issud'. 

Facilitating  autonomy  could  be  regarded  as  a broad  principle  in  the  philosophy  of  education,  which  can 
be  carried  out  in  different  learning  environments  and  with  learners  of  different  ages.  Autonomy  has  become  a 
part  of  distance  education  as  study  modes  in  adult  education  have  developed.  The  aim  of  Mrs.  Tiina  Front’s 
doctoral  thesis  is  to  specify  the  concepts  and  conceptions  of  the  learner’s  autonomy  and  of  open  learning 
environments,  especially  eLearning  aivironments.  The  main  goal  is  to  define  the  criteria  or  dimensions  that 
could  be  used  to  analyze  the  development  of  the  student’s  autonomy  and  the  roles  of  support,  interaction  and 
distance  in  this  developing  process.  The  learner’s  autonomy  is  defined  here  as  the  autonomy  of  action  and 
autonomy  of  thinking  (Launis  1994),  on  the  one  hand,  and  the  epistemological  autonomy  and  trans-situational 
autonomy,  on  the  other  hand  (Candy  1991).  The  development  of  critical  thinking  is  the  manifestation  of 
epistemological  autonomy,  which  is  the  focus  here.  Garrison  et  al.  (2000)  describe  the  development  of  critical 
thinking  in  the  practical  inquiry  mode  1. 

The  data  gathered  from  the  ©courses  at  the  Helsinki  University  of  Technology  will  be  coded  and 
analyzed  according  to  the  four  categories  derived  from  the  cognitive  presence  descriptors  (Garrison  et  al.  2001). 
The  results  will  show  how  the  7-Step  model  supports  the  development  of  critical  thinking  in  terms  of  increasing 
the  learner’s  autonomy.  The  preliminary  results  are  presented  in  ED-Media. 
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Abstract:  Project  COPE  (Computer  Organized  Physical  Education)  is  a data 
management  system  designed  to  help  teachers  productively  deal  with  the  inordinate 
amount  of  assessment  data  they  encounter.  When  teachers  collect  performance  data  on 
paper  based  forms  it  is  difficult  for  them  to  use  the  information  efficiently.  Project  COPE 
involves  the  use  of  a computerized  data  management  system  coupled  with  hand  held 
recording  units  to  record  and  organize  information  about  student  performance.  This 
technology  is  useful  in  all  educational  settings  including  the  classroom  and  the 
gymnasium. 


Introduction 

Project  COPE  (Computer  Organized  Physical  Education)  began  as  a data  management  system  to 
help  elementary  physical  education  teachers  productively  deal  with  the  inordinate  amount  of  assessment 
data  they  encounter  in  teaching  (Lambdin,  1997).  Elementary  physical  education  teachers  often  teach  300- 
600  students  and  have  contact  with  the  same  students  over  a five  to  seven  year  period  from  kindergarten  to 
fifth  or  sixth  grade.  However,  information  about  student  needs  often  is  not  passed  on  from  year  to  year  in 
useable  formats  because  of  the  overwhelming  number  of  students  and  difficulty  in  organizing  the  huge 
amount  of  data.  When  teachers  collect  performance  data  on  paper  forms  it  is  difficult  to  use  them 
efficiently.  It  is  very  time  consuming  for  teachers  to  transfer  data  from  paper  to  computer  systems  after 
class  is  over.  This  causes  teachers  either  to  spend  a great  deal  of  time  doing  paperwork  or  to  give  up 
attempts  at  tracking  student  progress.  Because  the  cost  to  benefit  ratio  of  data  recording  is  so  high,  teachers 
often  have  chosen  to  collect  only  the  simplest  data  on  student  learning  (test  scores).  The  data  often  are 
used  for  the  sole  purpose  of  calculating  a grade  for  the  report  card  and  then  discarded  when  a new  year  and 
new  grade  book  are  started.  Given  the  difficulty  of  recording  and  manipulating  student  data,  the  teacher's 
memory  has  been  considered  the  best  device  for  identifying  student  needs  when  planning  for  instruction. 


Dynamic  Data  Management  System  Components 

Project  COPE  uses  a database  created  in  FileMaker  Pro  to  organize  the  data,  and  hand  held  units 
to  aid  in  one  step  data  entry.  The  use  of  PDAs  (Personal  Digital  Assistant)  allows  direct  data  input  during 
class  activities  rather  than  having  to  transpose  from  teacher  notes.  When  dealing  with  hundreds  of  students 
and  scores  of  objectives  this  direct  entry  option  is  crucial.  Just  as  hand  held  units  are  used  by  clerks  in 
stores  and  meter  readers  on  the  streets,  teachers  across  the  country  are  beginning  to  use  PDAs  to  provide 
for  efficient  assessment  data  entry  during  teaching  and  learning  activities. 


The  current  system  has  been  developed  so  that  it  could  be  used  in  any  content  area.  There  are  six 
main  functions  in  Project  COPE.  They  are  Students,  Assessments,  Individual  Data,  Class  Summary,  Help 
Data,  Mastery  Data,  and  Reports.  Using  the  various  features  supplied  with  the  Project  COPE  templates, 
teachers  identify  which  grades,  classes,  and  semesters  they  are  teaching.  If  necessary,  teachers  can  create 
new  grades  or  a new  class  with  ease  within  the  program.  After  initial  setup  the  teacher  is  able  to  assign 
three  or  four  objectives  to  assess  on  any  given  day.  This  information  is  then  exported  from  the  desktop 
computer  to  the  personal  hand  held  device  using  Filemaker  Pro  Mobile.  Teachers  enter  data  for  students  on 
the  various  objectives  using  the  hand  held  device  during  their  teaching.  After  data  are  entered  in  the  PDA, 
the  sync  capabilities  of  the  PDA  transfer  the  collected  data  from  the  hand  held  device  to  a computer.  Using 
the  find,  sort,  and  script  capabilities  of  the  database  teachers  are  able  to  easily  determine  which  students 
have  mastered  each  objective  and  who  still  needs  additional  help.  The  organization  of  information  in  this 
way  is  invaluable  in  making  instructional  and  lesson  planning  decisions.  Another  advantage  of  utilizing 
Project  COPE  in  classroom  teaching  is  that  feedback  information  can  be  printed  out  in  customized  report 
cards  which  indicate  specific  individual  mastery  of  learning  objectives  rather  than  just  a letter  grade.  This 
enables  complete  and  useful  information  about  learning  and  performance  to  be  viewed  by  students’  and 
their  parents.  Finally,  summary  information  for  use  in  program  evaluation  can  easily  be  compiled. 


Conclusion 

The  impact  of  the  use  of  dynamic  and  interactive  systems  such  as  Project  COPE  is  enormous.  The 
general  system  is  equally  useful  in  recording  emergent  reading  characteristics,  use  of  problem  solving 
mathematical  strategies,  performance  characteristics  for  mature  throwing  and  catching,  and  the  exhibition 
of  pro -social  behaviors.  Teachers  are  able  to  efficiently  record  data,  easily  manage  data  and  effectively  use 
the  information  to  individualize  instruction  and  monitor  the  success  of  their  own  teaching.  Students  receive 
the  benefit  of  appropriately  targeted  instruction.  Parents  receive  specific  information  about  their  children’s 
learning  needs  and  teachers  and  administrators  are  able  to  obtain  often  elusive  summary  data  on  program 
impact.  If  knowledge  is  power.  Project  COPE  provides  teachers,  students,  parents  and  administrators  with 
increased  power  in  learning  environments. 
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Hard  Choices  for  Individual  Situations  by  Bruce  Landon,  Ph.D. 


Overview 

The  Decision  Dinner  - a parable  about  the  weighted  average  decision  strategy 
by  Bruce  Landon  and  Nancy 

The  novelty  of  the  proposed  decision  process  can  be  illustrated  by  a parable  about  a small  group  of  decision  makers  and 
relating  it  back  to  the  online  learning  product  selection  situation.  First,  assume  the  task  of  selecting  a restaurant  for  a board 
of  directors’  first  dinner  in  a new  city.  Also  imagine  that  this  dinner  is  pretty  important,  perhaps  not  quite  as  important  as  the 
Nixon  dinner  in  China,  but  a dinner  of  consequence,  nonetheless.  The  first  steps  in  the  process  would  be  to  scan  the 
available  resources  to  see  which  restaurants  (like  product  providers)  are  available  and  then  to  proceed  to  get  menus  (lists  of 
the  features  available  in  each  provider’s  product).  The  menu  information  could  be  augmented  with  reviews  of  the  items  on 
the  menu  but  this  would  still  not  be  enough.  What  the  decision  making  group  needs  to  know  is  what  kind  of  items  the 
members  of  the  board  actually  like  to  have  for  dinner,  i.e.  which  menu  items  are  important  to  them.  So  the  group  decides  to 
collect  some  more  information  by  sending  out  a stakeholder  survey  to  find  out  what  each  of  the  board  members  enjoys  for 
dinner. 

To  construct  this  survey,  the  decision  making  group  had  to  make  a short  list  of  restaurants  (product  providers)  and  from  their 
menus  make  a short  list  of  the  most  promising  items  (important  features)  because  the  board  members  would  have  never 
responded  to  a survey  that  was  as  long  as  all  of  the  menus  from  every  restaurant.  The  group  sent  out  its  survey  and 
analyzed  the  results  to  calculate  the  importance  weighting  for  each  of  the  dinner  menu  items.  Knowing  the  consequences  of 
making  a mistake  on  selecting  a restaurant  for  both  the  decision  makers  and  the  board  of  directors,  they  elect  to  use  the 
most  rational  decision  making  strategy  known:  the  weighted  averaging  strategy. 

To  be  able  to  use  this  strategy,  the  decision  makers  will  have  to  find  out  how  suitable  the  menu  items  actually  are.  So  they 
have  dinner  at  each  of  the  restaurants  to  taste  and  judge  the  quality  for  themselves.  Not  only  do  they  have  to  carefully  taste 
the  suitably  of  the  menu  items  (features)  but  they  must  rate  each  one  on  a consistent  scale  so  that  the  scores  can  be  used  in 
the  weighted  average  strategy.  They  decide  to  all  use  a scale  where  0 means  awful  and  10  means  wonderful  on  suitability 
rating  forms  used  when  they  go  to  the  restaurants.  Then  night  after  night,  dinner  after  dinner,  they  evaluate  the  suitability  of 
each  of  the  short  listed  restaurants  on  all  of  the  important  menu  items.  Finally  using  the  board  members’  weights  for  each 
menu  item  and  the  decision  makers’  suitability  scores  for  each  menu  item  at  each  restaurant,  they  calculate  the  winner  from 
the  data  collected. 

The  winning  restaurant  is  the  one  that  has  the  highest  average  score  when  the  menu  item  weights  are  multiplied  by  the  menu 
item  suitability  scores.  The  decision  makers  make  the  reservations  with  the  winning  restaurant  and  the  board  of  directors 
proceeds  to  have  a great  dinner.  The  board  of  directors  was  impressed  by  the  rationality  of  the  group  of  decision  makers  but 
in  truth  they  were  more  impressed  by  the  dinner  after  which  they  \oted  to  give  bonuses  to  the  decision  makers  and  all  live 
happily  ever  after. 

Selecting  an  online  eductional  delivery  application  is  analogous  to  selecting  the  restaurant  in  the  parable  above.  The 
members  of  the  board  of  directors  are  analogous  to  the  stakeholders  who  will  be  impacted  by  the  selected  product  i.e.  the 
students,  faculty,  staff,  and  administrators.  If  the  consequences  of  making  an  error  and  selecting  the  wrong  product  were  not 
important  there  would  be  no  need  to  be  so  rational  as  to  use  the  weighted  average  decision  strategy  and  the  group  of 
decision  makers  could  use  a less  rational  process.  When  the  consequences  are  important  such  as  in  the  assigning  of  course 
grades  then  a very  similar  rational  process  is  most  often  used  called  weighted  averaging  grading. 

This  chapter  will  focus  on  faculty  use  of  a decision-making  process  for  complex  situations.  The  analysis  part  of  the  process 
describes  and  compares  course  management  software  focusing  on:  technical  specifications,  instructional  design  values, 
tools  and  features,  ease  of  use,  and  standards  compliance.  The  extensive  comparisons  provide  faculty  with  side-by-side 
product  feature  descriptions.  The  decision-making  focus  of  the  chapter  and  of  the  companion  site  is  on  supporting  a detailed 
rational  decision  process  for  selecting  course  management  software.  This  decision  process  uses  a grading  style  model 
familiar  to  educators  that  compensates  for  the  cognitive  illusions  and  limitations  of  decision  makers  to  achieve  a transparent 
decision  process  for  selecting  the  "best"  application  for  each  local  situation.  The  current  site  obviously  meets  needs  for 
decision-making  support  as  evidenced  by  16k  visitors  a month  to  the  site.  By  the  fall  of  2002,  there  will  be  similar  decision- 
making supports  available  which  will  focus  on  student  services,  teaching  technologies  and  online  education  policies. 

Introduction 

Faculty  make  decisions  every  day  and  most  of  them  are  informal  choices  about  what  looks  good  on  the  menu  for  lunch. 
Sometimes,  faculty  make  quite  formal  and  rational  decisions  about  which  student  gets  an  A and  which  student  gets  an  F 
3 a strategy  of  weighted  averaging.  The  way  in  which  decisions  are  made  often  reflects  the  importance  of  the  decision 


and  the  situation  as  well  as  the  decision  maker.  The  research  in  psychology  over  the  last  few  decades  has  illuminated  some 
of  the  strengths  and  weaknesses  in  the  way  that  people  make  decisions  with  the  important  finding  that  in  many  situations 
people  are  less  than  rational  decision  makers.  When  a person  is  overloaded  with  information  or  under  time  pressure  then 
their  decision-making  can  be  quite  compromised  and  they  seem  to  use  “rules  of  thumb”  to  make  a decision.  The  case  of 
selecting  an  online  educational  delivery  system  is  examined  from  this  perspective  in  this  chapter.  First  the  difficulties  facing 
the  decision  maker  are  addressed  and  then  a remedy  is  demonstrated  to  show  how  decision-making  can  be  assisted  to 
prevent  the  decision  process  from  being  compromised  by  these  “rules  of  thumb”  using  a web  tool  for  comparative  analysis. 

Why  choices  are  difficult 

Too  many  products 

The  situation  is  reminiscent  of  the  early  automobile  industry  when  there  were  over  a hundred  automakers.  Like  the  early 
auto  industry  many  other  competitors  will  not  survive  in  the  long  run.  The  current  front-runners:  WebCT  and  BlackBoard 
started  out  as  small  almost  personal  projects  a few  years  ago  and  are  now  multi-million  dollar  enterprises  that  span  the  global 
education  marketplace.  There  may  well  be  others  out  the  current  products  that  will  evolve  into  the  major  brands  of  tomorrow 
as  the  market  matures.  The  point  of  this  analogy  is  that  selecting  a learning  management  system  is  much  more  complicated 
than  selecting  a new  car,  in  part,  because  there  are  so  many  competing  models.  There  are  over  100  products  in  this 
marketplace  that  range  from  relatively  simple  limited  function  applications  like  WebBoard  which  just  provides  threaded 
discussions  and  chat  to  complex  enterprise  wide  learning  systems  that  link  to  backend  databases  and  administrative  data 
systems  used  by  colleges  and  universities.  Previously  in  1997,  I had  attempted  to  characterize  these  products  as 
components  or  suites  by  this  categorization  turned  out  not  to  be  useful  because  there  were  so  many  variations  from  product 
to  product  he  became  difficult  to  classify  new  products.  Also,  the  products  in  this  marketplace  evolve  quite  rapidly  with  a new 
model  or  version  almost  every  year,  like  automobiles.  So  to  deal  with  this  rapidly  changing  market  it  seems  better  to 
characterize  products  in  terms  of  the  features  that  they  provide. 

Too  many  product  features 

The  feature  and  tools  are  organized  in  terms  of  the  group  that  they  are  most  likely  to  be  interested  in  them:  Learners,  Learner 
Supporters  and  Technical  Administrators.  These  major  groupings  are  further  divided  into  subtypes  of  features/tools  and 
finally  Into  the  individual  features  of  the  product.  This  schema  has  grown  over  the  years  and  the  language  has  changed  as 
well  so  that  some  of  the  terms  have  been  replaces  in  common  usage  by  other  terms  as  in  the  case  of  Newsgroup  being 
supplanted  by  the  term  discussion  or  forum.  A glossary  of  terms  provides  some  help  for  those  who  are  unfamiliar  with  this 
area  (httD://wwww.c2t2.ca/landon!ine/alossarv.htmn.  but  the  fundamental  problem  is  that  there  too  many  features/tool  about 
too  many  products  to  keep  track  of  easily  (products  * features  = 3474  facts)  especially  to  keep  track  of  in  your  head.  The 
most  frequent  problem  in  making  good  decisions  in  this  kind  of  situation  is  that  of  forgetting  or  omitting  some  important  part  of 
the  problem  when  making  the  decision.  Often  we  are  overly  optimistic  about  how  big  a decision  problem  we  can  do  “in  out 
head.” 

Too  little  working  memory  in  decision-makers 

Nobel  laureate,  Herbert  Simon  (1979)  characterized  the  human  decision  making  capacity  as  having  “Bounded  Rationality” 
(also  see  Gigerenzer  & Selten,  2001).  When  people  face  decisions  they  operate  within  the  bounds  or  capacities  of  the 
human  mind.  They  are  limited  by  fallible  perceptions,  wandering  attention,  faulty  memories,  and  fluctuating  information- 
processing abilities.  People  don’t  optimize,  but  instead  resort  to  simplifying  rules  of  thumb  in  order  to  proceed  with  their 
decision-making.  These  limitations  have  been  investigated  for  decades  with  the  repeated  finding  that  working  memory 
limitations  play  a major  role  in  limiting  the  ability  to  make  rational  decisions. 

The  notion  of  limited  memory  was  articulated  well  by  George  Miller  (1953)  as  the  magical  number  seven  plus  or  minus  two 
“things”  that  you  hold  in  your  mind  at  once  while  working  on  a problem  (presumably  due  to  a phonological  loop  process, 
Baddeley  1986,  cited  in  Anderson  2000).  This  limited  working  memory  is  a profound  handicap  for  a rational  decision  maker 
because  when  one  works  with  ideas  in  the  head  then  one  is  moving  them  around  sort  of  like  a juggler.  When  there  are  too 
many  “things”  then  some  of  them  will  get  dropped  or  set  aside  from  consideration  in  the  decision-making.  Stated  in  another 
way,  decision-making  becomes  more  difficult  as  the  number  of  things  to  consider  increases  and  quickly  reaches  a point  of 
overloading  working  memory. 

When  facing  difficult  decisions,  people  encounter  overload  and  then  they  resort  to  ways  of  coping  that  involve  cognitive 
shortcuts.  When  overloaded,  they  are  more  susceptible  to  cognitive  illusions  (Kahneman  and  Tversky,  1996).  Cognitive 
illusions  are  similar  to  perceptual  illusions  in  that  they  stem  from  the  fact  that  in  both  types  of  illusions  the  person  can  focus 
only  on  a small  part  of  the  situation  at  one  time.  In  the  case  of  perceptual  Illusions  this  is  because  the  fovea  of  the  eye  that 
sees  fine  detail  encodes  only  a small  segment  of  the  spatial  panorama  at  each  fixation.  In  the  case  of  cognitive  illusions  this 
Is  largely  because  the  conscious  working  memory  can  hold  only  a few  “things”  and  the  rest  must  be  inferred  or  extrapolated 
in  ways  that  are  sometimes  biased.  In  both  types  of  illusions  the  person  is  not  necessarily  aware  of  any  “illusion”  at  all  and 
may  believe  that  they  are  “seeing  everything  correctly.” 
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Too  many  Cognitive  iiiusions  in  Decision-Making 

The  discrepancies  associated  with  cognitive  illusions  may  be  seen  a little  more  clearly  in  contrast  to  an  idealized  rational 
decision  process.  In  the  idealized  process,  one  starts  by  selecting  relevant  features/criteria  and  then  assigning  an 
importance  weighting  to  each  of  those  features.  Next  one  evaluates  each  choice  option  on  each  of  those  important  features 
and  assigns  a suitability  score.  Finally  one  makes  the  rational  decision  to  choose  the  option  with  the  highest  weighted 
average  suitability  score  (the  average  of  the  weights  for  a feature  multiplied  by  the  feature’s  suitability  score).  Choosing  the 
option  with  the  best  score  is  the  rational  decision  (Keeney  and  Raiffa,  1976  as  cited  in  Roe,  Busemeyer  and  Townsend, 
2001).  With  only  a couple  of  optional  choices  and  a few  important  features  it  is  likely  the  limitations  of  working  memory  would 
be  overwhelmed  even  if  one  were  able  to  do  the  mental  multiplication  part.  When  one  is  overwhelmed  by  the  size  and 
complexity  of  the  decision  situation  then  one  is  susceptible  to  a host  of  cognitive  illusions.  A few  of  the  major  ones  are: 
Availability  Heuristic,  Representativeness  Heuristic,  Hindsight  Bias,  Gambler’s  Fallacy,  the  Framing  Effect,  Similarity  Effect, 
Decoy  Effects,  and  Overconfidence  Effect.  When  the  decision  situation  is  simple  and  small,  on  the  other  hand,  most  people 
can  and  do  make  good  judgments  unaffected  by  these  cognitive  illusions. 

The  Availability  Heuristic  investigated  by  Tversky  and  Kahneman  (1973)  is  a powerful  cognitive  distortion.  Essentially,  it  is 
captured  by  the  by  the  saying,  “out  of  sight  - out  of  mind.”  It  turns  out  that  the  believing  the  truth  of  a piece  of  information  is 
related  to  the  ease  of  recall.  If  something  does  not  easily  come  to  mind  then  the  inference  is  that  it  must  not  be  true  (Begg, 
Amour  and  Kerr,  1985).  Unfortunately,  the  ease  of  recall  is  affected  by  circumstances  that  have  nothing  to  do  with  the  truth 
of  the  information  such  as  the  vividness  of  the  information  and  the  number  of  times  it  has  been  repeated.  The  power  of  the 
Availability  Heuristic  is  not  lost  on  those  who  repeatedly  advertise  on  radio  and  television. 

Another  powerful  cognitive  illusion  is  the  Representativeness  Heuristic  (Kahneman  and  Tversky,  1973).  The  illusion  is  that  if 
something  looks  like  one  of  those  then  it  must  be  one  of  those  in  spite  of  relevant  base  rate  information  to  the  contrary. 
Common  versions  of  this  are  “you  can  tell  a book  by  its  cover”  and  “if  it  looks  good  then  it  works  well.”  These  rules  of  thumb 
leave  out  the  base  rate  information  such  as  about  how  likely  it  is  for  a random  book  to  be  a good  read.  The  cognitive  illusion 
is  to  be  unable  to  process  additional  information  about  the  likelihood  of  an  event  given  the  overwhelming  impact  of  how  it 
looks  on  the  surface. 

The  Hindsight  Bias  is  the  tendency  to  believe,  after  the  fact,  that  one  would  have  foreseen  a particular  event  (Fischhoff,  1982 
as  sited  in  Galotti,  1999).  This  has  also  been  referred  to  as  the  “l-knew-it-all-along-phenomenon”  that  inflates  self-esteem 
and  feelings  of  overconfidence  in  one’s  predictive  abilities.  The  notion  is  that  hindsight  is  always  20-20,  but  In  reality,  many 
events  like  the  weather  are  still  uncertain  predictions  before  the  fact. 

There  is  an  entire  industry  built  on  the  cognitive  illusion  of  the  Gambler’s  Fallacy.  This  Is  the  popular  Illusion  of  believing  that 
in  a game  of  chance  that  the  more  you  loose  the  greater  your  chance  of  winning  (Tversky  and  Kahneman,  1971).  The  truth 
is  that  the  odds  of  winning  stay  the  same.  The  illusion  is  belief  that  the  law  of  large  numbers  applies  In  the  situation  of  small 
numbers  and  that  there  would  necessarily  be  some  balance  in  the  situation  of  a specific  gambler.  This  shows  up  as  the 
gambler’s  belief  that  in  a fair  game  of  chance  after  you  have  lost  for  a while  it  becomes  more  and  more  likely  that  It  is  your 
turn  to  win.  Other  forms  of  this  can  occur  in  committees  who  believe  that  after  a string  of  poor  decisions  it  Is  their  turn  to  get 
lucky  in  picking  a good  vendor  or  whatever  has  been  the  focus  of  their  decision-making. 

The  Framing  Effect  refers  to  the  impact  of  the  context  on  the  decision.  People  tend  to  avoid  risks  that  are  described  In  terms 
of  benefits,  but  tend  to  take  the  same  risks  that  are  described  in  terms  of  loss  (Tversky  and  Kahneman,  1981).  This  Is 
reminiscent  of  the  rule  of  thumb:  win  - stay,  lose  - shift.  The  illusion  is  that  the  value  of  losses  is  greater  than  the  value  of 
gains  when  mathematically  they  are  equal.  This  misinterpretation  of  the  importance  of  risk  can  bias  the  importance  weighting 
of  features  and  thus  bias  the  decision  process. 

Similarity  Effects  (Tversky,  1972)  and  Decoy  Effects  (Roe  et  al,  2001)  are  both  products  of  confusion.  When  a new  option  is 
added  to  a choice  set  that  is  similar  to  one  of  the  existing  options  the  original  option  is  less  likely  to  be  chosen  because  this 
effect  sort  of  divides  the  mind  share  like  a similar  product  can  divide  the  market  share.  The  Decoy  Effect  is  a special  case  of 
the  Similarity  Effect  where  a range  decoy  that  is  more  extreme  than  the  original  choice  set  dramatically  reduces  the  choice 
likelihood  of  nearby  options  (Roe,  1999).  In  effect,  the  extreme  decoy  sabotages  the  nearby  options  by  altering  their  context. 

Decision  Deadlines  can  also  influence  the  role  of  Cognitive  Illusions.  When  deadlines  are  quite  short  the  decision  maker 
tends  to  rely  more  on  fewer  pieces  of  information.  In  effect  the  working  memory  is  further  limited  by  time  pressure.  Roe’s 
(1999)  analysis  suggests  that  the  option  chosen  under  time  pressure  is  simply  the  one  that  is  the  current  focus  of  attention 
when  time  runs  out.  With  longer  decision  deadlines  the  more  subtle  similarity  effects  exert  more  influence.  Longer  decision 
timelines  in  the  context  of  face-to-face  group  decision  making  tend  to  be  associated  with  more  extreme  decisions  due  to 
attitude  polarization  (Moscovici  and  Zavalloni,  1969).  It  is  rarely  the  case  that  truth  wins  in  the  group  decision-making 
situation  (White,  1993  as  cited  in  Myers  and  Spencer  2001). 

Overconfidence  is  a paradoxical  Cognitive  Illusion  in  that  it  is  the  illusion  that  one  has  no  cognitive  illusions  and  that  one’s 
'^''■sions  are  without  bias.  Kahneman  and  Tversky  (1996)  have  found  a replicated  tendency  for  people  to  be  more  confident 
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than  is  warranted  by  the  evidence  and  to  overestimate  the  accuracy  of  their  beliefs.  This  overconfidence  extends  to 
eyewitnesses  testimony  where  Loftus  (1979)  has  found  that  the  witness’s  confidence  in  their  testimony  to  be  unrelated  to 
accuracy  of  their  testimony.  This  overconfidence  has  the  side  effect  of  enhancing  personal  selfesteem  and  thereby 
contributing  to  resistance  to  being  influenced  otherwise  i.e.  stubbornness. 

In  summary,  there  are  many  mental  obstacles  for  those  who  would  be  rational  decision  makers.  Many  of  these  Cognitive 
Illusions  stem  from  the  limitations  of  working  memory.  These  are  the  same  limitations  that  cause  people  to  not  be  able  to 
remember  long  strings  numbers  like  international  telephone  numbers.  Because  large  and  complex  decisions  situations  are 
beyond  the  capacity  of  working  memory,  people  resort  to  rule  of  thumb  strategies  to  do  the  best  they  can.  In  such 
overwhelming  situations  Cognitive  Illusions  bias  judgments  and  compromise  decision-making. 

Making  the  decision  situation  smaller  and  simpler  can  circumvent  these  obstacles.  Dividing  up  a complex  decision  into 
several  small  and  simpler  decisions  keeps  the  load  on  working  memory  within  normal  capacity  and  then  decision  makers  can 
be  quite  rational  in  making  good  decisions. 

Donald  Norman  (1999)  has  made  the  case  that  human-computer  interactions  can  and  should  be  designed  to  take  account  of 
normal  human  fallible  perceptions,  wandering  attention,  faulty  memories,  and  fluctuating  information-processing  abilities. 

With  good  design  of  decision-making  processes  that  keep  the  in-the-head  part  small  and  simple,  a system  can  be  optimized 
to  enable  quality  decision-making.  The  normative  weighted  averaging  strategy  can  be  easily  divided  up  into  small  and  simple 
component  parts  and  then  later  recombined  (outside-of-the-head)  into  a rational  decision  process  that  leads  to  the  “best 
option.”  There  is  a powerful  decision-making  synergy  In  combining  the  focused  judgmental  prowess  of  people  with  the 
memory  and  computational  prowess  of  computers. 

How  people  normally  Make  Decisions:  The  5 Basic  Strategies 

There  are  five  basic  decisions  strategies  (Payne  & Bettman,  2001)  for  choosing  among  options  (products,  services  etc.) 
based  on  features  or  aspects  of  the  options.  All  of  these  five  strategies  are  compromised  by  the  reality  that  the  set  of  options 
is  limited  to  those  known  to  the  decision-maker  and  the  “best”  option  may  be  simply  missed  in  the  whole  process.  While  the 
names  of  these  strategies  on  may  be  unfamiliar  is  quite  likely  that  everyone  has  used  all  of  the  following  strategies  at  one 
time  or  another: 

Lexicographic  strategy  (aka  one  reason,  pick  the  best) 

Elimination  by  aspect  strategy  (aka  pick  the  last) 

Satisficing  strategy  (aka  Bounded  Rationality  Model) 

Equal  weight  strategy  (aka  scoring  strategy) 

Weighted  averaging  strategy  (aka  weighted  adding  strategy,  grading  model) 

The  Lexicographic  strategy  involves  two  steps:  selecting  the  most  important  feature  and  then  picking  the  product  from  all  of 
the  options  that  is  best  on  that  one  feature.  There  is  no  requirement  for  the  features  to  be  expressed  as  numbers.  If  there  is 
a tie  then  the  decision  maker  simply  repeats  the  process  on  the  second  most  important  feature.  This  non-compensatory 
strategy  does  not  allow  for  some  features  of  an  option  to  make  up  for  other  less  adequate  features.  The  only  involvement  of  a 
second  feature  is  in  the  case  of  a tie  that  is  broken  by  considering  the  next  most  important  feature  and  selecting  the  option 
that  is  best  on  that  feature. 

The  elimination  by  aspect  strategy  is  a simple  screening  strategy  involving  two  steps:  setting  the  requirements  for  each  of  the 
features  and  then  eliminating  options  one  at  a time  which  do  not  meet  any  one  of  the  requirements.  The  decision  maker 
examines  the  features  of  each  option  one  by  one  and  the  option  is  eliminated  as  soon  as  a feature  is  found  that  does  not 
meet  the  requirements.  This  popular  strategy  does  not  need  for  the  features  requirements  be  expressed  as  numbers  and  is 
easy-to-use.  The  elimination  by  aspect  is  a non-compensatory  strategy  so  that  once  an  option  is  rejected  on  any  feature  that 
option  is  eliminated  from  further  consideration.  This  strategy  does  not  necessarily  result  in  a single  best  option  since  more 
than  one  option  may  pass  on  all  of  the  requirements.  Including  any  features  that  are  irrelevant  or  biased  in  favor  of  one  of  the 
options  compromises  this  process. 

The  Satisficing  strategy  (Simon,  1955  cited  in  Gigerenzer  and  Selten,  2001)  is  more  psychologically  developed  model  of 
sequential  decision-making  that  includes  setting  requirement  cutoff  levels  for  each  feature.  An  option  is  examined  until  it  fails 
to  meet  one  of  the  cutoff  levels  then  it  is  rejected  and  the  next  option  is  considered.  The  first  option  that  passes  on  all  of  the 
features  is  selected.  If  none  of  the  options  pass  all  of  the  cutoff  requirements,  then  the  requirements  are  reduced  to  new 
lower  cutoff  levels  and  the  process  is  repeated.  The  satisficing  strategy  is  non-compensatory  and  effected  by  the  order  in 
which  the  options  are  presented  as  well  as  the  aspiration  level  of  the  initial  cutoff  requirements. 

The  Equal  weight  strategy  is  comprehensive  and  considers  all  of  the  features  on  ail  of  the  options.  The  suitability  values  of 
one  (passed)  or  zero  (failed)  are  assigned  to  each  to  each  feature  on  each  option.  The  total  suitability  score  for  each  option 
is  the  simple  sum  of  the  featire  suitability  values  for  that  option  (all  features  are  weighted  equally).  The  option  with  the 
highest  sum  of  the  values  is  selected.  This  strategy  is  directly  analogous  to  multiple  choice  test  scoring  where  correct 
^^  vers  are  awarded  one,  wrong  answers  are  awarded  zero,  and  the  highest  sum  score  identifies  (selects)  the  best  student 


on  the  test.  This  compensatory  numeric  strategy  enables  some  features  assigned  one’s  to  compensate  for  other  features 
that  were  assigned  zero’s.  This  strategy  is  more  useful  when  there  are  many  features  because  there  is  less  risk  of  a tie  for 
best  score. 

The  Weighted  averaging  strategy  is  a more  complex  version  of  the  simple  sum  score  model  where  each  feature  is  also 
assigned  an  importance  weighing.  This  idea  implies  that  the  decision  maker  is  willing  to  make  trade-offs  to  arrive  at  the 
selected  option.  The  process  involves  five  steps:  (1)  setting  the  importance  weights  for  each  of  the  features,  (2)  for  each 
option  then  determining  the  numerical  suitability  value  for  every  feature,  (3)  multiplying  the  feature  suitability  value  times  the 
feature  weight,  (4)  then  summing  these  weighted  features  subscores  into  an  option  total  score,  and  (5)  finally  selecting  the 
option  with  the  highest  average  score.  The  averaging  of  the  weighted  option  total  scores  simply  returns  the  scores  to  the 
scaling  metric  of  the  feature  suitability  value  units  (which  are  often  on  a rating  scale  such  as  1=poor  to  9=excellent).  This 
strategy  is  directly  analogous  to  the  grading  model  where  different  parts  of  the  grade  are  weighted  differentially  (or 
subsections  of  an  exam  are  weighted  differentially)  to  arrive  at  the  final  mark.  The  weighted  averaging  strategy  is 
compensatory  in  two  ways:  high  scores  on  one  feature  compensate  for  low  scores  on  another  feature  and  a lower  importance 
weight  on  one  feature  is  compensated  for  by  a higher  importance  weight  on  another  feature.  This  strategy  is  less  likely  to 
result  in  ties  between  options  than  is  the  equal  weight  strategy  when  using  a relatively  small  number  of  features. 

Furthermore,  the  weighted  averaging  strategy  is  considered  the  normatively  rational  decision  process  because  it  uses  all  of 
the  information  available  in  a consistent  manner  to  arrive  at  a selection.  This  model  is  similar  to  the  multiple  linear  regression 
model  used  in  statistical  analyses  and  suffers  from  some  of  the  same  shortcomings,  most  notably  it  is  compromised  when 
irrelevant  features  are  used  and  when  unreliable  suitability  values  are  used. 

Decision  makers  often  make  use  of  combinations  of  the  above  strategies.  In  combined  strategies  the  decision  is  made  in 
phases.  The  initial  phase  is  often  to  screen  out  some  of  the  potential  options  in  order  to  reduce  the  complexity  of  the 
situation  to  as  short  list  of  options.  This  phase  is  followed  by  a more  thorough  consideration  phase  to  select  the  best  option 
from  a short-list  of  candidate  options.  The  primary  reason  that  everyone  does  not  use  the  weighted  averaging  strategy  all  the 
time  for  all  decisions  is  because  it  would  be  too  mentally  difficult  and  too  time  consuming.  Other  strategies  are  heuristic 
attempts  to  make  good  decisions  with  less  effort  and  in  less  time  and  sometimes  they  seem  to  work  well  enough. 

Idealized  decision-making  process  for  comparison 

Selecting  a online  educational  delivery  application  is  a high-stakes  gamble  that  can  impact  the  future  prosperity  of  an 
educational  institution.  The  weighted  averaging  strategy  is  ideal  decision-making  process  because  it  Is  considered  to  be  the 
only  normatively  rational  decision  process.  In  its  most  rational  form  every  possible  product  option  would  be  considered  on 
every  possible  feature  - nothing  would  be  left  out.  The  process  involves  the  five  steps:  (1)  setting  the  importance  weights  for 
each  of  the  features,  (2)  determining  the  numerical  suitability  value  for  every  feature  of  every  product  option,  (3)  multiplying 
the  feature  suitability  value  times  the  feature  weight,  (4)  then  summing  these  weighted  features  subscores  into  a product 
option  total  score,  and  (5)  finally  selecting  the  product  option  with  the  highest  average  score  from  among  all  of  the  possible 
product  options.  In  this  ideal  there  are  three  major  difficulties:  discovering  every  possible  product  option  in  a dynamic 
expanding  market,  accurately  determining  the  importance  weightings  of  each  of  the  features,  and  accurately  determining  the 
suitability  of  each  feature  on  every  product.  The  practical  difficulty  is  simply  not  having  sufficient  time  and  resources  to  carry 
out  the  process.  The  recommended  process  is  below  intended  to  approximate  this  ideal  strategy  within  the  practical 
limitations  of  time  and  resources. 

Tailoring  decision  processes  to  Individual  Situations  by  involving  the  stakeholders 

The  ultimate  success  of  a decision  process  in  addition  to  making  a good  decision  is  that  others,  the  stakeholders,  cooperate 
with  the  decision  makers  and  carry  out  the  decision  and  implement  the  chosen  alternative.  Tetlock  (1983  as  cited  in 
Gigerenzer  2001)  has  studied  the  social  aspects  of  rationality  and  proposed  three  social  goals  that  are  important  for 
maintaining  this  cooperation.  The  first  social  goal  is  that  decisions  are  made  in  a transparent  way  that  is  understandable  and 
predictable.  The  second  social  goal  is  fairness  where  the  decisions  are  made  in  a way  that  does  not  violate  the  expectations 
of  people.  The  third  social  goal  is  where  decisions  are  made  in  an  accountable  way  that  can  be  justified  and  defended  in 
public.  Decisions,  which  meet  all  three  of  these  social  goals,  are  more  likely  to  pave  the  way  for  the  acceptance  by  the 
stakeholders  of  the  eventual  decision  to  select  a particular  product  over  the  competition. 

One  direct  way  of  accomplishing  these  goals  is  to  involve  the  stakeholders  in  the  overall  decision  process.  Including  several 
of  the  stakeholders  on  the  decision-making  or  decision-recommending  committee  often  does  this.  The  common  difficulty  with 
this  approach  is  that  there  are  many  times  more  stakeholders  than  there  are  members  on  the  committee.  By  dividing  up  the 
decision-making  tasks  it  is  possible  to  meaningfully  involve  more  stakeholders  and  still  keep  the  decision-making  committee 
to  a workable  size.  There  are  a couple  of  obvious  groups  of  stakeholders:  the  student  learners  and  the  faculty  who  will  be 
using  the  selected  system  to  deliver  online  courses  or  use  it  to  deliver  the  online  portion  of  blended  regular  courses.  In 
many  institutions  there  will  also  be  other  stakeholders  who  will  be  interested  in  some  involvement  in  the  decision  process. 
Other  obvious  stakeholders  are  the  systems  and  computing  personnel  who  would  be  doing  the  technical  administration  of  the 
Learning  Management  System  if  it  is  not  hosted  from  the  vendor’s  site.  Also,  there  may  be  a myriad  of  learner  support 
services  from  the  registrar  to  instructional  design  personnel  to  the  library  and  disabled  student  services  personnel  who  all 
a stake  in  the  decision  process.  Finally,  depending  on  the  level  of  involvement  the  institutional  Board  of  Directors  may 


well  have  political  role  to  making  their  priorities  felt  in  such  an  important  institutional  decision.  One  recommended  approach 
is  to  involve  some  of  the  stakeholders  in  deciding  the  importance  of  specific  product  features  and  also  involving  the  more 
technical  expert  stakeholders  in  judging  the  suitability  a specific  feature  in  a particular  product. 

When  the  initial  importance  weightings  of  the  product  features  for  the  comparative  analysis  site  were  developed  there  were 
many  differences  of  opinion  among  the  experts  involved  in  the  peer  review  committee.  These  differences  were  worked  out 
and  consensus  was  achieved  by  using  a couple  of  face-to-face  meetings  followed  a nominal  group  process  similar  to  the 
Delphi  process  (Delbecq,  Van  deVen  and  Gustafson  1975).  After  people  met  and  had  an  opportunity  to  talk  about  what  they 
thought  was  important  and  why,  they  were  then  asked  to  provide  importance  ratings  for  the  specific  features.  These  ratings 
were  simply  averaged  to  provide  a mathematical  consensus  without  resorting  to  additional  face-to-face  meetings.  The 
resulting  averages  were  also  sent  back  to  the  peer  review  committee  by  email  with  requests  for  further  comments  if  members 
had  strong  feelings  that  there  was  an  error  anywhere.  These  requests  produced  some  new  ratings  that  were  again  averaged 
and  circulated  again  for  comment  and  revision.  It  took  only  a couple  of  rounds  of  this  nominal  group  process  to  produce  a 
consensus  on  the  feature  weights.  This  consensus  was  in  part  because  the  process  was  transparent,  fair,  and  accountable 
as  recommended  by  Tetlock  (1983,  cited  in  Gigerenzer  2001). 

Focusing  on  what  is  important 

Colleges  and  Universities  are  hugely  different  as  are  institutions  of  3,000  students  and  institutions  of  30,000  students. 
Historically,  much  of  educational  market  has  been  a geographical  patchwork  of  niche  markets  with  some  exceptions.  In  each 
institutional  situation  of  there  are  some  considerations  that  are  more  important  than  others.  One  way  to  approach  the  problem 
of  too  many  products  and  too  many  features  is  to  limit  the  consideration  to  only  those  things  that  are  locally  important. 
Determining  what  features  are  needed  in  an  institutional  Learning  Management  System  is  often  a political  process  in  each 
institution  that  can  involve  many  people.  This  process  is  informed  by  the  institutional  history  of  what  has  worked  well  and 
what  has  not,  as  well  as  expectations  about  what  will  be  needed  in  the  future.  In  practice,  different  institutions  and  faculty 
within  those  institutions  use  products  in  different  ways  and  in  more  than  a few  cases  some  features  of  a product  are  never 
used.  This  situation  results  from  the  newness  of  the  market  and  the  rapid  pace  of  technological  change  in  recent  times.  In 
this  situation  the  wiser  approach  would  seem  to  be  to  focus  on  what  is  needed  rather  than  what  is  offered  in  the  market.  The 
idea  of  a needs  analysis  is  not  a new  idea,  but  it  can  be  particularly  advantageous  in  selecting  a Learning  Management 
System  by  informing  the  process  of  which  product  features  fit  into  the  institution’s  needs  and  which  ones  are  less  needed  or 
merely  duplicate  existing  institutional  systems. 

Each  institution  is  unique  and  that  uniqueness  can  be  augmented  or  diminished  by  the  choice  of  an  institutional  Learning 
Management  System.  Sometimes  institutional  mission  and  goal  statements  can  provide  a guide  to  which  directions  to 
pursue  and  sometimes  these  policy  documents  play  no  real  role  in  the  decision  making  of  the  institution.  What  is  at  issue  in 
the  selection  of  a Learning  Management  System  is  the  potential  commitment  to  a product  that  is  as  far-reaching  as  the 
institutional  decision  about  which  word  processor  application  to  support.  Once  the  decision  is  made  it  will  not  be  easy  or 
inexpensive  to  undo  that  decision.  By  basing  the  decision  around  the  unique  needs  of  the  specific  institution  there  is  a better 
chance  that  decision  made  will  further  the  success  of  the  institution  rather  than  the  opposite. 

Thinking  in  terms  of  the  audiences  of  users 

One  way  to  reduce  the  complexity  of  the  decision  process  is  to  divide  the  product  features  into  groupings  based  on  the  group 
of  people  who  are  more  likely  to  be  involved  with  those  features.  In  this  instance  one  can  think  about  three  different  groups 
of  people:  Learners,  those  who  support  the  learners  more  or  less  directly  and  those  who  are  involved  in  the  technical 
administration  of  the  software  installation.  These  three  groups  do  not  exhaust  the  possibilities  but  provide  a starting 
viewpoint. 

The  learners  or  students  are  usually  the  largest  group  of  users.  It  is  useful  to  consider  the  Learning  Management  System 
from  their  perspective  since  any  problems  or  short  comings  are  going  to  effect  so  many  people  and  because  many 
institutions  profess  a “learner  focus”  as  part  of  their  mission.  However  learners  are  not  a homogeneous  group  and  the  needs 
of  “first  time  users”  may  well  be  very  different  from  those  of  returning  successful  students. 

The  Learner  Support  group  includes  faculty,  teaching  assistants,  instructional  designers,  various  administrators  and  others 
who’s  job  it  is  to  help  learners  to  be  successful.  The  titles  and  descriptions  of  these  roles  vary  dramatically  from  situation  to 
situation 

The  audience  of  technical  administrators  consists  of  those  folks  who  operate  the  server  as  well  as  those  who  administer  the 
authorization  or  security  policies  out  of  sight.  In  many  situations  is  this  audience  also  includes  additional  administrators 
involved  in  setting  up  course  shells  on  the  server  and  arranging  for  access  to  registration  lists.  In  general,  the  technical 
administrators  operate  behind  the  scenes  and  are  able  to  access  parts  of  the  software  system  that  are  not  available  to 
instructors  or  course  developers. 
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Thinking  in  terms  of  product  features 

Within  each  of  the  three  audiences  the  features  of  the  products  can  be  further  grouped  into  clusters  of  features  and  this  can 
reduce  the  complexity  further.  On  the  Online  Educational  Delivery  Applications  site  fhttp://www.c2t2.ca/landonline/)  the 
following  conceptual  groupings  have  been  used  to  provide  targeted  views  for  the  learners,  learner  support,  and  technical 
administrator  audiences  respectively: 

Learner  Tools  (for  learner  audience) 

Web  Browsing 
Asynchronous  Sharing 
Synchronous  Sharing 
Student  tools 

Support  Tools  (for  learner  support  audience) 

Course 

Lesson 

Data 

Resource 
Administration 
Help  desk 

Techinfo  (for  technical  administrator  audience) 

Server  Platform 
Client  Platform 
Pricing 

Limitations  of  package 
Extra  Considerations 

Within  each  of  these  conceptual  categories  there  would  be  a small  number  of  specific  application  tools  or  features. 

Altogether  there  were  62  such  specific  features  that  could  be  a part  of  an  individual  application.  The  complete  listing  of 
application  features  inside  of  the  above  conceptual  structure  attempts  to  organize  the  otherwise  overwhelming  list  of  features: 
(adapted  from  httD.7/www.c2t2.ca/landonline/oDtion2.asp) 


Check  Features  to  inciude 


Learner_Tools 

Web_Browsing 

^ Accessibilitv 

^ Bookmarks 

n 

Multimedia 

Support_Tools 

Course 

n 

Course  olanning 
^ Course  managing 
^ Course  customizing 

Techinfo 

Server_Platform 

^ RAM 

^ Disk  Soace 

^ WindowsNT  40  Server 

r 

Securitv 

^ Course  monitoring 

^ Aoole  Server 

Asynchronous_Sharing  Lesson 

r 

p 

n 

Instructional  designing 

Unix  Server 

E-mail 

Client_Platform 

r 

BBS  file  exchange 

Presenting  information 

^ Minimum  Level 

□ 

NewsgrouDs 

^ Testing 

Ft 

Target  Level 

Synchronous.Sharing 

Data 

Pricing 

r 

r 

n 

Chat 

Marking  on-line 

Start-uD  Cost 

r 

Voice  Chat 

r 

r 

Manaaina  records 

On-going  Cost 

r 

Whiteboard 

^ Analvzing  and  tracking 

^ Technical  Suooort 

r 

AoDlication  sharing 

Resource 

n 

Limitations  of  package 

n 

□ 

Virtual  soace 

Curriculum  Managing 

pr 

IMS  Comoliance 

^ Number  of  courses 

p 

r 

Grouo  browsing 

Building  knowledge 
^ Team  Building 

r 

' Number  of  students 

O 

ERIC 


Teleconferencing 
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Videoconferencina 

Buildina  motivation 

Number  of  connections 

Student.tools 

Administration 

^ Number  of  instructors 

1 

Self-assessina 

L! 

n 

Installation 

^ Other  Limitations 

1 I 

Proaress  trackina 

Authorization 

Extra_Considerations 

r 

Searchina 

n 

Reaisterina 

^ Ootions 

r 

Motivation  buildina 

n 

On-line  fees  handlina 

^ Exit  Considerations 

n 

Studv  skill  buildina 

n 

Server  securitv 

r 

Resource  monitorina 

r 

Remote  access 

D 

Crash  recovery 

Hetp_desk 

n 

Student  suDDort 

^ Instructor  support 


No  product  had  all  of  the  possible  features  and  no  institutional  situation  needed  all  of  the  features  either.  The  Online 
Educational  Delivery  Applications  site  was  intended  to  help  institutions  find  products  with  the  features  that  they  need  and  for 
vendors  of  products  to  be  discoverable  on  the  basis  of  the  product  functionality  in  the  context  of  a rapidly  developing 
marketplace.  The  list  of  product  features  also  turns  out  to  be  a moving  target  over  time  as  new  technologies  are  incorporated 
into  the  evolving  applications. 

Product  Reviews  and  Rapid  Product  Change 

The  situation  of  finding  appropriate  application  candidates  is  made  substantially  more  difficult  by  the  fact  that  the  products 
themselves  are  changing.  Most  of  the  products  will  release  a new  version  one  or  more  times  per  year  and  In  many  situations 
this  “revision"  can  change  the  product  substantially,  especially  when  the  newness  comes  from  a company  merger  or 
acquisition.  Published  product  reviews  and  comparisons  quickly  become  dated  in  spite  of  the  best  of  Intentions.  This 
dynamism  among  the  vendors  is  heightened  further  by  technological  changes  such  as  the  advent  of  database  back-ends  on 
courseware  systems  that  has  occurred  during  the  last  several  months. 

In  the  context  of  the  field  of  rapidly  moving  products  it  is  very  difficult  be  base  decisions  on  what  is  available,  but  it  Is  still  quite 
feasible  to  base  decisions  on  what  Is  needed  in  the  individual  situation.  The  Institutional  context  of  online  learning  is 
changing  more  slowly  than  the  marketplace  and  it  can  be  argued  that  a needs  analysis  Is  more  valuable  than  a market 
analysis  in  such  a situation. 

Screening  phase  of  decision-making  based  on  requirements  of  Individual  Situations 

The  elimination  by  aspect  strategy  is  a screening  strategy  involving  two  steps:  setting  the  requirements  for  each  of  the 
features  and  then  eliminating  options  one  at  a time  which  do  not  meet  any  of  the  requirements.  To  be  able  to  decide  which 
features  are  Important  the  conventional  approach  is  to  conduct  some  sort  of  needs  analysis  of  what  Is  required  in  the 
Institutional  setting.  This  analysis  often  reveals  the  uniqueness  of  the  needs  of  the  future  users  of  the  online  system.  At  this 
stage  it  can  be  beneficial  to  involve  the  stakeholders:  students,  faculty,  student  support  personnel,  and  the  technical 
administrators  to  gain  later  acceptance  of  the  eventual  decision  as  well  as  to  become  aware  of  needed  features  that  might 
have  been  otherwise  overlooked.  Politically  it  can  be  beneficial  to  even  involve  representatives  from  the  governing  board  of 
the  institution  when  this  decision  will  represent  a new  pedagogical  direction  or  is  expected  to  involve  significant  long-term 
resource  commitment. 

Situational  Requirements 

Individual  situation  are  both  nominally  different  and  substantially  different.  Sometimes  the  differences  are  so  large  that  our 
tendency  to  attend  to  differences  makes  it  seem  like  some  institutions  are  as  different  as  day  and  night.  While  there  are  a 
host  of  differences  between  institutions  there  are  also  some  similarities  so  that  examining  an  example  of  decision-making  can 
be  instructive  about  similarities  in  the  processes.  As  the  online  educational  delivery  applications  have  begun  to  move  from 
the  one-size- fits-all  mold  of  the  industrial  era,  they  are  more  adaptable  to  individual  situations  but  still  the  applications  have 
different  strengths  and  weaknesses.  The  ideal  of  making  decisions  based  on  individual  local  institutional  situations  is  to 
exploit  the  synergy  of  matching  the  strengths  of  a product  to  the  specific  needs  of  the  specific  situation.  The  uniqueness  of 
local  situation  can  be  described  in  many  ways  but  by  using  the  features  of  the  products  as  the  focus  for  this  description  the 
Q ntial  synergy  is  easier  to  find  and  exploit.  This  involves,  for  example,  translating  a local  need  for  better  two-way 
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communication  between  instructors  and  students  into  the  importance  of  the  e-mail  feature.  Likewise  the  need  for  orienting 
and  assisting  instructors  in  how  to  use  an  online  educational  delivery  system  can  be  translated  into  the  importance  of  an 
instructor  help  desk  feature.  When  this  feature  importance  is  expressed  as  an  importance  weighting  then  this  information 
can  be  utilized  in  the  weighted  average  decision  strategy  which  combines  information  about  many  such  attributes  into  a 
overall  suitability  score  as  is  described  in  more  detail  in  the  example  below. 

The  deciding  of  which  features  are  more  important  versus  which  features  are  less  important  based  on  an  analysis  of  the  local 
situation  goes  a long  way  toward  reducing  the  risk  that  the  chosen  application  will  be  accepted  by  the  stakeholders  and  will 
benefit  the  institution.  Including  irrelevant  features  at  this  stage  will  bias  the  process  in  favor  of  those  applications  which  have 
those  irrelevant  features  so  this  deciding  which  features  are  required  is  a fundamental  part  of  the  overall  decision  process. 
This  is  a process  where  the  stakeholders,  as  described  above,  can  help  out  by  being  involved  in  determining  which  specific 
features  are  important  enough  to  be  “needs”  rather  than  “wants.”  The  elimination  by  aspect  strategy  does  not  require  that  the 
feature  requirements  be  numeric  so  simply  identifying  which  features  are  really  needed  is  enough.  There  are  62  feature 
categories  on  the  Online  Educational  Delivery  Applications  site  and  there  are  undoubtedly  some  additional  ones  are  critical  in 
some  situations. 

The  outcome  of  the  deciding  which  features  are  important  is  a Shortlist  of  important  product  features.  Any  product  that  does 
not  include  all  of  those  features  would  not  meet  the  needs  of  the  institution  and  should  be  screened  out  of  the  competitive 
options  for  the  final  selection.  This  initial  product  screening  can  effectively  be  done  by  the  elimination  by  aspect  decision- 
making strategy.  The  decision  maker  examines  the  features  of  each  option  one  by  one  and  the  option  Is  eliminated  as  soon 
as  a needed  feature  is  missing  or  completely  inadequate. 

Screening  tools  on  the  web 

Once  the  set  of  needed  features  has  be  determined  then  the  Online  Educational  Delivery  Applications  site  can  be  used  to 
identify  which  applications  provide  those  features  so  the  number  of  potential  options  can  be  reduced  to  just  those  that  are 
likely  candidates.  By  using  a page  like  the  above  table  the  decision  maker  can  check  the  checkbox  next  to  the  required 
features  and  then  click  submit  and  the  returning  page  will  include  a list  of  applications  which  support  those  features.  Using 
that  same  page  the  candidates  can  be  further  investigated  in  side-by-side  comparisons  of  feature  descriptions.  This  side  by 
side  comparison  may  help  in  further  reducing  the  list  of  applications  to  only  the  appropriate  options  for  the  local  decision 
situation. 

Making  a Short  List  of  Application  Options 

Using  the  web  tool  described  above  greatly  speeds  up  this  elimination  by  aspect  process  for  those  applications  that  are 
reviewed  on  the  site.  The  elimination  by  aspect  is  a non-compensatory  strategy  so  that  once  an  option  is  rejected  on  any 
feature  that  option  is  eliminated  from  further  consideration.  This  strategy  does  not  necessarily  result  in  a single  best  option 
since  more  than  one  option  may  pass  on  all  of  the  requirements.  The  usual  outcome  of  the  elimination  by  aspect  strategy  is 
to  result  in  a short  list  of  possibly  acceptable  options  that  become  the  narrowed  focus  for  a more  discriminating  decision 
strategy.  It  may  be  prudent  for  comparison  purposes  to  include  in  the  short  list  any  applications  that  are  used  by  consortia 
where  the  local  institution  is  a member  or  in  the  case  of  community  colleges  the  application  that  is  used  by  local  universities 
where  the  college  has  special  relationships. 

Evaluating  product  suitability  as  part  of  the  weighted  averaging  strategy 

Evaluating  products  (or  externally  hosted  product  options)  can  be  a very  large  task.  The  greater  the  number  of  products  and 
the  greater  the  number  of  important  features  multiplies  quickly  into  a potentially  overwhelming  product  research  task.  This 
task  can  be  partially  shifted  to  the  vendors  by  structuring  a competition  among  the  likely  candidates  where  they  are  invited  to 
bring  the  information  to  the  decision  makers  rather  than  the  decision  makers  seeking  out  the  vendors  and  trying  to  find  out 
the  most  current  product  information.  The  market  has  matured  enough  so  that  most  vendors  are  capable  of  responding  to  a 
request  for  proposal  (RFP)  to  supply  the  desired  product  functionality.  Many  institutions  already  have  RFP  procedures  but 
may  never  have  used  them  for  a situation  as  complex  as  selecting  an  online  educational  delivery  application. 

Inviting  the  shortlist  of  vendors  to  competitive  presentationsiproposals  (RFP  model) 

The  RFP  model  has  been  described  for  web  software  acquisition  recently  ( 
httD://www.technoloavnews.net/rfD/infotech  rfp.doc^  as  a necessary  part  of  the  decision  process. 

The  RFP  model  offers  several  advantages  over  less  formal  product  selection  processes  including:  all  vendors  get  the  same 
information  and  have  a fair  opportunity  to  compete  while  the  decision  makers  get  a proposal  document  that  can  be  used  to 
assess  both  the  vendor  interest  and  competence.  Local  procedures  may  vary  but  in  general  a useful  RFP  should  include  the 
following  sections  (according  to  the  info-tech  research  group  at  httD://www.technoloQvnews.net/rfD): 

1)  Executive  Summary 

2)  Reasons  for  the  online  educational  delivery  application  so  vendors  can  understand  local  situation 

3)  Explanation  of  the  vendor  selection  process  (the  elimination  by  aspect  strategy  as  above) 

^ 'Explanation  of  the  proposal  evaluation  process  (the  weighted  average  scoring  as  above) 


5)  Description  of  the  decision-making  team 

6)  Description  of  in-house  resources  that  will  also  be  used  in  the  online  educational  delivery 

7)  Background  Information  about  the  institution  and  the  competition 

8)  Time  line  for  the  vendor’s  deliverables 

9)  Penalties  for  missing  deadlines 

10)  The  Specifications  including  short,  medium  and  long-term  specification  lists  and  descriptions 

11)  General  terms  and  conditions  for  the  institutional  agreements 

12)  Special  terms  and  conditions  such  as  the  number  of  references  required,  legal  requirements  etc. 

The  RFP  would  be  sent  to  the  short  list  of  possible  vendors.  The  vendors  will  likely  have  questions  as  they  prepare  their 
proposals  so  it  is  useful  to  have  a designated  contact  person  who  can  supply  consistent  clarifying  information  to  all  of  the 
vendors’  requests  for  information.  When  the  proposals  are  received  there  is  often  another  round  of  elimination  by  aspect 
strategy  to  determine  which  proposals  are  in  the  competitive  range.  The  US  Department  of  Defense 
(http://web. deskbook.osd.mil/reflib/mfarsuDs/072ua/009/072ua009doc.htm)  further  advises  that  a proposal  may  be 
considered  outside  of  the  competitive  range  if  (1)  it  does  not  address  the  essential  requirements,  (2)  has  a substantial 
technical  drawback  the  would  essentially  require  a new  proposal  to  fix,  or  (3)  the  proposal  contains  major  deficiencies, 
omissions,  or  out-of-line  costs.  They  also  advise  that  these  proposals  are  part  of  a negotiation  process  and  at  the  end  of 
negotiations,  the  competitive  vendors  should  be  provided  with  one  additional  final  opportunity  to  submit  a revision  known  as 
the  “best  and  final  offer.’’ 

The  RFP  method  is  a time-consuming  and  involving  process  both  for  the  institution  and  for  the  potential  vendors  that  can  be 
used  to  produce  a fair  vendor  competition  where  the  best  candidate  is  selected  and  the  social  goals  of  rational  decision- 
making are  accomplished.  The  Online  Educational  Delivery  Application  site  provides  web  tool  support  for  the  Weighted 
Averaging  Strategy  calculation  part  of  the  of  decision-making.  This  decision  engine  tool  is  little  more  than  an  automated  score 
sheet  that  combines  the  feature  importance  and  the  feature  suitability  scores  into  an  appropriate  total  product  score.  The  tool 
does  however  enable  the  decision-makers  to  use  the  most  rational  strategy  of  decision-making,  which  would  be  impossible  to 
do  in  one’s  head.  The  following  example  illustrates  the  decision  engine  web  tool  using  two  vendors  (WebCT  and 
BlackBoard)  with  three  features  (accessibility  for  persons  with  disabilities,  student  help,  and  instructor  help). 

Quantifying  feature  importance  in  individuai  situations 

For  the  purposes  of  this  example  let  us  assume  that  our  institution  is  part  of  the  Consortium  that  is  offering  us  the  opportunity 
to  use  either  WebCT  or  BlackBoard  for  our  existing  system.  Further  lesson  soon  that  there  are  only  three  important  issues: 
meeting  the  508  accessibility  regulations  so  that  are  federal  funding  is  not  jeopardize,  meeting  the  help  desk  requirements  of 
our  students,  and  meeting  the  help  desk  requirements  of  our  faculty.  We  could  use  the  default  importance  weightings  from 
the  site  that  are  1 .0  for  accessibility,  1 .2  for  student  support,  and  0.8  for  instructor  support  but  it  would  be  better  to  tailor  the 
importance  weighting  to  our  institutional  setting.  So  for  this  example  let  us  assume  that  we  did  some  campus  wide 
information  meetings  and  formed  an  advisory  committee  of  interested  stakeholders. 

Potential  of  involving  stakeholders  in  setting  weights 

Then  with  the  help  of  our  stakeholders  we  put  together  a little  three-item  telephone  survey  of  a random  sample  students  and 
a random  sample  of  instructors.  The  results  of  our  little  survey  were  importance  ratings  of  9 for  the  accessibility  item,  7 for 
the  student  support  item,  and  6 for  the  instructor  support  item.  Those  numbers  can  be  just  entered  into  the  web  Decision 
Table  form  because  the  form  to  product  the  importance  weights  normalizes  the  numbers  before  they  are  used. 

Judging  product  feature  suitability  to  local  situation 

Different  persons  can  make  the  suitability  judgments  than  were  involved  in  the  setting  of  the  importance  weights.  In  this 
example  we  could  conceivably  involve  the  members  of  our  advisory  committee  to  nominate  from  among  themselves  a task 
group  who  have  experience  with  our  existing  system  and  are  willing  to  spend  a week  working  with  WebCT  and  a week 
working  with  BlackBoard  to  specifically  evaluate  the  accessibility,  the  student  help  desk,  and  the  instructor  help  desk.  Further 
our  task  group  could  also  get  others  to  help  with  their  evaluation  as  they  chose  with  the  only  caveat  that  at  least  two  persons 
with  disabilities  be  involved  in  checking  out  the  accessibility  aspects  of  the  two  products. 

If  we  had  allowed  more  time  it  might  have  been  possible  conduct  a request  for  proposals  and  have  representatives  from  the 
two  vendors  provide  information  and  trial  facilities  to  the  task  group  to  make  their  hands-on  experience  come  to  them.  After  a 
week  the  task  group  meets  with  the  advisory  committee  and  presents  their  considered  judgments,  which  are  averaged  to 
make  them  more  reliable.  Their  judgment  of  the  suitability  of  the  accessibility  for  WebCT  is  7,  for  BlackBoard  is  8,  and  for 
the  status  quo  Is  2 on  a scale  of  1 to  9.  Their  judgment  of  the  suitability  of  student  support  for  WebCT  is  8,  for  BlackBoard  is 
8,  and  for  the  status  quo  is  4.  Lastly,  their  judgment  of  the  suitability  of  instructor  support  for  WebCT  is  8,  for  BlackBoard  is 
6,  and  for  the  status  quo  is  3.  Now  this  is  where  the  doing  it  in  your  head  is  overwhelming  with  normalizing  three  numbers  to 
an  average  weight  of  1.0,  then  multiplying  nine  pairs  of  numbers  and  then  averaging  three  columns  to  arrive  at  the  winning 
Weighted  Average  Score  of  7.59  for  WebCT  followed  by  runner  up  BlackBoard  with  a Weighted  Average  Score  of  7.45  which 
is  well  ahead  of  the  status  quo  at  2.9  socore. 
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Demonstration  model  of  the  Comparative  Analysis  Decision  Table  with  Three  Options 

(Insert  table  similar  to  the  one  below  adapted  from  http://www.c2t2.ca/landonline/option4.asp) 

Instructions: 

To  compare  applications  A and  B just  enter  your  ratings  for  each  criteria  into 
the  boxes  in  the  column  and  then  press  Score.  Be  sure  to  enter  a rating  or  a 0 
(for  skipping  a criteria)  for  each  of  the  criteria  so  that  the  score  can  be  computed. 


Application  Evaluation  Engine  Table  for  Options:  A,  B,  Status  Quo 

' 

Importance 
1 Weighting  | 

A 

B 

Stat 

Evaluation  Criteria  Name 

I WebC 

1 Bla^ 

us 

t 

Quo 

9 

1 

1 

Accessibilitv 

7 

i 

8 ; 

2 

7 

student  suDDort 

8 

8 

4 

1 

' 6 

Instructor  suDDort 

8 j 

6 ! 

3 

1 

i 

J 

ScoreButton 

Weighted  Average  Scores  * 

7.59 

7.45  I 

2.9 

* NaN  - Not  a Number  error  results  from  a empty  rating  box  in  the  column  Help 


' Make  current  values  the  default  values 

! ~ I so  you  can  save  your  work 


Rechecking  by  doing  Sensitivity  Analyses 

One  of  the  benefits  of  using  a computerized  process  is  easy  to  have  the  machine  perform  calculations  again.  The  results  in 
the  decision  table  can  be  recalculated  after  tweaking  the  numbers  to  see  if  it  makes  any  difference  in  which  one  is  the  winner. 
The  adjusting  and  recalculating  can  be  done  on  the  basis  of  the  know  variability  of  the  numbers  involved  to  get  a better  idea 
of  the  stability  of  the  solution.  In  the  present  example  lets  say  we  were  curious  to  see  if  WebCT  would  still  be  the  winner  if 
the  average  instructor  support  rating  were  one  unit  lower  so  we  can  try  a rating  of  7 in  place  of  the  original  8 and  then  score 
the  whole  table  again.  In  this  case  the  change  of  a rating  from  a 8 to  a 7 would  in  fact  change  the  winner  to  BlackBoard 
because  the  Weighted  Average  Score  for  WebCT  would  drop  to  7.32  which  is  lower  than  BlackBoard.  The  point  of  doing  a 
bit  of  rechecking  is  that  it  can  become  clear  as  in  this  case  that  the  products  are  so  close  that  the  small  changes  in  the 
judgments  would  change  the  outcome. 

The  other  rechecking  that  can  be  done  is  to  examine  if  the  outcome  would  change  if  the  importance  weights  had  been  a little 
different.  In  the  present  example  if  the  importance  weight  for  accessibility  were  changed  to  90  from  9 to  examine  a 10  fold 
increase  in  the  importance  of  accessibility  then  BlackBoard  would  be  the  winner  with  a weighted  average  score  of  7.88 
compared  to  WebCT  with  a score  of  7.13.  Notice  in  both  of  these  instances  of  rechecking  the  status  quo  never  wins  so  one 
reasonable  conclusion  from  this  is  that  our  institution  should  prepare  for  a change  from  the  status  quo  as  at  least  that  much  is 
quite  clearly  an  outcome  of  this  decision  process. 

Summary  and  checklist 

The  main  reason  for  getting  so  involved  in  the  decision  of  which  learning  management  system  is  the  best  for  an  institution  to 
select  is  that  this  is  a very  important  decision.  Normally  when  people  make  a decision  or  even  groups  of  people  make  a 
decision  they  do  not  make  a rational  decision.  Only  when  the  judgments  that  people  make  are  quite  small  and  simple  can 
they  avoid  cognitive  illusions  and  biases.  Fortunately  this  kind  of  product  selection  decision  can  be  divided  up  into  many 
smaller  decisions  that  can  then  be  recombined  into  a weighted  averaging  decision-making  strategy  that  approximates  true 
rationality.  The  trade-off  is  suggested  to  first  use  an  elimination  by  aspect  strategy  to  arrive  at  a short  list  of  competitive 
products  worth  considering  and  then  do  a more  detailed  weighted  averaging  strategy  to  find  out  which  one  has  the  best 
After  that  the  decision  table  can  be  further  investigated  by  doing  sensitivity  analyses  to  find  out  how  stable  the 
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selection  of  the  winner  turned  out  to  be.  Using  the  web  tools  on  the  site  developed  for  this  decision-making  can  facilitate  this 
process: 

http://www.c2t2. ca/landonline 

The  online  educational  deliver  application  decision  checklist: 

1)  First  decide  what  features  are  important  in  the  local  situation 

2)  Screen  options  out  based  on  feature  criteria  using  web  tools 

3)  Making  a shortlist  of  product  and  features  required 

4)  Optional  Request  For  Proposals  (RFP)  from  vendors  on  the  shortlist 

5)  Judging  the  importance  of  features  and  assigning  feature  weights 

6)  Evaluating  the  product  features  for  each  of  the  shortlist  candidates 

7)  Using  the  Comparative  Analysis  Decision  Engine  Tool  to  pick  a winner 

8)  Rechecking  your  work  with  sensitivity  analyses 

9)  Communicating  how  the  decision  was  made  to  stakeholders 
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ABSTRACT 

The  aim  of  the  ECIC  project  is  the  dissemination  of  participative  methods  of  organizational  development 
originating  from  Scandinavian  countries  to  other  European  countries.  To  support  this  process  we 
developea  a hypertextual  learning  system.  Because  of  the  hetero^neous  target  group  and  the  ill- 
structured  domains  of  ECIC  detailed  usability  testing  is  necessary.  First  tentative  results  indicate  that 
graphical  overview  maps  play  an  important  role  and  interactive  examples  are  very  motivating. 

KEYWORDS:  hypermedia  application,  usability,  interactivity,  navigation 

INTRODUCTION 

The  aim  of  the  European  Continuous  Improvement  Circles  Project  (ECIC)  is  to  disseminate  participative 
methods  of  organizational  development  originating  from  Scandinavian  countries  to  the  rest  of  Europe. 
Examples  for  such  methods  are  improvement  circles  where  people  come  together  to  discuss  a given 
topic,  for  instance  implications  of  new  information  technology  for  their  workplace.  A facilitator  guides 
this  discussion  process  in  order  to  attain  agreed  goals. 

Manuals  on  paper  can  give  an  introduction  into  such  methods  but  sometimes  they  lack  the  capability  to 
convey  the  complex  nature  of  such  approaches.  More  interactive  forms  of  representation  like 
hypermedia  are  probably  more  appropriate  for  participative  methods.  In  addition,  the  ECIC  project 
consisted  of  several  fairly  similar  methods  strongly  related  to  each  other.  Therefore,  we  decided  that  a 
hypertext-based  learning  system  might  be  a viable  form  of  introducing  this  topic  to  European  countries 
not  acquainted  with  such  approaches.  The  potential  learners  of  the  ECIC  methodologies  are 
representatives  of  business  administration  or  Civil  Service  who  often  do  not  hold  academic  degrees.  The 
ECIC  learning  system  has  to  be  adapted  to  the  needs  of  this  user  group.  A thorough  evaluation  of  the 
learning  systems  is,  therefore,  necessary.  The  methods  of  ECIC  can  be  seen  as  an  ill- structured  domain 
[1],  therefore  it  is  the  main  interest  of  our  usability  study  to  verify  the  structural  representation  and  the 
user  interface  supporting  the  interpretations  of  the  learner  as  an  ongoing  process. 

EVALUATION  METHODS 

The  evaluation  has  several  aims.  The  most  important  one  is  to  find  out  how  users  navigated  the  learning 
system.  The  ECIC  methodologies  are  quite  similar  to  each  other  and  influence  each  other  in  their 
development.  Users  should,  therefore,  get  extensive  information  about  the  structure  of  the  material  to  be 
able  to  see  similarities  and  differences  of  the  different  approaches.  In  this  context,  several  aspects  are 
interesting.  The  learning  system  contains  several  pages  with  graphical  overviews.  We  wanted  to  know 
whether  spatial  visualization  of  information  is  effectively  realized  in  our  program.  Dee-Lucas  [6] 
assumes  tnat  spatial  visualization  is  especially  important  for  these  learners  who  compare  the  target 
content  and  generate  relationships  that  are  not  explicit  in  the  text.  Tosca  [4]  points  out  that  the  relevance 
of  links  is  the  result  of  an  ongoing  interpretation  process  by  the  learner.  To  get  information  about  this 
process,  we  analyze  whether  users  prefer  some  t^es  of  links  above  others  and  which  links  are  more 
understandable  than  others.  We  are  also  interested  in  how  long  learners  examine  single  pages  or  nodes. 

The  ECIC  learning  system  tries  to  present  the  information  in  an  interactive  way.  One  of  om  questions  is 
whether  users  are  motivated  by  interactive  examples  or  not.  If  they  engage  in  working  with  interactive 
examples  for  an  extended  period  of  time  this  mignt  be  an  indicator  for  the  attractiveness  for  such  pages. 
An  important  variable  is  also  whether  texts  are  clear  and  understandable.  A final  question  is  what  users 
keep  in  mind  when  they  have  finished  using  the  learning  system  and  if  their  recollection  is  based  on 
specific  representations  m the  program. 

There  are  several  methods  of  evaluating  computer  software  [3] [5].  We  chose  a combination  of  thinking 
aloud  protocols  and  software  logging.  The  thinking  aloud  protocols  are  supposed  to  convey  data  about 
users'  attitudes  towards  the  learning  system  and  the  motiyation  for  their  behavior.  We  have  formulated 
several  short  questions,  which  are  based  on  our  research  interests  formulated  above.  Users  are  asked  to 
keep  these  questions  in  mind  while  they  work  to  focus  their  utterances.  It  is  well-known  that  thinking 
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aloud  protocols  are  a fairly  disruptive  form  of  investigation.  Nevertheless,  we  find  that  this  method 
conveys  information,  which  cannot  be  obtained  otherwise,  like,  e.g.  learners'  motivation  for  using  or  not 
using  specific  links  or  graphical  overview  maps.  In  order  to  interpret  correctly  if  a user  stays  at  the  same 
topic  during  a long  time,  it  is  important  to  know  whether  the  user  finds  the  topic  very  interesting  or  very 
confusing. 

We  also  developed  a monitoring  tool  to  log  the  users'  actions.  This  tool  creates  a log-file  with  information 
about  the  duration  of  a session,  the  exact  time  when  a page  is  accessed  or  left  and  the  object,  which  is 
used  to  leave  a page. 


Figure  1:  Navigational  Graph  of  the  first  33  moves,  users'  moves  start  at  "Contents",  the  moves  are  shown  by  the  ascending 
numbers  next  to  the  directed  arrows;  “Map”  is  a graphical  overview  map,  “Contents”  is  the  main  content  of  ECIC  learning 
system,  “T_A”  to  “T_D”  stand  for  theme  A to  theme  D which  represent  four  different  topics  of  ECIC. 

From  these  data  we  can  infer  whether  users  traverse  the  document  hierarchical^  or  in  a sequential  way 
and  whether  they  visited  the  graphical  overviews  or  maps  to  a large  extent.  The  monitoring  tool  also 
provides  information  about  how  long  users  visit  a specific  node.  Based  on  this  information,  a directed 
graph  can  be  constructed  showing  the  moves  of  the  user  (Fig.  1).  McEneaney  uses  similar  graphs  to 
visualize  users'  behavior  [2]. 

If  analysis  of  users'  behavior  is  only  based  on  navigational  log- files,  wrong  conclusions  could  be  drawn. 
Both  methods  - thinking  aloud  protocols  and  software  logging  - are  supposed  to  complement  one 
another  to  give  a more  comprehensive  view  of  learners'  behavior. 

RESULTS 

So  far,  we  have  tested  two  persons  who  worked  with  the  ECIC  learning  system.  Those  tests  were  very 
extensive  and  yielded  a large  amount  of  data.  During  the  test  we  also  investigated  whether  the  test 
procedure  we  developed  was  viable  or  not.  We  realized,  for  example,  that  asking  the  users  too  many 
questions  during  the  test  was  rather  disruptive.  One  (tentative)  result  of  our  evaluation  is  that  the 
graphical  overviews  played  an  important  role  for  the  subjects,  although  one  of  the  two  persons  also 
employed  a sequential  mode  of  browsing  to  a large  extent.  For  both  subjects,  it  was  difficult  at  first  to 
adapt  to  the  hypertextual  form  of  organization  of  information.  As  expected,  users  spend  very  much  time 
examining  pages  with  complex  information.  But  Interactive  examples  engage  users'  attention  even  when 
they  do  not  convey  very  much  information.  One  user  mentioned  that  some  of  the  interactive  examples  in 
the  learning  system  gave  him  a more  vivid  impression  of  the  method  described.  After  the  test  we  asked 
the  users  to  write  down  the  most  salient  information  they  kept  in  mind.  One  of  the  subfocts  drew  a 
graphical  map  resembling  the  overview  map  in  the  program.  The  same  person  used  the  graphical 
overview  map  of  the  ECIC  learning  system  very  frequently. 
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CONCLUSION 

The  evaluation  of  the  ECIC  learning  system  is  an  ongoing  process.  So  far,  we  have  data  from  two 
subjects.  These  data  indicate  that  graphical  overviews  or  maps  probably  play  an  important  role  for 
learner  navigation.  We  intend  to  investigate  several  more  subjects  to  get  more  representative  results. 
Furthermore,  we  want  to  develop  a more  systematic  method  of  analysis  for  the  thinking  aloud  protocols 
and  for  the  directed  graphs  representing  the  learners'  paths  through  the  document. 
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Abstract:  When  it  comes  to  understanding  and  using  information  technology,  today’s  pre-service 
education  students  often  know  more  about  technology  than  their  college  professors  or  their  field-based 
teachers  in  K-12  classrooms.  University  and  K-12  faculty  members,  on  the  other  hand,  know  more 
about  teaching  and  learning  than  pre-service  students  and  less  about  technology.  By  bringing  these  two 
groups  together  as  student  mentor- faculty  pairs,  both  students  and  faculty  members  benefit  from  the 
expertise  of  the  other.  This  panel  will  describe  how  the  Faculty  Mentor  Program  developed  at  the 
University  of  Vermont  (an  initiative  of  the  Preparing  Tomorrow’s  Teachers  to  Use  Technology  [PT^] 
Implementation  Grant)  brings  education  students  and  university  faculty  together  to  support  new 
ways  of  learning  for  both  groups. 


Powerful  Partnerships 


Powerful  partnerships  can  occur  when  college  students  are  given  an  opportunity  to  teach  their  teachers.  By 
facilitating  student  mentor-faculty  pairs,  new  relationships  are  established,  new  skills  are  acquired  and  modeled, 
coursework  is  reinforced  by  practical  experience,  and  student  knowledge  is  tapped  and  utilized.  When  the  roles  are 
reversed,  when  the  learner  is  given  an  opportunity  to  become  the  teacher,  both  sides  gain. 

To  prepare  teachers  to  work  in  tomorrow’s  classrooms,  education  students  must  be  engaged  in  a program 
that  allows  them  to  learn  not  only  the  skills  to  use  technology  effectively  but  also  the  ability  to  integrate  information 
technology  to  improve  student  learning.  Part  of  the  pre -service  teacher’s  professional  preparation  should  include  a 
teacher  education  faculty  who  are  not  only  skilled  in  using  technology  systems  and  software  but  model  effective  use 
of  technology  as  a standard  component  of  coursework  (National  2000).  Pre -service  teachers  typically  are  exposed  to 
at  least  one  course  in  information  technology  during  their  preparatory  program.  Through  this  course,  students  either 
learn  new  technology  skills  and  applications  or  reinforce  existing  skills.  It  is  less  common,  however,  that  teacher 
education  programs  have  faculty  who  are  modeling  the  effective  integration  of  computer  technology.  Without  these 
role  models,  pre-service  teachers  are  less  likely  to  use  technology  in  their  own  teaching  (Beisser  2000).  National 
educational  technology  standards  recommend  that  pre -service  teachers  know  how  to  use  and  integrate  technology, 
both  for  instructional  and  professional  purposes  (National  2000).  In  effect,  pre -service  teachers  are  being  asked  to 
meet  standards  that  are  seldom  modeled  and  that  their  college  teachers  and  in-service  cooperating  teachers  often  fail 
to  meet  themselves. 

Because  college  students  often  have  or  acquire  information  technology  skills  through  coursework,  their 
knowledge  can  be  tapped  to  support  college  and  in-service  faculty.  “Using  graduate  students  to  mentor  college  of 
education  faculty  has  been  shown  to  be  an  effective  technique  for  integrating  technology  into  the  coursework  of  pre- 
service teachers”  (Beisser  1997).  When  graduate  students  are  unavailable,  undergraduate  students  have  also  been 
an  effective  option  (Beisser  1997). 


1.127 


Faculty  Mentor  Program 


Recognizing  the  need  to  promote  and  support  the  use  of  information  technology  in  its  educational  program, 
the  College  of  Education  and  Social  Services  at  the  University  of  Vermont  (UVM)  initiated  a Faculty  Mentor  Program 
in  the  Spring  semester  of  2001  as  part  of  the  Preparing  Tomorrow’s  Teachers  to  Use  Technology  (PT^) 
Implementation  Grant.  The  Faculty  Mentor  Program  was  created  to  help  education  faculty  (both  within  and  outside 
UVM)  strengthen  technology  skills  and  to  increase  the  integration  of  information  technology  in  college  coursework. 
Undergraduate  education  students  were  paired  with  K-16  faculty  members  to  accomplish  a set  of  goals  established 
by  the  faculty  member,  the  student  mentor  and  a PT^  grant  team  member.  Initially,  three  University  faculty  and  one  K- 
12  faculty  member  volunteered  to  work  with  undergraduate  students.  In  the  Fall  semester  of  2001,  five  University 
faculty  members  established  student  mentor  partnerships  and  additional  support  was  extended  to  three,  in-service  K- 
12  faculty  members  working  with  pre-service  teachers.  In  the  Spring  semester  of  2002,  the  number  of  student  mentor- 
faculty  pairs  is  anticipated  to  increase  to  twenty. 

Both  students  and  faculty  benefit  from  this  relationship.  Students  elect  to  earn  $ 10/hour  or  college  credit  for 
their  work.  Within  UVM,  education  faculty  members  have  worked  with  student  mentors  to  create  online  course 
syllabi  and  resources,  develop  new  online  courses,  and  to  improve  or  enhance  courses  already  in  place.  With  student 
mentorship,  faculty  members  have  learned  new  software,  explored  new  presentation  models,  developed  online 
courses  using  WebCT,  and  have  begun  to  model  the  use  of  information  technology  in  their  courses.  (Examples  may 
be  found  at:  http://www.uvm.edu/'^inquiryb/,  http://www.uvm.eduMitblock/,  http://wwwMvm.edu/-^crathbon/) 

Through  this  experience,  students  establish  a working  relationship  with  their  professors,  reinforce  their  own 
technology  skills  by  becoming  the  teacher,  and  participate  in  discussions  with  the  professor  about  the  rationale  and 
purpose  of  the  materials  under  construction.  In  fact,  some  student  mentors  have  been  instrumental  in  advising 
faculty  members  about  ways  to  improve  or  strengthen  course  work.  These  student  mentors  have  become  “critical 
friends”  of  their  professors  and  both  parties  have  become  co-leamers. 

In  the  K-12  arena,  student  mentors  have  the  opportunity  to  work  in  a local  school  district  with  a classroom 
teacher  and  their  students.  In  this  partnership,  the  student  mentor  may  work  directly  with  the  classroom  teacher  to 
strengthen  the  teacher’s  technology  skills  or  with  students  to  integrate  technology  into  the  curriculum.  Often,  the 
student  mentor  may  be  more  knowledgeable  than  the  classroom  teacher  about  the  use  of  technology  skills.  Student 
mentors  have  helped  K-12  faculty  learn  how  to  use  presentation  software  (such  as  PowerPoint)  with  middle  school 
students  to  create  science  presentations,  to  introduce  basic  word  processing,  keyboarding,  and  paint  program  skills 
to  second  grade  students,  to  research  educational  software  available  in  the  classroom,  and  to  create  web  pages  for 
curriculum  or  classroom  projects.  In  turn,  mentor  students  learn  more  about  the  complexities  of  being  a classroom 
teacher.  Here  again,  mentors  and  teachers  diare  their  knowledge  and  expertise  and  mutually  contribute  to  their 
learning. 

Studies  have  shown  that  it  takes  five  to  six  years  for  teachers  to  use  technology  effectively  in  their  teaching 
(Beisser  2000).  Ongoing  support  therefore  is  critical  to  support  professional  development  efforts  in  information 
technology.  Student  mentors  can  be  employed  to  reinforce  concepts  and  skills  learned  during  PT^  workshops  or 
summer  institutes.  This  continual  assistance  enables  the  faculty  member  to  move  forward  while  providing 
reinforcement  for  the  student  mentor  as  well.  Practical  applications  of  technology  provide  students  with  models  for 
their  future  growth  as  computer-using  teachers. 


Outcomes 

As  a result  of  the  Faculty  Mentor  Program,  notable  growth  has  been  seen  in  the  use  and  modeling  of 
information  technology.  More  University  faculty  members  have  begun  to  borrow  laptop  computers  and  more  time 
has  been  devoted  to  developing  new  online  projects.  Without  the  support  of  the  student  mentors,  this  increase  in 
use  and  interest  would  not  have  been  as  rapid.  Faculty  members  who  are  now  using  technology  to  support  their 
courses  are  gaining  recognition.  As  faculty  web  pages  improve,  other  faculty  members  become  increasingly 
interested  in  how  technology  can  improve  their  course  presentations.  Students  who  enroll  in  courses  offered  by 
professors  that  use  technology  “spread  the  word”  about  the  course  to  other  students.  This  interest,  in  turn, 
encourages  more  faculty  members  to  integrate  technology  into  their  coursework  and  to  consider  new  approaches. 
Faculty  members  are  beginning  to  publish  online  exemplars  of  student  work.  This  enables  the  professor  to 
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demonstrate  an  expectation  for  student  work  and,  as  a result,  the  standards  for  student  performance  have  been 
raised.  Some  faculty  members  have  begun  to  archive  student  work  online,  allowing  the  student’s  work  to  become  a 
knowledge  base  for  other  students.  This  growth  in  the  use  and  modeling  of  information  technology  has  been 
supported  by  the  efforts  of  student  faculty  mentors. 

The  Faculty  Mentor  Program  provides  an  opportunity  for  pre-service  teachers  to  reinforce  their  skills  in 
information  technology.  Students  establish  a collegial  relationship  with  their  professors  and  they  gain  more 
confidence  in  their  own  abilities.  It  provides  them  with  practical  experiences  that  they  can  document  and  use  when 
seeking  employment.  Students  who  work  in  K-12  classrooms  receive  the  benefits  of  working  directly  with  students  to 
see  how  information  technology  can  be  used  to  support  learning.  In  this  environment,  they  learn  about  different 
networking  systems  and  infrastructure,  educational  software,  curriculum  requirements,  school  rules  on  appropriate 
use,  the  challenges  of  out-dated  or  limited  resources,  and  many  information  management  lessons. 

Evolving  Strategies 

As  this  program  continues,  the  PT^  grant  team  will  refine  its  approach  and  methods  to  improve  and 
strengthen  student  mentor-faculty  pairs.  Expectations  and  guidelines  need  to  be  established  for  both  the  student 
and  the  faculty.  Commitment  must  be  mutual  on  both  sides.  Faculty  members  cannot  expect  students  to  produce 
content  for  their  courses,  but  they  can  be  active  collaborators  by  soliciting  and  sharing  constructive  feedback. 
Everyone  must  be  an  active  participant  in  the  learning  process.  Student  mentors  are  not  substitutes  for  secretarial 
assistance.  Student  mentors  must  learn  how  to  be  effective  teachers  and  supporters  of  information  technology 
integration.  With  collaboration  and  support,  both  parties  gain  from  this  partnership. 

Overall,  both  the  pre-service  teacher  and  the  education  faculty  benefit  from  this  model.  The  knowledge  and 
energy  that  a college  student  provides  is  a resource  to  professors,  K-12  faculty  and  K-12  students.  In  turn,  pre- 
service  teachers  gain  e:q>erience,  confidence,  and  practical  application  to  help  prepare  them  to  become  tomorrow’s 
teachers. 


Panelists 

Russell  M.  Agne,  Ph.  D.,  College  of  Education  and  Social  Services,  University  of  Vermont.  (Russell.  A gne@uvm.edul. 
Russell  M.  Agne  is  Professor  of  Education  at  the  University  of  Vermont  with  responsibilities  in  elementary  and 
secondary  teacher  education.  He  holds  a Ph.D.  in  science  education/curriculum  and  instruction  from  The  University 
of  Connecticut,  a M.S.  in  earth  science  education  from  Syracuse  University  and  a B.S.  in  Secondary  Education- 
Science  from  Central  Connecticut  State  College.  He  has  published  extensively  in  journals  such  as  The  Science 
Teacher^  The  Journal  of  Teacher  Education^  American  Journal  of  Physics,  Teachers  College  Record,  The  School 
Counselor,  Science  Education,  Phi  Delta  Kappan,  The  Journal  of  Research  in  Science  Teaching.  His  most  recent 
book,  co-authored  with  John  H.  Clarke  is  Interdisciplinary  High  School  Teaching:  Strategies  for  Integrated 
Learning  (1997)  published  by  Allyn  and  Bacon.  Professor  Agne  has  served  as  PI  for  more  than  20  grants  and 
projects  to  implement  science  curriculum  through  NSF,  Exxon  Education  Foundation,  The  Apple  Computer 
Foundation,  Eisenhower  Title  II  Funds,  and  Bell  Atlantic  Foundation.  He  has  been  elected  a Fellow  of  The  American 
Association  for  the  Advancement  of  Science,  and  currently  serves  as  IHE  Representative  on  the  Executive  Board  of 
the  Vermont  Science  Teachers  Association.  He  has  served  as  Department  Chair  of  Education  at  UVM  and  Associate 
Dean  of  its  Graduate  College. 

Adam  Deyo,  PT3  Graduate  Assistant,  College  of  Education  and  Social  Services,  University  of  Vermont. 
(Adam.Devo@uvm.edul  Mr.  Deyo  holds  a Bachelor  of  Science  in  Early  Childhood  Education  from  the  University  of 
Vermont  and  a Level  One  Teacher  Licensure  for  Kindergarten  through  Third  Grade.  He  was  a Child  Development 
Specialist  at  the  University  of  Vermont’s  Campus  Children’s  School  for  two  years  working  with  Infants,  Toddlers, 
and  Preschoolers.  In  addition,  he  mentored  two  separate  groups  of  student  teachers  in  the  classroom  throughout 
their  pre -service  work  with  young  children  each  year.  Mr.  Deyo  currently  is  seeking  a M.Ed  at  the  University  of 
Vermont  and  assists  on  all  aspects  of  the  Preparing  Tomorrow’s  Teachers  to  Use  Technology  Grant  in  the  College  of 
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Education  and  Social  Services.  He  is  also  is  a coordinator  for  the  Vermont  Association  for  the  Education  of  Young 
Children  and  a member  of  the  State  of  Vermont  Early  Childhood  Technology  Board. 


Sandra  A.  Lathem  M.Ed.,  PT3  Outreach  Coordinator,  College  of  Education  and  Social  Services,  University  of 
Vermont.  (Sandra.Lathem@uvm.edu)  As  PT3  Outreach  Coordinator,  Ms.  Lathem  recruits  and  supervises  student 
mentors  that  work  with  K-12  faculty  in  the  Burlington  and  South  Burlington  School  Districts.  She  supports  UVM 
student  mentors  to  assist  them  with  their  knowledge  and  integration  of  technology  in  the  K-12  classroom  and  works 
with  K-12  faculty  to  set  goals  and  objectives  for  the  student-faculty  mentorship.  Previously,  Ms.  Lathem  served  as 
the  Technology  Coordinator  for  the  Addison  Northeast  Supervisory  Union  in  Bristol,  Vermont  and  has  been  an 
educational  technologist  in  Vermont  school  systems  during  the  last  ten  years.  Formerly,  Ms.  Lathem  was  the 
Executive  Director  of  VITA -Learn,  a statewide,  non-profit  organization  supporting  information  technology  integration 
in  Vermont  K-12  schools.  Ms.  Lathem  received  her  Masters  of  Education  in  Professional  Development,  Curriculum 
and  Instruction  from  the  University  of  Vermont  in  1995. 


Joyce  L.  Morris,  Ed.  D.,  PT3  Professional  Development  Coordinator,  College  of  Education  and  Social  Services, 
University  of  Vermont.  (3ovce.Morris@uvm.edu)  Professor  Morris  is  an  assistant  professor  in  the  Department  of 
Education  at  the  University  of  Vermont.  Her  research  interests  include  electronic  portfolio  construction  for 
assessment  and  learning  and  professional  development  of  faculty  using  technology  tools.  At  the  University  of 
Vermont,  she  has  taught  undergraduate  courses  in  educational  technology,  science  methodology,  portfolio 
construction,  and  general  education.  At  the  graduate  level,  she  offers  courses  in  using  computers  in  the  classroom, 
web  design,  telecommunications,  and  connecting  technology  to  the  standards  for  in-service  teachers.  Prior  to  her 
appointment  at  the  University  of  Vermont,  Professor  Morris  was  a science  teacher  who  integrated  technology  into 
her  curriculum  at  a Bronx,  N.Y.  middle  school.  She  wrote  and  received  a grant  from  Apple  Computer  that  provided  a 
22  station,  networked  classroom  in  1989.  Using  these  computers,  students  created  science  animations,  used 
computer  probes  to  conduct  experiments  and  record  data,  published  science  pictionaries  for  younger  students,  and 
participated  in  a number  of  telecommunication  projects  with  TERC. 
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Abstract:  No  longer  just  a user  of  commercial  software,  the  21st  century  teacher  is  a designer  of 
interactive  software  based  on  theories  of  learning.  This  software,  a comprehensive  study  of  straight- 
line  equations,  enhances  conceptual  understanding,  sketching,  graphic  interpretive  and  word- 
problem  solving  skills  as  well  as  making  connections  to  real-life  and  scientific  phenomena. 
Developed  using  Maple  and  HyperStudio,  this  software  of  40  questions  actively  captures  students' 
visual  intelligence  and  evokes  thinking  to  provide  generative  responses.  Other  features  include 
corrective  feedback,  hide-and-show,  scoring,  timing  and  a student  output  file  containing  a student’s 
1st  and  2nd  responses  and  revealing  any  misconceptions  held  and  difficulties  encountered. 


Introduction 

A straight-line  equation,  the  most  basic  kind  of  mathematical  function,  is  one  of  the  most  unifying  ideas  in  mathematics.  It 
has  extensive  applications  in  the  world  of  science,  especially  physics,  business  and  social  science.  From  my  experience  as  a high 
school  mathematics  and  science  teacher  for  about  20  years,  I have  seen  how  the  lack  of  a conceptual  understanding  of  straight-line 
equations  has  hindered  students'  understanding  of  science,  especially  physics.  Some  difficulties  encountered  by  students  include 
the  inability  to  interpret  a graph,  to  determine  its  algebraic  representation,  to  verbalize  the  relationship  between  the  two  variables, 
to  sketch  a simple  line  without  going  through  a table  of  values,  or  to  see  the  graphical  representation  associated  with  an  algebraic 
expression. 

Dreyfus  & Eisenberg  (1982)  found  that  students  of  low  ability  had  difficulty  with  the  graphical  concept  of  a function.  In 
another  study  in  1991  they  contend  that  the  chief  source  of  difficulty  encountered  by  beginning  calculus  students  is  their  inability 
to  exploit  the  visual  representations  associated  with  the  concept  of  functions.  Vinner  and  Dreyfus  (1989)  surveyed  307  college 
students  on  their  concepts  of  a function  and  only  8 % of  the  students  made  reference  to  its  graphical  representation.  In  fact  many 
studies  investigated  students'  understanding  of  graphs  of  function  and  concurred  that  students  lack  the  skill  in  interpreting  graphs 
and  fail  to  see  the  connection  between  the  algebra  and  geometry.  (Knuth,  2000). 

In  this  computer  age  when  a large  quantity  of  data  can  be  easily  represented  in  different  graphical  forms,  it  is  vital  that 
students  be  proficient  in  the  interpretation  and  extrapolation  of  graphs  to  deduce  important  information  and  relationships 
between  the  variables.  Computer  technology  has  freed  students  from  the  laborious  task  of  graph  plotting  with  pencil  and  paper 
but  very  often  a student  is  not  even  aware  that  he/she  has  made  a mistake  in  keying  in  information,  resulting  in  the  display  of  a 
different  graph  or  function.  Thus  it  is  important  that  students  have  the  conceptual,  intuitive  and  visual  understanding  of  straight- 
line  equations  in  terms  of  positive  and  negative  slopes,  and  y intercepts  before  embarking  on  the  use  of  graphing  software. 

As  cited  in  the  Before  It's  Too  Late  report  (2000),  computer  technology  has  not  only  changed  the  way  we  live  and  the  way 
business  is  conducted,  it  has  changed  the  demands  of  our  work  force,  which  calls  for  students  to  be  equipped  with  the  ability  to 
think  and  solve  problems.  Educational  reforms  all  over  the  world  are  bringing  changes  to  meet  such  demands.  NCTM  (National 
Council  (f  Teachers  in  Mathematics  Standards)  (2000)  calls  for  a shift  in  emphasis  from  a curriculum  dominated  by  the 
memorization  of  isolated  facts  to  one  that  emphasizes  conceptual  understanding,  problem-solving,  functional  relationships, 
multiple  representations  and  connections,  in  particular  between  algebra  and  geometry,  as  well  as  to  the  real-world  and  other 
disciplines.  My  study  of  straight-line  equations  is  in  response  to  NCTM's  multi-faceted  frames  of  learning. 

In  Ontario,  the  New  Curriculum,  which  consists  of  4 years  of  high  school  compressed  from  5,  was  introduced  in  1999. 
Many  topics,  including  straight-line  equations,  have  been  moved  from  higher  to  lower  grades  causing  difficulties  for  many 
students.  A recent  report  confirms  a higher  failure  rate  of  our  grade  9 and  10  students.  (“Poor  Math”,  2001).  Teachers  with  an 
increased  workload  of  6 courses  from  5 are  also  faced  with  larger  class  sizes  and  new -immigrant  students  of  very  diverse  ethnic 
backgrounds,  languages  and  learning  abilities.  Professional  developments  for  teachers  and  the  use  of  computer  technologies  in  the 
form  of  software  or  the  Internet  are  high  on  the  agenda  to  support  student  learning. 
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Computer  Technology  and  Learning 

Douglas  Noble  (1988,  P.  241)  has  given  one  of  the  clearest  discussions  of  the  three  rationales  for  the  introduction  of 
computer  into  schools. 

"The  first  focuses  on...  technological  society  [which]  requires  new  skills,  including  computer  literacy... second 
rationale... technology  of  education:  computer-based  instruction  offers  new,  effective  and  efficient  ways  to  present  material,  to 
individualize  instruction... third  rationale  focuses  on  a technology  of  mind:  interactions  with  computers  enhance  cognitive  skills 
while  offering  the  possibility  for  intellectual  mindstorms". 

According  to  cognitive  theories,  computer  technologies  are  cognitive  learning  tools,  which  expand  human  abilities  such  as 
memory  and  processing.  Roschelle  (1994)  regarded  technology  as  a form  of  inquiry  with  three  functions  derived  from  Dewey  for 
the  following: 

a.  to  provide  a stable,  long-term  access  to  a problematic  situation  in  which  the  learner  can  repeatedly  replay  and  reflect. 

b.  to  provide  focus  and  context  to  identify  new  features  and  relationship. 

c.  to  augment  ways  of  acting  so  that  their  meaning  is  more  readily  available  to  others. 

Many  studies  have  shown  that  calculators  and  computer-based  materials  have  enhanced  learning.  Schwartz  (1999)  confirms 
the  5 aspects  of  mathematical  activities,  namely,  conjecturing  & exploring,  evaluating  & analyzing  data,  modeling,  conceptually 
grounding  manipulative  skills,  and  the  deepening  of  understanding,  which  can  be  enhanced  by  the  use  of  computer  technology.  In 
mathematics,  before  students  can  create  ideas  or  apply  their  ideas,  they  need  to  have  a good  grounding  of  basics,  facts  and 
conceptual  understanding.  Despite  all  the  merits  of  computer  technology,  a teacher  must  use  it  wisely  to  ensure  effective  learning 
and  must  give  clear  instructions  and  guidance.  I am  in  total  agreement  with  Bland  (1996,  p.2)  who  says,  "Technology  should  be 
used  to  expand  possibilities  for  students,  and  to  permit  them  to  explore  otherwise  inaccessible  problems.  However,  it  should  only 
be  introduced  after  they  understand  the  mathematical  concepts  involved,  and  are  able  to  manually  use  them  in  simple  problems". 

Exploring  and  conjecturing  are  stimulating  and  can  even  evoke  critical  thinking  for  the  average  and  above  average  students  but 
for  the  less  mathematically  inclined,  who  often  fail  to  discover  patterns  or  conjectures,  they  feel  very  lost  and  frustrated.  Mayes’s 
(1992)  study  concluded  that  the  average  students  performed  better  with  the  use  of  mathematical  software  with  exploratory  and 
programmable  functions,  but  not  the  weak  students.  Recently,  constructivism  has  been  met  with  opposition  by  parents  and 
educators  as  indicated  in  the  "Math  Wars"  brewing  in  California  which  called  for  the  adoption  of  the  “three  phase  approach", 
namely  the  direct  instruction  in  skills,  the  help  phase  and  the  self-regulated  drill  and  practice  (Becker  & Jacob,  1998).  Hirsch 
(1997,  p.6)  cites,  "...  only  through  intelligently  directed  and  repeated  practice,  leading  to  fast,  automatic  recall  of  math  facts,  and 
facility  in  computation  and  manipulation  can  one  do  well  at  real-world  problem  solving". 

Brady  (1991 , p.  149)  once  said,  “the  best  teaching  aid  is  often  those  designed  by  the  teachers  themselves”. 

Thus,  I have  embarked  on  this  project  of  designing  a CAI  software  ( mainly  to  serve  the  mathematically  less  inclined)  with  40 
questions  that  provides  lots  of  practice,  at  the  same  time  incorporating  a multi-faceted  frame  of  learning,  delving  in  depth  and 
width  into  all  the  topics  related  to  straight-line  equations.  The  research  also  examines  students'  conception  or  misconceptions  of 
straight-line  equations  as  well  as  ways  of  thinking  and  difficulties  encountered  in  word-problem  solving,  interpretation  of  graphs, 
and  scientific  applications. 

Software  Design  & Learning  Theories 

Piaget’s  and  Skinner’s  Theories.  Mellon  & Sass  (1981)  suggest  Piaget's  developmental  theory  as  a theoretical  foundation  for 
CAI.  Piaget  emphasized  the  movement  of  students  through  sequential  stages  of  cognitive  development;  each  stage  builds  on  the 
previous  one  to  develop  effective  learning.  Skinner  (1938)  listed  four  important  tenets  about  learning,  which  include  a short 
learning  process,  reinforcement,  immediate  feedback  and  provision  of  "stimulus  discriminations'  for  path  to  success,  most  of 
which  are  incorporated  in  the  design. 

Jonassen’s  Activity  Theory.  Interactivity  has  been  identified  as  an  important  characteristic  of  the  computer  environment  to  the 
acquisition  of  deep  learning.  According  to  Jonassen  and  Rohrer-Murphy  (1999),  Activity  Theory  posits  that  conscious  learning 
emerges  from  activity  or  performance.  This  software  is  designed  with  a high  degree  of  interactivity,  constantly  asking  the  student 
what  to  do  next  and  leading  her  through  a continual  process  of  interpreting,  judging,  constructing,  applying  concepts,  translating 
modes  of  representation,  and  reinforcing  algorithmic  skills. 

Real-Life  & Interdisciplinary  Learning.  Students  have  often  asked  me  “What  good  are  the  x andy  in  the  real  word?  How  are 
they  helping  me  to  find  a job?"  In  the  teaching  of  mathematics,  it  is  imperative  to  bring  in  applications  be  they  science,  business 
or  real-life  problems,  to  foster  conceptual  understanding  as  well  as  convince  students  of  the  power  and  usefulness  of  mathematics. 
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Beane  says  that  students  are  to  "...  integrate  learning  experiences  into  their  schemes  of  meaning... And  knowledge  is  called  forth  in 
the  context  of  problems,  interests  and  concerns  at  hand."  (1995,  p.  616).  Carifio  (1994)  identifies  key  contextual  features  as 
familiarity,  imageability  and  variable  type  (discreet  or  continuous  quantities).  The  questions  in  the  software  are  carefully  prepared 
to  incorporate  such  features  i.e.  real-life  and  scientific  phenomenon  that  students  are  familiar  with  or  can  relate  to. 

Problem-based  Learning.  Problem  solving  is  a form  of  higher-order  thinking  and  depends  on  a student’s  resources,  heuristics, 
control,  and  worldviews  (Schoenfeld,  1 985).  My  software  aims  at  developing  better  heuristics  and  control  in  students  as  well  as 
fostering  the  ability  to  identify  relevant  information,  to  transform  words  into  symbolic  equations,  to  assign  suitable  variables  and 
notation  for  immediate  recall  of  significant  information,  to  discern  the  reversal  error  (Rosnick,  1981).  and  to  apply  the  optimum 
algebraic  skills.  Through  working  with  the  word  problems,  students  will  learn  to  plan  and  sequence  their  tasks,  justify  their 
answers,  and  test  their  final  answers  by  integrating  algebraic  and  geometric  methods. 

Howard  Gardner's  Visual  Intelligence.  Gardner  (1983)  notes  that  many  students  have  preferred  ways  of  learning  depending  on 
their  "intelligence"  which  could  be  kinesthetic,  spatial  or  visual,  musical,  verbal,  intrapersonal,  interpersonal  or  naturalistic.  This 
software  captures  the  visual  intelligence  of  the  student  through  the  many  vivid  representations  of  graphs  created  by  Maple  (Char 
et  al,,  1991)  and  fosters  logical  reasoning  in  the  interpretation  of  graphs.  According  to  Mones-Hattal  & Mandes,  (1995,  p.891) 

visual  literacy  is  “a  cognitive  process  that  includes  mental  rehearsal,  introspection  and  visualization whereby  each  exposure  to 

the  visual  image  permits  the  observer  to  become  a keener  interpreter  of  the  visual  display". 


My  Interactive  Straight-Line  Equation  Software 

This  software  consists  of  4 modules,  namely.  A:  Slope  by  visualization  or  calculation,  B:  Graphical  to  algebraic  form,  C: 
Applications  and  D:  Word-problem  solving.  The  graphs  were  drawn  with  Maple.  First  the  grid  was  drawn  with  tiny  circles  each 
representing  integer  coordinates  or  some  multiples  of  integers.  A set  of  horizontal  points  was  generated  using  the  Maple  “seq” 
function  and  the  corresponding  points  were  drawn  with  the  “plot”  function.  These  two  functions  were  placed  inside  a loop  to 
draw  subsequent  lines  of  points  forming  a grid.  The  line  y = mx  + b was  then  drawn  and  the  whole  graph  was  exported  to 
HyperStudio.  (See  Appendix  I for  Maple  Program).  Hyperlogo  in  HyperStudio  is  used  in  achieving  the  more  sophisticated 
interactive  features  such  as  keeping  track  of  the  number  of  times  a student  has  answered  an  item  or  analyzing  if  the  response  is 
correct  and  furnishing  a corrective  feedback.  The  following  table  shows  the  correspondence  between  the  features  of  this  software 
and  Kuittinen’s  (1998)  four  criteria  or  demands  for  a good  CAT. 


Demands 

(Kuittinen) 

Characteristics  (This 

software) 

Domain  - 
Dependent 

-Questions  are  relevant  to  instructional  aims  and  curriculum  related  -Questions 

are  interdisciplinary  or  related  to  real-life  phenomena 

Instructional 

-Each  module  carries  at  least  one  worked  example  -Generative 

responses  as  well  as  multiple-choice  questions.  -Immediate  feedback; 

correct  answer  is  displayed  after  2 incorrect  inputs.  -A  high  level  of  interactivity  to 

elicit  continuous  student  participation;  step  by  step  guidance.  -Ample  questions  for  practice. 

-Questions  are  contextual  in  nature.  -Student 

performance  is  assessed  with  an  output  file,  which  has  a record  of  the  and  2"^*  responses. 

- Student's  achievement  score  is  recorded  giving  an  indication  of  the  student's  progress.  -Timing 

feature  keeps  a record  of  the  time  the  student  spends  on  a question  giving  an  indication  of  perceived 
difficulty. 

-Powerful  graphics  with  clear  and  precise  graphs  on  a dotted  grid  to  capture  student’s  visual 
intelligence  and  to  facilitate  the  skill  in  interpreting  graphs.  -Hide- 

1134 


User  Interface 

and-show  feature  to  capture  student's  attention  and  concentration  on  one  particular  idea  at  a time. 
-Drag-and-drop  feature  to  help  students  tack  on  x and  y coordinates  to  an  ordered  pair  of  a point. 
-Learner's  choice  of  method  in  word-problem  solving.  Only  when  the  student  fails  to  achieve  the 
answer  does  he  have  to  follow  certain  prescribed  steps,  those  steps  taken  by  experts!  Hopefully  this 
trains  students  to  emulate  experts.  -Accommodation  of 

upper  and  lower  case  input.  -Navigation  is  easy;  each 

card  is  linked  to  the  first  card,  which  has  the  exit  icon,  as  well  as  automatically  to  the  next  card  or  the 
menu  selection  card.  - In  some  modules,  questions  are  identified 

as  easy,  moderate  and  difficult.  -Texts  of  different  colours  and  highlighting  for 

easy  reading.  -Students  can  work  at  his  own  speed. 

Pragmatic 

HyperStudio  will  have  to  be  installed  in  the  computer  to  run  the  software.  I am  now  preparing  a compact 
disk  with  HyperStudio  embedded  in  so  it  can  run  independently.  Thus  the  software  is  portable  and  does 
not  require  any  technical  support  for  installation.  The  modules  could  easily  be  posted  on  the  Internet. 

Visit  httD://fcis.oise.utoronto.ca/‘^vlawrence/stlines.html  to  view  sample  computer  screens. 

If  you  have  Quick  Time  5 on  your  machine,  you  may  tryout  a sample  of  the  interactive  software  at 
http://fcis.oise.utoronto.ca/‘^vlawrence/h  vDertrl.html 


Module  D:  Word-Problem  Solving 


1 . Dimensions 

6.  Distance,  speed 

2.  Mixtures 

7.  Distance,  speed, 
(different  starting  times) 

3.  Money  (coins) 

8.  Tickets,  money 

4.  Age 

9.  Car  rentals 

5.  Concentrations 

10.  Numbers 

Table  of  Characteristics  of  Software 


Nature  of  Questions  & Links 
Module  C:  Applications 


1.  Ohm’s  Law 

6.  Speed  of  sound 

2.  Density 

7.  Motion 

3.  Friction 

8.  Snell’s  law  (light) 

4.  Electric  charge 

9.  Heat 

5.  Electrical  energy 

10.  Cooling  of  Water 

Proposed  Study 

A class  of  grade  10  students  who  have  covered  the  content  of  straight  line  equations  will  be  given  a pre-test  of  8 questions  , 1 
on  slope  by  inspection,  1 by  calculation,  1 on  transformation  from  graphical  to  algebraic  form,  1 from  algebraic  to  graphic  form,  1 
on  formulation  of  equation  given  relationships  expressed  in  words,  1 on  the  interpretations  of  a graph  representing  a scientific 
phenomenon,  and  2 on  word-problems.  From  the  results  of  the  pre-test,  weak  and  average  students  (selected  few)  will  be  solicited 
for  interviews.  The  whole  class  is  then  exposed  to  the  interactive  software  but  think-aloud  protocols  or  audio-video  taping  may 
be  employed  to  the  selected  few  who  work  individually  or  in  pairs.  Students  are  then  given  a questionnaire  about  their  learning 
experience  with  the  software,  followed  by  a post-test  and  a final  interview  (selected  few),  the  latter  to  probe  further  into  their 
misconceptions  captured  in  computer  files,  audio-video  taping  and  post-test. 


Pilot  Tests  & Findings 

In  the  first  pilot  study,  57  grade  9 students  ( from  3 classes)  wrote  a pre-test  followed  by  treatments  of  modules  B (graphic 
to  algebraic)  and  C(applications).  Students  were  then  given  a questionnaire  in  my  absence,  followed  by  a post-test.  A quantitative 
analysis  of  the  pre  and  post-test  using  the  comparison  t test  for  two  means  from  dependent  (correlated)  samples  pointed  to  a 
significant  difference  in  the  scores  or  gains.  Feedback  from  questionnaires  revealed  that  91%  of  the  students  affirmed  that  the 
software  had  enhanced  their  learning. 

In  the  second  pilot  test,  three  grade  10  students  were  interviewed  individually  followed  by  a pre-test.  Each  then  interacted 
individually  with  module  D (problem-solving)  followed  by  a questionnaire  and  a post  -test.  Both  feedback  from  the  questionnaire 
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and  results  of  the  post-test  indicated  significant  gains  in  learning.  Analysis  of  “captured  files”  also  revealed  certain 
misconceptions  held  and  difficulties  encountered  by  students,  some  of  which  were  the  inability  to  reduce  (9 A + 6S  = 7500), 
relating  to  time  frames  (to  add  or  subtract  when  leaving  a point  at  a later  time),  handling  special  cases  ( solving  C=40+0.2D  and 
C=30+0.25D,  did  not  see  coefficient  of  1 in  C provides  direct  subtraction  or  equating),  false  commutation  (length  is  2 cm  less 
than  width  as  L = 2-w),  faulty  conversion  between  decimal  and  percent  etc. 

In  general  the  students  agreed  that  the  software  had  helped  them  to  attain  a better  conceptual  understanding  of  straight-line 
equations,  had  improved  their  graphing,  graphic  interpretive,  algebraic  and  problem-solving  skills,  in  particular,  the 
transformation  of  relationship  in  words  to  symbols.  They  also  indicated  that  they  view  mathematics  as  more  meaningful  and 
have  less  fear  of  word-problem  solving. 


Conclusions 

In  view  of  positive  pilot  study  results,  this  interactive  software  will  not  only  enhance  the  learning  of  straight-line  equations 
related  to  graphical  interpretive  and  problem-solving  skills  but  also  foster  a better  understanding  of  scientific  principles  and 
usefulness  of  mathematics  in  the  real  world.  Misconceptions  and  students'  ways  of  thinking  can  shed  light  to  how  teachers 
should  teach  and  serve  as  a resource  for  curriculum  designers  and  planners.  To  accommodate  students’  learning  in  this  computer 
technology  era,  software  must  not  only  cover  all  aspects  of  a topic;  it  must  link  them,  actively  and  interactively  engaging  the  user 
as  a builder  of  his  own  knowledge.  These  principles  of  software  design  can  be  applied  to  other  topics  in  the  mathematics 
curriculum,  in  particular  the  extension  to  quadratic,  exponential,  logarithmic  and  trigonometric  functions. 


Appendix  I 

This  MAPLE  program  generates  the  graph  consisting  of  the  grid  points,  coordinate  axes  and  the  straight  line  y = lOx  + 20 
with(plots):hlines:={ }; 

hlines  :=  {} 

for  j from  1 to  6 
do 

yval:=  10*j: 

yrlist:=[seq([i,  yval],i=1..4)]: 

hlines:=hlines  union  {plot(yrlist,  style=point,symbol=circle,colour=blue)}: 
od: 

display(hlines); 
e:=plot(10*x  +20,  x=0..4): 
display({e}  union  hlines); 
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Abstract:  The  flexible  nature  of  hypermedia  allows  it  to  be  tailored  to  an  individual’s  needs.  Despite  the  many  degrees  of 
navigational  freedom,  however,  users  of  hypermedia  often  find  difficulty  locating  information,  feel  disoriented,  or  even  become 
“lost  in  hyperspace”  within  such  large  seas  of  data.  Research  findings  suggest  that  the  disorientation  problem  could  be  solved  if 
users  are  able  to  hold  a conceptual  overview  of  the  hypermedia  structure.  Variables  such  as  the  different  ways  of  representing 
information  structure  cues  and  cognitive  styles  may  affect  learners’  cognitive  abilities  in  knowledge  structuring  and  should  be 
taken  into  account  together. 


INTRODUCTION 

With  the  development  of  large  information  space  such  as  digital  libraries,  however,  users  of  hypermedia  often 
experience  difficulty  locating  information.  Scholars  (Beasley  & Waugh,  1995;  Dias  & Sousa,  1997)  have  suggested  that  the 
navigational  problem  could  be  solved  if  users  are  able  to  hold  a conceptual  overview  of  the  hypermedia  structure.  In  other 
words,  if  users  can  substantially  construct  a structural  knowledge  (Jonassen,  Beissner,  & Yacci,  1993)  of  the  information 
provided  in  the  hypermedia  system,  their  performance  could  be  improved.  In  a hypermedia  context,  graphical  browsers  have 
been  introduced  as  either  schematic  (concept  map)  or  spatial  (hyperlink)  representations  of  the  network.  Both  methods 
above  represent  what  Halasz  and  Moran  (1982)  have  called  "conceptual  models".  Yet,  it  may  not  always  be  easy  for  novice 
learners  to  use  an  abstract  conceptual  model.  If  a supplied  conceptual  model  is  not  familiar  to  the  learner,  then  this  strategy 
will  not  work  (Halasz  & Moran,  1982).  Consideration  of  this  issue  suggests  the  usage  of  the  metaphorical  GUI  (Carroll  & 
Thomas,  1982;  McKnight,  Dillon,  & Richardson,  1991;Rizk,  Streitz  & Andre,  1990),  e.g.,  the  office  metaphor,  to  help  with 
the  development  of  new  cognitive  structures  from  existing  ones. 


CONCEPT  MAP  AND  METAPHORICAL  GUI 

Despite  of  some  exciting  possibilities  in  the  realm  of  user-  centered  learning,  there  is  a fundamental  problem  regarding 
hypermedia;  it  provides  too  few  structural  cues  for  some  users  (Chamey,  1987;  McDonald  & Stevenson,  1996;  Dias  & 
Sousa,  1997).  They  need  to  depend  on  some  sort  of  consistent  layout  and  structure  for  their  understanding  and  would 
succumb  to  confusion  or  disorientation  if  their  expectations  were  violated.  Designers  have  to  realize  what  types  of  structure 
cues  hypermedia  could  adopt  and  then  evaluate  them  at  the  planning  stage. 

Concept  maps  (Nielsen,  1990;  West,  Farmer,  & Wolff,  1991)  are  maps  that  display  experts’  knowledge  structures 
graphically.  This  navigation  tool  is  similar  to  the  cognitive  representation  formed  by  the  learners  and  is  a mental 
representation  of  external  spatial  relations  used  to  navigate  in  physical  spaces  (Dias  & Sousa,  1997).  In  hypermedia -based 
instruction,  concept  maps  are  presented  as  templates  to  facilitate  the  novice’s  acquisition  of  experts’  knowledge  structures. 
The  concept  map  has  been  categorized  by  Jonassen,  Beissner,  and  Yacci  (1993)  as  an  organizing  strategy  which  might  help 
learners  to  construct  or  reorganize  their  knowledge  structures  by  showing  how  concepts  relate  to  each  other. 
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Another  type  of  structural  cues  is  the  metaphorical  GUI.  When  a computer  user  learns  a new  computer  program,  he 
calls  upon  his  prior  knowledge  (analogies  or  metaphors)  as  the  basis  on  which  to  form  a new  mental  model.  Designers  can 
take  advantage  of  users’  existing  mental  models  to  present  ways  of  conceptualizing  computer  functions  and  to  design 
interfaces  for  computing  systems  (Carroll  & Thomas,  1982).  Nepon  and  Cates  (1996)  state  that  appropriate  use  of  graphics 
embedded  in  interface  metaphors  may  have  the  potential  to  increase  "the  rate  at  which  users  can  process,  understand,  and 
respond  to  a display"  (p.  1).  They  note  Bielenberg's  (1993)  view  on  the  "transfer"  function  of  metaphor  and  suggest  that 
interface  designers  may  use  visual  metaphors  to  design  a conceptual  model  in  order  to  conform  to  users'  mental  models  in  a 
complete  and  painless  manner  while  the  users  are  learning  a new  domain.  Bishop  and  Cates  (1996)  also  suggest  that  a 
metaphorical  GUI  helps  users  to  establish  expectations  for  the  new  computer  systems  and  facilitates  the  understanding  of 
the  systems  to  be  learned  by  utilizing  the  users’  familiar  concepts  and  experiences. 

Apart  from  the  representational  aspect  of  hypermedia,  the  effects  of  learners'  cognitive  characteristics  on  the 
navigational  problem  are  another  concern  in  this 
study. 


COGNITIVE  STYLE 

Wickens  (1990)  and  McDonald  and  Stevenson  (1999)  indicate  that  once  the  correspondences  among  the  physical 
presentation  of  the  real  world,  the  user's  egocentric  view  of  the  real  world,  and  the  user's  mental  representation  of  the  world 
are  broken,  disorientation  will  happen.  The  physical  presentation  of  the  real  world  is  the  way  that  author  uses  to  convey 
information.  The  user’s  egocentric  view  of  the  real  world  could  be  viewed  as  an  internal  issue  regarding  the  way  that  users 
interpret  what  they  see.  How  users  interpret  what  they  see  highly  relate  to  the  individual  cognitive  styles.  Several  studies 
have  found  that  among  other  individual  elements,  the  effects  of  cognitive  styles,  is  arguably  one  of  the  most  important 
factors  that  might  affect  learners’  performances,  especially  in  a hypermedia  environment  (Andris  & Stueber,  1994;  Chang, 
1995;  Leader  & Klein,  1996).  Cognitive  style  refers  to  an  individual’s  cognitive  tendency  for  perceiving,  processing,  and 
organizing  information.  A subject's  structuring  ability  or  cognitive  style  might  be  more  strongly  manifested  in  a non-linear 
environment.  When  a learner  tries  to  link  incoming  information  with  established  ideas,  the  incidences  of  misconception 
might  increase  if  the  text's  organizational  pattern  can  not  be  used  appropriately.  Disorientation  and  confusion  thus  may 
occur  simultaneously  and  become  potential  hazards  for  studying  in  a hypermedia  environment  (Hammond,  1993).  In  this 
study,  cognitive  style  will  be  examined  within  the  dimension  of  "Wholist"  and  "Analytic"  (Riding  & Glass,  1997) 


SUMMARY 

It  is  hypothesized  that  learners’  cognitive  styles  and  two  types  of  structure  cues  could  affect  learners’  feelings  of 
disorientation.  Learners'  cognitive  characteristics  play  a central  role  in  the  regulation  of  learning  processes.  Yoder  (1994) 
and  other  researchers  (Rasmussen  & Davidson-Shivers,  1998)  feel  that  it  is  essential  for  educators  to  beware  of  the 
differences  of  individual  cognitive  styles  and  their  effects  on  performance.  However,  little  is  known  about  the  manner  in 
which  students  employ  these  characteristics  in  a hypermedia  context  to  search  information,  or  about  the  way  in  which  this 
execution  is  affected  by  different  information  representation  approaches  (Burton,  Moore,  & Holmes,  1995). 
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Abstract:  The  purpose  of  this  study  was  to  design  and  develop  multimedia-based  anchored 
program  and  to  examine  the  effects  of  students’  and  group  characteristics  on  the  problem-solving 
process  in  anchored  instruction  with  the  multimedia  program  in  a situated  learning  environment. 
Sixty-eight  students  were  assigned  to  small  groups  via  a stratified  random  sampling  procedure. 
The  students  were  working  cooperatively  as  a group  on  the  authentic  task  of  multimedia-based 
anchored  instruction.  The  results  of  the  analysis  showed  that  group  composition  as  well  as 
students’  characteristics  significantly  exerted  differential  effects  on  the  problem-solving  process. 
The  implications  of  the  results  for  anchored  instruction  were  discussed. 


Introduction 

Along  with  advances  in  information  and  communications  technology,  there  has  been  a shift 
towards  a new  paradigm  of  education  and  training.  The  emerging  paradigm  entails  the  characteristics  that 
distinguish  between  industrial-age  and  information-age  educational  systems  (Kang  & Kim,  1999; 
Reigeluth,  1999).  The  educational  system  based  on  a new  paradigm  supports  a constructivist  approach  to 
the  teaching  and  learning  process  and  fosters  learner-centered,  interactive,  contextualized,  and  meaningful 
learning  environments  with  diverse,  multiple  perspectives. 

There  have  been  an  increasing  number  of  studies  on  anchored  instruction  in  authentic  and  situated 
learning  environments  with  highly  interactive  and  engaging  forms  of  technology-supported  contents  (e.g., 
Bransford  et  al.,  1994;  Brown  et  al.,  1989;  Cognition  and  Technology  Group  at  Vanderbilt,  1991;  1992; 
1993a;  1993b;  Crews  et  al.,  1997;  McLellan,  1996;  Shih,  1997;  Ruokamo,  2001;  Young,  1993).  The 
anchored  instruction  approach  emphasizes  authentic  learning  in  solving  complex  problems  typically 
found  in  real  world  situations  and  collaborative  work  in  small  groups  to  generate  sub-problems  and 
develop  solutions  utilizing  embedded  data.  This  approach  can  provide  students  a situated  learning 
environment  that  facilitates  problem-solving  as  well  as  problem- definition  skills  and  collaborative  efforts 
among  students  during  group  work  on  the  authentic  task. 

In  general,  research  on  anchored  instruction  has  shown  positive  effects  on  students’  cognitive  and 
social-affective  outcomes  (e.g..  Cognition  and  Technology  Group  at  Vanderbilt,  1992;  1993a;  1993b; 
1997;  Hmelo  et  al.,  1993;  Lee  & Hwang,  2001;  Serafmo,  1999;  Shih,  1997;  Ruokamo,  2001).  Most  of  the 
studies  have  utilized  the  Adventure  of  Jasper  Woodbury  series  developed  by  Cognition  and  Technology 
Group  at  Vanderbilt  as  the  anchored  program.  Until  recently,  however,  few  studies  have  investigated 
anchored  instruction  with  different  forms  of  anchored  program  for  students  from  a different  cultural 
background.  To  this  end,  the  present  study  attempted  to  develop  multimedia -based  anchored  courseware 
that  is  adaptive  to  elementary  school  curriculum  in  Korea  and  to  examine  the  factors  affecting  the 
problem  solving  process  in  anchored  instruction  with  multimedia  courseware  in  a situated  learning 
environment. 

The  purpose  of  this  study  was  to  design  and  develop  multimedia -based  anchored  courseware  and 
to  examine  tihe  effects  of  cognitive  style  of  the  student,  the  type  of  problem-solving  task,  and  group 
composition  on  achievement  in  anchored  instruction  with  multimedia  courseware  in  a situated  learning 
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environment.  The  achievement  results  in  understanding  of  problems,  accuracy  of  problem  solving,  and 
reasoning  and  organization  of  problem  solving  of  students  working  on  original  and  analogy  tasks  were 
compared  in  heterogeneous  and  homogeneous  groups  featuring  individual  and  group  accountability. 


Method 


Sample 


The  subjects  for  the  study  were  sixty-eight  students  (33  females,  35  males)  in  the  sixth  grade  at 
two  elementary  schools  in  a metropolitan  city  in  Korea.  All  the  subjects  had  some  previous  experience 
with  computers  and  the  Internet  at  or  outside  of  school  in  a variety  of  subject  areas.  The  subjects  were 
assigned  to  small  groups  via  a stratified  random  sampling  procedure  on  the  basis  of  pretest  results.  They 
remained  in  the  same  groups  throughout  the  study. 


Procedure 

Before  the  study  began,  students  were  asked  to  complete  a background  survey  regarding  students’ 
previous  experience  with  computers.  The  pretest  was  administered  to  assess  students’  previously 
acquired  knowledge  with  the  subject  or  content  areas  of  the  authentic  task.  Stratified  random  sampling 
was  used  to  assign  students  to  heterogeneous  and  homogeneous  groups.  Heterogeneous  groups  consisted 
of  high-  and  low-score  students.  Homogeneous  groups  consisted  of  students  of  the  same-level  score. 
Students  were  unaware  of  the  composition  of  the  group.  Students  also  took  the  Group  Embedded  Figures 
Test  so  as  to  identify  those  with  field  dependence  or  field  independence.  Students  were  then  given 
practice  sessions  of  anchored  instruction  with  multimedia  courseware  developed  for  the  study.  They  were 
instructed  to  work  cooperatively  as  a group  on  the  authentic  task  of  multimedia  courseware,  to  help  each 
other  learn,  and  to  make  group  decisions  on  the  course  of  their  actions  in  solving  a set  of  problems. 
Students  worked  for  five  80-minute  instructional  sessions. 


Multimedia  Courseware 

For  the  purpose  of  this  study,  multimedia  courseware  was  designed  and  developed  for  Anchored 
Instruction.  This  courseware  appears  to  be  one  of  the  first  programs  for  anchored  instruction  that  was 
designed  to  be  adaptive  to  elementary  school  curriculum  in  Korea.  In  developing  multimedia  courseware, 
the  author  attempted  to  meet  the  design  principles  for  the  development  of  the  anchored  program  identified 
by  Cognition  and  Technology  Group  at  Vanderbilt  (1990;  1991;  1992;  1993a;  1993b;  1997).  The  design 
principles  include:  (1)  video-based  presentation  format,  (2)  narrative  format,  (3)  generative  learning 
format,  (4)  embedded  data  design,  (5)  problem  complexity,  (6)  pairs  of  related  adventures,  and  (7)  links 
across  the  curriculum.  The  subject  or  content  areas  are  cross-curricular,  including  proportion  and  ratio  in 
mathematics,  geography  in  the  social  studies,  and  environmental  awareness  in  science.  The  multimedia 
courseware  provides  a situated  learning  environment  where  students  work  together  as  a group  on  the 
authentic  task  so  that  they  can  generate  and  solve  a series  of  problems  and  sub-problems  and  relate  the 
solutions  to  the  complex  problem.  Through  their  learning  process,  students  can  explore  the  hyperlinked 
multimedia  contents  of  the  anchored  program  with  embedded  data.  In  designing  and  developing  the  user 
interface  of  the  courseware,  a special  emphasis  was  placed  on  user-fnendliness  and  efficiency.  A simple, 
intuitive  design  with  a text-based  menu,  rather  than  a complicated  design,  was  preferred  (e.g..  Harper  et 
al.,  1993;  Hedberg  at  al.,  1994). 


Research  Design  and  Data  Analysis 

The  study  employed  a randomized  block  factorial  design.  The  between-subjects  factors  included 
Cognitive  Style,  Task  Type,  and  Group  Composition.  The  within-subjects  factor  included  achievement 
scores  of  understanding  of  problems,  accuracy  of  problem  solving,  and  reasoning  and  organization  of 
problem  solving.  The  analysis  of  covariance  (ANCOVA)  was  performed  to  determine  the  interaction 
effects  as  well  as  the  main  effects  of  cognitive  style,  task  type,  and  group  composition  on  achievement  in 
understanding  of  problems,  accuracy  of  problem  solving,  and  reasoning  and  organization  of  problem 
solving  with  students’  pretest  results  serving  as  the  covariate.  The  level  of  significance  was  set  at  .05  in 
this  study. 


Results  and  Discussion 


The  means  and  standard  deviations  for  achievement  in  understanding  of  problems,  accuracy  of 
problem  solving,  and  reasoning  and  organization  of  problem  solving  are  presented  in  Table  1.  The  results 
of  the  analysis  of  covariance  for  achievement  by  cognitive  style,  task  type,  and  group  composition  are 
shown  in  Table  2. 

Understanding  of  Problems.  Significant  main  effects  were  found  for  cognitive  style,  F (1 , 59)  = 
4.306,  p < .05,  for  task  type,  F (1,  59)  = 6.189,  p < .05,  and  for  group  composition,  F (1,  59)  = 5.328,  p 
< .05.  Significant  interaction  effects  were  also  found  for  cognitive  style  and  group  composition,  F (1,  59) 
= 4.078,  p<  .Q5.  These  results  indicate  that  students  working  on  the  analogy  problem-solving  task  scored 
higher  than  did  those  working  on  the  original  problem-solving  task  and  that  cognitive  style  of  the  student 
and  group  composition  exerted  differential  effects  on  achievement  in  understanding  of  problems,  as 
shown  in  Tables  1 and  2.  Field  independent  and  dependent  students  tended  to  achieve  differentially 
across  the  groups  of  different  composition  on  the  posttest  of  understanding  of  problems.  The  students  in 
heterogeneous  groups  scored  higher  than  did  those  in  homogeneous  groups.  This  pattern  is  more 
noticeable  among  field- independent  students  than  field- dependent  students.  TTie  mean  score  of  the  field- 
independent  students  in  heterogeneous  groups  was  the  highest  (4.28);  the  mean  score  of  the  field- 
dependent  students  in  homogeneous  groups  was  the  lowest  (2.38). 

Accuracy  of  Problem  Solving.  There  were  significant  main  effects  for  cognitive  style,  F (1,  59) 
= 6.012,  p < .05,  and  for  task  type,  F (1,  59)  = 7.303,  p < .05,  and  significant  interaction  effects  for 
cognitive  style  and  task  type,  F (1,  59)  = 2.655,  p < .05.  Yet,  main  effects  for  group  composition  were 
not  statistically  significant,  as  seen  in  Table  2.  These  results  indicate  that  the  achievement  of  field- 
independent  and  field-dependent  students  was  dependent  on  the  type  of  problem-solving  task  on  which 
they  were  working  within  a group.  Field  dependent  students  with  analogy  task  tended  to  score  higher  on 
the  accuracy  posttest  than  tiiose  with  original  task  (3.85,  2.94,  respectively).  For  field  independent 
students,  however,  this  pattern  did  not  emerge:  There  were  only  slight  differences  between  students 
working  on  analogy  task  and  those  working  on  original  task  (4.31,  4.29,  respectively). 

Reasoning  and  Organization  of  Problem  Solving.  As  the  results  in  Table  2 show,  no  statistically 
significant  effects  were  found  for  cognitive  style,  task  type  or  group  composition,  or  for  the  interaction 
among  cognitive  style,  task  type  and  group  composition.  The  results  in  dicate  that  the  differences  between 
the  posttest  means  were  not  statistically  significant,  probably  due  to  the  relatively  large  standard 
deviations,  as  shown  in  Table  1.  For  reasoning  and  organization  of  problem  solving,  the  achievement 
scores  of  field- independent  and  field-dependent  students  were  not  significantly  different  fi*om  each  other, 
regardless  of  the  type  of  task  and  the  composition  of  the  group. 

This  study  examined  the  effects  of  cognitive  style  of  the  student,  the  type  of  task,  and  group 
composition  on  achievement  in  understanding,  accuracy,  reasoning  and  organization  of  problem  solving 
in  anchored  instruction  with  multimedia  courseware  in  a situated  learning  environment.  As  pointed  out 
earlier,  this  study  appears  to  be  one  the  first  studies  that  have  focused  on  the  factors  affecting  anchored 
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mstruction  at  the  elementary  school  level  in  Korea.  The  findings  of  the  present  study  corroborate  and 
extend  current  knowledge  of  anchored  mstruction.  The  results  of  the  analysis  of  covariance  indicate  that 
group  composition  as  well  as  students’  cognitive  style  and  task  type  significantly  exerted  differential 
effects  on  the  learning  outcomes.  In  general,  both  field -independent  and  field-dependent  students  in 
heterogeneous  groups  showed  higher  achievement  than  did  those  in  homogeneous  groups.  With  regard  to 
the  type  of  task,  students  working  on  analogy  tasks  tended  to  understand  the  problems  better  and  to  be 
more  accurate  in  problem  solving  than  did  those  working  on  original  tasks.  The  results  show  that  the 
effects  of  cognitive  style  of  students  were  dependent  on  the  composition  of  the  group  and  the  type  of 
problem-solving  task.  Thus,  the  results  of  the  present  study  allow  for  a closer  examination  of  the 
relationship  of  students’  individual  and  group  characteristics  in  the  problem  solving  process  in  a situated 
learning  environment,  and  corroborate  and  lend  further  support  to  the  previous  studies  on  anchored 
mstruction  (e.g.,  Bransford  et  al.,  1994;  Chinien  & Boutin,  1992;  Chou  & Lin,  1997;  Crews  et  al.,  1997; 
Leader  & Klein,  1994;  Lin  & Davidson,  1994;  Liu  & Reed,  1994;  Mevarech  et  al.,  1991;  Ruokamo,  2001; 
Shih,  1997;  Small  & Grabowski  1992;  Summerville,  1998;  Williams  et  al.,  2001). 

While  important  and  interesting  findings  have  been  revealed,  the  study  needs  to  be  replicated.  In 
addition,  there  are  several  suggestions  for  future  research.  First,  the  subjects  of  the  study  were  relatively 
homogeneous  in  terms  of  the  socioeconomic  background.  The  analysis  of  data  from  a nationwide 
probability  sample  across  various  student  populations  might  yield  more  generalizable  results.  Second,  the 
problem-solving  tasks  that  students  were  working  on  in  small  groups  included  original  and  analogy  tasks. 
Future  research  employing  other  types  of  problem-solving  tasks  may  be  worth  further  investigation. 


Table  1 . Means  and  standard  deviations  of  the  achievement  scores 


Understanding 

Accuracy 

Reasoning  and 
Organization 

Cognitive  Style 

Field  Independence 

M 

3.53 

4.30 

2.65 

SD 

1.45 

0.83 

1.80 

Field  Dependence 

M 

2.60 

3.40 

1.76 

SD 

1.57 

1.30 

1.65 

Task  Type 

Original  Task 

M 

2.37 

3.24 

1.74 

SD 

1.79 

1.57 

1.77 

Analogy  Task 

M 

3.30 

4.01 

2.26 

SD 

1.27 

0.76 

1.68 

Group  Composition 

Heterogeneous 

M 

3.25 

3.77 

2.33 

SD 

1.37 

1.24 

1.49 

Homogeneous 

M 

2.54 

3.57 

1.75 

SD 

1.70 

1.27 

1.90 

o 

ERIC 
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Table  2.  ANCOVA  results  for  problem  solving  by  cognitive  style,  task  type,  and  group  composition 


Source  of  Variation 

Sum  of 
Squares 

DF 

Mean 

Squares 

F 

P 

Understanding 

Cognitive  Style  (A) 

9.385 

1 

9.385 

4.306 

.042 

Task  Type  (B) 

14.137 

1 

14.137 

6.189 

.016 

Group  Composition  (C) 

11.614 

1 

11.614 

5.328 

.025 

AxB 

1.815 

1 

1.815 

.833 

.365 

A X C 

4.529 

1 

4.529 

4.078 

.049 

BxC 

.327 

1 

.327 

.150 

.700 

A X B X C 

1.263 

1 

1.263 

.579 

.450 

Residual 

128.595 

59 

2.180 

Accuracy 

Cognitive  Style  (A) 

8.357 

1 

8.357 

6.012 

.017 

Task  Type  (B) 

10.152 

1 

10.152 

7.303 

.009 

Group  Composition  (C) 

.733 

1 

.733 

.053 

.819 

AxB 

3.691 

1 

3.691 

2.655 

.049 

AxC 

.874 

1 

.874 

.063 

.803 

BxC 

.781 

1 

.781 

.056 

.813 

A X B X C 

.115 

1 

.115 

.579 

.775 

Residual 

82.014 

59 

1.390 

Reasoning  and  Organization 

Cognitive  Style  (A) 

9.997 

1 

9.997 

3.361 

.072 

Task  Type  (B) 

2.057 

1 

2.057 

.698 

.407 

Group  Composition  (C) 

9.516 

1 

9.516 

3.228 

.077 

AxB 

.256 

1 

.256 

.087 

.769 

AxC 

5.506 

1 

5.506 

1.868 

.177 

BxC 

.284 

1 

.284 

.096 

.757 

A X B X C 

1.413 

1 

1.413 

.479 

.491 

Residual 

173.896 

59 

2.947 
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Abstract:  User  data  and  contextual  information  about  anatomy  education  was  employed  to  guide 
efficient  and  effective  design  of  learning  applications.  The  research  focused  on  designing  educational 
software  to  provide  students  and  teachers  different  anatomy  learning  modules  corresponding  to  key 
topics  for  coursework  and  professional  training.  The  data  collection  emphasized  user  perspective, 
experience,  and  demands  in  anatomy  learning.  Collective  data  were  sorted  and  analyzed  using 
multidimensional  scaling  and  cluster  analysis.  The  learning  modules’  design  encompasses  both  unified 
core-components  required  by  all  users  and  specific  features  demanded  by  some  subgroups.  This  study 
begins  to  fill  an  important  void  by  describing  how  user  requirements  were  systematically  collected  and 
analyzed  and  then  transformed  into  guidelines  informing  the  iterative  design  of  multiple  learning 
modules.  A powerful  instrument  to  design  and  polish  the  content  and  user  interface  is  demonstrated  in 
this  study. 


Introduction 

A key  objective  of  the  Visible  Human  Project  at  the  University  of  Michigan  is  to  facilitate  the  use  of  medical 
images  from  the  Visible  Human  (VH)  Dataset  in  anatomy  learning.  It  is  our  observation  that  anatomy  teaching  differs 
from  one  group  to  another  in  heath  sciences  due  to  their  unique  requirements  in  the  class  and  professional  goals  in  the 
field.  We  are  interested  in  how  appropriate  and  useful  content  can  be  provided  in  the  specific  context  where  anatomy  is 
learned.  The  intent  is  to  use  user  data  of  contextual  information  about  anatomy  education  to  improve  iterative  design 
(Boyle,  1997),  and  thus  create  a learning  environment  that  is  efficient  and  effective. 

However,  no  efficient  methods  of  transforming  user  requirements  to  concrete  designs  have  been  suggested  in 
previous  research.  Hence,  this  study  attempts  to  begin  to  fill  an  important  void  by  describing  how  user  requirements, 
associated  with  users’  learning  experiences,  were  systematically  collected  and  analyzed  and  then  transformed  into 
guidelines  informing  the  iterative  design. 
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Methods 


One  or  multiple  focus  group  (Morgan,  1997)  sessions  were  conducted  with  each  group  of  target  users:  students 
and  faculty  in  professional  schools  of  nursing,  medicine,  dentistry,  kinesiology  and  surgery.  All  eleven  researchers 
then  met  as  a group  to  share  and  discuss  information  gathered  in  the  different  focus  group  sessions.  One  team  member 
created  a matrix  for  pooling  user  requirements  from  all  focus  groups.  After  each  researcher  rated  the  matrix 
independently,  a summary  containing  the  mean  rating  from  all  evaluators  for  each  cell  was  generated.  Examination  of 
these  rating  data  using  cluster  analysis  and  multidimensional  scaling  (MDS)  (SYSTAT,  1989)  was  performed  to  find 
target  groups  who  received  similar  scores  in  all  requirements.  Two  kinds  of  hierarchical  algorithms,  the  Complete 
Linkage  and  Ward’s  Method,  were  used  to  compare  the  similarity  of  all  requirements  across  different  users  and  further 
group  comparable  requirements  into  clusters. 

Results 

The  results  from  the  MDS  show  that  different  user  groups  are  distributed  into  the  four  quadrants  based  on  their 
requirements.  We  observed  that  users  who  share  similar  experience  and  training  objectives  in  their  health  care  education 
became  a natural  group.  The  two  optimized  dimensions  were  interpreted  by  corrparing  the  attributes  of  the  groups  on 
the  opposite  ends  along  each  dimension.  Thus,  the  vertical  dimension  was  conceptualized  as  anatomical  scope  or 
breadth  of  focus;  the  horizontal  dimension  was  interpreted  as  level  of  professional  development,  novice  versus  expert 
(right  versus  left).  These  results  indicate  at  least  four  clusters  of  users  that  the  design  should  accommodate. 

Results  from  the  hierarchical  algorithm  show  clusters  of  similar  requirements.  Each  cluster  of  requirements  was 
composed  of  numerous  related  functions  and  features  that  should  be  integrated  simultaneously  into  the  learning 
environment.  To  make  the  results  in  the  summary  matrix  more  informative  to  the  interface  design,  clusters  were 
categorized  by  level  of  demand  and  correlated  with  characteristics  of  the  users  who  requested  them. 

Conclusions 


In  interpreting  the  requirement  clusters  and  user  clusters  resulted  from  the  analysis,  we  suggested  at  least  two 
sets  of  functions  and  features  should  be  used  in  the  anatomical  learning  mo  dules.  One  set  of  core  components  used  as 
the  base  design  and  framework  would  be  common  across  groups.  Their  content  was  identified  in  the  focus  groups  as 
important  requirements  for  learning  anatomy  by  faculty  and  students  across  the  different  disciplines.  The  second  set  of 
functions  and  features  would  not  be  universal  across  all  groups  but  would  fulfill  the  needs  of  self-selected  users  groups 
that  parallel  their  specialties.  To  design  learning  modules  with  both  unified  core  components  and  user-specific 
applications,  the  program  should  be  flexible,  allowing  for  dynamic  insertion  of  different  learning  tools  for  different  users. 
One  possible  way  to  achieve  this  goal  is  to  provide  an  interface  that  users  can  employ  to  select  and  enable  suitable 
learning  materials. 

Interpretations  of  ideas  and  feedback  from  users  are  powerful  in  designing  and  polishing  the  VH  user  interface 
system.  In  this  study,  the  research  design  combines  both  quantitative  and  qualitative  research  methods  and  does  not 
rely  solely  on  one  or  the  other.  Through  these  methods,  we  were  able  to  (1)  identify  the  group  of  users  who  share  similar 
content,  (2)  suggest  the  general  design  framework  and  special  components  suitable  for  specific  users,  and  (3)  discern  the 
priority  of  implementing  each  requirement.  The  design  described  in  this  paper  will  continue  to  evolve  throughout  the 
software  development  process,  enabling  the  research  team  to  link  users’  needs  with  the  ability  to  construct  an 
appropriate  and  customized  instructional  design. 
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Abstract:  In  contrast  to  traditional  types  of  learning  and  teaching  processes  and  learning  medias,  such 
as  printed  material  for  Web  and  hypermedia  learning  resources,  Web  resources  and  Web-based 
activities  are  quite  often  unfortunately  more  or  less  separate  parts  of  the  planning  process  and 
curriculum  documentation  in  an  organization’s  traditional  organizational  level  of  educational.  This 
paper  introduces  a portal  system  based  on  curriculum  content.  An  attempt  is  made  to  solve  the  problem 
of  integrating  the  access  and  use  of  learning  through  both  traditional  and  modem  post-modern  types  of 
resources  into  an  online  interactive  coherent  system  of  curriculum  documentation  at  the  organizational- 
level.  At  our  university,  we  have  developed  an  Online  Curriculum  -based  Internet  Web  portal  as  a pilot 
for  one  course  in  order  to  integrate  all  learning  resources  through  Online  Curriculum  / Online  Syllabus, 
"" eCurriculum  / eSyllabuS\  content  into  one  location.  The  heart  of  the  system  is  located  on  the  public 
Internet  and  the  recent  beta  testing  system  is  currently  based  on  the  HTML  standard,  thus  avoiding  the 
commitment  of  the  organization  to  a single  platform  for  learning  resources. 


Introduction 

Recently,  there  has  been  a rapid  increase  in  the  use  and  the  will  to  use  and  develop  different  learning  types,  learning 
resources,  and  course  contents  and  even  entire  course  modules  that  are  partly  or  wholly  based  on  the  pedagogical  use  of 
computers  and  computer  networks.  A very  important  question  to  ask  is  how  our  curricula  and  the  formats  for 
curriculum  representation  should  be  developed  in  order  to  support  possible  new  post-modem  types  of  learning  and 
provide  easy  access  for  the  use  of  new  types  of  learning  resources. 

In  general,  the  pedagogical  planning  process  is  said  to  be  a process  that  generates  a more  or  less  coherent  stmctural 
model  of  resources,  systems,  and  activities  for  learning  and  education  and  the  use  of  different  systematic  tools  in  order 
to  achieve  the  goal. 

That  the  concept  of  a curriculum  and  its  importance  are  undoubtedly  a central  key  element  in  the  traditional  educational 
process  is  quite  clear.  It  can  be  said  that  the  entire  activity  of  learning  is  constmcted  around  a curriculum,  and  the 
curriculum  is  the  integrating  element  for  the  entire  educational  module  or  its  part  - a course  or  the  whole  programme. 
The  goals  of  a curriculum  also  form  the  basis  for  the  evaluation  and  assessment  of  the  outcome  of  learning.  In  general,  a 
curriculum  can  be  viewed  at  the  least  as  the  vision  of  the  goals  and  similarly  as  the  systematic  representation  of  the 
tools  (both  mental  and  material)  needed  in  order  to  achieve  that  vision. 


The  Traditional  Ways  of  Representing  Curricula 

Often,  a curriculum  and  its  different  representative  formats  are  also  used  as  the  requirements  for  a course.  It  may  be 
published  in  several  printed  and  electronic  formats  as  a book  or  as  an  Internet  page  and  given  to  students  or  to  school  or 
university  officials  and  teachers  as  a means  for  the  general  understanding  of  the  course,  its  goals,  contents,  and 
requirements.  It  specifies  the  tasks  or  achievements  during  a course,  the  literature  to  read  in  order  to  pass  the  course, 
plus  the  methods  of  assessment.  The  traditional  ways  of  generating  and  representing  curricula,  despite  the  publication 
platform  such  as  the  Internet,  can  be  said  to  have  typically  been  static.  The  metaphor  has  been  to  publish  and  not  to 
generate  an  interactive  integrated  system  of  tasks  and  resources  for  learning  by  collecting  them  all  into  one  location  as_a 
tool  for  easy  access  through  a single  resource.  In  this  new  situation,  rather  than  that  of  a static  presentation  of  content, 
the  metaphor  for  a curriculum  could  perhaps  be  more  as  an  interactive,  integrative,  iteratively  developed  living  tool  that 
is  applied  in  order  to  achieve  certain  goals  (See  e.g.,  Poldolskij  1997.) 


'sssir  COPT  mkiUMM 


Without  a more  specific  analysis  of  the  ideologies  enshrined  within  a curriculum  {such  as  e.g.,  PBL ) and  the  different 
view  of  education  they  impose,  as  well  as  the  need  for  representative  formats  of  a curriculum,  it  can  be  argued:  In  many 
cases,  the  present  ways  of  producing  a curriculum,  curriculum  epresentations,  and  the  implications  of  Instructional 
Design  based  upon  it,  are  quite  well  suited  to  traditional,  rather  static  institutionalised  and  non- virtual  types  of  learning, 
such  as  those  forms  of  learning  and  teaching  traditionally  organized  in  schools  and  universities. 

We  have  quite  good  tools,  both  mental  {e.g.,  well-defined  and  widely  used  concepts)  and  many  material  tools,  for  the 
traditional  type  of  educational  activity.  We  have,  for  example,  rather  satisfying  ways  finding,  distributing,  storing, 
using,  and  representing  traditional  written  material  as  books  or  handouts  or  other  types  of  traditional  media.  We  have 
generated  systems  such  as  libraries  and  library  databases,  publication  data  systems,  or  ISBN/ISSN  numbering  systems 
to  easily  handle  such  data;  there  are  such  things  as  traditional  course  readers,  and  ways  estimating  its  reliability  and 
value.  (Cp.  e.g.  Velman  1997.) 


The  New  Role  of  a Curriculum  as  an  Interactive  Online  Element  to  Integrate  all  Resources 

But  the  challenge  is  to  produce  curricula,  curriculum  representations,  and  Instructional  Design  for  the  modem  situation 
in  which  an  increasing  amount  of  party  or  wholly  location-free  teaching  and  learning  activity  is  present  and  in  which  the 
learning  resource  types  that  are  used  and  the  stmcture  of  learning  activity  i^he  ways  in  which  learning  is  organized)  are 
in  many  ways  very  different  from  the  traditional  format. 

It  can  be  said  that  in  the  present  situation,  where  there  are  many  new  types  of  learning  processes  and  resources  behhd 
all  processes,  the  role  and  essence  of  a curriculum  as  an  integrating  and  clarifying  key  element  in  the  planning  and 
realization  of  education  is  more  important  than  ever  before.  However,  how  and  by  using  which  kind  of  tools  could  this 
be  achieved?  Based  on  that  question  we  started  a pilot  research  project  at  our  university. 


How  and  in  which  ways  should  an  online  tool  metaphor  curriculum  to  be  generated  and 
represented? 

At  our  university,  the  University  of  Lapland,  the  curriculum  for  Technology  Education  for  the  years  2001  - 2003  has 
been  developed  as  a pilot  project  for  a new,  primarily  online  curriculum  / syllabus,  electronic  ''eCurriculum  / 
eSyllabus*\  that  is  located  behind  a Web  {World-Wide  Web)  portal  in  order  to  integrate  different  post-modern 
computerized  and  networked  means  and  resources  together  with  the  more  traditional  resources  for  learning  into  a 
coherent  learning  stmcture  <URL:hitp://www.uro va.fi/hoine/kik/opcttajankoulutus/tekninenJyo/>.  The  ultimate  aim  is 
to  develop  an  organizational -level  application,  a hierarchical  tree  of  Web  portals  through  which  different  courses  can 
take  advantage  of  the  generated  model. 

The  portal  is  primarily  used  in  research  projects  related  to  use  of  different  electronic  resources  and  mental  tools  such  as 
simulations  within  context  of  Technology  Education  (see  e.g.,  Lehtonen  2001),  thus  providing  the  integration  of  those 
resources  and  virtual  learning  activities  into  a coherent  part  of  student's  daily  learning  activity. 

At  the  beginning,  the  main  question  of  the  process  was:  What  kind  of  curriculum  and  its  representation  would  be 
needed  in  present  post -modem  age  to  integrate  different  parts  of  both  the  traditional  and  more  modem  electronic  and 
computerized  learning  ways,  resources,  and  materials  into  a meaningful  integrative  systemic  tool  for  our  students?  The 
other  main  question  was  how  we  could  integrate  as  well  as  possible  the  traditional  ways  of  curriculum  representation 
with  the  new  ones  in  order  to  create  as  coherent  a system  as  possible  for  the  use  for  our  students.  A special  interest  was 
also  the  possibility  to  link  easily  the  becoming  Finnish  Virtual  University  courses  <http://www.virtuaaliyliopisto.fi/^  as 
easily  as  possible  to  part  of  our  programmes  and  possibility  to  offer  parts  of  our  programmes  through  Finnish  Virtual 
University. 

The  realization  of  the  process  involved  many  phases  and  was  mainly  based  on  Cultural  Historical  Activity  Theory 
(“CHAT”)  (e.g.,  Poldolskij  1998;  Kapetelinin  & Nardi  1997;  Vygottsky  1978).  The  main  questions  were  to  plan  and 
understand  the  structure  of  learning  activity  well  enough  in  order  to  be  able  to  create  the  necessary  support  tools. 

The  questions  were  divided  into  three  main  areas: 

1.  The  users  of  the  tool:  Who  is  or  who  are  the  users? 

2.  The  situation  and  goals  the  tool  is  used:  For  what  purpose  does  the  user  use  the  tool  and  in  which  kind  of 
situation? 

3.  The  activity  for  which  the  tool  is  used  and  the  logical  structure  of  the  tool:  In  which  kind  of  (psycho)logical 
(activity)  stmcture  is  the  use  is  based  on? 
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Deflnidon  of  Tool  Using  Roles 


The  first  task  was  to  define  for  whom  the  tools,  the  tools  area,  and  the  portal  was  created.  Through  this  analysis,  we 
were  able  to  define  the  student  roles  for  the  targeted  users  of  the  tool. 

The  main  user  roles  were  found  to  be: 

■ the  basic  course  student, 

■ the  optional  courses  student, 

■ the  specializing  course  student, 

■ the  student  entering  training, 

■ the  continuing  education  student  {assumed  to  be  mainly  based  on  virtual  learning  activity). 

According  to  the  analysis,  these  were  the  most  important  user  roles  of  system  {the  additional  roles  considered  were,  for 
example,  resources  for  field  teacher  searches  s,  the  person  seeking  a specialist,  eta).  Through  this  analysis,  the  tools 
and  contents  were  generated  to  became  as  targeted  as  possible  in  order  to  fit  suitably  into  the  activity  structures  used  by 
the  learner  during  a specific  part  of  a course.  (See  Fig,  1 and  2,) 
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Figure  1 Main  page  of  the  beta  testing  portal, 
students  roles  and  shortcuts  to  commonly 
used  resources 


Figure  2 Second  level,  the  course  modules  and 
navigation  bar 


Definition  of  the  Situations  and  Goals  in  Which  the  Tool  Is  Used 

The  second  question  was  in  which  situations  students  need  the  portal  and  the  kinds  of  activities  students  performed  with 
the  tools. 

The  activity  structure  of  the  student  analysed  was  simplified,  in  that  the  student  wants  to  get  all  the  necessary  resources 
in  order  to  study  a certain  course  as  easily  as  possible.  What  does  he  or  she  want  to  find,  what  does  he  or  she  want  to  do, 
and  how  does  he  or  she  want  to  do  it? 

We  can  emphasize  that  obtaining  an  overall  representation  for  the  general  plan  of  achievement  and  gaining  an  overall 
view  of  what  to  do,  plus  obtaining  easy  access  to  the  tools  for  the  activity,  is  at  least  partly  formed  in  the  first  critical 
step  in  a well  defined  pedagogical  process  (cp.  E.g.  Poldolskij  1997).  In  other  words,  to  be  able  to  say  it  should  be  easy 
to  find  what  to  do  and  where  to  easily  find  the  needed  resources  in  order  to  concentrate  one's  activity  as  much  as 
possible  on  the  core  of  the  activity  itself  instead  of  on  some  other  activity. 

The  primary  answer  to  this  question  was  that  the  course  portal  should  be  integrative:  Integrate  all  the  modem  electronic 
and  online  resources  as  well  as  traditionally  used  resources  in  order  to  become  as  easily  accessible  as  possible. 
Secondly,  the  portal  iiould  be  iterative  Include  primarily  all  such  data  that  will  be  updated  during  the  curriculum  and 
which  cannot  or  is  not  sensible  to  represent  in  printed  format  in  a course  catalogue.  Such  objects  would  obviously  be 
online  linked  resources,  computer  programs,  literature,  and  the  more  specific  course  contents  that  are  updated  regularly. 
Thirdly,  the  portal  should  be  interactive  To  offer  as  much  as  possible  sensible  interactive  solutions  to  support  learners 
learning  activity. 


The  Logical  Structure  of  thePortal 
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The  third  question  was  upon  which  kinds  of  (psycho)logical  structure  use  could  be  based?  The  answer  was  sought  by 
analysing  the  present  activity  of  students  when  they  sought  for  similar  data  and  resources  from  the  more  traditional 
representation  of  the  curriculum  and  syllabuses.  Because  our  students  use  a printed  course  reader,  the  logical  structure 
of  the  portal  was  based  as  much  as  possible  on  the  structure  of  the  course  reader  and  the  terms  and  course  codes  found 
there.  The  navigation  model  is  based  on  the  following  steps:  the  student  role  (eg.  basic  studies)  — a list  of  basic  studies 
modules  - and  finally  an  interactive  representation  of  the  curriculum  where  all  the  needed  resources  are  linked  to  the 
required  places. 

In  addition,  an  attempt  was  made  to  achieve  visual  similarities  between  the  course  reader  and  the  Web  portal  (see  Fig.  3 
and  4).  The  idea  was  that  when  one  has  once  learned  to  use  one  type  of  representation  or  media,  one  could  use  that  same 
psychological  model  in  another  media  format.  Because  the  use  of  www-material  is  more  based  on  scanning  than 
continuous  reading  the  text  structure  contains  e.g.  clear  titles  and  other  helping  elements  (e.g.  Nielsen  2001).  Because 
according  the  observations  the  students  do  print  the  pages  quite  often,  the  format  is  also  optimized  for  that.  E.g.  some 
the  link  resources  contain  also  the  URL  section  in  printable  format. 


Figure  3 The  visual  outlook  of  the  portal  Figure  4 The  visual  outlook  of  the  course  reader 

page 

What  has  been  retained  as  a key  point  is  the  attempt  to  provide  a navigation  model  in  which  all  uimecessary  items  are 
done  away  with  in  order  to  reduce  the  cognitive  load  of  the  student  as  much  as  possible  (e.g.  Nielsen  1999;  Nielsen 
2000a,b;  Chandler  & Sweller  1991;  Sweller  & Chandler  1994;  Wilson  & Cole  1996).  However,  this  is  done  so  that  if 
the  student  is  willing  to  go  directly  to  a certain  offered  resources,  the  aim  is  to  provide  as  much  as  possible  support  for 
that  also  by  linking  some  of  the  most  used  tools  and  resources  directly  from  the  main  portal  page  and  in  some  cases 
from  a side  navigation  bars  etc. 


Online  Material  Interaction  with  Traditional  Resources 

Because  of  the  wide  use  of  more  traditional  resources  of  curriculum  representation,  as  course  readers  or  other  printed 
material,  the  attempt  was  made  to  keep  the  interaction  between  printed  and  electronic  media  as  easy  as  possible.  A 
course  reader  or  any  shared  paper  document  {or  e.g.  email  messages  to  students)  contains  only  one  link  to  a course 
portal  page. 

The  course  reader  only  contains  the  most  critical  parts  of  courses  and  the  more  specified  contents  and  linked  resources 
can  be  found  only  in  the  online  version.  The  organization  has  made  a commitment  to  store  the  main  page  address  as 
long  as  the  curriculum  is  valid.  The  most  important  part  of  the  system  is  the  idea  of  generating  the  curriculum  and  its 
primary  representation  into  an  online  format  and  organizing  all  the  resources  and  materials  through  the  curriculum 
representation 


Why  HTML  Content  on  the  Internet  Instead  of  a Special  “Virtual  Learning  Tool”? 

Why  did  we  choose  to  put  the  portal  onto  the  public  Internet  instead  of  using  a special  “ Virtual  Learning  Tool'll 
The  reason  for  using  the  public  Internet  instead  of  an  application  was  that  it  provides  the  possibility  to  avoid 
commitment  to  a single  program  or  environment.  It  also  provides  the  possibility  of  generating  a one-address  portal 
system  through  which  all  resources  can  be  accessed  easily  and  being  on  the  public  Internet,  there  is  no  need  to  log  in.  It 
has  the  benefit  of  integrating  very  different  resources  into  an  easy  accessible  part  of  a course.  It  holds  one  permanent 
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URL  address  on  the  organization’s  server,  thus  providing  easy  access  from  any  other  organization  without  the  fear 
changed  or  moved  links,  etc.  Another  benefit  is  that  students  only  need  to  know  one  location  in  order  to  access  all 
necessary  resources.  However,  a portal  is  only  a portal.  It  is  only  the  general  structure  for  integrating  elements  and 
tools.  If  a course  needs  a Web- authoring  tool  or  a ''Virtual  Learning  Environment”,  they  are  linked  from  the  outside. 
The  aim  is  to  keep  the  portal  as  small  as  possible.  The  portal  function  is  the  skeleton  for  the  overall  structure  of  studies; 
the  interactive  curriculum  representation  serves  the  integration  of  all  resources. 

All  the  pages  on  the  portal  are  linked  from  faculty’s  main  page  under  “Course  portal  pages”  and  the  structure  of  the 
portals  {and  sub  portals)  is  the  same  as  in  overall  structure  of  the  courses  in  the  course  reader. 


The  Present  Situation  in  the  Integration  of  Resources  Throughout  the  Portal 

Currently,  the  portal  serves  as  an  integrating  element  for  quite  different  resources.  Dependent  of  the  course  module,  the 
portal  integrates  most  of  the  items  listed  in  the  course  content: 

■ More  specified  course  objectives  and  contents, 

■ Changes  to  course  literature  or  articles  (e.g.  new  book  or  article  / required  pages  of  a book  or  a journal 

■ Links  to  required  electronic  resources  for  the  course  (e.g.  Internet  links  or  a link  to  “ Web  collaboration 
spaces”  such  as  BSCW®  or  "Web  learning  environment”',  many  different  alternatives  are  currently  in  use  at  our 
faculty)  etc., 

■ Registration  form  for  Web  learning  resources  (www-form), 

■ Readable  / printable  / downloadable  material  for  the  course  (includes  also  e.g.  steaming  media), 

■ Links  to  a additional  references  for  course  content  (such  as  external  links,  virtual  galleries,  or  references), 

■ Course  page  feedback  form  (a  html  text  entry  box  at  the  bottom  of  each  page) 

■ Course  feedback  form  button  (standardized  www-form  in  every  module), 

■ Buttons  to  university  library  and  city  libraries, 

■ ‘Search  from  library’  buttons  after  each  book  to  search  and  reserve  each  course  book  easily  from  the  library 
(sends  the  book  ISBN- number  and  starts  the  library  information  system  opening  it  in  a separate  browser 
window) 

■ Link  to  course  time  tables  etc. 

The  portal  page  contains  also  direct  links  for  several  most  used  resources;  additional  navigation  elements  e.g.  site  map 
and  contact  information  and  contact  form  for  faculty  personals,  etc. 


Conclusions 

The  Web  portal  based  on  curriculum  representation  has  now  been  used  for  year  as  a pilot  study.  Despite  its  visual 
simplicity,  according  the  beta  tests  and  more  precise  usability  testing,  students  have  been  quite  satisfied  with  it 
(e.g.,  see  the  beta  version  questionnaire 

<http://www.urova.fi/home/ktk/opettajankoulunis/tekninen_tyo/kouliiluspalaute/>) 

The  benefits  of  the  system  for  our  student  were  found  to  be: 

■ "Everything  can  be  found  at  one  location”, 

■ "Easy  to  find  needed  data  up  to  date” 

■ "Links:  E.g.  especially  library  links!” 

■ "Easier  to  return  essays  and  exams.  ” (e.g.  Linked  BSCW®  program)  and 

■ "The  possibility  to  collaborate  through  Web  tool^' 

Some  negative  aspects  of  the  system  have  been  mentioned: 

■ "Not  enough  computers  on  the  campus,  no  networked  computer  at  homd\ 

■ "The  linked  res  ources  like  BSCW  are  too  difficult  to  use”  and 

■ "More  pictures  ”. 

The  possibility  of  updating  iteratively  course  contents  has  also  been  a great  advantage  (which  makes  possible  the 
iterative  development  of  curriculum),  otherwise  it  would  have  been  very  difficult  to  update  the  course  contents  or  links 
to  several  Internet  resources  and  to  update  or  modify  course  literature  easily.  Probably  the  greatest  benefit  is  that  despite 
all  modifications  to  program  content,  a place  where  the  entire  coherent  content  can  be  found  still  exists  to  students. 

Research  in  the  future  shall  be  targeted  at  developing  the  students’  tools  and  the  overall  function  in  order  to  support 
learning  activity  as  much  as  possible.  The  plan  for  the  future  includes  adding  some  new  tools  for  surients,  such  as 
search-functions  {requires  also  better  content  metadata  format,  see  e.g.  Koper  2001),  offering  some  contents  also  in 
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mobile  Internet  {like  VML/XHTML)  format  for  mobile  phones  and  PDA's  and  making  it  possible  to  use  e.g.  redundant 
mobile  streaming  audio  format  instead  of  text  in  some  contents.  No  doubt,  the  future  version  will  be  generated  using 
dynamic  html  with  xml  content  base  or  a database  rather  than  static  html/JavaScript  solution.  What  has  been  learned  is 
the  fact  that  online  curriculum,  ''eCurriculum'\  and  this  kind  of  integrative  tool  have  proven  its  essence  from  the 
pedagogical  perspective.  Future  development  will  be  carried  out  in  order  to  develop  and  support  students  learning 
activity  with  as  many  easy  accessible  integrative,  interactive  and  iteratively  updated  tools  and  resources  as  possible. 
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Authentic  Performance  of  Complex  Problem-Solving  Tasks  with  an  EPSS 
By  Chet  Leighton  and  Cynthia  McCabe 


Teaching  People  to  Design 

Just-In-Time  Learning  (ITEC  805)  is  a semester-long  graduate  course  that  teaches  corporate  trainers  and 
instructional  designers  how  to  design  performance  improvement  interventions.  This  course  is  part  of  a Master's 
program  in  Instructional  Technology  at  San  Francisco  State  University.  The  course  has  been  offered  three  times  and 
has  been  completed  by  a total  of  38  people.  Learners  produce  two  major  deliverables  for  this  course—  a paper  of 
about  20  pages  describing  a performance  problem  and  the  design  of  a system  to  address  the  problem,  and  a paper  of 
about  5 pages  explaining  the  design  strategy. 

Based  on  final  grades  for  the  course,  participants  in  this  class  can  be  grouped  into  three  broad  categories.  The  A- 
level  performers  are  38%  of  the  total  students,  B-level  are  17%,  and  C-level  are  45%.  An  Electronic  Performance 
Support  System  (EPSS)  was  designed  and  implemented  to  help  the  B-level  and  C-level  people  improve  their 
performance.  (See  Leighton,  1996  for  a description  of  the  EPSS  model.)  The  JIT  Learning  EPSS  was  designed  and 
developed  by  Infomark  Software  Corporation  with  CGM  Communications  and  assistance  from  several  graduate 
students  in  the  SFSU  ITEC  Department  in  the  first  half  of  2001 . 

The  JIT  Learning  EPSS  was  implemented  with  15  learners  in  Fall  2001.  For  this  class  the  grade  distribution  was 
53%  in  the  A -level,  20%  in  the  B-level,  and  27%  in  the  C-level.  When  compared  with  prior  classes,  all  of  the  B- 
level  performers  improved  to  A -level  and  half  of  the  C-level  improved  to  B-level.  A formative  evaluation  of  the 
system  was  conducted  in  the  winter  of  2002.  The  results  suggest  that  the  remaining  C-level  performers  do  not  have  a 
sufficient  mental  model  of  the  design  task.  The  next  version  of  the  software  will  focus  on  helping  users  develop  and 
assess  their  mental  models  before  they  begin  the  design  task. 

Performance  Support  Systems  for  Complex  Problem-Solving 

Designing  a performance  improvement  intervention  is  a moderately -structured  complex  problem-solving  task 
(Foshay,  1997;  Jonassen,  1997).  Typically,  a performance  problem  can  be  defined  clearly,  it  is  clear  when  the 
performance  problem  is  solved,  and  there  are  multiple  ways  to  design  a correct  solution. 

There  are  numerous  examples  of  performance  support  systems  used  to  teach  complex  problem-solving  tasks.  There 
is  a subset  of  cases  where  the  user  performs  a task  primarily  involving  design,  analysis,  and/or  planning  and  the 
output  of  the  task  is  to  produce  a deliverable  document. 

■ Broadcast  News  allows  high  school  students  to  write  news  story  on  real  current  events  (Schank  & Kass,  1996). 
The  system  includes  a tool  for  writing  stories,  background  information  on  a story,  and  a library  of  information 
resources.  Video  advising  is  provided  based  on  the  concept  of  goal-based  scenarios. 

■ Project-Based  Learning  Support  System  (PBLSS)  allows  high  school  students  to  plan  and  conduct  a math  or 
scientific  inquiry  project  (Laffey,  Musser,  and  Tupper,  1998).  The  system  includes  guidelines  for  organizing  a 
project,  a tool  for  writing  a project  plan,  and  the  ability  to  publish  a project  plan  as  an  HTML  document.  PBLSS 
is  designed  to  support  project-based  learning,  where  students  work  in  groups  to  define  and  investigate  their  own 
project  ideas. 

■ Air  Campaign  Planning  Advisor  (ACPA)  allows  U.S.  Air  Force  officers  to  author  air  campaign  plans  to  attack 
military  targets  (Johnson,  Bimbaum,  Bareiss,  and  Hinrichs,  2000).  A seven-step  model  is  used  to  structure  the 
process.  Information  resources,  including  video-based  advising,  are  organized  around  questions  to  address  in 
producing  an  air  campaign  plan. 

■ Tactical  Readiness  Instruction,  Authoring,  and  Delivery  (TRIAD)  allows  U.S.  Navy  officers  to  create  tactical 
documents  that  are  used  to  share  information  on  tactical  problems  and  solutions  (Hannafin,  Hill,  and  McCarthy, 
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2001).  TRIAD  includes  an  authoring  tool  for  producing  multimedia  documents.  An  interview  process  is  used  to 
lead  the  user  through  writing  the  various  sections  of  a document. 

A common  theme  among  these  systems  is  support  for  two  major  activities.  First,  users  must  perform  the  complex 
problem-solving  task  that  involves  design,  analysis,  and/or  planning.  Second,  users  must  perform  the  task  of 
producing  a deliverable  document  that  contains  the  "output”  of  the  problem-solving  task. 

Underlying  the  design  of  these  systems  is  the  concept  of  situated  cognition  and  the  Cognitive  Apprenticeship 
framework  (Brown,  Collins,  and  Duguid,  1989;  Collins,  Brown,  and  Holum,  1991).  The  EPSS  contains  a tool  for 
producing  the  deliverable  and  uses  a series  of  guidelines  and  questions  to  scaffold  task  performance.  Information 
resources  are  organized  around  sub-tasks  in  order  to  model  performance.  Coaching  is  implemented  in  several  ways: 
the  process  is  structured  through  an  explicit  task  model,  the  guidelines  and  questions  provide  implicit  structuring  as 
performance  occurs,  and  video-based  advising  appears  in  the  context  of  authentic  task  performance. 

The  Just-In-Time  Learning  EPSS 

The  Just-In-Time  Learning  EPSS  uses  a six-step  process  to  design  a performance  improvement  intervention.  The 
first  five  steps  lead  the  learner  through  describing  the  problem  situation  and  explaining  the  design  of  the  proposed 
intervention.  The  last  step  reviews  the  design  strategy  of  the  intervention.  This  last  step  is  used  to  support 
articulation  and  reflection  on  the  current  design  effort,  and  to  explore  new  design  ideas  for  future  efforts. 

The  format  of  a typical  deliverable  produced  by  the  EPSS  is  like  the  format  of  this  article.  A paper  is  generated  as  a 
series  of  paragraphs  with  appropriate  subheadings  in  HTML.  The  formatting  of  the  document  is  simple,  as  the 
emphasis  is  on  the  content  not  on  presentation.  Of  course,  a user  can  import  the  HTML  into  a word  processing 
program  to  improve  the  formatting. 


Structuring  the  Performance  and  Le  arning  Process 

The  opening  web  page  for  the  EPSS  shows  the  status  of  work  on  each  step,  or  deliverable  document.  The  web  page 
for  one  of  the  deliverables  is  shown  in  Figure  1 . The  header  across  the  top  of  the  page  reiterates  all  the  steps  in  the 
process.  Access  to  performance  and  learning  resources  are  organized  around  producing  the  deliverable. 
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Figure  1 - Guidelines  and  Resources  for  Producing  a Deliverable  Document 


Resources  for  a deliverable  are  organized  into  three  categories,  as  noted  below.  The  user  clicks  a S symbol  to  see 
the  questions  under  a guideline  or  the  documents  for  a type  of  resource. 


Guidelines  and  specific  questions  to  answer  in  producing  the  deliverable  document 


BEST  COPT  AVAILABI.1 
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■ Performance  Resources-  Advice  on  key  design  decisions,  example  documents,  and  information  resources 
in  the  form  of  handout  documents  and  articles  to  read 

■ What  You  Need  to  Know-  These  learning  topics  represent  pre-requisite  knowledge  required  in  order  to 
understand  how  to  use  the  resources  and  produce  the  deliverable. 


The  items  under  "What  You  Need  to  Know"  correspond  to  the  learning  objectives  of  the  JIT  Learning  course.  A 
learning  topic  has  information  resources  like  article  readings,  text  discussions,  and  handout  documents.  It  also 
includes  exercises  (self-graded)  and  assignments  (graded  by  the  instructor)  for  practice.  These  learning  topics  allow 
learners  to  review  the  information  provided  in  class. 


Scaffolding  Task  Performance 


The  combination  of  guidelines  and  questions  structure  the  task  of  writing  each  deliverable  document.  By  clicking  on 
any  question  (or  the  question  number  following  "Continue  on  Questions  to  Answer  at:")  the  user  can  enter  an 
answer  for  a question  as  shown  in  Figure  2. 
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Figure  2 - Entering  the  Answer  for  a Question 


The  software  simplifies  the  task  of  writing  a document  by  prompting  a user  to  answer  a series  of  questions.  When 
the  user  clicks  "Save"  an  answer  is  saved  in  a relational  database.  Then  the  user  is  prompted  to  answer  the  next 
question.  A progress  meter  showing  how  many  questions  have  been  answered  for  the  deliverable  appears  on  both 
the  Deliverable  and  Question  pages. 

At  any  time  the  user  may  click  "View  My  Paper"  to  produce  the  output  document.  All  o f the  individual  answers  are 
compiled  into  a series  of  paragraphs.  Users  can  save  documents  in  HTML  format  and  import  them  into  a word 
processing  program  for  additional  formatting  or  revision. 

Another  scaffold  in  the  system  concerns  the  ability  to  prepare  an  outline  before  writing  a paper  (via  the  "Paper  or 
Outline"  button  in  Figure  1).  In  outline  mode,  the  user's  answers  are  shown  side-by-side  in  a two-column  table  with 
the  questions.  The  learner  can  submit  an  outline  version,  get  feedback  from  the  instructor,  and  then  revise  the  outline 
into  a narrative  paper. 

There  are  multiple  levels  to  fading  these  scaffolds.  The  learner  can  skip  producing  an  outline  and  move  directly  to 
writing  a paper.  The  learner  does  not  have  to  use  the  question  and  answer  page  as  the  writing  tool.  The  Deliverables 
page  could  serve  as  an  external  support  facility  and  the  user  could  work  entirely  in  a word  processing  program.  By 
design,  the  fading  of  the  scaffolds  is  controlled  by  the  user. 


o 

ERIC 


BEST  COPY  AVAIUBILIS 


1156 


Coaching  and  Advising 


Video-based  advising  takes  the  form  of  45  to  90  second  video  clips  with  advice  on  the  most  frequently  encountered 
design  problems  for  a deliverable.  This  advising  is  in  the  form  of  a story  about  a design  problem.  Most  of  the  videos 
feature  learners  who  have  completed  the  course  telling  their  own  painful  stories.  They  describe  a problem  situation 
and  then  discuss  their  solution.  There  are  also  clips  featuring  the  advice  of  the  instructor.  The  instructor  describes  a 
symptom,  the  underlying  problem,  and  advice  on  how  to  solve  that  problem.  For  example,  one  video  clip  explains 
the  trap  of  starting  to  design  the  solution  before  the  problem  is  fully  defined. 

Formative  Evaluation  Results 

A formative  evaluation  of  the  EPSS  was  conducted  in  the  winter  of  2002.  Data  was  collected  from  twelve 
participants  with  a 70 -point  questionnaire,  from  database  log  files  with  details  on  how  the  software  was  used,  and 
with  follow-up  interviews. 

The  average  amount  of  time  spent  writing  papers  with  the  system  was  927  minutes,  or  about  15!4  hours.  When  the 
participants  are  ranked  by  final  grade,  the  top  third  spent  an  average  of  975  minutes,  the  middle  third  spent  494 
minutes,  and  the  bottom  third  spent  1,313  minutes.  The  following  chart  shows  the  average  amount  of  time  spent  on 
each  of  the  four  majo  r papers  (Deliverables)  by  group.  The  differences  are  quite  obvious  between  the  three  groups. 
The  top  group  put  most  of  their  effort  into  the  problem  description,  while  the  bottom  group  spent  most  of  their  time 
on  describing  the  solution  (the  Components  and  Roles  and  Case  Scenario  papers).  The  middle  group  appears  to  be 
quite  efficient  in  using  their  time.  In  contrast,  given  the  low  quality  of  their  design  efforts,  the  bottom  group  spent  a 
lot  of  unproductive  time  in  devising  a solution. 

On  the  questionnaires,  the  bottom  group  scored  themselves  higher  than  the  top  group  in  using  the  feedback  from  the 
instructor  to  improve  their  project  designs.  However,  their  final  grades  were  lower  primarily  because  they  did  not 
correct  the  design  flaws  identified  in  feedback  on  their  papers.  The  Case  Scenario  papers  by  the  bottom  group 
revealed  a weak  understanding  of  the  task  being  performed  by  their  target  users.  Surprisingly,  the  bottom  group 
claimed  to  have  more  experience  doing  task  analysis  than  the  top  group.  Even  though  this  topic  is  covered  in  a 
prerequisite  course  on  the  ISD  model,  the  bottom  group  appears  to  have  a weak  understanding  of  task  analysis.  This 


Minutes  Per  Deliverable  By  Group 


Description  and  Roles  Scenario  Strategy 


□ Top  Third 

0 Middle  Third 

□ Bottom  Third 


limits  their  ability  to  effectively  use  the  instructor’s  feedback  in  defining  problems  and  designing  solutions. 
Implications  and  Future  Work 


The  approach  of  using  guidelines  and  questions  to  scaffold  the  task  of  writing  complex  papers  was  successful  in 
helping  improve  the  overall  performance  of  a class.  However,  there  is  still  a group  at  the  bottom  of  the  class  that 
needs  more  help  than  the  EPSS  provided.  They  spend  too  little  time  in  defining  the  problem  situation  and  too  much 
time  in  creating  a solution.  At  the  same  time,  they  do  not  make  effective  use  of  the  instructor's  feedback  in 
correcting  design  flaws. 


This  suggests  that  users  who  perform  poorly  have  weak  mental  models  of  the  design  task.  In  the  next  version  of  the 
JIT  Learning  EPSS  we  will  add  functions  to  help  users  develop  and  assess  their  own  mental  models  before  they 
begin  producing  a deliverable  paper.  This  will  include  more  emphasis  on  using  the  analysis  of  a target  user's  task  to 
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better  understand  the  problem  domain.  We  will  also  add  a feedback-and-response  function  so  that  users  explicitly 
acknowledge  the  instructor's  feedback  and  articulate  how  they  will  correct  design  flaws. 
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Addressing  Biodiversity  Probiems  Using  Web-Accessibie  Data:  A K-12, 

Hands-On  Exercise 

Jennifer  Leopold,  Anne  Maglia,  and  Jennifer  Dropkin 
Natural  History  Museum  and  Biodiversity  Research  Center,  University  of  Kansas, 

Lawrence,  KS  66045 


The  new  paradigm  in  science  education  calls  for  learning  in  which  1)  students  perform  real 
science  as  they  construct  meaning  and  acquire  understanding;  2)  students  develop  thinking 
processes  and  are  encouraged  to  seek  answers  that  enhance  their  knowledge  and  acquire  an 
understanding  of  the  physical  universe  in  which  they  live;  and  3)  students  are  presented  with 
problem-solving  activities  that  incorporate  authentic,  real-life  questions  and  generalization  to 
broader  ideas  and  applications  (North  Central  Regional  Educational  Laboratory; 
www.ncrel.org/sdrs/areas/issues/content/cntareas/science/sc500.htm).  Concurrently,  the  rapid 
development  in  technology  compels  us  to  include  computerization  in  the  classroom.  Web-based 
learning  can  provide  a solution  that  incorporates  all  of  these  objectives. 

One  of  the  more  common  areas  for  incorporating  Web-based  discovery  in  science  education 
is  investigating  biodiversity,  the  study  of  life  on  Earth.  This  knowledge  is  critical  to  science  and 
society  for  managing  natural  resources,  sustaining  human  health,  maintaining  economic  stability, 
and  improving  the  quality  of  human  life.  And  the  urgency  for  this  knowledge  increases  daily  as 
the  destruction  of  natural  systems  accelerates  the  decline  of  biological  diversity. 

Several  Web-based  projects  are  dedicated  to  biodiversity  education  at  the  K-12  level.  In 
fact,  most  museums  have  attempted  to  develop  some  project  designed  to  encourage  K-12 
biodiversity  education,  including  developing  “virtual”  museum  exhibits,  interactive  exercises,  and 
“Ask  a Scientist”  programs.  Unfortunately,  despite  the  excellent  education  programs  that 
museums  have  developed,  they  have  not  taken  advantage  of  their  biggest  strength— the  research 
data  that  they  collect  and  curate.  Data  associated  with  specimens  document  the  biotic, 
geological,  oceanic,  atmospheric,  and  geospatial  information  connected  to  the  world's  estimated 
1.8  million  species.  These  data  should  be  at  the  core  of  any  educational  endeavor  that  includes 
K-12  science  education.  With  the  development  of  robust  database  management  systems  and 
fast  and  accurate  Web  accessibility,  we  now  have  the  ability  to  make  collections  data  available  for 
public  access.  This  allows  us  to  develop  educational  exercises  that  can  bring  the  museum,  and 
the  data  collected  in  the  fight  to  preserve  biodiversity,  directly  into  the  classroom. 

Herein  we  describe  an  educational  exercise  for  K-12  grade  students  that  uses  museum 
collections  data  to  examine  biodiversity  problems  In  the  Amazon  rain  forest.  The  goals  of  this 
exercise  are:  1)  to  make  students  aware  of  current  biodiversity  issues,  2)  to  educate  them  about 
research  data  housed  in  natural  history  museums,  3)  to  motivate  student  interest  in  science,  4)  to 
increase  student  proficiency  with  computers,  databases,  and  Web  browsers,  and  5)  to  serve  as  a 
prototype  for  developing  other  exercises  that  use  Web-accessible  data  to  address  real  problems. 

Using  Soec/fv  Collections  Data  to  Examine  Amazonian  Biodiversity 

in  this  exercise,  students  use  a straightforward  Web  interface  to  access  data  collected  by 
natural  history  museums  and  apply  them  to  understanding  conservation  issues  in  the  Amazon 
rain  forest.  Students  are  challenged  with  making  decisions  based  on  data  they  have  collected, 
and  are  asked  to  re-evaluate  and  summarize  decisions  once  additional  data  are  added. 

Background. — The  Amazon  jungle  is  the  world’s  largest  tropical  rain  forest,  with  2.5  million 
square  miles.  Presently,  there  are  at  least  500  species  of  mammals,  475  reptile  species,  and  at 
least  a third  of  the  world’s  brds.  This  area  is  vital  to  the  economy  of  many  countries  in  South 
America,  which  send  much  of  the  natural  resources  (e.g.,  timber,  petroleum,  and  gold)  to  the 
United  States.  Unfortunately,  the  practice  of  harvesting  natural  resources  has  caused  severe 
declines  in  biodiversity  in  the  region.  Because  of  its  importance  to  so  many  economies,  it  is 
unlikely  that  industrial  practices  will  change,  and  therefore,  conservationists  are  trying  to 
compromise  with  industrialists  to  preserve  as  much  of  the  area’s  biodiversity  as  possible.  One 
popular  suggested  compromise  is  to  sacrifice  portions  of  the  Amazon  (such  as  the  eastern  or 
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western  half)  for  logging  and  mining,  and  leave  other  areas  untouched  to  maintain  their  pristine 
biodiversity.  By  using  the  information  stored  in  museum  collections,  conservationists  can  make 
informed  decisions  about  which  areas  are  more  prone  to  extinction  or  survival. 

Traditionally,  museum  collections  data  have  been  difficult  to  access  or  laborious  to  examine. 
However,  recent  technological  advancements  have  made  Web-accessibility  of  museum 
collections  data  a reality.  One  such  development  is  the  collections  management  software  tool. 
Specify  (www.usobi.org/specify/).  Specify  is  a robust  database  application  that  is  specifically 
designed  for  the  management  of  specimen  information.  This  software  provides  a single  user- 
interface  to  all  kinds  of  collections  data  (e.g.,  plants,  birds,  bacteria,  etc.)  and  a single  URL  for 
Web-accessibility.  Therefore,  users  do  not  need  to  learn  different  interfaces  to  examine 
heterogeneous  databases  housed  in  different  locations. 

Objectives. — Students  access  museum  collections  data  using  the  Specify  Web-query 
interface,  then  use  the  data  collected  to  generate  maps  and  charts  of  the  number  and  kinds  of 
species  (i.e.,  species  richness)  in  various  regions  within  the  Amazon  basin.  Students  apply  the 
data  they  uncover  to  determine  which  areas  in  the  Amazon  are  most  critical  to  preserve. 

Teaching  Strategy — The  presentation  begins  with  an  introduction  on  the  topic  of  biodiversity. 
Students  should  understand  what  extinction  is  and  how  it  can  be  both  a natural  and  man-made 
phenomenon,  and  what  a scientific  natural  history  museum  is  and  what  kinds  of  data  museums 
collect  to  study  biodiversity. 

The  Amazon  rain  forest  is  then  discussed  as  an  example  of  an  area  with  high  levels  of 
biodiversity,  but  that  is  also  in  decline.  This  might  include  a short  history  of  the  continent  and 
discussions  of  how  the  Amazon  rain  forest  became  so  biologically  rich.  Students  then  use  a 
search  engine  to  find  information  on  the  Internet  about  the  Amazon  rain  forest  and  the  different 
kinds  of  animals  and  plants  found  in  the  area. 

Once  students  are  familiar  with  using  the  Internet  to  collect  information,  and  are  familiar  with 
the  Amazon  rain  forest,  they  then  access  natural  history  collections  representing  different 
taxonomic  groups  using  Specify.  Students  are  shown  how  to  run  queries  of  species  by  locality  to 
determine  species  richness  (number  of  species)  and  endemism  (areas  of  high  concentration). 
Students  break  into  groups,  with  each  group  searching  for  different  kinds  of  organisms.  They  are 
then  given  outline  maps  onto  which  each  group  plots  the  data  they  discover.  Once  all  data  are 
collected,  students  combine  the  data  they  find  onto  a single  map  that  describes  species 
distributions.  Depending  on  the  level  of  computer  proficiency,  students  may  also  develop  charts 
and  graphs  of  the  data  using  a spreadsheet  application.  Working  together,  students  can  evaluate 
the  data  they  collected  to  determine  which  areas  should  be  conserved,  and  which  areas  may  be 
sacrificed.  Criteria  to  choose  could  include  number  of  species  and  presence  of  rare  species. 

Once  students  have  determined  which  areas  to  preserve,  the  teacher  may  want  to  access 
global  climate  change  databases  (e.g.,  edcwww.cr.usgs.gov/)  and  generate  climate  models  that 
can  be  used  to  predict  how  climate  changes  may  affect  the  Amazon  rain  forest  in  the  future.  The 
students  may  then  elect  to  re-evaluate  their  choices  based  on  possible  future  climate  changes. 
Students  conclude  the  exercise  by  writing  a description  of  their  findings  in  a lab  report,  or 
summarizing  their  findings  on  a poster  (for  possible  presentation  in  a science  fair). 

Extensions  of  the  Project. — In  the  near  future  we  intend  to  use  this  same  exercise  framework 
with  another,  more  sophisticated,  data  access  tool,  WebFormulate  (J.  Leopold,  M.  Heimovics, 
and  T.  Palmer,  ""WebFormulate.  A Web-Based  Visual  Continual  Query  System”,  [in  press]. 
Proceedings  of  ACM  11*^  International  WWW  Conference).  This  software  will  not  only  allow  the 
students  to  query  natural  history  specimen  collection  databases,  but  will  also  allow  them  to 
specify  how  often  the  database  should  be  queried  and  the  conditions  by  which  they  want  to  be 
notified  of  the  results  (e.g.,  whenever  a new  species  of  frog  is  added  to  the  database).  In  addition, 
any  user-specified  WebFormulate  computations  or  visualizations  based  on  those  query  results 
will  automatically  be  updated  every  time  the  notification  conditions  are  satisfied.  Thus,  the 
students  can  monitor  their  analyses  on  an  hourly,  daily,  weekly,  or  monthly  basis. 

Summary 

Herein  we  have  briefly  described  an  exercise  that  addresses  our  aforementioned  educational 
goals  at  the  K-12  level.  We  hope  that  this  endeavor  not  only  accomplishes  these  goals,  but  also 
helps  to  foster  the  next  generation  of  biodiversity  researchers  and  bioinformaticians. 


1160 


Teaching  with  Technology  through  Collaborative  Interdisciplinary  Instruction 
Mr.  Philip  Lewis,  M.S.  and  Mr.  John  Hildreth,  M.M. 


OVERVIEW: 

Incorporating  technology  into  the  classroom  is  paramount  at  Radford  University.  Both 
administrators  and  faculty  have  seen  pronounced  increased  in  effective  teaching  through 
the  use  of  multimedia  technology  when  presenting  difficult  material.  However,  not  all 
faculty  can  be  expected  to  learn  complex  applications  in  a timely  fashion  that  would 
enable  effective  instruction  to  any  students.  Therefore,  a unique  form  of  support  has 
developed  through  Radford  University  library’s  . In  a number  of  situations  and  courses, 
technologists  and  faculty  members  are  teaching  ‘high  tech’  classes  side  by  side. 
Increasingly,  this  support  is  giving  better  opportunities  for  students  to  learn  and  faculty  to 
become  more  familiar  with  the  technologies  that  dominate  nearly  every  field. 

This  paper  presents  the  findings  of  a class  that  became  a case  study  in  interdisciplinary 
academics.  The  course  incorporated  traditional  collegiate  education  themes  with 
technology  through  a collaborative,  co-instruction  effort  between  a university  music 
professor  and  multimedia  technologist.  The  course,  entitled  Introduction  to  Film  Music 
(MUSC  373,  Computer  Music  Composition),  incorporated  an  historical  survey  of  film 
music  media  including  literature,  recordings  and  film  as  well  as  compositional 
assignments.  In  these  assignments,  students  were  required  to  learn  the  parameters  of  a 
certain  era’s  film  music  (including  the  limitations)  and  then  create  their  own  soundtracks. 


Role  of  Faculty  Professor 

Role  of  Technologist 

Define  composition  dimensions  of 
coursework 

Provide  comparative  analysis  of  various 
industry  technologies,  past  and  present 

Prepare  lecture  materials 

Prepare  lecture  materials 

Program  examples  of  media  to  present 
during  class 

Program  examples  of  media  to  present 
during  class 

Present  and  discuss  traditional  composition 
practices  of  genre 

Discuss  technologies  utilized  during 
composition  and  production  process 

TIMELINE 


Radio  Silent  Film  Television  Film 

Technologies  and  composition  practices  of  each  medium  and  time  period  are  presented 
chronologically  and  are  studied,  discussed  and  experienced  through  assignments.  From 
Radio  to  modem  film,  each  era  (compositionally  and  technologically)  is  delivered. 


ASSIGNMENTS:  The  process 


Film  excerpts  were  digitized  using  a digital  video  editing  workstation.  The  audio  element 
was  removed,  and  the  film  was  then  compressed  into  a digital  format  (QuickTime)  that 
the  students  could  import  into  a software  application  used  to  sequentially  develop  their 
score  to  the  film.  In  this  marmer,  the  students  were  able  to  score  the  musical  and  sound 
excerpts  while  watching  them,  just  as  composers  working  in  the  industry  do. 

Assignments  were  handed  in  on  CD  or  cassette  media  and  evaluated  in  class  by  the 
professors  and  peers. 

STATEMENT  OF  PROPOSAL 

One  way  of  achieving  technology  in  the  classroom  is  to  have  collaborative  instruction.  In 
this  case  study,  the  course  was  co-taught  by  two  instructors,  each  an  expert  in  their  fields; 
one  in  music  and  the  other  in  media  technologies.  The  benefits  of  collaborative 
instruction  are  manifold: 

1 . Instructors  do  not  bear  the  burden  of  ‘learning  it  all’  before  they  incorporate 
technology  into  the  classroom. 

2.  Instructors  are  involved  in  faculty  development  as  they  simultaneously  impart  and 
absorb  expertise  in  their  relationship  with  the  co-teacher. 

3.  The  student  benefits  from  the  interdisciplinary  delivery  and  application  of  course 
materials,  part  of  which  include  the  dual  perspectives  and  expertise  of  the  faculty. 
Moreover,  students  enjoy  a fresh  approach  and  are  better  informed  as  a result. 

The  course  became  an  intuitive  utilization  of  resources,  and  the  student  reaction  made  it 
an  exceptional  experience  for  all  persons  involved. 

FILM 

A 25-minute  documentary  film  was  produced  from  in-class  footage  of  interviews  and 
discussions,  as  well  as  instructor  interviews,  commentary  and  samples  of  student  work. 
This  film  reveals  in  detail  the  efforts  made  by  the  instructors,  outcomes  of  the 
assignments  by  the  students  and  a candid  discussion  on  issues  of  teaching  collaboratively. 
The  film  will  be  screened  following  the  presentation,  and  participants  will  be  given  a 
copy  of  the  syllabus  and  class  timeline.  A discussion  will  follow. 
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An  examination  of  the  functional  capabilities  of  different  user  environments  in 
networked  classes  and  the  development  of  a school-based  language  lesson  and 
self-study  system  based  on  this  examination 
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Abstract  With  the  advent  of  high-speed  networks,  feasibility  studies  and  development  of  network -based  systems  to 
support  class  work  has  been  pursued  with  vigor,  resulting  in  new  education  environments  such  as  remote  lessons  and 
cooperative  learning.  These  studies  have  largely  been  carried  out  by  examining  an  existing  environment  and 
constructing  one  based  on  the  desired  functions.  However,  so-called  network-based  lesson  environments  contain  many 
different  factors,  such  as  the  different  computational  environments  used  by  teachers  and  students,  network 
configurations  between  computers,  and  a variety  of  subjects  that  serve  as  targets  for  study.  Moreover,  these  studies 
largely  focus  on  the  actual  classes,  whereas  in  the  real  world  of  school  education,  revision  of  material  after  each  class  is 
vitally  important.  This  aspect  has  received  little  attention  thus  far.  This  paper  examines  the  function  of  different 
environments  during  the  development  of  a support  system  for  network-based  classes.  By  examining  the  function  of 
different  environments,  an  actual  system  supporting  language  lessons  and  self-study  has  been  developed,  with 
appropriate  self-study  environments.  This  system  is  divided  into  a support  system  for  network-based  lessons,  and  a 
support  system  for  self-study.  The  lesson  support  system  mainly  offers  lesson  support  for  teachers,  while  the  self-study 
support  system  is  geared  towards  students,  providing  aids  such  as  the  reutilization  of  data  taken  from  lessons. 


Figure  1:  Model  for  language  learning 


1 . Introduction 

In  recent  years,  as  downsizing  has  lead  to  notable  advances  in 
computational  capabilities,  multimedia  environments  based  on 
computers  have  become  commonplace.  The  lifelike  expression  of 
sound,  video  and  animation  media,  unified  on  a computational 
platform,  presents  itself  as  an  opportunity  for  effective  use  in 
education.  Especially  in  the  field  of  language  education,  this  has 
enabled  the  creation  of  systems  that  deliver  lifelike  effects  not 
previously  available  to  the  language  learner.  Numerous 
applications  of  multimedia  in  language  education  have  been 
trailed  and  developed,  the  effectiveness  of  which  is  well  attested 
[1,2]. 

Also,  with  the  realization  of  high-speed  networks,  lesson 
support  systems  using  network  capabilities  have  been  brought 
under  investigation  and  developed  on  a large  scale.  New 
education  environments,  such  as  remote  lessons  and  cooperative 
learning,  have  been  realized  by  use  of  the  internet  and  intranet 
[3-6].  Research  so  far  has  mainly  aspired  to  construct  support 
systems  for  remote  lessons,  using  exclusive  connections  or  simply 
unprotected  data  exchange  on  the  internet.  They  aim  at  re-creating 
a classroom  environment  as  close  to  real  life  as  possible,  only 
network-based.  These  studies  basically  begin  with  an  existing 
environment,  proceed  through  to  an  examination  of  needed 
functions,  and  culminate  in  the  realization  of  network-based 
systems  for  support  of  lessons.  However,  lesson  support  systems 
include  many  factors  that  need  to  be  examined.  For  example,  do 
the  computers  used  by  the  teachers  and  students  adequately 
handle  sound  and  animation?  Can  the  band  area  necessary  for  the 
sending  of  animation  be  provided  for?  Special  needs  may  arise, 
depending  on  the  nature  of  the  target  subject  matter.  For  example, 
since  the  target  subject  matter  in  the  present  study  is  language 
lessons,  one  would  need  to  consider  the  possibility  of 
multi-lingual  input  and  display.  Hence,  a thorough  classification 
of  the  environment  to  be  constructed  is  necessary,  even  before 
examining  the  various  functions  needed  in  a network-based 
lesson  support  system.  It  would  be  impossible  to  provide  a 
trustworthy  support  system  without  following  such  a procedure. 

In  addition,  to  achieve  overall  learning  objectives  for  students  in 

an  educational  setting,  a three-stage  model  is  proposed,  basically  consisting  of  preview,  the  actual  lesson  and  revision.  This  is 
shown  in  figure  1.  In  this  model,  the  teacher  uses  established  principles  to  impart  knowledge  in  stage  2.  This  stage 
corresponds  to  the  procedures  of  current  classes.  There  is  a flood  of  information  to  convey  to  the  students,  but  there  is  little 
feedback  from  the  students  to  the  teacher,  because  of  the  large  size  of  each  class.  The  third  stage  corresponds  to  self-study. 
Students,  with  the  aid  of  notes  they  have  taken  in  class  and  their  own  comprehension  of  the  material,  develop  understanding  of 
the  imparted  knowledge.  A truly  effective  learning  support  system  requires  not  only  support  for  the  lesson  itself,  but  also  a 
support  system  that  includes  self-study.  To  my  knowledge,  there  are  no  examples  of  considered  application  of  these  matters. 


Net^^Clk  cnvironincnt 


er|c 


1163 


jBEST  copy  available 


The  authors  of  the  present  study  aim  to  pursue  the  research  goal  of  creating  a comprehensive  ‘‘integrated  support  system  for 
the  support  of  educational  environments.”  In  other  words,  a learning  environment  that  covers  the  spectrum  of  “lesson  to 
self-study,  classroom  to  home.”  This  paper  presents  a system  supporting  language  lessons  and  self-study,  developed  by  the 
authors.  Chapter  2 below  examines  the  functional  capabilities  of  different  user  environments.  Chapter  3 examines  the 
actualization  of  environments  and  system  architecture. 


(Teacher  to  student) 


None 


1 

2 

3 

0 

0 

0 

s 

None 

Sound 

Video 

(Student  to  Teacher) 
Table  1 : Classification  according  to  the  network  band  and  the 
computational  environment  used 


2.  Examination  of  the  functional  capabilities  of  different  user  environments 

2.1  Basic  factors 

Network-based  lessons  basically  consist  of  the 
following  three  factors: 

1 . Network  environment:  This  refers  to  how  much 
network  band  is  available  between  teachers  and 
students.  Satellite  communication,  exclusive 
connections  and  the  internet  may  be  used.  The 
issue  here  is  whether  sound  and  animation  can  be 
sent  from  teacher  to  student  and  vice  versa. 

2.  The  computational  environment  being  used:  One 

needs  to  consider  whether  the  computers  used  by  the  teachers  and  students  allow  the  transmission  of  sound  and  animation. 

3.  The  distance  between  the  teacher  and  students:  This  refers  not  to  physical  distance,  but  to  whether  the  teacher  and  students 
are  separated.  Students  may  be  gathered  in  one  spot,  or  be  scattered  around  different  places. 

In  addition,  there  are  differences  among  the  requirements  for  different  subject  matter.  For  example,  in  language  education, 
the  subject  matter  currently  under  consideration,  there  may  be  a need  for  multilingual  input  and  output  according  to  content  of 
each  lesson.  As  figure  2 shows,  the  user  environment  consists  of  the  1)  network  environment,  2)  the  computational 
environment  used,  3)  distance  between  the  teacher  and  students,  and  4)  the  subject  matter.  Classification  is  done  on  this  basis, 
but  for  the  sake  of  simplicity,  where  the  network  environment  has  sufficient  network  band  to  enable  the  sending  of  sound  and 
video  from  students  to  teachers,  the  computational  environment  used  is  assumed  to  be  one  where  each  student  has  his/her  own 
computer,  and  that  the  computers  are  able  to  take  in  sound  and  video. 


2.2  Classification  according  to  the  network  band  and  the  computational  environment  used 
As  shown  in  Table  1 , the  horizontal  axis  indicates  the  network  band  from  the  student  to  the  teacher.  The  vertical  axis  shows 
the  network  band  from  the  teacher  to  the  student.  The  analysis  has  been  simplified.  It  was  found  that  costs  would  be  relatively 
lower  to  provide  an  adequate  environment  for  teachers  only.  Hence  the  categories  only  show  “none,”  “video”  and  “sound.” 
The  results  of  this  classification  is  summarized  as  follows: 

0.  Neither  the  teacher  nor  the  student  has  access  to  a computer.  The  necessary  network  is  not  available.  Note  that  even  if  the 
learner  has  a computer  that  sends  out  sound  and  video,  it  would  be  useless  if  the  teacher  had  no  computer.  Hence,  in  both 
cases  the  category  0 is  assigned. 

1 . The  teacher  can  send  sound  and  video  information  to  the  student,  but  the  reverse  is  not  available.  This  is  typical  of  one-way 
transmission  situations  such  as  lessons  in  the  classroom  and  satellite  broadcasting.  In  this  environment,  the  student  does  not 
necessarily  need  a computer,  but  equipment  for  receiving  the  teacher’s  information,  such  as  projection  screens  and 
loudspeakers  might  become  essential. 

2.  The  teacher  can  send  both  sound  and  video  to  the  student,  but  the  student  can  only  send  sound  to  the  teacher. 

3.  Both  the  teacher  and  the  student  can  send  and  receive  sound  and  video.  In  this  case,  cameras  need  to  be  installed  on  all 
computers. 

The  above  classification  is  based  on  the  network  environment  and  the  computational  environment  used.  Cost  considerations 
yield  the  hierarchy  of  0<1  <2<3. 


2.3  Classification  according  to  the  distance  between  students  and  the  teacher 
Using  the  results  from  classification  according  to  network  environment  and  computational  environment  used,  we  may  now 
obtain  a further  classification  according  to  the  distance  between  students  and  the  teacher.  “Distance  between  students  and  the 
teacher”  refers  not  to  the  physical  distance,  but  to  whether  the  parties  are  gathered  in  the  same  place.  As  table  2 indicates,  the 
horizontal  axis  shows  1 ) teacher  and  students  in  the  same  place,  2)  students  in  the  same  place,  and  3)  students  scattered.  The 
vertical  axis  shows  the  results  of  classification  according  to  network  environment  and  computational  environment  used.  In  the 
current  analysis,  “teacher  and  students  in  the  same  place”  approximates  the  traditional  classroom  situation.  “Students  in  the 
same  place”  represents  the  situation  where  students  are  gathered  in  a classroom  environment,  while  the  teacher  is  elsewhere, 
possibly  in  a studio.  The  “students  scattered” 


scenario  refers  to  the  situation  where  students 


(Classification  according  to  network  band) 


are  in  their  own  homes.  The  results  of  the  above 
classification  are  as  follows: 

2’ 

5’ 

T 

0’  Both  the  students  and  the  teacher  are  in  the 

2 

2’ 

4’ 

6’ 

classroom,  representing  the  traditional 

1 

r 

3’ 

3’ 

classroom  situation  with  blackboards  and 

0 

0’ 

X 

X 

textbooks. 

r This  is  basically  the  same  as  O’,  only  the 
teacher  may  use  a personal  computer  and 
projector  for  presentation  of  material. 

2’  Both  the  students  and  the  teacher  have  access 
to  computers.  This  enables  the  transmission 


Teacher  and 
students  in 
the  same 


Students  in 
the  same 
place 


Students 

scattered 


(Distance 
between  students 
and  the  teacher) 


Table  2:  Classification  according  to  1)  the  distance  between  students 
and  the  teacher 
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3’  Students  and  teachers  are  not  in  the  same  place.  However  students  are  unable  to  send  information  to  the  teacher.  This  results 
in  a lesson  type  with  only  one-way  communication,  such  as  one-way  transmission  of  material  via  satellite  broadcasting  or 
education  broadcasts  on  television. 

4’  Students  and  teachers  are  not  in  the  same  place,  but  there  is  voice  communication  between  the  parties.  Gestures  and  facial 
expressions  cannot  be  sent. 

5’  Teachers  can  send  both  sound  and  video  to  the  students,  but  students  can  only  send  sound  signals  to  the  teacher.  Hence  the 
teacher  is  unable  to  ascertain  the  students’  facial  expressions.  However,  if  a moving  image  camera  was  installed  in  the 
classroom,  the  teacher  would  be  able  to  observe  the  classroom  environment. 

6’  Students  are  scattered  over  many  locations,  but  two-way  voice  communication  is  available.  The  teacher  is  unable  to  monitor 
the  actual  learning  environment. 

7’  Students  are  scattered  over  many  locations,  but  two-way  voice  and  video  communication  is  available. 

In  the  aforementioned  table,  the  combinations  of  “0  - teacher  and  students  in  the  same  place”  and  “0  - students  scattered” 
indicate  situations  where  there  can  be  no  communication  between  the  teacher  and  students.  Hence  an  X has  been  assigned. 

2.4  An  examination  of  basic  functional  capabilities 

The  functional  capabilities  available  in  various  environments  are  summarized  in  Table  3.  An  explanation  of  these 


1 . Sound  and  video  transmission  from  the  teacher  to  students:  Sound  and  video  data  taken  by  microphones  and  video  cameras 
are  sent  to  students.  This  function  is  essential  when  the  teacher  and  students  are  not  in  the  same  place. 

2.  Transmission  of  handwritten  information  from  the  teacher  to  students:  Information  written  on  the  blackboard  during  class  is 
sent  to  the  students.  This  function  becomes  very  important  when  further  explanation  is  needed,  apart  from  the  handout 
material  prepared  beforehand. 

3.  Sound  transmission  from  one  student  to  all  other  people:  This  function  operates  when  students  are  scattered  in  different 
locations.  When  the  teacher  is  asked  a question  by  a student,  this  function  enables  all  other  students  to  hear  and  share  in  the 
same  information. 

4.  Transmission  of  video  from  students  to  the  teacher:  By  sending  video  from  computers  equipped  with  cameras,  the  teacher  is 
able  to  monitor  the  students’  learning  environment. 

5.  Transmission  of  text  information  from  the  teacher  to  students:  This  enables  the  teacher  to  send  text  information  beforehand. 
Being  able  to  preview  information  is  ideal,  not  only  in  situations  where  the  parties  are  in  different  locations,  but  also  when 
students  gather  in  the  same  classroom. 

6.  Transmission  of  image  information  from  the  teacher  to  students:  This  enables  the  teacher  to  send  prepared  images 
beforehand,  such  as  photographs. 

7.  Relay  of  classroom  images  to  the  teacher:  This  is  achieved  by  cameras  that  take  in  images  of  the  whole  classroom,  and 
relaying  these  images  to  the  teacher.  This  enables  the  teacher  to  monitor  the  whole  classroom  situation. 

8.  Shared  cursor:  As  the  teacher  explains  material  during  class,  this  function  enables  all  students  to  view  the  position  of  the 
mouse  point  at  any  given  moment. 

9.  Easy-to-operate  testing  functions:  When  the  teacher  and  students  are  in  different  locations,  it  is  difficult  for  the  teacher  to 
ascertain  understanding  of  the  material.  With  easy-to-operate  test  functions,  the  understanding  of  students  can  be  tested. 

10.  Note  system:  This  function  enables  students  to  take  notes  on  their  own  computer. 

11.  Virtual  class  list  showing  seating  arrangements:  This  allows  the  teacher  to  check  where  the  students  are  sitting,  enabling 
the  teacher  to  ascertain  information  about  students. 

12.  A function  that  facilitates  the  asking  of  questions:  When  a student  wants  to  ask  a question,  this  function  is  used,  which 
enables  the  teacher  to  receive  questions. 

As  the  above  examination  of  basic  functional  capabilities  indicates,  we  are  concerned  with  the  retention  of  two-way 

communication,  which  is  fundamental  in  educational  classes  (functions  1-8,  12).  At  the  same  time,  the  merits  of  a 

computational  system  are  also  maximized  (functions  9,  10,  11). 

2.5  Language  education  and  the  functional  assessment  of  systems  supporting  language  lessons  and  self-study 
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In  language  education,  as  the  focus  shifts 
from  language  ability  based  solely  on 
reading  to  practical  language  proficiency, 
phonetic  education  has  become  an  essential 
component.  Especially  in  situations  where 
the  learner  is  not  in  a country  where  the 
target  language  is  spoken,  there  is  no 
environmental  support,  and  one  needs  to 
learn  the  sounds  of  the  language  with 
cassettes  or  videotapes.  Many  institutions 
offering  language  education  have  made  use 
of  LL  classrooms,  and  these  have  served 
their  purpose  well.  However,  with  the 
advent  of  downsizing,  many  are  shifting 
from  LL  to  LL/MM  classrooms.  With  the 
installation  of  computers  in  LL  classrooms, 
cassette  and  videotapes  in  current  use  may 
be  digitalized  and  made  available  by  online  retrieval.  Also,  the  availability  of  online  use  combined  with  the  use  of  multimedia 
language  education  software  is  seen  as  just  one  of  many  merits.  However,  as  lessons  have  shifted  to  LL/MM  classrooms,  the 
size  of  classrooms  has  increased,  making  it  hard  to  read  handwriting  on  blackboards.  Especially  with  languages  such  as 
German  (with  small  umlauts)  and  Thai  (with  letters  which  only  differ  slightly)  there  are  differentiation  problems.  Also,  it 
would  be  ideal  to  have  explanations  delivered  by  teachers  in  the  target  language  stored  on  the  network  for  self-study,  rather 
than  relying  solely  on  notes  taken  during  class.  For  self-study,  a student  should  be  able  to  store  notes  taken  in  class  together 
with  stored  data  of  pronunciation  drills,  enabling  a reenactment  of  the  actual  classroom  experience. 

At  this  point,  extra  functions  are  presented  in  a proposed  system  supporting  language  lessons  and  self-study.  These 
functions  are  expansions  on  the  basic  functions  covered  in  3Mn  Table  3.  Below  is  a description  of  the  added  functions. 

1.  Teacher  orientated:  Virtual  class  list  showing  seating  arrangements  (roll  call)  Student  orientated:  Login  function 

There  is  a need  to  store  roll  calls  on  a database,  enabling  unified  treatment  of  the  data.  Consideration  of  teachers’  needs, 
such  as  provision  of  class  lists  showing  seating  arrangements,  is  also  a necessity. 

2.  Teacher  orientated:  The  sending  of  text  information  Student  orientated:  The  receiving  of  text  information 

Text  information  prepared  by  teachers  would  be  made  available  online.  Multilingual  display  and  multilingual  input  needs 
have  to  be  addressed.  Text  information  should  be  made  available  at  the  same  time  as  the  lesson  contents  (to  be  discussed 
below).  This  information  would  be  used  for  self-study  by  the  student. 

3.  Teacher  orientated:  Sending  of  handwritten  information  Student  orientated:  Receiving  of  handwritten  information 

This  function  allows  the  teacher  to  send  diagrams  to  students  as  they  are  drawn,  via  the  internet.  This  should  also  be  made 
available  at  the  same  time  as  the  lesson  contents  (discussed  below),  to  be  used  for  self-study  by  the  student. 

4.  Teacher  orientated:  Sending  of  video  Student  orientated:  Receiving  of  video 
This  function  allows  the  transmission  of  video  files  to  students  via  the  network. 

5.  Teacher  and  student  orientated:  Dialogue-based  easy-to-operate  testing  functions 

Testing  functions  are  centred  on  the  filling  in  of  blanks.  These  tests  are  sent  by  the  teacher  to  the  students.  After  the 
completion  of  tests,  students  are  able  to  compare  their  own  work  with  the  answers  prepared  in  advance  by  the  teacher.  The 
data  is  then  registered  on  the  data  server  and  made  available  to  the  user  for  self-study.  Of  course,  all  of  this  is  available  for 
multilingual  display  and  multilingual  input. 

6.  Student  orientated:  voice  recording  and  replay  functions 

These  functions  allow  students  to  record  their  own  pronunciation  during  class,  and  to  compare  this  with  the  pronunciation 
in  the  teaching  materials. 

7.  Student  orientated:  Note  system 

This  function  enables  the  student  to  take  his/her  own  notes  in  class  and  to  save  these. 

8.  The  transmission  of  lesson  contents 

This  function  enables  the  transmission  of  video  of  the  lesson,  voice  data,  text  information  given  out  by  the  teacher,  and 
handwritten  information.  All  the  above  types  of  information  are  to  be  simultaneously  and  automatically  distributed  to  all 
students  after  each  lesson. 


3.  Composition  of  the  hardware  and  system  architecture 
3.1  Composition  of  the  hardware 

Figure  3 shows  the  overall  image  of  our  system  supporting  language  lessons  and  self-study  and  the  Table  4 shows  all  of  the 
function.  As  the  figure  shows,  one  computer  is  made  available  to  the  teacher,  with  50  computers  made  available  to  students.  A 
video  camera  for  recording  the  lesson  and  a cordless  microphone  is  included.  Two  servers  are  provided.  One  is  for  the 
transmission  of  moving  images,  the  other  is  a database  server  for  saving  all  data  used  during  class,  as  well  doubling  up  as  a 
file  server. 

(1)  The  teacher’s  computer 

The  teacher’s  computer  is  a windows  machine  which  is  connected  to  the  network. 

(2)  Computers  for  classroom  use 

Computers  for  self-study  can  be  existing  ones,  as  long  as  an  internet  connection  is  available  and  Windows  is  running.  It  is 
also  acceptable  to  use  a telephone  to  connect  them.  Below  are  some  specifications  for  the  various  parts  of  the  configuration. 

(3)  Server  for  database  and  files 
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This  server  has  Windows  NT  as  its  operating  system.  Since  it  is  used  to  store  applications  that  are  used  in  class,  a large  hard 
disk  is  installed.  Microsoft  Access  2000  database  server  is  also  installed,  and  all  types  of  data  used  during  class  are  stored 
here. 

(4)  Server  for  the  transmission  of  moving  images 

This  server  has  Windows  NT  as  its  operating  system.  A video  capture  card,  cordless  microphone  and  camera  are  installed 
for  the  purpose  of  recording  video  of  the  lesson.  A tracking  device  ensures  that  the  recording  of  the  instructor’s  face  remains 
at  a constant  size  on  tape.  Real  Server  has  been  selected  as  the  server  responsible  for  distributing  lesson  contents. 


Functions  supporting  teachers 

Functions  supporting  students 

Functions 
used  in 

class 

1 

Virtual  class  list  showing  seating  arrangements  (roll 
call  arrangements) 

Login  function  (*) 

2 

The  sending  of  text  information 

The  receiving  of  text  information  (*) 

3 

Sending  of  handwritten  information 

Receiving  of  handwritten  information  (*) 

4 

Sending  of  images 

Receiving  of  images  (*) 

5 

Dialogue-based  easy  -to-operate  testing  functions 

Dialogue-based  easy  -to-operate  testing  functions  (*) 

6 

— 

Note  system  (*) 

7 

— 

Voice  recording  and  replay  functions  (**) 

Functions 
used  in 

self-study 

8 

Video  recording  of  the  lesson  by  the  teacher 

Transmission  of  lesson  contents  (**) 

9 

Video  recording  of  the  text  transmission  information 
during  class 

Transmission  of  lesson  contents  (**) 

1 

0 

Video  recording  of  handwritten  information  during 
class 

Transmission  of  lesson  contents  (**) 

(*)  Basic  functions  based  on  Table  3.  (**)  Addition  functions  added  for  language  education 

Table  4 Functions  available  in  a system  supporting  language  lessons  and  self-study 


3.2  The  development  of  the  environment 

The  present  system  was  developed  using  VisualBasic,  and  the  various  types  of  data  utilized  in  class  were  realized  using 
Microsoft  Access  2000.  The  server  for  the  transmission  of  moving  images  was  constructed  with  Real  Basic  Server.  Access  for 
self-study  is  served  by  Web  Browsers,  with  Microsoft  Internet  Information  Server. 


3.3  Actual  installation  of  the  functions 

Figure  4 indicates  the  main  screens  as  seen  by  the  teacher  and  the  students.  The  following  explanations  pertain  to  this 

diagram. 

1 . The  installation  of  the  virtual  class  list  showing  seating  arrangements  (roll  call  system)  and  database  access  methods 

The  attendance  roll  runs  on  a Jet  database  engine.  The  attendance  database  was  created  with  Access.  The  roll  call  function 
begins  as  the  instructor's  EDLIN  is  started  up.  As  the  instructor  starts  up  his/her  side  of  the  system,  a seating  diagram  is 
shown,  as  seen  in  Figure  5.  Students’  reports  of  attendance  are  then  accepted.  Students  login  to  EDLIN,  and  their  data  is 
transmitted  to  the  teacher’s  computer.  The  IP  address  of  the  point  of  access  is  noted,  and  the  name  of  the  student  is  written 
onto  the  seat  diagram  and  preserved  as  data.  All  this  data  is  preserved,  so  that  the  user  is  free  to  check  his/her  own 
attendance  with  the  attendance  button  on  the  student  main  window.  The  teacher  can  use  the  AttendList  to  check  the  names 
of  attendees  on  that  day. 

2.  The  installation  of  the  text  transmission  function 

In  order  to  display  multilingual  data,  this  system  uses  the  Rich  Text  format  for  text.  Text  entered  by  the  teacher  is  saved  on 
file  at  the  touch  of  the  return  key,  and  automatically  sent  to  all  students.  The  time  is  recorded  with  the  saved  file,  and  the 
lesson  contents  transmission  function  (to  be  discussed  below)  uses  this  time  to  achieve  simultaneity.  On  Japanese  computers, 
umlauts  in  German  are  generally  not  available  on  the  keyboard,  hence  the  many  multilingual  input  assistance  tools,  as 
shown  in  Figure  6.  With  the  addition  of  these  multilingual  assistance  tools,  many  languages  can  be  dealt  with.  The  text  box 
in  the  main  window  of  Figure  3 shows  this  function. 

3.  The  installation  of  the  transmission  function  for  handwritten  information 
This  function  allows  the  teacher  to  transmit  to  the  students  (in  real  time)  the  data  drawn  on 
his/her  whiteboard  window.  When  WhiteBoard  is  activated  on  the  teacher’s  computer, 
whiteboards  appear  on  both  the  teacher’s  and  the  students’  screens.  Data  is  then  transmitted. 

Also,  every  ten  seconds,  information  about  time  and  the  image  on  the  whiteboard  are  saved  as  Figure  6 Special  char  input 
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an  image  file.  The  lesson  contents  function  also  refers  to  this  to  achieve 
simultaneity. 

4.  The  installation  of  the  sending/receiving  functions  for  images 
When  the  teacher  loads  a prepared  image  and  presses  the  transmission 
button,  the  students’  picture  windows  also  start  up  automatically, 
displaying  the  sent  image  (Figure  7),  The  file  types  that  can  be  displayed 
are  JPEG,  GIF  and  BMP, 

5.  Dialogue-based  easy-to-operate  test  functions 
When  the  teacher  presses  the  MiniTest  button,  a dialogue-based  test 
window  opens,  which  is  simple  to  operate  (Figure  8a),  When  the  teacher 
enters  the  questions  and  sends  these,  a test  window  appears  on  the  students’ 
computers  (Figure  8b).  When  a student  completes  the  answers  and  sends 
them  back,  a student  window  with  the  name  of  the  student  appears  (Figure 
8a).  If  the  teacher  sends  only  questions,  a question  sheet  appears  on  the 
student’s  window  (Figure  8c).  When  answers  and  inserted  and  sent,  a 
window  with  the  answers  appears  on  the  student’s  screen.  All  this  data  are 
stored,  and  students  may  use  this  to  revise  the  contents  of  the  lesson  during 
self-study  sessions. 

6.  Voice  recording  and  voice  replaying  functions  for  students 
As  shown  in  Figure  4,  the  student  may  use  the  record,  stop  and  replay 
functions  on  the  student  screen  to  record  his/her  own  voice  and  to  replay  this 
This  function  can  be  used  in  times  of  self-study,  or  utilized  to  compare  one’s 
own  pronunciation  with  that  of  multimedia  teaching  material. 

7.  When  the  Note  Button  is  selected,  a note  window  appears,  which  when 
utilized  and  saved,  can  be  used  in  the  next  self-study  session. 

8 Transmission  of  Lesson  Contents 

This  lesson  contents  system  enables  the  simultaneous  transmission  of  four 
types  of  data:  recorded  video  of  the  lesson,  sound,  text  and  handwritten  data. 

The  camera  used  has  a tracking  system  which  focuses  on  a red  mark 
attached  to  the  teacher’s  chest,  ensuring  that  the  teacher’s  face  is  always 
recorded  as  the  same  size  on  tape.  Sound  is  recorded  with  a cordless  microphone,  and  this  ensures  the  red  mark  remains 
unobstructed.  Both  sound  and  video  data  are  recorded  from  the  start  of  the  lesson  to  the  very  end.  When  the  lesson  is 
finished,  a Real  Text  file  with  time  information  is  generated,  referring  to  the  time  recorded  on  the  saved  text  files. 
Handwritten  data  is  made  into  Real  Picture  files  with  time  information,  again  using  time  data  on  the  saved  files.  Thus  text 
data,  handwritten  data,  together  with  sound  and  video  data  can  be  arranged  for  simultaneity.  All  these  are  sent 
simultaneously  as  SMIL  files.  Figure  8 displays  a screen  with  lesson  contents  being  transmitted.  On  the  left  is  Real  Text,  in 
the  middle  is  Real  Picture,  and  on  the  right  is  a video  screen. 

4 Conclusion 

In  this  study,  the  basic  functions  necessary  for  network-based 
learning  environments  were  examined.  In  addition,  further  functions 
necessary  for  language  education  were  added.  We  conclude  that 
network-based  learning  environments  must  not  only  re-create  the 
classroom  environment,  but  also  provide  support  for  study  sessions 
after  class.  In  particular,  language  learning  cannot  take  place  just  by 
understanding  of  concepts;  repeated  practice  is  necessary.  It  was  for 
this  purpose  that  the  authors  of  this  paper  developed  the  current 
system.  It  is  a system  that  supports  language  lessons  and  self-study,  based  on  the  transmission  of  lesson  contents,  and 
including  provisions  for  self-study.  As  this  system  comes  into  general  use,  evaluation  studies  will  be  conducted  as  part  of  the 
next  step  in  our  investigations. 
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Abstract:  Eight  Chinese  Language  and  Literature  teachers  participated  in  this  research  to  make  their  first 
attempts  at  integrating  information  and  communications  technologies  (ICT)  into  curriculum.  They  were 
provided  training  in  common  application  tools  and  then  paired  up  to  develop  ICT-based  courseware  for  use 
in  classroom  teaching.  Four  units  of  courseware  thus  developed  were  field-taught  to  169  students. 
Results  of  a questionnaire  survey  conducted  after  field-teaching  indicate  that  most  students  prefer 
ICT-based  lectures  to  conventional  ones  because  ICT  offers  such  benefits  as  making  the  lectures  more 
interesting,  enhancing  the  mood  of  the  class,  providing  richer  material  than  usual,  and  improving 
teacher- student  interaction. 


The  Research  Method 

A school-university  partnership  was  established  in  this  research  to  enhance  Chinese  Language  and  Literature 
(CLL)  teachers’  computer  competency  and  their  use  of  information  and  communications  technologies  (ICT)  in 
instruction.  The  partnership  project  was  operated  between  the  authors’  department  and  Hai-San  Junior  High  School,  a 
typical  junior  high  school  located  in  northern  Taiwan.  Within  this  partnership,  our  department  provided  three 
computers  and  project-related  software,  set  up  network  connection  for  these  computers,  and  offered  ICT  training 
courses  to  Hai-San ’s  CLL  teachers.  Reciprocally,  the  school  provided  full  administrative  support,  the  computer  lab  for 
the  training  courses,  the  conference  room  for  seminars,  and  the  equipment  to  be  used  for  field  teaching.  The  major 
participants  included  a professor,  three  graduate  students,  four  undergraduate  students  from  our  department,  and  eight 
CLL  teachers  from  Hai-San.  All  eight  participating  teachers  are  female.  Their  teaching  experiences  ranged  from  8 to 
28  years  and  they  ranged  in  age  from  33  to  51 . 

Considering  that  the  participating  teachers  had  limited  prior  experience  in  using  computers,  we  decided  to  offer 
them  an  up  front  training  course  in  ICT  skills.  The  training  course  was  also  open  to  other  interested  teachers  at 
Hai-San.  The  course  was  36  hours  in  length  and  was  divided  into  12  sessions.  Each  session  was  taught  by  a graduate 
student  as  the  lecturer  and  two  undergraduate  students  as  teaching  assistants.  Teachers  were  trained  in  common 
application  tools,  including  Microsoft  Windows  (3  hours),  Internet  tools  (email,  web  browsers,  ftp,  telnet,  etc.)  (6 
hours).  Word  (9  hours),  PowerPoint  (6  hours),  FrontPage  (9  hours)  and  Excel  (3  hours). 

We  took  care  to  incorporate  as  many  CLL-related  examples  and  exercises  into  the  training  material  as  possible . 
For  example,  when  we  were  teaching  the  use  of  browsing  tools,  we  asked  the  teachers  to  search  for  web  sites  that 
provided  good  databases  of  Chinese  idioms  and  proverbs  or  that  contained  a rich  collection  of  famous  Chinese 
calligraphic  works;  while  in  teaching  Word’s  features,  we  taught  them  how  to  format  a CLL  exam  paper;  we  also  used  a 
CLL  lesson  plan  as  an  example  in  introducing  formatting  “tables”  with  Word.  Our  purpose  was  to  keep  reminding 
these  teachers  that  ICT  skills  were  not  to  be  learned  in  isolation  from  the  subject  area.  In  addition,  we  arranged  three 
tutorials  to  introduce  these  teachers  to  useful  CLL-related  digital  resources,  including  CD  titles  and  web  sites.  In  the 
tutorials  we  also  demonstrated  exemplar  cases  of  ICT  integration  in  other  subject  areas. 

After  the  training  course  was  completed,  the  eight  teachers  paired  up  to  design  and  develop  ICT-based 
courseware.  We  assigned  one  of  our  students  to  assist  each  pair  of  teachers.  Each  team  picked  a unit  from  the 
textbook  and  wrote  a lesson  plan  in  which  they  embedded  ICT-integrated  learning  activities  making  use  of  the 
courseware  they  plan  to  develop.  The  teachers  generated  ideas  and  developed  the  courseware  with  technical 
assistance  from  our  students.  Individual  training  on  advanced  ICT  skills  were  provided  to  them  on  an  “as  needed” 
basis  during  the  development  process,  which  included  skills  in  using  web  page  design  tools,  video  editing  tools,  etc. 

Among  the  courseware  developed  by  the  four  teams,  one  team  focused  their  courseware  on  introducing  students 
to  an  author  who  has  been  famous  not  only  for  his  outstanding  literary  work  but  also  for  his  love  affairs  with  several 
beautiful  women  in  the  early  2(/^  century.  Another  team  used  PowerPoint  to  present  an  article  in  a storybook-like 
fashion.  They  involved  their  students  in  the  development  of  courseware  by  asking  them  to  create  illustration  for  the 
storybook.  Yet  another  team  of  teachers  asked  each  of  their  students  to  draw  a picture  to  visualize  what  he/she 
thought  an  ideal  world  would  be  like.  The  teachers  then  scanned  the  pictures  into  the  computer  so  they  could  be 
shown  during  classroom  discussion.  Teachers  of  the  fourth  team  asked  their  students  to  collaborate  in  writing  a play 
script  based  on  a famous  dialogue  occurring  between  an  emperor  of  the  Song  Dynasty  in  ancient  China  and  one  of  his 
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loyal  military  commanders.  The  script  was  then  presented  as  PowerPoint  slides  and  shown  in  the  background  while 
the  students  performed  the  play.  Even  though  the  learning  activities  supported  by  the  four  units  of  courseware  were 
quite  different  from  one  another,  all  exhibited  ingenuity  in  integrating  ICT. 

The  courseware  was  used  in  field  teaching  to  169  students  of  six  classes  at  Hai-San  Junior  High  School  in  April 
2001.  We  invited  the  principal,  the  director  of  the  academic  affairs,  and  all  CLL  teachers  to  sit  in  and  observe.  Six 
of  the  eight  participating  teachers  used  the  courseware  they  developed  in  teaching  a 45-minute  dass  session  each. 
The  presentation  equipment,  including  the  computer,  the  overhead  projector  and  the  screen,  was  set  up  by  our  graduate 
students  in  advance.  Our  students  also  helped  operate  the  computer  during  field  teaching  so  that  the  teachers  could 
concentrate  on  their  planned  teaching  activities  without  worrying  about  possible  technical  problems.  All 
field-teaching  sessions  were  videotaped  for  later  reviews  and  analyses. 

Findings 

A questionnaire  survey  was  conducted  right  after  field  teaching  to  collect  students’  feedback.  The  questionnaire 
contained  19  questions.  Eleven  of  them  focused  on  contrasting  the  use  of  ICT  in  instruction  with  conventional 
lectures;  three  questions  were  related  to  students  ’ attitude  toward  ICT  integration  in  classroom  teaching;  and  the 
remaining  five  questions  concerned  possible  changes  in  students  ’ motivation  toward  learning  computer  knowledge  and 
skills  as  a result  of  experiencing  ICT  integration  in  subject  teaching.  The  results  indicated  that  most  students 
preferred  ICT-based  lectures  to  conventional  ones.  A large  majority  of  students  agreed  that  integration  of  ICT  helped 
make  the  lectures  more  interesting  (97.64%),  enhanced  the  climate  of  the  class  (92.91%),  offer  richer  materials  than 
usual  (92.91%),  and  improved  teacher-student  interaction  (83.46%).  Most  students  would  like  to  see  the  new 
teaching  method  used  more  often  in  the  future  (92.91%);  they  would  like  students  of  other  classes  to  have  the  chance  to 
experience  the  new  teaching  method  (93.60%);  and  they  hoped  that  ICT  could  be  used  by  teachers  of  other  subjects 
(94.82%).  Moreover,  94.26%  of  students  admired  their  teachers  for  their  computer  skills  in  creating  the  courseware, 
and  95.24%  of  students  agreed  that  they  would  like  to  learn  more  about  computers  after  seeing  what  their  teachers  were 
able  to  do.  As  to  what  other  subjects  in  which  the  students  would  prefer  seeing  frequent  use  of  ICT  integration,  the 
top  three  subjects  were  History  (62.2%),  Geography  (40.94%)  and  English  (33.07%). 

Aside  from  students’  positive  feedback,  the  eight  participating  teachers  took  much  pride  in  their  own 
accomplishment,  and  they  found  the  experience  to  be  most  rewarding.  Furthermore,  their  achievements  were  well 
recognized  by  both  the  principal  and  the  director  of  the  academic  affairs.  In  a seminar  held  for  those  teachers  who 
came  to  observe  the  field  teaching,  the  eight  participating  teachers  shared  their  feelings  and  experiences  with  their 
colleagues  about  how  they  managed  to  overcome  their  own  technophobia,  how  they  came  up  with  the  idea  for 
developing  the  courseware,  their  alternate  feelings  of  frustration  and  self-fulfillment  along  the  way,  and  their  opinion 
of  what  distinguished  teaching  with  ICT  from  teaching  in  a regular  classroom.  They  encouraged  other  teachers  to 
give  it  a try  by  emphasizing  that  “if  we  can  do  it,  you  can  too.”  Most  other  teachers  remarked  that  they  were  highly 
impressed  by  what  their  peers  were  able  to  do  and  that  they  had  become  much  more  convinced  of  the  relevance  and 
value  of  integrating  ICT  in  CLL  teaching..  They  also  felt  more  motivated  toward  learning  computer  skills  and  had 
more  self-confidence  in  implementing  ICT  integration. 

In  the  seminar  the  teachers  also  discussed  about  obstacles  and  barriers  to  the  use  of  computers  and  the  Internet 
for  instruction.  What  was  identified  included  lack  of  computers  and  Internet  connection  in  teachers’  offices  and 
classrooms,  lack  of  proper  training  for  teachers  to  learn  how  to  use  computers  or  the  Internet,  lack  of  release  time  for 
teachers  to  develop  useful  courseware,  insufficient  class  hours  to  incorporate  frequent  use  of  computer  in  classroom, 
and  lack  of  information  about  useful  digital  resources  for  use  in  teaching  or  learning  CLL.  Some  teachers  were  more 
concerned  about  the  issue  of  technical  support,  while  some  others  hoped  that  there  could  be  one  or  more  multimedia 
classrooms  established  and  dedicated  to  CLL  teaching  to  facilitate  integration  of  ICT  in  the  curriculum. 

Conclusions 

Chinese  Language  and  Literature  has  long  been  considered  the  most  “traditional”  subject  in  our  schools,  and 
CLL  teachers  are  often  labeled  as  most  reluctant  to  adopt  innovative  teaching  practices.  In  this  study  we  guided  eight 
CLL  teachers  through  their  first  attempts  at  ICT  integration.  They  have  succeeded  in  convincing  themselves  as  well 
as  their  colleagues  that  integrating  ICT  into  CLL  curriculum  is  indeed  practicable.  They  have  also  proved  that  all 
CLL  teachers  have  the  potential  of  utilizing  ICT  for  enhanced  effectiveness  in  teaching  and  learning  if  only  they  are 
provided  with  adequate  training  and,  more  importantly,  if  only  they  are  willing  to  try. 
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Abstract:  Shadow  netWorkspace’*"'^  (SNS)  is  designed  and  developed  as  a free,  open 
source,  network-based  work  environment  specifically  for  use  in  K- 12  schools  to  support 
schools  and  learning.  In  this  paper,  we  describe  a learning  module  utilizing  features  of 
SNS  and  articulate  how  teacher  and  students  can  collaboratively  learn  and  work  to  build 
knowledge.  The  paper  also  proposes  a collaborative  knowledge  building  model  that 
illustrates  the  significant  phases  in  the  learning  module.  Teachers  in  the  local  community 
are  invited  to  evaluate  the  feasibility  of  learning  module  in  terms  of  its  benefits, 
challenges,  the  ease  and  appropriateness  of  use.  Feedback  from  the  evaluation  will 
provide  recommendations  for  the  design  and  development  of  knowledge  building 
activities. 


SNS  Learning  Communities 

Shadow  netWorkspace'*'^  (SNS)  is  an  Open  Source,  network-based  work  environment  designed 
and  developed  for  use  in  K-12  schools  by  the  Center  for  Technology  Innovations  in  Education  (CTIE)  at 
the  University  of  Missouri-Columbia.  Open  Source  licensing  provides  a new  framework  for  the 
development  and  evolution  of  software  systems,  and  network-based  learning  systems  enable  schools  to 
change  from  knowledge-acquisition  institutions  to  knowledge-building  learning  communities  (Laffcy  et  al. 
1999).  Specifically  SNS  has  developed  a network-based  software  system  and  is  building  a community  of 
developers  committed  to  open  source  licensing  and  the  SNS  learning  system. 

We  utilize  features  of  SNS  to  build  collaborative  learning  modules.  They  are  designed  to  gain 
participation  of  an  extended  community  of  researchers,  teachers,  administrators,  and  tech  coordinators  in  an 
effort  to  gain  better  understanding  of  network-based  learning  systems  and  their  potential  impact. 
Collaboration  could  happen  in  two  main  areas:  1)  development  and  refinement  of  learning  modules,  and  2) 
research  and  assessment  of  learning  modules.  Because  a widespread  use  of  technology  is  to  promote 
learning  and  facilitate  lesson  plans,  these  collaborative  efforts  can  build  the  body  of  learning  modules 
available  to  teachers  as  well  as  the  information  about  the  utility  and  efficacy  of  those  modules. 

Scardamalia  and  Bereiter  (1993)  describe  how  computer-supported  collaborative  learning  (CSCL) 
systems  can  support  and  optimize  both  the  process  and  the  representation  of  knowledge-building,  learning 
and  collaboration.  SNS  provides  a means  for  both  educators  and  software  developers  to  create  sharable  and 
continuously-improving  software  that  can  be  easily  and  flexibly  integrated  to  a local  learning  community. 
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Therefore,  rather  than  a rigid  learning  application,  SNS  is  more  likely  an  adaptable  operating  system  for 
collaboration  and  learning. 


A Learning  Module  in  SNS 

Objectives  and  Method  Review 

One  collaborative  learning  module  is  a note  taking  and  critiquing  module  that  outlines  methods  for 
using  SNS  to  facilitate  collaborative  knowledge  building.  The  objective  of  this  module  is  to  improve 
students’  note  taking  and  critical  analysis  skills  and  to  encourage  effective  integration  with  the  content 
lessons. 

Each  week,  different  students  are  assigned  to  be  the  “note-posters”.  They  are  responsible  for 
taking  their  best  notes  for  that  given  week  and  posting  them  into  appropriate  documents  (Shadowdoc)  to 
collaborate  and  brainstorm  within  their  science  class  group.  Each  night,  each  member  of  the  class  is 
assigned  to  critique  a manageable  number  (1-3  depending  on  volume)  of  note-sets  by  comparing  the  posted 
sets  to  their  own  and  commenting  via  the  discussion  boards.  Friday  is  review  day.  The  class  works  together 
to  guide  the  instructor  (and  vice  versa)  through  the  process  of  combining  the  note-sets  into  one  document 
that  represents  all  the  key  points  that  the  instructor  wanted  to  get  across  as  well  as  the  areas  of  detail  that 
should  be  included.  This  final  document  could  be  used  for  preparing  unit  assessment. 

This  exercise  aims  to  create  a “progressive  discourse”  between  the  students  and  the  instructor. 
This  discourse  embodies  the  concept  of  formative  evaluation  within  the  context  of  a knowledge-building 
task  (Scardamalia  & Bereiter,  1996a).  Thinking  can  be  shaped  by  writing  and  people  tend  to  think  more 
and  write  more  clearly  when  they  are  producing  content  for  others.  Critical  thinking  happens  in  good 
discussions.  For  years,  proponents  of  Writing-Across-the-Curriculum  (WAC)  have  been  saying  that 
students  learn  when  they  formulate  their  ideas  in  writing  (Lang,  2000).  Writing  is  reviewable,  revisable, 
and  recorded.  This  effort  to  optimize  the  process  of  building  expertise,  requires  students  to  be  intentional 
learners.  Analyzing  their  knowledge  and  organizing  new  information,  with  the  understanding  that  their 
words  will  be  under  the  scrutiny  of  their  peers,  should  encourage  thoughtfulness  and  learning.  As  the 
students  contribute  to  build  a collective  knowledge,  they  will  be  required  to  adapt  both  their  intentions  and 
their  methods.  As  they  gain  understanding  they  must  quickly  switch  roles  from  learner  to  teacher,  so  that 
they  can  help  their  peers  to  learn  and  improve  the  group  artifacts. 


Assessment  of  a collaborative  activity  might  be  the  biggest  challenge  for  teachers  when  using  this 
teaching  method.  The  reason  why  it  is  so  difficult  to  evaluate  collaboration  is  that  teachers  have  to  examine 
not  only  the  process  but  also  the  final  product  of  a group’s  work. 

This  learning  module  is  intended  to  fit  into  a traditional  curriculum,  so  the  weekly  note-sets  could 
serve  as  review  materials  for  quizzes  or  examinations.  Concept-mapping  assessment  could  be  used  as  an 
alternative  assessment  method.  A concept  map  is  a node -link-node  representation  of  content,  where  nodes 
represent  concepts  and  links  represent  relationships  between  connected  concepts  (Dansereau,  1995; 
Jonassen,  Beissner,  & Yacci,  1993).  When  large  numbers  of  concepts  are  connected,  a concept  map  is 
formed  which  ideally  represents  the  content  and  the  structure  of  a student’s  knowledge  framework.  For  a 
group  of  students,  concept  maps  can  be  viewed  as  expressing  the  meaning  shared  by  team  members.  The 
concept  maps  can  be  scored  for  (a)  number  of  unique  nodes,  (b)  number  of  links  between  nodes,  and  (c) 
number  of  levels  based  on  adaptions  of  concept  map  scoring  procedures  (Ayersman,  1995;  Novak  & 
Gowin,  1984;  Toro,  1995).  As  learning  occurs  from  instruction,  the  growth  of  learners’  knowledge  should 
be  reflected  in  the  elaborateness  and  structure  of  the  notes  they  construct  (Jonassen,  1987).  Analyzing  the 
complexity  and  accuracy  of  pre-  versus  post-lesson  concept  maps  can  illustrate  depth  of  students’ 
understanding. 

Learning  and  Knowledge  Building  in  SNS 


Assessment 


The  Knowledge  Building  pedagogy  envisions  learning  as  a social  process  of  collaboration  that  students 
can  become  active  learners  and  more  aware  of  their  own  and  others’  processes  of  constructing  knowledge 
(Brown  & Campione,  1994;  Scardamalia  & Bereiter,  1996b;  Pea,  1993).  This  note-taking  learning  module, 
the  significant  phases  of  collaborative  knowledge  building  are  illustrated  in  Figure  1:  cycles  of  personal 
understanding  and  collaborative  knowledge  building.  This  figure  represents  the  processes  of  internal 
personal  understanding,  and  socio -cultural  knowledge  building.  This  model  illustrates  the  environment 
where  individual’s  understanding  can  be  adapted  and  enhanced  collaboratively. 

SNS  Support  for  Social  Knowledge  Building  Activities 

Network-based  learning  system  cannot  provide  support  for  individual  cognition;  Individuals  must 
articulate  their  understanding  and  beliefs  as  public  statements  before  they  can  interact  within  network- 
based  learning  system.  Thoughts  must  be  even  more  formalized  for  computer  support  than  for  interpersonal 
interaction  (Stahl,  1993).  In  this  module,  students  can  use  Shadowdoc  to  create  documents  with  sections 
and  images  and  to  compile  the  class  notes.  The  full-featured  discussion  board  in  SNS  could  also  be  used  for 
representation  and  interaction. 


Figure  1:  Collaborative  Knowledge  Building  Using  SNS  Tools  (adapted  from  Stahl,  2000) 

(a)  Articulate  in  Words-  SNS  tools  will  facilitate  the  process  of  articulating  ideas  and  preserving  them  in 
convenient  forms.  Posting  notes  will  encourage  the  users  to  “verbalize”  their  thoughts  in  an 
appropriate  expression  and  accumulate  the  text  to  the  public  statements. 

(b)  Public  Statements  - Public  statements  are  the  domains  where  one  person  confronts  other  people’s 
perspectives.  Posted  notes  can  represent  the  different  perspectives  from  which  these  statements 
emerge.  It  also  makes  explicit  the  important  relationships  among  personal  and  group  perspectives,  as 
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well  as  providing  meanings  for  individuals  and  collaborative  teams  to  articulate  their  own  perspectives. 
The  instructor  or  other  users  can  co-construct  and  add/edit  commentary  or  critiques  on  writing 
documents. 


(c)  Discuss  Alternatives  - The  SNS  tools  provide  an  asynchronous,  interactive  communication  that  allows 
students  to  respond  to  notes  posted  by  others.  Unlike  a tree  of  divergent  opinions,  the  Shadowdoc  or  a 
scaffolded  discussion  forum  can  go  beyond  superficial  aimless  discussion  to  converge  on  shared 
understandings  and  acknowledged  opinions  (dePaula,  1998;  Guzdial  & Turns,  2000;  Hewitt  & 
Teplovs,  1999;  Stahl,  2000). 

(d)  Argumentation  & Rationale  - One  note  might  be  against  another  or  provide  evidence  to  support  the 
claim  of  another  note.  Donath  et  al.  (1999)  suggested  the  component  that  supported  argumentation  and 
rationale  could  contribute  to  participants’  meta-level  comprehension  of  their  knowledge-building 
process. 

(e)  Clarify  and  Negotiate  Meanings  and  Perspectives  - Constructing  group  knowledge  can  be  fostered  by 
clarifying  the  meaning  of  important  terms  or  key  concepts  (Stahl,  2000).  The  key  concept  discussion 
can  make  explicit  how  different  participants  understand  the  terms  they  use  and  should  result  in  a group 
glossary  of  the  agreed  upon  definitions  of  important  terms.  After  the  processes  of  clarification  and 
negotiation,  the  key  concepts  or  glossary  could  become  the  future  debate  or  the  topics  for  the 
assessment. 

(f)  Collaborative  Understanding  - The  accumulation  of  negotiated-shared  knowledge  results  in  the 
establishment  of  a group  perspective.  Individuals  can  then  build  on  this  shared  knowledge  within  their 
own  perspective  and  even  begin  to  critique  it  and  start  the  whole  cycle  over. 

(g)  Formalize  & Objectify  into  Cultural  Artifacts  and  Representations  - The  shared  knowledge  can  be 
further  formalized.  It  can  be  represented  in  another  symbolic  system  or  combined  into  a more 
comprehensive  system  of  knowledge.  For  example,  the  class  can  build  a note-set  to  outline  the  key 
concepts  of  the  unit.  The  note-set  will  provide  the  accepted  base  for  building  future  knowledge.  Other 
artifacts  could  include  an  annotated  bibliography,  like  a knowledge  repository. 
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Abstract:  This  paper  details  a pilot  study  into  alternative  forms  of  access  to  laboratory  hardware  for 
final  year  undergraduate  teaching.  Students  were  given  three  different  assignments  to  complete  using  an 
XY  contouring  table  - the  first  proximally  in  the  laboratory,  the  second  by  simulation  of  the  hardware 
and  the  third  by  remote  access  to  the  hardware  via  the  Web.  The  trial  was  evaluated  using  multiple 
tools,  with  attention  paid  to  the  differences  in  educational  outcomes  of  the  three  alternatives.  It  was 
observed  that  students’  performances  on  different  criteria  varied  depending  upon  the  form  of  access 
used.  Performance  on  a targeted  examination  question  was  also  found  to  be  dependent  upon  the  access 
method. 

Context  of  the  Project 

This  paper  outlines  the  development  and  implementation  of  new  laboratory  classes  within  the  final-year  Mechanical 
Engineering  subject  436-405  Advanced  Control  Systems.  In  this  subject,  students  learn  to  design  analog  and  digital 
controllers  for  complex  mechanical  engineering  systems.  A number  of  advanced  control  system  topics,  such  as 
nonlinear  control  system  analysis  and  design,  multivariable  control  systems  and  optimal  control  are  covered  in  the 
subject. 

In  previous  years  in  this  subject,  the  last  four  weeks  of  semester  were  devoted  to  a ‘case  study’  project,  in  which 
students  would  reproduce  the  results  of  a relevant  recent  journal  article,  and  report  back  to  their  peers.  While  this 
proved  to  be  a useful  exercise,  it  was  thought  that  students  would  benefit  more  from  direct,  personal  experience  with 
system  design,  implementation  and  experimentation,  using  physical  control  system  hardware.  We  also  sought  to 
expand  upon  our  previous  work  in  providing  alternative  access  methods  to  hardware  in  laboratory  classes  (Lindsay,  et 
al.,  2000). 

This  paper  reports  a pilot  study  involving  only  nine  students.  No  attempt  is  made,  therefore,  to  attribute  statistical 
significance  to  the  results.  Nevertheless,  the  results  are  indicative  of  meaningful  differences  in  learning  outcomes  with 
diffe  rent  access  modes. 

Theoretical  Framework 

The  development  of  new  assignment  material  for  the  course  allowed  us  to  pursue  a number  of  objectives.  We  shared 
many  of  the  motivations  of  (Hahn  and  Spong,  2000),  in  that  we  wished  to  increase  students’  exposure  to  control 
systems,  and  also  to  increase  the  flexibility  of  this  access. 

The  incorporation  of  web-based  access  offered  a number  of  flexibility  advantages,  both  immediately  and  also  for 
potential  future  developments.  Access  to  control  laboratories  via  the  Internet  dates  back  to  1996  (Aktan,  et  al.,  1996), 
with  a number  of  studies  since  being  published  to  show  the  feasibility  of  the  paradigm,  such  as  (Hahn  and  Spong, 
2000),  (Shen,  et  al.,  1999),  (Enloe,  et  al.,  1999). 

Of  particular  interest  in  our  development  of  the  projects  was  to  maximise  the  transparency  of  the  technology  involved. 
(Givens  and  McShea,  2000)  elaborates  extensively  upon  the  dual  amplification/reduction  effect  of  technology  upon 
human  experience.  The  beneficial  effects  offered  by  the  access  to  the  physical  hardware  could  be  partially  or  wholly 
offset  by  technical  difficulties  experienced  in  operating  the  hardware.  The  importance  of  a robust,  reliable  system  is 
particularly  emphasised  by  Givens  and  McShea,  and  was  a design  priority  for  this  work. 

Whilst  Givens  and  McShea  do  address  the  issue  of  amplification  vs  reduction,  the  bulk  of  the  literature  on  this  topic 
does  not  concentrate  on  evaluation  of  learning  outcomes.  Positive  feedback  from  students  is  often  reported,  and  indeed 
is  a desirable  outcome.  However,  it  is  secondary  to  the  students’  acquisition  of  desirable  skills  and  knowledge. 

Constructivist  theorists  such  as  (Ausubel,  2000)  tell  us  that  the  acquisition  of  these  skills  is  dependent  on  the  overall 
learning  experience  which  the  student  undergoes  - students  create  new  meanings  for  themselves  by  assimilating  new 
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experiences  into  their  existing  cognitive  structures . To  promote  the  flexibility  of  access,  we  offered  three  different 
learning  experiences  - proximal  access  to  the  hardware,  simulated  access,  and  remote  access. 

In  order  to  assess  the  effects  of  the  different  learning  experiences,  a number  of  different  evaluation  tools  were  developed 
- the  assessment  measures  of  the  assignments  and  the  final  examination,  as  well  as  student  journals,  access  logs  from 
the  hardware,  and  also  direct  student  feedback. 


The  System  Developed  For  the  Project 

In  the  past,  similar  work  in  the  Department  had  been  completed  from  scratch  with  a custom-built  system.  For  this 
project,  the  approach  taken  was  instead  to  integrate  commercially  available  components  to  form  an  overall  system.  This 
approach  was  taken  for  the  hardware  as  well  as  the  software. 

The  Hardware 

The  hardware,  illustrated  in  (Fig  1 ),  consists  of  two  linear  positioning  tables  - a y-axis  table  mounted  on  the  laboratory 
bench,  and  an  x-axis  table  mounted  perpendicularly  on  top  of  the  y-axis.  This  arrangement  provides  two  degrees  of 
freedom  for  the  “tool  point”  of  the  system.  The  linear  tables  are  powered  by  servomotors,  which  are  connected  by 
custom-made  flexible  couplings  that  deliberately  introduce  torsional  compliance  and  backlash  into  the  system. 


Figure  1 : X- Y table  system 


The  servomotors  are  each  controlled  by  an  independent  servo  drive,  located  in  the  cabinet  under  the  bench.  These  servo 
drives  are  in  turn  controlled  by  a dSPACE  ® DS1102  controller  board,  which  is  inside  the  PC  on  the  table.  The  PC  is 
used  to  define  tool  point  profiles  and  controllers  using  MATLAB®  and  Simulink®,  and  also  to  provide  the  web-based 
control  interface. 


How  the  System  Works 

The  system  utilises  the  Simulink  Real-Time  Workshop®  and  dSPACE  software  to  compile  Simulink  models  into  object 
code  for  the  DSP  board.  These  models  represent  the  controller  for  the  XY  table  system.  Depending  upon  operation 
mode,  the  DSP  board  sends  either  velocity  or  current  command  signals  to  the  servocontrollers.  The  servocontrollers 
power  the  servomotors  which  move  the  linear  tables.  See  (Fig  2) 
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Figure  2:  Schematic  of  the  control  system  (in  CNC  mode) 

When  the  system  is  in  operation,  the  PC  samples  the  memory  of  the  DSP  board  and  records  the  values  of  selected 
variables  such  as  motor  velocities,  position  commands  etc.  These  variables  are  stored  within  the  MATLAB  workspace 
on  the  PC. 
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Students  design  their  own  controllers  for  the  XY  table  system,  implement  them  in  Simulink  and  compile  them  to  the 
DSP  board.  They  then  initiate  data  capture  runs  and  collect  and  analyse  the  sampled  data.  The  inclusion  of  Simulink  in 
the  software  resources  allows  a wide  variety  of  alternative  control  strategies  to  be  readily  attempted  without  the  need  for 
time-consuming  and  error-prone  hand  coding. 

A comprehensive  Simulink  model  of  the  XY  table  system  was  developed,  including  many  of  the  non-linearities  of  the 
real  hardware.  This  model  was  used  in  the  place  of  the  connections  to  the  DSP  boards  to  provide  a simulation  of  the 
XY  table  system. 

The  MATLAB  Webserver  and  the  Apache  server  were  also  employed,  allowing  the  MATLAB  scripts  used  in 
controlling  the  XY  table  to  be  called  remotely  via  the  web.  Specific  input  and  output  HTML  templates  were  developed 
to  pass  the  required  commands  to  the  Webserver. 

Educational  Materials  Developed  For  the  Project 

Assignments 

Three  new  assignments  were  developed  for  the  course,  each  of  which  involved  a different  method  of  interacting  with 
the  hardware.  The  assignments  accounted  for  50%  of  the  overall  mark  that  students  received  in  this  subject. 

Assignment  One  — Proximal 

In  Assignment  One,  students  were  provided  the  opportunity  to  control  the  XY  Table  within  the  laboratory.  The 
controllers  of  the  table  axes  were  deliberately  mismatched  to  introduce  a correctable  contouring  error  into  the 
performance  of  the  table. 

Students  began  by  investigating  the  response  of  the  table  to  straight  line  command  profiles.  They  tuned  the  feedback 
gains  of  the  CNC  controller  to  optimise  this  performance.  They  then  investigated  the  behaviour  of  the  optimised  system 
attempting  to  reproduce  more  complex  profiles,  and  examined  the  ‘performance  envelope’  of  their  controller  - the  range 
of  tasks  for  which  the  system  provided  adequate  performance. 

Assignment  Two  - Simulation 

In  Assignment  Two,  students  were  provided  with  a detailed  Simulink  model  representing  the  XY  table.  This  model 
allowed  students  to  perform  “experiments”  as  though  they  had  direct  access  to  the  laboratory  equipment,  without  the 
scheduling  constraints  of  accessing  a single  piece  of  hardware.  The  simulated  system  generated  the  same  sampled  data 
as  was  available  in  the  laboratory  set-up. 

Students  were  asked  to  investigate  ways  of  improving  the  contouring  performance  with  alternative  control  strategies. 
For  some  strategies,  this  required  them  to  perform  ‘system  identification’  experiments.  They  were  required  to  select  an 
appropriate  measure  of  performance,  and  to  compare  their  alternative  control  strategies  using  this  criterion. 

Assignment  Three  — Remote 

In  Assignment  Three,  students  were  required  to  physically  implement  the  controllers  they  developed  in  Assignment 
Two.  This  implementation  was  through  the  remote  access  interface,  with  feedback  from  the  system  provided  via  a 
webcam  and  the  sampled  output  from  the  MATLAB  workspace. 

The  students  were  also  required  to  investigate  the  robustness  of  the  controllers  designed  using  the  simulation  model,  and 
to  tune  them  as  necessary  to  adapt  for  the  differences  between  simulation  and  reality. 

Examination  Question 

Deep  learning  of  the  course  material  is  important.  This  can  be  assessed  by  presenting  students  with  problems  similar  to 
those  encountered  in  the  assignments,  the  effective  solution  of  which  relies  upon  the  same  theory  and  concepts. 
Question  One  of  the  examination  for  the  subject  was  designed  to  address  the  key  concepts  covered  by  the  assignment 
work,  with  sub-sections  of  the  question  specifically  aimed  at  the  topics  in  the  three  different  assignments. 

Outcomes  of  the  Project 

A number  of  different  tools  were  used  to  measure  outcomes  of  the  project  - the  assessment  of  the  assignment  work,  and 
a follow  up  examination  question;  students’  self-reflections  from  study  journals;  analysis  of  access  logs;  and  direct 
feedback  from  students,  as  well  as  our  anecdotal  observations  of  their  work. 
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Assignment  outcomes 

To  allow  comparisons  to  be  drawn  between  the  three  approaches,  the  same  evaluation  tool  was  used  to  assess  each  of 
the  different  approaches.  To  further  reduce  the  variance  in  the  marking  process,  a single  marker  was  responsible  for 
marking  all  of  the  assignments. 

Eleven  criteria  were  chosen  for  the  purposes  of  evaluating  the  students’  work.  Each  of  the  criteria  is  a desirable 
outcome  of  the  assignment  work.  Some  are  generic  skills,  some  are  the  application  of  generic  skills  to  a specific  task, 
and  some  are  task  specific.  Comparisons  between  the  outcomes  for  the  task-specific  criteria  (such  as  criterion  1)  are 
less  meaningful  than  those  drawn  between  the  more  generic  criteria  (such  as  criterion  4).  Students  were  given  a grade 
from  1 (very  poor)  to  5 (very  good)  for  each  criterion. 

The  three  assignments  were  each  assessed  on  the  following  criteria: 

1.  Familiarity  with  the  system,  and  the  components  thereof. 

2.  Cognitive  achievement  - an  understanding  of  concepts  - rather  than  procedural  achievement  - just  “turning  the 
handle”. 

3.  Identification  of  the  important  variables/parameters  involved  in  problems  associated  with  the  hardware. 

4.  Generation  of  multiple  viable  solutions  to  given  set  tasks  with  the  hardware. 

5.  Evaluation  of  the  alternative  solutions  to  choose  an  optimal  solution. 

6.  Anticipation  of  possible  exceptions,  and  formulation  of  methods  of  overcoming  these  exceptions. 

7.  Adaptation  to  unforseen  exceptions  in  a professional  way. 

8.  Understanding  of  the  non-ideal  effects  of  compliance,  backlash  and  friction  on  control  systems 

9.  Understanding  and  demonstration  of  multiple  techniques  for  the  design  of  controllers  for  MISO  systems,  and  the 
strengths  and  weaknesses  of  both. 

1 0.  Demonstrated  understanding  of  the  1 imitations  of  the  system 

1 1.  Fluency  in  the  operating  language  of  the  control  system 

The  mean  performance  on  each  of  the  criteria  for  each  of  the  three  access  modes  is  shown  in  (Fig  3). 
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Figure  3: 


It  can  be  seen  that  the  Proximal  approach  and  the  Simulation  approach  produced  similar  outcomes  for  most  criteria,  but 
that  the  remote  approach  appears  to  differ  substantially  on  some  criteria.  Outcomes  are  notably  poorer  on  criteria  4 and 
5 (the  generation  and  evaluation  of  multiple  solutions),  and  on  criterion  9 (demonstration  of  multiple  design 
techniques).  The  remote  approach  is  stronger,  however,  on  criterion  7 (the  handling  of  exceptions).  These  results  must 


be  regarded  as  indicative  only,  because  of  the  small  class  size  and  possible  lack  of  uniformity  of  criterion  skill 
requirements  between  the  assignments. 

Examination  outcomes 

The  first  question  of  the  Examination  was  devoted  to  the  same  topics  covered  by  the  assignment  work,  with  the  question 
having  five  sub-questions  each  worth  8 marks.  Again,  all  of  the  examinations  were  marked  by  a single  marker  to  avoid 
any  inter-marker  deviation.  The  marks  for  each  question  were  averaged,  the  mean  mark  for  each  question  being  as 
follows: 


Question 

Topics  covered 

Avg  Mark  (/8) 

A 

General  / Proximal 

4.11 

B 

Proximal 

5.33 

C 

Proximal 

5.44 

D 

Simulation 

4.72 

E 

Remote 

4.78 

Again,  the  performance  on  the  Simulation-  and  Remote-topic  questions  was  similar.  Performance  on  two  of  the  three 
Proximakopic  questions  was  superior,  but  the  performance  on  Question  A was  in  fact  considerably  inferior.  We 
suspect  that  lack  of  precision  in  the  wording  of  Question  A may  have  partially  accounted  for  this  outcome. 

Journals 

Students  were  required  to  keep  a diary  of  their  progress  throughout  the  assignments  in  this  subject,  and  were  rewarded 
with  bonus  marks  if  they  did  so.  This  was  intended  as  a tool  to  help  with  the  cognitive  development  of  students  as  they 
progressed,  but  most  of  the  diaries  submitted  served  more  as  a chronicle  of  the  work  that  had  been  done.  Significant 
also  in  all  journals  was  the  amount  of  time  spent  by  students  in  completing  the  assignments  - easily  40  hours  over  an 
eight-week  period  for  a number  of  students.  This  represents  a considerable  assignment  workload  in  a subject  that  has  a 
scheduled  contact  of  two  hours  per  week. 

Access  Logs 

Most  of  the  students  involved  in  the  project  chose  to  postpone  their  use  of  the  equipment  until  as  late  as  possible  in  the 
semester.  Comprehensive  logs  of  their  access  during  the  remote  experiment  were  taken,  recording  each  command  that 
was  sent  to  the  system.  These  logs  showed  that  the  most  common  pattern  of  usage  was  to  change  the  controller,  run  a 
single  data  capture  with  that  controller,  and  then  to  change  the  controller  to  something  different.  Multiple  data  captures 
using  the  same  controller  architecture  were  rare. 

Student  Feedback  / Anecdotal  Observations 

There  was  general  consensus  amongst  the  students  that  the  project  work  required  too  much  time  to  complete 
successfully.  Students  also  expressed  frustration  at  any  technical  difficulties  that  arose  from  the  hardware  or  software 
not  functioning  as  expected,  although  the  complaints  were  somewhat  minor  when  compared  to  those  experienced  by 
Givens  and  McShea. 

Throughout  the  remote  access  assignment,  many  students  displayed  a desire  to  remain  in  physical  proximity  to  the 
hardware.  Despite  having  webcam  feedback  of  the  position  of  the  table,  and  the  output  data  files  of  their  commands 
executing,  students  still  felt  the  need  to  walk  into  the  laboratory  to  visually  inspect  the  hardware.  This  suggests  that  the 
laboratory  experience  provides  a richer  set  of  cues  to  system  behaviour  than  is  available  from  web-based  access.  Given 
the  opportunity,  students  will  seek  access  to  such  information.  We  suspect  that  auditory  feedback  would  result  in  a 
worthwhile  enhancement  of  the  remote  experience. 

Overall,  students  expressed  positive  responses  towards  the  introduction  of  the  hardware  into  the  course,  and  were 
appreciative  of  the  opportunity  to  apply  the  theory  that  they  had  learned. 

Conclusions  / Further  Work 

This  study  serves  as  another  demonstration  of  the  feasibility  of  using  web-based  access  to  physical  hardware  for  the 
teaching  of  control  theory  and  practice  at  an  undergraduate  level,  in  this  instance  at  a final-year  advanced  elective  level. 
As  with  previous  studies,  students  provided  positive  feedback  about  the  value  of  such  initiatives. 


1180 


Further  to  this,  the  results  were  indicative  of  some  differences  in  the  learning  outcomes  of  students  exposed  to  three 
different  modes  of  access  to  physical  hardware.  Proximal  access  generally  lead  to  superior  outcomes,  although  the 
differences  from  simulated  access  were  small. 

Learning  outcomes,  however,  are  the  product  of  the  overall  learning  experience,  and  the  mode  of  access  is  just  one 
dimension  of  this  experience.  Further  trials  are  presently  under  development  within  the  department  to  isolate  this  factor 
and  to  compare  more  rigorously  the  effects  of  the  three  access  modes. 
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Over  the  past  few  years,  the  Scottish  Higher  Education  Funding  Council  has  supported  a programme  of 
pioneering  projects  examining  staff  development  in  the  use  of  communication  and  information 
technology  within  Higher  Education  (http://www.scotcit.ac.uk/).  Two  of  these  projects  have 
specifically  focused  on  the  sharing  and  reuse  of  resources,  each  taking  a unique  approach,  but  having 
the  common  aim  of  enabling  institutions  to  adapt  existing  resources  for  use  across  the  sector,  while 
taking  into  account  diverse  institutional  contexts  and  requirements  (Alexander,  1999).  The  Scottish 
electronic  Staff  Development  Library  (SeSDL)  created  an  online  resources  base  that  allows  users  to 
upload  resources  in  granular  format  and  to  aggregate  these  resources  to  create  lessons  tailored  to  meet 
their  own  requirements.  The  Enabling  Large  Scale  Implementation  of  Communications  and 
Information  Technology  project  (ELICIT)  produced  a series  of  modular  courses  that  can  be  adapted  to 
suit  institutional  needs. 

ELICIT:  Reuse  and  adaptation  of  modular  courses  produced  by  ‘‘expert”  authors 

Enabling  Large  Scale  Implementation  of  Communications  and  Information  Technology  (ELICIT 
http://www.elicit.scotcit.ac.uk)  was  a 2 year  project  which  aimed  to  address  a shortage  of  high  quality, 
electronic  resouices  for  staff  development  by  developing  a common  core  of  short  modules  which  could 
be  adapted  and  reused  across  institutions.  These  resources  examine  underpinning,  educational  theories 
in  the  use  of  C&IT  for  teaching  and  learning  and  were  designed  for  reuse  in  a variety  of  ways:  within 
accredited  courses,  staff  development  workshops,  or  as  online,  supplementary  information. 

A core  of  modules  was  created,  each  consisting  of  a series  of  short  units  with  resources  materials.  The 
resource  materials  include  case  studies,  articles  and  texts,  with  associated  tasks,  which  can  either  be 
linked  to  online  discussions,  or  can  be  explored  in  face-to-face  sessions.  The  modules  include: 

• an  introduction  to  using  and  integrating  C&IT  in  teaching,  learning  and  assessment 

• using  virtual  learning  environments  in  teaching,  learning  and  assessment 

• using  computer  mediated  conferencing  in  teaching,  learning  and  assessment 

• developing  tutor  skills  for  computer  mediated  conferencing 

• developing  computer  aided  assessment 

• implementing  computer  aided  assessment. 

These  modules  are  freely  available  for  staff  developers  within  the  UK’s  Higher  and  Further  Education 
sectors  and  can  be  downloaded  from  the  project  website  in  HTML  or  MS  Word,  edited  to  suit 
individual  institutional  require  ments  and  implemented  within  a Virtual  Learning  Environment 
(Littlejohn  et  al,  200 1 ). 

SeSDL  - Constructing  courses  from  granular  resources 

The  Scottish  electronic  Staff  Development  Library  project  (SeSDL  http://www.sesdL.scotcit.ac.uk) 
takes  a radically  different  approach.  SeSDL  developed  an  online  resources  base,  which  facilitates  the 
retrieval  and  reuse  of  granular  staff  development  materials  focused  on  the  use  of  C&IT  for  teaching 
and  learning. 

SeSDL  provides  users  with  guidelines  on  how  to  create  granular,  educational  resources,  often  referred 
to  as  learning  objects,  and  then  allows  them  to  upload  these  resources  to  the  electronic  library.  All 
resources  submitted  to  the  library  are  described  using  IMS  metadata  and  classified  using  an  educational 
taxonomy  (Campbell,  Littlejohn  & Duncan,  2001).  Library  users  can  search  the  resource  base  for  other 
relevant  resources  using  an  advanced  or  a simple  search  tool,  or  a browse  tree,  which  corresponds  to 
the  educational  taxonomy  classifications  (see  diagram  1).  Once  users  have  located  relevant  resources 
these  granules  can  be  saved  to  a personal  folder  where  they  can  be  aggregated  using  a lesson  builder 
tool  to  form  lessons  tailored  to  meet  the  users  institutional  requirements  (see  diagram  2).  These  lessons 
can  be  further  customised  by  the  addition  of  a range  of  interface  styles.  Finally,  the  lessons  can  be 
downloaded  in  the  form  of  IMS  content  packages,  which  can  be  incorporated  into  the  users  chosen 
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virtual  learning  environment,  or  alternatively  they  can  be  resubmitted  to  the  electronic  library  where 
they  can  be  re-used  by  others. 

Despite  being  promoted  primarily  as  a resource  for  Scottish  HE  staff  developers,  the  Scottish  electric 
Staff  Development  Library  project  has  generated  a great  deal  of  interest  both  nationally  and 
internationally.  However,  although  an  initial  consultancy  procedure  indicated  overwhelming  support 
for  the  project,  the  staff  development  community  has  shown  some  reluctance  in  submitting  individually 
authored  resources  to  the  electronic  library.  It  seems  that,  while  the  staff  development  community  may 
be  willing  to  reuse  and  share  resources  produced  by  projects  or  expert  authors,  they  seem  less  willing 
to  share  their  own  materials. 

Institutional  adoption  of  these  approaches 

The  reuse  and  adaptation  of  modular  courses,  such  as  the  ELICIT  approach,  is  fairly  traditional.  It 
seems  as  if  careful  consideration  of  key  design  features,  such  as  the  development  of  tasks  which  are 
readily  adapted  for  institutional  use,  may  reduce  the  ‘not  invented  here*  syndrome  and  improve  uptake 
(Haywood  et  al,  1999;  Lee  et  al,  1999).  In  comparison,  the  SeSDL  approach,  the  submission  of  existing 
resources  by  authors  for  reuse  in  granular  format,  is  a relatively  new  method.  It  too  appears  to  eliminate 
the  ‘not  invented  her  syndrome*,  while  reducing  the  problem  of  duplication  of  materials.  However,  this 
approach  may  be  impeded  by  the  reticence  of  staff  to  contribute  resources. 

Therefore,  whilst  the  submission  of  resources  by  authors  in  granular  format  is  a radical  new  approach 
to  institutional  sharing  and  reuse  materials,  present  institutional  culture  and  practice  may  need  to 
evolve  to  encourage  the  adoption  of  this  approach.  The  authors  plan  further  research  to  examine 
factors,  which  will  support  the  adoption  of  multiple  approaches  to  sharing,  and  reuse  of  learning 
resources. 
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WebQuest:  an  Innovative  Way  of  Applying  Information 
Resources  to  Furnish  Performance  Tasks 


I.  Introduction: 

WebQuests  are  designed  to  offer  interdisciplinary  opportunities  that  enhance 
student  skills  of  learning  by  performing  tasks,  and  encourage  collegiality  and  ability  of 
information  processing  in  the  K-12  classroom.  Through  a multimedia  presentation,  the 
systems  and  implementation  of  WebQuest  will  be  shown.  Pros  and  cons  of  WebQuest 
on  both  the  learners  and  teachers  will  be  discussed.  Participants  will  be  facilitated  with 
sites  for  the  research  supporting  WebQuest  in  addition  to  sites  on  how  to  create 
WebQuest. 

II.  Definition  by  Dr.  Bernard  Dodge,  the  father  of  WebQuest: 

A WebQuest  is  an  inquiry-oriented  activity  in  which  some  or  all  of  the  information 
that  learners  interact  with  comes  from  resources  on  the  Internet,  optionally 
supplemented  with  videoconferencing.  (Dodge,  1995) 

III.  Two  Systems  of  WebQuest:  (as  Appendix  I) 
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Appendix  I:  Two  Systems  of  WebQuest: 


Bernard  Dodge 

Tom  March 

Theory 

Some  Thoughts  about  WebQuests 

http://edweb.sdsu.edu/courses/edtec596/ab 
out  webquests.html 

WebQuest:  A Strategy  for  Scaffolding 
Higher  Level  Learning 

http://edweb.sdsu. edu/webquest/necc98.ht 
m 

Why  WebQuest:  An  Introduction 

http://www.ozline.com/webquests/intro.html 

The  Ten  Stage  to  Web  Use  Nirvana 

htto://www.ozline.com/learninq/10staqes.ht 

ml 

Working  with  the  Web  for  Education 

http://www.ozline.com/learninq/theorv.html 

Structure 

Building  Blocks  of  a WebQuest 

http://edweb.sdsu. edu/people/bdodqe/webq 
uest/BuildinqBlocks.html 

Pre-writing  your  WebQuest 

htto://www.ozline.com/webquests/prewrite. 

html 

Process  & 
Taskonomy 

Chart  of  design  process 

http://edweb.sdsu.edU/webquest/Process/W 

ebQuestDesianProcess.html 

Chart  of  design  process 

htto://www.ozline.com/webquests/desiqn.ht 

ml 

Chart  of  taskonomy 

htto://proiects.edtech. sandi.net/staffdev/tps 
s99/tasksimap/ 

Uncovering  the  question/task 

http://www.ozline.com/webquests/question. 

html 

Process 

Process  Checklist 

Design  for  Success  (Designer’s 
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control 

httoi/Zoroiects. edtech.sandi.net/staffdev/tDS 
s99/processchecker.html 

Checklist) 

http://www.ozline.com/webquests/checklist. 

html 

Rubric 

A Rubric  for  Evaluating  WebQuest 

http://edweb.sdsu.edu/webauest/webquestr 

ubric.html 

Assessing  WebQuest 

http://www.ozline.com/webquests/rubric.ht 

rn] 

Template 

Webquest  Templates 

http://edweb.sdsu.edu/webauest/LessonTe 

mplate.html 

WebQuest  T emplate 

http://www.ozline.com/temDlates/webquest. 

html 

WebQuest 

collection 

Matrix  of  WebQuest  Examples 

http://webauest.sdsu.edu/matrix.html 

WebQuest  Collection 

http://edweb.sdsu.edu/webquest/webauest 

collections.htm 

WebQuests  and  other  Student  Activities 

http://www.ozline.com/learninq/index.htm 

About  oz -WebQuests 

http://rite.ed.qut.edu.au/oz- 

teachernet/proiects/webquests/index.html 

oz-WebQuests  search  engine 

httD://www.Da.ash.orq.au/rite/proiects/webq 

uests/forml/enqine.asp 

er|c 
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Designing  Interactive  Multimedia  Learning  Environments 
to  Support  Cognitive  Skills  Development 

Min  Liu 

University  of  Texas  at  Austin 


Most  of  us  will  probably  agree  that  developing  cognitive  skills  is  an  important  goal  for  education, 
and  Is  of  particular  significance  for  life  long  learners.  It  is  an  area  that  requires  continuous  attention  and 
further  research.  In  this  speech,  I will  examine  the  use  of  interactive  multimedia  technology  to  support 
cognitive  skills  development.  Specifically,  I'm  interested  in  examining  how  to  design  interactive  multimedia 
learning  environments  to  provide  necessary  support  for  developing  higher  level  cognitive  skills.  There  are 
many  different  ways  to  create  interactive  multimedia  learning  environments.  Here  I will  look  at  two 
approaches  based  upon  my  research  and  development  experience: 

1.  Engaging  learners  as  multimedia  designers  using  a project-based  learning  approach.  In  this 
environment,  students  at  high,  middle,  and  elementary  schools  take  on  the  role  of  a multimedia  designer, 
learn  state-of-art  multimedia  tools,  and  work  in  a group  to  create  a multimedia  product  for  others  to  use. 
Through  the  process,  they  are  to  acquire  design  skills  and  resource  management  skills  as  needed  in 
producing  a multimedia  product. 

2.  Enhancing  problem  solving  skills  through  a problem-based  multimedia  learning  environment  for  middle 
school  science,  Alien  Rescue.  In  this  environment,  sixth  graders  play  the  role  of  young  scientists,  and  are 
engaged  in  scientific  investigations  aimed  at  finding  solutions  to  complex  and  meaningful  problems.  Alien 
Rescue  provides  a rich  set  of  cognitive  tools  for  successful  use  of  PBL  in  6* -grade  classrooms. 

I will  discuss  the  design,  implementation,  and  research  of  each  environment.  I will  share  the  factors 
contributed  to  successful  implementations  and  challenges  of  designing  such  multimedia  learning 
environments;  and  will  explain  the  cognitive  support  built  in  each  environment.  After  examining  these  two 
approaches  and  reflecting  on  our  experiences,  I believe  the  following  factors  are  of  critical  consideration 
in  designing  such  environments: 

• The  design  of  a multimedia  learning  environment  must  be  solidly  grounded  in  educational 
theories  and  research  practice. 

• Cognitive  tools  should  be  built  in  the  environment  and  they  must  be  included  for  a purpose. 

• High-end  cutting  edge  software  tools  should  be  used  to  create  the  environment. 

• Interactions  among  the  learners,  the  environment,  and  the  teacher/facilitator  must  be  encouraged 
and  promoted  for  the  environment  to  be  effective. 

By  sharing  our  R & D experiences,  and  learning  the  audience’s  experiences  (hopefully  through  the 
question-and-answer  time),  I hope  we,  the  multimedia  community,  will  continue  to  have  our  dialogs  and 
search  for  new  ways  to  design  effective  multimedia  learning  environments  to  support  cognitive  skills 
development  for  learners  at  all  ages. 


The  Challenge  of  Being  an  Instructional  Designer  for  New  Media  Development: 

A View  From  the  Practitioners 


Min  Liu 
Scott  Gibby 
Ondrea  Quiros 
& 

Elaine  Demps 

The  University  of  Texas  at  Austin 


THEORETICAL  PERSPECTIVE 

The  rapid  changes  in  the  field  of  technology  are  redefining  the  process  of  developing 
technology-enhanced  educational  materials  as  well  as  the  roles  of  developers  involved  in  the 
process.  As  educators  whose  responsibility  it  is  to  prepare  future  designers  and  developers  using 
new  media,  and  as  students  who  are  about  to  enter  the  field  of  instructional  design,  we  must 
continually  update  our  knowledge  and  be  aware  of  these  changes.  In  our  previous  research  (Liu, 
Jones  & Hemstreet,  1998),  we  looked  at  the  multimedia  design  process,  and  the  different  roles 
involved  in  the  process  from  a practitioner's  perspective.  In  this  study,  we  chose  to  focus  on  the  role 
of  an  instructional  designer,  one  of  the  key  players  in  new  media  development.  We  looked  again  to 
the  practitioners  in  the  field  to  find  out  how  their  roles  are  defined,  how  they  handle  their  job 
challenges,  and  how  they  adapt  to  the  fast  changes  of  technology  and  market  demands. 

Instructional  design  refers  to  "the  systematic  process  of  translating  principles  of  learning 
and  instruction  into  plans  for  instructional  materials  and  activities"  (Smith  & Ragan,  1993,  p.  2).  An 
instructional  designer’s  task  is  to  plan  the  instruction  so  that  the  student  can  use  cognitive  strategies 
to  learn  the  material  actively  (West,  Farmer,  & Wolff,  1991).  The  term  "instructional  designer"  is 
less  familiar  outside  the  field  of  instructional  technology.  Instead,  one  hears  job  titles  such  as 
industrial  designer,  curriculum  developer,  learning  specialist,  instructional  technologist,  or 
sometimes  just  project  manager.  Yet,  people  in  these  titles  are  often  carrying  the  responsibilities  of 
an  instructional  designer,  especially  if  they  are  involved  in  developing  new  media-based 
instructional  products.  For  some  people,  “instructional  designer”  does  not  say  enough; 
‘“multimedia  producer,’  ‘webmaster,’  ‘developer  of  online  learning’  -these  are  much  more 
muscular  phrases”(Ganzel,  1997,^  14). 

The  reality  that  an  instructional  designer  might  be  called  upon  to  get  involved  in  different 
phases  of  producing  an  educational  product  could  spark  this  confusion.  As  such,  the  instructional 
designer  must  understand  the  needs  and  wants  of  the  client,  the  objective  and  the  audience  of  the 
finished  project,  the  capabilities  of  the  programmer,  graphic  artist,  and  available  tools;  and  must  have 
design  and  project  management  skills  (Liu,  Jones,  & Hemstreet,  1998).  Job  titles  other  than 
instructional  designer  reflect  recent  technological  changes  and  new  trends  for  creating  instructional 
materials.  If  a person  performing  the  role  of  an  instructional  designer  is  developing  online  courses, 
he  or  she  may  be  classified  as  an  online  learning  developer.  Regardless  of  the  job  title,  the  role  of  an 
instructional  designer  “depends  on  the  project,  the  composition  of  the  team,  and  the  skills  of  the 
team  members.  It’s  kind  of  a virtual  job  title  for  a very  real  job”(Jopson  & Smith,  1997,  p.  3). 
Since  the  role  of  an  instructional  designer  has  outgrown  traditional  textbook  definitions,  the 
perspectives  of  practitioners  in  the  field  will  provide  the  information  needed  to  accurately  assess  the 
evolving  responsibilities  of  an  increasingly  popular  position.  Therefore,  we  are  turning  to  the 
practitioners  to  provide  this  information  for  students  who  want  to  enter  the  field. 


PURPOSE  OF  THE  STUDY 


The  purpose  of  this  study  is  to  learn  from  the  practitioners  the  roles  and  responsibilities  of 
an  instructional  designer  in  developing  new  media  enhanced  instructional  materials.  We  are 
particularly  interested  in  the  challenges  designers  are  facing  and  how  they  handle  these  challenges. 
We  hope  the  findings  will  provide  useful  and  practical  information  to  students  who  are  about  to 
take  on  the  responsiWlities  of  an  instructional  designer.  Our  guiding  research  questions  were: 

1.  What  are  instructional  designers'  responsibilities  in  the  field  of  new  media? 

2.  What  challenges  do  instructional  designers  face? 

3.  How  do  instructional  designers  meet  these  challenges? 

4.  What  skills  are  important  for  being  a good  instructional  designer? 

METHOD 

About  Interviewees  and  Procedure 

To  answer  the  research  questions  proposed,  we  interviewed  instructional  designers  working 
at  various  multimedia  companies  in  Austin,  Texas.  Using  the  multimedia  directory  compiled  by  the 
Texas  Governor’s  Office  of  Music,  Film,  Television,  and  Multimedia  Industries,  we  identified  a list 
of  multimedia  companies  in  the  Austin  area.  We  selected  companies  that  produce  multimedia 
educational  or  training  programs,  and  excluded  one-person  companies.  Since  we  were  interested  in 
the  role  of  an  instructional  designer,  we  limited  the  selection  of  interview  subjects  to  those  who 
shoulder  the  responsibilities  of  instructional  designers.  A total  of  eleven  interviews  were  conducted 
with  these  individuals.  These  interviews  ranged  from  45  minutes  to  two  hours. 

All  eleven  interviewees  were  actively  engaged  in  instructional  design  at  the  time  of  the 
interviews.  Eight  were  female  and  three  were  male.  One  designer  had  a doctoral  degree.  Eight 
designers  had  a master's  degree  and  two  had  a bachelor  degree.  Some  had  as  many  as  20  years  of 
experience  in  designing  and  developing  multimedia  products  while  others  had  as  few  as  just  one 
year.  Some  designers  worked  for  large  technology  companies  while  others  worked  for  small 
multimedia  shops.  It  was  our  hope  that  by  looking  at  the  designers  working  in  different  settings,  we 
could  gain  a good  imderstanding  of  the  common  challenges  they  face. 

The  Development  of  the  Interview  Questions 

The  development  of  the  interview  questions  began  with  a survey  to  the  graduate  students 
enrolling  in  the  Instructional  Technology  Program  at  the  University  of  Texas  - Austin.  The  survey's 
purpose  was  to  identity  the  information  and  tips  that  future  instructional  designers  wanted  from  die 
practitioners.  We  also  modified  and  selected  relevant  interview  questions  from  our  previous 
research  (Liu,  Jones,  & Hemstreet,  1998),  which  examined  the  multimedia  design  and  development 
process  from  the  practitioners'  perspective.  Combining  the  two  sources  of  questions,  we  created  and 
refined  the  list  of  interview  questions  for  this  study  and  ensured  they  would  address  the  four 
research  questions.  A total  of  141  interview  questions  were  used.  Part  I of  the  questions  focused  on 
the  role  of  an  instructional  designers  (N=70),  which  was  our  primary  focus.  Part  II  of  the  questions 
(N=7I)  were  used  when  an  instructional  designer  also  performed  another  role,  which  is  often  the 
case  in  the  field.  The  questions  on  instructional  designers  addressed  the  following  ten  aspects:  (1) 
background  information,  (2)  roles  and  responsibilities,  (3)  design  and  production  process,  (4) 
interface,  instructional,  and  interaction  designs,  (5)  formative  evaluation,  (6)  teamwork,  (7)  client,  (8) 
prototype  development,  (9)  personal,  and  (10)  evaluation  of  instructional  designers.  Sample 
questions  included:  “What  are  your  roles  and  main  duties  as  an  instructional  designer?"  "What  are 
some  major  obstacles  you  face  in  doing  your  job?"  and  "What  aspects  of  your  job  do  you  like  the 
most?" 
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Data  Analysis 

The  analysis  of  the  data  followed  the  guidelines  by  Miles  and  Huberman  (1994).  The 
interviews  were  first  transcribed,  then  chunked,  and  coded.  Two  researchers  independently  coded 
the  data.  Codes  were  generated  directly  from  the  data  through  multiple  passes  of  data  examination. 
Two  researchers  then  checked  each  other's  coding  and  resolved  any  disagreement  in  the  coding.  A 
third  researcher  went  through  all  the  data  and  coding  for  any  gaps,  missing  codes  or  inconsistency. 
During  this  process  of  checking  and  rechecking,  the  codes  were  refined,  revised,  and  newly 
developed  as  emerging  themes  were  added.  Patterns  from  the  data  were  extracted,  and  the 
relationships  between  the  coded  segments  were  compared  and  contrasted.  The  data  were  then  sorted 
into  categories  and  sub-categories  according  to  their  common  themes  and  shared  relationships. 

RESULTS  AND  DISCUSSION 

Role  and  Responsibilities  of  an  Instructional  Designer 

Responsibilities.  Our  previous  research,  examining  the  multimedia  design  and 
development  process  from  the  practitioners'  perspective,  showed  that  the  development  process 
consisted  of  six  main  phases:  (a)  funding,  (b)  plamting,  (c)  designing,  (d)  producing,  (e)  testing,  and 
(f)  marketing  (Liu,  Jones,  & Hemstreet,  1998).  The  findings  of  our  current  study  indicated  that  in 
this  entire  process  of  shaping  an  idea  into  a finished  product,  an  instructional  designer  is  heavily 
involved  in  the  phases  of  planning  and  designing.  They  are  also  involved  in  the  phases  of  producing 
and  testing.  Sometimes  designers  may  also  take  on  the  responsibility  of  writing  proposals  and 
seeking  funding.  There  are  four  major  responsibilities  an  instructional  designer  performs:  (1) 
working  with  a client;  (2)  working  with  a subject  matter  expert  (SME);  (3)  working  on  the  design; 
and  (4)  working  with  other  members  in  a team.  An  instructional  designer  is  often  involved  in  the 
project  from  the  beginning.  He  or  she  interacts  with  the  client  to  understand  what  the  client  wants 
and  works  with  the  SMEs  to  understand  the  subject  matter  of  the  materials  to  develop. 

Sometimes  an  instructional  designer,  working  in  another  role  such  as  a project  manager, 
helps  to  secure  a project.  "Generally,  I’ll  get  the  first  contact  with  the  potential  client,  so  there’s  pre- 
work involved  and  just  talking  to  them,  finding  out  what  they  want,  answering  their  questions,  you 
know,  up  to  the  point  where  we  actually  have  a proposal.  ‘This  is  what  we  propose  to  do  for  you’ 
and  they  say  ‘yes.’"  With  a good  understanding  of  the  client's  needs  and  the  contend  an 
instructional  designer  will  develop  a blueprint  to  be  executed  by  other  team  members,  such  as 
programmers,  artists,  and  video/audio  specialists.  Such  a blueprint  comes  in  the  form  of  a design 
document  which  specifies  the  breakdown  of  the  content,  what  will  appear  on  each  screen,  and  what 
media  will  be  used.  In  short,  the  role  of  an  instructional  designer  is  to  translate  the  client's  needs 
into  a plan  that  will  be  used  to  produce  a product  which  meets  the  client's  needs. 

A Typical  Day.  What  a typical  day  is  like  for  an  instructional  designer  working  in  the  field 
of  new  media?  We  asked  the  interview  subjects  this  question.  Although  it  is  clear  that  the  days 
could  vary  from  one  project  to  another,  some  "typical"  tasks  they  do  in  a day  include:  checking 
emails,  having  meetings,  and  working  at  a computer  using  some  software.  Here  are  a few  responses 
to  this  question  that  capture  what  a day  is  like: 

First  thing  I do  in  the  morning  is  check  my  email.  Then  I'll  work  on  a project  or  start  a project  or  maybe 
attend  a meeting.  Around  [the]  midday,  [the]  changes  that  I've  made  I either  upload  to  the  servers  or  I update 
the  system  that  I'm  working  on.  In  the  afternoon  it's  basic  development.  I’ll  do  some  research  on  what  I'm 
doing.  I'll  try  to  collect  documents,  maybe  read  some  material  that's  necessary  for  me  to  do  my  job,  meet 
with  people,  things  like  that. 

First  thing  I do  is  check  email  before  anything  else,  and  usually  voice  mail  at  the  same  time.  I might  have 
some  meetings  based  on  that.  Somebody’s  researching  a project  and  I’ve  worked  on  it  so  I’ve  got  to  get 
them  some  information.  Maybe  there’s  a client  meeting.  Maybe  I’m  working  on  the  proposal  or  a piece  of 
the  design. 
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Work  Environment.  "Fast-paced,"  "collaborative,"  "casual,"  and  "flexible"  are  the 
adjectives  the  practitioners  used  to  describe  their  work  environments.  Most  designers  are  young,  in 
their  20's  and  30's.  Their  work  is  mostly  driven  by  the  projects  and  is  fast-paced.  When  they  are 
working  on  a project  and  facing  a deadline,  they  typically  work  long  hours  as  much  as  60  per  week 
or  more.  A designer  must  be  a team  player  and  a true  collaborator,  as  these  practitioners 
emphasized.  A designer  needs  to  be  able  to  work  with  other  designers  and  team  members  such  as 
programmers,  artists,  and  video/audio  specialists.  The  work  atmosphere  is  casual  in  general;  few 
wear  suits  or  ties.  Some  work  in  a large,  open,  studio-like  space  with  music  playing  in  the 
background.  Others  are  allowed  to  play  games  as  part  of  their  workday  to  relax.  Some  work  in 
physical  offices  while  others  work  in  a virtual  office  meeting  face-to-face  only  when  needed. 

Job  Satisfaction.  What  is  the  reward  for  being  an  instructional  designer?  For  some, 
job  satisfaction  comes  from  the  interaction  with  clients.  "I  love  the  clients,  actually.  I mean  they 
cause  the  most  frustrations  but  I love  them."  For  some,  it  is  the  challenge  of  learning  new  tools  and 
keeping  up  with  the  rapid  changes.  Others  found  it  rewarding  to  be  in  a position  where  they  had  an 
opportunity  to  deal  with  multiple  aspects  of  the  development  process.  For  many,  however,  their 
greatest  gratification  is  in  their  ability  to  be  creative— to  develop  a finished  educational  product  from 
just  an  idea.  One  stated,  "You  have  to  be  creative  every  single  time  to  give  the  clients  [what  they’re 
looking  for].  That’s  a good  spot  to  be  in." 

Approaches  in  Design.  Although  designers  follow  different  models  in  their  work, 

many  emphasized  the  importance  of  "learning  by  doing,"  "providing  experience,"  and  "learning 
from  mistakes"  in  approaching  design.  Some  employ  the  techniques  of  simulations,  scenarios,  and 
storytelling,  while  others  create  games  and  puzzles.  Engaging  a learner  through  various  interaction 
designs  is  emphasized.  "[We]  focus  on  providing  software  that  will  give  the  adults  an  opportunity 
to  interact,  where  they’re  not  just  reading  about  it  and  clicking  on  the  next  button." 

Software  Tools  used.  Students  in  instructional  technology  often  asked  "do  we  need  to 
know  how  to  use  software  tools?  If  we  do,  what  tools  do  we  need  to  learn?"  Our  findings  showed 
that  it  is  not  only  necessary  to  have  some  hands-on  experience  with  popular  software  tools,  but  it  is 
also  important  to  be  proficient  in  a few.  Although  instructional  designers  use  simple  software  such 
as  Microsoft  Word  to  write  design  documents,  it  is  clear  some  designers,  especially  those  who 
graduated  more  recently,  are  also  able  to  use  more  sophisticated  tools  such  as  Macromedia 
Director,  Flash,  Adobe  Photoshop,  Premiere,  Java,  and  HTML . Some  pointed  out  that  knowledge 
of  these  kinds  of  tools  can  help  them  quickly  put  together  a prototype  to  demonstrate  design  ideas 
to  the  client.  "We  have  at  least  one  designer  who  is  very  well  versed  in  Flash,  and  no  matter  what 
she  does,  she's  going  to  put  it  together  in  Flash.  She  puts  together  a model  of  instructional  design 
in  Flash.  I tend  to  be  a Director  guy.  If  I'm  trying  to  say  this  is  what  I want  to  be  animated.  I'll  slap 
it  together. . ..I  can  show  that  to  an  artist  or  show  it  to  an  audience  member  or  learner  and  say  do 
you  understand  this,  does  this  get  the  message  across?"  Others  said  that  such  knowledge  enables 
the  designers  to  participate  in  other  tasks  such  as  programming  or  creating  graphics  when  needed. 
Being  flexible  and  versatile  proves  very  valuable  in  a small  company  when  one  person  often  wears 
multiple  hats  (Liu,  Jones,  & Hemstreet,  1998). 

Challenges  of  Being  an  Instructional  Designer 

Based  upon  the  interviews,  it  appears  that  three  of  the  biggest  challenges  confronting  a new 
media  designer  arise  in  dealing  with  clients,  balancing  multiple  roles,  and  adapting  to  rapid 
technological  changes. 

Working  with  a Client.  An  important  task  of  being  an  instructional  designer  is  to 

determine  what  clients  really  need,  since  they  may  not  be  familiar  with  the  design  process  itself. 
Some  clients  need  assistance  in  producing  a clear  definition  of  the  problem  they  are  trying  to  solve. 
Other  clients  may  not  be  aware  of  the  steps  and  tasks  that  a designer  takes  to  get  to  the  end  product. 
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Some  clients  expect  the  designer  to  start  from  scratch  and  create  a polished  product  within  a short 
time,  without  providing  necessary  input.  "One  of  the  biggest  challenges  is. . . that  the  people. . . only 
want  to  see  the  end  product.  And  they  expect  you  to  develop  something  out  of  nothing.  They 
sometimes  don't  have  the  understanding  of  the  development  time  necessary.. ."  Being  able  to  get 
feedback  from  the  client  and  the  SMEs  about  the  product  throughout  the  design  process  is  a skill 
an  instructional  designer  must  acquire.  Many  clients  new  to  the  field  of  new  media  have  an 
oversimplified  view  of  the  design  process  necessary  to  tailor  a traditional  product  into  a new  media 
concept. 

Many  of  the  designers  we  interviewed  keenly  felt  the  responsibility  of  walking  the  clients 
through  the  process  and  teaching  them  along  the  way.  "You  have  to  continue  to  educate  others  in 
what's  involved  in  doing  this  job,"  noted  one.  This  education  includes  explaining  the  design  and 
production  process,  helping  the  clients  define  the  scope  of  the  problem,  learning  goals,  audience, 
and  outcomes.  A designer  should  be  able  to  ask  a lot  of  questions.  In  addition,  a designer  should  be 
aware  that  the  client  may  not  be  familiar  with  the  industry  jargon.  Designers  should  explain  the 
terminology  to  the  client,  or  simply  use  everyday  language.  When  instructional  designers  speak  of 
objectives,  goals,  entry-level  skills  and  the  like,  clients  often  need  to  hear  these  terms  clarified.  As 
one  designer  stated,  "There’s  some  kind  of  bridge  building  time  where  you  figure  out  what  are  you 
really  saying." 

Another  important  aspect  of  this  educational  process  is  to  help  the  client  make  the  right 
design  decisions  depending  on  a project's  needs.  Clients  sometimes  do  not  understand  the 
difference  between  an  instructional/training  module  and  merely  information  presentation.  They  may 
want  to  use  multimedia  when  there  is  no  need  to  do  so.  A designer  should  be  able  to  discuss 
knowledgeably  the  possibilities  of  various  technologies  and  help  the  client  make  the  appropriate 
choice.  “Sometimes  my  process  in  working  with  a client  is  to  say,  I don’t  think  you  really  need 
training.  I think  you’re  just  talking  about  documentation  here.”  A client  also  needs  to  be  informed 
that  design  decisions  will  have  an  impact  on  the  production.  For  example,  a decision  of  what  media 
to  use  in  the  product  will  affect  the  cost,  and  a change  in  such  a decision  later  in  the  process  may 
increase  the  production  cost. 

When  resources  and  budget  are  limited,  a designer  should  help  the  client  prioritize  what  is 
most  important  for  the  instructional  value  of  the  product  and  not  be  constrained  by  what  media  to 
use.  Although  the  clients  usually  have  the  final  say  in  any  design  decisions  as  they  pay  for  the 
development  and  often  know  the  market  better,  the  designers  should  offer  their  “best  judgment” 
based  on  their  expertise,  and  make  sure  the  clients  understand  their  position,  especially  in  the  case 
of  disagreement. 

Balancing  Multiple  Roles.  Although  many  textbooks  on  instructional  design  often 
separate  the  roles  of  instructional  designer  and  project  manager,  the  practical  experience  of  the 
interview  participants  showed  that  seldom  is  one's  role  limited  to  that  of  a designer.  About  half  of 
the  participants  were  also  project  managers  or  performed  duties  normally  associated  with  that 
position  in  addition  to  being  an  instructional  designer.  One  stated,  “my  heaviest  responsibility  is  in 
. . . getting  teams  up  and  running,  . . . .”  Another  said,  “. . . depending  on  what  the  project  is,  how  big 
it  is.  I’ll  start  assembling  the  people  who  I want  to  work  on  that  one... But  I’ll  continue  to  be  the 
primary  client  contact  as  far  as  letting  them  know  how  it's  going.” 

As  part  of  a team,  designers  are  often  called  upon  to  review  others’  work,  find  clients,  write 
scripts  for  video  and  audio  clips,  write  programming  codes,  write  technical  documents,  create 
animation  and  graphics,  work  on  character  development,  and  train  others.  That  is,  an  instructional 
designer  in  new  media  often  performs  multiple  roles  depending  on  a project's  needs.  Many  smaller 
companies  require  designers  to  have  different  skills  to  maintam  efficiency  and  low  cost  of 
operation.  Some  designers  feel  it  is  important  to  “be  able”  to  do  everything,  even  if  they  do  not  end 
up  doing  it  all.  It  is  interesting  to  note  that  within  all  of  these  responsibilities  lies  a main  objective 
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that  may  tie  all  the  roles  together.  As  one  participant  put  it,  “I’m  trying  to  help  people  understand 
where  we  are  now,  and  where  we  need  to  go,  and  help  arrive  at  a formula  for  getting  there.” 

Adapting  Oneself  to  Technological  Changes.  Rapid  technological  advances  continuously 
bring  changes  and  new  requirements  to  the  field  of  instructional  design.  "[Instructional]  design  is 
going  to  evolve  as  technological  capability  evolves."  One  designer  commented,  "designers  [in  her 
company]  will  need  to  be  extremely  proficient  in  Adobe  Acrobat , some  HTML  and  [Microsoft\ 
Access.  We  will  need  to  understand  technology  on  a higher  level  because  our  business  is  changing 
and  expanding  so  rapidly."  A designer  must  not  only  be  able  to  meet  multiple  responsibilities  and 
perform  different  roles  in  a team,  but  also  keep  abreast  of  these  technological  changes.  He  or  she 
must  stay  very  flexible  in  order  to  adapt  to  changes  quickly  and  continuously  gain  new  skills  to  be 
competitive.  A good  designer  is  a life  long  learner,  who  sees  the  changes,  and  is  willing  to  adjust 
herself  to  the  changes  so  as  to  produce  better  products  for  the  audience.  The  participants  mentioned 
the  challenges  they  faced  daily  in  producing  educational  products  using  new  technological  tools, 
and  the  need  to  stay  on  top  of  the  field.  Their  education  and  experience  prepared  them  to  some 
extent.  However,  as  they  emphasized,  they  must  keep  learning  to  stay  current.  "Really  in  keeping  up 
with  the  field  is  just  hard  work. . .You  just  gotta  read  and  read  and  read,  and  you  have  to  care  about 
it." 

Meeting  the  Challenges 

Staying  on  Top  of  the  Field.  An  important  question  to  these  designers  is  "How  do  you 
address  the  challenges  and  stay  current?"  Their  formal  education  in  instructional  design  equipped 
them  with  some  needed  knowledge  with  hands-on  courses  being  particularly  valuable.  Their  varied 
backgrounds  in  graphic  art,  video  production,  programming,  and  teaching  helped  lay  down  a solid 
foundation  for  being  a good  designer.  Some  pointed  out  that  the  experience  gained  from  working 
on  numerous  projects,  and  performing  different  roles  in  a project  helped  them  learn  to  be  flexible 
and  adapt  quickly  to  new  situations. 

While  on  the  job,  these  practitioners  try  to  keep  up  with  changes  by  taking  additional 
college  classes,  attending  conferences  and  training,  having  informal  meetings  within  their  company 
where  people  share  what  they  have  learned,  studying  products  from  their  competitors,  maintaining 
university  connections  and  involvement,  and  even  learning  from  the  clients. 

Attributes  of  a Good  Instructional  Designer.  The  practitioners  discussed  different 

qualities  a designer  should  possess.  Of  those  mentioned  attributes,  there  is  a consensus  that  a good 
designer  should  be  (1)  a quick  study  who  is  willing  to  learn  new  things;  (2)  a team  player  who  can 
work  with  others  well;  (3)  attentive  to  details;  and  (4)  a good  communicator  both  orally  and  in 
writing.  Apart  from  these,  the  practitioners  mentioned  they  would  also  look  for  people  who  have 
experience  and  are  self  reliant,  resourceful  problem-solvers.  Finally,  they  want  to  hire  individuals 
who  not  only  "know  what  they’re  talking  about,"  but  also  are  "passionate  about  what  they  do." 

Advice  to  Newcomers.  Given  the  challenges  of  being  an  instructional  designer,  what  can 
students  do  to  prepare  themselves  for  the  field  of  instructional  design?  Besides  taking  classes  and 
getting  a degree,  the  designers  offered  the  following  practical  advice: 

• Seek  out  to  gain  a variety  of  experiences. 

• Be  open  to  new  ideas  and  familiar  with  the  capabilities  of  technological  tools. 

• Learn  to  write  well. 

• Enjoy  what  you  do. 


CONCLUSION 


It  is  clear  that  an  instructional  designer  plays  a critical  role  in  developing  new  media-based 
instructional  products.  In  support  of  other  research  (International  Board,  1998;  Le  Maistre,  1998; 


Liang,  1999;  Liu,  Jones,  & Hemstreet,  1998;  Moallem,  1998),  the  findings  of  this  study  identified 
four  essential  competencies  for  being  an  instructional  designer  in  new  media  development: 

1 . Communication:  A good  designer  should  have  excellent  “people”  skills  and  be  able 
to  communicate  effectively  with  clients,  SMEs,  and  other  team  members  both 
verbally  and  in  writing. 

2.  Instructional  design:  A good  designer  should  be  well-versed  in  several  instructional 
design  models  and  strategies  from  which  to  choose  a case-specific  process.  He  or 
she  should  keep  up  with  new  education  or  training  theories  and  research  to  apply 
them  in  the  product  development. 

3.  Problem-solving/decision  making:  The  process  of  developing  a quality  new  media 
product  is  full  of  challenges  along  the  way.  A good  designer  should  be  able  to 
perform  multiple  responsibilities,  step  into  new  roles  when  necessary,  and  overcome 
obstacles  under  a deadline.  A good  designer  is  a problem-solver. 

4.  Knowledge  of  technology  tools:  A good  instructional  designer  should  have  a basic 
knowledge  of  important  software  tools  used  in  the  field  and  be  aware  of  newly 
advanced  tools  as  they  become  available. 

The  job  market  for  instructional  designers  will  continue  to  grow  and  expand,  and  with  it,  the 
definition  of  an  instructional  designer  will  evolve  at  the  same  pace  that  the  technology  changes. 
Since  the  role  of  an  instructional  designer  is  affected  by  the  possibilities  that  new  technologies 
create  daily  and  how  technological  tools  aid  communication,  so  shall  each  instructional  designer 
contribute  to  the  task  of  defining  their  profession.  Therefore,  it  was  to  these  practicing  instructional 
designers  that  this  study  turned  in  order  to  understand  the  evolving  responsibilities,  expectations, 
and  challenges  of  today's  instructional  designers. 
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Abstract:  The  design  and  implementation  of  advanced  tutorware  applications  is  a complex 
issue  requiring  detailed  information  about  the  studied  system,  and  a solid  knowledge  base 
enabling  suitable  tutorship  actions.  In  this  context,  existing  learning  systems  do  not  provide 
suitable  hooks  for  studying  and  reacting  to  information  produced  h the  environment.  This 
restrains  research  work  in  the  area  and  produces  hard-coded  proprietary  applications  involving 
raw  log  analysis. 

Our  approach  is  to  create  a distributed  awareness  infrastructure  providing  standard  hooks  for 
awareness  actuators  and  tutorware  agents  reacting  to  events  in  the  learning  environment.  We 
cover  different  steps  in  the  awareness  life  cycle:  event  persistence,  information  filters,  data 
analysis,  knowledge  base,  and  tutorware  Bots.  In  this  article  we  present  the  overall 
architecture,  outlining  the  services  offered  by  the  platform  and  explain  them  in  an  example  that 
provides  intelligent  knowledge  sequencing  in  course  navigation. 


Introduction 

The  impact  of  computers  and  network  technologies  upon  the  educational  process  has  been 
considerable.  The  possibilities  of  this  new  medium  are  open-ended  and  richer  interactive  scenarios  have  emerged 
under  the  umbrella  of  distance  learning  environments.  In  this  context,  traditional  Teacher  centred  scenarios  has 
been  easily  mapped  to  a computer  medium  in  web  based  learning  systems.  The  traditional  lecture/textbook/exam 
scenario  has  been  broadly  implemented  in  learning  environments  using  Audio  or  Video  broadcast  lectures,  web 
contents,  and  computer  evaluation  tools. 

Whereas  traditional  approaches  have  been  widely  adopted;  richer  interactive  scenarios  involving 
group-centered  design  and  collaborative  learning  are  also  being  supported  and  thus  benefits  from  the 
communication  possibilities  of  network  computing.  All  these  learning  scenarios  have  created  a fruitful  arena  for 
multi-disciplinary  research.  Nevertheless,  proprietary  technologies  and  systems  have  precluded  interoperability 
between  different  environments.  This  has  provoked  proprietary  non-realistic  research  efforts  that  suffer  the  lack 
of  a common  technology  base  for  contrasting  and  comparing  results. 

Fortunately,  standardisation  efforts  are  creating  open  specifications  that  set  the  base  for  future  learning 
environments.  In  this  line,  standardisation  efforts  have  succeed  in  establishing  standards  for  indexing  and 
packaging  contents  (LOM  and  IMS  content  packaging),  and  also  for  defining  evaluation  tools  (QTI). 
Unfortunately,  advanced  topics  like  system  monitoring,  tutoring  systems,  or  collaborative  learning  are  beyond 
the  scope  of  existing  standard  specifications. 

We  aim  to  create  a generic  monitoring  awareness  service  that  considerably  facilitates  data  acquisition  in 
distributed  learning  environments.  The  system  also  provides  standard  hooks  for  creating  filter  agents  and 
information  bots  getting  information  from  the  information  bus  and  reacting  in  different  ways  according  to  their 
task.  The  system  is  based  on  open  technologies  and  provides  standard  hooks  for  obtaining  and  reacting  to 
information  in  the  environment.  Although  we  do  not  aim  to  create  a standard  specification,  the  overall  model 
could  be  mapped  to  create  a standard  distributed  awareness  service.  The  system  is  constructed  on  top  of 
available  free  middleware  and  is  publicly  available  as  an  open  source  project. 

As  a proof  of  concept  implementation,  we  have  created  a prototype  intelligent  tutoring  system  that 
uses  the  Awareness  infrastructure  to  obtain  information  from  the  system  and  react  accordingly  to  events 
produced  in  the  environment  using  tutorware  bots.  In  this  paper,  we  present  AWS:  a generic  distributed 
awareness  service  that  aims  to  provide  a common  base  for  monitoring  applications  and  proactive  applications 
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and  agents  getting  information  from  a common  bus.  In  section  2,  we  define  the  services  provided  by  the 
Awareness  infrastructure;  in  section  3 we  explain  a system  that  benefits  from  the  Awareness  Service:  a 
prototype  intelligent  tutoring  system  extending  standard  platform  hooks.  In  section  4,  we  apply  the  tutoring 
system  to  possible  simulated  scenarios.  Finally,  we  draw  interesting  conclusions  and  future  trends  for  the  ANTS 
Awareness  service. 

ANTS  AWS 

The  Awareness  service  is  a server-side  infrastructure  supplying  the  required  services  to  capture  and 
react  to  information  events  produced  in  a distributed  learning  environment.  ANTS  AWS  is  a key  component  of 
the  ANTS  CSCL  framework  (Garcia,  2001)  and  thus  benefits  from  applications  and  components  developed  for  the 
ANTS  System.  AWS  is  however  application  independent  and  can  thus  work  with  any  learning  environment. 

Based  on  existing  CSCW  Awareness  systems  (Prinz,  1999),  AWS  include  an  event  repository,  a 
notification  service,  and  an  infrastructure  to  filter  or  get  information  from  events  in  the  distributed  bus.  Every 
application  in  the  monitored  system  should  trigger  events  to  the  Information  bus  (Notification  system)  following 
some  standardised  notation.  In  an  ideal  scenario,  all  these  event  fields  should  be  standardised  by  an 
international  committee.  This  would  imply  that  all  chat  tools,  all  question  tools,  or  all  content  navigation  tools 
should  trigger  events  in  a specified  compatible  format.  Monitoring  applications  could  thus  directly  listen  to  the 
information  bus  and  know  what  is  happening  in  any  learning  environment. 

An  initial  attempt  to  achieve  such  standardisation  was  accomplished  by  the  initial  IMS  specification 
and  prototypes  (Instructional  Management  Systems).  IMS  proposed  to  use  the  CORBA  Notification  system, 
and  defined  standard  programmatic  hooks  (Event  Agent)  for  triggering  events  to  IMS  groups.  Furthermore,  it 
also  proposed  an  initial  draft  for  defining  field  notation  for  events  thrown  by  learning  tools.  Unfortunately,  these 
standardisation  efforts  did  not  succeed  and  IMS  spent  more  efforts  in  other  specifications  like  LOM,  QTI,  or 
content  packaging.  Due  to  these  lack  of  standard  initiatives,  our  approach  aims  to  be  open  and  flexible.  We  do 
not  impose  any  special  notation  but  require  that  every  application  must  provide  an  XML  file  containing  event 
information  about  triggered  events  to  the  bus. 

AWS  Services 

The  overall  system  is  based  on  sensors/mediator/actuators.  The  mediator  is  the  central  point  of  the 
server-side  awareness  system  and  is  responsible  of  binding  sensors  and  actuators.  We  consider  two  kind  of 
sensors:  event  sensors  and  time  sensors.  Event  sensors  are  activated  in  response  to  application  events 
produced  in  the  information  bus.  An  event  sensor  is  created  using  a subscription  following  the  constraint  filter 
language  available  in  the  Notification  system.  Time  sensors  are  activated  by  the  ANTS  time  scheduler;  we  can 
create  a time  sensor  for  a specified  date  “Sat  Jun  23  18:31:06  CEST  2001”  or  even  set  repetitive  dates  “all 
mondays  at  16:00”.  The  time  scheduler  notifies  the  awareness  service  when  a time  sensor  has  been  activated. 

Actuators  represent  applications  that  must  be  triggered  in  response  to  AWS  sensors.  We  provide  a set 
of  possible  actuators  including:  Persistent  actuator  (database),  mail  actuator,  Bot  actuator,  EJB  actuator,  CGI 
Actuator,  XMLRPC  Actuator  and  Event  Actuator.  Nevertheless,  the  system  aims  to  be  extensible  and  it  is 
straightforward  to  create  new  actuators. 

In  a possible  scenario,  we  could  create  a subscription  telling  that  we  are  interested  in  events  produced 
by  the  user  called  “pedro”  (event  sensor)  and  we  want  to  make  them  persistent  (actuator).  This  would  log  each 
event  coming  from  user  Pedro  occurred  in  the  environment.  We  could  also  create  our  own  infoBots  that  would 
react  to  events  specified  by  our  subscription. 

The  most  important  aspect  of  the  AWS  is  its  extensibility;  its  set  of  actuators  enables  us  to  create 
different  monitoring  applications  over  the  information  bus.  As  we  will  see,  the  system  is  a powerful  alert  tool  and 
establishes  the  base  for  filter  agents  and  information  bots  getting  information  from  the  bus  and  reacting  in 
different  ways  according  to  their  task. 

From  awareness  to  tutorware 

As  explained  before,  the  ANTS  AWS  infrastructure  supplies  the  required  services  to  create  and  use 
software  actuators  reacting  to  events  produced  in  the  information  bus.  In  this  section,  we  explain  how  we  have 
extended  the  AWS  system  in  order  to  create  a prototype  tutorware  system.  The  knowledge  of  what  is  happening 
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in  a running  environment  is  a hot  issue  that  enables  users  to  interpret  the  context  of  their  own  activities.  Being 
aware  of  user  and  activity  events,  is  the  base  to  perform  proactive  reactions  in  order  to  improve  the  existing 
scenario.  Awareness  information  thus  represents  the  required  feedback  that  facilitates  the  iterative  incremental 
process  of  refining  a running  scenario.  Nevertheless,  raw  data  stored  as  event  logs  or  in  a relational  database  is 
useless  as  is;  it  is  necessary  to  extract  relevant  knowledge  in  order  to  understand  what  is  really  important  and 
what  is  not. 

Intelligent  Tutoring  systems  (ITS)  represent  a challenging  problem  involving  a knowledge-intensive 
process  that  requires  a significant  amount  of  engineering  to  encode  knowledge  into  a usable  form.  This  process 
can  be  quite  complex,  expensive,  and  time-consuming.  In  fact,  the  knowledge  acquisition  bottleneck  is  one  of 
the  major  challenges  for  real-world  AI  systems  like  ITS  (Wenger,  1987). 

We  believe  that  our  overall  awareness  model  and  distributed  services  can  be  used  to  smooth  and 
alleviate  the  knowledge  acquisition  bottleneck  in  many  learning  environments.  Furthermore,  we  have  create  a 
prototype  tutorware  system  extending  AWS  services  that  demonstrates  our  overall  approach. 

Intelligent  Tutoring  Systems 

Whereas  “intelligent”  tutors  may  be  far  to  be  achieved,  machine-learning  techniques  can  be  applied  as 
a useful  tool  for  facilitating  the  knowledge  acquisition  task  to  educational  practitioners.  We  find  essential  the 
combination  of  a domain  expert  and  machine  learning  algorithms  to  create  useful  ITS  systems.  This  kind  of 
supervised  learning  can  devise  interesting  results  difficult  to  achieve  without  these  powerful  tools. 

We  will  follow  the  Wenger’s  structure  of  a classic  ITS  model.  Wenger’s  three  main  components  of  an 
ITS  are:  an  expert  module  that  represents  domain  expertise,  a student  model  that  represents  the  student’s 
knowledge  state  and  expertise,  and  a tutor,  or  pedagogical  module  that  structures  the  interaction  between  the 
tutor  and  the  student  (Wenger,  87).  In  this  article  we  concentrate  on  the  knowledge  acquisition  problem  of 
intelligent  tutoring  systems  by  “pushing  the  frontier”  of  technology  for  distributed  awareness  services.  The 
major  goals  of  our  prototype  tutorware  system  are: 

1)  Benefit  and  extend  AWS  services  in  order  to  receive  and  store  selected  event  information  from  a 

running  learning  environment. 

2)  Use  machine-learning  techniques  to  isolate  key  feature  knowledge  from  the  stored  data. 

3)  Develop  a tool  to  automatically  generate  simulated  scenarios  for  training. 

4)  Implement  a prototype  tutorware  example  that  benefits  from  the  available  infrastructure. 

Information  retrieval:  Using  AWS  subscriptions  the  user  can  specify  which  events  he  is  interested  in, 
and  how  they  should  be  persistently  stored.  AWS  provides  a persistent  EJB  actuator  that  stores  events  in  a 
database.  We  can  thus  specify  that  we  are  interested  in  events  in  a concrete  workspace,  and  that  we  want  those 
events  to  be  stored  using  the  Persistent  Actuator.  From  this  point,  all  events  in  that  workspace  will  be  saved  in 
the  persistent  event  store  transparently  to  the  user. 

Data  Analysis:  Using  data  filters  and  machine  learning  algorithms  (Clustering,  classificators)  the  user 
must  identify  key  variables  that  produce  seamless  groups.  From  the  stored  event  data  is  thus  obtained  a 
knowledge  base  or  trained  algorithms  that  can  onwards  classify  users  from  their  behaviours.  Raw  data  stored  as 
logs  or  in  a relational  database  is  useless  as  is;  it  is  necessary  to  perform  filters  in  the  data  to  retrieve  key 
classification  variables.  We  have  created  data  filters  for  six  grouping  variables: 

1.  Time  connected  to  the  system  or  to  a specific  workspace  for  each  user. 

2.  User  activity  (number  of  events)  in  the  system  or  in  a specific  workspace. 

3.  Tool  activity  (number  of  events  for  this  tool)  in  the  system  or  in  a specific  workspace. 

4.  Thing  activity  (number  of  events  for  this  thing)  in  the  system  or  in  a specific  workspace. 

5.  Used  tools  in  the  system  (total)  or  in  a specific  workspace. 

6.  Navigation  time  (specific  for  the  contents  tool)  or  time  navigating  a specific  course. 

Of  course,  it  is  possible  to  develop  more  filter  variables  depending  on  the  specific  problem  to  study.  As 
possible  filters  to  develop,  we  have  considered  the  possibility  of  specifying  different  strengths  for  events 
triggered  by  a concrete  tool. 
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Figure  1 . Data  Analysis  module 


The  data  analysis  module  receives  two  input  data  sources  and  generates  three  output  files.  The  input 
sources  are  a persistent  event  log  and  a XML  Filter  file  defining  what  filter  variables  should  be  used,  what 
clustering  algorithms  should  be  applied,  the  number  of  resulting  clusters,  and  the  date  range  where  events 
should  be  studied. 

We  are  using  the  freely  available  WEKA  (Witten,  1999)  machine  learning  algorithms  and  tools.  “Weka” 
stands  for  the  Waikato  Environment  for  Knowledge  Analysis  and  provides  Java  APIs  for  accessing  to 
algorithms  as  well  as  a visual  environment  for  studying  and  processing  data  sets.  WEKA  algorithms  require  a 
standard  file  format  called  arff  using  a special  notation;  our  data-analysis  phase  extract  key  variables  found  in 
the  XML  filter  file,  and  generate  a WEKA  compatible  arff  file.  Using  this  file,  we  apply  clustering  algorithms  to 
the  data  set  in  order  to  obtain  student  grouping.  The  results  are  stored  in  the  output  file  with  a comprehensive 
format. 

Output  data  generated  by  clustering  algorithms  give  us  information  about  prior  probability.  Normal 
Distribution  Mean,  and  standard  deviation  for  each  variable  in  every  classified  cluster.  This  is  important  to 
define  what  variables  are  important  in  each  cluster,  and  also  permit  us  to  know  prototype  data  about  existing 
clusters.  We  obtain  information  for  each  cluster  regarding  each  studied  variable,  as  well  as  a correspondence  list 
of  student  to  clusters.  We  can  thus  know  in  which  cluster  every  student  is  located  using  these  filter  variables. 
Finally,  we  can  also  know  what  is  the  prototype  student  in  every  cluster.  A pedagogical  expert  should 
conveniently  study  all  this  information  in  order  to  find  relevant  variables  that  produce  seamless  groups.  Once 
the  expert  find  a seamless  classification,  this  knowledge  can  be  used  to  later  categorise  other  group  of  students 
in  the  same  scenario.  The  third  output  file  generated  by  the  Data  Analysis  module  is  a Java  serialized  trained 
algorithm  that  our  bots  will  use  to  classify  students  in  the  running  learning  environment. 

Data  analysis  is  the  most  important  phase  in  the  overall  model.  If  the  pedagogic  expert  can  choose  the 
key  variables  that  create  seamless  groups,  this  information  can  be  later  used  to  categorise  students  in  the  same 
scenario.  Useful  tutor  reactions  can  thus  be  accomplished  using  this  important  information.  In  a context  where 
massive  information  is  stored  about  a high  number  of  students,  it  is  a hard  task  for  educators  to  study  raw  logs 
even  with  the  help  of  visualization  interfaces.  Machine  learning  algorithms  can  facilitate  this  task  by  telling  the 
educator  what  students  fall  in  a specific  group.  This  information  can  be  later  used  to  perform  appropriate 
reactions  in  order  to  improve  the  learning  scenario. 

Scenario  simulation:  In  order  to  demonstrate  the  usability  of  the  framework  we  have  created  a tool 
that  generates  simulated  event  data.  We  also  tested  AWS  using  real  data  with  the  assessment  of  educational 
experts.  We  aim  to  provide  a technological  infrastructure  that  is  ready  to  be  used  by  pedagogical  experts.  It  is 
thus  an  interesting  research  work  for  educational  experts  to  use  such  tool  in  running  scenarios.  Furthermore,  the 
commercial  EduStance  learning  environment  is  including  AWS  as  its  standard  monitoring  and  knowledge 
analysis  system. 

In  our  case,  we  wanted  to  provide  simulated  tests  as  a proof  of  concept  implementation.  The  tests 
should  incorporate  user  guidance  as  well  as  provide  a random  generation  factor.  Our  approach  is  to  employ  filter 
variables  found  in  the  data  analysis  module  in  order  to  define  the  generated  output  data. 

Tutonvare  example:  intelligent  contents 

Once  we  have  developed  the  required  infrastructure  to  obtain  knowledge  from  the  running 
environment,  our  major  aim  is  to  create  a simple  Tutonvare  sample  that  benefits  from  the  overall  infrastructure. 
Among  other  efforts  to  bring  ITS  to  WWW,  we  have  found  relevant  the  technique  cdWtdi  intelligent  knowledge 
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sequencing  and  problem  sequencing  (Brusilovsky,  1992).  Intelligent  knowledge  sequencing  implies  adaptative 
selection  of  the  next  topic  to  be  learned  using  the  student  model  and  the  knowledge  about  the  learning  material. 
In  hypermedia  systems,  this  technique  is  mapped  in  a straightforward  approach  by  sequencing  hypermedia 
pages  or  hypemodes.  Normally,  the  student  is  provided  with  dynamic  “intelligent  buttons”  that  activate  the 
knowledge  sequencing  mechanism. 

Because  our  approach  is  to  work  with  open  standards,  we  decided  to  benefit  from  IMS  content 
packaging  specification.  More  concretely,  we  have  used  the  IMS  Table  of  contents  (TOC)  that  is  included  in 
every  packaged  content  unit.  In  the  specification  it  is  specified  that  one  content  unit  can  include  several  tables 
of  contents.  These  TOCs  can  be  used  to  create  different  paths  in  a course  and  thus  provide  different 
navigational  alternatives  for  a packaged  material. 

<o  rganizatio  ns  <lef  ault  = "TOCl " > 

<tableo  f CO  nte  nt  s i dent  if  ie  ir=  ” TOC  1 "> 

(...) 

</ table of content s> 

<tableof contents  identif  ie  ir=''TOC2  ”> 

(...) 

> </tableof content s> 

</oirganizations> 

Figure  2.  IMS  Table  of  Contents 

In  general,  when  a course  is  designed  and  developed  by  domain  experts  and  educational  experts, 
different  content  paths  can  be  defined  depending  on  student’s  skills  and  knowledge  state.  In  our  prototype 
tutorware  system,  the  expert  model  of  Wenger  ‘s  categorization  is  comprised  in  the  course’s  table  of  contents. 


Figure  3.  Tutor  Architecture 


The  Student  model  is  obtained  from  the  data  analysis  phase  as  a trained  algorithm,  ready  to  classify 
students  related  to  their  previously  studied  behaviours.  The  Tutor  model  is  in  fact  quite  simple:  every  clustered 
group  correspond  to  a content  path  defined  in  the  course  TOC.  For  each  student,  the  tutor  module  ask  the 
Student  model  in  which  group  is  included,  and  consequently  redirects  the  sequence  to  the  appropriate  path 
found  in  the  IMS  TOC. 

In  order  to  make  the  system  more  realistic  and  broaden  our  test  case,  we  have  created  an  adaptative 
cluster  bot  that  periodically  revises  the  knowledge  base  to  be  always  up-to-date.  If  one  or  several  students 
change  their  behaviours  and  thus  belong  to  another  group,  the  bot  will  redirect  them  to  the  appropriate 
knowledge  sequence.  The  system  is  not  static  and  provides  adaptative  revision  of  the  Student  model.  Finally,  we 
outline  that  AWS  services  have  played  an  essential  role  in  the  Tutorware  sample: 

• In  the  data  acquisition  phase,  binding  an  event  sensor  subscription  with  a persistent  actuator  provides 
the  needed  server-side  logging  mechanisms  for  the  distributed  learning  environment.  In  our  case, 
however,  events  are  generated  by  the  event  simulation  tool  and  directly  stored  in  the  database. 

• In  the  knowledge  acquisition  phase,  the  filter  component  is  a very  important  service  that  pedagogical 
experts  can  use  to  obtain  key  variables  that  construct  seamless  groups.  The  knowledge  acquisition 
phase  can  be  combined  with  the  use  of  visualization  and  data-analysis  tools  such  as  WEKA  data 
exploration  interface. 

• Bot  development  can  also  be  facilitated  using  event  and  time  sensors.  In  our  case,  the  adaptative 
cluster  bot,  must  periodically  reprocess  event  data  in  order  to  decide  to  what  group  belongs  every 
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student.  Using  a repetitive  time  sensor  the  hot  is  triggered  and  can  thus  re-evaluate  knowledge  in  order 
to  classify  users  according  to  their  actions. 

Conclusions 

One  of  the  hot  issues  in  future  learning  environments  will  be  how  to  extract  and  interpret  useful 
knowledge  from  the  running  systems.  This  required  information  feedback  is  the  base  for  system  improvements 
as  well  as  a key  information  source  for  Intelligent  Tutoring  Systems. 

Data  mining  and  machine  learning  algorithms  can  alleviate  the  knowledge  acquisition  bottleneck 
inherent  to  these  complex  environments.  Nevertheless,  such  environments  require  powerful  distributed 
awareness  and  monitoring  services  enabling  fine-grained  monitoring  and  reactions  over  a plethora  of 
heterogeneous  learning  tools. 

We  have  created  a technological  infrastructure  providing  a well-defined  event  repository,  a time 
scheduler,  and  a server-side  Mediator  enabling  to  bind  event  and  time  sensors  with  generic  Actuator  agents.  We 
also  created  a Data  acquisition  component  using  clustering  algorithms  that  permits  to  filter  data  and  produce  a 
suitable  Student  Model.  At  last,  we  have  implemented  a prototype  example  of  a intelligent  kiowledge 
sequencing  ITS  that  extracts  information  from  the  Student  model  and  the  standard  IMS  table  of  contents  to 
dynamically  define  content  paths. 

We  focus  on  the  data  and  knowledge  acquisition  bottleneck  existing  in  nowadays  learning 
environments.  We  provide  a technological  solution  that  can  considerably  facilitate  the  development  of 
monitoring  and  proactive  agents  working  in  the  running  environment.  We  believe  that  such  infrastructure  can 
be  useful  in  a number  of  learning  scenarios.  First  of  all,  alleviating  knowledge  acquisition  over  massive  logs  is  a 
considerable  service  for  educators;  technology  can  help  them  to  identify  groups  or  student  profiles. 
Furthermore,  Intelligent  Tutoring  systems  supported  by  human  expert  intelligence  can  really  improve  and  refine 
learning  scenarios. 

We  are  collaborating  with  commercial  vendors  and  educational  experts  in  order  to  validate  and  use 
these  systems  in  real  life  experiments.  Furthermore,  we  find  interesting  to  include  domain  expertise  and 
pedagogical  information  on  well-defined  data  sources  such  as  LOM  and  IMS  content  packaging.  Tutorware 
bots  can  thus  access  well-defined  information  using  standard  widespread  specifications.  To  conclude,  we  find 
very  interesting  to  apply  machine  learning  techniques  and  tutoring  systems  to  collaborative  learning  scenarios 
with  richer  interactions  and  collaborative  communication  tools  and  interactive  simulations. 
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Case  Study 

The  Human  Touch:  Incorporating  Service-Learning  into  an  Online 

Course 

Can  a student  learn  how  to  build  rapport  with  an  older  adult  through  an  online  course? 
Can  students’  firmly  held  stereotypes  about  what  being  old  is  like  be  affected  by  a web 
site?  Can  a distance  education  student  coordinate  a warm  hug  between  a third  grader  and 
a frail  older  adult  in  an  electronic  chat  room? 

Technology  has  made  access  to  many  types  of  learning  possible.  In  courses  that  focus  on 
human  service,  where  field  experience  has  been  a widely  used  element  in  constructivist 
learning,  the  online  distance  education  student  misses  the  opportunity  to  learn  from  the 
rich,  often  unpredictable  interactions  of  the  physical  and  social  world  of  human  service. 
(Wulff,  Burke,  & Hurley,  2001)  Virtual  reality  and  simulations,  still  technologically 
inaccessible  to  many,  are  also  limited  by  the  perceptions  and  understandings  of  the  author 
and  instructional  designer.  No  one  could  program  the  many  unexpected  challenges  of  12 
students  at  a distance  responsible  for  setting  up  a three  part  service-learning  project  with 
older  adults  in  their  local  communities! 

This  project,  funded  by  the  Corporation  for  National  Service,  Learn  & Serve  Higher 
Education,  through  a grant  to  The  Association  for  Gerontology  in  Higher  Education  in 
partnership  with  Generations  Together/University  of  Pittsburgh  developed  a service- 
learning  model  that  extended  service-learning  to  gerontology  distance  education  students. 
The  project  integrated  a service-learning  component  into  the  online  course  “Therapeutic 
Interventions  with  the  Elderly.” 

http://www.hper.indiana.edu/lehuber/r366r566/service/index.htm 

Service-Learning  is  an  experiential  learning  opportunity  that  integrates  community 
service  into  students’  academic  learning.  Service-learning  gives  students  an  opportunity 
to  apply  disciplinary  practices,  theories,  and  concepts  in  real-world  contexts.  Students  in 
the  community  interact  and  learn  from  people  in  social  service-agencies,  schools,  cultural 
centers,  and  grass  roots  organizations.  (Jacoby,  1996) 

The  services  provided  by  students  in  this  class  included  welcoming  older  adults  to  the 
community,  one  on  one  student/older  adult  interaction,  introduction  of  gerontological 
curriculum  into  an  elementary  school,  an  intergenerational  activity  with  the  older  adults 
at  the  elementary  school,  and  recognition  by  the  community  of  the  uniqueness  and 
potential  contributions  of  its  older  citizens. 

Important  Features 


Important  elements  of  any  service-learning  class  include: 

1 . Service  is  clearly  connected  to  the  academic  component,  treated  as  a text  via 
readings,  discussions,  speakers,  etc. 
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2.  A reciprocal  relationship  between  the  university  and  the  community  makes  each  a 
partner  in  the  education  of  students. 

3.  Service  meets  a genuine  community  need  as  defined  by  the  community-based 
organization. 

4.  The  philanthropic  and  civic  content  of  the  students’  service  is  discussed  and 
examined.  It  is  the  practice  of  citizenship,  broadly  defined,  that  distinguishes  service- 
learning  from  practica  or  internships,  which  focus  more  on  professional  preparation. 
(COPSL,  2000) 

Important  elements  specific  to  this  online  class  include: 

1 . Overcoming  barriers  to  distance  education.  Distance  education  students  frequently 
report  feelings  of  isolation  and  lack  of  connectivity.  While  web-based  courses  offer 
flexibility  for  the  already-employed  or  student  at  a distance,  these  students  miss  out 
on  class  activities  and  field  trips  that  offer  opportunities  for  guided  hands-on  practice 
with  older  adults.  Adding  a structured  service-learning  component  greatly  enhanced 
the  understanding  of  course  concepts  through  application  and  critical  reflection. 

2.  Increased  understanding  of  course  concepts.  This  course  is  about  applying  knowledge 
about  aging  to  therapeutic  activity  intervention  with  older  adults.  This  project  was 
effective  in  taking  students’  understanding  of  the  assessment,  analysis,  planning, 
implementation  and  evaluation  elements  of  therapeutic  activity  intervention  to  a 
deeper,  more  holistic  level.  By  becoming  completely  immersed  in  an  extensive 
project,  students  had  the  opportunity  to  apply  concepts  and  knowledge  from  each 
content  area  of  the  course. 

3.  Increased  understanding  of  aging  and  the  aged.  This  project  offered  the  opportunity  to 
develop  two  different  one-to-one  sustained  relationships  with  older  adults.  Past 
experience  in  this  course,  as  well  as  reports  from  other  faculty,  report  a positive 
affective  experience  from  ongoing  one-to-one  relationships  with  older  adults,  as 
opposed  to  students  coming  in  as  a group  to  offer  a one-time  activity  with  a group  of 
older  adults.  The  rationale  behind  selecting  two  different  older  adults  of  varying 
functional  abilities  was  to  give  students  an  appreciation  of  the  difference  between 
aging  with  disease  as  opposed  to  normal,  or  even  successful  aging.  Students  with 
limited  experience  with  older  adults  often  underestimate  the  individuality  and 
heterogeneity  of  the  older  adult  population. 

4.  Building  competence.  Besides  the  ability  to  work  knowledgeably  with  older  adults, 
practicing  activity  directors  need  to  be  able  to  offer  intergenerational  activities, 
develop  community  partnerships,  and  obtain  media  recognition  for  facility  events. 

This  project  went  beyond  requiring  student  contact  with  older  adults  to  building 
student  competence  in  areas  of  needed  expertise. 

A description  of  the  problem 

Pedagogical  Needs 

In  any  learning  environment,  there  is  a need  to  engage  the  student.  Theoretical  constructs 
in  web-based  learning  have  particularly  focused  on  the  need  for  learning  to  be  problem- 
based,  and  the  importance  of  learner-centered  instruction.  (Bonk  & Cummings,  1998) 
After  a semester  pilot-testing  the  course  “Therapeutic  Interventions  with  the  Elderly,”  it 
was  clear  that  incorporating  principles  of  learner-centered  instruction  would  benefit  what 
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was  learned  and  how  it  was  learned.  Of  particular  importance  in  this  course  which 
focuses  on  human  service  were  building  in  opportunities  to  learn  communication  and 
interaction  skills  with  older  adults. 

Of  Bonk  and  Cummings’  Twelve  Learner-Centered  Web  Recommendations,  the  most 
salient  for  this  course  were: 

• Facilitate  don ’t  dictate.  Students  at  a distance  were  responsible  for  every  aspect  of 
the  service-learning  assignment.  I facilitated  where  possible  with  phone  calls,  letters, 
and  additional  materials.  In  contrast,  two  students  were  local  lU  students.  I had  set  up 
their  project  in  advance  over  the  summer  so  that  I had  a model  to  use  for  trouble 
shooting  community  partner  relationships  at  a distance.  These  two  students  were  the 
least  engaged  in  the  project,  showing  the  importance  facilitation  of  student  ownership 
and  leadership  of  the  project  over  dictating  project  elements. 

• Use  public  and  private  forms  of feedback  Students  posted  all  reflective  journals  and 
analytical  papers  in  public  spaces.  I gave  both  public  and  private  feedback  on 
assignments,  giving  students  a shared  understanding  of  expectations  as  well  as  the 
opportunity  to  learn  from  students  already  employed  in  the  field. 

• Varying  the  forms  of  electronic  mentoring  and  apprenticeship.  Students  learned 
from  each  other,  through  peer  feed  back,  but  also  learned  from  older  adults,  activity 
directors,  family  members,  and  grade  school  teachers.  The  best  lessons  were  those 
learned  from  the  older  adults  with  whom  they  partnered;  achieving  the  ultimate  goal 
of  the  course,  an  appreciation  and  respect  for  the  dignity  of  the  aged. 

• Employing  recursive  assignments  that  built  on  personal  experience.  In  addition  to 
asking  for  personal  memories  of  experiences  related  to  aging,  students  shared  barriers 
and  successes  in  their  common  service-learning  experiences.  All  assignments  asked 
students  to  integrate  and  synthesize  course  concepts  with  personal  experience,  and 
students  learned  from  peer  application  of  theory  to  practice. 

• Varying  the  forms  of  electronic  writing,  reflection,  and  other  pedagogical  activities. 
Students  wrote  journals,  participated  in  asynchronous  discussion  forums,  wrote  press 
releases,  and  one  page  analytical  papers  in  addition  to  service-learning  activities. 

• Provide  clear  expectations  and  prompt  task  structuring.  Before  the  beginning  of  the 
semester,  students  were  mailed  a “Course  in  a Box”  including: 

56  page  booklet  outlining  every  detail  of  the  service-learning  assignments  and 
analytical  papers. 

A video  showing  an  associate  instructor  completing  one  of  the  service- 
learning  assignments  with  an  older  adult 

Community  needs  met  by  the  service-learning  activities  in  this  course 

• Integrate  older  adults  into  the  community.  Bloomington  has  been  identified  as  one  of 
the  top  20  places  to  retire  in  the  US.  In  the  past  3 years,  4 new  residential  facilities 
for  older  adults  have  been  built;  previously  there  was  only  one  such  facility.  Many  of 
the  residents  of  these  facilities  are  new  to  the  area.  Because  they  live  in  a facility, 
many  remain  uninvolved  in  community  citizenry. 

While  many  of  the  communities  of  the  DE  students  might  not  have  experienced  the 
in-migration  of  older  adults  that  Bloomington  has,  older  adults  that  are  in  facilities  are 
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segregated  from  the  greater  community.  Most  experience  a sense  of  isolation  and 
disconnect  from  community  life.  Many  have  few  contacts  with  other  generations.  The 
community  loses  a potential  active,  interested  citizen.  Thus,  the  primary  community 
need  addressed  by  this  project  is  the  integration  of  facility-residing  older  adults  into 
community  life,  benefitting  both  older  adults  and  their  respective  communities. 

• Provide  intergenerational  opportunities  to  three  different  age  groups.  Because  of  changes 
to  the  nuclear  family  and  an  increasingly  mobile  society,  many  adults  and  children 
have  few  opportunities  to  interact  with  those  of  other  generations.  This  project 
provided  interaction  opportunities  to  three  generations:  older  adults,  college  students, 
and  elementary  school  students. 

• Integrate  gerontology  into  grade  school  curriculum.  Today’s  children  are  ill-prepared  to 
live  in  a world  where  1 out  of  every  4 people  are  over  65.  Very  little  material  about 
aging  is  available  in  the  K- 12  curriculum,  and  many  children  hold  misconceptions 

and  stereotypes  about  aging  and  the  aged.  This  project  provided  one  unit  from  a 
series  called  “Positively  Aging”  developed  by  the  University  of  Texas.  The  unit  is 
called  “All  About  Them,”  and  describes  the  diversity  of  the  older  adult  population.  It 
address  stereotypes  and  myths  about  aging,  and  engages  children  in  critical  thinking 
activities  to  develop  their  understanding  of  the  aging  process. 

Who.  What.  When.  Why.  How 

At  the  beginning  of  the  semester,  students  identified  a facility  in  which  older  adults 
reside,  and  working  with  the  Activity  Director,  identified  two  residents  with  whom  to 
work  during  the  semester.  Working  with  Newcomers  Club  or  similar  community  partner 
they  assembled  a “Welcome  Basket”  for  each  resident,  welcoming  and  introducing  the 
resident  to  community  life.  When  they  brought  the  basket  to  the  resident,  they  collected 
a narrative  autobiography  as  they  heard  the  story  of  the  resident’s  life.  On  their  next 
visit,  the  student  and  older  adult  created  a “Visible  Lives”  storyboard,  introducing  the 
resident  to  the  community  as  well  as  to  staff  and  residents  within  the  facility.  “Visible 
Lives”  is  a collage  of  artifacts  from  a person’s  life,  a way  for  a person  to  tell  their  life 
story  as  they  would  like  to  tell  and  share  it.  (Nguyen,  Hill,  & Cole,  2000).  Photos  of  the 
class  project  are  available  at 

http://www. hper. indiana. edu/lehuber/r 3 66r 566/service/pictures  l.htm 

The  resident  was  then  “introduced”  to  the  community  by  participating  in  an 
intergenerational  celebration  at  a local  grade  school  facilitated  by  the  DE  student.  Grade 
school  students  and  teachers  prepared  for  this  activity  through  a curriculum  packet,  “All 
About  Them”  from  the  “Positively  Aging”  curriculum  project 
(http://salud.uthscsa.edu:8900/public/posae2A  which  led  students  to  appreciate  the 
diversity  among  aged  individuals. 

Students  then  prepared  a media  release  for  their  local  media  about  the  project, 
which  frirther  integrated  the  older  adults  into  the  community,  as  well  as  providing  project 
dissemination. 

After  each  activity,  students  used  web-based  communication  to  reflect  on  their 
experiences.  At  the  end  of  the  semester,  the  class  had  a weeklong  asynchronous  class 
“party,”  complete  with  jokes,  snacks,  Christmas  wish  lists  and  reflections  on  the 
experiences  of  the  semester. 
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From  class  party,  12/8/01 

http://ssfindiana.edu/lehuber/dispatch.csi/modulel7/docProfile/100004/d2001 12072033 
5 0/No/t 100004. him 


Joke  punchline:  Dad  says,  Where's  Susie? 

The  little  girl  says,  Susie  ran  out  of  gas  about  halfway  down  the  block  and  there's  another 
dog  pushing  her  home. 

Hope  that  was  not  too  bad,  my  grandfather  told  me  that  one. 

Anyway,  my  favorite  snack  is  cashews  and  I am  eating  them  now. 

My  greatest  barrier  that  I ran  across  while  taking  this  class  was  definately  using  the 
Internet  for  communication  purposes.  I have  had  tremendous  difficulty  with  this  but  thats 
it.  The  class  itself  was  great. 

The  greatest  success  that  I received  from  this  class  was  gaining  new  friends  and  learning 
more  about  myself  and  how  growing  doesn 't  bother  me. 

What  I want  for  Christmas  up  until  yesterday  was  a guitar  but  I received  that  yesterday 
for  my  graduation  present.  Now  I guess  I will  ask  for  a Red  Rider  B-B  GUN!  Don't  worry 
I won't  shoot  my  eye  out.  HA,HA. 

Great  party 

Implications  for  the  local  setting 

Building  a service-learning  component  into  an  online  course  makes  something 
already  time-consuming  (online  teaching)  even  MORE  time-consuming!  At  the 
beginning  of  the  fall  semester  I had  decided  this  type  of  model  to  increase  student 
involvement  in  a human  services  online  course  was  just  too  problematic.  As  the  semester 
ended,  overwhelming  positive  student  feedback  made  me  reconsider.  Over  the  previous 
summer,  I had  developed  substantive  materials  to  guide  students  in  their  service  activities 
and  online  reflections.  I will  continue  to  use  those  materials,  but  give  students  a choice 
about  service-learning.  Students  who  are  already  employed  will  find  this  type  of  project 
useful  in  accomplishing  learning  and  professional  objectives,  and  can  implement  the 
assignment  in  their  place  of  employment.  Some  undergraduates,  themselves  not  well 
integrated  into  their  community,  might  find  it  easier  to  write  a long  paper  than  develop 
community  partners.  From  an  instructional  time  management  point  of  view,  it  might  be 
easier  to  grade  those  long  papers  than  try  to  develop  and  maintain  community  partners  at 
a distance.  From  the  learner’s  perspective,  developing  community  contacts  might  be 
more  frustrating  than  rich  with  learning  opportunities. 

Implications  for  others  outside  the  local  setting 

Service-learning  is  a valuable  concept  for  online  learning  as  well  for  traditional  brick  and 
mortar  learning  enviroiunents.  Human  services  can  hardly  be  adequately  taught  solely 
through  electronic  communication.  Many  courses  involve  a field  experience,  but  that 
places  an  unrepaid  obligation  to  mentor  on  kind  souls  at-a-distance.  Service-learning 
asks  for  mentoring,  but  promises  student  service  in  return. 
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Like  many  ideas  in  instructional  technology,  this  model  program  deserves  consideration 
by  others  inspired  to  provide  the  best  possible  learning  environment  and  opportunities  for 
students.  Also  like  many  ideas  in  instructional  technology,  this  program  comes  with  the 
caveat  that  it  is  time  consuming,  potentially  frustrating,  and  not  guaranteed  to  succeed  for 
each  student. 

The  turning  point  in  my  decision  to  continue  to  include  this  component  came  with  the 
following  email  from  a student:  (http://oncourse.iu.edu/fal2001/oncourse.asp) 

Dr.  Huber,  first  let  me  say  thank  you  for  this  project!  It  was  great!  Both  my  Buddy  and  I 
had  such  a good  experience.  I don 't  know  who 's  more  excited  about  the  outcome,  her  or 
me!  ...Some  of  the  nurses  came  to  me  tonight  to  tell  me  what  a difference  they  had  seen  in 
the  resident  known  as  Buddy  One.  They  had  been  worried  about  her  withdrawal 
associated  with  the  deterioration  of  sight.  They  had  to  tell  me  what  a difference  the 
Visible  Lives  project  made  in  this  Resident,  who  by  their  accounts,  is  now  eager  to  greet 
the  day  and  filled  with  enthusiasm  to  finish  our  projects.  She  talked  a lot  this  past  week 
about  how  she  can  still  do  things  that  matter.  Just  thought  you  might  enjoy  that  tidbit  as 
much  as  I do.  Thanks  again,  Cheryl. 
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Viewing  the  Reviewing 


An  observational  study  of  the  use  of  an  interactive  digital  video  to  help  teach 
the  concepts  of  Design  Inspection  Reviews 

Matthew  Love 

Digital  Media  Research  Centre 
Sheffield  Hallam  University 
Howard  Street,  Sheffield,  England 
m.love@shu.ac.uk 

Abstract:  “Design  Inspection  Reviews”  are  structured  meetings  in  which  participants 
follow  certain  rules  of  procedure  and  behaviour  when  conducting  detailed  readings  of 
design  plans  to  identify  errors  and  misunderstandings.  The  technique  is  widely  used  in  the 
software  engineering  industry,  where  it  is  demonstrably  more  effective  than  testing  at 
identifying  errors  in  software,  but  it  is  by  no  means  restricted  to  this  domain.  Similar 
practices  can  be  found  in  many  other  creative  industries. 

This  paper  reports  on  an  interactive  multimedia  program  used  to  reinforce  the  teaching  of 
the  technique  to  software  engineering  and  computer  network  students.  The  results  of  the 
end-of-studies  examination  papers  of  the  student  group  that  used  the  multimedia  program 
are  compared  to  those  of  the  previous  group.  The  results  appear  to  show  an  improvement 
of  over  7%  for  the  Review  topic  question  for  exams  that  were  otherwise  comparable  in 
standard.  The  quality  of  the  research  data  and  the  validity  of  it's  conclusion  are  discussed. 

Introduction 

A common  saying  is  “prevention  is  better  than  a cure”.  In  the  industrial  design  environment  this  is  certainly  true.  It  is 
known  to  be  far  more  cost  effective  to  take  extra  time  to  identify  problems  at  the  design  stage  rather  than  try  to  work 
around  the  problems  at  the  product  installation  or  user  operational  stages.  For  example,  this  has  been  a central  tenant 
of  the  quality  assurance  movement  of  the  last  two  decades,  as  typified  by  the  ISO  9000  standard. 

The  truth  of  the  saying  has  been  known  to  the  software  development  industry  for  many  years.  Fagan  [1976]  describes 
the  application  of  Design  Inspection  Review  techniques  at  IBM.  Recent  surveys  of  industrial  practice  show  Design 
Reviews  are  today  very  widely  used  for  ensuring  quality  of  software.  Indeed  [Love,  1999]  found  that  the  reviews  are 
the  top-most  used  approach  for  ensuring  quality,  more  widely  adopted  than  internal  quality  audits  and  statistical 
quality  control  procedures. 

Using  lectures  to  teach  the  basics  of  Design  Inspection  Reviews  can  be  ‘dry’  and  for  the  student  a rather  passive 
experience.  This  paper  describes  the  concepts  and  technologies  of  an  interactive  video  that  was  developed  to  allow 
students  to  “chair”  a Review  meeting  for  themselves,  giving  individual  viewers  the  opportunity  to  judge  when  the 
rules  of  the  technique  are  being  broken.  The  author  makes  observations  on  the  success  of  the  use  of  the  interactive 
video,  drawing  upon  data  from  examination  results.  However,  these  are  only  observations  - the  difficulties  of  drawing 
empirical  conclusions  are  noted. 

Although  the  paper  uses  examples  from  software  design  to  illustrate  Design  Inspection  Reviews,  the  principles  of  the 
Review  technique  - and  of  multimedia  program  used  to  teach  it  - can  be  applied  to  many  other  situations. 


Design  Inspection  Reviews 

The  premise  of  the  Design  Inspection  Review  technique  is  that  the  author  / designer  / originator  is  often  too  ‘close’  to 
their  own  piece  of  work  be  able  to  properly  and  objectively  judge  its  fitness  for  purpose  or  technical  qualities.  Instead 
a small  group  of  people  assist  the  author  to  find  all  significant  errors,  omissions,  inconsistencies  and  areas  of 
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confusion  in  the  product  [Kelly,  1993].  Research  has  shown  that  an  effective  Inspection  Review  regime  is  more  than 
paid  back  by  reductions  in  the  subsequent  test-and-correct  phase  [Freedman,  1990]. 

The  Design  Inspection  Review  technique  has  three  central  tenants 

• The  inspection  team  must  be  knowledgeable  about  both  the  product  requirements  and  about  the  tools  and 
methods  used  to  design  it.  They  must  also  have  a vested  interest  in  wanting  to  find  problems  before  the  product 
design  is  released.  Typically  this  means  that  the  team  is  comprised  of  direct  peers  of  the  author,  plus  the  author 
him/herself  Fagan  [1976,  1986]  and  Freedman  & Weinberg  [1990]  recommend  that  anyone  with  line 
management  responsibilities  over  the  other  team  members  should  not  be  present  as  this  could  inhibit  the  open 
discussion  of  problems.  Crawford-Hines  [1996]  dissents,  pointing  out  that  in  industries  such  as  software  where 
seniority  of  position  often  is  based  on  technical  prowess,  an  immediate  supervisor  may  have  valuable  expertise  to 
share. 

• The  Review  is  carried  out  according  to  schedule  typically  comprising  of  six  phases:  planning;  participant 
briefing;  individual  preparation;  inspection  and  defect  recording;  rework;  and,  if  necessary,  re-inspection.  Each 
phase  is  structured  towards  the  efficient  use  of  time  and  towards  the  successful  identification  (rather  than  the 
solving)  of  problems.  For  example,  the  use  of  checklists  of  common  errors  is  strongly  recommended  (together 
with  a department-level  process  to  review  those  lists  to  help  Inspectors  learn  from  other  reviews  about  likely 
sources  of  problems).  To  ensure  breadth  of  consideration,  the  team  members  are  each  allocated  specific  view- 
points from  which  to  review  the  product:  accuracy  of  working,  ergonomics,  and  future  maintainability,  etc. 

• A number  of  conduct  rules  apply  to  the  Inspection  meeting  itself.  Two  examples  are:- 

In  the  months  leading  up  to  the  Review  the  author  will  have  invested  many  hours  in  their  work.  It  represents,  and 
reflects,  their  skills.  At  the  Review  meeting  it  is  perhaps  natural  for  the  author  to  want  to  show  to  their  colleagues 
f the  complex  or  inventive  parts  of  the  solution.  However,  an  author-lead  discussion  may  not  identify  that  the 
author  either  misunderstood  or  completely  overlooked  parts  of  the  requirement  specification.  A rule  of  the 
meeting  is  that  the  Chair  leads  the  discussion  and  the  author  just  responds  to  questions. 

The  product  should  only  be  inspected  when  the  author  believes  it  to  be  defect-free  and  ready  for  release.  It  can  be 
a significant  challenge  to  his/her  ”ego”  if  the  reviewers  do  then  finds  serious  errors.  The  author  may  become 
defensive  and  attribute  problems  to  others.  Alternatively,  they  might  even  counter-attack  and  question  the 
technical  skills  of  the  reviewer.  In  either  case  the  effective,  free- flowing  discussion  of  potential  problems  would 
be  lost.  To  prevent  this  the  Chair  of  the  meeting  must  ensure  that  all  communication  ~ verbal,  and  non-verbal  too 
^ must  suggest  that  “there  is  a problem  with  the  product”  and  not  “there  is  a problem  with  the  producer”. 

This  paper  refers  to  the  technique  under  the  general  title  of  “Design  Inspection  Review”.  Over  the  years  a number  of 
variants  - Fagan  Inspections,  Formal  Technical  Reviews,  Active  Design  Reviews,  Phased  Inspections  - have  been 
proposed.  Interested  readers  are  referred  to  Freedman  and  Weinberg  [1990]  as  the  most  comprehensive  printed  work. 
The  Software  Inspection  and  Review  Organization  web  site  has  useful  information  though  is  a little  dated  [SIRO, 
1996].  The  WWW  Formal  Technical  Review  Archive  [FTRA,  2001]  is  a further  source  of  online  information. 
Trainers  may  find  Powerpoint  presentations  of  Design  Inspection  Reviews  [Johnson,  1998]  useful. 


Previous  approach  to  teaching  Design  Inspection  Reviews 

Students  on  one  of  the  undergraduate  computing  degrees  at  Sheffield  Hall  am  University,  England,  are  taught  a 
version  of  Design  Inspection  Review  based  on  Freedman  & Weinberg.  Students  attend  a lecture  that  describes  the 
principles,  and  (prior  to  the  development  of  a multimedia  teaching  tool)  then  attended  classes  in  groups  of  around 
twenty  where  six  volunteers  would  read  aloud  the  dialogue  of  a “play”  written  by  a course  tutor.  In  the  play  the 
“characters”  break  many  of  the  procedural  and  behavioural  rules  described  in  the  lecture.  All  the  students  - the 
readers  and  the  others  - were  encouraged  take  the  perspective  of  the  Chair  of  the  meeting,  and  to  interrupt  when  they 
identified  transgressions.  The  tutor  led  the  class  in  discussions  as  to  the  likely  consequences.  The  class  would  then 
resuming  with  reading  the  script. 

These  classes  appeared  to  be  popular  with  the  students,  with  lively  discussions  in  most  sessions.  However,  there  were 
some  notable  problems  with  the  format: 
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• Tutors  found  it  hard  to  get  all  students  involved  equally.  Typically,  a few  noisy  or  confident  students  would 
dominate  the  discussions. 

• An  educational  aim  was  that  the  "play"  should  help  inculcate  in  the  students  a sense  of  appropriate  use  of  tone 
and  language  when  making  criticism.  As  with  many  issues  of  chairing  a meeting,  it  is  a matter  of  judgement 
knowing  how  far  to  let  a discussion  run  before  interjecting.  However,  the  first  student  to  interject  pre-empted  the 
other  students  from  making  this  judgement. 

• Students  found  it  hard  to  revise  the  topic  when  preparing  for  end-of-year  examinations.  Perhaps  because  they 
were  actively  involved  in  reading  and  discussing  the  script,  few  took  notes.  Although  the  scripts  themselves  were 
available  for  aiding  their  revision,  if  a student  did  not  pick  up  on  a particular  issue  during  the  class  session  it  is 
believed  unlikely  that  they  would  spot  it  when  re-reading  the  script  weeks  later. 


A modifled  approach  to  teaching:  A multimedia  program 

A multimedia  program  was  created  to  address  the  problems  identified  above.  A re-scripted  version  of  the  Design 
Inspection  Review  play  was  enacted,  videoed,  digitised  and  placed  onto  a CD,  together  with  a media-controlling 
program  to  manage  the  interaction.  The  intent  of  the  program  was  to  permit  individual  students  to  take  the  role  of 
“chair”  of  the  meeting.  They  could  interrupt  the  video  whenever  they  thought  that  one  of  the  Inspection  rules  was 
being  broken  and  then  use  clickable  buttons  to  identify  the  problem.  The  program  would  confirm  the  correctness  or 
otherwise  of  the  interruption,  and  give  further  feedback  on  the  features  of  that  class  of  problem.  An  important  part  of 
the  design  of  the  multimedia  program  was  to  guide  students  towards  noticing  problems  that  they  had  missed  on  the 
first  viewing. 

The  original  concept  for  the  program  interaction  was  to  create  a ‘network’  of  navigation  paths  through  the  video 
presentation.  After  students  had  interrupted  the  video  and  used  the  buttons  and  menus  to  identify  what  rule  they 
thought  was  being  broken,  the  program  would  supply  a little  further  information  (as  text  or  extra  video)  about  the 
consequences  of  this  problem  and  where  to  get  further  information.  It  would  then  restart  with  a different  video  clip 
that  showed  a corrective  action  being  taken.  For  example,  if  the  viewer,  in  their  role  as  “chair”,  indicated  that  the 
language  of  criticism  was  too  harsh  then  the  next  clip  would  start  with  an  unseen  voice  (the  “chair’s”  voice)  asking 
for  more  moderate  tones.  The  subsequent  clip  would  then  progress  towards  a desired  outcome.  However,  if  the 
interjection  was  made  at  an  inappropriate  time  the  next  clip  would  have  a similar  start,  but  the  subsequent  language 
might  be  so  polite  as  to  fail  to  uncover  the  next  defect.  In  this  design,  not  all  the  restart  clips  led  to  desirable 
outcomes. 

The  network  concept  was  not  used  in  the  final  version  of  the  program.  It  turned  out  that  too  many  scripts  were 
required  for  the  time  available  to  the  project.  It  was  also  hard  to  find  “natural”  ways  of  giving  feedback  about  the 
correctness  of  student  interjections,  especially  if  students  selected  faults  that  were  not  intended  to  be  the  primary 
problem  at  that  point.  A further  problem  was  that  the  student  might  undermine  the  value  of  the  package  by  navigating 
a route  that  missed  all  the  important  teaching  points.  Instead  the  program  was  designed  to  reflect  that  Inspection 
meetings  have  six  sub-phases,  and  that  most  errors  are  most  likely  to  occur  in  specific  phases.  For  example,  the 
attendance  of  an  inappropriately  skilled  reviewer  is  likely  to  be  detected  at  the  meeting  start-up  phase. 

In  the  modified  program  design  the  student  were  presented  with  a predefined  sequence  of  six  video  clips  of  between 
one  and  two  minutes.  They  could  interact  to  identify  problems  either  as  they  came  across  them  or  at  the  end  of  the 
clip.  If  they  interacted  during  a clip  the  video  would  be  paused  and  feedback  given  as  above,  but  on  restart  the  same 
clip  would  be  shown.  Students  could  opt  to  see  the  clip  again  from  its  beginning,  or  from  just  before  where  they  left 
off.  To  handle  unexpected  interjections,  the  machine  would  respond  that  “this  problem  can  happen,  but  it  isn’t 
happening  here”.  A central  part  of  the  interaction  was  that  the  program  did  not  tell  the  viewer  whether  or  not  they 
had  identified  all  the  problems  present  in  a clip.  This  was  to  encourage  the  student  to  be  critical  in  their  interactions 
rather  than  routinely  identifying,  say,  two  problems  per  scene,  and  simulates  “real  life”,  where  some  sections  of 
Design  Inspection  Meetings  might  run  smoothly  and  require  little  correction,  whilst  others  might  require  many 
interjections. 
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At  the  end  of  the  whole  program,  the  package  lists  which  errors  had  been  detected  and  which  had  been  missed. 
Students  then  have  the  choice  of  watching  again  extracts  of  clips  that  contained  identified  or  missed  problems.  The 
program  concludes  by  giving  a “score”  based  on  the  percentage  of  problems  identified  on  the  first  viewing  of  the 
scenes.  This  “score”,  though,  is  not  considered  a strong  metric  of  performance  (it  takes  no  account  of  whether  the 
problems  was  spotted  during  the  showing  of  the  clip  or  at  the  end,  for  example)  and  is  not  collected  for  evaluation 
purposes. 


Technical  details  of  the  creation  of  the  Multimedia  product 

A standard  home  VHS-S  video  camera  was  used  to  film  a group  of  students  role-playing  an  inspection  meeting. 
Scenes  from  the  video  were  played  back  through  the  camera  and  saved  in  AVI  digital  format  by  a computer  (a 
Pentium  II  with  256Mb  RAM  and  4 Gbytes  of  unused  disk  space  - a fairly  standard  specification  machine,  but 
equipped  with  a digital  video  capture  card).  Adobe  Premier  software  was  used  to  edit  together  different  ‘takes’  of  the 
same  piece  of  script,  either  to  remove  errors  and  hesitations  in  the  acting  or  to  give  different  camera  angles  such  as 
close-ups  on  individual  speakers.  The  resulting  sections  of  video  film  were  saved  in  Real  format.  This  editing  and  re- 
encoding into  Real  format  reduced  the  storage  size  of  the  clips  from  several  gigabytes  to  a few  hundred  megabytes. 

A computer  program  written  in  Macromedia’s  Director  language  was  used  to  control  the  playing  of  the  video  to  a 
computer  screen.  A series  of  menu  buttons  were  displayed  beside  the  video  (see  figure  2).  Director  allows  both  the 
buttons  and  the  actions  taken  when  they  are  pressed  to  be  varied  by  the  program,  so  the  buttons  could  be  made  to  be 
appropriate  to  the  content  of  the  video  being  shown  at  the  time. 

The  video  clips  and  program  were  written  to  CDs,  which  students  could  borrow  to  watch  on  a home  or  university 
machine.  Alternatively  they  could  download  a copy  across  a network  to  watch  on  a university  machine  (downloading 
across  a standard  modem  to  a home  machine  would  take  prohibitively  long).  The  two  delivery  formats  gave  identical 
results  when  watched.  We  did  not  try  to  stream  the  video  because  with  current  technologies  there  can  be  a significant 
delay  - often  around  20  seconds  - before  sufficient  video  is  received  to  start  the  viewing.  Eventually,  advances  in 
technologies  may  overcome  this  difficulty,  and  allow  on-demand  access  to  interactive  learning  materials  without  any 
administrative  overheads. 

None  of  the  above  technical  steps  were  found  to  be  difficult.  They  were  all  within  the  leaning  capability  of  the 
program’s  author,  a Final  Year  undergraduate  Software  Engineering  student.  It  is  noteworthy,  too,  how  little  attention 
needed  to  be  paid  to  the  filming  environment.  Acceptable  results  were  gained  using  just  the  normal  fluorescent  lights 
of  the  meeting  room  and  the  camera’s  built-in  microphone.  We  did  use  a room  with  carpets  to  soften  the  sound  and 
reduce  the  noise  of  chairs  scraping. 
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Evaluation  of  the  learning  from  the  package 


Learning  from  the  multimedia  teaching  tool,  measured  eight  weeks  after  the  use  the  video,  was  evaluated  by  repeating 
the  same  examination  question  given  to  the  cohort  prior  to  the  use  of  the  video  onto  the  examination  paper  of  the 
cohort  that  did  have  it  available.  This  is  shown  as  Q1  in  the  tables  below.  To  check  that  any  change  in  performance 
levels  was  not  due  just  to  question  familiarity,  a question  from  an  unrelated  topic  was  also  repeated  (Q2).  The  other 
questions  were  from  different  topics  or  gave  a different  emphasis  to  the  same  topic,  and  were  not  directly  comparable. 
This  is  emphasised  in  the  tables  by  labelling  them  Q3-Q7  and  Qiii-Qvii  respectively.  The  two  papers  each  required 
the  students  to  answer  four  questions  from  a choice  of  seven. 

The  same  group  of  tutors  taught  and  examined  the  course,  and  the  teaching  approaches  were  broadly  the  same  except 
for  the  topic  of  Inspection.  The  author  of  this  paper  was  not  a tutor  on  the  course  for  either  year,  do  direct 
experimenter  influence  is  reduced.  It  is  stressed,  however,  that  still  this  was  not  intended  to  be  a strict  experiment  and 
a number  of  important  variables  were  not  controlled.  For  example,  students  in  the  later  year  group  had  access  to  any 
new  literature  published  during  the  later  year.  For  this  reason,  this  research  paper  makes  observational  findings  based 
on  data,  it  does  not  claim  that  these  are  empirically  derived  results. 

Tables  1 and  2 respectively  show  the  examination  results  from  the  year  before  the  use  of  the  video  and  for  the  year  of 
use.  For  each  question  they  show  the  number  of  student  answers,  the  question’s  “popularity”  (i.e.  the  percentage  of 
students  choosing  to  answer  the  question)  and  the  mean  of  the  marks  given  to  the  students’  responses.  Similar  data  is 
shown  for  the  exam  overall,  and  for  overall  but  excluding  the  Inspection  question.  To  make  the  tables  simpler  to 
understand  Q3-Q7  and  Qiii-Qvii  are  presented  in  decreasing  mean-score  order.  The  questions  did  not  appear  in  this 
order  on  the  examination  papers. 


Ql 

Q2 

Q3 

Q4 

Q5 

Q6 

Q7 

Overall 

Overall 

Inspection 

Exam 

Excluding 

Inspection 

NO  of  ANSWERS 

92 

85 

46 

27 

23 

107 

94 

122 

122 

POPULARITY(%) 

19 

18 

10 

6 

5 

22 

20 

MEAN  MARK 

51.2 

55.9 

55.2 

52.4 

49.9 

48.4 

47.6 

51.1 

51.1 

Table  1 Results  for  cohort  previous  to  video 


Ql 

Q2 

Qiii 

Qiv 

Qv 

Qvi 

Qvii 

Overall 

Overall 

Inspection 

Exam 

excluding 

Inspection 

NO  of  ANSWERS 

170 

158 

137 

83 

126 

57 

72 

202 

202 

POPULARITY(%) 

21 

20 

17 

10 

16 

7 

9 

MEAN  MARK 

58.7 

55.7 

52.9 

51.9 

50.7 

48.4 

46.8 

53.4 

51.9 

° Inspection  ■Other  qns 


Previous  "Video" 

Cohort  Cohort 


Table  2 Results  for  cohort  that  had  video  available 


Figure  1 : Comparison  of  results 


Tables  1 & 2 show  that  mean  mark  for  the  Inspection  question  (Ql)  for  the  cohort  before  the  video  was  51 .2  and  was 
58.7  after.  It  can  also  be  seen  that  the  mean  mark  for  the  "control"  question  (Q2)  is  only  0.2%  different,  suggesting 
that  repeating  a question  does  not  itself  explain  the  rise  in  mark  for  Ql.  It  can  further  be  seen  that  the  exam  overall 
excluding  the  Inspection  question  is  only  0.8%  different  between  the  two  cohorts,  suggesting  that  the  two 
examinations  were  of  more-or-less  comparable  levels  of  difficulty.  Hence  it  might  be  concluded  that  the  change  in 
QTs  result  from  5 1 .2  to  58.7  is  due  to  the  change  in  method  of  teaching,  i.e.  due  to  the  introduction  of  the  interactive 
video  CD.  Under  the  United  Kingdom  examination  system,  a 7.5  mark  improvement  represents  not  far  off  a whole 
degree-classification  difference,  and  is  a worthy  achievement. 

(A  less  favourable  interpretation  could  also  be  postulated:  perhaps  it  was  not  the  use  of  the  video  that  lead  to  the 
improvement  in  answers  to  Ql,  but  that  the  previous  use  of  the  “play”  so  confused  the  students  that  their  ’previous- 
cohort’  mark  was  actually  depressed.  The  research  does  not  have  the  evidence  to  refute  this  conclusion.  It  can  only  be 
stated  that  the  professional  judgement  of  the  teaching  team  is  that  this  was  not  the  case.) 

An  examination  of  the  data  was  made  to  see  if  the  profile  of  marks  within  the  Inspection  question  had  changed  for 
those  students  who  declared  that  they  had  watched  the  video.  A total  of  5 1 students  out  of  202  either  told  staff  that 
they  had  downloaded  a copy  or  signed  as  having  borrowed  a CD.  Figure  2 presents  a comparison  of  the  exam  results 
in  graphical  format.  In  the  previous  year  the  score  for  the  Inspection  question,  at  5 1 .2,  was  nearly  equal  to  the  mean  of 
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all  the  other  questions.  The  Students  who  are  known  to 
have  used  the  multimedia  programme  scored  a mean  of 
61.5  for  the  inspection  question,  an  improvement  of  over 
10  marks.  However,  the  “not  watched”  figure  also  raised 
by  5 marks,  to  55.5.  We  consider  it  likely  that  many 
students  watched  the  video  but  did  not  notify  the  staff, 
and  the  “not  watched”  column  should  really  be  labelled 
as  "not  reported  as  having  watched  the  video".  Although 
the  graph  shows  that  students  that  did  watch  the  video 
did  marginally  better  on  their  other  questions  (51.5)  than 
the  non-reporters  (49.1),  this  is  not  considered  to  be  a 
significant  difference. 


Conclusions 

The  “headline”  conclusion  of  this  paper  is  that  eight  weeks  after  teaching  of  a topic  was  supplemented  by  an 
interactive  video,  the  marks  for  the  relevant  question  on  the  examination  paper  were  raised  from  51.2  to  58.7  (and  to 
61.5  for  those  who  explicitly  said  that  they  had  seen  the  video).  This  is  after  discounting  factors  such  as  experimenter 
influence,  differences  in  the  overall  standard  of  the  examination  papers,  or  the  re-use  of  a question  leading  to  students 
being  better  prepared. 

The  author  is  careful  to  state  that  whilst  the  above  conclusion  is  based  on  numerical  data  this  does  not  amount  to 
empirical  evidence,  and  has  pointed  out  many  factors  that  invalidate  the  results  from  being  a correctly-formed 
“scientific  experiment”. 

On  a different  aspect,  a second  finding  of  the  research  was  the  ease  with  which  the  interactive  video  was  conceived 
and  implemented.  The  researchers  were  surprised  how  few  technical  difficulties  were  met  in  the  filming  and  digitising 
of  the  video,  or  in  writing  the  presentational  program.  The  largest  difficulty  was  a pedagogic  issue,  namely  of 
deciding  the  program’s  content. 

The  overall  conclusion  is  that  the  apparent  gain  in  retained  learning  of  over  7%  is  noted,  and  that  the  project  does 
merit  the  design  of  a carefully  structured  empirical  study. 
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Figure  2 - Analysis  of  mean  question  results 


References 

Crawford-Hines 

Software  Inspections  & Technical  Reviews 
http;//www.izdsw.orf/projects/FTR/dogma,  June  1996 

Fagan,  M.E(1976) 

Design  and  code  inspections  to  reduce  errors  in  program  development. 
IBM  Systems  Journal  1976  Volume  3 pp  1 82-21 1 . 

Fagan,  M.E  (1986) 

Advances  in  software  inspections. 

IEEE  Trans  in  Software  Engineering  1986  Volume  7 pp744-751. 

Freedman;  Weinberg 

Handbook  of  Walkthroughs,  Inspections  and  Technical  Reviews. 
Dorset  House  Publishing,  3^^  Edition,  1990 

Kelly,  J 

Inspection  and  Review 

Software  Inspection  and  Review  Organisation  newsletter,  Vol  2,  April  1993 

Johnson,  P 

Link  from  the  WWW  Formal  Technical  Review  Archive 
http://www2.ics/hawaii.edu/'^johnson/FTR,  Jan  1998 

Love,  M;  Siddiqi,J;  Kay,N  Commitment  to  Quality:  A survey  of  South  Yorkshire  software  producers. 

Procs  of  Int.  Conf  on  Software  Process  Improvement  (INSPIRE),  London,  1999. 

Promoting  Student  Learning  through  Peer  Tutoring  - A Case  Study 


Joe  Luca,  Edith  Cowan  University,  Multimedia  Department,  Australia 
j.luca@cowan.edu.au 

Barney  Clarkson,  Edith  Cowan  University,  Multimedia  Department,  Australia 
b.clarkson@cowan.edu.au 


Abstract:  The  literature  abounds  with  information  about  peer  tutoring  and  the  benefits  that  it  can  bring 
to  student  learning.  This  case  study  sought  to  explore  ways  of  using  peer  tutoring  to  enhance  the 
learning  experience  of  a group  of  higher  education  students  in  a multimedia  course,  who  had  access  to 
learning  resources  in  an  on-line  environment.  It  illustrates  how  easily  and  effectively  the  basic 
principles  of  peer  tutoring  can  be  adapted  and  implemented  following  explicit  guidelines  ftom  the 
literature. 


Introduction 

It  is  ironic  that  schools  and  tertiary  institutions  are  often  chided  for  not  providing  a real  world  experience  for 
students,  yet  they  can  provide  a perfectly  realistic  learning  environment  for  students  to  tutor  others.  This  permits 
authentic  practice  of  a number  of  useful  generic  skills  like  working  collaboratively  with  peers,  which  can  enhance 
teamwork  and  interpersonal  skills.  This  study  presents  support  for  using  peer  tutoring  and  peer  assessment  for  students  in 
higher  education.  After  all,  evidence  suggests  that  peer  tutoring  can  greatly  enhance  the  learning  experience  of  both  the 
student  tutor  and  leamer(Goodlad,  1999;  Topping,  1996). 

At  a time  when  there  is  a push  for  higher  education  institutions  “to  do  more  with  less”  and  promote  the 
development  of  students’  generic  skills  (Australian  National  Training  Authority,  1998;  Bennett,  Dunne,  & Carre,  1999; 
Candy,  Crebert,  & O'Leary,  1994;  Dearing,  1997;  Mayer,  1992),  peer  tutoring  can  provide  an  effective  system  which  not 
only  assists  student  tutors  and  tutees  to  learn  better,  but  also  helps  promote  the  development  of  generic  skills,  as  well  as 
freeing  up  time  for  tutors  (Topping,  1996).  This  provides  an  alternative  teaching  and  learning  approach  in  which  students 
take  a pro-active  role  in  thinking,  questioning  and  sharing  knowledge. 

In  this  paper,  we  examine  design  issues  needed  when  implementing  a teaching  program  using  peer  learning,  and 
also  present  the  results  of  our  evaluation.  We  begin  by  considering  some  theoretical  underpinnings  and  design  aspects  of 
implementing  peer  tutoring  and  learning. 

Peer  Tutoring  - theory  and  design  aspects 

The  concept  of  learning  through  peer  tutoring  is  based  on  a social  constructivist  view  of  learning  that 
emphasises  the  role  of  the  students  to  generate  learning  where  students  coach  peers  through  social  interaction  within 
their  zones  of  proximal  development  (Vygotsky,  1978).  Rather  than  applying  a stimulus/response  process,  users  are 
actively  engaged  in  making  meaning  through  cognitive  accommodation  and/or  assimilation  (Piaget,  1969).  Vygotsky 
argued  that  learning  comes  about  through  social  negotiation  within  a cultural  context,  with  language  as  the  primary 
enabling  tool.  This  social  constructivist  philosophy  has  been  expanded  on  recently,  introducing  the  notion  of  cognitive 
apprenticeship  (Brown,  Collins,  & Duguid,  1989)  through  which  students  learn  in  a manner  similar  to  traditional 
apprenticeships.  The  students  access  expertise  through  mentors,  whose  role  is  to  facilitate  rather  than  teach,  and  the  aim 
of  learning  is  to  solve  realistic  and  practical  problems  in  an  authentic  setting.  For  a peer  tutor,  this  setting  is  a very 
realistic  human  setting.  Just  as  in  traditional  apprenticeships,  learners  engage  in  activities  ‘on-the-job’  rather  than 
through  the  didactic  teaching  of  abstract  concepts.  The  argument  is  that  students  are  better  equipped  to  approach  non- 
familiar problems  and  produce  solutions  that  are  appropriate  to  a given  culture.  Peer  tutoring  is  aligned  with  these 
aspects  of  social  constructivist  theory  by  enhancing  social  negotiation  with  the  student  tutor  and  tutee,  where  knowledge 
construction  is  promoted  through  communication  and  dialogue,  which  is  helpful  for  the  tutees. 
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Peer  tutoring  is  also  valuable  for  the  tutor,  ie  “learning  is  enhanced  through  teaching”.  In  an  evaluation  study 
conducted  by  Hartman  (1990),  a reported  outcorhe  of  peer  tutoring  was  an  increase  in  student  motivation  toward 
learning.  These  results  are  supported  by  Whitm^(1982),  Annis  (1983)  and  Benware  & Deci  (1984)  who  argue  that  peer 
tutoring  can  be  the  most  intellectually  rewarding  experience  of  a student’s  career,  and  that  they  perform  better  on  higher 
order  conceptual  understanding  scales  than  students  who  read  the  material  simply  for  study  purposes.  The  benefits  of 
peer  tutoring  are  summarised  by  Goodlad(1999)  as  follows: 

• Student  tutees  found  lessons  more  interesting,  easier  to  follow,  more  enjoyable  and  seemed  to  learn  more; 

• Student  tutors  practiced  communication  skills,  felt  that  they  were  doing  something  useful  with  their  knowledge,  got 
to  know  people  from  different  social  backgrounds,  gained  insights  into  how  other  students  saw  subjects,  increased 
self-confidence  and  reinforced  subject  knowledge; 

• Teachers  found  lessons  easier  to  handle,  teaching  was  more  enjoyable  and  reported  that  pupils  seemed  to  learn  more. 

Implementing  a peer-tutoring  program  is  not  a trivial  process,  as  there  are  many  salient  issues.  Topping  (1996) 
describes  nine  different  peer  tutoring  formats  suiting  different  circumstances  ie  cross-year  small -group  tutoring,  a 
personalised  system  of  instruction,  supplemental  instruction,  same  year  dyadic  fixed -role  tutoring,  same  year  dyadic 
reciprocal  peer  tutoring,  dyadic  cross  year  fixed-role  tutoring,  same  year  group  tutoring,  peer  assisted  writing  and  peer 
assisted  distance  learning.  In  our  case  study,  the  personalised  system  of  instruction  was  akin  to  type  seven,  ie 
implemented  at  the  same  year  group  level,  where  tutors  assisted  tutees  who  were  working  at  their  own  pace  on  set 
exercises. 

Even  though  different  formats  meet  different  needs,  there  is  a commonality  of  purpose  as  well.  Goodlad(1999)  lists 
seven  “golden  rules”  as  criteria  for  designing  and  implementing  peer-tutoring  schemes.  His  criteria  are: 

1.  Clearly  define  the  aims  of  the  tutoring  scheme  by  writing  a statement  of  intent  which  shows  “who  is  teaching  what 
to  whom  and  for  what  purpose”; 

2.  Define  roles  and  responsibilities  in  the  scheme  being  implemented,  which  may  include  rules  for  matching  or  pairing 
students  by  sex,  friendship  or  ethnicity; 

3.  Train  the  tutors  in  task/content  requirements  and  also  in  tutoring  techniques  such  as  “pause,  prompt  and  praise”; 

4.  Structure  the  content  so  that  there  are  clearly  defined,  meaningful  tasks  for  the  tutees  which  involve  maximum 
participation  and  reinforcement; 

5.  Support  the  tutors  with  regular  feedback  through  de-briefing  sessions  and  well  structured  materials; 

6.  Keep  logistics  as  simple  as  possible  ie  make  the  scheduled  time  and  space  for  meetings  convenient  to  all  parties; 

7.  Evaluate  the  scheme. 

Context  of  Study 

“IMM  1 122  - Publishing  on  Web”,  was  designed  as  an  introductory  unit  to  teach  media  students  at  Edith  Cowan 
University  how  to  publish  information  on  the  web.  Students  were  required  to  create  web  pages  that  incorporated 
graphics,  sound  and  animation,  design  suitable  media  for  web  delivery  and  use  a variety  of  web  development  tools  to 
build  well-designed  sites.  An  outline  of  the  weekly  topics  included:  Introduction  to  HTML,  Designing  a Web  Page, 
Tables,  Frames,  Creating  Forms,  Programming  with  Javascript,  WWW  Images,  Bandwidth  and  Compression,  Flash  and 
using  Word,  PowerPoint  and  Acrobat  for  on-line  publishing.  All  notes,  exercises  and  the  syllabus  were  made  available  to 
the  students  through  the  use  of  a web  site. 

There  were  1 10  students  from  a wide  variety  of  backgrounds  in  the  course,  which  comprised  of  a one-hour 
lecture  and  a two -hour  tutorial.  This  required  four  academic  tutors  to  run  five  classes,  with  about  twenty-three  students  in 
each  class.  The  unit  is  an  elective  in  the  “Communications  and  Multimedia”  course,  and  can  be  taken  by  first,  second  or 
third  year  students.  Also,  students  taking  this  unit  are  from  disparate  backgrounds  ie  some  students  in  the  program  have 
the  intention  of  majoring  in  Media  Studies,  Film  & Television  or  Photomedia,  while  others  would  be  majoring  in 
Multimedia,  Computer  Science  or  Information  Systems.  So,  in  the  same  class  it  is  possible  to  have  a third  year  student 
majoring  in  Multimedia  and  Computer  Science  alongside  a first  year  student  majoring  in  Media  Studies  and  Photomedia. 
This  large  discrepancy  in  student  expertise  and  background  created  a great  diversity  of  abilities,  which  suggested 
implementing  a peer-tutoring  program. 
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The  students  were  required  to  complete  three  assessment  tasks.  Two  assessments  were  based  on  developing 
web-based  portfolios,  showing  their  “digital  achievements”  which  could  serve  as  potential  CV’s  for  employment 
opportunities.  The  peer  tutors  were  involved  in  the  third  assessment  block  of  weekly  tasks  which  required  students  to 
complete  ten  tasks  (worth  3 marks  each),  based  on  the  previous  week’s  work.  Academic  tutors  would  then  assess  each 
student’s  work  during  the  tutorial  session,  which  was  a time  intensive  task  and  took  up  most  the  academic  tutors’  time 
during  the  two -hour  tutorial.  Tutors  would  discuss  the  task  with  students  and  give  them  a mark  and  justification. 

Students  could  ask  questions,  but  generally  the  amount  of  time  spent  on  each  student  was  only  five  or  ten  minutes.  While 
this  was  in  progress,  it  was  difficult  for  students  having  problems  to  ask  their  normal  academic  tutor  any  questions. 

So,  given  that  there  was  a great  discrepancy  in  student  abilities,  and  also  that  academic  tutors  were  busy 
assessing  weekly  tasks,  it  seemed  appropriate  to  implement  a peer-tutoring  scheme  in  this  context. 

Implementation 

Within  the  syllabus,  clear  guidelines  were  given  to  students  about  peer  tutor  responsibilities  and  assessment  of 
these.  Four  peer  tutors  were  required  in  each  class,  which  had  20-25  students.  The  scheme  commenced  in  week  three  of 
the  semester,  which  allowed  time  for  nominations,  and  was  based  on  a short  competency  test  given  by  the  academic 
tutors. 

What  was  the  incentive  for  peer  tutors  wanting  to  take  on  this  responsibility?  Simple  - they  were  “bribed”! 
Successful  peer  tutors  would  be  exempt  from  doing  weekly  workshop  activities  (worth  3 marks  each).  However,  to 
remain  as  peer  tutors  they  were  expected  to  obtain  positive  feedback  from  their  peers.  This  was  acquired  through  an  on- 
line database  at  the  end  of  each  tutorial  session.  If  they  obtained  two  consecutive  weeks  of  poor  feedback  they  were 
warned  that  they  could  be  taken  off  the  role.  This  commitment  was  established  through  a “Student  Contract”  which  had 
these  criteria  and  was  signed  by  both  peer  tutor  and  lecturer. 

Goodlad’s  (1999)  “seven  golden  rules”,  were  generally  adhered  to  except  for  training  tutors  in  tutoring 
techniques  and  supporting  tutors  with  regular  feedback.  Over  the  semester,  it  was  found  that  all  the  tutors  across  all  the 
classes  were  keen  to  maintain  their  role  and  worked  hard  at  maintaining  a high  standard  of  delivery.  However,  as  time 
was  always  limited  in  the  tutorial  sessions,  not  much  feedback  was  given  to  the  peer  tutors.  This  was  largely  due  to 
weekly  student  feedback  being  so  positive  in  the  early  weeks,  and  then  gradually  tapering  off. 

Student  Feedback  - Tutees 

Student  tutees  were  asked  to  complete  a post-course  questionnaire,  of  which  thirty-five  were  collected.  Students 
were  asked  to  rate  a range  of  questions  using  a Likert  scale  (0_2_4_6_8_10)  from  0 (low)  to  10  (high).  Key  results  are 
shown  in  Figure  1 and  summarised  as  follows: 

• “I  think  the  peer  tutors  were  successful”  had  an  average  of  6.4,  and  is  clearly  bi-modal.  Students  were  largely 
strongly  positive  or  strongly  negative  about  the  peer-tutoring  program.  By  closely  examining  the  responses,  it 
appeared  that  students  who  were  capable  and  didn’t  need  help  felt  that  peer  tutoring  was  a waste  of  time.  Also,  a 
small  number  of  weaker  students  also  criticised  the  peer  tutors,  who  felt  that  peer  tutors  should  have  done  more  for 
them.  However,  the  majority  of  weaker  students  rated  the  peer  tutoring  system  highly,  and  felt  that  it  supported  their 
learning 

• Students  who  needed  help  rated  the  peer  tutoring  system  highly  ( 1 0 or  close  to  1 0),  whereas  the  more  able  students 
who  didn’t  need  as  much  help  rated  the  peer  tutoring  system  low  (0  or  close  to  0),  and  considered  it  irrelevant  or  of 
little  use 

• “I  think  that  peer  tutors  can  explain  concepts  better  than  academic  tutors”  had  a wide  range  but  an  average  score  of 
5.0,  which  implied  that  there  were  probably  multiple  diverse  feelings  about  this  issue 

• “This  unit  helped  me  learn  the  subject  content”  and  also  “I  would  recommend  this  unit  to  other  students”  both 
gained  high  scores  of  8.0  and  7.3  respectively,  which  implied  that  overall  the  unit  was  perceived  as  being  useful.  We 
attributed  a good  part  of  that  success  to  the  support  provided  by  the  peer  tutoring  system. 
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I think  the  peer  tutors  were  successful 
(Avg=6.4) 


I think  that  peer  tutors  can 
explain  concepts  better  than 
academic  tutors  (Avg=5.0) 


The  unit  helped  me  iearn  the 
subject  content  (Avg=8.0) 


I would  recommend  this  unit  to 
other  students  (Avg=7.3) 


Figure  1 : Results  of  four  questions  from  the  post-questionnaire  survey 


Students  were  also  asked  to  give  written  opinions  about  what  they  thought  about  the  peer  assessment  program. 

Positive  comments  included: 

• Peer  tutoring  was  a good  idea.  Less  wait  for  help  and  they  did  the  same  work  as  us  so  they  knew  the  material 

• Peer  tutors  were  an  effective  concept.  Offers  a less  formal  way  of problem  solving  on  a more  personal  and  intimate 
level.  Also,  it  doesn '/  hinder  the  tutors  marking  of  weekly  tasks 

• they  did  a good  job.  I think  in  helping  others,  they  also  learn  new  stuff,  too.  Very  co-operative  in  helping  to  solve 
problems. . 

• the  peer  tutoring  system  worked  well  for  the  students  who  were  struggling.  Our  peer  tutors  very  patient  and  spent 
time  explaining  things  to  students 

• peer  tutors  are  an  effective  concept.  They  offer  a less  formal  way  of problem  solving  on  a more  personal  and 
intimate  level,  also  it  doesn '/  hinder  the  tutor  *s  marking  of  the  weekly  tasks 

• I think  using  the  peer  tutors  was  excellent,  as  the  tutor  cannot  be  in  two  places  at  once 

• .*>  good,  cool  thing! 

• peer  tutors  are  good  for  a go-between  student  and  tutor.  Sometimes  if  the  peer  tutor  doesn  7 know  the  answer  we 
work  on  it  together  to  solve  it. 


Negative  comments  included: 

• it  would  have  helped  if  the  peer  tutor  ever  came.  First  few  weeks  were  good  but  then  it  dropped  off 
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♦ no-one  knew  who  they  were;  the  system  died  without  a sound.  While  they  did  a good  job,  there  wasn  V any  real 
definition  of  they  were  supposed  to  be  doing,  and  a lot  of  them  were  just  plain  bored! 

♦ They  need  to  do  more  - offer  more  help  to  people 

♦ They  need  to  be  more  prominent.  Also,  maybe  we  should  be  able  to  email  them! 

♦ I found  I did  not  need  much  basic  help.  I tended  to  work  problems  out  myself 

♦ I didn  V use  a peer  tutor,  the  books  were  easy  to  follow 

♦ Good  idea,  but  maybe  need  to  make  it  clearer  who  they  are  and  what  they  are  there  for 

Considering  some  of  the  negative  comments  given  by  the  students,  it  appears  that  Goodlad’s  (1 999)  “seven  golden 
rules”  were  not  applied  as  well  as  we  had  wanted.  Clearly  some  students  were  unsure  about  who  the  peer  tutors  were  and 
what  their  specific  roles  were.  A closer  analysis  of  the  data,  showed  that  these  comments  were  class  dependent,  ie  how 
the  academic  tutor  in  each  class  introduced  and  promoted  the  peer  tutoring  system  probably  affected  its  implementation. 

Student  Feedback  - Peer  Tutors 

Peer  tutors  were  also  asked  to  complete  a post-course  questionnaire.  Ten  responses  were  collected  which 
represented  about  two  thirds  of  the  peer  tutors.  These  students  were  asked  to  rate  questions  using  a Likert  scale  from  0 
(low)  to  10  (high).  The  results  are  shown  in  Table  2. 


Question  Asked 

Average  Score  /lO 

Being  a peer  tutor  helped  me  learn  the  subject  content  ie  HTML,  Flash  etc.. 

10 

Being  a peer  tutor  helped  me  practice  and  develop  communication  & interpersonal  skills 

10 

Using  peer  tutors  would  be  useful  in  other  units 

10 

The  idea  of  assessing  the  peer  tutors  with  an  on-line  database  was  successful 

4 

Table  1:  Average  scores  of  post-course  questionnaire 


Participants  were  also  asked  to  give  written  opinions  about  improving  the  system.  Comments  included: 

• Not  a bad  idea  in  theory,  although  many  people  possibly  didn  7 use  us  enough  and  it  got  boring 

• More  feedback  on  our  performance  would  have  been  useful 

• I didn  *t  know  if  students  were  filling  out  feedback  on  our  efforts,  so  I wasn  7 sure  if  my  help  was  satisfactory? 

• Would  have  liked  more  information  about  how  the  system  worked  before  we  started 

• Have  the  peer  tutor  tied  with  the  same  students  all  semester.  This  way  it  would  be  easier  to  help  them,  as  you  know 
where  they  are  at,  their  history  and  motivation.  This  also  would  help  in  developing  some  trust  between  the  peer  tutor 
and  the  student  which  would  help  smooth  things  out  at  assignment  time  when  both  are  stressed 

• The  tutors  need  more  tasks  to  perform.  The  students  were  not  using  us  enough. 

Clearly,  all  the  peer  tutors  felt  peer  tutoring  had  been  a useful  exercise  and  the  experience  had  helped  them  leant  the 
content  and  develop  better  communication  and  interpersonal  skills.  Also,  they  felt  that  peer  tutoring  should  be 
implementation  across  other  units.  However,  they  were  not  so  content  with  the  level  of  feedback  received,  as  they  were 
unsure  if  they  were  “doing  a good  job”.  Clearly,  it  is  critical  to  keep  tutors  well  informed  of  their  progress  to  maintain 
their  motivation  and  level  of  success.  This  is  stipulated  by  Goodlad  (1999)  in  point  five  “Support  the  tutors  with  regular 
feedback  through  de-briefing  sessions.. 

Conclusions 

Both  peer  tutors  and  tutees,  as  other  studies  predicted,  tended  to  enjoy  the  peer  tutoring  process  and  were 
largely  supportive.  It  is  difficult  to  determine  if  tutor  support  was  based  on  the  “rewards”  attached  to  being  a peer  tutor 
(ie  not  having  to  do  the  weekly  tasks),  or  a case  of  being  intellectually  and  interpersonally  stimulated  by  the  exercise.  To 
avoid  problems  it  seems  evident  that  Goodlad's  (1 999)  principles  are  worth  following  when  implementing  peer-tutoring 
strategies.  Also,  in  larger  implementations  such  as  this,  where  there  were  over  100  students  and  multiple  academic  tutors 
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required,  clear  instructions,  regular  follow-up  and  even  training  sessions  are  needed  to  ensure  an  on-going  peer-tutoring 
implementation.  This  should  reduce  inconsistencies  across  classes,  as  noticed  in  the  student  feedback  where  some 
students  were  unaware  of  the  role  peer  tutors.  In  the  next  implementation,  we  would  be  more  specific  and  procedural 
about  giving  peer  tutors  training  and  support  with  clear  feedback  about  their  progress. 

In  summary  incorporating  peer  tutoring  into  this  tertiary  course  was  not  taxing  when  we  used  the  “seven  golden 
rules”  outlined  by  Goodlad(1999).  Further,  the  rules  appeared  to  be  robust  criteria.  The  results  in  this  case  study  showed 
strong  positive  feedback  for  and  support  from  the  peer  tutors,  high  student  satisfaction  with  the  course,  and,  obviously, 
satisfaction  for  the  academic  staff.  It  demonstrates  how  easily  peer  tutoring  can  be  integrated  with  reasonable  success 
into  an  existing  unit  of  work  with  tertiary  students. 


References 

Annis,  L.  F.  (1983).  The  processes  and  effects  of  peer  tutoring.  Human  Learning,  I0(\\  39-47. 

Australian  National  Training  Authority.  (1998).  Australia’s  National  Strategy  for  Vocational  Education  and  Training 
1998-2003.  Canberra:  Commonwealth  of  Australia. 

Bennett,  N.,  Dunne,  E.,  & Carre,  C.  (1999).  Patterns  of  core  and  generic  skill  provision  in  higher  education.  Higher 
Education,  37{\\  71-93. 

Benware,  C.  A.,  & Deci,  E.  L.  (1984).  Quality  of  learning  with  an  active  versus  passive  motivational  sti.  American 
Educational  Research  Journal,  27(4),  755-765. 

Biggs,  J.  (1999).  Teaching  for  Quality  Learning  at  University.  Buckingham:  Open  University  Press. 

Brown,  J.  S.,  Collins,  A.,  & Duguid,  P.  (1989).  Situated  cognition  and  the  culture  of  learning.  Educational  Researcher, 
75(1),  32-42. 

Candy,  P.,  Crebert,  G.,  & O'Leary,  J.  (1994).  Developing  Lifelong  Learners  through  Undergraduate  Education. 
Canberra:  Australian  Government  Publishing  Service. 

Bearing.  (1997).  Higher  education  in  the  Learning  Society.  London:  HMSO. 

Gibbs,  G.  (1992).  Improving  the  Quality  of  Student  Learning.  Plymouth:  Technical  and  Educational  Services  Ltd. 

Goodlad,  S.  (1999).  Never  Knowingly  Oversold:  a watchword  for  tutoring  and  mentoring  schemes?  Paper  presented  at 
the  2nd  BP  Regional  Conference  on  Tutoring  and  Mentoring,  Perth,  Western  Australia. 

Hartman,  G.  (1990).  Peer  learning  and  support  via  audio-teleconferencing  in  continuing  education  for  nurses.  Distance 
Education,  77(2),  308-319. 

Mayer,  E.  (1992).  Employment-related  key  competencies:  a proposal  for  consultation.  Melbourne:  Australian 
Government  Publishing  Service. 

Moses,  I.,  & Trigwell,  K.  (1993).  Teaching  Quality  and  Quality  of  Learning  in  Professional  Courses.  Canberra: 
Australian  Government  Publishing  Service. 

Piaget,  J.  (1 969).  Science  of  education  and  the  psychology  of  the  child.  New  York:  Viking. 

Ramsden,  P.  (1992).  Learning  to  Teach  in  Higher  education.  London  & New  York:  Routledge. 

Topping,  K.  J.  (1996).  The  effectiveness  of  peer  tutoring  in  further  and  higher  education:  A typology  and  review  of  the 
literature.  Higher  Education,  32,  321  -345. 

Vygotsky,  L.  S.  (1978).  Mind  in  Society.  Cambridge,  MA:  Harvard  University  Press. 

Whitman,  N.  A.  (1982).  Peer  Teaching:  To  Teach  is  To  Learn  Twice.  Washington  DC:  ERIC  Cleaminghouse  on  Higher 
Education. 


1219 


Turning  a Current  Trend  into  a Valuable  Instrument: 
Multidimensional  Educational  Multimedia  based  on  XML 


Ulrike  Lucke,  Djamshid  Tavangarian 
Department  of  Computer  Science,  Chair  for  Computer  Architecture 
University  of  Rostock 
Germany 

ulrike.lucke  | tav  @informatik.uni-rostock.de 


Abstract:  This  paper  gives  an  overview  on  the  work  of  the  joint  project  „Wissenswerkstatt 

Rechensysteme“  (WWR,  engl.:  Knowledge  Factory  for  Computing  Systems),  that  is  done  by 
twelve  German  universities  under  leadership  of  the  University  of  Rostock.  The  project  is  based 
on  an  innovative  and  multidimensional  module  concept.  Its  primary  goal  is  to  offer  a large 
number  of  widely  scalable  teaching  and  learning  modules  in  the  field  of  computer  engineering, 
which  may  be  easily  combined  to  courses.  Moreover,  an  infrastructure  for  the  management  and 
use  of  the  educational  material  is  developed.  The  concept  is  completely  based  on  the  Exten- 
sible Markup  Language  (XML). 


Introduction 

The  integration  of  computers  and  networks  into  education  has  led  to  the  creation  of  numerous  educational  multi- 
media  systems,  reaching  from  isolated  solutions  (Mayer  et  al.  1998)  and  combined  platforms  (Neumann  et  al. 
2001)  up  to  complete  studies  via  internet  (Open  University  2002).  Projects  like  these  are  now  followed  by  several 
current  works,  which  are  trying  to  develop  complex,  flexible,  expandable  and  easily  usable  educational  multimedia 
material  based  on  the  existing  experiences. 

There  are  a lot  of  tools  for  the  realization  of  educational  contents.  For  the  presentation  via  internet,  HTML  and 
dynamical  elements  like  JavaScript  or  media  plug-ins  are  well  suited.  Moreover,  authoring  tools  allow  to  create 
material  with  a high  amount  of  interactivity  or  dynamical  multimedia  components.  Programming  languages  like 
Java  or  C++  are  rarely  used  for  the  implementation  of  educational  software.  A new  possibility  is  the  integration  of 
different  multimedia  elements  into  a XML  description  (Pawlowski  2001)  and  their  transformation  into  different 
output  formats. 

The  goal  of  the  (mostly  expensive)  production  of  multimedia  courseware  is  the  development  of  flexible  platforms, 
which  are  suited  for  several  methods  of  learning.  Within  the  project  „Wissenswerkstatt  Rechensysteme“  (Lucke  et 
al.  2001b),  these  requirements  are  fulfilled  by  use  of  the  Extensible  Markup  Language  (XML). 


Project  Overview 

The  work  was  started  in  April  2001.  Recently,  the  conceptual  phase  has  been  finished,  and  the  implementation  of 
educational  material  and  tools  is  going  on.  The  following  paper  gives  an  overview  on  the  whole  WWR  project:  the 
structure  of  modules  and  courses,  their  implementation,  and  the  architecture  to  develop  and  deploy  this  material. 
Some  further  aspects  of  this  project  are  presented  in  more  detail  in  other  Ed-Media  papers;  the  integration  of  our 
module  concept  and  didactical  aspects  into  existing  XML  description  languages  (Dettlaff-Giinther  et  al.  2002),  the 
criteria  and  results  of  an  evaluation  of  existing  learning  environments  and  content  management  systems  (Wiesner 
et  al.  2002),  and  an  inquiry  of  the  integration  of  external  simulators  and  experiments  into  the  project’s  run  time 
environment  (Kalfa  et  al.  2002). 

Compared  to  other  works,  there  are  mainly  the  following  differences  to  WWR;  The  innovative  concept  of 
multidimensional  modules  simplifies  the  reuse  of  modules,  since  the  extent  of  a module  is  not  statically  defined  by 
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the  author,  but  dynamically  by  the  teacher.  A module  can  be  simply  adapted  to  different  didactical  strategies.  In 
contrast  to  other  projects,  WWR  is  distinguished  by  the  consequent  use  of  XML  technology  and  a very  large  group 
of  developers  and  users  (teachers  as  well  as  learners). 


Concept  of  the  educational  material 

The  content  to  be  implemented  within  the  project  has  been  structured  into  150  modules,  each  for  4 weeks  of  lessons 
and  exercises  or  practical  training.  A module  has  the  following  characteristics  (Lucke  et  al.  2002): 

• It  is  scalable  in  size  respectively  intensity.  That  means,  basic,  advanced  and  expert  versions  can  be  generated 
from  a single  description. 

• It  contains  special  elements  for  learners  (for  example  exercises  and  tests)  and  for  teachers  (for  instance  sample 
solutions  and  didactical  or  pedagogical  instructions). 

• From  an  abstract  description,  several  output  formats  can  be  generated,  e.g.  HTML  for  interactive  use,  PDF  for 
printable  manuscripts  or  PPT  for  a presentation  with  slides. 

In  this  way,  three  dimensions  of  a module  are  achieved,  as  visualized  in  figure  1.  This  leads  to  18  potentially 
different  variants  of  each  module.  For  the  generation  of  a specific  output  from  the  internal  data  representation,  the 
value  of  each  dimension  has  to  be  specified. 


Figure  1 Module  dimensions 

In  order  to  ensure  the  reuse  of  the  developed  modules,  a metadata  set  following  the  IEEE  Learning  Objects  Meta- 
data standard,  LOM  (IEEE  2000)  is  generated  for  each  module.  There  are  35  attributes  defined  as  obligatory, 
because  otherwise  users  are  not  willing  to  fill  out  optional  fields  (Monthienvichienchai  et  al.  2001).  18  attributes 
are  fixed  due  to  the  project  context,  13  can  be  automatically  extracted  from  the  module  description,  and  4 have  to 
be  specified  by  the  author.  Furthermore,  4 attributes  can  be  completed  by  own  values.  The  generation  of  metadata 
is  done  during  the  upload  of  a module  to  the  system. 

Teachers  are  able  to  compose  individual  courses  from  this  modules.  To  define  a course,  a combination  of  modules 
with  their  intensity  (basic,  advanced  or  expert)  has  to  be  specified.  From  this  description,  documents  in  different 
output  formats  and  for  both  target  groups  (teachers  as  well  as  learners)  can  be  generated  (see  figure  2). 
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Figure  2 Generation  of  course  material 
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Additional  information  resources  are  generated  from  the  module  and  course  descriptions,  e.g.  table  of  contents, 
index,  glossary,  list  of  abbreviations,  list  of  cited  persons.  This  is  done  by  examining  appropriately  marked  text 
elements  in  all  modules  and  sorting  them  into  the  corresponding  index. 

Module  implementation 

To  apply  this  high  amount  of  flexibility  to  the  implementation  of  modules,  we  have  chosen  document  processing 
based  on  the  Extensible  Markup  Language,  XML  (Harold  et  al.  2001).  In  contrast  to  other  projects,  XML  is  not 
only  used  for  modelling  of  metadata  or  protocols,  but  for  the  whole  encoding  of  a module.  The  description  of 
documents  is  splitted  into  their  syntactical  structure  (Document  Type  Description,  DTD),  the  contents  (described 
in  XML),  and  its  semantical  interpretation  (described  with  the  Extensible  Stylesheet  Language,  XSL).  Because  of 
the  separation  of  content  and  its  presentation,  several  kinds  of  output  may  be  generated  from  a single  XML  file. 
For  example,  all  elements  can  be  marked  with  attributes  to  specify  the  dimensions  mentioned  above.  Furthermore, 
XML  files  are  completely  independent  from  any  hardware  or  software  environment  and  may  be  handled  easily  in 
web-based  applications. 

The  creation  of  the  XML  description  language  did  not  start  from  scratch.  Existing  languages  have  been  evaluated, 
their  concepts  have  been  partially  integrated  and  combined  with  the  project’s  innovative  concepts,  e.g.  flexible 
definition  of  module  dimensions  and  different  didactical  contexts  (Dettlaff-Gunther  et  al.  2002).  This  allows  a 
conversion  between  the  formats  by  application  of  appropriate  stylesheets. 

The  XML  encoding  of  educational  contents  is  done  by  the  author  of  a module,  using  available  XML  editors  or  spe- 
cial tools.  Since  a lot  of  educational  resources  are  already  existing  in  other  formats,  import  mechanisms  (e.g.  from 
word  processing  software)  are  an  important  topic.  Many  of  the  common  tools  are  able  to  create  XML  files,  which 
need  to  be  converted  into  files  following  the  project’s  DTDs.  Currently,  converters  from  MS  Powerpoint,  LaTeX, 
Adobe  FrameMaker,  and  MS  Word  to  the  WWR  document  format  are  available. 

In  order  to  evaluate  and  to  improve  the  XML  concept,  it  has  been  validated  by  several  test  implementations.  Firstly, 
an  example  module  on  workstation  cluster  architectures  has  been  developed.  It  realizes  all  characteristics  of  the 
DTDs,  for  instance  elements  with  different  intensities,  multimedia  components  with  alternative  definitions  for 
distinct  output  formats,  several  types  of  exercises,  didactical  markings,  and  so  on.  Furthermore,  the  project’s  XML 
working  group  has  provided  a specific  XML  tutorial  as  well  as  the  documentation  of  the  project’s  DTD  set  follow- 
ing the  DTD  itself,  and  gained  a lot  of  useful  hints  from  this.  A step-by-step  transformation  of  the  local  educational 
material  into  this  format  is  currently  going  on  at  all  partner  universities.  In  figure  3,  parts  of  the  source  files  and 
different  output  types  of  a developed  sample  module  are  shown. 


Figure  3 Input  and  output  files  of  a sample  educational  module  (in  German  language) 
a)  XML,  b)  XSL,  c)  online  manuscript,  d)  printable  PDF,  e)  presentation  slides 
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Interesting  aspects  of  XML  and  stylesheet  implementation  are,  for  instance,  the  definition  and  interpretation  of  a 
multimedia  component’s  attributes.  It  is  necessary  to  offer  a conversion  mechanism  between  dynamic  and  static 
media  types  (as  used  in  interactive  and  printable  manuscripts),  e.g.  sound,  video  or  animations.  Another  topic  is 
the  description  of  didactical  intensities  for  the  elements  of  a module,  which  have  a strong  influence  on  the  extent 
of  the  generated  material.  Finally,  the  integration  of  external  programs  like  simulators  is  subject  of  current  work, 
too  (Kalfa  et  al.  2002). 

An  important  point  that  is  often  neglected  is  how  to  enable  authors  to  produce  quality  content.  Within  our  project, 
a special  training  of  involved  authors,  e.g.  for  transformation  of  existing  manuscripts  into  multimedia  scripts,  took 
place.  Additionally,  the  DTD  requires  the  specification  of  important  didactical  information,  like  the  educational 
objectives  or  the  explanation  of  used  terms.  All  universities  regulary  give  reports  on  and  discuss  their  results  in  the 
project  meetings.  Finally,  all  modules  are  evaluated  regarding  to  content,  design  and  didactics  by  other  partners 
(from  universities  as  well  as  external  partners)  before  publication.  This  assures  maximum  quality  of  each  module. 

A more  XML-oriented  description  of  this  project  has  been  given  in  some  further  publications  (Lucke  et  al.  2002). 


System  architecture 

Three  aspects  need  to  be  considered  to  define  the  system  architecture: 

• typical  user  groups  of  the  system 

• the  data  which  needs  to  be  stored 

• the  processing  of  this  data 

Existing  educational  systems  are  distinguishing  between  the  following  types  of  users:  administrator,  author, 
teacher,  tutor,  student,  and  guest.  Thereby,  guests  and  tutors  are  subsets  of  other  roles  with  restricted  permissions, 
and  therefore  they  do  not  need  to  be  considered  for  the  identification  of  architectural  components. 

Regarding  to  the  storage  components,  there  are  two  main  groups  of  data:  Unstructured,  large  objects  like  modules, 
multimedia  elements  or  DTDs  are  better  stored  in  a conventional  file  system;  whereas  smaller  or  fine-grained 
information  like  user  data,  module  metadata  or  course  descriptions  should  be  managed  in  a data  base  system,  in 
order  to  guarantee  fast  access  and  comfortable  search. 

Moreover,  the  following  steps  of  data  processing  can  be  identified  within  such  a system: 

1 . content  production : 

An  author  is  downloading  the  DTD  set  and  additional  instructions.  Based  on  this  information,  he  encodes  his 
educational  content  in  a module  using  a converter  and/or  editor. 

2.  content  upload: 

The  author  is  uploading  the  module  with  referenced  multimedia  elements  on  the  server.  It  is  verified  against 
the  DTDs,  and  the  metadata  is  generated.  Further  evaluation,  e.g.  from  didactical  or  content  point  of  view,  can 
take  place  here. 

3.  course  definition: 

Teachers  identify,  parameterise  and  combine  the  modules  needed  for  their  courses,  based  on  the  metadata 
information.  The  course  description  is  stored  on  the  server. 

4.  course  delivery: 

Teachers  and  learners  are  now  able  to  request  target  group  and  media  specific  course  material,  which  is  gener- 
ated from  the  module  and  course  descriptions. 

5.  feedback  management: 

Answers  to  exercises,  questions  for  teachers,  or  remarks  for  authors  need  to  be  collected  and  redirected  to  their 
destination. 

This  leads  to  an  architecture,  which  is  simplified  depicted  in  figure  4. 


Figure  4 Architecture  of  the  system 


Unfortunately,  we  did  not  find  a platform  that  fits  our  requirements  (Wiesner  et  al.  2002).  For  this  reason,  several 
existing  tools  are  currently  combined  and  integrated  into  an  overall  solution  for  our  project.  Feedback  manager, 
publisher,  composer  and  verifier  can  be  integrated  in  a web-based  interface,  which  is  mainly  realized  using  tools 
like  teaching  and  learning  platforms  or  content  management  systems.  They  need  to  be  adapted  to  the  project 
specifics,  e.g.  DTD  descriptions,  XSL  transformations  or  specifics  of  workflow  management.  Finally,  any  existing 
XML  editor  can  be  used  to  produce  the  modules.  Project  specific  support  for  authors  will  be  given,  too. 

This  architecture  not  only  implements  the  principles  of  content  on  demand.  It  could  also  be  called  format  on 
demand,  because  it  allows  the  online  generation  of  course  material,  which  is  adapted  to  a specific  educational 
intensity,  target  group  and  output  medium.  Furthermore,  the  distributed  architecture  is  able  to  handle  large  amounts 
of  data  as  well  as  users,  which  is  a strong  requirement  in  this  field. 


Conclusion  and  further  work 

Within  the  joint  project  „Wissenswerkstatt  Rechensysteme“,  about  150  multimedia  teaching  and  learning  modules 
for  the  field  of  computer  engineering  will  be  developed.  All  modules  can  be  combined  together  to  generate  specific 
courses,  according  to  given  educational  objectives.  Trough  the  use  of  the  Extendable  Markup  Language  (XML), 
each  module  is  scalable  in  three  dimensions: 

• intensity:  for  basic,  advanced  or  expert  users 

• target  group:  for  teachers  or  learners 

• output  format:  e.g.  HTML,  PDF  or  PPT 

A platform  has  been  presented,  which  will  be  used  for  deployment  of  the  educational  modules.  The  results  will  be 
used  by  all  project  partners.  Furthermore,  the  partners  from  the  industry  and  special  interest  groups  will  evaluate 
the  modules  relating  to  their  content,  design,  and  didactical  aspects.  The  concept  to  define  multimedia  educational 
material  completely  in  XML  in  order  to  achieve  more  flexibility  in  the  dimensions  named  above  is  adaptable  to 
any  fields  of  education. 

Current  work  is  done  on  module  development,  on  filter  implementation,  and  on  the  integration  of  interactive  ele- 
ments like  simulators.  Further  research  will  be  done  on  the  implementation  of  a real-time  extension  for  the  teaching 
and  learning  environment.  The  objectives  of  such  a system  are  to  allow  the  users  to  access  applications  from  one 
or  more  central  servers  via  the  web,  and  to  manage  data  and  applications  shared  between  many  users.  In  this  way, 
cooperative  education  using  the  material  developed  within  the  project  can  take  place  (Lucke  et  al.  2001a). 

All  mechanisms  mentioned  above  are  valuable  instruments  to  further  improve  the  quality  of  education,  not  only  in 
the  field  of  computer  science. 
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Abstract:  Immediate  feedback  and  assessment,  active  and  interactive  learning,  and  timely  changes  to 
adaptively  deliver  the  curricular  objectives  within  an  integrated  environment  promote  effective  learning. 
This  paper  presents  a flexible  framework  to  develop  technology -based  solutions  to  augment  learning 
experiences  by  coordinating  and  synchronizing  instructional  multimedia  streams,  matching  lesson  plans 
to  student  class  experience  and  to  program  requirements. 


Introduction 

There  are  many  different  factors  that  influence  whether  a student  has  successfully  attained  core  program 
knowledge.  Though  the  causes  for  poor  student  performance  are  hard  to  pinpoint,  providing  timely,  factual  information 
to  the  stakeholders  (students,  instructors  and  program  administrators)  creates  a foundation  for  more  productive  learning. 
A comprehensive  assessment  strategy  facilitating  two-way  communication  between  each  stakeholder,  focused  on 
immediate  feedback,  needs  to  be  defined  and  implemented.  This  strategy  should  skew  student  performance  above  the 
desired  threshold,  generating  “A  Significant  Difference”  (Russell  1999). 

It  is  the  instructor’s  responsibility  to  ensure  that  the  course  provides  the  appropriate  content  for  the  student, 
supporting  program  requirements.  Course  learning  outcomes  are  accomplished  through  course  delivery  and  verified 
through  test  evaluation.  Co-relationships  exist  between  the  program-defined  content  and  the  lecture -delivered  content; 
and  the  lecture -delivered  content  and  the  student-received  content.  Unfortunately,  a gap  between  defined,  delivered 
and  received  content  frequently  exis  ts. 

Educational  assessment  has  multiple,  distinct  uses  in  instructional  improvement  including:  school  and  student 
accountability  for  academic  achievement,  feedback  for  teachers,  data  for  administrators  to  allocate  resources,  and 
stimulation  for  students  to  receive  a deeper  understanding  (Baker  1999).  Typically,  student  performance  is  evaluated  at 
irregular  intervals.  Students  who  were  confused  during  lectures  would  not  find  out  how  far  behind  they  were  until  it  was 
too  late  to  catch  up.  Without  timely  feedback  connecting  student  learning  to  instructor  evaluation,  none  of  the 
stakeholders  have  the  power  to  affect  change  or  to  correct  problems. 

A typical  classroom  experience  involves  multiple,  simultaneous  communications  or  streams  of  information. 
These  may  include  verbal  and  visual  presentations,  board  notes,  electronic  exercises,  overhead  presentations,  hyper- 
linked  multimedia  references,  as  well  as  student  questions,  discussion,  and  notes.  Unfortunately,  these  different  material 
formats  may  overwhelm  students,  in  terms  of  absorption  or  proper  note  taking.  Student  disengagement  in  the  classroom 
is  an  issue  that  may  hinder  student’s  learning  in  an  information  overload  situation.  Research  focusing  on  getting  more 
out  of  the  classroom  experience  shows  that  user-interaction  in  selecting  data  keeps  the  students  interested  in  material 
(Hannafin  1996).  According  to  Brien  and  Eastmond  the  ideal  situation  is  one  where  the  learner  knows  the  final  goal  as 
well  as  the  sub-goals  that  support  the  final  goal.  Therefore,  it  is  important  that  learning  outcomes  be  explicit  and 
feedback  frequent  (Brien  1994).  These  studies  indicate  that  students’  needs  and  viewpoints  have  to  be  taken  into 
consideration  when  course  material  is  given  in  technology -based  classrooms. 

Many  classrooms  at  the  University  of  Cincinnati,  like  other  campuses  across  the  nation,  are  being  equipped 
with  the  latest  e-media  technology.  As  students  bring  laptops  into  classrooms,  they  will  have  capability  not  only  for 
note  taking,  but  also  for  interacting  with  the  instructional  system.  These  machines  present  an  excellent  opportunity  for 
enhanced  interaction  via  immediate  student  feedback  and  assessment. 


Approach 

To  facilitate  a connected  learning  environment  for  students,  teachers  and  educational  organizations,  there  is  a 
need  for  a flexible  instructional  infrastructure  that  can:  Augment  both  classroom  and  post-classroom  learning;  Provide  a 


channel  for  student  feedback;  Connect  program  requirements,  course  objectives  and  student  learning  explicitly  so  that 
effective,  adaptive  instruction  can  be  realized. 

The  approach  is  to  electronically  record  instructional  content,  synchronized  with  student  feedback  and 
assessment  data,  providing  real-time  monitoring.  Content  will  be  parsed,  grouped  into  content  units  (CU),  labeled,  and 
indexed.  These  CU  can  be  connected  to  lesson  plan  topics  and  program  requirements,  connecting  delivered  content  to 
defined  content  and  received  content.  Each  stakeholder  will  be  given  the  opportunity  to  mark  or  index  into  the  content. 
As  such,  it  should  become  immediately  apparent  if  a requirement  is  not  supported  or  a topic  not  received.  This  will 
foster  two-way  communication,  facilitate  review,  and  promote  incremental,  constructive  change.  The  intent  is  to  make 
connections  between  stakeholders  explicit,  immediate  and  effective. 

A new  framework,  CaSA  (Classroom  and  Student  Achievement  assessment),  designed  to  develop  new  systems 
and  instructional  software  is  proposed.  CaSA  is  a flexible  framework  to  augment  classroom  experience  by  coordinating 
and  synchronizing  instructional  streams,  matching  class  plans  to  student  class  experience,  and  presenting  instruction  in 
a variety  of  media  forms  to  promote  self-directed  learning.  As  such,  it  will  provide  a foundation  to  facilitate  timely 
feedback  and  collect  assessment  data. 

CaSA  will  synchronize  information  streams,  providing  context  and  valuable  insight  during  assessment, 
instructor  feedback,  and  student  review.  The  strategy  will  take  four  forms:  student  topic  marking,  periodic  electronic 
concept  questions  both  during  and  after  class  sessions,  real-time  feedback  monitoring,  and  instructional  stream  review 
and  evaluation.  By  providing  tools  that  assist  students  in  retrieving,  evaluating,  comprehending  and  memorizing 
information  while  performing  learning  tasks  the  system  will  provide  clear  benefits  (Aroyo,  et  al.,  2001). 


Conclusion 

E-classrooms,  and  CaSA,  provide  a unique  opportunity  to  augment  current  classroom  experience,  aligning  the 
expectations  of  students,  instructors  and  institutions.  Given  a system  where  both  instructional  and  presentational 
aspects  of  a lecture  can  be  evaluated  promptly,  instructors,  students  and  program  administrators  can  react  quickly  when 
there  is  a gap  between  program  requirements,  course  objectives  and  feaming  outcomes,  between  the  intent  of  one 
stakeholder  and  the  understanding  of  another.  As  such,  more  effective  communication  can  occur.  Information  can  be 
gathered  to  measure  student,  faculty  and  organizational  achievement  and  to  assist  in  improvement.  An  e-classroom 
framework  will  improve  instructor  effectiveness  with  prompt  and  iterative  feedback,  giving  them  information  and 
guidance  regarding  the  kinds  of  educational  materials  and  practices  that  most  effectively  enhance  student  achievement. 
It  will  increase  student  engagement,  providing  an  environment  that  produces  demonstrable  gain  in  student  achievement. 
Creation  and  implementation  of  projects  such  as  this  will  demonstrate  the  usefulness  of  technology  in  education, 
especially  in  higher  education  where  significant  technological  resources  are  already  invested.  Using  technology  to  open 
clearly  defined  communication  from  students,  instructors,  and  program  administrators  will  close  the  gap  from  defined, 
delivered  and  received  content  and  make  "A  Significant  Difference". 
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Abstract:  This  presentation  will  describe  and  report  the  findings  from  two  offerings  of  a 
new  online  course  on  "engaged  learning  with  technology"  delivered  to  K12  inservice 
teachers.  The  course  emerged  as  a university-business  partnership  activity.  An  overview  of 
the  partnership,  the  framework  for  course  content  and  design,  and  research  results  will  be 
presented.  Recommendations  for  best  practices  for  assisting  inservice  teachers  in 
developing  and  implementing  technology  -rich  and  engaging  learning  activities  will  be 
offered. 


Background 

School  reformers  hold  that  teaching  and  learning  in  schools  needs  to  be  reconceptualized  from  a new 
paradigm  incorporating  meaningful  and  challenging  content,  higher  order  thinking,  and  project-centered, 
collaborative  activities.  Technology  may  be  a means  for  arriving  at  such  a vision  (Means,  Olson,  & Singh, 
1995).  As  a result  government,  local  school  boards,  and  administrators  infuse  more  technology  into  our 
schools.  By  now  it  is  finally  being  recognized  that  technology  is  not  a "silver  bullet"  and  that  it  can  create 
challenges  for  teachers  expected  to  use  it  in  their  classrooms  (Ertmer  & Hruskocy,  1999;  Means,  Olson,  & 
Singh,  1995).  Schools  will  need  a “next  generation”  professional  development  system  that  uses  the 
technology  to  cost  effectively  engage  teachers  in  ongoing,  everyday  professional  development  tightly  linked 
to  student  learning  (NCREL,  2001). 

Partnership  models  have  evolved  and  been  recommended  as  one  method  of  approaching  solutions  to 
school  innovation,  technology  integration,  and  teacher  training  (Osguthorpe,  Harris,  Harris,  and  Black,  1995). 
In  1998,  a “Partnership  for  Technology  Integration”  was  formed  between  the  College  of  Education  (COE)  at 
Northern  Illinois  University  and  the  Dukane  Corporation.  The  mission  of  the  partnership  was  “To  act  as 
agents  of  change  toward  the  adoption  and  educated  use  of  integrated  instructional  technologies  in  K12 
settings  and  higher  education”.  One  of  the  principle  activities  charted  to  carry  out  this  mission  was  to  develop 
an  online  course  to  assist  inservice  teachers  in  developing  engaging  and  technology  enhanced  lessons.  The 
research  described  here  focused  on  the  lessons  learned  during  the  development  and  first  two  offerings  of 
“Engaging  with  Technology  (EWT):  An  Online  Course.” 


The  Course 

The  goal  of  EWT  is  to  help  teachers  develop  effective  technology-supported  instructional  activities 
that  promote  engaged  and  worthwhile  learning  as  well  as  enhance  student  achievement.  "Engaged  learning"  is 
the  concept  based  on  the  North  Central  Regional  Education  Lab's  (1998)  promotion  for  "Learning  with 
Technology."  Using  the  NCREL  materials  as  a starting  point,  online  activities  were  designed  to  include 
individual,  small  and  large  group -based  instructional  strategies.  These  strategies  included  Student 
HomePages,  Private  Discussion  Areas  for  Peer  Feedback,  WebQuests,  Synchronous  Role  Play  and 
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Discussion  Activities,  Streaming  Video  and  Print  Case  Studies,  Existing  Web-based  Tutorials,  and  Threaded 
Discussion  Brainstorming  and  Debriefing  Activities. 


Methodology 

The  nature  of  this  inquiry  was  collaborative  and  formative.  Data  sources  included  Engaged 
Learning  Plans,  online  discussion  (synchronous  and  asynchronous)  transcripts,  participant  autobiographies, 
interviews,  field  notes,  researcher  debriefings.  Each  of  these  was  examined  using  both  interpretational  and 
reflective  analysis. 

Research  questions  that  guided  the  study  included:  1)  How  do  the  instructional  and  communication 
strategies  employed  in  the  online  course  and  course  tools  assist  learners  in  developing  engaged  learning 
lessons  integrating  technology?  2)  How  do  participants  react  to  learning  to  integrate  technology  through  a 
technology -driven  course?  and  3)  What  tools  and  techniques  help  promote  participant  interaction? 

Findings  Highlights 

Self-report  on  pre -survey  data  for  all  participants  indicated  that  students  beginning  the  course  ranged 
in  their  experience  with  technology  skills  from  novice  to  expert,  but  all  had  access  to  and  had  used  email  and 
word -pro  cessing  before  enrolling  the  course.  The  majority  of  the  students  also  described  themselves  as 
having  some  knowledge  of  engaged  leaning  and  technology  integration,  however  only  one  had  used  a 
systematic  framework  for  planning  such  lessons.  Some  had  tried  various  technologies  in  lessons.  Results  on 
the  post-survey  showed  an  increase  in  familiarity  and  interest  in  the  systematic  planning  of  engaged  learning 
lessons  that  integrate  technology.  Students  also  reported  a wider  range  of  technologies  that  they  were  familiar 
with  and  comfortable  considering  for  future  lessons.  The  Internet  was  cited  most  often  as  the  technology  most 
useful  for  their  lesson  planning. 

A review  and  report  from  the  course  facilitators  on  the  lesson  plans  indicated  that  teachers  developed 
effective  lesson  plans  incorporating  engaged  learning  strategies  and  technology.  Interviews  with  the  students 
supported  consistently  a high  level  of  satisfaction  with  the  course,  with  integration,  and  with  learning  to  use 
technologies. 

Facilitator  and  design  team  reflections  from  debriefing  transcripts  and  journal  entries  yielded  the 
following  reflections  and  recommendations: 

Participants  need  regular  and  repeated  feedback  in  asynchronous  discussions. 

Synchronous  discussions  have  better  participation  and  depth  when  guided  by  questions  and  when 
participation  is  limited  to  small  groups. 

Assessment  is  more  time  intensive  and  more  essential  in  the  online  environment. 

Instructors  comfortable  with  face-to-face  instruction  have  to  learn  and  employ  many  “personal” 
techniques  to  gain  the  same  or  similar  relationships  with  students  (own  web  page  with  personal 
comments  and  picture,  chat  type  communication  with  individuals  and  small  groups). 

Regular  communication  between  the  design  team  and  facilitators  was  necessary  even  when  content 
was  clear  and  understood. 
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Abstract;  The  case  of  learning  in  digital  economy  ‘s  era  is  a fascinating  phenomenon.  The 
complexity  of  learning  integrated  with  the  diversification  of  new  technologies  set  a dynamic 
landscape  where  the  performance  seems  to  be  the  overall  objective  of  combined  efforts.  Our 
approach  tries  to  investigate  the  phenomenon  from  a ‘‘myopic”  point  of  view.  The  starting 
point  of  our  research  effort  is  the  acceptance  for  the  existence  of  a direct  relation  that  brings 
together  learning,  new  technologies,  pedagogy  and  knowledge  management.  In  this  paper  we 
present  the  initial  findings  from  the  application  of  qualitative  research  methodology.  More 
specifically  we  discuss  three  case  studies  (multiple  case  strategy)  concerning  eleaming 
systems  implementations  and  we  discuss  the  way  that  knowledge  management  theory  can 
support  in  practical  ways  the  performance  of  e-1  earning  systems. 


Introduction 

A common  practice  in  academic  institutions  or  business  units  is  the  employment  of  new  learning 
technologies  in  order  to  support  the  human  resources  management.  In  most  of  cases  the  adoption  of  several 
Learning  Management  Systems,  provides  a context  for  exploitation  but  unfortunately  the  achieved  learning 
performance  is  rather  inadequate  (Lytras  and  Pouloudi  2001a).  Several  conflicting  opinions  about  the 
convergence  of  knowledge  management  and  e-leaming  realized  a dialectic  context  of  reasoning  and  justification. 
The  fact  that  knowledge  management  concentrates  on  knowledge  processes  and  knowledge  artifacts  (Mentzas 
et  ai  2001)  promoted  the  initial  step  in  our  approach.  If  in  an  e-leaming  environment  we  can  justify  several 
knowledge  processes  that  construct  a kind  of  knowledge  artifact  that  is  finally  diffused  to  learners  then  the 
information  technology  can  analyze  the  technical  standards  and  the  required  applications  for  the  support  of  our 
conceptualization. 

Research  Methodology 

In  the  general  literature  of  research  methodologies  in  information  systems,  (Orlikowski  and  Baroudi  1991),  we 
can  found  many  approaches  which  in  general  can  be  placed  among  the  qualitative  or  in  the  quantitative  research 
methodology.  More  over  the  analysis  of  our  research  problem  and  the  justification  of  our  research  methodology 
can  be  supported  by  the  concepts,  which  Klein  & Myers  (Klein  and  Myers  1999)  proposed.  The  hermeneutic 
cycle  and  its  basic  emphasis  on  the  complementarities  of  a phenomenon  and  its  component  promote  the  basic 
idea  that  a phenomenon  such  as  e-leaming  has  to  be  analyzed  through  the  qualitative  research  methodology. 
The  case  studies  in  our  research  are: 

• The  Global  E-management  Master  e-leaming  system  and  especially  the  development  of  the  T1  course 
entitled  E-technology,  which  was  delivered  to  the  student  of  the  GEM  consortium, 

• The  XEXO  project  (Educational  space  without  limits),  which  objective  was  to  establish  an  e-leaming  system 
and  to  deliver  a master  degree  course  to  the  students  of  three  universities  in  Greece 

• The  Teleducation  Center  of  Athens  University  of  Economics  and  Business  that  utilizes  the  WebCT  platform 


in  order  to  provide  learning  content  to  students. 

Conclusions 

The  limited  space  of  the  article,  does  not  allow  to  present  the  findings  in  detail  even  though  that  we 
believe  that  qualitative  research  reveals  many  hidden  aspects  of  the  phenomenon  of  e-leaming  which  no 
quantitative  research  can  do.  Figure  1 summarizes  the  e-leaming  content  development  life  cycles. 


Figure  1.  The  knowledge  management  perspective  in  3 cases 
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(Mentzas  et  al  2001) , 
(Lytras  et  al  2002b),  claim 
that  knowledge  processes 
are  a value  carriers.  This 
theoretical  abstraction 
utilized  the  propositions  of 
other  researchers , (Nissen 
et  al  2000)  who  provided 
several  life  cycle  descriptive 
models  for  KM 

All  3 cases  provided 
information  that  justifies  e- 
leaming  as  a KM  process. 
Specific  phases  usually 
informal  provide  a context 
for  the  diffusion  of  learning 
content  to  users.  Several 
problems  unfortunately 
and  limited  management 
mechanisms  decrease  the 
performance 

The  conclusion  is  that  the 
knowledge  process 
concerning  e-leaming 
realization  range  to  a wide 
spectrum  that  incorporates 
two  general  pillars;  General 
knowledge  management 
processes  and  process  for 
the  adoption  of  knowledge 
to  learning  objects. 

LEARNING  OBJECT 
AS  A KNOWLEDGE 
ARTIFACT 

(Mentzas  et  al  2001) 
claimed  that  knowledge 
artifact  is  a way  for  realizing 
the  knowledge  management. 
Several  researchers  e.g. 

In  all  cases  it  was 
recognized  that  learning 
content  was  not 
knowledge  in  its  true  sense 
since  there  was  an  absence 

The  development  of 
learning  content  requires 
the  attachment  of  metadata 
in  order  to  become  a 
qualitative  transformation  of 
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(Nonaka  and  Takeuchi 
1995),  where  a questioning 
concerning  what  is 
knowledge  is  provided.  The 
justification  of  learning 
object  as  a knowledge 
artifact  drive  an  excessive 
analysis 

of  metadata  or  semantics 
that  would  enrich  the 
semantic  density  of 
learning  material.  But  it  was 
also  stated  that 
codification  and  diffusion 
is  of  critical  importance  for 
the  learning  objects. 

information  to  knowledge. 
This  requires  excess  effort 
from  educators.  The 
overview  of  several 
metadata  schemas  (LOM, 
GEM,  Ariadne,  IEEE  LOM, 
IMS,  DC)  seem  to  bee 
inadequate  to  support  the 
learning  dimension  of  e- 
leaming 

METADATA 
JUSTIFICATION 
AND  E-LEARNING 
PEDAGOGY 
SUPPORT 

Shuell  (1992)  & Bloom  and 
Krathwohl  (1984)  provide  a 
context  for  providing 
different  learning  value. 
Learning  processes  or 
learning  objectives  are  the 
value  drivers. 

Unfortunately  the  learning 
performance  of  three  cases 
was  based  on  static 
learning  scenarios.  It  was 
evident  that  this  dimension 
of  e-leaming 
implementations  was  too 
weak  and  needed  support. 

The  detailed  analysis  of 
learning  processes  can 
cause  a significant 
contribution  specifying 
metadata  elements  for 
specific  learning  processes. 

Table  1.  Comparing  the  Theoretical  and  Empirical  Evidence 
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Abstract:  In  recent  years  many  researchers  try  to  investigate  the  direct  or  indirect  impact  of 
knowledge  management  approaches  to  the  achievement  of  strategic  corporate  objectives. 
Several  approaches  have  been  promoted,  with  main  emphasis  to  be  paid  at  the  justification  of 
knowledge  exploitation  processes.  Our  research  effort  is  concentrated  on  the  demand  for  high 
quality  interchangeable  knowledge  objects  capable  to  support  dynamic  learning  initiatives.  The 
general  metadata  models  (Dublin  Core,  IMS,  LOM,  SCORM)  for  knowledge  objects 
enrichment  are  been  reviewed  and  a well  justified  critique  is  provided  in  order  to  claim  the 
importance  for  the  justification  of  an  “interchangeable  knowledge  objects  standard”  for  learning 
purposes.  The  overall  objective  is  the  justification  of  a dynamic  learning  environment  capable 
of  supporting  the  value  diffusion. 


Introduction 

The  development  of  dynamic  learning  environments  requires  a systematic  justification  of  methods  and 
processes  that  vary  learning  (unique  value  proposition)  for  every  learner.  This  challenging  situation  sets  a 
context  that  promotes  the  research  in  reusability  of  knowledge  resources.  A realization  of  reusability  can  be 
based  on  semantics  that  enrich  general  knowledge  resources  (e.g.  an  article,  a journal  paper,  a PowerPoint 
presentation,  e.t.c.).  The  ultimate  objective  is  to  expand  the  customization  capabilities  of  learning  environments 
and  knowledge  management  seem  to  be  a well  justified  mean  for  the  achievement  of  such  a goal  (Lytras  and 
Pouloudi  2001c),  (Lytras  et  al.  2002a),  (Lytras  et  al.  2002b)  Our  research  intents  to  justify  a metadata  schema  that 
supports  a dynamic  e-leaming  environment.  The  current  situation  on  metadata  is  described  by  a narrow  vision 
on  learning. 

A synopsis  of  conceptual  abstractions 

The  organization  of  knowledge  in  distinct  objects,  small  structural  components,  is  the  basis  of  reuse.  In 
the  case  of  customized  learning  the  same  learning  content  can  be  used  in  many  ways  due  to  its  integration  with 
other  knowledge  objects.  Our  approach  is  based  on  two  clear  assumptions  that  initiate  the  whole  research:  A 
learning  object  is  a synthesis  of  knowledge  and  metadata  that  expand  further  the  learning  effect  of  knowledge. 
From  this  perspective  an  e-leaming  environment  has  to  utilize  a learning  objects  base.  A critical  question  is 
which  metadata  can  we  use  in  order  to  increase  the  learning  effectiveness.  A learning  process  is  an  abstraction 
of  a logical  sequence  of  learning  activities  or  tasks.  The  conceptualization  of  learning  processes  determines  an 
interface  in  which  the  components  of  a learning  object  can  be  realized.  This  layout  is  a value  container,  which  is 
utilized  by  the  embodiment  of  learning  object.  The  critical  question  is  which  learning  processes  can  we  use?  And 
is  there  any  hierarchy  of  learning  processes  that  determine  different  value  layers? 

The  underlying  issue  behind  the  packaging  of  knowledge  is  the  reveal  of  a key  idea  that  will  support 
the  accomplishment  of  a metadata  scheme.  Our  proposition  that  has  been  developed  after  many  implementations 
of  e-leaming  projects  in  our  research  unit,  ELTRUN.  (www.eltrun.aueb.grj  is  that  the  convergence  of  knowledge 
management  and  e-leaming  is  realized  through  an  integrated  life  cycle  model  (Lytras  et  al.  2002a).  Each  step  in 
this  model  represents  a value  adding  process.  Each  step  sets  a context  for  questioning  about  the  required 
metadata  that  have  to  be  attached  to  a generic  knowledge  source  in  order  to  formulate  learning  object.  The 
key  proposition  in  our  approach  is  that  a number  of  metadata  can  supports  each  step  in  the  whole  model  and 
also  the  learning  processes  as  learning  objects  containers.  For  the  specification  of  the  elements  of  this  metadata 
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schema  a review  of  well-known  metadata  schema  has  been  undertaken. 


LEARNING 

OBJECTS 

BASE 


Which  metadata? 


LEARNING 

PROCESSES 

POOL 


How  many  / Which 
Learning 
Processes? 


LEARNING  OBJECT  = KNOWLEDGE  + METADATA  LEARNING  PROCESS  = UFE  CYCLE  + TEMPLATE 


Each_Metadata  element 
Is  used] A certain  locations 
oMhe^creen  / interface 


A generic  interface  / A computer  screen 

The  metadata  and  the  knowledge  define  the  educational  space 


Figure  1;  The  basic  conceptual  abstraction 

Multidimensional  Dynamic  Learning  Proposed  Metadata  Schema 

Metadata  are  data  about  data.  In  the  context  of  learning,  metadata  are  data  about  learning  objects.  This 
definition  provides  the  major  characteristic  of  metadata:  They  are  descriptive  indexing  labels.  Unfortunately  for 
the  case  of  learning  this  is  not  enough.  The  indexing  of  learning  object  it  seems  to  be  a management  facility  with 
limited  impact  on  the  realization  of  engagement  of  learners.  A lot  of  approaches  concerning  metadata  have  been 
proposed  for  general  or  learning  purposes.  Some  of  the  most  well  known  organizations  that  work  in  metadata 
specifications  are  the  Aviation  Industry  Computer-Based-Training  Committee  (AICC),  the  IEEE  Learning 
Technology  Standards  Committee,  the  ARIADNE,  the  Educause  Instructional  Management  Systems  (IMS) 
Project,  the  W3C,  and  the  ADL.  The  critical  review  of  several  metadata  schemas  including  IEEE  Learning  Objects 
Metadata,  IMS,  Dublin  Core,  SCORM,  GEM  realizes  a gap  in  learning.  This  gap  in  the  propositions  of  the  famous 
metadata  schemas  sets  a context  for  significant  contribution. 

Conclusions 

The  overall  approach  described  in  the  limited  space  of  this  paper  provides  a context  for  exploitation 
focusing  on  learning.  The  incremental  refinement  of  conceptualization  drives  to  a set  of  metadata  that  can  assist 
the  transformation  of  knowledge  resources  to  reusable  learning  objects.  This  stcp-by-step  justification  finally 
drives  an  extensive  XML  specification  of  the  metadata  schema.  The  selection  of  learning  processes  as  the  carrier 
of  value  expands  the  dynamic  nature  of  an  e-leaming  system.  The  whole  approach  seems  to  integrate  the  two 
well-defined  and  distinct  approaches  to  knowledge  management,  the  knowledge  artifact  and  the  knowledge 
activities  approaches.  Future  research  is  concentrated  on  the  analysis  and  the  development  of  a dynamic  e- 
leaming  environment  based  on  the  key  ideas  of  Knowledge  Interchange  Concept.  For  this  scope  an  1ST 
proposal  is  going  to  be  prepare.  We  welcome  any  collaboration  that  will  support  the  refinement  of  this  proposal. 
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Relate  / Value 

Language,  Subject,  Quality,  Main  Concept 

Acquire 

Purpose,  Title,  Description,  Creator,  Publisher,  Contributor,  Identifier,  Authors,  and 
Institution 

Organize 

Date,  Format,  Location,  Type,  Source,  Relation,  Discipline,  Sub  discipline,  Main 
concept.  Main  concept  synonyms,  other  concepts,  granularity 

Transfer 

Cost,  Copyright  & other  restrictions,  rights.  Document  type.  Document  handle. 
Document  format.  File  size 

Use 

Operating  system  type,  OS  yersion,  other  platform  requirements.  Installation  remarks. 
Access  rights,  restrictions.  Usage  remarks 

Table  1 : Metadata  for  General  Knowledge  Management 


Relate 

Interactiyity  Type,  Interactiyity  Leyel,  Intended  End  user  role.  Learning  context,  Typical  Age 
Range,  Typical  Learning  Time,  Language,  Relation,  Coyerage,  Audience,  Grade,  Pedagogy, 
End  user  type.  Didactical  context 

Adapt 

Metadata  (author,  creation  date,  last  modified  date,  language,  yalidator,  yalidation  date) 

Attract 

Semantic  density,  difficulty.  Description,  Standards,  Quality,  Duration,  Difficulty  Leyel, 
Interactiyity  Leyel 

Engage 

Essential  Resources,  Pedagogy,  and  Pedagogical  Duration 

Learn 

Pedagogy  teaching  methods,  pedagogy.  Assessment,  semantic  density,  annotation  (annotator, 
creation  date,  content) 

Table  2:  Metadata  for  Learning  Exploitation 


Presentation 

Summary,  Purpose,  Essential  Resources,  Annotation,  Topics,  and  Search  Guidance 

Analysis 

Summary,  Purpose,  Conceptual  Components,  Typical  Relations,  Annotation,  Relevant 
Topics,  Search  Guidance,  Analysis  conclusions.  Collaboration  Details 

Synthesis 

Summary,  Purpose,  Relevant  Knowledge  Objects,  Typical  Relations,  Conceptual 
Components,  Annotation,  New  Meaning,  Draw  Relations,  Search  Guidance, 
Recommended  Conclusions,  Conclusions,  Collaboration  Details,  Guidance 

Evaluation 

Purpose,  Relevant  Knowledge  Objects,  Summary,  Collaboration  Details,  Guided 
Theories,  Guidance,  Application  Session,  Simulation  Session,  Annotation,  New 
Meaning,  Conclusions,  Recommended  Conclusions 

Reasoning 

Summary,  Purpose,  Relevant  Knowledge  Objects,  Annotation,  New  Meaning,  Guidance, 
Conclusions,  Recommended  Conclusions,  Starting  Points,  Collaboration  Details 

Explanation 

Summary,  Purpose,  Relevant  Knowledge  Objects,  Topics,  Typical  Relations,  Draw 
Relations,  Annotation,  New  Meaning,  Guidance,  Collaboration  Details,  Recommended 
Conclusions 

Relation 

Summary,  Purpose,  Relevant  Knowledge  Objects  , Main  Concepts,  Draw  Relations, 
New  Meaning,  Annotation,  Guidance,  Typical  Relations,  Conclusions,  Collaboration 
Details 

Problem 

Solving 

Summary,  Purpose,  Relevant  Knowledge  Objects,  Present  Problem,  Sub  problems.  Main 
Concepts  , Collaboration  Details,  New  Meaning,  Annotation,  Guidance, 

Conclusions 

Collaboration 

Summary,  Purpose,  Recording,  Annotation,  Findings 

Table  3:  Metadata  for  Learning  Processes 
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The  study  examines  the  support,  training  and  environment  likely  to  make  possible 
effective  online  teaching  and  learning.  It  focuses  on  an  innovative  new  senior 
college  in  Perth,  Western  Australia  that  is  committed  to  creating  an  outcomes- 
focused  curriculum  by  integrating  unique  Information  and  Communication 
Technology  infrastructure  into  the  delivery  of  educational  programs,  on-line 
curricula,  and  management  systems  for  both  teachers  and  students.  Sevenoaks 
Senior  College  has  provided  the  means  to  manage  education  through  an 
infrastructure  that  supports  a learning  community  via  strategic  partnerships  and 
online  collaboration.  This  is  an  important  direction  for  schools  as  it  addresses  the 
educational  needs  of  students  with  lifelong  learning  strategies  for  future  work  and 
university  life.  The  study  being  carried  out  over  2001-2003  combines  quantitative 
and  qualitative  research  methods  to  examine  the  ways  in  which  ICT  is  used  and  the 
learning  environment. 


Background/Thcoretical  Framework 

In  Backing  Australia  5 Ability:  Innovation  Action  Plan,  $68.2  million  over  five  years  (2001-2006)  was 
committed  to  the  Schools  Online  Curriculum.  The  commitment  was  made  on  the  premise  that: 

“ While  investment  by  all  schooling  systems  in  equipment  and  connectivity  has  paved  the  way 
for  change,  it  is  the  availability  of  quality  online  content  that  will  begin  to  improve  learning” 
(Learning Federation  Schools  Online  Curriculum  Content  Initiative,  2001) 

At  the  local  state  level,  the  Western  Australian  government  has  invested  approximately  $100  million 
on  information  technology  in  schools,  including  approximately  $8.3  million  to  build  the  Sevenoaks 
Senior  College. 

However,  increased  spending  on  ICT  in  schools  and  educational  institutions  does  not  necessarily 
guarantee  improved  teaching  and  learning  environments  and  improved  student  outcomes  (Centre  for 
Research  on  Information  Technology  and  Organisations,  1999;  EdNA,  1999a;  White,  1999). 

Education  has  a poor  history  of  meeting  the  challenges  of  major  shifts  in  Information  and 
Communications  Technology  (Trinidad,  1998). 

Findings  To  Date  - Multi  Layered  And  Multi-Faceted  Case  Study  of  Sevenoaks  Senior  College 
The  Online  Courses 

To  date,  the  teachers  at  Sevenoaks  Senior  College  have  developed  90  courses  in  WebCT;  for  the  first 
year  of  operation,  this  can  be  considered  a significant  success  for  the  teachers  and  support  facilities 
found  at  Sevenoaks. 

Some  of  the  courses  developed  online  at  Sevenoaks  Senior  College  include  an  English  unit  dealing 
with  TV  drama,  a Digital  Media  unit  based  on  an  anti-smoking  campaign,  a Physics  Nuclear 
Technology  unit  and  a Political  and  Legal  Studies;  all  used  a constructivist  framework.  Discussions 
with  teachers  indicate  that  time  constraints,  the  need  to  recognise  workloads  and  set  limits  within  a 
given  time  frame  is  critical. 

The  Teachers 

Interviews  have  reflected  a positive  attitude  to  the  vision  of  Sevenoaks  Senior  College.  However,  one 
of  the  biggest  hurdles  that  the  teachers  have  experienced  is  getting  students  to  use  their  time  wisely  and 
effectively.  “I  found  that  some  students  did  all  the  reading  that  was  required  but  had  to  be  pushed  to 
produce  anything  substantial,  and  that  the  computer  nerds  talked  the  language  but  tended  not  to  do 
much  else”  [IDT  1:2001].  Observations  and  interviews  with  teachers  and  students  indicated  that  the 
supportive  learning  environment  created  by  the  teachers  was  vital  to  the  success  of  the  unit  (Aldridge 
et  al,  2002). 


A key  factor  in  the  development  of  online  learning  at  Sevenoaks  has  been  the  integral  role  of  the 
Teacher  Librarian,  The  Teacher  Librarian  has  become  responsible  for  facilitating  staff  development  by 
acting  as  a staff  trainer,  a mentor  and  an  initiator  in  the  development/creation  of  modules  of  online 
curriculum 

The  Students 

According  to  students  studying  at  Sevenoaks  whom  were  interviewed,  computers  are  used  mainly  for 
researching  information  on  the  Internet,  typing  up  assignments  and  accessing  course  information  from 
the  WebCT  platform  during  class  time  (Trinidad,  Macnish,  Aldridge,  Fraser  & Wood,  2001).  These 
results  reflect  previous  research,  which  has  reported  that  Australian  school  students  spend  more  time 
using  word-processing  software  than  any  other  software,  and  that  they  use  the  Internet  primarily  for 
email,  research  and  entertainment  (EdNA,  1999a;  White,  1999). 

A significant  feature  of  the  Nuclear  Physics  course  was  the  opportunity  available  to  students  to  engage 
in  discussion,  debate  and  present  and  argue  about  ideas.  One  student  interviewed  commented:  “I  liked 
the  online  course  because  sometimes  you  can  learn  more  from  other  students,”  Some  students 
indicated  their  displeasure  at  the  amount  of  time  that  was  required  to  research  and  look  for 
information.  On  the  other  hand,  one  student  commented  on  the  advantage  of  the  unit  in  terms  of 
inquiry  rather  than  being  given  information  by  the  teacher:  “I  usually  find  that,  when  you  actually  find 
out  something  on  your  own,  you  learn  more  from  it”  (ID98:2001). 

All  of  the  students  whom  were  interviewed  held  very  positive  attitudes  towards  the  use  of  ICT  in  the 
classroom.  Student  comments  on  the  benefits  of  online  learning  included:  “You  know  exactly  what  to 
do  and  there  are  no  excuses  for  not  doing  assignments”  (IDS  12:2001)  and  “[We  are  able  to]  look  up 
more  information,  which  is  better  than  books  and  gives  a wider  range  to  look  at”  (IDS2:200I), 

Discussion  and  Conclusion 

As  Sevenoaks  Senior  College  moves  into  its  second  year,  it  is  timely  to  consider  ways  in  which 
teachers  can  be  helped  to  use  the  ICT  environment  in  more  innovative  ways  in  their  classroom  and  to 
move  towards  a higher  level  of  course  development.  There  is  a need  for  appropriate  professional 
development  to  move  towarck  developing/adapting  courses  with  innovative  models;  achieving  quality 
measures  and  standards;  and  planning  the  integration  of  ICT  into  classes. 

In  summary,  the  learning  environment  at  Sevenoaks  is  one  where  teachers  and  students  feel  supported. 
It  is  an  environment  that  enables  students  to  assume  more  responsibility  for  their  individual  learning 
and  where  teachers  are  provided  with  the  means  to  manage  the  transition  from  a traditional  classroom 
to  one  that  effectively  integrates  ICT  in  teaching  and  learning  for  the  benefit  of  the  student. 
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Abstract:  The  Didactics  of  Informatics  research  group  at  the  University  of  Paderbom  is 
involved  in  efforts  to  design,  implement  and  evaluate  a curriculum  for  Media  education 
for  prospective  teachers  at  the  secondary  school  level.  One  major  issue  is  the  question 
of  whether  it  is  necessary  for  future  teachers  to  learn  the  basic  concepts  of  informatics, 
and  if  so,  which  topics  within  informatics  are  relevant,  and  which  main  objectives 
should  be  achieved.  After  two  years  of  seminars  on  basic  informatics  for  students  of 
education  at  the  University  of  Paderbom,  it  is  now  possible  to  submit  the  preliminary 
results.  The  presented  paper  discusses  the  rationale  behind  introducing  informatics  into 
media  education  for  teachers,  and  describes  elements  of  the  curriculum,  as  well  as  the 
main  teaching  objectives  for  a better  understanding  of  computer-based  media. 


The  Role  of  Computer-Based  Media  in  School 

From  the  Perspective  of  the  Student 

Students  today  should  have  many  different  abilities  in  using  computers  and  computer-based  media.  First  of 
all,  they  must  have  the  ability  to  use  hardware  and  software  (i.e.  to  turn  on  a computer,  use  a word 
processor,  work  cooperatively  over  a network,  etc.)  Because  of  the  ongoing  revolution  of  products  and 
ideas,  training  for  specific  software  products  will  not  sufficiently  prepare  students  for  future  demands. 
Today’s  students  should  be  able  to  analyse  and  evaluate  the  social  impact  of  using  computer  systems  and 
computer-based  media  (CBM).  Therefore,  it  is  not  sufficient  to  teach  only  the  basics  of  computer  operation. 
Students  should  learn  about  concepts  of  computer-based  media,  digital  coding  of  data,  and  computer 
networks  (especially  the  Internet),  as  well  as  the  relationship  between  traditional  and  computer-based 
media.  They  should  acquire  basic  skills  and  understand  the  functionality  of  new  media  such  that  they  can 
evaluate  its  future  impact  on  society.  These  are  the  elements  of  competency  in  media  that  should  be  taught 
to  students. 


From  the  Perspective  of  the  Instructor 

Educators  have  tasks  which  range  from  planning  and  instructing  courses  to  administration  of  schools  or 
counselling  of  students,  and  they  can  incorporate  CBM  into  these  areas.  But  if  students  should  learn  more 
than  simple  computer  operations,  then  teachers  must  as  well.  They  should  not  only  have  media  competency 
so  that  they  can  work  with  computers,  but  also  didactical  media  competency  in  order  to  integrate  computers 
into  the  learning  and  teaching  processes.  Finally,  teachers  need  media  educational  competency  to  teach 
computer-based  media.  It  is  essential  that  teachers  learn  to  use  CBM  in  different  classroom  settings 
(scenarios),  in  order  to  enhance  the  learning  process.  These  scenarios  are  helpful  in  designing  a curriculum 
for  media  education  in  teacher  training  (Magenheim  & Schubert,  2000). 


Organisational  Aspects 


Administration  can  also  benefit  from  the  use  of  computer-based  media.  Although  its  needs  differ  from 
those  of  the  classroom,  administration  is  concerned  with  the  improvement  of  the  school  system,  and  this 
can  be  done  with  CBM.  It  can  be  used  to  create  a more  customized  learning  environment,  design  curricula 
for  media  education  or  create  new  ways  of  teaching  and  learning,  make  the  school  more  responsive  to 
community  needs  and  vice  versa,  administrate  school  equipment  and  networks,  and  make  the  school 
dynamic  and  open  to  change. 

The  role  of  computer-based  media  in  school  involves  learning  by  the  students,  instruction  by  the  teachers, 
and  organisation  and  imp  rovement  by  the  administration.  Teachers  need  not  only  a specific  tool,  but 
knowledge  about  how  it  works  in  order  to  adapt  the  tool  to  their  specific  needs.  This  means  that  the 
curriculum  must  be  generic,  and  must  include  the  basics  of  informatics.  If  future  instructors  do  not  obtain 
grounding  in  informatics  in  their  media  classes,  they  will  not  be  able  to  use  the  full  potential  of  computer- 
based  media. 

To  gain  an  understanding  of  the  possibilities  of  computer  based-media  that  cover  the  above  perspectives 
and  allow  prospective  teachers  to  keep  their  understanding  up  to  date,  a curriculum  for  media  education 
must  supply  the  learners  with  a solid  foundation.  This  foundation  is  built  by  incorporating  the  specific 
qualities  of  CBM  into  the  curriculum. 

One  of  these  qualities  is  digitalization.  CBM  uses  digitally-represented  data,  regardless  of  form  (text,  video, 
sound,  etc).  This  quality  allows  certain  features  such  as  transforming,  copying,  etc.  To  understand  this 
aspect,  one  must  consider  the  technical  and  informatics  background  of  digitalization. 

Another  important  aspect  is  to  understand  how  the  processing  of  digitally -coded  data  and  its  representation 
is  interwoven  with  the  human  processing  of  information  in  media  use.  CBM  are  tools  for  constructing, 
transforming,  and  copying  digital  data,  and  simultaneously,  they  convey  information.  One  must 
comprehend  this  duality  of  CBM  in  order  to  fully  grasp  the  effect  on  our  daily  lives. 

To  understand  and  use  CBM  effectively,  its  construction  process  must  be  considered;  software 
development  is  not  only  a technical  process,  but  must  take  into  account  cognitive  processes  and  needs  of 
the  users  (Magenheim,  2001).  This  is  especially  important  in  an  educational  context  where  prospective 
teachers  should  learn  to  apply  pedagogical  software  tools,  to  evaluate  and  (hopefully)  to  improve  them  in 
cooperation  with  software  developers. 

It  is  clear  that  these  goals  are  only  reachable  by  a curriculum  that  includes  basic  elements  of  informatics. 
These  will  be  discussed  later,  but  first,  we  consider  other  elements  necessary  for  the  development  of  a 
curriculum. 


Elements  of  a Curriculum  for  Media  Education  in  Teacher  Training 


Figure  1 : Influencing  Factors  on  a Curriculum  for  Media  Education 


Many  factors  influence  a curriculum  for  media  education.  Arts  and  sciences  (not  a topic  of  this  paper) 
contribute  to  its  concepts,  content,  and  teaching  methods.  However,  these  cannot  entirely  dictate  the 
curriculum,  as  they  are  filtered  through  normative  settings  (society,  educational  conventions  and  politics). 
They  do  assist  in  forming  the  requirements,  which  are  necessary  in  the  use  of  computer-based  media.  From 
these  requirements,  tasks  and  scenarios  can  be  generated  and  worked  into  the  construction  of  the 
curriculum.  Specific  contents  can  be  taken  from  the  general  curriculum  to  cover  different  tasks. 

The  curriculum  for  media  education  in  teacher  training  developed  by  the  research  group  at  the  University 
of  Paderbom  does  not  claim  to  be  complete;  rather,  it  is  focused  on  those  aspects  of  a curriculum  which 
require  fundamentals  of  informatics. 


Basic  Areas  of  Informatics  in  Media  Education  for  Teachers 

As  we  have  seen,  instructors  must  have  different  skills  and  knowledge  areas  to  be  successful.  There  must 
be  a way  to  distinguish  the  use  of  specific  software  tools  from  the  use  of  computer-based  media.  Useful 
here  is  Basic  Areas  of  Informatics  (BAI),  which  describes  the  fundamentals  of  informatics  for  general 
media  education  (Schulte  & Scheel,  2001).  BAI  serves  as  a theoretical  foundation  for  forming  basic  skills  and 
knowledge.  BAI  was  developed  in  the  context  of  didactics  of  informatics  to  specify  the  role  of  informatics 
in  secondary  education  for  a general  media  education.  Thus,  didactical  concepts,  common  media  education 
concepts,  and  corresponding  work  in  informatics  were  analyzed,  with  a resulting  abstract  consisting  of  four 
tasks  for  building  the  informatical  foundations  of  a general  media  education.  These  areas  are: 

1.  Concepts  of formalism  and  problem  solving  strategies  in  informatics 

In  informatics,  modelling  languages  and  tools  are  developed  for  the  representation  of  information.  We  call 
them  representation  techniques.  Modelling  tools  such  as  UML  are  usable  outside  of  the  software 
development  process.  The  aim  here  is  to  supply  students  with  the  background  knowledge  necessary  to 
competently  use  these  tools.  On  the  one  hand,  students  should  be  able  to  express  representations  of  things 
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(structures  and  processes  in  different  subject  areas)  formally  with  the  aid  of  representation  techniques.  On 
the  other  hand,  they  should  learn  skills  for  using  software  products  (i.e.  word  processors,  spreadsheets)  to 
represent  things.  The  aim  is  not  to  teach  a specific  user  interface,  but  rather  to  teach  understanding  for  the 
typical  media  functions  of  these  computer-based  tools  - for  example,  the  proper  use  of  style  sheets.  These 
functions  are  1)  to  represent  things,  2)  to  (re)arrange  representations,  3)  to  save  representations  in  data  files, 
and  4)  to  provide  others  with  information  about  these  products  (see  Hubwieser,  1997).  This  Area  does  not 
cover  questions  concerning  the  content  or  aesthetics  of  representations. 

2.  Technical  basics 

This  Area  describes  the  technical  and  informatics  skills  necessary  for  using  computer-based  media.  It 
covers  skills  such  as  choosing  the  appropriate  data  format  and  software -tool,  building  suitable  directory 
trees  to  manage  data,  and  querying  databases  and  search  engines.  Students  should  gain  an  overview  of  the 
technical  infrastructure  in  which  they  use  computer-based  media:  For  instance,  the  global  networks, 
protocols  and  software  tools  that  build  the  World  Wide  Web. 

3.  Semiotics 

In  computer-based  media,  symbols  have  a dual  purpose:  They  form  programs  to  control  machines,  but  they 
also  serve  as  media  in  a broader  sense;  for  example,  source  code  for  a program  is  executed  by  a machine 
but  understood  by  a human.  The  learning  goal  of  this  Area  is  to  reflect  on  these  specific  semiotic 
implications  of  computer-based  media.  The  user  communicates  with  other  users,  but  interacts  with  the 
machine  (human-computer  interaction).  There  lies  an  area  between  which  is  a combination  between 
human-computer  interaction  and  communication  between  humans.  This  can  be  seen  in  automated  email 
systems,  in  which  a message  is  generated  by  computer  as  a result  of  statistical  information,  but  it  is  possible 
for  a human  to  add  a personal  note  (i.e.  a user  might  receive  a note  thanking  him  or  her  for  contributions) 
(Maurer,  1999). 

Students  should  also  be  able  to  understand  the  possibilities  and  the  role  of  technology  in  automated  and 
partially-automated  communication  processes.  Therefore  they  should  learn  to  differentiate  between  the 
ways  in  which  humans  and  computers  process  data.  Media  functions  can  be  addressed  to  further  this 
understanding. 

4.  Social  impacts  of  computer-based  media 

The  goal  here  is  to  become  aware  that  software -tools  and  description  techniques  - in  short,  the  technical 
infrastructure  - a)  is  purposely  created  to  meet  specific  aims,  b)  is  changeable,  and  c)  embodies  implicit 
values  and  regulates  individual  behaviour  (Lessig,  1999). 

It  is  important  here  to  contextualise  the  current  computer,  software,  or  internet  generation  within  its 
development  history.  In  many  cases,  the  functionality  of  a current  version  is  only  clear  with  knowledge  of 
its  evolution. 

All  in  all,  the  BAI  describes  essential  basic  skills  for  prospective  teachers,  but  does  not  cover  an  important 
aspect  which  was  previously  mentioned:  The  specific  function  of  computer-based  media  in  school  and 
typical  scenarios  of  its  use  in  teaching  and  learning. 

Scaffolding  a curriculum  of  basic  informatics  within  media  education 

The  professional  future  of  an  educator  is  determined  by  activities  within  the  scope  of  the  classroom, 
administration,  or  other  didactic  area,  as  well  as  communications  with  students,  parents,  and  community 
administrators.  These  are  termed  action  scenarios.  Often,  it  is  useful  or  even  necessary  to  integrate 
computer-based  media  into  teachers*  professional  lives. 

These  situations  can  be  incorporated  into  seminars  for  future  teachers,  on  the  topic  of  media  education  and 
fundamentals  of  informatics.  First,  we  analyse  professional  scenarios  of  teachers  and  identify  typical 
situations  of  computer-based  media  use.  Then,  we  cover  the  fundamentals  of  informatics  which  teachers 
should  know,  in  order  to  foster: 

a fundamental  understanding  of  CBM 

elementary  skills  in  handling  a sample  of  computer-based  software  tools 

an  effective  use  of  computer-based  media  in  school-related  action  scenarios 

the  ability  to  organise  learning  and  teaching  processes  with  CBM  in  different  subjects 

the  ability  to  organise  media  education  in  the  classroom,  with  CBM  itself  an  educational  topic 
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Finally  we  must  structure  these  situations  based  on  fundamentals  of  informatics  with  regard  to  the  tasks 
previously  described.  This  will  generate  a generic  curriculum  enabling  us  to  create  a selection  of  contents 
and  objectives  for  university  courses  on  the  BAI  of  CBM.  The  table  below  describes  how  to  assign  each 
action  scenario  to  a Basic  Area  of  Informatics,  as  well  as  describing  outlines  for  problem-centred  seminars. 
The  contents  listed  in  the  table  below  should  not  all  be  covered  in  a single  seminar,  but  represent  possible 
options  describing  how  to  teach  the  basics  of  informatics  in  a problem-centred  seminar.  In  some  cases,  it 
will  be  possible  to  re-assign  contents  to  different  Areas.  For  example,  xml  is  assigned  to  Area  1 (as  a 
formalism)  but  can  also  demonstrate  how  documents  are  divided  into  content,  structure,  and  layout,  thereby 
allowing  specific  semiotic  functions  and  automated  processing  (see  Area  3). 

Nevertheless,  the  curriculum  will  be  helpful  in  organising  student  seminars.  These  seminars  should  focus 
on  practical  problems  and  issues  regarding  the  use  of  CBM  in  teachers’  professional  lives.  University 
seminars  and  related  materials  with  the  topic  of  BAI  should  be  organised  around  a certain  action  scenario. 
The  project  should  contain  an  assignment  to  produce  specific  computer-based  media  to  be  used  later  in  a 
real  pedagogic  action  scenario.  While  finding  solutions  for  this  relatively  complex  problem,  students  shall 
not  only  learn  about  different  functions  of  CBM  and  how  to  use  them  in  a pedagogic  context  but  also  gain  a 
fundamental  understanding  of  basic  areas  of  informatics.  The  university  seminars  should  not  be  organised 
simply  as  lectures,  but  should  offer  students  different  action  scenarios  in  which  they  can  exercise  the  use  of 
CBM  and  develop  media  products  of  their  own  as  a contribution  to  the  complex  problem-solving  process. 
The  concept  also  includes  aspects  of  constructivist  learning,  because  students  should  have  access  to  a 
hypertext  learning  environment,  with  relevant  materials  such  as  supplement  texts,  pictures,  animations, 
videos,  interactive  forms  and  so  on.  By  navigating  this  web-based  learning  environment,  they  will  be  able 
to  organise  learning  processes  of  their  own  and  gain  the  knowledge  they  need  to  solve  the  problem 
presented  in  a seminar. 

This  will  give  them  the  ability  to  use  their  newly  acquired  knowledge  and  skills  on  CBM  in  a variety  of 
similar  scenarios.  At  minimum,  after  attending  another  one  or  two  such  courses,  they  will  finally  have  the 
media  competency  described  in  the  objectives  above,  and  be  able  to  independently  manage  complex 
assignments  using  Information  and  Communication  Technologies  (ICT). 
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Abstract:  The  need  for  accurate  and  current  scientific  information  in  the  fast 
paced  Internet-aware  world  has  prompted  the  scientific  community  to  develop 
tools  that  reduce  the  scientist’s  time  and  effort  to  make  digital  information 
available  to  all  interested  parties.  The  availability  of  such  tools  has  made  the 
Internet  a vast  digital  repository  of  information.  But  the  ad  hoc  nature  in  which 
information  is  gathered  and  organized  on  the  web,  makes  access  to  such 
information  a time  consuming  and  sometimes  fhistrating  affair.  Digital  library 
systems  have  the  potential  for  solving  problems  in  maintaining  high  quality 
scientific  content  delivered  via  the  web  by  providing  tools  for  scientists  to 
collect,  verify,  organize,  manage,  and  update  their  collections.  This  paper 
describes  an  environment  that  reduces  the  effort  and  time  required  by  scientists 
to  share  their  data  with  other  collaborators  in  an  automated  and  asynchronous 
manner,  thereby  allowing  them  to  focus  mostly  on  their  own  scientific  practice. 
The  data  is  maintained  as  a collaborative  collection  in  a digital  library  that  can 
also  be  used  as  an  educational  resource. 


1.  Introduction 

Digital  library  systems  have  the  potential  for  solving  problems  in  maintaining  high  quality  scientific 
content  delivered  via  the  web  by  providing  tools  for  scientists  to  collect,  verify,  organize,  manage,  and 
update  their  collections.  The  ability  of  the  Internet  to  bring  geographically  distant  scientists  and 
researchers  together  as  a community  provides  the  opportunity  to  build  collaborative  digital  libraries  that 
can  provide  a larger  quantity  and  higher  quality  of  data.  The  notions  of  ownership  and  attribution  of 
intellectual  and  scientific  property  must  be  retained  and  the  contributors  must  control  sharing  of  their  data. 

This  paper  describes  an  environment  that  reduces  the  effort  and  time  required  by  scientists  to  share  their 
data  with  other  collaborators  in  an  automated  and  asynchronous  manner,  thereby  allowing  them  to  focus 
mostly  on  their  own  scientific  practice.  The  data  is  maintained  as  a collaborative  collection  in  a digital 
library  that  can  also  be  used  as  an  educational  resource  [1].  The  collection  can  be  continually  updated  by 
the  contributors  and  used  or  reused  to  build  course  content  via  tools  provided  by  the  digital  library  [2]. 

2.  Research  Context 

2.1  Digital  Flora  of  Texas 

The  Digital  Flora  of  Texas  (DFT)  [3]  project  was  established  as  an  open,  web-based  digital  resource. 
Participants  currently  include  scientists  from  43  contributing  herbaria  as  well  as  many  researchers  at  public 


and  private  organizations  [4].  Each  participating  herbarium  (or  collection)  has  its  own  curator  or  group  of 
curators  who  verify  and  validate  the  data  they  contribute.  The  DFT  project  has  a wide  array  of  web  tools, 
collections  and  prototype  systems  that  can  be  used  for  research  or  education.  The  DFT  contains  information 
visualization  tools  for  displaying  statistical  and  distributional  maps  of  taxa  of  the  flora  of  Texas.  In 
addition,  upload  and  data  conversion  tools  provide  an  automated  way  for  researchers  to  easily  build 
collections  and  share  them  with  the  scientific  community.  The  DFT  collections  are  accessible  through  the 
web,  making  it  easier  for  researchers,  scientists,  instructors  and  students  to  access  accurate  and  up  to  date 
information.  Instructors  can  use  the  collections  to  construct  web-based  content  for  their  courses.  These 
educational  and  scientific  resources  can  be  used  and  reused  in  many  different  contexts  [5,6,7]. 

2.2  Collections 

DFT  collections  are  built  either  by  a group  of  collaborating  researchers  or  by  individual  researchers.  In 
either  case,  the  collections  are  publicly  available  and  all  scientists/researchers  can  participate  in  verifying 
and  validating  the  result.  Researchers,  who  are  geographically  distributed,  collaborate  using  the  various 
tools  provided  by  the  DFT  project  to  build  and  maintain  the  following  collections: 

• The  Herbarium  Specimen  Browser  is  the  Internet’s  only  multi-herbarium  specimen  data  portal.  It 
offers  unique  filtering,  mapping  and  listing  displays  for  a growing  mass  (more  than  240,000  currently) 
of  specimens  taken  from  the  participating  Texas  herbaria  [8]. 

• The  Image  Gallery  provides  access  to  8000  images  of  plants  representing  families  found  in  Texas  [9]. 

• The  Contributor  collection  contains  contact  information  for  all  contributors  who  generate  digital 
content  for  this  community-based  collaborative  digital  library  [10,1 1]. 

The  following  collections  are  a result  of  individual  research: 

• The  Bibliography  (Wilson)  provides  over  3000  bibliographic  references  for  the  Texas  Flora  [12]. 

• The  Checklist  of  the  Vascular  Plants  of  Texas  (Hatch,  et  al)  refers  to  180  families,  --1300  genera,  and 
---5000  species  [13]. 

• The  Centex  Flora  (Reed)  is  a manual  of  the  dicot  flora  of  Brazos  and  surrounding  counties  that 
provides  keys  and  descriptions  for  104  families,  489  genera  and  1 104  species  [14,15]. 

• The  Texas  Grasses  (Hatch  and  Dawson)  has  keys,  descriptions  and  images  for  142  genera  and  -^550 
species  in  the  grass  family  [16]. 

3.  Design  of  a Collaborative  Digital  Library 

3.1  Model  of  Collaboration/Tnteraction  with  System  Components 

In  the  design  of  a scientific  collaborative  digital  library,  it  is  of  utmost  importance  for  the  contributor  to 
maintain  ownership  of  his/her  scientific  and  intellectual  property.  The  notion  of  ownership  and  control  is 
maintained  in  the  DFT  because  the  contributors  decide  when  to  share  their  data  and  what  part  of  their  data 
to  share.  Automation  is  used  only  under  control  of  the  contributor  and  is  a two  step  process:  1)  uploading 
data  to  share  and  2)  incorporating  data  into  the  public  collection.  The  sharing  occurs  at  different  times  and 
from  different  geographic  locations. 

3.2  Rationale  for  Choice  of  Tools 

The  DFT  is  using  the  previously  defined  Flora  of  Texas  Consortium  (FTC)  data  model  [17]  as  the  common 
data  format  for  its  collections,  extensible  Markup  Language  (XML)  [18]  is  ideal  for  the  FTC  data  model 
since  it  renders  a strong  sense  of  structure  to  data  and  can  be  validated  and  maintained  with  little  or  no 
programming.  A wide  variety  of  Application  Programming  Interfaces  (APIs)  are  also  available  that  can 
convert  a wide  range  of  data  formats  to  XML. 

Greenstone  is  an  open  source  digital  library  system  that  automatically  provides  multiple,  user-defined 
browsing  and  searching  indexes  on  the  collections  it  maintains.  It  provides  plugins  for  converting  many 
standard  format  files  (e.g.  Word,  PDF)  to  XML  and  allows  rebuilding  of  collections  while  they  are  in  use 


(no  down  time).  It  also  provides  a standard,  yet  customizable  user  interface  and  compresses  its  documents, 
thereby  saving  space  and,  more  importantly,  mitigating  the  effect  of  long  XML  tags. 

3.3  System  Architecture 

Contributors  can  upload  their  data  and  build  or  rebuild  a collection  in  the  DFT  at  any  time.  The  contributors 
may  use  any  program  they  wish  to  maintain  their  personal  collections  (e.g.  Word,  Excel,  Access,  Filemaker 
Pro).  When  they  are  ready  to  upload  their  data,  they  dump  it  in  a flat  ASCII  file  (an  operation  that  is 
standard  with  all  data  management  tools).  The  upload  system  converts  the  data  into  XML  and  validates  it 
against  the  FTC  data  model.  Errors,  if  any,  are  reported  to  the  contributors.  Records  without  errors  are 
uploaded  and  are  ready  to  be  added  to  the  collection.  The  uploaded  data  is  not  shared  with  the  other 
collaborators  until  the  contributor  explicitly  builds  or  rebuilds  a collection  of  this  validated  data,  making  it 
available  for  public  use.  Other  collaborators  as  well  as  non-specialists  with  an  interest  in  the  domain  can 
access  the  collection  using  the  common  web-based  user  interface  provided  by  Greenstone  or  by 
customizing  their  own  user  interface.  This  process  relieves  the  scientists  of  the  tasks  of  converting  data 
into  various  acceptable  formats  enabling  them  to  channel  more  time  and  effort  toward  practicing  their 
science  (Figure  1). 

The  Herbarium  Specimen  Browser,  Image  Gallery  and  Contributor  collections  mentioned  in  section  2 are 
collections  that  are  built  and  maintained  by  geographically  distributed  contributors  and  follow  the  same 
methodology  as  described  above.  To  upload  his/her  data,  the  contributor  uses  the  web-based  upload  system 
[19]  to  specify  the  number  and  order  of  fields,  field  separators  and  field  enclosures.  Figures  2 and  3 show 
the  upload  form  and  an  example  search  results  page  for  the  Image  Gallery  collection. 


4.  Lessons  Learned 

4.1  Scientific  Practice 

“The  products  of  Systematic  Botany,  previously  generated  locally  as  static,  hardcopy  documents,  can  now 
be  presented  as  collaborative  enterprises  from  distributed  centers  as  high-content,  dynamic  data  resources 
that  are  constantly  updated  and  refined.”  [1]  The  system  architecture  of  the  DFT  enhances  the  sharing  and 
browsing  aspects  of  collaboration  by  providing  a web-based  input  system  that  converts  the  uploaded  data  to 
standard  XML  format.  This  data  is  then  added  to  the  shared  collection  under  control  of  the  contributor. 
Thus,  scientists  can  follow  this  process  without  disrupting  their  normal  working  style.  This  automated 
process  shortens  the  time  to  share  scientific  information  and  the  notion  of  ownership  of  intellectual  and 
scientific  property  is  maintained,  as  the  scientists  control  their  own  collections  and  share  whenever  they  are 
ready.  The  upload  system  also  filters  the  data  and  performs  various  error  checks  (e.g.  typographical  error 
checks)  in  order  to  verify  the  data.  Scientists  appreciate  the  fact  that  their  data  is  verified  by  the  input 
system  and  by  the  information  visualization  tools  (e.g.  the  mapping  system)  provided  to  browse  through  the 
digital  library. 

4.2  Education 

We  have  observed  that  collaborators  use  and  reuse  either  their  own  information  content  or  that  of  the  other 
collaborators  in  the  digital  library  to  build  new  course  structures  and  educational  packages.  Tools  provided 
with  the  digital  library  expedite  the  process  of  reuse  and  construction  of  new  packages.  Students  are  also 
presented  with  a larger  and  more  diverse  collection  and  are  not  restricted  by  the  limitations  of  a localized 
collection.  In  most  cases,  we  observed  that  collaborators  found  it  useful  to  have  cross  collection 
querying/browsing,  wherein  the  results  of  a search  on  one  collection  are  used  to  search  an  entirely  different 
collection.  This  exploratory  learning  methodology  helps  students  find  information  they  are  looking  for 
while  following  trails  that  interest  them  the  most.  Greenstone  supports  this  cross  collection 
querying/browsing  model,  enabling  the  user  to  seamlessly  move  from  one  collection  to  another. 


4.3  Digital  Libraries 

The  immediate  benefit  of  having  a collaborative  digital  library  is  the  continuous  updating  and  refinement  of 
the  information  content  and  accuracy  of  information  provided  by  the  collaborative  interactions  of  the 
scientists.  Since  it  is  fairly  easy  to  maintain  and  browse  the  digital  library,  we  find  more  and  more 
collections  being  built  using  the  system  we  have  described.  For  example,  the  Image  Gallery  collection  was 
developed  because  the  scientists  involved  find  the  model  of  collaboration  easy  to  use.  Collections  do 
indeed  have  their  own  unique  identities  and  are  a scientifically  valid  resource,  but  they  also  tend  to  become 
integrated  in  the  notion  of  a digital  library  that  allows  cross  collection  query  ingA)rowsing. 
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Figure  1.  Architecture  for  DFT  Collaborative  Digital  Library  Collections, 
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Figure  2,  Example  upload  Form  For  the 
Image  Gallery  Collection. 


Figure  3.  Example  search  results  page  For  the 
Image  Gallery  Collection. 
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5.  Future  Work 


Currently,  the  system  does  not  have  collaborative  tools  to  support  a peer  review  process  by  a group  of 
curators  [20].  With  this  set  of  tools,  any  person  with  botanical  interests  and  a computer  connected  to  the 
Internet  could  contribute  to  the  digital  library.  The  curators  would  be  responsible  for  accepting  or  rejecting 
contributions  and  for  maintaining  high  quality  content.  Considering  the  ease  of  creating  and  maintaining 
collections,  we  would  like  to  put  more  content  in  the  digital  library  in  the  form  of  a monthly  journal,  a 
collection  of  thesis  and  dissertations,  and  a collection  of  videos  of  lectures  and  field  trips;  thus  adding 
various  media  formats  to  the  digital  collection  and  enhancing  the  value  of  the  collections.  Currently, 
collaborating  scientists  and  university  students  are  the  main  beneficiaries  of  the  DFT.  It  would  also  be 
interesting  to  see  how  well  the  system  scales  to  the  needs  of  primary  and  secondary  students. 
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Abstract:  Chydenius  Institute  (Research  and  Training  Centre  of  Jyvaskyla  University) 
implements  the  Finnish  Virtual  University.  In  Kaustinen  and  Kokkola  regions,  which  are  rural 
areas,  is  created  a network  of  learning  centres  in  municipalities.  In  this  way  every  citizen  has 
the  possibility  to  develop  him/herself  occupationally  or  because  of  interest.  The  learning 
centres  have  computers  with  ATM -network  and  videoconferencing  facilities.  Every  citizen  is 
able  to  come  to  study  in  web-courses,  to  surf  in  internet  or  to  take  part  in  conversations  in 
newsgroups  or  posting  lists.  There  are  organized  open  lessons  in  real-time  via  two-way 
videoconferencing  where  people  can  follow  experts  in  different  areas  of  life.  Speeches  of 
Studia  Generalia  offer  interesting  themes  and  discussions  of  current  topics.  In  this  way  people 
have  the  possibility  to  study  and  distance  makes  no  difference:  we  are  all  equal. 


Chydenius  Institute  is  a Network  University  for  adults.  It’s  located  in  Kokkola,  in  the  western  coast  of  Finland. 
Administrative  it’s  an  autonomous  Research  and  Training  Centre  of  Jyvaskyla  University.  Functionally 
Chydenius  Institute  belongs  also  to  the  University  of  Oulu  because  of  the  agreement  of  cooperation.  Under 
discussion  is  the  agreement  with  the  University  of  Vaasa. 

Practically  Chydenius  Institute  cooperates  with  universities  in  these  areas 
University  of  Jyvaskyla:  e.g.  educational  and  social  sciences, 
information  technology 

University  of  Oulu:  e.g.  medicine,  electronic  engineering 
University  of  Vaasa:  economics 

In  addition  to  this,  Chydenius  Institute  implements  the  Finnish  Virtual 
University.  This  consortium  of  all  the  universities  in  Finland  has  a 
purpose  to  develop  web-based  learning  and  implement  the  strategy  of  the 
National  Information  Society  announced  by  the  Ministry  of  Education. 

Finland  is  one  of  the  largest  countries  in  Europe.  There  are  plenty  of  rural 
areas  and  long  distances.  That’s  why  it  has  been  natural  to  develop 
distance  learning  methods.  Videoconferencing  has  become  very  common 
way  of  delivering  teaching  in  educational  institutes  and  schools,  but  also 
companies  are  used  to  have  it  as  a communication  tool  between  them. 

Chydenius  Institute,  an  Adult  Network  University,  offers  most  of  the 
lessons  for  Master  of  Electronic  Engineering  via  videoconferencing, 
using  IP-  and  ISDN-based  technology.  The  students  are  working  daytime  and  participate  in  lessons  during 
evenings  and  weekends.  Videoconferencing  makes  possible  for  them  to  work  and  stay  with  their  families 
without  travelling  to  the  University  of  Oulu  (at  250  km  from  Kokkola).  The  same  model  is  implemented  with  the 
Master  studies  of  Information  Technology  organized  by  University  of  Jyvaskyla.  Because  the  Ostrobothnia 
region  has  an  ATM-network,  there  has  been  delivered  earlier  mentioned  lessons  also  for  homes  of  some 
students. 
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During  the  academic  year  2001-2002  the  Open  University  of  Oulu  University  has  offered  15  credits  of 
Educational  Technology.  This  training  has  delivered  for  Chydenius  Institute  in  Kokkola  and  Kaustinen  using 
ISDN-bridge.  There  has  been  given  lectures  also  from  other  universities  using  videoconferencing. 

In  Kaustinen  and  Kokkola  regions,  which  are  rural  areas  in 
Central  Ostrobothnia,  has  been  created  a network  of 
learning  centres  in  municipalities.  In  this  way  every  citizen 
has  the  possibility  to  develop  him/herself  occupationally  or 
because  of  interest.  The  learning  centres  have  computers 
with  ATM-network  and  videoconferencing  facilities.  Every 
citizen  is  able  to  come  to  study  in  web-courses,  to  surf  in 
internet  or  to  take  part  in  conversations  in  newsgroups  or 
posting  lists.  There  are  organized  open  lessons  via  two-way 
videoconferencing  where  people  can  follow  experts  in 
different  areas  of  life.  Speeches  of  Studia  Generalia  offer 
interesting  thems  and  discussions  of  latest  happenings.  In 
this  way  people  have  the  possibility  to  be  aware  of  recent 
research  and  information. 

There  has  been  offered  training  for  enterprizes  and 
companies  in  Kokkola  and  Kaustinen  region  using  this 
learning  centre  network  that  Chydenius  Institute  has 
organised.  Municipality  of  Lestijarvi  has  own  IP  and  ISDN 
bridge/gateway,  which  has  been  utilised  in  this  training. 

Also  associations  and  communities  use  the  network  of 
learning  centres. 

Learning  centres  are  located  in  municipalities  depending  on  their  own  possibilities.  They  might  be  in  school, 
library,  folk  institute  or  municipal  office.  Every  municipality  has  invested  for  the  infrastructure:  in  a learning 
centre  there  should  be  a number  of  computers  with  ATM-network  and  videoconferencing  facilities.  These 
learning  centres  are  open  either  office-hours  or  in  the  evenings.  Every  municipality  decides  according  to  the 
needs  and  practicalness. 

Citizens  who  don’t  have  computers  at  home  or  they  don’t  have  low  modems  are  very  welcome  to  use  the 
facilities  of  the  learning  centre.  In  this  way  everybody  has  the  possibility  to  study  interesting  subjects  in  open 
universities,  in  web-courses  and  also  in  learning  environments  that  the  Finnish  Virtual  University  offers.  People 
are  able  to  develop  their  knowledge  of  ICT  and  utilize  the  use  of  ICT.  University  studies  and  networks  come 
close  to  everybody  and  distance  makes  no  difference:  We  are  all  equal! 


References:  www.chydenius.fi 
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Abstract:  In  this  paper  we  present  the  way  a multi-criteria  decision  making  methodology  is  applied  in 
the  case  of  agent-based  selection  of  offered  learning  objects.  The  problem  of  selection  is  modeled  as  a 
decision  making  one,  with  the  decision  variables  being  the  learner  model  and  the  learning  objects’ 
educational  description.  In  this  way,  selection  of  educational  content  is  based  on  dynamic  data  input 
collected  at  the  time  of  the  decision.  This  methodology  is  studied  in  the  context  of  an  agent-based  e- 
market  for  educational  content  brokering,  and  is  engaged  by  the  broker  agents  recommending  learning 
objects  to  learners,  according  to  their  cognitive  style. 

Introduction 

In  agent-mediated  educational  content  brokering,  artificial  agents  are  responsible  of  locating  offered 
learning  objects,  negotiating  with  the  content  providers  on  the  terms  of  the  service  provision,  and  for  presenting  the 
learner  with  those  offers  that  best  match  his/her  needs  and  preferences.  Successful  completion  of  this  task  is  based 
on  two  parameters:  the  capability  of  the  mediating  agent  to  understand  and  model  the  user  needs,  and  its  capability 
to  evaluate  all  available  learning  objects  and  recommend  the  most  suitable.  The  most  common  ways  to  introduce 
intelligent  brokering  for  educational  content  are  based  on  techniques  from  Artificial  Intelligence  (AI):  knowledge 
representation,  reasoning,  and  expert  systems,  often  offer  tools  and  techniques  in  the  developers  of  such  systems 
since  they  allow  the  agent  to  behave  intelligently  in  two  ways;  first,  with  the  ability  to  store  the  knowledge  of  the 
experts  (for  example  using  a knowledge  base);  second,  using  the  previous  knowledge  to  infer  rational  decisions. 

In  the  last  years  though,  most  agent-based  e-commerce  systems  engage  methodologies  and  tools  which 
originally  come  from  sciences  other  than  AI;  these  include  methods  and  techniques  from  Game  Theory,  Optimization 
and  Decision  Making  (Guttman  et  al,  1998).  In  this  paper,  we  are  going  to  approach  the  problem  of  content  selection 
(know  as  the  recommendation  problem  in  e-commerce  systems)  from  a decision-making  point  of  view.  We  are  going 
to  introduce  how  a multi-criteria  decision  making  (MCDM)  methodology  is  applied  in  the  case  of  agent-based 
educational  content  brokering,  and  we  are  going  to  discuss  the  benefits  and  drawbacks  of  this  approach. 

There  is  an  interesting  feature  in  this  approach:  it  replaces  searching  knowledge  inside  a knowledge  base, 
using  constraints  posed  by  logical  expressions  and  logic  rules,  with  an  evaluation  of  knowledge  using  mathematical 
formulations.  It  can  therefore  provide  measures  of  “how  much  better  one  decision  is  compared  to  another”,  even  if 
both  decisions  are  rational  and  solve  the  constraints  problem.  This  usually  sounds  rather  awkward  though  - how  can 
we  represent  the  expert  knowledge  in  terms  of  mathematical  expressions?  Logic  variables  (CategoryOfUser=’This’) 
and  logic  rules  (IF  CategoryOfUser=’This’  THEN  TypeOfContent=’That’)  are  simple,  and  easily  comprehended;  how 
is  the  learner  going  to  be  modeled  using  mathematical  variables,  and  how  is  the  expert  going  to  express  the 
experience  in  a mathematical  form? 

We  will  try  to  give  answere  to  these  questions  by  introducing  the  example  of  modeling  the  learners  with  the 
classic  Honey  and  Mumford  cognitive  styles  model  (1992).  We  will  introduce  the  basic  principles  of  a multi-criteria 
decision  making  methodology,  and  study  which  parameters  of  the  Honey  & Mumford  model  serve  as  criteria  for  the 
selection  of  the  available  content  (learning  objects);  we  will  also  study  a method  that  can  be  used  to  carry  out  this 
selection.  Finally,  we  will  present  how  this  decision-making  procedure  is  incorporated  into  an  agent-based  system, 
and  how  the  broker  agents  operate  it. 


ERIC 


The  Honey  & Mumford  Model 


Tennant  (1988)  defined  cognitive  style  as  "an  individuals  characteristic  and  consistent  approach  to 
organizing  and  processing  information".  There  are  several  learning  and  cognitive  styles  theories  and  models,  which 
categorize  learners  in  terms  of  instructional  preferences,  information  processing  and  personality  styles,  and  are 
usually  employed  for  the  realization  of  individualized  instruction  (for  a survey,  see  Sampson  et  al,  2002).  In  order  to 
demonstrate  how  a cognitive  style  model  can  be  used  in  order  to  create  the  multi-criteria  selection  model,  we  will 
engage  the  classic  Honey  and  Mumford  model.  The  Honey  and  Mumford  model  (1992)  is  a cognitive  style  model, 
developed  for  use  in  commerce,  management  and  training  situations.  It  categorizes  learners  according  to  the 
following  dimensions  of  a person’s  learning  style:  Theorist,  Activist,  Reflector  and  Pragmatist.  In  order  to  rate 
learners  on  each  one  of  the  categories,  the  model  uses  the  answers  in  a specially  designed  Learning  Styles 
Questionnaire  (LSQ)  of  80  questions,  with  binary  answers  of  “Correct”  and  “False”,  and  after  an  internal  processing 
of  the  results,  it  provides  a percentage  weight  for  each  category.  Therefore,  the  learner’s  style  is  defined  depending 
on  the  weight  at  each  learning  style  category. 

Leamlng  Styles  Analysts  Honey  A.Hum£Brtl 
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Figure  1:  An  example  learner’s  model  after  a LSQ  is  processed.  It  provides  a measure  of  how  the  learner  ‘scored*  in 
every  category  (source:  PSI-Press,  http://www.psi-press.co.uk/) 

From  a practical  point  of  view,  it  is  important  that  a learner’s  model  describes  not  only  how  learners  are 
categorised,  but  also  how  the  instruction  method  should  be  adapted  for  each  learner  category  (Spector,  2001).  The 
real  complexity  for  the  designers  of  ©-learning  systems  arises  when  they  try  to  match  subject  matter  with  learner 
characteristics  and  appropriate  instructional  methods.  Such  a process  includes  both  learner  modeling  (using  the 
Honey  & Mumford  model  results  ^ see  Figure  1)  and  description  of  the  educational  properties  of  the  learning  content 
(that  is  e.g.  a description  of  how  suitable  a course  is  for  the  learners  having  a specific  cognitive  style).  The  matching 
procedure  involves  all  these  parameters  as  input  - and  still  a proper  matching  mechanism  has  to  be  found.  We 
address  this  problem  as  a decision  making  one. 


Multi-Criteria  Decision  Making  Methodology 

According  to  Roy  (1996)  the  general  methodology  of  decision  making  problems  includes  four  steps:  (i) 
defining  the  object  of  the  decision,  that  is  the  set  of  potential  actions,  and  the  problematic  of  the  decision;  (ii) 
studying  the  parameters  influencing  decision  and  defining  so  a set  of  criteria;  (iii)  choosing  an  appropriate  multi- 
criteria aggregation  method;  (iv)  proceeding  at  the  activity  of  decision  aid. 

Step  One:  The  Object  of  Decision 

The  first  step  includes  definition  of  the  decision  variables  in  a form  of  a consideration  set>4.  In  the  case  of 
educational  content  brokering,  this  set  includes  all  the  available  learning  objects,  which  will  be  evaluated  by  the 
decision  maker  (in  our  case,  the  brokering  agent).  What  is  called  ‘the  problematic  of  the  decision’  is  the  definition  of 
what  kind  of  evaluation  or  choice  does  the  decision  maker  want  to  make  upon  the  different  objects  available  in  setv4; 
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in  the  case  of  broker  agents  recommending  a learning  object,  the  decision  problematic  is  “selection  of  one”.  That  is 
the  selection  of  one  action  from  the  consideration  set  of  the  form  ?-  [aj,  Q2,  a„]. 

Step  Two:  The  Criteria 

The  next  step  is  deciding  what  will  be  the  criteria  upon  which  the  learning  object  will  be  evaluated.  This 
means  defining  a consistent  family  of  criteria,  assuming  that  these  criteria  are  non-decreasing  value  functions, 
exhaustive  and  non-redundant.  Each  criterion  is  defined  on  A as  it  follows: 

[g,,  ,g,’  ] c SR  / a ->  g(a)  e SR  , where  [g,.  ,g*]  is  the  criterion  evaluation  scale,  with  the  worst 

level  of  the  iih  criterion,  g*  the  best  level  of  the  ith  criterion,  (a)  the  evaluation  or  performance  of  action  a on 

the  ith  criterion  and  g(fl)  the  vector  of  performances  of  action  a on  the  n criteria  (Jacquet-Lagreze  & Siskos,  2001). 

In  the  case  of  educational  content  brokering,  there  are  two  ways  to  deal  with  the  selection  of  the  criteria. 
Recommendation  is  usually  based  on  criteria  as  price,  time  of  delivery,  form  of  delivery,  etc.  - a common  practice  in  e- 
commerce  systems.  We  would  like  to  introduce  another  aspect  of  recommendation  in  agent-based  learning 
environments:  based  on  the  pedagogical  profile  of  the  learner.  That  is  using  the  categories  of  the  learner  model  as 
criteria;  each  learning  object  will  be  evaluated  on  how  suitable  it  is  for  each  category  of  learners.  This  is  exactly  the 
same  as  making  an  expression  as  “ TypeOfContent=’That’  ”.  To  be  more  specific,  in  a similar  case  an  expert  system 
would  need  a definition  as:  TypeOfContent="SuitableForActivists".  In  the  case  multiple  criteria  are  used  for 
the  description  of  the  content,  this  definition  is  transformed  (remaining  fully  equivalent)  to  something  similar  to  this: 

• SuitabilityOfContentForTheorists=  "NotSuitableAtAll" ; 

• SuitabilityOfContentForActivists=  "Perfectly  suitable"; 

• Suitabi lityOf Content ForReflectors=  "NotSuitableAtAll" ; 

• SuitabilityOfContentForPragmatists=  "NotSuitableAtAll" ; 

Therefore,  as  criteria  we  will  use  the  four  categories  of  the  Honey  & Mumford,  which  can  take  their  values  from  a 
5-scaled  climax  of  qualitative  descriptions  [“Not  suitable  at  all”,  “Not  very  suitable”,  “Moderately  suitable”,  “Very 
suitable”,  “Perfectly  suitable”]  showing  the  evaluation  of  each  learning  object  upon  each  categories.  It  is  obvious 
that  a learning  object  (e.g.  an  on-line  course)  is  never  “Perfectly  Suitable”  for  a learner  category  and  “Not  Suitable  At 
All”  for  the  rest  -it  rather  addresses  some  of  the  needs  of  other  categories  of  learners  too.  Therefore,  such  an 
evaluation  provides  the  expert  with  a way  to  precisely  describe  the  pedagogical  ‘profile’  of  a learning  object.  This 
meta-data  description  is  a criteria  definition  that  fully  complies  with  the  definition  of  the  criteria  according  to  Roy 
(1996). 

Step  Three:  The  Utility  Model 

First  of  all,  we  have  to  define  the  preference  model  that  will  be  used  from  the  broker  agent  to  make  the 
decision;  as  we  have  already  stated,  in  our  case  we  are  using  the  Honey  & Mumford  learner  model  as  a preference 
model.  The  expression  of  the  model  is  then  pr\pi , P2»  P3>  where  Ph  Ps^Ps  ^ and p4  are  the  weights  inferred  after 
the  learner  has  completed  the  Honey  & Mumford  LSQ  and  the  results  have  been  processed. 

This  is  the  point  where  the  analyst  has  to  define  which  evaluation  method  will  be  used;  that  is,  which  multi- 
criteria decision  making  method  will  be  engaged  in  order  for  the  decision  to  be  best  modeled  and  simulated.  In  this 
paper  we  will  use  one  of  the  most  traditional  approaches,  that  leads  to  a functional  representation  g that  can  be 
formed  directly  from  the  criteria  g/,...^v  that  constitute  A.  The  goal  of  using  this  approach  is  to  present  how  the 
methodology  works  - depending  on  the  application,  other  more  complex  methods  can  be  used.  Thus,  the 
comprehensive  preference  model  is  characterized  by  a unique  synthesizing  criterion  g:  g(a)  = V\gi(a),...,g„(a)],  where 
V is  an  aggregation  function.  The  function  will  be  in  the  form: 

y(a)  = '^Pigi(a) 

where  gi(a)  are  the  evaluations  of  each  learning  object  a regarding  the  suitability  for  each  category  of  learners.  The 
weights  Pi  consist  the  preference  model,  which  are  the  parameters  calculated  by  the  Honey  and  Mumford  LSQ.  The 
final  value  of  V is  the  total  utility  of  each  learning  object  for  the  learner  under  study. 
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Step  Four:  The  Activity  of  Decision  Aid 


Let  us  now  introduce  the  way  the  activity  of  decision  making  will  be  carried  out,  in  the  context  of  an  agent- 
based  system  We  define  a generic  architecture,  with  agents  capable  of  handling  and  communicating  descriptions  of 
learning  objects,  as  defined  in  the  previous  paragraph.  The  three  basic  roles  are: 

The  Assistant.  The  assistant  agents  are  responsible  for  user  needs  and  requirements  elicitation,  and  formulation  of 
requests  into  messages  understandable  by  the  broker  repository  system.  The  LAssistant  is  the  agent  that  interacts 
with  the  user  and  can  identify  the  request  for  a learning  object.  The  CPAssistant  is  the  agent  that  gets  the  learning 
object  descriptions  from  the  Content  Providers,  and  publishes  them  as  offers  in  the  e-market.  The  TAssistant  serves 
as  the  pedagogical  counselor  of  the  learner:  it  creates  the  learner  model  of  the  user  and  provides  this  model  to  the 
LBroker.  The  TAssistant  can  also  be  the  expert  that  describes  a learning  object  using  the  cognitive  styles;  this  is  a 
task  that  can  be  directly  carried  out  by  the  Content  Author  too,  so  we  will  not  focus  more  on  this  point. 

The  Broker.  The  broker  agents  represent  each  human  user  in  the  e-market  and  facilitate  educational  content  seeking 
or  advertising.  The  Broker  agents  interact  among  each  other  in  the  Brokerage  Pool  and  either  advertise  published 
learning  objects,  or  make  requests  when  looking  for  them.  The  LBroker  evaluates  the  available  offers  from  the 
CPBrokers  in  the  way  described  in  the  previous  paragraphs.  The  LBroker  then  recommends  the  content  selection  to 
the  LAssistant.  The  Brokers  are  also  responsible  for  negotiation  among  the  participating  parties  upon  the  terms  of 
the  service  provisioning  (beyond  the  scope  of  this  paper). 

The  Matchmaker.  This  facilitator  agent  provides  mediating  services,  by  informing  agents  about  other  agents  of  the 
broker  repository  and  their  availability.  The  Matchmaker  provides  all  necessary  administrative  support  information  to 
the  human  administrator  of  the  agent  pool. 

We  can  see  that  the  Broker  agent  engaging  the  multi-criteria  methodology  previously  described  is  the 
LBroker.  In  order  to  pedagogically  evaluate  the  available  learning  objects  and  recommend  one  to  the  learner,  the 
LBroker  needs  the  learner  model  (provided  by  the  user)  and  the  meta-description  of  the  available  learning  objects 
(provided  by  the  CPBroker  or  the  Tutor).  Other  kinds  of  evaluation  of  the  offers  (e.g.  based  on  the  price,  time,  and 
other  terms  of  delivery)  are  beyond  the  scope  of  this  paper. 


Usage  Scenario 

Let  US  demonstrate  how  the  recommendation  procedure  works  in  this  agent-based  system.  A content  author 
creates  different  e-learning  courses  concerning  the  same  subject.  The  author  wishes  those  courses  to  address 
different  learner  needs,  depending  on  the  cognitive  style  of  the  learner.  When  a learning  object  (course)  is  created, 
the  content  author  also  provides  an  evaluation  of  its  suitability  for  each  different  category  of  the  Honey  & Mumford 
model.  The  author  creates  thus  five  different  learning  objects,  with  different  pedagogical  descriptions  (as  depicted  in 
Table  1). 


LOl 

L02 

L03 

L04 

LOS 

Activist 

3 

2 

5 

3 

5 

Reflector 

1 

3 

1 

2 

2 

Theorist 

4 

5 

2 

1 

1 

Pragmatist 

3 

2 

1 

5 

2 

Table  1:  The  content  author  provides  an  description  of  each  learning  object  (LO)  on  a scale  from  ‘1-  Not  Suitable  At 
Air  to  ‘5-Perfectly  Suitable’,  regarding  each  one  of  the  cognitive  categories. 
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Activist 

Reflector 

Theorist 

Pragmatist 

LSQ  values 

90% 

55% 

25% 

85% 

Normalized 

0,35 

0,22 

0,10 

0,33 

Table  2:  The  cognitive  styles  preference  model,  as  derived  from  the  Honey  and  Mumford  Learning  Styles 
Questionnaire  (values  are  properly  normalized  in  order  to  provide  a sum  of  1). 


LOl 

L02 

L03 

L04 

LOS 

Utility 

2,67 

2,51 

2,51 

3,25 

2,96 

Table  3:  The  total  utilities  of  the  available  learning  objects  (a  score  upon  the  ’1-Not  Suitable  At  All*  to  *5-  Perfectly 
Suitable’  climax). 

Introducing  the  multi-criteria  methodology  presented  in  the  previous  section  the  Broker  will  select  a learning  object 
according  to  these  steps: 

1.  The  Tutor  provides  a multi-attribute  cognitive  model  of  the  user,  according  to  the  results  of  the  Honey  and 
Mumford  model  (Table  2). 

2.  When  the  Learner’s  Broker  receives  the  five  offers  for  learning  objects,  it  calculates  the  total  utility  (that  is 
suitability)  of  each  one,  using  the  methodology  described  in  the  previous  section. 

3.  The  Learner’s  Broker  selects  the  most  appropriate  learning  object  to  be  presented  to  the  user  (Table  3). 

It  is  interesting  to  note  in  this  example  the  difference  between  the  application  of  a simple  rule-based  selection  of  the 
learning  object  (in  this  case  the  agent  would  classify  the  learner  as  ‘Activist’  and  would  propose  the  learning  object 
L03)  and  the  proposed  methodology.  Using  the  multi-attribute  cognitive  model  and  the  multi-criteria  evaluation 
model,  we  observe  that  the  most  appropriate  learning  object  seems  to  be  L04,  being  the  only  one  suitable  for  the 
combination  of  categories  that  characterize  the  specific  learner’s  cognitive  style. 


Conclusions  and  Future  Work 

In  this  paper  we  presented  an  agent-based  recommendation  system  for  educational  content  brokering  that 
implements  a multi-criteria  decision  making  methodology.  Due  to  the  differences  of  learners’  cognitive  models, 
recommendation  can  be  either  carried  out  by  classifying  learners  into  broader  categories  that  then  are  directly  linked 
to  learning  objects  ^.g.  with  rules  like  IF  ‘thistypeofleamer’  THEN  ‘thistypeofmaterial’)  or  by  enhancing  the 
mediating  agent  with  the  appropriate  intelligence  to  propose  the  most  suitable  learning  object  available  at  the  market. 
By  engaging  a multi-attribute  utility  (MAUT)  model,  the  brokering  agents  do  not  need  to  posses  intelligence  in  the 
form  of  a knowledge  base  or  hard-coded  rules  but  only  to  include  the  multi-attribute  evaluation  logic  in  their  decision 
making  module.  In  this  way  learners  are  modeled  in  a multi-attribute  way,  instead  of  being  simply  classified  into  a 
general  learning  category,  learning  objects  are  described  according  to  their  educational  scopes,  brokering  agents 
dynamically  evaluate  all  available  proposals  of  content,  and  select  the  learning  objects  most  suitable  for  the  specific 
learner’s  profile,  and  finally  the  knowledge  the  broker  agents  have  to  contain  in  permanent  storage  is  minimized  since 
only  the  MAUT  evaluation  formula  is  needed;  all  other  input  is  provided  at  run-time. 

In  this  paper,  the  decision  problem  was  modeled  using  the  Honey  and  Mumford  LSQ  parameters  as  the 
preference  model,  but  we  believe  that  the  proposed  methodology  can  be  similarly  applied  in  the  case  of  other 
cognitive  style  models  too.  Future  research  will  be  focused  on  applying  of  several  other  LS  models,  either  separately 
or  combined.  An  issue  to  be  also  studied  is  the  application  of  more  complex  multi-criteria  decision  making 
methodologies  for  modeling  the  user  preferences;  such  methodologies  provide  easier  and  more  realistic  means  of 
expressing  preferences,  but  require  more  complicated  MCDM  methods  to  conclude  to  the  best  proposal  for  each 
learner,  as  one  of  the  famous  family  of  the  ELECTRE  methods  (Roy,  1996). 

Current  international  standardization  efforts  regarding  syntax  and  semantics  of  a learner  model  that 
characterize  a learner  and  his  or  her  knowledge/abilities;  i.e.  IMS  LIP  (IMS,  2001),  IEEE  LTSC  PAPI  (IEEE,  2000)  do 
not  explicitly  define  information  about  the  cognitive  style  of  the  learner.  However,  both  the  IMS  LIP  and  IEEE  PAPI 
Learner  preference/performance  data  types  provide  data  elements  that  allow  cognitive  style  related  information  to  be 
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stored  in  an  appropriate  form.  These  data  types  could  be  used  in  order  to  represent  a multi-attribute  representation  of 
the  cognitive  style  model  of  the  learner,  by  an  appropriate  extension  of  the  corresponding  XML  schema. 

The  1ST  Project  KOD  “Knowledge-On-Demand'*  (KOD,  2001)  engages  a multi-agent  brokering  system  for 
selection  of  learning  objects  and  dynamic  synthesis  of  on-line  courses.  In  the  case  of  KOD,  intelligence  is  included 
into  the  learning  objects  in  the  form  of  rules  that  express  the  suitability  of  the  object  for  each  type  of  learners,  so  that 
courses  can  be  constructed  based  on  how  the  learning  objects  are  characterized.  This  is  a case  very  close  to  the 
problem  we  introduced,  and  we  are  currently  studying  ways  to  introduce  the  multi-criteria  descriptions  in  the  learning 
objects  of  KOD,  in  order  for  the  KOD  agents  to  be  able  to  dynamically  assess  and  synthesize  courses  based  on  the 
introduced  decision  making  methodology  and  not  on  prescribed  rules. 
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Abstract:  The  University  of  Idaho  has  been  redesigning  its  teacher  preparation  program  for 
four  years.  The  redesign  has  included  the  development  of  a set  of  core  courses  that  are 
required  for  all  teacher  certifications.  These  four  courses  include  a “strand”  of  content  about 
students  with  disabilities  and  methods  for  working  with  them  in  inclusive  settings.  The 
special  education  content  in  the  final  course  is  focused  on  mediating  the  interns’  experiences 
in  their  yearlong  internship  placements,  which  are  all  over  the  state.  We  are  currently 
developing  an  online  support  structure  for  the  students  to  complete  the  special  education 
component  of  the  final  core  course.  This  paper  describes  that  online  structure  that  also 
includes  a method  for  collecting  data  about  the  interns’  use  of  the  system  and  the  content  they 
submit. 


Introduction 

Research  on  models  of  training  teacher  candidates  for  their  work  in  inclusive  environments  has  shown 
that  special  education  content  integrated  into  existing  coursework  with  an  accompanying  field  experience  is 
more  effective  than  other  models  studied.  Further,  recent  research  has  shown  that  the  integration  of  online 
delivery  modes  with  traditional  instructional  practice  is  effective.  Therefore,  the  integrated  model  we  are 
implementing  to  provide  continued  support  during  the  internship  holds  a great  deal  of  promise  for  helping  our 
teacher  candidates  to  become  effective  inclusive  educators. 


Issues  in  the  Preparation  of  Inclusive  Educators 

Teacher  candidates  enter  their  programs  with  clusters  of  beliefs  based  on  their  experiences  as  students, 
values,  and  other  background  determinants.  These  determinants  influence  how  they  approach  their  programs 
and  their  developing  perceptions  of  what  it  means  to  be  a teacher  (Richardson  1996). 

Teacher  candidates’  entering  beliefs  about  students  and  learning  will  be  challenged  when  they 
encounter  student  with  disabilities  in  the  general  education  classroom.  The  belief  that  teaching  is  a matter  of 
telling  or  showing  and  learning  is  committing  information  to  memory  will  come  into  question  since  many 
students  with  disabilities  have  difficulties  with  this  kind  of  teaching  practice  (McDiarmid  1989). 

These  beliefs  about  teaching  and  learning  make  it  difficult  for  teacher  educators  to  help  teacher 
candidates  understand  instruction  about  accommodating  students  with  disabilities.  Anderson  and  Bird  (1995) 
point  out  that,  based  on  their  lack  of  exposure  to  inclusive  practices,  teacher  candidates  may  have  difficulties 
assimilating  instruction  about  accommodating  students  with  disabilities  into  their  current  understandings  of 
teaching  and  learning.  Mediating  these  beliefs  is  especially  critical  for  facilitating  the  progress  of  inclusive 
efforts  in  schools. 

Teacher  educators  have  attempted  numerous  designs  to  train  general  education  teacher  candidates 
specifically  for  effective  inclusive  practices.  These  designs  have  ranged  from  one-time  training  events,  to 
separate  courses,  to  the  infusion  of  special  education  content  into  the  general  education  curriculum  (Bradley, 
King-Sears,  & Tessier-Switlick  1997). 

Rademacher,  Wilhelm,  Hildreth,  Bridges,  and  Cowart  (1998)  studied  changes  in  the  attitudes  of 
general  education  teacher  candidates  toward  students  with  disabilities  who  participated  in  one  of  three  possible 
course  configurations.  The  researchers  concluded  that  the  teacher  candidates  who  had  direct  contact  over  time 
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with  students  with  disabilities  in  regular  and  special  education  placements,  along  with  reflective  discussions  and 
assignments,  appeared  to  have  a significant,  positive  impact  on  their  attitudes  toward  students  with  special 
needs,  inclusion,  collaborative  teaching,  and  their  self  confidence. 

Description  of  Online  Support  Structure 

The  online  support  structure  will  offer  resources  to  help  the  teacher  candidates  as  they  make  the 
transition  from  student  to  intern  and  work  within  the  inclusive  environments  found  in  today’s  schools.  First  and 
foremost,  an  online  support  system  will  provide  a syndironistic  communication  system  to  allow  for  immediate 
discussion  of  issues  and  concerns  with  field  placement  personnel.  As  has  been  demonstrated,  (Seabrooks, 
Kenney,  & LaMontagne  2000)  interns  benefit  from  online  mentoring.  The  interns  will  benefit  from  online 
conversations  among  themselves.  They  will  learn  from  each  others’  experiences  and  gain  information  about 
various  inclusive  settings.  The  web  site  will  provide  the  interns  with  links  to  information  that  will  help  them  to 
problem  solve  and  make  instructional  decisions.  The  interns  will  also  use  the  site  to  post  completed 
assignments  for  the  special  education  portion  of  the  final  core  course. 

All  of  the  online  data  will  be  archived  for  purposes  of  qualitative  and  quantitative  research  analysis. 
By  examining  the  frequency  of  use,  nature  of  the  discussions,  and  coursework,  these  data  will  provide  a 
continuation  of  the  data  collection  that  started  in  earlier  core  courses  around  inclusive  attitudes  and  practices. 
We  will  be  contributing  to  the  present  research  about  how  teacher  candidates  learn  from  their  programs  and 
utilize  that  information  in  field  experiences.  We  will  also  be  able  to  learn  more  about  the  challenges  currently 
faced  by  teachers  practicing  in  inclusive  environments  through  the  content  of  the  discussions.  All  of  the  data 
will  help  the  teacher  preparation  faculty  at  the  University  of  Idaho  to  improve  the  content  and  structure  of  the 
core  courses. 


Future  Directions 

Along  with  continuing  to  improve  the  online  support  structure  based  on  the  data  collected,  we  plan  to 
expand  the  content  to  serve  a broader  audience,  i.e.,  the  supervising  teachers.  We  would  like  to  offer  an  option 
of  the  final  core  course  that  would  invite  the  participation  of  supervising  teachers.  They  could  take  “Inclusive 
Strategies”  for  one  credit  and  complete  projects  with  their  interns  based  on  models  of  action  research.  This 
practice  would  help  the  supervising  teacher  to  improve  the  work  being  done  in  the  classroom  and  provide  the 
intern  with  a model  of  data  collection  and  analysis  that  could  be  utilized  in  the  future. 
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Abstract:  Because  the  explosion  of  electronic  and  print  information  requires  that  productive 
citizens  be  able  to  locate,  evaluate,  synthesize,  and  communicate  ever-increasing  amounts  of 
text,  hypertext,  visual  stimuli,  and  data,  the  need  for  information  literacy  among  students  is 
paramount.  However,  perceptual  barriers  to  implementation  by  both  teachers  and  librarians 
keep  information  literacy  a much  talked  about,  much  written  about,  yet  infrequently  attained 
goal  in  K-12  education.  This  paper  discusses  obstacles  to  K-12  information  literacy  and 
presents  possible  solutions. 


Introduction 

Despite  the  attention  given  to  the  topic  of  information  literacy,  it  has  yet  to  be  realized  among  K-12 
students  in  a widespread  manner.  So,  what  is  in  the  gap  between  the  definition  information  literacy  and  the 
realization  of  information  literacy?  Although  students  are  the  ultimate  beneficiaries  of  information  skills,  gap 
analysis  indicates  that  teachers  and  librarians  in  the  settings  of  classroom  and  libraries  are  key  factors  in  the 
transfer  of  information  literacy  skills.  An  examination  of  the  literature  about  teacher  Internet  use,  perceived 
barriers  to  implementation,  and  potential  vehicles  for  transformation  identifies  some  ways  that  teachers  and 
librarians  can  diagnose  the  distance  from  information  literacy  and  begin  to  close  the  gap. 

How  Do  Teachers  Feel  Well-Prepared  To  Use  Computers  And  The  Internet  In  Teaching? 

A recent  survey  conducted  by  the  National  Center  for  Education  Statistics  (NCES)  found  that  99 
percent  of  full-time  regular  public  school  teachers  reported  they  had  access  to  computers  or  the  Internet 
somewhere  in  their  schools.  Sixty-six  percent  of  public  school  teachers  reported  using  computers  or  the  Internet 
for  instruction  during  class  time;  thirty  percent  reported  assigning  research  using  the  Internet  to  a moderate  or 
large  extent.  The  ways  teachers  direct  students  to  use  computers  or  the  Internet  varied  by  instructional  level, 
school  poverty  level,  and  hours  of  professional  development.  Elementary  school  teachers  are  more  much  likely 
than  secondary  school  teachers  to  assign  students  practice  drills  using  computers  and  to  have  their  students  use 
computers  or  the  Internet  to  solve  problems.  Secondary  school  teachers,  however,  are  more  likely  to  assign 
research  using  the  Internet  (NCES,  2000). 

This  sense  of  technology  confidence  affects  the  ways  teachers  control  knowledge  in  the  classroom. 
Because  information  literacy  is  an  outcome  that  is  linked  to  technology  related  skills,  it  requires  a shift  to  self- 
directed  and  recursive  instruction.  Teaching  strategies,  therefore,  need  to  be  revamped.  This  need  for  alteration 
in  teacher  practice  can  cause  mental  and  practical  confusion  and  discomfort  (Farmer,  1992).  Most  often,  this 
discomfort  manifests  in  perceptions  about  information  literacy  skills  including  insufficient  class  time,  teacher 
fear  of  failure,  and  perception  that  information  literacy  skills  are  the  role  of  another  class. 

Librarians:  Untapped  Colleagues  and  Collaborators 

Librarians  recognize  the  critical  importance  of  their  participation  in  curriculum  development;  however, 
their  actual  involvement  in  collaboration  with  classroom  teachers  does  not  match  the  theoretical  role  and  the 
role  they  were  trained  to  perform  (O'Sullivan,  2000).  Collaborative  planning  requires  a knowledgeable  and 
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flexible  librarian,  with  good  interpersonal  skills  and  a commitment  to  integrated  information  literacy 
instruction,  and  the  active  support  of  the  principal 

Because  students  are  familiar  with  computers,  assumptions  are  made  about  their  knowledge  of 
information  sources.  Researchers  have  demonstrated  that  students  teach  themselves  to  search  for  information  by 
looking  on  the  Web  for  entertainment  related  information  (Dresang,  1999).  Certainly,  this  information  is  not 
closely  scrutinized  for  quality  and  veracity.  Rarely  do  students  need  to  synthesize  information  about 
entertainment  related  issues  and  communicate  in  a sensible  and  holistic  way 

Coordinating  information  gathering  activities  with  a librarian  can  often  a crucial  step  in  the  lesson’s 
success.  Librarians  know  whether  print  or  electronic  resources  would  be  appropriate  for  the  lesson  as  well  as  if 
the  lesson  can  even  be  completed  with  the  school’s  resources.  By  bringing  the  librarian  and  other  colleagues 
into  the  planning  process,  teachers  can  help  the  information  skills  be  distributed  across  classes  as  well  as 
relevant  to  material  being  taught  in  all  classes. 

For  information  literacy,  the  library  is  a crucial  venue  and  the  librarian  is  a crucial  agent  of  change 
(Kulthau,  1 990).  Information  literacy  is  essentially  about  getting  a sense  for  the  range  of  resources  available  and 
how  to  organize  the  information  these  resources  provide.  Librarians  are  able  to  work  with  teachers  and  students 
to  obtain  knowledge,  share  knowledge,  and  keep  track  of  knowledge  systematically. 

Collaborative  Action  Research  Can  “Fix  It  In  The  Mix” 

Action  research  can  help  teachers  and  media  specialists/librarians  understand  important  variables 
related  to  student  Internet  use,  how  students  perceive  it  as  a source  of  learning,  and  the  challenges  educators 
face  in  effectively  introducing  the  Internet  into  the  school  curriculum.  Action  research  linked  directly  to  the 
learning  needs  of  students  and  validated  through  empirical  research  has  the  potential  to  be  the  foundation  for 
introducing  information  literacy  on  a school-wide  basis  (Bruce  & Easley,  2000). 

. The  immensity  and  complexity  of  information  literacy  forces  teachers  into  collaborative  mode.  No 
one  teacher  or  librarian  knows  how  to  locate,  evaluate,  synthesize,  and  communicate  text,  hypertext,  visual 
stimuli,  and  data.  The  integration  of  research  and  teaching  methods  must  be  a shared  endeavor  aimed  at  helping 
students  learn.  (Kieman  & Shaw-Reeves,  2001).  The  empirical  findings  of  action  research  can  act  as  a measure 
of  how  far  students  are  from  being  information  literate.  The  final  component  of  collaborative  action  research  is 
to  present  the  methods  and  findings  of  the  research  to  the  larger  educational  community  (O'Sullivan,  2000). 

By  recognizing  the  power  of  individuals,  collaborative  planning,  using  the  library  as  a setting,  and 
applying  action  research  to  test  results,  perceptual  barriers  to  adopting  information  literacy  activities  such  as 
lack  of  readiness,  insufficient  time,  and  improper  context  can  be  rethought.  When  educators  begin  to 
acknowledge  and  address  these  barriers,  students  will  benefit.  Information  literacy  will  provide  students  with 
the  critical-thinking  skills  they  need  to  transform  information  into  the  valuable  knowledge  required  to  make 
informed  decisions  in  the  21st  century. 
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This  panel  will  present  various  perspectives  on  designing  and  implementing  community-based 
educational  digital  libraries.  The  educational,  social,  and  technical  challenges  of  digital  library 
development  will  be  explored,  in  addition  to  the  promises  that  digital  libraries  and  other  types  of 
distributed  learning  environments  hold  for  improving  accessibility,  developing  community,  and 
advancing  the  educational  reform  agenda  of  particular  disciplines.  An  emerging  digital  library  program, 
the  Digital  Library  for  Earth  System  Education  (www.dlese.org)  will  serve  as  an  instance  of  digital 
library  development  issues  for  the  panel  discussion.  The  presentation  will  also  include  a demonstration  of 
this  library  effort  to  date. 

Position  Statement 

Community  libraries  are  evolving  with  the  information  age.  The  past  decade  has  witnessed  the  increasing 
ubiquity  of  the  World  Wide  Web  in  homes  and  schools,  the  emergence  of  new  kinds  of  “electronic 
communities,”  and  the  widespread  creation  and  distribution  of  digital  educational  materials.  Digital 
libraries  are  becoming  a means  for  disciplinary  communities  to  share,  organize,  and  assess  their 
intellectual  holdings.  To  date,  there  are  many  digital  library  efforts  underway  aimed  at  improving  K-12 
and  undergraduate  science  education. 

Over  the  past  three  years.  The  Digital  Library  for  Earth  System  Education  (DLESE)  has  emerged  to 
support  the  specific  educational  needs  of  the  geoscience  community  within  this  larger  national  library 
network.  In  the  tradition  of  community  libraries,  DLESE  can  fundamentally  change  the  way  students 
learn,  instructors  teach,  and  researchers  interact,  by  providing  new  ways  of  sharing  information,  tools,  and 
services.  As  such,  DLESE  provides  unprecedented  opportunities  for  scientific  learning  and  discovery,  for 
increased  access  to  and  diversity  within  the  Geosciences,  and  for  revolutionizing  teaching  and  learning 
about  the  Earth. 

The  value  of  DLESE  lies  in  its  potential  to  scale  success  on  a national  level.  At  the  technical  level,  we  are 
building  on  the  ubiquity  of  Internet  and  web  technologies,  and  the  maturation  of  digital  library 
technologies.  At  the  educational  level,  DLESE  scales  successful  innovations  by  providing  a venue  and  a 
forum  for  creators  of  educational  resources  to  share,  on  a national  scale,  both  their  educational  products 
and  their  creativity  and  expertise.  At  the  social  level,  DLESE  provides  a community  platform  for  social 
creativity  and  the  creation  of  intellectual  capital. 

What  is  DLESE? 

DLESE  is  conceived  as  a community-based  resource  that  will  serve  the  unique  needs  of  Earth  system 
educators  and  learners  at  all  academic  levels,  in  both  formal  and  informal  settings,  by  providing: 

Interfaces  and  tools  to  allow  student  exploration  of  geospatial  materials  and  Earth  datasets.  Though  a 
wealth  of  Earth  data  exists  on  the  Web;  much  of  it  is  difficult  for  educators  to  use.  DLESE  will  provide 
student-friendly  access  to  a wide  variety  of  archived  and  real-time  datasets. 

Rapid,  sophisticated  access  to  collections  of  peer-reviewed  teaching  and  learning  resources.  Earth 
science  educators  have  been  frustrated  in  attempts  to  find  high  quality  teaching  resources  appropriate  for 
their  teaching  style  and  educational  level  on  the  Web  in  a timely  manner.  This  resource  discovery 
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challenge  is  being  met  with  the  creation  of  metadata  schemas,  controlled  vocabularies,  and  cataloging 
best  practice  recommendations,  all  informed  by  community  participation. 

Services  to  help  users  effectively  create  and  use  materials,  A full  array  of  digital  and  human-mediated 
services  for  both  users  and  contributors  to  the  library  is  critical  to  the  vision  of  DLESE  as  an  active 
organization  that  both  builds  and  serves  its  community. 

A community  center  to  facilitate  sharing  and  collaboration,  DLESE  will  serve  as  an  intellectual  commons 
for  the  global  Earth  system  community  by  being  the  primary  contact  for  educators,  learners,  and  citizens 
who  seek  reliable  information  about  the  Earth. 

A federated  collection  of  holdings,  DLESE  is  being  designed  from  the  beginning  to  support  resource 
discovery  across  a diverse,  federated  network  of  holdings  and  collections  (e.g.  NASA,  Alexandria  Digital 
Library).  These  distributed  resources  will  be  complemented  by  a small,  centrally  managed  core  collection. 

The  DLESE  Community  Framework 

DLESE  is  a significant  undertaking,  and  our  library  building  effort  is  as  much  a social  experiment  as  it  is 
a technical  challenge.  DLESE’s  community  participation  framework  and  participatory  design  process 
emphasize  inclusiveness  and  promote  a process  of  cultural  change,  and  is  essentially  the  intersection  of 
three  primary  functions:  policy,  operations,  and  community. 

Policy,  Critical  to  the  construction  and  management  of  DLESE  is  a governance  structure  that  supports 
wide  involvement  in  policy  decisions.  Library  policy  is  set  by  the  DLESE  Steering  Committee,  broadly 
representative  of  Earth  science  education  in  K-16  and  informal  education,  informed  by  Standing 
Committees  (Collections,  Technology,  Users,  and  Services)  reflecting  the  diverse  needs  of  the  community 
DLESE  seeks  to  serve. 

Operations,  The  DLESE  Program  Center  (DPC)  performs  key  operational  functions  for  the  library,  and  is 
headquartered  at  the  University  Corporation  for  Atmospheric  Research  (UCAR)  in  Boulder,  CO.  Critical 
activities  underway  include  the  development  of  the  DLESE  system  architecture,  user  interface,  and 
standards  and  protocols  for  metadata  and  interoperability. 

Community,  A third,  and  crucial,  leg  of  the  DLESE  framework  is  an  informed  and  engaged  community. 

By  shaping  the  conditions  of  library  use,  community  members  are  subsequently  better  positioned  to 
influence  policy  decisions.  Community  members  have  contributed  user  scenarios,  resources,  developed 
review  criteria,  reviewed  metadata  standards,  and  articulated  policies  for  academic  recognition  and 
intellectual  property. 

Accomplishments  over  the  past  year  include:  developing  and  evaluating  a library  testbed,  significantly 
engaging  the  community  in  the  process;  establishing  a community-led  governance  mechanism;  initiating 
collections  development  activities;  and  designing  and  developing  a metadata  schema  and  cataloging  tools. 

Conclusion 

A distinguishing  mark  of  DLESE  is  its  grass-roots  foundation  and  its  emphasis  on  participatory  design. 

As  the  library  grows  and  matures,  DLESE  will  face  the  special  demands  of  scalability  and  sustainability 
that  accompany  such  ambitious  efforts.  Although  there  are  aspects  of  our  experience  that  are  unique  to  the 
discipline  of  Earth  system  education,  we  believe  that  most  of  the  challenges  that  we  have  encountered  are 
applicable  to  the  development  of  distributed  learning  environments  across  a wide  variety  of  disciples. 
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Abstract:  The  purpose  of  this  paper  is  to  present  an  ongoing  research  project  that 
evaluates  and  identifies  the  requirements  and  needs  for  Web  Based  Support  to 
learning  environments  in  an  on-campus  context.  These  environments  supplement, 
with  different  methodologies,  the  traditional  teaching  and  learning  in  engineering 
institutions  of  higher  education.  A set  of  undergraduate  and  postgraduate  courses  of 
the  Porto  Polytechnic  School  of  Engineering  (ISEP)  and  the  Faculty  of  Engineering 
of  Porto  University  (FEUP)  will  be  used  as  test  beds 


Introduction  and  motivation 

Academic  institutions  and  companies  operating  in  the  education  “market”  are  increasingly 
exploring  the  use  of  Information  and  Communication  Technologies  (ICT)  to  develop  teaching  and 
learning  environments  accessible  through  a Web  interface  (Flanagan,  2000,  Collis,  1998).  Diverse 
models  and  patterns  for  these  environments  have  been  developed  and  implemented  in  undergraduate 
and  postgraduate  courses  (Nachmias,  2000,  Seufert,  2000,  Cardoso,  2001). 

like  most  higher  education  institutions,  ISEP  and  FEUP  have  launched  a few  eleaming 
initiatives  on  an  experimental  basis,  whose  results  are  being  evaluated.  However  these  initiatives  have 
not  been  conducted  in  a systematic  way  and  have  not  been  integrated  with  the  information  systems  of 
the  institutions,  taking  profit  of  the  management  and  communication  facilities  offered  by  them.  This  is 
a serious  limitation,  as  “Information  systems  also  constitute  an  important  tool  to  support  the  evolution 
of  the  pedagogical  process  and  it  is  expected  to  contribute  to  the  continuous  education  effort”  (David, 
2000). 


Project  Development 

The  project  is  being  conducted  during  the  present  academic  year,  with  a special  focus  on  the 
Computer  Science  Departments  of  both  Institutions. 

The  first  phase  goal  was  to  identify  the  requirements  for  a Web  Based  Environment,  which 
advantageously  complements  the  traditional  learning  process.  For  this  purpose,  a set  of  tools  was 
designed  to  provide  this  information,  taking  in  account  the  nature  of  the  stakeholders  (higher  education 
institutions,  teachers  and  managers).  The  first  tool  was  a questionnaire  created  to  study  the  perception 
of  the  teaching  staff  in  relation  to  the  introduction  of  e-leaming  in  each  institution  and  to  identify 
possible  areas  where  e-leaming  can  be  used.  The  questionnaire  also  allows  determining  the  statistical, 
technological  and  e-leaming  profile  of  the  teaching  staff  and  the  institutional  profile  of  each  institution. 
The  questionnaire  was  initially  delivered  to  a dozen  teachers  to  assess  its  reliability  (internal 
consistency  and  stability).  For  the  pilot  test;  using  Cronbach's  Alpha  formula;  reliability  for  the  total  of 
the  items  was  + .80.  There  is  general  agreement  that  + .75  or  above  indicates  appropriate  instmment 
internal  consistency  (Woodward,  1983  ). 

Later,  the  questionnaire  was  delivered  in  paper  to  each  teacher  of  the  two  academic 
institutions.  We  have  gotten  above  of  100  responses  in  each  institution,  which  represents  about  21%  of 
the  total  target 

Based  on  the  questionnaire  results  a directed  interview  was  conducted  to  a small  numbers  of 
teaching  staff  and  managers  in  crucial  leading  roles  (department  directors,  scientific  and  pedagogic 
council,  etc.). 


Applicability  in  Engineering  Colleges 


Simultaneously,  an  analysis  was  made  on  existing  Web  Based  Environments.  We  synthesized 
existing  studies  on  these  tools  as  well  as  case  studies  in  other  academic  institutions. 

The  tri angulation  of  the  results  from  the  previous  tools,  allowed  to  organize  the  requirements, 
in  a preliminary  way,  in  terms  of  features  concerning  trainer’s  support  (authoring  facilities  and  course 
management),  learner’s  support  (learning;  collaborative  and  interaction  facilities)  and  technical 
administrators  support 

The  second  phase  of  the  project  (currently  ongoing)  will  look  into  the  adequacy  and 
advantages  of  integrating  Web  Based  Learning  Environments  with  the  information  system  of  the  two 
academics  institution.  The  evaluation  will  be  based  on  the  requirements  and  in  terms  of  its  value  of  use 
and  analysis  of  cost^ene^lt. 

The  first  step  was  to  select,  acquire  and  install  a Web  Based  Learning  Environment  (WBLE) 
for  each  Institution,  based  on  the  respective  requirements. 

The  next  step  in  t his  phase  is  to  integrate  the  WBLE  with  each  information  system,  test  and 
evaluate  the  final  environment.  This  is  currently  being  done  but  has  proved  to  be  a more  difficult 
process. 

We  have  selected  pilot  courses  (and  a corresponding  teaching/leaming  met  hodology)  in  each 
Institution  to  evaluate  the  efficiency  of  the  WBLE  and  its  integration  with  the  Information  System. 
Before  the  courses  start,  an  assessment  of  student’s  technological  profile  was  conducted  through  a 
questionnaire. 

As  it  is  difficult  to  demonstrate  that  a change  in  practice  or  in  knowledge  over  time  can  be 
attributable  to  a single  isolated  resource  (the  WBLE,  in  our  case),  the  Integrative  approach  therefore 
looks  at  how  the  resource  is  used  in  conjunction  with  all  the  other  resources  that  are  available  to  its 
users.  It  will  focus  on  users*  opinions  and  level  of  use  of  the  resource,  and  on  how  it  is  embedded  into 
the  course  as  a whole.  Also  the  opinions  of  the  administrative  staff  and  technician  involved  in  this 
study  will  be  heard. 


Conclusions  and  Expected  Results: 

The  expected  results  are  twofold:  identification  of  the  requirements  and  evaluation  of  the 
effectiveness  of  an  integrated  e-learning/information  system  environment  in  engineering  education. 

We  expect  that  the  results  of  this  project  may  be  useful  for  other  engineering  institutions  of 
higher  education  that  wish  to  create,  manage,  access  and  share  Web  Based  Infrastructures  for  "campusr 
learning”.  We  also  think  that  the  experience  of  this  work  represents  an  alternative  view  about  the  role 
of  learning  Web-based  environments/systems  in  educational  processes. 
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Abstract:  ToolBook  Instructor  is  a comprehensive  desktop  authoring  software  [7]  designed  for  use 
by  a broad  spectrum  of  course  developers  and  instructional  designers  to  create  standards-based 
content.  The  purpose  of  this  paper  is  to  describe  the  development  of  an  interactive  multimedia 
courseware  for  college  algebra  and  precalculus.  The  course  work  developed  using  the  Toolbook 
Instructor  provides  lectures,  examples,  simulations  and  exercises  on  topics  usually  covered  in  a 
precalculus  course. 


1.  Introduction 

The  multimedia  Interactive  courseware  is  in  the  form  an  electronic  book.  The  presentation  of  a topic  is  done  via 
hypermedia.  Text,  sound,  graphics  and  animation  are  used  to  present  a topic.  This  is  followed  by  examples.  The  user 
is  then  provided  with  a chance  to  do  interactive  exercises.  The  multimedia  authoring  software  used  in  writing  the 
precalculus  book  is  Toolbook  Instructor  II  ([7]).  Most  of  the  computer  algebra  used  in  arriving  at  the  solutions  of 
the  mathematics  problems  in  the  book  is  written  in  the  scripting  language  of  the  authoring  software.  Many  of  the 
decision  in  the  presentation  of  the  mathematical  expressions  are  dictated  by  the  limitation  of  the  authoring  software. 
The  book  has  been  used  as  tutorials  for  trigonometry,  algebra  and  precalculus  courses.  Sbveral  chapters  were 
integrated  into  interactive  classroom  lectures.  However,  reliable  evaluation  of  the  courseware  is  still  pending. 
Students  have  given  high  marks  for  the  software  in  the  student  evaluation  of  the  courseware.  The  development  of 
the  courseware  got  its  start  from  the  Minority  Science  Improvement  Program  (MSIP)  funded  by  the  U.  S. 
Department  of  Education  in  1994-  96.  One  of  the  objectives  of  the  project  was  to  develop  interactive  multimedia 
computer  assisted  instructional  (CAI)  modules  for  topics  in  trigonometry. 

3.  Courseware  Design  And  Educational  Issues 

In  developing  the  electronic  book  the  author  looked  at  several  design  issues  that  one  must  consider  in  developing 
interactive  instructional  material.  Many  of  the  earlier  problems  in  designing  and  developing  interactive  educational 
software  are  well  stated  by  Jones,  Farquahar  and  Surrey  [4].  The  uses  of  design  elements  such  as  pull  down  menus, 
multiple  windows,  navigational  buttons,  help  buttons,  and  the  uses  of  program  metaphors  are  discussed  by  Jones  [4]. 
Jones  & Okey  [5]  give  helpful  suggestion  to  the  designers  of  interactive  software.  Burbulas  and  Callister  ([1]) 
discuss  the  roll  of  Hypermedia  for  the  presentation  of  educational  material.  Integrated  text,  sound,  graphics  and 
animation  in  the  presentation  of  information  should  not  only  enhance  means  of  providing  rapid  information  but 
should  also  motivate  and  keep  the  student  interested  in  the  material  presented  (Hooper  [2]  and  Romiszowski  [6]). 

4.  General  Technical  Issues 

Toolbook  provides  object  oriented  programming  with  integrated  multimedia  tools.  Toolbook  has  its  drawbacks, 
especially,  when  one  is  writing  a book  for  doing  mathematics.  The  main  drawback  has  to  do  with  mathematical 
typography.  There  is  also  a serious  problem  with  the  fonts  substitution  in  a text  field.  There  also  exists  the  problem 
of  displaying  mathematical  expressions  and  formulae  on  the  screen.  Mathematical  equations  and  formulae  have  to 
be  imported  as  graphics  and  pasted  in  the  text  field.  One  can  cleverly  get  around  this  difficulty  but  it  requires  great 
programming  effort.  Beside  the  presentation  problem,  there  are  obvious  problems  of  creating  interactive 
mathematics  tutorials.  There  is  a problem  on  how  to  evaluate  student’s  responses  to  a problem  generated  by  ftie 
computer.  There  may  be  several  correct  responses  to  a question.  The  program  has  to  parse  the  student’s  answer  and 
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compare  it  with  the  possible  correct  answers  generated  by  the  program.  Directing  the  student  to  answer  a question 
using  a certain  pre-arranged  format  can  reduce  some  of  these  difficulties.  There  are  some  basic  requirements  that  are 
common  to  the  field  of  electronic  book  publishing  in  the  category  of  educational  software  Romiszowski  ([6]). 

Computer  Algebra  and  Computation 


After  all,  the  pre  calculus  electronic  book  teaches  mathematics.  So,  how  are  the  calculations  and  algebraic 
manipulations  performed?  Toolbook  provides  basic  functions  that  are  available  in  any  programming  language.  All 
the  calculations  and  operations  are  programmed  using  the  Toolbook  script.  For  example,  the  programming  for 
interactive  algebra  section  took  about  three  months.  There  are  programs  that  factor  polynomials  of  degree  six  or  less. 
The  program  can  find  real  and  complex  zeroes  of  polynomials  of  degree  six  or  less.  If  no  rational  zero  exists,  the 
program  attempts  to  find  approximate  zero  using  the  standard  numerical  analysis  methods  such  as  Newton  Law  or 
the  Intermediate  Value  Theorem.  If  rational  zero  exists,  it  attempts  to  provide  rational  zeros  and  factors.  Since,  the 
purpose  of  the  software  is  to  provide  instruction  only,  the  software  is  not  a sophisticated  computer  algebra  system 
like  Maple  or  Mathematica.  However,  it  is  able  to  do  the  algebra  problems  that  are  found  in  an  elementary  algebra 
book.  It  will  do  algebra  of  rational  functions.  It  will  solve  polynomial  equations  of  degree  six  or  less.  It  will  graph 
polynomial,  rational,  trigonometric,  logarithmic,  and  exponential  functions.  And  it  will  do  so  interactively.  It 
demands  the  students  to  know  these  functions  and  challenges  them  to  interactively  graph  them  using  various 
important  properties  of  the  functions.  The  whole  theme  of  the  book  is  about  student  involvement.  It  demands 
understanding  of  topics  discussed.  There  are  no  multiple -choice  questions.  All  question  require  that  student  perform 
calculation  to  arrive  at  the  answers.  The  book  is  capable  of  giving  objective  type  tests  and  could  provide 
instantaneous  results. 

Web  Connection  and  External  Help 

One  can  provide  connection  to  Web  by  creating  buttons  with  hyperlinks.  In  the  Precalculus  book  each  chapter 
contains  a button  for  launching  a web  page  specially  designed  for  the  courseware.  The  page  provides  links  to 
existing  precalculus  material  on  the  web.  Moreover,  the  page  can  be  used  to  provide  an  instructor  means  of 
providing  external  help  through  links  to  other  web  pages  and  other  material  developed  by  the  instructor.  The  content 
of  the  page  can  be  changed  to  fit  each  instructor  individual  needs.  This  gives  a instructor  the  chance  to  customize  the 
page  as  desired.  In  fact,  the  purpose  of  this  web  page  is  to  provide  links  to  interactive  tutorials,  homework 
assignments,  and  other  support  material  for  the  course. 
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Abstract:  This  paper  synopsizes  the  central  findings  of  a two-year  empirical  study  into  how  the  new  rhetorical 
situations  presented  by  hypertext  affect  the  writing  process  and  thus  impact  upon  literacy  and  education.  It  theorizes 
a conceptual  model  based  on  these  findings. 


New  technologies  are  transforming  literacy  in  general  and  writing  in  particular.  Hypertext  is  perhaps  the  most 
radical  transformation  to  date.  In  hypertext,  writers  struggle  to  master  a new  process  that  includes  electronic  links, 
visual  images,  sound,  animation,  and  other  forms  of  data  within  a single  digitized  writing  space.  This  transformation 
challenges  educators  to  reffame  our  roles  and  points  of  reference  as  we  increasingly  use  hypertext  in  the  form  of 
websites  to  enhance  curriculum.  Hypertext  offers  new  opportunities  for  optimizing  learning  if  we  use  it  well.  We 
need  new  process  models  to  help  us  rethink  our  conceptions  of  writing  in  order  to  adapt  this  new  form  of  writing  to 
best  pedagogical  practices.  This  paper  presents  such  a model. 

This  paper  synopsizes  the  key  findings  of  a completed  two-year  conceptual  and  empirical  study  of  the  hyperwriting 
process.  My  research  was  guided  by  the  overarching  question:  How  are  writers*  perceptions  of  the  new  rhetorical 
situations  presented  by  hypertext  affecting  their  attitudes  towards  writing  and  the  consequent  decisions  they  make  in 
response  to  these  perceptions?  The  study  was  the  basis  of  my  recently  completed  doctorate.  My  position  within  the 
study  was  that  of  an  ‘‘indwelling  researcher”  who  inhabits  the  world  s/he  is  studying.  That  is,  my  dissertation  was 
composed  and  submitted  as  a hypertextual  website.  It  was  the  first  dissertation  of  its  kind  at  my  university — both 
content  and  form — and  one  of  the  first  in  the  world.  It  challenged  academic  norms  in  both  content  and  form  and 
successfully  redefined  academic  notions  of  literacy  to  some  degree. 

The  findings  of  this  study  are  contextualized  within  a significant  body  of  pre-hypertextual  writing  theory  and  a small 
but  growing  body  of  hypertext  theory.  Unlike  writing  theory,  hypertext  theory  consists  principally  of  speculative 
theorizing.  Little  attention  has  been  paid  to  the  particulars  of  the  writing  or  reading  processes,  and  empirical  studies 
are  virtually  non-existent.  This  study  begins  to  address  that  imbalance. 

The  design  of  my  study  is  based  on  phenomenological  concepts  and  methods  common  to  qualitative  research  using 
an  emergent,  field-based,  ethnographic  approach.  This  framework  is  well-adapted  to  the  complex  nature  of  the 
writing  process — a process  that  is  rooted  in  individual  human  cognition  contextualized  within  a social  experience. 
Because  our  “social  universe”  is  constructed,  it  does  not  exhibit  the  same  kind  of  cause  and  effect  consistency  as  the 
physical  universe.  A qualitative  methodology  allows  for  the  unpredictability  and  variability  inherent  in  this  socio- 
cognitive  process  (Denzin,  Donmoyer,  North,  Villanueva).  The  emergent  interdisciplinary  paradigm  of  the 
ethnographic  approach  (Clifford  and  Marcus,  Geertz)  fiirther  supports  the  goal  of  discovery  and  deeper 
understanding,  important  in  a discipline  such  as  hypertext  theory  that  is  itself  interdisciplinary  and  emergent. 
Purposive  sampling  allows  for  a range  of  experience  with  maximum  variation,  and  data  collection  in  natural  settings 
respects  the  importance  of  context  in  an  intrinsically  social  activity.  Inductive  analysis  helps  to  ensure  that  what  is 
seen  as  important  is  determined  not  only  by  the  researcher,  but  by  the  informants  and  the  data  they  generate  (Eisner, 
Glesne  and  Peshkin,  Maykut  and  Morehouse,  Strauss  and  Corbin).  Data  collection  methods  include  interviews, 
observations,  correspondence,  journals,  and  artifacts.  A significant  amount  of  data  was  collected  over  the  Internet 
using  asynchronous  and  synchronous  communication  (Markham,  Mason,  “Ethnography  in  Cyberspace”). 

The  informants  in  this  study  include  writers  whose  orientations  range  from  academic,  to  business,  to  creative.  At  the 
time  of  data  collection,  they  all  composed  regularly  in  a hypertextual  writing  space  and  published  on  the  World 
Wide  Web.  Although  it  is  possible  to  compose  in  hypertext  that  is  not  destined  for  the  Web — thus  integrating  only 
internal  hyperlinks — I did  not  encounter  writers  who  used  hypertext  this  way.  To  my  informants,  hypertext  and  the 


1268 


Web  were  virtually  synonymous.  I collected  data  from  seven  “major”  writers  who  used  hyperwriting  in  their 
professions,  and  approximately  ninety  “minor”  writers  in  the  group  setting  of  three  classroom-based  writing  courses. 
Further,  having  authored  extensively  in  hypertext  myself — and  having  assumed  the  posture  of  the  “indwelling 
inquirer” — I also  drew  on  my  own  experiences.  In  my  dissertation,  I reported  my  complete  findings  in  the  manner 
of  an  “emic”  or  “thick”  description  extended  to  include  interpretations,  speculations,  and  pedagogical  applications 
(Mason,  From  Gutenberg's  Galaxy).  In  theorizing  a conceptual  model  based  on  my  findings,  I believe  I have 
achieved  the  ultimate  goal  of  qualitative  research  to  develop  “the  highest  level  of  interpretation  and  abstraction  from 
the  data  in  order  to  arrive  at  the  organizing  concepts  and  tenets  of  a [grounded]  theory  to  explain  the  phenomenon  of 
interest”  (Maykut  and  Morehouse  122). 

As  stated  above,  the  principal  conceptual  framework  for  my  interpretations  is  a significant  body  of  pre-hypertextual 
writing  theory,  and  a small  but  growing  body  of  hypertext  theory.  I will  elaborate  on  this  framework  by  giving  a 
brief  overview  of  writing  and  hypertext  theories  as  follows: 

The  1960’s  witnessed  a major  shift  away  from  a behaviorist  understanding  that  viewed  writing  principally  in  terms 
of  models  and  rules  derived  from  finished  texts.  This  shift  is  embodied  in  what  we  now  call  writing  theory  or 
composition  theory.  Writing  theory  is  grounded  in  a philosophy  of  rhetoric  that  rejects  a traditional  product-oriented 
understanding  of  writing  in  favor  of  a process-oriented  approach.  Early  writing  theory  draws  heavily  on  the 
philosophies  of  Kenneth  Burke  who  regards  language  first  and  foremost  as  symbolic  action  and  only  second  as 
representation.  The  work  of  Richard  Rorty,  Thomas  Kuhn,  and  Clifford  Geertz  further  shaped  writing  theorists’ 
growing  understanding  of  the  role  of  social  and  cultural  contexts  in  the  writing  process.  Moreover,  the  influences  of 
Michael  Polanyi,  John  Dewey,  Suzanne  Langer,  Lev  Vygotsky  and  Jerome  Bruner  inspired  the  notion  of  writing  as 
both  a tacit  and  a transactional  process  with  both  cognitive  and  social  implications.  Early  explorations  of  the  writing 
process  focused  on  an  individual  writer’s  cognitive  process.  Tensions  soon  broke  out  between  theorists  who  focused 
on  individual  cognition  and  those  who  viewed  writing  as  a socially  contextualized  process.  Bizzell  and  Faigley 
synopsize  this  complicated  intellectual  exchange  in  theoretical  models  that  help  us  to  understand  the  evolution  of 
writing  theory.  Exchanges  among  process  theorists  set  in  place  a framework  for  “reinventing”  the  rhetorical  tradition 
(Freedman  and  Pringle)  in  such  a way  as  to  emphasize  ultimately  both  the  cognitive  and  social  dimensions  of 
writing  in  order  to  reveal  the  full  complexity  of  writing  as  “a  manifestation  of  complex  and  interpenetrating 
cognitive,  social,  and  cultural  processes”  (Kennedy  243).  It  is  upon  this  dynamic  that  the  sub-division  of  writing 
theory  known  as  genre  theory  is  built.  Genre  theory  is  the  most  fully  elaborated  theory  of  writing  to  date  because  it 
joins  the  micro  level  of  writing  to  the  macro  level  of  discourse,  unites  process  with  product,  and  connects  the 
individual  (cognitivist)  and  the  social  (constructivist)  approaches.  It  is  the  link  between  writing  theory  and  hypertext 
theory. 

Genre  theory  bridges  the  theoretical  gap  between  traditional  writing  and  hypertext  because,  like  the  broader  conceptual 
framework  of  discourse  theory,  genre  theory  recognizes  writing  as  a form  of  social  practice.  Genre  is  understood 
differently  than  in  its  more  traditional  literary  sense  of  textual  features  (Freedman  and  Medway).  Genre  here  includes 
the  interests,  goals,  and  shared  assumptions  implicit  in  different  discourse  commimities.  Genre  theorists  explore  the 
ways  in  which  these  factors  function  as  heuristics  of  process  that  culminate  in  textual  practices  and  features 
(Berkenkotter  and  Huckin).  Noted  discourse  theorist  Norman  Fairclough  details  the  ways  in  which  discourse  is  social 
practice.  In  his  1995  study.  Critical  Discourse  Analysis:  The  Critical  Study  of  Language,  Fairclough  defines  discourse 
not  only  as  language  in  sociocultural  practice;  he  further  observes  that  “texts  in  contemporary  society  are  increasingly 
multi-semiotic. . . .not  only  because  they  incorporate  photographs  and  diagrams,  but  also  because  the  graphic  design  of 
the  page  is  becoming  an  ever  more  salient  factor”  (4).  Writing  in  1995,  Fairclough ’s  definition  of  discourse  anticipates 
the  impact  of  hypertext  on  text.  Genre  theory  provides  a much-needed  link  to  discourse  theory  and,  by  extension,  to 
hypertext  theory  since  writing  theory,  with  few  exceptions,  (Hawisher  and  Selfe,  Selfe  and  Hilligoss)  treats  hypertext 
as  a displacement  of  print-based  literacy  (Johnson-Eilola).  Like  Lemke,  I find  it  more  logical  to  view  hypertext  in  the 
larger  ecosocial  context  of  a communication  continuum  that  begins  with  grunts  and  cave  drawings. 

Hypertext  theory  is  an  interdisciplinary  framework  that  that  draws  on  postmodern  theory  in  general,  and  literary, 
communication,  and  social  theories  in  particular.  Best  known  among  hypertext  theorists  are  George  Landow,  Jay 
David  Bolter,  Paul  Gilster,  Richard  Lanham,  Michael  Joyce,  Paul  Levinson,  Steven  Johnson,  Janet  Murray,  Stuart 
Moulthrop,  James  O’Donnell  and  Sherry  Turkle.  What  all  these  theorists  share  in  common  is  a view  of  hypertext  as 
a paradigm  shift,  and  as  a physical  embodiment  of  the  ideal  text  envisioned  by  the  postmodern  theorists  such  as 
Foucault,  Barthes,  Derrida,  and  Bakhtin;  that  is,  text  whose  structure  consists  of  multi-sequential  nodes,  links,  and 
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networks,  whose  content  can  include  the  aural  and  the  pictorial,  and  whose  interactive  nature  re-negotiates  the 
relationship  between  reader  and  writer.  Hypertext  is  thus  understood  by  hypertext  theorists  as  a phenomenon  that 
reconfigures  the  writing  space  and  the  thinking  processes  behind  it.  As  Gilster  observes,  “hypertext  is  a mental 
process  as  well  as  a digital  tool”  (138). 

It  is  within  the  theoretical  framework  described  above  that  I interpret  my  data.  In  the  original  full  length  study 
(Mason,  From  Gutenberg's  Galaxy)  I identified,  reported,  and  interpreted  a number  of  immediate  contextual 
concerns.  These  included  the  Internet  as  a public  writing  space,  multimedia,  collaboration,  and  “techno-angst.”  I 
further  extended  my  interpretations  to  suggest  pedagogical  implications.  Finally,  I used  my  interpretations  as  a 
scaffold  upon  which  to  build  my  own  theoretical  speculations  about  the  impact  of  hypertext  on  ways  of  thinking  and 
knowing.  In  the  limited  space  of  this  paper,  I offer  a much-reduced  synopsis  of  the  central  focus  of  my  research:  the 
particulars  of  the  hyperwriting  process.  From  my  interpretation  of  these  particulars  I theorize  a new  process  model. 
While  avoiding  the  positivist  pitfall  common  to  generalization,  I do  believe  it  is  both  the  prerogative  and 
responsibility  of  the  qualitative  interpreter  to  identify  commonalities  as  a basis  for  theoretical  speculation.  It  is  these 
common  practices  among  my  informants  that  I will  now  describe. 


The  Importance  of  the  Visual 

The  need  to  visualize  both  the  “look”  on  the  screen  and/or  the  overall  design  of  their  “docuverse”  was  uppermost  in 
the  minds  of  most  informants  when  they  faced  a new  hjqjerwriting  task.  (“Docuverse”  is  a term  coined  by  Theodor 
Nelson  to  describe  a hypertextual  document.)  Different  informants  had  different  ways  of  addressing  this  challenge, 
but  each  relied  on  some  form  of  visual  schematization.  Terms  such  as  story-boarding,  mind-mapping,  sketching,  and 
drawing  were  commonly  used  by  the  writers  under  study  to  describe  their  process.  The  words  of  my  informant,  Mac, 
best  represent  this  experience.  Mac  said,  “Well,  I think  the  first  thing — and  everything  plays  off  this — is  that  you 
start  essentially  with  a storyboard,  much  more  the  way  you  would  approach  putting  together  a cartoon  or  a movie 
than  a simple  piece  of  writing.”  Concern  over  how  to  visualize  their  writing  space  ranged  from  envisioning  the 
macro  level  of  the  site  as  a whole — colors,  shapes,  and  structural  relationships— to  envisioning  the  micro  level  of 
individual  page  layouts.  Further,  the  potential  to  include  images— both  still  and  animated — seemed  to  invite  writers 
to  imagine  the  pictorial  as  an  intrinsic  part  of  whatever  text  they  were  composing.  The  need  to  visualize  occurred 
early  in  most  writers’  processes  and  persisted  throughout. 


Considerations  of  Structure 

According  to  hypertext  theorist  Jay  David  Bolter,  “In  this  shifting  electronic  space,  writers  will  need  a new  concept 
of  structure.  In  place  of  a closed  and  unitary  structure,  they  must  learn  to  conceive  of  their  text  as  a structure  of 
possible  structures”  (144).  A concern  with  structure  in  terms  of  navigational  paths  was  common  to  all  my  major 
informants.  They  acknowledged  this  to  be  a new  writing  concern.  My  informant  Jay  put  it  this  way:  “I  had  to 
determine  how  people  might  look  through  the  material  and  what  people  looking  through  the  material  might  be 
searching  for,  and  what  kinds  of  questions  they  might  have,  to  first  and  foremost  anticipate  the  kinds  of  twists  and 
turns  that  my  readership  might  take.  I had  to  go  there  before  they  did.  This  is  different,  this  trying  to  anticipate  the 
thought  processes  of  other  people.”  I believe  Jay  identifies  a fundamental  difference  between  printed  text  and 
hypertext.  Writing  in  hypertext  forces  a higher  degree  of  audience  anticipation  on  the  part  of  the  writer.  It  is  easier  to 
predict  reading  patterns  in  print;  options  are  fewer  and  more  obvious.  How  to  divide  text,  what  to  include  in  menus, 
and  where  to  place  hyperlinks  in  order  to  build  trails  and  “leave  breadcrumbs”  thus  become  crucial  decisions  in 
hypertextual  structuration.  In  building  these  trails,  however,  writers  must  envision  the  stability  of  their  text  much  the 
way  Schryer  describes  genre  in  the  context  of  social  practice  as  “stabilized-for-now”  (Freedman  and  Medway  1 OS- 
124).  Janet  Murray  comments  on  this  complex  socio-cognitive  process  in  Hamlet  on  the  Holodeck  when  she 
observes,  “authorship  in  electronic  media  is  procedural.  Procedural  authorship  means  writing  the  rules  by  which  the 
texts  appear  as  well  as  writing  the  texts  themselves”  (152).  It  is  questionable,  however,  to  what  extent  those  writer- 
based  rules  will  be  followed.  Readers  in  h}q)ertext  are  free  to  follow  whatever  paths  they  choose  and  create  their 
own  structures.  Hypertext  theorists  comment  widely  on  the  role  of  the  reader  in  creating  coherence.  It  may  well  be 
that,  as  Myka  Vielstimmig  suggests,  when  it  comes  to  hypertext  “the  coherence  is  performative”  (31).  Nonetheless, 
writers  appear  to  be  challenged  to  envision  a range  of  paths  in  order  to  create  a navigable  structure.  Furthermore,  the 
writer’s  acute  awareness  of  the  reader’s  role  in  negotiating  meaning  emphasizes  the  redefinition  of  the  constructivist 


1270 


transaction  between  writer  and  reader  that  is  emerging  in  this  medium  (Bolter,  Landow,  Johnson-Eilola,  Sosnoski). 
Fairclough’s  definition  of  discourse  as  “language  seen  as  a form  of  social  practice”  is  clearly  evident  (6). 


Content  and  the  Place  of  Words 

Composing  in  a multi -medial  “writing”  space,  my  informants  began  to  question  the  role  of  words.  The  majority  tried 
to  eliminate  words  when  possible,  sensing  that  in  this  medium,  “a  picture  is  worth  a thousand  words.”  Among  my 
informants,  Mary,  an  English  teacher,  noted  that  when  she  hyper-writes  words  are  the  last  element  she  considers 
whereas  in  traditional  text  they  are  the  first.  Mary  also  observed  her  tendency  as  a hyper-writer  to  avoid  complex 
sentences  in  favor  of  subject- verb -object,  and — much  to  her  amazement — a sudden  acceptance  of  incomplete 
sentences.  Informant  Don  described  how  he  went  to  extraordinary  lengths  to  minimize  words.  He  even  claimed  his 
primary  concern  was  “how  the  page  looks  rather  than  the  content  it  contains.”  To  get  the  “look”  he  wanted,  Don 
admitted  he’d  do  anything  to  “make  the  words  fit  into  a specific  size  cell  in  the  table.”  Rose,  a writer  of  hypertext 
fiction,  still  valued  words  but  said  that  in  hypertext,  “I  think  I tend  to  be  shorter — both  my  stories  and  the  sentences 
and  words  in  them.”  Several  informants  remarked  that  the  ability  to  hyperlink  to  external  resources  often  eliminated 
the  need  to  “write”  text  at  all  in  some  circumstances.  In  sum,  my  data  suggests  that  hyperwriting  requires  more  than 
word-based  language  skills,  and  that  content  is  defined  as  more  than  words.  Writing  in  this  medium  thus  appears  to 
draw  on  more  than  “linguistic  intelligence”  (Gardner),  and  re-configures  the  role  of  words  within  the  semiotic 
configuration  of  the  writing  space.  This  multi-medial  reconfiguration  of  the  writing  space  may  be  welcome  news  to 
intelligences  and  learning  styles  that  have  been  previously  undervalued  in  the  traditional  curriculum. 


Conventions 

As  with  any  new  medium,  conventions  develop.  In  printing,  for  example,  we  take  for  granted  pagination,  margins, 
standardized  spelling,  indexes,  and  so  forth.  Hypertext  in  the  form  of  a website  is  no  exception.  My  informants 
concerned  themselves  with  the  increasingly  expected  conventions  of  this  medium.  They  knew,  for  example,  that 
readers  would  expect  a navigation  bar  on  each  page,  and  they  deliberated  about  where  to  place  it.  Informants 
concerned  themselves  with  page  titles,  menus,  banners,  headers,  footers,  back-to-top  icons,  home  icons,  authorial 
information,  and  dates.  They  also  pondered  what  to  do  about  conventions  connected  to  the  “Gutenbergian  artifact” 
(Negroponte)  of  copyright.  They  worried  about  if  and  how  sources  for  material  they  included  in  their  websites 
should  be  cited,  and  whether  and  how  to  copyright  their  own  original  material.  Informants  also  had  to  unlearn  and 
relearn  common  habits  based  on  print-based  conventions.  Underlining,  for  example,  creates  confusion  if  used  to 
highlight  text  since  hyperlinks  themselves  are  underlined.  Finally,  within  a medium  as  new  as  Web-based  hypertext, 
conventions  metamorphose  continually  as  technology  creates  new  options.  Thus,  my  informants  were  confronted 
with  the  stress  of  a medium  whose  points  of  reference  were  constantly  shifting. 

The  particularities  of  the  hyperwriting  process  that  I have  highlighted  above  constitute  a brief  overview  of  the  key 
findings  in  my  study.  A more  extended  analysis,  and  a range  of  less  commonly  cited  habits  are  detailed  in  the  full  study 
(Mason,  Fro/w  Gutenberg's  Galaxy). 


A Conceptual  Model  for  the  Hyperwriting  Process 

Perhaps  it  was  working  in  such  a visual  medium  that  prompted  me  increasingly  to  visualize  the  substance  of  my 
interpretations  as  a concept  map.  “A  Process  Model”  (Fig.  1)  represents  the  interpretations  of  my  empirically 
grounded  theory  in  a single  conceptual  image.  This  model  is  not  meant  to  be  definitive  or  prescriptive  in  any  way. 
Writing  is  a highly  individual  yet  complex  activity  that  cannot  be  reduced  to  a single  model.  Furthermore,  as 
research  evolves  new  ways  of  understanding  the  hyperwriting  process  will  emerge.  The  value  of  any  model, 
however,  is  that  it  allows  researchers  a concrete  point  of  reference  from  which  to  question  and  conceptualize  further. 
As  a result  of  my  findings,  I have  also  come  to  believe  that  graphic  representations  engage  a part  of  the  brain  that 
words  do  not  and  thus,  when  combined  with  words,  can  transform  scholarly  theorizing  into  a more  whole  brain 
activity. 
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Fig.  1:  A New  Process  Model  for  Hyperwriting 

Here  is  how  I would  verbally  describe  the  hyperwriting  process  as  it  is  represented  in  the  “new  model”  presented 
(Fig.  1).  The  “typical”  hyper- writer  appears  to  hold  in  mind  and  make  key  decisions  about  many  elements  of  a given 
writing  task  at  the  same  time.  Some  of  these  elements  are  common  to  a Gutenbergian  writing  space  and  are  a given 
in  any  writing  medium  (e.g.,  audience,  purpose,  words);  however,  many  elements  are  new  rhetorical  considerations 
(e.g.,  multi-media,  hyperlinks,  technical  functions).  It  is  these  new  elements  that  this  model  embodies. Because  of  the 
number  of  diverse  elements  the  hyper- writer  appears  to  consider  both  simultaneously  and  sequentially,  it  seems 
logical  that  different  levels  of  consciousness  may  be  involved.  Moreover,  the  hyper-writer  must  consider  both  the 
macro- level  (e.g.,  navigational  system,  style  templates)  and  the  micro- level  (e.g.,  page  layouts,  content) 
simultaneously  as  they  impact  on  each  other  in  an  electronic  environment  that  is  always  in  a kind  of  inter-dependent 
controlled  motion  rather  than  frozen  on  a printed  page.  To  accomplish  this  multi-faceted  procedure,  the  hyper-writer 
likely  processes  these  elements  internally  at  different  levels  of  consciousness  both  simultaneously  and  sequentially, 
while  at  the  same  time  representing  these  elements  and  their  evolution  externally  on  either  paper  or — more  likely — 
by  using  web  authoring  software  or  even  hard-coding  in  HTML.  These  external  representations  appear  to  be  highly 
graphical  in  nature  as  compared  to  the  traditional  writing  process  that  is  constituted  largely  by  words.  Holistically, 
this  model  represents  what  may  be  the  most  crucial  distinction  between  traditional  writing  and  hyperwriting, 
between  Gutenberg’s  Galaxy  and  Cyberspace — as  most  of  my  informants  noted  in  various  ways — there  is  just  so 
much  more  to  deal  with  all  at  the  same  timel 
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1  Abstract 

In  the  Information  Age,  Help  Desks  play  an  increasingly  important  and 
highly  visible  role  in  providing  effective  customer  service  and  satisfaction. 
Effective  communication  is  a key  component  in  delivering  customer  satis- 
faction. Therefore,  we  at  the  ISS  Help  Desk,  at  The  George  Washington 
University,  purchased  Desktop  Streaming^^  - software  that  allows  remote 
screen  sharing,  without  client  side  installation,  over  a standard  internet 
connection. 


2  Introduction  to  Environment  and  Users 


Any  large  organization,  like  The  George  Washington  University  (GW),  has  an  eclectic 
mix  of  computer  users  and  an  equally  varying  computer  skill  set. 

And  it  is  when  computers  do  not  function  properly,  that  clients  call  the  Help  Desk. 


3  What  Does  the  Help  Desk  Do? 


Ideally, 


• Resolve  the  issue  over  the  phone 
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If  issue  cannot  be  resolved,  assign  the  call  to  another  division 


Computer  problems  are  not  easy  to  describe  and  sometimes,  neither  are  their  solu- 
tions. Verbally  instructing  clients,  particularly  novices,  on  the  steps  to  troubleshoot 
computer  issues  can  be  time  consuming  and  frustrating  to  clients  and  IT  Support 
Staff. 

In  a small  organization,  it  is  feasible  to  walk  over  to  the  cubicle  down  the  hall  to 
solve  the  problem  in  person,  but  that  is  not  always  possible  with  large  and/or  off 
campus/site  users  because 

• of  distance^ 

• of  time  to  get  to  off-site  locations^ 

• it  reduces  Help  Desk  coverage. 

• you  always  forget  something  at  your  desk 

and  you  almost  wish  you  could  see  the  user’s  computer  from  your  desk. 


4 Enter  Desktop  Streaming™  (DTS) 


DTS  is  a great  tool  for  remote  screen  sharing.  HelpAlert^^  - “call”  routing  software 
- is  about  2 MB  is  size  and  works  on  any  Windows  based  machine^.  The  client  visits 
a web  site  using  a Java  enabled  browser  over  a standard  internet  connection,  and  you 
can  see  the  client’s  screen  within  a minute. 


No  Client  Side  Install  The  was  very  important  to  us.  It  was  not  is  to  install/push 
software  on  the  client  machines  (10,000-f  computers).  All  the  client  needs  is  an 
internet  connection  and  a Java  enabled  browser.  Both  IE  and  Netscape  worked 
great,  but  we’ve  heard  of  (not  experienced  so  far)  issues  with  Netscape. 

Screen  Sharing  This  is  neat  part  about  DTS.  You  can  share  the  client’s  keyboard 
and  mouse,  but  the  client  has  ultimate  control  over  their  computer.  We  try  to 
emphasize  this  before  we  access  the  client’s  screen.  The  screen  sharing  session 
starts  in  draw  mode,  and  the  Help  Desk  Analyst  has  to  manually  switch  over 
to  share  the  client’s  keyboard  and  mouse"*. 

^Distance  Learning  students  as  far  as  California,  Remote  Campuses,  Faeulty  at  Conferences 
4 hour  flight  to  California 

^You  can  screen  share  with  any  Windows  and  Solaris  based  machine 

^This  prevents  computer  accidents 


Web  Push  Type  in  a URL  and  click  Push,  and  the  web  page  loads  on  the  client’s 
screen.  Great  tool  for  getting  to  sites  quickly. 

File  Transfer  Exchange  files  using  regular  FTP  - you  don’t  have  to  forget  CDs/fioppies 
at  your  desk. 

ChatLink^^  Allows  the  Help  Desk  Analyst  to  instant  message  with  client  if  need 
be.  We  hardly  use  this  as  we  usually  have  clients  on  the  phone  when  we  are 
screen  sharing. 

Whiteboard  You  can  draw  on  the  client’s  machine.  Lets  say  the  client  cannot  find 
something  on  the  computer,  you  can  circle  it.  The  software  always  starts  out 
in  whiteboard  mode,  which  is  very  useful. 

Management  Console  Gives  access  to  logs  for  screen  sharing,  and  chat  sessions. 
Handy  for  metrics,  if  you  are  into  that  kinda  stuflF. 


5 Conclusion 


DTS  oflFers  several  features  that  make  technical  support  efficient  and  reliable. 
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Abstract:  This  paper  presents  the  basic  knowledge  about  the  influence  of  hypermedia 
educational  technology  on  the  selection  and  design  of  the  teaching  process  for  new 
generations  of  adolescent  and  adult  students.  The  global  teaching  model  which  is 
predominant  in  the  present-day  school  could  be  called  “the  class-subject -lesson 
system”.  This  didactic  system  cannot  meet  the  needs  and  expectations  of  young  people 
and  adults  at  the  start  of  this  millennium. 


Introduction 

In  the  mid  17**^  century  J.  A.  Komensky  (1592-1670)  established  the  teaching  model  which  is 
termed  “the  class -subject -lesson  model”.  This  model,  according  to  which  a group  of  students  meets  every 
day  in  a school  building  for  common  teaching  activities,  is  still  predominant  today  in  most  schools  around 
the  world.  This  group  of  students  is  taught  by  one  teacher  in  one  classroom.  Activities  alternate  every  45 
minutes.  Such  a teaching  model  has  provided  cheap  education  for  a large  number  of  students  in  compulsory 
education.  It  is  regarded  that  mass  education  of  young  people  was  made  possible  by  the  most  revolutionary 
invention  of  that  time,  i.e.  the  book. 

Although  the  model  established  by  Komensky  has  been  criticised  during  the  last  three  and  a half 
centuries,  it  still  represents  the  basis  of  the  teaching  process  even  today.  This  system  received  a most  severe 
blow  at  the  beginning  of  the  century  from  the  well-known  movement  of  reformist  pedagogy  (J.  Dewey, 

M.  Montessori,  R.  Steiner,  C.  Freinet,  P.  Petersen,  etc.). 

Today,  young  people  and  adults  live  in  an  essentially  different  media  environment  compared  to  the 
one  J.  A.  Komensky  did.  How  would  Komensky  have  designed  the  teaching  process  in  school  if  he  had 
been  surrounded  by  the  present-day  communication  media? 


Some  empirical  results 

In  Croatia,  children  spend  at  school  about  a thousand  teaching  periods  a year  (one  teaching  period 
lasts  45  minutes).  One  year  has  8760  hours,  which  means  that  the  students  spend  at  school  about  one  ninth 
of  the  total  time  of  one  year.  Moreover,  the  teaching  activities  for  students  are  organized  during  180  days 
within  a year.  This  means  that  students  do  not  atte  nd  school  more  than  half  of  the  year. 

On  a sample  of  primary  school  students  attending  lower  grades  of  primary  school  (N=3245)  it  has 
been  tested  how  many  students  have  a personal  computer  at  home  and  how  many  of  them  have  had  the 
chance  to  use  the  Internet.  The  sample  comprised  children  from  6 to  10  years  of  age.  The  results  showed 
that  31%  of  the  students  have  a PC  at  home  and  25%  have  experienced  the  benefits  of  the  Internet.  More 
than  50%  of  the  teachers  teaching  these  students  are  computer  literate  and  use  a PC  in  their  everyday  work. 

A sample  of  grammar  school  students  (N=308)  showed  that  61%  of  them  have  a PC  at  home  and 
70%  have  had  a chance  to  use  the  Internet.  The  sample  comprised  students  between  15  and  19  years  of  age. 

In  Croatian  schools  teacher-oriented  teaching  is  still  predominant.  Students  are  dissatisfied  with 
such  situations  and  react  by  not  attending  classes  and  by  passively  boycotting  their  teachers’  occasional 
appeals  to  become  involved  in  the  common  activities.  A number  of  secondary  school  students  decide  to 
withdraw  from  full-time  education  and  enrol  in  some  kind  of  distance  education.  These  forms  of  education 
ensure  a more  flexible  planning  of  their  school-  and  free  time  (Matijevic,  1996). 


Teaching  aims  and  strategies 

Nineteenth-century  school  is  considered  by  experts  to  have  been  intellectualistic.  This  means  that 
teaching  activities  were  mainly  oriented  towards  cognitive  development.  The  educational  ideal  was  to  learn 
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as  many  new  facts  as  possible.  The  emotional  and  psychomotor  development  was  in  a position  of  secondary 
importance.  In  the  school  of  the  20*^  century  some  things  changed. 

Apart  from  the  teaching  and  learning  of  facts  the  school  is  required  to  take  into  consideration  the 
emotional  and  psychomotor  development  of  students.  However,  this  is  more  of  a declaratively  stated  wish 
than  the  real  state  of  affairs  in  schools.  Because  of  an  essentially  different  media  environment  and  the 
revolution  in  technology  and  information  science  which  has  been  going  on  in  the  last  20  years  of  the  2Cf 
century,  some  new  aims  of  teaching  and  learning  have  been  gaining  ground.  Experts  have  observed  the 
existence  of  much  better  media  for  storing  and  processing  information  than  students’  heads.  Instead  of 
learning  facts  the  students  should  learn  how  to  learn  independently  and  how  to  find  and  use  information. 
Besides,  school  has  started  to  prepare  young  people  for  changes  which  will  become  part  of  their  lives.  The 
educational  value  of  learning  social  and  communication  skills  has  also  been  observed.  These  aims  could  not 
be  realized  by  means  of  traditional  didactic  solutions  based  on  teacher-oriented  teaching,  where,  generally, 
the  role  of  the  teacher  was  to  speak  and  show,  and  that  of  the  students  to  sit,  listen  and  watch. 

The  students,  who  live  in  a new  learning  environment  and  media  environment  could  not  be 
satisfied  with  a passive  role  in  the  teaching  process.  There  are  many  things  going  on  after  the  time  spent  at 
school,  which  are  much  more  interesting  than  the  contents  explained  and  shown  by  the  teachers.  School 
faces  many  changes.  Komensky,  in  his  time,  had  only  the  book  at  his  disposal  as  a teaching  medium  and  the 
voice  of  the  teacher.  If  he  were  alive  today  he  would  probably  offer  a more  diverse  didactic  repertoire  for 
school  activities  than  the  class-subject -lessonsystem. 

The  class,  as  a relatively  constant  group  in  terms  of  experience  and  previous  knowledge  on  similar 
topics,  does  not  exist  any  more.  There  are  noticeable  differences  among  individuals  of  the  same  age.  These 
differences  are  determined  by  greater  mobility  (travelling)  and  by  the  rich  media  environment.  The 
differences  in  the  interests  and  in  the  mental  condition  are  evident.  It  is  very  difficult  to  organize  a teaching 
activity  aimed  at  cognitive  learning,  which  can,  at  the  same  time,  satisfy  the  needs  and  expectations  of  a 
group  of  students. 

This  media  environment  essentially  determines  the  new  roles  of  the  teacher.  Traditionally,  teachers 
were  trained  to  implement  a certain  curriculum  with  groups  of  students.  Today  the  school  needs  teachers 
who  can  help  with  and  successfully  solve  the  educational  problems  of  individual  students.  The  school  needs 
teachers -mentors  and  not  teachers- lecturers. 

Hypermedia  educational  technology  surpasses  the  didactic  framework  of  the  class-subject-lesson 
system.  Experts  in  European  schools  frequently  make  the  mistake  of  trying  to  find  ways  of  using  the 
advantages  of  the  Internet  and  multimedia  exclusively  within  the  class-subject -lesson  system.  In  this  way, 
the  possibilities  of  hypermedia  are  substantially  reduced. 


Conclusions 

Hypermedia  educational  technology  not  only  provides  opportunity  for  the  solution  of  many 
problems  of  traditional  teaching  but  it  also  raises  numerous  new  questions.  Some  of  the  most  important 
ones  are  the  following:  To  what  extent  and  how  to  use  this  technology  to  accomplish  the  aims  of  education 
in  modem  school?  Which  teaching  aims  should  be  ceded  to  this  technology  and  to  what  extent,  and  what 
should  be  left  for  direct  human  communication  between  students  and  teacher?  What  kind  of  relationship 
should  exist  between  learning  by  means  of  hypermedia  educational  technology  and  learning  in  authentic 
reality?  Which  teaching  aim  cannot  be  achieved  through  the  use  of  hypermedia?  What  are  the  new  roles  of 
the  teacher  with  regard  to  hypermedia  educational  technology?  The  question  posed  by  J.  Kemp  (1973) 
acquires  a new  meaning  in  relation  to  hypermedia  educational  technology.  Experts  for  teaching  and 
learning  issues  must  search  for  answers  to  the  above-mentioned  questions.  A lot  of  multimedia  software 
exists  on  the  market  which  cannot  be  accepted  as  satisfactory  by  educational  experts  and  which  is  harmful 
for  the  development  of  young  people  in  the  same  way  as  the  worst  solutions  in  traditional  teaching  are.  One 
must  not  lose  one’s  way  in  technology,  or,  in  other  words,  technology  must  not  be  more  important  than 
man. 
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Fostering  Inquiry-Based  Learning  Online:  A Case  Study 
Inquiry  Online:  The  Challenge 

Using  examples  from  TERC/Lesley  University  Master's  in  Science  Education  online  courses, 
this  presentation  will  explore  strategies  for  fostering  inquiry  within  a distance  learning 
environment.  The  topic  will  be  investigated  from  two  perspectives:  that  of  a course  developer 
and  that  of  a program  participant. 

This  presentation  is  a case  study  in  two  respects:  a case  study  of  the  online  program  and  a 
case  study  of  a program  participant's  response  to  the  experience.  Each  course  component 
discussed  will  be  paired  with  an  example  of  that  participant's  work. 

To  place  our  program  within  the  larger  context  of  online  learning,  we  will  address  a range  of 
questions  relevant  to  the  de\elopment  and  implementation  of  online  environments: 

• What  online  course  features  support  or  detract  from  a learner's  inquiry  experience?  What 
assumptions  about  inquiry  are  encoded  into  an  online  course's  structure  and  presentation? 

• Are  some  aspects  of  the  online  environment  actually  more  conducive  to  inquiry  than 
face-to-face  classrooms?  How  can  educators  take  best  advantage  of  the  Internet- 
mediated  learning  context? 

• How  can  online  course  developers  tell  if  the  understanding  goals  are  achieved?  How 
does  one  assess  the  extent  to  which  learners  genuinely  become  engaged  in  inquiry? 

We  have  grappled  extensively  with  these  questions  in  the  process  of  developing  and 
implementing  our  own  courses.  This  presentation  is  an  opportunity  to  share  what  we  have 
learned,  grounding  our  theories  and  insights  about  online  learning  with  actual  examples  from 
our  program. 


About  the  Program 

The  Science  in  Education  Master's  program  is  a two-year  program  for  K-8  teachers  who  are 
currently  teaching  in  classroom  settings.  The  program,  a partnership  between  TERC  and 
Lesley  University  whose  development  is  funded  by  NSF  and  FIPSE,  launched  its  first  course 
in  the  summer  of  2000.  The  first  cohort  of  degree  recipients  will  graduate  in  the  winter  of 
2003. 

Course  Implementation:  Our  program  is  delivered  entirely  online  using  a combination  of 
Blackboard  online  courseware  and  a multimedia-rich  web  site  interface.  All  of  our  courses 
are  team  taught  by  a scientist  and  a science  educator. 

In  each  course,  program  participants:  1)  engage  in  sustained  science  investigation;  2)  learn 
strategies  for  supporting  inquiry  in  the  classroom;  and  3)  put  new  pedagogical  ideas  into 
practice  in  their  own  classrooms.  They  further  their  understanding  of  science  content  by 
carrying  out  a series  of  investigations  at  home,  then  go  online  for  in-depth  discussions  to 
share  results,  look  for  patterns  in  their  findings,  and  generate  explanations  based  on  evidence. 
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Their  coursework  carries  over  to  the  classroom  where  they  try  out  ideas  for  supporting 
learning  through  inquiry  and  for  assessing  inquiry-based  experiences. 

Course  Development:  A development  team  that  includes  a scientist,  educator,  and 
instructional  designer/multimedia  specialist  produces  each  course.  In  the  formative  phase  of 
course  development,  the  team  works  with  an  advisory  group  of  teachers.  The  course  is  then 
piloted  twice,  with  weekly  formative  assessment  informing  course  revisions. 

Our  multi- year  project  also  includes  an  extensive  research  study  of  our  model  for  online 
learning.  Wynne  Harlen’s  Model  of  Inquiry,  used  as  a key  organizer  in  our  online  Master's 
program,  has  informed  the  development  of  instmments  to  evaluate  the  impact  of  the  program 
on  participant  understanding.  Indicators  of  inquiry  learning  are  obtained  from  the  following 
sources: 

• data  collected  via  weekly  online  formative  evaluation  questionnaires; 

• participants'  comments  to  colleagues  and  faculty  in  course  discussion  forums  and  in 
email  correspondence  with  faculty; 

• videotaped  interviews  with  program  participants; 

• observation  of  teachers  working  with  students  in  their  classrooms;  and 

• comparison  of  pre-  and  post-  course  thought  experiments  performed  by  participants. 

This  extensive  research,  coupled  with  our  thoughtful  development  model,  has  given  us 
valuable  time  for  group  dialogue  about  the  nature  of  inquiry  and  the  unique  characteristics  of 
online  learning. 

Implications  for  Others 

The  ultimate  goal  of  this  project  is  has  been  to  rethink  what  teaching  and  learning  can  look 
like  in  the  best  of  all  worlds,  then  translate  their  new  vision  into  high-quality  online  courses 
for  teachers. 

Preliminary  research  results  indicate  that  our  model  for  online  learning  is  successful.  Our 
finding  indicate  that  program  participants: 

• devote  more  time  to  their  coursework  online  than  when  the  same  course  is  taught  face-to- 
face; 

• demonstrate  a greater  increase  in  science  content  knowledge  than  their  face-to-face 
counterparts; 

• develop  a deep  sense  of  responsibility  and  community  with  their  fellow  learners; 

• develop  the  perspective  of  scientists  - behaving  more  like  graduate  level  science  majors; 

• become  deeply  invested  in  the  program  (our  96%  retention  is  unusual  for  distance 
learning);  and 

• change  how  they  teach  as  a result  of  their  online  learning  experiences. 

Our  work  should  be  of  interest  to  anyone  contemplating  teaching  online,  but  it  also  has  larger 
implications  for  furthering  our  understanding  about  teaching  and  learning  in  general. 
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Abstract 

The  Master  of  Agribusiness  program  developed  by  The  University  of  Melbourne  is  offered  to  off- 
campus  students  using  an  online  mode  of  study.  The  approach  to  study  is  based  on  collaborative 
learning,  to  develop  graduate  skills  in  communication,  negotiation  and  teamwork  in  addition  to  the 
program’s  focus  on  Agribusiness.  This  study  outlines  the  theoretical  basis  and  the  development  of 
the  program  and  online  learning  materials.  Evaluation  data  from  post-course  questionnaires  and  a 
focus  group  highlight  the  features  and  difficulties  of  the  program  from  the  student  perspective. 


Introduction 

A major  concern  for  teachers  using  online  delivery  is  how  to  ensure  that  their  students  attain  a high  level  of 
understanding  and  ability  from  the  course.  Performance  outcomes  need  to  meet  the  expectations  of  the 
professional  area  in  which  the  students  will  work.  The  development  of  a Master  of  Agribusiness  program 
at  The  University  of  Melbourne’s  Institute  of  Land  and  Food  Resources  was  based  on  two  key 
considerations:  it  needed  to  reach  a geographically  widespread  rurally-based  target  group  of  students 
nationally  and  internationally;  and  it  needed  to  use  the  methods  of  teaching  and  learning  that  had  been 
found  to  be  most  effective  in  learning  business  management  skills.  To  achieve  these  aims  the  course  was 
modeled  on  a similar  course  offered  by  the  University  of  Guelph  in  Canada  (Sparling,  1998;  McAlpine, 
2000).  Online  delivery  is  used  as  the  means  of  reaching  the  students.  Collaborative  learning  is  a 
fundamental  aspect  of  the  approach  to  teaching  and  learning.  An  important  reason  for  this  choice  is  that 
the  collaborative  learning  approach  was  developed  in  business  schools  as  an  effective  means  of  developing 
the  communication  and  teamwork  skills  that  are  needed  in  the  business  environment,  in  addition  to  the 
course  subject  matter  (Milter  & Stinson,  1995a).  Online  delivery  enables  this  approach  to  be  applied  in  a 
distance- learning  environment,  by  providing  both  the  means  of  access  to  course  materials  and  the 
communication  channels  necessary  to  implement  a collaborative  learning  approach.  This  article  considers 
some  of  the  teaching  and  learning  issues  relating  to  the  implementation  of  collaborative  learning  in  an 
online  environment,  using  the  Master  of  Agribusiness  as  a case  study. 


Improved  learning  using  technology 

One  of  the  principal  reasons  for  using  online  technology  is  to  achieve  higher  quality  learning  outcomes 
than  are  being  achieved  through  current  means  (Bates,  2000).  The  use  of  instructional  technology  is 
increasing  as  an  aid  to  learning  by  the  design  and  development  of  courseware  to  facilitate  student-centered 
outcomes-based  learning  (Oliver  & Omari,  1999).  Online  delivery  has  been  identified  as  having  the 
potential  to  increase  access  to  education,  promote  improved  learning,  and  to  achieve  this  while  containing, 
rather  than  increasing,  the  cost  of  education  (Owston,  1997).  The  desired  outcome  of  instructional  design 
and  courseware  development  is  materials  that  promote  effective  learning. 
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Research  on  the  application  of  technology  in  education  has  focused  on  how  the  technology  can  be  used  to 
enable  students  to  learn  more  effectively.  This  is  not  based  on  a view  that  the  use  of  technology 
automatically  improves  learning  outcomes.  Rather,  the  research  examines  ways  that  the  technology  can  be 
applied  to  provide  the  resources  that  facilitate  Meamer-centered’  activities.  These  resources  may  be 
information  sources  or  tools,  such  as  databases,  that  promote  enquiry  and  information  seeking  (Hannafm  & 
Land,  1997).  Technological  resources  such  as  computer  conferencing,  databases,  spreadsheets,  and 
simulators,  may  be  used  as  ‘cognitive  tools*  if  their  purpose  is  to  support  thinking  and  learning  processes 
that  will  enhance  the  quality  of  learning  (Jonassen  & Reeves,  1996). 

The  theoretical  basis  for  this  research  is  the  cognitive-constructivist  view  that  students  learn  most 
effectively  by  actively  creating  meaning  from  their  own  learning  experiences  (Jonassen,  Mayes,  & 
McAleese,  1993).  To  ensure  that  the  meaning  created  is  effective  learning,  the  students*  learning  needs  to 
be  centered  on  activities  that  develop  critical  thinking,  problem-solving  and  reasoning  skills  (Hannafm  & 
Land,  1997).  Investigations  have  found  that  the  most  effective  learners  make  learning  an  active,  engaging 
process  that  develops  a rich  pattern  of  meaningful  associations.  An  important  aim  for  teachers  and 
educational  developers  is  to  design  learning  activities  that  require  learners  to  engage  in  the  active  learning 
processes  that  the  most  effective  learners  do  naturally  (Biggs,  1999). 

From  this  theoretical  position,  technological  resources  can  be  developed  to  enhance  and  guide  the  process 
of  student  learning.  The  active  process  of  constructing  a meaningful  interpretation  from  complex  learning 
tasks  is  supported  by  rapid  access  to  resources  and  tools,  by  creating  communication  channels  that  enable 
speedy  resolution  of  problems  and  the  development  of  new  ideas  through  interaction  with  teachers,  other 
learners,  and  experts  who  may  be  external  to  the  teaching/leaming  situation.  They  enable  the  learners  to 
work  with  information  and  data  quickly  in  order  to  develop  solutions.  By  being  designed  and  developed  to 
support  these  activities,  technology  can  enable  effective  learning  outcomes. 


Collaborative  Learning 

Students  need  to  experience  the  cultural  context  and  the  environment  in  which  knowledge  will  be  applied. 
An  important  aspect  of  collaborative  learning  is  that  the  students  can  apply  their  own  experience  to  the 
learning  process,  and  benefit  from  the  experience  of  other  students.  In  this  way,  students  can  use  their  own 
interests  and  experience  as  ‘springboards*  to  the  achievement  of  a deeper  level  of  knowledge  generation. 
This  has  been  described  as  a move  from  dependence  to  interdependence,  using  dialogue  as  a fundamental 
mode  of  enquiry  (Palloff  & Pratt,  1999).  An  important  aim  is  to  develop  critical  thinking,  reasoning  and 
problem  solving  skills.  Glaser  (1990,  cited  in  Jonassen  et  al.,  1993)  argues  that  cognitive  development 
occurs  through  processing  concepts  that  are  originally  experienced  in  social  contexts,  and  that  while 
meaning  may  be  an  individual  construct,  shared  understandings  result  from  social  negotiations  of  meaning. 

The  vital  part  of  using  online  discussion  in  any  format  is  to  keep  the  students  actively  engaged.  Palloff  and 
Pratt  (1999)  suggest  a range  of  activities  that  foster  active  engagement,  including: 

• Posting  instructions  and  learning  expectations; 

• Forming  teams  and  posting  guidelines  for  their  performance; 

• Encouraging  a search  for  real  life  examples; 

• Using  dialogue  as  enquiry  by  encouraging  thought  provoking  expansive  questioning; 

• Sharing  responsibility  for  facilitation  among  group  members; 

• Promoting  constructive  feedback. 

Open-ended  questions  such  as  “What  do  you  consider  to  be  the  ethics  of  a computing  professional?*’ 
(McLoughlin  & O'Sullivan,  2000,  p.  47)  are  useful  to  facilitate  dialogue  as  enquiry.  A statement  that  is 
designed  to  promote  discussion  should  be  very  effective  in  promoting  dialogue  (Palloff  & Pratt,  1999). 

The  role  of  the  online  tutor  is  critical  to  attaining  effective  learning  outcomes.  Tutors  need  to  set 
appropriate  activities  and  provide  the  right  kind  of  guidance.  Feedback  from  experienced  tutors  indicates 
that  online  tutoring  can  be  more  demanding  than  tutoring  in  a face-to-face  situation.  Some  students  show  a 
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high  level  of  appreciation  for  online  conferences,  but  students  are  likely  to  be  dissatisfied  if  there  is  a lack 
of  participation  by  others  (Mason  & Weller,  2000).  For  online  conferencing  to  be  successful,  participation 
needs  to  be  directly  assessed  or  closely  associated  with  assessment  tasks. 


Building  a learning  community 

One  of  the  important  outcomes  of  collaborative  learning  online  is  a sense  of  community  among  the 
learners.  In  some  studies  both  learners  and  tutors  report  the  feeling  of  getting  to  know  their  fellow 
participants  even  better  than  they  would  in  face  to  face  tutorials,  and  how  this  process  enabled  students  to 
build  confidence  and  to  extend  themselves  further  (Mason  & Weller,  2000).  Bruffee  (1999)  argues  that  the 
development  of  a learning  community  is  a critical  part  of  the  education  process.  He  sees  the  role  of  the 
teacher  as  helping  students  to  negotiate  their  way  into  membership  of  communities  they  aspire  to  join,  that 
is  the  practitioners  of  the  knowledge  they  are  learning.  Klemm  and  Snell  (1996)  propose  that  students  need 
to  adopt  roles,  and  to  change  roles,  to  help  to  build  collaborative  communities. 


Case  study  of  the  Masters  of  Agribusiness 

Design 

The  online  study  courseware  is  based  on  a similar  course  that  is  offered  by  the  University  of  Guelph  in 
Canada  (Sparling,  1998).  Materials  are  structured  to  facilitate  collaborative  learning.  To  achieve  this,  the 
courseware  is  organized  to  provide  access  to  course  content  materials,  study  guides  for  each  course,  and 
guidance  about  study  in  this  mode,  contacts  for  support  if  this  is  needed,  a means  of  assignment  submission 
online,  access  to  library  materials,  and  access  to  a range  of  online  discussions.  Several  separate  discussions 
are  each  focused  on  a specific  purpose.  In  this  way,  discussion  about  a specific  assignment,  information 
about  administrative  or  technical  issues,  and  specific  questions  to  the  course  tutor  do  not  all  appear 
together,  as  this  may  be  confusing  for  many  students  and  be  distracting  from  the  main  issues  to  be 
addressed.  Each  course  has  a homepage  that  links  to  the  content  materials  and  study  guides,  to  the  various 
online  discussions,  to  information  on  assessment  and  assignment  submission,  and  to  library  resources. 

Course  content  is  organized  around  weekly  topics.  Each  topic  has  reading  material  from  a text  book  or 
other  sources.  The  online  content  materials  provide  guidance  as  to  what  should  be  done  each  week  and, 
sometimes,  additional  readings  if  the  information  in  the  text  book  is  insufficient.  A discussion,  called  the 
Online  Workshop,  is  used  for  ongoing  discussions  of  the  weekly  topics.  Students  are  required  to  make  a 
contribution  to  these  discussions.  Typically  there  is  one  new  topic  each  week.  There  may  however  be 
additional  topics  also.  Students  are  encouraged  to  interact  with  each  other  in  the  discussions,  rather  than 
making  postings  that  are  written  as  a comprehensive  individual  presentation.  The  course  tutor’s  role  is  to 
keep  the  discussion  on  track,  to  raise  important  matters  that  the  students  may  miss,  and  to  assist  the 
students  to  develop  their  ideas. 

Students  are  given  individual  and  collaborative  group  assignments  to  submit.  Specific  online  discussions 
are  established  for  small  group  collaborative  tasks,  with  access  restricted  to  the  students  in  the  group  and 
the  course  tutor.  These  enable  the  students  to  discuss  how  they  will  do  the  group  assignment,  to  allocate 
roles,  and  to  send  drafts  to  each  other.  Students  can  access  library  materials  online,  in  the  form  of  access  to 
full  text  articles  from  some  journals.  Assessment  for  each  course  is  based  on  individual  and  group 
assignments,  and  on  the  student’s  contribution  to  the  online  discussion  of  weekly  topics. 


Evaluation 

Questionnaires 

A questionnaire  that  could  be  completed  online  was  developed  for  this  evaluation.  The  questionnaire  used 
a combination  of  specific  statements  to  which  the  students  were  asked  to  indicate  agreement/disagreement 
on  a 5 point  Likert  scale,  and  questions  that  asked  for  open-ended  comment.  Statements  were  arranged  in 
groups  addressing  the  following  issues: 


• technical  issues  of  access  and  clarity  of  use; 

• effectiveness  of  content  information,  how  to  learn  and  assessment; 

• computer  conferencing  and  collaboration; 

• involvement  of  the  course  lecturer  and  perceived  effectiveness  of  learning. 

Each  group  of  statements  was  followed  by  an  open-ended  question.  Responses  to  the  statements  most 
relevant  to  this  paper  have  been  reproduced  and  are  discussed  below. 


SA  - strongly  agree  SD  - strongly  disagree SA  A NS  D SD 


1 

The  way  to  use  the  online  technologies  was  made  Cl 

clear  to  me  from  the  beginning.  C2 

C3 

1 

1 

2 

5 

1 

4 

2 

2 

I experienced  difficulties  in  gaining  access  to  the  Cl 

online  course  materials.  C2 

C3 

3 

1 

1 

2 

2 

1 

2 

2 

2 

3 

After  the  initial  problems  of  connection  were  Cl 

overcome,  access  to  the  online  materials  and  C2 

conferences  was  consistent  and  effective.  C3 

1 

1 

1 

1 

4 

1 

1 

2 

1 

1 

1 

Cl  - Course  1 C3  - Course  3 


Table  1 Responses  to  questions  on  technical  issues  and  clarity  of  use  for  three  courses. 

Data  from  the  questionnaires  for  the  first  three  courses  offered  online  have  been  included  in  this  study. 

One  course  is  notable  as  it  provided  the  students  with  more  online  content,  and  had  more  input  from  the 
lecturer  into  the  online  workshop  discussions  than  the  other  courses.  This  course  was  rated  more  positively 
by  the  students  than  the  others.  The  positive  rating  was  consistent  across  nearly  all  of  the  questions, 
indicating  the  pervasive  nature  of  the  perceived  benefit  from  this  input. 

The  responses  in  Table  1 show  a diminishing  level  of  issues  and  problems  relating  to  the  use  of  the 
technology.  The  students  had  many  difficulties  with  access,  as  they  needed  to  install  software  (lotus  Notes 
Client)  as  well  as  establish  an  Internet  connection  to  the  server.  This  caused  some  delays  in  fully  getting  to 
grips  with  the  first  course.  One  student  commented;  ‘It  was  extremely  difficult  to  try  to  come  to  grip  with 
the  course  material  whilst  using  the  browser  and  trying  to  get  Notes  running.  More  consideration  should  be 
given  to  supporting  student  with  the  set-up.  We  did  feel  like  guinea  pigs’.  These  initial  problems  were 
sorted  during  the  first  course,  and  there  were  few  problems  relating  to  access  after  that.  A student’s 
comment  in  the  second  course  was:  ‘Technical  problems  did  improve  with  time’. 

Questions  9 and  10  in  Table  2 focused  directly  on  collaborative  aspects  of  the  course. 

Students  were  consistently  positive  in  their  responses  to  the  question  about  working  closely  with  others. 
They  were  more  divided  on  the  issue  of  learning  from  each  other,  with  a minority  either  neutral  or  in 
disagreement  with  this  statement.  One  student  commented  on  the  value  of  other  students'  responses  as 
follows:  ‘The  answers  to  the  compulsory  questions  from  the  other  students  were  very  informative  and  gave 
me  a broader  understanding  of  the  course.’  Comments  in  two  courses  pointed  to  difficulties  with  the 
computer  conference  medium,  which  did  not  allow  a free  flowing  discussion.  After  the  first  course,  a 
student’s  comment  was;  ‘It  is  almost  impossible  to  "discuss"  on  this  basis  and  the  freedom  of  expression 
one  has  verbally  cannot  be  replicated  literally.’  Some  students  felt  that  meaningful  discussion  needed  to  be 
by  phone.  This  does  not  appear  to  have  been  a problem  in  the  third  course.  One  of  the  students  wrote  of 
this  course: 

We  are  at  last  getting  a real  methodology  of  handling  an  ideas  session  online.  Each  contributing 
many  points  for  elaboration  by  others  and  by  ourselves  in  turn.  Then  editing  and  re-editing.  A real 
online  brainstorming  session! 
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SA  A NS  D SD 


SA  - strongly  agree  SD  • strongly  disagree 


7 

The  computer  conferences  enabled  me  to  exchange  Cl 
information  and  ideas  with  the  tutor  and  other  C2 

students.  C3 

3 

I 

I 

I 

3 

I 

3 

2 

8 

The  computer  conference  discussions  helped  me  to  Cl 
learn  more  about  the  subject  than  I would  have  C2 

learned  working  on  my  own.  C3 

3 

I 

2 

3 

I 

2 

2 

I 

9 

I worked  closely  with  other  students  on  the  group  Cl 

learning  tasks.  C2 

C3 

3 

I 

I 

5 

4 

I 

10 

I learned  a lot  from  the  other  students  while  Cl 

working  on  the  group  learning  tasks.  C2 

C3 

I 

2 

I 

I 

3 

2 

2 

I 

2 

Cl  - Course  1 C3  - Course  3 


Table  2 Responses  to  questions  on  computer  conferencing  and  collaboration  for  three  courses. 

It  is  likely  that  the  students  were  becoming  more  accustomed  to  using  the  medium.  More  important, 
however,  was  the  way  that  the  course  tutor  managed  the  online  discussions.  This  is  elaborated  further  in 
the  discussion  content  analysis.  The  overall  success  of  the  course,  and  importance  of  guidance  for  group 
work  is  evidenced  by  the  following: 

I really  enjoyed  the  ...  course.  The  workload  was  appropriate  and  the  fact  that  it  was  well  related 
to  my  area  of  work  was  very  beneficial  for  me.  Although  it  is  up  to  groups  to  work  out  how  to 
work  together,  suggestions  from  past  experiences  could  be  made.  It  took  us  three  goes  to  get  a 
really  good  group  way  to  approach  the  assignments.  I guess  you  could  say  that  this  is  continual 
improvement  as  we  went.  Some  suggestions  may  be  useful  to  others. 


Focus  group 

A focus  group  discussion  was  held  with  the  students  after  the  courses  included  in  this  evaluation  had  been 
completed.  Topics  included  in  the  discussion  included  technical  issues  related  to  connection  with  the 
system  and  working  collaboratively  in  an  online  environment.  The  students  expressed  a high  level  of 
interest  in  the  way  that  the  program  was  run  and  how  this  affected  the  way  that  they  learned.  The 
discussion  focused  on  the  overall  program,  rather  than  differences  between  one  course  and  another. 

The  students  all  had  vivid  memories  of  the  initial  phase  of  the  program,  in  which  they  had  to  install  the 
client  software  and  make  a connection  to  the  server  through  an  ISP.  As  this  happened  during  the  initial 
weeks  of  the  first  course,  students  had  to  deal  with  installation  while  to  keep  up  with  the  course  activities. 
Two  student  comments  illustrate  some  of  this  fhistration: 

I actually  spent  several  days  of  actual  assignment  time  in  economics  trying  to  get  to  a computer  to 
run  with  Lotus  notes.  It’s  not  as  if  I am  a recent  convert  to  computers  - I’ve  been  using  them  for  a 
long  time. 

It’s  a huge  issue;  it  affects  our  ability  to  study. 

The  students  proposed  earlier  installation  with  more  support  as  a solution: 

We  need  a lot  more  IT  advice  on  how  we  are  actually  going  to  get  connected  and  make  sure  that 
everyone  is  connected  right  from  the  start. 
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After  the  connection  issues  were  overcome,  the  students  needed  to  learn  to  work  with  the  technology  as  a 
learning  support  tool.  One  student  summed  this  up  by  saying: 

I had  a few  initial  teething  problems  which  were  really  I mean  they  just  came  down  to  IP 
addresses  and  that  type  of  thing  with  Lotus  Notes,  but  on  the  whole  once  we  got  over  that  initial 
hurdle  everything  ran  - it  just  took  a week  or  two  to  get  everything  sorted  but  after  that  it  did  work 
quite  well  I thought. 

Another  commented  on  the  use  of  the  discussion  groups  specifically: 

I think  the  Lotus  Notes  worked  quite  well  as  a group  discussion  basis  as  well,  I mean  most  of  the 
discussion  groups  we  had  there  were  some  hiccups  at  the  beginning,  but  really  as  a tool  to  use  as  a 
group  discussion  basis  and  to  get  your  work  done,  once  all  those  teething  problems  were  finished  I 
think  it’s  quite  a good  use  of  that  technology  and  it  went  well,  I think,  for  most  of  us. 

Collaborative  tasks  were  the  most  challenging  aspect  of  the  course  to  the  students,  as  they  had  issues  of 
negotiation  and  organization  to  deal  with  as  well  as  the  information  and  interpretation  associated  with  the 
content  of  the  task.  As  they  progressed  through  several  courses  they  improved  their  ability  to  work 
together.  Two  comments  illustrate  this  process: 

. . . when  [we]  ...  did  our  first  one  we  actually  divided  up  the  task. . . and  then  each  one  had  a task  - 
well  that  was  alright  except  I think  it  worked  better  with  actually  building  it  up  and  moving  the 
assignment  around  and  each  person  contributing  ... 

Other  comments  also  reflect  that  the  process  was  difficult  at  first,  but  became  easier  with  practice  as  the 
students  got  to  know  each  other  and  improved  their  organization.  The  students  generally  saw  the 
collaborative  tasks  as  a good  way  to  learn  despite  the  added  stress: 

Yes  it  was  good  - nothing  will  ever  replace  sitting  in  a room  with  people  and  talking  to  them  but 
to  do  the  group  work  that  way  - you  certainly  got  to  know  the  group  members  much  better  and 
you  interacted  with  their  thoughts  and  ideas  and  that’s  the  way  you  learned. 


Conclusion 

In  general  the  students  were  strongly  supportive  of  the  Masters  program.  They  set  a high  value  on  the 
collaborative  aspects.  They  got  to  know  each  other  well  and  appreciated  working  together  on  learning 
tasks.  Apart  from  the  initial  problem  of  access,  the  students  found  that  they  learned  to  work  with  the  online 
technology,  and  that  it  was  an  effective  way  of  facilitating  their  learning.  The  course  focus  is  to  develop 
knowledge  and  skill  in  the  management  of  Agribusiness.  The  online  technologies  enabled  those  students 
based  in  rural  areas  to  participate  in  the  program.  Learning  to  work  collaboratively  overcame  any  feelings 
of  isolation  the  students  may  have  experienced,  and  led  to  skill  development  that  built  on  the  knowledge 
and  experience  of  the  students  as  well  as  the  staff. 
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Abstract 

Developing  a course  for  online  study  by  off-campus  students  requires  a major  change  of 
approach  to  teaching  and  learning.  This  study  outlines  the  constructivist  theoretical  basis  and 
the  approach  to  teaching  applied  in  the  development  of  a humanities  course.  The  study 
includes  analysis  of  the  turning  points  in  the  students’  learning  as  identified  in  asynchronous 
discussion  postings  and  evaluation  feedback  from  a post-course  questionnaire.  Analysis  of 
discussions  and  student  feedback  are  used  to  demonstrate  student  learning  and  attitudes  to  the 
new  mode  of  study. 


Introduction 

Transforming  a traditional  course  to  an  online  offering  to  off-campus  students  requires  a 
reconsideration  of  the  approach  to  learning  and  teaching.  Using  online  discussions  may  be  a new 
experience  for  both  teachers  and  students.  Consequently,  learning  new  ways  of  interacting  via  an 
unfamiliar  medium  may  be  a major  part  of  doing  the  course,  and  the  student  has  a few  hurdles  to  jump 
over  before  learning  the  course  topic  can  begin. 

This  paper  considers  theoretical  aspects  of  online  learning  in  relation  to  the  design,  development,  and 
implementation  of  an  online  course.  The  project  began  with  a need  to  offer  the  course  to  off-campus 
students.  The  course  requires  the  students  to  analyze  art  and  literature  to  identify  the  influences  of 
culture  and  ideology  on  perception.  An  important  aspect  of  the  teaching  and  learning  is  the 
development  of  the  student’s  ability  to  analyze  issues,  compare  and  contrast  different  art  works  and  art 
forms,  and  to  synthesize  a level  of  understanding  and  a new  perspective.  The  major  requirement  of  the 
course  is  the  development  of  a specific  set  of  higher-order  cognitive  abilities.  The  online  courseware 
and  the  learning  and  teaching  strategy  were  developed  to  support  this  particular  type  of  student 
learning. 


Online  teaching  and  learning 

Research  on  issues  relating  to  online  teaching  and  learning  reveals  a number  of  issues  that  are  critical 
to  the  tasks  of  design,  development  and  teaching  of  an  online  course.  Major  issues  that  must  be 
critically  examined  in  relation  to  current  research  include: 

• Using  online  technologies  to  enhance  student-centered  approaches  to  teaching  and  learning  to 
ensure  high  quality  learning  outcomes; 

• The  use  of  online  discussion  to  support  teaching  and  learning; 

• Providing  effective  support  mechanisms  for  off-campus  students. 

The  following  sections  will  examine  these  issues  to  identify  the  key  points  from  current  research  that 
are  important  for  online  course  design. 
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Online  technologies  for  learning  enhancement 

In  a situation  in  which  a course  is  offered  entirely  online  the  application  of  the  online  technologies  has 
two  critical  roles:  to  enable  an  effective  teaching  and  learning  approach  by  supporting  and  enhancing 
the  essential  teaching  and  learning  processes;  and  to  make  the  course  accessible  for  the  off-campus 
student.  From  an  instructional  design  perspective  it  is  critical  for  the  design  of  the  course  materials  to 
be  based  on  sound  educational  theory.  The  constructivist  approach  to  teaching  and  learning  has  been 
developed  to  empower  students  to  attain  higher-order  levels  of  understanding  and  the  ability  to  work 
with  complex  issues  (Hannafin  & Land,  1997).  An  online  course  in  which  students  are  engaged  in 
exploring  to  find  new  meanings  and  to  engage  in  knowledge  construction  can  be  considered  to  be  a 
constructivist  learning  environment  (Jonassen,  1 999). 

To  enable  the  most  effective  knowledge  construction  the  student  must  engage  in  learning  activities  that 
foster  the  desired  learning  outcomes.  Learning  activities  should  be  structured  to  actively  engage  the 
learner  in  higher-order  thinking  and  learning  by  analyzing  topic  material,  comparing  alternative 
opinions  and  perspectives,  and  synthesizing  a new  level  of  understanding  and  awareness.  Biggs  (1999) 
describes  as  ‘alignment’  a process  of:  formulating  learning  outcomes  that  reflect  the  appropriate  level 
of  knowledge  and  skill;  designing  learning  tasks  that  will  foster  knowledge  construction  and  skill 
development;  and  setting  assessment  tasks  that  require  the  student  to  apply  the  appropriate  level  of 
knowledge  and  skill.  To  achieve  this,  learning  tasks  and  assessment  tasks  are  either  the  same  task,  or 
closely  related.  Collaborative  work  with  other  students  and  the  teacher,  possibly  in  the  form  of 
collaborative  assessment  tasks,  should  be  a part  of  the  learning  process  as  this  fosters  deeper  learning 
outcomes  (Biggs,  1 999). 

The  design  of  the  online  learning  environment  needs  to  be  structured  to  support  the  most  effective 
learning  activities.  In  an  online  course,  critical  considerations  are  the  way  that  the  course  materials 
support  the  student  to  know  how  to  use  the  courseware,  how  to  find  the  learning  resources,  how  to 
access  guidance  about  coming  to  an  understanding  of  the  course  topics,  and  how  to  use  the 
communication  systems.  The  structures  to  achieve  this  can  be  conceptualized  as  tools,  such  as 
cognitive  knowledge  construction  tools,  or  communication  tools  (Jonassen,  1999).  Student  guidelines 
can  be  considered  as  scaffolding,  or  supports  that  can  be  provided  where  necessary  and  removed  when 
the  students  have  attained  a level  of  ability  and  no  longer  need  them.  Important  types  of  scaffolding 
are: 


• Conceptual 

• Metacognitive 

• Procedural 

• Strategic 


- guiding  the  learner  in  what  to  consider 

- how  to  think  during  learning 

- how  to  use  the  features 

- how  to  analyze  and  approach  learning  tasks 
(Hannafin,  Land,  & Oliver,  1999,  p.  131). 


In  the  instructional  design  process  it  is  important  to  ensure  that  these  supports  are  in  place  and 
accessible. 


Online  discussions 

A key  element  in  a constructivist  approach  is  active  learning  through  discussion  and  sharing  of  ideas. 
Laurillard  (1993)  outlines  a process  of  discussion,  adaptation,  interaction,  and  reflection  that  represents 
an  exchange  of  ideas  that  is  essential  before  cognitive  change  can  occur.  This  process  has  a strong 
social  element,  as  it  is  through  interaction  and  sharing  that  a sense  of  a learning  community  is  created 
and  a high  level  of  engagement  with  the  learning  task  occurs.  Learning  is  seen  to  revolve  ‘around 
learners’  conversations  about  what  they  are  learning,  not  teacher  interpretations’  (Mills  & Cottell, 

1998,  p.  229).  In  this  context,  group  activities  are  considered  to  develop  generic  skills  that  are 
transferable  to  other  situations,  such  as  communication,  negotiation,  and  interpersonal  skills,  and  to 
enhance  learning  leading  to  deeper  learning  outcomes  (Morris  & Hayes,  1997). 

In  the  context  of  online  learning,  discussion  and  interaction  occur  using  online  conferencing  tools. 

These  tools  are  considered  to  be  some  of  the  essential  ‘cognitive  tools’  provided  by  a computer- 
supported  learning  environment  to  aid  knowledge-building  through  sharing  ideas  and  collaboratively 
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constructing  knowledge  (Jonassen,  1999).  Online  discussions  are  applied  to  enable  the  cognitive 
processes  described  above  to  occur  in  an  online  environment.  They  should  not,  however,  be  seen  as  a 
direct  equivalent  of  group  discussions  in  a classroom.  Asynchronous  online  conferences  are  conducted 
in  writing,  which  changes  the  nature  of  the  discussion.  Comparative  research  indicates  that  while 
online  discussions  may  be  less  spontaneous  than  the  classroom  equivalent,  they  are  more  reflective  and 
often  demonstrate  a higher  quality  of  learning  as  a result  (Newman  1997).  Consequently  while  the  use 
of  online  conferencing  is  growing  as  a method  of  supporting  off-campus  students  it  is  also  being  used 
in  conventional  on-campus  courses  (Romiszowski  & Mason,  1996). 

As  online  discussion  is  new  to  many  teachers  it  is  appropriate  for  online  teachers  to  consider 
appropriate  methods  for  facilitating  teaching  and  learning  in  this  way.  Studies  of  courses  in  which  an 
online  discussion  is  an  optional  extra  without  teacher  involvement  often  report  some  shortcomings  in 
student  use  of  the  discussion.  To  use  online  discussion  as  an  essential  element  in  the  learning  process 
an  appropriate  level  of  involvement  by  the  teacher  is  essential  (McAlpine,  2000). 

Salmon  (2000)  presents  a five-stage  model  for  the  development  of  student  ability  to  engage  with  online 
discussions.  She  describes  the  teacher’s  role  in  an  online  discussion  as  ‘e-moderating’.  In  this  role  the 
teacher  must  ensure  that  the  students  move  through  the  stages  of: 

• Access  and  motivation 

• Online  socialization 

• Information  exchange 

• Knowledge  construction 

• Development. 

Salmon  recommends  that  at  the  early  stages  of  an  online  discussion  the  e-moderator  needs  to  provide 
online  activities  that  will  facilitate  the  student’s  ability  to  manipulate  the  conferencing  tools  and  to 
develop  knowledge  by  exchanging  and  expressing  ideas  through  the  online  discussion  process.  It  is 
only  when  the  students  have  reached  the  fourth  and  fifth  levels  of  ability  that  the  students  are  able  to 
engage  in  the  type  of  knowledge  construction  through  dialogue  that  is  the  goal  of  the  learning  activity. 


Review  of  theory 

The  use  of  online  technologies  to  promote  knowledge  construction  and  active  learning  requires  a range 
of  critical  supports  and  guidelines  for  learners.  They  need  to  know  how  to  use  the  online  courseware 
and  where  to  find  the  necessary  tools  and  resources.  A critical  tool  is  the  online  discussion.  Through 
this  means  the  learners  can  be  guided  in  how  to  share  ideas  and  work  with  other  learners,  and  how  to 
develop  their  understanding  of  the  topic.  A staged  process  beginning  with  simple  tasks  for  online 
discussion  building  to  more  complex  ones  is  helpful  to  learners  as  they  come  to  terms  with  a new  mode 
of  study. 

The  Course 

The  course  on  which  this  study  is  based  was  designed  as  part  of  a UNSW  initiative  to  encourage  online 
teaching  in  General  Education.  General  Education  is  a requirement  by  UNSW  that  all  students  take 
some  courses  in  disciplines  outside  their  faculty,  to  broaden  their  education  and  introduce  them  to 
forms  of  knowledge  and  styles  of  thinking  outside  their  normal  study. 

The  course  is  called  ‘Seeing  Australia’,  an  examination  of  the  ways  in  which  Australia  has  been 
represented  through  its  history  in  different  modes  of  cultural  production.  The  content  addresses  various 
forms  of  written  and  visual  representation,  including  fiction,  poetry,  non-fiction,  painting,  photography, 
and  prospectively,  film  and  television.  The  learning  outcomes  of  the  course  involve  high  level 
cognitive  and  inteipretative  abilities.  At  its  completion  the  students  should  not  only  be  able  to  identify 
various  ways  of  representing  Australian  place  and  society,  and  to  distinguish  between  two  predominant 
traditions  of ‘seeing’  Australia,  but  also  be  able  to  recognize  that  ‘seeing’  itself  is  not  a simple  ‘natural’ 
activity,  but  based  upon  cultural  assumptions  and  traditions  that  largely  go  unnoticed. 

The  course  was  run  for  the  first  time  on  a trial  basis  in  the  second  semester  2001.  For  this  trial  a small, 
totally  external,  student  cohort  was  selected.  Nine  students  were  Singaporean  Chinese  enrolled  in  an 


external  Safety  Science  degree  at  UNSW,  one  was  an  accounting  student  from  rural  NSW.  None  had 
taken  a humanities  subject  before.  These  students  were  relatively  familiar  with  WebCT,  but  had  not 
previously  engaged  in  external  study  that  used  online  technologies  as  the  main  focus  for  learning 
activities. 


Course  Content  versus  Learning  Outcomes 

The  course  was  divided  into  Six  Modules  containing  topics  that  approximated  a weekly  schedule.  Once 
the  students  (eventually)  came  online  the  introductory  module,  involving  their  ‘seeing’  of  their  own 
neighbourhood  and  of  Australia  worked  well.  However,  it  quickly  became  clear  that  the  students  were 
not  able  to  maintain  a weekly  schedule  of  topics  as  might  be  demanded  in  a face  to  face  course.  This 
immediately  suggested  that  ihe  online  format  provided  a ready  diagnosis  of  student  progress,  otherwise 
unavailable  if  they  had  sat  quietly  and  uncomprehending  in  an  on-campus  class. 

The  most  interesting  outcome  was  the  discovery  that  course  content  could  be  sacrificed  in  favor  of  the 
learning  outcomes.  Although  the  course  was  not  structured  according  to  a progressive  format,  in  which 
the  student  could  only  move  ahead  once  a topic  had  been  mastered,  it  was  clear  that  there  was  no  point 
moving  forward  unless  they  had  frilly  grasped  the  conceptual  skills  involved  in  a module.  By 
privileging  the  learning  outcomes  over  content,  four  topics  in  a module  could  be  reduced  to  three,  or 
even  two,  in  practice,  although  the  content  was  available  in  the  Reader  for  the  better  students.  This 
meant  that  difficult  theoretical  topics  were  engaged  by  students  with  no  background,  by  ensuring  that 
they  could  work  at  the  issues  through  discussion.  In  the  workshops  a certain  momentum  built  up  so  that 
even  slower  students  became  engaged  with  the  more  difficult  topics  by  entering  into  the  discussion. 
Conversely,  it  was  clear  that  once  a new  topic  began  it  took  some  time  to  build  up  momentum.  In 
effect,  it  became  necessary  for  the  content  to  be  reduced  so  that  the  learning  outcomes  could  be 
achieved  by  all.  One  positive  corollary  of  this  apparently  slow  rate  of  progress  was  that  the  online 
delivery  of  content,  particularly  of  paintings  and  images,  allowed  the  students  considerable  time  for 
study  and  reflection  at  their  leisure. 

In  addition  to  the  slower  rate  of  progress,  it  was  clear  that  some  issues  of  cultural  analysis  were  simply 
too  difficult  for  students  of  non-English  speaking  background  with  no  experience  in  the  Humanities. 
Nevertheless,  by  maintaining  a clear  focus  on  the  learning  outcomes,  these  problems  could  all  be 
circumvented.  Whereas  a concentration  on  the  content  might  have  suggested  that  the  course  was  falling 
short  of  its  aims,  it  was  clear  that  not  all  conceptual  issues  or  forms  of  content  were  necessary  for  the 
skill-based  learning  outcomes  to  be  reached. 


Diagnosis  of  Learning 

The  discussion  provided  a ready  indication  of  the  degree  to  which  students  had  grasped  the  conceptual 
point  or  the  extent  to  which  they  needed  to  be  encouraged  to  progress.  The  following  comment  is  from 
a slower  student  who  is  entering  a discussion  about  Oliver  Sacks’  story  “The  Man  Who  Thought  His 
Wife  was  a Hat”  which  touches  on  the  neurological  bases  of  seeing  as  an  introduction  to  the  ways  in 
which  ways  of  seeing  can  vary. 

Hi  Bill  I am  a down  to  eartli  person  and  believe  in  seeing  with  my  own  eyes.  In  my  opinion, 

Dr  P can  be  describe  as  followed:  Like  most  musician  and  painter,  he  is  a man  with  creative 
mind  as  shown  in  his  previous  painting  work  and  his  love  on  music.  Most  people  see  with 
their  eyes,  but  after  reading  his  story,  I found  that  one  can  also  see  with  their  ear  and  nose.  In 
the  story  ’journey  to  the  west',  I remember  the  monk  preach  his  followers  to  see  with  their 
heart.  At  that  time,  I thought  it  was  only  a story,  but  now  I found  that  in  real  life,  they  are 
actually  people  seeing  things  in  another  angle.  In  Dr  P case.  Music  is  the  whole  world  to  him. 
His  love  for  music  is  so  great  that  he  can  teach  music  to  his  student  till  the  last  day  of  his  life. 
That  make  me  wonder  how  powerful  is  our  mind. 

Here  we  find  a student  at  an  interesting  juncture  between  his  understanding  of  himself  as  ‘a  down  to 
earth  person’  and  therefore  not  amenable  to  exotic  ideas  about  seeing,  and  his  beginning  to 
comprehend  the  complex  cultural  bases  of  vision.  Critically,  this  turning  point  comes  when  he 
incorporates  previous  knowledge  to  construct  his  understanding.  At  this  stage  he  is  not  quite  aware  of 
the  implications  of  someone  ‘seeing’  with  ears  and  nose,  but  the  phrase  ‘how  powerful  is  our  mind’  is  a 


significant  entry  to  the  understanding  of  the  cultural  bases  underlying  ways  of  seeing  Australia.  This 
was  a beginning  that  could  be  built  on  by  the  instructor.  But  it  was  a beginning  constructed  by  the 
student’s  incorporation  of  previous  knowledge. 


Turning  Points 

It  became  possible  to  tell  when  the  learning  outcomes  were  being  reached  by  observing  the  emergence 
of  active  learning  in  the  online  discussion.  This  was  particularly  so  when  the  Singaporean  students 
constructed  the  requisite  knowledge  from  their  own  experience.  Throughout  the  discussions 
constructivist  ‘turning  points’  could  be  distinguished,  at  which  the  students  reached  learning  outcomes 
by  actively  constructing  the  learning  central  to  the  course. 

This  learning  construction  could  sometimes  occur  in  a frivolous  way,  as  we  see  here  from  a student 
who  proved  to  be  the  mentor  and  driving  force  of  the  Singapore  students. 

We  often  go  through  life  "doing",  "seeing",  "hearing"  and  "feeling"  things  out  of  habits,  which 
we  actually  don't  intend  to  do  or  be  conscious  about!  My  personal  experience  it  that  I have  a 
wife  that  is  so  tidy  that  very  often  annoys  me.  Everyday  she  simply  goes  about  doing  her 
routine  out  of  habit  more  than  necessity.  No  item  is  allowed  to  be  misplaced  in  the  house  and 
that  include  children  toys,  so  you  can  imagine  the  amount  of  shouting  going  on  around  the 
house  every  day!  Well,  this  is  just  her  way  of  seeing  life!  I certainly  do  not  intend  to  see 
Australia  through  her  way! 

A much  more  significant  example  occurred  during  discussion  of  what  proved  to  be  an  difficult,  but 
ultimately  extremely  productive  topic.  This  topic  was  based  on  a complex  essay  on  the  Eurocentric 
bases  of  the  Mercator  projection  map.  While  die  content  proved  to  be  very  difficult  for  this  class,  other 
content  was  sacrificed  so  that  the  link  between  the  map  and  ways  of  seeing  the  world  could  be 
confirmed.  The  discussion  threads  provide  a concrete  record  of  the  gradual  movement  from  the  idea  of 
a map  as  a simple  description  of  the  real  world,  to  the  concept  of  the  map  as  based  on  the  cultural 
phenomenon  of  Eurocentrism. 

One  turning  point  was  significant  for  the  whole  class.  The  following  comment  is  from  the  student  most 
ready  to  go  beyond  the  confines  of  the  material.  It  represents  the  kind  of  moment  that  teachers  look  for, 
and  which  are  all  too  hard  to  isolate  or  analyze  in  oral  discussion. 

I am  still  looking  for  the  Japanese  map  of  the  world  which  put  Japan  at  the  centre  mentioned 
in  Message:  142.  However,  I managed  to  view  a 12th  Century  map  of  the  world  which  is 
called  a "T  and  O"  map.  The  world  it  portrays  is  a circle,  divided  by  a T-shaped  cross,  the 
center  of  which  is  in  Jerusalem.  The  circle  represents  perfection;  the  world  as  the  object  of 
God's  affection.  The  world  is  divided  into  three  continents,  named  after  the  sons  of  Noah.  This 
map  represented  the  Church's  truth.  It  conveyed  everything  important  to  know  for  those  who 
lived  in  a world  where  nature  was  unfolding  according  to  God's  plan. 

I will  try  to  look  for  the  Japanese  map  as  mentioned. 

The  student  has  actively  constructed  knowledge  of  the  cultural  bases  of  maps  by  means  of  a cultural 
comparison  that  is  germane  to  his  own  experience  but  also  critical  in  the  progression  of  the  whole  class 
towards  the  learning  outcomes. 

It  is  moments  like  these  that  the  instructor  can  justifiably  say  “Bingo!”  Fascinatingly,  it  wasn’t  until  I 
was  reviewing  the  postings  on  the  second  reading  that  I realized  just  how  significant  the  turning  point 
had  been.  The  review  of  written  responses  was  crucial  in  both  diagnosis  and  a perception  of  the  gradual 
achievement  of  the  learning  outcomes. 

Evaluation 

An  evaluation  questionnaire  was  completed  at  the  end  of  the  course.  Questions  relating  to  content  and 
online  discussion  are  included  below. 


The  online  discussion  and  group  learning  tasks  elicited  a strong  favorable  response  (see  Table  1). 

While  the  content  itself  was  regarded  as  difficult  by  this  cohort,  the  online  discussion  meant  that 
students  felt  that  their  learning  had  been  facilitated.  Typical  responses  were: 

I am  very  satisfied  with  the  online  Group  Project  method  of  learning.  It  was  something  very 
new  to  me  and  my  group  members.  I would  not  have  acquired  such  a method  of  learning  if  not 
for  this  online  Group  Project.  Thank  you. 

I am  happy  with  the  group  learning  tasks.  I enjoyed  working  with  the  other  group  members. 

Consequently,  although  content  was  often  regarded  as  difficult,  the  response  to  the  content 
was  very  positive. 


Strongly  agree  SA 

Agree  A 

Not  sure/does  not  apply  NS 

Disagree  D 

Strongly  disagree  SD 


Q 

SA 

A 

US 

D 

SD 

9 

The  online  discussions  enabled  me  to  exchange 
information  and  ideas  with  the  tutor  and  other 
students. 

5 

1 

10 

The  computer  conference  discussions  helped  me 
learn  more  about  the  subject  than  I would  have 
learned  working  on  my  own. 

4 

1 

11 

I worked  closely  with  other  students  on  the  group 
learning  tasks. 

2 

3 

1 

12 

I learned  a lot  from  other  students  while  working  on 
the  group  learning  tasks 

2 

3 

r 

Table  1 Student  feedback  on  online  collaboration 


Although  the  content  was  reduced  the  responses  to  the  questionnaire  confirmed  the  fact  that  the 
learning  outcomes  were  met  by  allowing  longer  time  to  discuss  difficult  concepts  (see  Table  2). 

When  I enrolled  for  this  subject,  I thought  Seeing  Australia  was  a ‘holiday  subject’  not 
requiring  any  effort  to  learn  new  concepts.  It  was  in  the  thick  of  the  subject  that  I learned  that 
Seeing  is  not  necessarily  what  I used  to  think  but  a whole  new  concept  which  I never  knew 
existed. 


Q 

SA 

A 

US 

D 

SD 

14 

I found  the  subject  matter  interesting 

4 

2 

15 

The  course  helped  me  understand  the  concept  of 
'seeing’  in  a new  way. 

6 

16 

The  course  helped  me  understand  a lot  more  about 
Australia. 

3 

3 

Table  2 Student  feedback  on  learning  from  the  course 


Conclusion 

Courses  taught  in  the  Humanities  are  extremely  amenable  to  the  active  construction  of  knowledge 
because  they  do  not  come  with  ready-made  answers.  Skills  of  analysis  and  discernment  are  crucial  to 
the  nature  of  the  knowledge  these  courses  present.  Nevertheless  active  learning  is  habitually 
discouraged  by  an  obsession  with  content  which  is  usually  held  to  be  synonymous  with  learning.  The 
presentation  of  courses  online  enhances  the  processes  of  active  learning  because  the  need  to  write 
postings  and  the  length  of  time  allowed  to  consider  content,  increases  reflection  and  engagement,  and 
provides  a ready-made  diagnostic  tool  for  the  instructor’s  evaluation  of  the  learning  process.  But  the 
course,  ‘Seeing  Australia,’  also  demonstrates  that  high  level  cognitive  skills  can  be  achieved  by 
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students  of  all  abilities  if  content  is  subordinated  to  learning  outcomes.  The  written  record  provides 
examples  of  turning  points  in  the  students’  learning  at  which  major  advances  were  made  as  they  found 
the  sldlls  required  by  the  course  by  moving  outside  the  content  to  engage  the  material  in  a more 
culturally  familiar  context.  These  turning  points  no  doubt  occur  in  all  teaching,  but  the  written  record 
of  online  discussion  provides  confirmation  that  turning  points  occur  when  the  learning  is  constructed  as 
authentic  for,  and  by  the  student. 
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Interactive  Web  Page  Development  for  Undergraduate  Science  Courses;  A Discussion  of 
Process  and  Problems 

In  2000  and  2001,  three  institutions  of  higher  education  attended  workshops  at  the 
Marshall  Space  Flight  Center  (MSFC)  and  Johnson  Space  Center  (JSC)  to  develop 
interactive  web-based  inquiry  activities  for  science  courses  that  serve  preservice  teachers. 
These  web-based  lab  activities  were  developed  to  provide  active  learning  modules  that 
linked  course  content  to  research  conducted  by  scientists  at  the  National  Aeronautics  and 
Space  Administration  (NASA)  centers.  The  participating  colleges  and  universities  are 
members  of  consortium  of  universities  that  form  the  NOVA  network.  NOVA  is  a NASA 
initiative  to  improve  the  quality  of  undergraduate  science  and  mathematics  content 
courses  that  serve  preservice  teachers.  NOVA  was  created  to  develop  and  disseminate  a 
national  framework  for  enhancing  science,  mathematics,  and  technology  literacy  for 
preservice  teachers  in  the  2U‘  century. 

In  order  to  foster  the  goal  of  increasing  the  use  of  technology  to  enhance  learning 
in  science  and  mathematics,  NOVA  offered  a NOVA  Phase  III  grant  opportunity  to 
network  members.  These  grants  provided  funds  to  develop  interactive  web-based 
activities  that  linked  faculty  at  NOVA  institutions  with  NASA  scientists  whose  research 
informed  concepts  in  science  courses  supported  by  NOVA.  Workshops  were  held  at 
NASA  centers.  Science  faculty  spent  five  days  at  a NASA  center.  The  morning  was 
spent  with  scientists  whose  research  related  to  topics  covered  in  the  courses  taught  by  the 
faculty.  Afternoons  were  devoted  to  interactive  web  page  development.  NOVA 
provided  technology  experts  to  assist  faculty  with  software  and  technology  training. 
NASA  staff  provided  resources  in  the  form  of  images,  data,  and  information  on  the 
location  of  web-based  resources. 

NOVA  team  leaders  were  available  to  provide  expertise  in  instructional  design  for 
active  learning  environments.  The  NOVA  model  promotes  instructional  strategies  that 
are  inquiry  based  and  student  centered  with  extensive  use  of  technology  to  facilitate 
learning.  A learning  cycle  approach  is  the  approach  used  by  all  three  institutions  in  the 
development  of  the  interactive  web  sites.  A common  version  of  the  learning  cycle  has 
three  phases,  engagement,  concept  and  term  introduction,  and  elaboration.  Students  are 
introduced  to  new  material  by  engaging  in  an  activity,  investigation  or  discussion  of  a 
question  posed  to  the  class.  The  explanation  phase  allows  the  students  to  verbalize  their 
conceptual  understanding.  After  students  have  had  the  opportunity  to  explain  their 
understandings,  the  instructor  may  introduce  scientific  terms  and  explanations.  Group  and 
whole  class  discussion  during  the  explanation  phase  replace  the  lecture  component  of  the 
course.  Students  apply  what  they  have  learned  during  the  elaboration  phase  of  the 
learning  cycle. 

Two  web  modules  have  been  completed  and  are  currently  being  used  by  the 
faculty  teams  at  two  schools.  One  team  has  completed  work  on  a second  module  which 
will  be  available  in  the  spring  of  2002  for  classroom  use.  The  third  university  is  in  the 
process  of  completing  their  interactive  web  site.  Each  web-based  activity  was  developed 
by  a team  consisting  of  a science  faculty  member,  a science  educator  and  technical 
support  personnel.  NOVA  staff  were  available  for  technical  assistance  with  the  projects. 
Faculty  were  given  stipends  to  support  the  development  of  the  web  activities.  These  web 
activities  are  presently  used  as  a web-based  component  in  face-to-face  courses. 
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One  of  the  web  modules  links  students  to  research  on  the  phenomenon  of  urban 
heat  islands.  Students  explore  this  topic  by  making  observations  of  thermal  images  of 
cities  that  were  gathered  by  satellites.  Web-based  resources  provide  students  with 
information  on  the  concept  of  heat  islands,  the  causes  of  heat  islands  and  the  means  of 
reducing  heat  in  urban  areas.  Students  collect  data  on  temperature  variation  in  an  urban 
area  using  graphing  calculators  with  temperature  probes  and  GPS  units.  Data  is  recorded, 
results  are  analyzed  and  conclusions  are  reported  on  the  web  site.  This  web  activity  was 
developed  at  the  MSFC  and  is  based  on  research  by  Dr.  Dale  Quattrochi  of  the  Global 
Hydrology  and  Climate  Center  (GHCC). 

A second  module  was  developed  at  the  JSC  and  is  based  on  the  work  of  Dr.  David 
McKay.  This  module  relates  to  the  requirements  and  characteristics  of  life,  classification 
of  cells  by  cell  infrastructures,  and  evidence  of  life  in  extreme  environments.  Students 
engage  in  the  concept  by  formulating  ways  to  gather  evidence  of  life  on  Mars.  Web 
based  resources  provide  students  with  information  about  life  in  extreme  environments 
and  about  research  that  may  indicate  the  possibility  of  life  on  Mars.  Students  examine 
and  compare  size  of  cells  and  cells  structures  from  different  groups  of  organisms.  Web- 
based  research  of  new  discoveries  of  nanobacteria  and  debates  among  scientists  provide 
additional  resources  for  students  to  draw  their  own  conclusion  about  the  evidence  for  life 
on  Mars.  Student  design  an  experiment  to  find  microbes  in  harsh  environments  in  the 
immediate  area.  These  experiments  are  performed  in  the  laboratory. 

The  members  of  the  panel  will  present  the  web-based  modules  and  describe  the 
process  and  problems  of  interactive  web  page  development.  The  presentation  will 
include  discussion  of  the  problems  encountered  in  finishing  the  projects  after  the  NOVA 
Phase  III  workshops.  Technical  support  issues  will  be  included  in  the  discussion. 
Implementation  of  the  project  in  the  classroom  and  student  problems  with  the  modules 
will  be  discussed.  Issues  of  faculty  time  and  evaluation  are  important  issues  in 
integrating  these  web  modules  into  the  curriculum.  Each  panel  member  will  have  1 0 
minutes  to  present  the  interactive  web  module. 

The  presentation  of  the  module  development  will  be  followed  by  a thirty  minute 
discussion  that  focuses  on  the  problems  of  development  and  implementation  of 
interactive  web  modules  for  undergraduate  science  courses.  Science  is  an  active  process 
and  developing  inquiry  based  web  activities  is  necessary  when  e-leaming  is  used  in 
science  teaching.  Development  of  these  sites  is  time  and  resource  intensive.  Participants 
will  be  encouraged  to  share  their  experiences  in  using  and  developing  interactive  science 
web  sites.  The  discussion  will  focus  on  barriers  to  web-based  instruction  in  science  and 
how  to  overcome  those  barriers. 

The  panelist  are  scientists  and  science  educators  that  are  members  of  the  NOVA 
Network.  All  are  faculty  members  at  colleges  and  universities  and  have  participated  in 
NOVA  workshops  at  NASA  Centers.  Each  panelist  has  developed  an  interactive  web 
page  that  is  used  in  an  undergraduate  science  course. 
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Transforming  Student  Technology  Use  through  Faculty  Development  Grants 

Science  classrooms  at  a small  religious,  liberal  arts  university  located  in  an  urban 
area  of  South  Texas  have  been  transformed  technology  use  in  the  classroom  through 
faculty  development  programs.  Participation  in  a National  Aeronautics  and  Space 
Administrations  (NASA)  program  for  college  faculty,  campus  faculty  development 
programs  and  Department  of  Education  Title  V (Title  V)  faculty  development  funds 
paved  the  way  for  integration  of  technology  into  biology  classrooms  and  increased 
opportunities  for  students  to  apply  technological  tools  to  solving  problems,  to 
understanding  fundamental  concepts  and  to  research  in  current  issues  in  biology. 

The  university  enrolls  3,500  undergraduate  students.  Seventy-one  percent  of  the 
student  population  can  be  designated  as  minorities  with  the  majority  (54%)  coming  from 
Hispanic  households.  Most  of  the  students  are  first- generation  college  attendees  who  use 
financial  aid  to  pay  for  their  education  (77%).  The  university  qualifies  as  a Hispanic 
Serving  Institution  (HSI). 

In  1996  two  biology  faculty  members  received  a grant  from  Project  NOVA. 
NOVA  is  a NASA  initiative  to  create  a national  model  for  change  in  the  way  that 
introductory  science  classes  for  pre- service  teachers  are  taught.  NOVA  supports  college 
faculty  and  administrative  teams  in  activities  that  apply  the  principles  of  constructivism 
and  national  reform  to  content  courses  in  mathematics  and  science.  The  National  Science 
Foundation  (NSF)  has  called  for  more  opportunities  for  students  to  use  graphing 
calculators  and  computers  in  the  classroom  and  outside  the  classroom  as  a necessary 
element  of  reform  in  science  learning  (NSF,  1996a).  The  National  Science  Education 
Standards  (NRC,  1 996)  call  for  making  “the  available  tools,  materials,  media,  and 
technological  resources  available  to  students”  (NRC,  1996  p.  43)  and  for  providing  a 
learning  environment  that  engages  the  student  in  actively  constructing  knowledge  by 
engaging  in  the  process  of  science. 

At  the  time  the  NOVA  funding  was  granted,  the  biology  faculty  members 
involved  in  the  project  did  not  have  the  technological  skills  needed  to  implement  the  use 
of  technology  in  the  classroom.  This  was  a university-wide  problem  that  was  recognized 
by  the  Office  of  Instructional  Technology  (OIT,  www.uiwtx.edu/~cheryla).  In  the 
summer  of  1998,  OIT  obtained  university  faculty  development  funds  to  develop  a 
program  to  train  faculty  in  use  of  instructional  technology  in  the  classroom.  A key 
element  of  the  training  was  to  introduce  faculty  to  the  instructional  design  process. 
Although  most  participants  in  the  workshops  had  taught  for  years,  many  had  little  formal 
training  in  instructional  design.  The  STI  is  currently  funded  by  a Title  V grant  for  HIS. 

One  result  of  the  training  for  the  biology  faculty  members  was  that  they  received 
a supplementary  grant  from  NOVA.  This  NOVA  Phase  III  grant  provided  the  faculty 
members  with  the  opportunity  to  spend  a week  at  the  Johnson  Space  Center  (JSC)  to 
develop  an  interactive  web  site  for  their  revised  course.  This  faculty  development 
opportunity  was  a valuable  experience  and  facilitated  development  of  interactive 
technology  materials  for  student  learning. 

As  a result  of  attending  OTI  training  workshops  and  receiving  technology 
fellowships,  the  biology  faculty  gained  the  technological  and  instructional  design  skills  to 
develop  instructional  products  to  increase  student  technology  use  in  the  classroom.  One 
important  use  of  technology  is  the  ability  to  link  students  to  real-world  problems  and  to 
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create  an  interactive  environment  for  students  to  solve  problems  (Bransford,  et  al,  1999). 
The  biology  faculty  used  NOVA  funds  to  develop  a course  web  site  with  activities  linked 
to  NASA  research  (www.uiwtx.edu/~mccormic/nova).  These  activities  require  web- 
based  research,  data  analysis,  and  electronic  submission  of  research  results. 

The  course  has  evolved  from  a classroom  that  had  almost  no  technology  use  to 
one  that  is  multimedia  enriched  and  requires  students  and  faculty  to  become  proficient  in 
technology  use.  The  success  of  the  inclusion  of  technology  in  the  NOVA  supported 
course  has  been  extended  to  other  classrooms  in  the  department.  As  the  biology  faculty 
became  more  proficient  in  incorporating  technology  in  instructional  design  they  began  to 
incorporate  technology  use  in  other  courses. 

An  additional  problem  for  the  students  at  the  university  was  the  fact  that  many 
students  come  to  us  families  who  statistically  had  a low  use  of  home  access  to  computers 
and  Internet  service.  In  August  of  2000,  the  U.S.  Department  of  Commerce  (2000) 
reported  that  only  37%  of  Hispanic  households  owned  computers  and  only  16.1%  of 
Hispanics  accessed  the  Internet  from  home.  To  address  this  problem  the  University 
became  the  largest  IBM  ThinkPad  University  in  the  South.  All  full-time  students, 
sophomores  through  seniors,  have  laptops  with  software  and  wireless  cards.  The  laptop 
program  provided  the  means  to  fully  integrate  technology  use  in  the  classroom.  Faculty 
development  led  to  increased  demand  for  student  access  to  technology  and  contributed  to 
the  need  to  change  the  method  of  providing  computer  access  to  students. 

The  National  Science  Foundation  (NSF  1996b),  calls  for  all  students  to  “have 
access  to  supportive,  excellent  undergraduate  education  in  science,  mathematics, 
engineering,  and  technology,  and  all  students  learn  these  subjects  by  direct  experience 
with  the  methods  and  processes  of  inquiry."  Faculty  development  programs  model 
requirements  for  teaching  using  technology  consistent  with  national  reform  movements. 
As  students  collaborate  with  faculty  to  apply  technology  to  solve  problems,  students  and 
faculty  become  part  of  a technologically  literate  community.  Providing  laptops  and 
Internet  access  bridges  the  digital  divide  and  reduces  competition  for  scarce  resources. 
Everyone  has  the  technological  tools  available  in  the  classroom. 
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Abstract 

Cognitive  tools  are  well  documented  in  multimedia  learning  environments  but  there  has 
been  limited  transfer  to  online  learning  environments,  particularly  to  Web-based 
Learning  Management  Systems  (LMS).  This  paper  describes  specific  tools  appropriate 
for  LMS  and  advocates  purposeful  integration  of  cognitive  tools  in  support  of  adaptive 
learning  for  diverse  populations. 

Industry,  government,  and  higher  education  have  been  pressed  to  offer  training  that  meets  the  needs  of  the 
21®*  century  worker  and  student  offering  flexible  scheduling  and  self-paced  courses  (Woods  & Cortada,  2002; 
Fjortoft,  1995)  at  reasonable  costs  (Aldrich,  2001).  Some  experts  believe  that  conventional  education  and  training 
are  obsolete  in  view  of  increasingly  sophisticated  and  readily  available  learning  management  systems  (LMS) 

(Delphi  Group,  20002.  Because  the  Web-based  learner  is  physically  isolated  from  peers  and  typically  learns 
autonomously  he  or  she  may  or  may  not  be  prepared  to  construct  effective  learning  supports.  Therefore  the 
integration  of  tools  that  support  learning  are  most  critical,  even  more  so  when  the  instructor  is  not  present  to 
determine  learner’s  progress  or  understanding.  This  paper  briefly  discusses  cognitive  supports  as  they  relate  to  the 
nature  of  Web-based  Learning  Environments  (WBLEs). 

Conceptualizing  Cognitive  Tools 

Web-based  Learning  Environments  (WBLEs)  typically  require  that  the  learner  work  autonomously  and 
independently.  Feedback  mechanisms  are  mostly  user-generated  rather  than  technology -generated  and  yet  there  are 
many  instances  in  which  WBLEs  have  provided  mechanisms  that  support  the  learner  in  a variety  of  ways:  with 
study  strategies  (Morgan,  Dingsdag,  & Saenger,  1998),  prior  knowledge  evaluation  (Portier  & Wagemans,  1995), 
resources  for  information  retrieval  and  problem  solving  (Oliver,  1999) , tutorials,  and  advising  services  (Wright, 
1991).  These  supports  are  primarily  cognitive,  supporting  the  mental  functions  required  of  the  online  learner. 

The  term  cognitive  tool  has  been  defined  in  several  ways.  Jonassen  (1994)  states  that  computers  and  the 
software  that  provides  interactivity  are  cognitive  tools  that  allow  the  learner  to  use  them  “as  tools  for  knowledge 
construction  rather  than  media  of  conveyance  and  knowledge  acquisition”  (p.2)  as  learners  are  actively  involved  in 
constructing  their  own  knowledge  and  learning  wfr/i  technology  (Jonassen,  1991).  Others  define  cognitive  tools  by 
the  function  served  within  an  electronic  learning  environment  (Lajoie,  1993).  This  approach  is  not  in  conflict  with 
Jonassen’s  general  notion  but  more  specifically  identifies  cognitive  devices  which  support  learning  strategies  that 
vary  by  learning  context.  Such  tools  may  support  cognitive  processes,  reduce  cognitive  load  of  the  learner,  extend 
the  cognitive  capabilities  of  the  learner  or  allow  the  learner  to  test  ideas  within  problem  solving  contexts  (Lajoie, 

1 993).  Katz  (1997)  suggests  that  cognitive  tools  should  support  processes  of  learning  that  are  associated  with  study 
strategies  such  as  review,  rehearsal,  attention,  practice,  and  application.  Regardless  of  how  cognitive  tools  is 
conceptualized,  learning  at  a distance  is  essentially  learner  centered  and  as  such  requires  supports  that  are  responsive 
to  the  learner’s  immediate  needs  (Thompson,  1998. 

Sugrue  (2000)  identifies  four  aspects  of  learning  environments  that  require  cognitive  strategies  that  could 
be  supported  by  tools,  only  one  of  which  can  be  reasonably  applied  to  learner-only  interaction.  Information 
Organization  and  Access* . Helping  the  learner  navigate  and  interact  with  the  electronic  environment  supports 
Lajoie’s  (1993)  idea  that  cognitive  tools  can  reduce  cognitive  load  and  assist  with  lower  level  thinking  processes  so 
that  the  learner  may  engage  in  higher  order  thinking.  Many  cognitive  tools  identified  and  used  in  interactive 
multimedia  learning  environments  resemble  devices  found  in  print  materials  that  assist  in  identifying  location  within 


* The  other  strategies  require  opportunities  to  interact  with  others:  Authentic  Activities,  Collaborative  Learning,  and 
Student  Modeling. 
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a system  or  finding  one’s  place,  such  as  bookmarks  (Katz,  2001).  Erickson  and  Lehrer  (2000)  suggest  that 
hyperlinks  serve  as  cognitive  tools  if  they  are  used  intentionally  and  for  specific  purposes  such  as  for  navigation 
functions,  offering  elaborative  information,  or  for  structural  organization.  Index  or  navigational  interface  windows 
(Jonassen,  1998)  provide  the  user  with  information  about  their  location  and/or  progress,  possibly  through  a program 
or  site  map. 

Other  tools  assist  the  learner  in  locating  information  such  as  find,  glossary  (Katz,  2001;  Sinitsa,  Mizoguchi, 
& Serdjuk,  2001),  and  help  functions  which  may  also  provide  cues,  elaboration  or  explanations  (Katz,  2001).  Tags, 
pop-up  graphics,  or  rollovers  may  also  direct  the  user  to  useful  or  clarifying  information.  It  may  be  that  visual  cues 
such  wipes  and  dissolves  direct  the  user’s  attention  and  support  the  perceived  coherency  of  the  material  presented 
(Slatin,  1991).  Additionally,  explanatory  interface  windows  (Jonassen,  1989)  may  elucidate,  give  details,  or  coach 
the  user.  Others  have  identified  cognitive  tools  that  may  serve  as  an  organizer  of  knowledge  or  illustrate  the 
relationship  of  parts  to  the  whole  illustrating  how  knowledge  is  represented  in  memory  (Sugrue,  2000).  Concept  and 
knowledge  maps  can  help  learners  see  the  big  picture  of  the  relationship  of  concepts  within  a specific  structure  more 
clear  (Rewey,  Dansereau,  Skaggs,  Hall,  & Pitre,  1989),  making  connections  between  prior  and  future  learning. 
Metaphorical  interface  windows  provide  a representation  of  material  designed  to  link  prior  knowledge  to  materials 
being  learned  and  organization  interface  windows  support  the  user’s  process  of  making  sense  out  of  information 
(Jonassen,  1998).  Devices  that  allow  the  user  to  record  and  organize  information  are  also  described  as  cognitive 
tools.  Harper,  Hedberg,  Coderoy,  and  Wright  (2000)  suggest  that  authoring  tools  are  a means  for  students  to  present 
ideas  for  examination  by  others.  Not  all  learners  work  well  with  2 -dimensional  electronic  representations  so  notepad 
(Katz,  2001)  or  downloadable  documents  may  also  function  as  a cognitive  tool  if  they  are  used  to  store  ideas  and 
thoughts  that  are  referenced  in  the  process  of  learning. 

Implications  and  Further  Studies 

It  may  be  that  the  nature  of  Learning  Management  Systems  (LMS)  is  grounded  in  the  notion  of  knowledge 
management  as  opposed  to  knowledge  construction.  Activating  the  learner’s  schema  is  less  likely  in  a prescribed 
and  pre -determined  WBLE,  however,  adaptive  cognitive  tools  may  allow  the  learner  to  construct  understanding  by 
facilitating  the  learner’s  organization  of  information  (Woolf,  1992)  which  can  then  be  substantiated  through 
interaction  with  other  learners  and  the  course  instmctor  (Derry,  1992;  Jones,  Greer,  Mandicah,  du  Boulay,  & 
Goodyear,  1992).  If  a learner  enters  a WBLE  and  is  offered  a path  that  appropriately  supports  his  or  her  entry  level 
skills,  technology  or  content  knowledge,  or  learning  orientation,  the  cognitive  load  will  be  diminished  and 
embedded  supports  will  facilitate  understanding  and  more  authentically  respond  to  the  unique  and  individual  needs 
of  the  learner  (Laurillard,  1992).  This  is  the  intent  of  cognitive  tools,  particularly  in  systems  which  presuppose  or 
require  the  learner  to  work  to  some  degree  without  interacting  with  others. 
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Abstract:  This  paper  details  the  work  performed  toward  building  a reliable,  automatic, 
content  analysis  suite  of  tools  that  analyzes  online  educational  discussion  list  messages.  This 
project  is  nearing  completion  and  is  conducted  under  contract  by  Georgia  Tech’s  Center  for 
Education  Integrating  Science,  Mathematics,  and  Computing  (CEISMC)  with  the  Institute  of 
Higher  Education  at  the  University  of  Georgia  under  a service  agreement  with  the  University 
System  of  Georgia  Board  of  Regents’  Advanced  Learning  Technologies  Unit. 


Introduction 


In  the  state  of  Georgia,  the  number  of  credit  hours  offered  online  jumped  from  59,593  in  fiscal  year 
2000  to  94,531  in  FY  2001,  an  increase  of  59%  (YahoolFinance,  12/11/01).  Meanwhile,  IDC  estimates  that  the 
e-leaming  market  will  grow  from  $2.2  billion  in  2000  to  $18.5  billion  in  2005  (Moore,  2001).  This  growth  in 
electronic  learning  means  that  the  discourse  from  many  of  these  learning  environments  is  very  easily  captured 
providing  an  opportunity  for  researchers  to  study  the  process  of  learning  in  a way  that  has  never  before  been 
available.  This  study  outlines  the  initial  phase  of  the  construction  and  use  of  an  artificial  neural  network  (ANN) 
to  perform  a content  analysis  of  a large  body  of  student  messages  for  cognitive  presence.  This  research  seeks  to 
answer  two  questions.  First,  can  neural  networks  be  used  to  analyze  and  describe  the  cognitive  landscape  of 
online  educational  discussions?  Second,  at  this  phase,  how  is  cognitive  presence  revealed  in  an  online  course? 
This  work  is  based  upon  the  cognitive  presence  element  of  Garrison,  Anderson,  and  Archer’s  (2000,  2001) 
community  of  inquiry  model.  Unlike  Garrison,  Anderson  and  Archer,  this  work  uses  a neural  network 
combined  with  the  General  Inquirer  (see  Danielson  & Lasorsa,  1997;  http://www.wjh. harvard. edu/^inquirer)  to 
analyze  messages.  Additionally,  this  analysis  extends  the  General  Inquirer  by  adding  self-defined  categories 
designed  to  improve  the  classification  into  cognitive  presence  categories.  The  suite  of  tools  developed  for 
analyzing  these  courses  may  ultimately  be  used  to  gauge,  guide,  direct,  and  manipulate  any  learning 
environment  employing  online  discussions. 

The  Study 
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To  answer  the  two  questions,  can  neural  networks  be  used  to  analyze  and  describe  the  cognitive 
landscape  of  online  educational  discussions  and  how  is  cognitive  presence  displayed  in  an  online  course,  a 
content  analysis  of  six  eCore'^^  freshman  and  sophomore  history  and  political  science  courses  was  performed 
during  phase  one  of  this  study.  Phase  two  of  the  study  will  include  modeling  the  decision-making  process  of 
the  content  analysis  coders  using  artificial  neural  network  software.  Six  coders  were  trained  in  the  use  of  a 
rubric  based  Garrison,  Anderson,  and  Archer’s  (2000)  cognitive  presence  hierarchy,  which  consists  of: 
triggering  event,  exploration,  integration,  and  resolution.  After  completion  of  the  training,  each  coder  used  the 
rubric  to  categorize  a set  of  randomly  selected  100  reliability  messages  and  another  set  of  200  randomly 
selected  course  messages.  Since  the  coders  represent  a subset  of  a larger  population  of  teachers,  Shrout  and 
Fleiss’  two-way  random  effects  model  was  used  (ICC(2,6)  = 0.8231).  Barrett  (2001)  considers  an  intraclass  r 
statistic  above  .74  to  be  excellent.  Pairwise,  the  Cohen’s  kappa  (k)  rater  reliability  scores  ranged  from  .504  to 
.747  with  a mean  of  ,608.  This  compares  well  with  Garrison,  Anderson  and  Archer’s  reliability  data  for  two 
coders,  which  yielded  k =.35,  .49  and  ,74  for  three  successive  trials.  Scores  below  .8  are  acceptable  for  cutting 
edge  research,  that  which  is  breaking  new  ground,  a category  under  which  this  research  clearly  fits  (Riffe,  Lacy, 
and  Fico , 1998). 

Conclusion 

The  study  yielded  insight  into  potential  improvements  for  the  rubric  and  coder  training.  Coders  had 
difficulty  in  correctly  coding  Triggering  Events  that  were  intentionally  initiated  by  the  instructor.  In  some 
instances,  these  messages  were  coded  as  Unrelated/Course  Management.  This  was  due  to  the  fact  that  the 
Subcategories  under  Triggering  Events  Sense  of  Puzzlement  and  Recognizes  Problem,  were  inconsistent  with  an 
intentional,  instructor  initiated  Triggering  Event.  Clearly,  the  instructor  was  not  puzzled,  nor  was  he/she 
recognizing  a problem.  In  future  studies,  we  intend  to  modify  the  rubric  by  adding  a new  subcategory  under 
Triggering  Event  for  instructor  initiated  Triggering  Events.  Further,  increased  reliability  may  be  achieved  in 
future  studies  by  providing  additional  training  to  coders  on  distinguishing  between  exploration  vs.  integration, 
exploration  vs.  not  cognitive,  and  triggering  event  vs.  exploration. 
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Abstract:  In  the  context  of  higher  education,  the  development  of  students’  problem 
solving  skills  continues  to  be  an  area  of  much  ongoing  research.  Effective  teaching  of 
problem  solving  requires  the  adoption  of  process-based  approaches  that  reveal  to  students 
the  ways  that  experts  solve  problems,  and  the  coaching  of  students  in  higher  order  and 
metacognitive  skills  that  lead  them  away  from  a preoccupation  with  finding  solutions  and 
towards  building  up  a repertoire  of  problem  solving  strategies.  It  is  suggested  that  online 
environments  and  computer  resources  can  scaffold  the  acquisition  of  domain  knowledge 
and  systematic  problem  solving  skills.  This  article  acknowledges  that  there  are  multiple 
ways  to  support  complex  problem  solving,  and  that  online  environments  hold  great 
promise  in  creating  effective  instructional  interactions.  This  study  focuses  on  the 
metacognitive  aspects  of  problem  solving  and  more  particularly  on  those  aspects  of 
technology  -based  scaffolding  that  support  reflection,  process  support,  and  the  anchoring 
of  skills  to  domain  knowledge. 


Problem  solving  skills  as  central  to  learning 

In  tertiary  education,  there  is  an  urgent  need  for  professionals  who  can  solve  real  problems,  anticipate  and 
predict  problems  and  find  realistic  solutions.  To  meet  this  need  effectively  tertiary  educators  must  now 
reexamine  methods  of  teaching  problem  solving.  In  discussing  pedagogical  approaches,  Hobden  (1998; 
223)  states  that: 

“It  (problem  solving)  is  a routine  activity  occupying  a large  proportion  of  curriculum  time  and 
plays  a central  role  in  student's  experience  of  classroom  life.  From  the  first  days  of  science 
instruction,  sets  of  routine  problem  tasks  assigned  by  the  teacher  have  been  part  of  classroom  life. 

As  a teaching  strategy,  they  have  largely  been  used  uncritically.  It  would  appear  that  nearly  all 
physical  science  education,  and  especially  the  physics  component,  seems  to  be  based  on  the 
optimistic  assumption  that  success  with  numerical  problems  breeds  an  implicit  conceptual 
understanding  of  science. " 

These  comments  apply  to  the  way  problem  solving  has  been  taught  in  the  past,  and  to  approaches  that  rely 
on  traditional  ‘show  and  tell’  where  students  do  not  engage  actively  wth  the  problem  or  context.  The 
traditional  method  of  showing  examples  of  solutions  to  problems  followed  by  student  practice  represents  a 
rather  primitive  approach  in  contrast  to  the  teaching  of  acting,  music  or  a sport,  where  the  skills  required 
for  competent  performance  are  taught  in  context,  often  in  real  world  scenarios,  and  integrated  so  that  expert 
performance  results.  Taconis  et  al  (2001)  have  recently  analyzed  articles  appearing  in  international 
journals  between  1985  and  1995  on  the  effectiveness  of  teaching  strategies  for  science  problem  solving. 
Very  briefly,  this  analysis  showed  that  in  student  performance  and  achievement,  knowledge  of  strategy  and 
practice  of  problem  solving  turned  out  to  have  little  effect,  whereas  accounts  of  effective  teaching  of 
problem  solving  all  give  attention  to  contextualised  strategies  related  to  mastery  of  domain  knowledge.  The 
learning  conditions  recognised  by  Taconis  et  al  (2001)  as  significant  for  building  problem  solving  skills 
were  those  which  provide  learners  with  guidelines  and  criteria  they  can  use  in  judging  their  own  problem 
solving  processes  and  products.  The  provision  of  immediate  feedback  to  learners  is  also  essential.  These 
conclusions  are  congruent  with  earlier  research  carried  out  by  Alexander  & Judy  (1988),  Clarke  (1992)  and 
Lajoie(1993). 


The  learning  paradox 


A number  of  contemporary  designs  for  learning  environments  require  student  self-direction  and  high-level 
metacognitive  control.  Exploratory  and  open-ended  learning  environments  enable  students  to  generate 
knowledge  and  engage  in  critical  thinking.  Similarly,  project  based  approaches  assume  that  students  are 
able  to  generate  questions  and  produce  a final  product  that  represent  knowledge  integration.  Many  students 
however  lack  essential  metacognitive  skills  and  a repertoire  of  learning  strategies  to  enable  them  to 
maximise  their  learning  in  innovative  learning  environments.  Table  1 shows  how  a range  of  contemporary 
learning  designs  assume  metacognitive  knowledge. 


Example 

Learning  scenarios 

Metacognitive  skill  needed 

Anchored  instruction 

Narratives,  stories,  real  life  anchors 

Capacity  to  define  problems 
and  abstract  from  case 

Open  ended  learning 
environment 

Multiple  scenarios  and  viewpoints 

Self-direction  and  self 
management 

Project  based  learning 

Collaborative,  task  based  learning 
environments 

Management  of  information, 
self  and  others 

Problem  based  learning 

Presentation  of  cases  and  events  that 
present  potential  problems 

Capacity  to  identify  the 
problem  and  select  resources  to 
solve  it 

Table  1 : Metacognitive  requirements  of  contemporary  student -centered  learning  environments 

These  learning  environments  though  highly  successful,  assume  that  students  are  goal  driven  and  self- 
directed.  Yet,  these  expectations  bring  with  them  a range  of  assumptions,  primarily  that  students  have 
metacognitive  skills  to  enable  them  to  cope  with  self-direction  learning.  Other  research  has  shown  that  the 
processing  demands  of  these  environments  are  problematic  and  need  to  be  investigated.  One  of  the  issues 
of  most  concern  is  that  of  the  learning  paradox  noted  by  Schank  & Cleave  (1995)  **How  can  students  learn 
by  doing  what  they  do,  when  they  do  not  know  how  to  do  what  they  have  to  do  to  learn?”.  Stated  quite 
simply,  project  and  problem  based  learning  assumes  that  students  can  access  and  apply  knowledge  and 
metacognitive  strategies  and  engage  in  self-regulated  learning.  It  is  well  established  that  in  order  to  learn 
effectively,  a repertoire  of  learning  strategies  and  the  capacity  to  manage  one’s  own  learning  are 
fundamental  (Boekaerts,  2000;  De  Corte  et  al,  2000).  It  is  this  range  of  skills  that  we  refer  to  as 
metacognition. 


The  role  of  metacognition  in  learning  and  problem  solving 

Metacognition  is  a learner’s  knowledge  about  his  or  her  processes  of  cognition  and  the  ability  to  control  and 
monitor  those  processes  as  a function  of  the  feedback  the  learner  receives  via  outcomes  of  learning  (cf. 
Metcalfe  & Shimamura,  1994;  Schraw  1998b).  Metacognitive  knowledge  refers  to  what  the  learner 
understands  and  believes  about  a subject  matter  or  a task,  and  the  judgments  s/he  makes  in  allocating 
cognitive  resources  as  a result  of  that  knowledge.  Metacognitive  control  refers  to  the  strategies  the  learner 
uses  to  achieve  specific  learning  goals  - strategies  like  planning  aid  organizing,  allocating  attention  to 
relevant  and  irrelevant  factors,  looking  for  relationships  and  patterns,  monitoring  comprehension, 
identifying  and  testing  procedures,  evaluating  outcomes,  and  reflecting  on  learning  (cf.  Jacobs  & Paris, 
1987).  Schraw  (1998a)  explains  that  attentional  resources,  existing  cognitive  strategies,  and  awareness  of 
breakdowns  in  comprehension,  are  all  enhanced  by  training  in  metacognitive  knowledge  and  skills.  While 
there  are  examples  of  successful  metacognitive  instruction  in  the  literature,  the  most  effective  ones  involve 
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providing  the  learner  with  both  knowledge  of  cognitive  processes  and  strategies,  and  experience  or  practice 
in  using  them.  Simply  providing  knowledge  without  experience  or  vice  versa  does  not  seem  to  be  sufficient 
for  the  development  of  metacognitive  control  (Volet,  1991).  It  is  also  essential  that  learners  have  an 
opportunity  to  evaluate  the  outcome  of  their  efforts,  to  reflect  and  self-assess  their  own  approaches  to 
learning. 

Four  categories  of  metacognitive  knowledge  are  recognised  as  important  and  affect  task  performance  and 
achievement  of  distance  learners  (White,  1999).  These  are  as  follows: 

□ Self-knowledge:  Self-knowledge  entails  individual  capacity  to  recognise  their  strengths  and 
weaknesses  and  to  evaluate  themselves. 

□ Task-knowledge:  This  involves  understanding  the  demands  of  tasks  and  what  they  require. 

□ Strategic  knowledge:  This  refers  to  the  knowledge  of  usefulness  of  strategies  available  for  achieving 
learning  goals. 

□ Knowledge  of  plans  and  goals:  This  refers  to  learner's  capacity  to  set  and  maintain  goals  and  to 
record  what  they  intend  to  do  through  their  learning. 


Design  of  learning  environments  for  metacognitive  support 

In  response  to  a perceived  need  to  support  problem  solving,  an  on-line  tutorial  designed  in  WebCT,  called 
metAHEAD,  has  been  created  for  first  year  science  students  in  Biology,  Biophysics  and  Chemistry.  The 
aim  of  the  tutorial  is  to  help  students  explicitly  develop  their  metacognitive  skills  in  science  problem 
solving.  The  tutorial  comprises  four  modules  - an  introduction  to  thinking  and  learning,  knowledge  maps, 
explanations  and  descriptions,  problem  solving.  In  developing  the  resource,  instructional  design  was  guided 
by  current  research  on  constructive  learning,  and  metacognitive  skills  development.  Lin  et  al  (1999)  in 
considering  the  technological  design  of  learning  environments  have  noted  that  four  features  are  important 
for  effective  scaffolding  of  reflective  thinking  by  students.  These  are  process  displays,  process  prompts, 
process  models  and  reflective  social  discourse.  The  first  module  of  metAHEAD  introduces  students  to  key 
aspects  of  learning  and  problem  solving  and  allows  them  to  rate  their  own  problem  solving  skills  on  a 
standard  metacognitive  inventory  test.  The  second  module  helps  them  to  build  concept  maps  for  topics  in 
their  subject  areas.  The  last  two  modules  concentrate  on  the  sorts  of  skills  needed  for  explanations  and 
descriptions  (as  most  often  needed  in  Biology)  and  quantitative  problems  (more  often  needed  in  Biophysics 
and  Chemistry). 

As  students  tackle  problems  in  the  modules  they  are  asked  to  engage  in  a number  of  the  following 
activities:  predict  their  own  expected  success  on  problems;  discuss  their  success  with  other  students  during 
and  after  attempting  the  problem;  explicitly  consider  strategies  for  use  with  particular  problems;  examine 
other  students'  answers  and  comments  on  them;  listen  to  audio  clips,  view  video  clips  or  read  transcripts  of 
other  students  as  they  worked  on  problems  and  comment  on  these;  rate  other  students'  answers;  post  their 
answers  to  a bulletin  board  for  discussion  with  other  students;  view  lecturer's  model  answers  and  comment 
on  them. 

Students  maintain  an  on-line  logbook  to  record  their  responses  to  the  various  prompts  throughout  the 
tutorial  and  to  record  their  answers  to  problems.  In  metAHEAD  particular  attention  has  been  paid  to 
supporting  the  four  categories  of  knowledge  identified  as  essential  for  development  of  metacognitive  skills 
(White,  1999).  Process  prompts  and  process  models  are  in  evidence  in  the  learning  activities,  which 
precede  the  solution  of  example  problems,  and  in  the  many  worked  answers  of  students  and  lecturers  and 
comments  thereon.  Bulletin  board  discussion  of  strategies  and  problem  solving  skills  together  with  the 
creation  of  collaborative  teams  provide  opportunities  for  reflective  social  discourse. 


Evaluation  of  metAHEAD 

It  was  intended  to  carry  out  a formative,  integrative  evaluation  of  metAHEAD  in  order  to  improve  the 
design  and  to  refine  various  aspects  of  the  resource.  Oliver  (2000)  has  remarked  that  while  a range  of 
methodologies  exists,  each  may  be  restricted  in  its  use  and  in  the  range  of  situations  it  can  be  applied  to. 
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From  our  pragmatic  perspective,  we  found  the  Open  University  model  as  described  by  Jones  et  al  (1996)  a 
useful  framework.  This  approach  focuses  on  three  main  themes:  context,  interaction  and  outcomes.  This  is 
outlined  in  table  2,  and  shows  that  range  of  data  that  was  collected  and  analyzed.  As  practitioners,  we 
sought  an  evaluation  approach  that  was  broad  and  flexible  enough  to  suit  our  situation,  which  was  the 
introduction  of  an  innovative  resource  within  a university  context.  From  the  outset,  data  has  been  gathered 
on  the  design  and  use  of  metAHEAD  from  pilot  studies,  practitioners'  opinions,  instructional  designers  who 
have  used  and  developed  the  program  and  academic  staff  who  have  used  sections  in  their  teaching.  It  was 
important  to  include  this  data  in  our  evaluation,  as  we  needed  a holistic  picture  of  its  benefits. 


Context 

Interactions 

Outcomes 

Rationale 

Need  for  metAHEAD  at 
UNE;  curriculum  context 

Need  to  look  at  student 
interactions  with  the 
resource. 

Learning  outcomes;  problem  solving 
outcomes;  changes  of  perception  and 
attitude  must  be  considered 

Data 

Designers  aim;  principles 
underpinning  design; 
pressing  iterations 

Records  of  students' 
interactions,  student 
diaries,  and  online  logs 

Measures  of  effective  problem 
solving,  changes  in  attitude,  strategy, 
perception  of  self. 

Methods 

interviews;  written  records; 

Observations,  videos, 
diaries,  computer  records, 
product  data  generated  by 
students 

Focus  groups,  tests  and  questionnaires 

Table  2:  Features  of  the  evaluation  approach  adopted 


As  metAHEAD  is  embedded  in  Web  CT,  it  allows  for  the  gathering  of  the  following  types  of  data  for  the 
evaluation  of  metacognition,  data  collection  was  relatively  standard  self-rating  quiz  data  taken  at  beginning 
and  end  of  semester;  students'  self  predictions  of  level  of  success  with  particular  problems  and  reflection 
upon  these  on  completion  of  the  problem;  on-line  log  book  entries  including  students'  notes  on  strategy  use 
and  actual  problem  solutions;  bulletin  board  discussions.  These  data  are  complemented  by  further  data 
gathered  from  small  face-to-face  focus  group  discussions.  As  a further  elaboration  of  our  evaluation 
approach,  we  collected  a range  of  data  relating  to  the  interactive  and  scaffolding  features  of  metAHEAD. 
These  are  summarised  in  table  3. 


Data  source 

Type  of  evidence 

Journal  entries 

Progress  logs  (process  data) 

Oral  discussion 

Online  transactions  (process  data  ) 

Learner’s  self  assessment 

Prediction  of  success  (process  data) 

Solution  logs 

Solutions  to  problems  (product  data  ) 

Table  3:  Evaluating  metacognition  in  online  environments 


Evaluation  and  data  analysis 

Data  has  been  gathered  from  focus  groups  giving  feedback  on  technical  issues  and  development  of  problem 
solving  skills.  There  are  two  main  areas  of  feedback  gained  from  this  evaluation.  Students  have  greatly 
appreciated  the  availability  of  other  students'  answers  and  particularly  the  comments  on  them.  This  has 
allowed  them  to  put  their  own  answers  into  a better  perspective  and  to  give  a clearer  idea  of  what  the 
lecturer  expects.  (The  fact  that  assessment  and  lecturers'  expectations  drive  the  majority  of  students  cannot 
be  escaped.)  These  do  provide  process  models  and  support  reflection  by  the  students  on  many  aspects  of 
problem  solving.  Coincidentally,  other  student  answers  help  some  students  to  feel  more  of  a part  of  a 
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group  and  that  they  are  "not  so  stupid  after  all",  which  can  affect  student  motivation  to  succeed  along  with 
others.  Students  have  identified  that  their  motivation  often  parallels  their  success  in  the  subject.  Becoming 
part  of  a community  of  students  working  on  a unit  is  particularly  important  for  distance  education  students, 
who  may  be  otherwise  rather  isolated.  The  ability  to  discuss  issues  on  the  bulletin  board  has  been  helpful 
for  these  students. 

Students  have  also  mentioned  that  planning  and  analysis  of  problems,  whereby  parts  can  be  tackled  step  by 
step  has  been  helpful  to  them.  Becoming  more  aware  of  and  practicing  such  skills  has  been  beneficial  for 
students  who  in  the  past  may  have  been  daunted  by  a problems  and  given  up  too  soon  when  it  seemed  too 
difficult.  While  we  have  provided  some  models  and  coaching  online,  students  appear  to  need  a more  staged 
approach,  with  step-by-step  examples  and  support.  This  ‘cognitive  apprenticeship’  approach  is  in  tune  with 
the  constructivist  principles  underpinning  the  design  of  metAHEAD.  Table  4 shows  a summary  of  student 
comments  gained  from  the  evaluation. 


Comments  on  what  students  gained  from  metAHEAD 

Helps  think  about  level  of  confidence  and  predict  degree  of  success. 

Helps  motivation  somewhat. 

Helps  with  ways  to  understand  the  question  better  in  order  to  tackle  it  successfully. 

Helps  with  breaking  up  tasks  - something  they  didn’t  commonly  consider  explicitly. 

Talking  to  other  students  is  helpful,  hear  other  views. 

Getting  feedback  from  lecturer,  hearing  them  explain  and  comment  on  lecturer's  answers. 

Appreciated  other  students'  answers,  so  they  could  compare  their  own  answers,  and  lecturer's  "metacomments"  on  these  were 
important. 

Other  students’  answers  also  show  other  ways  of  solving  problems. 

Information  about  learning  and  thinking  processes  - eg  chunking  information. 

Comments  on  limitations  of  metAHEAD 

Can’t  replace  face  to  face.  Need  personal  tutor  for  motivation. 

Notebook  tool  has  great  technical  deficiencies  particularly  for  science  answers  involving  formulas  and  equations. 

Needs  to  help  students  with  more  practice. 

Needs  more  models  and  demonstrations  of  how  to  work  through  problems  step  by  step. 

Technical  difficulties.  Eg  Plug-in  for  Quicktime  movies. 


Table  4:  Summary  of  comments  from  evaluation  data 
Implications  for  instructional  design 

The  aim  of  metAHEAD  was  to  design  a learning  resource  to  offer  students  support  in  metacognition  and 
problem  solving  and  a learning  experience  that  was  different  from  traditional  teaching  in  first  year 
(freshman)  study.  The  focus  was  on  building  student  self-awareness  of  metacognition  and  strategy 
development  infused  into  disciplinary  content  (chemistry,  physics  and  biology).  Our  initial  evaluation  has 
shown  both  positive  and  negative  aspects  of  the  resource,  and  insights  gained  will  be  used  to  design  the 
resources  and  improve  its  capacity  to  support  problem  solving.  Our  choice  of  evaluation  approach  has  been 
effective  and  worthwhile,  providing  valuable  data  on  context,  interactions  and  outcomes  of  the  resource. 
Continuing  evaluation  wll  assist  in  refining  the  resource,  tailoring  to  student  needs  and  improving  its 
relatedness  the  context  in  which  it  is  being  used  i.e.  the  support  of  metacognition  at  tertiary  level. 
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Abstract 

The  needs  of  learners  are  changing  rapidly  and  continually  in  response  to  an 
environment  that  is  characterised  by  change  at  economic,  political  and  technological 
levels.  There  is  greater  pressure  on  universities  to  work  more  closely  with  employers 
in  contributing  to  the  process  of  economic  development  through  the  creation  of  a 
skilled  workforce.  Universities  are  becoming  increasingly  flexible  in  their  responses 
to  meeting  the  lifelong  learning  agenda.  Online  experiential  learning  is  an  essential 
element  in  the  move  towards  more  situated  and  professional  orientations  and  with 
the  drive  to  providing  students  with  real  world  working  knowledge.  Asynchronous 
communication  tools  and  tasks  are  ideal  conduits  for  the  refinement  of  professional 
skills.  This  case  study  profiles  an  on-line  approach  to  developing  professional 
project  management  skills  for  multimedia  developers  and  presents  snapshot  views  of 
an  online  learning  environment  in  which  students  developing  real  projects  for 
industry  clients. 

Theoretical  Framework:  Experiential  Learning 

Through  the  design  of  an  asychronous  online  learning  space,  learners  to  were  introduced  to  situations 
and  ways  of  working  with  others  that  were  experientially  based.  The  processes  of  learning  were 
emphasised  from  the  outset  and  students  were  encouraged  to  have  ownership  of  the  learning, 
assessment  and  reflection  processes.  For  Kolb  (1984),  the  actual  experiences  people  go  through 
become  the  starting  points  for  learning,  and  this  underlying  educational  approach  was  reflected  in  the 
design  of  the  online  environment.  Emotion  and  reflection  are  also  an  integral  part  of  the  cycle  of 
learning,  and  reflective  processes  are  intrinsic  to  learning  from  experience. 

In  an  experiential  learning  cycle,  the  learner  passes  though  each  of  four  stages:  concrete  experience, 
reflective  observation,  abstract  conceptualisation,  and  active  experimentation.  Our  design  ensured  that 
we  linked  the  experiential  learning  cycle  to  the  online  environment  and  the  collaborative  tools  afforded 
by  the  technology  as  follows: 

• Active  experimentation:  Students  engaged  in  the  discussion  and  resolution  of  problems  through 
online  discussion  using  multimedia  tools  to  support  the  display  of  responses  (eg,,  Cox,  1 994). 

• Task  engagement:  Peers  analysed  the  output  of  the  task,  and  compared  problem-solving 
approaches  through  discussion,  email  and  conferencing  activity  (Bonk  & Cummings,  1 998). 

• Reflection:  Individuals  analysed  and  reflected  on  the  learning  tasks,  group  processes  and  self- 
development  as  they  tested  new  ideas  and  perspectives  in  virtual  learning  groups  (English  & 
Yazdani,  1999). 

• Formation  of  abstract  concepts:  Utilising  peer  and  tutor  feedback,  students  developed 
understandings  of  key  course  concepts  through  engagement  with  new  ideas  supported  via 
communications  networks  (Collis,  1998). 

Experiential  learning  was  reinforced  in  various  ways  through  the  course  pedagogy,  which  focussed  on 
both  process  and  content  aspects  of  learning.  Students  participated  in  decision-making  processes  by 
choosing  and  defining  a topic  for  their  project,  creating  and  managing  their  own  development  team  and 
negotiating  peer  assessment.  The  design  was  based  on  pedagogical  and  curriculum  philosophies  that 
acknowledge  group  work,  peer  feedback  and  support  as  essential  to  the  development  of  independent 
learners  and  practitioners  (Candy,  1994;  Schrum,  2000).  Reciprocity  in  giving  and  receiving  peer 
feedback  provides  learners  with  opportunities  to  deepen  their  interpersonal  skills.  Through  these 
processes  they  also  developed  reflective  skills,  and  learnt  to  monitor  their  own  learning  and  appreciate 
the  contribution  of  others  to  their  learning  (Biggs,  1999). 
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Thus,  by  focussing  on  learning  process  and  peer  supported  activities,  the  learning  activities  took  on  a 
broader  perspective,  to  include  the  totality  of  the  learning  experience  while  consolidating  those  aspects 
of  professional  learning  that  would  enable  the  learners  to  develop  transferable  skills.  This  process  is 
illustrated  in  Figure  1,  which  shows  a focus  on  using  learner-centered  strategies  that  encourage  learner 
independence  and  peer  support,  which  in  turn  promotes  the  development  of  professional  skills  and 
process  knowledge.  These  then  directly  contribute  to  deep  and  meaningful  learning  experiences,  and  as 
part  of  the  authentic  learning  experience,  students  were  able  to  share  their  knowledge  and  experiences 
with  their  peers  through  the  Listserv. 


Collaborative 


Figure  1:  Student  Learning  Process 
Context  of  the  study:  Course  description 

At  Edith  Cowan  University,  final  year  multimedia  students  are  required  to  complete  IMM3330/4330 
‘‘Industry  Project  Development”.  The  aim  of  the  unit  is  to  consolidate  core  multimedia  skills  learnt  in 
other  units,  while  at  the  same  making  industry  contacts  and  developing  a portfolio  item  to  assist  with 
job  applications.  Students  are  provided  with  industry  projects  made  available  through  the  Faculty,  or 
they  can  negotiate  a project  of  their  own,  as  long  as  it  fulfils  the  course  requirements,  team-based, 
commercial  in  nature  and  not  trivial.  As  part  of  the  unit  requirements,  students  are  required  to  perform 
a needs  analysis,  provide  a design  specification,  develop  the  web  site,  evaluate  it,  implement  it  and 
produce  the  required  documentation  (legal,  procedures,  metrics,  templates  and  standards).  The  course 
objectives  were  closely  linked  to  the  professional  competencies  required  for  multimedia  development 
in  the  industry  and  integrated  authentic  assessment  tasks  where  learners  could  develop  multimedia 
products  in  a team  environment,  thus  replicating  the  skills  required  of  them  in  the  workplace. 

The  unit  runs  over  a fifteen-week  semester,  with  three  hours  allocated  per  week  for  tutorials  and 
lectures.  There  are  no  formal  face-to-face  classes,  the  unit  is  largely  run  online  from  a Listserv,  which 
provides  a rich  arena  for  advice,  comments  and  feedback  as  there  is  about  one  hundred  participants 
subscribed  on  the  List,  including  industry  representatives  and  ex-students.  The  constructivist  approach 
used  in  structuring  the  project  work  and  the  Listserv  activities  was  that  students  should  benefit  from  the 
experiences  of  other  students,  industry  representatives  and  academic  tutors  in  an  environment  that 
promotes  the  exchange  of  ideas. 
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A salient  feature  of  the  assessment  tasks  was  the  focus  on  learning  processes,  rather  than  mastery  of 
content  as  this  matched  the  orientation  of  the  course  towards  skills  development.  The  assessment  tasks 
and  associated  learning  processes  are  presented  in  Table  1. 

Table  1:  The  assessment  tasks  and  associated  learning  processes 


Assessment  task 

Learning  processes 

Web-based  product  development 

• Team  work,  planning,  decision 
making  project  management 

Self  and  peer  assessment  of  team 
products 

• Higher  order  thinking  and  analysis 

• Development  of  criteria  for 

assessment 

• Giving  feedback  to  peers 

• Reflection  on  feedback  leading  to 

revision  of  ideas 

Student  learning  contract 

• Self-directed  learning 

Team  progress  reports  to  the  list 
serv 

• Metacognitive  skills  and 

To  initially  motivate  students  in  using  the  Listserv,  assessment  was  included  which  acknowledged 
individual  contributions  (for  “quality”  feedback  and  advice)  as  well  as  submitting  team  progress 
reports,  which  included  students  posting  design  ideas  and  prototypes  and  requesting  feedback  from 
others.  Students  were  given  templates  to  use  in  preparing  these  reports  as  well  as  rules  or  “netiquette” 
they  were  required  to  use  when  posting  information  to  the  Listserv.  Encouraging  students  to  initially 
use  the  Listerv  by  allocating  marks  was  necessary,  as  proven  by  previous  experience.  However,  in  all 
cases,  as  the  semester  progressed,  students  became  more  comfortable  with  using  the  Listserv,  and  were 
posting  on  a variety  of  topics. 


Non  assessable 
comments 


Student  questions 
& comments 

L J 


Figure  1 . A model  for  online  discussion  based  on  peer  partnerships 

The  tutors’  involvement  on  the  Listserv  was  structured  to  be  non-interventionist  so  that  they  acted 
more  as  facilitators  by  focusing  discussion,  rather  than  as  the  “fonts  of  all  wisdom”.  Figure  1 outlines 
the  model  used  for  promoting  discussion  on  the  Listserv.  Issues  were  generated  from  team  progress 
reports  or  from  individual  queries  about  technical,  content,  procedural,  client,  communication, 
team/peer  through  to  how  to  get  employment  in  the  industry!  However,  as  time  progressed,  students 
more  freely  contributed  to  the  Listerv  with  many  other  issues,  some  of  which  were  more  philosophical 
in  nature  and  posted  without  the  objective  of  obtaining  marks. 


An  example  of  how  the  Listserv  was  used  to  promote  real  world  learning  is  as  follows.  The  unit  had  72 
students,  and  the  number  of  posts  made  to  the  Listserv  over  the  semester  was  1035  in  total.  In  the 
example  given  below,  a student  is  seeking  help  on  a difficult  issue  i.e.  how  do  you  develop  web  sites 
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which  are  compatible  across  all  browsers?  This  problem  has  no  real  solution  and  is  also  a problem  for 
professional  developers  in  the  industry.  So  this  is  a rich,  authentic  topic  for  discussion,  in  which 
students  can  assist  their  peers  by  giving  their  opinions  or  help  on  the  issue.  Another  12  replies  follow  in 
this  thread,  where  the  students  kept  debating  the  issue: 

• Initial  Post  “BROWSERS  GRRRRR  ” - Hi  all  Scamers,  I have  come  to  a stage  in  the  project  where 
I just  wish  that  there  were  a universal  Web  browser  that  combines  Internet  Explorer  and 
Netscape!!  It  is  Frustrating  to  design  and  program  a website  that  will  look  exactly  the  same  on 
both  browsers,  well  it  doesn't!!  Not  even  with  CSS  style  sheets  that  is  the  biggest  nightmare  I have 
been  going  through...  Furthermore  how  the  browsers  for  the  different  platforms  ex  Mac  and  Pc 
reads  the  content  but  does  not  display  it  the  same..  I believe  that  it  is  a common  problem  that  web 
developers  have  to  deal  with  and  I wonder  if  one  have  to  be  a fascist  and  say..  Ok  The  site  is 
gonna  look  best  on  IE  assuming  that  70%  of  the  market  is  using  it!  And  forget  about  the  rest!!!  It 
is  a nightmare  and  I just  cant  get  my  head  around  it , Just  want  to  continue  the  development  of  the 
clients  site.  Oh  and  our  client  is  aware  of  the  problem  and  has  given  us  a Ok  to  ignore  the  small 
errors  between  the  browser  but  still  this  is  something  that  we  have  to  deal  with  in  future  projects/ 1 
will  be  glad  is  somebody  has  a good  solution  for  my  problem.. 

• Reply  I - It  is  just  a problem  that  exists,  and  seemingly  always  will,  the  CSS  sheets  will  look 
different  because  Microsoft  have  one  set  of  codes,  and  there  is  another  for  Netscape  preferences. 

If  you  think  the  differences  between  Mac  & PC  are  bad,  I wouldn't  recommend  looking  at  what 
Unix  would  do  to  your  code,  furthermore  the  world  does  not  end  at  MAC  & PC  if  you  get  lucky 
and  enough  money  see  what  a Sun  system  does  they  're  maybe  around  $20k  - apparently  lots  of  fun 
with  graphics  and  3d.  The  only  thing  I could  perhaps  suggest  is  survey  a portion  of  the  target 
market  early  in  the  project  to  see  what  the  majority  of  users  have  as  browsers. 

• Reply  2-  Don 't  forget  Amigas.  I actually  know  one  guy  who  has  browsed  the  net  with  a 
Commodore  64  :)  And  soon  Dreamcasts  and  Playstation  2.  X-Box?  But  you  have  to  draw  the  line 
somewhere,  don't  you? 

• Reply  3- 1 don't  know  that  you  can,  when  it  comes  to  your  customers.  It  seems  fairly  certain  that 
web  developers,  and  web  development  technologies  themselves,  will  have  to  grow  to 
accommodate  the  fact  that  the  web  is  now  being  viewed  on  everything  from  35"  monitors  to  mobile 
phones. 

• Reply  4-As  designers  we  need  to  design  *for*  the  medium,  not  expect  the  medium  to  conform  to 
our  design.  The  nature  of  the  web  is  just  so  variable  that  its  stupid/lazy  to  just  design  for  one 
browser  or  platform.  As  mentioned  by  Oliver,  its  just  going  to  get  worse  when  people  start  wanting 
to  get  web  content  on  phones  and  PDA 's.  Remember  that  content  is  what  people  Content  is  mostly 
textual  and  text  will  go  anywhere.  Its  not  impossible  to  design  an  engaging  site  that  will  view 
correctly  in  all  browsers  from  v3  upwards  on  all  platforms  (including  *nix  / webtv  / dreamcast  / 
palm)  its  just  requires  a bit  of  creativity.  Something  every  designer  should  have  an  abundance  of.  I 
really  want  to  see  that  c64  browser. 

• Reply  5-1  don't  think  there  is  a solution  - other  than  creating  very  standard  html  sites  that  can  be 
viewed  from  lynx  to  ie  5.5.  But  that's  a bit  boring...  Create  for  the  majority,  that's  my  solution.  I'm 
concerned  with  the  ie4. 75  on  Mac  that  does  not  seem  to  support  dhtml.  Other  than  that  I've  been 
able  to  create  a site  that  works  the  same  on  ie  and  NN  4.x  and  above.  That's  good  enough  for  the 
client  so  I'm  happy. 

This  example  shows  how  students  supported  each  other  through  the  virtual  community  and  in  doing  so 
helped  to  develop  their  skills  in  communication,  problem  solving,  and  giving/accepting  positive 
criticism  from  others.  This  community  provided  a “conduit”  for  the  students  in  which  they  could  share 
their  experiences  and  knowledge,  and  provided  a platform  to  constructively  help  each  other. 

Evaluation 

As  part  of  the  project  requirements,  students  were  responsible  for  making  contact  with  the  client  and 
discussing  the  scope  and  legal  aspects  of  the  development  (educational  software,  IP,  etc).  These  had  to 
be  negotiated  so  that  the  client  had  their  needs  satisfied,  while  at  the  same  time  the  students  were 
involved  in  producing  a web-site  that  conformed  to  the  requirements  of  an  academic  unit.  This  often 
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involved  firstly  understanding  the  client’s  needs  and  “educating”  the  client  about  web  production, 
maintenance  and  costs. 

From  the  evaluation  questionnaire  conducted  at  the  end  of  the  unit  it  appeared  that  the  students  were 
very  positive  about  the  structure  of  the  unit  and  the  use  of  the  Listserv.  Almost  all  the  students  wanted 
to  remain  subscribed  on  the  List  as  a form  of  graduate  Listserv  (as  well  as  offering  advice  to  other 
students  who  will  be  enrolled  in  the  future).  Grounding  the  industry  project  in  a commercial 
environment  with  the  use  of  a Listserv  for  inter-team  and  inter-student  communication  was  clearly 
successful,  and  added  value  to  the  students'  experience  in  this  unit.  Contributing  factors  that  led  to  its 
success  include: 

• The  Listserv  was  treated  as  a mainstream  activity  which  replaced  face-to-face  classes,  and  was 
integrated  into  the  assessment  system 

• The  industry  partnership  involved  students  dealing  with  authentic  commercial,  client  and  technical 
problems  in  the  workplace  and  using  the  Listserv  to  ask  for  advice  and  share  experiences 

The  tutors  acted  as  a facilitators  and  not  experts  by  not  fostering  discussion  rather  than  questions; 
dealing  with  inappropriate  responses  offline;  and  identifying  underlying  issues  and  redirecting  these  as 
questions  back  to  the  Listserv  for  students  to  reflect  on  and  discuss  where  appropriate. 

Critical  success  factors 

In  this  online  unit,  project  based  learning  combined  with  asynchronous  learning  tools  enabled  students 
to  develop  a relationship  with  a client,  create  solutions  to  a design  problem  and  develop  a project  brief. 
In  the  approach  adopted  here,  the  practice  has  added  to  it  the  learning  aims,  specific  learning  processes, 
assessment  processes  and  the  learning  outcomes.  Each  of  these  involved  partnerships  with  industry 
clients  with  learners  taking  advantage  of  the  real  world  context  to  gain  professional  knowledge. 

The  success  of  the  unit  can  be  attributed  to  the  following  factors: 

• A virtual  community  was  formed  through  the  use  of  a Listserv,  which  encouraged  collective  and 
collaborative  learning  rather  than  individualistic.  The  online  social  environments  provided  scope 
for  group  interaction,  sharing  and  discussion  and  in  depth  exploration  of  issues. 

• The  content  of  the  course  was  opened  ended,  and  students  had  access  to  a website  where  they 
could  add  resources  or  choose  the  most  relevant  ones  (see  Figure  3).  Access  to  relevant 
knowledge  resources  was  oar  of  the  “just  in  time  “approach  to  planning,  i.e.  the  choice  of  selecting 
resources  is  left  to  students  who  have  to  identify  a learning  need; 

• The  development  of  networked  collaborative  learning  requires  a focus  on  the  processes  of 
collaboration,  and  the  well-being  and  development  of  the  collaborative  group  (McGrath,  1 990).  In 
summary,  this  involves: 

o openness  in  the  educational  process 

o self-determination  in  learning 

o a real  purpose  in  the  cooperative  learning  process 

o a supportive  learning  environment 

o collaborative  assessment  of  learning 

o assessment  and  evaluation  of  the  ongoing  learning  process. 

The  notion  of  professional  development  and  growth  was  recognised  by  learners  to  be  more  important 
than  learning  facts,  figures  and  project  management  procedures.  Students  had  to  work  closely  with  an 
industry  client,  develop  a project  brief  and  then  communicate  and  negotiate  the  final  product  to  a group 
of  peers.  Peer  learning  enabled  student  to  reflect  on  the  experience  and  learn  from  it. 
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Conclusion 


The  study  illustrates  an  effective  learning  design  involving  experiential  real  world  learning,  which  was 
integrated  into  the  final  year  of  tertiary  multimedia  course.  The  unit  aimed  to  consolidate  core 
multimedia  skills  learnt  in  other  units,  while  at  the  same  time  having  students  making  industry  contacts 
and  developing  portfolios  item  to  assist  with  their  job  applications. 

Through  authentic  settings,  students  were  engaged  with  industry  projects,  clients  and  assessment  tasks. 
This  was  complemented  with  a virtual  environment  and  asynchronous  conferencing  in  which  students 
exchanged  ideas  and  helped  each  other  cope  with  these  authentic  tasks.  This,  in  combination  with 
client  negotiations,  promoted  the  development  of  professional  skills  and  process  knowledge,  which 
supported  both  process  and  product  skills. 
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Abstract:  In  the  past  decade,  the  use  of  technology  for  foreign  language 
instruction  has  expanded  rapidly  and  with  it  has  come  the  expectation  that 
online  methods  of  teaching  will  supersede  other  more  traditional  methods  of 
language  learning.  This  article  reports  on  a longitudinal  study  of  Australian 
students  studying  French  in  three  different  modes.  The  results  indicate  that 
performance  and  achievement  in  ‘open  learning  mode  where  students  self- 
direct  their  own  learning,  has  the  most  successful  approach,  and  that  multimedia 
enhanced  teaching  of  foreign  languages  is  highly  successful. 


Literature  review 

A number  of  benefits  for  students  related  to  the  general  use  of  technology  for  foreign  language 
learning.  These  include  improved  motivation,  self-concept  and  mastery  of  basic  skills,  increased 
student  centered  learning  and  engagement  in  the  learning  process  (Stepp -Greany,  2002).  Use  of 
multimedia  technologies  for  low  achieving  students  have  been  shown  to  have  facilitate  learning  when 
used  to  illustrate  concepts  and  factual  information  (Nowaczyk,  1 998).  For  networked  classrooms, 
Beauvois  (1998)  found  that  more  student-student  interaction  promoted  communicative  awareness,  as 
self-disclosure  through  conversation  was  common.  While  technology  enhanced  learning  has  been 
applauded  in  the  research  literature  as  enhancing  and  supporting  language  learning,  it  is  the  theory  of 
constructivism  that  underpins  instructional  design  of  technology  that  drives  new  approaches  to 
language  learning  (McLoughlin  & Oliver,  1998).  The  present  study  provides  evidence  that  technology 
enhanced  learning  promotes  successful  outcomes  when  compared  with  more  traditional  methods  of 
language  teaching. 


A longitudinal  study  of  different  language  teaching  modes 

Longitudinal  research,  which  has  monitored  three  discrete  types  of  students  learning  French  ab  initio, 
has  been  carried  out  at  the  University  of  New  England,  Australia,  since  1992.  The  three  modes 
investigated  were  for  a (i)  on-campus  (on  campus)  students,  who  received  face-to-face  teaching  (ii)  off- 
campus  (distance  education)  students,  studied  at  a distance  except  for  a four-day  intensive  on-campus 
workshop  each  year;  and  (iii)  Open  Learning  studaits,  who  learnt  entirely  by  correspondence  and  had 
no  face-to-face  contact  with  academic  staff.  See  table  1. 


Student  Cohort 

Methodology 

Learning  approach 

Gnoi^  1 - On  campus, 
face-to  face  mode  of 
study 

Bonjour,  ^ va?  is  a traditional  method  of  teaching  French 
developed  in  North  America.  It  is  a traditional,  grammar - 
based  course  with  explanations  in  English. 

Traditional  ie  teacher 
directed 

Group  2 - Off-campus 
distance  learners 

The  off-campus  students  self  taught  and  self  directed  their 
own  learning  except  for  a brief,  intensive  residential  school 
six  months  after  the  start  of  their  studies. 

Independent,  with  some 
facet  0- face  sessions 

Groi^  3 - Open 

Learning 

(Selfpaced,  resource- 
based  learning) 

The  open  learning  students  used  an  innovative  multimedia 
course,  French  in  Action,  developed  at  Yale.  A semi- 
immersion course,  it  comprises  twenty-six  30minute  videos 
entirely  in  French,  together  with  related  text/wOrkbooks. 

Independent,  with  no 
faceto-face  sessions 

Table  1 : Student  groups  and  modes  of  study 
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The  study  provides  evidence  that  technology  enhanced  learning  promotes  successful  outcomes  when 
compared  with  more  traditional  methods  of  language  teaching  (Hutchinson  & Koplin,  1998). 

Confirmatory  evidence  of  the  success  of  the  ope  n learning  mode 

In  1993,  the  results  showed  that  Open  Learning  students  performed  significantly  better  than  the 
students  in  the  other  two  groups  but,  again,  a controlled  statistical  study  was  not  possible,  as  there  were 
still  data  limitations  (See  Table  2).  The  question  was  raised,  of  course,  as  to  whether  this  was  merely  a 
chance  occurrence  and,  if  it  were  not,  then  why  the  Open  Learning  students  outperformed  the  others  so 
convincingly  when  they  appeared  to  be  the  most  pedagogically  disadvantaged? 


Mode  of  Study 

Average  Final  Mark  (with  standard 

deviation  in  parentheses) 

Open  Learning 

77.80  (9.26) 

On-campus 

68.47  (13.28) 

Oif^  campus 

72.46(11.70) 

On-campus  + Off-campus 

71.18(12.25) 

Table  3:  Relative  performance  of  modes  of  study 

Further  investigation  involving  a comparison  of  the  performance  over  three  years  of  all  French  in 
Action  (multimedia)  students  (from  1996-2001)  with  that  of  all  students  studying  a more  traditional 
method,  showed  that  students  learning  with  the  multimedia  course  had  outperformed  those  learning  in  a 
more  traditional  manner  (Hutchinson  & Koplin,  1997).  The  results  of  the  study  support  the  findings  of 
Dywer  (1996)  and  Weiss,  1994).  In  summary,  the  results  indicated  that  students  studying  without  any 
direct  teacher  contact  outperformed  students  in  the  other  two  groups  because  of  the  use  of  a multimedia 
learning  package  which  allowed  self-paced,  self  directed  learning  to  occur. 

Conclusions 

The  purpose  of  the  longitudinal  study  was  to  investigate  the  comparative  academic  achisvement  of 
these  modes  of  language  learning,  and  to  apply  lessons  learnt  to  the  creation  of  improved  models  of 
language  teaching.  The  solid  performance  of  students  using  a self-directed  multimedia  package,  when 
compared  with  other  students  of  French  following  a more  traditional  course,  even  with  online 
enhancement,  shows  the  pedagogical  benefits  of  the  multimedia  learning  environment. 


References 

Hutchinson,  H.  and  Koplin,  M (1998).  "Home  alone  with  French  in  Action:  successful  language  learning  in 
isolation  at  tertiary  \QyeV\  AUMLA,  89,  67-78. 

Hutchinson,  H.  and  Koplin,  M.  (1997).  'French  in  Action:  is  face-to-face  language  teaching  really  necessary?" 
AULLA  Paper  presented  at  the  Australasian  Language  and  Literature  Association)  conference,  Sydney  University, 
February,  1997 

Beauvois,  M.  (1998).  Conversations  in  slow  motion:  Computer -mediated  conununication  in  the  foreign  language 
classroom.  The  Canadian  Foreign  Language  Review,  54(2),  198-217. 

Dywer,  D.  (1996).  A response  to  Douglas  Noble:  We're  in  this  together.  Educational  Leadership,  54(3),  24-27. 

McLoughlin,  C.  & Oliver,  R.  (1998).  Maximising  the  language  and  learning  link  in  computer  learning 
environments.  British  Journal  of  Educational  Technology,  29  (2),  125-136. 

Nowaczyck,  R.  (1998).  Student  p erceptions  of  multimedia  in  the  undergraduate  classroom.  InternationalJournal 
of  Instructional  Media,  25y  367-368. 

Stepp-Greany,  J.  (2002).  Student  perceptions  of  language  learning  in  a technological  environment:  implications  for 
the  new  millennium.  Language  Learning  and  Technology,  5(1),  165-180. 

Weiss,  J.  (1994).  Keeping  up  with  the  research.  Technology  and  Learning,  74(5),  30-34. 


1318 


Multiple  perspectives  on  the  evaluation  of  online  discussion 


Catherine  McLoughlin 
Australian  Catholic  University,  Australia 
c.mcloughlin@signadou.acu.edu.au 

Mary  Panko 
UNITEC,  New  Zealand 
mapanko@unitec.ac.nz 

Abstract;  This  paper  attempts  to  examine  different  methods  of  content  analysis  for 
student  inline  discussion  that  takes  place  on  bulletin  boards.  The  purpose  of  that 
analysis  is  to  determine  whether  higher  order  thinking  can  be  distinguished  vwthin 
transcripts  of  dialogue.  The  context  of  the  analysis  was  a higher  education 
undergraduate  course.  A group  of  students  were  presented  with  a discussion  topic  and 
were  given  a series  of  criteria  against  which  their  work  would  be  assessed.  At  the  end 
of  the  discussion  their  dialogue  was  analysed  using  three  different  techniques  which 
were  then  compared  to  identify  whether  any  one  method  could  be  recommended  to 
other  practitioners.  Suggestions  for  evaluation  of  bulletin  board  transcripts  are  made 
on  the  basis  of  this  investigation. 


The  development  of  discussion  boards  as  learning  tools 

Discussion  boards  or  bulletin  boards  are  part  of  a general  category  of  CMC.  Discussion  boards  allow 
students  to  interact  with  one  another  and  with  a facilitator  and  can  be  used  for  a variety  of  teaching 
techniques  designed  to  stimulate  cognitive  and  metacognitive  skills. 

An  extensive  literature  base  (Harasim,  1990;  Mason,  1994)  has  developed  which  supports  the  educational 
value  of  CMC  as  a teaching  and  learning  strategy.  However,  most  investigations  into  the  benefits  of  this 
form  of  interaction  have  tended  to  be  quantitative  in  nature  (McKenzie  & Murphy,  2000;  Nastasi  & 
Clements,  1992)  concentrating  on  either  the  number  of  exchanges  that  take  place,  or  on  diagrammatic 
representations  of  the  pattern  of  interaction  between  participants.  During  the  last  decade  however,  methods 
of  analysis  have  been  applied  to  this  communication  technique  to  identify  more  precisely  the  varied 
educational  dimensions  that  may  be  found  within  the  online  text  (Henri,  1992;  Gunwardena  et  al,  1997). 
Different  forms  of ‘content  analysis’  have  been  developed  and  refined  to  aid  the  categorisation  of  dialogue 
produced  in  any  debate  or  online  seminar.  MacKinnon  and  Aylward,  (2000)  for  example,  have  used  a 
system  of  coding  online  dialogue  using  ‘cognotes*,  where  the  e-moderator  attaches  icons  to  ongoing  text  in 
order  to  indicate  the  nature  of  the  interactions  which  are  occurring.  This  is  intended  to  encourage  higher 
student  achievement  and  more  positive  relationships  between  the  participants. 


Aim  of  study 

Currently,  content  analysis  has  been  seen  as  a time-consuming  activity  which,  while  it  can  produce 
valuable  insights  for  educational  researchers,  does  not  lend  itself  to  practical  application  by  teachers  who 
might  want  to  evaluate  the  online  discussion  element  of  their  programmes.  This  pilot  study  seeks  to 
compare  the  relative  strengths  and  limitations  of  established  content  analysis  techniques  and  then  propose  a 
simp  ler  method  that  is  useful  and  practical.  This  would  then  allow  course  providers  to  decide  whether  their 
course  participants  are  displaying  cognitive  and  metacognitive  aspects  of  learning  at  the  level  they  expect. 


Depth  of  learning 

McLoughlin  and  Oliver  (1998)  have  indicated  that  Higher  Order  Thinking  (HOT)  is  not  easy  to  define  but 
can  be  recognised  when  encountered.  These  authors  have  summarised  recent  debate  on  the  definition  of 
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HOT  as  referring  to  thinking  which  is  complex,  multi-faceted,  self-directed  and  one  in  which  the  learner 
plays  an  active  role  (1998,  p.  242).  This  is  in  line  with  Sternberg’s  (1985)  theory  of  intelligence  that 
considers  the  basis  for  higher  order  cognitive  processing  to  be  executive  processes  used  in  planning, 
monitoring  and  decision  making.  For  the  purpose  of  this  paper  cognitive  skills  are  generally  regarded  as 
applying  to  student  understanding,  reasoning,  and  the  development  of  critical  thinking  and  problem  solving 
skills,  while  metacognition  relates  to  knowledge  of  one’s  own  cognition  and  the  further  regulation  of  that 
cognition  (Hara,  Bonk  and  Angeli,  2000).  Henri  and  Parer  (1992),  in  addition  to  their  model  of  content 
analysis,  also  identified  the  level  of  information  processing  as  ‘surface’  or  ‘in-depth’.  They  recognised 
surface  processing  to  include:  making  judgements  without  justification;  stating  that  one  shares  opinions 
already  stated;  repetition;  and  irrelevant  questions.  In-depth  processing,  Henri  stated,  was  demonstrated 
when  participants  linked  facts  and  ideas;  offered  new  elements  of  information;  discussed  advantages  and 
disadvantages;  and  made  justified  judgements. 


The  study 

A group  of  10  adult  learners  on  a higher  education  programme  took  part  in  an  online  discussion  which  was 
an  obligatory  part  of  their  course  assessment.  None  of  these  participants  had  previously  experienced  online 
discussion.  The  asynchronous  computer  conferencing  software  used  was  the  discussion  board  contained 
within  BlackBoard  Courseinfo,  a web  course  management  system.  The  discussion  was  entitled: 
“Integrating  technology  into  education  - a constructivist  approach”.  The  students  were  required  to  take  part 
on  at  least  four  occasions  in  a discussion  on  the  topic  within  a four-week  time  frame.  The  criteria  upon 
which  their  contributions  were  assessed  in  the  course  were  obtained  from  Campbell’s  experiences  (1999) 
and  were:  regularity  of  contribution,  links  to  the  comments  of  others,  sharing  of  ideas  and  experiences,  and 
reference  to  readings  or  Websites. 


Evaluation  by  content  analysis 

Content  analysis  involves  comparing,  contrasting  and  categorising  elements  of  written  dialogue  for 
meaning,  as  opposed  to  discourse  analysis  which  examines  the  process  of  communication  including 
specific  speech  acts.  A range  of  examples  of  models  have  been  selected  for  this  study  including  those 
developed  by  Henri  (1992)  and  Gunwardena,  Lowe  and  Anderson  (1997)  and  an  earlier  method  of 
determining  the  complexity  of  student  responses,  the  Biggs’  SOLO  taxonomy  (1982).  Modifications  to 
these  schemes  are  also  incorporated  in  the  analysis,  ‘since  every  computer  conference  will  have  its  own 
unique  attributes,  researchers  may  have  to  design  electronic  discussion  group  evaluation  criteria  on  a case 
by  case  basis,’  (Hara,  Bonk  and  Angeli,  2000,  p.l43).  In  particular  the  models  were  chosen  to  enable  higher 
orders  of  thinking  to  be  identified  in  the  dialogue,  as  this  is  frequently  identified  as  a required  outcome  in 
tertiary  level  education  programmes. 

Table  1:  Gunawardena,  Lowe  and  Anderson  content  analysis  model  (1997) 


Categories  For  Data  Analysis 

Examples 

Phase  1 : Sharing  and  Comparing 

Observations,  examples  and  descriptions,  basic  agreement 

Phase  2:  Discovery  and  exploration  of 
difference 

Questions  re  differences,  clarifying  statements 

Phase  3 : Negotiation  of  meaning  and  co- 
construction of  knowledge 

Identification  of  common  ground,  statements  of  compromise 

Phase  4:  Testing  and  revision  of  ideas 

Testing  of  ideas  against  personal  knowledge 

Phase  5:  Awareness  of  newly  constructed 

Metacognitive  statements,  reflection 

Model  1:  Gunawardena,  Lowe  and  Anderson’s  (1997)  criteria  for  content  analysis 
This  model  was  originally  designed  to  examine  the  construction  of  knowledge  within  online  debates  in  a 
similar  manner  to  the  design  of  Nastasi  and  Clements  (1992)  who  considered  that  the  resolution  of 
cognitive  conflict  could  be  used  as  a measure  of  cognitive  learning.  See  table  1. 
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Model  2:  Henri  and  Parer's  qualitative  criteria  for  content  analysis  (1 992) 

This  model  identifies  five  dimensions  within  the  framework  of  the  content  analysis:  participative, 
(examined  by  quantitative  analysis),  social,  interactive,  cognitive  and,  metacognitive.  Each  of  these  have 
operational  definitions  and  related  indicators,  however  only  the  latter  two  have  been  examined  in  this  study 
as  the  focus  of  the  study  was  to  analyse  participants’  learning  outcomes  rather  than  the  collaborative 
process  of  achieving  that  result.  Nevertheless,  it  is  recognised  that  in  a constructivist  learning  environment, 
where  collaborative  learning  is  occurring  in  a social  setting,  interactions  between  the  participants  must  play 
a significant  role  in  the  achievement  of  understanding. 


Categories  for  data  analysis 

Definitions/Indicators 

Cognition 

1 . Elementary  classification 

Observing  a problem  and  its  linkages,  questions, 
basic  descriptions 

2.  In-depth  classification 

Analysing  a problem  re  beliefs  and  assumptions, 
referential  criteria 

3.  Inferencing 

Inducting  and  deducting  (based  on  prior 
propositions),  generalisations,  conclusions 

4.  Judgment 

Making  decisions  (I  agree),  appreciations,  criticisms 

5.  Strategy  application 

Proposing  actions,  evaluations,  decisions 

Metacognition 

1 . Metacognitive  knowledge 

Perceptions  of  oneself  as  learner  and  thinker, 
realisation  of  strategic  knowledge 

2.  Metacognitive  skills 

Planning,  regulation,  evaluation  and  self-awareness 

Table  2:  The  Henri  and  Parer  content  analysis  model 

Hara,  Bonk  and  Angeli  (2000)  have  added  the  categories  ‘reflection’  and  ‘self-questioning’  to  Henri’s  list 
of  metacognitive  skills  and  these  have  been  applied  in  this  study.  Even  with  these  modifications, 
metacognition  is  not  often  clear,  possibly  because  it  occurs  infrequently  in  the  discussion  being  studied. 

Model  3 - Biggs's  SOLO  taxonomy  (Biggs  and  Collis,  1982) 

This  form  of  textural  analysis,  designed  primarily  for  the  print-based  word,  was  an  earlier  attempt  to 
examine  the  complexity  of  students’  responses.  Although  it  was  not  designed  to  function  as  a model  of 
content  analysis,  it  was  trialed  in  this  study  to  see  if  Biggs’s  categories  of  HOT  would  provide  a useful 
alternative  technique.  The  outcomes  have  the  advantage  of  being  generic,  and  provide  a fairly  clear  cut-off 
between  students’  surface  and  deep  learning  approaches,  at  a point  separating  level  three  from  level  four. 


Level  of  outcome 

Indicators 

1 . Prestructural 

Irrelevant  information,  no  meaningful  response 

2.  Unistructural 

Answer  focuses  on  1 relevant  aspect  only 

3.  Multistructural 

Several  relevant  ideas,  but  not  co-ordinated 

4.  Relational 

Integrated,  meaning  is  understood 

5.  Extended  abstract 

Answer  goes  beyond  information  given. 

Table  3 : Levels  of  Biggs’s  SOLO  taxonomy 

Method  applied  in  this  study 

The  fifty  messages  contributed  by  students,  primarily  in  chronological  order,  were  first  listed  by  dialogue 
number.  Remarks  made  by  the  e-moderator  were  deleted  from  this  list.  Initial  entries  placed  on  the  board 
during  the  first  face-to-face  session  were  also  removed  as  these  did  not  pertain  to  the  assignment  but  were 
designed  to  familiarise  students  with  the  process  of  contributing. 

Following  this  preparatory  stage  the  three  main  techniques  described  above  were  applied  to  the  whole 
sequence  at  an  entire  dialogue  message  level.  In  addition,  surface  processing  compared  to  in-depth 
processing  following  the  Henri  model  was  also  evaluated.  Within  each  message  the  category  was  selected 
on  the  basis  of  the  highest  order  of  thinking  demonstrated  within  the  dialogue.  In  other  words,  messages 
were  not  sub-divided  into  smaller  units  of  meaning.  There  were  three  reasons  for  this;  firstly,  subdivision 
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would  have  simply  produced  additional  data  indicating  an  increased  amount  of  lower  value  learning  levels 
without  affecting  the  extent  to  which  higher  order  thinking  might  be  discerned.  Secondly,  this  whole 
message  analysis  overcame  the  problem  of  deciding  where  to  slice  the  dialogue  to  achieve  appropriate 
subsections.  Thirdly,  it  saved  a large  amount  of  analytical  time.  It  did  however  obscure  the  precise 
identification  of  student  performance,  as  described  in  Gunawardena’s  model  where  each  of  the  five  phases 
is  categorised  in  more  detail.  To  increase  the  rigor  of  this  subjective  process  a second  researcher  examined 
every  fifth  entry  and  applied  the  same  series  of  content  analyses  to  this  20%  sample. 


Results 

All  models  showed  a distribution  of  rating  scores  from  1-5,  with  relatively  few  responses  at  the  higher 
level  of  cognition.  Table  4,  indicating  the  relative  ratings  allocated  by  each  model,  shows  that  the 
Gunawardena  model  produced  a greater  proportion  of  low-level  ratings  than  the  other  two.  In  the  Henri 
model  metacognitive  references  were  also  listed  (M)  which  occurred  mainly  in  the  cognition  category  5 
ratings  but  also  one  occasion  in  the  category  4.  This  table  indicates  overall  comparative  frequencies  but 
does  not  indicate  similarities  or  differences  between  the  models  per  individual  dialogue. 


Content  analysis 
rating 

Gunawardena 

Henri 

Biggs 

1 

24 

15 

6 

2 

9 

16 

8 

3 

9 

8 

16 

4 

6 

5 + IM 

15 

5 

2 

6 + 3M 

5 

Table  4:  Content  analysis  rating  scores  for  all  three  models 
To  examine  whether  the  three  models  did  show  consistency  between  the  individual  ratings  a simple 
correlation  analysis  was  applied  to  show  degrees  of  association  or  similarities  between  them  all  (see  Table 

5). 


Gunawardena 

Henri 

Gunawardena 

- 

- 

Henri 

.7405 

- 

Mifigs 

.6624 

.7025 

Table  5:  Correlation  Coefficient  between  content  analysis  models 

Table  5 shows  that  there  is  a strong  correlation  between  the  Henri  and  Gunawardena  models  and  a lower 
one  between  the  Gunawardena  and  the  Biggs  analyses.  The  reason  for  this  difference  is  that  results  from 
the  Gunawardena  rating  allocate  50%  extra  scores  at  the  lower  end  of  the  learning  level  scale  in  comparison 
to  Biggs  (see  Table  4).  Nevertheless,  all  three  models  do  show  statistically  strong  degrees  of  correlation. 


Conclusions 

Two  main  conclusions  can  be  drawn  from  this  study.  Firstly,  practitioners  wanting  to  apply  one  of  these 
models  will  be  able  to  do  so  quite  readily  if  they  select  their  categories  at  the  level  of  the  whole  message, 
rather  than  sub -categorising  the  text.  The  consistent  application  of  any  appropriate  technique  will  then 
indicate  levels  of  HOT  in  their  own  online  discussions  which  can  then  be  used  for  modifying  teaching 
techniques. 

Secondly,  the  nature  of  the  discussion  and  the  expectations  of  the  course  providers  should  determine  the 
choice  of  the  primary  model  selected.  If  the  aim  is  to  encourage  a collaborative  learning  environment,  then 
the  Gunawardena  model  provides  more  insight  into  whether  or  not  that  has  been  achieved.  This  model  also 
incorporates  metacognition  into  the  main  framework,  which  is  essential  if  any  evaluation  of  HOT  is 
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required.  The  Henri  model,  with  its  metacognitive  aspect  applied  as  a separate  framework  dimension  tends 
to  highlight  individuals’  success  in  displaying  internal  cognition  in  a more  Piagetian  sense,  although  within 
some  of  her  categories  there  are  opportunities  to  acknowledge  the  interactive  nature  of  the  discussions.  A 
major  criticism  of  Henri’s  model  (McLoughlin  and  Luca,  1999,  p.223)  is  that  ‘it  was  designed  for  contexts 
where  there  was  a strong  teacher  presence,  and  is  not  readily  applicable  to  a learner-centred  conferencing 
environment’.  These  authors  consider  that  the  Gunwardena  model  a more  suitable  tool  for  reflecting 
collaborative  and  social  factors,  on  the  basis  that  it  proposes  a social  constructivist  approach  to  knowledge 
building  in  a online  environment. 

In  the  Biggs  model  the  categories  are  clearly  hierarchical  which  leads  to  an  overall  value  judgement  about 
the  content  of  each  message.  This  rules  out  completely  the  possibility  that  socially  interactive  functions  are 
acknowledged  as  part  of  cognitive  development.  Although  all  three  are  structured  to  indicate  levels  of 
learning  and  deeper  understanding  demonstrated  by  the  text,  it  is  only  the  Gunawardena  cognition  model 
that  relates  this  achievement  to  the  degree  of  collaboration  demonstrated. 

In  considering  what  model  to  apply  and  what  modifications  might  be  required,  the  nature  of  the  student 
body  should  also  be  taken  into  account.  The  learning  style  of  participants  is  one  issue  that  should  be 
considered  and  ‘it  is  suggested  that  current  research  literature  in  the  area  of  learning  styles  and  strategies 
can  provide  instructional  designers  with  insights  into  differences  in  learning  and  performance  that  can  be 
factored  into  the  design  process’  (McLoughlin,  1999,  p.  223).  Research  has  shown  (Klemm  and  Snell, 
1996)  that  unless  discussions  are  summatively  assessed,  dialogue  will  be  minimal.  It  might  be  asked  if 
instructional  designers  are  merely  providing  increasingly  elegant  hoops  for  learners  to  jump  through. 


Summary 

The  significance  of  this  paper  is  that  it  should  encourage  online  teachers  to  develop  more  effective  teaching 
techniques  by  enabling  them  to  evaluate  the  effects  that  their  own  online  practices  play  in  students’ 
cognition  and  metacognition.  A summary  of  the  analysis  approach  for  online  transcripts  can  be  summarised 
in  there  steps,  and  requires  the  selection  of  a content  analysis  approach  that  is  congruent  with  the  intended 
learning  outcomes  and  pedagogy  of  the  course  of  study. 

The  three  stages  for  analysis  are  as  follows: 

1.  First,  analyse  dialogue  content  at  whole  message  level. 

2.  Second,  allocate  a rating  according  to  highest  level  of  thinking  evidenced  in  message  (See  for  example 
the  scheme  proposed  by  McLoughlin  & Oliver,  1998). 

3.  Third,  select  or  modify  a content  analysis  technique  that  reflects  the  philosophy  of  the  course  in  respect 
of  student  learning  outcomes  and  pedagogy. 

Data  analysis  and  coding  of  transcripts  can  then  be  carried  out  to  determine  the  forms  of  thinking  and 
cognitive  skills  that  are  embedded  in  the  dialogue. 
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Abstract:  Conducting  scientific  investigations  and  thinking  scientifically  are  among  the 
most  complex  intellectual  activities  that  learners  can  pursue,  yet  the  teaching  of  science  in 
general  has  not  fully  adopted  innovative  and  constructivist  approaches  to  develop  these 
skills.  In  addition,  progress  towards  the  practical,  widespread  implementation  of 
technology  to  support  constructivist  teaching  approaches  has  been  limited.  Where  there  is 
a smaller  number  of  leaders  who  do  embrace  newer  educational  theories  and  attempt  to 
incorporate  research  findings  in  their  teaching,  there  continues  to  be  a substantial  gap 
between  educational  research  and  development  projects  and  actual  practice  in  science 
teaching  overall.  This  paper  discusses  present  barriers  to  adoption  of  ICT  in  science 
teaching,  current  thought  on  how  ICT  can  transform  science  teaching  and  how 
technologies  can  support  the  development  of  scientific  understanding. 


Overview 

A preliminary  survey  was  taken  of  tertiary  teaching  staff  to  seek  comments  on  their  use  of  ICT  and  its 
impact  on  their  teaching  and  their  students'  learning.  Data  obtained  from  first  year  science  teachers  in  the 
few  Australian  universities  offering  science  by  distance  education,  together  with  the  Open  University  in  the 
United  Kingdom  and  Athabasca  University,  the  open  university  in  Canada,  reveals  widespread  use  of  ICT 
but  fewer  instances  of  innovative  learning  designs  supported  by  principled  application  of  technology.  This 
evidence  suggests  that  there  may  be  a dearth  of  innovative  teaching  in  the  sciences  us  tiling  ICT,  and  that 
while  technology  has  the  potential  to  play  a role  in  supporting  authentic  scientific  activity  in  the  classroom, 
there  are  nevertheless  barriers  to  the  adoption  and  integration  of  ICT  to  teach  science  subjects  at  a distance. 
Specifically,  while  technology  an  place  a greater  range  of  tools  and  resources  at  the  disposal  of  teachers 
and  students,  and  increase  the  opportunity  for  social  interchange  at  the  heart  of  authentic  science  practice, 
there  remain  a range  of  problems  to  be  addressed. 


Barriers  to  ICT  adoption  in  science 

The  focus  of  the  remainder  of  this  paper  is  on  the  teaching  of  first  year  university  science  by  distance. 
Teaching  tertiary  level  science  subjects  with  practical  components  in  the  distance  education  (DE)  mode 
present  some  particular  difficulties  and  this  may  be  one  reason  why  teaching  some  science  subjects  at  a 
distance  is  still  relatively  new  in  Australia.  For  exampel,  simply  makign  resources  available  in  flexible 
mode  a online  and  expecting  students  to  learn  in  neither  realistic  nor  effective.  Attention  to  new 
pedagogies  and  curriculum  structures  is  essential  to  ensuring  that  technology  is  integrated  with  learning. 
There  has  been  some  reporting  of  teaching  science  at  a distance  in  the  literature,  (Rowntree,  1992), 
(Bennett,  1994),  (Druskovich,  Klease,  Teague,  Lyall,  & Mayes,  1996),  (Hollingworth,  1998),  (Kennepohl 
& Last,  2000). 

Difficulties  and  problems  which  must  be  addressed  in  the  teaching  of  science,  include: 

• There  is  a tendency  for  teachers  to  opt  for  ‘content  coverage”  rather  than  teaching  of  higher  order  thinking 
skills. 

• The  need  to  foster  understanding  of  concepts  related  to  thinking  processes  in  science.  Hypothesis  formation 
and  generation,  problem  solving,  critical  thinking  are  higher  order  cognitive  skills  require  advanced  level 
teaching  skills  and  use  of  ICT  in  innovative  ways  (Hollingworth  & McLoughlin,  2000). 
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• Teachers  need  to  focus  on  development  of  metacognitive  awareness  of  students'  own  learning  and  their  skills 
of  reflection.  At  first  year  level  many  science  students  are  at  Perry’s  basic  dualism  stage  of  development 
(Perry,  1970). 

Some  further  difficulties  which  must  be  addressed  in  online  and  distance  delivery  of  science  are: 

• The  need  to  provide  adequate  practical  experience  in  the  distance  mode.  This  includes  the  physical 
difficulties  of  providing  convenient  laboratory  facilities  and  timing  difficulties,  if  it  is  desired  to  give  a 
parallel  presentation  of  theory  and  practical. 

• The  need  to  provide  students  with  opportunities  for  social  discourse  with  peers  and  lecturers  in  which 
knowledge  of  science  concepts  is  constructed  and  more  generic  communication  skills  are  developed. 

These  problems  and  others  restrict  the  usefulness  of  educational  technology  in  distance  learning 
contexts.  Other  issues  may  be  lack  of  teacher  expertise  with  computers,  issues  of  access  and  uptake 
and  funding  pressures  that  restrict  innovative  applications. 


Current  & future  practice  in  science  teaching  at  a distance 

The  average  ratings  on  statements  in  the  survey  about  the  impact  of  ICT  on  lecturers'  approach  to  teaching 
and  their  students'  approach  to  learning  are  given  in  Table  1.  Ratings  were  based  on  a 5 point  Likert  scale 
ranging  from  1 = Strongly  disagree  to  5 = Strongly  agree.  Not  all  respondents  were  involved  in  both  on 
campus  and  off-campus  teaching,  so  some  statements  were  not  applicable  to  them. 


ICT  has  impacted  on: 

Rating 
(Out  of  5) 

my  approach  to  teaching  on  campus  students 

4.25 

my  approach  to  teaching  off  campus  students 

3.8 

my  on  campus  students' approach  to  their  learning. 

3.25 

my  off  campus  students'  approach  to  their  learning. 

3.2 

the  way  I support  my  on  campus  students' learning. 

3.75 

the  way  I support  my  off-campus  students'  learning. 

3.8 

The  two  modes  of  teaching  are  converging. 

4.3 

Table  1 : Rating  of  impact  of  ICT  on  science  teaching  and  learning 

The  most  significant  points  to  note  from  Table  1 are  that  strongest  agreements  are  on  the  impact  of  ICT  on 
campus  teaching  and  on  the  convergence  of  the  on  campus  and  off-campus  modes  of  teaching.  The  lack  of 
impact  of  ICT  on  distance  teaching  may  be  due  to  the  fact  that  off-campus  students  do  not  have  as  ready 
access  to  computer  facilities  as  on  campus  students,  and  that  print  materials  tend  to  dominate.  One 
respondent  commented:  ''We  have  continuous  competition  between  exploring  new  technologies  to  improve 
delivery  versus  staying  with  low  technology  to  make  the  course  materials  universally  accessible.  ** 
Convergence  of  on  campus  and  off-campus  teaching  may  be  because  of  the  contraction  of  staff  numbers  in 
science  departments  around  Australia,  there  has  been  pressure  to  rationalise  the  teaching  of  on  campus  and 
off-campus  students  as  much  as  possible.  In  addition,  the  availability  of  the  Internet  means  that  more  and 
more  of  the  learning  resources  being  developed  are  as  readily  accessible  to  off-campus  students  as  on 
campus,  increasing  convergence  between  the  two  modes  of  delivery. 

Current  practices  in  science  teaching  in  distance  learning  contexts  and  the  implementation  of  ICT  may  be 
evaluated  with  respect  to  a number  of  relevant  criteria.  Essentially,  constructivist  learning  with  technology 
should  enable  students  to  develop  skill  in  designing  and  constructing  new  knowledge  (Jonassen  & Peck, 
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1999).  Best  educational  practice  would  explicitly  aim  to  enhance  these  skills.  We  discuss  current  practice 
and  use  of  ICT  under  three  main  headings  in  the  following. 


Communicating  and  constructing  new  knowledge 

Survey  quotes:  ’’Communications  with  off  campus  students  is  the  major  change."  "Off  campus  students 
are  sometimes  reluctant  to  use  discussion  groups,  but  it  is  improving." 

The  survey  showed  asynchronous  communication  tools  such  as  e-mail,  bulletin  boards,  telephone 
conferencing  and  voicemail  are  in  common  use.  The  number  of  DE  students  having  access  to  Internet 
facilities  has  increased  markedly  in  the  last  five  years  from  about  20%  to  80%.  Some  lecturers  have  noted 
a reluctance  on  the  part  of  students  to  participate  in  on-line  forums,  etc.  experience  in  other  areas  shows 
this  could  be  overcome  in  part  by  making  participation  assessable,  (Fox,  Herrmann,  & Taylor,  1999)  but 
there  are  still  equity  issues  to  be  faced  before  making  use  of  these  avenues  for  communication  mandatory. 
In  the  authors'  experience  reluctance  to  participate  in  discussion  may  be  greater  with  first  year  science 
students,  because  they  may  have  had  limited  previous  experience  in  this.  More  mature  students  in  DE  units 
are  often  willing  to  participate.  Few  guidelines  exist  on  how  skills  in  communicating  in  the  sciences  are 
best  developed  online,  but  it  certain  that  more  attention  needs  to  be  given  to  supporting  learning  online 
(McLoughlin,  Winnips  & Oliver,  2000).  Online  teaching  can  improve  the  level  of  conversation  beyond 
that  of  content  transmission  by  allowing  students  to  enter  into  dialogue  with  their  peers  and  lecturers.  This 
is  in  contrast  to  the  more  traditional  transmissive  style  of  pedagogy  often  encountered  by  on  campus 
students  in  lectures. 


Learning  complex  concepts 

Survey  quotes:  Methods  used  in  teaching  DE,  which  require  a more  careful  analysis  of  the  way  we  teach,  are 

useful  when  it  comes  to  teaching  on  campus. " 

"More  supplementary  resources  are  being  built  up  (e.g.  web  pages  in  science).  When  enough  of  our 
students  have  access  to  technologies  to  make  use  of  these,  the  materials  will  be  become  required." 

The  understanding  of  science  requires  an  ability  to  operate  and  move  between  three  levels,  the  microscopic, 
the  macroscopic  and  the  symbolic  (Johnstone,  1991).  Chemists  have  long  been  at  the  forefront  of 
developments  in  computer  visualisation,  because  of  their  need  to  visualise  molecules  and  chemical 
processes,  which  cannot  be  seen  directly.  There  are  many  examples  of  multimedia  materials  now  being 
produced  to  address  these  problems  (Tasker,  1998),  (Russell  et  al.,  1997).  Example  applications  readily 
accessible  on  the  Internet  include  Chime  molecular  visualisation  plug-in  program  (MDL,  2001)  and  a range 
of  science  Java  applets  modelling  chemical  processes  (Gargus,  1999).  CD  ROMs  and  CAL  packages  are 
also  now  available  from  many  of  the  large  publishing  houses  to  assist  students’  understanding  of  chemical 
concepts  (Kotz  & Vining,  1999). 

Numerous  attributes  of  multimedia  learning  materials  involving  computer  simulations,  which  are  beneficial 
to  learning,  have  been  identified  from  recent  literature  (Gaddis  & Anderson,  2000).  They  provide  feedback 
to  enhance  conceptual  understanding;  provide  dynamic  animations  to  emphasize  the  molecular  level  of 
chemical  reactions;  permit  students  to  generate  and  test  hypotheses;  engage  students  with  high  level  of 
interactivity;  present  a simplified  version  of  reality  by  distilling  abstract  concepts  into  their  most  important 
elements,  making  abstract  concepts  more  concrete;  standardize  instructional  pedagogy,  teaching,  and 
content  across  multiple  topic  areas.  Reports  in  the  literature  tend  to  highlight  the  technology,  but  the 
pedagogical  benefits  of  computer  simulations  are  the  ingredients  that  make  them  such  a valuable  learning 
tool.  "It  is  the  pedagogy,  not  the  technology  that  counts"  (Cottman,  1997).  Participation  in  collaborative 
assignments  can  increase  student  motivation,  which  then  leads  to  increases  in  both  active  participation  and 
the  quality  of  learning.  According  to  Hilz  et  al  (Hiltz,  Coppola,  Rotter,  & Turoff,  2000)"  when  students  are 
actively  involved  in  collaborative  (group)  learning  on-line,  the  outcomes  can  be  as  good  or  better  than  those 
of  traditional  classes,  but  when  individuals  are  simply  receiving  posted  material  and  sending  back 
individual  work,  the  results  are  poorer  than  in  the  traditional  classrooms." 


Experiential  learning 


Survey  quotes:  use  computer  lab  simulations  (digitized  stop-action  video)  to  prepare  students  for  in-lab  work 

....the  simulation  students  did  slightly  better  on  the  in-lab  quiz  which  tests  for  practical  in- lab  knowledge. " 

Providing  a convenient  laboratory  experience  of  sufficient  quality  is  one  of  the  major  difficulties  to  be 
faced  in  teaching  practical  science  subjects  by  distance  education.  This  is  a major  reason  for  the  reluctance 
of  science  educators  to  provide  DE  courses  of  study.  In  science  some  use  has  been  made  of  microscale  or 
full-scale  kits,  which  allow  experiments  to  be  done  in  the  home  (Kennepohl,  1996)  (Bennett,  1994) 
(Druskovich  et  al.,  1996).  Efforts  have  been  made  to  produce  low  cost,  low  maintenance  equipment 
(Hodges,  Pillekers,  & Hodges,  1991).  In  practice,  students  usually  either  attend  short  residential  schools 
for  concentrated  practical  experience  or  attend  designated  labs  over  the  semester. 

ICT  can  play  a role  in  laboratory  work  through  programs  to  collect  and  analyse  data  and  increasingly 
through  computer  control  of  instruments,  but  at  first  year  bvel,  this  is  not  yet  very  important.  More 
significantly,  with  respect  to  teaching  and  learning,  a range  of  simulations  of  simple  experiments  and 
instrument  simulator  software  are  now  available  on  CD  ROM  and  on-line  and  may  provide  a pedagogically 
sound  DE  laboratory  component  (Cartwright,  2000).  The  benefits  of  computer  simulations  are  numerous: 
they  can  replace  experiments  that  use  hazardous  materials;  reduce  cost;  replace  experiments  that  occur  too 
quickly  or  too  slowly  to  be  done  in  a regular  laboratory  period;  reduce  cognitive  noise,  so  that  students  can 
concentrate  on  the  concepts  involved  in  the  experiments;  allow  rapid  data  collection;  serve  as  pre- 
laboratory  preparation  to  aid  understanding  of  the  lab.  "In  fact,  computer  simulations  may  provide  a more 
cognitively  meaningful  laboratory  experience  than  an  on-campus  laboratory  class"  (Gaddis,  2000). 

Remote  controlled  labs  are  starting  to  become  available  on-line,  for  example  CNC  machines  in  engineering 
(Wong,  Ferguson,  Florance,  Bantwal,  & Jones,  1999)  and  the  Scanning  Probe  Microscope.  (IN-VSEE, 
2000)  The  availability,  variety  and  sophistication  of  such  labs  is  expected  to  increase  rapidly  in  the  next 
few  years  and  will  bring  the  lab  experience  of  DE  students  much  closer  to  that  of  on-campus  students. 
Such  on-line  labs  will  in  the  future  provide  the  opportunity  to  perform  experiments  which  have  never 
before  been  possible  for  first  year  students  at  all.  The  distinct  benefits  of  computer  mediated  collaborative 
learning  tasks  have  been  well  documented  (Saltiel,  1998).  As  an  example  in  science,  a virtual  organic 
science  lab,  where  on  campus  students  worked  collaboratively,  produced  significantly  increased 
understanding  in  students.  (Gaddis  & Anderson,  2000) 


Self-monitoring  and  metacognitive  skills 

In  science  subjects  there  is  a great  emphasis  on  content  to  the  point  that  often  this  leads  to  surface 
approaches  to  learning  as  a coping  skill  by  students.  Science  teachers  may  feel  little  inclination  to  include 
more  generic  skills  as  a part  of  their  teaching.  Studies  show  that  interventions  which  improve  strategy  use, 
develop  metacognitive  awareness  and  skills  and  increase  student  skills  of  reflection  can  be  very  beneficial 
to  students  (Pressley  & Wharton-McDonald,  1997).  It  is  time  to  give  more  consideration  to  these  factors 
as  on-line  learning  resources  are  developed.  As  an  example  of  an  innovative  ICT  project,  an  on-line 
tutorial,  metAHEAD,  develop  metacognitive  awareness  and  to  strengthen  the  metacognitive  skills  of  first 
year  science  students.  It  provides  students  with  ample  opportunity  for  personal  reflection  and  for  reflective 
social  discourse  on  their  metacognitive  and  problem  solving  skills  (Hollingworth  & McLoughlin,  2001). 


Table  2 summarises  the  current  use  and  possible  future  directions  for  ICT  in  science  education.  It  is  evident 
that  multimedia  resources  offer  opportunities  far  superior  to  print  materials  for  visualisation  of  chemical 
concepts.  They  can  provide  opportunities  to  explore  chemical  concepts  and  processes  in  more  depth,  to 
repeat  simulations,  to  change  variables  in  different  scenarios,  all  leading  to  deeper  understanding.  Such 
materials  have  usually  been  professionally  designed,  often  at  very  high  cost  by  teams  of  professionals  with 
a breadth  of  skills. 


Summary 


Technique 

Current  Use 

Future  Directions 

CD  ROM,  multimedia 
materials 

Widespread  use.  Publishing 
company  packages  with  texts 
provide  ease  of  distribution. 
Limitation  is  student  access  to 
computer  facilities. 

Will  increase.  More  attention  needs  to  be  given  to 
pedagogical  aspects  of  the  use  of  materials  for  the 
most  effective  learning. 

Computer  programs  for 
concepts,  theory 

In  general  use.  Some  problems  in 
distribution  of  individual 
applications  to  students. 

Increasing  numbers  of  applications  of  greater 
sophistication  will  be  available  to  all  on  Internet, 

Computer  programs  for 
laboratory 

Limited  usage.  Pre-labs,  data 
gathering  & analysis,  some  CAL 
sessions. 

Major  educational  benefits  in  development  of 
simulated  labs,  on-line  labs,  collaborative  labs  at  a 
distance 

On-line  unit 

Some  availability.  Limitation  is 
student  access  to  computer 
facilities. 

Increasing  availability  and  range  of  learning 
resources  and  increasing  student  access. 

On-line  collaborative 
tasks 

Assessment  still  presents  a barrier 
to  change 

Essential  area  to  increase  for  improved  quality  of  DE 
experience. 

Other  web  based 
activities 

Some  use,  eg  library  & research 
projects 

Need  to  increase  student  information  gathering  skills 
in  Internet  era. 

email  listserver 

In  general  use  mainly  as  a 
communication  tool. 

More  use  as  an  additional  learning  tool 

Bulletin  board 

Some  use,  not  compulsory. 

More  use  as  a collaborative  learning  tool 

Telephone  conferencing. 
Voicemail 

Some  use. 

Will  be  overtaken  by  applications  on  Internet 

AudioWideo  cassettes 

In  general  use 

Will  probably  phase  out  as  Internet  bandwidth 
increases 

As  many  have  pointed  out  implementation  of  ICT  in  teaching  and  learning  does  not  usually  involve  cost 
savings.  In  these  times  of  financial  stringencies  it  must  also  be  noted  that,  to  assure  quality  online  courses, 
selection,  support  and  development  of  faculty  teaching  skills  are  critical  (Sener  & Stover,  2000), 
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Abstract:  A great  deal  of  research  has  been  conducted  on  the  roles  of  online 
moderators  and  on  support  structures  needed  for  successful  online  collaboration  and 
productive  learning.  While  some  research  indicates  the  need  for  highly  structured 
environments  with  facilitation  and  tutor  support,  this  case  study  suggests  that  by 
utilising  well  designed  inquiry  oriented  tasks,  students  learn  to  self-direct  their  own 
activities.  In  the  study  presented  here,  the  assessment  of  the  task  increased  student 
motivation  and  task  based  interactions  indicated  that  supportive  feedback  and  social 
exchange  was  a characteristic  feature  of  successful  task  engagement. 


Introduction 

On-line  forums  provide  opportunity  and  potential  for  collaborative  work,  dialogue  and  study  that  can 
increase  the  flexibility  of  learning  while  motivating  participants.  By  enabling  teacher-learner  and  learner- 
learner  dialogue  computer-based  course  systems  can  support  the  essential  elements  of  a learning 
conversation  by  providing  scope  for  discussion,  interaction,  dialogue  and  reflection  (Laurillard,  1995). 
Clearly,  the  opportunities  for  learner-centered  activity,  flexible  access  and  asynchronous  discussion  creates 
new  opportunities  for  adult  learners  and  scope  for  self-direction  and  inquiry  based  learning. 

Instructional  designers  and  teachers  alike  must  confront  an  envision  the  need  for  learner-centered 
pedagogies  and  provide  opportunities  for  learners  to  have  control  over  their  own  learning  (Doherty,  1998). 
According  to  Hannafin  & Land  (1997)  open-ended  learning  environments  provide  opportunities  for  self- 
regulated  learning,  collaboration  learner  driven  inquiry  and  exploration.  Instead  of  providing  direct 
instruction,  the  teacher  will  foster  expectations  for  increased  learner  control  and 

knowledge  generation.  This  study  takes  as  its  starting  point  the  need  for  self-directed  learning  and  provides 
an  example  of  how  a task  focused  environment  online  provides  opportunities  for  learner  self-direction. 

Study  Context 

Computers  in  Teaching  and  Learning  is  a one  semester  unit  that  forms  part  of  the  Bachelor  of  Education 
and  Master  of  Education  programmes  at  the  University  of  New  England.  It  is  offered  online  via  the 
Distance  Education  mode  using  WebCT  software.  In  this  unit,  students  explore  the  use  and  integration  into 
classrooms  of  a variety  of  software  tools  such  as  word  processors,  spreadsheets,  databases,  drawing  and 
paint  programs,  multimedia  and  the  Internet.  To  help  achieve  the  objectives  of  the  unit  a number  of 
compulsory  learning  activities  and  associated  assessment  items  were  designed  around  the  online  tools 
provided  by  the  WebCT  environment.  These  online  activities  contributed  80%  of  the  final  grade  for  the 
unit.  The  remaining  20%  was  allocated  to  a paper  based  assessment  task. 
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Table  1 provides  a description  of  the  online  assessment  tasks  and  their  associated  WebCT  tools. 


Assessment  Task 

Online  tool 

Weighting 

Introductory  posting 

Bulletin  Board 

5% 

Online  summaries  & 
discussions 

Bulletin  Board 

5% 

Group  WebQuest 

Bulletin  Board,  Assignment  drop  box 

20% 

Individual  WebQuest 

Student  homepage 

30% 

Table  1:  Summary  of  Online  Assessment  Tasks 

Assessment  tasks  were  designed  to  provide  students  with  a range  of  online  experiences  from  interacting  on 
bulletin  boards  to  working  collaboratively  online.  These  tasks  also  modelled  for  students  the  potential  of 
the  online  environment  as  learning  and  teaching  tool.  As  a consequence,  the  experience  of  completing  the 
assessment  tasks  helped  achieve  the  unit’s  objectives  as  much  as  the  completed  tasks. 

Research  Questions 

The  purpose  of  this  study  was  to  determine  that  to  what  degree  students  would  moderate  themselves  if 
provided  with  a task  with  very  little  organisational  structure.  Berge  (1995)  has  identified  four  roles  of  e- 
moderators  and  classified  them  as:  Pedagogical,  Social,  Managerial  and  Technical.  These  roles  as  used  as 
the  basis  of  classification  for  the  current  study.  In  particular  the  following  questions  were  to  be 
investigated: 

If  a task  is  structured  to  encourage  self  moderation  is  there  evidence  of  any  of  these  roles  being 
dominant  over  the  others? 

Would  any  increased  responsibility  required  for  self  moderation  interfere  with  the  successful 
completion  of  the  task? 

Is  there  any  educational  advantage  for  structuring  a task  to  encourage  self  moderation? 


Group  WebQuest  Activity 


The  assessment  task  forming  the  focus  of  this  study  was  a group  WebQuest  activity. 

Dodge  (1995)  describes  a WebQuest  as  “an  inquiry -oriented  activity  in  which  some  or  all  of  the 
information  that  learners  interact  with  comes  from  resources  on  the  Internet”.  The  fundamental  idea  behind 
WebQuests  is  that  they  are  designed  to  prevent  students  from  aimlessly  surfing  on  the  Internet.  A well 
designed  WebQuest  will  help  give  students  structure  and  guidance  as  they  use  the  Internet  as  a tool  to 
support  their  own  knowledge  construction.  For  the  group  WebQuest  activity,  students  were  expected  to 
complete  a WebQuest  about  WebQuests  Figure  1 . shows  the  introductory  page  for  the  WebQuest  activity. 
This  task  involved  students  being  placed  in  groups  of  four  (except  one  group  which  was  reduced  to  two 
members  shortly  before  the  task  started)  and  each  adopting  one  of  four  possible  and  evaluating  the 
WebQuest  using  the  criteria  associated  with  each  particular  role. 


These  roles  were: 
Efficiency  expert: 
The  Affiliator  : 
The  Altitudinist: 
The  Technophile: 


Does  the  WebQuest  keep  students  on  task? 

Does  the  WebQuest  encourage  collaboration? 

Does  the  WebQuest  encourage  higher  order  thinking? 

Does  the  WebQuest  make  the  best  possible  use  of  the  available  technology? 


The  individual  component  of  this  assessment  task  involved  evaluations  being  done  by  the  students  within 
their  selected  roles.  The  collaborative  activity  involved  students  negotiating  with  each  other  via  the 
bulletin  board  tool  to  answer  the  following  questions: 


1.  Which  two  of  the  example  WebQuests  listed  were  the  best?  Why? 
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2.  Which  two  were  the  worst?  Why? 

3 What  criteria  were  used  to  base  the  group  decisions  upon? 

The  lecturer  gave  no  guidance  or  direct  assistance  on  how  to  complete  this  activity.  Students  were 
expected  to  organise  themselves  in  order  to  successfully  complete  the  task.  This  include  setting  up 
deadlines  for  individual  submissions,  providing  advice,  feedback  and  encouragement  to  each  other  and 
developing  selection  criteria  to  identify  the  best  and  worst  WebQuests. 

Once  completed,  students  had  to  submit  their  tasks  via  e-mail.  This  submission  included  each  student’s 
individual  evaluations  and  the  collective  evaluations  from  the  group.  Marking  was  based  upon  a 
combination  of  the  individual  and  collaborative  activities.  The  educational  objective  of  this  task  was  to 
provide  students  with  the  opportunity  to  observe  how  the  concept  of  a WebQuest  worked  from  a 
participant’s  point  of  view  while  at  the  same  time,  gain  some  exposure  as  to  what  constituted  an  effective 
WebQuest.  Figure  1 shows  a screen  dump  of  the  activity. 


I Netscape:  A WebQuest  about  WebQuestsi 
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A WebQuest  About  WebQuests 


Introduction 

Since  early  in  1995,  teachers  everywhere  have  learned  how  to  use  the  web  well  by  adopting  the 
WebQuest  format  to  created  inquiry-oriented  lessons.  But  what  exactly  is  a WebQuest?  What  does  it  feel 
like  to  do  one?  How  do  you  know  a good  one  when  you  see  It?  With  the  other  members  of  your  group 
you're  going  to  grapple  with  these  questions  and  more. 


Task 

To  develop  great  WebQuests,  you  need  to  develop  a thorough  understanding  of  the  different 
possibilities  open  to  you  as  you  create  web-based  lessons.  One  way  for  you  to  get  there  is  to  critically 
analyze  a number  of  webquest  examples  and  discuss  them  from  multiple  perspectives.  That's  your  task 
in  this  exercise. 

By  the  end  of  this  lesson,  you  and  your  group  will  answer  these  questions: 


1.  Which  two  of  the  example  WebQuests  listed  below  are  the  hast  ones?  Why? 

2.  Which  two  are  the  worst  ? Why? 

3.  What  do  hast  and  worst  mean  to  you? 

p— , ..  ^ ■ ■ ■ ■■ 


I 


Methodology 

Description  of  Coding  System 

Berge  (1995)  has  identified  four  broad  roles  that  e-moderators  play  in  an  online  environment.  These  are: 
pedagogical,  social,  managerial  and  technical.  Bulletin  Board  posts  from  students  in  the  Group  WebQuest 
assessment  task  were  analysed  and  classified  according  to  these  categories.  Decisions  were  based  on  the 
following  criteria: 
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Pedagogical : Posts  were  classified  as  pedagogical  if  the  majority  of  the  post  was  directly  related 
to  the  assessment  task. 

Social:  These  posts  involved  interaction  primarily  at  the  social  level  for  example:  “How  is  the 
weather  where  you  are?” 

Managerial : Posts  that  contained  organisational  information  for  instance  setting  up  posting  times 
or  deadlines. 

Technical:  Posts  that  contained  advice  at  the  hardware  or  software  level.  An  example  would  be 
instructions  on  how  to  upload  an  attached  file. 

Once  coded,  the  number  of  each  type  and  the  number  of  words  per  post  was  recorded.  From  this  data  the 
average  words  per  post  was  calculated. 


Results 

Table  2 shows  the  number  of  messages  classified  by  role  and  the  percentage  of  each  to  the  total  number  of 
messages  posted  (n  = 306).  The  results  indicate  that  the  largest  number  of  posts  to  the  WebQuest  forum 
were  managerial  in  nature.  Posts  centred  on  discussion  of  the  actual  activity  (Pedagogical)  were  the  second 
highest,  followed  by  social  and  technical  respectively. 


Type  of  Role 

Number  of  posts 

Percentage  of  posts 

Pedagogical 

89 

29% 

Social 

86 

28% 

Managerial 

95 

28% 

Technical 

36 

12% 

Total 

306 

100% 

Table  2 Posts  Classified  According  to  Roles 

Table  3 provides  the  total  number  of  words  posted  for  each  category  and  the  average  number  of  words  per 
post.  Posts  classified  as  Pedagogical  were  on  the  average  the  longest  messages  (average  = 96  words  per 
post).  Post  classified  as  Managerial  and  Technical  followed  next,  with  posts  classified  as  social  in  nature 
being  the  shortest  messages  on  average. 


Type  of  Role 

Number  of  posts 

Percentage  of  total 

Pedagogical 

8535 

96 

Social 

1611 

19 

Managerial 

3618 

19 

Technical 

1188 

33 

Total 

149952 

100 

Table  3:  Total  Words  and  Average  Words  per  Post 


Discussion 

Due  to  the  activity  being  largely  student  directed,  it  is  probably  not  surprising  that  the  highest  number  of 
posts  were  classified  as  managerial  in  nature  (n  = 95,  31%  of  total).  Students  had  to  spend  time  as  a group 
organising  themselves  in  order  to  complete  the  collaborative  activities.  However  in  terms  of  actual  word 
length,  managerial  orientated  posts  did  not  dominate  the  discussions  with  the  average  length  of  each  post 
being  38  words.  Students  were  able  to  set  up  an  organisational  structure  that  did  not  appear  to  interfere 
with  the  educational  objectives  of  the  task.  In  fact,  as  a consequence  of  organising  themselves,  students 
may  have  been  more  motivated  towards  the  task  they  had  to  undertake. 
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Posts  classified  as  being  pedagogical  in  nature  were  the  second  most  extensive  type  of  post  in  numerical 
terms.  However,  the  average  length  of  these  posts  far  exceeded  any  other  category  (96  words  per  post  on 
average).  This  suggests  students  were  deeply  engaged  in  the  process  of  collaborating  with  each  other  in  the 
construction  of  new  knowledge. 

Implications  of  the  study 

Despite  a large  percentage  of  social  posts  (n  = 86,  28%  of  total),  these  were  relatively  short  (19  words  on 
average)  and  consisted  of  mainly  short  social  exchanges  or  pleasantries.  Nipper  (1989)  expressed  concern 
that  the  process  of  social  interaction  online  may  overshadow  its  educational  focus.  However,  in  this 
instance,  this  appears  not  to  be  the  case.  These  social  exchanges  in  the  group  WebQuest  activity  appeared 
to  form  a “social  glue”  helping  support  group  cohesiveness.  Berge  (1995:22)  believes  these  types  of 
exchanges  are  necessary  for  the  creation  of  an  environment  that  will  “promote  high  quality  and  individual 
and  social  interaction  for  the  student  during  the  construction  of  knowledge”. 

There  were  relatively  few  posts  that  were  technical  in  nature  (n  = 36,  12%  of  total).  The  unit  had  been  in 
progress  for  several  weeks  before  this  activity  so  it  appears  that  students  had  been  given  the  opportunity  of 
becoming  familiar  with  using  the  WebCT  software.  The  exchanges  were  also  relatively  short  in  nature  (33 
words  per  post  on  average)  so  it  is  unlikely  that  technical  aspects  had  any  major  impact  upon  the 
collaborative  activities  and  the  shared  construction  of  knowledge  amongst  students. 


Conclusion 

According  to  (Ragoonaden  & Bordeleau  2000:368) 

the  success  of  collaboration  via  the  internet  lies  in  the  design  of  virtual 
courses  and  in  the  embedded  pedagogical  model.  The  collaborative  process 
has  to  be  an  integral  part  of  the  course  framework  and  activities  based  upon 
constructivist  approach.  In  this  manner,  students  are  compelled  to 
communicate  and  to  work  with  distant  partners  in  order  to  construct  new 
knowledge. 

Such  was  the  case  for  the  WebQuest  activity.  The  deliberate  lack  of  lecturer  direction  forced  students  into 
having  to  take  on  many  of  the  multi-faceted  roles  of  e-moderators  individually.  It  can  be  argued,  that  as  a 
consequence  of  this  expanded  role,  students  were  more  than  simply  participants  but  rather  organisers  of  the 
activity.  This  led  to  deeper  levels  of  engagement  in  the  task  at  hand.  By  being  given  the  opportunity  of 
being  able  to  take  on  the  roles  generally  associated  with  e-moderators  actually  enhanced  the  students’ 
ability  to  communicate  and  work  with  others  on  the  successful  completion  of  the  group  WebQuest  activity. 
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Abstract:  Increasingly,  universities  are  being  asked  by  industry,  government  and  funding  authorities  to  help 
prepare  students  for  industry  with  appropriate  professional  and  life  long  learning  skills.  How  can  tertiary 
educators  design  and  implement  learning  activities  to  help  promote  the  development  of  these  skills?  In  this 
paper  we  investigate  a method  of  scaffolding  the  development  of  these  skills  through  a conceptual 
framework  based  on  self-regulated  learning.  This  framework  is  applied  to  a third  year  higher  education 
learning  environment,  and  supported  with  a variety  of  on-line  tools  to  scaffold  students  learning.  These 
include  an  on-line  validated  professional  skills  testing  instrument,  an  on-line  self  and  peer  assessment 
journals  which  are  consolidated  into  streamlined  reports  for  the  tutor,  and  on-line  student  contracts,  designed 
to  encourage  commitment  and  strategy  formation  early  in  the  semester.  Each  of  these  are  discussed,  and  also 
pre  and  post  testing  results  are  shown  for  a validated  testing  instrument,  given  to  students  to  identify  their 
strengths  and  weaknesses. 

Introduction 

With  the  rapid  growth  of  Interactive  technologies  and  on-line  learning,  there  is  a consistent  demand  for 
graduating  students  to  have  life  long  learning  skills  that  enable  learners’  to  continually  upgrade  their  skills  and 
knowledge  through  their  own  self-motivation  and  learning  skills  (Bennett,  Dunne,  & Carre,  1999;  Candy,  Crebert,  & 
O'Leary,  1994;  Bearing,  1997).  An  important  aspect  for  achieving  this  goal  is  to  help  students  take  more  responsibility 
for  managing  their  own  learning  by  helping  them  become  more  strategic  learners.  Biggs  (1999)  argues  that  there  are 
certain  limits  to  what  certain  students  can  achieve,  and  these  are  beyond  the  teacher’s  control.  However  many  claim  that 
such  skills  can  be  taught,  that  while  they  may  be  developmentally  based,  the  fostering  of  general  skills  still  requires 
proactive  involvement  and  strategy  forming  (Zimmerman,  1989).  The  challenge  for  educators  then  is  to  find  teaching 
and  learning  methods  that  can  support  the  generation  of  life  long  learning  skills  that  are  relevant  to  a wide  variety  of 
professional  contexts. 

This  paper  will  discuss  three  specific  techniques  that  are  used  both  to  assess  and  enhance  students’  abilities  to 
create  life  long  professional  and  learning  strategies.  These  strategies  are  framed  within  a theoretical  model  of  self- 
regulation that  identifies  the  psychological  states,  processes  and  outcomes  that  are  inherent  in  independent  learning.  The 
aim  of  this  study  was  to  implement  strategies  that  could  be  used  within  each  of  these  levels  to  support  the  development 
of  these  skills. 

A Deflnition  of  Self  Regulation 

Self-regulation  is  somewhat  easier  to  define  than  understand.  It  has  been  described  as  ‘the  process  whereby 
students  activate  and  sustain  cognitions,  behaviours,  and  affects,  which  are  systematically  oriented  toward  attainment  of 
their  goals’  (Schunk  & Zimmerman,  1994,  cited  by  Boekarts,  1997,  p.  171).  This  definition  is  reinforced  by  Brooks 
(1997),  who  argues  that  that  it  is  active  and  goal  directed,  resulting  from  self  control  of  behaviour  motivation  and 
cognition.  This  emphasis  on  multiple  constructs  places  Self-regulated  Learning  at  the  junction  of  several  fields  of 
research  (Boekaerts,  1997).  It  emphasises  students'  reliance  on  their  own  internal  resources  to  govern  their  learning,  but 
these  resources  are  not  easy  to  delineate.  Self  regulated  behaviour  is  an  end  process,  dependent  upon  the  affects  and 
cognitions  that  precede  it.  These  are  to  a certain  extent  inaccessible,  since  they  are  internally  constructed  and  not  always 
explicitly  articulated  by  individuals. 

Also,  the  notion  of  self-regulation  is  prone  to  multiple  interpretations  based  upon  educational  philosophy. 
Zimmerman  (1989)  identifies  it  in  terms  of  Phenomenological,  Social  Cognitive,  Volitional,  Vygotskian  and  Cognitive 
Constructivist  Theories.  All  of  these  approaches  bring  a unique  framework  to  the  concept.  Behaviourist  approaches 
emphasise  self-monitoring,  self-instruction  and  self-reinforcement,  while  a phenomenological  approach  defines  it  in 
dimensions  such  as  self  worth,  planning,  and  goal  setting.  Common  to  most  of  these  however,  is  an  acknowledgment  of 
the  interaction  of  affective  and  cognitive  processes  at  a level  of  abstraction.  Self  awareness  at  a cognitive  and  emotional 
level  appears  to  be  the  key  process  in  the  development  of  self-regulatory  strategies. 
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A model  of  Self  Regulation 

A number  of  models  have  been  developed  to  explain  the  processes  that  underpin  Self- Regulated  Learning. 
(Boekaerts,  1997)  provides  a six  component  model  based  upon  the  notions  relating  to  cognition,  metacognition,  content 
and  motivation. 

These  elements  are  co-dependent  and  interact  with  each  other  in  the  application  and  development  of  goals, 
strategies  and  domain -specific  knowledge.  Garcia  & Pintrich  (1994)  articulate  self-regulation  in  terms  of  knowledge 
and  beliefs,  strategies  used,  and  outcomes.  Each  of  these  is  moderated  by  motivational  and  cognitive  components  such 
as  personal  beliefs  and  conceptual  knowledge,  motivational  and  cognitive  strategies,  and  quantity  and  quality  of  effort. 
Common  to  both  models  is  an  integration  of  both  affective  and  cognitive  issues: 

Neither  motivational  nor  cognitive  models  alone  can  fully  describe  the  various  aspects  of  student 
academic  learning,  yet  the  two  types  of  models  are  complementary  due  to  the  respective  strengths  and 
weaknesses  of  motivational  and  cognitive  models  ^ (Garcia  & Pintrich,  1994,  p.  127) 

Figure  1 represents  a synthesis  of  the  above  frameworks.  It  accommodates  the  role  of  both  affective  and  cognitive 
aspects  of  self  regulation,  but  also  acknowledges  the  effects  of  external  environmental  factors  upon  an  individual's 
ability  to  regulate  their  learning.  Self  regulation  is  viewed  here  as  the  intersection  of  self  awareness  at  both  a rational 
and  emotional  level.  Metacognition  and  self  concept  are  seen  as  the  primary  enabling  process  in  this  model,  with  self 
monitoring  and  motivation  as  subordinate  processes  which  are  involved  in  the  development  of  cognitive  and 
motivational  strategies. 


Environment  & Context 

/ Cognitive 

Affective  \ 

/ Domain  / 

\ Domain  \ 

1 Metacognition  [ 

' Self-concept 

1 Self-monitoring 

7 Motivation  j 

V Strategy  \ y 

^ Volition  control  / 

\ formation  \y 

strateeies  / 

Figure  1 : Conceptual  Framework 

Enhancing  Cognitive  Self-Regulation  Through  Scaffolding 

There  is  a large  body  of  work  which  examines  ways  in  which  the  affective  components  of  self-regulation  can  be 
targeted  to  increase  students’  motivation  and  persistence  in  their  learning.  Emotional  factors  are  generally  seen  to  be 
more  accessible  and  amenable  to  change  than  the  cognitive  aspects.  In  fact,  the  two  are  not  unrelated.  Como  (1986),  for 
example,  argues  for  metacognition  as  the  dominant  controlling  process;  that  “affect  is  the  subjective  perception  of 
emotional  states;  thus  associated  attempts  to  control  negative  affect  fall  within  the  domain  of  metacognitive  control”  (p. 
334).  However,  the  focus  of  this  paper  is  on  the  specific  knowledge  that  students  have  about  their  abilities  and  the 
cognitive  strategies  they  use  rather  than  the  subjective  values  which  they  attribute  to  themselves  and  their  abilities. 

The  primary  enabling  state  for  cognitive  self-regulation.  Metacognition  is  a concept  that  this  fraught  with 
contention.  Some  have  argued  that  it  is  an  inherent  psychological  state  that  cannot  be  changed,  although  this  view  has 
come  under  increasing  criticism.  There  is  a growing  consensus  for  example,  that  Metacognition  is  only  mildly 
correlated  with  unalterable  measures  of  ability  such  as  IQ  (Schraw,  1986).  Recent  theorists  are  starting  to  examine  the 
construct  of  Metacognition  from  social  and  environmental  perspectives.  Rather  than  being  developmentally  fixed,  the 
acquisition  of  Metacognition  may  be  subject  to  instructional  intervention  (Boekaerts,  1997) 

This,  however,  doesn’t  solve  a major  problem,  which  is  how  to  design  instruction  that  will  actually  enhance 
students’  ability  to  regulate  their  learning.  Does  it  come  through  modelling  regulatory  strategies,  having  students 
monitor  their  performance,  or  by  attempting  to  raise  general  metacognitive  awareness?  This  paper  argues  a process  of 
scaffolding  that  may  be  implemented  at  each  of  the  three  cognitive  levels  of  Metacognition,  Self-Monitoring,  and 
Strategy  formation  (Figure  1).  At  the  broadest  level,  students  need  to  be  placed  in  the  mode  of  reflecting  on  their  own 
cognitive  abilities.  This  may  be  general  in  nature,  but  can  be  enhanced  through  the  use  of  self-monitoring.  Also, 
cognitive  strategies  for  regulation  need  to  be  enacted  in  a purposeful  and  structured  way.  This  is  not  going  to  happen 
automatically;  the  role  of  scaffolding  is  to  support  the  learner’s  transition  from  dependent  learner  to  self-regulating 
learner. 
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Scaffolds  are  forms  of  learning  support  provided  to  bridge  the  gap  between  existing  skills  and  potential  skills. 
Central  to  the  notion  of  scaffolding  is  Vygotsky’s  Zone  of  Proximal  Development  (Vygotsky,  1978),  which  can  be  seen 
as  space  between  the  level  of  actual  achievement  and  the  level  of  achievement  possible  with  assistance.  Just  as  physical 
scaffolding  provides  an  framework  during  early  phases  of  building,  instructional  scaffolds  act  as  initial  support  which  is 
gradually  removed  as  the  learner  becomes  more  independent.  Scaffolding  can  take  many  forms.  Winnips  & 

McLoughlan  (2001),  for  example,  propose  video  commentary,  providing  Web  links,  and  linking  to  good  examples  of 
student  work  among  others;  but  such  scaffolds  can  exist  within  the  following  broader  macro  strategies  proposed  by 
Hogan  & Pressley  (1997): 

• Pre -engagement 

• Establishing  a shared  goal 

• Actively  diagnosing  the  understandings  and  needs  of  learners 

• Providing  tailored  assistance 

• Maintaining  pursuit  of  a goal 

• Giving  feedback 

• Controlling  frustration  of  risk 

• Assisting  internalisation,  independence,  and  generalization  to  other  contexts 

While  many  of  these  strategies  are  targeting  the  affective  rather  than  rational  components  of  self-regulation  (for 
example  ‘controlling  frustration  and  risk’),  they  still  provide  a useful  heuristic  in  developing  scaffolds  to  support  the 
constructs  of  Metacognition,  Self-Monitoring,  and  Strategy  Development  shown  in  Figure  1. 

Scaffolds  for  Metacognition 

Metacognition  is  self  awareness  at  a global  level,  and  exists  independently  from  a specific  context.  Scaffolding 
for  Metacognition,  then,  must  be  done  at  a level  that  can  exist  outside  a particular  knowledge  domain.  This  may  exist  as 
explicit  instruction,  though  the  internalisation  of  such  abstract  concepts  may  be  somewhat  challenging.  In  this  case. 
Metacognition  was  stimulated  through  the  use  of  the  WORKING  assessment  instrument  (Miles  & Grummon,  1996). 

The  WORKING  inventory  is  a self-assessment  package  for  workplace  skills,  designed  to  help  students  understand  what 
skills  employers  require,  and  how  they  rate  against  these.  These  skills  are  generally  quite  different  to  the  regular 
academic  and  technical  competencies  students  are  normally  tested  for.  They  include  the  following  nine  skills:  taking 
responsibility,  working  in  teams,  persisting,  a sense  of  quality,  life-long  learning,  adapting  to  change,  problem  solving, 
information  processing  and  systems  thinking. 

The  inventory  does  not  just  provide  a tool  to  diagnose  the  understandings  and  needs  of  learners,  but  also  acts  as 
a means  of  giving  customised  feedback  to  the  learners  and  is  a vital  tool  to  assist  with  learners’  pre -engagement.  In 
stimulating  their  perceptions  of  their  own  working  abilities,  the  WORKING  inventory  works  both  to  assess  and  assist  in 
the  development  of  Metacognition.  The  instrument  was  used  in  semester  1,  2001  with  a class  of  ninety  final  year 
multimedia  tertiary  students  studying  project  management  methodology.  It  was  given  at  the  beginning  and  end  of 
semester  and  showed  an  overall  increase  of  1 3%  across  the  nine  scales. 


Being 

Responsible 

Team 

Skills 

Persisting 

A sense  of 
Quality 

Life  Long 
Learning 

Adapting  to 
Change 

Problem 

Solving 

Info 

Processing 

Systems 

Thinking 

Total 

Pre 

52 

49 

44 

48 

51 

56 

49 

52 

59 

170 

Post 

67 

69 

68 

64 

66 

65 

74 

77 

76 

192 

Table  1:  Pre/Post  WORKING  Results 


Feedback  was  given  to  each  student,  based  on  their  overall  scores.  For  example,  “Level  1 : Beginning  ABE 
Literacy”  is  explained  as:  “Has  few  or  no  effective  workplace  habits  and  skills;  likely  to  lack  a well-developed  sense  of 
flexibility,  persistence,  and  quality;  poor  at  problem-solving  and  learning.  May  have  a low  sense  of  responsibility  ”. 
Students  were  also  given  tailored  feedback  on  each  scale,  which  allowed  them  to  target  specific  weaknesses,  by  trying 
to  develop  strategies  to  enhance  these  skills  over  the  semester,  supported  with  tutor  help. 

Scaffolds  for  Self-Monitoring 

Weinstein  & Mayer  (1986)  argue  that  all  metacognitive  activ  ities  incorporate  to  some  extent  the  monitoring  of 
comprehension,  and  all  models  of  self-regulation,  from  behaviourist  to  social-cognitive  involve  self-monitoring  as  the 
core  metacognitive  process,  whether  described  in  terms  such  as  self-monitoring,  self-observation,  or  inner  speech 
(Zimmerman,  1989).  Self-monitoring  strategies  may  involve  tracking  of  attention  while  reading  a text  or  listening  to  a 
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lecture,  self-testing  through  the  use  of  questions  about  text  material  to  check  for  understanding,  monitoring 
comprehension  of  a lecture,  use  of  test  taking  strategies  (eg  timing  questions)  in  an  exam  situation  (Garcia,  1994).  Core 
to  all  of  these  however  is  the  evidence  of  a reflective  process  on  the  part  of  the  learner.  One  method  of  encouraging  self- 
monitoring is  to  use  the  strategy  of  journaling.  Journaling  has  been  identified  as  a useful  tool  to  increase  self-awareness 
(Brooks,  1997)  and  is  an  ideal  scaffold  in  that  it  assists  the  internalisation  of  new  understandings  as  well  as  the  setting 
of  goals  and  monitoring  of  their  achievement. 


I Student  1 | Student  2 H Student  3 1 StudentT 


Figure  2:  On-line  journal  for  weekly  peer  assessments 

In  this  study,  students  completed  self  and  peer  assessment  journals  each  week  through  an  on-line  application. 
They  completed  self-assessment  journals  in  which  personal  views  of  their  progress  were  recorded.  Here,  students 
considered  their  success  in  finishing  assigned  tasks  (scale  from  1 to  5),  the  quality  achieved  (scale  from  1 to  5),  how 
successful  they  had  been  in  managing  their  time  (scale  from  -2  to  +2),  and  comments  justifying  these  scores.  This  was 
available  for  peers  to  view  and  consider  before  peer  assessment  took  place,  which  was  based  on  agreed  tasks  for  that 
week  (Figure  2).  Students  confidentially  rated  their  peers  on  four  criteria:  attendance  at  meetings,  collaboration,  success 
in  completing  required  tasks  and  quality  of  delivered  tasks.  Tutors  used  this  confidential  information  to  discuss  progress 
and  make  decisions  about  transferring  marks  at  “tutor  led  peer  assessment  sessions”.  These  sessions  provided  a forum  in 
which  students  could  see  the  results  of  their  self  and  peer  assessments,  and  how  accurate  they  had  been  in  relation  to  the 
teams’  interpretation  and  the  tutor’s  summary. 

Scaffolds  for  Strategy  formation 

Cognitive  strategies  are,  in  effect,  the  outcome  of  Metacognition  and  the  self-monitoring  process.  These 
strategies  are  numerous,  and  include  approaches  for  rehearsal,  elaboration,  and  organization  (Weinstein  & Mayer, 

1986).  Certainly  self-regulating  students  use  techniques  such  as  paraphrasing,  highlighting  and  concept  mapping,  and 
these  are  skills  that  can  be  taught  through  direct  instruction.  Ultimately,  however,  the  implementation  of  such  strategies 
must  be  enacted  by  the  learner.  Assisting  learners  to  articulate  at  the  outset  the  strategies  that  they  will  be  implemented 
within  their  projects  provides  a strong  initial  scaffold,  as  it  allows  the  establishment  of  a goal,  and  the  plan  for 
maintaining  the  pursuit  of  that  goal.  The  teacher  can  use  this  plan  to  tailor  assistance,  and  the  plan  itself  acts  as  a 
milestone  against  which  learners  can  continue  to  assess  their  performance. 

As  a pre -engagement  tool  and  to  enhance  the  authenticity  of  their  projects,  learners  in  this  unit  were  required  to 
enter  into  a student  contract  (Figure  4).  The  contract  was  an  agreement  between  themselves,  their  team  members’,  and 
the  tutor.  Its  purpose  is  to  outline  each  student’s  responsibilities  such  as  team  role,  responsibilities,  topics  for  their 
portfolio,  amount  of  time  committed  to  achieve  these  tasks,  and  contributions  they  would  make  to  help  the  group 
achieve  its  goals.  This  is  completed  in  week  3,  with  a meeting  in  which  a “Team  Contract”  is  signed,  were  all  team 
members  agree  that  the  roles,  tasks  and  times  allocated  for  all  team  members’  are  acceptable.  This  helped  students 
commit  to  specific  responsibilities  in  order  to  create  a collaborative  environment  in  which  they  would  establish  shared 
goals  through  an  agreed  plan. 
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Figure  4:  Sample  Student  Contract 

Conclusion 

There  is  currently  much  debate  focused  on  creating  optimal  conditions  in  learning  environments  for  self- 
regulated  learning,  to  help  students  develop  as  independent  learners.  This  paper  has  provided  a conceptual  framework  to 
help  identify  important  elements  needed  for  self-regulation,  and  provided  scaffolds  to  support  Metacognition,  Self- 
Monitoring  and  Strategy  Formation.  However,  providing  appropriate  learning  environments  and  valid  assessment 
instruments  to  help  students’  develop  these  skills  is  not  an  eas  y task.  At  which  level  should  Self-Regulated  learning  be 
targeted?  Should  Metacognitive  strategies  be  targeted  before  Strategy  Formation,  or  should  Self-Monitoring  be 
practised  first?  We  suggest  that  all  of  these  cognitive  elements  need  to  be  considered  concurrently.  The  design  of  the 
learning  environment  must  integrate  appropriate  learning  activities  as  outlined  above,  to  scaffold  the  development  of 
these  skills. 

Students  must  actively  engage  with  authentic  learning  activities,  which  are  student-centred  but  scaffolded  to 
give  guidance  and  support.  In  the  above  examples,  the  WORKING  inventory  helped  interrogate  students’  professional 
skills,  and  provided  customised  feedback  on  how  to  improve  these,  as  well  as  providing  tutor  and  on-line  support.  At 
the  same  time,  students  were  continually  monitoring  self  and  peer  performance  through  an  on-line  application  that 
provided  regular  and  consolidated  feedback  on  their  progress  and  viewpoints.  These  activities  enabled  students  to 
regularly  gain  feedback  on  how  successful  their  strategies  had  been  as  outlined  in  their  contracts,  and  through  reflective 
practice  allowed  them  make  continual  improvement. 
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Abstract: 


The  paper  will  explore  the  environment  of  intense  change  that  characterises  most  Australian 
universities  (and  indeed  universities  in  much  of  the  world).  In  this  environment,  universities 
have  had  to  reassess  their  fundamental  business  and  the  way  they  go  about  it.  Information 
Technology  (IT)  is  viewed  as  an  important  factor  in  streamlining  their  operations  and  all 
universities  are  investing  heavily  in  systems  and  services.  If  this  implementation  is  not 
managed  in  a collegial  way  there  is  a very  real  danger  that  academic  teachers  will  resent  the 
changes  being  made  and  undermine  the  investment.  Polarity  theory  will  be  described  and  a 
model  will  be  developed  which  sho\\s  that  the  ‘zone  of  effective  change’  in  universities 
requires  that  several  dimensions  need  to  be  managed  at  the  same  time.  Examples  of  these 
dimensions  are: 

• top  down  versus  bottom  up  decision  making 

• management  versus  scholarship 

• systems  ver^w^services 

• central  devolved 

• focus  versus  variety 

• mass  change  versus  growing  individuals 

• competition  versus  collaboration 

It  will  be  argued  (with  examples)  that  changing  'versus"  to  'and  allows  a new  perspective  to 
be  added  to  university  implementation  of  policies  and  practice. 


The  Changing  Terrain  of  Higher  Education 

This  paper  is  based  on  two  sets  of  experiences;  the  first  is  the  ongoing  experience  of  working  in  academic  staff 
development  in  three  large  Australian  universities  over  the  last  ten  years,  and  the  second  is  directing  a recent 
national  study  about  the  uptake  of  technology  for  teaching  and  learning  across  the  Australian  higher  education 
sector. 

Universities  are  currently  in  an  environment  of  intense  change.  They  are  being  required  to  educate  more 
students,  from  an  increasing  variety  of  backgrounds,  with  decreasing  government  funding.  Universities  are 
required  to  compete  vigorously  for  student  enrolments  and  external  sources  of  funding.  In  this  environment, 
universities  have  had  to  reassess  their  fundamental  business  and  the  way  they  go  about  it.  Information 
Technology  (IT)  is  viewed  as  an  important  factor  in  streamlining  their  operations  and  all  universities  are 
investing  heavily  in  systems  and  services.  The  implementation  of  technology  covers  aspects  like  IT 
infrastructure  (networks,  standards,  hardware,  etc.);  online  learning  management  systems  (such  as  WebCT  or 
Blackboard);  academic  management  systems  (e.g.  PeopleSoft)  which  may  be  fully  integrated  with  the  learning 
management  systems  and  financial  systems;  digital  library  investments;  and  extensive  staff  development. 

Yetton  et  al.  (1997)  make  it  clear  that  communication  and  information  technologies  will  be  a major  part  of  future 
university  planning.  Yetton ’s  research  team  examined  20  universities’  management  of  IT.  They  noted  that  an 
organisation’s  performance  is  a finction  of  fit  among  five  factors:  strategy,  roles  and  skills,  management 
processes,  structure  and  technology.  They  noted  the  need  to  change  terms  and  conditions  of  employment  in 
universities  and  the  need  for  rationalisation.  “There  will  be  winners  and  loser^’  (p.  xiii).  In  this  paper  I will 
discuss  a few  ideas  about  how  universities  might  manage  internal  policy  development  and  their  approaches  to 
collaborative  ventures  in  order  maximise  their  chances  of  being  ‘winners’. 


o 
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Innovation  and  Change 


We  should  rot  underestimate  the  difficulties  involved  in  innovation  and  change.  Marris  (1986)  parallels  the 
sense  of  loss  during  bereavement  to  the  resistance  one  can  feel  when  letting  go  of  known  ways  of  doing  things 
and  embarking  on  new  strategies.  For  many  academics  the  increasing  emphasis  on  the  use  of  computer 
technology  for  administration,  research  and  teaching  is  highly  threatening.  We  need  to  recognise  these  fears  and 
devise  plans  that  build  staff  confidence  and  motivation,  and  provide  adequate  support  and  training  onportiinities. 
Changing  educational  practices  and  styles  can  produce  many  negative  reactions  and  this  negativity  needs  to  be 
acknowledged  and  managed  effectively.  Change  should  be  introduced  and  implemented  within  a supportive 
environment.  The  culture  of  the  organisation  needs  to  be  able  to  embrace  change  while  offering  staff 
opportunities  to  manage  their  own  levels  of  comfort  with  the  change. 

Conventional  texts  on  management  often  define  organizations  as  groups  of  people  united  by  a common  goal,  but 
our  common  experience  (and  our  common  sense)  would  tell  us  that  organizations  are  only  rarely  so  united  and 
so  rational.  Ford  et  al.  (1996)  acknowledge  the  complexity  of  modem  universities  and  provide  descriptions  of 
various  perspectives  of  people  who  w)rk  in  and  with  higher  education.  However,  some  of  the  passion  and 
trauma  of  change  seems  to  have  been  sanitised  out  of  this  book  and  others  like  it.  The  authors  operate  on  an 
evolutionary  approach — “the  principle  that  diversity  and  change  are  forces  to  be  managed  and  harnessed  rather 
than  resisted”  (p.  2).  The  strength  of  individual  innovation  seems  stifled  and  yet  such  innovation  is  one  catalyst 
for  change. 

It  is  important  to  consider  how  innovation  leads  change,  and  how  inappropriate  management  strategies  can  either 
stifle  the  innovation  or  cause  a long  time  delay  between  the  innovators  developing  and  evaluating  their  work, 
and  the  results  moving  into  other  areas  of  the  organisation. 

Argyris  (1991)  discusses  how  relatively  simple  models  of  problem-solving  such  as  action  learning  do  not  go  far 
enough  because  they  focus  on  “identifying  and  correcting  errors  in  the  external  environment”  (p.  99).  He 
suggests  that  the  way  problems  are  perceived  and  defined  needs  attention  as  well. 

Senge  (1990)  argues  that  the  modem  organisation  needs  not  only  knowledge  at  all  operational  levels,  but  also  the 
capacity  to  learn.  His  model  of  a ‘learning  organisation’  draws  on  the  work  of  Argyris  and  Schon  (1978)  on 
single  and  double  loop  learning;  single  loop  learning  is  simple  problem  solving;  double  loop  learning  looks  at 
fundamental  organisational  stmcture  and  embeds  individuals’  discoveries,  inventions  and  evaluations  in 
organisational  memory  (Argyris  & Schon,  1978,  p.  19).  For  Senge,  teams  are  the  learning  agents  that  can 
achieve  double  loop  learning  and  translate  individual  work  into  new  theories  of  action  for  the  organisation.  This, 
at  least,  is  closer  to  what  my  experience  of  effective  change  has  beea  Senge  suggests  a range  of  strategies — 
personal  mastery,  building  mental  models  and  shared  visions,  and  forming  learning  teams. 


Polarity  Theory 

Instead  of  thinking  of  a range  of  problems,  issues  or  options  that  needs  to  be  considered  during  the 
implementation  of  an  innovation,  Johnson  (1992)  suggests  that  it  is  more  realistic  to  consider  a series  of 
polarities.  He  claims  that  polarities  are  sets  of  opposites  which  can’t  function  well  independently.  Because  the 
two  sides  of  a polarity  are  interdependent,  it  is  not  possible  to  choose  one  as  a solution  and  neglect  the  other.  The 
aim  of  polarity  management  is  to  get  the  best  of  both  opposites  while  avoiding  the  limits  of  each.  The  solution 
resides  within  the  tension  between  polarities.  For  example,  we  can  view  collaboration  and  competition  as  being 
at  two  poles;  they  are  not  mutually  exclusive,  but  rather  both  need  to  be  accommodated  in  our  strategic  planning. 
Polarity  theory  does  not  offer  defined  solutbns  to  organisational  problems.  It  emphasises  that  change  is  a messy 
and  dynamic  situation;  as  the  appropriate  balance  point  for  one  set  of  polarities  shifts,  this  will  influence  others. 
Fig.  1 shows  a series  of  polarities,  intersecting  in  what  I have  called  the  ‘zone  of  effective  change’.  I will  make  a 
few  comments  on  and/or  provide  illustrations  for  each  of  these  polarities  in  turn;  no  priority  is  indicated  by  the 
order  of  discussion. 


Figure  1:  Effective  change  as  the  management  of  multiple  dimensions  or  polarities 


Top  down  vs  Bottom  up 

Large  scale  ‘social  re-engineering’  from  the  top  can  get  a process  going  but  it  is  too  difficult  to  sustain  unless 
genuine  commitment  develops  locally.  Over  the  past  few  years,  RMIT  University  in  Australia  has  made  a 
substantial  investment  into  the  use  of  online  learning  technology , aimed  at  improving  the  quality  of  programs  by 
reviewing  their  educational  design  and  adding  greater  flexibility  to  modes  of  studying  them.  This  initiative, 
called  the  IT  Alignment  Program  (ITAP),  has  involved  investment  in  infrastructure,  enterprise  computer-based 
systems  which  are  IMS-compliant  (http://www.imsproject.org/),  Library  resources,  staff  development,  and 
program  and  course  renewal  (http://www.lts.rmit.edu.au/renewal/).  RMIT  has  tried  to  move  on  several  fronts  at 
once,  and  many  of  the  policies  and  processes  that  have  been  developed  are  still  being  refined  (Kenny  & 
McNaught,  2000).  While  the  original  vision  for  how  technology  might  enhance  the  processes  and  procedures  for 
the  University  came  from  the  top  (Fallshaw,  2000),  the  success  of  the  implementation  will  be  determined  by  the 
take-up  in  faculties  and  departments.  Kenny  (2001)  explores  this  tension  in  the  area  of  courseware  development 
at  RMIT.  Based  on  several  years  experience  of  working  with  RMIT  staff,  he  proposes  a model  to  combine  the 
benefits  of  project  management  for  ensuring  quality  educational  outcomes  and  a return  on  investment  to  the 
institution,  with  scholarship  and  ownership  within  the  project  teams  to  ensure  individual  commitment  and 
satisfaction. 


Management  vs  Scholarship 

Bowden  and  Marton  (1998)  depict  the  21®^  century  university  as  the  University  of  Learning.  All  aspects  of 
university  work — teaching,  research  and  community  involvement — are  forms  of  learning  for  the  individual,  the 
collective  scholars  at  a discipline  level,  and  for  the  local  (and  increashgly  global)  society.  All  these  types  of 
learning  involve  growth  and  change,  often  in  unpredictable  directions.  Yet,  universities  are  large  organizations 
and  need  to  have  some  defined  parameters.  Willets  (1996)  has  written  a humorous  article,  entitled  ‘The  best 
ways  to  survive  reengineering.  Expert  tips  on  how  to  reinvent  your  attitude’.  But  there  is  a great  deal  of  irony 
beneath  his  li^  advice.  The  sense  of  powerlessness  that  organizational  change  can  often  engender  is  very  real. 
One  strategy  for  universities  is  to  keep  the  nature  of  the  business  which  is  scholarship  at  the  forefront  of 
management  meetings.  ‘Will  this  process  enhance  scholarship?’  is  the  question  that  must  constantly  be  debated. 
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Systems  vs  Services 

Quality  assurance  systems  are  a good  way  to  illustrate  this  tension.  Evidence  must  be  scholarly,  not  just  an  audit 
trail  If  information  about  the  quality  of  academic  work  does  not  feed  back  into  quality  improvement,  it  is  hard  to 
justify  the  resources  used  t o collect  the  information.  Quality  assurance  systems  must  also  provide  service  to  the 
organisation.  One  example  is  the  IS09000  system  fittp://www.iso9000,org/).  It  has  125,000  registered  and 
certified  organizations  who  use,  and  pay  for,  its  methodology.  This  may  be  fine  for  several  types  of  businesses 
but  universities  need  to  question  how  applicable  it  is  for  higher  education.  “Despite  more  than  a decade  and  three 
evolutions  of  IS09000,  the  model  has  made  little  significant  impact  on  higher  education.  This  is  not  surprising 
because,  despite  tinkering  with  it,  IS09000  was  never  fundamentally  designed  for  an  educational  context” 
(Harvey,  2001,  p.  59), 


Central  vs  Devolved 

There  are  many  tensions  in  the  central/  devolved  debate.  One  major  issue  is  whether  university  funding  for 
courseware  design  and  production  should  be  through  central  or  faculty-based  processes.  Faculty  staff  want  the 
skills  and  expertise  that  exists  in  central  units,  but  wish  to  have  it  provided  without  reduction  in  funding  to 
faculties.  The  requirement  to  pay  for  services  from  central  units  can  set  up  resistances.  Most  universities  use  both 
approaches  (McNaught,  Phillips,  Rossiter  & Winn,  2000).  It  is  finding  the  appropriate  balance  point  that  is  the 
challenge.  Tab,  1 summarises  the  arguments  for  and  issues  associated  with  each  approach. 


Centralised  funding 

Devolved  funding 

Points  in  favour  of: 

Can  reduce  duplication  of  expensive  services  by 
funding  a range  of  projects,  the  design  ideas  and 
products  of  which  can  be  used  in  other  faculties. 
Can  foster  cross- faculty  collaboration  and 
communication. 

Can  fund  projects  based  on  local  knowledge  of 
curricula  and  faculty  culture. 

Can  develop  stable  ongoing  teams  for  future 
developments. 

Can  allow  university  strategic  priorities  to  be 
enacted. 

Can  allow  local  ownership  and  commitment  to  grow. 

Can  foster  the  integration  of  outside  funding  with 
university  priorities. 

Can  source  funding  from  discipline  and  industry- 
related  bodies. 

Issues  associated  with: 

If  the  funding  committee  is  not  broadly  constituted, 
this  can  result  in  a restricted  range  of  models  being 
favoured. 

Traditional  practices  in  the  discipline  can  dominate, 
and  it  may  be  difficult  for  some  innovative  projects 
to  be  funded. 

Can  be  dominated  by  a few  strong  university 
personalities;  this  may  disadvantage  certain  faculties. 

Can  be  dominated  by  a few  strong  faculty 
personalities;  this  may  disadvantage  certain 
departments/  schools. 

Table  1:  Pros  and  cons  for  centralised  and  devolved  funding  (McNaught  et  al.,  2000,  p.  113) 

Focus  vs  Variety 

Coordination  is  the  key  here.  If  administrative  or  academic  support  units  have  too  many  functions,  their  energy 
can  be  dissipated.  If  they  have  too  narrow  a focus,  then  the  result  may  well  be  isolated  units  who  do  not  see  ‘the 
big  picture’.  Clearly  articulated  functions  and  effective  coordination  between  units  is  the  answer. 

An  example  of  how  diverse  the  range  of  services  offered  by  university  units  is  given  in  this  list  I made  at  a 
national  meeting  of  20  Directors  of  Staff  Development  of  Australian  universities  in  May  2001.  Each  of  these 
items  was  supported  by  several  of  the  universities  present;  minor  or  unusual  services  are  not  noted.  This  list 
refers  to  teaching  and  learning  support.  This  list  is  in  addition  to  the  courseware  production  aspect  of  several 
units  which  are  part  of  large  flexible  learning  centres  . 

• Graduate  Certificate  in  Higher  Education  programs; 

• series  of  workshops  and  seminars  across  the  university; 

• academic  induction  programs; 

• activities  based  at  local  faculty  or  departmert/  school  level; 

• student  evaluation  service; 


• support  for  sessional  staff,  both  onshore  and  offshore; 

• post-graduate  supervisor  training; 

• support  for  heads  of  departments/  schools  linked  to  development  of  local  policy; 

• support  for  graduate  capabilities,  especially  for  programs  leaders/  coordinators;  and 

• liaison  withleaming  skills  and  library  staff  in  course  renewal  teams. 

So,  academic  staff  development  units  are  in  danger  of  having  too  much  variety.  While  some  liaison  with  student 
support  and  library  staff  was  noted,  the  focus  of  discussion  remained  on  coping  with  the  variety  of  internal 
activities  rather  than  on  coordination  with  other  units  within  each  university. 


Mass  Change  vs  Growing  Individuals 

One  example  to  illustrate  how  individual  growth  can  (bvetail  with  overall  strategic  directions  is  a recent 
academic  staff  development  program  run  at  RMIT  University.  A Learning  Technology  Mentor  (LTM)  program 
at  RMIT  ran  from  mid-1999  to  the  early  part  of  2001.  The  LTM  program  provided  for  120+  academic  staff  to 
have  one  day  per  week  time  release  over  one  semester,  in  order  to: 

• learn  how  to  use  the  University’s  recently  established  online  education  system, 

• design  and  implement  online  learning  in  their  faculty’s  education  programs,  and 

• promote  and  support  similar  activities  among  colleagues  in  their  departments. 

The  aim  of  making  a significant  investment  in  learning  technology  mentoring  by  academic  staff— rather  than 
establishing  a specialist  online  design  and  production  unit  to  service  them,  for  example — ^was  to  achieve 
widespread  adoption  of  online  learning  as  part  of  effecting  a change  in  the  culture  of  academic  work.  Extended 
time  release  of  more  than  one  semester  was  required  to  achieve  useful  outcomes  in  some  cases;  these  academic 
staff  were  called  Experienced  Learning  Technology  Mentors  (ELTMs).  Over  this  time,  in  several  faculties,  a 
network  of  individuals  developed  that  remained  after  the  formal  end  of  the  program  ^cNaught,  2001;  Gray  & 
McNaught,  2001 ). 


Competition  vs  Collaboration 

There  are  benefits  in  both  perspectives.  Fig.  2 summarises  the  drivers  for  both  collaboration  and  competition.  If 
the  aim  is  to  produce  high  quality  educational  offerings  there  are  drivers  for  both  collaboration  and  competition 
relating  to  quality  standards,  financial  viability  and  raised  prestige. 


Policy-driven 

Mechanisms  for  quality 

Collaboration 

competition  and 

control  and  effective 

through  official  and 

marketing  strategies 

dissemination 

unofficial  alliances 

Quality  standards  based  on : 
student  demand; 
industry  needs 

Quality  standards  based  on: 
combined  intellectual  input;  broader 
contextual  experiences 

Financial  viability  based  on: 
developing  potential  for  new  markets 
for  courses 

Financial  viability  based  on : 
efficiencies  in  resource  terms; 
sharing  of  expenses 

Raised  prestige  based  on: 
creating  a niche,  a particular  cultural 
identity  in  the  marketplace 

Raised  prestige  based  on: 
pooled  reputation  of  all  partners; 
possible  increased  publication  output 

Figure  2:  Summary  of  drivers  for  both  competition  and  collaboration 
(after  McNaught  etal,  2000,  p.  160) 

Summary 

One  simple  summary  of  polarity  theory  and  its  applicability  to  higher  education  today  is  the  reframing  of  the  set 
of  dimensions  I chose  as  follows: 
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top  down  and  bottom  up  decision  making 
management  and  scholarship 
systems  Art  (/services 
central  devolved 
focus  and  variety 

mass  change  and  growing  individuals 
competition  and  collaboration 
Remove  all  oppositional  ‘versus’  thinking  and  replace  it  with  ways  to 
embracing  both  ends  of  poles.  The  zone  of  cffcctivx  change  can  only 
of  both  ends  of  each  dimension. 


consider  how  to  gain  maximum 
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benefit  by 
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Abstract 

liiliouuclory  college  chemistry  can  be  *'taught“  by  a mode  of  instruction  other  than  lecture  and 
careful  use  of  the  web  can  alter  the  way  chemistry  is  learned.  The  web  provides  a means  to  deliver 
educational  resources  via  a universally  accessible  platform.  The  medium  is  relatively  easy  to  develop 
materials  for  and  provides  a level  of  interaction  that  cannot  be  provided  in  a textbook.  The  interactive 
nature  can  be  used  to  deliver  simulations  of  chemical  and  physical  processes  (animations  of  phenomena) 
or  interactive  simulations  of  experiments  that  may  (or  may  not)  be  done  in  the  laboratory  portions  of  a 
general  chemistry  course.  This  paper  discusses  a variety  of  the  interactive  materials  that  have  been  tested 
in  the  author’s  first  semester  general  chemistry  course.  The  simulations  presently  involve  material  from 
approximately  three  quarters  of  the  course.  The  interactive  web-based  activities  serve  as  the  starting  point 
for  small  group  classroom  discussion  and  assignments.  It  is  important  to  make  the  use  of  simulations  an 
integral  part  of  the  course,  provide  a motivation  for  individual  students  to  perform  the  simulations,  and  to 
provide  a setting  in  which  the  students  can  exchange  ideas  and  insights.  The  positive  aspects  of  the 
learning  tools  discussed  in  the  paper  will  be  highlighted.  However,  the  web  cannot  solve  all  of  the 
challenges  of  teaching  chemistry.  Therefore,  brief  discussion  of  the  difficulties  encountered  in  using  web- 
based  simulations  for  small-group  assignments  will  also  be  discussed. 

Introduction 

When  deciding  upon  making  changes  to  any  course,  there  are  several  questions  that  instructors 
typically  consider.  First,  "What  works  in  the  present  course  and  are  there  'problems'  that  need  to  be 
addressed?"  Second,  "Who  are  the  students  in  the  course?"  and  "What  is  the  nature  of  the  institution?". 
The  answer  to  these  questions  will  suggest  what  ought  to  be  changed  and  help  the  instructor  decide  what 
changes  best  fit  the  needs  of  the  students  and  the  institution.  A final  question  that  is  key  to  the  success  of 
any  changes  in  the  course  is  "Who  is  the  instructor  and  what  role  are  they  comfortable  with  in  the 
classroom?". 

The  following  conditions  apply  at  the  author’s  institution.  Wilkes  University  is  a private,  co-ed, 
comprehensive  institution  founded  in  1933.  Wilkes  serves  approximately  2,000  full-time  undergraduate 
students  from  20  states  and  15  foreign  countries  and  it  has  a School  of  Pharmacy.  The  freshman  chemistry 
courses  serve  approximately  175  students  in  the  fall  semester.  The  author's  course  is  the  four  credit 
"Elements  and  Compounds"  course  (offered  in  the  fall  semester)  that  meets  for  four  50  minute  periods  per 
week  plus  one  “three  hour”  lab.  The  students  enrolled  in  this  course  are  nearly  all  traditional  age  students, 
over  95%  are  first  semester  freshman,  and  are  nearly  all  required  to  take  the  course  as  part  of  their  major. 
Most  are  pursuing  a bachelor  degree  in  engineering  or  science  (but  few  are  chemistry  majors).  The  largest 
majors  represented  in  the  class  are  pre-Pharmacy,  Pre-med,  biology,  and  engineering  majors.  The  class 
size  ranges  from  45  to  nearly  85  at  the  start  of  the  semester  (this  year  there  were  three  sections  offered  at 
different  times).  The  on-line  simulations  were  originally  developed  and  implemented  at  the  author’s 
previous  institution.  The  Pennsylvania  State  University  - Hazleton  Campus.  There  were  two  problems  the 
author  wanted  address:  (a)  students  were  not  spending  adequate  time  engaging  the  course  material  and  (b) 
some  topics  covered  in  the  lecture  portion  of  the  course  either  had  too  little,  or  no,  coverage  in  lab. 

Aside  from  the  adjustments  to  the  "normal"  structure  of  their  lives  that  traditional  age  freshman 
must  make  when  they  enter  college,  there  are  a number  of  factors  that  make  the  first  year  college 
chemistry  course  challenging: 

• the  quantity  of  the  material  covered 

• the  variety  of  material  covered 


• the  requirement  to  perform  calculations  (especially  solving  "word  problems") 

• the  requirement  to  relate  the  abstract  to  the  concrete 

• the  misconceptions,  alternate  conceptions,(l)  from  prior  courses  or  life  experience 

Traditional  lecture  and  recitation  sections  do  not  capture  the  interest  of  students  and  suffer  from 
other  difficulties:  (a)  little  time  for  remediation  of  basic  skills,  (b)  ineffective  for  some  learning  styles,  (c) 
students  cannot  control  time,  place,  or  pace,  and  (d)  interaction  with  the  instructor  is  avoided  (especially 
in  medium  to  large  classes).  As  a result,  students  do  not  actively  engage  the  course  material  nor  do  they 
develop  the  depth  of  understanding  that  is  desirable.  For  some  students,  learning  and  grades  suffer 
because  interest  in  the  course  is  low.  A recent  study  also  indicates  that  a general  chemophobia  may 
contribute  to  lower  achievement  in  chemistry.(2) 

Some  of  these  difficulties  can  be  addressed  by  using  modes  of  instruction  other  than  traditional 
lecture.  As  many  instructors  know,  a major  key  to  improving  student  learning  is  getting  the  students  to 
spend  time  engaging  the  course  material.  It  is  in  this  area  that  the  web  can  be  useful  as  a way  to  "teach" 
chemistry.  The  web  provides  a universally  accessible  platform  for  the  delivery  of  interactive  educational 
resources.  The  medium  is  relatively  easy  to  develop  materials  for  and  provides  a level  of  interaction  that 
cannot  be  duplicated  by  a textbook.  Additionally,  students  can  control  (to  some  extent)  the  place,  time, 
and  pace  of  interaction  with  the  material.  The  interactive  nature  can  be  used  to  deliver  simulations  of 
chemical  and  physical  processes  (animations  of  phenomena)  or  interactive  simulations  of  experiments 
that  may  (or  may  not)  be  done  in  the  laboratory  portions  of  a general  chemistry  course.  A variety  of 
interactive  materials  have  been  developed  and  tested  in  the  first  semester  of  the  author's  general  chemistry 
course.  Attempts  (both  successful  and  not  so  successful)  to  implement  web-based  course  materials, 
especially  small  group  use  of  simulations,  will  be  described. 

Interactive  Web  Based  Simulations 

The  author  has  developed  a series  of  interactive  simulations  for  use  in  the  first  semester  of  a 
college  general  chemistry  course.  Students  are  exposed  to  assignments  that  require  active  participation 
and  these  simulation  assignments  serve  as  the  starting  point  for  small  group  discussions.  The  activities 
require  students  to  "perform  experiments",  collect  and  analyze  data,  and  arrive  at  conclusions  about  the 
behavior  of  matter.  This  is  more  consistent  with  the  practice  of  science  than  the  traditional  lecture  mode 
of  teaching.  This  allows  the  students  to: 

• become  involved  in  the  process  of  learning, 

• develop  a sense  for  the  process  of  doing  science, 

• and  learn  to  think  more  independently  relying  less  on  a textbook  for  answers  and  more  on  their 
own  ability  to  understand  and  correlate  data. 

The  simulations  (found  at  http://www.hn.psu.edu/faculty/dmencer/chemsims.htm,  the  author’s  old  web 
site)  address  areas  of  content  typically  found  in  the  first  year  college  curriculum. 

• density  of  solids  and  liquids-  perform  density  determinations  on  liquids  and  solids 

• mass  laws  - investigate  mass  changes  in  reactions  to  "discover"  (a)  mass  conservation,  (b)  law  of 
definite  proportions,  and  (c)  law  of  multiple  proportions 

• compound  stoichiometry  - determine  empirical  formulas  using  a simulation  of  combustion 
analysis 

• reaction  stoichiometry  - investigates  mass  changes  in  reactions  to  "discover"  (a)  limiting  reactant 
and  (b)  theoretical  yield 

• ideal  gases  - investigate  relationships  between  pressure,  volume,  temperature,  and  number  of 
moles  of  an  ideal  gas 

• investigations  of  bonding  - the  effect  of  electronegativity  difference  on  bond  strength 
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• calorimetry  - simulations  of  both  a bomb  calorimeter  and  a coffee  cup  calorimeter 

• heat  and  work  - investigate  relationships  between  heat  and  work  in  both  constant  pressure  and 
constant  volume  systems 

Other  activities  (without  simulations)  explore  periodic  properties,  the  atomic  mass  scale  and  the  mole, 
atomic  structure,  and  aspects  of  intermolecular  forces. 

Small  Group  Work 

Small  group  work  was  employed  because  research  has  shown  that  cognitive  learning  has  an 
important  social  aspect.(3)  Small  group  use  of  the  interactive  web-based  activities  has  been  attempted 
using  two  methods  in  the  author's  course.  In  the  first  and  second  year  of  use  (at  PSU-Hazleton),  students 
were  clustered  into  groups  of  three  to  five  students  and  assigned  a simulated  experiment  to  complete.  A 
worksheet  was  available  to  guide  them  through  the  exercise.  They  had  to  analyze  the  data  and  determine 
the  significance  of  the  results.  Based  upon  their  interpretation  of  the  data,  the  group  then  developed  a 
lesson  for  the  rest  of  the  class.  Prior  to  teaching  these  lessons,  the  groups  met  with  the  instructor  as  a 
quality  control  check  for  the  lesson  plan.  Students  received  an  individual  and  a group  grade  for  their 
efforts. 

The  "Ideal  Gases"  simulation  was  used  by  four  groups  to  investigate  simple  gas  laws  as  shown  in 
Table  1 . Students  worked  as  a group  to  collect  and  analyze  the  data.  They  were  not  told  the  name  of  the 
gas  law  they  were  to  investigate  prior  to  the  meeting  with  the  instructor.  Each  group  collected  several  sets 
of  the  requested  data.  Each  data  set  was  collected  using  different  values  for  the  constant  variables.  For 
example,  in  the  Boyle's  Law  case,  P vs.  V data  would  be  collected  with  one  set  of  n and  T values,  then 
again  with  a new  set  of  n and  T values,  and  finally  a third  time  with  one  more  set  of  n and  T values.  The 
worksheet  prompted  the  students  to  determine  whether  the  relationship  between  the  two  variables  was 
direct  or  indirect  (inverse)  and  to  prepare  a linear  graph  of  the  relationship.  Students  were  also  asked  to 
write  the  relationship  for  their  data  as  a proportionality.  Students  could  then  solve  initial  state/final  state 
problems  using  the  simple  gas  law  they  have  "discovered".  Students  were  also  prompted  to  calculate  ^^/nx 
for  every  data  point.  The  four  groups  presented  their  finding  in  class  and  the  instructor  would  use  this  as  a 
starting  point  for  discussion  of  the  combined  gas  law  and  the  ideal  gas  law. 


TABLE  1.  ASSIGNMENTS  FOR  "IDEAL  GASES"  SIMULATION 

Group 

Gas  Law  Assigned 

General  Directions 

1 

Boyle's  Law 

Vary  V and  measure  P while  holding  n and  T constant. 

2 

Charles'  Law 

Vary  T and  measure  V while  holding  n and  P constant. 

3 

Amonton's  Law 

Vary  T and  measure  P while  holding  n and  V constant. 

4 

Avogadro's  Law 

Vary  n and  measure  V while  holding  P and  T constant. 

This  small  group  method  was  not  as  successful  the  author  had  hoped.  Students  in  the  small 
groups  learned  the  material  well:  they  completed  the  assignment,  developed  a lesson  plan,  met  with  the 
instructor,  taught  the  material,  and  fielded  questions.  This  process  increased  the  group  members'  success 
with  the  material.  However,  students  who  were  not  assigned  to  teach  the  material  did  not  respond  well  to 
peer  teaching:  they  did  not  use  the  simulations,  they  "turned  off  during  in-class  peer  teaching,  and  they 
did  not  develop  good  questions  for  the  presenters.  This  method  of  using  the  simulations  with  small  group 
peer  teaching  was  also  time  consuming  for  the  instructor.  Assignments  were  developed  and  coordinated 
the  for  all  of  the  groups  (approximately  20)  with  each  group  requiring  a minimum  of  a single  half-hour 
meeting  to  review  their  lesson  plan.  Some  groups  needed  a follow-up  meeting.  The  same  method  was 
repeated  for  a second  year  with  the  hope  that  it  could  be  made  to  work  better.  Questions  based  on  the 
presentations  were  included  on  quizzes  and  exams  to  try  to  increase  student  involvement.  However, 
student  reaction  to  peer  teaching  was  still  poor.  Other  instructors  might  be  able  to  employ  this  method  of 
peer  teaching  successfully,  however,  the  author  tried  an  alternative  method. 
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During  the  fall  1999  and  2000  (at  PSU)  and  Fall  2001  (at  Wilkes  University),  the  author 
committed  to  getting  all  students  involved  in  every  activity.  To  achieve  this  goal,  the  class  is  divided  into 
either  three  or  four  large  groups.  Each  group  is  given  a different  investigation  to  perform  using  a 
simulation.  In  the  case  of  the  "Ideal  Gases"  example,  four  groups  are  employed  and  the  assignments  are 
the  same  as  those  listed  in  Table  1.  Each  student  is  required  to  complete  his  or  her  group’s  assignment 
working  alone  outside  of  class.  Every  student  now  has  to  access  the  web  pages  and  use  the  simulation.  On 
the  assigned  day,  all  students  are  required  to  bring  the  results  of  their  investigations  (compiled 
data/completed  worksheets)  to  class.  The  students  then  split  into  small  groups  comprised  of  one  member 
from  each  large  group  (in  the  "Ideal  Gases"  example  each  small  group  would  have  members  from  groups 
1,  2,  3,  and  4).  The  students  share  what  they  learned  within  their  small  group.  Since  each  of  the  large 
groups  have  a different  assignment,  every  student  makes  a unique  contribution.  Also,  since  each  student 
completes  a worksheet  and  graph  for  the  assignment,  they  can  more  readily  interpret  data  presented  by 
others. 

In  order  to  provide  external  motivation  to  complete  the  assignments,  the  instructor  can  randomly 
collect  completed  data/worksheets  from  students.  Students  needn’t  know  in  advance  whose  assignment 
will  be  collected.  Make-up  assignments  can  be  given,  or  not,  and  the  assignment  grades  can  be  factored 
into  the  overall  course  grade.  This  method  proved  to  be  successful  in  increasing  student  involvement  with 
the  assignments.  The  logistics  are  somewhat  simplified  since  the  meetings  with  individual  groups  are 
eliminated.  However,  there  are  written  assignments  to  grade,  and  some  students  hand  in  poorly  completed 
worksheets  or  none  at  all. 

Assessment  of  Effectiveness  of  Web-based  Activities 

Student  surveys,  and  the  author's  personal  assessment,  have  been  used  to  assess  the  use  of 
simulations  and  other  on-line  materials.  Surveys  assessed: 

• reaction  to  the  use  of  simulations  (along  with  questions  about  the  other  on-line  materials) 

• interest  in  and/or  attitude  toward  learning  science/chemistry 

• attitude  toward  science/chemistry  as  a process  (vs.  a set  of  facts  and  equations) 

• self-assessment  of  their  level  of  ability  to  carry  out  scientific  data  collection  and  analysis 

The  results  of  these  surveys  indicate  a positive  assessment  of  the  simulations  and  other  web-based 
materials  (for  example,  on-line  course  notes).  After  completing  the  course  with  the  simulations  student 
self-evaluation  also  indicated  improvement  in  the  other  three  areas.  However,  data  from  a control  group 
(a  section  of  the  course  taught  without  the  simulations)  are  not  complete  at  the  time  of  paper  submission. 

The  author  has  also  examined  other  means  of  assessing  the  new  course  materials.  An  interesting 
trend  was  observed  in  student  course  evaluations  at  Penn  State  (Student  Ratings  of  Teaching 
Effectiveness,  SRTEs).  The  students'  overall  evaluation  of  both  the  course  and  the  instructor  declined  for 
the  first  two  years  that  on-line  course  materials  and  simulations  were  employed  (fall  semesters  1997  and 
1998).  Several  colleagues  (in  a number  of  different  disciplines)  at  the  same  PSU  campus  have  also 
noticed  a decrease  in  course  evaluation  upon  adoption  of  "nontraditional  components"  to  their  courses. 
The  two  most  recent  semesters  for  which  data  is  available  (fall  1999  and  2000)  showed  a reversal  of  this 
trend  with  evaluations  climbing  back  to  the  level  from  the  fall  of  1996.  Course  retention  rates  have  been 
evaluated  from  late  drop  data  and  have  been  compared  to  data  from  previous  years  of  the  author's  course 
(1996  to  present).  Retention  rates  in  the  author's  course  have  been  declining  steadily  when  referenced  to 
his  first  year  at  PSU,  1996  (the  only  year  with  no  on-line  component).  Class  grade  point  average  has  also 
declined  during  the  same  period.  Exams  comparable  to  previous  years  were  given  and  student  averages  on 
each  quiz  and  exam  have  been  compared  to  the  same  quiz  or  exam  in  the  previous  years.  However,  these 
comparisons  do  not  account  for  differences  from  one  semester  to  another  in  other  variables  linked  to 
student  success  rates  (average  preparation  level,  motivation  level,  and  other  class  demographics).  The 
reasons  for  these  trends  may  be  related  to  the  teaching  methods  used  in  this  course,  or  they  may  reflect 
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trends  at  the  national  or  institutional  level.  In  order  to  place  the  trends  observed  in  the  author's  course  into 
context,  more  data  is  needed  (for  example,  average  math  and  chemistry  placement  scores  could  be 
compared). 

Improved  performance  of  the  "learning"  taking  place  in  the  course  is  much  trickier  to  assess.  The 
author  has  anecdotal  experiences  that  indicate  use  of  the  simulations  and  small  groups  can  help  students 
overcome  misconceptions,  better  understand  concepts  and  phenomena,  and  better  learn  "problem 
solving"  skills  (vs.  using  rote  algorithmic  methods  to  solve  problems).  For  example,  many  students  have 
deeply  rooted  misconceptions  about  density,  displacement,  and  buoyancy.  The  density  determination 
simulations  serve  as  a good  starting  point  to  discuss  phenomena  related  to  density  (for  example 
Archimedes  Principle).  The  simulation  of  solid  density  simulation  relies  on  displacement  for  the  volume 
determination.  During  the  in-class  discussions,  the  instructor  carefully  reviews  the  requirements  for  a 
solvent  that  can  be  used  to  determine  the  volume  of  a water  reactive  low  density  solid  (like  sodium,  Na). 
The  discussions  invariably  turn  to  other  "real  world  examples"  of  displacement  and  buoyancy  (for 
example  the  class  has  discussed  ice  cubes  floating  in  liquid  water  and  hot-air  balloons).  Students 
completing  the  simulated  density  experiments  and  the  small  group  work  appear  to  have  a better 
understanding  of  the  phenomena  of  displacement. 

A second  example  involves  mass  conservation  and  the  concept  of  limiting  reactants.  When 
performing  mass  conservation  simulations  some  students  are  surprised  to  observe  that  excess  reactant  can 
remain  after  a reaction  goes  to  100  percent  completion.  Class  discussion  time  is  spent  in  careful 
examination  of  atomic  theory  to  see  how  it  can  explain  the  Law  of  Conservation  of  Mass,  the  Law  of 
Constant  Composition,  and  the  Law  of  Multiple  Proportions.  Even  after  this  effort  some  students  are  still 
confused  when  the  concept  of  limiting  reactant  is  introduced  in  the  following  chapter  in  the  context  of 
mole  calculations.  The  simulations  are  employed  again,  but  this  time  students  are  expected  to  work  in 
moles  instead  of  in  mass.  Although  the  average  quiz  score  on  this  chapter  has  not  improved,  the  author 
has  noticed  a number  of  students  employing  mass  conservation  concepts  (although  sometimes 
incorrectly),  mass  ratios  of  elements  in  compounds,  and  other  "interesting  options"  when  solving 
stoichiometry  problems.  Although  these  methods  are  not  always  employed  properly,  it  seems  that  some 
students  are  doing  more  than  simple  algorithmic  problem  solving  on  these  questions;  that  is  they  seem  to 
be  thinking.  The  use  of  the  simulations  along  with  the  small  group  work  allows  students  to:  (a)  engage  the 
material  actively,  (b)  develop  ideas/relationships,  (c)  test  them,  and  (c)  learn  on  their  own  and  from  other 
students.  In-class  examples  and  instructor  led  discussion  are  used  to  supplement  the  group  work  as 
needed. 

One  final  difficulty  with  the  assignments  is  the  vast  range  of  prior  knowledge  of  the  students  in 
the  course.  The  "best"  students  found  some  of  the  assignments  trivial,  while  other  students  found  some 
assignments  (or  perhaps  just  the  instructions)  confusing.  It  is  clear  from  this  work,  and  from  other  studies 
(ref.  4 and  references  within),  that  computer-based  assignments  can  improve  student  performance  and 
attitude  toward  chemistry.  However,  it  is  important  that  the  level  of  the  student  and  the  assignment  be 
well  matched.  It  is  also  important  to  recognize  that  improved  student  performance  on  exams  may  not 
reflect  increased  under standing{\).  Also,  gains  on  exam  scores  may  not  be  attributable  to  the  particular 
type  of  assignment  or  the  mode  of  delivery.  A recent  study  of  a web-based  system  of  homework  and 
testing  showed  a strong  positive  effect  on  exams  scores.  (5)  One  finding  was  that  scores  on  practice 
exams  increased  with  the  number  of  times  they  were  attempted.  This  may  simply  be  a function  of  the  time 
spent  engaging  the  material.  This  needn’t  negate  the  positive  aspects  of  computer-based  assignments;  but 
similar  gains  might  be  achieved  by  other  assignments  that  increase  "time  on  task". 

Individual  and  In-Class  Use 

The  simulations  can  be  used  to  provide  practice  problems  or  to  provide  a starting  point  for  in- 
class  discussion.  Students  can  attempt  the  problems  and  then  check  their  answers  against  the  simulation's 
result.  The  author  also  uses  them  to  develop  problems  during  office  hours.  The  author  plans  to  add  more 
"drill  and  practice"  types  of  exercises  to  the  course  site.  These  exercises  are  intended  for  students  needing 
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skill  development  or  remedial  work.  These  types  of  exercises  should  also  help  to  address  the  increasing 
number  of  students  who  use  their  calculators  poorly. 

Conclusion 

The  web  is  an  effective  medium  for  delivering  interactive  course  materials.  Interactive  tutorials 
and  simulated  experiments  provide  the  means  alter  the  way  chemistry  is  taught.  Although  it  is  not  the 
final  solution  to  all  of  the  challenges  of  teaching  college  chemistry,  it  does  have  some  striking  strengths. 
It  is  imperative  to  make  the  use  of  simulations  an  integral  part  of  the  course,  to  provide  motivation  for 
individual  students  to  perform  the  simulations,  and  to  provide  a setting  in  which  the  students  can 
exchange  ideas  and  insights.  If  small  groups  are  used,  it  is  critical  to  take  steps  to  minimize  the  difficulties 
created  by  group  members  that  do  not  "do  their  parts".  Also,  students  must  be  given  time  and  guidance  to 
successfully  adjust  to  teaching  and  being  taught  by  themselves  and/or  their  peers. 
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Resources 

In  addition  to  the  links  found  on  various  pages  of  my  website  (http://course.wilkes.edu/mencer),  I also 
recommend  the  following. 

1 . A more  detailed  version  of  the  small  group  methods  discussed  in  this  paper,  and  some  additional 
on-line  materials,  can  be  found  on-line  at  CONFCHEM:  Conferences  on  Chemistry 
http://www.ched-ccce.Org/confchem/2000/b/mencer/mencer.htm 

2.  More  excellent  papers  on  innovative  teaching  methods  in  chemistry  can  also  be  found  at  the 
CONFCHEM:  Conferences  on  Chemistry  archive  site  http://www.ched-ccce.org/confchem/past.html 

3.  A nice  “Resource  for  Students  and  Teachers”  (http://www.hwscience.com/smarsden/)  is 
maintained  Steve  Marsden.  Follow  the  link  to  topics  and  click  on  a topic  (for  example  "Gases"). 
There  are  many  links  to  tutorials,  notes,  and  simulations  (the  links  are  footnoted  with 
bibliographic  information). 
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Choosing  an  Online  Learning  Platform  focusing  on 
Reusability  of  Learning  Objects  and  its  Implications  for 
Comparison  Schemata  Design 


Abstract:  Project  decisions  about  suitability  of  existing  online  learning  platforms  are  often  discussed  by  using 
project  specific  proprietary  comparison  schemata  or  via  a comparison  service  offered  in  the  world  wide  web.  As 
educational  material  in  form  of  learning  objects  becomes  more  available  focusing  on  content  reusability  aspects 
is  becoming  more  important  for  online  learning  platforms  due  to  cost  minimization  in  content  production  and 
preparation  for  content  reusal.  Exactly  these  reusability  aspects  can  hardly  be  found  in  these  comparison 
schemata.  In  learning  platforms  learning  objects  are  stored  in  database  management  systems  by  using  document 
management  systems.  Modularization  of  courses  and  course  building  elements  as  objects  are  essential  for 
reusability.  For  effective  administration,  storage,  search,  and  retrieval  of  these  elements  as  a part  of  a reusability 
process  metadata  descriptions  of  the  processed  learning  objects  are  neccessary  and  fundamental.  Our  approach 
to  enrich  comparison  schemata  bf  adding  reusability  specific  elements  fills  the  existing  gap  between  metadata  and 
reusability  features  implemented  in  available  online  learning  platforms  and  popular  used  comparative  analysis 
tools  for  online  learning  platforms,  respectively. 

Keywords:  Reuse,  Learning  Object,  LOM,  Metadata,  Online  Learning,  Comparative  Analysis 

1 Introduction 

This  paper  investigates  which  criterions  for  comparative  analysis  of  online  learning  platforms  are  considered  to  be 
relevant  for  decisions  focusing  on  content  reusability  aspects  of  these  platforms  when  using  currently  available  and 
often  reviewed  information  systems  like  [LanOl],  [EduOl].  The  term  online  learning  platform  in  this  scope  is 
defined  by  webbased  applications  to  deliver  or  present  educational  multimedia  course  materials  on  a learner’s  side 
and  to  offer  administration  features  on  a tutor  side  through  management  of  course  relevant  metadata  while 
authoring  support  is  offered  by  content  management  and  content  creation.  Due  to  the  lack  of  reusability  specific 
categories  in  these  information  systems  a set  of  already  existing  criterions  is  collected.  Referring  to  this 
investigation  this  paper  describes  an  approach  for  handling  a criterion  enrichment  by  adding  reusability  specific 
categories  to  comparative  analysis  tools. 

Therefore  the  term  reusability  is  described  as  it  is  understood  in  the  context  of  the  content  of  a learning  platform 
as  an  object  in  section  2.  As  a result  of  this  description  a metadata  schemata  for  learning  objects  named  Learning 
Object  Metadata  (LOM)  which  is  proposed  by  the  Learning  Technology  Standards  Committee  (LTSC)  of  the  IEEE 
[LWGOl]  is  introduced  which  is  especially  useful  for  the  attribution  of  learning  resources  [HFM+01].  After 
mentioning  use  cases  working  on  LOM  and  currently  not  solved  granularity  problems  in  LOM,  section  3 handles 
the  reusability  aspects  of  learning  objects  implemented  in  currently  available  learning  platforms.  Afterwards  this 
paper  is  going  on  discussing  included  reusability  specific  criterions  in  the  comparison  schemata  of  [Bat99], 
[EduOl]  and  [Land].  The  following  section  4 states  a proposal  of  a hierarchically  structured  set  of  criterions  with 
specific  categories  and  elements  to  allow  comparative  analysis  among  online  learning  platforms.  Finally  this  paper 
ends  with  a conclusion  and  an  outlook  of  its  discussed  subject  in  section  5. 

2.  Reusability 

Reusability  in  general  is  a possible  method  for  saving  monetary  and  non  monetary  costs.  In  the  context  of  learning 
platforms  reuse  of  learning  system  components  - the  implementation  itself  in  modularized  form  as  it  is  used  in 


Oliver  Merkel,  Cornelia  Seeberg  and  Ralf  Steinmetz 


KOM  - Multimedia  Communications 
Department  of  Electrical  Engineering  and  Information  Technology 
Darmstadt,  University  of  Technology 
Merckstr.  25  • 64283  Darmstadt  • Germany 


{merkel,  seeberg,  steinmetz}@kom.tu-darmstadt.de 


different  online  courses  - or  its  content  can  be  established.  While  modularized  configurable  implementation  is  a 
well  solved  problem  in  recent  online  learning  platforms  like  Lotus  Learning  Space  by  Mindspan  Solutions^, 
Netcoach  by  Or  bis  Communications^  or  Hyperwave  eLearning  Suite  by  Hyperwave^  - e.g.  by  configuration  of 
hiding  or  including  a chat  or  a discussion  forum  to  specific  courses  for  online  learning  platform  users  - reusing 
content  as  learning  objects  in  form  of  documents,  images,  other  multimedia  components  or  a collection  of  such 
components  is  recently  a matter  of  research  [LWGOl].  In  the  scope  of  this  paper  reusability  has  to  be  understood 
as  reusability  of  content  representing  documents  or  objects  as  a part  of  a document  in  form  of  educational 
multimedia  material  used  for  testing  of  knowledge  or  transfering  knowledge  to  a learner. 

Content  o**'ented  reuse  of  course  material  can  be  done  by  copying  a selectable  collection  of  course  docurnents  in 
whole  or  building  templates  from  formerly  used  courses  - e.g.  like  it  is  done  in  Lotus  Learning  Space,  WebTycho^ 
or  Hyperwave  eLearning  Suite  which  are  based  on  document  management  systems  (DMS)  on  top  of  Lotus  Notes 
/Lotus  Domino,  or  Hyperwave,  respectively.  At  least  if  content  oriented  reuse  on  a more  detailed  layer  is  focused, 
the  typical  automatically  generated  metadata  of  such  DMS  have  to  be  replaced  by  more  abstract  and  learning 
specific  metadata  schemata. 

2.1  Knowledge  Representation  through  LOM 

To  describe  a coherent  context  of  a complete  reusable  unit  in  [LWGOl]  the  term  learning  object  is  used.  The 
metadata  of  a learning  object  LOM  is  the  information  about  this  object.  When  storing  learning  objects  in  local  or 
distributed  repository  systems,  this  metadata  can  be  used  to  provide  effective  retrieval,  management,  transfer,  and 
use  of  learning  objects,  e.g.  from  a repository  system  into  a course  context,  by  working  on  the  corresponding  LOM 
entries  instead  analysing  the  content  of  a learning  object.  Beside  that  additional  metadata  information  which  is  not 
part  of  the  learning  object  itself  can  be  stored  in  LOM.  Consequently,  the  result  of  the  use  of  metadata  is  a reduction 
of  costs  through  reusability  of  the  described  learning  objects  while  facilitating  the  maintainance  of  learning  objects. 
LOM  offers  data  elements  and  a structure  for  these  data  elements  to  describe  metadata  of  a learning  object.  It 
contains  nine  main  categories  of  metadata  elements  representing  information  about  the  related  learning  object  as 
described  in  [LWGOl]. 

Since  other  metadata  schemata  like  those  from  ARIADNE^  - a consortium  of  20  European  universities  and  5 
international  corporations  - provide  mappings  to  LTSC^'s  LOM  and  the  IMS"^  (Instructional  Management  System) 
metadata  specification  model  which  is  compliant  with  LOM  [MBG+01],  the  relevance  of  LOM  is  becoming 
increasingly  fundamental  for  online  learning  platforms. 

2.2  Retrieval  and  Repository  Administration 

In  online  learning  platforms  working  on  top  of  a DMS  the  automatically  generated  metadata  can  be  used  and  often 
is  used  on  its  own  to  establish  the  administration  of  learning  documents.  A more  detailed  access  to  learning 
resources  on  a sub-document  layer  results  in  the  need  of  adding  an  additional  metadata  schemata,  if  comfortable 
retrieval  mechanisms  for  learning  objects  should  be  realized. 

So  the  learning  platform  repositories  should  consist  of  a content  repository  separated  from  the  metadata  repository. 
Retrieval  is  done  through  explicit  usage  and  search  on  metadata  which  references  other  metadata  or  its  represented 
learning  object. 

2.3  Granularity 

As  mentioned  before  it  is  of  enormous  importance,  if  either  reuse  should  be  done  on  a document  oriented  layer  or 
on  a sub-document  layer  inside  the  DMS  environment  of  a learning  platform,  too,  because  in  a DMS  accessibility 
of  documents  is  fundamental,  but  accessibility  of  element  parts  of  a document  is  usually  not  supported  by  the  DMS 
without  programmatical  effort.  Furthermore  it  can  easily  be  seen  that  metadata  descriptions  of  more  detailed 
reusable  content  compared  to  a document  level  can  also  be  used  on  higher  non  detailed  abstraction  levels. 


1 . URL:  http://www.lotus.com/home.nsf/welcome/leamspace 

2.  URL:  http://www.orbis-communications.de/index_nc.htm 

3.  URL:  http://www.hyperwave.de/e/products/els.html 

4.  URL:  http://tychousa.umuc.edu 

5.  URL:  http://www.ariadne-eu.org 

6.  URL:  http://ltsc.ieee.org 

7.  URL:  http://www.imsproject.org 
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For  an  increased  support  of  this  functionality  a granularity  element  is  included  in  the  LOA/ schemata  describing  the 
type  and  abstraction  layer  of  a learning  object.  One  not  yet  solved  problem  in  LOM  is  that  this  granularity  describes 
a two  dimensional  area  while  [LWGOl]  describes  the  granularity  as  a single  one  dimensional  field  with  no  further 
specification  or  description  about  the  vocabulary  which  build  the  entries  for  this  LOM  element.  The  horizontal 
direction  of  the  granularity  represents  the  type  - not  format  - of  the  learning  object  (e.g.  explanatory  text  section  of 
a learning  unit  or  in  contrast  questionary  text  of  an  exam),  while  the  vertical  direction  of  the  granularity  represents 
the  abstraction  layer  (e.g.  icon,  chart,  image,  explanatory  text  section,  chapter,  document,  lesson,  course;  see 
[HFM+01]). 

3.  Current  Situation 

At  the  moment  commercial  and  non-commercial  well-known  online  learning  platforms  only  attempt  to  support 
IMSov  lEEE/LOM  or  rather  announce  development  of  this  features  in  near  future  [EduOl].  So  on  the  producer  and 
researcher  side  developers  of  online  learning  platforms  see  the  need  to  support  detailed  metadata  specifications  and 
those  features  are  partly  implemented. 

The  currently  available  versions  of  comparison  schemata  in  general  do  not  contain  any  category  named  reuse  or 
reusability  and  only  one  well  referenced  comparison  schemata  [EduOl]  contains  information  regarding  metadata 
in  its  general  section. 

3.1  Learning  Platforms 

The  following  learning  platforms  were  investigated  for  this  paper  to  see  whether  they  already  support  any  type  of 
metadata  for  learning  objects  or  whether  support  is  planned  or  if  it  is  possible  to  add  an  own  support  via  a 
framework  or  an  application  programmers  interface  if  existent. 

WebTycho 

At  the  UMUC  home  campus  in  Adelphi,  Maryland,  a web-based  education  delivery  system  named  WebTycho  has 
been  developed.  It  is  based  on  Lotus  Notes  / Lotus  Domino  and  due  to  the  replication  features  of  this  platform 
WebTycho  is  offered  via  several  servers  located  in  Germany,  Japan,  and  the  United  States.  WebTycho  itself  is 
neither  freely  available  nor  commercially  offered  but  often  counts  as  a reference  system  when  comparing  online 
learning  platforms. 

Its  current  authoring  support  for  reusability  of  course  content  offers  storing  of  template  objects  ranging  from  whole 
courses  to  single  documents.  Subdocument  layer  elements  reuse  has  to  be  done  manually.  Metadata  support  beside 
that  of  Lotus  Notes  / Lotus  Domino  is  not  implemented  so  far. 

Lotus  Learning  Space 

For  practical  investigation  purposes  version  3.5  of  the  Lotus  Learning  Space  was  used  in  this  scope.  In  general  the 
same  reusability  options  like  those  from  WebTycho  are  supported. 

Corresponding  to  [EduOl]  Lotus  Learning  Space  version  4.0  allows  import  ofAICC  courses  while  IMS  support  is 
announced.  Import  and  export  of  course  content  and  course  state  in  XML  format  is  already  supported. 

Since  Lotus  Learning  Space  is  a database  template  for  Lotus  Notes  /Lotus  Domino  with  partly  open  course  design 
(partly  changeable  sources),  actors  with  designer  or  manager  rights  in  their  Notes  access  control  list  can  apply 
implementation  specific  changes  to  the  learning  platform  via  C/C++-API^  Java-API^  Lotus  Script  and  Lotus 
Formula  Language. 

Netcoach 

The  Netcoach  system  is  implemented  in  Lisp  and  uses  the  server’s  local  filesystem  as  the  storage  system  and 
repository  for  all  user  and  course  specific  data.  Courses,  exams  and  user  data  are  stored  in  different  but  task 
immanent  files  in  single  Lisp  structures. 

Although  this  results  in  a well  modularized  form,  reusability  of  data  is  only  supported  per  exporting  of  courses. 
Exported  data  is  proprietary  so  that  it  can  only  be  imported  in  Netcoach  servers  again. 

Netcoach  uses  its  own  metadata  structure  of  course  content.  The  current  implementation  does  not  offer  mappings 
or  imports  and  exports  to  other  metadata  formats. 
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Hyperwave  eLearning  Suite 

The  online  learning  platform  Hyperwave  eLearning  Suite  is  running  on  top  of  the  Hyperwave  Information  Server 
which  is  a document  management  system  which  stores  its  documents  in  an  object  relational  DBMS  {Hyperwave 
recommends  Oracle).  The  Hyperwave  Information  Server  and  its  related  additional  components  are  developed  at 
the  Technical  University  Graz,  Austria. 

The  underlying  document  management  system  offers  superficial  support  of  metadata  while  the  Hyperwave 
eLearning  Suite  itself  does  not  support  metadata  description  of  learning  resources.  System  immanent  development 
of  metadata  support  or  access  to  course  content  via  an  application  programmer’s  interface  is  supported  but  results 
in  high  efforts. 

Direct  reuse  of  courses  or  course  documents  above  subdocument  layer  is  possible  through  system  functions. 

Ilias^ 

The  open  source  Ilias  online  learning  platform  is  developed  at  the  University  of  Cologne,  Germany.  It  uses  a 
modularized  architecture  based  on  LAMP  (Linux,  Apache,  MySQL,  PHP).  Ilias  uses  a proprietary  format  named 
VRI  (Virtual  Resource  Index)  to  refer  to  the  modularized  learning  objects  dynamically.  It  directly  uses  this 
addressing  type  of  learning  objects  to  support  reusability. 

The  Ilias  metadata  system  VRI  is  based  on  the  concepts  of  IMS,  ARIADNE  and  Dublin  Core.  This  allows  direct 
mappings  from  VRI  to  these  target  formats.  It  is  used  on  the  course  layer,  the  learning  unit  layer,  the  documents 
layer,  and  subdocument  layers  like  pages  or  page  elements  of  the  learning  platform  contents. 

Since  Ilias  version  2.0  XML  templates  can  be  used  for  increased  reusability  of  course  content  and  course 
properties. 

3.2  Comparison  Schemata 

In  [Bat99]  Bates  suggests  an  online  learning  platform  comparison  schemata  named  ACTIONS.  This  name  is 
derived  from  its  main  comparison  categories;  Access,  Costs,  Teaching,  Interactivity,  Organisational  issues, 
Movelty  and  Speed.  Without  classifying  a specific  online  learning  platform  it  can  be  implied  that  the  reusability 
criteria  costs  and  speed  are  influenced  in  form  of  pre-programmed  multimedia  and  the  amount  of  needed  re- 
programming of  course  materials  when  stored  in  repositories  or  not.  The  terms  reuse  or  reusability  of  learning 
materials  itself  are  not  mentioned.  Further  aspects  that  can  be  treated  as  content  reusage  oriented  aspects  are  not 
discussed. 

The  LandOnline  service  offered  in  the  world  wide  web  [LanOl]  compares  different  commercial  and  non- 
commercial online  learning  platforms.  It  is  updated  and  reviewed  frequently  and  its  comparison  schemata  consists 
of  3 main  categories  with  a total  of  1 5 subcategories  containing  a total  of  62  criterions.  At  the  moment  55  different 
platforms  are  reviewed.  Regarding  the  classification  of  mentioned  platforms  in  reusability  contexts  the  criterion 
IMSjcompliance  is  the  only  criterion  giving  information  about  learning  object  reuse. 

While  Edutech  is  offering  a detailed  comparison  analysis  service  in  general  it  is  superficial  regarding  reusability. 
It  is  updated  frequently  and  compares  1 08  criterions  on  9 learning  platforms.  Only  its  general  section  handles  all 
available  authoring  support  with  reusability  oriented  criterions. 

The  obtained  list  of  the  considered  criterions  with  reusability  aspects  over  all  mentioned  comparison  schemata 
therefore  consists  of  the  following  elements:  IMS  compliance,  standards  / metadata  support  {IMS,  IEEE,  AICC)., 
XML  support,  programming  interfaces,  web  technology  compatibility  (compatibility  of  presented  learning  objects 
with  common  web  media  types),  import  / convert  existing  material,  flexible  resource  pool  (for  the  course 
developer). 

4.  Adding  Reusability  Features  in  Comparison  Schemata 

Summarized  it  can  easily  be  seen  and  implied  that  currently  offered  popular  comparative  analysis  tools  do  not 
comply  with  online  learning  projects  needs  when  decisions  have  to  be  made  regarding  which  online  learning 
platform  should  be  used. 

In  this  section  of  this  paper  a proposal  is  introduced  to  add  reusability  features  in  comparison  schemata.  The 
introduced  main  category  of  the  comparison  schemata  enrichment  is  named  reusability  and  consists  of  subsections 
named  basic^  learningObject^  and  additional. 


1 . URL:  http://www.ilias.uni-koeln.de/ios/index-e.html 
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The  notation  follows  the  rule  that  a higher  section  identifier  is  separated  from  the  included  subsection  identifier  by 

the  separation  character  (e.g.  reusability, learningObject  means  that  learningObject  is  a subsection  of  the 

category  reusability), 

4.1  Fundamental  Reusability  Specific  Criteria 

• reusability.basic:  Category  grouping  general  information  for  reusability  support. 

• reusability.basic.architecture:  Category  describing  the  general  architecture  of  the  online  learning  platform. 

• reusability.basic.architecture.type:  Data  element  describing  the  fundamental  type  of  the  online  learning 
platform  (e.g.  monolithic  or  modular  architecture). 

• reusability.basic.architecture.DMS:  Data  element  describing  the  document  management  system  of  the 
online  learning  platform  (e.g.  no  clean  separation  in  architecture,  Lotus  Notes  /Lotus  Domino,  Hyperwave 
Information  Server,  Zope), 

• reusability.basic.architecture.ServerServerReplication:  Data  element  decribing  if  the  system  provides 
server-server  replication  to  reuse  the  same  course  content  at  a different  location  through  distribution. 

• reusability .basic.development:  Category  grouping  developmental  aspects  of  the  online  learning  platform 
code. 

• reusability .basic.development.systemTemplate:  Data  element  describing  if  the  online  learning  platform  is 
a template  system  as  an  included  application  for  a DMS  {Lotus  Learning  Space,  Webtycho,  Hyperwave 
eLearning  Suite)  or  if  it  is  an  encapsulated  code  {Netcoach), 

• reusability .basic.development.sourceCodeAccess:  Data  element  describing  if  platform  source  code  is 
accessible  {Ilias,  Hyperwave  eLearning  Suite,  partly  available  in  Lotus  Learning  Space), 

• reusability .basic.development.APls:  Data  element  describing  which  programming  languages  are  supported 
through  APIs. 

4.2  Learning  Object  Specific  Criteria 

• reusability.learningObject:  Category  grouping  learning  object  specific  information. 

• reusability.learningObject.access:  Category  grouping  information  about  accessibility  of  learning  objects. 

• reusability.learningObject.access.type:  Data  element  describing  the  type  of  access  to  a learning  object 
(programmatic  through  API  functions,  manually  by  author  through  platform  function). 

• reusability.learningObject.access.granularity:  Data  element  describing  on  which  content  layer  the  access 
of  learning  objects  is  supported  (course  layer,  document  layer,  subdocument  layer,  element  collections,  single 
elements). 

• reusability.learningObject.metadata:  Category  describing  metadata  support. 

• reusability.learningObject.metadata.type:  Data  element  describing  the  type  of  metadata  representation  in 
the  platform  (proprietary,  LTSC  LOM,  ARIADNE,  IMS  Global  Learning  Consortium,  Dublin  Core), 

• reusability.learningObject.metadata.version:  Data  element  describing  the  version  of  supported  metadata. 

• reusability.learningObject.metadata.mappings:  Data  element  describing  which  mappings  exist  to  other 
metadata  formats. 

• reusability.learningObject.metadata.supportLevel:  Data  element  describing  the  level  of  metadata  support 
in  the  used  metadata  format  (fully,  partly,  percentage  value). 

• reusability.learningObject.metadata.cardinalityType:  Data  element  describing  the  number  of  metadata 
sets  associated  to  the  number  of  learning  objects  (1:1,  1 :n,  m:  1 , m:n  associations). 

• reusability.learningObject.metadata.usage:  Data  element  describing  how  metadata  is  used  by  the  platform 
(used  in  retrieval  process;  import-/export  functions). 

4.3  Additional  Criteria 

• reusability  .additional:  Category  describing  additional  criteria  with  content  reusability  support. 

• reusability .additional.interoperability:  Data  element  describing  the  ability  to  integrate  the  platform  in  an 
existing  computer  infrastructure  (e.g.  to  use  common  webservers,  protocols,  data  standards  and  programming 
languages). 

• reusability .additional.contentStorage:  Category  describing  the  platform  content  storage  system. 

• reusability.additional.contentStorage.type:  Data  element  describing  the  type  of  database  used  (filesystem; 
proprietary  platform  build-in;  external  RDBMS,  ORDBMS,  XML  DB,  object-oriented  DB). 

• reusability .additional.contentStorage.dataFormat:  Data  element  describing  the  format  of  the  stored 
content  (XML,  relational  tables,  proprietary,  BLOB,  CLOB). 
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• reusability .additional.contentStorage.accessType:  Data  element  describing  the  content  access  type 
(programmatically  through  API,  macro  defmition/manually  by  platform  function). 

• reusability .additional.metadataStorage:  Similar  to  reusability.additional.contentStorage  but  for  metadata. 

• reusability.additional.metadataStorage.type:  Similar  to  reusability.additional.contentStorage.type  but  for 
metadata. 

• reusability.additional.metadataStorage.dataFormat:  Similar  to 
reusability. additional. contentSto rage. dataFormat  but  for  metadata. 

• reusability.additional.metadataStorage.accessType:  Similar  to 
reusability.additional.contentStorage.accessType  but  for  metadata. 

• reusability.additional.export:  Category  grouping  data  export  specific  information  about  the  platform. 

• reusability.additional.export.datatype:  Data  element  describing  which  type  of  data  can  be  exported 
(courses,  documents,  images,  text  sections,  audio,  video,  applications,  exams,  &c.) 

• reusability.additional.export.format:  Data  element  describing  the  format  used  to  export  the  corresponding 
data  type  (XML,  object  specific  format) 

• reusability.additional.import:  Similar  to  reusability.additional.export  but  for  import. 

• reusability.additional.import.datatype:  Similar  to  reusability.additional.export.datatype  but  for  import. 

• reusability.additional.import.format:  Similar  to  reusability.additional.export.format  but  for  import. 

• reusability.additional.transcoding:  Data  element  describing  to  which  formats  learning  objects  or  metadata 
can  be  automatically/manually  transcoded  (format  conversion)  by  the  platform. 

5.  Conclusion  and  Outlook 

This  work  proposes  a criterion  enrichment  by  adding  content  reuse  specific  categories  to  comparative  analysis 
tools  to  support  architecure  decision  making  in  an  online  learning  environment.  After  analysis  of  existing  learning 
platforms  and  their  reusability  features  implemented  in  currently  available  versions  an  abstract  description  of 
decision  making  on  learning  platforms  was  investigated  and  afterwards  concrete  existing  often  refered  comparative 
analysis  tools  were  analysed.  Referring  to  this  analysis  a proposal  has  been  stated  following  a logical  structure  and 
associates  online  learning  platform  properties  and  features,  stored  learning  object  and  their  metadata  properties  to 
significant  and  meaningful  identifiers  in  comparison  schemata  design  for  online  learning  platforms. 

Investigated  online  learning  platforms  fit  in  the  proposed  schemata  enrichment  very  well.  Future  work  might  result 
in  implementing  the  proposal  into  existing  comparative  analysis  tools.  Furthermore  an  introduction  of  project 
characteristic  coefficients  describing  a typical  specific  property  rating  spectrum  of  a project  will  be  investigated. 
This  might  allow  a more  specific  choice  of  an  available  set  of  target  platforms  due  to  the  large  scale  and  variety  of 
different  types  of  reusability  demands  in  online  learning  projects. 
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WCET  e-Learning  Projects  and  Resources  (http://www.wcet.info) 


Introduction 

This  brief  paper  offers  an  overview  of  several  key  e- learning  projects  at  the  WCET  (Western 
Cooperative  for  Educational  Telecommunications).  These  projects  include:  a web-based 
resource  that  is  designed  to  develop  specific  tools  for  the  llighcr  education  COirnnurnty;  a project 
developing  web-based  student  services  including  models,  resources  and  processes  for  their 
implementation;  and  the  “technology  costing  methodology”  project  that  enables  institutions  to 
calculate  the  cost  of  technology  within  and  across  higher  education. 

e-Learning  Technologies  Projects 

WCET  was  awarded  a major  grant  from  the  William  and  Flora  Hewlett  Foundation  to  support 
several  critical  eLearning  projects.  This  grant  is  providing  generous  support  for  several  projects: 

• The  first,  e-Learning  Technologies:  Web  Resource  for  Comparisons,  will  provide  a 
resource  to  help  educators  analyze  and  compare  vendor-produced  course  management 
tools,  software  products,  outsourcing  vendors,  and  e- learning  policies. 

• A second,  just  underway,  is  exploring  the  need  for  develop  and  implement  a simple-to- 
use  system  for  evaluating  distance  learning  courses  available  from  third-party  vendors. 

• The  final  project  involves  the  Hewlett  Foundation  and  WCET  staffs  working  together  to 
explore  policy  issues  that  are  likely  to  have  an  impact  on  policy  for  the  successfully 
integrating  technology  into  higher  education.  Based  on  this  work,  a paper  on  open  source 
course  management  systems  will  be  released  early  this  summer. 

e-Learning  Technologies:  Web  Resource  for  Comparisons  (www.edutools.info ) 

Of  the  Hewlett  Foundation  funded  projects,  the  Web  Resource  for  Comparisons  is  well 
underway  to  being  implemented  and  will  be  highlighted  in  this  section.  For  colleges  to  quickly 
implement  online  courses,  many  course  management  software  products  have  been  developed. 
These  tools  make  it  easier  to  adapt  curriculum  to  the  Web  and  to  give  the  student  a common 
“look  and  feel”  in  each  course. 

Over  the  next  two  years,  this  project  will  redesign  and  expand  the  existing  site.  The  new  website 
will  be  called  EduTools  (www.edutools.info).  We  will  also  develop  three  new  companion  sites 
that  apply  the  same  decision-making  models  for  comparisons  of  other  tools.  The  three  new  sites 
will  analyze:  1)  electronic  student  support  services,  2)  instructional  technologies,  and  3)  policies 
relating  to  the  use  of  educational  technologies. 

Beyond  the  Core:  Creating  Web-based  Student  Services  for  Online  Learners 

Students  participating  in  distance  education  academic  programs  must  also  have  access  to  student 
support  services.  However,  one  of  the  biggest  gaps  in  online  education  is  institutions'  inability  to 
provide  time-  and  location- independent  access  to  a complete  array  of  student  support  services. 
Higher  education  institutions  and  other  providers  must  take  advantage  of  technology  to  deliver 
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student  services  to  online  learners  in  a way  that  effectively  meets  their  unique  needs.  This  project 
will  focus  on  Web-based  student  support  services,  defined  broadly,  and  will  result  in: 


• Student  services  models  developed  at  three  partner  institutions. 

• A set  of  guidelines  for  institutions  interested  in  building  their  own  "home-grown"  Web- 
based  services. 


* Detailed  case  studies  of  the  instiUitional  change  processes  required  to  implemeut  W eb- 


based  student  services. 


The  project  uses  the  Web  to  collaborate  and  disseminate  information  on  online  student  services. 
The  project  leaders  also  produce  a monthly  Webcast  series  called.  Providing  Student  Services  to 
Distance  Learners.  Guest  experts  define  best  practices  and  demonstrate  effective  and  innovative 
online  services.  This  project  will  result  in  a number  of  models  and  resources  for  institutions  to 
draw  upon. 


Technology  Costing  Methodology  (TCM)  and  TCM2:  Completing  the  Toolbox  Project 

The  original  Technology  Costing  Methodology  (TCM)  project  began  in  September  of  1998  with 
the  goal  of  developing  an  authoritative  costing  methodology  (and  related  procedures)  for 
calculating  technology  costs  (1)  within  an  institution  to  determine  if  proposed  instructional 
approaches,  that  make  heavy  use  of  technology,  actually  do  serve  to  contain  costs;  and  (2)  across 
institutions  allowing  data  to  be  compared  legitimately  for  different  instructional  or  technological 
approaches,  which  will  benefit  legislatures,  state- governing  boards,  state  coordinating  boards, 
and  federal  agencies. 

A follow-on  project.  Technology  Costing  Methodology:  Completing  the  Toolbox  Project 
(TCM2),  focuses  on  refining  the  methodology  and  assisting  decision-makers  in  exploring  and 
understanding  the  policy  implications  resulting  from  additional  costing  information. 

About  WCET 

The  Western  Cooperative  for  Educational  Telecommunications  (WCET)  is  a recognized  leader 
in — and  advocate  for — the  effective  use  of  educational  telecommunications.  Its  membership  is 
open  to  higher  education  institutions  and  government  agencies,  non-profit  organizations,  schools, 
and  corporations — regardless  of  location — that  are  providers  and  users  of  educational 
telecommunications.  Six  countries  and  43  states  are  represented  among  the  nearly  250  members. 
WCET,  established  by  the  Western  Interstate  Commission  for  Higher  Education  (WICHE)  in 
1989,  serves  its  members  and  the  broader  higher  education  community  in  the  following  ways. 

• Central  clearinghouse  for  information  and  expertise. 

• Source  of  professional  development  and  assistance  including  the  annual  conference  will 
be  held  in  Denver,  CO  Nov.  6-9,  2002,  and  the  Institute  for  Managing  and  Developing  E- 
Learning  (MDE)  will  be  held  in  Park  City,  UT  on  July  21-25,  2002. 

• Leader  in  facilitating  cross- institutional  projects  in  mediated  education. 

• Advocate  for  effective  policies  and  practices  supporting  e- learning. 

• Evaluator  and  researcher  on  quality  uses  of  educational  telecommunications. 


From  PBL  to  e-PBL 


C.  Jacqmot  and  E.  Milgrom 
Department  of  Computing  Science  and  Engineering 
University  catholique  de  Louvain 
Place  Sainte-Barbe,  2 B-1348  Louvain-la-Neuve,  Belgium 
cj@info.ucl.ac.be  / em@info.ucl.ac.be 

Abstract:  A completely  new  engineering  ciariculum  has  been  intrcduccd  in  September  2000  by  tlic  Schuul  uf 
Engineering  of  the  University  catholique  de  Louvain  (UCL).  This  curriculum  is  entirely  based  upon  active  and 
collaborative  learning  by  students.  Surprisingly  enough,  very  little  is  readily  available  to  effectively  support  this  type  of 
pedagogy  by  means  of  technology.  This  contribution  looks  at  available  general  and  special-purpose  tools  from  the  point 
of  view  of  their  suitability  for  problem-  and  project-based  learning.  It  also  points  to  areas  in  which  adequate  support  is 
still  needed.  Finally,  it  looks  into  technology  as  a way  to  allow  participation  by  remote  learners  and  tutors  and  it  makes 
the  case  for  ubiquitous  access  to  learning  tools. 

Introduction 

Active  learning  as  embodied  in  either  problem-  or  project-based  learning  has  been  around  for  more  than  30  years.  Many 
authors  testify  to  the  benefits  of  active  learning  both  from  the  point  of  view  of  learning  outcomes  and  from  that  of  student  and 
teacher  satisfaction  [Vernon,  Blake  1993],  [Gijselaers  1996]. 

After  a long  and  extensive  study,  the  School  of  Engineering  of  UCL  decided  in  1 999  to  embark  upon  a complete  overhaul  of 
its  curriculum  in  order  to  implement  active  learning  approaches.  We  took  advantage  of  this  opportunity  to  precisely  define 
the  expected  learning  outcomes  - not  to  be  confused  with  the  customary  tables  of  contents.  The  new  curriculum  was 
launched  in  September  2000  (for  the  first  year  of  our  5 year  program)  and  will,  over  5 years,  gradually  replace  the  traditional 
approach. 

Active  learning  is  the  central  principle  of  this  model:  students  are  continuously  exposed  to  activities  which  require  direct 
participation  and  which  stimulate  learning.  Problems  are  designed  and  run  in  much  the  same  fashion  as  in  by  now  "classical" 
Problem  Based  Learning  (PBL)  [Boud,  Feletti  1997].  Projects  are  different  from  what  one  often  finds  in  engineering 
curricula  in  that  they  are  a means  for  acquiring  new  knowledge  and  competences,  not  merely  for  demonstrating  an  ability  to 
apply  previously  acquired  capabilities.  We  use  projects  mostly  to  develop  interdisciplinary  and  longer-term  (11  weeks) 
approaches,  while  problems  are  used  mostly  within  a single  discipline  and  during  a shorter  time  span  (1  week).  Our 
hypothesis  is  that  problems  allow  students  to  delve  in  a more  controlled  way  into  disciplinary  topics  than  projects,  thereby 
ensuring  that  essential  topics  would  not  be  merely  glanced  over. 

A second  major  principle  of  our  approach  is  collaborative  learning:  groups  of  7 or  8 students  are  constituted  randomly  at  the 
onset  of  every  trimester.  Many  - but  not  all  ~ activities  are  tackled  collectively  by  all  students  belonging  to  a group:  this 
concerns  mostly  the  preparation  of  individual  learning,  the  planning  of  subtasks,  and  the  sharing  of  results.  Some  of  the  group 
meetings  occur  in  the  presence  of  a tutor;  much  necessary  work  is  done  either  individually  or  in  groups  without  presence  of 
tutors.  Meetings  are  run  (by  the  students  themselves)  according  to  a set  of  guidelines,  but  a lot  of  leeway  is  given  for 
organizing  both  collective  and  individual  work  (provided  it  js  oiganized). 

A third  essential  concept  in  our  new  curriculum  is  a focus  on  (self-)assessment  of  progress  and  on  systematic  reflection 
about  one's  learning  process.  Indeed,  we  try  to  make  the  learning  process  as  explicit  as  possible,  since  we  believe  that 
awareness  about  one's  learning  process  is  a necessary  condition  for  success. 

This  instructional  model  fits  nicely  with  the  learning  theory  we  adhere  to:  socio-constructivism  [Jonnaert,  Vander  Borght 
1999]  [Seely,  e.a  1989].  Problems  and  projects  are  situated  in  realistic  professional  contexts  and  incite  our  students  to  build 
upon  existing  knowledge  to  acquire  new  knowledge;  communities  of  learners  stimulate  individual  learning. 

In  this  pedagogical  approach,  the  role  of  faculty  and  other  staff  is  totally  transformed:  from  teachers  and  assistants  we 
morphed  into  tutors  and  facilitators,  entirely  new  roles  for  most  of  us.  From  providing  answers  we  switched  to  asking 
questions,  to  guiding  progress,  to  pointing  to  resources. 

The  role  of  technology 

We  decided  to  look  at  ways  to  use  technology  firstly  to  facilitate  and  enhance  learning  by  students  and  secondly  to  lighten  the 
overall  load  on  our  staff.  It  would  of  course  have  been  preferable  to  design  and  implement  necessary  tools  along  as  we 
designed  and  implemented  our  new  curriculum,  but  scarcity  of  resources  and  the  fact  that  curriculum  design  and 
implementation  is  an  ongoing  iterative  process  prevented  us  from  following  a concurrent  approach.  It  is  thus  only  after  one 
year  of  experience  with  the  new  curriculum  that  we  started  looking  into  ways  to  use  technology  to  support  our  pedagogical 
approach. 

Two  kinds  of  tools  may  be  considered  when  looking  into  ways  to  use  technology  in  education:  widely  available  general 
purpose  presentation  and  communication  tools  and  special  purpose  teaching/leaming  tools.  We  looked  into  both  categories 
from  the  point  of  view  of  their  suitability  for  supporting  our  own  brand  of  active  learning.  The  next  part  of  the  paper  presents 
a typical  problem  based  learning  cycle  and  reviews  possible  uses  of  technology. 
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From  PBL  to  e-PBL 

In  our  flavor  of  PBL,  a problem  is  tackled  by  a group  of  students  during  a time  span  of  one  week.  The  work  is  organized  as 
follows: 

• In  the  first  scheduled  meeting,  the  kick-off  meeting,  the  group  will  analyze  a new  problem  situation;  it  will  take  stock 
of  existing  knowledge  which  may  help  tackle  the  problem  and  it  will  try  to  produce  a set  of  specific  (learning)  tasks 
which  it  feels  should  be  accomplished  in  order  to  increase  its  available  knowledge  in  the  areas  related  to  the  problem 
and  in  order  to  produce  whatever  outcomes  are  needed  according  to  their  understanding  of  the  requirements  of  the 
given  problem  statement.  Resources  will  then  be  allocated  to  the  various  tasks.  The  group  will  also  update  its  weekly 
schedule  in  order  to  reflect  its  own  planning  of  the  work  related  to  the  problem  at  hand.  A tutor  is  present  to  make 
certain  the  group  reaches  the  objectives  of  the  meeting  (not  to  answer  questions  about  the  material).  Since  we  have 
quite  a large  number  of  groups,  a single  "floating"  tutor  handles  three  groups  simultaneously.  A series  of  questions 
about  the  problem  may  be  handed  over  to  the  group  to  help  focus  their  analysis. 

• After  the  kick-off  meeting,  students  will  do  whatever  they  have  themselves  decided  to  do  to  get  closer  to  the  solution 
of  the  problem,  trying  to  keep  to  the  schedule  they  have  set  for  themselves.  They  may  work  alone,  in  subgroups  or  in 
the  full  group,  according  to  their  own  preferences  and  availability.  This  phase  involves  identifying  and  reading  of 
course  material  and  other  resources,  discussion  among  group  members  about  issues  raised  by  individual  learning, 
resolving  exercises,  etc. 

• Two  or  three  days  after  the  kick-off  meeting,  there  is  a second  scheduled  meeting  in  the  presence  of  the  tutor.  In  this 
meeting,  progress  towards  solving  the  problem  is  analyzed  and  discussed,  and  intermediate  results  are  shared  by  the 
members.  This  is  usually  an  occasion  which  raises  further  learning  issues  and  which  may  impact  the  weekly  schedule 
since  new  subtasks  may  be  identified. 

• After  the  intermediate  meeting,  students  will  again  pursue  their  own  learning  activities  and  execute  whatever  tasks 
the  group  has  allocated  them. 

• At  the  third  and  final  scheduled  meeting,  the  group  finalizes  its  production;  it  may  be  asked  by  the  tutor  to  present  its 
solution  or  any  other  product  of  the  work.  Sometimes,  the  group  is  asked  to  answer  questions  about  the  course 
material  which  is  relevant  for  the  problem. 


We  won’t  dwell  further  on  projects^  which  are  run  in  a way  similar  to  the  one  outlined  above  for  problems,  albeit  over  a 
longer  time  span  and  with  weekly  scheduled  meetings  between  groups  and  their  tutor. 

If  one  looks  at  technology  and  wonders  how  it  could  be  marshaled  in  order  to  support  the  process  outlined  above,  one  will 
immediately  figure  out  straightforward  ways  to  use,  say,  the  Internet.  Problem  statements,  resources,  focus  questions  and  the 
like  can  be  made  readily  available  through  a Web  site.  E-mail  may  be  used  to  communicate  among  members  of  a group  and 
with  the  tutor.  Chats  may  help  group  members  pursue  discussions  even  when  not  all  members  are  physically  present  and 
threaded  discussions  forums  may  help  a group  focus  on  a given  issue  until  it  is  satisfactorily  resolved.  Calendar  and  planning 
software  may  be  used  to  handle  the  scheduling  and  planning  tasks.  Finally,  students  may  be  encouraged  to  write  their  own 
Web  pages  in  order  to  record  information  they  wish  to  share  with  other  members  of  their  group  or  with  other  groups. 

Of  course,  one  should  immediately  point  out  that  using  the  Web  to  make  resources  available  to  students  is  still  traditional 
transmissive  pedagogy:  a book  with  hyperlinks  is  still  a book.  Very  few  course  delivery  platforms  manage  to  provide  a truly 
personalized  learning  experience  and,  to  judge  by  the  amount  of  paper  printouts  of  Web-delivered  course  materials,  students 
have  not  yet  been  convinced  to  opt  for  a paperless  society  ! It  is  also  worth  noting  that  chats,  while  eminently  suitable  for 
synchronous  communication  between  remote  people,  are  very  volatile:  they  leave  no  or  little  useful  record  of  the  exchanges 
for  later  perusal.  On  the  other  hand,  if  threaded  discussion  forums  are  not  monitored  closely,  they  may  perpetuate  and 
disseminate  incomplete  or  erroneous  information.  Also,  their  very  nature  forces  them  to  grow  and  become  unwieldy:  here 
again,  intervention  by  a qualified  teacher  is  needed  to  summarize  and  condense  information  into  an  exploitable  form. 

A first  conclusion  may  be  drawn  from  this  mundane  usage  of  the  Internet:  even  though  setting  up  everything  that’s  needed 
may  require  more  than  a casual  effort,  very  little  of  this  usage  is  actually  related  to  learning  ! We  shall  therefore  discuss  some 
tools  which  could  provide  true  scaffolding  for  the  learning  process  in  PBL  by  looking  at  some  of  the  activities  which  occur  in 
this  process,  some  of  which  are  managerial,  while  others  are  truly  cognitive: 

Analyzing  a problem  situation  Making  choices 

Formulating  hypotheses,  ideas,  proposals  Making  all  ideas,  opinions,  decisions  known  to  all  group  members 

Restating  the  problem  Finding  out  how  work  has  progressed  with  respect  to  the  schedule 

Identifying  sub-problems  Proposing  paths  to  solutions 

Preparing  or  modifying  a list  of  learning  issues  Evaluating  paths  to  solutions  and  solutions 

Preparing  or  modifying  a list  of  tasks  and  milestones  Assessing  the  group’s  progress  in  the  various  tasks 

Allocating  resources  to  tasks  Assessing  one’s  own  mastery  of  the  course  material 

Scheduling  tasks  and  meetings  Preparing  the  expected  production,  results,  outcomes 

Structuring  knowledge  Assessing  other  groups’  production 

Sharing  knowledge  Identifying  and  resolving  conflicts 

What  we  are  looking  for  are  tools  which  support  and  encourage  learning  through  discovery  and  peer  interaction  (i.e. 
interactions  which  affect  learning  positively  and  deepen  student  understanding)  and  which  promote  collaboration  and  group 
cohesion. 
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BEST  COPT  AVABUBIiE 


Shared  workspaces 

Shared  workspaces  such  as  BSCW  [Appelt  1999]  provide  the  basic  mechanisms  for  cooperative  work  between  members  of 
various  commimities,  thus  also  - potentially  - between  the  members  of  a PBL  group  (including  the  tutor).  Shared  workspaces 
are  used  to  collect,  structure,  and  present  any  kind  of  information  needed  for  the  work  at  hand,  usually  arranged  in  a 
hierarchy  based  on  commonly  agreed  upon  structuring  principles.  They  may  also  provide  event  services,  which  will  inform 
members  of  a group  that  something  worth  their  attention  has  happened.  Additional  features  may  include  authentication, 
version  management,  search  capabilities,  discussion  forums,  etc.  Integration  of  various  synchronous  and  asynchronous 
communication  means  is  usually  provided,  up  to  a point.  There  may  be  some  support  for  the  scheduling  of  meetings.  Several 
of  the  group  communication  and  management  activities  in  the  list  above  may  be  supported  in  some  way  by  such  shared 
workspaces,  provided  the  learning  curve  is  not  too  steep  (for  teachers  as  well  as  for  students).  However,  as  general-purpose 
tools,  such  workspaces  fall  short  of  providing  support  for  cognitive  activities.  Still,  our  intent  is  to  deploy  such  a system  next 
year  in  order  to  find  out  which  of  the  features  will  be  adopted  by  our  students  to  support  their  collaboration. 

A next  step  up  the  evolutionary  ladder  is  exemplified  by  environments  such  as  the  CoVis  Collaborative  Workbook  [Edelson 
e.a.  1996].  In  this  system,  the  objects  manipulated  by  the  tools  are  not  just  any  kind  of  document,  but  they  are  typed 
according  to  their  use  in  a learning  process.  For  instance,  in  the  Scientific  Notebook,  one  distinguishes  the  following  types  of 
objects  (called  "pages"):  question,  conjecture,  evidence-for,  evidence-against,  plan,  step-in-plan,  information,  and  commentary. 
Students  add  pages  and  links  between  pages  as  they  proceed  along  their  inquiry,  which  helps  structure  the  process  and  thus 
leaves  more  time  for  working  on  the  content.  The  resulting  web  of  pages  provides  a trace  of  the  process  and  certainly  helps 
tutors  provide  better  feedback  than  that  which  may  be  provided  by  looking  only  at  the  final  results  of  a group's  work.  The 
authors  of  the  system  report  on  several  successful  uses  of  their  tool  in  project-based  learning. 

Still  higher  on  the  scale  of  tools  providing  effective  support  for  cognitive  activities  are  tools  exemplified  by  IHMC's  Concept 
Mapping  Tool  CMap  [Canas,  e.a.  1999].  With  this  tool,  students  may  collectively  build  a graphical  representation  of  concepts 
and  relationships  between  concepts  as  their  imderstanding  of  a domain  evolves  over  time.  Threaded  discussions  may  be 
associated  with  concepts  and  with  links,  and  a system  for  managing  propositions  is  also  available.  The  way  students  model 
their  knowledge  by  means  of  such  tools  provides  deep  insight  in  their  thinking  processes  and  allows  even  more  effective  tutor 
interaction  and  feedback.  We  shall  soon  look  into  ways  to  integrate  CMap  in  our  problem  and  project-based  approach  since 
we  feel  that  its  knowledge  representation  power  will  help  us  make  learning  more  explicit  (as  sequences  of  evolving  concept 
maps). 

Time  management 

As  they  find  out  quite  quickly  for  themselves,  the  scarcest  resource  for  students  (and  for  teachers)  is  time.  Very  few  of  our 
students  manage  to  successfully  juggle  the  various  assignments  and  tasks  which  run  concurrently.  Since  we  wish  to  develop 
self-sufficiency,  we  refrain  from  providing  complete  work  schedules  for  every  group.  Instead,  we  schedule  only  pre-planned 
activities  such  as  lectures  and  tutored  meetings.  We  wish  therefore  to  incite  our  students  to  pay  serious  attention  to  issues 
such  as  planning,  scheduling,  milestones,  deadlines  and  accountability.  This  is  an  area  in  which  none  of  the  available 
software  packages  we  looked  into  fit  our  requirements,  our  needs,  and  our  budgets.  Most  calendar  and  planning  software  is 
designed  for  professional  use  and  is  much  too  complex  and  constraining  for  student  use.  Besides,  such  tools  may  allow  one  to 
enter  a schedule,  check  its  consistency,  and  verify  its  execution,  but  do  they  help  one  learn  how  to  prepare  a valid  schedule  ? 

Assessment  and  feedback 

A central  tenet  of  problem  and  project-based  learning  is  that  a student  should  be  constantly  aware  of  his/her  progress  (or  lack 
of  it).  Self  assessment,  peer  assessment,  and  assessment  by  teaching  staff  are  thus  an  intrinsic  part  of  any  PBL  approach. 
Indeed,  some  form  of  assessment  necessarily  precedes  reflection  upon  one's  learning.  Since  it  would  be  too  costly  to  provide 
on-demand  assessment  by  staff,  numerous  attempts  have  been  made  to  provide  on-line  interactive  assessment  tools.  Here  too, 
the  situation  is  less  than  encouraging,  since  most  tools  fail  to  allow  assessment  of  higher  order  cognitive  skills:  how  does  one 
write  a tool  to  check  the  validity  of  a proof,  the  correctness  of  a design,  the  adequacy  of  a line  of  reasoning  ? How  does  one 
distinguish  between  shallow  and  deep  learning  ? How  does  one  reveal  incomplete  understanding  ? 

Our  experience  with  on-line  multiple  choice  / multiple  answers  quizzes  shows  that  they  tend  to  give  a false  sense  of 
knowledge  and  understanding  and  thus  fail  to  promote  self  questioning  and  meta-reflection:  students  tend  to  stop  trying  to 
learn  as  soon  as  their  marks  reach  a "satisfactory"  level:  scoring  high  becomes  the  main  goal,  instead  of  acquiring  knowledge. 
Besides,  we  would  like  our  students  to  set  their  own  learning  goals,  not  only  to  try  and  reach  those  we  set  for  them.  Providing 
effective  on-line  self-assessment  for  student-set  learning  goals  is  even  harder  to  imagine  ! 

About  the  human  cost  of  technology 

When  we  decided  to  switch  from  traditional  pedagogy  to  active  learning,  we  knew  it  would  require  a lot  of  effort  to  design 
and  implement  the  new  curriculum.  It  turns  out  that  using  technology  to  support  collaborative  learning  has  significantly 
added  to  our  work  load:  deploying  and  maintaining  tools  to  help  our  students  is  far  from  cheap  with  respect  to  staff  costs. 
Several  of  us  have  paid  dearly  to  set  up  devices  whose  pedagogical  efficiency  remains  in  doubt.  The  lesson  we  draw  is  that, 
when  planning  for  the  use  of  technology  in  education,  one  should  run  a cost/benefit  study  taking  into  account  the  real  added 
value  of  technology  rather  than  expectations. 


From  e-PBL  to  d-PBL  to  m-PBL  ? 

We  now  briefly  contemplate  what  happens  when  one  adds  new  dimensions  to  the  issue  of  using  technology  to  facilitate 
problem-  or  project-based  learning. 

A first  possible  extension  consists  in  allowing  some  of  the  members  of  a group  to  participate  in  the  various  activities  while 
not  physically  on  the  premises  with  the  rest  of  the  group.  In  this  case,  one  mostly  worries  about  alleviating  the  effects  of 
distance  on  the  dynamics  of  collaborative  work  [Davis,  Schlais  2000], 

Some  of  the  tools  such  as  e-mail,  chats  and  threaded  discussion  forums,  which  fail  to  provide  significant  support  for  PBL 
when  a!!  participants  are  present  in  the  same  room,  now  start  playing  a much  more  effective  role  since  they  helo  bridge  the 
physical  distance  between  participants  and  allow  more  asynchronous  operation  of  the  group.  The  other  tools  described  earlier 
will  certainly  be  as  useful  as  in  the  case  of  "traditional"  PBL.  Of  course,  one  should  not  underestimate  the  negative  impact  of 
distance  on  social  issues  such  as  group  identity  building,  group  cohesiveness,  and  individual  motivation  (every  one  of  which 
is  essential  to  the  success  of  PBL)  and  one  should  probably  not  overestimate  the  power  of  technology  to  truly  compensate  for 
this  impact.  Still,  this  case  may  be  one  when  videoconferencing  might  prove  useful  (certainly  for  Europeans  learners)  to 
overcome  the  distance  barrier  between  participants. 

Another  issue  which  must  be  faced  is  that  of  the  availability  of  technology-based  tools  to  learners.  We  feel  that  learning  tools 
should  be  available  whenever  and  wherever  a learner  feels  like  using  them.  This  means  that,  whenever  possible,  a variety  of 
terminals  (PC,  PDA,  cell  phone)  should  allow  access  to  these  tools  with  a minimum  of  technical  fuss:  what’s  the  use  of  a 
powerful  agenda  tool  if  it  is  not  available  day  and  night  ? We  are  now  in  the  process  of  providing  wireless  access  to  a part  of 
the  city  of  Louvain-la-Neuve  in  order  to  facilitate  ubiquitous  connectivity  and  mobility.  We  expect  that  generation  GSM 
mobile  phones  will  widen  the  coverage  within  a few  years.  Still,  not  many  of  the  existing  tools  cater  to  terminals  in  the  PDA 
to  cell  phone  range,  which  rather  limits  the  usefulness  of  wireless  coverage. 


A single  year  has  been  concluded  at  the  time  of  the  writing  of  this  paper:  it  is  therefore  too  early  to  draw  conclusions  about 
the  actual  learning  outcomes  of  our  new  curriculum.  Still,  one  thing  is  quite  clear:  many  of  us  feel  a strong  need  to  use 
technology  in  order  to  enhance  the  quality  of  students'  learning  and,  if  possible,  to  relieve  staff  from  some  of  the  pressure 
inherent  to  active  pedagogy.  An  overview  of  existing  technology-based  tools  reveals  that  few  of  those  which  are  readily 
available  are  really  effective  in  their  support  of  the  learning  process  in  PBL,  A first  challenge  is  thus  to  identify  and  deploy 
tools  specifically  to  enhance  and  facilitate  learning  and  related  activities,  rather  than  merely  automate  some  of  the  clerical 
and  managerial  work. 

A second  challenge  is  to  provide  ubiquitous  access  to  existing  and  future  tools:  learning  activities  should  be  possible  in  every 
occasion  when  a learner  is  (1)  available  and  (2)  willing  to  learn.  We  feel  that  too  many  learning  opportunities  are  missed 
because  technology  is  not  yet  available  in  every  case  when  both  conditions  are  met. 

A last  - but  not  the  least  - issue  concerns  the  overhead  for  both  staff  and  students  for  using  technology-based  tools:  these 
tools  need  to  be  simple  and  easy  to  use  in  order  to  allow  both  categories  of  participants  to  focus  their  attention  and  devote 
their  efforts  to  the  essential  aspects  of  their  respective  jobs  (i.e,  instructional  design,  tutoring,  collaborating,  and  learning), 
and  not  on  the  use  of  the  tools  themselves  or  the  mastering  of  their  interfaces. 
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Abstract  Language  learners  often  leave  school  able  to  do  little  more  than  parrot  second  language  formulae.  This  paper 
proposes  a Web-based  system  for  the  construction,  delivery  and  management  of  online  language  courses.  The  system 
and  courses  it  supports  are  aimed  at  providing  more  timely  and  personalized  instruction  than  learners  normally  have 
available.  They  also  provide  opportunities  for  practice  in  collaborative  and  creative  communication.  This  paper 
illustrates  an  online  ESL  course  driven  by  this  system,  and  highlights  a role-playing  activity  that  encourages  the 
appropriate  use  of  language  in  social  and  business  contexts.  It  allows  students  to  gain  facility  with  basic  grammar, 
vocabulary  and  conversational  language  devices,  as  well  as  ‘higher  order’  pragmatic  linguistic  skills. 


A problem  in  language  education 

The  need  to  educate  a large  and  varied  population  in  English  as  a lingua  franca  in  SE  Asia  has  given  rise  to  a number 
of  problems,  which  are  in  a general  sense  also  faced  in  the  teaching  and  learning  of  other  types  of  knowledge  and  skills. 

One  of  the  most  frequently  discussed  problems  in  SE  Asia  is  the  unhappy  force-fed  pedagogy  in  which  many  teachers 
and  students  find  themselves  trapped  (e.g.  Zeng,  1999).  This  problem  is  complicated  by  a dearth  of  qualified  ESL 
teachers  and  a lack  of  time  and  resources  to  develop  effective  lessons.  Educational  theorists  generally  believe  the 
teaching  and  learning  practices  based  on  ‘teaching  to  the  exam’,  which  are  the  preferred  methods  in  many  public  and 
private  institutions  in  SE  Asia,  dampen  students’  desire  and  ability  to  learn  (e.g.  Paris,  1995).  However,  because  of  the 
face  validity  that  norm-referenced  standardized  assessment  promises,  these  practices  often  dominate  the  curriculum. 
While  this  problem  is  most  keenly  felt  at  primary  (!)  and  secondary  levels,  even  tertiary  education  is  not  free  of  it. 

One  of  the  shortcomings  of  these  practices  is  that  they  habitually  result  in  students  merely  learning  how  to  parrot 
formulae.  In  the  case  of  second  language  learning,  stock  phrases  taught  at  school  are  often  made  to  substitute  for 
grammatical  and  communicative  competence  in  the  target  language  (Milton,  2001).  This  practice  is  not  always  the  fault 
of  the  teachers,  who  are  often  victims  of  social  pressure  to  teach  test-taking  strategies  for  short-term  gains. 

One  proposal  for  dealing  with  the  problem 

The  author  was  commissioned  to  produce  online  ‘workplace  English’  courses  for  professionals,  and  ‘bridging’  courses 
for  secondary  school-leavers  who  have  been  out  of  school  for  a number  of  years  and  who  wish  to  upgrade  their  English 
or  to  enroll  in  tertiary  courses  taught  in  English.  These  students  are  mainly  Cantonese  and  Putonghua  speakers  at  a 
wide  range  of  English  proficiency  levels,  requiring  a broad  array  of  receptive  and  productive  English  skills. 

It  was  not  immediately  apparent  how  current  multimedia  technologies  could  be  deployed  quickly  and  cheaply  in  order 
to  develop  and  deliver  high-level  interactive  online  language  learning  content.  Despite  the  Web’s  potential  to  provide 
courses  on  demand,  most  content  delivery  systems  appear  to  offer  an  unsatisfactory  range  of  activity  types  and  learning 
options  for  dynamic  language  learning. 

It  seems  reasonable  to  expect  an  online  content  development  and  management  system  to  assist  materials  developers  and 
instructors  (who  often  have  limited  time  and  resources)  to  create  engaging  and  meaningful  communicative, 
collaborative  and  problem  solving  activities.  We  should  also  expect  the  courses  created  on  such  a system  to  follow 
generally  agreed  educational  principles  and  pressing  logistic  needs  by: 
o appealing  to  a wide  range  of  learner  proficiencies  and  interests; 
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o motivating  low-proficiency  students,  who  often  suffer  from  low  self-esteem,  to  engage  in  high-level 
interaction,  while  also  enabling  a transparent  system  of  student  and  teacher  accountability; 
o providing  quality,  individualized  instruction  to  a large  number  of  varied  learners; 

o ensuring  a coherent,  planned  and  progressive  curriculum,  and  the  transmission  of  core  information,  while  also 
providing  opportunities  for  self-discovery,  problem  solving,  participation  and  collaboration,  creativity  and  the 
pursuit  of  individual  learning  paths  (i.e.  aim  at  the  type  of  quality  assurance  outlined  by  Alley  and  Jansak, 
2001); 

o enabling  and  encouraging  the  teacher  to  act  more  often  as  a mentor,  and  less  often  as  a pedagogue; 
o integrating  intrinsic,  criterion-referenced  and  performance-oriented  formative  assessment  into  the  learning 
process,  thereby  minimizing  the  need  for  extrinsic,  norm-referenced  summative  assessment; 
o providing  quicker  and  more  reliable  reporting  to  the  instructor,  allow  for  easy,  consistent  grading,  and  also 
more  effective  and  quicker  feedback  to  learners;  and 
o allowing  ongoing  assessment  of  the  content  and  methodologies  by  the  students. 

It  was  crucial  that  this  online  content  development  system  allow  for  the  specific  pedagogical  needs  of  language  courses. 
This  meant  making  it  possible  for  students  who  may  have  little  perceived  opportunity  to  use  the  target  language  in  their 
daily  life  to  gain  accuracy  and  fluency  in  all  language  skills.  These  skills  include  authentic  listening,  speaking,  writing 
and  reading  activities. 

Internet  technologies  potentially  allow  for  a highly  interactive  educational  model  through  retrieval  of  online  data,  as 
well  as  through  enhanced  communication  among  students  and  with  the  instructor.  However,  while  access  to  expert 
support  is  important,  it  is  both  pedagogically  and  economically  counter-productive  to  base  online  courses  entirely  on  a 
teacher-led  model.  In  the  absence  of  reliable,  affordable  and  widely  available  Artificial  Intelligence  systems  for  human 
language  acquisition  and  production  (e.g.  natural  language  parsing),  this  requires  the  careful  preparation  of  objectively 
scored  activities  - ideally  aimed  at  ‘higher-order’  outcomes  (e.g,  Haladyna,  1997).  These  activities,  content  and 
technologies  must  of  course  be  sufficient  and  appropriate  to  meet  the  learners’  needs  and  learning  styles,  which  in  this 
case  are  extremely  varied. 

More  importantly,  an  effective  online  language-learning  framework  requires  the  development  of  collaborative  activities 
that  make  communicative  use  of  the  language  the  responsibility  of  the  students  themselves.  Various  studies  (e.g. 
Johnson,  1993)  suggest  that  a well -organized  learner-centered  curriculum,  especially  one  that  puts  extensive  resources 
at  the  disposal  of  teachers  and  students,  enhances  effective  mentoring  of  students  and  increases  the  accountability  of 
students  for  their  own  learning.  Students  in  an  online  environment  normally  produce  far  more  language  (in  this  case, 
both  writing  and  speech)  than  those  in  a traditional  class  (Selfe  & Cooper,  1990),  and  the  instructor  has  more  often  to 
play  a mentoring  than  the  conventional  classroom  pedagogical  role. 

Several  currently  available  course  delivery  systems  were  evaluated,  and  while  all  permit  basic  information  delivery  and 
assessment,  none  applied  the  range  of  mechanisms  for  language  learning  suggested  above.  Also,  most  systems  leave 
much  to  be  desired  in  the  ease  with  which  content  can  be  authored,  the  degree  to  which  individual  learners  can  be 
accommodated  and  social  interaction  encouraged,  Internet  technologies  continue  to  make  available  new  expanded 
forms  of  communication  and  participation,  but  these  new  technologies  often  take  considerable  time  to  be  integrated 
into  commercial  course  delivery  systems,  and  most  of  these  systems  do  not  allow  new  technologies  to  be  easily 
incorporated  on  an  ad  hoc  basis.  A careful  look  at  the  technologies  involved  suggests  that  - if  it  is  pedagogically 
justified  - database-driven  Internet  technologies  have  matured  to  the  extent  that  it  is  technically  and  economically 
feasible  to  develop  original,  a customizable  Web  system  with  the  features  outlined  above  that  can  be  used  to  create, 
deliver  and  manage  interactive,  individualized  online  content. 

Among  the  parameters  set  for  this  system  were  that  it  must  allow  courses  to  be  undertaken  through  a series  of 
sequenced  activities  and  units:  learners  are  generally  held  accountable  for  completing  a ‘unit’  within  a set  period.  It 
must  nevertheless  allow  for  streaming  so  that  more  or  less  proficient  and  independent  learners,  or  learners  with 
different  academic  or  job-related  needs,  can  pursue  different  learning  paths  and  activities  within  the  same  course.  It 
must  allow  the  creation  of  conventional  ‘objective’  activities  (i.e.  activities  that  can  be  automatically  graded,  such  as 
multiple  choice,  gap  fill,  drag-and-drop,  matching  etc.).  Less  conventionally,  it  must  also  allow  the  creation  of  open- 
ended  activities  beyond  standard  essay  submission  forms  and  threaded  text  discussions  - for  example,  sequenced 
access  by  learners  (via  ‘Voice  over  Internet  Protocol’  - VOIP  technologies)  to  asynchronous  and  synchronous  voice 
discussion  Web  boards.  The  remainder  of  this  paper  illustrates  yet  another,  even  less  conventional  open-ended  module 
implemented  in  an  online  English  course  currently  driven  by  this  system:  a module  that  learners  can  use  to  create  and 
post  animated  role-plays  on  the  Web. 
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Online  Role  Plays 

Oscar  Wilde  observed,  “Education  is  an  admirable  thing.  But  it  is  well  to  remember  from  time  to  time  that  nothing  that 
is  worth  knowing  can  be  taught.”  (1913).  How  to  make  known  the  unteachable  may  in  some  sense  be  answered  by 
Muriel  Rukeyser’s  1960  remark  that  “The  universe  is  made  of  stories,  not  of  atoms.”  Allowing  the  language  learner  to 
adopt  the  role  of  narrator  as  a means  of  acquiring  language  skills  is  particularly  relevant. 

While  not  negating  the  role  of  the  instructor,  most  academics  accept  that  ‘higher’  levels  of  learning,  such  as  ‘critical 
thinking’,  which  are  not  easily  taught  or  assessed,  are  the  preferred  outcomes  of  the  educational  process.  There  is 
justifiable  concern  whether  universities  are  currently  producing  students  capable  of  achieving  the  higher  order  learning 
outcomes  listed  in  Bloom’s  taxonomy  of  educational  objectives  (i.e.  recall  ->  comprehension  ->  application  ->  analysis 
->  synthesis  ->  evaluation).  One  term  often  used  by  linguists  to  describe  the  advanced  communicative  proficiency  that 
make  advanced  level  communication  possible  is  ‘pragmatic  competence’,  which  includes  the  ability  to  critically 
evaluate  what  is  said  or  written  and  to  respond  in  a contextually  appropriate  manner. 

One  way  such  outcomes  have  been  encouraged  is  through  role-playing  activities,  usually  face-to-face  or  at  a distance. 
These  are  used  in  many  disciplines,  with  various  online  projects  using  ‘virtual  worlds’  (e.g.  MUDs)  in  simulations  of 
business  or  academic  projects.  The  benefits  of  role-playing  for  generalized  educational  settings  are  detailed  in  Jones 
(1985),  and  specifically  for  language  learning  in  Ladousse  (1987),  Crookall  & Oxford  (1990)  and  Bambrough  (1994). 

In  the  course  delivery  system  developed  in  response  to  the  requirements  sketched  above,  students  use  an  online 
‘scripting  tool’  to  manipulate  animated  characters  on  screen,  assign  them  movements  and  gestures  (including  some 
body  language)  and  to  write  dialogue  that  is  synthesized  and  ‘spoken’  by  the  characters,  whose  mouths  move 
synchronously  with  the  dialogue.  The  technology  that  runs  the  animated  characters  (‘Microsoft  Agent’)  was  originally 
developed  to  give  a social  dimension  to  the  computer  interface. 

Via  this  technology,  students  can  listen  to  their  dialogues  (or  any  other  text  on  the  Web  page)  rendered  with  standard 
American  or  British  synthesized  pronunciation.  The  current  stage  of  development  in  freely  available  text -to-speech 
technology  is  such  that  the  synthesized  suprasegmentals  (e.g.  stress,  rhythm  and  intonation)  are  not  entirely  human,  but 
the  students  who  have  taken  the  course  (almost  300  to  date)  rarely  complain  about  the  quality  of  the  synthetic  voices. 
Indeed,  they  report  finding  the  relatively  high  accuracy  in  pronunciation  useful  (the  voice  synthesis  can  be  replaced  by 
a recording  of  a human  voice  if  the  student  or  instructor  wishes  - e.g.  where  prosodic  modeling  is  crucial).  This  activity 
has  the  benefit  of  allowing  teachers  and  students  to  script  and  edit  conversational  dialogue.  It  also  allows  the  instructor, 
who  has  access  to  the  written  script,  to  supply  detailed  written  feedback  on  the  accuracy,  fluency  and  effectiveness  of 
the  students’  dialogues.  Students  have  additional  opportunities  for  direct  practice  in  synchronous  and  asynchronous 
conversation  with  the  instructor  and  with  other  students  on  the  voice  discussion  boards  mentioned  above  (e.g. 
http://ihome.ust.hk/-^lciohn/Intemational  Voice. htni). 

Several  Second  Language  Acquisition  (SLA)  theorists  recommend  the  use  of  dramatization,  ‘actualization’  and 
narrative  techniques  for  effective  SLA,  regardless  of  the  age  of  the  learners  (e.g.  Oiler,  1993).  Scarcella  and  Crookall 
(1990)  also  review  research  to  show  how  simulation  facilitates  second  language  acquisition  by  giving  students  the 
opportunity  to  try  out  new  language  in  a safe  environment.  The  learning  theories  they  discuss  claim  that  learners 
acquire  language  when: 

o they  are  exposed  to  large  quantities  of  comprehensible  input  through  being  engaged  in  genuine 
communicatbn  as  part  of  the  roles  assigned  to  characters; 
o they  are  actively  involved  in  worthwhile,  absorbing  interaction,  which  tends  to  make  students  forget  they  are 
learning  a new  language;  and 
o they  experience  positive  feelings  and  attitudes. 

Such  studies  argue  that  the  dramatization  of  ‘real  life’  problems  helps  students  develop  their  critical  thinking  and 
language-related  problem  solving  skills.  Role-playing  would  certainly  appear  to  be  a useful  language  acquisition 
activity  to  the  extent  to  which  we  view  language  as  a vehicle  for  the  realization  of  interpersonal  relations  and  social 
transactions. 

The  particular  role-playing  activity  which  this  system  supports  allows  students  to  practice  and  demonstrate  ‘higher 
order’  linguistic  skills  such  as  the  pragmatic  devices  necessary  for  developing  a narration,  resolving  a conflict, 
conducting  a negotiation  etc.,  while  also  being  a useful  vehicle  for  practice  in  basic  grammar,  vocabulary  and 
conversational  language  devices.  Students  are  guided  in  commonly  avoided,  misused  or  overused  language  devices. 
The  activity  gives  students  the  opportunity  to  express  and  acquire  cultural  values  and  to  be  creative,  while  developing 
practical  language  strategies  for  handling  business  and  social  communication.  Those  who  are  normally  shy  to  speak 
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spontaneously  in  a classroom  out  of  concern  for  their  imperfect  English  appreciate  the  opportunity  to  practice  using 
the  language  in  this  context.  A serendipitous  benefit  of  the  activity  is  that  it  appears  to  give  students  who  have  limited 
technology  fluency  more  confidence  in  the  use  of  computer  applications  and  enhanced  Internet  skills.  Incidentally,  it 
also  appears  less  prone  to  plagiarism:  students  seem  less  inclined  or  able  to  copy  holophrastically  from  other  sources 
when  they  are  scripting  for  the  digital  puppets  - perhaps  because  the  activity  is  not  so  closely  associated  with 
academic  assessment  and  right/wrong  answers,  as  is  the  standard  essay  question. 

This  role-play  activity  (normally  assigned  once  in  each  unit  or  in  alternative  units)  was  designed  to  encourage  students 
to  work  coiiaboratively  at  a distance.  However,  while  role-playing  may  be  an  ideal  pailicipaioiy  excicisc  in  active 
learning,  this  ideal  cannot  be  fully  realized  if  students  find  it  difficult  to  schedule  online  meetings  in  order  to  co-design 
plays.  Most  students  who  have  taken  the  course  to  this  point  are  professionals  seeking  to  improve  their  English,  and 
most  have  chosen  not  to  collaborate  with  other  students  in  writing  their  role  -plays.  A number  of  students  do,  however, 
work  with  family  members  in  designing  and  scripting  the  plays.  Even  when  they  write  the  plays  alone,  they  socialize 
through  the  public  production  of  the  scenarios,  by  giving  and  receiving  classmates*  feedback,  and  through  the  virtual 
relationships  they  establish  among  the  animated  characters.  Several  of  the  higher-order  outcomes  that  are  claimed  for 
team-based  role-playing  activities  are  also  possible  in  the  creation  of  individually  written  role-plays,  such  as  increased 
confidence  in  language  production,  heightened  discriminatory  skills,  and  the  sheer  fun  that  comes  from  the  act  of 
personal  expression  and  creation.  Collaborative  elements  can  nevertheless  be  retained,  even  when  the  students  choose 
not  to  work  together.  In  addition  to  commenting  on  and  grading  each  other’s  plays  (by  assigning  ‘stars’),  students  often 
spontaneously  continue  narrative  lines  from  their  classmate’s  plays. 

A functional  thematic  context  (e.g.  telephone  skills,  encouraging  and  criticizing,  the  language  of  negotiation)  is 
developed  in  each  unit.  Also,  relevant  grammar  and  lexis,  along  with  the  features  of  spoken  language  (e.g.  stress, 
rhythm,  intonation  and  pronunciation),  are  provided  to  students  through  a variety  of  high-interest  interactive  activities 
(including  via  licensed  popular  songs  and  lyrics  streamed  from  our  server).  The  spoken  and  written  input  are  based  on 
extensive  research  into  the  language  learning  difficulties  of  the  students,  and  are  more  authentic  and  copious  than  can 
normally  be  made  available  in  the  classroom. 

Instructors  and  fellow  students  supply  feedback  to  each  play,  and  students  can  re-write  their  scripts.  Care  is  taken  to 
make  the  purpose  and  structure  of  the  role-plays  clear,  since  many  students  are  more  accustomed  to  conventional  essay 
assignments  than  they  are  to  communicative  methods  of  evaluation  (see  Li,  1999  for  a discussion  of  the  mismatch 
between  student  expectations  for  teacher-led  instruction  and  the  participatory  methods  often  preferred  by  western - 
educated  teachers). 

The  instructor  can  exemplify  a wide  range  of  language  features  in  communicative  contexts  with  this  technique, 
including  problematic  or  unknown  grammatical  structures  and  lexis.  Sbripts  written  by  learners  usually  illustrate 
surprisingly  authentic  (albeit  imperfect)  dialogue.  Learners  are  enabled  to  adopt  opposing  points  of  view  and  often 
make  an  effort  to  use  problematic  grammatical  structures  that  are  frequently  avoided  in  conventional  settings.  Their 
errors  provide  teaching  opportunities,  and  are  evidence  of  involvement  in  genuine  communicative  acts,  rather  than  the 
mere  completion  of  academic  exercises  in  error  avoidance. 

Conclusion 

The  system  outlined  in  this  paper  allows  language  teachers  and  course  writers  to  take  advantage  of  ongoing  advances  in 
Internet  technology  to  deliver  instruction  while  encouraging  participatory  learning  for  ‘higher  order’  outcomes.  The 
online  English  course  developed  on  this  system  seeks  to  address  cognitive  and  affective  learning  needs  and  allows 
students  to  pursue  their  own  language  learning  goals  free  of  many  of  the  logistic  limitations  of  the  classroom.  The 
specific  modality  highlighted  in  this  paper  combines  Internet  delivery  with  database  connection,  computer  animation 
and  speech  technology  to  enable  language  learners  to  experiment  with  manipulating  the  second  language  in  simulated 
and  supported  contexts. 

This  role-playing  exercise  is  one  of  the  most  popular  activities  in  the  current  online  English  course:  almost  all  students 
find  this  activity  ‘very  interesting’,  and  at  the  completion  of  the  course,  most  mention  it  as  one  of  their  favourite 
activities.  The  approximately  300  students  who  have  taken  the  course  so  far  showed  an  average  increase  of  35%  in  their 
listening  comprehension  and  24%  in  grammatical  accuracy  over  the  length  of  the  course,  based  on  standardized  pre- 
and  post-tests.^ 
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Changes  in  performance  of  productive  skills  (writing  and  speaking)  have  not  yet  been  measured. 
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There  are  a number  of  plans  to  enhance  the  use  of  the  narrative  and  dramatization  devices  discussed  here.  In  addition  to 
being  able  to  write  their  own  scripts,  students  should  be  given  more  control  over  the  narratives  provided  by  the  course 
author.  This  will  mean  building  in  various  forms  of  student -driven  controls,  including  possibly  voice  recognition,  and 
writing  branched  scripts  so  that  interactions  proceed  according  to  choices  made  by  the  student. 
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Abstract:This  paper  describes  a computer  based  English  exercise  system  through  pictures  using  TTS(  Text  to 
Speech  ).  With  the  help  of  TTS  technology,  this  system  can  provide  the  phonic  check  function  of  a learner’s 
input  answers  in  order  to  overcome  the  shortcoming  of  language  learning  coursewares  that  must  rely  on 
sound  files.  This  system  can  also  provide  check  functions  of  word  spelling  and  grammar  for  the  learner.  The 
TTS  technology  can  fully  display  the  advantage  of  computer  multimedia  teaching  and  realize  the  real 
computer  based  interactive  teaching. 


English  Exercise  System  Base  on  TTS 

In  an  English  lesson,  a teacher  often  draws  the  content  of  course  on  a picture  in  advance  and  asks  the 
students  some  questions  about  the  picture  in  classroom  This  kind  of  teaching  method  makes  a point  of  the 
student’s  participation  and  enhances  the  student’s  initiative  to  a certain  degree.  So  this  method  is  in  favour 
of  training  the  learner’s  ability  of  studying  the  foreign  language.  Different  learners  will  have  individual 
perceptions  as  well  as  different  language  foundations,  so  they  will  give  different  interpretations  of  the 
pictures  and  give  different  answers.  Because  of  the  restriction  of  course  time  and  the  teacher’  energy,  it  is 
impossible  for  the  teacher  to  check  all  answers.  Some  partial  outstanding  student's  reply  may  usually 
replace  the  entire  class  student's  reply,  some  partial  outstanding  student's  reply  may  usually  replace 
teacher's  explanation,  simultaneously  the  teacher's  answer  only  can  take  the  reference. 

Under  this  condition,  the  majority  of  students  may  be  occupied  in  the  passive  study  and  therefore  the 
student's  initiative  study  and  the  positive  thought  can  not  be  manifested  fully.  Therefore  it  is  very  important 
for  a language  courseware  to  provide  interactive  functions. 

In  every  kind  of  couresewares,  the  process  of  sound  mostly  adopts  sound  files,  namely  recording  sound 
of  teaching  contents  in  advance  to  become  the  sound  files,  and  plays  them  in  the  class.  The  learners  can’t 
usually  check  the  pronunciation  of  their  own  answers  and  it  is  also  impossible  in  advance  to  record  the 
sound  files  of  all  the  answers.  Regarding  any  language  practice  system,  it  is  inpossible  to  enumerate  all 
possible  answers,  but  if  it  only  stipulates  one  standard  answer,  it  will  not  conform  to  real  life  language.  So 
it  is  impossible  that  the  coursewares  to  provide  interaction  functions  for  the  learners.  The  TTS  technology 
solves  this  problem  in  computer-based  language  teaching. 

This  paper  designed  a computer-based  English  exercise  system  using  TTS.  This  system  takes  the 
advantage  of  computer  multimedia,  specially  the  vividness  and  interaction  of  computer  multimedia  for 
overcoming  the  shortcoming  of  the  traditional  teaching,  at  the  same  time  this  system  also  can  overcome  the 
shortcoming  of  the  usual  computer-based  language  exercise  systems  that  are  unable  to  pronounce  the 
learner’s  input  answer.  Therefore  this  system  has  the  ability  for  the  learner  to  feel  the  different 
pronunciation  between  the  different  sentences. 

The  system  is  effective  because  a learner  must  see  the  picture  using  his  eyes,  and  think  of  the  answer 
using  his  head,  then  type  the  answer  using  his  hand,  pronounce  the  answer  using  his  mouth  at  last.  All  of 
above  may  bring  the  learner’s  enthusiasm,  enhance  the  study  effect. 

Database  design 


Designing  a database  for  storing  all  the  picture  files  path  and  exercise  contents.  Furthermore,  all  the 
learner’s  information  is  stored  in  the  database. 
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Most  of  the  language  exercise  systems  use  database  to  store  the  datum.  This  paper  also  selects  database  to 
store  the  picture  file  path,  exercise  contents  and  the  learner’s  information  including  the  study  degree  of 
progress.  At  the  same  time  in  order  to  use  the  sound  files  provided  by  the  teachers,  the  function  is  set  for 
switching  TTS  and  sound  files  which  means  that  the  fields  of  question  sound  file  path  and  answer  sound 
file  path  are  established. 

Word  spelling  and  grammar  check 


Providing  the  functions  of  word  spelling  and  grammar  for  the  learner  to  check  the  input  answer  by  himself 
in  exercise. 

Because  the  self-practice  is  provided  for  the  learners,  it  is  very  important  to  provide  the  functions  for 
checking  word  spelling  and  grammar.  This  paper  uses  the  check  function  of  word  spelling  and  grammar  in 
Microsoft  WORD  for  providing  the  reference  about  word  spelling  and  grammar.  This  function  may  not 
only  let  the  learner  understand  the  possible  mistakes,  and  but  also  can  help  the  learner  to  understand  the 
related  knowledge,  similar  to  providing  a privately  owned  instructor.  Of  course,  one  question  can  proceed 
iteratively. 

Checking  standard  answer 

It  will  be  useful  for  the  learner  to  understand  the  study  contents  by  the  way  of  comparing  reference 
answers. 

Comparing  the  reference  answers  will  let  the  learners  understand  the  point  contents  of  the  exercise. 

Pronunciation  Using  TTS 

This  paper  selects  the  Microsoft's  Speech  API  (SAPI)  which  is  well  established  as  the  leading  platform 
for  adding  speech  recognition  and  text  to  speech  capabilities  to  desktop  PC  applications.  TTS  is  used  to 
convert  the  learner’s  answer  into  speech.  S imultaneously  when  a question  and  its  answer  does  not  have  the 
sound  files,  TTS  will  convert  them  into  speech.  This  enables  the  teacher  to  ask  ad  hoc  some  questions  in 
class  and  let  the  learners  practice  them  at  once. 

Conclusion 

In  brief,  the  union  of  computer  multimedia  technology  and  the  traditional  teaching  method  will  cause  the 
students  to  become  fully  involved.  Through  studying  the  scene  created  by  computer  multimedia,  the  study 
enthusiasm  of  students  is  stimulated,  the  thought  angle  will  be  opened  up  and  cause  the  student's  thought 
quality  to  take  a great  leap  forward.  The  stimulation  of  the  student's  thought  process  will  incur  an  avalanche 
effect  and  result  in  a very  high  role  of  learning  retention. 
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Abstract:  This  paper  describes  a method  for  supporting  multi-language  display  in  a web-based  Chinese 
learning  system  created  by  ASP(ActiveX  Server  Page).  The  proposed  method  uses  both  databases  and  ASP 
dictionary  objects  to  display  different  language  prompts  for  a learner  of  Chinese  with  different  mother 
languages.  With  multi-language  support,  the  developer  of  web-based  Chinese  learning  system  can  avoid 
massive  repeated  development.  The  proposed  method  based  on  database  and  ASP  technology  provides  one 
extremely  good  method  for  realizing  tbe  expansion  of  tbe  web-based  language  learning  system  from  one 
language  to  other  languages. 

Chinese  Learning  System  Supporting  Multi-Languages 


When  a web-based  Chinese  learning  system  is  designed,  Chinese  character  must  be  displayed  in  all  the 
web-pages.  Meanwhile  it  must  be  considered  that  the  learner  may  come  from  different  countries.  It  means 
the  mother  language  will  be  different.  For  a Japanese,  Japanese  will  be  also  displayed  but  for  an  American, 
English  will  be  displayed  in  the  same  web  pages.  So  it  is  necessary  for  a web-based  Chinese  Learning 
System  to  support  multi-language  display.  This  question  may  be  considered  as  using  some  method  to 
realize  fast  transformation  of  the  page  contents  by  obtaining  the  suitable  data  from  a database  and  then 
display  the  data  using  the  ASP  dynamic  demonstration.  All  the  text  and  image  files  of  different  languages  in 
the  web  site  are  stored  in  the  database.  All  the  pages  camy  a variable  to  identify  which  language  the  site  is 
also  to  be  displayed  except  Chinese.  Based  on  the  language  of  every  learner’s  browser,  the  content  is  pulled 
out  from  the  respective  tables  for  the  language  chosen,  and  displayed. 

Choosing  Character  Encoding 

The  first  thing  is  decision  of  character  encoding  when  designing  a web  site.  For  a web  site  of  Chinese 
learning  system,  it  needs  to  use  two  languages  on  a web  page.  The  web  page  might  include  scripts  with 
both  Japanese  and  Chinese  string  litera  Is.  In  these  languages,  the  web  pages  may  also  have  static  HTML 
segments  and  dynamic  information  acquired  from  a database.  For  instance,  if  Chinese  is  declared,  then  the 
Japanese  text  will  be  misinterpreted.  So  this  paper  declared  UTF-8  which  supports  many  languages  as 
character  encoding  to  format  all  pages. 

Design  Text  and  File  Path  Database 

When  the  browser  language  changes,  it  is  hoped  that  there  is  no  need  to  modify  the  ASP  pages  but  only 
add  data  to  database.  For  example,  if  the  prompt  text  in  English  is  ”yes”,  then  when  the  browser  language 
changes  to  Japanese,  the  word  "yes”  will  be  altered  to  Japanese  word  ” ,t,0  ” meaning  yes  in  Japanese. 
Hopefully,  no  modification  of  any  ASP  pages  is  needed.  Based  on  above  requirement,  the  database  is 
designed  as  below. 

Two  tables  are  created  for  saving  the  prompt  text.  One  table  is  common  strings  to  save  the  repetitious 
words  and  phrases.  Moreover  one  table  is  specialized  strings  to  save  the  words  and  phrases  only  used  in  a 
specific  page.  It  has  advantages  that  the  standard  maintenance  is  simple  to  store  the  public  text  and  the 
specific  text  separately.  The  tables  use  both  the  field  string  name  and  the  filed  language  code  as  the  key 
filed.  The  field  string  name  and  the  filed  language  code  are  united  to  create  the  main  index.  The  field 
display  string  is  used  to  preserve  the  word,  the  phrase  or  the  sentence.  Only  one  index  is  created  for  all 
records  because  all  the  repetitious  input  of  a word,  phrase  or  sentence  must  be  avoided. 

Regarding  specialized  string  table,  it  needs  an  extra  field  for  the  connection  with  the  common  string  table. 
It  must  be  considered  that  not  all  data  can  be  stored  in  the  dictionary  object,  in  fact  most  of  them  are 
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retrieved  from  database  when  needed.  So  a special  string  table  is  needed  to  save  these  data.  Only  the  data  in 
common  string  table  is  loaded  to  the  dictionary  object  because  these  are  not  necessary  to  change  frequently. 
The  fields  of  the  tables  are  showed  as  below. 

Regarding  the  picture  files,  the  table  can  be  created  with  the  same  step  described  above  to  save  the  file 
path.  Only  replace  the  field  display  string  in  the  common  image  table  and  specialized  image  table  using  the 
filed  of  file  name. 

Read  Data  from  Database  to  Dictionary  Object 

The  Dictionary  object  in  ASP  is  a data  structure  that  can  contain  sets  of  pairs,  where  each  pair  consists  of 
an  item,  which  can  be  any  data  type,  and  a key,  which  is  a unique  String  value  that  identifies  the  item.  The 
values  can  be  extracted  from  a Dictionary  object  only  simply  placing  the  object.key  identifier  on  the  right 
side  of  the  assignment  statement.  Such  an  assignment  returns  the  value  of  the  referenced  key.  The 
dictionary  object  can  be  set  as  global  action  for  the  entire  application  scope.  Bu  it  is  no  need  to  worry  the 
loss  performance. 

After  the  connection  to  a database  is  created,  the  data  in  database  will  be  received  and  loaded  to  the 
dictionary  object.  The  language  code  and  string  name  are  united  together  as  a dictionary  key  and  the  value 
of  field  displaystring  is  set  as  the  key’s  value.  The  result  is  to  put  the  value  of  the  field  display  string  to  the 
corresponding  dictionary  data  area.  Therefore  good  performance  can  be  obtained,  because  the  lasting 
database  connection  is  avoided  that  will  influence  the  performance.  In  fact,  database  is  only  connected  one 
time  when  application  start,  after  that  the  ASP  page  will  read  the  necessary  data  from  the  dictionary  object. 

Quotation  of  Text  and  File  Path  in  ASP  Page 

After  the  data  including  text  and  file  path  are  stored  to  the  dictionary  object,  they  can  be  quoted  in  the 
ASP  pages.  But  at  first  it  is  needed  to  know  the  language  of  the  current  user’  browser.  It  can  be  obtained 
the  variables  of  the  request  object.  The  text  or  picture  file  path  saved  in  the  dictionary  object  nay  be 
obtained  through  the  corresponding  key  value,  the  key  value  is  composed  of  the  name  and  the  language 
prefix.  According  to  experience,  there  is  no  disadvantage  caused  by  quoting  data  from  the  dictionary  object 
instead  of  the  input  data  directly.  This  is  because  the  text  or  the  picture  file  path  which  is  stored  in  the 
dictionary  object  is  buffered  by  the  server. 

Example/Discussion 

The  proposed  method  is  evaluated  using  a web  based  Chinese  exam  system.  This  system  allows  the 
examiners  with  different  mother  languages  join  the  test  at  same  time.  When  an  examiner  accesses  the 
system,  the  web  server  obtains  the  language  information  of  browser,  then  retrieves  data  from  the  common 
string  table.  For  the  special  string  table,  the  language  code,  common  string  name  and  string  name  are  united 
as  key  to  retrieve  data  from  the  special  table.  For  example,  if  the  language  code  is  en  and  the  common 
string  name  is  section!  and  the  current  string  name  is  point!  then  the  content(word)  stored  in  the  special 
table  will  be  displayed  in  the  web  page.  So  there  is  no  need  to  modify  any  place  in  the  web  page,  it  can  be 
used  for  different  mother  language  examiners. 

Certainly,  the  realization  of  multi-languages  support  has  more  other  factors  to  be  considered.  When  an 
education  system  is  designed,  it  is  very  important  that  good  design  is  useful  both  for  the  user  and  the 
developer.  In  this  paper,  how  to  set  the  isolation  between  the  content  the  change  and  the  development  is 
explained.  This  plan  may  not  only  regard  Chinese  leamhg  system,  but  may  be  useful  for  orther  learning 
systems.  The  database  and  the  dictionary  object  as  well  as  the  ASP  technology  provided  one  extremely 
good  realizations  from  one  language  to  other  languages  in  web  based  learning  systems. 
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Decisions,  Decisions,  Decisions.. .how  to  approach  the  impiementation  of  a distance  iearning 
program. 

Description: 

The  rush  to  develop  online,  telecommunicated  and  hybrid  degree  and  certificate  programs  has  left  a 
trail  of  failed  programs,  bankrupted  corporations  and  disillusioned  individuals  both  academically  and 
commercially.  Still  drawn  to  the  promise  of  distance  education  institutions,  administrators  and  faculty 
alike,  are  taking  a second  look  and  attempting  to  learn  from  the  successes  and  failures  of  others.  This 
panel  discussion  aims  to  stimulate  thinking  in  the  interest  of  ensuring  that  institutions  interested  in 
distance  education  are  reminded  of  what  to  consider  and  how  to  plan  for  program  development  and 
implementation.  Three  experienced  distance  learning  creators  will  put  it  all  on  the  table  with  lessons 
learned  from  the  implementation  of  both  online  and  telecommunicated  degree  programs. 

This  panel  will  present  and  answer  questions  covering  a broad  range  of  topics  from  planning  within 
campus  wide  systems,  choosing  technology,  instructional  design  and  faculty  support,  intellectual 
property,  marketing  and  student  services.  A general  outline  of  what  will  be  covered  in  each  topic  area 
is  offered  below: 


Interactive  Video,  Synchronous 
Programs:  Tony  Klejna 

Online,  A-synchronous 
Programs:  Melissa  Miszkiewicz 

Marketing  & Student  Services: 
Christine  Chelus 

■ Lessons  Learned  for 
Instructors  & Students 

■ Basics  of  Presentation 
Design 

■ Student  Involvement 

■ Bringing  Unity  to  the 
Force  - Ownership, 
Copyright,  Faculty 
Issues 

■ Extending  your 
interactive  sessions. 

■ Planning  at  the 
Department  level 

■ Planning  at  the  Program 
level 

■ Instructional  Design 

■ Marketing  & Recruitment 

■ Specialist  or  Generalist: 
How  do  you  set  up 
relationships  with 
specialists  in  your 
institution  to 
accommodate  these 
needs  for  students  at  a 
distance? 

■ What  are  the  pros  and 
cons  of  offering  full 
degree  programs  using 
DL  technology? 

■ How  do  you  choose  the 
right  program,  certificate 
or  course  to  offer  via  DL 
technology? 

Detailed  descriptions  from  each  presenter  appear  on  the  next  page. 


Ed  Media  2002 

World  Conference  on  Educational  Multimedia,  Hypermedia  & Teiecommunications 

Panel  Proposal:  Decisions,  Decisions,  Decisions. ..how  to  approach  the  impiementation  of  a distance  learning 

program. 
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Detailed  Presentation  Information 

Interactive  Video,  Synchronous  Programs;  Tony  Klejna 

■ Lessons  Learned  for  Instructors  & Students 

o Anticipate  time/distance/connection  issues 
o I’m  not  technical,  where  can  I get  help? 
o Why  Should  I Change?  You  Mean  I’m  Not  Already  Interactive? 
o Developing  Your  Knowledge  Base  of  Teaching  Techniques 
o New  Ways  to  Package  Your  Material 
o Practice  Makes  Permanent,  for  better  or  worse 
o Instructor  Dynamics,  Flexibility,  and  Style 
o Evaluation  and  Redesign 

o Why  Should  We  Even  Consider  Interactive  Video? 

■ Basics  of  Presentation  Design 

o Is  Interactive  Video  Fundamentally  Different  From  What  I Do  in  the  Classroom? 
o Less  Is  More,  Keep  It  Simple 

o Why  Fonts,  Layouts,  and  Styles  Are  Critical  in  Interactive  Video 
o How  Long  Should  I Be  On-screen? 

o Graphics  and  Animations-  The  Good,  the  Bad,  and  the  Really  Ugly 

■ Student  Involvement 

o “Am  I Less  Boring  By  Using  Technology?” 
o Why  Room  Design  and  Setup  Matter 
o Should  I Tell  Students  My  Plan  of  Action? 
o How  to  Create  a “Single”  Class 
o “Hello  Out  There?”  - Names  and  Faces 
o Can  I Visit  an  End  Site? 

o Collaboration  Across  the  Network  - Opportunities  for  Students  and  Instructors 
o Developing  Technology  Savvy  Students 


■ Bringing  Unity  to  the  Force  -Ownership,  Copyright,  Faculty  Issues 

o What’s  Yours  Is  Mine,  and  I Might  Not  Pay  You  For  It! 
o The  “Ownership”  Committee,  the  Dean,  and  the  Lawyer 
o Strategies  for  Equitable  Course  Ownership  and  Delivery 

■ Extending  Your  Interactive  Sessions 

o Generating  Live  and  Archival  Sessions 
o Generating  Narrated  Presentations 
o Hardware/  Software  Products  and  Considerations 
o Hard  and  Softcopy  Distribution 


Ed  Media  2002 

World  Conference  on  Educational  Multimedia,  Hypermedia  & Telecommunications 

Panel  Proposal:  Decisions,  Decisions,  Decisions... how  to  approach  the  implementation  of  a distance  learning 

program. 

p.3 
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Online,  A-synchronous  Programs  - Lesson  Learned:  Melissa  Miszkiewicz 

■ Planning  at  the  Department  Level  - Why  create  an  online  program? 

o Find  your  niche 

o Set  realistic  goals  and  desired  outcomes 
o Consider  geography 
o Consider  faculty  issues: 

■ Time 

■ Teaching  load  and  other  responsibilities 

■ Compensation 

■ Intellectural  property 

■ Sharing  the  burden:  teaching  courses  designed  by  others 

■ Faculty  support  at  the  planning  level:  instructional  design,  training 

■ Faculty  support  at  the  course  level:  teaching  assistants 

■ Planning  at  the  Program  level 

o Choosing  your  platform 
o Identifying  resources 
o Technical  support 
o Technical  requirements 
o Logistics 

o Supporting  Documentation  and  redundancy 
o Instructional  design 

■ Faculty  values 

■ Classroom  techniques  vs.  online  techniques 

• Instructor  presence 

• Learning  styles  and  proof  of  mastery 

• Structure  and  time  on  task 

■ Organization 

■ Choosing  appropriate  technology,  or  choosing  not  to  choose 

■ Identifying  teaching  resources 

■ Creating  for  the  future:  reusable  learning  objects 

■ Team  work 

■ Formative  evaluation  plans:  responding  to  constructive  criticism 

■ Documents  for  student  support 

■ Rules  for  student  participation 

Marketing  & Student  Services  - Lessons  Learned:  Christine  Chelus 

■ Marketing  & Recruitment 

o How  do  I reach  my  intended  audience?  - If  my  targeted  audience  is  500  miles 
away,  why  would  they  pick  up  a publication  with  my  institutions  name  on  it? 
o Moving  beyond  parochial  views  of  OUTREACH. 

o What  are  your  boundaries,  how  do  you  fill  the  gap  for  other  student  services 
offices  and  faculty  members  in  the  recruitment  process? 
o How  do  I disseminate  important  information  quickly  when  my  students  are 
located  so  far  away? 

■ Specialist  or  Generalist 

Transferring  a call  is  not  an  option  with  Distance  Learning  students,  now  that  you  have 
your  distance-learning  students,  how  will  they: 

o avoid  fees  imposed  by  due  to  policies  and  procedures  originally  established  for 
on-campus  students; 
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o register  for  courses; 

o access  journals  and  books  to  complete  assignments: 
o buy  books  for  their  courses; 

o pay  their  bill;understand  and  complete  financial  aid  forms;receive  advisement 
and  counseling; 

o turn  in  assignments  and  projects; 
o feel  like  an  authentic  student  of  your  institution: 
o contact  you  toil  free; 

How  do  you  set  up  relationships  with  specialists  in  your  institution  to  accommodate  these  needs  for 
students  at  a distance? 

■ Programs,  Certificates  and  Courses 

o What  are  the  pros  and  cons  of  offering  full  degree  programs  using 
distance  learning  technology  for  the  student  and  the  institution? 
o How  do  you  choose  the  right  program,  certificate  or  course  to  offer  via  distance 
learning  technology? 


Presenter  bios  follow  on  attached  pages 


Phone:716-839-8571 


Anthony  J.  Klejna 

Daemen  College 
Amherst,  New  York  14226 
FAX:  716-839-8261  E-mail:  tklejna@daemen.edu 


Director  of  Distance  Learning  and  Continuing  Education 
DAEMEN  COLLEGE 
Amherst,  New  York 

Implementation  of  distributed  learning  systems  and  programs  offered  through  interactive 
video,  on-line  Internet  based,  and  residential  delivery  systems.  Guide  college  utilization  of 
multimedia  and  emerging  technologies,  including  streaming  media  and  wireless  delivery,  to 
provide  synchronous  and  asynchronous  learning  opportunities.  Faculty  support  and  training 
in  distance  learning  and  technology  integration  into  the  curriculum. 

Presenter  and  Trainer  - ED-MEDIA  2002,  Denver,  Colorado  - The  World  Conference  on 
Educational  Multin^edia,  Hypermedia  & Telecommunications  - Streaming  Media 2002  -Media for 

All  - development  of  streaming  multimedia  assets,  various  methods  of  creating  content,  conversion  to 
appropriate  format,  and  broadcasting.  Developing  media  for  lo\/v-bandwidth  handheld  computing 
applications  and  demonstration  of  high-performance  videoconferencing  using  Internet  2. 

Presenter  and  Trainer  - ED-MEDIA  2001,  Tampere,  Finland  - The  World  Conference  on 
Educational  Multimedia,  Hypermedia  & Telecommunications  - STREAMING  MEDIA  2001  - A 
Media  odyssey  -developing  and  integrating  media  assets  and  instructional  technologies. 

Accepted  Presenter-  CIT2001  - The  10th  Annual  SUNY  Conference  on  Instructional 
Technologies,  presentation  - Developing  Streaming  Media  in  Educational  Applications 
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Presenter  and  Trainer  - ED-MEDIA  2000,  Montreal,  Quebec,  Canada  - The  World  Conference  on 
Educational  Multimedia,  Hypermedia  & Telecommunications  - creating  on-line  media  content, 
presentations,  audio,  video,  and  live  broadcasts. 

Presenter  and  Trainer  ICCE  99,  Chiba,  Japan  - 7th  International  Conference  on  Computers  in 
Education,  “New  Human  Abilities  for  the  Networked  Society”.  Adding  dynamic,  multimedia 
content  to  educational  website  development  and  distance  learning  applications. 

Presenter  and  Trainer  - ICCE  98,  Beijing,  China  - The  Sixth  International  Conference  on  Computers 
in  Education.  The  conference  focus,  "Global  Education  on  the  Net"  served  to  foster  the  creation 
and  dissemination  of  knowledge  about  the  use  of  information  technology  in  education.  Presentations 
and  training  sessions  on  basic  and  advanced  educational  website  development,  electronic  curriculum 
content  distribution,  and  distance  education. 


Christine  Chelus  holds  a B.S.  in  Sociology,  a M.S.  in  Student  Personnel  Administration  and  is 
currently  A.B.D.  in  the  Department  of  Communication  at  the  University  at  Buffalo.  Christine  was 
recently  selected  as  the  sole  recipient  of  the  The  Magness  Institute’s  research  grant  program  for  2002 
and  will  receive  funding  for  her  dissertation  research  regarding  educational  access  to  the  Internet. 
Christine  also  earned  her  distance  learning  professional  certification  from  Texas  A&M  University. 

Christine  has  worked  for  the  State  University  of  New  York  since  1996  holding  numerous  positions  at 
Buffalo  State  College  and  the  University  at  Buffalo.  At  Buffalo  State,  Christine  assisted  with  student 
recruitment  for  the  Office  of  Graduate  Studies.  She  also  served  as  program  coordinator  for  the 
Center  for  Applied  Research  in  Interactive  Technologies.  In  this  role,  Christine  provided  training, 
marketing,  student  services  and  facilitation  for  over  30  distance  learning  courses  annually. 

Upon  transferring  to  the  University  at  Buffalo,  Christine  served  as  the  primary  international  education 
contact  for  the  Center  for  Applied  Technologies  in  Education  and  held  the  positions  of  Manager  of 
Technology  Application  Development  and  Director  of  New  Media  Projects.  In  these  roles  she  served 
as  Project  Director  for: 

Buffalo  CityNet-  a 20  site  interactive  video  network  comprised  of  educational, 
government  and  community  agencies.  Project  coordination  included  consulting 
with  various  community  agencies  to  develop  educational  programming  for  the 
video  network. 

Taking  Gombe  to  the  World  through  Technology  - partnership  project  between  UB  and 
the  Jane  Goodall  Institute  to  connect  their  global  Roots  & Shoots  teams  through 
videoconferencing  (HOPENet)  and  to  extend  Dr.  Goodall’s  impact  on  people,  animals,  and 
the  environment  through  an  interactive  educational  web-based  curriculum  (Lessons  For 
Hope)  based  on  her  research. 

Project  Loop  - partnership  project  with  CRUSA  (Costa  Rica  - USA  Foundation) 
to  use  technological  resources  to  provide  citizens  in  geographically  isolated  areas 
access  to  qualified  instructors  regardless  of  their  socioeconomic  status.  Provide 
students  in  Costa  Rica  and  Western  New  York  the  opportunity  to  interact  with 
people  and  situations  of  different  cultures. 

New  Media  Project  - partnership  project  with  the  Office  of  the  Provost,  the 
Graduate  School  of  Education  and  the  College  of  Arts  & Sciences  to  provide 
educators  in  the  outlying  areas  of  New  York  State  access  to  University  at  Buffalo 
graduate  programs. 
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Christine  has  presented  and  worked  with  groups  on  educational  technology  material  regionally, 
nationally  and  internationally.  In  addition  to  working  on  site  in  Costa  Rica,  Panama,  England,  and 
Germany,  through  videoconferencing  technology,  she  has  worked  with  educators  around  the  world 
including  Mexico,  Tanzania,  South  Africa,  Columbia,  Guatemala,  Argentina,  France,  and  Japan. 

Currently,  Christine  serves  on  the  following  committees:  New  York  State  Evaluating  Technology  in 
Education  Team,  New  York  State  Distance  Learning  Consortium,  Western  New  York  Distance 
Learning  Consortium,  Western  New  York  Higher  Education  Distance  Learning  Consortium,  and  the 
Blackboard  Advisory  Committee. 

Melissa  J.  Miszkiewicz  holds  a B.S.  in  English  Literature  from  the  University  at  Buffalo  and  an  M.S.  in 
Curriculum  Development  and  Instructional  Technology  from  the  University  at  Albany.  She  is  a 
member  of  several  educational  associations  including  the  Association  for  the  Advancement  of 
Computing  in  Education  (AACE),  the  Association  for  Educational  Communications  and  Technology 
(AECT)  and  the  American  Dental  Educators  Association  where  she  participates  in  special  interest 
groups  in  Academic  Affairs  and  Informatics. 

Melissa  has  worked  for  the  State  University  of  New  York  since  1991  in  various  roles,  beginning  her 
career  in  higher  education  at  Empire  State  College  where  she  successfully  started  up  operations  for 
the  college’s  Workforce  Development  wing  in  the  Western  region  of  New  York  State.  There  she 
facilitated  all  aspects  of  new  program  development  and  delivery  including  conceptualization,  proposal 
development,  contract  negotiation,  staffing,  and  maintenance. 

Melissa  moved  to  the  University  at  Buffalo  to  start  up  the  online  MBA  program  in  the  University’s 
School  of  Management.  As  director  of  the  online  MBA  program  Melissa  collaborated  with  all  areas  of 
the  university  to  uncover  and  develop  faculty  resources,  rework  University  systems  and  policies  to 
accommodate  the  new  program,  develop  inter-department  partnerships,  and  to  develop  a policy  on 
intellectual  property  and  course  ownership.  In  this  position  Melissa  successfully  selected 
technologies,  developed  strategies  for  instructional  design,  and  a five-stage  plan  for  formative 
evaluation  culminating  in  the  development  of  six  complete  online  courses.  Formative  evaluation  and 
testing  took  place  overseas  with  the  school’s  international  programs  in  Singapore,  and  on  campus 
with  traditional  MBA  students.  Two  courses  were  delivered  to  students  in  the  fall  of  2001  with  great 
success. 

Melissa  has  recently  accepted  a position  with  the  School  of  Dental  Medicine  as  Director  of  Academic 
Services.  In  this  position  she  has  been  asked  to  develop  the  school’s  electronic  curriculum  by 
integrating  appropriate  tools  and  technologies,  applying  appropriate  methods  of  evaluation,  and 
working  directly  with  faculty  on  pedagogical  issues.  The  University  at  Buffalo  School  of  Dental 
Medicine  is  one  of  five  Dental  School’s  across  the  nation  to  have  adopted  an  electronic  curriculum 
and  one  of  two  to  first  implement  the  curriculum. 

In  addition  to  her  formal  duties  in  the  School  of  Dental  Medicine,  Melissa  is  currently  a consultant  to 
the  University  at  Buffalo  School  of  Nursing  for  the  development  of  an  Acute  Care  Nurse  Practitioner 
program  that  will  be  delivered  online.  She  has  also  presented  and  worked  with  groups  on  educational 
technology  and,  while  with  the  School  of  Management,  routinely  met  and  provided  guidance  to 
academic  and  technical  officials  from  China,  Russia,  Brazil,  the  World  Trade  Centers  Association  and 
the  Singapore  Institute  of  Management. 
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The  impact  of  “communicative  effectiveness”  of  computers  in  use  of 
computers  for  communication 


Ananda  Mitra,  Ph.D. 
Wake  Forest  University 
Department  of  Communication 
Winston-Salem,  NC  27109 
ananda@wfu.edu 


Abstract:  This  paper  uses  the  data  from  a longitudinal  study  to  demonstrate  that  the 
perceptions  of  the  communicative  effectiveness  of  a computer  system  has  a 
significant  impact  on  the  desire  to  use  computer  for  communication.  The  results 
suggest  that  it  is  important  to  attend  to  the  perceptions  when  introducing  the  use  f 
computers  for  pedagogic  communication. 


Introduction 

One  of  the  key  questions  that  have  been  raised  about  the  increasing  use  of  computers  and  digital 
technology  in  the  educational  setting  is  the  way  in  which  students  tend  to  use  the  computer  when  they  are 
provided  with  ubiquitous  computing  opportunities  (see,  e.g.,  Mitra  and  Hullett,  1997).  Studies  in  the  past 
have  demonstrated  that  usage  does  vary  with  different  demographic  categories  just  as  the  very  definition  of 
usage  remains  a contested  construct.  Indeed,  over  the  time  that  computers  have  entered  the  pedagogic 
arena,  the  idea  of  usage  has  become  increasingly  complex  as  the  term  “computer  use”  has  become  multi- 
dimensional. Past  research  has  demonstrated  that  the  computer  usage  can  be  considered  from  various 
perspectives  such  as  those  of  use  for  specific  pedagogic  tasks  to  those  of  communication  (Oblinger  and 
Rush,  1997). 

There  is  a growing  body  of  literature  that  would  suggest  that  the  computer  has  taken  on  a very 
critical  role  as  a tool  for  communication  as  the  digital  technologies  of  electronic  mail,  instant  messaging 
and  the  Internet  in  general  have  opened  up  opportunities  of  enhanced  communication  options  between 
students,  teachers  and  institutions  (Mitra  and  Hazen,  1999).  There  has  also  been  a body  of  literature  that 
has  suggested  that  computer  use  could  be  determined,  in  part,  by  specific  attributes  of  users  and  user  groups 
who  share  certain  attitudinal  and  demographic  similarities.  This  relationship  is  particularly  important  to 
explore  because,  as  a corollary,  it  suggests  that  knowing  certain  characteristics  of  the  user,  it  might  be 
possible  to  predict  the  usage  patterns  of  an  individual  or  group  (Woodrow,  1994). 

Such  studies  on  profiling  of  users  have  typically  used  two  different  metrices.  On  one  hand,  users 
have  been  classified  on  the  basis  of  demographic  characteristics,  such  as  those  of  age  and  gender.  Past 
studies  have  sometimes  indicated  that  men  and  women  have  different  patterns  of  computer  use  (Mitra,  et. 
al.,  2000a).  Similarly,  studies  on  age  categories  within  a general  population  of  computer  users  have 
demonstrated  that  usage  trends  could  be  determined  by  the  age  of  the  user  (Honey,  et.  al.,  2000).  A second 
relational  criteria  that  has  been  mobilized  to  understand  and  predict  usage  deals  with  attitudes  of  the  users. 
Personal  opinions  towards  computers  and  computing  have  known  to  have  an  influence  on  the  way  in 
computers  are  used.  In  many  studies  it  has  been  demonstrated  that  those  with  a positive  attitude  towaids 
computing  can  have  a more  positive  use  valence  as  well  (Mitra  and  Stefensmeier,  2000;  Mitra  et.  al.,  1999). 

As  suggested  earlier,  there  are  several  different  ways  to  measure  attitudes  towards  computers.  It 
has  recently  been  suggested  that  there  are  several  different  ways  in  which  different  attributes  of  attitudes 
can  be  measured  and  one  of  those  attributes  could  be  the  perceptions  of  the  communicative  effectiveness  of 
computers  (Mitra,  2001).  Given  the  relationship  between  attitudes  towards  computers  and  the  use  of 
computers,  it  can  be  speculated  that  those  with  a positive  attitude  towards  the  communicative  effectiveness 
of  computers  would  possibly  use  the  computers  for  communication  purposes  as  well.  In  this  paper  the 
communicative  effectiveness  factor  is  used  as  the  independent  variable  to  test  whether  differences  in  the 
attitude  level  actually  reflect  statistically  significant  differences  in  the  communicative  use  of  computers  as 
well. 
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Methodology  and  Data 


This  paper  utilizes  the  data  from  a longitudinal  study  to  assess  the  impact  of  technology 
introduction  in  liberal  arts  education  at  a non-technical  small  private  university  in  the  United  States.  The 
data  was  collected  over  four  academic  years  and  thispo^'^  hoc  analysis  uses  the  data  collected  in  the  spring 
of  1999  from  a sample  of  students  who  responded  to  a questionnaire  mailed  to  them.  The  questionnaire 
was  developed  by  conducting  a series  of  focus  group  meetings  that  explored  the  various  technology-related 
issues  that  the  students  were  facing  with  the  universal  and  standardized  introduction  of  technology  in 
teaching.  The  questionnaire  included  nearly  fifty  attitude  items  measured  on  a five-point  Lickert-type  scale. 
Of  these  fifty,  there  were  several  items  that  were  specific  to  the  local  conditions  of  the  university.  The 
relevant  items  from  the  questionnaire  have  been  selected  to  measure  attitude  and  usage. 

Instrumentation 

The  items  used  to  measure  communicative  effectiveness  is  based  upon  four  questions  that  have 
been  found  to  be  reliable  measurements  of  this  particular  attitude.  The  measurement  of  use  level  is  based 
upon  self-reported  frequency  of  use  of  the  computer  for  specific  tasks.  Using  the  work  of  Mitra  (2001),  the 
attitude  item  is  measured  on  a five-point  Lickert-type  scale  while  the  use  level  is  measured  on  a four-point 
interval  scale  as  well.  The  items  used  to  measure  attitude  and  usage  have  all  been  tested  previously  to 
demonstrate  that  they  are  reliable  measures  of  the  constructs. 

The  specific  items  used  to  measure  communicative  effectiveness  included: 

□ Computers  are  effective  for  communicating  with  other  students  about  class  related  work 

□ Computers  are  effective  for  communicating  with  faculty  about  class  related  work 

□ Computers  are  effective  for  communicating  with  other  students  about  non-course  related  subjects 

□ Computers  are  effective  for  communicating  with  faculty  about  non-course  related  topics 

Thus  communicative  effectiveness  includes  attributes  that  focus  primarily  on  the  role  of  the  computer  in  the 
pragmatic  act  of  communicating.  It  can  be  argued  that  a change  in  the  perception  of  the  effectiveness  of 
the  computer  in  facilitating  communication  could  have  an  impact  on  the  use  of  computers  for 
communication  and  eventual  learning  outcomes. 

The  specific  items  used  to  measure  the  use  of  computers  for  communication  included  questions 
about  the  frequency  of  use  of: 

• Use  the  Internet  to  get  information 

• Electronic  mail  with  teachers 

• Electronic  mail  with  friends 

• Electronic  mail  with  family 

• Electronic  mail  with  classmates 

• Electronic  mail  with  students  organizations 

All  of  these  items  test  the  way  in  which  the  computer  can  become  an  effective  tool  for  communicating  with 
a variety  of  groups  that  a student  can  be  involved  with  in  the  academic  setting. 

Analysis  and  Results 

As  a first  step  the  attitude  items  were  summed  to  create  an  overall  communicative  effectiveness 
measure.  The  item  was  then  dichotomized  into  a high  and  low  level  creating  to  nearly  similar  sized  groups. 
Tests  of  difference  in  mean  level  of  use  was  then  conducted  between  the  groups  using  an  independent 
sample  t-test  at  a level  of  p<0.05  to  identify  if  there  were  significant  differences  in  mean  usage  for  the  low 
and  high  attitude  groups.  The  results  are  presented  in  Table  1 . 
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Table  1 : Computer  use  level  based  on  High  and  Low  scores  for  Communicative  Effectiveness  Scale 


Computer  Use 

Hi  (n=162) 

Low  (n=157) 

Use  the  Internet  to  get 
information 

3.7 

3.8 

Electronic  mail  with  teachers 

3.1 

3.5 

Electronic  mail  with  friends 

3.7 

3.9 

Electronic  mail  with  family 

3.3 

3.7 

Electronic  mail  with  classmates 

3.3 

3.6 

Electronic  mail  with  students 
organizations 

2.8 

3.0 

The  next  part  of  the  analysis  explored  the  way  in  which  the  respondents  self-reported  their 
perception  of  the  effectiveness  of  computer-mediated  communication  (CMC)  for  task  and  non-task  related 
contexts.  The  results  for  the  two  groups  are  presented  in  Table  2. 

Table  2:  Perceptions  of  CMC  based  on  High  and  Low  scores  for  Communicative  Effectiveness  Scale 


Effectiveness  of  CMC 

Hi  (n=162) 

Low  (n=157) 

Task  Context 

3.6 

3.9 

Non-task  Context 

3.7 

4.0 

Discussion 

First,  the  data  suggests  that  those  who  feel  that  computers  do  play  a role  in  enhancing  their 
communicative  effectiveness  tend  to  use  the  computer  more  for  communication  as  well.  Furthermore,  the 
analysis  also  suggested  that  the  primary  contributor  to  the  higher  communicative  use  was  the  perception 
and  other  possible  determinants  such  as  demographic  attributes  did  not  produce  statistically  significant 
differences.  Consequently,  it  is  safe  to  suggest  that  it  is  important  to  be  able  to  reliably  measure  the 
attitudes  of  the  users  towards  the  communicative  effectiveness  of  the  computer.  Indeed,  knowing  the  range 
of  the  perceptions  can  help  to  determine  the  way  in  which  the  computer  could  be  used  by  a group  of 
students  and  predicting  that  use  can  have  a significant  impact  in  policy  decisions  about  the  way  in  which 
computer  networks  are  designed  and  maintained  on  college  campuses.  This  becomes  particularly  true  in 
view  of  the  way  in  which  band  width  availability  has  become  an  issue  on  college  campuses  as  students  are 
using  the  computer  more  for  non-task  related  CMC  (see,  e.g.,  Carlson,  2001).  As  the  data  suggests  here,  a 
positive  predisposition  towards  the  communicative  effectiveness  can  result  in  a statistically  significant 
higher  level  of  use  of  the  computer. 

Secondly,  the  findings  from  this  study  also  underscore  the  importance  of  measuring  the 
perceptions  of  the  communicative  effectiveness  of  the  computer.  Although  there  are  different  perceptions 
factors  that  can  be  measured  with  respect  to  the  computer,  none  of  the  other  factor  identified  by  Mitra 
(2001)  were  significant  in  distinguishing  between  the  different  levels  of  computer  use.  While  the  data 
suggests  that  there  are  significant  differences,  it  is  really  the  communicative  effectiveness  factor  that 
emerges  as  the  critical  one.  Thus,  when  considering  the  measurement  of  attitudes  of  students  towards 
computers,  it  is  necessary  and  essential  to  evaluate  the  differences  in  this  factor  since  those  differences  can 
have  far  reaching  usage  and  effectiveness  consequences  both  for  task  and  non-task  activities. 

Finally,  the  importance  of  the  communicative  effectiveness  should  come  as  no  surprise  as 
computers  are  increasingly  being  introduced  in  the  pedagogic  space.  It  has  been  the  case  that  the 
introduction  of  the  computer  in  the  teaching  environment  has  transformed  the  process  of  teaching  and 
learning  in  so  far  as  that  the  pedagogic  process  has  become  a more  dialogic  process  where  the  different 
participants  in  the  teaching  and  learning  process  are  engaging  in  the  acts  of  communication.  In  a way  the 
different  players  in  the  pedagogic  space  have  now  found  their  own  voices  and  the  traditional  relations  of 
power  and  authority,  often  predicated  upon  who  was  able  to  ©mmunicate  with  whom,  are  being 
dismantled.  Those  students  who  are  quicker  to  realize  that  a shift  is  occurring  in  the  way  teaching  and 
learning  is  being  played  out  with  the  help  of  computers  are  perhaps  also  the  ones  who  have  a positive 
attitude  towards  the  communicative  effectiveness  of  the  computer.  Eventually  these  students  with  the  more 
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positive  attitude  would  be  more  likely  to  use  the  computer  more  for  communicating  with  the  various  groups 
they  have  to  interact  with  in  the  learning  process. 

To  some  degree  the  positive  attitude  could  also  be  related  to  the  issues  of  communication 
apprehension  that  have  been  researched  with  respect  to  the  use  of  CMC  (Sproull  and  Kiesler,  1992; 
Walther  1992,  1994,  1996).  Work  on  interpersonal  communication,  communication  apprehension  and  the 
opportunities  provided  by  the  use  of  CMC  in  teaching  has  demonstrated  that,  first,  more  voices  can  be 
heard  during  the  teaching  and  learning  process.  The  increasing  number  of  voices  in  the  pedagogic  space,  as 
the  positively  predisposed  students  increasingly  use  the  communicative  functions  of  the  computer  can  have 
implications  on  the  process  of  teaching  and  the  organization  of  the  teaching  institution.  For  instance,  using 
a tool  such  as  electronic  mail  to  communicate  with  the  instmctor  is  often  accompanied  by  an  increasing 
burden  on  the  teacher  to  attend  to  and  respond  to  all  the  electronic  letters  that  arrive  with  missives  and  from 
the  students  who  are  often  the  quiet  ones  in  the  classroom  (Mitra,  et.  al.,  2000b).  As  the  findings  from  this 
data  suggest,  there  can  be  an  anticipation  that  there  will  be  increasing  use  of  electronic  mail  to 
communicate  with  teachers  and  it  is  in  the  interest  of  the  teachers  and  the  educational  institution  to  be  able 
to  reliably  measure  the  various  attitudes  of  the  students  with  respect  to  the  use  of  computers  in  teaching. 

In  summary,  this  data  and  finding  from  earlier  studies  all  suggest  that  there  is  a need  to  measure 
attitudes  towards  computers.  Such  measurements  can  offer  insights  into  the  way  in  which  computers  can 
eventually  be  used  in  teaching.  Furthermore,  the  findings  from  such  measurements  can  offer  ways  to 
predict  and  anticipate  what  the  usage  patterns  could  be.  Finally,  the  usage  patterns  can  have  an  impact  on 
the  learning  outcomes  related  to  the  use  of  computers  in  teaching.  The  data  here  suggests  that  among  all 
the  different  attitude  factors  that  can  be  measured,  one  of  the  most  important  ones  is  the  attitude  towards 
the  communicative  effectiveness  of  computers  in  teaching.  To  be  sure,  the  value  of  this  measurement  is  not 
only  empirically  evident  but  can  be  ontologically  supported  given  the  fact  that  good  pedagogy  ultimately 
remains  a dialogic  and  discursive  process. 
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Abstract:  A typical  learning  method  using  the  Internet  is  exploratory  learning,  where  learners 
frequently  reach  a learning  impasse  caused  by  insufficient  hyperlinks.  In  this  paper,  we  propose  an 
environment  that  supports  open-ended  web-based  exploratory  learning.  This  environment  avoids 
the  impasse  by  enabling  learners  to  create  hyperlinks  in  the  open  web  and  share  the  hyperlinks. 
Furthermore,  the  hyperlinks  have  been  adapted  with  collaborative  filtering. 


Introduction 

Rapid  spread  of  the  Internet  enables  us  to  learn  with  informative  web  pages  that  exist  all  over  the  world.  A typical 
learning  method  using  the  Internet  is  exploratory  learning,  where  learners  construct  knowledge  through  exploring 
the  web  autonomously.  However,  the  learners  frequently  reach  a learning  impasse  caused  by  insufficient  hyperlinks. 
If  a page  is  not  complete  with  sufficient  hyperlinks  to  fulfill  their  goal  or  interest,  the  learners  cannot  visit  the  next 
page  smoothly.  Thus  insufficient  hyperlinks  lead  the  learners  to  the  impasse. 

Search  engines,  which  can  supply  suitable  hyperlinks  (pages),  help  escape  from  the  impasse  but  occasionally  return 
meaningless  hyperlinks.  Adaptive  web-based  learning  systems,  which  alter  the  hypertext  contents/structure  to 
learners'  characteristics,  can  avoid  the  impasse.  The  existent  adaptive  web-based  learning  systems  do  not  present 
sufficient  hyperlinks  to  fulfill  learners'  expectation.  Almost  all  such  systems  limit  the  adaptation  to  the  closed  web 
(i.e.  web  pages  inside  one  server),  since  teachers  can  hardly  describe  information  indexes  and  adaptation  rules  to  the 
open  web  (i.e.  arbitrary  web  pages).  Recently,  this  limitation  has  become  a major  focus  in  the  domain  of  adaptive 
hypermedia  (Brusilovsky,  2001). 

In  this  paper,  we  propose  an  environment  that  supports  open-ended  web-based  exploratory  learning.  This 
environment  avoids  the  impasse  by  enabling  learners  to  create  hyperlinks  in  the  open  web  and  share  the  hyperlinks. 
Furthermore  the  hyperlinks  are  adapted  with  collaborative  filtering.  The  prototype  system  with  this  environment, 
which  is  MITS  (Multi -hyperlink  Tailoring  System),  is  currently  under  constmction. 


Free-Hyperlink  Environment 

The  key  idea  for  avoiding  the  impasse  is  a free -hyperlink  environment,  where  learners  can  create  hyperlinks  (i.e. 
free-hyperlinks)  in  the  open  web  and  share  the  hyperlinks.  The  free-hyperlink  environment  will  organize  scattered 
pages  and  consequently  form  large-scale  teaching  material  with  many  sufficient  hyperlinks,  which  avoids  the 
impasse.  In  other  words,  numerous  learners  substitute  for  the  teachers  who  have  to  complete  sufficient  hyperlinks. 
When  reaching  the  impasse,  learners  usually  try  to  find  suitable  pages  by  using  search  engines.  Taking  this  process 
into  account,  a search  engine  has  been  integrated  into  the  process  of  the  free-hyperlink  creation.  Although  this 
integration  is  time-consuming,  the  time  will  be  minimized  eventually  since  the  free-hyperlinks  can  be  shared.  The 
free-hyperlinks  are  created  in  the  following  process. 

(1)  Search  Query  Input:  Learners  pick  a word(s)  as  the  query  and  send  the  word  to  an  existing  search  engine. 

(2)  Search  Output  Receipt:  They  receive  the  search  output  modified  by  MITS. 

(3)  Visit  and  Free- Hyperlink  Creation:  By  their  selecting  a page,  the  page  and  a comment  window  are  displayed.  If 
considering  the  page  to  be  informative,  they  input  comments  on  the  page  (e.g.  "This  page  has  useful  diagrams"). 
Immediately  after  inputting  the  comments,  the  free-hyperlink  between  the  word  and  the  page  is  created. 
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IntemetCop  is  a comprehensive  software  for  managing/monitoring  Internet  L 
Internet  connection  of  any  type  (Dialup  TCP/IP,  ISDN,  ADSL,  Cable  modem 
among  several  computers  in  a networl<. 

It  allows  organizations  to  integrate  management  of  various  security  functions 
protection  facilities. 

Advantages: 

IntemetCop  provides 

• Maximum  utilisation  of  the  existing  Hardware. 

• Distribution  of  Internet  time  to  a number  of  persons  over  a small  or  larg 

• It  logs  individual  user  data. 
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Figure  1.  Free -hyperlinks 

Figure  1 shows  a snapshot  of  the  free-hyperlinks.  The  free-hyperlinks  are  highlighted  with  a star-shaped  icon  in 
order  that  learners  can  distinguish  the  free-hyperlinks  from  original  hyperlinks.  By  clicking  on  a free-hyperlink,  a 
pop-up  box  containing  the  information  about  the  free-hyperlink  is  immediately  displayed.  The  information  consists 
of  page  title,  URL,  free-hyperlink  creator's  e-mail  address,  creation  date,  and  comments.  The  learners  select  the  next 
suitable  page,  referring  to  the  information. 


Collaborative  Filtering  in  MITS 

It  is  apparent  that  the  increase  of  the  free-hyperlinks  causes  option  overflow.  To  improve  this  problem,  collaborative 
filtering  is  used,  which  is  a technique  for  recommending  items  estimated  to  be  suitable  by  similar  learners  (Resnick 
et  al,  1994).  To  calculate  the  similarity  between  learners,  we  assume  that  the  learners  who  made  the  free-hyperlink 
of  the  same  word  have  similar  interest.  MITS  counts  the  number  of  free-hyperlink  of  the  same  word.  It  calculates  the 
similarity  by  means  of  Pearson's  correlation  coefficient  (a*),  and  shows  the  free-hyperlinks  of  five  learners  with  high 
similarity  (r>0).  This  is  the  adaptation  in  MITS,  which  strikes  a proper  balance  between  enriching  information  and 
eliminating  redundancy. 


Conclusion 

A free-hyperlink  environment  avoids  the  learning  impasse  caused  by  insufficient  hyperlinks.  Specifically,  it  enables 
learners  to  create  hyperlinks  in  the  open  web  and  share  the  hyperlinks,  which  are  adapted  on  the  basis  of  the 
learners'  interest  using  collaborative  filtering.  However  there  are  some  problems  with  the  MITS  prototype.  One  is  a 
failure  to  modify  original  web  pages.  For  example,  JavaScript  embedded  by  MITS  contends  with  JavaScript  in  the 
original  web  page.  Currently,  this  study  is  trying  to  improve  the  problem  and  prepare  an  experiment  for  evaluation. 
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Abstract:  This  paper  examines  the  use  of  the  Web  to  support  continuing  professional  development 
(CPD).  It  outlines  the  factors  driving  the  adoption  of  CPD  and  highlights  areas  where  the  Web  can  aid 
in  the  development  of  successful  professional  communities.  A survey  examining  the  use  of  the  Internet 
to  support  professionals  working  in  the  domains  of  the  Arts  and  Sience  is  presented.  The  study 
reviews  twenty  four  sites  for  the  presence  and  degree  of  adoption  of  several  key  features  including:  - 
community  building,  range  and  value  of  content,  user  friendliness  and  guidance,  sophistication  of 
employed  Web  technology. 


1.  Introduction 

The  use  of  the  Internet  and  the  World  Wide  Web  is  growing,  and  the  number  of  applications  where  it  is 
employed  to  support,  or  even  to  replace,  many  traditional  activities  associated  with  work  and  recreation  continues  to 
expand.  Such  new  technology  means  that  the  nature  of  the  jobs  we  do  has  changed,  as  has  the  professions  in  which  we 
work  and  the  skills  required.  Life-long  learning  has  never  assumed  greater  importance  than  it  does  today,  allowing  us  to 
remain  competent  and  achieve  personal  satisfaction  and  job  fulfilment,  throughout  our  careers.  From  an  employer’s 
viewpoint,  lifelong  learning  contributes  to  competitive  advantage  through  excellence  of  employees.  At  a national  level, 
it  allows  us  to  remain  competitive  in  a world  participating  in  free  trade  and  experiencing  rapid  changes  through 
technology. 

In  this  setting  it  is  not  surprising  that  many  professional  communities  have  turned  to  the  Internet  to  augment  or 
replace  continuing  professional  development  (CPD)  support.  Traditional  approaches  to  professional  development  rely 
on  a mixture  of  activities  including  conferences,  meetings,  scholarly  reading,  newsletters  and  short  courses.  However, 
no  matter  how  excellent  any  of  these  activities  prove  to  be,  they  are  of  little  value  if  they  are  not  accessible  to  those  who 
need  them  most,  when  they  need  them.  While  many  will  argue  that  the  Web  cannot  provide  all  the  things  that  come  with 
more  ’traditional'  approaches,  Web  technology  can  provide  flexible  support  to  professional  communities,  especially 
under  circumstances  where  the  community  members  are  geographically  dispersed  and  are  subject  to  time  constraints. 

The  level  of  support  will  vary  between  communities,  depending  on  the  subject  requirements,  the  technical 
sophistication  of  providers  and  audience,  as  well  as  resources  available  for  the  upkeep  of  the  support  mechanism  itself 
Resources  available  on  websites  supporting  professional  communities  can  range  from  links  to  other  informative  sites,  to 
provision  of  original  content,  to  full-blown  community  facilities  including  online  discussion  forums.  In  the  latter  case, 
the  professional  community  can  support  itself  via  a 'Web  community'. 

Significantly,  the  use  of  new  technology  does  not  guarantee  success  in  encouraging  and  supporting  CPD.  Any 
successful  Web  community  must  be  built  upon  solid  foundations.  "Web  communities  need  'social  scaffolding'  to  grow 
and  thrive.  Social  scaffolding  refers  to  those  aspects  of  a site  - roles,  rituals,  features,  events,  and  leadership  - that 
facilitate  community  development.  Much  like  a trellis  enables  a plant  to  grow,  social  scaffolding  enables  members  to 
become  progressively  more  involved  in  the  community"  (Kim,  1998).  Simply  launching  a website  with  a bulletin  board 
and  chat  facilities  does  not  automatically  generate  a community  (Mager  and  Karlenzig,  2001).  There  are  numerous 
examples  of  quickly  launched  message  boards  with  many  topics  but  no  responses.  Creating  gathering  places  alone  is  not 
enough  - they  need  to  be  organised  and  integrated  into  the  community.  The  central  issues  surrounding  communities  are 
people  issues  - Web  technology  merely  acts  as  a facilitator,  providing  the  tools  for  helping  people  come  together 
(Cothrel  and  Williams,  1999). 

This  paper  begins  by  outlining  the  factors  driving  and  inhibiting  the  formation  of  Web  communities  to  support 
professional  development.  It  then  goes  on  to  survey  two  disparate  professional  groups,  in  the  Arts  and  in 
Science/Technology,  and  attempts  to  gauge  their  level  of  adoption  of  Web  technology  in  their  professional  development 
activities.  Twenty  four  sites  are  examined,  twelve  in  each  group,  and  a number  of  measures,  ranging  from  ‘level  of 
adoption  of  community  interaction*  to  ‘technical  sophistication*,  are  considered. 


2.  Continuing  Professional  Development  and  the  Web 


Getting  people  to  ’buy  into’  continuing  professional  development  can  often  prove  difficult.  Individuals  often  need 
to  be  reminded  that  they  have  a career  - not  just  a job  (Arnold  and  Smith,  1998).  In  order  to  be  truly  effective, 
individuals  need  to  take  ownership  of  their  approach  to  personal  development  and  life  long  learning.  The  Web  enhances 
this  process  by  allowing  individuals  to  extend  their  professional  interaction,  seeking  out  and  exchanging  new  ideas  and 
opinions,  beyond  the  boundaries  of  the  organisation  in  which  they  work.  Web  technology  can  be  used  to  complement 
existing  forms  of  learning  and  development,  delivering  CPD  to  a widely  diverse  and  dispersed  audience.  Individuals 
may  get  less  benefit  Irom  face-to-face  conferences  as  conference  size  increases.  The  Web  can  mitigate  these  effects  by 
enabling  users  to  take  a more  ’personal’  approach  to  professional  development,  so  that  individuals  can  tailor  a 
developmental  programme  to  their  own  requirements.  "Every  learner  can,  at  his  or  her  own  choice  of  time  and  place, 
access  a world  of  multimedia  material...  immediately  the  learner  is  unlocked  from  the  shackles  of  fixed  and  rigid 
schedules,  from  physical  limitations...  and  is  released  into  an  information  world  which  reacts  to  his  or  her  own  pace  of 
learning"  (Benjamin,  1994,  p49).  While  the  use  of  Web  technology  does  mean  that  individuals  can  work  at  their  own 
pace,  there  still  needs  to  be  a forum  for  obtaining  feedback  on  their  progress.  Further,  there  are  some  subjects,  requiring 
more  nuance  or  hands-on  involvement,  which  may  prove  more  difficult  to  translate  to  an  online  environment  (Stuart, 
1999). 

Professional  development  can  be  substantially  enhanced  by  opportunities  to  collaborate  with  others  within  Web 
communities  (Hixson  and  Tinzmann,  1990).  The  opportunity  to  take  advantage  of  the  expertise  of  others  can  provide 
community  members  with  important  reinforcement  and  incentive  for  continuing  growth  and  development,  along  with 
enhanced  personal  status  and  respect  that  comes  from  membership  in  a learning  community  with  their  professional 
colleagues.  The  asynchronous  nature  of  much  of  the  communication  that  takes  place  online  lends  itself  to  CPD  and 
supports  synthesis  of  knowledge  (Salmon,  2000).  Individuals  have  24-hour  access  to  the  system  and  can  log  on 
whenever  they  wish  and  for  however  long  they  wish.  Users  are  able  to  reflect  on  issues  raised  online,  and  their  own 
ideas  and  thoughts  can  germinate  through  composing  replies. 

CPD  is  an  ongoing  process  and  as  such,  the  use  of  Web  technology  to  aid  this  process  is  not  a one-shot  event. 
Traditionally,  individuals  within  an  organisation  would  take  one  or  two  weeks  off  from  their  daily  grind  to  go  on  a 
training  course.  The  use  of  Web  technology  enables  professional  development  to  be  a constant  continuing  process. 
However,  technology  is  no  panacea  for  professional  development.  In  order  to  be  truly  effective,  the  use  of  technology 
must  be  linked  to  the  objectives  and  goals  of  any  professional  development  programme,  and  must  deliver  real  value  to 
the  community. 


3.  Web  Communities 

The  use  of  the  Internet  to  develop  Web  communities  has  meant  that  localised  communities  can  now  have  a more 
global  outlook.  ’’The  Net  erases  boundaries  created  by  time  and  distance,  and  makes  it  dramatically  easier  for  people  to 
maintain  connections,  deepen  relationships,  and  meet  like  -minded  souls  that  they  would  never  have  met’’  (Kim,  2000, 
p.x).  In  order  to  be  successful  and  deliver  true  value  to  the  users,  the  aims  of  the  community  must  be  clear  and  the  first 
step  to  building  a loyal  community  of  members  is  to  understand  the  purpose  of  the  community  (Kim,  1998).  Web 
communities  grow  and  thrive  when  members  are  able  to  fulfil  their  purpose  and  accomplish  those  goals  that  require 
other  members  to  participate.  The  concept  of  collaborative  purpose  is  one  of  the  Web's  premier  strengths  as  a means  of 
building  community  (Real  Communities  Inc,  2000).  There  are  a number  of  features  that  aid  the  development  of  an  on- 
line community  (Kim,  2000).  These  include:  - 

Backstory:  The  community’s  backstory  provides  a powerful  tool  in  shaping  members'  expectations  about  the 
purpose  and  personality  of  the  site  (Kim,  1998).  The  term  backstory  comes  from  filmmaking  and  refers  to  the  part  of  the 
movie’s  story  that  has  happened  before  the  first  frame  of  the  film.  Backstories  introduce  the  community  founders, 
communicate  their  motivation,  and  impact  a sense  of  the  community's  core  values. 

Site  Map:  The  site  map  gives  an  overall  picture  of  the  community  space  and  may  include  links  to  each  section  of 
the  community.  Site  maps  should  be  updated  as  sites  evolve  and  sections  of  the  community  are  added. 

Feedback:  Through  implementation  and  maintenance  of  feedback  loops,  communities  evolve  over  time  and  may 
react  to  the  requirements  of  its  members.  Regular  surveys  will  help  gauge  the  community  opinions  and  re-enforce  a 
sense  of  shared  purpose. 

Database:  A member  database  system  is  crucial  for  creating  and  maintaining  member  profiles  that  evolve  over 
time.  This  database  can  be  used  to  control  access  to  Web  pages,  mailing  lists,  chat  rooms,  conference  areas  and  member 
profiles.  Many  communities  also  allow  members  to  develop  their  own  home  pages,  thus  helping  each  member  develop 
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their  sense  of  belonging  to  the  community  (White,  2001). 

Frequently  Asked  Questions:  These  address  the  needs  and  questions  of  newcomers  without  alienating 
established  members.  FAQs  serve  a key  role  in  breaking  down  initial  barriers  for  new  users  and  making  them  feel  more 
at  ease  with  the  technology  and  the  community  environment  itself. 

Communication  Technology:  Communication  is  at  the  heart  of  online  communities.  The  technologies 
employed  can  be  public  (interactions  between  several  people)  or  private  (one-to-one  interactions);  synchronous 
(messages  are  exchanged  in  real  time)  or  asynchronous  (messages  are  accumulated  and  users  need  not  be  online 
simultaneously).  All  communities  need  a mixture  of  public  and  private  meeting  places  and  it  can  often  prove  difficult  to 
identify  a single  space  where  the  Web  community  'lives'.  This  is  a result  of  the  range  of  tools  adopted  by  members 
(Cothrel  and  Williams,  1999).  The  most  commonly  used  tools  include:  - Electronic  Mail-  The  use  of  e-mail  is 
widespread  and  as  such,  it  is  an  extremely  powerful  communication  medium  that  may  be  used  to  hold  a community 
together.  E-mail  distribution  lists  may  be  used  for  making  announcements  and  for  encouraging  communication  between 
users.  A community  newsletter  can  serve  a key  role  in  keeping  members  up  to  date  with  key  events  or  issues  that  may 
arise  within  the  community.  E-mail  can  also  be  used  to  encourage  feedback  and  suggestion  from  community  members. 
Poling  (1994)  found  that  the  use  of  e-mail  enhanced  the  quality  of  communication  within  groups  and  ultimately  aided 
group  cohesion:  Mailing  Lists  and  Bulletin  Board  - Community  mailing  lists  can  facilitate  conference-style  interaction 
between  community  members.  However  mailing  lists  do  not  create  the  same  sense  of  gathering  in  a location  with  fellow 
community  members  that  conference-style  interaction  can  provide  (Steuer,  1998):  Chat  - Real-time  chat  is  a frequently 
misused  community  technology  (Steuer,  1998).  Nothing  discourages  users  more  than  ai  empty  chat  room,  or  an 
interactive  event  that  has  very  little  or  zero  interaction  between  users.  However  when  utilised  correctly,  reakime  chat 
can  be  a very  effective  communication  medium. 


4.  A Survey  of  Arts  and  Science  Websites 

Our  investigation  of  Web-based  support  for  professional  development  focussed  on  two  broad  subject  areas,  viz., 
Arts/Humanities  and  Science/Technology.  Twenty  four  websites  were  chosen  at  random  - twelve  from  the  Arts  domain 
and  twelve  from  the  Science  domain  ^ee  Table  A).  The  aim  of  this  survey  was  to  gauge  the  degree  and  nature  of 
support  afforded  by  these  subject-oriented  sites.  To  this  end,  a feature  list  was  formulated  (see  Table  B)  to  map  the 
characteristics  across  the  sample  sites.  The  features  in  question  span  three  broad  categories:  Usability,  Resources,  and 
Community  Interaction;  with  each  sample  site  examined  for  the  presence/absence  of  these  features. 


Arts  and  Humanities  Sites 

Science  Sites 

History  Classics  and  Archaeology 

LTSN  Centre  for  Information  and  Computer  Sciences 

Humbul  Humanities  Hub 

Association  for  Computing  Machinery 

CORD 

Joint  Information  Systems  Committee 

Philosophical  and  Religious  Studies 

Scottish  Teachers  Online  Resource  Modules 

Art,  Design  and  Communication 

IEEE 

Performing  Arts  (PALATINE) 

IEEE  Computer  Society 

Sociology,  Anthropology  and  Politics  (C-S  AP) 

International  Federation  for  Information  Processing 

English  Subject  Centre 

Computer  Science  Discipline  Network 

Hospitality,  Leisure,  Sport  and  Tourism 

Geography,  Earth  and  Environmental  Sciences 

LeisureTourism.com 

LTSN  Engineering 

Social  Policy  and  Social  Work  (SWAP) 

LTSN  Physical  Sciences 

European  League  of  Institutes  of  the  Arts 

Royal  Astronomical  Society 

Table  A:  Sample  Sites 


Presence  of  Features:  The  feature  survey  covers  aspects  that  are  commonly  employed  on  websites  that  actively 
try  to  develop  a sense  of  community.  Naturally,  a professional  group  may  decide  not  to  adopt  all  such  features  for  their 
particular  community.  Every  site  gained  one  point  for  each  extant  feature  (see  Table  B). 


Usability 

Backstory,  Contact  details,  Details  of  last  site  update.  Search  engine,  Elementary 
search  and  navigation  instructions.  Site  map 

Resources 

Details  of  conferences/  seminars.  Resources  - academic,  Resources  - magazines/ 
newsletters.  Links,  FAQ  section 

Community  Interaction 

Feedback  forms/  surveys,  Mailing  lists.  Chat  rooms.  Bulletin  boards.  Registration 
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required  in  order  to  contribute  to  site 

Table  B:  Features 

Scoring  of  Features:  Having  surveyed  the  presence  of  various  features  for  supporting  communities  online,  a 
deeper  analysis  of  each  feature  sought  to  gauge  the  range  and  depth  of  material  on  each  site,  estimate  site  reliability,  and 
assess  site  appearance.  The  appearance  and  overall  feel  of  the  site  is  considered  significant  since  this  is  likely  to 
influence  the  return  of  users  to  a site.  Each  of  the  feature  categories  was  examined  in  detail  to  afford  a further  dimension 
of  ranking  for  our  twenty  four  websites. 

As  a basis  fo^"  ‘rankinj^*,  each  Site  was  awarded  Up  to  4 points  for  features  in  each  category.  Points  are  awarded 
on  a qualitative  basis,  dependant  upon  areas  where  there  are  discemable  differences  between  the  features  present  on 
each  individual  site.  In  cases  where  a site  had  none  of  the  features  examined  in  a particular  category,  the  site  was 
assigned  zero  points  for  that  category.  For  example,  a site  that  does  not  have  any  of  the  Community  Interaction  features 
would  be  awarded  zero  points  in  that  particular  category. 

Technological  Features:  This  final  measure  gauges  the  sophistication  of  the  technology  used  on  the  website. 
The  technological  features  present  on  a site  do  not  necessarily  yield  better  results  in  terms  of  supporting  a professional 
community.  Four  simple  to  detect  measures  were  used  to  estimate  the  ‘sophistication’  of  each  site.  Namely,  the 
presence  and  use  of  HTML,  different  Data  Types,  use  of  Scripting,  and  use  of  Server-Side  Scripting.  HTML  examines 
the  sophistication  of  the  HTML  code  employed  on  the  site  (see  Table  C). 


HTML 

Raw  HTML  code.  Image  maps,  Frames,  Cascaded  Style  Sheets,  Tables 

Data  Types 

Adobe  Acrobat  files,  Windows  Media  files.  Word  documents,  PowerPoint 
presentations,  Zip  files.  Flash,  Rich  Text  Format  files 

Scripting 

Instances  of  scripting.  Number  of  functions.  Size  of  functions.  Number  of  times 
function  called 

Server-Side  Scripting 

Use  of  GET  method  in  forms.  Use  of  POST  method  in  forms.  Use  of  cookies.  Site 
adding  information  to  their  database 

Table  C:  Technological  Features 


The  source  code  for  pages  from  each  of  the  twenty  four  sites  was  analysed  across  each  of  the  four  categories  to 
gauge  the  level  of  complexity  and  technical  sophistication  of  the  code.  In  a similar  manner  to  the  Scoring  of  Features 
section,  the  categories  were  assessed  in  order  to  arrive  at  a site  ranking  that  was  based  on  those  areas  where  there  are' 
discernible  differences  between  the  websites,  with  3 points  being  awarded  to  the  most  sophisticated  site  in  each 
category. 

Results  and  Analysis 

Almost  predictably,  study  of  the  twenty  four  sample  sites  has  highlighted  the  fact  that  Science  sites  score  higher 
than  Arts  sites  in  every  respect.  This  may  be  explained  by  the  quicker  uptake  of  technology  in  science-based  subject 
areas,  and  the  greater  experience  of  those  maintaining  these  sites. 

Presence  of  Features 

When  analyzing  the  Presence  of  Features  on  each  of  the  twenty  four  sample  sites.  Science  sites  consistently 
score  higher  across  each  of  the  three  categories  examined.  However  it  is  interesting  to  note  that  Arts  and  Science 
subjects  can  actually  be  found  to  be  progressing  entirely  in  parallel.  Science  sites  may  be  a step  ahead  all  of  the  time, 
but  nonetheless  it  holds  true  throughout  that  where  Science  sites  score  highly  in  categories  such  as  Resources  and 
Usability,  so  do  Arts  sites.  Similarly,  in  the  Community  Interaction  category  where  Science  sites  gain  lower  scores.  Arts 
sites  also  gain  lower  scores.  It  could  perhaps  be  argued  that  Arts  sites  are  following  in  the  footsteps  of  Science  sites  (see 
Figure  A). 
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Figure  A:  Presence  of  Features  Figure  B:  Scoring  of  Features 


If  there  is  one  thing  that  the  results  do  not  seem  to  imply,  it  is  that  Arts  websites  are  good  at  providing  one 
particular  thing  and  Science  websites  at  providing  another.  Essentially,  the  experiences  of  both  communities  are  very 
similar.  However,  Arts  sites  do  appear  to  cater  more  to  the  novice  Internet  user,  with  50%  of  the  Arts  sites  sampled 
offering  elementary  navigation  instructions  to  users.  Conversely,  Science  sites  appear  to  assume  a higher  level  of 
technological  proficiency  from  those  accessing  their  sites,  with  only  16%  offering  any  form  of  help  to  users  in 
navigating  between  pages. 

Both  Arts  and  Science  communities  have  potential  to  improve  the  level  of  community  interaction.  Neither  area 
has  fully  embraced  the  use  of  communication  technology  to  enhance  the  level  of  interaction  between  community 
members.  The  reasons  behind  this  could  be  varied.  Maybe  current  implementation  of  these  technologies  is  still 
relatively  new  and  fragile.  Perhaps  many  existing  tools  do  not  mirror  well  ‘real  life  interactions’.  Maybe  the 
communities  see  no  benefit  from  these  forms  of  interaction. 

Scoring  of  Features 

Science  sites  appear  to  score  higher  across  all  of  the  categories  examined.  Similar  to  the  results  in  the  previous 
section,  both  groups  of  sites  appear  to  be  progressing  in  parallel,  with  Science  sites  scoring  higher  in  each  category  (see 
Figure  B).  Both  sets  of  sites  scored  highly  when  considering  the  Usability  of  the  sites.  Equally,  both  Arts  and  Science 
sites  show  less  evidence  of  Community  Interaction.  This  concurs  with  the  findings  of  the  previous  section,  where  the 
majority  of  sites  in  each  category  are  not  fiilly  making  use  of  communication  technology  as  a means  of  building 
community  and  developing  a level  of  community  interaction. 

The  difference  between  the  scores  of  Arts  and  Science  sites  is  greater  in  the  Resources  category.  Across  the  sites 
surveyed,  it  is  interesting  to  note  that  Science  sites  tend  to  have  a greater  range  and  depth  of  material,  while  Arts  sites 
tend  to  rely  more  on  providing  links  to  material  of  interest. 

Technological  Sophistication 

Again  Science  sites  scored  more  strongly  across  all  categories  (see  Figure  C).  This  perhaps  indicates  a quicker 
uptake  of  technology  or  greater  experience  from  those  maintaining  Science  sites.  Alternatively,  Arts  sites  may  have  felt 
that  this  level  of  technological  sophistication  was  not  necessary. 
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Once  more,  there  appears  to  be  a degree  of  parallel  progression  between  the  two  sets  of  sites:  Arts  sites  score 
highly  (though  not  quite  as  highly)  in  the  same  categories  where  Sbience  sites  achieve  high  scores.  This  holds  true 
especially  in  the  HTML  and  Data  Types  categories,  where  Science  sites  show  only  marginally  greater  sophistication 
than  Arts  sites.  The  only  anomaly  is  the  category  of  Scripting,  which  achieved  the  third  highest  ranking  in  Science,  but 
comes  fourth  in  the  Arts.  Interestingly,  this  category  also  shows  the  highest  divergence  between  Arts  and  Science  sites, 
with  a difference  of  almost  10  points. 

The  main  difference  between  Arts  and  Science  sites  appears  in  the  Scripting  and  Server-Side  Scripting 
categories,  where  Science  sites  score  substantially  higher  than  their  Arts  counterparts.  There  is  certainly  more  complex 
processing  around  any  areas  of  interaction  on  Science  sites,  and  this  may  be  the  result  of  the  maintainers  of  Science  sites 
being  more  comfortable  with  using  complex  technological  features. 


5.  Concluding  Comments 

The  use  of  Web  technology  for  professional  development  is  perceived  as  being  useful  and  is  currently  being 
employed  by  a wide  spectrum  of  professional  communities.  The  resources  and  features  available  on-line  can  range  from 
lists  of  useful  links,  to  the  provision  of  original  content,  to  full-blown  community  facilities  including  online  discussion 
forums. 

Our  survey  of  websites  providing  support  in  the  Arts  and  Science  domains  has  found  that  overall  there  is  more 
correspondence  than  divergence.  An  analysis  of  site  features  shows  little  difference  between  the  two  domains.  The  most 
interesting  point  to  come  out  of  this  analysis  is  the  relative  lack  of  community  interaction  facilities  on  sites  in  both 
domains.  There  may  be  several  reasons  for  this.  Possibilities  include  a lack  of  perceived  need  and/or  local 
administrative  or  technological  constraints.  This  is  an  appropriate  area  for  further  study. 

One  of  the  few  elements  to  show  marked  differences  between  sites  in  the  two  domains  was  the  presence  or 
absence  of  elementary  navigation  instructions.  Arts  sites  are  more  sipportive  here,  perhaps  because  their  user  base  is 
assumed  to  be  less  proficient  in  the  use  of  websites  in  general.  Another  marked  difference  emerges  within  the 
technological  features,  where  Scripting  is  clearly  employed  more  with  sites  supporting  the  Science  community.  This 
may  be  the  result  of  the  maintainers  of  Science  sites  being  more  comfortable  with  using  complex  technological  features. 
Ultimately,  however,  the  examination  of  the  sites  shows  that,  barring  the  exceptions  mentioned  above,  differences 
between  Arts  and  Science  websites  are  ones  of  degree  rather  than  kind. 
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Abstract 

The  psper  describes  the  experience  of  the  Fscult}'  of  Hconomics  ot  the  University'  of  Trento,  Itnly,  in 
delivering  online  courses  through  the  Web,  The  faculty  chose  to  design  and  build  its  own  system  from  scratch 
rather  than  buy  one  off  the  shelf  We  discuss  the  considerations  informing  this  decision,  and  present  the  main 
functionalities  of  the  system,  which  was  built  with  a server-side  approach.  There  are  asynchronous  functions 
provided,  such  as  the  ability  for  the  teacher  to  upload  educational  materials  and  for  the  student  to  download 
these  materials  through  the  Web,  Some  interesting  synchronous  functions  built  into  the  system  include  the 
teacher-student  chat  forum  and  the  low-cost  videoconferencing  tool  where  a single  student  can  interact  with 
the  teacher  who  is  available  on-line  at  the  same  time.  Everything  is  integrated  in  a unique,  browser-based 
system  accessible  from  everywhere.  We  also  describe  some  important  administrative  functions  of  the  system. 


1.  Introduction 

Web-based  instruction  has  become  a convenient  education  medium  [Aggarwal  2000]:  course  offerings 
using  web- based  instruction  are  very  common  in  universities  [Teare  et  al,  1999],  even  if  the  opportunities  for  and 
means  of  student  -teacher  interaction  can  vary  with  the  programme.  This  paper  describes  the  development  of  an 
interactive  online  teaching  system  at  the  University  of  Trento,  and  reports  the  results  of  the  first  experiment  with 
course  development  and  delivery  in  the  Faculty  of  Economics.  The  faculty  has  about  3000  students  and  more 
than  one  hundred  teachers,  some  of  whom  come  from  other  faculties.  At  present,  few  teachers  have  their 
educational  material  available  online.  Some  teachers  in  the  Faculty  of  Economics  took  the  initiative  to  produce 
their  educational  material  in  electronic  format,  and  created  their  own  personal  Web  sites  with  the  syllabus  of  the 
course,  the  slides  of  the  lectures,  and  the  information  related  to  practical  aspects  of  the  course  (room,  timetable 
etc.).  However,  each  of  these  Web  sites  had  its  own  design  and  logic,  resulting  in  a confusing  array  of  links, 
material,  format  of  lessons,  and  overall  structures  that  students  had  to  sift  through  in  order  to  find  the  desired 
information.  The  plea  arose  from  the  students  for  a more  uniform  presentation  of  online  educational  materials 
and  for  a standard  procedure  to  access  them. 

On  this  basis,  in  September  2000  the  faculty  decided  to  fulfil  this  request  for  niformity.  Firstly,  we 
determined  which  functionalities  should  be  provided  and  then  looked  into  whether  such  a system  could  be  found 
on  the  market  or  should  be  built  from  scratch.  We  will  briefly  discuss  this  latter  issue.  At  the  evaluation  stage  it 
was  important  to  investigate  the  cost  of  building  a system  using  internal  resources.  This  was  because,  although 
there  are  valid  reasons  for  purchasing  an  off-the-shelf  Course  Management  System  such  as  TopClass,  Lotus 
Learning  Space  or  WebCT,there  are  many  instances  where  universities  have  successfully  developed  a system  by 
themselves.  It  was  evident  that  homegrown  systems  are  often  more  tailored  to  a university's  specific  needs.  For 
an  analysis  of  this,  see  [A’herran  2001]  [McMahon  et  al,  2000].  Courseware  management  systems  are  being 
promoted  as  a means  of  simplifying  the  creation  and  management  of  instructional  Web  sites.  There  are  many 
studies  that  compare  specific  courseware  management  tools  (e.g. [Info World,  1998], [PC  Week,  1997]),  but  it 
seems  that  the  efficiency  and  effectiveness  of  these  tools  with  respect  to  customized  Web-based  systems  is  less 
easy  to  measure. 

In  summary,  we  decided  to  build  rather  than  to  buy  for  the  following  reasons: 

• the  cost:  the  systems  that  manage,  distribute  and  maintain  the  online  learning  materials  are  high- cost 
systems  that  also  imply  secondary  costs  for  adequate  hardware  and  assistance. 

• the  skills:  the  faculty  has  no  personnel  specifically  qualified  to  work  in  these  environments.  Complex 
systems  such  as  Lotus  Leamingspace  ™ or  Blackboard  ™ require  personnel  specifically  dedicated  to 
their  management,  thus  requiring  more  resources  than  the  faculty  can  provide. 

• the  highly  specific  needs  of  end-users,  in  particular  the  integration  with  administrative  and  bureaucratic 
functions:  many  of  the  functionalities  that  normally  relate  with  the  didactic  activity  of  the  teacher  are 
closely  related  with  administrative  tasks  that  should  be  carried  out  in  a specific  way  and  at  a specific 
time,  thus  constituting  a big  constraint  for  packaged  ^sterns  that  have  no  personalization  features  or 
that  require  a vast  amount  of  adjustments  in  order  to  be  personalized. 

• the  time:  a prototype  was  to  be  ready  for  the  second  part  of  the  2000  academic  year  (March  2000)  with 
a pilot  group  of  teachers  and  students,  and  a beta  version  of  the  system  was  to  be  available  for  use  by  all 
the  teachers  and  the  related  students  in  the  first  semester  of  2001.  This  schedule  was  not  compatible 
with  our  time  of  implementation  of  already-built  learning  environment  like  the  ones  cited  above. 


o 

ERIC 


1395 


For  all  these  reasons,  we  decided  to  build  the  system  on  our  own,  creating  only  what  we  needed.  The 
main  objectives  of  the  project  were  the  following: 

• to  allow  the  teacher  to  keep  a constant  relationship  with  students,  in  the  different  ways  that  we  will 

explain  in  the  following  sections; 

• to  allow  the  teacher  to  distribute  educational  material  to  students; 

• to  keep  data  for  administrative  and  reporting  tasks  that  must  be  performed  by  the  teacher  and  by  the 

secretary  of  the  Faculty. 

Some  general  considerations  must  be  kept  in  mind  to  understand  the  complexity  of  the  relationships 
among  the  different  entities  of  the  data  schema.  We  have  the  following  constraints: 

• A teacher  can  have  multiple  courses. 

• More  than  one  teacher  could  teach  a course. 

• A student  can  follow  multiple  courses,  and  therefore  must  have  access  to  the  different  faculty  areas 
of  the  teachers. 

• The  educational  material  must  be  kept  online  over  a period  of  some  years,  in  order  to  let  students 
download  the  correct  material  for  the  year  they  followed  the  course,  and  also  to  give  another 
possibility  to  students  who  have  failed  to  complete  their  courses  within  the  prescribed  time. 

A difficult  choice,  currently  under  debate  in  many  institutions,  regards  whether  the  system  should  be 
open  to  everyone  or  limited  to  only  registered  students,  i.e.,  only  regular,  fee-paying  students.  Some  teachers 
believe  that  the  educational  material  should  be  freely  distributed  to  everyone  in  the  world,  while  others  prefer 
that  only  their  students  (freely)  dowiload  the  educational  material.  For  the  time  being  the  more  restrictive  vision 
has  been  adopted:  the  students  must  register  with  the  system  by  providing  their  personal  data  and  their  faculty 
registration  number.  A login  and  password  is  released  for  newly  registered  students.  The  same  registration 
procedure  is  used  for  the  teacher,  thus  assuring  that  only  the  course  instructor  can  load  material  into  the  system. 

The  system  is  built  with  several  functions,  most  of  which  are  available  to  the  teacher,  a subset  8 
available  to  the  student,  and  a specific  set  of  functions  are  devoted  to  administrative  purposes.  There  are  also 
common  functions,  though  with  different  privileges:  teachers  can  modify  the  data  while  students  can  only  read 
them.  We  will  now  present  these  functions,  distinguishing  them  by  type  of  user.  The  main  function  that  the 
system  guarantees  is  to  transfer  a file  from  the  teacher’s  computer  to  the  system,  and  from  the  system  to  the 
students’  computer.  By  inserting  the  correct  login  and  password,  the  teacher  can  upload  the  files,  categorize  them 
and  assign  them  to  the  different  lessons.  The  students  can  work  from  their  home  workstations  to  connect  to  the 
system  as  long  as  they  have  a login  and  password.  They  can  then  check  for  updated  information  or  mat  erials  for 
a their  courses,  and  download  the  relevant  materials  if  desired.  The  didactic  material  or  administrative 
information  is  thus  transferred  to  the  student,  who  can  then  read,  print,  or  store  it  as  desired.  At  the  moment,  the 
graphic  layout  of  the  Web  site  has  not  been  decided.  When  all  the  functionalities  of  the  system  have  been 
completed,  the  designers  will  look  more  closely  at  the  “cosmetic”  aspects  of  the  site. 

2.  The  Teacher  tools 

The  teacher  is  the  user  having  access  to  almost  all  the  functions  of  the  system.  After  having  inserted  her 
own  login  and  password,  the  teacher  chooses  the  course  on  which  she  wants  to  operate.  Courses  are  also 
identified  by  year,  so  the  course  entitled  “Information  Systems”  taught  in  1999  is  found  in  a different  section 
with  respect  to  the  same  course  taught  in  2000,  even  if  it  uses  the  same  educational  material. 

After  having  chosen  the  course,  the  teacher  has  all  the  tools  at  her  disposal.  All  the  tools  provide  the 
principle  functions  for  managing  the  material  or  information,  i.e.,  inserting,  updating,  deleting  and  listing.  These 
tools  are  principally: 

• Materials  (lecture  slides,  tests,  solutions  of  tests,  notes,  graphics,  charts,  tables  etc.)  are  categorized; 
the  list  of  types  of  materials  is  dynamic. 

• A dynamic  syllabus,  where  the  educational  material  can  be  viewed.  The  teacher  can  upload  and 
download  material  at  this  location. 

• The  course  programme,  timetable  and  bibliography  of  related  material. 

• “Live”  links  related  to  the  course,  dynamically  updated  by  the  teacher  and  directly  usable  by  students. 

• Frequently  Asked  Questions  related  to  the  topics  covered  in  the  course. 

• The  diary  of  the  lessons  (fig.  1),  which  differs  from  the  syllabus  in  that  it  has  a more  administrative 
organization,  while  the  syllabus  can  also  be  used  to  upload/download  the  materials. 

• The  newsgroup,  up  to  three  for  each  course:  the  details  are  discussed  in  the  next  section. 

• The  list  of  the  students  that  are  registered  for  the  course,  with  their  email  address.  The  teacher  can 
use  it  as  a mailing  list  t)  send  cumulative  or  multiple  messages  to  the  students  on  the  list;  such 
messages  can  refer  to  lesson  cancellation,  announcement  for  seminars  etc. 

• A videoconferencing  system  (fig.  2),  made  using  a personalized  version  of  the  Netmeeting  ™ 
software,  integrated  inside  the  browser.  When  the  teacher  is  online  with  the  videoconferencing  system 
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opened,  the  students  that  connect  to  the  videoconferencing  page  will  automatically  see  a green  light 
indicating  that  the  teacher  is  available  for  a live  session.  The  system  automatically  stores  the  IP 
address  of  the  teacher,  thereby  helping  the  student  and  teacher  to  overcome  the  technical  difficulties 
of  managing  a conferencing  system  via  Internet. 
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Figure  1:  The  teacher's  diary 
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Figure  2:  A videoconferencing  session 


In  the  “syllabus”  and  in  the  “materials”  sections  the  teacher  can  upload  the  material  by  means  of  a 
simple  form,  just  choosing  the  relevant  file  from  her  own  hard  disk.  The  system  will  transfer  the  file  to  the  server 
and  record  the  information  in  the  underlying  database.  In  this  way,  the  material  is  securely  transferred  to  a 
central  station,  available  for  the  students  to  be  downloaded.  The  file  is  also  associated  to  a specific  lesson,  thus 
helping  to  compile  the  syllabus.  An  interesting  feature  of  the  uploading  process  is  the  possibility  of  excluding  the 
students  from  seeing  that  the  file  is  present  on  the  system.  This  is  very  useful  in  cases  where  there  are  multiple 
teachers  on  the  same  course.  In  fact,  the  system  could  be  used  to  create  a closed  group  composed  only  of 
teachers  and  tutors,  where  some  documents  are  shared  only  among  them  and  are  not  visible  to  the  students  that 
enrol  for  the  course.  This  has  been  done  commonly  in  courses  where  tutors  and  teachers  exchanged  the  test 
answers  before  the  lessai  or  test  was  given. 


3.  The  Student  tools 
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The  tools  available  to  the  students  are  the  same  as  those  seen  by  the  teacher,  although  mostly  in  read- 
only format.  In  practice  this  means  that  the  syllabus  can  be  used  only  to  download  material  from  the  server, 
while  the  course  programme  can  be  viewed  but  not  modified.  A specific  functionality  provided  to  the  student  is 
course  enrolment  (Fig.3,  item  3).  As  with  any  enrolment  procedure,  the  student  must  choose  the  courses  desired. 
In  this  area  the  student,  previously  registered  with  the  institution,  is  presented  with  the  list  of  the  courses  offered 
by  the  faculty  that  have  online  materials,  and  the  student  simply  chooses  the  courses  she  wants.  However,  the 
most  important  feature  available  that  is  specifically  for  students  is  the  “news’*  section.  (Fig.  3,  item  2) 
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Figure  3:  The  main  window  of  the  student 


This  function  is  presented  to  the  student  when  she  connects  to  the  system.  On  the  left  is  the  list  of 
courses  in  which  the  student  is  enrolled,  while  the  right  part  of  the  video  shows  a frame  with  all  the  documents 
uploaded  since  the  last  time  the  student  logged  in.  This  feature  allows  the  student  to  go  directly  to  a document 
presented  in  the  “news”  list  without  having  first  to  choose  a course  and  search  for  the  most  recent  material.  This 
is  particularly  useful  to  the  student  who  cannot  be  present  in  the  classroom  for  one  or  several  lessons  but  who 
nevertheless  wants  to  keep  up  with  the  course. 

Another  important  feature  of  the  system  is  the  newsgroup.  The  validity  of  this  tool  for  didactic  purposes 
has  been  widely  analysed  and  proved.  [Collis  et  al,  1997]  [Eastmond  1995]  flanna  2000]  [Horton  2000] 
[Palloff  et  al,  1999].  The  newsgroup  inside  the  system  is  related  to  the  course,  thus  allowing  the  teacher  to  have 
different  discussion  forums  for  every  course  taught.  The  teacher  moderates  the  newsgroup  by  acting  on  the 
messages  in  varying  ways:  recommending  a message,  deleting  messages,  voting  on  messages  etc..  The  messages 
from  the  teacher  are  of  a different  colour  (fig.  4),  this  to  confirm  the  different  role  of  the  teacher  in  managing  the 
newsgroup.  Each  message,  both  in  the  summary  and  in  the  detailed  view,  reports  the  number  of  times  it  has  been 
read  by  other  users  (fig.  5,  item  1).  Students  can  participate  in  the  forum  not  only  by  writing  and  responding  to 
messages,  but  also  by  voting  on  the  messages  (fig.  5,  item  2). 
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Figure  4:  The  newsgroup  with  the  different  types  of  messages 
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This  vote  is  visible  to  everyone  and  serves  to  evaluate  the  importance  of  the  message  for  the  current 
topic.  The  forum  has  threads,  so  every  reply  is  associated  with  the  corresponding  message.  The  thread  in  which 
the  message  is  involved  is  always  visible  to  the  reader,  also  when  she  writes  a response,  (fig.  5,  item  3).  All  the 
messages  are  stored  in  and  appended  to  text  files,  not  in  the  database.  The  files  have  a preconfigured  maximum 
dimension,  so  when  this  dimension  is  exceeded  a new  text  file  is  created.  This  proved  to  be  the  most  logical 
choice  when  taking  into  account  the  following  facts: 

• the  number  of  newsgroups  available:  there  are  more  than  one  hundred  teachers  and  courses 

• the  variable  dimension  of  the  messages:  in  most  cases  the  messages  are  short  (a  few  hundred 
characters),  but  some  messages  (FAQs,  for  example)  could  be  very  long,  so  the  database  fields  should 
be  dimensioned  accordingly,  with  a great  waste  of  space. 

On  these  text  files  an  indexing  engine  makes  it  possible  to  use  the  traditional  search  and  advanced 
search  mechanisms.  As  mentioned  in  the  discussion  of  teacher  tools,  in  the  student  section  it  is  possible  to  check 
if  the  teacher  is  on-line  at  that  moment  via  videocamera,  and  therefore  if  the  teacher  is  available  for  interaction  in 
real-time  (fig.  3,  item  1) 
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Figure  5:  a single  message  with  different  options 


4.  The  administrative  functions 

This  area  includes  several  functions,  not  only  related  to  ICT  administration.  It  also  includes  functions 
necessary  for  the  faculty  secretary.  The  first  important  function  is  the  change  of  academic  year.  In  September  of 
every  year,  the  course  can  change  teacher/s,  title,  group  of  discipline,  and  most  of  all  students.  So,  every  year  in 
this  period  a specific  function  that  allows  the  faculty  secretary  to  redefine  the  association  between  the  courses 
and  the  teachers  must  be  provided.  Because  of  the  quantity  of  information  associated  with  a course,  this  task 
requires  the  provision  of  several  functions  that  could  be  fully  automated,  partially  automated  or  completed 
manually.  These  functions  are  listed  as  follows: 

• A function  to  transfer  automiically  all  the  materials  from  the  previous  to  the  current  year,  this  in  case  the 
teacher  decides  to  reuse  the  material  of  the  previous  year.  Everything  is  therefore  copied  with  a new 
identifier,  so  the  old  and  the  new  courses  have  different  identities  but  the  same  material. 

• A function  to  selectively  transfer  materials  and  other  information  related  to  the  course  from  one  year  to 
the  other.  The  major  problem  is  represented  by  the  newsgroup,  where  threads  could  be  in  progress  and 
therefore  an  automatic  transfer  between  the  two  years  could  confuse  the  students  on  one  side,  and  could 
be  inconsistent  with  a potentially  different  course  programme  on  the  other.  The  default  option  is  that  the 
newsgroup  remains  connected  only  to  its  original  course,  so  a new  course  will  have  a new  newsgroup. 

• A function  to  store  previous  courses  with  all  the  related  materials  so  that  students  can  access  to  them. 

Other  more  traditional  administrative  functions  are  related  to  the  ordinary  activity  that  the  secretary 
carries  out  during  the  academic  year,  especially  at  the  beginning  of  the  year.  The  main  function  is  the 
management  of  the  teacher  database,  with  all  the  administrative  information  necessary  to  fulfil  bureaucratic 
obligations.  In  addition,  a function  that  assigns  the  teacher  to  courses  is  provided;  this  association  is  fundamental 
for  the  rest  of  the  system,  as  it  is  used  in  every  area  that  grants  access  to  the  online  material.  All  the  materials, 
newsgroups,  links,  students  etc.  depend  on  this  unique  association:  teacher-course-academic  year. 


5.  Feedback  and  future  developments 

The  prototype  of  CMS  we  developed  has  been  under  a stress  test  since  the  second  semester  of  academic 
year  2000/2001  and  will  be  released  in  its  final  version  at  the  end  of  this  experimentation  period.  The  system  has 
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been  tested  with  about  1.600  students  and  200  teachers  for  one  year,  with  a download  rate  of  1.300  documents 
per  day.  We  submitted  some  questionnaires  to  students,  both  manually  and  via  e-mail,  and  collected  direct 
feedback  from  colleagues.  Fran  these  responses  we  can  assert  that  the  reaction  to  the  system  has  been  very 
positive.  Our  main  concern  was  the  teachers  with  less  computer  experience,  which  meant  a majority  of  our 
faculty.  The  system  has  proven  very  easy  and  intuitive  to  use,  winning  praise  from  everyone  due  to  its 
simplicity.  We  must  admit,  however,  that  we  made  a great  effort  in  this  area,  in  terms  of  providing  direct 
assistance  to  teachers,  sometimes  by  using  students  to  help  teachers  in  difficulty.  We  also  received  positive 
comments  from  the  students.  In  the  students’  questionnaires  we  asked  for  comments  on  functionality,  usefulness, 
quality  of  materials  and  performance  of  the  system. 

In  short,  of  the  approximately  200  questionnaires  returned  from  students,  79%  expressed  satisfaction  with 
the  system.  In  particular,  the  contact  with  the  professor  and  the  availability  of  new  material  are  the  most 
welcome  features.  Eighty-six  percent  were  satisfied  with  the  quality  of  the  material,  although  a few  underlined 
the  need  for  open  (not  read- oily)  material.  The  overall  usefulness  of  the  system  was  largely  connected  to  the 
news  section,  where  the  student  is  notified  at  every  connection  of  any  new  material  added  since  his/her  last 
connection.  A second  positive  aspect  was  the  teacher -student  contact  and  the  related  improvement  in  the  learning 
process.  In  this  part  we  received  91%  positive  feedback,  an  embarrassing  indication  that  change  was  definitely 
needed.  The  performance  of  the  system  was  found  to  be  satisfactory  for  78%  of  students,  while  the  rest  were  not 
very  satisfied.  An  important  point  to  note  is  that  teachers  often  don’t  look  at  the  amount  of  megabites  their 
materials  represent  (they  use  office  connections  to  the  Web  site)  but  students  using  home  connection  with 
analogic  modems  at  56Kbps  suffer  in  this  aspect,  which  should  be  considered  more  by  teachers  when  uploading 
materials.  As  regards  the  frequency  of  use,  more  that  50%  of  students  used  system  daily,  and  the  rest  from  two  to 
four  times  a week. 

In  conclusion,  we  can  be  very  satisfied  by  our  first  experiment,  though  there  are  some  points  where  the 
system  must  be  further  integrated.  One  important  point  of  integration  is  with  the  student  secretary  database.  In 
our  university  the  student  secretary  is  an  autonomous  entity,  with  its  own  procedures,  database  and  technology. 
On  one  hand  this  allows  single  projects  to  be  started  without  having  to  depend  on  the  big  infrastructure  of  the 
student  secretary.  On  the  other  hand,  however,  there  is  a time  where  the  different  autonomous  systems  should  be 
integrated.  The  simplest  example  regards  the  login  and  password  to  access  the  different  facilities  that  the 
university  offers  to  students,  examples  being  e-mail,  access  to  laboratories,  account  of  the  refectory,  etc..  At  the 
moment,  the  uniformity  of  logi  n and  password  in  the  two  systems  is  not  guaranteed,  it  is  up  to  the  student  to 
register  herself  in  both  systems  with  the  same  login  and  password. 

In  conclusion,  the  system  presented  in  this  paper  is  one  alternative  to  Course  Management  Systems,  built 
with  the  main  target  of  distributing  educational  material  to  students  in  a uniform  way.  Some  real-time  functions 
have  been  added  to  the  system  and  are  currently  under  testing;  however  we  believe  that  the  basic  functionalities 
provided  by  the  system  to  distribute  dynamic  information  and  educational  material  are  the  minimum  services 
that  should  be  provided  to  students.  The  positive  results  of  the  first  experiments  seem  to  confirm  this  assertion. 
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Abstract:  This  paper  will  provide  overview  of  the  redesign  of  two  corporate  training 
programs,  New  Supervisor’s  Basics  and  Ethics  and  Compliance.  These  two  courses  are  part 
of  an  enterprise  wide  e-learning  initiative  underway  at  Harris  Corporation;  a Melbourne, 
Florida  based  international  communications  and  electronics  company. 

Components  of  successful  repurposing  training  programs  and  interactive  methodologies 
incorporated  in  redesign  will  be  shared  with  participants.  The  lessons  learned,  and  challenges 
encountered  will  also  be  included  in  the  presentation.  Attendees  will  have  an  opportunity  to 
discuss  the  components  of  successful  programs  and  uses  of  interactive  methodologies  to 
determine  applicability  to  their  own  organizations. 


Evolution  of  Corporate  Web-Based  Training 

This  paper  presents  an  overview  of  two  separate  stand-up  (instructor-led)  courses  that  were  repurposed  into  web- 
based  training  (WBT).  The  paper  further  describes  the  lessons  learned  by  the  Harris  Knowledge  Network  (HKN) 
team  during  this  ongoing  effort.  These  lessons  learned  are  not  restricted  to  a stand-upAVBT  repurposing  effort;  they 
are  readily  applicable  to  other  projects  in  which  an  existing  educational/training  system  is  modified  into  a different 
delivery  system. 


Corporate  Example 

An  ongoing  goal  of  the  HKN  team  was  to  create  a site  that  supports  employees’  need  for  training  and  information 
while  maximizing  return  on  investment  with  Internet  technology  and  training.  Empowering  employees  with  the 
elements  and  tools  to  become  independent  knowledge  workers  was  also  an  objective  of  the  Harris  team. 
Preliminary  research  strongly  indicated  that  although  the  application  of  web-based  training  and  other  e-leaming 
initiatives  would  play  significantly  in  the  overall  organizational  learning  strategy,  they  would  have  to  be  carefully 
integrated  into  the  curriculum.  Adapting  the  blending  learning  methodology  for  two  of  the  first  courses  rolled  out 
worked  well  within  this  strategy.  The  initial  blended  model  included  a Bulletin  Board  System  (BBS)  and 
collaborative  tools.  These  features  will  be  added  incrementally  as  HKN  solidifies  its  curriculum. 


Requirements 

An  analysis  of  business  requirements  and  technical  environment  was  performed  in  order  to  determine  the  best 
overall  methodology  and  design  strategy  for  these  courses.  The  methodology  had  to  account  for  multiple  training 
and  information  needs  within  a large  geographically  dispersed  corporation  in  an  economic  manner.  A needs 


analysis  was  also  conducted  to  determine  the  skills  and  skill  levels  of  the  end  users.  This  data  identified  what  were 
the  gaps  or  “holes**  of  knowledge.  It  also  gave  strong  design  clues  as  to  which  materials  should  be  presented  online 
and  which  material  would  be  optimally  delivered  via  live  instructor. 

Business  requirements  identified  were  diverse  training  and  information  needs,  and  personnel  distributed  across 
several  states  and  countries.  Fiscal  drivers  included  the  desire  to  reduce  travel  and  training  costs  while  improving 
the  skills  and  knowledge  of  employees.  The  technological  factors  taken  under  consideration  included  accelerated 
desktop  computer  processing  speed,  improved  browser  technology,  and  such  features  as  Java  enabled  client  server 
interactivity,  improved  backend  database  technologies,  the  ubiquitous  availability  of  commercial  internet  service 
providers,  and  improved  access  to  the  bandwidth  needed  for  large  file  transmission  (Wagner,  1999). 

After  performing  a comparative  analysis  of  business  and  technical  requirements,  as  well  as  needs  analysis  of  end 
users,  the  chosen  methodology  for  both  Harris’  New  Supervisor  Training  and  Ethics  and  Compliance  Courses  was  a 
blended  design  consisting  of  synchronous  and  asynchronous  learning. 

Glossary  of  Terms 

E-learning:  Covers  a wide  set  of  applications  and  processes,  such  as  Web-based  learning,  computer-based  learning, 
virtual  classrooms,  and  digital  collaboration.  It  includes  the  delivery  of  content  via  Internet,  intranet/extranet 
(LANAVAN),  audio-  and  videotape,  satellite  broadcast,  interactive  TV,  and  CD-ROM. 

Blended  learning:  Learning  events  that  combine  aspects  of  online  and  face-to-face  instruction 

Asynchronous  learning:  Learning  in  which  interaction  between  teachers  and  students  occurs  intermittently  with  a 
time  delay.  Examples  are  self-paced  courses  taken  via  the  Internet  or  CD-ROM,  Q&A  mentoring,  online  discussion 
groups,  and  email. 

BBS  (bulletin  board  system):  An  online  community  run  on  a host  computer  that  users  can  dial  or  log  into.  BBS 
users  can  post  messages  on  public  discussion  boards,  send  and  receive  email,  chat  with  other  users,  and  upload  and 
download  files.  BBSs  are  text-based  and  often  related  to  the  specific  hobbies  or  interests  of  their  creators 

Business  requirements:  Conditions  an  e-leaming  solution  should  meet  to  align  with  needs  of  such  stakeholders  as 
content  developer,  subject  matter  expert,  learner,  manager,  and  training  administrator. 

Collaborative  tools:  Allow  learners  to  work  with  others  via  email,  threaded  discussions,  or  chats 

Customer-focused  e-learning:  Web-based  learning  programs  targeted  at  current  and  prospective  customers.  By 
training  customers  online,  companies  attract  new  business  and  make  people  more  comfortable  with  e-transactions. 

End  user:  The  person  for  whom  a technology  is  designed;  the  individual  who  uses  a technology  for  its  designated 
purpose.  In  e learning,  the  end  user  is  usually  the  student 

Enterprise-wide  e-learning:  E-leaming  that  is  intended  for  all  or  most  employees  within  a company.  Often  part  of 
a strategic  change  of  direction  with  a very  short  timeline.  Also  used  to  support  a core  process  such  as  sales. 

Synchronous  learning:  A real-time,  instmctor-led  online  learning  event  in  which  all  participants  are  logged  on  at 
the  same  time  and  communicate  directly  with  each  other.  In  this  virtual  classroom  setting,  the  instructor  maintains 
control  of  the  class,  with  the  ability  to  "call  on"  participants.  In  most  platforms,  students  and  teachers  can  use  a 
whiteboard  to  see  work  in  progress  and  share  knowledge.  Interaction  may  also  occur  via  audio-  or 
videoconferencing,  Internet  telephony,  or  two-way  live  broadcasts 

Ongoing  Endeavor 

The  New  Supervisor’s  Basics  Course  initially  piloted  in  November  2001  and  is  scheduled  for  its  second  session  in 
January  2002.  Ethics  and  Compliance  Course  is  scheduled  for  initial  pilot  in  February  2002.  Also  Environmental 
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Health  and  Safety  WBT  is  scheduled  for  its  first  course  pilot  in  January  2002  and  customer  focused  e-leaming 
courses  are  currently  under  development.  This  corporate  training  and  information  site  will  continue  to  expand  as  the 
portal  to  various  knowledge  portals  for  specific  audiences  throughout  the  international  company.  It  follows  the 
strategic  plan  for  continually  increasing  the  breadth  and  depth  of  information  and  training  offered  via  the  corporate 
training  portals  and  knowledge  hubs. 

Harris  Knowledge  Network  has  continued  to  successfully  integrate  itself  as  a corporate  tool  as  technology  advances 
and  business  factors  address  employee  needs  as  specialized  knowledge  workers.  Successful  deployment  and 
maintenance  of  a knowledge  network  depends  upon  accurately  defining  organizational  business  goals,  creating  a 
viable  learning  strategy  and  on  having  a clear  understanding  of  requirements  and  terminology. 


ASTD’s  Learning  Circuits  E-Leaming  Glossary  Compiled  by  Eva  Kaplan-Leiserson  (2001). 

Wagner,  E.  ( 1 999).  Beyond  Distance  Education:  Distributed  Learning  Systems.  Handbook  of  Human  Performance  Technology, 
626-648. 
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Abstract:  Previous  research  on  video  instruction  shows  that  the  learning  is  enhanced  when 
small  groups  of  users  watch  and  discuss  lectures  and  videos  together.  Using  specialized^ 
high-end  videoconference  systems,  these  improved  results  have  been  shown  to  apply  even 
when  the  users  are  in  different  locations.  In  this  paper,  we  explore  two  issues  in  making  Digital 
Library  scenarios  widely  supported  at  low  cost.  First,  we  explore  designing  of  a model 
system  that  allows  distributed  individuals  to  collectively  watch  a Digital  Library  video  using 
shared  controls  such  as  play,  pause  and  stop.  Second,  we  explore  the  impact  of  four  alternative 
discussion  channels  on  users  leaning  and  interaction  behavior. 


1 Introduction 

This  paper  contains  the  research  plan  from  the  original  project  proposal.  More  recent  ideas  and 
progress  will  be  periodically  made  available  from  the  Yano  Lab.  home  page*.  With  rapid  changes  in  technology, 
what  we  learn  today  become  obsolete  or  irrelevant  within  a short  time  span.  We  need  to  continuously  update  our 
skills,  and  cost-effective  support  for  such  lifelong  learning  is  a key  challenge  facing  our  education  systems.  To 
address  this  problem,  one  solution  adopted  by  many  universities  has  been  to  broadcast  classes  via  TV  networks 
in  a Digital  Libraries,  and  strategy  Stanford  has  pursued  successfully  for  over  25  years.  However,  much  work  is 
needed  to  enable  a more  advanced  utilization  of  the  available  information  and  communications  resources.  This 
research  focuses  on  the  exploration  of  issues  in  develop  of  digital  library  environments  that  facilitate  CSCL, 
CSCW  (Computer  Supported  Collaborative  Work)  to  their  specific  individuals  needs  and  preferences.  The 
strategy  that  Stanford  has  pursed  as  synchronous  model  of  learning  has  drawbacks:  it  often  conflicts  with  users’ 
work  commitments,  and  the  model  is  not  scalable.  Even  if  the  class  cold  be  broadcast  to  a million  people 
concurrently,  there  are  few  chances  that  interesting  interaction  would  happen  between  specialized  users  and  end 
users  in  such  environment. 

The  flexibility  and  scalability  can  be  solved  by  using  Digital  Library  video,  videoconference  streaming 
technologies  and  the  Internet  to  create  an  on-demand,  anytime,  anywhere  learning  model.  However,  research 
shows  that  learning  can  suffer  when  users  watch  videos  individually.  This  concern  is  identified  and  addressed  in 
a classic  study  reported  in  Science  by  Gibbons  et  al.  (1997).  As  a solution,  they  propose  the  tutored  video 
instruction  model,  wherein  remote  users  watch  videos  in  small  groups  with  a discussion  facilitator.  As  they 
watch  the  video,  they  periodically  pause  and  discuss  it.  The  study  shows  that  the  tutored  video  instruction 
users  outperform  users  who  attend  the  live  class  in  the  classroom,  as  well  as  users  who  watched  classes  live 
from  remote  locations  and  users  who  watched  classes  video  individually.  The  advantage  of  tutored  video 
instruction  over  classroom  attendance  was  replicated  at  the  University  of  Massachusetts  where  some  users 
groups  met  without  tutors  (Stone,  1990).  The  advantage  of  tutored  video  instruction  was  also  found  by  Smith, 
Sipusic,  and  colleagues  (Sipusic,  1999,  Smith  1999).  In  addition,  they  tested  and  extension  of  tutored  video 
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instruction  called  distributed  tutored  video  instruction,  which  does  not  require  users  to  meet  in  the  same  room. 
The  researchers  had  groups  of  up  to  seven  users  watch  and  discuss  pre-recorded  lectures  from  multiple 
locations.  In  addition  to  the  lecture  video,  users  were  connected  with  a high-quality,  low-latency  audio  and 
video  connection.  In  experiments  involving  six  university  courses  and  several  hundred  users,  course  grades  of 
distributed  tutored  video  instruction  and  tutored  video  instruction  users  were  better  that  those  of  users  who 
attended  the  live  lecturers  and  were  indistinguishable  from  one  another.  The  success  of  distributed  tutored 
video  instruction  and  tutored  video  instruction  is  attributed  to  the  collaboration  and  discussion  that  occurs 
among  the  users  watching  the  video. 

Distributed  tutored  video  instruction  is  clearly  a desirable  model  in  many  ways.  It  allows  uso-s  to 
participate  from  anywhere,  thus  eliminating  location  constrains.  It  allows  users  to  participate  almost  anytime, 
subject  to  finding  a few  partner  users  willing  to  participate  at  the  same  time.  It  is  scalable,  as  hundreds  or 
thousands  of  such  small  groups  can  exist  and  make  progress  independently.  Best  of  all,  it  provides  all  these 
features  while  statically  showing  better  learning  outcomes  than  being  in  the  classroom. 

Given  the  desirability  of  distributed  tutored  video  instruction,  this  paper  extends  earlier  work  in  two 
principal  directions,  allowing  for  wide-scale  deployment  of  distributed  tutored  video  instruction  in  a Digital 
Library.  First,  a key  component  of  distributed  tutored  video  instruction  is  the  ability  to  collaboratively  view 
pre-recorded  lecture  video.  For  example,  when  one  user  presses  the  pause  button,  the  video  should  pause  for  all 
remote  users.  The  “what  you  see  is  what  I see”  principle  needs  to  be  preserved  for  all  group  members. 

Second,  distributed  tutored  video  instruction  results  presented  and  Sipusic  at  al.  (1999)  were  based  on 
the  remote  participants  having  a specialized,  high-quality  videoconference  system,  e.g.,  the  system  presented 
four  remote  videos  currently  on  a monitor,  each  at  30  frames  per  second,  while  also  encoding  the  video  of  the 
local  participant.  Such  infrastructure  is  unlikely  to  be  available  to  most  DL-users  (digital  library  users),  and  past 
literature  shows  little  benefit  from  including  talking-head  video  in  somewhat  similar  contexts  (Boyle,  1994) 
Consequently,  h this  paper  we  propose  a new  model  of  communication  for  effective  collaboration  in  a 
videoconferencing-based  CSCL  environment  with  videoconferencing,  audio-conferencing,  text  chat  and 
face-to-face  communication.  And  also  the  question  of  how  this  model  is  affected  if  the  communication  channel  is 
less  rich.  Does  is  affect  learning  and  interaction  behavior? 

2 Distributed  collaborative  video 

Enabling  the  distributed  collaborative  video  instruction  in  Digital  Library  scenarios  requires  two  components: 

a)  A distributed  lecture  video  viewing  system  with  shared  controls  like  play,  stop  and  pause. 

b)  A communication  system  for  discussion  around  the  library  holds  materials  including  video  content. 

Although  the  distributed  tutored  video  instruction  requirements  are  quite  basic  in  many  ways,  today’s  streaming 
media  products  and  applications  sharing  products  do  not  support  it. 

Streaming  media  players  (e.g.,  Microsoft  Windows  Media  Player)  allow  multiple  people  to  watch  the 
same  stored  video  from  a shared  video  server.  However,  users  do  not  shared  the  VCR  controls.  Users  could  say 
“pausing  video”  on  the  conference  call  when  pausing  the  player,  but  remote  users  will  end  up  pausing  their 
video  players  at  slightly  different  times.  In  addition  to  being  a cumbersome  process,  over  multiple  stop-starts, 
they  can  easily  get  out  of  synchronization. 

Unfortunately,  this  almost  works,  but  not  adequately.  The  remote  Digital  Library  users  can  see  and 
control  the  navigator  or  video,  but  the  video  is  jittery  and  of  low  quality.  Instead  of  the  video  data  stream 
going  from  the  Digital  Library  video  server  to  all  participants  and  being  independently  decoded  and  rendered  on 
their  workstations  or  PCs  (shown  in  Figure  1),  the  video  stream  goes  only  to  the  Digital  Library  user  who  starts 
the  media  player  (DL-userl)  (see  Figure  2).  It  is  decoded  and  rendered  on  DL-userl’s  machine,  and  the 
pixel-level  screen  changes  are  detected  by  the  application  sharing  system  (e.g.,  NetMeeting),  re-encoded  using  a 
separated  algorithm,  and  sent  to  others’  desktops.  Because  none  of  the  application  sharing  products  is 
optimized  for  this  scenario  (they  are  targeted  towards  transmitting  infrequent  changes  to  the  screen  as  may 
occur  during  web  browsing)  the  performance  is  dismal  compared  to  highly  optimized  video  codecs , and  the 


network  bandwidth  consumed  is  large.  Although  application  sharing  allows  DLrUsers  to  share  documents, 
presentations  and  DL  web  pages,  current  systems  do  not  work  for  dynamic  media  like  video. 

To  solve  the  above  problem,  what  we  need  is  a partial  sharing  navigation  browser  and  media  player 
application.  We  want  the  Digital  Library  content  to  stream  independently  to  each  users’  navigation  browser  and 
media  player  and  be  decoded  and  rendered  there  (as  in  Figure  1),  but  we  want  to  have  a Navigation  with  VCR 
control  UI  of  the  browser  and  media  player  to  work  via  our  model  application  system.  To  implement  this  solution, 
we  need  to  combine  the  functionality  of  JMF  (Java  Media  Framework),  NetMeeting  SDK  and  JDK  1 .3. 

3 Video-communication  model 


In  this  section,  we  propose  an  interaction  model  that  would  solve  the  problems  of  distributed  materials  and 
video-communication  viewing  that  commonly  exists  in  others  implementations,  while  also  providing 
opportunities  for  instant  communication  for  unintended  interactions  with  unexpected  DL-users  by  introducing 
the  sense  of  distance  amo^  DL-usemn  the  Digital  Library  environment. 


Figure  1:  Conventional  video  flow  when  each  Media  Player 
independently  connects  to  the  digital  video  server 


Figure  2:  video  flow  when  players  are  shared  under 
NetMeeting. 


In  the  first  implementation  we  tried  to  introduce  the  sense  of  distance  in  a hybrid  space  by  providing 
public  and  private  places,  as  well  as  by  varying  the  amount  of  digital  information  exchange  between  users  (see 
Figure  3).  Public  places  are  used  by  end  DL-users,  while  private  places  are  used  by  specific  DLrusers.  A place  for 
DL  communication  consist  of  a public  place  that  is  shared  by  users  as  a hallway,  and  private  places,  that  are 
used  by  these  users  as  their  classrooms.  The  hallway  is  provided  in  a virtual  space  on  the  network,  while  the 
private  classroom  space  is  located  in  a Digital  Library.  DLrusers  can  be  defined  independently  from  their  private 
places  locations.  The  virtual  hallway  and  the  private  classrooms  are  interconnected  by  virtual  links  that  are 
provided  by  the  messenger  GUI  located  in  their  networks  terminals. 

When  one  DL^iser  wishes  to  initiate  a video-conversation  with  other  DL-users,  the  first  user  send  a 
instant  message  through  the  Digital  Library  messaging  service  showing  his  or  her  picture  and  information  to  the 
DLrusers  group.  The  user  caller  can  wait  till  the  DUusers  group  is  available  without  interrupting  the  activities  of 
the  group  when  the  group  looks  busy.  After  the  user  decides  to  contact  the  DL-users  group,  the  user 
approaches  the  private  place  of  the  DLrusers  group,  then  our  implementation  combines  the  Windows  Media 
Player  and  NetMeeting  in  a manner  that  can  be  displayed  the  DLrvideo  image  along  with  the  basic  CVR  controls 
start,  stop  and  pause  (see  Figure  4)  . All  events  generated  by  the  VCR  controls  are  captured  by  the  UI  shell 
surrounding  the  media  player  and  transmitted  to  remote  DL-user  group’s  media  players  using  the  T.  120  data 
channel  provided  by  NetMeeting,  JMF  and  JDK. 


Figure  3:  Proposed  interaction  model. 


These  race  conditions  are  inherent  in  distributed  systems,  and  many  classes  of  solutions  are  possible. 
It  is  necessary  to  give  an  apparent  order  to  all  tasks  being  issued,  but  without  first  passing  them  through  a 
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central  messaging  service  to  avoid  video-communication  the  user  felt  uncomfortable  and  the  Digital  Library  does 
not  know  where  the  users  are.  In  our  first  software  implementation,  we  introduced  a notion  of  ordering  of  the 
transmitted  messages  using  the  time  stamps  of  the  DL-user  sender’s  video  stream  in  each  of  the  messages.  Since 
the  media  player  records  the  video  stamps  relative  to  the  start  of  the  video  stream,  these  timestamps  are  obsolete 
and  not  influenced  by  the  system  clock.  Any  player  that  receives  a message  with  a timestamp  earlier  than  the 
one  on  it  had  record  on  its  last  sent  message  ignores  the  received  message.  This  ensures  that  all  the  players 
record  the  message  that  has  the  latest  timestamp  relative  to  the  start  of  the  video  stream. 


Figure  4:  User  interface.  The  modified  media  player  used  by  the  participants  on  the  left. 

As  mentioned  in  Section  I,  we  wish  to  explore  the  impact  of  three  digital  communication  channels  in  a 
Digital  Library.  The  first,  videoconferencing  (transmission  of  life  video  of  all  DDusers)  required  extra  work.  A 
separate,  special-purpose  application  was  built  to  allow  participants  to  see  each  other.  This  application  only 
transmitted  video;  the  second,  audio-conference  was  provided  by  other  channel.  This  window  displayed  medium 
quality  video  (full  color,  15  frames/second)  of  all  DDusers  group.  Each  group  member’s  image  was  approximately 
5 centimeters  wide  and  4 centimeters  tall.  Video  was  captured  using  a camera  on  top  of  each  participant’s  monitor. 
The  third,  text  chat  was  supported  using  the  standard  chat,  with  these  tools  all  DDusers  were  communicated  in  a 
network-based  system. 

4 Study  focus 


A major  goal  of  this  ongoing  research  is  to  develop  a new  model  for  effective  collaboration  in  a 
videoconference-based  CSCL  environment  examining  how  various  communication  channels  affect  this  new 
model  in  a Digital  Library  environment.  Specifically,  we  are  interest  in  determining  whether  videoconferencing 
(audio  and  video)  is  the  best  option  relative  to  text  chat. 

We  are  examining  thKe  major  sets  of  variables  to  determine  the  utility  of  these  communication  channels. 
First,  because  this  model  is  been  designing  for  the  educational  context,  we  measured  participant  learning  and 
comprehension.  Second,  because  of  the  tight  relationship  between  learning  and  discussion  in  our  model,  we 
measured  various  aspects  of  the  users’  interaction  and  discussion.  Finally,  we  want  to  know  if  our  new  CSCL 
model  experience  is  the  best  way  for  effective  collaboration  in  a Digital  Library  environment.  Figure  5 shows  an 
example  screen  for  the  collaborative  video  viewing  condition.  For  this  first  implementation,  each  participant  was 
seated  alone  in  a room  with  PC.  All  participants  used  microphone  to  communicate,  but  when  the  shared 
DDvideo  played,  the  sound  from  the  computer  speakers  was  relayed  through  each  microphone,  causing  an 
unacceptable  distraction  (public  place).  For  the  second  test,  participants  were  seated  together  at  the  same  room. 
One  computer  class  was  played  using  our  first  implementation  model  (private  place),  except  the  computer’s 
screen  was  displayed  on  a projector  screen  at  the  end  of  the  room.  The  professor  who  was  teaching  used  a 
computer  and  mouse  to  pause  and  play  the  DDvideo  during  discussions.  Group  members  would  ask  the 
professor  to  pause  the  video  if  they  had  something  to  say;  often  they  would  just  being  talking  and  the  professor 
would  automatically  pause  the  video. 
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Figure  5:  Shows  an  example  screen  for  the  collaborative  video  viewing  condition. 

Although  the  groups  of  our  first  implementation  paused  the  DL-video  more  often,  the  difference  was 
not  significant.  In  addition,  both  test  groups  felt  that  the  number  of  pauses  was  about  right,  and  both  test 
groups  were  equally  comfortable  with  pausing  video.  Both  groups  were  also  equivalent  in  terms  of  time  spend 
discussing  the  DL-video  and  comfort  with  asking  questions  and  making  comments.  However,  the  first  test  users 
felt  that  there  was  too  much  discussion,  while  second  test  users  felt  that  the  amount  of  discussion  was  about 
right. 

5 Discussion 

First  we  should  note  shortcomings  of  this  ongoing  study.  First,  the  video  we  used  for  these  studies 
was  didactic  material  property  of  Digital  Library.  However,  if  the  distance  education  takes  hold,  it  is  likely  that 
more  resources  will  be  invested  in  producing  materials  of  higher  quality  than  a recorded  lecture.  Second,  this 
ongoing  research  has  the  generalizability  issues  common  to  our  lab  studies.  The  majority  of  our  participants 
were  students,  and  although  our  tests  settings  were  not  very  similar  to  a college  classroom  environment,  it  was 
similar  to  watch  is  found  in  distance  leaning  training  settings.  As  a result,  this  ongoing  research  could  benefit 
from  future  field  study  replication. 

The  first  goal  of  this  ongoing  research  is  to  build  and  effective  collaboration  model  in  a 
videoconference-based  CSCL  Digital  Library  environment  on  top  of  existing,  commercial  technology.  Ideally,  we 
hope  to  build  a model  that  would  recreate  the  collaboration  video  viewing  experience  for  students  and  DDusers 
who  could  not  meet  in  the  same  room.  Our  ongoing  studies  found  no  significant  differences  between  the  first 
and  second  test  groups  on  most  major  measures,  including  our  measures  of  leaning. 

However,  our  study  did  find  one  area  in  which  the  model  could  benefit  from  additional  design. 
Distributed  tutored  video  instruction  test  groups  were  not  as  comfortable  with  pausing  and  discussing  the 
Dl^video,  reflected  by  both  the  survey  data  and  the  reduced  time  spend  discussing  the  video  when  it  was 
paused.  This  result  is  worrisome  given  the  tight  assumed  connection  between  discussion  and  learning  in  the 
tutored  video  instruction  literature.  The  second  goal  of  this  ongoing  research  was  to  examine  the  effects  of 
using  different  communication  channels  to  link  Dl^users  together.  Aside  from  this  issue,  our  data  suggest  that 
providing  videoconference  is  a helpful  addition.  If  this  is  consistently  replicated,  it  is  good  news  for  Digital 
Libraries  implementations  that  wish  to  implement  distributed  tutored  video  instruction  providing  the  video 
channel. 


6 Concluding  Remarks 

The  effective  collaboration  in  a videoconference-CSCL  environment  model  in  a Digital  Library  provides 
significant  benefits:  scalability,  good  learning  outcomes,  and  location  and  time  flexibility  for  DL-users.  In  this 
paper  we  have  addressed  two  issues  that  together  allow  our  model  to  become  more  widespread.  First,  we  have 
shown  how  a collaborative  video  viewing  solution  can  be  built  on  existing  technologies  by  any  third  party. 
Seconds,  we  have  provided  results  on  Dl^users  learning,  interaction,  and  satisfaction  as  a function  of 
communication  channels  used  for  interaction.  Both  of  these  solutions  illustrate  that  existing  barriers  to 
distributed  tutored  video  interaction  can  be  overcome  without  tremendous  expense. 
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Abstract:  Problem  Based  Learning  is  of  great  use  in  Medical  Schools.  This  method 
is  based  on  patients’  cases  discussions  and  its  success  lies  in  the  fact  that  it 
stimulates  student’s  investigation  and  critical  thinking.  This  strategy  complements 
the  conceptual  learning  and  allows  the  students  to  deal  with  real  situations.  This 
paper  describes  a web-based  cooperative  learning  environment, 
CardioCaseDiscussion,  to  support  the  discussion  of  patients’  cases  among  medical 
students  and  professors  of  cardiology.  This  environment  is  part  of  CardioEducar,  an 
educational  meta-environment  which  purpose  is  to  integrate  several  educational 
environments. 


1 Introduction 

Problem  Based  Learning  (PBL)  has  become  the  outstanding  method  of  teaching  in  pre-clinical 
undergraduate  medical  education  (Eisner,  C.  et.  al.,  2000).  PBL  makes  students  more  active  during  the  learning 
process  because  they  feel  empowered  to  have  impact  on  the  outcome  of  the  investigation.  The  web  can  be  used 
to  support  on-line  discussion  in  PBL  approach  in  order  to  enhance  the  student  communication.  Online  discussion 
can  assist  learners  to  interact  and  collaborate  with  peers  or  experts  for  knowledge  construction  purpose. 


2 Cooperative  Learning 

Cooperative  learning  allows  students  to  help  each  other  in  the  learning  process  by  working  together  and/or 
with  an  instructor.  According  to  Comeaux,  P.  & Nixon,  M.  A.  (2000),  collaborative  technologies  allow  the 
construction  of  common  ways  of  seeing,  act  and  understand.  These  technologies  allow  people  to  develop  shared 
knowledge  and/or  new  group  activities.  The  communication  technology  used  in  the  learning  environment 
depends  on  the  educational  goals  of  the  learning  theory  adopted.  The  web  support  to  medical  cases  discussion 
provides  a virtual  environment  where  users  can  interact  from  different  remote  locations,  in  different  or 
synchronized  moments,  and  share  knowledge  information  (Osuna,  C.  A.  & Dimitriadis,  Y.  A.,  1999). 

CardioCaseDiscussion  is  designed  to  incorporate  problem-based  learning  into  a multi-user  computer- 
assisted  information  exchange  in  an  asynchronous  communication  way. 
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3 CardioCaseDiscussion  Environment 


CardioCaseDiscussion  is  one  of  the  learning  environments  of  CardioEducar,  a meta-environment  for 
cardiology  education  that  offer  computer-based  support  for  the  teaching  conferences  and  meetings  held  in  the 
Medical  School  and  the  University  Hospital  of  the  Federal  University  of  Bahia,  Brazil. 

CardioCaseDiscussion  supports  meetings  for  discussion  of  patients’  cases.  It  v^^as  developed  to  teach 
medical  students  to  analyze  patient’s  data  (history,  physical  examination  and  tests)  and  to  formulate  a diagnosis. 
The  environment  supports  cardiology  cases  discussion  in  three  different  situations:  (i)  a group  of  students 
oriented  by  a professor;  (ii)  a single  student  oriented  by  a professor,  and,  (iii)  a group  of  students  v^ithout  the 
professor  guidance.  It  can  also  be  used  by  a single  student  with  individual  learning  objective.  In  this  scenano 
there  will  be  no  opportunity  for  discussion. 

CardioCaseDiscussio/7  is  composed  by  four  environments: 

- Authoring  Environment,  which  supports  a professor  when  preparing  the  discussion  of  a patient’s  case.  It 
offers  facilities  to  insert  the  patient’s  data  (history,  physical  examination  and  tests  results). 

- Discussion  Environment,  which  presents  the  cases  and  monitors  the  discussion  among  students  and  the 
professor. 

- Meeting  Environment,  which  supports  the  case’s  data  presentation  in  a live  classroom  meeting. 

- Tutorial  Environment,  which  provides  medical  students  with  detailed  information  about  cardiology 
diseases  and  cardiology  tests. 

The  communication  among  participants  takes  place  by  means  of  text  messages  interchange. 
CardioCaseDiscussion  uses  categorization  and  structuring  of  messages  in  order  to  facilitate  the  argumentation 
and  to  guide  the  participants  to  reflect  about  their  messages.  The  users  can  participate  in  the  discussion  in  four 
different  ways:  (i)  asking  a question,  (ii)  answering  a question  of  another  participant,  (iii)  introducing  a 
commentary,  or,  (iv)  disagreeing  of  an  answer  or  a commentary.  CardioCaseDiscussion  supply  representative 
pieces  of  information  about  the  message  contents  to  help  user  to  identify  the  message  relevance  and  context. 
This  argumentation  model  is  used  to  facilitate  the  dynamic  of  the  discussion,  orienting  the  participants  on  how  to 
express  their  opinions  and  ideas  (Gerosa,  M.  A.  et.al.,  2000).  Awareness  is  implemented  in 
CardioCaseDiscussion  through  the  author  identification  of  each  contribution  for  the  discussion.  The  discussions 
are  asynchronous,  which  provide  more  flexibility  to  the  students  allowing  them  to  use  the  environment  according 
to  their  possibilities  or  preferences. 


4 Conclusions 

The  cardiology  cases  discussion  can  be  held  easily  with  the  CardioCaseDiscussion  support.  The  benefits  of 
this  environment  lie  on  its  support  to  problem  based  learning  and  its  ability  to  facilitate  user’s  access  and 
cooperative  work  through  the  web.  The  use  of  an  Internet  browser  provides  platform-independent  access  and 
allows  its  users  to  participate  in  the  discussions  at  different  time  and  locations.  The  environment  can  be  used  in 
Medical  Schools  or  as  a support  for  distance  learning  and  continuing  medical  education. 
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Abstract:  In  this  paper,  I review  a set  of  studies  that  examined  what  students  learn  in 
various  virtual  reality  environments  (VREs)  designed  to  promote  an  understanding  of 
environmental  science.  The  goal  of  the  reported  studies  was  to  provide  an  update  to  the 
classic  distinction  between  the  role  of  media  versus  method  in  promoting  learning  (Clark, 
1999).  Media  was  varied  by  comparing  how  students  learn  from  an  instructional  game 
delivered  via  a desktop  display  (D),  head  mounted  display  without  walking  (H),  and  head 
mounted  display  with  walking  (W).  The  instructional  method  was  varied  by  comparing 
how  students  learn  when  words  are  presented  as  on-screen  text  (T),  narration  (N),  or  both 
(NT). 


The  Role  of  Media  in  Virtual  Reality  Environments 

The  first  goal  of  this  research  was  to  examine  the  role  of  media  in  VREs.  Thus,  an  important  research  issue 
concerns  whether  more  immersive  VREs  (e.g.,  via  head  mounted  displays)  result  in  qualitatively  different  learning 
outcomes  than  less  immersive  VREs  (e.g.,  via  desktop  computer  displays).  Immersion  is  defined  as  the  extent  to 
which  computer  displays  are  capable  of  delivering  an  inclusive,  extensive,  surrounding,  and  vivid  illusion  of  reality 
to  the  senses  of  a human  participant  (Slater  & Wilbur,  1997).  How  does  immersion  affect  learning? 

First,  higher  levels  of  immersion  may  promote  a higher  sense  of  presence  which  in  turn,  may  promote  more 
engagement  and  deeper  learning.  Presence  is  a subjective  state,  the  psychological  sense  of  being  in  a VRE  (Slater  & 
Wilbur,  1997).  Past  research  in  VREs,  found  that  higher  levels  of  immersion  in  the  learning  environment  induce  a 
higher  sense  of  presence  during  the  learning  experience  as  measured  by  participants’  ratings  on  a presence 
questionnaire  (Welch  et  al.,  1996).  The  fundamental  idea  is  that  a higher  sense  of  being  in  the  environment  may 
encourage  the  deep  processing  of  the  learning  task  by  engaging  students  in  an  active  search  for  meaning  (Moreno  & 
Mayer,  2000). 

Second,  more  immersive  VREs  may  improve  transfer  of  learning  to  the  real  world  because  the  more  similar  a 
learning  environment  is  to  a real  environment,  the  better  the  transfer  of  learning  (Durlach  & Mavor,  1995). 
Preliminary  evidence  supports  the  idea  that  the  richer  the  perceptual  cues  and  multimodal  feedback  (e.g.,  visual, 
auditory,  haptic,  etc.)  the  more  likely  the  transfer  of  VR  training  to  real-world  skill  (Regian,  Shebilske,  & Monk, 
1992).  This  idea  is  also  consistent  with  one  of  the  grand  aims  of  VREs  for  learning:  to  provide  interface 
transparency  by  making  the  interaction  with  technology  more  “natural”.  The  cognitive  advantage  of  a transparent 
interface,  which  is  generally  concomitant  with  a more  immersive  environment,  is  that  it  drives  students’  limited 
attentional  resources  to  learning  the  content  material  rather  than  to  the  interface. 

The  Role  of  Method  in  Virtual  Reality  Environments 

The  second  goal  of  this  research  was  to  examine  the  role  of  method  in  VREs.  Method  refers  to  the  instructional 
method  implemented  in  the  design.  For  the  case  of  the  reported  studies,  the  role  of  the  modality  of  the  verbal 
information  was  examined.  More  specifically,  the  instructional  method  was  varied  by  presenting  the  words  printed 
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as  on-screen  text  (T),  spoken  as  narration  (N),  or  both  (NT).  How  does  the  modality  of  the  verbal  information  affect 
learning?  To  answer  this  question,  I start  with  a dual-processing  theory  of  multimedia  learning. 

According  to  dual-processing  theory,  students  who  can  hold  corresponding  pictorial  and  verbal  representations 
in  working  memory  at  the  same  time  (such  as  the  dynamic  graphics  of  the  VREs  held  in  visual  working  memory  and 
the  corresponding  narrated  explanations  held  in  auditory  working  memory)  are  better  able  to  build  referential 
connections  between  them  (Paivio,  1986).  On  the  other  hand,  given  the  limited  resources  students  have  for  visual 
working  memory,  using  a visual  modality  to  present  both  pictorial  and  verbal  information  can  create  an  overload 
situation  for  the  learner  (Baddeley,  1992;  Chandler  & Sweller,  1991;  Sweller,  1989).  Previous  research  in  a desktop 
environment  has  shown  that  students  learn  better— as  measured  by  tests  of  retention  and  transfer— when  the 
instructional  method  involves  spoken  rather  than  printed  explanations  (Mayer  & Moreno,  1998;  Moreno  & Mayer, 
1999;  Moreno  & Mayer,  2002;  Moreno,  Mayer,  Spires,  & Lester,  2001).  Similar  results  were  obtained  in  an 
immersive  VR  training  environment  (O’Neil  et  al.,  2000).  This  effect  of  instructional  method  has  been  called  the 
modality  effect  (Moreno  & Mayer,  1999;  Moreno  et  al.,  2001). 

Does  redundant  verbal  information  (identical  narration  and  on-screen  text  presented  simultaneously)  facilitate 
students’  understanding  of  an  interactive  VR  program  further?  Recent  findings  on  learning  with  short  multimedia 
explanations  have  shown  that  when  instructional  materials  contain  simultaneous  pictorial  information  such  as 
animations,  adding  identical  on-screen  text  to  a spoken  explanation  hurts  rather  than  helps  students’  learning 
(Mayer,  Reiser,  & Lonn,  2001;  Moreno  & Mayer,  2002).  The  negative  effects  of  redundant  verbal  messages  have 
been  interpreted  as  due  to  learners’  need  to  split  their  limited  visual  attention  between  simultaneous  text  and 
graphics.  This  effect  of  instructional  method  has  been  called  the  redundancy  effect. 

Media  Versus  Method 

The  decision  to  use  different  delivery  media  (D,  H,  or  W)  or  methods  (N,  T,  or  NT)  in  instructional  design  may 
depend  on  one’s  conception  of  learning.  In  this  section,  I consider  the  following  two  views:  media-affects  learning 
and  method-affects-leaming. 

Media-Affects-Learning 

On  one  hand,  a media-affects-leaming  view  is  that  the  delivery  medium  can  affect  what  is  learned.  Media 
refers  to  the  delivery  device;  in  this  case  I varied  the  degree  of  immersion  by  presenting  the  instructional  program 
via  a desktop  computer  screen  (D),  via  a head  mounted  display  while  the  learner  is  seated  (H),  or  via  a head 
mounted  display  while  the  learner  can  walk  (W).  For  example,  Seidel  and  Chatelier  (1997,  p.  2)  call  for  research 
comparing  how  people  learn  from  desktop  workstations  versus  from  head  mounted  displays  (HMDs)  by  asking, 
“What  is  the  value  added  by  HMD,  and  what  purposes  or  what  types  of  tasks  does  it  serve  best  for  learning 
purposes?” 

According  to  the  media-affects-leaming  view,  immersive  VREs  (such  as  in  conditions  W and  H)  have  the 
potential  of  making  computer-based  learning  feel  more  real  by  promoting  a sense  of  presence,  which  in  turn,  could 
promote  deeper  learning.  The  effect  of  media  would  be  reflected  in  superior  performance  on  tests  of  retention  and 
transfer  for  students  who  experience  more  immersive  environments.  If  media  affects  learning,  then  a media  effect 
would  be  expected,  that  is,  performance  on  tests  of  retention  and  transfer  depends  on  whether  the  instructional 
program  was  presented  by  desktop  display,  head  mounted  display,  or  head  mounted  display  with  walking.  This 
pattern  would  be  reflected  in  a main  effect  for  media  and  no  interaction  between  method  and  media. 

Method-Affects-Learning 

On  the  other  hand,  the  method-affects-leaming  view  is  that  instmctional  methods  influence  learning. 
Instmctional  methods  can  guide  cognitive  processing  in  the  learner,  which  influences  the  knowledge  that  the  learner 
is  able  to  constmct.  According  to  the  method-affects-leaming  view,  it  is  not  technology  per  se  that  promotes 
learning  but  rather  how  the  technology  is  used.  As  long  as  instmctional  methods  promote  appropriate  cognitive 
processing  during  learning,  then  media  does  not  seem  to  matter  (Clark,  1 999).  Although  VR  offers  a compelling 
technological  breakthrough,  the  medium  itself  does  not  promote  learning  but  rather  the  instmctional  methods  that  it 
affords  promote  learning. 

In  the  present  review,  the  modality  of  the  verbal  information  was  used  as  instmctional  method.  Therefore,  the 
effect  of  method  would  be  reflected  in  two  ways.  First,  by  demonstrating  a modality  effect,  that  is,  superior 
performance  on  tests  of  retention  and  transfer  for  students  who  receive  verbal  explanations  as  narration  (N)  rather 
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than  as  on-screen  text  (T).  Second,  by  demonstrating  a redundancy  effect.  Students  who  receive  verbal 
explanations  as  narration  (N)  will  outperform  those  who  receive  redundant  explanations  (NT)  on  tests  of  retention 
and  transfer. 

The  main  interest  of  the  reported  studies  is  the  general  issue  of  whether  method  affects  learning  across  media- 
that  is,  whether  instructional  methods  have  the  same  effects  within  different  media  environments.  If  method  affects 
learning,  then  a modality  or  redundancy  effect  can  be  expected  within  each  of  the  VREs  examined— desktop  display, 
head-mounted  display,  and  head  mounted  display  with  walking.  This  pattern  would  be  reflected  in  a main  effect  for 
method  and  a lack  of  interaction  between  method  and  media. 

A Virtual  Reality  Agent-Based  Micorworld 

The  learning  environment  used  in  the  reported  studies  is  a computer  game  called  "Design-A-Plant",  developed 
by  the  Multimedia  Laboratory  at  the  Department  of  Computer  Science  of  North  Carolina  State  University  (Lester  & 
Stone  1997).  In  this  program,  the  student  travels  to  an  alien  planet  that  has  certain  environmental  conditions  (e.g., 
low  rainfall,  light  sunlight)  and  must  design  a plant  that  would  flourish  there  (e.g.,  including  designing  the 
characteristics  of  the  leaves,  stem,  and  roots).  It  includes  a pedagogic  agent  who  offers  individualized  advice 
concerning  the  relation  between  plant  features  and  environmental  features,  encouragement  when  students  encounter 
difficulties,  and  feedback  on  the  choices  that  students  make  in  the  process  of  designing  plants.  The  program  starts 
with  the  agent  introducing  the  student  to  the  first  set  of  environmental  conditions.  Then,  he  asks  the  student  to 
choose  the  appropriate  root  from  the  library  of  roots'  names  and  graphics  shown  on  the  computer  screen.  After  the 
student  had  chosen  a root,  the  agent  gives  a narrated  explanation  for  the  correct  root.  The  same  procedure  applies  to 
the  stem  and  leaves,  with  the  agent  first  asking  the  student  to  make  a choice,  and  giving  the  student  feedback 
afterwards.  Once  the  student  is  given  the  last  explanation  on  the  leaves  for  each  environment,  he  is  taken  to  the  next 
environment.  The  same  procedure  follows  for  the  rest  of  the  environments. 

Experiment  1:  Media  Versus  Modality  Effects 

Method  and  Results 

The  participants  were  89  college  students.  Each  participant  served  in  one  cell  of  a 2 x 3 between-subjects 
factorial  design,  with  the  first  factor  being  modality  of  the  verbal  information  (narration  or  text)  and  the  second 
factor  being  the  level  of  immersion  during  the  computer  interaction  (desktop,  head  mounted  display,  or  head 
mounted  display  plus  walking).  There  were  17  participants  in  the  narration  and  desktop  group  (ND  Group),  17 
participants  in  the  text  and  desktop  group  (TD  Group),  13  participants  in  the  narration  and  head  mounted  display 
group  (NH  Group),  13  participants  in  the  text  and  head  mounted  display  group  (TH  Group),  13  participants  in  the 
narration  and  head  mounted  display  plus  walking  group  (NW  Group),  and  14  participants  in  the  text  and  head 
mounted  display  plus  walking  group  (TW  Group). 

After  interacting  with  the  respective  program,  participants  were  tested  on  three  important  measures  of  learning: 
retention-in  which  memory  for  the  basic  factual  information  was  assessed;  problem-solving  transfer-in  which 
students  were  asked  to  solve  new  problems  based  on  the  principles  learned  in  the  program;  and  program  ratings-in 
which  students  were  asked  to  rate  their  level  of  motivation,  interest,  understanding,  and  the  perceived  difficulty  and 
friendliness  of  the  lesson.  Participants  also  completed  a presence  questionnaire.  We  determined  whether  the  groups 
differed  on  measures  of  retention,  transfer,  and  program-ratings  by  conducting  two-factor  analyses  of  variance  for 
each  dependent  measure  with  modality  (T  or  N)  and  level  of  immersion  (D,  H,  or  W)  as  between-subject  factors, 
and  retention,  transfer,  and  program  ratings  as  the  respective  dependent  measure. 

Do  students  experience  a stronger  sense  of  presence  in  more  immersive  VREs? 

Consistent  with  past  research  in  VREs,  higher  levels  of  immersion  in  the  learning  environment  induced  a higher 
sense  of  presence  during  the  learning  experience  (Welch  et  al.,  1996).  Using  presence  as  a dependent  measure,  a 
one-factor  ANOVA  revealed  that  students  in  D groups  rated  their  sense  of  presence  significantly  lower  than  students 
in  H and  W groups  which  did  not  differ  from  each  other,  Ms  = -2.59,  7.31,  and  8.22;  SDs  = 12.49,  10.03,  and  12.33, 
for  D,  H,  and  W groups,  respectively,  p = .0006. 
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Do  more  immersive  VREs promote  deeper  learning  than  less  immersive  VREs? 


The  media-affects-leaming  view  holds  that  more  immersive  VREs  are  more  likely  to  promote  students'  learning 
of  the  science  lesson,  by  virtue  of  inducing  higher  levels  of  presence.  Using  retention  as  a dependent  measure,  a 
two-factor  ANOVA  failed  to  reveal  a main  effect  for  immersion.  Groups  presented  with  higher  levels  of  immersion 
did  not  differ  in  the  mean  number  of  recalled  items  about  the  plant  library  from  those  presented  with  lower  levels  of 
immersion  (Ms  = 6.01,  6.50,  and  5.82;  SDs  = 1.85,  1.85,  and  2.04,  for  D,  H,  and  W groups,  respectively).  There 
was  no  significant  interaction  between  immersion  and  modality. 

Using  transfer  as  a dependent  measure,  a two-factor  ANOVA  failed  to  reveal  a main  effect  for  immersion. 
Groups  presented  with  higher  levels  of  immersion  did  not  differ  in  the  mean  number  of  answers  from  those 
presented  with  lower  levels  of  immersion  (Ms  = 31.06,  30.69,  and  30.52;  SDs  = 9.54,  9.20,  and  1 1.68,  for  the  D,  H, 
and  W groups,  respectively).  A significant  interaction  was  found  between  immersion  and  modality  (p  < 0.05).  Post 
hoc  analysis  of  simple  effects  for  modality  and  immersion  indicated  that  both  for  D and  H conditions,  receiving 
information  via  on-screen  text  rather  than  via  narration,  proved  to  significantly  hinder  students’  learning  as 
measured  by  transfer  scores  (,p  = .006  and  .0001,  respectively).  Similar  to  the  case  for  retention,  there  was  no 
immersion  effect  for  students’  program  ratings  and  no  interaction.  The  respective  ratings  for  the  D,  H,  and  W 
groups  were:  Ms  = 28.67,  31.83,  and  31.52,  SD  = 6.61,  7.02,  and  6.88. 

Do  students  who  learn  with  narration  learn  more  deeply  than  students  who  learn  by  reading  on-screen  text? 

The  method-affects-leaming  view  holds  that  presenting  verbal  material  as  speech  is  more  likely  to  promote 
students'  understanding  of  a multimedia  lesson  than  presenting  the  same  material  as  on-screen  text,  regardless  of 
delivery  medium.  Using  retention  as  a dependent  measure,  a two-factor  ANOVA  revealed  a main  effect  for  modality 
(p  = .0003),  with  a mean  number  of  ideas  recalled  of  6.84  and  5.43  respectively  for  the  narration  and  text  groups 
(SDs  =1.51  and  2.01,  respectively). 

Using  transfer  as  a dependent  measure,  a two-factor  ANOVA  revealed  a main  effect  for  modality  (p  = .0001), 
with  a mean  number  of  correct  answers  of  36.12  and  25.57  respectively  for  the  narration  and  text  groups  (SDs  = 
8.34  and  8.80,  respectively).  Groups  presented  with  the  verbal  information  in  the  form  of  speech  gave  significantly 
more  correct  answers  than  those  presented  with  the  verbal  information  in  the  form  of  text.  Groups  also  differed  in 
the  overall  ratings  for  the  program.  A two-way  ANOVA  using  program  ratings  as  the  dependent  measure  revealed 
that  the  narration  groups  rated  the  program  more  favorably  than  the  text  groups  { p < .05),  with  a mean  rating  of 
32.33  and  28.72  respectively  for  the  narration  and  text  groups  (SDs  = 5.85  and  7.42,  respectively). 

The  results  from  Experiment  1 supported  the  method-affects-leaming  hypothesis  by  demonstrating  a modality 
effect  on  retention,  problem-solving  transfer,  and  program  ratings.  No  evidence  was  found  in  favor  of  the  media- 
affects-1  earning  hypothesis.  The  purpose  of  Experiment  2 was  to  test  the  alternative  hypothesis  using  a different 
type  of  instmctional  method.  The  effects  of  presenting  redundant  verbal  information  by  means  of  a VR  desktop 
display  (D)  or  by  means  of  a head  mounted  display  without  walking  (H)  was  examined.  Students  received 
explanations  from  an  agent  via  narration  alone  (N),  via  on-screen  text  alone  (T),  or  via  simultaneous  narration  and 
on-screen  text  explanations  (NT). 

Experiment  2:  Media  Versus  Redundancy  Effects 

Method  and  Results 

The  participants  were  75  college  students.  Each  participant  served  in  one  cell  of  a 3 x 2 be  tween-subjects 
factorial  design,  with  the  first  factor  being  modality  of  the  verbal  information  (narration,  text,  or  narration  and  text) 
and  the  second  factor  being  the  level  of  immersion  during  the  computer  interaction  (desktop  or  head  mounted 
display).  There  were  14  participants  in  the  narration  and  desktop  group  (ND  Group),  14  participants  in  the  text  and 
desktop  group  (TD  Group),  14  participants  in  the  redundant  desktop  group  (NTD  Group),  1 1 participants  in  the  text 
and  head  mounted  display  group  (TH  Group),  10  participants  in  the  narration  and  head  mounted  display  group  (NH 
Group),  and  12  participants  in  the  redundant  head  mounted  display  group  (NTH  Group).  The  procedure  was 
identical  to  that  of  the  first  experiment.  To  determine  whether  treatment  groups  differed  on  measures  of  retention, 
transfer,  and  program  ratings,  separate  two-factor  analyses  of  variance  were  conducted  for  each  dependent  measure 
with  modality  (N,  T or  NT)  and  level  of  immersion  (D  or  H)  as  between- subject  factors,  and  retention,  transfer,  and 
program  ratings  as  the  respective  dependent  measure. 
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Do  students  experience  a stronger  sense  of  presence  in  more  immersive  VREs? 

Similar  to  Experiment  1,  the  results  showed  that  programs  delivered  via  head-mounted  displays  rather  than 
desktop  displays  induce  higher  levels  of  presence  (p  = .03).  Students  in  the  D groups  rated  their  sense  of  presence 
significantly  lower  than  students  in  the  H groups.  (Ms  = .81  and  8.73;  SDs  = 12.95  and  17.93,  for  D and  H groups, 
respectively. 

Do  more  immersive  VREs  promote  deeper  learning  than  less  immersive  VREs? 

Using  retention  as  a dependent  measure,  a two-factor  ANOVA  failed  to  reveal  a main  effect  for  immersion  or 
an  interaction  between  immersion  and  modality.  Groups  who  learned  in  the  desktop  conditions  did  not  differ  in  the 
mean  number  of  recalled  items  about  the  plant  library  from  groups  who  learned  with  higher  levels  of  immersion  (Ms 
= 6.81,  and  6.55;  SDs  = 1.55  and  1.68,  for  D and  H groups,  respectively). 

Using  transfer  as  a dependent  measure,  a two-factor  ANOVA  failed  to  reveal  a main  effect  for  immersion  or  an 
interaction  between  immersion  and  modality.  Groups  presented  with  higher  levels  of  immersion  did  not  differ  in  the 
mean  number  of  answers  from  groups  presented  with  lower  levels  of  immersion  (Ms  = 28.17,  and  30.55;  SDs  = 7.47 
and  7.36,  for  the  D and  H groups,  respectively).  Similarly,  a two-way  ANOVA  using  the  overall  program  rating  as 
the  dependent  measure  failed  to  reveal  an  immersion  effect.  The  respective  ratings  for  the  D and  H groups  were:  Ms 
= 30.14  and  28.49,  SD  = 6.95  and  7.02.  There  was  no  significant  interaction  between  immersion  and  modality. 

Do  students  who  learn  with  redundant  explanations  learn  more  deeply  than  students  who  learn  with  explanations  in 
only  one  modality? 

The  method-affects-leaming  view  holds  that  presenting  verbal  material  as  narration  alone  is  more  likely  to 
promote  students’  understanding  of  a multimedia  lesson  than  presenting  the  same  verbal  material  as  narration  and 
on-screen  text,  regardless  of  delivery  medium.  Using  retention  as  a dependent  measure,  a two-factor  ANOVA 
revealed  a main  effect  for  modality  (p  = .006),  with  a mean  number  of  ideas  recalled  of  7.25,  6.96,  and  5.88 
respectively  for  the  N,  NT,  and  T groups  (SDs  = 1.57,  1.46,  and  1.51,  respectively).  Supplemental  Tukey  tests  (with 
alpha  at  .05)  revealed  that  groups  presented  with  on-screen  text  alone  recalled  significantly  fewer  items  from  the 
plant  library  than  groups  presented  with  narration  alone  or  with  narration  plus  on-screen  text,  which  did  not  differ 
from  each  other. 

Using  transfer  as  a dependent  measure,  a two-factor  ANOVA  revealed  a main  effect  for  modality  (p  = .0004), 
with  a mean  number  of  correct  answers  of  32.08,  30.77,  and  24.84,  respectively  for  the  N,  NT,  and  T groups  (SDs  = 
6.16,  8.15,  and  5.96,  respectively).  Similar  to  the  case  of  retention,  supplemental  Tukey  tests  (with  alpha  at  .05) 
revealed  that  groups  presented  with  on-screen  text  alone  gave  significantly  less  correct  answers  to  problem-solving 
transfer  tests  than  groups  presented  with  narration  alone  or  with  narration  plus  on-screen  text,  which  did  not  differ 
from  each  other.  Groups  also  differed  in  the  overall  ratings  for  the  program  (p  = .01),  with  a mean  rating  of  32.54, 
29.31,  and  26.52  respectively  for  the  N,  NT,  and  T groups  (SDs  = 5.36,  6.87,  and  7.43,  respectively).  Supplemental 
Tukey  tests  (with  alpha  at  .05)  revealed  that  groups  presented  with  on-screen  text  alone  gave  significantly  lower 
program  ratings  than  students  presented  with  narration  alone. 

Overall,  modality  effects  were  obtained  on  the  retention,  transfer,  and  program  ratings  replicating  the  pattern 
found  in  Experiment  1 and  thus  supporting  the  method-affects-leaming  hypothesis.  However,  redundant  verbal 
information  did  not  hurt  or  help  students’  understanding  as  compared  to  spoken  explanations  alone.  A possible 
interpretation  for  NT  groups  performing  comparably  to  N groups  is  that  students  in  NT  groups  may  have  been 
inclined  to  attend  to  the  narration  alone  due  to  the  experiential  mode  of  VREs.  When  students  are  exploring  a VRE 
(either  by  moving  the  computer  mouse  or  by  moving  their  head),  it  is  less  likely  that  they  will  read  a box  containing 
text  if  they  can  obtain  the  same  information  by  listening  to  a narration. 


General  Discussion 

The  present  review  yields  evidence— based  on  questionnaires— that  students  feel  a stronger  sense  of  presence  in 
more  immersive  VREs.  In  addition,  the  reported  studies  provide  new  evidence-from  retention  and  transfer  tests- 
that  students  who  experience  more  immersive  VREs  do  not  necessarily  learn  a computer-based  lesson  more  deeply 
than  students  who  experience  a lower  sense  of  presence.  This  is  consistent  with  past  research  in  VREs  where  no 


1416 


relationship  between  presence  and  performance  was  found  (Slater  & Wilbur,  1997)  and  fails  to  support  a media- 
affects- learning  hypothesis. 

Theoretically,  this  research  provides  one  of  the  first  methodologically  rigorous  studies  of  conditions  that  foster 
productive  learning  in  a virtual  environment.  The  replication  of  the  modality  effect  across  different  media  shows 
that  effective  learning  depends  on  which  instructional  techniques  help  guide  the  learner’s  cognitive  processing  of  the 
presented  material  rather  than  on  the  medium  per  se.  Importantly,  this  review  has  shown  that  the  same  factors  that 
improve  student  understanding  in  one  medium  (such  as  modality  effects  in  a desktop  environment)  improve  student 
understanding  in  another  medium  (such  as  the  present  findings  concerning  immersive  VREs).  In  both  cases, 
ineffective  instructional  messages  can  be  converted  into  effective  ones  by  applying  the  same  instructional  design 
principles. 

Finally,  the  conclusions  drawn  are  limited  by  the  nature  of  the  learning  materials.  The  learning  materials 
consisted  of  an  environmental  science  VRE  with  short  agent  and  student  interventions.  It  is  possible  that  in  VREs 
where  students’  physical  interventions  are  essential  to  the  learning  process,  such  as  if  the  goal  of  the  instructional 
material  is  to  train  a procedure,  the  use  of  more  immersive  environments  might  play  an  important  role  in  adding 
psychomotor  feedback  to  the  learning  experience  (Seidel  & Chatelier,  1997;  Thurman  & Russo,  2000).  Because 
some  media  may  enable  instructional  methods  that  are  not  possible  with  other  media,  it  might  be  useful  to  explore 
instructional  methods  that  are  possible  in  immersive  environments  but  not  in  others. 
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Abstract:  Who  leams  best  with  multiple  representations?  In  this  paper,  I present  a cognitive 
theory  of  multimedia  learning  from  which  predictions  on  individual  differences  in  learning  are 
derived  and  tested.  Elementary  students  learned  how  to  add  and  subtract  integers  with  an 
interactive  multimedia  game  that  included  visual  and  symbolic  representations  of  the  arithmetic 
procedure.  They  learned  either  with  or  without  verbal  guidance  in  their  first  language.  Verbal 
guidance  was  expected  to  help  minimize  cognitive  load,  especially  for  students  with  low  prior 
knowledge,  low  computer  experience,  and  a less  reflective  cognitive  style.  The  theoretical  and 
practical  implications  of  the  results  are  discussed. 


Multiple  Representations  for  Meaning  Making  in  Math 

Multimedia  environments  allow  learners  to  integrate  information  from  different  representation  formats  and 
sensory  modalities  into  one  meaningful  experience  (Moreno  & Mayer,  2000).  Therefore,  when  an  arithmetic 
procedure  is  to  be  taught  with  a multimedia  program  or  game,  the  instructional  designer  is  faced  with  the  need  to 
choose  between  several  combinations  of  modes  and  modalities  to  promote  meaningful  learning.  In  the  present 
review  of  studies,  students  learned  to  add  and  subtract  integers  by  interacting  with  a multimedia  game  consisting  of 
symbolic  representations  in  the  traditional  number  sentence  format.  In  addition,  based  on  recent  advances  in 
multimedia  learning,  the  computer  game  depicts  the  execution  of  the  computational  procedures  as  visual 
representations  of  movements  along  a number  line  (Moreno  & Mayer,  1999).  According  to  the  arithmetic-is-motion 
metaphor,  "numbers  are  locations  on  a path”,  "the  mathematical  agent  is  a traveler  along  that  path”,  "arithmetic 
operations  are  acts  of  moving  along  the  path”,  and  "the  result  of  an  arithmetic  operation  is  a location  on  the  path” 
(Lakoff  & Nunez,  1997,  p.  37). 

For  example,  students  select  one  of  eight  problems  to  solve  from  the  problem  menu.  Then,  students  see  the 

problem  presented  in  symbolic  form  (as  4 - -5  = ) and  a number  line  showing  integers  from  -9  to  9 with  a bunny 

standing  at  the  0 point.  A simulated  joystick  consisting  of  the  following  four  alternative  moves  for  the  bunny 
appears  in  the  lower  right  comer:  face  to  the  left,  face  to  the  right,  jump  forward  one  step,  or  jump  backwards  one 
step.  Students  may  click  on  any  combination  of  the  four  joystick  options  and  instantly  see  the  resulting  change  in 
the  bunny  on  the  number  line.  The  program  instmcts  learners  to  try  to  figure  out  the  problem  by  moving  the  bunny 
along  the  number  line  using  the  joystick.  When  students  are  ready  to  answer,  they  type  in  a numeral  (and  negative 
sign,  if  needed). 

If  the  student’s  answer  is  not  correct,  the  student  may  try  again  or  see  the  solution  to  the  problem.  If 
students  type  in  the  correct  answer,  they  hear  the  word  “Yes”  followed  by  an  animated  sequence  which  consists  of 
four  major  steps  in  solving  the  problem.  First,  the  symbol  ”4”  is  highlighted  and  the  bunny  moves  to  position  4 on 
the  number  line.  Second,  the  minus  sign  is  highlighted  and  the  bunny  turns  to  the  face  the  left  side  of  the  screen. 
Third,  the  symbol  ”-5"  is  highlighted  and  the  bunny  makes  five  jumps  to  the  right.  Finally,  the  number  “9”  is 
highlighted  and  the  bunny  faces  forward  on  the  9th  position  of  the  number  line.  Thus,  the  learner  sees  how  each 
step  in  the  procedure  can  be  represented  in  symbols  and  movements.  At  this  point,  students  may  click  on  the  "Back 
to  Menu”  button  which  takes  them  back  to  the  main  menu. 


Would  the  addition  of  a verbal  representation  in  the  form  of  spoken  words  explaining  the  relationship 
between  the  visual  metaphor  and  the  symbolic  sentence  help  students’  learning?  Moreover,  do  all  students  benefit 
from  learning  with  more  rather  than  fewer  representations?  To  help  answer  these  questions,  I conducted  two  studies 
where  the  learning  outcomes  of  students  who  learned  to  add  and  subtract  integers  with  symbolic  and  visual 
representations  (Group  SV)  was  compared  to  those  of  students  who  learned  the  same  lesson  with  symbolic,  visual, 
and  verbal  representations  (Group  SVV).  In  particular,  I concentrated  on  three  learning  measures:  learning  rate, 
pretest-to-posttest  gain,  and  using  what  they  have  learned  to  solve  word  problems.  The  overarching  goal  of  the 
reported  studies  was  to  explore  the  cognitive  implications  of  training  students  with  multiple  representations  in  a high 
cognitive  load  situation  and  infer  some  preliminary  principles  of  instructional  design  for  individual  differences  in 
multimedia  learning.  In  the  next  sections,  I provide  a cognitive  theory  of  how  people  learn  with  multiple 
representations,  introduce  three  possible  sources  of  individual  differences  in  learning  from  multimedia 
environments,  and  derive  predictions  based  on  the  theory. 

A Cognitive  Theory  of  Multimedia  Learning 

The  cognitive  theory  of  multimedia  learning  used  in  the  present  studies  draws  on  two  main  assumptions: 
multiple  representations  help  learning  and  cognitive  load  hurts  learning.  One  one  hand,  according  to  a multiple 
representation  hypothesis,  teaching  with  more  representations  facilitates  and  strengthens  the  learning  process  by 
providing  several  mutually  referring  sources  of  information  (Kozma,  Russell,  Jones  & Marx,  1996;  Grouws,  1992; 
National  Council  of  Teachers  of  Mathematics,  1989).  For  example,  past  studies  in  multimedia  learning  have  shown 
that  students  learn  math  better  when  information  is  presented  in  verbal,  visual,  and  symbolic  formats  rather  than  in 
symbolic  formats  alone  (Moreno  & Mayer,  1999),  and  that  students  learn  science  better  from  animations  that 
include  words  rather  than  from  animations  or  words  alone  (Mayer,  2001). 

On  the  other  hand,  according  to  the  thesis  that  cognitive  load  hurts  learning,  students’  working  memories 
are  limited  in  capacity  (Baddeley,  1986;  Chandler  & Sweller,  1992;  Sweller,  1994).  Therefore,  presenting  multiple 
representations  of  the  same  arithmetic  procedure  may  overload  the  learner  by  creating  the  need  to  integrate  the 
multiple  sources  of  mutually  referring  information.  Students  with  larger  capacity  are  better  able  to  maintain  and 
coordinate  multiple  representations  in  working  memory  than  are  students  with  lower  capacities  (Just  & Carpenter, 
1992).  Thus,  although  cognitive  theory  of  multimedia  learning  supports  learning  with  multiple  representations  in 
general,  individual  differences  cannot  be  discarded  (Moreno  & Mayer,  1999).  What  are  some  possible  sources  of 
cognitive  load  differences  between  students? 

Cognitive  Tempo 

Cognitive  tempo  (or  Impulsivity/Reflectivity)  refers  to  a person’s  tendency  to  reflect  or  not  reflect,  before 
responding  on  a problem-solving  task  with  response  uncertainty  (Kagan,  1966).  According  to  this  individual 
difference,  reflective  students  make  fewer  errors  and  have  a longer  response  time  than  impulsive  students  on 
inductive  reasoning,  recognition  memory,  visual  discrimination  tasks,  and  serial  recall  (Borkowski,  Peck,  Reid,  & 
Kurtz,  1983).  From  a theoretical  perspective,  it  can  be  hypothesized  that  a persistent  impulsive  style  limits  students’ 
time  to  reflect  on  the  accuracy  of  an  answer.  Reflective  students  take  more  time  to  develop  a solution  hypothesis 
before  making  a choice  (clicking  on  the  computer  mouse).  A consequence  is  that  deep  learning  becomes  less  likely 
for  impulsive  than  for  reflective  students,  especially  where  tasks  carry  high  uncertainty  and  little  guidance  (Jonassen 
& Grabowski,  1993). 

Prior  Knowledge 

Based  on  a cognitive  theory  of  multimedia  learning,  students  with  high  prior  knowledge  in  arithmetic  have 
already  built  their  own  mental  model  for  the  presented  material  (at  least  partially,  for  the  addition  and  subtraction  of 
natural  numbers).  Therefore,  multiple  representations  are  easier  to  integrate  with  prior  knowledge  and  less  likely  to 
overload  their  working  memory  (Mayer,  2001;  Moreno  & Mayer,  1999).  In  addition,  computer  knowledge  or 
experience  (mastery  of  the  task-specific  skills  that  derive  from  interacting  with  a multimedia  program  such  as 
clicking  on  buttons,  typing  in  numbers,  etc.)  may  affect  multimedia  learning.  Low  computer  experience  may 
impose  an  extra  cognitive  load  if  students  are  not  very  familiar  with  computers  and  the  tasks  involved  in  the 
interaction  have  not  become  automatic  yet  (Chandler  & Sweller,  1991).  This  is  particularly  relevant  for  the  present 
study  because  many  participants  are  children  who  come  from  low-income,  immigrant,  non-English  backgrounds 
with  very  few  having  access  to  computers  at  home. 
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Language  Background 


What  are  the  cognitive  consequences  of  learning  verbal  materials  in  a more  versus  less  familiar  language? 
An  additional  goal  of  the  present  study  was  to  examine  if  students  would  choose  to  learn  the  arithmetic  procedure  in 
their  first  or  second  language  given  the  option  to  do  so.  Knowledge  of  a less  familiar  language  constrains  problem 
solving  in  that  language  (Duran,  1985).  As  the  computer  game  used  for  the  present  studies  presented  verbal 
explanations  in  English  with  the  option  to  listen  to  Spanish  translations,  it  was  expected  that  limited  English 
proficient  students  (LEP)  would  make  significant  use  of  the  translations  during  training  sessions.  The  rationale  is 
based  on  the  assumptions  that  attentionai  resources  are  limited  and  that  in  learning  situations  of  high  cognitive  load 
such  as  the  present  one,  students  will  be  likely  to  lower  the  attentionai  demands  by  choosing  to  use  a more  familiar 
language. 

Predictions 

Based  on  a cognitive  theory  of  multimedia  learning,  students  trained  with  example  problems  presented  in 
three  different  forms  encode  the  material  more  deeply  than  those  trained  with  problems  presented  in  two  forms 
(Clark  & Paivio,  1991;  Paivio,  1986).  Moreover,  cognitive  load  for  the  SW  group  is  minimized  respect  to  the  SV 
group  because  the  extra  verbal  representation  guides  students’  discovery  of  the  relationship  between  the  visual 
metaphor  and  the  number  sentence  (Chandler  & Sweller,  1991;  Solter  & Mayer,  1978;  Tarmizi  & Sweller,  1988). 
Consequently,  it  was  predicted  that  compared  to  the  SV  group,  the  SW  group  would  show  a larger  pretest-to- 
posttest  gain,  faster  rate  of  learning,  and  better  transfer  to  solve  word  problems. 

However,  individual  differences  in  cognitive  load  based  on  students’  cognitive  tempo,  prior  knowledge, 
and  language  background  were  expected.  Therefore,  the  positive  effects  of  multiple-representation  learning  should 
be  stronger  for  high  prior  knowledge  or  high  computer  experience  rather  than  for  low  prior  knowledge  or  low 
computer  experience  students.  Second,  the  positive  effects  of  multiple-representation  learning  should  be  stronger 
for  reflective  rather  than  impulsive  students.  Finally,  as  students  in  Group  SW  were  given  the  option  to  look  up  the 
explanations  in  Spanish,  it  was  predicted  that  limited  English  proficient  (LEP)  learners  would  show  a greater  use  of 
Spanish  explanations  over  sessions  compared  to  first  English  proficient  (FEP)  learners.  The  rationale  is  based  on 
the  idea  that  students  who  are  still  in  the  transition  to  becoming  proficient  English  speakers  will  be  likely  to  choose 
their  first  language  to  lower  the  cognitive  load  required  to  understand  the  explanation  of  the  arithmetic  procedures  in 
English. 

Experiment  1:  A Low  Computer  Experience  Scenario 

The  first  study  was  conducted  at  the  beginning  of  the  school  year  in  a southern  California  elementary 
school  where  many  children  come  from  emigrant,  low-income,  Spanish-speaking  homes.  A significant  proportion 
of  students  is  limited  English  proficient  and  has  almost  no  computer  experience.  Training  with  multimedia  games  in 
this  scenario  is  very  interesting  as  it  can  be  argued  that  the  visual  presentation  of  a mathematical  procedure  such  as 
the  one  used  in  the  present  studies  can  minimize  the  importance  of  language  and  therefore  has  great  potential  for 
enhancing  the  learning  of  non-native  speakers  of  English.  Second,  visual  instruction  can  build  on  the  existing 
intuitive  knowledge  of  the  learner  and  can  be  particularly  appropriate  for  less  skilled  students  who  lack  formal 
academic  training  in  the  subject  domain  (Mayer,  1997).  Third,  visual  forms  of  presentation  can  help  learners  build 
mental  models  of  complex  systems  that  enable  problem  solving  rather  than  being  subjected  to  rote  methods  of 
instruction  that  emphasize  solely  the  acquisition  of  isolated  facts  and  procedures  (Grouws,  1992). 

Method  and  Results 

The  participants  were  61  students  from  three  fifth  and  sixth-grade  classrooms  of  an  elementary  school  in 
southern  California  who  lacked  substantial  prior  knowledge  about  addition  and  subtraction  of  signed  numbers. 
Thirty  students  participated  in  the  SV  group  and  31  students  participated  in  the  SW  condition.  Based  on  the  school 
records  for  their  English  language  proficiency  level,  students  were  classified  as  FEP  or  LEP.  Fourteen  FEP  and  16 
LEP  students  served  in  Group  SV  and  15  FEP  and  16  LEP  students  served  in  Group  SW.  First,  each  student  was 
given  the  computer  experience  questionnaire,  the  paper-and-pencil  pretest,  and  the  Matching  Familiar  Figures  test 
(MFFT)  to  determine  children’s  cognitive  tempo  (Kagan,  Rosman,  Day,  Albert,  & Phillips,  1964).  Second,  they 
were  randomly  assigned  to  learn  with  either  a SV  or  SW  version  of  the  computer  game  and  participated  in  each  of 
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four  training  sessions  held  on  different  days  over  a two-week  period  during  regular  class  time.  Finally,  they  were 
given  the  paper-and-pencil  posttest  and  computerized  word  problem  test. 

For  each  student,  the  number  of  correct  answers  on  the  pretest  was  subtracted  from  the  number  of  correct 
answers  on  the  posttest  to  yield  a pretest-to-posttest  gain  score  and  the  number  of  correct  answers  on  the  word 
problem  test,  the  number  of  correct  answers  on  each  of  the  training  sessions,  and  the  number  of  times  students  asked 
for  a Spanish  translation  was  recorded  from  a computer-generated  log  during  training  sessions. 

Do  students  who  learn  interactively  with  symbolic,  visual,  and  verbal  representations  show  deeper  understanding 
from  a muitimedia  math  lesson  than  students  who  learn  with  symbolic  and  visual  representations  alone? 

Students  in  Group  SW  had  a larger  mean  pretest-to-posttest  gain  score  than  students  in  Group  SV  (p  = 
.02),  a marginally  larger  transfer  to  word-problem  score  (p  = .075),  and  no  significant  difference  in  the  learning  rate. 
These  findings  suggest  that  instructional  programs  that  use  multiple  representations  (such  as  symbolic 
representations  and  visual  interactive  metaphors)  are  more  effective  when  verbal  explanations  are  explicit. 
However,  the  benefit  of  the  verbal  representation  only  affects  students’  pretest-to-posttest  gain  of  the  arithmetic 
procedure  itself  Providing  verbal  representations  does  not  affect  students’  learning  rate  and  only  marginally  helps 
students  transfer  the  procedure  to  solve  word  problems. 

On  the  other  hand,  the  results  on  individual  differences  showed  that  although  prior  knowledge  did  not 
affect  the  pretest-to  posttest  gain,  computer  experience  showed  a significant  interaction  with  the  type  of  treatment: 
Students  who  had  low  computer  experience  did  not  benefit  from  the  verbal  representations  but  students  with  high 
computer  experience  benefited  considerably  (p  = .05).  In  addition,  reflective  students  scored  marginally  higher  on 
pretest-to-posttest  gain  (p  = .06).  Moreover,  high  prior  knowledge  students,  high  computer  experience  students,  and 
reflective  students  learned  faster  and  scored  significantly  higher  than  their  counterparts  in  transferring  the  arithmetic 
procedure  to  solve  word  problems  (p  = .0001,  p = .003,  and  p = .01  respectively  for  learning  rates  and  p = .0001,  p = 
.05,  and  p = .01  respectively  for  transfer  scores).  These  results  show  that  prior  knowledge  (both  in  content  and 
computer  skills)  and  a reflective  cognitive  style  help  students’  learning  by  minimizing  cognitive  load  in  complex 
learning  environments. 

Do  students  who  are  in  the  transition  to  becoming  English  proficient  use  verbal  explanations  in  their  first  language? 

An  additional  issue  was  if  LEP  students  in  Group  S W would  make  significantly  more  use  of  the  Spanish 
translations  than  FEP  students  in  Group  SW.  To  answer  this  question,  for  each  session  and  student,  the  number  of 
times  students  listened  to  Spanish  translations  was  recorded.  Consistent  with  the  prediction,  an  ANOVA  with 
language  background  as  a between-subjects  factor  and  number  of  translations  as  a within-subject  factor  revealed  a 
significant  effect  for  language  background,  number  of  translations,  and  a significant  interaction  (p  = 0.02,  p = 
0.0001,  and  p = 0.001,  respectively).  LEP  students  used  significantly  more  the  Spanish  translations  than  FEP 
students  and  the  number  of  translations  decreased  significantly  over  time  for  LEP  students.  This  result  is  consistent 
with  the  idea  that  students  prefer  to  use  a more  familiar  to  reduce  the  high  cognitive  load  that  results  from  learning 
with  multiple  representation  interactive  environments. 

Experiment  2:  A High  Computer  Experience  Scenario 

Experiment  2 was  conducted  at  the  end  of  the  school  year  in  a fifth  and  sixth  grade  classroom  of  the  same 
elementary  school  that  participated  in  Experiment  1 . Contrary  to  the  case  of  the  first  experiment,  all  participating 
students  had  been  intensively  trained  in  computer  use  for  almost  an  entire  school  year.  Therefore,  the  first  goal  of 
the  second  study  was  to  test  the  prediction  that  due  to  students’  training  with  computers,  computer  experience  would 
lose  its  impact  as  a determinant  factor  in  learning  from  the  interactive  program. 

An  additional  goal  of  Experiment  2 was  to  investigate  if  transfer  to  word  problems  could  be  improved  by 
providing  explicit  word  problem  training.  The  results  of  Experiment  1 showed  an  advantage  of  SW  instruction 
over  SV  instruction  for  only  one  of  three  dependent  measures:  pretest-to-posttest  gain.  Providing  verbal 
explanations  did  not  help  students’  learning  rate  or  transfer  to  solve  word  problems.  In  Experiment  2,  the  learning 
of  a group  of  students  trained  with  symbolic,  verbal,  and  visual  representations  (Group  SW)  was  compared  to  that 
of  an  identical  group  presented  with  an  additional  word  problem  for  each  problem  type  (Group  SWW).  Based  on 
past  research  on  problem-solving  transfer  (Cox^  1997;  Mayer  & Wittrock,  1996),  it  was  predicted  that  explicit 
training  in  solving  word  problems  would  result  in  better  performance  on  this  measure.  However,  as  the  treatments 
did  not  differ  in  any  other  aspect,  differences  in  pretest-to-posttest  gain  or  learning  rate  were  not  expected. 
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Method  and  Results 


The  participants  were  twenty-four  students  from  a fifth  and  sixth-grade  classroom  of  an  elementary  school 
in  southern  California  who  lacked  substantial  prior  knowledge  about  addition  and  subtraction  of  signed  numbers. 
Twelve  students  participated  in  the  SW  group  and  12  students  participated  in  the  SWW  condition.  The  procedure 
was  identical  to  that  used  in  Experiment  1 except  that  after  students  were  given  the  computer  experience 
questionnaire  and  the  paper-and-pencil  pretest,  they  were  not  given  the  MFFT  because  individual  differences  on 
cognitive  tempo  were  not  the  focus  of  the  study. 

Do  students  who  learn  how  to  add  and  subtract  integers  with  word  problem  examples  show  better  transfer  to  solve 
word  problems  than  students  who  are  not  presented  with  word  problems? 

Consistent  with  the  predictions,  students  in  Group  SWW  and  SW  did  not  differ  in  their  mean  pretest-to- 
posttest  gain  or  in  their  learning  rate.  However,  the  main  prediction  was  that  providing  word  problem  examples  to 
Group  SWW  would  make  a significant  difference  in  the  transfer  to  a word  problem  test  respect  to  Group  SW. 
Consistent  with  this  prediction,  using  transfer  to  word  problems  as  a dependent  measure,  a one-factor  ANOVA 
revealed  a treatment  effect  (p  = .05).  Students  in  Group  SWW  were  significantly  better  able  to  transfer  the 
arithmetic  procedure  to  solve  word  problems  as  compared  to  students  in  Group  SW. 

What  is  the  role  of  computer  experience  for  students  who  learn  with  multiple  representations  in  a computer-trained 
classroom? 

Due  to  students’  overall  high  experience  with  computers,  it  was  expected  that  computer  knowledge  would 
not  affect  overall  performance  because  they  had  automated  the  computer  skills  necessary  to  interact  with  the 
program.  Consistent  with  the  predictions,  the  statistical  analyses  revealed  no  computer  experience  effect  on  pretest- 
to-posttest  gains,  learning  rate,  or  transfer  to  word  problems.  High  experience  students  scored  comparably  to  low 
experience  students  on  all  dependent  measures.  As  argued  in  the  predictions,  the  different  pattern  of  results  between 
both  studies  can  be  interpreted  as  due  to  the  computer  training  program  that  the  teacher  had  implemented  in  the 
participating  class.  Experiment  1 was  conducted  at  the  beginning  of  the  school  year  and  many  students  reported 
having  very  low  or  even  no  basic  computer  experience.  On  the  other  hand.  Experiment  2 was  conducted  in  a 
classroom  where  the  teacher  placed  priority  on  students’  practice  of  technologically  based  instructional  materials. 
For  example,  one  of  the  goals  of  the  participating  class  consisted  of  having  each  student  develop  their  own  web  page 
by  the  end  of  the  year. 


General  Discussion 

Multimedia  programs  allow  students  to  work  easily  with  multiple  representations  of  complex  systems.  The 
reported  studies  demonstrate  that  presenting  a symbolic  and  visual  representation  of  how  an  arithmetic  procedure 
works  does  not  insure  that  all  students  will  understand  the  explanation  unless  cognitive  theory  is  applied  to  the 
design. 

The  reported  studies  have  important  implications.  Theoretically,  this  research  supports  a cognitive  theory 
of  multimedia  learning  in  two  ways.  First,  by  demonstrating  that  teaching  with  symbolic,  visual,  and  verbal 
representations  facilitates  and  strengthens  the  learning  process  by  providing  several  mutually  referring  sources  of 
information.  Second,  by  suggesting  that  the  additional  cognitive  load  imposed  on  the  SV  group  (i.e.,  the  need  for 
students  to  use  their  limited  cognitive  resources  to  discover  the  meaning  of  a procedure  and  generate  their  own 
verbal  representation)  is  detrimental  to  learning  by  exhausting  working  memory  capacity.  Third,  the  results  from 
Experiments  1 and  2 show  that  even  when  instructional  methods  are  grounded  on  visual  metaphors,  transfer  skills 
require  explicit  training.  Although  interactive  visual  metaphors  may  help  students  build  connections  between 
arbitrary  sets  of  procedures  that  use  symbols  and  their  informal  conceptual  knowledge,  this  method  only  seems  to 
help  students  understand  the  arithmetic  procedure  itself.  Finally,  congruent  with  cognitive  load  theory,  when 
learners  are  required  to  mentally  integrate  disparate  sources  of  mutually  referring  information,  such  split-source 
information  may  create  a heavy  cognitive  load  that  disrupts  learning,  especially  for  learners  who  have  fewer 
cognitive  resources  (Moreno  & Mayer,  1999;  Sweller,  1991).  The  present  results  on  individual  differences  are 
consistent  with  the  interpretation  that  students  learn  deeper  in  multimedia  interactive  environments  when  the 


1422 


advantages  of  multiple  representations  are  not  outweighed  by  individual  differences  in  cognitive  load  (Chandler  & 
Sweller,  1991;  Sweller,  1994). 

On  the  practical  side,  the  overall  results  indicate  that  visual  metaphors  need  verbal  guidance  and  word 
problem  examples  if  they  are  to  be  used  as  an  instructional  tool  to  foster  mathematical  understanding  and  transfer. 
Multimedia  environments  have  the  capability  of  creating  dynamic  visual  representations  of  constructs  that  are 
frequently  missing  in  the  mental  models  of  novices  (Kozma,  1991).  However,  the  present  findings  show  that  for 
multimedia  learning  to  be  effective  it  is  important  to  design  the  materials  in  a manner  that  minimizes  cognitive  load. 
One  way  to  reduce  cognitive  load  is  by  providing  guidance  in  the  form  of  verbal  explanations  in  students’  first 
language.  A contribution  of  the  reported  studies  is  to  identify  computer  experience,  prior  knowledge,  and  cognitive 
tempo  as  important  factors  that  help  students  learn  from  multimedia  by  minimizing  cognitive  load. 
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Abstract:  There  exist  two  commonly  held  views  regarding  the  use  of  multimedia  and  Internet 
technology  within  learning  environments.  Many  educators  believe  that  the  presence  of  content  material 
on  CD-ROM  or  the  Internet  will  help  students  reach  their  learning  goals,  and  students  Mike’  multimedia 
and  /or  Internet  based  delivery.  This  paper  explores  these  views  through  analysis  of  the  design  and 
development  of  an  eLearning  solution. 


Introduction 

“Television  can  teach.  It  can  illuminate.  Yes,  it  can  even  inspire.  But  it  can  do  so  only  to  the  extent  that  humans 
are  determined  to  use  it  to  those  ends.  Otherwise,  it  is  merely  lights  and  wires  in  a box”  Edward R.  Murrow 

While  this  statement  is  obviously  discussing  the  use  of  television  as  a learning  resource,  the  same 
sentiment  can  be  directed  toward  eLearning  solutions.  However,  the  thought  of  an  expensive  eLearning  solution 
as  merely  ‘lights  and  wires  in  a box’  is  probably  enough  to  strike  fear  into  the  hearts  of  most  eLearning 
providers.  Given  the  general  view  that  the  use  of  educational  multimedia  applications  and  online  learning 
solutions  enhance  learning  effectiveness,  such  a stance  is  akin  to  heresy.  Instructional  designers  developing 
eLearning  solutions  claim  ‘improved  learner  outcomes*  (Colorito,  2001)  and  ‘more  effective  learning 
experiences’  (Maki  et  al,  2000).  These  are  nice  ideas.  Happy  learners  and  even  happier  teachers  skipping  down 
the  yellow  brick  path  toward  successful  learning  outcomes.  However,  not  a lot  unlike  the  young  woman  in  the 
sparkling  red  shoes,  it  does  not  take  long  to  discover  that  fantasy  and  reality  are  somewhat  distanced  from  one 
another. 


There  exist  two  commonly  held  views  regarding  the  use  of  multimedia  and  Internet  technology  within 
learning  environments.  Many  educators  believe  that  the  presence  of  content  material  on  CD-ROM  or  the 
Internet  will  help  students  reach  their  learning  goals,  and  students  ‘like’  multimedia  and  /or  Internet  based 
delivery.  Many  factors  play  a part  in  shaping  an  individual’s  learning  experience,  not  least  of  all  the 
assumptions  of  the  instructional  designer.  For  example,  using  Brookfield’s  paradigmatic  assumption,  learners 
within  the  university  context  will  be  predominantly  proactive,  highly  motivated  consumers  of  knowledge. 
Associated  with  this  assumption  is  the  prescriptive  assumption  that  as  the  students  are  proactive,  self-directed 
learners  then  the  best  teaching  will  be  that  which  encourages  students  to  take  control  over  designing,  conducting 
and  evaluating  their  own  learning  (Brookfield,  1995).  Therefore,  if  the  paradigmatic  and  prescriptive 
assumptions  are  correct,  it  follows  that  if  an  environment  is  created  to  support  self  directed  learning  such 
learning  will  (naturally)  take  place  (causal  assumption).  This  is  largely  the  reasoning  applied  to  the  integration 
of  multimedia  and  Internet  technology  within  learning  environments.  Students  ‘like’  multimedia  and/or  Internet 
based  content  therefore  if  learning  facilitators  provide  such  environments  students  will  (naturally)  have 
successful  learning  outcomes. 

While  many  learners  may  prefer,  or  perhaps  expect,  to  access  content  material  via  the  Internet  recent 
research  does  not  support  the  view  that  students  prefer  this  mode  of  delivery  (Mclnnis,  James  & Hanley,  2000). 
New  technology  is  seductive,  this  danger  is  especially  apparent  when  multimedia  and  Internet  technology  are 
viewed  as  a providing  improved  learning  experiences.  However,  it  is  important  to  not  allow  technology  to 
overwhelm  the  foundations  of  an  effective  learning  experience.  Whether  delivered  via  CD-ROM,  the  Internet, 
cassette  or  print,  we  should  seek  to  provide  a learning  experience  that  is  meaningful  to  all  users.  Jonassen 
(2001)  discusses  eight  qualities  inherent  in  meaningful  learning  environments.  These  are: 

- Active:  Learners  are  engaged  by  the  learning  process  in  mindful  processing  of  information  where  they  are 
responsible  for  the  result. 
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- Constructive:  Learners  integrate  new  ideas  with  prior  knowledge  in  order  to  make  sense  or  make  meaning 
or  reconcile  a discrepancy,  curiosity,  or  puzzlement. 

- Collaborative:  Learners  naturally  work  in  learning  and  knowledge  building  communities,  exploiting  each 
others  skills  while  providing  social  support  and  modeling  and  observing  the  contributions  of  each  member. 

~ Intentional:  All  human  behavior  is  goal  directed. 

- Complex:  Unless  learners  are  required  to  engage  in  higher  order  thinking,  they  will  develop  oversimplified 
views  of  the  world. 

- Contextual:  A great  deal  of  recent  research  has  shown  that  learning  tasks  that  are  situated  in  some 
meaningful  real  world  task  or  simulated  in  some  case-based  or  problem  based  learning  environment  are  not 
only  better  understood,  but  also  are  more  consistently  transferred  to  new  situations. 

- Conversational:  Learning  is  inherently  a social,  dialogical  process.  ...  Technologies  can  support  this 
conversational  process  by  connecting  learners  across  town  or  across  the  world. 

- Reflective:  Learners  should  be  required  by  technology-based  learning  to  articulation  what  they  are  doing, 
the  decisions  they  make,  the  strategies  the  use,  and  the  answers  that  they  found. 

(Jonassen,  2001) 

In  view  of  these  effective  practice  guidelines  and  qualities  as  articulated  Jonassen  (2001)  the  challenge 
for  educators  and  eLearning  providers  is  to  build  in  to  the  distributed  educational  context  dynamic, 
collaborative,  and  conversational  environments.  However,  integrating  collaboration  and  conversation  within  a 
distributed  learning  environment  is  easier  said  than  done.  The  following  case  study  explores  the  way  in  which 
these  guidelines  informed  the  development  of  an  eLearning  environment  for  a core  subject  for  second  year 
university  computing  students. 


Case  Study  - ^Issues  in  the  Design  and  Delivery  of  Multimedia’  eLearning  resource 

History:  The  subject  content  and  Internet  site  were  partially  developed  in  1999  and  both  were  extensively 
revised  in  2000.  The  structure,  sequence  and  content  of  the  subject  and  the  Internet  site  have  been  used  as 
teaching  and  learning  resources  for  two  years  (2000  and  2001). 

Subject:  ‘Issues  in  the  Design  and  Delivery  of  Multimedia*  focuses  on  examining  and  analysing  ethical,  legal, 
social  and  professional  issues  in  the  development  and  delivery  of  multimedia.  In  this  subject  issues  such  as 
software  piracy,  computer  crime,  online  pornography,  professional  ethics  and  the  impact  and  implications  of  the 
use  of  technology  are  critically  analysed. 

Learners  and  Mode:  Typically  the  student  cohort  comprises  (generally)  motivated  adult  learners  from  a variety 
of  experiential  and  cultural  backgrounds.  Approximately  15%  of  learners  are  international  students  and  between 
20%  and  40%  are  aged  over  25  years.  The  subject  is  delivered  in  on  campus  mode.  An  extensive  Internet  site 
was  developed  to  support  lectures  and  workshops. 


Methodology:  The  creation  of  educational  environments,  both  in  class  and  online,  that  fostered  collaboration, 
conversation  and  activity  (Jonassen,  2001)  underpinned  the  philosophical  orientation  of  the  instructional  design 
process.  In  many  respects,  the  process  of  designing  and  developing  online  instructional  materials  closely 
resembles  a software  development  life  cycle  model  in  that  the  instructional  design  process  is  an  iterative  one. 
Progression  relies  on  analysis  and  evaluation  of  the  objectives  and  outcomes  of  the  preceding  phase.  The 
instructional  design  process  used  in  the  development  of  the  eLearning  environment  relied  on  four  primary 
phases  that  guided  and  shaped  the  development  process.  The  phases  are  illustrated  in  Figure.  1. 
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Figure  1.  Four  stage  instructional  design  process. 


Phase  1:  Front  end  analysis.  What  do  we  currently  have?  (What  are  the  global  aims  and  specific  objectives? 
What  resources  can  be  utilised  to  achieve  those  goals?)  In  this  phase,  existing  subject  content  was  reviewed  to 
update  specific  content  areas.  This  phase  examined  the  current  teaching  and  learning  process  and  specifically 
analysed  the  assessment  instruments  in  relation  to  the  stated  subject  aims  and  objectives.  Feedback  regarding 
content  and  teaching  strategy  was  sought  from  learners  and  colleagues.  The  major  focus  of  the  review 
concentrated  on  addressing  three  primary  areas  identified  in  the  initial  problem  analysis: 

- Explicitly  stating  global  aims  and  specific  objectives  and  strengthening  the  relationships  between  aims, 
objectives  and  content. 

- Identifying  learner  entry  and  exit  points  (including  assessment)  and  the  relationships  between  them. 

- Addressing  inconsistencies  in  delivery  media  and  identifying  opportunities  for  implementation  of 
innovative  delivery  strategies. 

Global  aims:  The  solution  sought  to  incorporate  the  following  global  educational  aims  in  order  to  strengthen  the 
relationship  between  aim,  objective  and  content.  At  the  completion  of  the  subject  learners  should: 

- have  developed  an  awareness  and  understanding  of  the  concepts,  practices  and  ethical  codes  and 
conventions  regarding  the  design  and  delivery  of  multimedia  applications  and  technology  generally 

- have  developed  an  understanding  of  the  issues  and  implications  regarding  management  of  knowledge 

- exhibit  high  level  critical  thinking  and  problem  solving  skills 

- have  developed  high  level  verbal  and  written  communication  skills  and  be  encouraged  to  think  critically 
about  the  uses  (and  abuses)  of  multimedia  and  information  technology  generally 

- exhibit  responsible  and  discerning  use  of  technology  in  general 

Specific  objectives:  Further  to  strengthening  the  relationships  betw'een  aim,  objective  and  content  in  relation  to 
global  educational  aims,  the  following  areas  were  identified  as  desired  specific  objectives  of  the  subject. 
Accordingly,  at  the  conclusion  of  the  course,  learners  should: 

- have  detailed  understanding  of  the  composition  and  application  of  their  personal  ethical  framework 

- have  detailed  knowledge  of  the  concepts  and  principles  regarding  the  ethical,  legal,  social  and  professional 
issues  relevant  to  the  development  and  delivery  of  multimedia  and  an  ability  to  apply  these  concepts  and 
principles  in  practice. 

- be  competent  in  making  informed  judgments  about  the  impact  of  multimedia  and  information  technology 
on  the  individual,  organisation  and  society 

- understand  the  ‘current  state  of  play’  of  the  information  economy  in  Australia  and  understand  the  strategic 
direction  of  state  and  federal  governments  with  regard  to  the  information  economy  and  understand  the  role 
of  Australia  in  the  global  information  economy 

- understand  and  engage  in  the  discourse  surrounding:  Ethical  codes  for  computing  and  information 
technology  professionals;  International  domain  name  control  and  registration;  Domain  name  registrar 
accreditation;  Disability  discrimination  in  online  environments;  Digital  image  manipulation;  Content 
control;  Online  pomography/erotica;  Privacy,  security  and  encryption;  Hacking  and  cybercrime/terrorism; 
Copyright  and  intellectual  property  law 

- effectively  incorporate  ethical  practice  in  the  design,  development  and  evaluation  of  multimedia  solutions 

- apply  knowledge  of  the  ethical,  social  and  legal  impact  of  the  use  of  multimedia  applications  and 
technology  to  a business  situation  in  the  form  of  specific  analysis  and  recommendation. 

Resources:  The  resources  developed  exist  primarily  in  an  online  format.  Resources  for  in  class  workshops  were 
developed  in  hard  copy.  Workshops  in  weeks  2,3  and  4 concentrated  on  exploration  of  personal  ethical 
frameworks  and  utilised  scenario  analysis  as  a delivery  strategy.  Student  seminar  and  discussion  sessions  were 
scheduled  to  take  place  in  weeks  5-14. 

Phase  2:  Knowledge/content  analysis.  What  do  we  want  to  achieve?  (What  is  the  terminal  objective?  What 
type/s  of  knowledge  does  the  learner  need  to  develop?)  In  this  phase,  subject  content  was  analysed  and 
evaluated  in  relation  to  the  stated  aims  and  objectives  of  the  subject,  and  to  ensure  that  a variety  of  learning 
approaches  were  appreciated.  The  teaching  strategy  adopted  included  lectures,  workshops  and  online  fora, 
integrating  scenario  analysis,  and  facilitating  formal  and  informal  discussion.  As  part  of  this  process  major 
learning  goals  and  objectives  were  defined  -“(i)...the  terminal  objective  of  instruction”  (Taylor,  1994)  and 
subject  content  was  assembled  from  multiple  sources  and  evaluated  -“(ii)... analyse  the  underlying  declarative 


knowledge  base  of  an  expert  (or  experts)”  (Taylor,  1994),  and  arranged  into  relatively  broad  topic  areas.  The 
broad  topic  areas  comprise  four  main  modules,  providing  multiple  entry  points  for  learners. 


Entry  points:  Four  primary  organising  factors  were  identified  and  implemented  to  provide  overall  structure  to 
the  subject  content.  These  were:  Module  1:  Ethical  considerations;  Module  2:  Social  factors;  Module  3:  Legal 
framework;  Module  4:  Professional  issues.  Students  are  required  to  undertake  all  four  modules  of  study.  Initial 
modules  are  presented  linearly  to  enable  learners  to  progressively  develop  understanding  of  primary  concepts 
and  pnnciples  (Figure  2 — Detail  ‘Gov.au’). 


Exit  points:  Four  assessment  items  were  specified.  They  are:  Participation  in  online  fora;  20  minute  seminar 
with  oral  defence;  Academic  paper  - Article  reflection;  Consultancy  report  for  ‘real  world’  client. 

As  the  subject  is  taught  as  part  of  an  undergraduate  program  exit  points  are  prescribed  in  that  learners  must 
undertake  all  assessment  items.  However,  learners  have  control  over  a number  of  aspects  of  assessment.  For 
example,  at  the  commencement  of  the  semester  students  submit  suggested  fora  topics.  Students  then  ‘vote’  for 
four  preferred  fora  topics.  When  selecting  seminar  topics  learners  can  select  from  a provided  list  or  negotiate  a 
topic  of  choice.  Articles  for  reflection  are  supplied,  however  learners  are  also  able  to  analyse  (approved)  articles 
of  choice.  Further,  students  are  encouraged  to  self  select  a client  partner  for  the  consultancy  report. 

Knowledge:  In  order  to  achieve  desired  learning  outcomes  learners  needed  to  develop  (at  least)  two  types  of 
knowledge  - explicit  and  implicit  (Sternberg,  1998,  p 13).  For  example,  as  primarily  novice  learners,  the 
students  require  explicit  knowledge  “ ...  knowledge  of  the  facts,  formulas,  principles,  and  major  ideas  of  a 
domain  of  inquiry.”  (Sternberg,  1998,  p 13),  in  this  case  knowledge  of  ethical,  social,  legal  and  professional 
issues  required  to  function  as  multimedia  designers,  they  also  required  high  levels  of  implicit  knowledge  -tacit 
knowledge  of  the  domain  (multimedia)  and  the  social  organisation  (their  classroom  and  the  University) 
(Sternberg,  1998,  p 13).  According  to  Gagne  et  al,  the  locus  of  control  in  learning  rests  with  the  learner,  in  that 
the  learner  brings  to  the  process  of  learning  inherent  cognitive  strategies  that  facilitate  understanding  (Gagne, 
1987,  p66).  This  view  is  shared  by  Simonson  and  Thompson  when  they  discuss  cognitive  theory  and  computer 
based  instruction  “CBI  needs  to  be  organised  and  delivered  in  a way  that  compliments  the  cognitive  structure 
and  level  of  sophistication  of  the  learner  “(1994,  p36).  Further,  Gagne  states  that  learner’s  process  information 
according  to  the  cognitive  strategies  used  by  the  learner,  and  when  new  strategies  are  acquired  metacognition 
takes  place.  That  is  the  process  of  “learning  to  learn”  is  analysed  and  edited  by  the  learner,  increasing  the  level 
of  understanding  (Gagne,  1987,  p66).  Similarly,  Bruner  discusses  the  concepts  central  to  cognitive  theory  as 
being:  how  knowledge  is  organised  and  structured;  readiness  for  learning;  intuition;  motivation  (in  Simonson  & 
Thompson,  1994,  p37).  Hypermedia  is  used  as  an  example  of  a “powerful  tool”  in  computer  based  instruction 
“that  is  non-linear  and  non-sequential  ...  [and]  used  by  cognitive  scientists  to  examine  how  students  interact 
with  instruction  during  the  process  of  learning”  (Simonson  and  Thompson,  1994,  p37). 

Phase  3:  Sequencing/synthesising  content  How  can  we  get  there?  (What  are  the  major  instructional 
strategies?  What  are  the  learning  activities  that  will  be  used?  ) In  early  iterations  of  the  subject,  any 
opportunity  for  peer  to  peer  discussion  in  any  formal  or  semi  formal  educational  environment  was  non-existent. 
The  opportunity  to  provide  students  with  content  in  a way  that  integrated  their  areas  of  professional  interest 


Figure  2.  Screenshot  detail  ‘Gov.au’  module. 
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(computer  mediated  communication,  multimedia  and  the  Internet),  and  directly  related  to  content  issues  they 
were  discussing  (legal,  social,  professional  and  ethical  issues  in  a networked  multimedia  environment)  was 
identified.  Students  were  encouraged  to  learn  through  understanding  by  articulating  and  defending  their  views 
through  delivery  of  a student  seminar  with  oral  defence  and  to  become  proactive  learners  by  setting  their  own 
research  direction  within  the  criteria  for  the  major  assessment  item.  In  addition,  tailoring  delivery  of  resources 
in  this  way  also  provided  opportunities  to  help  learners  identify  and  expand  bridging  points  between  subjects 
being  studied  simultaneously.  For  example,  students  are  required  to  complete  the  subject  ‘Computer  Mediated 
Communication’  in  the  same  semester  as  ‘Issues  in  the  Design  and  Delivery  of  Multimedia’.  ‘ Computer 
Mediated  Communication’  requires  students  to  examine  and  apply  concepts  and  principles  related  to 
communication  in  ‘virtual’  environments  while  ‘Issues  in  the  Design  and  Delivery  of  Multimedia’  provides 
such  an  environment  through  the  online  fora. 

Instructional  strategies:  Instructional  strategies  were  identified  “strategy  and  role-playing  games,  intentional 
learning  environment,  case  studies,  coaching  and  scaffolding,  learning  by  design,  group,  cooperative, 
collaborative  learning”  (Wilson,  1997).  In  practice  teaching  strategies  focusing  on  the  needs,  preferences  and 
expectations  of  the  learner  were  implemented.  These  instructional  strategies  were  modelled  on  the  effective 
practice  guidelines  and  qualities  as  articulated  by  Collins  (1998)  and  Jonassen  (2001)  and  comprised: 

- striving  to  create  a classroom  atmosphere  (both  on  campus  and  online)  that  is  conducive  to  collaborative 
learning  and  is  supportive  and  inclusive  of  all  learners 

- encouraging  learners  to  form  learning  alliances  with  their  peers  in  order  to  share  information  and 
knowledge  within  a peer  supported  learning  framework 

- promoting  strategies  that  foster  a sophisticated  level  of  understanding  of  subject  content  and  related 
concepts  by  encouraging  learners  to  relate  content  to  personal  schema. 

Early  modules  utilised  short  video  segments  to  coach  and  support  novice  learners,  and  to  ‘personalise’  the 
eLearning  environment  (Figure  3.).  Students  were  given  simple  instructions  and  study  direction  for  the  module. 


Figure  3.  Screenshot  showing  video  segment  and  transcript. 

The  rationale  for  using  the  online  fora  was  based  on  two  concepts.  Firstly,  high  level  understanding  of  the 
subject  content  and  specifically  complex  ethical  issues,  comes  not  through  transmissive  teaching  but  from 
engagement  with  the  discourse  surrounding  specific  issues.  Secondly,  based  on  anecdotal  evidence  suggesting  a 
high  rate  of  usage  of  computer  mediated  communication  technology  among  multimedia  students  it  seemed  that 
multimedia  students  would  adapt  quickly  to  the  medium  and  engage  readily  with  the  concept  of  an  online 
discussion  environment.  This  view  has  been  supported  by  the  subsequent  rapid  development  of  the  fora  into  a 
dynamic  ‘virtual’  discussion  environment  where  students  feel  comfortable  in  forming  and  expressing  their 
opinions  on  issues  related  to  the  subject  content.  Evaluation  feedback  showed  that  69%  of  respondents  stated 
that  they  would  have  contributed  to  the  fora  even  if  it  were  not  assessed. 

A further  area  of  innovation  in  this  subject  involved  the  use  of  a 20  minute  seminar  with  oral  defence  to  assess 
student  understanding  of  specific  concepts.  As  multimedia  students  are,  in  general,  high  achieving  students  the 
seminar  was  implemented  as  an  assessment  item  to  provide  a high  level  of  cognitive  challenge  and  stimulation 
for  second  year  students.  This  aspect  reflected  one  of  Jonassen ’s  (eight)  qualities  of  meaningful  learning 
environments  - namely  the  provision  of  a complexity  (2001).  Additional  challenge  was  provided  in  the  form  of 
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weekly  reflection  questions.  The  questions  were  designed  to  provide  extension  to  in  class  discussion  and 
activity. 

Phase  4:  Evaluation,  How  do  we  know  we  have  achieved  our  aims?  (What  are  the  results  of  formative  and 
summative  evaluation?)  Students  were  surveyed  (informally)  at  the  start  of  the  semester  “[to]...  measure  the 
extant  knowledge  bases  of  the  relative  novices...”  (Taylor,  1994).  The  purpose  of  the  survey  was  twofold. 
Firstly,  it  was  important  to  discover  existing  levels  of  knowledge  and  expectation,  and  for  students  to  begin  to 
examine  the  validity  of  their  assumptions  and  expectations.  Students  were  asked  to  examine:  what  they  knew 
now,  what  they  hoped  to  know  and  what  they  thought  the  subject  would  help  them  learn. 

Based  on  the  outcome  of  the  survey,  the  subject  design  focussed  on  taking  the  learners  from  where  they  were 
situated  at  the  start  of  the  subject  (self  identified  novices)  to  where  they  expected  to  be  (more  or  less)  at  the 
conclusion  of  the  subject  (multimedia  producers  cognisant  of  ethical  and  professional  issues  related  to  the  field). 
This  progression  reflected  Taylors  “.. . gradual  elaboration  of  a series  of  organisers...”  (Taylor,  1994). 

Conclusion 

The  use  of  a supported  eLearning  environment  for  the  subject  Tssues  in  the  Design  and  Delivery  of 
Multimedia’  provided  students  with  a valuable  learning  resource.  While  students  enjoyed  ease  of  access, 
intellectual  challenge  and  willingly  contributed  to  the  online  fora,  the  majority  of  high  level  discussion  and 
complex  reasoning  still  took  place  within  the  on  campus  classroom. 

Within  distributed  learning  environments,  multimedia  and  Internet  technology  provide  a vehicle  for  the 
transmission  of  content  material,  in  the  same  way  print  materials,  and  audio/video  cassettes  have  traditionally 
done.  Technical  capabilities  aside,  it  matters  little  whether  content  is  delivered  via  CD-ROM,  the  Internet,  audio 
or  video  cassette,  or  even  print  based  materials,  as  these  technologies  merely  provide  the  substrate  to  the 
learning  resource.  Unless  the  subject  structure  and  content  is  based  on  soxmd  pedagogical  theory,  and  sits  within 
a coherent,  supported  learning  framework,  it  will  make  no  difference  whether  cutting  edge  multimedia  and 
Internet  technology  have  been  used.  The  content  will  make  little  sense  to  the  learner. 
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Abstract 

This  study  is  an  ongoing  examination  of  the  effectiveness  of  an  online  professional  development  course 
designed  to  examine  the  connection  between  learning  theory  and  technology  used  to  support  learning  in  the 
classroom.  The  course  was  taught  by  three  methodologies:  a traditional  face-to-face  class  that  used  the 
online  materials  as  a resource;  an  online  class  that  received  all  instruction  via  the  Internet;  and  a hybrid 
class  that  used  the  online  resources  in  addition  to  meeting  weekly  for  teleconferencing  with  the  instructor. 
At  the  completion  of  the  course,  data  was  collected  from  the  participants  relating  to  course  design  and 
implementation  resulting  in  positive  feedback  from  most  students.  To  ascertain  changes  in  attitude  and 
classroom  practice,  further  data  was  requested  from  the  students  six  months  after  completion  of  the  course. 
This  study  focuses  on  the  second  round  of  data  collection  and  examines  the  changes  in  attitude  and  practice 
reported  by  the  teachers. 


Introduction 

Information  Technology  in  the  Teaching  and  Learning  Process  is  a core  course  in  a seven  course  Diploma 
of  Information  and  Communication  Technology  at  the  Christchurch  College  of  Education  in  Christchurch, 
New  Zealand.  The  diploma  attracts  practicing  teachers  who  want  to  upgrade  qualifications  and  gain  skills 
in  using  information  and  communication  technologies  (ICT).  The  course  is  designed  to  examine  the 
connection  between  learning  theory  and  technology  used  to  enhance  learning  in  the  classroom,  and  was 
formerly  taught  as  a face-to-face  course  with  provision  for  the  occasional  distance  student  made  available 
through  teleconferencing.  At  the  beginning  of  2001,  an  increased  demand  from  students  around  the 
country  wishing  to  enrol  in  the  course  as  distance  learners  led  to  the  decision  to  develop  the  course  for 
online  delivery.  Due  to  the  configuration  of  students  requesting  enrolment  in  the  course,  three  different 
methodologies  were  identified:  a traditional  face-to-face  group  that  would  meet  bi-weekly  with  the 
instructor  on  campus  and  use  the  online  materials  as  a resource;  an  online  group  that  would  receive  all 
instruction  via  the  Internet;  and  a hybrid  group  involving  a group  of  teachers  in  a remote  location  who 
would  use  the  online  resources  in  addition  to  an  initial  two -day,  face-to-face  start  to  the  class  with  the 
instructor.  A weekly  phone  conference  between  the  instructor  and  the  assembled  group  was  also  a 
dimension  of  this  hybrid  class.  The  existence  of  the  three  methodologies  presented  a unique  opportunity  to 
explore  how  the  use  of  telecommunications  impacted  each  of  the  groups  as  they  interacted  with  the  course 
materials,  the  instructor,  other  class  members  and  with  the  technological  interface  used  to  deliver  the 
course.  Also  of  interest  was  the  impact  that  this  class  had  on  the  teachers’  attitude  and  practice  in  their 
individual  classrooms. 

Literature  Review 

Online  professional  development  is  a new  methodology  that  is  increasing  rapidly.  New  demands  are  being 
placed  on  teachers  and  school  as  an  outgrowth  of  the  demands  of  the  knowledge  age  society,  new 
understandings  of  the  teaching  and  learning  process,  and  an  upsurge  in  the  design  and  availability  of  new 
technologies.  These  new  demands  are  forcing  changes  in  professional  development  (Moon,  2000; 

Trewem,  1999)  and  have  contributed  to  the  growth  of  professional  development  offered  through 
telecommunications  (Bates,  1995;  Blanton,  Moorman,  & Trathen,  1998).  One  such  change  is  the 
availability  of  professional  development  presented  to  practicing  educators  online,  allowing  them  to  take 
advantage  of  new  interactive  technologies  that  offer  study  at  a time  and  place  convenient  to  them  (Leonard, 
1999;  Mclsaac  & Gunawardena,  1996). 

Berge  (1998)  describes  two  major  frameworks  from  which  to  view  instruction.  The  first  is  a transmission 
framework,  examples  being  lectures,  textbooks  and  videotape  and  is  based  on  theories  of  positivism  and 
behaviourism.  The  second,  a transformation  framework,  in  which  the  learner  transforms  information  by 
generating  hypotheses,  making  decisions  and  constructing  knowledge  either  individually  or  through  a 
social  interaction  with  others,  is  based  on  the  theory  of  constructivism.  Contrasting  these  methods  as 
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teacher  centred  versus  learner  centred,  Berge  states  that  instructors  have  implicit  or  explicit  personal 
theories  about  good  instruction  and  will  choose  teaching  methodologies  and  technologies  according  to 
theses  preferences.  Bates  (1995)  cautions  that  the  transmission  model  no  longer  meets  the  changing  needs 
of  the  knowledge  economy  workers,  where  communicating  effectively,  working  in  teams,  analysing 
information  and  generating  new  knowledge  are  creating  complex  new  educational  needs. 

The  design  of  online  instruction  needs  to  shift  from  prescriptive,  directed  learning  situations  to 
environments  that  allow  learners  to  solve  real-world  problems  and  engage  in  dialogues  with  a community 
of  practitioners  (Jonassen,  Davidson,  Collins,  Campbell,  & Haag,  1995).  Describing  constructivist 
environments  as  those  that  “engage  learners  in  knowledge  construction  through  collaborative  activities  that 
embed  learning  in  a meaningful  context  and  through  reflection  on  what  has  been  learned  through 
conversation  with  others  learners,*’  (p.  13)  Jonassen  et.al  suggests  a framework  of  context,  construction, 
collaboration  and  conversation  to  facilitate  the  making  of  meaning  for  learners.  Within  the  online  learning 
environment,  this  framework  can  be  mediated  by  a variety  of  technologies.  Computer-mediated 
communication  is  available  through  electronic  mail,  computer  based  discussion  and  conferencing. 
Collaboration  is  available  online  through  the  ability  of  the  technology  to  support  groups  across  a distributed 
environment  where  learners  can  actively  work  toward  a negotiated  meaning.  Using  situated,  case-based 
learning  to  give  authentic  contexts  in  which  students  can  work,  is  supported  by  both  video  and  hypermedia 
environments  and  gives  learners  an  opportunity  to  reflect,  communicate  and  negotiate  a shared  meaning 
within  the  group  (Jonassen  et  al.,  1995).  Wiggins  (1993)  defines  the  characteristics  of  good  learning 
environments,  both  distance  and  local,  as  being  centred  on  worthy  problems  or  important  questions, 
including  tasks  that  are  replicas  of  real-world  problems  faced  by  professionals  in  the  field,  providing  access 
to  resources  commonly  available  to  those  professionals,  and  presenting  problems  that  require  a repertoire  of 
knowledge,  judgement  and  skills.  In  defining  the  constructivist  online  environment  Jonassen  et.al  (1995) 
states:  “Constructivism  can  provide  theoretical  bases  for  unique  and  exciting  distance  learning 
environments.  These  environments  should  emerge  from  authentic  tasks,  engage  the  learners  in  meaningful, 
problem-based  thinking  and  require  negotiation  of  meaning  and  reflection  on  what  has  been  learned”  (p. 

21). 


Design  and  Implementation  of  the  Course 

The  outcomes  of  this  course  required  that  students  examine  a variety  of  learning  theories  and  teaching 
strategies,  look  at  current  research  in  the  field  and  apply  practical  skills.  To  accomplish  these  goals  the 
instructor  used  a combination  of  directed,  transmission  oriented  learning  such  as  lectures  or  explanatory 
content,  practical  activities  and  readings;  and  more  transformational,  constructivist  approaches  such  as 
reflection,  evaluation  and  discussion.  To  give  the  students  a chance  to  work  through  these  concepts  and 
make  personal  meaning  from  the  idea  presented,  students  were  asked  to  reflect  on  what  they  had  read, 
apply  the  learning  to  their  own  situation  and/or  evaluate  materials  within  the  context  of  their  own  teaching 
situation.  Questions  were  posed  through  the  discussion  forum  where  students  could  offer  opinions, 
solutions  and  share  experiences  based  on  their  personal  reflections  and  interactions  with  the  course 
materials. 


Methodology 

This  study  was  begun  in  February  2001  during  the  first  semester  of  the  academic  year  in  the  southern 
hemisphere.  Twenty-nine  students  enrolled  in  the  course  with  14  in  the  hybrid  group,  8 in  the  online  group 
and  7 in  the  on-campus  group.  To  evaluate  the  effectiveness  of  the  course  design  and  understand  the 
students’  perceptions  and  satisfaction  with  the  course,  the  following  data  from  the  hybrid,  online  and 
campus  groups  was  available:  initial  survey  of  online  students,  statistics  from  the  courseware  used  to 
present  the  course  materials,  e-mail  between  lecturer  and  students,  discussion  board  archives  and  end  of 
course  survey. 


Since  the  data  collected  during  and  immediately  after  the  course  reflect  only  participants  perception  of  the 
design  and  implementation  of  the  course,  further  data  was  needed  to  examine  any  change  of  attitude  or 
practice  in  the  teachers’  classroom  practice.  Six  months  after  the  end  of  the  course,  students  were  asked  to 
reflect  and  comment  on  changes  in  attitude  or  classroom  practice  that  they  could  attribute  to  their 
participation  in  the  course. 


1431 


Results 


Results  of  the  end  of  course  survey  showed  that  students  reacted  positively  to  the  structure  and  design  of 
assignments,  with  only  one  campus  student  marking  ‘disagree’  to  the  question  “Assignments  were  of 
definite  instructional  value,”  and  all  others  marking  either  ‘strongly  agree’  (33%)  or  ‘agree’  (62%).  When 
asked  to  comment  on  the  effectiveness  of  the  instructional  methods  used  in  the  class,  62%  of  students 
marked  ‘strongly  agree’  and  38%  of  students  marked  ‘agree.’  Students  were  equally  positive  about  the 
student-student  interaction  and  student-instructor  interactivity,  and  100%  of  the  students  felt  that  the  course 
helped  them  to  see  the  relationship  between  learning  theory  and  information  and  communication 
technologies,  with  57%  marking  ‘strongly  agree’  and  43%  marking  ‘agree.’  Students  felt  that  the  course 
would  help  them  integrate  ICT  into  their  classroom  activities  with  62%  marking  ‘strongly  agree’  and  24% 
marking  “agree.”  Overall,  differences  in  satisfaction  between  the  participants  in  the  three  course 
methodologies,  although  they  did  exist,  were  minor  and  not  statistically  significant  due  to  the  small  sample. 
The  majority  of  the  students  were  positive  about  their  participation  in  the  course,  their  interactions  with  the 
instructor  and  the  other  students,  their  success  with  the  technological  interface  and  the  knowledge  gained 
about  teaching  and  learning  with  information  and  communication  technologies. 

While  these  results  showed  that  students  responded  well  to  the  combination  of  both  directed  and 
constructivist  instructional  methods,  and  were  pleased  with  the  content  of  the  course,  these  findings  did  not 
indicate  whether  the  course  would  have  significant  impact  on  the  attitudes  and  classroom  practice  of  the 
teachers  involved.  To  determine  the  impact  of  the  course  content  six  months  later,  answers  to  an  e-mail 
questionnaire  were  examined.  Ten  teachers  returned  the  questionnaire,  which  represented  a third  of  the 
participants. 

The  first  question  asked  the  teachers  to  comment  on  any  change  in  attitude  or  perception  about  either 
technology,  learning  theory  or  teaching  in  general  that  they  attributed  to  the  content  or  interactions  in  the 
course.  The  answers  to  this  question  fell  into  three  categories:  confidence,  leadership  and 
teaching/leaming. 

Confidence 

All  the  teachers  reported  feeling  more  confident.  Four  teacheis  mentioned  specifically  that  they  felt  more 
confident  about  their  own  use  of  information  and  communication  technologies.  Four  teachers  reported  that 
they  were  more  assured  about  their  own  academic  abilities  or  more  confident  about  taking  part  in 
discussbns  about  learning  theory  and/or  research  relating  to  the  use  of  technology  in  the  classroom. 

“I  also  feel  that  the  research  we  had  to  do,  as  part  of  the  course,  was  good  for  me  personally,  as  it 
made  me  feel  more  confident  in  myself  Doing  the  readings,  the  bibliography  and  the  presentation 
were  all  hard  work,  but  I felt  extremely  pleased  with  my  efforts  and  finished  products.” 

“I’ve  certainly  gained  more  direction  and  confidence  to  mix  it  with  those  whom  I perceive  to  be 
more  technologically  literate  ...” 

“ . . . and  being  able  to  wax  eloquent  with  such  terminology  as  constructivism  and  scaffolding 
while  discussing  lesson  plans  proves  that  my  IT  self-confidence  has  gone  up  the  scale 
tremendously.” 


Several  teachers  noted  that  they  found  themselves  more  confident  as  they  planned  instruction. 

“This  has  helped  me  to  have  greater  confidence  in  using  ICT  with  my  classes  and  to  be  less 
concerned  about  not  knowing  how  to  do  everything.” 

“It  has  also  helped  me  to  become  more  comfortable  in  an  ICT  environment  where  collaboration 
and  teacher  facilitation  are  often  the  most  effective  strategies  to  use  when  managing  a class.” 
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But  more  so,  the  readings  have  given  me  the  confidence  and  knowledge  to  trial  things.  The  stress 
seems  to  have  alleviated  when  setting  up  items  for  the  class,  eg.  HyperStudio  stacks  at  the 
beginning  of  the  year  were  effortless.  I now  think  the  big  stresses  came  from  me,  as  the  children 
pick  it  up  so  quickly.” 

Leadership 

Several  teachers  commented  on  leadership  roles  they  had  assumed.  Some  of  these  roles  were  formal  such 
as  becoming  the  technology  coordinator  for  the  school,  running  professional  development  sessions  for  staff 
or  posting  technology  rich  lesson  plans  on  a website;  while  others  were  more  informal: 

“This  course  has  also  made  me  think  more  closely  how  to  regard  computer  skills  - or  lack  of  them 
- amongst  colleagues  and  how  to  extend  the  common  pool  of  knowledge  within  the  school.” 

“I  read  information  [in  the  course]  that  I have  referred  to  since  and  passed  on  to  other  teachers  I 
work  with.” 

“Attitude  was  positive  to  start  with,  but  knowledge  gained  has  led  to  informal  discussion  with 
peers  and  increasing  their  options.” 

Teaching/ Learning 

Comments  were  mentioned  by  the  teachers  that  pointed  out  changes  in  perceptions  toward  teaching  and 
learning. 


“I  have  found  myself  looking  at  the  work  that  I am  preparing  and  presenting,  and  reflecting  on 
whether  the  knowledge  fits  the  constructivist,  directed  or  behaviourist  style  and  in  this  reflection  I 
guess  comes  an  improved  evaluation  of  the  product  as  well  as  a question  of  the  value  of  the  work 
and  how  it  is  presented.” 

“One  of  the  perceptions  that  I have  become  more  aware  of  is  sorting  out  whether  the  computer  is  a 
tutor,  tool  or  tutee.  It  has  made  me  look  closely  at  what  the  computer  in  each  situation  requires  of 
the  learner.” 

“Whereas  I used  to  be  fascinated  by  what  computers  and  technology  could  do  I am  now  (since  the 
course)  refocused  on  the  fascinating  business  of  learning  and  how  to  set  up  the  optimum 
conditions  for  it  to  take  place.” 

“I  have  found  it  vital  that  I know  what  research  says.” 


The  second  question  focused  on  the  practical  classroom  applications.  Teachers  were  asked  to  comment  on 
any  activities  they  had  implemented  or  changes  they  had  made  resulting  from  the  knowledge  gained  in  the 
course.  Some  answers  focused  on  changes  in  the  classroom  that  involved  the  use  of  technologies. 

“I  have  focussed  on  information  literacy  in  our  library  and  tried  to  refine  the  library  use  to  include 
the  Internet.” 

“I  taught  and  used  PowerPoint  and  Internet  skills  in  my  IT701  assignment.  I have  since  gone  on 
to  teach  spreadsheets,  databases  and  HTML  in  basic  ways.” 


Teachers  also  reported  making  changes  in  the  way  they  approached  the  teaching  task,  taking  the  learners’ 
needs  into  consideration.  The  following  quotes  are  examples  of  this. 

“My  focus  with  ICT  now  is  to  get  the  children  asking  focus  questions  that  they  will  be  able  to 
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answer. 


This  course  has  made  me  think  about  direct  instruction  and  how  I can  present  it  when  teaching 
computer  skills  to  ensure  that  the  students*  learning  styles  are  used  and  higher  order  thinking  skills 
are  stimulated.’* 

“Probably  when  I analyse  the  impact  of  IT701  on  my  teaching  practice,  I believe  it  has  given  me 
ideas  on  how  I can  make  better  use  of  the  computer  as  a tool  and  how  it  fits  with  learning 
theories.’* 


From  the  ten  questionnaires  returned  there  is  evidence  that  the  course  has  had  an  impact  on  teachers  and 
they  way  they  approach  teaching  and  learning.  Many  teachers  report  increases  in  confidence  in  computer 
use,  planning  for  classroom  instruction  and  academic  abilities.  Some  mention  increased  leadership  within 
their  educational  communities.  All  teachers  noted  changes  in  their  perceptions  and  attitudes  about  using 
information  and  communication  technologies  in  the  teaching  and  learning  process.  These  included  a new 
appreciation  of  constructivist  versus  directed  approaches  to  teaching  and  learning,  fostering  higher  order 
thinking,  catering  to  individual  learning  styles,  the  teacher  in  the  role  of  facilitator,  an  increased 
appreciation  of  the  benefits  of  research  findings  on  classroom  practice  and  reflection  and  evaluation  of  their 
own  teaching  methods.  One  teacher  described  the  changes  in  this  way: 

“1  don*t  know  sometimes  I thin  k that  I must  have  been  walking  around  teaching  with  my  eyes 
closed.  Not  realising  why  I am  teaching  this  way,  where  did  these  teaching  strategies  come  from 
and  what  was  I trying  to  achieve  in  the  class.  This  course  has  opened  my  eyes  - even  though  it  is 
just  now  that  I have  realised  how  much  I have  changed  over  the  year  - it’s  quite  exciting!” 

Results  from  the  questionnaire  about  changes  in  classroom  practice  are  less  clear.  Some  teachers  reported 
on  specific  practices  they  had  changed  while  others  reported  on  changes  in  focus.  In  general,  the  responses 
from  the  second  question  were  extensions  and  variations  of  the  responses  to  the  first  question  and  did  not 
list  explicit  changes.  A limitation  of  this  study  is  the  low  number  of  questionnaires  that  were  returned  and 
the  limited  scope  of  the  questions  asked.  Further  study  would  be  required  to  determine  the  long-term 
impact  of  the  course  content  and  methodology  on  the  participants’  classroom  practice. 
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Abstract:  This  multimedia  programme  consists  of  colourful  animations  and  video 
footage,  supplemented  with  commentary,  text,  slides,  and  photographs.  It  is  used  for 
training  students  and  laboratory  personnel  in  basic  serology.  The  educational  rationale 
behind  the  design  and  development  of  this  programme  is  discussed.  The  compilation 
of  the  project  team  and  the  roles  and  responsibilities  of  each  member  is  presented  and 
the  programme  will  be  demonstrated. 

Introduction 

The  development  of  Applied  Serology  programme  was  initiated  by  the  Department  of  Veterinary  Tropical 
Diseases,  Faculty  of  Veterinary  Science,  University  of  Pretoria.  Multimedia  development  is  a costly 
procedure,  but  due  to  funding  from  a German  institution,  it  was  possible  to  obtain  the  necessary  resources 
to  develop  this  programme. 

Educational  rationale 

The  aim  of  the  programme  is  to  present  students  and  laboratory  personnel  with  a multimedia  tutorial  on 
adapted  serological  techniques  for  use  in  undergraduate  as  well  as  post-graduate  teaching  in  Immunology, 
Applied  Veterinary  Immunology  and  Serology. 

Practical  serological  procedures  as  used  in  diagnostic  and  research  laboratories,  are  relatively  complicated 
tests  that  require  dedicated  and  expensive  apparatus  and  reagents.  To  demonstrate  these  procedures  is 
labour-intensive  and  costly.  Therefore  it  would  be  prudent  to  utilize  communication  and  information 
technologies  to  provide  cost-effective  and  innovative  learning  opportunities  in  applied  serology  and 
immunology.  The  nature  of  serological  techniques  requires  that  students  be  afforded  maximum  visual 
exposure  and  hands-on  training.  By  developing  a programme  richly  illustrated  with  diagrams,  video  clips 
and  structured  animations,  it  will  provide  opportunities  for  students  to  be  repeatedly  exposed  to  procedures 
that  would  otherwise  only  have  been  demonstrated  once,  in  a teaching  laboratory. 

A self-assessment  question  databank  was  included  to  provide  the  students  the  opportunity  to  assess  their 
own  level  of  competence.  A student  can  do  the  self-assessment  as  often  as  he/she  wants  to  - the  questions 
for  each  exercise  will  differ  because  it  is  drawn  from  the  database  of  questions. 

Planning  phase 

The  content  of  the  programme  is  based  in  part  on  a book  compiled  by  C.  Staak,  F.  Salchow  and  N.  Denzin 
- ''Practical  Serology  ~ From  the  Basics  to  Testing"".  The  rest  of  the  content  was  added  and  it  was  very 
well  structured  before  the  development  of  the  programme  started. 

Collaboration  between  the  Department  of  Veterinary  Tropical  Diseases  of  and  the  GTZ-BgVV  Service 
laboratory  in  Germany  exists  and  the  development  of  this  project  was  made  possible  by  the  generous 
financial  support  provided  by  the  German  Ministry  of  Economic  Cooperation  and  Development  through  the 
GTZ.  Although  some  services  needed  to  be  paid  for,  the  Department  of  Telematic  Learning  and  Education 
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Innovation  provided  the  services  of  the  instructional  designer  and  programme  developer.  A project  team 
consisting  of  members  of  the  Department  of  Telematic  Learning  and  Education  Innovation,  lecturers  from 
the  Department  of  Veterinary  Tropical  Diseases  and  the  GTZ,  was  compiled. 

The  subject  matter  experts  from  the  GTZ  and  the  University  of  Pretoria  compiled  a very  extensive 
storyboard  even  before  the  project  proposal  was  submitted  to  the  Department  of  Telematic  Learning  and 
Education  Innovation  of  the  University  of  Pretoria.  This  department  is  responsible  for  the  development  of 
different  forms  of  electronic  educational  media  used  in  the  different  faculties. 

Project  team 

The  project  team  for  this  project  consisted  of  a project  manager,  content  specialists,  an  instructional 
designer  and  a programmer,  a graphic  designer,  a sound  engineer,  a photographer  and  video  photographer. 
Each  team  member  played  a significant  role  in  the  success  of  the  final  programme. 

Design  phase 

The  design  phase  consisted  of  the  development  of  an  appropriate  “look  and  feel”  and  the  selection  of  an 
appropriate  development  platform.  QuestNeH,  an  authoring  tool  was  used  for  the  development  of  the 
programme.  The  functional  utilisation  of  different  media  e.g.  animations,  voice-over,  video,  graphics  etc. 
was  very  well  researched  by  the  appropriate  members  of  the  project  team.  The  design  of  the  processes  and 
procedures  demonstrated  by  the  programme,  was  based  on  the  fact  that  the  principles  applied  in  serological 
techniques  are  often  very  difficult  for  students  to  understand  and  it  also  takes  a lot  of  time  and  laboratory 
equipment  to  perform  the  different  tests.  The  programme  provides  the  student  with  a graphic  and  animated 
explanation  of  each  test  principle.  The  test  procedures  are  explained  in  systematic  animations  and  are 
supplemented  with  video  footage.  The  aim  is  to  expose  the  learner  as  much  as  possible  to  the  real-life 
situation  in  the  laboratory.  Photographs,  slides  and  other  graphics  are  used  to  present  results. 

Development  process 

Numerous  obstacles  had  to  be  overcome  during  the  development  process  e.g.  the  distance  between  one  of 
the  subject  experts  (in  Germany)  and  the  developers  (in  South  Africa).  Failure  of  technology  at  a crucial 
phase  of  the  project,  also  hampered  the  continuity  of  the  project.  Emphasis  was  placed  on  the  effective 
utilisation  of  resources  and  time  management.  Quality  assurance  strategies  were  also  implemented  to 
ensure  that  all  the  team  members  were  involved  with  the  testing  and  evaluation  of  the  programme  and 
everybody  was  satisfied  before  the  next  phase  commenced.  Technical  aspects  as  well  as  content  matter 
were  thoroughly  assessed.  A lot  of  time  was  devoted  to  these  quality  assurance  strategies. 

Demonstration 

The  programme  will  be  demonstrated  to  provide  examples  of: 

• The  functional  use  of  the  different  media. 

• The  content  of  the  programme. 

• The  self-assessment  exercises. 

Conclusion 

Applied  Serology  is  an  excellent  example  of  the  power  of  multimedia.  During  the  design  and  development 
phases  of  this  project  many  valuable  lessons  were  learnt  that  will  be  implemented  in  the  process  of 
developing  of  future  multimedia  programmes.  It  was  surely  a worthwhile  experience. 
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Abstract:  This  paper  discusses  the  growing  need  for  a virtual  university  facility  in 
Lebanon  and  the  creation  of  a system  (VUSIL)  to  meet  this  need.  Although  general 
public  access  to  the  system  is  possible,  the  prototype  server  was  designed  with  two 
main  user- groups  in  mind:  students  and  professional  engineers.  The  ystem  was 
designed  to  provide  lifelong  learning  opportunities  based  on  the  potential  of  the 
Internet  and  the  World  Wide  Web  to  deliver  distance  education.  A needs  analysis 
was  conducted  and  the  results  were  used  to  inform  the  design  of  the  VUSIL  system. 
This  paper  describes  the  results  of  the  needs  analysis  and  outlines  the  design  of  the 
system. 


Introduction 

As  we  enter  the  21  century  and  embark  on  the  Information  Age,  there  is  a growing  belief  that  learning 
is  the  key  to  economic  success  and  competitiveness.  This  belief  is  developing  at  many  levels  - within 
individuals,  organisations  and  nations.  In  a fast  changing  world,  knowledge  rapidly  becomes  outdated. 
Employees  therefore  need  to  enhance  their  employability  through  ongoing  self-development. 
Furthermore,  employers  need  to  continually  update  the  skills  of  their  workforce  in  order  to  remain 
competitive.  Bearing  this  in  mind.  Keeling  et  al  (1998)  suggest  that  "learning  in  the  worlq?lace  ...  is 
essential  to  promote  a more  highly  skilled,  flexible  workforce  and  economic  stability'. 

Over  the  last  few  decades  a new  approach  to  learning  has  evolved  in  order  to  overcome  the  problems 
outlined  above.  This  new  approach  is  often  referred  to  as  "continuing  professional  development' 
(CPD).  CPD  is  just  one  aspect  of  the  broader  field  of  lifelong  learning.  The  development  of  lifelong 
learning  has  taken  place  in  two  basic  stages.  The  first  of  these  was  initiated  in  about  1970,  when  a 
number  of  international  organisations  such  as  UNESCO,  OECD,  the  Council  of  Europe  and  the 
Standing  Committee  of  European  Ministers  introduced  the  concept  of  lifelong  learning  into 
international  discussions  and  committed  themselves  to  its  realisation.  In  the  next  decade,  interest  in 
lifelong  learning  waned  but  it  still  retained  its  importance  (Peters,  1998). 

The  second  stage  in  the  development  of  lifelong  learning  started  in  the  mid-1990s.  Interest  in  the 
concept  is  now  greater  than  ever  before  and  is  of  a completely  different  nature  (Knapper  and  Cropley, 
2000).  The  economic,  technical  and  social  changes  and  the  new  challenges  set  by  the  Information 
Society  have  made  this  possible. 

The  lifelong  learning  approach  to  educational  provision  is  important  because  it  can  facilitate  the 
realisation  of  the  "education-on-demand'  and  "Just-in-time*  training  paradigms.  These  techniques  are 
increasingly  being  implemented  through  the  use  of  appropriately  designed  web -based  systems  (based 


on  both  intranets  and  the  Internet).  Web-based  training  (WBT)  is  a method  for  the  delivery  of  training, 
assessment  and  reference  material  via  the  Internet  or  an  intranet  (Steed,  1999).  Like  computer-based 
training  (CBT),  it  is  self-directed  and  self-paced.  It  allows  students  to  access  training  material  on  their 
own  personal  computers.  Unlike  CBT,  web-based  training  materials  may  be  rapidly  changed  and 
delivered  to  students,  without  the  cost  and  ‘latency’  involved  in  creating  and  mailing  out  a computer 
diskette  or  a compact  disk  (CD-ROM).  Another  important  facet  of  WBT  is  that  it  allows  the  tracking  of 
course  use  and  the  assessment  of  users’  progress  - in  order  to  better  determine  the  success  of  the 
training  course.  In  addition,  WBT  provides  interactivity  through  online  communication  and  multimedia 
techniques  such  as  chat  rooms,  e-mail,  discussion  groups  and  audio/video  presentations  (via  the 
Internet).  Some  of  the  benefits  of  web-based  training  include:  reduction  of  training  expenses  and 
training  time,  ease  of  delivery,  ease  of  updating  content,  multimedia  capabilities,  controllable  access 
and  accessibility-  anytime  and  anywhere. 

A genuine  virtual  university  has  no  physical  place  where  people  go  to  attend  classes.  The  classrooms 
are  nowhere  and  everywhere  at  the  same  time.  Indeed  according  to  Donoghue  et  al  (2001),  it  is 
anticipated  that  "virtual  classrooms  will  take  the  place  of  the  solid  buildings  where  students  currently 
attend  lectures,  at  set  times  in  set  rooms’.  Within  these  systems,  he  lecturers  are  instructors  and 
facilitators  who  are  trained  to  use  networking  facilities  to  teach  and  lead  interactive  discussions 
(Duggleby,  2000;  Barker,  2002a;  2002b).  Furthermore,  the  class  materials,  books  and  other  products 
can  be  ordered  via  the  telephone  or  on  the  Internet.  These  books  may  be  conventional  paper-based 
publications  or  they  may  be  electronic  documents  that  exist  within  a digital  library  facility.  These 
resources  can  be  augmented  by  the  use  of  email  and  other  means  of  electronic  communication  such  as 
video  conferencing.  The  students  and  facilitators  communicate  with  each  other  - just  as  they  might  if 
they  were  meeting  in  a student  lounge,  a library  or  during  a lecturer’s  office  hours.  Emerging  Internet 
technologies  can  be  used  to  create  online  learning  communities  that  are  not  restricted  by  geography. 


Project  scope 

The  fundamental  issue  that  this  research  is  intended  to  address  is  the  impact  that  digital  electronic 
libraries,  and  hence,  virtual  university  facilities  are  having  on  post-compulsory  educational  provision 
within  the  professional  sector  of  the  Lebanese  community.  Particular  emphasis  has  been  given  to 
exploring  the  implications  of  realising  the  education  on  demand  and  just-in-time  training  paradigms 
through  the  utilisation  of  a distributed  digital  library  system  based  on  the  use  of  a globally  available 
digital  information  superhighway.  A cyber  caf6  was  used  as  one  of  the  access  options.  Other  access 
possibilities  include  home  PCs,  libraries  and  conventional  universities. 

The  initial  work  in  this  research  project  involved  identifying  the  needs  of  particular  sectors  of  the 
Lebanese  professional  community  and  designing  a prototype  virtual  university  facility  to  meet  those 
needs.  Courseware  materials  have  been  designed,  and  are  in  the  process  of  being  implemented  and 
evaluated.  These  materials  are  to  be  delivered  using  the  prototype  VUSIL  facility.  &ibsequently,  they 
will  be  employed  as  a basis  for  an  empirical  study  of  lifelong  learning  in  Lebanon.  As  well  as 
courseware  creation,  particular  emphasis  was  given  to  mechanisms  for  course  accreditation,  skill 
assessment  and  the  mechanisms  underlying  the  optional  award  of  ‘digital  diplomas’.  A critical 
evaluation  of  the  potential  advantages  and  limitations  of  the  system  was  conducted  from  two 
perspectives:  those  involved  in  using  the  system;  and  those  responsible  for  maintaining  it. 


Needs  Analysis 

The  needs  analysis  was  undertaken  within  the  Internet  cafe  (’the  PC  club’)  in  Beirut,  Lebanon;  most  of 
the  participants  were  students  at  universities  and  professional  people  (mainly  engineers).  The  main 
aims  were:  (1)  to  identify  the  need  for  a prototype  system,  VUSIL;  and  (2)  to  expbre  different  topics  of 
interest  that  would  become  subjects  of  study.  Because  the  intended  users  of  the  system  are  both 
students  and  professional  people,  he  perspective  of  both  of  these  groups  were  studied.  For  this 
purpose,  an  online  questionnaire  was  used;  this  could  be  accessed  via  a URL  that  was  embedded  in  an 
electronic  message  that  was  sent  to  members  of  an  electronic  mailing  list.  This  contained  337 
addresses.  In  some  situations,  ‘follow  up’  face-to-face  interviews  were  carried  out  in  order  to  clarify 
issues  in  order  to  obtain  more  information  about  potential  study  areas. 


Seventy-seven  responses  were  received.  Sixty -three  percent  of  the  respondents  were  males;  thirty- 
seven  percent  were  students,  twenty-eight  percent  were  professionals  (mainly  engineers).  The  rest 
followed  a range  of  careers  such  as  computer-related,  management,  self-employed  and  consulting. 
Amongst  those  who  took  part  in  the  survey,  nearly  45%  held  (or  were  studying  for)  an  undergraduate 
degree  and  32%  possessed  (or  were  eligible  for)  a graduate  degree. 

The  results  of  the  needs  analysis  survey  for  VUSIL  were  encouraging.  This  is  reflected  by  the  fact  that 
more  than  75%  of  those  who  responded  reported  a positive  interest  in  online  learning.  Further  interest 
was  shown  in  obtaining  a quafification  through  the  Internet,  Four  levels  of  online  qualification  were 
distinguished:  diploma  or  certificate,  technical  degrees,  undergraduate  and  post-graduate  degrees.  A 
diploma/certificate  represents  one  or  a set  of  modules  relating  to  a subject  of  study;  at  this  level,  a full 
degree  is  not  aimed  for,  only  recognition  of  successfully  completing  the  required  modules.  A technical 
degree  is  based  on  a highly  specific  set  of  modules.  Undergraduate  (BSc)  and  post-graduate 
(MSc/PhD)  levels  represent  normal  degrees.  The  majority  of  respondents  (61%)  showed  an  interest  in 
the  post-graduate  degree  level,  while  19%  and  12%  expressed  an  interest  in  studying  at 
diploma/certificate  and  undergraduate  levels,  respectively.  Only  8%  were  interested  in  further  study 
leading  to  a technical  degree.  These  results  demonstrate  the  need  for  and  the  potential  importance  of  an 
online  learning  system  (such  as  VUSIL)  in  Lebanon. 

The  survey  has  demonstrated  the  need  for  a wide  variety  of  subjects  of  study.  It  revealed  that 
information  technology  topics  were  amongst  the  most  popular  subjects  of  interest.  Computers  and  the 
Internet  were  the  most  popular  topics  - accounting  for  more  than  25%.  There  was  also  considerable 
interest  in  other  subjects  - such  as  e-business  (1 1.5%),  money  and  finance  (10.7%),  science  and  nature 
(9.9%)  and  health  and  fitness  (9.1%).  Based  on  these  findings,  appropriate  topics  were  chosen  for 
delivery  through  VUSIL. 

An  attempt  was  made  to  study  the  reasons  behind  respondents’  choices  relating  to  the  selected  areas  of 
interest.  The  reasons  were  divided  into  four  main  categories:  36%  for  new  skills  relevant  to  jobs,  25% 
for  current  topics  of  importance,  25%  for  personally  relevant  topics  and  nearly  6%  for  particular  leisure 
interest  or  hobbies.  Other  minor  reasons  given  amounted  to  8%. 

The  findings  from  the  survey  we  have  conducted  fall  in  line  with  the  results  of  other  research  in  this 
area-  see,  for  example,  Sinitsa  (2000).  Our  results  provide  a sound  justification  for  the  development  of 
the  VUSIL  system. 


Development  of  the  Prototype 


The  VUSIL  project  involved  a number  of  different  design  and  development  stages.  Following  on  from 
the  initial  design,  the  first  prototype  was  programmed.  Table  1 shows  the  functions  that  were  included 
in  the  first  implementation  of  the  VUSIL  prototype.  The  next  stage  of  the  development  involved 
producing  two  short  courses  and  mounting  them  on  the  server.  At  the  conference,  further  progress  on 
the  development  of  the  system  will  be  reported. 


Table  1.  Functions  of  VUSIL 


Functions 

Description 

Administration  Office 

performs  the  tasks  of  a university’s  registration  office 

Lecture  Hall 

place  where  users  go  in  order  to  ‘take’  lectures 

Library 

gives  access  to  electronic  documents  and  bookshop  facilities 

Course  Catalogue 

provides  university  information  and  a list  of  available  modules 

Forum 

place  to  post  questions,  comments  and  get  replies 

Manager’s  Toolbox 

Provides  facilities  for  adding  and  deleting  courses 

The  main  aims  of  our  study  are  to  test  the  first  prototype  virtual  university  in  Lebanon,  to  help  tutors 
perform  their  teaching  tasks  and  activities  and  to  assist  students  and  professionals  in  their  learning 
activities.  As  can  be  seen  in  Table  1,  the  system  supports  six  main  functions.  Each  of  these  is  briefly 
discussed  below. 
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The  Administration  Office  performs  the  tasks  of  a university’s  registration  facility.  This  includes  the 
ability  for  users  to  register  for  online  courses,  view  and  edit  their  personal  profiles,  and  view  their 
transcripts.  The  Lecture  Hall  is  the  place  where  users  who  are  registered  for  a module  can  go  in  order 
to  take  the  corresponding  lectures.  The  Library  provides  two  main  facilities:  first  the  provision  of 
access  to  electronic  documents  relevant  to  particular  courses  of  study  (for  example,  course 
specifications  and  course  notes);  and  second  it  provides  a bookshop  facility.  As  VUSIL  does  not  handle 
commercial  transactions,  there  are  links  to  Web  sites  (such  as  www.amazon.com)  vhere  students  can 
buy  books  for  the  courses.  Students  and  online-tutors  can  enter  the  Library  in  order  to  purchase 
required  books  for  courses,  university  merchandise,  souvenirs,  etc.  These  can  be  placed  in  a shopping 
cart  and  purchased  from  third*party  suppliers.  The  Course  Catalogue  provides  users  with  general 
university  information  along  with  a list  of  available  modules.  Upon  choosing  a certain  module,  users 
are  presented  with  a more  detailed  description.  The  Forum  gives  users  the  ability  to  post  questions  and 
information  related  to  the  modules  they  are  taking.  Modules  are  automatically  added  to  the  Forum  once 
they  have  been  entered  in  the  Administration  Office.  In  the  Forum,  users  can  select  the  module  to 
which  they  wish  to  post  questions  and  comments.  In  turn,  learners  and  tutors  can  reply  to  these  posted 
items.  The  Forum  also  serves  as  a place  for  tutors  to  post  information  about  modules  such  as  test  dates, 
assignments,  and  so  on.  The  Manager’s  Toolbox  is  primarily  intended  for  course  instructors  and 
managers.  It  enables  new  courses  to  be  added  to  the  system  and,  if  necessary,  old  ones  to  be  deleted.  In 
addition,  particular  parts  of  any  given  course  can  be  modified  and  deleted  and  new  parts  added.  New 
functions  can  be  added  to  the  toolbox  as  needs  arise. 


Future  Developments 


VUSIL  has  been  designed  and  built  using  a basic  set  of  requirements.  There  is  therefore  still  a number 
of  features  and  improvements  that  need  to  be  incorporated  into  the  system.  These  fall  into  short-term 
and  long-term  development  goals.  Three  of  the  most  important  short-term  developments  that  should  be 
included  in  the  system  are:  (1)  the  addition  of  a ‘user-friendly’  help  system  that  is  capable  of  providing 
new  users  with  general  help,  guided  walk-throughs  and  advice  related  to  using  the  system;  (2)  the 
provision  of  a topic  index  and  a search  facility  to  enable  students  to  search  the  VUSIL  web  in  order  to 
locate  particular  items  of  interest  - software  to  perform  each  of  these  latter  functions  is  available 
commercially;  and  (3)  linking  the  VUSIL  system  to  external  academic  support  sites.  This  is  discussed 
below. 

For  the  last  eighteen  months,  the  University  of  Teesside  (UK)  has  been,  evaluating  the  use  of  a 
commercial  software  package  - called  Blackboard  - to  create  an  online  learning  environment. 
Blackboard  (www.blackboard.com)  offers  a complete  suite  of  enterprise  software  products  and  services 
that  provides  online  education  infrastructure  for  schools,  colleges,  universities  and  other  education 
providers.  Using  the  power  of  the  Internet,  it  provides  software  solutions  that  facilitate  the  management 
of  the  many  diverse  demands  of  an  academic  organisation  and  helps  to  bring  educational  experience 
online.  One  of  Blackboard’s  important  features  is  its  forum  facility.  As  a further  development  of  the 
VUSIL  system,  we  plan  to  provide  our  users  with  access  to  Blackboard’s  forum  - thus  giving  them  the 
ability  to  access  some  of  Blackboard’s  facilities . 

From  the  perspective  of  long-term  developments,  there  are  three  important  enhancements  to  be  made  to 
the  system.  First,  the  addition  of  a large  variety  of  subjects  of  study;  this  would  cover  most  of  the  topics 
in  which  students  have  expressed  potential  interest  (based  on  the  survey).  This  variety  of  subjects 
would  attract  more  users,  thus  adding  to  the  overall  utility  of  the  whole  system. 

The  second  main  development  would  involve  the  incorporation  of  XML  as  the  basic  mark-up  language 
(Maris,  2000).  Use  of  XML  is  important  because  it  facilitates  the  preparation  of  the  catalogues  and 
indexes  needed  in  order  to  implement  efficient  and  effective  searching  for  material  and  modules  in  the 
system.  It  also  simplifies  data  exchange  with  respect  to  the  administrative  tasks  associated  with 
monitoring  and  assessment  - and  subsequent  reporting  of  feedback  to  both  students  and  staff 

The  third  enhancement  would  involve  the  addition  of  streaming  audio  and  video  facilities  (Beckstrand, 
Barker  and  van  Schaik,  200 1).  This  type  of  technology  does  not  require  files  to  be  downloaded  before 
use.  In  addition,  the  use  of  streaming  technology  does  not  require  specific  class  meeting  times  while 
still  simulating  on-campus  presentations. 


1440 


Conclusion 


Computers  and  electronic  communications  technologies  are  increasingly  being  used  to  provide 
mechanisms  for  the  support  of  open  and  distance  learning.  These  technologies  make  it  possible  to 
provide  mechanisms  to  facilitate  an  ‘education  on  demand’ approach  in  a variety  of  different  situations. 
This  can  occur  at  home,  in  the  workplace,  in  libraries  and  in  colleges.  Of  course,  through  the  use  of 
mobile  computing  facilities  (such  as  notebook  computers  and  hand-held  devices),  there  is  no  limit  to 
where  electronic  learning  can  take  place  provided  a wireless  signal  of  some  sort  can  be  received.  This 
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iPaqhand4ield  computerare  now  being  put  - see,  for  example,  Schneider  (2001) . 

There  is  now  a significant  number  of  virtual  university  systems  available  to  support  open  and  distance 
learning  - particularly  in  the  UK  and  North  America.  We  have  designed  and  developed  a prototype 
virtual  university  server  system  to  support  the  needs  identified  in  this  paper  Quite  naturally,  these 
systems  are  based  on  (and  embed)  ‘Western  culture  - as  they  are  primarily  designed  to  serve  American 
and  British  societies.  However,  within  Europe,  the  use  of ‘all  English’  material  is  problematic  because 
of  the  many  different  languages  and  cultural  differences  that  exist  This  too  is  a problem  in  many 
Middle  Eastern  countries  where  Arabic  is  the  predominant  language.  Although  the  VUSIL  project  is 
currently  being  developed  in  English,  a long-term  objective  of  the  project  is  to  create  an  online  system 
that  can  be  used  by  both  English  and  Arabic  speaking  people.  Naturally,  two  important  aspects  of  this 
work  are  the  development  of  a bilingual  facility  both  in  terms  of  the  end-user  interface  to  the  system 
and  its  academic  content.  Such  a development  is  quite  challenging  and  makes  it  different  from  many  of 
the  other  monolingual  virtual  university  systems  that  currently  exist.  For  Arabic  speaking  nations,  this 
is  an  important  development  because  although  many  students  and  most  professional  people  in  Lebanon 
can  speak  and  use  English,  there  are  many  other  people  in  the  Middle  East  who  are  likely  to  want  to 
use  lifelong  learning  opportunities  but  who  are  unable  (or  unwilling)  to  use  English  as  their  mode  of 
communication.  The  VUSIL  system  could  easily  cater  for  this  type  of  audience. 
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A Neural-Network  system  for  Automatically  Assessing  Students 


Absiraci:  Tnis  paper  is  concerned  with  an  automated  system  for  grading  students 
into  an  ability  level  in  response  to  their  ability  to  complete  tutorials.  This  is  useful  in 
that  the  student  is  more  likely  to  improve  their  knowledge  of  a subject  if  they  are 
presented  with  tutorial  material  at  or  just  beyond  their  ability.  However,  dynamically 
responding  to  a student’s  changing  knowledge  about  a subject  usually  requires  the 
presence  of  a human  teacher,  an  altogether  expensive  resource.  The  system  discussed 
here  can  grade  both  a student  and  the  questions  in  a tutorial  with  mininal  input  from 
the  human  teacher.  In  order  to  accomplish  this  a specialist  neural  network  is 
employed.  The  design  and  operation  of  our  system  is  discussed  along  with 
arguments  as  to  why  a neural  network  approach  is  suitable  for  this  problem. 


Introduction 

The  Tutorial  Supervisor  (TS)  is  an  automatic  system  for  grading  a student  into  an  ability  level 
in  response  to  the  student’s  interaction  with  tutorial  questions.  Once  the  student  has  been  graded  then  a 
question  or  tutorial  can  be  selected  which  is  inline  with  best  pedagogic  practice  (Bergeron  1989).  Our 
TS  is  an  expansion  of  the  TS  designed  by  Bergeron  et  al  (1989).  Our  TS  improves  on  Bergeron’s 
original  specification  by  having  the  additional  ability  to  adapt  to  both  students  and  questions/tutorials 
as  the  system  is  in  use,  as  opposed  to  requiring  the  system  to  be  taken  off-line  and 
reprogrammed/trained.  The  TS’s  ability  to  respond  in  real-time  to  a student’s  changing  ability  and  how 
a population  of  students  perceive  a particular  tutorial  or  question  is  brought  about  by  the  use  of  a 
specialist  neural  network  device  that  is  able  to  learn  and  adapt  without  human  intervention.  A thorough 
description  of  our  TS  along  with  complete  details  of  its  design  and  testing  can  be  found  in  Mullier 
(1999). 

Operation  of  the  Tutorial  Supervisor 

The  TS  is  simple  in  operation.  A set  of  questions  to  be  given  to  the  student  is  recorded  in  a 
database  and  is  graded  with  a difficulty  level  by  the  author  of  the  question.  A student  can  then  be  given 
an  appropriate  question,  depending  upon  the  student’s  ability  and  the  difficulty  of  the  question. 
However,  in  order  for  a student  to  progress  in  their  learning  it  is  necessary  to  pitch  questions  so  that 
they  sufficiently  tax  the  student  without  it  being  impossibly  difficult  (Bergeron  1989).  Therefore  it  is 
necessary  to  track  the  student’s  change  in  ability  as  they  progress  through  the  learning  material.  Our 
system  achieves  this  by  recording  the  student’s  interactions  with  questions  and  mapping  this  onto  an 
ability  level.  The  system  is  robust  to  exceptions  in  the  student’s  behaviour,  since  a student  may 
generally  perform  well  but  make  a mistake  with  one  particular  question.  Similarly,  the  system  is  able  to 
regrade  questions  in  the  question  database.  For  example,  a question  may  have  been  graded  by  the 
author  as  being  relatively  easy.  However,  it  may  transpire  that  a population  of  students  actually  find  it 
difficult.  This  will  be  bourn  out  by  most  students  who  should  have  performed  well  with  the  question 
actually  performing  poorly.  Such  a situation  negates  the  pedagogy  stated  above.  Our  system  is  able  to 
statistically  determine  that  a question  has  been  misgraded  and  is  able  to  remedy  the  situation.  The 
remainder  of  this  paper  will  discuss  the  issues  relating  to  the  design  and  implementation  of  our  TS 
system. 

Rationale  for  Using  a Neural  Network 

A neural  network  is  an  Artificial  Intelligence  (AI)  system  that  is  able  to  learn  rules  in  response 
to  being  presented  with  many  examples.  The  neural  network  is  said  to  learn  the  rules  from  the 
examples.  In  contrast  a traditional  rule-based  system  would  have  rules  encoded  within  it  that  a designer 
has  previously  identified.  The  advantage  of  neural  network  systems  is  that  it  is  not  always  possible  for 
a human  designer  to  express  and  encode  rules  in  a reasonable  time  frame  or  even  express  them  at  all.  A 
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further  disadvantage  of  rule-based  systems  is  that  if  the  rules  change  for  some  reason  then  it  is 
necessary  for  the  designer  to  reincorporate  the  new  rules  within  the  rule-base  (Hagan  et  al  1 996). 

A further  reason  for  choosing  a neural  network  for  the  TS  in  preference  to  a rule-based  system 
is  that,  unlike  a rule -based  system,  a neural  network  can  be  domain  independent.  It  is  unlikely  that,  for 
example,  a high  level  student  produces  results  in  the  same  range  for  every  type  of  domain.  Thus  the 
rule  "IF  SCORE  >70  THEN  LEVEL  10"  is  only  likely  to  apply  to  the  domain  that  it  was  initially 
defined  for.  This  is  the  reason  why  Bergeron  et  al  (1989)  use  a neural  network  for  their  Tutorial 
Supervisor.  Their  neural  network  holds  the  rules  that  it  has  learnt  from  its  training  data  (the  first 
domain).  It  is  then  able  to  change  its  rules  in  response  to  new  data  (new  domains),  by  retraining  off- 
line. In  this  manner  the  neural  network  can  adapt  to  misconceptions  or  inaccuracies  in  the  original  rules 
and  adapt  to  new  situations.  This  would  be  a difficult  and  time  -consuming  process  for  a symbolic  rule- 
based  system,  since  it  would  require  the  re-engineering  of  the  rule -base  by  a designer,  in  that  the  new 
rules  would  have  to  be  identified  and  then  encoded.  In  essence,  the  neural  network  is  doing  the  job  of 
the  human  rule  designer.  Designing  such  rules  is  not  necessarily  a simple  matter,  since  it  requires  the 
human  designer  to  examine  many  student  interactions  with  various  questions  and  tasks  so  that  a valid 
grading  of  each  question  can  be  made  (e.g.  this  question  was  answered  well  by  novice  students,  it  is 
therefore  easy  and  can  be  presented  to  other  novice  students).  The  situation  is  further  complicated  by 
the  possibility  that  different  populations  of  students  (students  from  different  classes  or  tutorial  groups) 
may  have  different  previous  knowledge  of  the  domain  and  therefore  the  initial  question  gradings  may 
not  apply  to  them.  It  would  therefore  be  helpful  if  an  Al  system  could  be  employed  to  accomplish  the 
task  of  dynamic  question  grading  and  therefore  remove  some  burden  from  the  author.  However, 
Bergeron  et  aPs  (1989)  TS  is  unable  to  readapt  to  different  domains  without  being  manually  provided 
with  new  training  data  and  then  retra  ined  off-line.  The  remainder  of  this  paper  describes  the  neural 
networks  used  for  our  Tutorial  Supervisor,  which  improves  upon  Bergeron  et  aPs  (1989)  design  by 
allowing  the  automatic  on-line  adaptation  to  different  domains. 

Tutorial  Supervisor  Architectu  re 

The  Kohonen  self-organising  map  is  a specialist  type  of  neural  network  with  the  ability  to 
learn  without  a designer  specifically  indicating  that  the  networks  results  are  right  or  wrong,  this  is 
termed  unsupervised  learning  (Callan  1999).  This  is  useful  in  our  system  since  the  distinction  between 
a novice  student  and  an  expert  student,  in  terms  of  marks  at  tutorial  nodes,  may  be  small,  or  may  vary 
significantly  from  domain  to  domain.  For  example,  the  majority  of  students  may  achieve  marks 
between  50%  and  60%,  with  a few  results  between  60%  and  75%  percent  and  a few  between  40%  and 
50%.  There  are  therefore  two  large  ranges  of  numbers  that  occur  infrequently  (040  and  75-100).  If  a 
standard  type  of  neural  network,  such  as  a back-propagation  network  which  is  the  most  commonly  used 
neural  network,  were  to  be  used  to  model  the  above  problem  then  these  mark  ranges  must  be  identified 
beforehand  or  the  neural  network’s  outputs  would  have  to  be  designed  to  produce  student  ability  levels 
between  0 and  100,  in  order  to  accommodate  a broad  range  of  scores.  Identifying  scores  beforehand  is 
not  likely  to  be  practical  since  it  would  require  the  collection  of  a large  amount  of  data  (with  no  initial 
benefit  for  the  student).  Designing  such  a generic  neural  network  also  introduces  the  following 
difficulties.  A back-propagation  neural  network  must  have  enough  ability  levels  (outputs)  to  clearly 
demonstrate  the  distinction  between  students  in  these  highly  clustered  areas,  necessitating  an  increase 
in  outputs  for  all  areas  to  cover  for  all  possibilities,  even  those  that  are  unlikely.  The  increase  in  outputs 
renders  the  neural  network  more  complex,  resulting  in  a network  that  is  more  difficult  to  train.  Further, 
and  most  crucially,  once  the  trained  back-propagation  neural  network  is  used  for  different  domains, 
then  there  is  no  direct  correspondence  between  an  ability  level  for  one  domain  and  an  ability  level  for 
another.  This  is  because  an  output  of  the  back-propagation  neural  network  does  not  correspond  directly 
to  a student  ability  level,  since  the  student  ability  level  may  vary  between  domains.  The  upshot  of  this 
is  that  the  back-propagation  neural  network  would  require  complete  retraining  for  different  domains. 

A Kohonen  network  can  solve  the  above  problem  by  continually  adapting  to  input  stimuli 
whilst  it  is  being  used  by  students.  This  is  because  of  the  way  a Kohonen  network  operates.  A Kohonen 
network  is  given  a number  of  outputs  by  the  network  designer,  representing  the  number  of  categories 
that  the  network  designer  wishes  the  network  to  identify  (the  number  of  required  student  levels  in  our 
case).  It  is  then  left  to  the  network  itself  to  sort  the  input  data  into  this  number  of  categories,  since  it  is 
not  made  explicit  by  the  training  data  itself.  In  the  case  of  the  back-propagation  neural  network  it 
would  be  required  to  include  an  example  solution  with  the  training  data.  If  there  are  ten  or  more  distinct 
patterns  in  the  data  then  a correctly  trained  Kohonen  will  learn  by  itself  to  distinguish  them  (Kohonen 
1989,Callan  1999).  Therefore,  it  is  irrelevant  to  the  network  if  the  actual  values  of  the  input  data 
change;  it  will  still  attempt  to  separate  them  into  the  number  of  categories  represented  by  the  number  of 


outputs  that  it  has.  The  outputs  of  a Kohonen  neural  network  therefore  behave  as  fuzzy  sets,  whose 
boundaries  may  change  over  time.  The  Kohonen  neural  network  is  therefore  simpler  than  the  standard 
neural  network  for  the  TS  in  this  case,  since  the  number  of  outputs  can  be  kept  small  since  it  is  not 
necessary  to  design  for  all  unlikely  possibilities  (the  network  will  adapt  to  them  if  they  occur). 

Training  Data 

A neural  network  has  a number  of  inputs,  which  in  our  case  represent  the  student’s  responses 
to  questions  and  a iiunibei  of  outputs,  each  One  represciuing  a Unique  ability  level.  The  inputs  to  the 
Kohonen  neural  network  must  incorporate  history  data  in  order  to  make  a more  informed  evaluation  of 
the  student  and  therefore  avoid  a restriction  of  Bergeron  at  al’s  (1989)  neural  network,  namely  reacting 
to  a one  off  error  (or  success)  from  a student.  History  data  can  be  used  to  prevent  the  TS  from  making 
snap  judgments  of  the  student.  For  example,  if  the  student  is  generally  performing  well,  but  gets  one 
question  wrong,  then  if  no  history  data  is  taken  into  account  the  TS  is  forced  to  make  a decision  based 
only  upon  the  most  recent  presentation  and  the  student  is  likely  to  drop  a level.  The  student  ability  itself 
represents  a degree  of  history  data,  in  that  if  a student  is  regarded  to  be  an  expert  student,  then  they 
must  have  performed  well  in  the  past.  However,  the  direct  incorporation  of  history  data  prevents  a 
continual  changing  of  levels  based  upon  one  interaction  only.  The  incorporation  of  history  data  can  be 
achieved  by  presenting  a number  of  previous  interactions  with  tutorial  nodes  to  the  neural  network. 
Each  time  a new  interaction  is  presented  the  previous  interactions  are  shifted  along  the  inputs  to 
accommodate  the  new  input  and  the  oldest  interaction  is  lost.  Training  data  supplied  to  the  neural 
network  are  figures  that  represent  a percentage  value  of  a student’s  interaction  with  a tutorial.  For 
example,  if  the  student  achieved  a 50%  success  level  with  a tutorial  question,  then  it  is  this  figure  that 
is  passed  to  the  TS. 

Re-grading  Questions  Using  Fuzzy  Logic 


Each  question  level  is  generally  presented  to  a student  of  the  same  level,  or  just  below,  a 
pedagogy  used  with  success  by  Bergeron  et  al  (1989),  in  that  a level  x student  should  be  able,  overall, 
to  answer  a level  x question.  A question  may  however,  be  graded  hcorrectly  by  the  domain  author. 
This  can  be  determined  by  the  system  after  a number  of  interactions  with  different  students  (a 
population  of  students  who  should  be,  generally,  getting  a question  right  are  getting  it  wrong  or  vice 
versa).  It  is  not  suitable  to  immediately  re-grade  a question  with  respect  to  an  interaction  with  one 
student,  however.  As  has  been  discussed  earlier,  a student  is  a complex  entity  and  it  is  difficult  to 
formulate  rules  describing  them  accurately.  In  order  to  resolve  this  problem  each  question  level  is 
modeled  as  a fiizzy  set.  This  allows  a question's  level  to  be  adjusted  slightly,  within  the  level,  without 
necessarily  affecting  the  overall  level  (as  presented  to  the  student),  thus  there  is  a buffering  effect  and 
the  question  does  not  rapidly  leap  back  and  forth  between  levels.  The  use  of  fuzzy  sets  also  provides  a 
mechanism  for  allowing  a question  to  belong  to  more  than  one  question  level  set,  providing  a smoother 
transition  between  levels.  This  differs  from  Bergeron  et  al’s  (1989)  approach  in  that  they  collect  data 
from  the  students  and  then  periodically  use  it  to  update  the  training  of  the  neural  network.  There  is 
therefore  a delay,  which  ensures  that  the  question  levels  do  not  suddenly  change,  which  could 
potentially  result  in  the  question  level  continually  changing  and  thus  be  distracting  to  the  students. 
However,  the  drawback  is  that  this  is  a manual  process  that  requires  the  direct  intervention  of  the 
system  designer.  The  process  for  regrading  questions  described  below  is  achieved  automatically,  whilst 
still  maintaining  the  delay  between  the  question  being  presented  to  a student  and  changing  its  level. 

A question  is  re-graded  by  a population  of  students’  interactions  with  the  question  being 
determined  as  incorrect  by  the  TS,  for  the  reasons  described  above.  Such  erroneous  interactions  cause 
the  question  ability  to  move  within  the  fuzzy  set  until  it  crosses  into  a different  fuzzy  set.  A question’s 
ability  level  is  therefore  only  changed  after  a number  of  erroneous  interactions  with  students,  the  actual 
number  being  dependent  upon  the  size  of  the  fuzzy  set,  the  fiizzy  set  is  therefore  acting  as  a buffer.  A 
simple  fiizzy  processor  accomplishes  question  re-grading.  The  fiizzy  processor  compares  the  level  of 
the  current  question  and  the  student  level  output  of  the  TS  neural  network.  The  buffer  can  be 
implemented  using  three  fiizzy  rules  (Kosko  1996): 


1 IF  S_LEVEL  > Q_LEVEL  THEN  Q^LEVELf-  Q_LEVELf  + 1 

2 IF  S_LEVEL  < Q^LEVEL  THEN  Q_LEVELf-  Q_LEVELf  - 1 

3 IF  S_LEVEL  - Q^LEVEL  THEN  Q_LEVELf  = Q_LEVELf  (remain  unchanged) 


where 

S_LEVEL  is  the  level  assigned  to  a student 

Q_LEVEL  is  the  level  of  the  question,  used  to  decide  whether  it  is  suitable  for 
the  student. 

Q_LEVELf  is  the  fuzzy  membership  number  of  the  question. 
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Figure  A Membership  of  the  fuzzy  sets 


Using  figure  A,  a question  may  belong  to  one  or  two  of  four  levels.  If  the  question  has  a value 
of  Q_LEVELf  corresponding  to  x,  then  the  question  is  regarded  as  both  level  three  and  level  four.  If 
however,  as  a result  of  interactions  with  several  students,  rule  one  is  repeatedly  fired,  then  the  value  of 
Q_LEVELf  will  increase  and  the  question  will  become  graded  as  level  four  only.  Conversely,  if  rule 
two  is  repeatedly  fired  then  the  value  of  Q_LEVELf  will  decrease  and  the  question  will  be  graded  as 
levels  only.  Such  changes  may  result  in  further  increases  or  decreases  in  the  value  of  Q_LEVELf, 
which  may  become  any  of  the  levels  available.  The  utilisation  of  this  fuzzy  system  ensures  that  the 
level  of  a question  is  not  changed  (in  terms  of  presenting  to  students)  in  response  to  individual 
interactions  with  students.  The  fuzzy  logic  is  able  to  distinguish  overall  trends  and  is  therefore  robust  in 
the  presence  of  exception  data. 

The  changing  of  levels  is  dependent  upon  the  size  of  the  fuzzy  sets,  since  the  size  of  the  fuzzy 
set  directly  affects  how  many  interactions  are  required  before  a question  migrates  from  one  level  to 
another.  These  start  and  end  points  also  define  whether  a question  can  belong  to  more  than  one  level  or 
not.  It  is  intuitively  sensible  to  provide  a small  overlap  of  neighbouring  fuzzy  sets.  This  provides  a 
simple  buffer  that  will  help  the  question  to  find  its  true  grade.  For  example,  if  fuzzy  sets  were  to  be 
defined  separately  (or  as  a discrete  set)  then  a question  may  be  presented  as  a difficult  level  but  become 
graded  as  an  easier  level  after  several  interactions.  Once  this  change  has  occurred  then  the  question  is 
no  longer  presented  to  the  original  class  of  students  that  graded  it.  This  may  result  in  the  question 
becoming  stuck  at  a particular  level.  If  instead  the  question  is  presented  to  both  the  original  class  of 
student  and  the  new  class  of  student,  then  a smoother  progression  from  one  level  to  another  may  result 
and  the  extra  information  enables  the  question  level  to  settle  more  easily.  If  the  fuzzy  sets  are  defined 
so  that  a question  may  belong  to  more  than  two  levels,  then  the  question  level  may  not  settle  at  all,  as 
one  level  of  student  may  push  the  value  of  Q_LEVEL  one  way  and  another  level  may  push  it  in  the 
opposite  direction. 


Experimental  Trials 

Our  TS  was  designed  using  a Neural  Network  package,  NeuroShell2  by  Ward  Systems. 
Several  configurations  of  neural  network  were  built  and  tested  with  both  simulated  data  and  real 
student  interactions.  The  most  successful  architecture  is  discussed  below.  A full  description  of  the 
experimental  trials  can  be  found  in  Mullier  (1999). 

The  Kohonen  neural  network  proved  to  be  a successful  architecture  for  the  problem  of  grading 
students  into  ability  levels.  Most  permutations  of  parameters  produced  neural  networks  that  converged 
upon  a solution.  A fast  and  reliable  neural  network  could  be  produced  with  between  five  and  twenty 
inputs  or  outputs.  It  is  possible  to  increase  this  number,  but  this  is  unlikely  to  be  required,  since  it  is  not 
desirable  to  use  information  from  too  far  in  the  past,  since  the  network  does  not  have  any  knowledge  of 
time.  It  has  been  experimentally  determined  that  the  number  of  outputs  should  not  rise  above  twenty.  If 
more  levels  are  required  then  the  outputs  may  be  combined  to  form  fuzzy  sets. 

Parameters  for  a generic  Kohonen  Tutorial  Supervisor,  i.e.  one  that  will  converge  on  a 
solution  and  provide  a high  degree  of  student  grading  for  a variety  of  domains  are  suggested  as  the 
following: 


i 
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• 5 Inputs  - this  has  been  found  to  provide  the  network  with  enough  information  with  which  to 
evaluate  the  student.  It  is  suggested  that  an  input  filter,  a simple  algorithm  that  combines  many  history 
elements  into  the  five  required,  is  employed  if  more  than  this  number  of  inputs  is  required. 

• 10  Outputs  - corresponding  to  ten  student  levels,  if  more  levels  are  required  then  it  is  advised  that 
thirty  levels  is  the  upper  limit,  beyond  this  the  neural  network  becomes  less  likely  to  activate  all  of  its 
outputs. 


s Any  form  of  '^ata  cxtracticn  may  be  used  to  form  the  training  set  however  a large  number  of 
examples  are  required  to  produce  a fully  trained  network.  One  thousand  student  interactions  provide 
enough  examples.  Note  therefore,  that  for  a network  to  be  trained  with  real  student  data  is  probably 
impractical.  However,  since  the  Kohonen  network  is  fully  adaptable,  it  is  suggested  that  a network  be 
trained  upon  generated  data  and  then  allowed  to  adapt  to  real  students.  The  generated  data  could  be 
designed  to  represent  realistic  but  uncomplicated  situations.  For  example,  training  the  network  to 
model  simple  rules  such  as  “If  result  in  the  range  40-50  THEN  set  student  level  to  5”.  The  neural 
network  is  then  able  to  adapt  to  any  misconceptions  present  in  these  rules.  Once  a neural  network  has 
been  trained  it  may  be  saved  and  replicated. 


Discussion  and  Conclusion 

The  Tutorial  Supervisor  is  intended  to  be  an  automatic  system  for  gauging  a student’s  abilities 
with  tutorials.  This  imposes  a restriction  upon  the  kind  of  material  that  can  be  offered  in  a tutorial, 
since  it  must  be  suitable  for  automatic  assessment.  Automatic  assessment  effectively  rules  out 
assessments  that  cannot  be  graded  in  a relatively  simple  fashion.  For  example,  it  would  not  be  possible, 
with  current  technology,  to  have  an  essay  automatically  assessed,  since  this  would  require  a complex 
understanding  of  the  essay  on  the  part  of  the  assessor.  Automatic  assessment  is  limited  to  tasks  that  can 
be  broken  down  into  elements  that  can  then  be  individually  marked.  A tutorial  may,  for  example,  take 
the  form  of  ten  questions,  each  of  which  could  be  answered  by  the  student  and  graded  as  right  or 
wrong,  or  graded  as  containing  relevant  keywords.  However,  assessment  is  not  the  role  of  the  TS, 
which  is  presented  with  completed  assessments  (results).  Assessment  is  therefore  limited  to  multiple- 
choice  questions  or  identifying  that  the  student  has  visited  certain  nodes  (or  a combination  of  both). 

A key  issue  of  concern  regarding  the  TS  is  the  number  of  student  levels  that  the  TS  is  to 
recognise  and  output.  Each  student  level  should  have  tutorial  material  generated  for  it;  since  it  is 
important  to  target  tutorial  tasks  at  the  student’s  ability,  this  is  seen  as  being  of  more  educational 
benefit  than  offering  the  same  tutorials  to  all  students  and  then  assigning  a student  level  based  upon  the 
grade  that  the  student  achieves  (Bergeron  1989),  although  there  is  no  technical  reason  why  the  latter 
could  not  be  done.  The  number  of  student  levels  therefore  may  change  between  domains,  since  some 
domains  may  have  a richer  set  of  assessment  questions  than  others  (for  a number  of  possible  reasons). 
A possible  conflict  therefore  could  arise  between  the  number  of  student  levels  that  has  been  designed 
into  the  TS  by  the  system  designer  and  the  number  of  student  levels  that  are  required  by  the  current 
domain  author.  A possible  solution  to  this  problem  is  for  the  system  designer  to  provide  a TS  that  is 
capable  of  outputting  a large  number  of  student  levels  and  then  each  domain  author  can  allow  it  to 
adapt  to  their  domains  and  ignore  the  inactive  outputs  from  the  TS  that  will  naturally  arise  if  there  is 
not  sufficient  input  student  levels.  The  benefit  of  this  approach  is  that  one  TS  configuration  could  be 
used  for  many  domains  without  the  need  for  reconfiguration.  However,  the  drawback  is  that  some 
outputs  of  the  TS  will  always  remain  inactive,  although  the  experiments  carried  out  as  part  of  the 
research  demonstrated  that  active  outputs  tend  to  cluster  together  and  so  are  easily  identifiable.  A 
problem  related  to  the  number  of  outputs  is  the  number  of  inputs. 

The  amount  of  history  data  presented  to  the  TS  directly  affects  the  grading  of  the  student,  in 
that  the  more  history  data  presented  to  the  neural  network,  the  greater  the  effect  of  previous  results  with 
tutorials.  This  is  a similar  situation  to  that  of  the  number  of  student  levels,  in  that  it  is  possible  to  design 
a TS  with  a large  number  of  inputs  and  then  use  only  the  required  amount.  However,  it  is  not  a simple 
matter  to  determine  how  much  history  data  to  present  to  the  neural  network  in  order  to  aid  the  student 
the  most.  This  issue  is  difficult  to  reconcile  without  extensive  trials  with  real  students  and  even  if  this 
were  done  it  would  still  be  unlikely  that  any  real  conclusions  could  be  drawn  since  proving  the 
effectiveness  of  educational  systems  is  notoriously  difficult  in  the  educational  field  (Dillon  and 
Gabbard  1998).  The  purpose  of  the  TS  here  is  to  explore  the  technical  issues  relating  to  the  feasibility 
of  providing  an  automatic  student  grading  system.  Whether  this  facility  is  useful  is  open  to  educational 
debate.  However  it  is  likely  to  be  the  case  that  it  will  be  useful  should  the  correct  set-up  of  the  TS  be 
achieved  during  trials  with  real  students,  since  Bergeron  et  al  (1989)  found  their  TS  to  be  useful. 
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Further  issues  arise  concerning  the  adaptability  of  the  neural  network  used  for  the  TS.  The 
neural  network  architecture  used  by  Bergeron  et  al  (1989)  requires  off-line  training  and  is  therefore 
under  the  control  of  the  system  designer.  The  drawb  ack  with  this  approach  is  that  it  requires  the  manual 
intervention  of  a person  who  can  interpret  the  student  interaction  data  with  tutorials  and  determine 
whether  it  should  be  represented  to  the  neural  network.  The  advantage  of  the  Kohonen  neural  network 
architecture  is  that  it  is  able  to  train  continually  without  any  intervention  from  a human.  However,  there 
are  situations  where  this  adaptation  is  undesirable,  most  notably  when  different  skill  levels  of  students 
use  the  same  domain  at  different  times.  For  example,  if  a class  of  first  year  students  use  the  system 
Ky  2i  class  of  final  year  s^Jden^s.  However,  this  is  not  a problem  if  the  questions  and  tutorials 
have  been  adequately  assigned  a difficulty  level,  since  the  first  year  students  will  only  be  offered  easier 
tutorials  and  so  can  be  graded  only  as  lower  level  students  (although  they  can  still  progress  if  they 
continue  to  achieve  success  with  the  tutorial).  Problems  can  arise  only  if  both  the  student  abilities  and 
the  question  difficulties  are  unknown  beforehand.  This  is  because  the  TS  acts  as  a bi-directional 
mapping  device,  in  that  if  either  the  student  abilities  or  the  question  difficulties  are  known  beforehand 
then  the  TS  can  produce  the  unknown  parameter.  It  is  not,  however,  able  to  produce  \alues  when 
nothing  is  known  beforehand.  The  TS’s  ability  to  re-grade  questions  automatically  is  an  exploitation  of 
this  bi-directional  mapping  facility,  in  that  the  student  ability  can  be  changed  in  response  to  improving 
results  and  the  question  difficulty  can  be  changed  if  a significant  proportion  of  students  who  should  get 
the  question  right  in  fact  get  it  wrong. 

Research  into  the  TS  has  demonstrated  that  a fully  adaptable  system  for  automatically  grading 
students  is  possible  and  practical.  The  approach  of  using  an  automatic  tutorial  supervisor  has  been 
practically  justified  by  Bergeron  et  al  (1989).  However,  their  system  requires  manual  periodic 
retraining  which  renders  it  unsuitable  for  a generic  tutorial  system,  or  a tutorial  system  that  can  be  used 
without  the  need  for  reprogramming  or  otherwise  rearranging  the  program  code  of  the  system. 

Further  research  is  concerned  with  incorporating  the  TS  within  a hypermedia  tutoring  system 
(Mullier  et  al  2002,  Mullier  1999)  so  that  the  students*  interactions  with  the  TS  can  be  studied.  It  is 
anticipated  that  such  a study  will  prove  useful  for  determining  how  the  TS  reacts  to  different  domains, 
where  the  rules  that  describe  ability  are  different,  with  a view  to  reengineering  the  TS  so  that  it  is  able 
to  learn  such  a vast  set  of  rules  without  conflict.  A “superTS”  such  as  this  would  be  useful  in  a more 
generic  tutoring  environment  such  as  may  become  more  prevalent  on  the  WWW. 
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Abstract:  A variety  of  attempts  have  been  made  to  address  the  concerns  of  new  teachers  as  they 
enter  the  profession.  A meaningful  support  system  has  been  recommended  as  a key  component  to 
guiding  new  teachers  as  they  become  socialized  into  the  profession.  Professors  realize  the  gap 
between  theory  and  practice  is  never  perceived  as  wider  than  perhaps  during  this  period  of  a novice 
teacher's  career  life.  Two  professors  decided  to  explore  if  technology  could  be  used  to  enhance  their 
senior  seminar  programs  for  student  teachers.  The  project  designers  were  interested  designing  a 
format  where  student  teachers  could  participate  in  an  interactive  telementoring  experience.  A pilot 
program  was  implemented  to  determine  the  interest,  use,  and  benefits  of  an  interactive  website  for 
new  teachers. 


Introduction 

Numerous  studies  have  identified  the  most  critical  issues  that  cause  teachers  to  leave  teaching  before 
they  complete  three  years.  Concerns  particularly  associated  with  novice  teachers  include  those  of  classroom 
management,  overwhelming  job  responsibilities,  and  finding  neaningful  methods  of  support  (Nelson  & 
Low,  2000). 

A variety  of  attempts  have  been  made  to  address  these  concerns.  School  districts,  regional  service 
centers,  and  university  induction  programs  have  all  struggled  to  develop  programs  to  help  support  new 
teachers,  particularly  in  that  first  critical  year  of  teaching  (TEA,  2000).  Providing  some  type  of  a support 
system  have  been  recommended  by  many  as  a key  component  to  guiding  new  teachers  as  they  become 
socialized  into  the  profession  (Cole  & Knowles,  2000). 

Conceptual  Framework 

Support  systems  for  new  teachers  take  on  many  forms  and  designs.  Various  configurations  include 
individual  outside  mentors  assigned  to  new  teachers,  seminars,  and  other  opportunities  for  professional 
dialogue  and  development  (Cook-Sathcr,  2001;  Cole  & Knowles,  2000). 

Emergent  issues  resulting  from  related  research  indicates  that  perhaps  there  is  a vital  need  for  support  in 
areas  other  than  those  traditionally  focused  on:  teaching  skills  and  knowledge.  Successful  retention  of 
teachers  may  need  to  reach  beyond  the  first  year  and  attempt  to  help  teachers  develop  internal  resources 
necessary  to  pursue  a long  and  satisfying  teaching  career  (Cook-Sather,  2001). 

Designing  effective  student  teacher  experiences  and  induction  programs  are  of  primary  interest  to 
schools  of  education.  Professors  realize  the  gap  between  theory  and  practice  is  never  perceived  as  wider 
than  perhaps  during  this  period  of  a novice  teacher's  career  life  (Posner,  2000). 

Providing  structured  meetings  or  seminars  are  invaluable  ways  of  allowing  new  teachers  a way  to 
schedule  and  prepare  for  discussions  about  teaching.  However,  with  new  technologies  continually 
emerging,  there  may  be  additional  ways  in  which  professionals  can  connect  without  the  structure  of  brick 
and  mortar  (Harris  & Figg,  2000).  As  new  teachers  graduate  with  more  skills  in  technology  use  and 
awareness,  other  avenues  could  be  included  in  meaningful  teacher  induction  programs. 


The  Study 

Two  professors  decided  to  explore  if  technology  could  be  used  to  enhance  their  programs  for 
student  teachers.  The  project  designers  were  interested  designing  a format  where  student  teachers  could 
participate  in  a telementoring  experience.  A pilot  program  was  implemented  to  determine  the  interest,  use, 
and  benefits  of  an  interactive  website  for  new  teachers. 

Each  group  of  student  teachers  was  enrolled  in  predominately  rural  serving  settings.  Students  in  both  of 
these  programs  had  completed  a traditional  teacher  education  program.  Regionally,  both  groups  were 
located  in  similar,  rural  areas.  Participation  in  the  pilot  project  was  voluntary,  with  a total  of  twenty-three 
student  teachers  enrolled  on  the  website. 

The  site  includes  resources  for  teaching  strategies,  such  as  lesson  plans,  journals,  and  classroom 
management  ideas.  Additionally,  the  site  has  a link  to  an  interactive  component  where  participants  have 
access  to  a discussion  board,  email,  and  a virtual  chat  room.  The  purpose  of  the  interactive  component  is  to 
provide  student  teachers  a way  to  communicate  at  times  other  than  during  scheduled  seminar  opportunities. 
Participants  have  the  capability  to  share  experiences,  reflect  on  their  own  experiences,  and  have  other 
students  with  which  to  discuss  ideas  and  concerns. 

Implications 

By  incorporating  technology  into  a peer  mentoring  program,  students  could  have  the  advantage  of 
a “just  in  time”  format  for  addressing  their  needs.  If  it  appears  this  type  of  interactive  site  is  beneficial  to 
participants,  the  benefits  may  expand  to  others  outside  of  our  two  programs.  Introducing  a telementoring 
component  into  the  final  semester  of  student  teaching  may  be  one  way  to  encourage  new  professionals  in 
establishing  supportive  relationships  early  in  their  careers.  These  first  year  teachers  would  already  have 
established  a network  of  colleagues  to  continue  sharing  concerns,  issues,  and  challenges  during  this  critical 
point  in  their  careers. 

Conclusion 

Those  entering  the  teaching  profession  are  benefited  by  various  methods  of  mentoring  and  support 
as  they  navigate  the  routines  of  daily  classroom  life.  Emerging  technologies  can  be  integrated  into  existing 
mentoring  programs  as  a method  of  extending  support  systems.  Data  collected  from  student  teachers  who 
participated  in  a pilot  project  of  an  interactive  website  reported  benefits  in  both  reflective  practices  as  well 
as  valuable  dialogue  between  others  experiencing  similar  situations.  Of  the  twenty-three  eligible 
participants,  eight  students  were  classified  as  frequent  users  of  the  interactive  portion  of  the  site. 

The  authors  propose  that  teacher  retention  begins  even  before  teachers  step  into  their  first 
classroom.  If  support  programs  can  be  implemented  during  the  final  semesters  of  teacher  education 
programs,  it  may  be  possible  for  novice  teachers  to  build  support  systems  and  professional  development 
resources  early  in  their  careers.  Developing  resiliency  skills  through  telementoring  may  be  one  avenue  to 
enhance  first  year  experiences. 
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Abstract:  This  paper  describes  a research  project,  which  seeks  to  showcase  the  experience  of 
educators  with  technology -enhanced  teaching  and  learning.  A particular  focus  of  this 
investigation  is  on  how  the  use  of  information  and  communications  technology  is  influencing 
teaching  practices  and  students’  approaches  to  learning  at  the  University  of  Melbourne.  As 
such  it  comprises  a naturalistic  inquiry  into  the  experience  base  of  practitioners  who  have  been 
engaged  in  technology -enhanced  teaching  and  learning.  Our  goal  is  to  look  beyond  objective 
data  and  examine  closely  how  information  and  communications  technology  is  fundamentally 
influencing  the  nature  of  the  teaching  and  learning  transactions.  As  such  we  are  interested  in 
the  ’’untold"  stories  of  practitioners  and  participants  in  this  work.  Data  that  is  collected  is 
archived  on  a website,  and  used  in  a variety  of  ways  for  faculty  development. 


Aims  and  Outcomes 

Educational  institutions  all  around  the  world  are  beginning  to  pay  greater  attention  to  the  improvement 
of  their  teaching  and  learning  practices  with  the  innovative  use  of  information  and  communications  technology 
(ICT).  Although  a great  deal  of  work  has  gone  on  in  the  investigation  of  the  effects  of  computer-based  learning, 
there  is  a lack  of  reliable  knowledge  about  what  works,  why  and  in  what  waysl  This  paper  describes  a research 
project  that  is  trying  to  seek  for  some  answers  to  the  foregoing  questions  from  the  perspective  of  practitioner 
experiences.  The  goal  of  this  investigation  is  to  look  beyond  survey  data  derived  from  questionnaires  into  the 
experiences  of  practitioners,  to  ascertain  how  ICT  is  fundamentally  influencing  the  nature  of  the  teaching  and 
learning  processes  in  various  subject  matter  domains.  As  such  the  aim  is  to  tell  the  ’’untold"  stories  of 
practitioners  and  participants.  The  stories  we  are  collecting,  and  the  profiles  of  practice  that  we  are  developing 
will  comprise  the  data  for  the  development  of  conceptual  models  of  best  practice.  These  models  may  then  form 
the  subject  of  empirical  study  in  the  future. 

The  outcomes  of  this  research  are  expected  to  be  a deeper  level  understanding  of  how  the  use  of  ICT  is 
influencing  teaching  and  learning  in  fundamental  ways.  In  that  regard,  this  is  "exploratory"  research  as  it  seeks  to 
compile  stories  of  the  experiences  of  teachers  and  students  with  technology-enhanced  teaching  and  learning.  As 
an  immediate  output,  these  "stories"  or  "vignettes"  of  practitioners  will  be  used  to  build  a "gallery  of  stories"  on 
technology -enhanced  teaching  and  learning  that  will  be  accessible  on  a website  for  the  benefit  of  all,  but  most 
importantly,  for  novices.  Furthermore,  models  of  behavior  and  practice  derived  from  this  research  will  provide 
the  context  for  more  empirical  studies  in  the  future,  such  as  the  study  of  any  correlation  between  innovative 
teaching  and  learning  designs  and  specific  learning  outcomes,  and/or  approaches  to  learning  and  teaching. 


Context  and  Scope 

The  application  of  ICT  in  teaching  and  learning  has  the  potential  to  change  educational  practices  in 
significant  ways  (Ben-Jacob,  Levin,  & Ben-Jacob,  2000;  Rogers,  2000).  For  example,  the  integration  of  e-mail 
and  computer  conferencing  with  multimedia  databases  and  electronic  libraries  has  enabled  the  emergence  of  a 
whole  new  kind  of  educational  activity,  namely  eLearning.  Information  and  communications  technology  is  also 
enabling  established  campus-based  providers  to  rethink  and  re-engineer  the  nature  of  their  teaching  and  learning 
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practices.  The  University  of  Melbourne,  like  many  other  educational  institutions,  is  currently  involved  in  just 
such  a process  as  part  of  a strategy  to  position  the  University  as  a global  player  in  higher  education.  As  a direct 
result  of  this  and  along  with  the  adoption  of  ICT,  innovative  approaches  to  teaching  and  learning  such  as 
problem-based  learning  and  collaborative  learning  are  being  encouraged.  These  initiatives  have  led  to  the  rise  of 
new  roles  for  teachers  such  as  "facilitators  of  learning"  as  opposed  to  "deliverers  of  content"  (Evensen  & Hmelo, 
2000;  de  Vemeil  & Berge,  2000;  Salmon,  2000).  They  have  also  exposed  students  to  new  models  and  approaches 
to  learning  such  as  "computer  supported  collaborative  learning"  (Dillenbourg,  1999;  Koschmann,  1996; 
O'Malley,  1995),  and  "computer  supported  problem  based  learning"  (Bernard,  Rojo  de  Rubalcava,  & St-Pierre, 
2000;  Koschmann,  Kelson,  Feltovich,  & Barrows,  1996). 

While  interest  is  growing  in  the  integration  of  technology  in  learning  and  teaching,  there  is  still  very 
little  known  about  how  the  use  of  ICT  is  changing  teachers  approaches  to  teaching,  and  students*  approaches  to 
learning  (Rumble,  2000).  The  need  to  investigate  what  is  happening  with  technology -enhanced  teaching  and 
learning  is  now  imperative.  This  includes,  among  other  things,  understanding  how  approaches  to  teaching  are 
being  impacted,  how  teacher-thought  about  teaching  and  learning  is  being  modified,  how  students'  approaches  to 
learning  are  changing,  and  how  student  support  is  changing  with  the  use  of  ICT? 


Methodology 

The  research  described  in  this  paper  set  out  to  seek  some  answers  to  the  foregoing  questions  from  the 
practitioners’  perspective.  It  comprises  a naturalistic  inquiry  into  the  modus  operandi  of  educators  (Lincoln  & 
Cuba,  1985).  Naturalistic  inquiry  is  particularly  suited  to  questions  and  settings  such  as  this  where  the  context  is 
heavily  implicated  on  meaning.  Such  a contextual  inquiry  demands  the  use  of  human  instruments  for  gathering 
data,  s humans  have  the  capacity  to  use  their  tacit  knowledge  with  qualitative  data  gathering  tools  such  as 
interviews y direct  observations^  self-reporting  and  think -aloud,  and  document  analysis.  Once  in  the  field,  the 
inquiry  takes  the  form  of  successive  iterations  of  these  elements:  purposive  sampling,  inductive  analysis  of  the 
data,  development  of  grounded  theory  based  on  the  inductive  analysis,  and  projection  of  next  steps  in  a constantly 
emergent  design  (Lincoln  & Cuba,  1985;  p.  187).  Throughout  the  inquiry,  and  especially  at  the  end,  the  data  and 
interpretations  are  continuously  checked  with  respondents,  and  differences  of  opinion  are  negotiated  until  the 
outcomes  are  agreed  upon  or  differences  of  opinion  are  understood  and  reflected  as  such.  This  information  is  then 
used  to  develop  a case  report  or  profile,  which  is  tested  for  "credibility"  and  "confirmability"  (Lincoln  & Cuba, 
1985;  p.  189).  This  testing  begins  early  in  the  study  and  continues  throughout,  culminating  in  a final 
representation  of  what  was  observed. 


Data  Gathering 

Interviews  are  being  used  as  the  principal  instrument  for  data  gathering  in  the  first  instance.  The  initial 
sample  comprises  practitioners  who  are  known  for  spearheading  the  use  of  ICT  into  teaching  and  learning  at  The 
University  of  Melbourne.  This  sample  will  grow  to  include  other  practitioners  at  the  University  of  Melbourne. 
There  is  a possibility  that  this  may  grow  even  further  to  include  practitioners  from  other  organizations  including 
tertiary  educational  institutions,  and  also  commercial  enterprises.  In  the  spirit  of  naturalistic  inquiry,  interviews 
are  being  conducted  on  location.  An  interview  protocol  has  been  developed  which  sets  out  the  goals  of  the 
interview  and  trigger  questions  to  guide  the  interview  (see  boxed  text  below). 


Teaching  and  Learning  Experiences  with  Educational  Technology 

We  are  interested your  story  andyowr  experience  with  technology-enhanced  teaching  and  learning.  We  would 
therefore  like  you  to  reflect  on  your  experience  with  technology-enhanced  teaching  in  terms  of  the  following: 

Planning 

• What  are  your  goals  and  motivations? 

• What  aspects  of  your  teaching  and  learning  are  you  trying  to  influence  (e.g.,  approaches  to  subject  matter 
content  representation,  activation  of  learning,  assessment,  socialization,  or  provision  of  feedback)? 

• Why  are  they  important,  to  whom  and  to  what? 
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• Describe  your  approach  to  learning  and  teaching? 

• What  is  unique  or  innovative  about  this  approach? 

• What  limitations  of  theoretical  perspective  have  you  encountered? 

• What  unique  challenges  have  you  faced  in  planning  your  approach? 

• How  did  you  know  if  you  were  on  the  right  track? 
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What  you  are  doing? 

fxGVV  did  you  gc  about  choosing  the  tools  und 
What  influenced  your  choice  of  these  tools  and  technologies? 

• What  challenges  did  you  face  in  selecting  these  tools  and  technologies? 

• What  limitations  did  you  experience;  financial,  technical  or  organizational? 


What  happened? 

• What  worked  and  what  didn’t  work? 

• What  criteria  did  you  use  for  evaluating  the  effectiveness  of  your  progress? 

• How  did  you  make  use  of  the  data  that  was  gathered? 

What  you  learned? 

• How  did  this  innovation  influence  your  view  of  teaching  and  learning? 

• How  did  it  influence  your  students*  approaches  to  studying  and  learning? 

• Did  it  impact  your  understanding  of  your  students*  studying  and  learning? 

• What  are  your  successes,  failures,  serendipitous  findings,  lessons  learned? 

• What  would  you  do  differently  next  time? 


Interviews  routinely  begin  with  a discussion  of  this  interview  protocol  and  the  trigger  questions.  This  is 
to  ensure  that  interviewees  understand  the  questions,  and  are  comfortable  with  the  their  motives.  With  the 
permission  of  the  interviewees  interviews  are  audio  taped  and  subsequently  transcribed.  These  transcripts 
comprise  the  raw  data  for  the  development  of  profiles  of  practice.  The  interview  protocol  follows  the  action 
research  methodology,  which  comprises  planning,  doing/taking  action,  observing  and  reflecting. 

Developing  Profiles  of  Practice 

Excerpts  derived  from  the  interview  transcripts  comprise  the  raw  data.  Researchers  sift  through  these 
transcripts  to  develop  individual  profiles  of  practice  along  the  lines  of  the  interview  protocol.  These  are  then 
presented  to  each  interviewee  to  allow  the  filling  of  gaps  in  the  profiles,  verification  of  existing  materials  and 
addition  of  any  other  thoughts  on  the  matters  raised  during  the  interview.  This  in  itself  is  an  iterative  process  and 
involves  further  consultations  with  interviewees.  The  profiles  are  entered  onto  the  database  only  when  complete 
agreement  has  been  reached  between  the  interviewee  and  the  researchers  on  the  content  of  the  profiles. 

Development  of  the  Database 

The  database  is  used  to  generate  profiles  of  practice  for  a website  that  is  available  to  all  University  of 
Melbourne  academics.  Data  is  entered  using  a simple  web-based  form  to  populate  the  following  fields: 


Planning 

Doing 

Observing 

Reflecting 

Summary 

Summary 

Summary 

Summary 

Goals  & motivations 

Choice  of  tools 

What  worked 

Impacts  on  your  view  of 
teaching  & learning 

Approach  to  learning 

Influences  on  choice  of 
tools 

What  did  not  work 

Impacts  on  your  student’s 
approach  to  studying 

Limitations  of  the 
approach 

Challenges  in  selecting 
tools 

What  criteria  for  success 
were  used 

What  did  you  learn 

Challenges 

Limitations  experienced 

How  those  criteria  were 
measured 

What  you  would  do 
differently  next  time 
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Data  is  also  included  for  standard  project  details  such  as  the  names  and  affiliations  of  faculty  and 
project  leaders,  date  of  implementation,  and  project  type.  Each  profile  also  contains  a brief  summary  of  the 
project  and  current  issues  in  the  discipline.  There  is  no  requirement  that  all  fields  contain  data. 

Where  appropriate,  data  can  be  provided  in  formats  other  than  text.  The  database  is  used  to  generate  a 
showcase  of  academic  practice.  Users  of  the  site  can  customize  the  display  of  stories  by  focusing  on  a particular 
action  research  process,  group  of  trigger  questions,  faculty  or  the  complete  profile  of  a particular  project. 

Display  options  include  a choice  of  predefined  categories,  as  well  as  browseable  lists  and  individually 
con.structed  .search e.s.  Browseable  lists  allow  the  display  of  projects  by  faculty  and  department,  as  well  as  by 
action  research  process,  including  approaching  the  data  from  varying  perspectives.  Search  functions  provide  for 
user-constructed  views  of  the  data  that  enable  investigation  based  on  specific  interests.  Keyword  searching  is 
also  available  across  all  fields  of  the  database.  Each  story  segment  is  displayed  with  fully  linked  metadata  for  the 
corresponding  project.  This  provides  a link  to  departmental  databases  containing  other  project  information,  and 
also  to  academics  involved  with  the  project.  Links  are  provided  to  live  course  content  where  possible. 

Sample  Profile  (Abbreviated) 


Project  Leader:  Tim  Van  Gelder 

F acuity/ Departm  en  t:  A rts/Ph  ilosophy 

Summary  of  Project 

Reason! Able  is  a stand-alone  PC  package  designed  to  assist  students  at  all  levels,  including  those  with  no  explicit 
training  in  logic  or  argument,  to  acquire  general  informal  reasoning  skills.  We  looked  at  research  about  critical 
thinking  courses  and  it  showed  that  they  just  weren’t  having  the  effect  that  they  were  claiming  to  have  had,  and 
were  perhaps  actually  hurting  students*  critical  thinking. 

Planning:  Summary  Comment 

We  wanted  to  help  students  to  learn  to  acquire  general  and  fundamental  skills  of  reasoning  and  argument.  We 
teach  general  skills  that  can  be  applied  in  any  domain  whatsoever  e.g.  in  students’  other  academic  subjects  and  in 
their  chosen  profession. 

Planning  - Goals  and  motivations 

We  wanted  the  students  to  go  through  a certain  fairly  standard,  straightforward  routine.  We  had  initially  used  a 
HyperCard  stack,  but  wanted  to  come  up  with  a dynamic  forming  type  of  software  tool,  that  would 
simultaneously  teach  the  students  all  the  concepts  and  procedures  that  they  needed  to  know. 

Planning  - Approach  to  learning 

1 was  fundamentally  concerned  with  the  problem  that  the  students  did  not  learn.  They  were  trying  to  learn,  but 
not  succeeding,  and  not  reflecting  on  the  fact  that  they  weren’t  succeeding.  If  you  want  to  acquire  skills,  you’ve 
got  to  practice.  It’s  true  for  cognitive  skills  just  as  much  as  practical  ones.  But  it’s  not  just  any  old  practice;  it’s 
got  to  be  the  right  kind  of  appropriately  guided,  scaffolded,  and  motivating  practice. 

Planning  - challenges 

The  idea  that  targeted  practice  will  lead  to  an  improvement  in  skills  is  the  one  thing  that  has  been  absolutely 
constant.  What  we  were  trying  to  create  with  the  Reason! Able  software  is  what  I have  come  to  call  an 
‘environment  tool’  for  quality  practice  in  reasoning. 

Doing  — Summary  Comment 

Teaching  is  still  a cottage  industry  where  a whole  lot  of  people  are  just  assumed  to  be  able  to  do  it  and  they  go 
by  unquestioned.  There  hasn’t  been  the  pressure  to  force  change. 

Doing  - Choice  of  tools 

We  thought  that  maybe  there  were  more  ways  of  representing  complex  structures  of  reasoning,  which  take 
advantage  of  representational  resources,  which  for  practical  reasons  couldn’t  be  used  very  effectively.  In 
ReasonlAble  you’ll  see  an  almost  complete  switch  from  a HyperCard  approach  to  an  all-in-one  workspace.  All 
the  information  is  at  all  times  is  available  on  the  screen. 

Doing:  Challenges  in  selecting  tools 
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Despite  difficulties,  it  was  the  first  time  ever  that  somebody  had  built  a way  of  handling  argumentation  that  made 
it  visual,  manipulable,  and  graphical.  It’s  a real  turning  point. 

Doing  - Limitations  experienced 

In  a certain  sense,  we  failed  in  our  first  attempts,  because  we  found  out  that  the  task  was  a lot  more  difficult  and 
challenging.  I don’t  think  it  was  a success  from  the  point  of  what  we  said  we  would  do  at  the  beginning  and  what 
we  delivered  at  the  end.  Although,  I think  that  we’ve  succeeded  at  the  end  of  three  attempts. 

Observing  - Summary  Comment 

We  are  dealing  with  a huge  spectrum  of  intuitiveness  and  familiarity.  It  is  very  interesting,  that  many  university 
students  are  having  difficulty  with  critical  thinking  skills  and  yet,  elementary  school  students  are  coping  with  it. 

Observing  - What  did  and  did  not  work 

Students  have  been  more  successful  in  learning  how  to  think  critically  because  of  these  visual  elements.  It  is  the 
visual  element  that  is  having  the  most  impacts. 

Observing  - What  criteria  for  success  were  used? 

So  far,  our  students  gain  three  to  four  times  as  much  as  any  other  students  in  the  world,  in  terms  of  critical 
thinking  skills.  The  studies  showed  that  this  technology  helps  people  learn  critical  thinking  skills. 

Observing  - How  those  criteria  were  measured 

Last  semester,  we  did  a number  of  studies,  comparing  the  results  from  the  University  of  Melbourne,  with  that 
from  Monash  University  in  Melbourne  and  McMaster  University  in  Canada.  Their  results  were  only  half  as  good 
as  ours.  There  was  something  about  our  approach,  which  was  working  much  better. 

Reflecting  - Summary  Comment 

It  has  changed  me  a lot,  there’s  no  question  about  that.  We  don’t  think  hard  enough  about  effectiveness  and 
quality.  We  inherit  a framework  of  practices  and  assumptions  and  we  work  within  that  so  that  what  everybody 
else  accepts  is  deemed  fair  and  reasonable  practice  is  what  obviously  I would  accept  as  reasonable  practice. 

Reflecting  - Impact  on  your  view  of  teaching  and  learning 

The  software  is  tapping  into  a much  larger  set  of  brain  resources  than  the  typical  ways  of  presenting  arguments. 
We  are  getting  students  to  use  more  of  their  brain.  The  software  is  partly  visual,  partly  manipulable  and  that 
helps  them  and  it  makes  life  easier  for  them.  It  reduces  the  cognitive  burden. 

Reflecting  - Impact  on  your  students  ’ studying  and  learning 

The  students  are  being  affected  far  more  than  they  realize.  At  a deeper  level,  they  are  being  exposed  for  the  first 
time  to  what  it  is  to  be  critical  and  what  it  is  to  have  a rational  opinion.  They’re  getting  an  understanding  of  what 
it  takes,  how  complex  the  world  is,  what  the  issues  are  and  how  much  work  is  involved  in  actually  thinking 
through  an  issue. 

Reflecting  - What  you  learnt 

If  the  primaiy  role  of  this  course  is  to  improve  students’  critical  thinking  skills,  then  yes,  I think  I have  become 
better  at  it.  I am  using  better  methods,  better  tools,  and  getting  better  results.  I have  also  learned  that  if  you’re 
engaged  in  a challenging  project,  you  have  got  to  expect  that  it  may  not  pan  out  the  way  you  think  it  will.  You 
have  got  to  be  prepared  to  change  your  direction,  re -conceive  the  project,  and  keep  pushing  the  boundaries. 


Concluding  Remarks 

The  work  that  is  described  and  discussed  in  this  paper  grew  out  of  a growing  call  for  "evidence  of  the 
impacts"  of  information  and  communications  technology  in  tertiary  teaching  and  learning.  While  this  is  a 
question  that  has  been  asked  many  times  before,  answers  to  it  have  not  been  conclusive  one  way  or  the  other. 
There  is  a lot  of  evidence  to  suggest  that  the  use  of  information  and  communications  technology  in  tertiary 
teaching  and  learning  has  many  advantages.  There  are  also  suggestions  that  these  benefits  do  not  justify  the  cost. 
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time  and  effort  that  this  kind  of  work  entails.  Many  of  these  findings  are  however,  problematic  as  they  are  based 
on  neither  reliable  nor  valid  research  techniques.  The  work  that  is  reported  in  this  paper  incorporates 
investigation  techniques  that  depart  from  the  commonly  used  approaches  to  the  quantification  of  user 
perceptions  with  questionnaires  and  surveys  of  sorts. 

Our  goal  in  this  work  is  to  capture  the  experience  base  of  practitioners  with  the  use  of  a range  of  data 
gathering  techniques  that  are  grounded  in  the  principles  of  naturalist  inquiry.  We  realize  that  data  derived  from 
these  kinds  of  approaches  are  not  easily  "generalizable”  to  other  contexts.  Among  other  things,  generalizability  is 
a function  of  sampling  and  we  expect  that  over  time,  we  will  have  in  this  gallery,  the  amount  of  information  and 
data  that  is  necessary  to  make  meaningful  generalizations  from  it  to  similar  situations  and  contexts.  We 
anticipate  that  this  gallery  will  grow  into  an  extremely  rich  resource  of  the  experience  base  of  not  only  many  of 
our  pioneering  efforts  but  some  of  the  most  innovative  work  that  is  being  undertaken  in  this  regard  at  the 
University  of  Melbourne  and  elsewhere.  A larger  collection  of  profiles  in  the  database  would  enable  the 
examination  of  patterns  and  models  of  behavior  among  practitioners  that  could  become  the  subject  of  study 
further  down  the  road.  Questions  that  might  seem  relevant  would  include:  the  reasons  for  particular  approaches 
to  teaching,  and  prominent  approaches  to  student  study  behavior.  We  are  hopeful  that  these  questions  would 
make  interesting  study  towards  explaining  how  the  use  of  technology  is  impacting  teaching  and  learning. 
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Abstract:  The  desire  to  learn  and  teach  in  an  online  setting  often  presents  great  difficulty  for  students  and  instructors  due  largely  to  the 
lack  of  face-to-face  interaction.  Online  instructors  are  continually  looking  for  methods  to  improve  student  engagement  without 
monopolizing  instructional  time.  Students  desire  feedback  on  their  progress  in  a non-test  setting.  Using  JavaScript  and  html  to  develop 
client-side  online  testing  fulfills  both  of  these  goals  and  provides  instructors  and  students  with  better  tools  for  successful  online  education. 

Introduction 

One  of  the  most  significant  challenges  experienced  by  instructors  of  online  courses  is  keeping  students  engaged  and 
confirming  that  students  are  making  satisfactory  progress  throughout  the  semester.  The  lack  of  face-to-face  interaction 
makes  it  difficult  for  the  instructor  to  determine  which  students  are  fully  participating  and  comprehending  the  materials,  and 
which  topics  are  causing  particular  difficulty.  Minimal  interaction  between  the  instructor  and  an  individual  online  student 
could  mean  that  the  student  has  grasped  all  concepts  and  needs  no  further  assistance,  the  student  is  inactive  and  will  remain 
so  until  a mandatory  deadline  occurs,  the  student  is  unaware  that  he  or  she  is  not  grasping  the  course  material,  or  that  the 
student  is  overwhelmed  by  the  material  and  needs  help  to  succeed. 

In  addition,  the  lack  of  immediate  feedback  from  the  instructor  often  leaves  students  in  a quandary  with  respect  to  their 
current  progress  in  the  course.  Frequently  students  receive  minimal  feedback  in  an  online  setting  until  an  examination  is 
graded,  at  which  time  it  is  too  late  to  take  remedial  action.  While  these  problems  can  also  exist  in  the  traditional  face-to-face 
classroom,  the  misperceptions  caused  by  the  lack  of  mandatory  meetings  between  the  students  and  the  instructor  is  magnified 
in  the  virtual  classroom.  To  combat  this,  scheduled  intermediate  quizzes,  perhaps  weekly,  can  be  created  which  measure  the 
students’  progress  in  a non-threatening  environment,  and  let  the  students  know  which  concepts  the  instructor  feels  should  be 
emphasized  to  better  meet  the  educational  goals  of  the  course.  The  benefits  of  the  quizzes  for  the  instructor  and  student  are 
many. 

1.  Students  are  engaged  weekly  in  verifiable  activities. 

2.  Instructors  can  identify  problem  areas  where  more  examples  or  clarification  are  required. 

3.  The  quiz  process  is  asynchronous  so  students  can  schedule  time  appropriately. 

4.  Students  can  measure  their  progress  against  a course  standard. 

5.  Each  student  can  take  the  individual  time  required  to  complete  the  quiz  without  fear  of  being  judged. 

It  is  certainly  possible  to  meet  these  goals  by  merely  posting  a series  of  quiz  questions  for  the  students  to  answer,  and  then 
manually  grading  the  responses.  However,  the  use  of  automated  grading  on  the  client-side  reduces  the  workload  for  the 
instructor  while  allowing  the  students  to  receive  immediate  feedback  on  their  quiz.  By  providing  results  in  this  manner,  the 
students  have  the  opportunity  to  further  research  the  questions  they  answered  incorrectly  while  the  topics  are  fresh  in  their 
minds. 

Online  Testing 

In  order  to  solve  the  problem  efficiently  and  effectively  in  an  online  setting,  content-specific  quiz  questions  were  selected  and 
put  in  a standard  format,  an  html  version  of  the  quiz  was  produced,  a JavaScript  program  to  grade  the  quiz  was  produced.  To 
optimize  the  above,  the  system  was  prototyped  once,  and  then  automated  to  save  the  instructor  time  and  minimize  any 
associated  errors.  In  this  case  Microsoft  Excel  was  used  to  store  the  content-specific  quiz  questions. 

Question  Selection 

Although  a variety  of  formats  for  creating  online  quizzes  exist,  the  system  implemented  includes  html.  Visual  Basic  and 
JavaScript.  Prototypes  were  developed  and  then  Visual  Basic  was  used  to  generate  future  quizzes.  The  format  for  the 
example  questions  is  multiple -choice,  although  any  method  that  can  be  graded  algorithmically  is  theoretically  programmable. 
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Html  Prototype 


Prototype  quizzes  were  created  for  online  graduate  courses.  There  are  three  buttons  at  the  end  of  the  quiz  that  provide 
students  with  the  options  to  Check  Answers ^ Submit  Solutions^  and  Clear  Quiz.  The  three  options  provide  students  with  the 
capability  to  check  their  answers  in  a non -threatening  setting.  They  can  check  their  answers  as  many  times  as  they  like  prior 
to  submitting  their  answers.  The  amount  of  information  provided  in  response  to  selecting  the  Check  Answers  button  is 
controllable  by  the  instructor,  and  can  range  from  merely  reporting  the  number  of  correct  answers  to  detailed  information 
about  each  question  that  was  answered  incorrectly. 

JavaScript 

The  immediate  feedback  to  the  student  is  provided  through  JavaScript.  The  easiest  method  of  using  JavaScript  on  a web 
page  is  to  embed  the  code  in  the  html  source.  However,  this  allows  anyone  to  view  the  JavaScript,  which  in  this  case  will 
include  the  quiz  answers,  simply  by  viewing  the  page's  html  source.  Since  allowing  the  user  to  view  the  web  page  source  is  a 
basic  function  supported  by  all  popular  browsers,  embedded  JavaScript  is  not  particularly  effective  for  online  quizzes.  To 
ensure  that  the  quiz  answers  are  not  trivially  accessible,  the  JavaScript  can  be  stored  in  a separate  file  on  the  server,  and  that 
file  is  then  referenced  from  the  html  document.  Anyone  viewing  the  html  source  in  this  case  will  no  longer  have  immediate 
access  to  the  JavaScript  code.  Unfortunately  this  does  not  completely  solve  the  problem.  The  file  in  which  the  JavaScript  is 
now  stored  is  likely  saved  in  the  browser's  local  cache  directory,  and  can  be  opened  by  the  student  using  any  text  editor, 
revealing  the  JavaScript  source.  However,  it  is  likely  that  storing  the  JavaScript  in  a separate  file  is  a sufficiently  secure 
solution  for  most  quizzes,  while  still  providing  the  benefits  of  a client-side  controlled  interactive  page.  In  the  event  that  a 
more  secure  system  is  required  the  interactivity  could  be  controlled  on  the  server-side  using  CGI  programming,  which  would 
also  require  a greater  level  of  client-server  communication. 

Quiz  Generation 

The  extensive  ability  inherent  in  current  technologies  to  automate  redundant  tasks  can  greatly  simplify  the  quiz  construction 
process.  The  html  for  a collection  of  quizzes  varies  only  in  the  quiz  question  contents,  and  any  header  information  that  the 
quiz  creator  chooses  to  include  for  supplementary  information.  The  JavaScript  varies  in  the  construction  of  the  answer  array 
as  a list  of  correct  solutions.  Thus  for  the  collection  of  quizzes  associated  with  a particular  course,  the  html  and  JavaScript 
files  should  be  identical  except  in  the  references  between  the  files,  question  contents,  and  header  information.  The  references 
between  the  files  include  the  items  necessary  to  correctly  name  the  files  and  to  associate  the  correct  JavaScript  file  and  path 
with  the  correct  html  file  and  path.  The  question  contents  include  the  textual  questions  and  associated  solutions.  The  header 
information  includes  quiz-specific  information  such  as  quiz  number,  course,  etc.  With  so  much  similarity  between  these 
items,  it  certainly  is  possible  to  create  one  html  and  JavaScript  prototype  and  modify  each  to  create  subsequent  quizzes.  With 
the  ease  of  automation,  this  process  can  be  greatly  simplified  with  the  possibility  of  associated  error  being  minimized  as  a 
side  effect.  Perhaps  one  of  the  simplest  methods  is  to  put  all  of  the  content  questions  and  associated  solutions  in  a file,  and 
then  use  Visual  Basic  to  generate  the  associated  html  and  JavaScript  files.  In  fact,  that  is  how  the  files  in  the  figures  above 
were  generated,  with  minimal  effort  on  the  part  of  the  quiz  generator. 

Conclusions 

The  client-side  quizzing  has  been  tried  in  several  courses  with  very  positive  results.  At  first  consideration,  many  students 
have  difficulty  immediately  seeing  that  they  can  determine  the  correct  answers  to  all  questions  by  checking  answers  for  each 
question  individually.  Beginning  students,  and  those  with  minimal  experience  in  the  area  of  combinatorics  change  multiple 
answers  at  once,  thereby  greatly  complicating  the  process  of  determining  which  answers  are  correct.  The  option  to  submit 
the  answers  has  met  with  mixed  results.  It  requires  the  students  to  remain  connected  to  the  server  for  submission  and  also 
increases  the  possibility  of  submitting  answers  before  the  student  is  ready  through  accidental  means.  Using  client-side 
quizzes  created  by  combining  html  and  JavaScript  provides  students  with  the  capability  to  measure  their  progress  each  week. 
The  instructor  can  also  choose  to  verify  the  results  depending  on  the  needs  of  the  particular  virtual  community.  This  can 
guide  the  instructor  in  developing  content  focus  areas,  and  can  provide  a valuable  means  to  ensure  that  students  are  engaged 
throughout  the  course. 
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This  case  study  reviews  a new  model  that  we  have  developed  for  our  introductory  economics  courses  that 
focuses  on  the  use  of  technology  to  engage  students  in  active  learning  strategies.  This  project  is  the  result  of 
a three-year  grant  awarded  to  the  Economics  and  Mathematics  departments  at  Fairfield  University  by  the 
Davis  Educational  Foundation.  Under  the  grant,  each  department  has  restructured  the  way  it  teaches 
introductory  courses.  The  objective  has  been  to  use  technology  to  improve  quality  and  lower  the  costs  of 
delivering  high-quality  learning  opportunities  to  students.  In  this  paper  we  discuss  the  economics  portion 
of  the  project. 

Our  contention  is  that  students  can  learn  about  economics,  but  to  understand  and  use  the  theoretical  models 
and  quantitative  techniques  that  we  teach  them,  they  must  also  do  economics.  Economics  texts  often  say 
that  students  need  to  be  able  to  understand  the  "economic  way  of  thinking";  we  argue  that  they  also  need  to 
experience  the  "economic  way  of  doing",  to  pose  interesting  questions,  formulate  hypotheses,  collect  data, 
analyze  the  data,  and  generate  conclusions  regarding  the  results  of  their  experiments.  In  order  to  make  this 
environment  both  active  and  effective  in  improving  student  learning  outcomes,  students  need  to  be  engaged 
in  the  process.  Our  solution  is  to  infuse  technology  into  the  activities  of  the  course  so  that  it  becomes  the 
common  denominator  around  which  the  course  in  constructed.  Though  our  colleagues  in  the  natural 
sciences  have  had  students  working  in  labs  as  active  learning  environments  for  many  years,  social  scientists 
have  not  had  easy  access  to  similar  settings  for  students.  Computer  time  was  expensive,  and  data  files 
were  often  difficult  or  expensive  for  students  to  retrieve.  The  recent  explosive  growth  in  computer 
technology  now  makes  it  possible  for  economists  to  get  students  into  a meaningful  leaming-by-doing 
environment  in  a cost  effective  way. 

Introductory  economics  classes  at  our  school  have  traditionally  been  taught  in  a 3x50  minutes  or  2x75 
minutes  per  week  lecture  format.  Since  the  demand  for  introductory  economics  is  great  (the  courses  are 
required  for  our  School  of  Business  students),  faculty  members  in  economics  often  teach  two  sections  of 
“intro”  each  semester.  To  give  our  students  the  opportunity  to  do  economics  we  restructured  our  sections 
of  introductory  economics  in  the  following  way:  Two  sections  were  combined  into  one  large  (60+)  student 
lecture.  The  class  meets  as  a “large”  lecture  twice  a week  and  in  smaller  groups  (20-22)  once  a week  in  a 
computer  classroom.  There  are  two  practical  advantages  to  this  restructuring.  The  first  is  that  students  are 
able  to  use  the  technology  within  the  format  of  a three-credit  course.  No  extra  lab  meetings  or  outside  class 
assignments  are  needed  to  “add  on”  the  technology.  Rather,  the  technology  has  become  an  integral  part  of 
the  class.  Second,  the  instructor  saves  one  class  meeting  per  week.  Typically,  the  two  sections  would  have 
six  50-minute  meetings  per  week;  with  the  new  model,  there  are  two  50-minute  lectures  and  three  50- 
minute  labs. 

Outside  of  class,  we  have  two  important  assignments  that  engage  students  with  course  material  using 
technology.  Our  department  has  used  spreadsheet  exercises  in  introductory -level  courses  for  many  years. 
These  spreadsheet  exercises,  written  by  department  members,  guide  students  through  introductory  and  then 
more  complex  spreadsheet  operations  in  the  context  of  economic  problen>solving  exercises.  Students 
learn  to  use  technology  as  they  would  paper,  pencil,  or  calculator,  as  a tool  to  aid  in  their  understanding  of 
sometimes  complex  economic  concepts.  As  students  move  into  upper-division  economics  courses, 
additional  spreadsheet  concepts  (like  regression  analysis)  are  added,  and  they  learn  other  statistical 
software  and  simulation  packages  (like  E-Views  and  the  Fair  model).  We  believe  that  this  has  become  a 
distinctive  component  of  our  departmental  curriculum;  technology  is  infused  into  the  learning  process  from 
the  introductory  level  on  so  students  are  able  to  engage  actively  in  the  learning  process. 
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To  further  enhance  student  engagement  with  technology  outside  of  class,  students  in  our  sections 
participate  in  online  threaded  discussions  on  issues  of  their  choosing  related  to  the  course.  This  allows 
them  to  continue  to  develop  discussions  that  started  in  class  but  that  were  cut  short  as  the  class  period 
ended.  It  also  encourages  them  to  try  out  new  terms  and  ideas  that  they  are  applying  to  the  economic  world 
around  them.  The  discussion  board  encourages  students  to  experiment  with  new  economic  ideas  and 
applications  in  a non-threatening  environment. 

There  have  been  several  challenges,  particularly  during  the  pilot  phase  of  the  project.  Our  computer 
classroom  facilities  have  a high  level  of  utilization  and  there  was  no  guarantees  that  we  would  receive 
priority.  Our  restructured  format  has  also  required  that  students  who  enroll  in  our  sections  have  two  free 
time  codes  in  their  schedule  - one  for  the  lecture  and  one  for  the  lab.  This  has  perhaps  deterred  students 
from  enrolling  in  our  sections.  We  have  also  experienced  our  share  of  technology-related  "glitches",  which 
frustrate  and  discourage  students  and  instructors  alike. 

Also,  the  students  themselves  were  not  always  as  receptive  to  the  technology  and  hands-on  learning 
environment  as  we  had  hoped  they  would  be.  Acquiring  some  “ownership”  of  the  learning  environment 
was  not  a concept  that  they  universally  desired.  We  need  to  do  a better  job  of  refining  our  lab  materials  so 
that  the  students  are  more  engaged  and  eager  to  complete  them.  To  do  this,  we  have  developed  exercises 
that  must  be  completed  outside  class  and  we  have  provided  "homework"  credit.  This  form  of  reward  often 
works  with  our  students,  who  like  to  know  what  the  "payoff’  is  to  their  effort  and  attention. 

In  terms  of  the  course  material  itself,  "losing"  a third  of  our  lecture  time  presented  a challenge.  During  year 
one  of  the  grant  we  wondered  how  we  could  cover  the  same  amount  of  material  and  still  provide  students 
with  the  hands-on  experience  we  so  valued.  What  we  learned  is  that  some  concepts  are  covered  better  in 
the  lab  and  we  were  able  to  eliminate  or  greatly  reduce  discussion  of  them  in  the  lecture.  Experiential 
learning  is  a much  more  efficient  way  to  deliver  some  course  concepts,  we  just  need  to  understand  which 
concepts  these  are  and  develop  lab  materials  that  encourage  students  to  take  more  responsibility  for  their 
own  learning.  We  realized  that  we  were  initially  thinking  of  the  lab  as  a place  to  work  on  material  that  we 
had  already  lectured  over;  what  we  realized  was  that  students  did  not  need  to  be  "briefed"  before  they 
entered  the  lab.  The  lab  exercises  could  be  structured  as  self-contained  units,  presenting  students  with 
entirely  new  concepts  and  applications. 

As  yet,  assessment  results  regarding  improvements  in  student  learning  are  inconclusive.  We  are 
administering  pretests  in  introductory  micro  in  the  fall  semester,  the  same  multiple -choice  questions  will 
appear  on  course  final  exams.  We  hope  that  we  can  use  the  results  of  these  tests  to  gain  insights  into  how 
student-thinking  processes  are  affected  by  our  technology-enhanced  approach  to  the  courses  relative  to 
more  standard  formats.  We  are  using  the  Flashlight  cost  assessment  model,  which  has  allowed  us  to  track 
resources  allocated  to  this  project.  We  have  concluded  that  extensive  use  of  technology  for  teaching  is  a 
time-intensive  activity. 

In  terms  of  student  satisfaction,  results  have  been  mixe  d.  Some  students  embrace  the  technology  and  enjoy 
working  in  the  lab  environment.  Others  see  the  lab  work  as  an  "add  on",  or  as  extra  work,  and  prefer  to 
take  strictly  lecture  sections.  Going  into  the  project  we  assumed  that  since  students  are  using  computer 
technology  to  shop,  to  communicate  with  their  friends,  and  to  download  the  latest  music  they  would 
recognize  the  importance  of  using  technology  to  analyze  data  and  enhance  learning.  We  have  found, 
however,  that  we  have  to  work  hard  to  convince  students  of  the  importance  of  using  technology  in  their 
academic  lives.  In  terms  of  instructor  satisfaction,  having  the  opportunity  to  develop  a new  course  with  a 
colleague  has  been  one  of  the  most  rewarding  aspects  of  this  project.  We  have  also  found  that  this  sort  of 
project  has  encouraged  us  both  to  think  carefully  about  the  goals  that  we  have  for  our  courses  and  students. 
In  order  to  center  our  pedagogy  around  technology  and  active  learning,  we  have  relinquished  “center  stage” 
and  find  ourselves  playing  the  role  of  co-leamer  and  coach  rather  than  all-knowing  sage.  This  has  at  times 
been  difficult,  but  we  believe  that  our  students  will  benefit  by  better  retaining  what  they  learn  about  the 
economy.  We  also  believe  that  these  active  learning  skills  will  serve  them  well  as  they  move  through  the 
rest  of  their  academic  and  work  place  lives. 
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Abstract:  This  paper  describes  the  need  for  and  the  creation  of  an  innovative,  web-based 
learning  tool  intended  to  provide  interactive,  hands-on  learning  opportunities  for  students  and 
employees  in  various  institutions  and  businesses.  Specifically,  the  Computer  Information 
Literacy  Modules  (CILM)  educate  people  about  the  relevant  topics  of  computer  crime 
prevention,  ethics  and  morals  in  digital  communities,  and  effective  information  processing. 
Each  module  provides  basic  information  about  the  subject  matter,  interactive  case  studies  and 
simulations,  a self-evaluation,  and  a test  to  evaluate  the  learner’s  competency. 


Introduction 

Computer  and  communication  technologies  continue  to  be  at  the  heart  of  both  changes  and  important 
development  in  the  new  century.  It  is  vital  to  our  global  educational  and  economic  systems  that  online  users 
gain  knowledge  about  computer  crime  prevention,  ethics  in  digital  communities,  and  effective  information 
processing.  This  knowledge  becomes  increasingly  important  considering  the  number  of  people  entering  the 
online  community,  as  well  as  the  number  of  problems  reported  as  a result  of  computer  crime  and  misuse.  The 
Pew  Internet  & American  Life  Project  reports  that  the  “online  adult  population  has  hit  56%,  totaling  104 
million,  and  16  million  new  users  ventured  online  in  the  last  6 months  (JJSA  Today  Website,  2001).  A 1997 
Computer  Security  Institute  survey  indicates  that  “75%  of  US  institutions,  corporations,  and  government 
agencies  reported  financial  losses  in  the  past  year  as  a result  of  computer  crime”  (McKenzie  1998).  While  many 
isolated  web  sites  cover  these  issues,  they  do  not  provide  an  interactive,  hands-on,  problem-centered,  and 
competency-based  learning  opportunity  for  online  users.  The  Computer  Information  Literacy  Modules  (CILM) 
learning  tool  assures  employers  and  educational  institutions  that  every  employee  and  student  has  an  equal 
opportunity  to  learn  a basic  level  of  competency  about  healthy  participation  in  digital  communities  and  effective 
utilization  of  computer  and  communication  technologies  in  one  comprehensive  package  without  limitation  of 
time  and  place.  The  CILM  has  the  potential  to  be  a learning  tool  not  only  for  all  incoming  freshman  classes  at 
Ball  State  University,  but  also  for  students  in  high  school  systems,  students  in  other  higher  educational 
institutions,  and  employees  in  different  branches. 

The  web-based  Computer  Information  Literacy  Module  was  chosen  because  1)  there  are  national 
concerns  and  demands  for  today’s  online  users  in  general  and  students  in  particular  to  have  adequate  knowledge 
and  skills  about  computer  crime  prevention,  ethical  and  moral  issues,  and  information  processing;  and  2)  there 
is  not  a comprehensive,  web-based,  interactive  learning  tool  similar  to  CILM  on  the  market.  The  Internet 
technology  fits  well  with  most  learners,  who  have  other  responsibilities  and  concerns  such  as  job,  family, 
distance,  and  insufficient  time.  In  2000,  grant  money  was  provided  in  order  to  design  and  develop  phase  I of 
the  CILM  project. 

A Description  of  the  Computer  Information  Literacy  Modules 

The  initial  design  of  the  CILM  project  includes  the  following  four  modules:  Module  1,  Computer  Crime 
Prevention,  is  intended  to  help  learners  understand  the  main  issues  of  computer  crimes  and  their  prevention  in 


computer  and  communication  technologies  and  digital  communities.  Module  2,  Ethics  and  Morals^  is  intended 
to  prepare  learners  in  the  areas  of  resource  utilization,  privacy,  intellectual  integrity,  copyright,  and  other 
common  standards  in  open-access  environments  and  digital  global  communities.  Module  3,  Information 
Processing,  is  intended  to  help  learners  gain  needed  knowledge  in  searching  for  resources,  selecting  needed 
resources  from  overloaded  databases,  and  processing  vast  anounts  of  information  for  needed  knowledge. 
Finally,  Module  4,  Ball  State  University's  Infrastructure,  is  intended  to  provide  an  opportunity  for  students  to 
gain  needed  knowledge  about  Ball  State  University’s  technology-based  facilities  and  resources,  procedures  and 
standards  for  their  utilization,  and  available  support  systems.  This  module  can  also  help  new  faculty  and  staff 
members  to  learn  about  technology  infrastructure  and  support  systems  at  Ball  State  University. 

The  instructional  design  of  these  web-based  learning  modules  is  based  on  the  constructivism  model, 
which  supports  the  active  participation  of  learners  in  learning  activities.  The  subject  matter,  case  studies,  and 
simulations  are  designed  and  developed  based  on  current  issues,  events,  concepts,  models,  and  concerns  at  local 
and  global  levels.  Each  module  has  four  sections.  The  key  elements  of  each  module’s  four  sections  include  the 
following:  1)  A description  provides  basic  information  about  the  subject  matter  of  each  of  the  four  modules  for 
learners.  This  section  helps  learners  to  develop  theoretical  knowledge  about  the  subject  matter  and  prepares 
them  for  activities  in  the  next  section.  2)  Rich  interactive  case  studies  and  simulations  help  learners  to  gain 
practical  skills  and  knowledge  about  the  subject  matter  through  hands-on  activities.  This  section  of  each  module 
has  several  case  studies  and  simulations  to  cover  a variety  of  situations.  3)  A self  evaluation  provides  learners 
with  an  opportunity  to  evaluate  their  level  of  knowledge  about  each  subject  matter.  If  a learner  does  not  meet  a 
certain  level  of  readiness,  then  he/she  will  be  advised  about  necessary  additional  study.  4)  Finally,  a test  section 
evaluates  the  learner’s  competency  in  each  module  and  permits  the  learner  to  advance  to  the  next  module.  A 
central  database  system  records  all  learners’  test  results.  The  results  of  each  module  can  be  available  to  various 
academic  departments  for  class  credit  and  to  organizations  for  future  planning  and  action. 

The  instructional  design  of  the  CILM  is  perfectly  suited  for  an  audience  with  different  styles, 
backgrounds,  abilities,  locations,  and  commitments.  The  asynchronous  learning  modules  will  provide  learners 
with  the  flexibility  to  complete  each  module  at  times  that  are  most  convenient  for  them.  Individual  participants 
will  gain  adequate  skills  and  knowledge  from  the  CILM  learning  tool  through  varying  amounts  of  effort  and 
time.  The  World  Wide  Web  is  perfect  for  the  Graphical  User  Interface  (GUI)  design  and  interactive  multimedia 
programming  of  the  CILM  project.  Today,  any  reasonably  equipped  PC  with  a basic  connection  to  the  Internet 
provides  a good  platform  to  access  the  CILM  learning  tool  from  anyplace  at  anytime. 


Conclusion 

The  four  sections  of  each  module  are  designed  in  a way  that  serve  all  different  learners’  basic  needs. 
The  description  section  will  help  all  participants  gain  some  theoretical  knowledge  about  subject  matter  in  each 
module.  The  case  studies  and  simulations  section  will  help  learners  gain  practical  knowledge  in  addition  to 
theoretical  knowledge.  The  self-evaluation  section  will  help  learners  to  evaluate  their  progress  in  each  module 
and,  when  needed,  advise  them  about  additional  learning  activities.  Finally,  the  test  section  will  certify  their 
adequate  competency  in  subject  matter  and  will  record  the  results  in  a database. 

Ball  State  University’s  Information  Technology  Office,  Student  Affairs  Office,  University  Computing 
Services,  College  of  Continuing  Education,  and  Computer  Science  Department  already  support  the  development 
and  implementation  of  the  CILM  project  for  students  at  Ball  State  University  and  online  learners  in  different 
branches  in  Indiana.  The  quality  of  the  modules  will  be  ensured  by  adherence  to  the  formative  evaluation  plan 
and  content  review  by  experts  in  the  field.  For  legal  issues  and  accuracy  of  the  content,  local  experts  will  be 
asked  to  review  content  before  it  is  published  on  the  Web.  Phase  I of  the  CILM  project,  completed  in 
December  2001,  can  be  found  at  <web.bsu.edu/cilm/>. 
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Abstract:  Effectively  applied,  the  lessons  the  eleaming  community  has  learned  about  designing  and 
delivering  education  and  training  at  a distance  can  enable  organizations  to  evolve  new  and  potentially  better 
ways  of  conducting  conferences.  Travel's  high  cost  and  inconvenience  have  often  been  cited  as  the  impetus 
for  rethinking  the  role  of  the  classroom.  These  reasons,  coupled  with  the  economic  downturn,  have  led  to 
rethinking  the  need  for  and  the  role  of  professional  gatherings.  Technology  has  transformed  education  and 
training,  and  similarly  has  the  potential  transform  conferences  in  ways  that  can  be  beneficial  both  to 
participants  and  those  who  would  like  to  participate. 
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INTRODUCTION 

Based  on  the  number  of  e-mails  Tve  seen  recently  offering  extended  deadlines,  it  seems  that  a lot  of 
conferences  are  not  attracting  as  many  submissions  or  advance  registrations  as  they  would  like.  It  made  me 
wonder  if  conferences  are  evolving  in  a way  that  parallels  corporate  training  in  that  dwindling  enrollments, 
budgetary  pressures,  and  new  technologies  have  led  to  an  interest  in  online  alternatives.  Will  the  same  factors 
lead  to  the  emergence  of  the  “blended  conference”? 

I can  track  much  of  my  professional  career  by  conference  attendance  — absorbing  as  much  wisdom  as  possible 
at  the  early  ones  and  eventually  networking  and  presenting  with  relative  ease.  I find  conferences  a source  of 
ideas  and  inspiration,  as  well  as  a way  to  assess  trends  and  the  state  of  the  art.  I like  writing  conference  papers 
and  giving  talks, because  it  forces  me  to  reflect  and  think  clearly  about  a subject.  The  conferences  I like  best  are 
the  ones  where  I know  a lot  of  people  and  meet  new  people  with  interests  similar  to  mine'.  I collect  business 
cards  and  stay  in  touch  with  people,  and  have  even  made  some  close  friends  at  conferences. 

Today’s  lower  attendance  levels  — caused  by  security  issues  and  the  economic  recession  — concern  many  of 
us  who  savor  the  learning  and  networking  opportunities  conferences  represent.  Now,  these  circumstances  have 
brought  us  to  a crossroads  where  the  basic  model  for  conferences  is  being  reexamined.  For  example,  instead  of 
being  multi-day  events  at  a convention  center  or  hotel,  some  conferences  are  conducted  online.  Many  more  take 
a "blended”  approach,  using  technology  for  submissions,  registration,  and  advance  programs,  pre -conference 
activities,  and  broadcast  or  archive  sessions. 

To  look  at  the  effectiveness  of  online  or  blended  conferences,  you  first  have  to  consider  why  people  attend 
conferences  in  the  first  place:  To  learn  and  network.  They're  a break  from  work,  and  they  offer  a chance  to  meet 
people,  get  new  ideas  and  insights,  reflect,  have  a change  of  scenery,  and  eat  good  food.  These  are  the  same 


[1]  “The  many  manifestations  of  organizational  change  - downsizing,  outsourcing,  merging,  splitting,  acquiring,  partnering, 
and  the  constant  redrawing  of  internal  organizational  charts”  - have  led  to  a situation  in  which  workers  become  more 
dependent  on  personal  social  networks  that  cross  organizational  boundaries.  Conferences  are  among  the  best  ways  to 
establish  and  refresh  the  “intensional  networks”  defined  in  http://www.firstmondav.org/issues/issue5  5/nardi/index.html.. 
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reasons  employees  like  off-site  training.  Conferences  are  also  an  immersive  experience  where  people  focus  on 
conference  activities  and  other  attendees  while  professional  and  personal  concerns  are  largely  left  behind. 
Conferences  foster  a sense  of  membership  or  belonging  in  a professional  society  or  discipline.  None  of  these 
attributes  requires  technology. 


MOVING  CONFERENCE  ACTIVITIES  ONLINE 

Methods  can  be  developed  and  refined  for  integrating  online  and  in-person  conference  components  in  a kind  of 
blended-leaming  approach.  Teachers  are  well  aware  of  the  advantages  of  combining  online  and  classroom 
activities:  Pre-class  activities  reduce  the  quantity  of  in-class  time  spent  covering  basics,  and  they  help  level  the 
playing  field  on  which  students  tackle  a new  subject.  Classroom  time  is  spent  on  activities  where  face-to-face 
contact  is  most  beneficial.  In  theory,  these  advantages  can  be  applied  to  conferences. 

When  conference  Web  sites  link  to  abstracts  and  papers,  attendees  can  plan  their  schedules  better  and  prepare 
for  sessions.  Both  attendees  and  people  who  can't  attend  the  conference  can  participate  in  e-mail  discussions, 
virtual  meeting  places,  or  discussion  forums.  One  blended-leaming-like  result  is  that  actual  conference  time  is 
used  more  efficiently.  ASTD,  www.astd.org.  for  example,  has  successfully  implemented  some  of  these 
components  for  its  annual  conference. 

CHI  2002  offers  CHIpIace,  www.chiplace.org.  where  anyone  can  set  up  a profile,  see  who  else  is  there  and 
what  their  interests  are,  preview  and  comment  on  papers,  and  discuss  conference-related  topics.  I'm  sure  the 
most  heavily  accessed  part  is  the  personal  profiles.  I especially  like  the  feature  that  generates  random  pictures  of 
participants.  It's  similar  to  the  serendipity  of  running  into  someone  at  a conference  break.  Unfortunately  it 
doesn’t  provide  the  interaction  or  awareness  indicators  that  make  breaks  so  much  fun. 


MOVING  CONFERENCE  SESSIONS  ONLINE 

Some  conferences  Webcast  sessions,  which  is  not  unlike  COMDEX’S  broadcasts  of  popular  sessions  to 
overflow  ballrooms.  Some  conferences  archive  sessions  online  as  well.  When  I consider  the  benefits  of 
Webcasting  and  archiving,  I am  reminded  of  my  experiences  teaching  online:  In  general,  it's  hard  to  engage 
both  present  and  distant  audiences.  The  one  time  I was  Webcast  while  presenting  at  a conference,  I had  trouble 
ignoring  the  cameras  and  focusing  only  on  the  audience.  Then  there  are  the  speakers  who  present  by 
videoconferencing  to  a "live"  audience,  which  rarely  is  the  same  as  being  there  for  speaker  or  audience. 
However,  I imagine  the  people  "watching"  — hard  to  call  it  participating  — these  sessions  think  it's  better  than 
not  being  able  to  listen  to  the  talks  at  all. 


PUTTING  A CONFERENCE  ONLINE 

How  can  fully  online  conferences  be  made  effective?  Ultimately,  the  answer  parallels  the  familiar  e- 
leaming/classroom  debate:  Rather  than  trying  to  figure  out  how  to  reproduce  a conference  as  an  online  event, 
we  should  try  something  different  online  that  exploits  our  delivery  technology’s  capabilities  for  superior 
results.^  Cost,  comfort,  and  convenience  are  potential  benefits  of  the  online  format.^ 


[2]  David  Eastment,  in  the  Guardian,  http://education.guardian.co.Uk/eleaming/storv/Q.lQ577.6Q3226.QQ.html.  wrote  about 
attending  a fully  online  conference  and  thought  it  was  "better  online  because  you  can  flip  forward  and  back  through  the 
presenter's  screens  rather  than  simply  follow  what  is  being  said,  and  you  can  get  up  and  stretch  when  you  want"  and  switch 
sessions  without  stepping  on  toes.  Eastment  pointed  out  that  "only  a few  presenters  had  had  previous  experience  of  speaking 
to  an  unseen  audience,"  but,  in  general,  conference  presenters  vary  widely  in  how  good  they  are,  and  there  may  be  better 
opportunities  to  provide  coaching  for  first-time  online  presenters. 

[3]  In  some  recent  experiences  organizing  an  on  online  seminar  (Kim  Vicente  speaking  about  Cognitive  Task  Analysis, 
www.ctaresource.com)  feedback  was  largely  positive,  including  '1  thought  that  it  was  a great  way  to  hear  a speaker.  No 
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How  much  are  people  willing  to  pay  for  an  online  conference  or  online  portions  of  an  in-person  conference? 
Online  conferences  and  seminars  are  usually  free,  although  I have  seen  a few  charging  high  fees.  There  is  a cost 
to  using  technology,  and  conference  organizers  may  fear  that,  in  using  a blended  approach,  they  lose  attendees 
who  prefer  to  participate  online."^  Over  time  weMl  find  out  if  online  programs  provide  a viable  option  for  those 
who  wouldn’t  otherwise  attend  a particular  conference  at  all,  or  if  these  programs  are  most  successful  at  getting 
people  excited  about  in-person  attendance. 


THE  FUTURE  OF  CONFERENCES 

Let’s  go  back  to  my  original  list  of  reasons  people  attend  conferences  in  person:  education,  networking,  break 
from  work,  chance  for  reflection,  new  ideas,  a change  of  scenery,  good  food,  and  a sense  of  connectedness. 

I got  my  job  by  networking  at  a traditional  conference,  and  it's  hard  to  imagine  replicating  that  experience 
online.  And  what’s  the  likelihood  of  an  online  conference  giving  you  a break  from  work  and  a chance  for 
reflection?  Limited,  at  best;  in  fact,  the  opposite  is  probably  true.  Because  online  conferences  are  typically 
crammed  into  existing  work  hours,  frustration  and  fragmentation  are  the  likely  outcomes.  The  change  of 
scenery  and  good  food  I can  get  on  my  next  vacation.  But  the  sense  of  membership  or  belonging  is  harder  to 
acquire  online.  In-person  conferences  excel  at  promoting  connectedness,  scheduling  breaks  and  receptions  to 
increase  physical  presence,  and  using  badges,  t-shirts,  and  bags  for  a sense  of  inclusion. 

Where  can  technology  help  further?  At  any  multi-track,  in-person  conference,  it  would  be  great  to  know  more 
about  what  you  missed.  I'd  like  to  be  able  to  replay  some  sessions,  although  I understand  the  dilemmas 
organizers  face  h determining  how  to  price  those  capabilities  and  offer  them  in  a way  that  doesn't  lower 
registration  numbers.  I miss  sessions  because  I’m  talking  to  someone  in  the  hall,  and  that  experience  is  much 
harder  to  facilitate  with  technology. 

Since  I'm  not  great  at  standing  up  during  sessions  to  ask  questions.  I’d  like  the  opportunity  to  make  queries 
online  and  see  others’  questions  and  answers.  Presenters,  attendees,  and  people  participating  at  a distance  could 
all  use  such  a resource.  While  the  prospect  of  piled-up  work  and  full  in-boxes  may  inhibit  participation  in  post- 
conference discussions,  extending  the  conference  experience  online  may  be  the  best  way  of  increasing 
conferences'  relevance  back  in  the  "real"  world. 

Based  on  where  technology  is  now  and  how  people  and  organizations  respond  to  technology-delivered  courses, 
I don’t  believe  that  online  conferences  are  a substitute  for  on-site  ones.  Online  conferences  may  be  valuable  for 
what  they  do  offer:  more  diversity  in  the  participants  and  more  flexibility  to  reflect  on  and  apply  concepts. 
Current  technologies,  even  telepresence,  cannot  replicate  the  immersive  experience  of  in-person  conferences. 

We  are  still  at  an  embryonic  stage  in  the  application  of  technology  to  conferences.  Conference  organizers  are 
experimenting  with  and  evaluating  different  approaches;  the  results  to  date  have  been  interesting,  as  have  been 
the  diversity  in  their  approaches.  People's  expectations  of  and  comfort  levels  with  technology  change  over  time, 
and  this,  in  conjunction  with  further  experimentation  and  innovations,  may  be  what  ultimately  leads  to  new  and 
more  successful  models  for  technology-enhanced  conferences. 


airfare,  no  conference  fees,  no  hotel  to  pay  for.  It  ...  allowed  people  who  would  otherwise  never  get  the  chance  to  hear  an 
accomplished  person  in  the  CWA  field."  “I  loved  being  able  to  sit  in  my  office...  and  put  my  feet  on  my  desk.” 

[4]  Duke  University’s  Fuqua  School  of  Business  charges  more  for  their  online  MBA  program  because  people  are  willing  to 
pay  for  the  convenience.  When  they  started  their  online  MBA,  they  must  have  been  concerned  about  it  drawing  students 
away  from  the  campus  program. 
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Introduction 

Modeling  of  technology  integration  by  university  faculty  into  their  respective  course  delivery 
structures  is  a crucial  element  in  the  preparation  of  technologically  proficient  teachers  (OTA, 

1995;  Topp,  Mortenson,  & Gradgenett,  1995;  Wetzel,  1993).  These  “teachers  in  training”  need  to 
experience  technology  within  the  context  of  both  their  learning  opportunities  at  the  university 
and  their  student  teaching  experiences  in  the  field  setting.  In  order  to  model  this  type  of 
framework  within  their  pedagogy,  university  faculty  must  change  attitudes  and  behaviors  toward 
the  use  of  technology  (Joyce  & Showers,  1983;  Schrum,  1999).  As  Strudler  and  Wetzel  (1999) 
assert,  the  “goal  of  technology  integration  is  a moving  target”  (p.80),  and  therefore  cannot  be 
studied  in  a vacuum.  Rather,  faculty  must  be  dedicated  to  beginning  and/or  continuing 
professional  development  of  their  own  technology  proficiencies  within  the  dynamics  of  their 
learning  organization.  Furthermore,  they  must  be  cognizant  of  the  extensive  technology 
proficiencies  required  by  state  teacher  certification  standards.  The  following  proposal  details  a 
local  experience  relating  to  a technology  integration  project  designed  to  support  the  ongoing 
refinement  of  faculty  technology  skills  within  the  context  of  the  teaching  and  learning  situation. 

Important  Features  of  Project 

The  Learning  and  Integrating  Knowledge  and  Skills  (LINKS)  project  is  a three-year  technology 
project  is  designed  to  integrate  established  and  emerging  technologies  into  the  teacher 
preparation  curriculum  at  the  Texas  Woman’s  University  (TWU)  and  is  supported  by  a U.S. 
Department  of  Education,  Preparing  Tomorrow’s  Teachers  to  Use  Technology  (PT^) 


implementation  grant.  The  LINKS  program  supports  both  changes  in  university  faculty  roles  and 
changes  in  curriculum  content  and  delivery. 

This  paper  addresses  two  primary  questions:  (1)  how  were  the  development  of  faculty 
technology  proficiencies  supported  and  (2)  what  was  the  progress  of  the  participating  faculty  in 
infusion  of  technology  in  university  web-based  course  delivery?  To  address  these  questions, 
second  year  findings  regarding  the  examination  of  changes  in  faculty  attitudes  and  behaviors  and 
the  effectiveness  of  the  implemented  LINKS  project  are  provided. 

Description  of  Problem 

Integration  of  technology  into  university  course  delivery  provides  an  essential  model  for  pre- 
service teachers.  Pre-service  teachers  recognize  that,  throughout  the  teacher  preparation  required 
coursework,  they  are  expected  to  demonstrate  technology  integration  skills  prior  to  student 
teaching.  Therefore,  these  future  teachers  expect  university  faculty  to  model  these  behaviors 
within  the  context  of  their  university  learning  experiences.  Increasingly,  faculty  from  Arts  and 
Science  as  well  as  from  Teacher  Education  recognize  the  need  to  model  within  their  own  course 
delivery  a variety  of  ways  to  integrate  technology  into  teaching  and  learning.  This  faculty 
challenge  encompasses  both  the  new  technology  proficiencies  and  contemporary  issues  in 
pedagogy. 

Description  of  Need,  Population,  and  Implementation  of  Project 

Need 


A needs  assessment  from  a purposive  sample  of  current  faculty  addressed  faculty  needs  relating 
to  the  integration  of  advanced  technology  tools  into  course  delivery.  Their  perceived  critical 
needs  were  in  four  areas  of:  (1)  productivity;  (2)  connectivity;  and  the  integration  of  technology. 

Faculty  members  see  productivity  as  a priority  including  support  in  development  of  their  skills 
related  to  multi-media  presentations  and  the  creation  of  a professional  web  page  for  use  in 
classroom  instruction  and  communication  with  pre-service  teachers.  The  survey  data  indicated 
the  need  for  training  to  extend  their  multimedia  presentations  with  more  advanced  media  tools 
and  the  use  of  connectivity  tools  such  as  the  World  Wide  Web  and  asynchronous  and 
synchronous  forms  of  communication. 

In  order  to  provide  a model  for  our  pre-service  teachers,  faculty  must  integrate  technology  into 
the  design  and  delivery  of  all  instruction.  Advanced  forms  of  technology  must  be  infused 
throughout  all  courses,  not  just  a separate  technology  method  course,  as  transparent  tools  for 
thinking,  learning,  and  constructing  new  knowledge.  According  to  survey  data,  support  for  the 
use  of  web-based  curriculum  and  instructional  materials  were  high  priorities  for  the  faculty.  They 
identify  their  need  for  training  in  evaluation  and  utilization  of  advanced  forms  of  media,  of 
content  related  Internet  resources  related  to  content  areas,  and  of  appropriate  CD-ROMs  and 
content-specific  software. 
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Beginning  in  the  spring  of  1999  the  LINKS  program  for  faculty  was  created  and  implemented  to 
respond  to  these  perceived  needs. 

Population 

Faculty  volunteers  from  a variety  of  disciplines  were  recruited  to  attend  technology-training 
sessions  began  September  2000  and  were  completed  in  April  2001.  These  faculty  members  were 
recruited  from  the  group  responsible  for  coursework  within  the  teacher  education  sequence. 

Their  only  compensation  was  the  use  of  a University  laptop  outfitted  with  web  page  browsers, 
Microsoft  Office,  multimedia  software,  and  Ethernet  connections  for  the  duration  of  their 
participation  in  the  LINKS  project.  This  volunteer  sample  of  university  faculty  (N=20)  registered 
for  each  LINKS  training  session  one  week  before  the  session  date.  If  vacancies  became  available 
for  any  session,  the  training  on  that  day  was  opened  to  interested  faculty  from  the  College  of 
Professional  Education  and  the  College  of  Arts  and  Sciences.  At  each  session  data  were  collected 
on  faculty  stages  of  concern,  levels  of  use,  and  perceptions  of  the  effectiveness  of  the  training 
they  had  received. 

Implementation  of  Project 

Training  Goals.  The  primary  training  goals  were:  (1)  the  introduction  of  LINKS  standards  and 
resources,  and  (2)  support  for  faculty  delivery  of  web-based  courses  as  models  for  future 
teachers.  The  goals  were  addressed  through  two  types  of  sessions.  Hands-on  professional 
development  sessions  focused  on  specific  areas  for  remediation  or  for  more  advanced  work  and 
whole-group  development  sessions  addressed  broad  topics  essential  to  all  participants. 

Training  Session  Overview.  All  sessions  served  as  an  orientation  to  the  required  technology 
proficiencies  Texas  pre-service  teachers  are  expected  to  master  and  demonstrate.  To  provide  a 
cohesive  model,  faculty  were  encouraged  to  integrate  technology  into  the  design  and  delivery  of 
their  instruction.  These  sessions  also  served  as  a training  vehicle  for  the  preparation  of  web- 
based  course  delivery  via  Blackboard,  the  University’s  web-based  course  delivery  template. 

Fourteen  sessions  were  available  throughout  the  2000-2001  year,  seven  per  semester.  These  two- 
hour  sessions  were  delivered  in  a University  computer  lab  providing  Blackboard  access.  All 
materials  were  available  on  a “class”  Blackboard  site  for  later  reference.  By  using  a web-based 
course  delivery  system  to  communicate  with  faculty  participants,  meaningful  learning  was 
integrated  into  their  own  web-based  course  development. 

LINKS  infused  advanced  forms  of  technology  throughout  training  sessions  as  transparent  tools 
for  thinking,  learning,  and  constructing  new  knowledge.  Integrating  technology  as  a transparent 
tool  includes  the  use  of  advanced  forms  of  media  such  as  optical  technology,  scanners,  video  and 
sound,  digital  and  document  cameras,  and  streaming  video. 

Many  sessions  were  designed  as  hands-on  workshops.  Faculty  were  encouraged  to  bring  course 
materials,  syllabi,  graphics,  and  other  materials  to  use  as  they  converted  traditional  courses  to  the 
electronic  medium.  LINKS  provided  practical  printed  reference  materials,  related  web  page 
URLs,  and  individual  assistance  as  needed.  Additional  one-on-one  assistance  was  available  from 
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LINKS  staff  by  request.  Personnel  from  Information  Technology  Services  (ITS),  the  Distance 
Education  Support  Team,  and  from  the  library  services  collaborated  with  LINKS  personnel  in 
organization  and  implementation  of  the  training. 

Training  Session  Topics. 

• Session  I:  Orientation  and  Data  Collection.  A whole-group  session  which  served  as  an 
orientation  to  the  LINKS  project  and  PT^,  including  student  and  faculty  benefits.  The 
research  methodology  was  outlined,  and  data  was  collected  via  online  tools. 

• Session  II:  Exploring  Internet  Resources.  A whole-group  session  designed  as  an 

orientation  to  TWU  Library  resources,  search  engines,  and  other  tools. 

• Session  III:  Searching  Without  Losing  Your  Composure.  A hands-on  session  with  step- 
by-step  procedures  for  searching  the  Internet  using  tools  available  on  the  Internet. 

• Session  IV:  Getting  Your  URLs  Organized  and  Creating  a Virtual  Office.  A hands-on 
session  designed  to  organize  favorite  and  frequently  used  resources  into  directories  for 
Bookmarks  or  Favorites. 

• Session  V:  Introduction  to  Blackboard.  A whole-group  session  designed  to  address 
questions  concerning  the  creation  of  courses,  getting  access  from  a remote  location,  and 
managing  information  and  sharing  information  in  the  course. 

• Session  VI:  ABC’s  of  Preparing  Documents  for  Blackboard.  A hands-on  session  which 
discussed  the  philosophy  of  organizing  course  materials. 

• Session  VII:  Shortcuts  and  More.  A hands-on  session  with  an  emphasis  on  keyboarding 
and  toolbar  shortcuts  in  Microsoft  Word  and  PowerPoint. 

• Session  VIII:  Exploration  of  Blackboard.  A whole-group  session  which  covered  upgrades 
from  Blackboard  4.0  to  5.0.  Collection  via  online  tools  was  also  administered. 

• Session  IX:  Adventures  in  Uploading  for  the  Tentative.  A hands-on  session  that  related  to 
the  uploading  of  documents  into  Blackboard  for  the  novice  user. 

• Session  X:  Cliff  Walking  on  the  Rocky  Blackboard  Range.  A session  for  the  more 
experienced  user  that  related  to  uploading  different  document  types. 

• Session  XI:  Refinement  of  Blackboard.  A whole-group  session  specifically  related  to 
information  dissemination  addressing  pedagogical  issues  pertaining  to  Blackboard. 
Speakers  discussed  recent  finding  from  EduTex,  a regional  conference  of  EDUCAUSE, 
in  San  Antonio.  Spotlighted  were  best  practices  and  how  other  TWU  faculty  members  are 
using  Blackboard. 

• Session  XII:  Beyond  Simple  Documents.  Adding  PowerPoint  presentations  and  graphics 
to  online  classes. 

• Session  XIII:  Braving  New  Frontiers.  Using  HTML  coding. 

• Session  XIV:  Symposium  and  Data  Collection.  A whole-group  session  that  consisted  of 
closure,  overview  of  lessons  learned,  best  practices  models  and  presentations,  and  online 
data  collection. 

Data  Sources.  The  purpose  of  related  research  inquiry  and  associated  program  evaluation  of  the 
LINKS  project  was  to  assess  the  changes  in  faculty  behaviors  and  attitudes  related  to  the 
integration  of  technology  and  to  measure  the  effectiveness  of  training.  Intrinsic  to  the 
examination  of  behaviors  and  attitudes  of  participating  faculty  was  the  use  of  the  Concerns 
Based  Adoption  Model  (CBAM).  The  module  was  developed  by  Hall,  Wallace,  and  Dosset 
(1973)  at  the  University  of  Texas  at  Austin  Research  and  Development  Center  for  Teacher 
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Education.  Because  the  faculty  could  be  followed  and  supported  as  they  adopted  the  innovation 
(i.e.,  use  of  a web-based  course  delivery  system),  the  CBAM  model  was  implemented  with  this 
population  through  the  Stages  of  Concern  Questionnaire  (SOCQ)  and  the  use  of  a modified 
online  survey  stmctured  from  the  Levels  of  Use  (LoU)  interview  process. 

Hall  et  al.  (1973)  discovered  that  an  individual  progresses  through  three  clusters  of  concern  as 
she  or  he  adopts  an  innovation— self  concerns,  task  concerns,  and  impact  concerns.  Three 
diagnostic  dimensions  were  identified  that  are  used  to  monitor  a person’s  progress  through  these 
stages:  Stages  of  Concern  Questionnaire  (SoCQ),  Levels  of  Use  Interview  (LoU),  and  Innovation 
Configuration  (IC).  Both  the  SoCQ  and  the  LoU  were  used  to  assess  progress  toward  innovation 
adoption  or  the  IC. 

To  gauge  the  effectiveness  of  the  provided  training  opportunities,  faculty  completed  evaluation 
questionnaires  at  the  end  of  each  two-hour  session.  The  evaluation  questions  related  to  the 
session’s  impact  on  their  motivation  to  apply  new  knowledge  and  skills  and  the  ability  to  use 
technology  more  effectively  in  the  classroom.  Additionally,  an  overall  perception  of  session 
quality  was  obtained.  The  evaluations  of  individual  training  sessions  were  utilized  to  refine 
management  and  enhance  training  in  subsequent  semesters. 

Analysis.  Pre-  and  posttest  data  were  collected  for  the  SoCQ  and  Levels  of  Use.  Parametric 
paired-sample  t-tests  assessed  differences  across  time.  Qualitative  analyses  were  conducted  for 
open-ended  items  on  session  evaluation  forms. 

Implications  for  the  Local  Setting 

The  primary  goals  for  the  university  faculty  were  introduction  of  the  LINKS  standards  and 
resources,  and  support  for  faculty  delivery  of  web-based  courses  as  models  for  the  future 
teachers.  Descriptive  statistics  and  profiles  for  the  SoCQ  suggested  that  the  faculty  had  high 
informational  and  personal  concerns  as  well  as  increased  consequence  and  collaboration 
concerns.  These  types  of  concerns  are  consistent  with  movement  through  the  change  process. 
Analysis  of  the  LoU  revealed  that  all  individual  faculty  made  significant  progress  in  levels  of 
use,  although  they  began  at  different  levels.  Qualitative  analyses  of  open-ended  evaluation 
questionnaire  items  suggested  issues  related  to  individual  technical  ability,  to  the  time  needed  for 
implementation  and  integration  of  technology,  and  to  the  applicability  of  new  learning  to  specific 
teaching  situations.  Additional  analyses  of  implementation  data  have  been  conducted  and  will  be 
reviewed  during  the  course  of  this  paper  presentation. 

The  LINKS  project  provided  a wealth  of  training  opportunities  for  TWU  faculty  volimteers 
relative  to  the  teacher  preparation  program.  The  following  are  key  findings: 

• Persoimel  from  various  campus  departments  collaborated  to  deliver  staff  development  for 
faculty. 

• LINKS  training  was  tailored  to  accommodate  faculty’  broad  range  of  technology 
abilities. 

• LINKS  activities  raised  university  faculty’  awareness  of  technology  proficiencies  needed 
by  future  Texas  teachers. 
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• LINKS  training  positively  impacted  university  faculty’  motivation  to  use  and  capacity  to 
integrate  technology. 

• Suggestions  for  training  improvement  centered  on  the  individualization  of  training  to 
meet  faculty’  particular  needs. 

• Faculty’  main  concerns  centered  on  time,  personal  skill  proficiency,  skill  retention,  and 
resources. 

^ Faculty  concerns  with,  web -based  course  implementation,  as  measured  by  the  SoCQ, 
changed  over  time. 

e Progress  toward  higher  levels  of  web-based  course  delivery  use  varied  for  particular 
faculty. 

Implications  for  External  Settings 

Even  though  the  benefits  of  technology  integration  has  been  well  established  in  the  literature,  the 
United  States  Department  of  Education  (USDE)  finds  that  “relatively  few  teachers  (20%)  report 
feeling  well  prepared  to  integrate  educational  technology  into  classroom  instruction”  (2000a).  In 
a separate  report,  USDE  suggested  that  “teachers  have  been  prepared  for  a model  of  teaching 
dramatically  out  of  step  with  what  is  needed  to  prepare  the  nation’s  students  for  the  challenges 
they  will  face  in  the  future”  (USDE,  2000b).  Findings  presented  regarding  the  implementation 
and  effectiveness  of  the  LINKS  project  has  implications  for  increasing  the  technology 
proficiencies  of  entry-level  teachers  as  well  as  providing  a model  for  other  universities 
undertaking  similar  changes  and  institutionalization  efforts.  Findings  have  particular  relevance  to 
explain  how  university  faculty  can  be  nurtured  and  supported  as  effective  models  of  technology 
use  in  web-based  course  delivery  and  electronic  communication  with  students. 

Through  the  examination  of  the  change  process  and  the  utilization  of  this  information  to  support 
faculty  to  become  users  and  modelers  of  technology,  multiple  benefits  are  evident.  By  providing 
a supporting  scaffold,  university  faculty  become  less  fearful  of  change  and  their  attitudes  take  on 
a willingness  to  engage  in  increased  learning  experiences.  When  pre-service  teacher  witnesses 
technology  integration  within  the  auspices  of  their  university  coursework  they  are  more  likely  to 
adopt  the  same  technology  integration  and  model  that  back  to  their  future  students  thus 
encouraging  a true  technology  integration  model. 
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Abstract:  First  steps  in  Teleleaming  is  the  title  of  our  project  (started  at  the 
beginning  of  the  schoolyear  2001/02)  in  which  we  try  to  evaluate  methods  of 
teleleaming  for  children  at  the  age  of  6-10  years.  We  are  iiterested  in  the  aims, 
chances  and  borders  of  teleleaming  at  this  age. 

1.  Introduction  and  Basic  Information 

Our  main  intention  is  to  find  out,  how  models  of  teleleaming  could  be  adapted  for  children  at  the  age  of  6 to  10 
years:  We  try  to  find  out,  which  basic-skills  are  absolutely  necessary  to  enable  children  to  use  teleleaming  tools. 
We  are  interested  to  find  and  evaluate  ways  to  guide  children  to  a self-motivated  and  self-guided  learning  and  try 
to  find  out  the  limits  of  tele  learning-projects  for  children  at  this  age. 


2.  First  steps:  Training  of  basic  competences  in  using  IT 

Before  starting  the  tel  el  earning-project  itself,  we  had  to  train  the  kids  (and  some  teachers)  the  basic  skills  in 
using  computers  and  modem  IT.  To  motivate  children  we  always  start  with  very  simple  and  easy  computer 
games  and  similar  things:  Using  Windows  and  its  basic  functions,  use  of  very  simple  standard- software 
(wordpad,  paint,  ...),  first  steps  in  handling  data, ... 


3.  Network-Technology 

) 

First  steps  in  understanding  network -technology:  Right  from  the  beginning  we  are  using  the  network:  We 
“force”  the  children  to  save  their  data  on  network- drives.  Beside  that  we  begin  to  communicate  using  the  LAN. 
Then  we  start  the  first  steps  in  the  www  (“guided  tours”,  finding  & saving  information, ...)  with  a short  basic 
training.  After  that  children  should  find  different  informations  (using  different  search-engines).  While  training 
the  basic  internet  -skills  children  are  also  learning  to  use  mailing  to  communicate:  Each  child  creates  his  own 
mail- account  (webbased)  and  sends/receives  e-mails.  Another  part  of  our  “basic -training”  is  to  use  chat  - engines: 
Pupils  begin  to  chat  using  an  Austrian  chat- engine,  created  by  Prof.  Dr.  Rita  Humer  (Pedagogic  Academy) 
especially  for  children  (www.antonwelt.at). 


4.  Netmeeting 

As  a consequence  of  our  idea  to  base  our  project  on  standard-software  and  very  common  used  tools,  we  mainly 
use  software-products  of  Microsoft.  MS  Netmeeting  enables  us  to  communicate  in  a simple  way  in  our  LAN  as 
well  as  in  in  the  www. 

5.  First  Telelearning-Projects 

Our  first  projects  & tests  were  based  on  projects  and  contents  of  regular  lessons:  The  pupils  began  to  use  MS 
Netmeeting  as  a part  of  the  “daily  school -life”:  Contacts  to  parents,  contacts  to  dijferent  "professionals  ”.  The 
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first  challenge  was  to  take  part  at  the  " Interpaedagogika  2001 " (November  S*''  -10**^,  2001 ) with  20  of  our 
pupils:  Ten  pupils  in  Linz  (at  the  fair)  and  ten  in  Vienna  presented  how  to  communicate  with  netmeeting  & 
“antonchat”  (www.antonwelt.at).  As  a consequence  of  the  special  knowledge  of  those  children,  they  could  also 
present  the  possibilities  of  application -sharing  and  worked  together  with  different  software-products  (to  show, 
how  to  solve  several  problems  together  using  the  web).  Another  part  of  the  presentation  was  working  with  word 
and  excel  in  a “long- distance-team”.  In  the  future  we  plan  to  establish  National  & International  Partnerships: 
We  initiated  a partnership  with  a small  school  in  Salisbury  (country).  Now  the  children  have  the  opportunity  to 
learn  to  communicate  with  "strangers”:  They  have  to  get  to  know  each  other  by  the  web-based  communication. 
The  latest  project  and  next  step  is  to  organize  an  international  partnership  with  schools  of  other  European 
countries:  The  "Comenius-Programm"  is  the  basis  for  this  plans,  - We  are  interested  in  establishing  a longer 
lasting  partnership:  Pupils  of  the  different  countries  shall  learn  about  each  other  and  about  the  different 
hometownsAcountries. 


6.  Accompanying  projects  and  outcomes 

Another  part  of  our  work  concerning  our  teleleaming-project  is  to  enable  the  pupils  to  use  the  opportunities  of 
modem  distance- communication  and  IT  for  their  school-duties,  their  individual  learning  and  their  daily  life, 
"Learning  at  home”  - Home  exercises,  using  e-mail,  encourage  special  interests:  We  noticed,  that  some  pupils 
began  to  communicate  with  members  of  their  family,  they  did  not  see  every  day:  For  instance  there  are  some 
children,  whose  parents  are  divorced.  They  by  themselves  began  to  communicate  by  using  e-mail,  chat-engines 
and  MS  Netmeeting  to  establish  contact. 


7.  Problems  & Limits 

First  there  are  technical  and  software- problems:  e.  g,  bandwidth;  most  of  the  user- interfaces  (hardware  and 
software)  are  not  created  for  children's  use. 

On  the  other  hand  there  are  problems  and  limits  of  telelearning  especially  for  children  at  this  age{6  - 1 0 years) 
as  there  are: 

Usage  of  the  mother  tongue.  Children  are  taught  to  read  and  to  write  at  the  age  of  six  years.  But  this  culture 
technique  is  the  minimum  base  for  the  use  of  "usual"  tools  of  modem  IT  and  web-based  communication: 

- Capability  of  reading:  Reading  speed  of  pupils  at  this  age  is  very  low;  so  they  need  much  time  to  read  the 
informations  they  are  given  by  the  computer-human- interfaces, 

- Capability  of  writing:  Children  are  able  to  type  letters,  but  they  are  typing  very  slowly  - the  speed  of  text  -based 
communication  often  is  to  low.  Therefore  it  is  very  important,  to  work  with  video-Zaudio -support  or  by  using 
very  intuitive  graphic- interfaces  (bandwidth!), 

- Language -style  of  the  children:  The  development  of  the  language-style  is  very  intensive  at  this  age.  So  children 
are  often  not  able  to  express  what  they  mean.  While  they  mostly  can  speak  out  their  thoughts  orally,  they  are  not 
able  to  do  this  by  written  language  in  the  same  way). 

Foreign  language:  The  most  important  “computer- language”  is  English  - but  usually  children  begin  to  learn  this 
foreign  language  when  they  start  school.  But  most  of  the  the  web  - information  is  in  English  and  so  our  German 
speaking  children  are  not  able  to  use  them! 


8.  Accompaning  Measures 

Teacher  education  and  advanced  classes  for  teachers  with  experience  in  working  with  computers:  We  always  try 
to  involve  students  of  the  Pedagogic-Academy  to  our  project:  It  is  also  a very  important  aim  for  us,  to  motivate 
and  enable  them  to  use  IT  for  their  future  educational  work.  Another  very  significant  part  of  our  project  is  to 
teach  the  teachers  : Most  of  those  teachers  were  able  to  handle  their  computers  but  did  not  have  the  necessary 
skills  to  train  their  children  in  using  web-based  communication  tools.  So  we  had  to  organize  special  workshops 
and  lessons  to  train  teachers.  Since  that  we  are  organizing  special  workshops  in  cooperation  with  the  Pedagogic 
Institute  of  the  Archdiocese  of  Vienna. 
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Abstract:  This  work  stems  from  a project  funded  by  the  Arthritis  Research  Campaign 
(ARC)  to  develop  a web  site,  JointZone©,  for  the  study  of  Rheumatology  for  both 
undergraduate  medical  students  and  practicing  doctors.  The  educational  application 
incorporates  both  declarative  and  procedural  knowledge,  providing  students  with  a 
chance  to  acquire  knowledge  on  rheumatic  disorders  as  well  as  develop  clinical  reasoning 
skills  through  a series  of  graded  case  studies.  In  order  to  enhance  learning  and  reduce 
cognitive  overload,  which  can  be  associated  with  hypermedia  environments,  adaptive 
hypermedia  techniques  (Brusilovsky  2001)  have  been  integrated  into  the  core  of  a web- 
based  learning  environment.  This  paper  discusses  the  adaptive  features  employed  and  the 
pedagogical  rationale  involved  in  developing  the  web  site. 

1.  Introduction 

Resources  in  an  information-rich  hypermedia  environment  allow  users  to  browse  the  material  in  an  informal  way 
until  something  of  interest  commands  greater  attention.  Alternatively,  they  nay  search  the  material  in  a goal- 
directed  manner  to  find  the  information  they  need.  The  former  is  analogous  with  an  informal  learning  environment 
(Duchastel  1989)  where  one's  attention  skims  a wide  range  of  information.  However,  a corpus  of  information  in  a 
hypermedia  environment  is  not  necessarily  an  ideal  learning  environment  as  it  is  essentially  'non-pedagogical*,  and 
generally  provides  minimal  structural  support  (Duchastel  1992).  However,  this  mode  of  learning  can  be  effective  if 
the  reader  already  has  a fairly  well  developed  schema  (a  personal  construct  of  related  information)  for  the  material 
being  read.  A schema  acts  to  filter  incoming  information  and  provides  a framework  for  processing  information.  For 
a well-developed  schema,  paying  scant  attention  to  material  as  often  happens  when  browsing,  is  very  often  enough 
to  modify  a schema,  and  hence  learn.  Practitioners,  or  those  knowledgeable  in  a field  therefore,  may  use  this 
technique  in  a casual  manner  and  informally  upgrade  their  knowledge  of  a subject.  Novices  on  the  other  hand,  with 
less  well-developed  schemas,  are  more  easily  overwhelmed  and  quickly  suffer  from  cognitive  overload.  General 
browsing  therefore  is  not  necessarily  an  effective  learning  method  for  them  (Mayes  et  al.  1990).  Goal-directed 
learning  on  the  other  hand  is  more  focused,  affording  a greater  opportunity  to  attend  to  the  material.  It  is  important 
therefore  to  provide  guided  support,  or  scaffolding  for  novices  while  allowing  the  more  advanced  reader  to  move 
more  quickly  through  the  material.  The  nature  of  hypermedia  as  a searchable  network  of  information  can  enhance 
goal-directed  learning  through  effective  search  facilities  while  also  allowing  easy  access  to  a wide  variety  of 
material  for  browsing.  However,  both  these  functions  of  hypermedia  can  still  lead  to  a cognitive  overload  (Thuring 
et  al.  1995)  as  even  with  goal-directed  learning  a vast  array  of  linked  resources  can  be  presented  to  the  learner. 
Adaptive  hypermedia  is  one  approach  that  takes  into  account  the  reader's  knowledge  level  and  can  present  an 
appropriate  selection  or  an  adapted  set  of  content  nodes  - adaptive  presentation.  It  can  also  provide  adaptive 
navigational  support  offering  a selection  of  links  based  on  reader  knowledge  and,  or  readers'  browsing  history 
(Brusilovsky  2001).  In  both  these  cases,  adaptive  mechanisms  support  the  reader  by  offering  a guided  set  of 
documents/  or  links  with  the  aim  of  reducing  the  cognitive  overload,  thus  enhancing  the  reader's  ability  to  attend  to 
pertinent  documents  and  learn.  In  view  of  these  issues,  our  task  within  this  Rheumatology  project  has  been  to  place 
learning  materials  online  which  ensure  coherency  and  a good  accessibility  to  information  without  causing 
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disorientation.  Since  we  are  using  a closed  corpus  of  informatbn,  we  are  able  to  collect  information  from  the  user 
on  entry  which  is  used  to  develop  a supportive  learning  environment  through  adaptive  hypermedia  techniques. 

Three  major  roles  have  been  involved  in  the  development  of  this  project  to  encourage  effective  web-based 
learning.  There  is  the  web  author^  whose  major  responsibility  is  to  develop  a coherent  information  model  employing 
adaptive  hypermedia  techniques,  changing  a static  website  into  a dynamic  one  with  tailored  links  and  navigational 
aids  to  enhance  orientation  and  reduce  cognitive  load.  There  is  the  domain  expert,  whose  role  is  to  provide 
information  (declarative  knowledge)  on  various  topics  within  the  domain,  and  learning  material  that  encourage 
readers  to  structure  this  knowledge  by  constructing  relations  between  pieces  of  information  (structural  knowledge). 
In  addition,  an  opportunity  is  provided  for  readers  to  interpret  various  elements  within  a virtual  clinical  interview 
(procedural  knowledge),  emulating  the  activity  of  an  expert  practitioner  (fonassen  1991).  Finally,  there  is  the 
education  expert,  whose  role  is  to  understand  the  structure  of  the  domain,  anticipate  the  needs  of  readers  and  their 
potential  understanding  of  the  content  at  any  given  point.  This  role  offers  advice  on  learning  support  and 
scaffolding  strategies  that  enhance  readers'  ability  to  develop  their  current  schemas,  construct  new  ones  and  develop 
cognitive  skills  that  allow  them  to  apply  the  knowledge  they  have  within  their  professional  context. 

In  this  paper  we  shall  first  outline  the  implementation  of  JointZone©  and  our  pedagogical  approach  in 
developing  readers’  procedural  knowledge  through  a cognitive  apprenticeship  model.  Then  we  shall  explain  the 
adaptive  mechanism  employed  mJointZone. 

2.  Implementation 

When  constructing  a framework  for  the  JointZone©  application,  an  information  model  (see  Fig.  1)  was  developed  to 
feature  three  types  of  'separation’  (as  described  in  (Lowe  & Hall  1999,  p.67))  namely:  the  separation  of  presentation 
code  from  information  (content),  the  separation  of  links  from  content,  and  the  separation  of  logic  from  information. 

These  three  features  were  taken  into  consideration  in  the  design  in  order  to  reduce  the  authoring  effort  by 
re-using  the  resources  and  to  facilitate  the  maintenance  of  the  application.  As  shown  in  Fig.  1,  the  information  model 
outlines  the  different  entities  in  the  application  (content,  user  data,  link  data,  presentation  code  and  logical  data)  and 
they  are  implemented  using  different  web  technology  to  emulate  the  notion  of  separation  of  concerns. 


Figure  1:The  information  model  of  the  Jo/nfZone©  application 


The  latest  web  technologies  such  as  XML^,  XSL^  and  JavaServer  Pages  ™ have  been  used  to  implement  the  model. 
XML  is  used  to  store  and  structure  the  content  while  XSL  is  responsible  for  its  presentation  style.  This  allows  data 
to  be  separated  from  information  and  encourages  information  re-use.  For  example,  the  same  copy  of  XSL  style  sheet 

XML  (Extensible  Markup  Language)  is  the  markup  language  that  provides  a mechanism  for  inventing  new 
elements. 

[^]  XSL  (Extensible  Stylesheet  Language)  is  the  style  sheet  that  renders  the  presentation  of  XML  on  a web  browser. 
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can  be  used  to  display  more  than  one  XML  page.  Likewise,  only  one  copy  of  XML  content  is  maintained  even  if  it 
is  presented  in  different  ways  in  the  browser.  On  the  other  hand,  the  logical  bits  of  the  web  site,  i.e.  session  tracking 
of  users*  activity,  the  adaptivity  engine  and  the  search  mechanism  have  been  implemented  using  JavaServer  Pages 
(JSP).  The  JSP  technology  allows  the  integration  of  Java"^”  codes  into  HTML  pages  to  facilitate  dynamic  content 
generation,  which  makes  the  adaptation  of  links  and  contents  possible.  In  short,  all  the  components  described  above 
contribute  to  the  notion  of  separation  of  concern,  which  facilitates  maintenance. 

We  employ  link  databases  or  linkbases  (Hall  et  al.  1996)  to  store  links  separately  from  the  documents  to 
which  they  refer.  This  implies  the  separation  of  links  from  content.  Links  are  not  only  a means  to  traverse  the  web 
site  but  also  embody  the  semantic  relationship  between  the  nodes  (web  pages).  Proper  link  management  therefore  is 
essential  for  the  orientation  and  usability  of  the  application.  From  the  perspective  of  learning,  good  management  of 
links  can  reduce  users*  cognitive  overload  that  can  result  from  poor  orientation  and  navigation  facilities.  Hence  we 
employ  the  use  of  different  types  of  links  to  better  manage  the  linking  of  information  pieces.  We  adopted  the 
following  link  taxonomy  from  (Lowe  1996,  p.  33): 

Structural  links  illustrate  the  structural  layout  of  the  web  site.  These  include  ‘jump*  links  within  a page  to 
provide  direct  access  to  page  fragments,  ‘next*  and  ‘previous*  page  links  that  ensure  coherency  between  the 
documents,  and  the  provision  of  the  ‘table  of  contents*  to  give  a meaningful  overview  of  the  information  site. 

Referential  links  are  essentially  glossary  links  that  relate  a word  in  context  to  its  definition.  The  main 
difference  between  structural  links  and  referential  links  is  in  their  linking  mechanism.  Referential  links  are 
abstracted  from  the  idea  of  generic  linking  (Hall  et  al.  1996)  where  links  are  not  embedded  inside  the  documents  but 
stored  in  the  link  databases.  They  can  appear  anywhere  in  any  document  where  a glossary  word  exists.  Hence  these 
links  are  not  handcrafted  but  automatically  generated  using  an  offline  pre-processing  mechanism,  another  feature 
that  eases  authoring  effort.  The  glossary  links  are  highlighted  in  green  and  are  resolved  in  real  time  from  the  link 
database  once  followed  by  the  users.  The  effect  will  be  a pop  up  window  explaining  the  word  that  a user  has 
selected.  From  the  learning  perspective,  the  glossary  links  are  useful  for  learners  to  expand  their  knowledge  of  the 
terminology  within  the  knowledge  domain. 

The  idea  of  gtntxdXmg  associative  links  comes  from  the  usual  phenomenon  of  users  browsing  in  the  current 
context  to  find  more  information  about  a particular  concept.  For  example,  when  users  are  reading  a document  or 
solving  a problem  that  concerns  concept  ‘X*,  links  to  other  documents  are  suggested,  which  they  could  find  out 
more  about  ‘X*  in  other  documents  (Lowe  & Hall  1999).  Hence,  these  links  are  called  associative  links  because  they 
associate  two  or  more  documents  based  on  a common  concept.  Associative  links  are  stored  in  a linkbase  where  each 
of  them  is  indexed  or  retrieved  based  on  a keyword  or  concept.  From  a learning  perspective,  this  link  type  is  used  for 
goal  directed  learning  where  the  user  will  search  on  a concept  to  find  a series  of  related  documents. 

3.  Learning  Issues 

JointZone©  provides  a rich  source  of  material  to  enrich  users'  domain  (declarative)  knowledge  in:  basic  science, 
rheumatic  disorders,  approach  to  patient,  investigation  and  disease  management.  This  network  of  information  can 
be  used  by  all  readers  in  a browsing  or  goal-directed  learning  mode  and  is  suitable  for  medical  students,  and 
practising  doctors.  Information  can  be  found  using  the  site's  internal  search  engine  or  selecting  sections  from  the 
side  menu  bar  to  reveal  documents,  and  as  documents  are  selected,  their  linked  structure  becomes  visible  on  the  side 
menu.  However,  domain  competence  is  more  than  domain  knowledge,  it  also  comprises  a whole  range  of  skills  and 
for  medicine,  skills  based  competence  is  vital.  These  skills  are:  a)  physical,  e.g.  equipment  and  procedures  and,  b) 
cognitive,  e.g.  analysis,  interpretation  and  decision-making.  It  is  the  cognitive  skills  that  demand  a more 
sophisticated  learning  process  if  they  are  to  develop  (Kinshuk  2001)  and  with  adaptive  hypermedia  mechanisms  we 
are  able  to  match  the  content  level  with  the  student's  knowledge  through  interactive  case  studies.  A case  study 
approach  was  adopted  for  JointZone©  as  it  can  holistically  represent  the  complexity  of  clinical  reasoning.  Students 
are  then  able  to  develop  their  procedural  knowledge,  which  elaborates  how  doctors  approach  problems,  interpret 
clinical  information  and  make  decisions  (DesCoteaux  & Harasym  1998). 

3.1  Developing  Cognitive  Skills  through  Procedural  Knowledge 

Some  of  these  critical  skills  and  processes  that  medical  students  need  to  learn  are  decision-making,  reasoning  and 
problem  solving  (Greening  1998);  essentially  skills  that  relate  to  a clinical  diagnosis  and  management  which  is  a 
pivotal  activity  for  all  physicians  (Nkanginieme  1997).  To  develop  these  skills,  it  is  important  for  students  to 
practice  relating  disparate  pieces  of  information  from  a patient  within  the  clinical  context.  The  graded  case  studies 
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in  JointZone©  provide  clinical  medical  education  that  develops  students’  clinical  reasoning  skills  within  the  domain 
of  Rheumatology. 

The  advanced  case  studies  were  developed  to  encompass  a wide  range  of  information  available  in  the 
clinical  context.  These  were  generated  following  a heuristic  analysis  of  the  clinical  decision  making  process  by  an 
expert  practitioner.  Our  analysis  reflects  the  cognitive  skills  identified  by  Nkanginieme  ^Ikanginieme  1997), 
namely  that  physicians:  a)  obtain  and  recognise  symptoms  in  the  patient,  b)  identify  the  appropriate  system 
involved,  c)  speculate  on  the  pathological  processes,  d)  differentiate  pathological  processes,  e)  identify  the  possible 
causes  of  the  pathology,  f)  evaluate  all  pieces  of  information  and  make  a clinical  diagnosis.  To  support  student 
development  of  these  cognitive  skills,  explicit  procedural  steps  for  the  clinical  interview  were  presented:  the  referral 
letter,  patient  history,  the  examination,  investigations,  diagnosis  and  clinical  management.  On  working  through 
these  stages,  students  are  confronted  with  a wide  range  of  options,  such  as  results  from  investigations:  diagnostic 
imaging,  haematology,  immunology,  microbiology,  serology,  and  synovial  fluid  aspiration,  which  give  students 
information  pertinent  to  the  case  under  study.  The  clinical  observations  they  gather  by  working  through  the  case 
study  are  recorded  so  the  student  has  ongoing  access  to  this  information  as  they  proceed.  At  the  diagnostic  stage, 
students  are  given  a complete  record  of  the  examination  findings  and  investigations.  If  the  system  feels  insufficient 
or  inappropriate  information  has  been  gathered  this  will  be  pointed  out  to  the  student.  From  a list  of  possible 
diagnoses,  students  select  their  preference  based  on  their  interpretation  of  clinical  information  gathered.  They  then 
obtain  subsequent  feedback  on  why  that  diagnosis  is  likely  to  be  appropriate.  This  feedback,  and  the  feedback 
across  all  case  studies,  makes  explicit  the  expert’s  heuristics  of  clinical  reasoning.  This  is  an  attempt  to  share  that 
clinical  expertise  through  a modelling  or  a cognitive  apprenticeship  (Collins  et  al.  1989)  approach,  and  it  displays 
the  underlying  principles  and  rationale  of  the  clinical  diagnosis  and  completes  the  process.  This  enables  students  to 
develop  their  cognitive  skills,  providing  a framework  for  problem  solving  and  hence  clinical  reasoning. 

Case  studies  for  beginners  and  intermediate  level  students  are  much  narrower  in  scope  and  the  choices  are 
more  restricted.  However,  as  with  the  advanced  case  studies,  the  feedback  mechanism  serves  to  model  expert 
thinking  in  how  to  solve  these  problems.  The  goal  for  beginner  case  studies  is  to  recognise  the  value  of  particular 
pieces  of  clinical  information,  identify  the  pertinent  investigations  and  make  a diagnosis.  For  the  intermediate 
students,  the  goal  is  to  reassess  a case  in  which  the  original  diagnosis  is  under  question.  For  both  these  levels, 
students  are  given  scores  on  their  performance  in  reaching  a diagnosis.  A random  selection  of  items  is  penalised. 
Through  these  case  studies,  students  learn  to  recognise  and  interpret  the  value  and  relevance  of  disparate  pieces  of 
information  that  are  integral  to  any  clinical  scenario.  In  this  way  students  construct  their  own  understanding  of  the 
relationship  between  the  pieces  of  information  rather  than  having  it  imposed  upon  them.  This  encourages  a 
cognitive  flexibility  (Spiro  et  al.  1992)  that  is  vital  for  the  complex  world  of  the  physician. 

3.2  Bridging  the  Declarative  and  Procedural  Knowledge  Adaptively 

At  the  end  of  all  case  studies,  students  have  access  to  an  ’adaptive  reading  room’  (see  section  4)  where  links  to 
pertinent  domain  knowledge  are  made  available.  This  adaptivity  feature  gives  advanced  users  a wider  range  of 
pertinent  documents  than  those  with  less  knowledge.  An  example  of  this  is  for  the  concept  ’osteoporosis’,  advanced 
students  have  access  to  the  following  sub-concepts:  osteoporosis,  crush  fracture,  densitometry,  bone  scan,  and 
corticosteriod  while  intermediate  students  have  access  to  osteoporosis,  crush  fracture  and  densitometry,  and 
beginners  have  access  simply  to  osteoporosis.  However,  there  is  no  restriction  to  information  for  any  user  group 
should  they  wish  to  investigate  further.  The  list  of  suggested  reading  titles  is  identified  for  each  case  study  and 
presented  adaptively.  Each  document  title  in  the  list  will  indicate  the  amount  of  attention  that  reader  has  already 
paid  to  the  particular  document  through  a green  icon  indicator  alongside  the  title,  (see  Section  4).  Hence,  the 
‘adaptive  reading  room’  and  the  textual  glossary  links  throughout  all  cases  act  as  a bridge  between  declarative  and 
procedural  knowledge  in  the  application. 

4.  Integrating  Adaptive  Features 

We  have  briefly  mentioned  adaptive  case  studies  (for  beginner,  intermediate  and  advanced  level)  and  the  adaptive 
reading  room  in  the  previous  section.  For  some  background,  adaptive  hypermedia  (AH)  (Brusilovsky  2001)  systems 
essentially  revolve  around  the  idea  of  adapting  information  to  users’  differences  (knowledge,  background,  interest 
etc).  One  of  its  aims  is  to  protect  users  from  getting  lost  in  hyperspace  by  limiting  their  browsing  space  or  providing 
advice.  In  our  case,  the  integration  of  adaptive  hypermedia  will  hopefully  counteract  some  of  the  negative  effects  of 
self-exploratory  learning  on  the  web. 

Our  main  reason  for  integrating  adaptive  features  into  a web-based  learning  environment  is  to  cater  for 
users’  needs  that  arise  due  to  individual  differences.  This  can  be  accomplished  by  adapting  information  to  users  in 
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several  ways.  Firstly,  their  knowledge  level,  where  information  is  presented  based  on  users*  current  level  of 
knowledge.  Each  user’s  knowledge  level  is  evaluated  by  means  of  either  a prior  knowledge  test  upon  initial  entry  to 
the  application  or  a self-selecting  level  based  on  registration  details.  Secondly,  the  user's  browsing  history  provides 
personalized  browsing  aids  for  individuals,  so  that  they  can  concentrate  on  pages  they  have  not  read  adequately.  The 
browsing  history  is  a record  kept  by  the  system  to  keep  track  of  pages  that  have  been  read  by  the  users  and  how 
much  time  they  have  spent  on  those  pages.  The  system  estimates  the  degree  of  effort  a user  has  spent  on  a particular 
page  by  comparing  an  optimal  reading  time  (set  by  the  system  on  each  page)  with  the  actual  time  spent  by  the  user 
on  the  page.  In  order  to  make  the  estimation  more  accurate,  the  system  will  consider  the  individual’s  effective 
reading  speed  (Jackson  & McClelland  1979)  and  the  length  of  the  page  when  estimating  the  effort  that  has  been 
spent  on  each  page  (Ng  et  al.  2001).  Thirdly,  the  goal  of  the  user  is  employed,  by  pointing  users  to  pages  based  on 
their  selected  reading  goals.  These  short-term  goals  are  captured  explicitly  from  the  users’  input,  when  they  select  a 
reading  goal.  From  a learning  perspective,  short-term  goals  provide  guidance  in  a self-exploratory  learning 
environment  where  the  user’s  declarative  knowledge  of  the  domain  can  be  enhanced.  Instead  of  free  browsing,  users 
can  therefore  focus  on  a topic  or  goal. 

In  JointZone©  we  have  used  adaptive  features  based  on  users’  knowledge  level,  browsing  history  and 
reading  goals  mentioned  above.  In  adaptive  case  studies  (as  explain  in  Section  3.1),  users  are  directed  to  the  graded 
case  study  section  based  on  their  current  knowledge  level.  The  personalised  site  map  (see  Fig.  2)  provides  an 
overview  of  the  web  site  showing  all  the  web  pages  and  their  physical  layout  (headings,  subheadings)  on  the  site. 
Once  a user  has  registered  with  the  system,  their  browsing  history  and  time  spent  on  each  document  is  logged, 
enabling  this  personalised  site  map  to  be  presented.  Pages  are  annotated  with  a three-color-band  code  to  indicate 
their  reading  gauge  - a visual  representation  of  the  amount  of  time  a user  has  spent  reading  individual  pages.  The 
personalised  topic  map  is  similar  to  the  personalised  site  map,  but  instead  of  showing  all  pages  on  the  site,  it  shows 
only  the  pages  that  are  related  to  the  current  reading  topic.  To  initiate  this,  a user  is  required  to  choose  a reading 
topic  from  a pull  down  menu,  for  example  ‘gout’.  A topic-based  map  will  then  be  presented  showing  only  pages  that 
are  related  to  the  reading  topic  and  the  pages  are  highlighted  adaptively  according  to  the  user’s  browsing  history. 
Finally,  the  adaptive  reading  room  supports  user-driven  tasks.  Unlike  the  two  navigational  maps  mentioned  above, 
which  apply  to  general  free  browsing,  adaptive  reading  hints  are  integrated  with  the  interactive  case  studies.  When  a 
user  has  finished  working  on  a case  study,  he  or  she  might  want  to  know  more  about  the  concepts  related  to  that  case 
study.  The  adaptive  reading  room  hence  offers  a reading  list  that  is  adapted  to  the  user’s  knowledge  level,  as 
described  in  section  3.2 . 
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Figure  2:  A partial  screen  shot  of  the  personalised  site  map 

6.  Conclusion  and  Work 

The  overall  aim  of  this  web-based  learning  environment  was  to  provide  a rich  source  of  information  that  effectively 
supports  the  learner  through  the  adaptive  features  and  allows  users  to  engage  with  material  more  effectively.  We  set 
out  to  achieve  this  through  careful  hypermedia  authoring  while  maintaining  a constant  awareness  of  users’  needs. 
Our  main  strategies  for  developing  an  effective  learning  environment  on  the  web  were  to: 


• provide  a personalised  information  delivery  system  that  through  adaptive  hypermedia  technique,  allowed  users 
to  maintain  an  overview  of  the  documents  they  had  read  or  not  read.  We  have  also  linked  the  procedural  and 
declarative  knowledge  in  the  application  adaptively,  to  prevent  cognitive  overload  and  facilitate  the  process  of 
learning. 

• design  learning  materials  that  develop  medical  students'  cognitive  skills  in  clinical  reasoning  via,  graded 
interactive  case  studies. 

• reduce  authoring  effort  and  increase  maintainability  of  the  application  through  the  separation  of  concerns  in 
authoring. 

• increase  the  usability  and  accessibility  of  information  through  various  linking  strategies. 

A comprehensive  evaluation  of  the  adaptive  features  is  currently  being  undertaken  to  scrutinize  their 
effectiveness  in  terms  of  learning  and  information  access.  This  evaluation  will  give  us  some  insights  into  whether 
adaptive  web-based  learning  really  benefits  the  students  or  whether  it  is  a mere  technology-driven  technique  in  a 
learning  paradigm. 
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Abstract:  The  Swiss  Federal  Institute  of  Technology  in  Lausanne  is  currently  developing  an 
environment  for  Web-based  experimentation  in  engineering  education.  This  environment 
aims  at  providing  students  with  the  facilities  to  reinforce  their  learning  process  by  carrying 
out  practical  experimentation.  This  paper  focuses  on  the  concept  of  Laboratory  Journal,  a 
component  of  the  environment,  which  has  been  developed  to  support  knowledge  acquisition 
and  reinforcement  in  a collaborative  way.  The  Laboratory  Journal  is  designed  and 
implemented  as  an  integrated  space  that  supplies  the  needs  for  collaborative  learning  and 
fulfils  hands-on  session  requirements,  using  the  latest  communication  and  information 
technologies.  It  has  been  developed  as  a Web-based  application  with  client/server 
architecture.  The  implementation  is  based  on  the  use  of  Java  and  XML  technologies. 


l.Introduction 

The  rapid  development  of  computer  networking  and  the  Internet  in  the  last  decade  has  provided  new 
possibilities  and  also  new  challenges  for  designing  and  deploying  distance  and  collaborative  learning  systems. 
Nowadays,  one  of  the  useful  and  interesting  trends  to  support  active  learning  is  to  expand  the  available 
educational  resources  by  providing  virtual  and  real  experimentation  facilities  (Gillet,  D.  & Farkas,  G.,  2001). 
This  is  especially  important  in  engineering  education  where  the  practical  approach  to  learning  is  as  necessary  as 
the  theoretical  one.  Thus,  there  is  now  a growing  demand  of  developing  remote  experimentation  environments 
for  educational  purposes. 

The  deployment  of  Web-based  experimentation  facilities  is  carried  out  at  the  Swiss  Federal  Institute  of 
Technology  in  Lausanne  (EPFL)  in  the  framework  of  the  eMersion  project.  This  project  aims  at  providing  an 
environment  for  engineering  education  based  on  new  learning  technologies  resources,  which  facilitates 
experiments  through  Web-based  simulation  and/or  remote  manipulation  of  laboratory  setups.  eMersion  proposed 
an  integrated  environment  that  allows  students  to  execute  remotely  Experimentation  Modules  over  the  Internet  to 
support  their  learning  activities.  The  environment  has  a Cockpit-like  graphical  user  interface.  This  so-called 
Cockpit  environment  contains  all  the  components  necessary  to  complete  successful  experimentations.  In 
particular,  the  Cockpit  includes  three  main  parts:  the  Experimentation  Console,  the  Toolkit  Console  and  the 
Laboratory  Journal.  The  Experimentation  Console  can  be  regarded  as  the  interaction  part  that  enables  the  actual 
realization  of  experiments.  The  Laboratory  Journal  constitutes  the  collaboration  part  that  facilitates  reporting  as 
well  as  knowledge  integration  and  sharing.  The  Toolkit  Console  provides  students  tools  to  carry  out  interactive 
design  and  analysis  activities  related  to  the  experiment.  The  portal  of  the  Cockpit  environment  for  automatic 
control  modules  is  shown  in  (Figure  1).  Currently,  the  environment  is  used  for  pilot-courses  in  automatic  control, 
fluid  mechanics  and  biomechanics  at  the  EPFL. 

This  paper  introduces  the  collaborative  part  of  the  Cockpit  namely  the  Laboratory  Journal  It  is 
organized  as  follows:  section  2 discusses  about  the  concepts  of  collaborative  learning  in  engineering  education. 
Sections  3 and  4 present  the  features  and  implementation  of  the  Laboratory  Journal.  Section  5 shows  the  user 
case  validation.  Section  6 talks  about  the  future  work  and  section  7 concludes  the  paper. 
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Figure  1 : The  portal  screen  for  the  automatic  control  module. 


2.  Collaborative  Learning  in  engineering  education 

CSCL  (Computer  Supported  Collaborative  Learning)  studies  how  computer  science  can  support 
learning  processes  promoted  by  the  collaborative  efforts  of  students  working  on  a given  task  (Santoro,  F.  M.  et 
al.,  2000).  CSCL  supports  active  collaborations.  That  means  it  not  only  provides  the  delivery  mechanism  for 
collaborative  interactions  provided  by  CSCW  (Computer  Supported  Collaborative  Work)  but  also  provides 
additional  controls  and  services  required  to  guide  collaborative  learning  in  an  active  fashion,  based  on  the 
requirements  of  the  peers  involved  in  the  interactive  learning  (Clemson  University  Notes).  Collaborative 
learning  not  only  helps  group  works  but  also  encourages  individual’s  activities  and  contributions  in  the  group. 
Each  member  in  a group  has  some  specific  authorities,  responsibilities  and  roles  when  taking  part  in  the  group  ‘s 
activities. 

Taking  ideas  from  CSCL,  many  collaborative  learning  environments  (CLEs)  have  been  developed  and 
proposed  for  the  tutor  and  student  to  work  together.  These  CLEs  are  generally  made  of  a collection  of  tools  and 
services  that  supports  teaching  and  learning  processes  to  help  tutors  and  students  show  their  opinions  without 
face-to-face  judgement.  Nowadays,  the  WWW  is  becoming  an  ideal  platform  for  developing  CLEs. 

In  engineering  education,  the  practical  part  is  as  important  as  the  theoretical  part.  In  the  spirit  of  flexible 
learning  (Gillet,  D.  & Farkas,  G.,  2001),  the  students  should  be  allowed  to  carry  out  the  experimental  work  in 
laboratories  as  well  as  from  distance.  Hence,  to  facilitate  the  practical  learning,  the  physical  and  virtual 
experimentation  setups  should  be  both  supported.  Those  are  used  by  the  tutor  for  demonstrating  and  also  by  the 
students  for  hands-on  exercises  (Gillet,  D.  et  al.,  2001).  The  virtual  experimentation  could  be  a combination  of 
graphical  animations  and  numerical  simulation  tools.  During  the  experimentation  process,  students  need  a shared 
working  space  (or  shared  workspace),  i.e.  the  collaborative  space,  where  they  can  take  notes  about  their 
experimental  works,  store  results  and  observations,  discuss,  share  information  and  materials,  ask  or  provide 
assistance  with  other  students. 

In  academia  as  well  as  in  industry,  there  are  a lot  of  simulation  tools,  which  support  practical  activities 
in  engineering  education.  For  example,  in  (Mateos  et  al.,  2001)  PROSIMAX,  a Windows  based  tool  for  testing 
control  programs  implemented  on  different  control  devices,  is  presented.  It  is  intended  to  minimize  the  main 
drawbacks  of  classical  tools  used  for  test  in  educational  environments.  DEVRE  is  presented  in  (Ridao,  P.  et  al., 
2001)  with  the  aim  to  assist  engineers  during  the  software  development  and  testing  in  the  laboratory  prior  to  real 
experiments.  (Sepe  et  al.,  2001)  has  constructed  VE-LAB,  a Web-based  virtual  engineering  laboratory  for 
collaborative  experimentations  on  a hybrid  electric  vehicle  starter.  Using  these  environments,  users  can  perform 
virtual  experimentations.  DEVRE  and  VE-LAB  are  built  in  a distributed  environment  (Web-based  environment 
in  case  of  VE-LAB).  However,  users  have  no  way  to  collaborate  (the  collaboration  is  poor  supported  in  VE- 
LAB)  and/or  to  share,  to  exchange  their  ideas,  experimental  results  and  information.  In  other  words,  these 
environments  don’t  support  working  in  groups. 

Recently,  there  is  a growing  interest  to  support  collaborative  work  over  the  Internet.  Maybe  one  of  the 
most  popular  collaborative  tools  in  academia  is  BSCW  (Basic  Support  for  Cooperative  Work)  (Kldckner,  2000). 
This  is  a shared  workspace  where  users  can  share  information;  collaboratively  perform  their  activities  over  the 
Web.  However,  BSCW  doesn’t  support  task  manager;  i.e.  the  capability  of  the  supervisor  to  fragment  a big 
project  into  small  tasks  and  distribute  to  different  groups,  assign  responsibilities  to  group  members,  monitor  the 
progress  of  groups  when  performing  tasks.  In  addition,  the  synchronous  collaboration  among  users  in  BSCW  is 
not  much  either  supported. 

The  goal  of  the  INVITE  project  (Bouras  et  al.,  2001)  is  to  build  a platform  for  synchronous  tele- 
learning  that  can  be  interfaced  with  standardized  content  management  and/or  industrial  management  system. 
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INVITE  is  a collaborative  virtual  environment  that  supports  e-leaming.  It  is  based  on  distributed  virtual 
environment  technologies.  However,  the  avatar  representations  in  INVITE  are  not  really  comfortable  for 
information  sharing  and  discussions  for  engineering  students. 

Edelson  et  al.  from  Northwestern  University  has  developed  the  CoVis  Collaboratory  Notebook  (Daniel 
C.  Edelson,  D.  Kevin  O’  Neil,  1994),  which  is  a shared,  hypermedia  database  that  supports  communication  and 
collaboration  both  locally  and  remotely  over  the  Internet.  Nevertheless,  the  synchronous  communication  in 
CoVis  is  not  supported.  And  in  fact,  CoVis  Notebook  is  used  only  for  high  school  science  learning.  Thus,  the 
idea  of  collaboration  in  this  project  is  still  simple.  It  is  mainly  a medium  for  students  to  record  their  thoughts  and 
actions  as  they  perform  scientific  inquiry. 

Other  Web-based  collaborative  tools  or  architectures  could  be  considered  in  (Fredj  Diidi  oc  Gustaf 
Neumann,  1999;  Beca  et  al.,  1998;  Dan  Suthers  & Dan  Jones,  1997;  TeamWave;  Lee,  M.  J.  et  al.,  2000). 
Because  all  of  the  collaborative  tools  presented  above  are  not  well  designed  for  engineering  education,  remote 
experimentation  and  simulation  cannot  be  sustained. 

As  presented  above,  there  are  a lot  of  simulation/manipulation  as  well  as  collaboration  tools  in 
academia  and  industry.  But  so  far,  it  seems  that  there  is  no  product  that  integrates  these  two  aspects.  In  eMersion 
project,  the  goal  is  to  develop  a global  and  flexible  infrastructure  that  supports  the  remote  execution  of 
Experimentation  Modules  over  the  Internet  and  the  collaboration  between  actors  (students,  tutors,  pedagogical 
team,  etc.).  One  important  thing  to  notice  is  that  these  two  parts  of  the  Cockpit  environment  (Experimentation 
Console  and  Laboratory  Journal,  or  Interaction  space  and  Collaboration  space)  are  connected  so  that  materials 
(files,  experimental  results,  configurations,  etc.)  can  be  automatically  transferred  from  one  part  to  another.  By 
this  way,  the  collaboration  between  the  student  and  the  tutor  as  well  as  between  the  students  when  performing 
remote  experimentations  is  well  supported.  The  next  sections  will  describe  the  features  and  the  implementation 
of  the  Laboratory  Journal. 


3.  Laboratory  Journal  features 

The  Laboratory  Journal  is  developed  based  on  the  concept  of  electronic  journal.  Normally,  text, 
sketches,  graphs,  tables,  signatures,  images  and  other  kinds  of  data  are  “written”  (or  “recorded”)  on  electronic 
journal  “pages”,  which  can  be  read  and  navigated  (Daniel  C.  Edelson,  D.  Kevin  O’  Neil,  1994).  The  electronic 
journal  allows  easily  input  of  scientific  data  (for  example,  the  experimental  results  in  the  form  of  image  or 
mathematical  formula).  As  indicated  in  (Al  Geist  & Noel  Nachtigal,  2001),  an  electronic  journal  has  many 
advantages,  it  can  be  shared  by  a group  of  users,  can  be  accessed  remotely,  cannot  be  lost  (if  carefully  backed  up 
but  this  task  is  easily  done),  can  be  searched  for  information,  can  include  multimedia  or  hyperlinks  to  other 
information,  etc. 

In  our  system,  the  Laboratory  Journal  is  constructed  as  a collaborative  working  space  (or  collaborative 
workspace  for  simplicity)  for  students  when  performing  Web-based  experimentations.  In  other  words,  it  is  a 
Web-based  medium  where  engineering  students  can  remotely  record  aspects  of  experiments  that  are  conducted 
in  the  framework  of  a virtual  practical  exercise.  The  Laboratory  Journal  lets  students  access  their  preparation 
materials,  store  and  post-process  their  experimental  results.  Logically,  we  have  two  different  spaces  for  working. 
The  private  Journal  components  can  only  read  or  modified  by  its  single  owner,  and  group  Journal  space  is  the 
shared  space  for  a group.  The  student  and  tutor  can  work  on  their  private  Journals  before  sharing  them  with 
others.  Each  group  Journal  is  dedicated  to  one  group  for  one  experimentation  module.  Authorization  or  access 
rights  relate  to  the  operations  available  to  the  user.  The  authorization  is  changed  from  one  Journal  to  another.  It 
depends  on  the  role  as  well  as  on  the  context.  To  prevent  conflicts  when  co-editing  the  Journal,  the  locking 
mechanism  is  provided. 

The  Laboratory  Journal  provides  the  possibility  for  documenting  and  reporting,  and  it  facilitates  the  key 
activities  of  knowledge  integration  and  knowledge  sharing  (Gillet,  D.  et  al.,  2002).  It  takes  many  advantaged 
features  from  structured  document  concepts  (Adre,  Furuta  & Quilt,  1989).  The  structured  document  permits  the 
flexible  reuse  of  the  document  and  of  its  elements.  The  relationships  between  elements  are  based  on  the 
document’s  logical  structure  and  not  the  physical  appearance  of  the  components  displayed  on  the  page.  Students 
just  concern  with  the  content  of  the  document.  In  the  Cockpit  environment,  they  use  the  Laboratory  Journal  to 
collaboratively  prepare  reports.  Thanks  to  the  hierarchical  structure  of  the  Journal  page,  users  can  co-edit  the 
whole  or  just  some  parts  of  the  report.  Also  thanks  to  the  DTD  (Document  Type  Definition)  (Harold,  E.  1999) 
defined  for  the  Cockpit  and  the  Laboratory  Journal  interface,  the  Laboratory  Journal  ‘s  views  can  be  changed 
depending  on  the  user’s  profiles  and  the  report  can  be  automatically  generated  based  on  the  inputs  from  the  user. 

In  the  Laboratory  Journal,  many  kinds  of  elements  can  be  defined,  for  instance:  text,  image,  table,  etc. 
The  artifacts  in  the  workspace  are  called  fragments^  which  can  be  composed  of  objects  and  used  to  construct  the 
report.  The  text  fragment  can  be  edited  by  hand  or  extracted  from  documents  imported  by  different  students.  The 
imported  documents  are  transformed  into  structured  document  format.  When  the  imported  documents  cannot  be 


transformed,  they  remain  as  attached  documents.  Other  kinds  of  fragment  can  be  uploaded  from  a file  server  or 
transferred  from  the  Experimentation  Console  using  the  Data  Tunnel,  which  is  constructed  as  a tunnel  for  data 
transmission  between  different  parts  of  the  Cockpit  environment.  In  addition,  the  student  can  create  and  edit 
notes,  which  will  be  used  in  the  report,  to  memorize  the  observations  or  anything  else  when  performing  the 
experiment. 

The  report  is  constructed  by  inserting  fragments.  The  fragment  is  chosen  in  the  workspace  and  then 
inserted  to  the  report.  The  delete,  modify  possibility  is  also  allowed  in  the  report  editing  space.  Every  time  the 
report  is  modified,  the  previous  version  is  saved.  This  version  control  functionality  helps  students  to  review  their 
working  history.  After  being  edited,  the  report  is  sent  to  the  tutor  as  HTML  or  PDF  file  by  email.  After  being 
subiiiilleu  lu  the  lulur,  the  gruup  repui  l camiul  be  luuuifieu  auyiiiOie. 

The  member  of  a group  communicates  with  other  members  in  the  same  group  or  with  the  tutor  either 
directly  through  message  exchange  or  indirectly  through  annotations,  which  are  added  to  the  fragments  (or  the 
shared  artifacts)  in  the  workspace.  The  annotation  provides  the  meta- information  about  the  fragment.  For 
example,  it  could  help  students  and  tutors  know  that  this  fragment  was  created  by  whom  and  when.  Different 
annotations  from  different  users  can  be  attached  to  the  same  fragment.  The  communication  between  members  is 
also  enabled  by  the  way  the  student  provides  the  “write”  permission  to  other  members  in  the  same  group  or  the 
“read/write”  access  to  the  tutor.  Thus,  the  tutor  or  other  members  can  also  modify  the  fragment  or  to  give  some 
feedbacks  to  the  student-author.  The  communication  mechanism  is  in  fact  provides  students  and  tutors  the 
awareness  information  in  collaboration  process.  The  general  concept  of  awareness  was  defined  by  (Paul  Dourish 
and  Victoria  Belloti,  1992)  as  an  understanding  of  the  activities  of  others,  which  provides  the  context  for  each 
member’s  activities. 

A group  task  manager  is  also  constructed  to  solve  the  need  to  fragment  a whole  task  (for  example, 
group  1 prepare  a report  for  experimentation  module  A)  into  specialized  smaller  sub-tasks  that  are  to  be 
performed  by  different  students  working  within  a single  group  (for  example,  who  does  what).  Common  tasks 
may  be  shared  and  manipulated  independently  by  a number  of  users  with  different  kinds  of  access  control.  With 
this  tool,  students  monitor  their  progress  or  organize  cooperative  tasks;  for  example,  create,  refine  or  modify  a 
task.  To  create  a task  means  to  define  this  task,  to  allocate  some  resources  and  assign  it  to  some  specific  students. 


4.Laboratory  Journal  implementation 

(Figure  2)  shows  the  Cockpit  environment  when  performing  experimentation  module  about  electrical 
drivers.  In  the  top  of  the  figure  is  the  Navigation  and  Supervision  Area.  In  this  part,  we  could  find  the  objectives 
and  the  status  of  the  task  currently  being  processed  (for  example,  the  task  is  finished  or  currently  implemented). 
The  Protocol  space  in  this  Area  describes  step-by-step  procedures  necessary  to  perform  the  experimental  work. 
The  Navigation  and  Supervision  Area  also  provides  the  possibility  to  access  other  relevant  supplementary 
information  such  as  reminders  or  links  representing  the  theory,  a description  of  the  environment,  etc.  The  bottom 
left  of  the  figure  represents  the  Experimentation  Console  (the  higher  part)  and  the  Toolkit  Console  (the  lower 
one).  The  Laboratory  Journal  is  showed  in  the  bottom  right  of  the  figure. 
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Figure  2:  The  Cockpit  environment  for  the  automatic  control  module. 
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To  represent  the  concept  of  structured  document,  XML  (extensible  Markup  Language)  (Harold,  E., 
1999)  is  chosen.  It  is  the  universal  format  for  structured  documents  and  data  on  the  Web.  XML  data  only 
contains  data;  that  means  what  is  to  be  presented.  The  presentation  part  is  done  by  stylesheets.  Using  a 
stylesheet,  the  correct  style  will  then  be  generated  in  different  formats  such  as  PDF  or  HTML. 

The  Laboratory  Journal  is  constructed  using  Web-based  client-server  architecture.  Users  can  use  Java- 
enabled  browsers  to  access  Web  pages,  which  contain  codes  (Java  codes  and  scripting  codes  such  as  Javascript) 
used  to  invoke  services.  The  Web  pages  in  our  system  are  in  fact  the  JSP  (JavaServer  Pages)  (Aveda  et  al.  2000) 
ones.  JSP  pages  can  provide  the  full  power  of  the  Java  language  to  process  and  transform  XML  documents.  The 
DTD  (Ducuiiiciii  Type  Dcfiiiiuuii)  is  deriiicu  at  tl'ic  server  side.  The  server  side  is  an  exLended  HTTP  server,  i.e. 
an  HTTP  server  with  some  application  servers  (written  as  Java  applications)  to  enrich  processing  functionalities. 
In  the  Cockpit  environment,  a TOMCAT  version  4.0  from  the  Apache  Software  Foundation  is  used  as  the  Web 
server.  A relational  database  (mySQL  (see  MySQL)  in  our  case)  and  a file  system  are  used  to  store  data  in  the 
central  server.  Depending  on  the  requests  from  the  clients,  the  appropriate  XML  files  are  created.  The  XML  files 
are  then  transformed  by  XSLT  stylesheet  (see  XSLT)  into  formatting  object  trees.  The  object  tree  is  parsed  by 
Java  codes  to  create  objects  in  the  Journal. 


5.  User  case  validation 

Since  October  2001,  a group  of  25  EPFL  volunteer  students  for  each  semester  has  used  the  Cockpit 
environment  to  carry  out  laboratory  assignments  in  automatic  control  under  observation.  They  were  asked  some 
preliminary  questions  listed  in  the  Cockpit  protocol  and  then  the  answers  are  added  to  the  Laboratory  Journal  by 
creating  notes  or  attaching  documents.  This  is  considered  as  the  pre-lab  task  and  when  finished,  it  is  made 
visible  (by  providing  permissions)  so  the  tutor  is  able  to  grade  it. 

The  answers  are  corrected  and  annotated  by  the  tutor  within  the  Laboratory  Journal  context.  The 
students  are  then  authorized  to  perform  the  practical  part  of  the  experimental  assignments  using  the 
Experimentation  Console.  (Figure  2)  illustrates  a laboratory  assignment,  in  which  the  student  performs  the 
experimentation  in  modeling  and  digital  controlling  of  servo  drive. 

During  performing  experimentations,  the  student  takes  notes  in  the  Journal.  The  experimental  results 
and  observations  are  saved  in  the  workspace  in  order  to  create  the  reports. 

The  volunteers  have  been  observed  and  interviewed  by  pedagogues  to  evaluate  their  reactions  regarding 
the  Web-based  environment  and  the  flexible  learning  approach.  The  evaluation  detail  is  described  in  (Gillet,  D. 
et  al.,  2002). 


6.  Future  work 

Currently,  the  Laboratory  Journal  is  in  the  phase  of  design  and  implementation.  There  are  still  many 
theoretical  and  technical  problems  to  deal  with  when  developing  the  system.  The  future  version  of  the 
Laboratory  Journal  will  frilly  support  the  collaboration  when  performing  activities  on  Journal  pages  and 
Experimentation  Console.  The  ability  to  generate  automatically  the  report  based  on  the  user's  inputs  (thanks 
mainly  to  the  well-defined  XML  files  stored  in  the  server)  will  be  implemented  as  well. 


7.  Concluding  remarks 

In  this  paper  we  described  the  Cockpit  environment  as  a paradigm  introduced  at  EPFL  in  engineering 
education  to  sustain  hands-on  learning  activities.  We  have  defined  the  Laboratory  Journal,  the  collaboration  part 
of  the  Cockpit,  as  an  infrastructure  that  supports  collaborations  among  students  when  performing  Web-based 
experimentation.  Our  system  relies  on  a variety  of  flexible,  scaleable  and  platform  independent  components.  We 
have  shown  the  concepts  related  to  the  Laboratory  Journal,  its  features  and  implementation.  Currently,  the 
environment  supports  three  courses  taught  to  students  in  automatic  control,  biomechanics  and  fluid  mechanics. 
An  evaluation  is  currently  carried  out  in  collaboration  with  a usability  team  and  pedagogical  experts  in  order  to 
assess  the  effectiveness  of  the  deployed  solutions. 
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Abstract:  This  paper  describes  the  design  and  implementation  of  a real  time  on- 
line collaboration  tool,  EDUCO.  The  main  focus  is  to  demonstrate  how  the  tool  is 
applied  to  a cal  life  on-line  distance-education  course,  discuss  the  preliminary 
research  findings  of  qualitative  empirical  study,  and  propose  directions  for  future 
work  and  implications.  The  preliminary  results  show  that  some  of  the  features  used 
in  this  tool  can  be  directly  applied  to  a wider  context  of  modem  computer-based 
learning  environments. 

Introduction 

The  main  objective  of  the  EDUTECH  project  is  to  develop  methods  for  applying  probabilistic 
modeling  techniques,  such  as  Bayesian  network  models,  in  building  and  using  personalized,  adaptive 
software  components  for  digital  learning  environments.  The  research  focuses  on  developing  intelligent 
autonomous  but  interoperabile  modules  for  educational  purposes  from  assessment  techniques  to  the 
support  of  co-operative  activities  and  personalized  adaptive  learning  materials. 

1\[Q  first  phase  of  EDUTECH  study  period  (1999-2000)  concentrated  on  the  development  of 

intelligent  questionnaire  software  EDUFORM  (Kurhila,  Miettinen,  Niemivirta,  Nokelainen,  Silander 
and  Tirri  2001).  EDUFORM  allows  for  an  adaptive  and  dynamic  optimization  of  questionnaire 
propositions  and  profiling  of  learners  on-line.  This  software  module  is  applied  to  varying  types  of 
assessment  applications  both  in  business  and  educational  domains. 

The  ongoing  second  (2001 -2002)  of  the  project  has  two  sub -phases.  Firstly  the  research 
group  has  developed  an  experimental  version  of  a tool  for  on-line  computer  supported  collaborative 
learning.  The  latter  ongoing  part  of  the  research  is  focused  on  utilizing  and  combining  the  co-operative 
aspects  of  profiling  information  gained  from  EDUFORM  by  providing  mechanisms  for  intelligent 
matching  with  co-students. 

The  major  goal  of  this  paper  is  to  describe  the  design  and  implementation  of  a real  time  on-line 
collaboiation  tool,  EDUCO.  The  main  focus  is  to  demonstrate  how  the  tool  is  applied  to  a real  life  on- 
line distance -education  course,  discuss  the  preliminary  research  findings  of  qualitative  empirical  study, 
and  propose  directions  for  future  work  and  implications. 

The  Role  of  Real  Time  Interaction  in  Web-Based  Learning 

The  concept  of  computer  supportive  collaborative  learning  (CSCL)  consisting  intentional  active 
learning,  process  aspect  knowledge  and  methods  of  knowledge-building  resources  necessitates  a 
flexible  and  synchronous  learning  situation  (Scardamalia  and  Bereiter  1994).  When  we  compare  these 
conditions  for  learning  and  the  implications  from  CSILE  and  Knowledge  Forum  to  modem  platforms 
for  web-based  learning,  such  as  WebCT  and  Learning  Space,  we  must  admit  that  the  real  time 
component  of  on-line  learning  process  is  still  very  static  and  inflexible. 

How,  then,  the  task-related  social  interaction,  which  should  play  a significant  role  in  a learning 
process,  is  implemented  in  available  educational  software?  Comparison  between  the  software  is 
relatively  simple  to  carry  out  due  to  the  fact  that  popular  platforms  have  relatively  similar  functionality. 
They  all  provide  the  learners  with  tools  to  go  through  the  learning  material,  perform  tasks  like 
answering  to  quizzes  and  submitting  assignments,  and  communicating  with  peers  or  teachers  (see  e.g. 
Landon  2001).  However,  in  these  “traditionaf*  web-based  learning  environments  the  real  time  social 
interaction  is  limited  to  a simple  chat  and  web  board  services.  When  a mer  is  actually  using  the 
platform,  the  learning  experience  can  be  perceived  as  a lonesome  activity,  since  there  is  no  feelingof 
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other  live  learners  or  teachers  within  the  environment.  The  concept  of  social  navigation  (Munro,  Hook 
and  Benyon  1999)  provides  one  viable  solution  to  the  problem. 

Considering  social  navigation  from  the  standard  perspective,  the  concept  means  providing  the 
users  with  information  from  the  actions  of  other  users  of  the  environment.  For  example,  showing  the 
hits  for  different  web  pages  is  a simple  form  of  providing  means  for  social  navigation.  In  the  seminal 
book  of  social  navigation,  Munro  et  al.  (1999)  use  an  on-line  grocery  store  as  an  example:  if  people 
visiting  the  store  are  given  recommendations  what  other  people  have  bought,  it  is  a form  of  indirect 
social  navigation.  If  a shopper  in  the  grocery  store  has  a sense  of  other  people  moving  about  the  store 
and  can  engage  in  seeking  e.g.  assistance,  it  is  a case  of  direct  social  navigation 

Experiments  with  social  navigation  have  mostly  fallen  into  the  category  of  indirect  social 
navigation.  Still,  in  some  sense  the  feeling  of  other  users  can  be  achieved  even  if  the  actions  are  not 
delivered  to  other  users  in  real  time.  For  example,  in  CoWeb  (Dieberger  1999)  the  feeling  of  aliveness 
is  carried  out  with  persistent  discussion  spaces  that  allow  any  user  to  modify  any  hyperdocument. 
CoWeb  uses  indirect  social  navigation  in  generating  footprint  markers  that  indicate  the  amount  of 
recent  traffic  to  pages  and  when  the  pages  were  last  modified.  The  markers  provide  an  additional  sense 
of  awareness  of  activities  in  the  information  space  and  thus  increase  the  sense  of  the  CoWeb  as  a social 
“place”. 

However,  the  indirect  social  navigation  approach  does  not  add  to  the  need  for  really  /ive users. 
Therefore,  we  have  implemented  a tool  for  real  time  on-line  collaboration  called  EDUCO.  EDUCO 
appears  to  the  users  as  a visual  collection  of  web  sites,  where  the  users  can  navigate  the  documents  and 
see  when  other  users  are  navigating  those  same  documents.  Users  can  engage  in  chat  simple  by 
clicking  the  dots  representing  users  in  EDUCO.  Furthermore,  EDUCO  users  are  able  to  set  “alarms” 
which  are  triggered  when  a certain  person,  group  member  (Hoppe  and  Ploezner  1999)  or  any  user 
arrives  to  the  systems  or  to  a certain  document.  Search  functions  to  locate  users  or  documents  with 
desired  keywords  in  the  title  exist  as  well. 

The  Design  of  the  Study 

The  first  experiment  with  EDUCO  was  a course  given  at  the  Department  of  Computer  Science, 
University  of  Helsinki, Finland.  The  topic  of  the  course  was  Web-based  learning.  The  first  meeting  was 
an  introduction  to  the  topic  and  to  EDUCO.  The  following  weekly  or  bi-weekly  sessions  were  carried 
out  on-line  with  EDUCO  as  an  interaction  tool  until  the  final  day  when  students  presented  their  papers. 
Students  (N=24)  taking  part  of  the  course  were  expected  to  form  a group  of  two,  pick  a topic  in  the 
field  of  web-based  learning  and  prepare  a paper  and  a presentation  about  the  topic.  In  addition,  there 
were  several  time -limited  mini-tasks  given  by  the  teacher  on-line,  and  tasks  where  the  students  were 
supposed  to  comment  on  research  papers  on  web -based  learning.  The  data  set  was  gathered  in  three 
stages:  (1)  Pre  test  on  the  first  day  of  the  course  measured  motivational  level  and  learning  strategies,  (2) 
user  log  was  gathered  during  the  course,  and  (3)  post  test  after  the  course  measured  how  students’ 
expectations  faced  the  reality. 

Pre  Test 

Motivational  profiling  in  this  study  is  based  on  the  Motivated  Strategies  for  Learning 
questionnaire  (MSLQ),  which  is  developed  on  the  basis  of  motivational  expectancy  model  (Garcia  and 
Pintrich  1994).  MSLQ  measures  both  motivational  factors  and  learning  strategies  and  has  been  adapted 
to  the  research  field  of  Finnish  vocational  and  higher  education  (Ruohotie  2000).  The  motivation 
section  (A)  of  the  questionnaire  consists  of  17  items  that  were  used  to  assess  students’  value  for  a 
course,  their  beliefs  about  their  skill  to  succeed  in  the  course,  and  their  anxiety  about  tests  in  the  course. 
The  learning  strategy  section  (B)  includes  13  items  regarding  student’s  use  of  different  cognitive, 
metacognitive  and  resource  management  strategies.  A 5-point  Likert  type  scale  ranging  from  1 (“Not  at 
all  true  of  me”)  to  5 (“Very  true  of  me”)  was  used  for  all  items.  Students  were  divided  into  three  groups 
based  on  their  motivational  level  scores  from  “A”  section  of  the  questionnaire:  Group  1 (blue,  N=10) 
characteristics:  Extrinsic  goal  orientation,  test  anxiety  and  meaningfullness  of  studies.  Group  2 (green, 
N=8)  characteristics:  Efficiency  beliefs,  intrinsic  goal  orientation  and  meaningfiillness  of  studies. 
Group  i(red,  N=6)  characteristics:  Control  beliefs  and  intrinsic  goal  orientation. 

Data  Collection  During  the  Course 

User  log  from  EDUCO  (time  stamp,  user  id,  action)  was  recorded  during  the  course  from 
September  24  to  November  20,  2001.  (Figure  1). 
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12:25  1. 10.2001;userl;login; 

12:30  1. 10 . 2001 ; userl; search; Al 

12: 31  1. 10 . 2001; userl; get ; http: //www7 . scu. edu . au/ programme /fullpapers/ 1893/ coml893 . htm 
12: 41  1. 10. 2001; user2 ; comment ; cmts47 . html 

12:  42  1.  10 . 2001;  use r3;  alarm;  use  rl,  http:  cs . pitt . eduZ-chang/booIcds/GrowingBook.  htm 


Figure  1;  A sample  of  EDUCO  log  file. 


Post  Test 


Ail  email  survey  consisting  of  15  open  propositions  (10  responses  out  of  24)  was  conducted  two 
weeks  after  the  course  in  December  2001.  Propositions  measured  users  experiences  and  expectations 
towards  computer  supported  education  together  with  EDUCO-related  attributes  (usability  issues,  user 
interface,  features  etc.). 


EDUCO  Architecture 

From  a technological  point  of  view,  EDUCO  consists  of  a server,  a Java  applet  for  every  user 
and  several  cgi-scripts.  The  server  tracks  the  state  of  the  distributed  system  and  informs  the  clients  as 
changes  occur.  Examples  of  possible  changes  in  the  environment  are  users  moving  to  another  page  and 
initiating  a chat.  The  implementation  of  this  kind  of  a scheme  in  real  time  requires  the  clients  to 
maintain  an  open  connection  to  the  server  throughout  the  session. 

To  avoid  any  copyright  or  other  intellectual  property  right  issues  we  have  taken  the  approach 
that  the  documents  (i.e.  HTMLfiles)  chosen  to  be  part  of  a particular  instance  of  EDUCO  are  not 
copied  to  our  server.  Instead,  they  can  be  located  anywhere  on  the  Web.  This  being  the  case,  the  server 
needs  to  know  which  page  each  user  is  viewing.  We  have  solved  this  problem  by  using  our  EDUCO- 
server  as  a kind  of  proxy,  which  means  that  the  documents  are  routed  through  our  server  instead  of 
being  sent  to  the  client  directly  from  their  actual  location  (Figure  2). 


EDUCO  User  Interface 

The  EDUCO  user  interface  consists  of  six  elements  of  which  only  one  is  concurrently  visible: 
(1)  Map,  (2)  Chat,  (3)  Search,  (4)  Alarm,  (5)  Preferences  and  (6)  Help.  The  EDUCO  interface  is  small, 
only  200  x 300  pixels,  and  thus  placed  into  the  left4iand  side  of  a www-browser  frameset.  The  right- 
hand  side  of  the  browser  frameset,  usually  more  than  440  pixels,  is  reserved  for  the  content  (i.e. 
hyperdocuments)  and  the  space  below  EDUFORM  is  reserved  for  the  document  specific  comments. 
Commenting  is  based  on  ordinary  html- forms.  Each  document  has  an  associated  comment  file,  which  is 
opened  to  its  designated  frame  and  can  be  updated  by  the  users.  The  server  keeps  track  of  modifications 
and  the  visits  of  individual  users.  This  way  the  documents  that  have  been  commented  after  the  last  visit 
can  be  distinguished  visually  from  those  that  contain  only  comments  the  user  has  already  seen.  (Figure 

3.) 

“Map”  is  a view  to  a visualization  of  the  hyperdocument  structure  and  the  people  present  in  the 
learning  environment.  The  material  is  organized  in  clusters  consisting  of  related  hyperdocuments. 
People  are  shown  as  coloured  dots.  Colour  indicates  different  group  membership  or  types  of  user 


1488 


profile,  and  the  location  of  the  dot  depends  on  the  document  the  person  is  currently  viewing.  When 
mouse  is  placed  on  top  of  the  symbol  of  document  or  person,  a tool  tip  appears  showing  the  name  of 
the  person  or  the  document.  Clicking  a symbol  selects  it  for  further  use;  double  clicking  a document 
opens  it  in  the  browser  window.  The  documents  change  their  colour  on  the  map  depending  on  how 
much  they  have  been  read  relative  to  the  others.  The  total  time  all  users  have  spent  reading  each 
document  is  recorded  by  the  server  on  an  hourly  basis.  The  change  in  the  cobur  of  an  individual 
document  is  determined  by  the  distance  of  its  moving  average  for  the  last  24  hours  from  the  same 
average  for  all  documents.  (Figure  4.) 
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Figure  3:  EDUCO  user  interface  with  Map  view  active  and  open  document  with  related  comments. 

“Chat”  enables  synchronous  communication  with  other  peers.  EDUCO  Chat  sends  messages 
only  to  those  people  sender  has  actively  contacted  by  e.g.  clicking  dots  on  the  Map.  The  number  of 
participants  in  the  discussion  is  unlimited,  but  one  person  may  use  only  one  chat  channel 
simultaneously.  (Figure  4.) 


] 1 Duttefs: 

pf 

i . r ~ . 1 

1 : , 

Ditcorywot  I | 

II  Discustm  1 

T»wgo^  jitiM  M»  quil«  '^11 

us«(ul 

JjjM(o:Yw.  indtad  ... 
ejtch  tha  majning  of 

did  you 

‘modutjrizad  laamlng 

eomoonants* 

Chat 

d 

|Ya»| 

tSi!l 

(/.V- 

4 g ct  ! 

) 

sT  ? 

1 

Figure  4:  EDUCO  Map  (left)  and  Chat  (right)  views. 


EDUCO  “Search”  is  useful  for  finding  people  or  documents.  The  search  is  targeted  to  the  titles 
of  documents  and  names  (and  nicknames)  of  users  online.  Search  selection  affects  also  to  the  other 
views  (“Map”,  “Chat”  and  “Alarm”).  “Alarm”  defines  a condition  to  be  monitored  by  the  system.  If  a 
user  wishes  to  seek  for  a work  mate  who  is  also  interested  in  certain  document  (or  topic),  he  can  tell  the 
system  to  send  a notifying  message  to  the  title  bar  of  EDUCO  when  predefined  conditions  are  met,  i.e. 
other  person  arrives  to  the  document  specifying  the  task.  Alarm  is  helpful  when  user  wishes  to  conduct 
a collaborative  time- related  assignment  with  a friend  who  has  not  yet  logged  into  the  system.  See 
Figure  5 for  screen  dumps  of  Search  and  Alarm  views. 
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Figure  5:  EDUCO  Search  (left)  and  Alarm  (right)  views. 


On  the  “Preferences”  view  the  user  is  allowed  to  change  personal  settings  i.e.  what  is  the 
preferred  nickname,  is  he  or  she  visible  to  other  users  (and  vice  versa)  or  is  the  chat  enabled.  Help  view 
provides  information  about  the  program.  (Figure  6.) 
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Figure  6:  EDUCO  Preferences  (left)  and  Help  (right)  views. 


Results 

The  main  focus  of  this  study  is  on  the  design  and  implementation  of  EDUCO  software  and  thus 
profound  analysis  of  the  empirical  results  of  all  three  forementioned  stages  is  unbearable  to  carry  out. 
Next  we  will  present  some  preliminary  results  of  the  post  test.  Seventeen  respondents  (71  %)  out  of  24 
replied  to  the  email  survey.  In  this  paper  we  analyse  briefly  following  four  propositions  (Table  1 ): 

Table  1:  Selection  of  preliminary  propositions  of  the  post  test.  

5.  What  is  your  opinion  of  the  essential  added  value  of  EDUCO  compared  to  traditional  computr- based  learning? 

9.  Give  your  comment  on  the  Map  view's  functionality  considering  it  as  a tool  for  a)  navigating  from  one 
hyperdocument  to  another,  b)  commenting  (annotation)  hyperdocuments,  and  c)  social  navigation  and  interaction. 

10.  How  relevant  th  e Chat  was  to  you? 

11.  What  was  the  relevance  of  EDUCO 's  profiling  info  to  you  when  you  were  seeking  for  work  mate(s)  to  carry  out 
collaborative  tasks? 


Preliminary  results  of  the  post  testing  show  that  the  EDUCO  was  found  to  be  a useful  tool  in  the 
matters  like  adaptation  to  respondents  learning,  cognitive  and  motivational  strategies,  and  means  to 
implement  collaborative  actions. 

*7r  was  very  useful  to  see  what  documents  other  users  were  reading  ...  it  gave  me  many  hints  and  saved 
time.  ” 

''It  was  truly  nice  to  be  able  to  see  what  is  the  most  interesting  document  at  the  moment  and  who  is  reading 
it " 

"Actually,  in  several  cases  / wanted  to  start  a chat  conversation  with  someone  reading  same 
hyperdocument  with  me  ...I guess  this  is  social  navigation? " 
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Presence  of  EDUCO  increased  task-related  participation  and  was  valued  tool  for  those  who  had 
difficulties  to  participate  in  face-to-face  meetings: 

'^Learning  material  was  easy  to  access.  ” 

‘‘EDUCO  gives  more  flexibility  to  studying  process.  ” 

“It  was  possible  for  me  to  participate  to  this  course  and  cany  out  all  those  tasks  regardless  of  my 
domicile. " 

Real  time  interaction  of  EDUCO  also  elicited  negative  comments: 

“EDUCO  hindered  formation  of  REAL  social  contacts! " 

“Chat  never  beats  traditional  face  to  face  meetings. 

EDUCO’s  tools  for  seeking  work  mates  (group  membership,  search  function)  were  truly  useful  for 
most  of  the  respondents: 

“I  was  in  a blue  group,  and  when  another  blue  was  looking  for  a mate,  I replied  instantly.  He  had  already 
chosen  an  article,  I glanced  at  it  and  found  that  it  was  suitable  for  me  too.  " 

“I  had  a group  proposal  via  email  message.  As  my  forthcoming  work  mate  had  the  same  colour  than  I did, 
it  was  easy  to  make  the  decision  to  start  collaboration.  Afterwards  I thought  that  I agreed  so  quickly 
because  of  the  same  motivational  group,  normally  it  takes  more  consideration.  But  to  be  honest,  the  topic 
was  the  most  important  factor.  ” 

Conclusions 

The  paper  has  described  the  design  and  implementation  of  a real  time  on-line  collaboration  tool. 
EDUCO  was  build  to  seek  solutions  to  some  unsolved  problems  concerning  real  time  collaboration 
between  live  users  in  both  traditional  and  modem  web-based  education  platforms.  The  main  focus  of 
the  study  was  to  demonstrate  how  EDUCO  was  applied  to  a real  life  on-line  distance-education  “Web- 
based  learning*’  course.  The  preliminary  research  findings  of  small-scale  qualitative  empirical  study 
indicated  that  some  of  the  features  used  in  this  restricted  evaluation  task,  namely  “motivational  group 
membership  information”,  “graphical  view  to  the  hyperdocument  space”,  “hyperdocuments  that  change 
their  colour  depending  on  the  popularity”,  can  be  directly  applied  to  a wider  context  of  modem 
computer-based  learning  environments  (see  e.g.  Dillenbourg  1999). 
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A Study  of  Changes  in  Attitude  Towards  Science  in  a 
Technology  Based  K-8  Preservice  Preparation  Science  Classroom 
Rena  Faye  Norby,  Assistant  Professor,  Black  Hills  State  University 

The  purpose  of  this  study  was  to  identify  and  analyze  changes  in  attitudes  towards  science  in  a 
technology  based  science  content  course  for  K-8  pre-service  teachers.  Given  a science  classroom  where 
Internet  research,  hands-on  science  activities,  cooperative  group  work  and  a constructivist  learning 
environment  is  promoted,  research  can  identify  how  students’  attitudes  towards  science  change  before  and 
after  taking  a technology  integrated  class.  Technology  is  required  in  standards  based  teaching,  and  the 
science  teacher  needs  guidance  in  integrating  technology  into  her  teaching  while  continuing  learning 
experiences  that  work  in  her  classroom.  A positive  attitude  towards  different  components  of  science  in 
pre-service  teachers  can  facilitate  more  effective  learning  in  science  in  K-8  classes. 

Attitudes  associated  with  science  appear  to  be  affecting  student  participation  in  science  as  a 
subject  (Koballa  et.  al.,  1990) ' and  impacting  performance  in  science  (lAEP,  1992)^  An  international 
assessment  of  nine-and-thirteen-year-old  students  in  twenty  countries  (lAEP,  1992)^  revealed  that  positive 
attitudes  toward  science  were  related  to  higher  student  performance. 

Science  teachers  in  grades  K-12  have  developed  hands-on,  experiential  methods  that  have  been 
effective  without  adding  in  the  mix  of  technology.  Teacher  accreditation  agencies  are  requiring 
technology  literacy  for  teachers,  but  the  connection  to  improving  attitudes  towards  science,  perception  of 
the  science  teacher,  and  the  value  of  science  in  society,  had  not  yet  to  be  analyzed.  The  hands-on  science 
teacher  has  some  real  questions  about  the  value  of  replacing  class  time  spent  in  practicing  the  processes  of 
science  with  computer  and  other  technology  based  experiences. 

For  this  study  twenty-nine  students  in  a one  semester  four  hour  class  took  the  “Test  Of  Attitude 
Towards  Science”^  on  the  first  day  of  the  Earth  and  Physical  Science  for  K-8  Teachers  at  Black  Hills 
State  University.  These  students  spent  4 hours  per  week  for  15  weeks  in  a university  computer  lab  that 

* Koballa,  Jr.,  T.  R.,  Crawley,  F.  E.,  & Shrigley,  R.  L.  (1990).  A summary  of  science  education-1988.  Science  Educatioa  74  (3),  369-381. 

^ lAEP.  (1992)  International  Assessment  of  Educational  Progress.(1992).  Learning  Science.  Princeton,  NJ:  Educational  Testing  Service. 

^ Weinberg,  M.  (1998).  Assessing  student  attitudes  towards  science  in  an  urban  elementary  classroom.  Paper  presented  at  the  Annual  Meeting  of 
the  National  Association  for  Research  in  Science  Teaching,  San  Diego,  CA. 
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provided  Internet  access,  and  use  of  standards  software  packages.  Students  studied  a science  textbook 
outside  of  class,  and  completed  project  based  activities  in  earth  and  physical  sciences  during  class.  They 
used  the  Internet  to  research  science  content,  lesson  plans  for  K-8  teachers,  and  to  access  state  and 
national  science  standards,  including  STS  standards.  The  students  took  the  same  survey  as  a post 
assessment  in  the  last  week  of  the  class. 

The  pre  and  post  responses  were  analyzed  using  nonparametric  statistics.  The  perception  of  the 
science  teacher  was  the  variable  that  showed  the  smallest  increase  between  pre  and  posttest.  Other 
subscales  on  the  instrument  showed  decrease  in  anxiety  towards  science,  and  increases  in  motivation  and 
enjoyment  of  science.  More  than  80%  of  the  students  indicated  that  their  overall  attitude  toward  science 
had  increased  appreciably  since  the  beginning  of  the  course. 

The  students  were  pre-service  teachers  so  it  would  be  expected  that  they  already  had  a fairly  positive 
perception  of  science  teachers.  On  a formative  assessment  that  asked  the  students  about  their  most 
positive  experience  in  science  learning,  almost  all  of  the  individuals  who  responded  to  this  question 
described  hands-on  activities  and  an  enthusiastic  science  teacher.  Adding  in  the  use  of  technology  in  this 
class  appears  to  have  been  the  cause  of  a significant  increase  in  feelings  of  motivation  and  enthusiasm 
about  science,  since  so  many  pre-service  teachers  are  uncomfortable  with  the  requirement  that  they  teach 
science  after  obtaining  employment.'^  Further  research  with  similar  classrooms  that  are  taught  without  the 
presence  of  extensive  access  to  technology  could  make  an  even  stronger  case  for  integration  of 
technology  into  science  teaching. 

In  this  study,  hands-on,  learner-centered  classroom  activities  as  well  as  the  use  of  technology, 
provided  interesting  and  motivating  experiences  for  the  students  at  the  same  time  that  their  attitudes 
towards  science  improved.  Results  of  this  research  can  be  generalized  and  validated  for  larger  populations 
and  can  encourage  science  teachers  to  use  technology  and  Internet  access  in  their  classrooms  for  other 
purposes  than  record  keeping  and  word  processing. 


Cantema,  Ron.  (1994).  University  of  Wyoming.  Personal  Communication. 
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A Framework  for  Online  Professional  Development 

Online  professional  development  serves  the  needs  of  individuals  as  well  as  organizations.  Professional  development  can 
be  viewed  from  these  two  perspectives:  the  individual  who  plans  to  participate  in  additional  education  and  training 
opportunities  and  a system  that  provides  those  opportunities  to  its  workforce  (Bellanca,  1995).  Adding  an  online 
perspective  to  professional  development  activities  provides  an  individual  with  the  chance  toparticipate  in  education  and 
training  opportunities  at  times  and  places  that  are  convenient.  Systems  benefit  from  online  professional  development  by 
reducing  travel  and  absent  employees  (McCampbell,  2000).  This  proposal  explores  a framework  of  designing, 
developing,  maintaining,  and  implementing  an  online  professional  development  program  for  educators  who  are 
challenged  by  national  and  state  reform  and  accountability  initiatives.  The  example  OPD  workshops  described  in  this 
paper  have  been  developed  and  implemented.  As  part  of  the  framework  for  OPD,  issues  related  to  Accountability  and 
Reform  Initiatives  are  addressed.  In  addition,  a set  of  guiding  questions  that  focus  on  High  Quality  Course  Design, 
Course  Management,  and  Learner  Support/Course  Facilitation  are  used  to  design,  develop  and  implement  the  OPD. 
Before  these  questions  are  addressed,  however,  let  us  begin  by  defining  OPD. 

The  Challenge  of  School  Reform  and  Accountability 

During  the  past  decade,  many  states  across  the  country  have  engaged  in  rigorous  school  reform  and  accountability 
efforts,  most  in  direct  response  to  the  national  Goals  2000  initiative.  In  some  states,  high-stakes  testing  accompany 
rigorous  standards,  all  with  the  hope  of  transforming  education  and  improving  student  performance. 

Ultimately,  the  improvement  of  performance  comes  down  to  how  well  teachers  understand  the  reform  efforts,  the 
standards,  and  the  content  for  which  they  are  being  held  accountable  (Sparks  & Hirsch,  1997).  Teachers  must  have  the 
tools,  support,  and  training  to  radically  change  teaching  and  to  infuse  change  at  the  school  level  (Darling-Hammond  & 
McLaughlin,  1995;  Sparks  & Hirsch,  2000).  Schools  trying  to  improve  must  consider  professional  development  as  a 
cornerstone  to  success  (Fullan,  1999)  with  strong  linkages  between  professional  development  and  the  reform  agenda 
(Darling-Hammond  & McLaughlin,  1995).  In  the  past,  fragmented  approaches  to  professional  development  have 
yielded  minimal  progression  toward  a clear,  compelling  vision  for  the  future  (Fullan,  1991).  This  vision,  apparent  in 
most  school  reform  agendas,  should  be  considered  when  linking  professbnal  development  with  reform.  By  linking 
professional  development  with  participants’  ongoing  professional  practice,  the  effects  may  begin  to  visibly  move  the 
reform  agenda  in  a positive  direction,  thus  promoting  change  within  the  educational  system  as  a whole  (Sparks  & 
Hirsch,  1997). 

Lieberman  (1995)  suggests  that  a radical  re-thinking  of  professional  development  must  occur.  Given  that  learning 


occurs  through  active  engagement,  it  is  time  to  consider  alternatives  to  more  traditional  direct  -instruction  approaches. 
With  a national  average  of  only  1%  of  the  budget  slated  for  staff  development,  with  less  than  50%  of  teachers  receiving 
release  time  for  professional  development,  and  with  nearly  25%  of  teachers  given  no  time,  credit  or  support  (NCES, 
July  98),  rethinking  the  practice  of  professional  development,  as  well  as  the  mechanism  for  delivery,  is  critical.  It  is  no 
surprise  that  teachers  report  increased  use  of  strategies  and  change  in  their  teaching  practice  with  increased  professional 
development  (NCES,  July  98).  Thus,  support  for  professional  development  is  a most  important  variable  in  school 
reform. 
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Online  Professional  Development  (OPD)  is  the  delivery  of  professional  development  where  participants  and  instructors 
are  separated  by  time  and  usually  by  distance,  using  the  World  Wide  Web  as  the  tool  for  instruction,  communication, 
and  collaboration.  As  with  all  forms  of  distributed  learning,  there  are  a variety  of  learning  goals  and  approaches  used 
within  various  OPD  settings.  Based  on  the  learning  goals,  the  available  technologies,  and  the  level  of  interaction  that  is 
deemed  necessary,  an  appropriate  model  or  combination  of  models  may  be  selected.  With  interaction  being  a significant 
contributor  to  the  success  of  online  learning,  it  is  important  to  consider  the  levels  of  interaction  being  used.  Both  social 
and  instructional  interactions  contribute  to  the  success  of  online  learning.  Social  interaction  promotes  communications 
and  collaboration,  while  instructional  interaction  promotes  interaction  with  the  content.  Three  levels  of  instructional 
interaction  are  commonly  discussed  for  distributed  learning,  to  include:  (1)  interaction  with  the  content,  (2)  interaction 
with  the  instructor,  and  (3)  interaction  with  peers  (Moore,  1989;  Northrup,  2001;  Rasmussen  & Northrup,  2000). 
Models  of  Online  Professional  Development  will  vary  in  their  type  and  amount  of  social  and  instructional  interaction. 
Models  for  OPD  that  encourage  various  levels  of  interaction  include:  (1)  Self-Paced,  Individual  Study,  (2)  Lecture- 
Discussion,  and  (3)  Learning  Community. 


Self-Paced,  Individual  Study 


This  model  of  OPD  has  low  levels  of  interaction  and  closely  resembles  a correspondence  course.  Interaction  would 
most  likely  occur  between  the  student  and  instructor  and  between  the  student  and  the  course  materials.  This  self-paced 
model  may  be  “open-entry,  open-exit”  where  participants  log  on  and  complete  their  course  using  their  own  time  table 
rather  than  a specific  beginning  and  ending  date  for  the  course.  For  participants  who  are  very  motivated  and  self- 
directed,  this  model  may  yield  successful  results.  For  most,  this  is  a difficult  approach.  In  addition  to  being  difficult  for 
students,  instructors/facilitators  have  no  boundaries  and  find  it  hard  to  communicate,  assess  assignments  or  to  establish 
a relationship  with  students. 


Lecture-Discussion  Model 


Typical  lecture-discussion  models  provide  online  lectures  via  PowerPoint,  streamed  video,  or  even  through  videotapes 
delivered  via  surface  mail.  This  approach,  if  deemed  appropriate,  may  provide  the  typical  lecture  discussion  approach 
where  students  are  encouraged  to  read  materials,  view  lectures,  and  respond  accordingly  through  listservs,  chat  rooms, 
or  via  threaded  discussions.  The  levels  of  interaction  in  this  model  would  include  instructor-to-student  interaction, 
student-to-materials,  and  student -to-student  interactions  most  closely  aligned  to  discussion  of  given  content.  Less  social 
interaction  would  occur  in  this  model. 


Learning  Community  Model 

This  model  places  its  emphasis  on  interactions  among  participants,  instructors,  and  facilitators.  Many  learning 
community  models  rely  on  activities  to  spark  ideas  and  generate  collaboration  and  sharing.  This  many-to-many  model 
makes  use  of  listservs,  threaded  discussion,  and  chats,  with  a focus  on  collaboration  and  communications.  Success  with 
this  approach  requires  embedded  scaffolding  of  expectations,  guidelines  for  setting  up  teams,  and  some  general  rules  of 
dialog.  Interaction  in  this  model  heavily  emphasizes  social  interaction  as  well  as  student-to-student  and  student -to- 
instructor  instructional  interactions.  What’s  missing  in  this  approach  is  the  tutorial-focused  instructional  component  of 
instruction,  as  most  assignments  are  centered  around  activities.  Depending  on  the  instructional  content,  this  may  or  may 
not  suffice  as  the  sole  vehicle  for  instructional  delivery. 

Guiding  Questions  for  Online  Professional  Development 

As  a result  of  the  need  for  radical  rethinking  of  professional  development,  this  paper  documents  how  Online 
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Professional  Development  (OPD)  can  be  used  as  one  of  many  professional  development  strategies  within  a school 
district.  This  paper  advocates  OPD  as  one  solution;  however,  there  are  several  other  learner-centered  approaches  that 
should  be  considered  when  developing  a school  or  district -wide  professional  development  plan.  The  guiding  questions 
addressed  in  this  paper  center  around  the  elements  of  professional  development,  whatever  the  medium  (EDC  Center  for 
Online  Professional  Development,  2000).  The  following  guiding  questions  form  the  basis  for  High  Quality  Design, 
Course  Management,  and  Learner  Support/Course  Facilitation  of  OPD: 

• How  do  we  provide  equal  professional  development  opportunities  for  all  teachers  at  a time  and  place 

• coriveriiciit  for  therri? 

• How  do  we  provide  opportunities  for  collaboration  and  sharing  among  faculty? 

• How  can  we  provide  a forum  for  inquiry,  reflection,  and  mentoring? 

• How  do  we  model  successful  practice? 

• How  can  professional  development  translate  into  professional  practice? 

• How  can  we  promote  ongoing  communication  and  conversation  among  educators? 

• How  can  we  foster  “deep”  understanding  of  the  subject  matter  and  its  relation  to  the  school  reform  and 
accountability  agenda? 

• How  can  we  instill  confidence  in  participants  who  are  not  familiar  with  online  learning? 

• How  can  we  scaffold  strategies  for  time  management  and  for  embedded  support  throughout  the  professional 
development  experience? 

Using  the  Framework  for  OPD  Development 

Information  Engagement:  Reading  in  the  Content  Area  and  The  Making  of  a Technology-Rich  Classroom  are  two 
Online  Professional  Development  (OPD)  workshops  designed  and  field  tested  through  grants  in  collaboration  with  The 
University  of  West  Florida’s  Division  of  Technology,  Research  and  Development  and  partnering  Florida  school 
districts.  In  response  to  standards- driven  reform,  these  OPDs  provide  opportunities  in  continuous  improvement  of 
professional  skills  as  teachers  participate  in  an  active,  engaging  learning  community.  The  OPDs  are  highly  interactive, 
activity-focused,  and  rich  in  collaboration  and  discussion  among  participants,  instructors,  and  facilitators. 

Each  of  these  OPDs  are  four-week,  10-hour  workshops  (approximately  2.5  hours  per  week)  designed  to  guide  teachers 
through  related  processes  and  content.  In  the  Making  of  a Technology-Rich  Classroom, 
(http://mentor.coe.uwf.edu/onlinepd/)  the  goal  of  the  workshop  is  to  provide  teachers  with  knowledge,  skills,  and  abilities 
that  are  needed  to  successfully  integrate  technology  into  the  teaching  and  learning  environment  which,  in  turn,  will 
enhance  the  overall  learning  environment  and,  ultimately,  increase  student  performance.  Similarly,  Information 
Engagement:  Reading  in  the  Content  Area  fhttp://dev.ibinder.uwfedu/cvbersphere/profdev/onlineindex.cfm')  is  designed  to 
provide  middle  and  high  school  teachers  with  the  skills  they  need  to  effectively  integrate  reading  techniques  and 
strategies  into  content  area  classes  with  the  objective  of  increasing  reading  scores. 

The  OPDs  combine  elements  of  the  lecture-discussion  model  (i.e.,  information  acquisition)  and  the  learning  community 
model  (i.e.,  discussion -based,  sharing  of  experiences,  both  prior  and  current).  The  OPDs  were  systematically  designed 
to  promote  exploration  and  discovery,  encourage  experimentation,  and  to  provide  rich  examples  and  opportunities  for 
practice.  In  addition,  the  OPDs  are  designed  to  be  highly  interactive  and  visually  rich,  taking  teachers  into  others’ 
classrooms  via  online  video  examples. 

Information  is  presented  over  the  four-week  period  (see  Figure  1 for  the  main  site  of  The  Making  of  a Technology-Rich 
Classroom)  to  the  teachers  in  manageable,  small  chunks  that  focus  on  specific  objectives  through  several  delivery 
methods:  text,  graphics,  and  video.  Teachers  are  guided  through  the  workshop  by  a facilitator.  The  facilitator  promotes 
interaction  in  threaded  discussions  and  through  the  listserve,  and  serves  as  an  expert  for  individual  questions,  offering 
advice  and  best  practices. 
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Figure  1. 

The  Making  of  a Technology-Rich  Classroom  main  site  with  weekly  topics. 


Research  on  traditional  teacher  staff  development  suggests  that  typical  in-service  activities  for  teachers  is  faulty  - 
teachers  spend  isolated  time,  away  from  their  classroom,  learning  the  topic  and  never  fully  implement,  either  by  choice 
or  by  situation,  the  changes  into  their  classroom.  These  examples  of  OPD  combat  these  issues  by  enabling  teachers  to 
participate  in  the  workshop  at  times  that  are  convenient  for  them  and  by,  more  importantly,  guiding  the  teachers  in  a 
step-by-step  processes  that  are  immediately  implemented.  In  other  words,  teachers  field  test  techniques  and  strategies 
suggested  by  the  OPD  and  other  teachers  in  their  own  classroom  laboratory.  This  integration  of  implementation  into  the 
workshop  environment  provides  teachers  with  the  motivation  to  start  to  implement  and  then,  hopefully,  to  provide  them 
with  enough  support  so  that  they  may  continue  the  implementation  after  the  workshop  is  completed.  Teachers  then 
reflect  with  other  learners  and  facilitators  as  to  what  worked  through  communication  activities  (chats,  listserves,  and 
threaded  discussbns  [see  Figure  2])  which  also  permits  them  to  learn  about  challenges  and  successes  experienced  by 
their  colleagues.  The  structure  of  the  OPDs,  over  an  extended  period,  require  that  sharing  and  reflection  take  place  - 
giving  teachers  the  time  that  they  need  to  actually  try  out  innovations  in  their  classrooms  and  evaluate  the  success  of 
that  innovation. 
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Figure  2. 

Information  Engagement:  Reading  in  the  Content  Area  weekly  lesson  example. 


Field  Testing  the  OPDs 

Information  Engagement:  Reading  in  the  Content  Area  and  The  Making  of  a Technology-Rich  Classroom  have  been 
developed  and  implemented  with  teachers.  An  evaluation  was  conducted  after  each  field  test.  Both  evaluations  yielded 
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positive  results,  with  teachers  focusing  on  the  convenience  of  the  OPD  and  applicability  of  the  strategies  to  the 
classroom.  Teachers  were  able  to  integrate  innovative  strategies  into  their  classrooms  with  little  disruption  or 
interference  in  the  day-to-day  operation  of  the  class.  Using  anecdotal  data  from  the  four -week  period,  teachers  believed 
that  student  performance  would  be  improved  with  continued  use  of  the  new  strategies. 


Addressing  the  Guiding  Framework  Questions 


The  questions  outlined  earlier  relating  to  High  Quality  Course  Design,  Course  Management,  and  Learner 
Suppoiu'Coursc  Facilitation  are  extremely  important  in  the  ultimate  success  of  the  OPD  participant.  The  first  area  of 
concern  that  needs  to  be  evaluated  is  the  course  quality.  Without  a strong  foundation  of  quality,  the  professional 
development  will  not  meet  the  needs  of  the  participants.  Learning  outcomes  are  aligned  to  relevant  standards  (e.g., 
NETS,  NCTE/IRA).  In  addition,  throughout  the  OPDs,  teachers  align  their  own  revised  curriculum,  using  suggested 
strategies  that  focus  on  improving  student  performance.  Materials,  including  web  pages,  activities,  supporting  resources, 
have  been  created  by  professional  instructional  designers  in  consultation  with  subject  matter  experts  who  are  experts  in 
their  fields.  The  instructional  materials  contain  web  links,  readings,  and  assignments  that  are  aligned  to  standards  and 
workshop  goals  and  objectives.  Information  is  presented  and  activities  are  incorporated  in  a variety  of  fashions  to  meet  a 
variety  of  learning  styles.  All  activities  are  designed  to  be  able  to  be  immediately  used  in  the  classroom  with  a minimum 
of  disruption.  The  weekly  activities  are  designed  to  take  no  more  than  the  allocated  time  of  2.5  hours,  which  includes 
the  time  set  aside  for  chat  sessions,  threaded  discussion  responses,  and  listserve  communication.  All  elements  of  the 
OPD  are  designed  to  promote  effective  and  efficient  learning  that  meets  the  needs  of  the  teachers  and  the  goals  of  the 
workshop. 


Expectations  for  teacher  performance  are  clearly  outlined  in  the  first  week  - the  final  outcomes  are  outlined  and 
teachers  immediately  begin  to  work  on  their  final  projects,  which  are  based  upon  implementing  topic  strategies  by  the 
end  of  the  OPD.  Facilitators  communicate  with  teachers  at  least  twice  a week  to  make  sure  that  they  remain  motivated 
and  on  track. 


Course  management  issues  revolve  around  two  major  categories:  Administrative  (for  the  providers  of  the  OPD  as  well 
as  for  those  who  supervise  professional  development  activities  at  the  district-  and  school-level)  and  the  Participant. 
Concerns  of  registration,  recognition  and  notification,  cost,  and  alignment  of  the  OPD  to  the  district  schools’ 
accountability  and  improvement  initiatives  must  be  addressed.  Districts  handle  these  issues  differently.  In  some  cases, 
teachers  register  through  their  home  districts.  At  other  times,  participants  will  register  through  the  OPD  web  site.  At  the 
completion  of  the  workshop,  teachers  are  provided  with  a certificate  and  the  district  staff  development  office  is  notified 
via  e-mail  that  the  teacher  has  successfully  completed  the  OPD.  Costs  are  negotiated  with  the  district  for  district - 
sponsored  workshops.  Some  districts  may  choose  to  negotiate  to  implement  the  OPD  and  assume  the  responsibilities  for 
access  and  facilitation. 


OPD  can,  in  many  cases,  be  overwhelming  to  participant  on  several  levels.  Even  expert  technology  users  must  become 
comfortable  with  the  delivery  system  and  the  demands  of  learning  at  a distance.  Novice  users  face  the  challenges  of 
learning  both  to  learn  at  a distance  as  well  as  being  able  to  efficiently  and  effectively  use  new  technologies.  Teachers 
should  have  a basic  level  of  technology  literacy,  including  the  ability  to  use  the  Internet,  email,  and  use  basic 
application  software.  An  online  orientation  assists  teachers  in  developing  the  skills  they  need  to  be  distance  learners.  In 
addition,  the  first  week  serves  as  an  orientation  to  the  content  of  the  OPD.  When  districts  participate  in  the  OPD,  face- 
to-face  orientation  meetings  can  be  held.  For  teachers  who  may  not  possess  all  of  the  technology  skills,  tutorials  arc 
available  to  provide  them  with  additional  support  (which  may  require  additional  time  on  the  teacher’s  part). 

Before  a teacher  participates  in  OPD,  they  must  commit  to  the  time  required  for  the  workshop  and  have  access  to  the 
minimum  technology  resources  needed.  They  should  also  accept  the  philosophy  of  the  OPD  in  terms  of  interaction  and 
relationship  development.  In  the  example  OPDs,  teachers  commit  to  accessing  the  site  at  least  two  to  three  times  a week 
as  they  actively  share  their  experiences  and  implement  technology -rich  activities  in  their  own  classrooms.  Teachers 
work  in  groups  and  must  be  able  and  willing  to  participate  in  group  activities.  The  final  area  of  concern  when 
considering  OPD  involves  learner  support  and  course  facilitation.  First,  teachers  should  evaluate  the  level  and  amount 
of  support  and  determine  whether  it  fits  their  style  and  needs.  The  accessibility  of  the  facilitator  and  experts  should  be 
evident  — both  facilitators  and  experts  should  be  available  at  various  times  throughout  the  OPD  to  offer  support  and 
guidance.  Responses  to  questions  should  be  returned  to  the  teacher  within  24  hours  at  most.  Online  support,  such  as 
tutorials,  coaching,  embedded  assistance,  and  other  resources,  should  be  available  to  supplement  needs  of  the  distance 
learner.  Student -to-student  interaction  should  be  encouraged  to  promote  and  develop  an  active  learning  community  that 


will  continue  beyond  the  OPD  experience.  The  learning  environment  should  be  non -threatening  and  supportive  to 
permit  teachers  to  explore  and  discover  strategies  that  are  meaningful  to  their  own  instructional  situation.  Finally,  the 
site,  or  resources  from  the  site,  should  be  available  so  that  teachers  can  revisit  resources  as  they  follow-up  and  continue 
to  implement  strategies  from  the  OPD. 

Support  and  facilitation  are  of  utmost  importance.  In  the  example  OPDs,  extensive  web  resources  including  tutorials, 
web  resources,  activity  databases,  standards,  and  previous  teacher  ideas  are  available  whenever  the  teacher  needs 
additional  information.  Teachers  are  encouraged  to  share  via  the  listserve  new  sites  that  might  also  interest  their  peers. 
These  new  resources  are  captured  by  the  facilitator  who  then  improves  the  web  resources  by  new  relevant  sites. 
Facilitators  are  themselves  expert  teacher  who  successfully  integrate  technology  in  their  own  classrooms  and  serve  as 
active  participants,  sharing  their  own  experiences,  as  they  encourage  and  motivate  the  teachers.  After  the  OPD 
concludes,  the  facilitator  maintains  contact  with  the  teacher  participants  for  a period  of  two  months,  providing  follow-up 
information  and  encouraging  the  teacher  to  continue  to  integrate  technology  into  his  or  her  classroom.  In  addition, 
teachers  are  encouraged  to  keep  in  contact  with  their  new  peers  to  develop  their  own  learning  community  that  promotes 
technology  integration. 

Conclusion 

Online  Professional  Development  is  an  effective  way  to  provide  teachers  with  the  new  skills  that  they  need  to  meet  the 
challenges  of  state  and  national  accountability  and  reform  initiatives.  Available  at  their  convenience,  teachers  can  take 
advantage  of  OPD  workshops  that  meet  their  needs.  The  appropriate  design  and  development  of  a workshop,  in 
conjunction  with  learner  support  of  the  OPD,  facilitates  participant  success.  Following  the  framework  for  online 
professional  development  ensures  a successful  OPD  experience. 
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Interaction  has  been  defined  from  many  perspectives.  Most  simply  stated 
interaction  is  engagement  in  learning  (Hillman,  Willis  & Gunawardena,  1994).  It  is 
agreed  that  interaction  must  be  designed  into  an  instructional  program  and  that  it  is  an 
important  variable  for  online  learning.  Berge  (1999)  suggests  that  interaction  is 
important  to  learner  satisfaction  and  that  it  assists  in  maintaining  student  persistence  in 
courses.  With  retention  in  online  learning  programs  being  as  low  as  50%  in  some  cases 
and  course  completion  rates  in  traditional  courses  at  10-20  percentage  points  higher  than 
in  online  courses  (Carr,  2000),  learner  satisfaction  is  a key  variable.  With  interaction 
being  a component  of  overall  student  satisfaction,  interaction  should  be  considered  when 
trying  to  increase  retention  in  online  courses.  However,  from  the  online  learners  point  of 
view,  too  much  interaction  may  be  perceived  as  busy  work  and  lead  to  frustration, 
boredom,  and  overload  (Berge,  1999);  while  too  little  interaction  may  result  in  student 
isolation.  Both  are  considered  frustrating  and  a balance  has  to  be  found. 

Several  interaction  frameworks  and  taxonomies  are  available  as  guidelines  for 
designing  for  online  interaction.  Moore  (1989)  identifies  three  types  of  interactions;  (1) 
interaction  with  the  content,  (2)  interaction  with  peers,  and  (3)  interaction  with  the 
instructor.  Gilbert  and  Moore  (1998)  view  the  process  from  a different  perspective 
indicating  that  interactions  encompass  both  instructional  and  social  situations. 

Sorting  through  interaction  frameworks  to  determine  the  most  appropriate 
interactions  for  given  learning  outcomes  is  difficult  at  best.  Northrup  (2001a)  provides  a 
set  of  interaction  attributes  that  can  be  used  to  select  strategies  and  tactics  to  facilitate 
online  interaction.  The  attributes  encompass  levels  of  content  interaction,  types  of  dialog 
through  communications  and  collaboration,  levels  of  student  self-directedness,  and  types 
of  support  for  the  learner  anytime,  anyplace. 

With  most  research  on  interaction  focused  on  classifying  the  types  of  interactions 
or  building  frameworks  from  which  designers  would  select  appropriate  interactions  for 
given  learning  outcomes,  it  seemed  apparent  that  there  should  be  an  upper  and  lower 
limit  to  the  types  of  interactions  used  for  a given  set  of  instruction.  Additionally,  with 
student  perception  of  interaction  being  complete  as  such  an  important  variable  for 
ongoing  participation  in  the  course  (Zhang  & Fulford,  1994),  the  relationships  of  student 
perception  to  the  attributes  of  interaction  should  be  considered. 

Purpose  of  the  Study 

The  purpose  of  this  study  was  to  investigate  the  types  of  interactions  that  students 
perceived  to  be  important  for  online  learning.  The  interaction  attributes  investigated 
included  content  interaction,  conversation  and  collaboration,  intrapersonal/metacognitive 
skills,  and  need  for  support.  Also  investigated  were  reasons  why  learners  were  taking 
online  courses.  It  was  presumed  that  students  taking  courses  for  convenience,  flexibility, 
or  preference  would  likely  be  more  pleased  with  interaction  in  online  course  than  those 
required  to  take  an  online  course  because  it  wasn’t  offered  on  campus. 
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Method 


Introduction 

The  purpose  of  this  study  was  to  investigate  the  types  of  interactions  that  students 
perceived  to  be  important  for  online  learning.  Interaction  attributes  studied  in  this 
investigation  included  content  interaction,  conversation  and  collaboration, 
intrapersonal/metacognitive  skills,  and  need  for  support.  This  study  was  an  initial 
iTTVwStigatioir  of  1 earner  perceptions  of  online  interaction.  Data  were  collected  through 
the  administration  of  the  Online  Learning  Interaction  Inventory  (OLLI)  (Northrup, 
2001b). 

Participants 

This  study  consisted  of  52  graduate  students  in  an  online  masters  program  in 
instructional  technology.  Thirty- four  of  the  students  were  female  and  18  were  male. 
Students  were  selected  to  participate  in  this  study  based  on  where  they  were  in  the 
program  of  study.  Intact  classes  of  students  were  selected  from  two  courses  at  the 
beginning  of  their  online  learning  sequence  and  two  courses  at  the  end  of  their  online 
learning  sequence. 

Instrumentation 

The  instrument  used  for  this  study  was  the  Online  Learning  Interaction  Inventory 
(OLLI),  with  a reliability  coefficient  of  .95.  The  OLLI  focused  on  the  four  interaction 
attributes  of  content  interaction,  conversation  and  collaboration, 
intrapersonal/metacognitive  skills,  and  need  for  support. 

The  OLLI  was  divided  into  six  sections  with  a total  of  50  items.  Section  1 dealt 
with  demographic  information.  Section  2 included  five  questions  on  reasons  why 
students  selected  to  take  an  online  course.  Section  3-6  addressed  each  of  the  interaction 
attributes  and  were  rated  on  a five  point  Likert  scale  with  1 representing  strongly  disagree 
to  5 representing  strongly  agree.  Section  3 dealt  with  Content  Interaction.  There  were 
13  items  relating  to  the  indicators  of  content  interaction.  Section  4 addressed 
Conversation  and  Collaboration  with  14  items  relating  to  the  indicators  of  interaction. 
Section  5 addressed  Intrapersonal/Metacognitive  Skills  with  7 items  relating  to  the 
indicators  of  interaction.  Section  6 addressed  Support  with  7 items  relating  to  the 
indicators  of  interaction. 

Procedure 

In  the  current  study,  students  from  four  online  classes  were  sent  a detailed  email 
stating  that  the  purpose  of  the  Online  Learning  Interaction  Inventory  (OLLI)  was  to 
gather  information  to  continue  to  make  the  online  courses  and  the  program  more 
appropriately  interactive.  The  email  indicated  that  data  would  be  reported  and  used  as 
research  as  well  as  be  used  for  formative  evaluation  purposes.  Students  were  provided 
with  the  url  to  take  the  OLLI  online.  In  two  of  the  four  courses,  the  OLLI  was  posted  as 
a weekly  assignment.  In  the  other  two  courses  taking  the  OLLI  was  optional.  Students 
were  provided  with  one  week  to  complete  the  50- item  instrument. 

Data  Analysis 

Data  were  analyzed  by  item  using  frequency,  means,  and  standard  deviations  to 
report  areas  of  interaction  that  are  perceived  to  be  valuable  or  a hindrance  to  success  for 
online  learning.  Research  questions  for  the  study  are  as  follows: 

Question  1:  Why  do  students  learn  online?  Question  2:  What  interaction 
attributes  do  students  perceive  as  important  for  online  learning? 
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Results  and  Discussion 

Data  collected  from  the  OLLI  were  analyzed  by  attribute,  with  frequency,  means 
and  standard  deviations  reported.  Reported  first  will  be  responses  from  the  first  research 
question  related  to  students  learning  online.  The  second  research  question  related  to  the 
interaction  attributes  will  be  reported  by  each  of  the  four  interaction  attributes. 

Learning  Online 

Learning  online  is  related  to  the  first  research  question,  IVhy  do  xtudonts  learn 
online?  The  majority  of  students  selected  to  take  online  courses  for  convenience 
(M=4. 13,  SD=1 .14)  and  flexibility  (M=4.65,  SD=1 .33).  Most  of  the  students  reported 
that  they  could  attend  school  even  if  the  course  was  campus-based  (M=3.58,  SD=1.58), 
indicating  that  many  of  the  students  lived  close  enough  to  the  campus  to  take  campus - 
based  courses.  Only  12  students  (23%)  reported  that  it  would  be  impossible  to  take  the 
course  if  it  were  not  offered  online. 

Interaction  Attributes 

There  are  four  interaction  attributes  related  to  the  second  research  question. 
Attributes  included:  (1)  content  interaction,  (2)  collaboration  and  conversation,  (3) 
intrapersonal/metacognitive  strategies,  and  (4)  support.  Responses  are  included  by 
attribute  for  the  following  research  question:  What  interaction  attributes  do  students 
perceive  as  important  for  online  learning? 

Content  Interaction.  In  general,  it  appears  that  students  agree  that  interacting 
with  the  content  is  important  to  their  online  learning  experiences.  Overall,  they  report 
that  they  like  partially  individualized  courses  with  some  instructor  direction  (M=3.77, 
SD=.85).  Participants  also  reported  a desire  to  interact  with  content  delivered  via  audio- 
narrated  online  presentations  (M=3.65,  SD=1 .22).  Interacting  with  innovative 
instructional  strategies  also  was  reported  to  be  important  to  their  online  experience. 
Strategies  such  as  case  studies  (M=2.83,  SD=.92);  structured  games  (M=3.10,  SD=1.11): 
and  readings  followed  by  online  discussion  (M=4.56,  SD=1.09)  were  all  rated  popular 
with  participants.  Interestingly,  participants  expressed  strong  frustrations  about  being 
required  to  participate  in  too  many  interactive  assignments  in  a weekly  segment  of  the 
course  (M=4.08,  SD=1.06). 

Interacting  with  the  content  is  a major  component  of  an  online  course  and  the 
primary  location  where  new  knowledge,  skills,  and  abilities  are  presented.  Typically 
instruction  online  is  presented  as  instructor-centered  or  student -centered.  Both  are 
appropriate  given  the  learning  outcome  and  topics  of  the  course  content.  Students  in  this 
study  seem  to  prefer  a variety  of  techniques,  yet  seem  to  feel  most  comfortable  with  the 
“feeling”  of  a traditional  class.  With  the  highest  reported  perceptions  of  positive 
interaction  in  the  areas  of  audio- narrated  presentations  and  readings  text  followed  by 
discussion.  The  lecture  itself  (the  audio-narrated  presentations)  can  provide  a foundation 
for  other  attributes  of  interaction  including  conversation,  collaboration  and  informal 
discussion. 

Conversation  and  Collaboration  Results  of  the  interaction  attribute  of 
conversation  and  collaboration  indicated  that  participants  rely  on  their  peers  and  their 
instructor  in  forming  and  maintaining  the  online  learning  community.  The  majority  of 
participants  (M=4.94,  SD=1.06)  reported  that  it  is  essential  to  build  a community  of 
learners  in  the  online  environment.  Participants  reported  liking  to  discuss  ideas  and 
concepts  with  peers  (M=4.00,  SD=.  71)  and  also  perceive  that  sharing  information  with 


peers  is  important  (M=3.83,  SD=.  71).  In  relationship  to  teaming,  participants  reported 
that  working  in  teams  was  difficult  for  them  (M=3.08,  SD=1.19)  and  that  once  a team  is 
formed;  they  prefer  to  maintain  the  same  team  for  the  entire  semester  (M=3.62, 

SD=1.05).  In  terms  of  innovative  instructional  strategies  for  interacting  online, 
participants  reported  liking  online  debates  (M=3.04.  SD=1.12)  and  posing  questions  to 
experts  (M=4.02.  SE)=1.02).  Students  weren’t  as  receptive  to  the  idea  of  posing  as  the 
guest  presenter  in  class  (M=2.71,  SD=1.18).  Finally,  in  terms  of  feedback  from  the 
instructor,  participants  reported  that  it  is  important  to  them  (M=4.35,  SE)=.76)  and  that 
the  instructor  should  make  every  attempt  to  provide  some  kind  of  feedback  to  them  at 
least  two  times  per  week  (M=3.77,  SD=.85).  Interestingly,  participants  reported  that  it 
was  unnecessary  for  instructors  to  provide  feedback  on  a daily  basis  (M=4.25,  SD=.84). 

Promoting  collaboration  and  conversation  online  is  an  attribute  of  online  learning 
that  participants  considered  important.  Overall,  forming  the  community  of  learners, 
collaborating  with  peers,  and  getting  feedback  from  the  instructor  were  the  most  highly 
rated  indicators  of  this  attrbute.  Given  that  groups  of  students  do  not  just  become 
collaborative  because  they  are  assigned  together  (Johnson  & Johnson,  1994)  means  that 
designers  and  instructors  should  provide  clear  expectations  for  collaboration  online. 

Interestingly,  note  the  positive  responses  on  innovative  instructional  strategies. 
The  variety  of  strategies  presented  within  the  confines  of  a course  appears  to  yield 
positive  perceptions  among  students.  Providing  both  synchronous  and  asynchronous 
conversation  and  communication  online  can  extend  learning  and  at  the  same  time 
motivate  the  learner  (Sherry,  2000). 

Intrapersonal/Metacognitive  Skills.  Analysis  of  items  related  to 
intrapersonal/metacognitive  skills  suggest  that  self-directedness  and  embedded  cognitive 
strategies  designed  into  the  online  learning  environment  are  perceived  to  be  important  to 
participants.  Participants  reported  that  it  is  important  to  monitor  their  own  progress  each 
week  (M=4.58,  SE)=.72).  With  regard  to  embedded  cognitive  strategies,  participants 
reported  that  it  is  important  to  have  structured  times  that  assignments  are  due  (M=4.33. 
SD=.83),  to  have  an  advance  organizer  to  assist  them  through  the  assignments  each  week 
(M=4.10,  SE)=1.00).  to  provide  graphical  representations  of  the  steps  that  should  be  taken 
to  complete  assignments  (M=3.96.  SD=1.31),  and  to  have  notetaking  guides  to 
accompany  audio- narrated  presentations  (M=4.04,  SD=1.12). 

Overall,  self- regulating  one’s  own  learning  is  an  important  aspect  of  online 
learning.  Not  only  do  students  need  to  monitor  their  progress  in  an  ongoing  fashion  and 
adjust  their  strategies  for  learning  based  on  their  progress,  they  also  need  to  maintain  a 
time  management  schedule  in  order  to  complete  online  learning  activities  in  the  allotted 
timeframes.  To  assist  and  guide  learners  through  online  learning,  strategies  like  advance 
organizers  and  graphical  representations  are  used  to  guide  the  learner  through 
assignments,  while  notetaking  guides  and  posted  times  for  assignment  due  dates  are  also 
included. 

Support.  Results  indicate  that  support  is  also  a key  attribute  in  the  success  of 
online  learning.  Designing  online  learning  with  a solid  support  system  in  place  enables 
timely  responses  to  questions,  mentoring,  tutorials,  and  tips  from  peers.  This  support 
system  may  very  well  provide  a foundation  for  successful  learning.  Participants  report 
that  timeliness  of  response  (M=4.48.  SD=.64)  is  a major  indicator  of  support.  Most 
participants  reported  also  that  having  a mentor  in  place  to  provide  assistance  is  also 
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important  (M=3.52,  SD=1.35y  Participants  also  reported  that  having  tutorials  available 
as  needed  (M=3.12,  SD=1.55)  will  assist  them  in  performing  tasks  such  as  being  in  a chat 
room,  posting  to  a threaded  discussion,  etc.  And  no  surprise,  participants  report  that 
when  the  technology  doesn’t  perform  as  intended,  they  are  extremely  frustrated  (M=4.17, 
^1.15). 

Overall  Perceptions  of  Interaction 

Overall,  participants  provided  the  reasons  why  they  chose  to  take  courses  online. 
They  also  rated  items  in  each  attribute  of  online  interaction  as  important  to  their  success 
as  online  learners.  The  top  reason  for  taking  a course  online  was  the  flexibility  (M=4.65, 
SD=.74)  followed  closely  by  convenience  (M=4. 1 3,  SD=1.14)  With  regard  to  the 
interaction  attributes,  Intrapersonal/Metacognitive  had  the  most  highly  rated  indicators 
with  self- monitoring  of  individual  progress  (M=4.58,  SD=.72)  rated  at  the  highest 
frequency.  The  support  attribute  also  rated  at  the  top  with  timely  responses  by  the 
instructor  (M=4.48,  SD=.64)  rated  as  the  number  two  indicator  of  an  interactive  online 
course.  Although  indicators  exist  in  each  of  the  interaction  areas,  the  idea  of  self- 
regulating learning  and  having  timely  feedback  from  the  instructor  was  reported  as  most 
valued  by  participants. 

Conclusion 

In  conclusion,  it  is  agreed  that  interaction  should  be  designed  into  online 
instruction.  It  is  also  agreed  that  interaction  is  an  important  variable  for  learning, 
primarily  because  it  is  important  to  learner  satisfaction  and  motivation  (Berge,  1999).  In 
this  study,  online  learners  echo  the  importance  of  interaction  by  requesting  interactive 
elements  in  their  online  experiences.  Participants  in  this  study  are  still  most  comfortable 
with  the  idea  of  simulating  a campus-based  class  online,  as  reflected  in  their  statements 
regarding  the  desire  for  instructors  to  use  online  audio-narrated  lectures,  provide 
notetaking  guides,  and  discuss  learned  experiences  in  some  type  of  online  conversation. 
Although  their  comfort  is  with  the  “known”  they  still  favorably  rated  using  more 
innovative  strategies  in  the  online  environment  including  case  studies,  debates,  role- 
plays,  and  gaming.  The  foundation  of  the  online  learning  environment  however,  included 
the  notion  of  solid  student  support  and  self-directedness.  Participants  strongly  stated  that 
the  need  for  timely  responses  from  peers  and  from  their  instructor  was  of  utmost 
importance.  They  also  indicated  that  it  was  essential  for  students  to  self- monitor  their 
progress  for  survival  in  the  online  course. 

This  study  was  an  initial  investigation  into  the  perceptions  of  online  learners’ 
interaction  needs.  Future  studies  should  consider  other  variables  that  may  affect  the 
individual  learner,  the  learning  environment,  and  instructional  strategies  that  may  be  most 
appropriate  for  specific  learning  outcomes. 
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Abstract:  The  groundwork  for  interactive  learning  applications  at  the  Laboratory  for 
Technical  Mechanics,  begun  ten  years  ago,  has  led  to  a new  type  of  highly  interactive 
application  called  an  Exploration.  Explorations  combine  aspects  of  construction, 
modeling,  and  simulation;  they  allow  students  to  graphically  create  their  own 
constructions,  and  supply  the  corresponding  mathematical  description.  The  program 
executes  the  necessary  calculations  and  displays  the  results  in  graphs  and  as  analytic 
equations.  The  idea  behind  Explorations  was  to  support  learning  by  constructions  and 
immediate  response  by  the  system.  In  cooperation  with  the  research  group  Computers 
and  Society  of  the  University  of  Paderbom,  a number  of  Explorations  were  built  for 
student  use. 


Introduction 

The  problem  that  arises  in  learning  Technical  Mechanics  is  finding  the  connection  between  mechanical 
structures  and  their  complex  mathematical  and  physical  relations.  The  practice  of  teaching  shows  that  esp  e- 
cially  for  visual  conceptualization,  both  of  static  and  of  dynamic  mechanical  processes,  the  use  of  anima- 
tion and  simulation  programs  can  be  a good  extension  to  the  classical  method  of  presentation  with  chalk 
and  blackboard.  Even  for  mechanical  problems  covering  static  structures,  an  animation  effectively  shows 
how  one  can  find  a solution.  The  necessity  for  the  visualisation  of  mechanical  effects  was  the  internal  spark 
for  developments  of  simulation  programs  at  the  Laboratory  for  Technical  Mechanics  of  the  University  of 
Paderbom. 

In  the  following  paragraph  is  given  a historical  overview  about  our  approach  and  our  previous  experience 
in  the  development  of  animations  and  simulations. 

The  second  part  of  this  paper  will  introduce  a new  kind  of  interactive  learning  application,  the  Exploration. 
These  applications  have  capabilities  that  are  in  many  ways  more  powerful  than  the  capabilities  of  anima- 
tions and  simulations.  They  give  students  the  ability  to  build  constructions  in  a graphical  way  while  con- 
struction, simulation,  visualisation  and  analytic  description  of  technical  models  are  linking. 

Previous  works  in  the  field  of  animations  and  simulations 

In  1991,  the  Laboratory  for  Technical  Mechanics  procured  its  first  PC  systems,  leading  to  the  development 
of  computer  programs  for  instruction  purposes.  The  first  application  (see  Ferber  et  al.,  1992)  was  a program 
for  image  processing,  simulation  and  scientific  visualisation.  This  program  created  simulations  of  experi- 
ments done  by  students  in  the  laboratory  section  of  a technical  mechanics  course.  The  shadow -optical  im- 
ages created  by  students  during  these  courses  are  digitized  and  analyzed  by  image  analysis  algorithms*, 
thereby  allowing  the  unknown  parameters  of  the  experiment  to  be  determined.  With  these  parameters, 
based  on  the  fundamental  equations  of  optics,  numeric  simulations  will  be  carried  out.  The  pictures  gained 
by  visualization  of  the  simulation  results  are  compared  with  those  pictures  which  are  experimentally  ob- 
tained, and  students  discuss  the  variations.  Even  the  testing  of  parameter  combinations  before  carrying  out 


* In  this  basic  experiment,  a specimen  of  a transparent  composite  material  containing  an  existing  crack  is  illuminated 
with  light  generated  by  a point  light  source.  The  stress  intensification  in  the  region  surrounding  the  crack  tip  leads  to  a 
reduction  of  both  the  thickness  of  the  specimen  and  the  refractive  index  of  the  material.  As  a consequence,  in  the 
transmission  case,  the  light  passing  through  the  specimen  is  deflected  outwards,  and  a dark  shadow  spot  is  formed  on 
an  image  plane  behind  the  specimen.  The  spot  is  bounded  by  concentration  of  bright  light,  the  caustic.  This  caustic 
contour  determines  the  properties  of  the  material. 
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time-consuming  experiments  is  thus  made  simple.  In  hindsight,  the  longevity  of  this  application  is  remark- 
able. Function  libraries  which  were  developed  in  the  programming  language  FORTRAN  can  still  be  used 
by  modem  compilers  to  simulate  complex  mechanical  processes.  Between  1994  aid  1998,  several  applica- 
tions and  non-related  animations  were  created.  The  didactic  goal  was  to  offer  a means  beyond  mere  text 
and  visuals  by  which  students  could  prepare  for  this  infrequently  offered  course. 
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as  a co-operation  between  the  chairs  of  Technical  Mechanics  and  for  Computers  and  Society,  the  efforts 
within  the  area  of  the  new  media  which  focus  on  new  concepts  of  multimedia  teaching  were  resunied  and 
strengthened. 

The  acronym  mechANIma  establishes  a connection  between  “mechanics**  and  “animation**.  However, 
mechANIma  is  not  only  a computer  program  or  a collection  of  different  materials  for  a mechanics  course, 
but  rather  a new  method  for  teaching  and  understanding  mechanics.  Our  biggest  concern  in  the  project  was 
the  inclusion  of  students  in  the  production  of  materials.  This  goal  led  to  the  phrase,  “Software  created  by 
students  for  students**,  which  remains  an  objective  for  us  today.  This  phrase  can  be  realized  in  auxiliary 
worker  activities  or  as  a part  of  seminary  - and  diploma  theses.  It  was  important  to  us  that  the  chosen  pro- 
gramming language  was  easy  to  learn  and  use,  so  even  students  without  extensive  programming  knowledge 
could  obtain  a quick  aid  easy  introduction  to  programming.  The  programming  language  should  not  contain 
pitfalls  which  normally  lead  to  errors,  and  should  be  capable  of  calculating  complex  simulations.  Indispen- 
sable are  extensive  documentation  and  code  samples  which  can  be  understood  by  the  non- expert  program- 
mer. It  should  also  contain  programming  examples  for  typical  problems  which  arise  in  the  development 
processes.  Therefore,  the  suitability  of  a number  of  popular  programming  languages  and  environments  has 
been  evaluated  and  tested  for  our  special  purposes.  Because  of  its  popularity  as  a teaching  language,  it  is 
assumed  that  future  generations  of  students  will  use  Java.  Therefore,  following  the  creation  of  independent 
modules  in  Visual  Basic  and  Borland  Delphi,  Java  was  finally  selected  as  the  favoured  programming  lan- 
guage for  this  project.  Because  of  the  availability  of  different  development  environments  without  license 
fees,  Java  is  also  well  suited  for  usage  at  universities.  Its  extensive  standard  class  library  allows  Java  to  be 
used  for  developing  user  interfaces.  Another  advantage  is  the  availability  of  Java  for  all  four  significant 
operating  systems  (Windows,  Linux,  MacOS  and  Solaris)  used  in  university  computer  labs.  The  desire  to 
more  clearly  elucidate  complex  mechanical  processes  by  suitable  visualization  methods  was  the  main  mot  i- 
vation  for  the  creation  of  animations  and  simulations  for  learning  purposes.  The  most  important  aspect  is 
that  all  our  animation  systems  visualize  the  precise  effects  and  parameters  addressed  in  the  lectures  and 
tutorials.  Hence,  by  modifying  these  parameters  in  animation  systems,  students  get  a much  better  idea  of 
the  underlying  physical  and  mathematical  effects. 

In  general,  the  interactive  capabilities  of  simulation  programs  are  limited.  Only  particular  simulation  pa- 
rameters may  be  modified  but  the  mathematical  model  of  an  experiment  is  fixed  by  the  program  and  cannot 
be  changed.  Normally  even  small  changes  by  the  user  are  not  possible. 

With  the  intention  of  building  a new  kind  of  interactive  digital  media,  we  began  the  creation  of  mechADO 
(Hampel,  1999),  a construction  environment  which  covers  the  topic  of  static  analysis  of  plane  trusses.  With 
mechADO,  students  are  able  to  construct  a truss  and  apply  forces  to  it  in  an  interactive  way.  The  program 
then  automatically  calculates  internal  and  external  reactive  forces  of  the  construction.  Even  if  mechADO 
lacks  some  important  aspects  of  current  Explorations,  it  was  an  important  step  towards  the  development  of 
the  Explorations-concept.  This  concept  comprises  the  core  of  this  paper,  and  is  further  described  in  the  fol- 
lowing paragraphs. 


The  Exploration  Concept 

Explorations  are  a new  kind  of  highly  interactive  learning  application  which  combine  aspects  of  construc- 
tion, modelling  and  simulation.  They  allow  students  the  graphical  creation  of  their  own  constructions 
whose  mathematical  description  and  calculation  are  executed  by  the  program.  The  results  are  shown 
graphed  and  as  analytic  equations.  The  basic  idea  was  to  support  learning  by  construction  and  immediate 
response  from  the  system.  Designed  as  modular  units,  a single  Exploration  always  covers  one  well-defined 
topic.  Those  modules  should  be  usable  in  different  contexts:  e.g.  lectures,  tutorials,  individual  self- 
instruction,  or  even  in  an  internet -based  cooperative  learning  environment. 
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A piece  of  software  comparable  to  the  Explorations  in  that  it  links  construction  and  simulation  is  Interac- 
tive Physics  2000  (IP),  a commercial  application  produced  and  sold  by  MSC  Working  Knowledge  (see 
http://www.krev.com/products/ip.html),  IP  is  suitable  for  simulating  a wide  range  of  physical  topics  such  as 
kinematics,  kinetics  and  rotation  dynamics.  Virtual  experiments  covering  the  elastic  collision  of  masses  are 
especially  easy  to  simulate  with  IP.  It  was  created  as  a monolithic  application  to  cover  all  the  above  topics 
in  one  program.  The  disadvantage  of  this  is  that  it  may  take  the  user  longer  to  discover  all  the  features  of 
construction  and  parameter  in-  and  output. 

We  tested  IP  as  an  authoring  environment  for  building  animations,  but  soon  reached  the  limitations  of  the 
system.  It  was  proven  to  be  quite  difficult  to  precisely  reproduce  the  exercises  which  students  had  been 
solving  for  years  with  pencil  and  paper.  However,  the  main  reason  for  developing  Explorations  rather  than 
using  IP  in  the  teaching  of  engineering  was  its  approach  to  mechanics.  In  IP,  a physicist’s  point  of  view  is 
taken,  whereas  the  Explorations  view  mechanics  from  an  engineering  perspective. 

However,  the  major  difference  between  IP  and  Explorations  is  that  IP  has  no  means  of  describing  the  con- 
structed experiment  with  an  analytic  equation. 

The  ambitious  goals  of  the  Explorations  were  fulfilled  in  a two-year  project.  Applications  which  are  now  in 
use  have  been  developed  for  fundamental  topics  in  mechanics,  including  statics,  dynamics  and  failure  theo- 
ries. It  was  not  our  aim  to  create  an  entire  multimedia  script,  but  rather  a few  modules  which  cover  signifi- 
cant topics  and  are  usable  in  a highly  interactive  way.  That  is  why  the  Explorations  are  not  bound  to  a spe- 
cific learning  medium,  but  can  be  used  in  many  different  environments.  In  an  internet -based  usage,  such  as 
in  our  prototypical  environment,  the  Explorations  can  be  started  using  Java  Web  Start  in  the  browser.  On  a 
CD-ROM,  the  Explorations  can  be  started  immediately  without  any  further  software.  Two  out  of  eight  con- 
structed modules  are  presented  in  detail  below. 

Example  of  an  Exploration;  one-dimensional  spring-absorber-mass  system 

A fundamental  topic  in  the  study  of  engineering  are  vibratory  systems.  In  electrical  engineering,  they  can 
be  implemented  using  a combination  of  inductive  and  capacitive  components,  and  in  mechanics  they  can  be 
implemented  with  masses  and  springs.  For  this  part  of  a mechanics  course,  the  Exploration  Swingx  should 
enable  free  construction  of  a vibratory  mechanic  system  with  various  components. 
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Figure  1;  Exploration  Swingx  (one- dimensional  spring- absorber -mass  system) 

With  this  Exploration,  it  is  possible  to  demonstrate  the  element  which  mainly  describes  such  a system;  that 
is,  the  connection  between  drawing,  equation  of  motion  and  the  motion  itself  The  picture  above  shows  one 
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student’s  construction  and  how  the  equation  of  motion  is  deduced  and  presented.  In  this  way,  the  student 
simultaneously  sees  the  graphical  and  mathematical  representation  of  the  mechanical  system.  The  follow- 
ing paragraph  deals  with  this  specific  feature  of  the  instructional  program,  which  is  called  multi-level  rep- 
resentation. 

Multi-level  representation  in  learning  modules 

Technical  systems  can  be  described  not  only  with  drawings,  but  also  through  mathematical  notation  (e.g.  an 
equation).  Because  the  mathematical  notation  can  automatically  be  deduced  from  some  kind  of  construc- 
tion, it  is  possible  to  transfer  this  work  to  a computer  program.  Thus,  the  most  important  feature  of  an  Ex- 
ploration beside  the  visualisation  of  the  numeric  solution  (i.e.  in  a graph  or  animation  of  the  drawing)  is  the 
ability  to  depict  the  analytical  solution.  These  two  ways  of  representing  a mechanical  system  can  be  viewed 
simultaneously  in  order  to  help  the  student  comprehend  the  complex  structures. 

The  multi-level  representation  and  the  ability  of  the  student  to  change  the  construction  with  its  components 
and  connections  demonstrate  the  correlations  that  describe  the  system . Thus,  different  points  of  view  can  be 
developed  for  a single  task  and  set  in  correlation  to  one  other.  This  will  help  the  student  to  gain  a deeper 
understanding  of  the  topic. 

In  a nutshell,  the  main  task  of  an  Exploration  is  to  simulate  and  animate  systems  constructed  by  students 
and  to  ensure  consistency  between  the  different  ways  of  describing  this  mechanic  system. 

Example  of  an  Exploration:  Calculation  of  internal  forces  for  straight  beams 

A comprehensive  topic  in  technical  mechanics  is  the  static  analysis  of  plane  trusses.  An  introduction  to  this 
topic  generally  uses  the  example  of  straight  beams.  Therefore  the  Exploration  Beamex  is  a learning  tool  in 
which  students  design  the  load  and  bearing  of  beams.  Afterwards  the  resulting  internal  forces  are  visual- 
ised. 
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Figure  2:  Exploration  Beamax  (internal  forces  in  a beam)  with  an  example  construction 

A selection  of  bearings  is  given,  and  the  correct  combination  placed  under  the  beam  allow  the  calculation 
of  the  internal  forces.  Such  an  example  is  shown  in  the  picture  above.  Distributed  forces  can  be  combined 
and  superposed  in  various  ways  on  the  beam,  leading  to  different  internal  forces.  In  a frame  beneath  the 
designed  structure  it  is  possible  to  visualise  different  internal  force  diagrams.  Particular  to  the  Exploration 
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is  not  only  the  depiction  of  reactive  forces  in  the  schematic  as  force  vectors,  but  also  a corresponding  calcu- 
lation formula. 

It  is  possible  to  change  the  construction  even  after  the  calculation.  As  a result,  the  student  can  continuously 
vary  the  loads  and  simultaneously  observe  the  new  diagram. 

To  summarize.  Explorations  have  capabilities  which  surpass  those  of  previous  tools  in  several  ways.  They 
allow  students  to  design  on  their  own,  with  error-checking  by  the  system  to  prevent  impossible  construc- 
tions. They  enable  the  student  to  observe  and  link  both  the  visual  and  mathematical  descriptions  of  me- 
chanical systems.  Furthermore,  they  can  be  integrated  into  different  learning  environments. 

Conclusion 

With  the  Explorations,  a new  kind  of  higjily  interactive  learning  application  was  developed  for  engineering 
science  which  allows  the  construction  and  simulation  of  virtual  experiments  by  the  students  themselves. 
What  is  more,  the  Explorations  represent  a new  quality  of  learning  program;  they  combine  the  graphical 
representation  of  technical  systems  with  their  corresponding  mathematical  equations  and  numerical  simula- 
tions. 

Future  Work 

The  discoveries  made  during  this  two-year  project  are  entering  a new  stage  of  development,  forming  the 
basis  for  cooperative  Explorations.  After  implementing  a communication  interface,  these  Explorations  will 
become  a part  of  the  cooperative  learning  environment  sTeam  (see  Open  sTeam  2001  and  Hampel  & Keil- 
Slawik  2001).  Explorations  which  implement  the  so  called  sTeam  module  framework  could  use  compo- 
nents for  synchronous  and  asynchronous  communication  as  well  as  awareness  components  directly  from 
the  sTeam  environment. 

With  cooperative  Explorations,  distributed  teams  of  engineering  students  can  be  formed,  allowing  them  to 
receive  feedback  from  each  other,  as  well  as  from  the  simulation  system. 
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Abstract  Business  organizations  and  their  employees  face  ever-increasing  complexity  and 
accelerating  changes  in  their  everyday  work.  This  brings  along  the  need  for  training  models 
able  to  facilitate  the  development  of  knowledge  and  skills  needed  in  this  kind  of  environment. 
This  paper  introduces  a certain  computer-based,  continuously  processed  business  simulation. 
The  purpose  of  the  described  learning  environment,  created  with  the  use  of  highly  interactive 
sim  ulation  is  to  represent  a realistic  view  of  business  processes  to  the  students  and,  enhance 
students'  perception  of  business  processes  as  a whole.  We  introduce  some  pedagogical  and 
theoretical  guidelines  for  designing  of  this  business  game,  and  describe  also  an  example  of  a 
learning  environment  based  on  the  use  of  this  game.  Some  preliminary  results  from  the 
research  based  on  an  example  course  are  also  presented. 


Introduction 

Modem,  so-called  knowledge  society  induces  many  new  challenges  to  human  competence  and  skills  required  for 
learning  and  working.  As  a result  the  ways  of  working,  studying,  and  collaborating  are  changing  dramatically. 
Work  is  increasingly  becoming  organized  in  teams  and  groups,  and  often  supported  by  technology.  Other  main 
characters  of  the  modem  work  are  the  distribution  of  expertise  and  working  in  networks,  both  of  which  require 
social  and  collaborative  skills  to  share  and  constmct  together  knowledge,  ideas,  symbolic  artifacts  etc.  (Lehtinen, 
2000).  From  the  pedagogical  perspective,  students  need  to  learn  to  learn,  develop  metacognitive  capabilities  and 
constmct  meaningful  knowledge  in  interaction  with  others  and  with  tools  afforded  by  activity  environments. 
This  is  tme  especially  in  the  fields  of  science,  but  also  in  business  world. 

Our  rationale  for  using  a dynamic  simulation  in  teaching  within  the  domain  of  business  comes  from  the  alarming 
findings  from  previous  studies.  Namely,  the  biggest  challenges  and  obstacles  in  business  education  as  well  as  in 
other  subject  domains  are  difficulties  in  applying  theoretical  subject  knowledge  in  real  life  settings,  inabilities  to 
handle  complex  and  ill-defined  problems,  and  lack  of  consistent  and  holistic  conception  of  business  processes. 
All  of  these  problems  are  more  or  less  connected  to  the  question  of  contextualizing  the  content  being  learned.  If  s 
been  found  that  graduate  students  of  business  management  are  not  well  prepared  for  the  complex  problems  and 
situations  they  will  face  in  real-life  working  contexts.  That*s  partly  due  to  the  traditional  teaching  methods, 
which  don’t  help  students  to  face  the  complexity  and  cope  with  ambiguity  and  uncertainty  of  real  business  world 
they  will  inevitably  face  when  taking  positions  in  working  organizations  (Aram  & Noble,  1999).  These 
traditional  forms  of  instmction  (e.g.  lectures)  are  often  found  to  produce  inert  knowledge,  which  cannot  be 
applied  in  complex  situations  (Bransford  et  al.,  1991),  and  even  intermediate  experts  of  economics  may  have 
enormous  difficulties  in  applying  their  knowledge  and  skills  in  realistic  problem-solving  situations  (Mandl  et  al., 
1994).  One  of  the  main  reasons  for  the  inabilities  of  traditional  teaching  methods  to  facilitate  the  development  of 
flexible  and  useful  knowledge  and  skills  is  the  lack  of  contextualizing  or  anchoring  the  content  being  learned.  If 
content  is  separated  from  its  authentic  context  were  content  knowledge  and  skills  are  used  in  real  life,  it  will 
produce  inert  or  impoverished  knowledge.  In  such  impoverished  environments,  learning  becomes  the 
memorization  of  seemingly  abstract,  self-contained  entities,  not  useful  tools  for  understanding  and  interacting 
with  the  world  (Barab  et  al. , 2000).  Furthermore,  there  exists  a lack  of  integration  between  different  functional 
areas  of  business  studies  (e.g.  accounting,  marketing,  management),  which  makes  it  very  difficult  for  students  to 
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develop  coherent  mental  models  about  the  business  world  and  strategies  in  whole  (Selen,  2001).  This  is  a clear 
example  of  stripping  content  out  of  context  and  conditions  in  which  it  is  used;  separating  different  domain  areas 
from  each  others  and  from  their  realistic  intertwined  existence.  Walter  and  Black  (2000)  have  recognized  similar 
problem:  what  is  needed  in  business  education  is  a process  approach.  They  state  that  the  flexible  aid 
organizational  understanding  of  the  business  world  can  only  be  accomplished  by  presenting  the  study  of  business 
as  a series  of  integrated  activities  (c.f.  connecting  the  areas  together  in  realistic  way).  However,  all  of  this 
embedding  of  learning  experences  in  practical  situations  or  contexts  must  not  neglect  the  learning  of  formal 
knowledge  and  construction  of  abstract  ideas  (Lehtinen,  2002).  On  the  basis  of  the  issues  discussed  above 
changes  are  definitely  needed  in  the  teaching  methods  in  the  domain  of  business  studies,  and  one  possible 
answer  for  these  problems  might  be  the  use  of  a dynamic  computer  simulation.  The  main  reason  for  this 
simulation  use  is  to  create  an  authentic  and  collaborative  learning  environment,  where  computer  simulation  is 
used  as  a tool  for  situating  the  business  content  being  learned  in  its  authentic  contexts  in  challenging  way. 

The  Case  of  Dynamic  Business  Game 

We  assume  that  the  potential  of  simulations  and  games  in  business  education  is  not  fully  exploited.  One  of  the 
reasons  for  this  is  that  present,  traditional  batch-processed  business  games  neither  realistically  represent  the 
actual  processes  occurring  inside  and  outside  organizations,  nor  correspond  to  the  real-world  decision-making 
situations.  However,  there  are  no  technical  obstacles  for  constructing  continuously  processed  business 
simulations  that  differ  from  the  main  stream  of  business  games  by  how  the  participants  experience  the  subject 
matter  of  the  learning  session.  So  far  the  computer  based  business  games  have  worked  in  batch-process  mode. 
Whicker  and  Sigelman  (1991)  describe  how  ‘business  strategy  games’  are  processed:  “Typically,  the  player 
feeds  information  into  a computer  program  and  receives  back  a series  of  optional  or  additional  data  that  are 
conditional  upon  the  player’s  initial  choices.  The  game  proceeds  through  several  series  of  these  interactive, 
iterative  steps.”  The  problem  with  the  batch-processing  method  is  that  world  very  rarely  works  in  such  a pure 
sequential  order.  There  are  hardly  any  business  areas  where  the  decision  makers  first  enter  all  their  decisions  for 
the  next  budgeting  term,  then  rest  during  all  the  actual  term,  and  enter  again  the  business  in  the  end  of  the  term  to 
analyze  the  term  results  executed  and  to  prepare  the  next  budget. 

The  simulation.  Dynamic  Business  Game  (DBG),  we  are  introducing  in  this  paper  is  in  several  ways  a typical 
business  game,  and  the  aim  of  the  new  construction  is  the  same  as  with  the  conventional  business  games:  to 
increase  business  perception  of  the  participants . However,  as  the  new  construction  is  operated  in  a different  way 
compared  to  conventional  games,  it  brings  along  some  advantages  not  met  in  conventional  games.  What  is 
suggested  here  is  a real-time  processed  business  simulation.  Decision-making  and  having  results  from  the 
decisions  made  should  be  in  interactive  real-time  mode  as  they  are  in  the  real-world  environment.  Interactive 
mode  means  that  decisions  are  made  continuously  when  in  the  game  model  and  game  market  situations  occur 
which  need  to  be  reacting  to  by  the  participants.  In  the  interactive  model  decisions  are  made  as  soon  as  they  are 
needed  or  at  least  as  soon  as  the  decision-maker  notices  that  the  market  situation  needs  actions  from  him. 

DBG  is  a computer-based  simulation,  which  creates  a complex  and  authentic -like  environment  for  learning  of 
business  studies.  Ifs  developed  by  one  of  the  authors  of  this  paper.  DBG  models  the  managing  of  a 
manufacturing  company  with  main  decision-making  functions,  and  a market  engine  creating  demand  and  supply. 
This  is  done  by  a computer  model  representing  realistic  contexts,  in  which  students  are  required  to  perform 
complex  problem  solving  and  discover  functional  market  strategies  when  running  their  virtual  company. 
Therefore  DBG  can  be  defined  as  certain  kind  of  a hybrid  form  of  simulation  between  typical  types  of 
conceptual  model  and  operational  model  simulation  as  classified  by  de  Jong  and  van  Joolingen  (1998).  On  one 
hand  DBG  requires  discovery  inferring  the  simulation  model  as  a way  to  achieve  functional  business  strategy 
(conceptual  model),  and  on  the  other  hand  it  urges  an  active  functioning  and  responding  to  that  model 
(operational  model).  DBG  can  also  be  seen  as  a combination  of  educational  games,  experiential  and  conceptual 
simulations  as  divided  by  Gredler  (1996),  because  DBG  environment  contains  competition  between  participants, 
it  places  students  into  responsible  roles  in  complex  situation,  and  requires  discovering,  predicting  and  developing 
of  mental  models  about  business  functions.  What  makes  DBG  very  special  is  the  changing  nature  of  its  business 
model  as  a result  of  all  participating  companies’  actions.  That’s  why  this  game  is  called  as  dynamic. 
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Theoretical  Principles  of  Dynamic  Business  Game  Simulation 

The  most  important  theoretical  principles  hat  have  guided  our  efforts  to  design  DBG  can  be  described  as 
follows : 

• PROVIDE  REALISTIC  AND  COMPLEX  MODEL  OF  BUSINESS  FUNCTIONS.  DBG  provides  dynamically 
changing,  rich  information  about  the  current  situation  of  companies  and  market  trends  in  whole  as  a reaction 
to  the  students’  actions  and  decisions  made  in  the  managing  of  their  firms  in  an  interaction  with  other 
companies’  inputs.  The  main  idea  is  to  embed  learning  in  realistic  and  relevant  contexts. 

• JlSiR  4T  iTMP.hJTfC  tools  AND  MULTIPLE  REPFE^SENTA.TIONS.Th^T^  are  realistic  tools  to  examine  this 
rich  information  provided  by  the  simulation  (e.g.  different  market  and  cash  flow  reports),  which  might 
support  the  understanding  of  the  business  functions  more  deeply  and  facilitate  interaction  between  students 
(DBG  is  functioning  as  a shared  frame  of  reference) 

• FACILITATE  CONTINUOUS  PROBLEM  SOLVING.  DBG  is  functioning  in  real-time  -based  virtual  market 
in  LAN  network.  Therefore  the  time  to  react  and  make  decisions  is  an  essential  element  of  successful 
business  fiinctio  ning,  and  the  students  need  to  continuously  watch  the  market,  interpret  available  information 
and  if  necessary  change  their  strategies. 

• EMBED  LEARNING  IN  SOCIAL  EXPERIENCE.  DBG  is  designed  to  be  used  in  small  groups,  because 
learning  by  its  very  nature  is  a social  and  collaborative  process.  DBG  tries  to  facilitate  task  oriented 
interaction,  social  negotiation  and  collaborative  meaning  making  by  demanding  the  students  to  discuss  about 
their  decisions  in  the  management  of  their  company. 

• SUPPORT  STUDENTS’  LEARNING  BY  TAILORING  THE  GAME  There  are  opportunities  to  adjust  the 
simulation  to  support  the  needs  of  learners  (e.g.  the  speed  of  game  clock  can  be  changed  or  stopped,  the 
model  can  be  made  more  complex  during  the  gaming,  the  company  context  with  all  of  its  variables  can  be 
modified,  the  supply  and  demand  of  the  market  can  be  changed,  economical  depression  is  possible  to  model 
etc.)  (c.f  de  Jong,  200 1 ) 

The  objectives  of  using  DBG  can  be  very  diverse  and  versatile.  For  example  the  students  can  learn  substance 
knowledge  and  skills  of  business  studies,  enhance  their  collaboration  and  interaction  skills,  learn  to  face  and 
master  the  complexity  of  real  economical  world,  and  acquire  flexible  and  robust  competence  which  is 
transferable  to  other  contexts  and  real-life  situations.  In  short,  the  working  with  simulation  aims  to  facilitate  both 
formal  and  informal  knowledge.  The  main  pedagogical  basis  behind  this  kind  of  a learning  environment  is  the 
idea  of  “learning  with  technology”  (Jonassen  et  al.,  1999),  which  is  congruent  with  constructivist  perspective  as 
seeing  learning  as  an  active  meaning  making  and  knowledge  building  in  an  interaction  between  learner,  available 
tools  and  his/her  social  and  physical  environment.  In  this  view  the  role  of  computer  technology  is  to  encourage 
the  thinking  of  the  learners,  facilitate  the  mindful  engagement  with  a deeper  level  of  learning,  support  the 
collaboration  between  participants,  and  function  as  a cognitive  partner  of  learners.  The  essential  feature  of  this 
learning  environment  is  the  use  of  simulation  in  small  groups  (or  teams),  in  such  a way  that  every  small  group 
manages  their  own  company.  Collaborative  action  becomes  very  important,  because  every  decision  has  to  be 
discussed  and  processed  in  small  groups  the  same  way  as  in  real  business  contexts. 

DBG  is  highly  interactive  in  several  ways  (see  Fig . 1 ).  First,  there  is  collaboration  between  the  participating 
students  within  each  team  around  the  same  computer.  Secondly,  the  participants  enter  their  decisions  to  the  game 
decision-making  application  (computer  model  of  manufacturing  company)  in  their  computer.  Thirdly,  each 
company  application  is  in  continuous  interaction  with  the  market  server.  Actually  simulation  does  not  operate  in 
true  real-time  processing.  However,  the  processing  is  continuous  in  the  sense  that  the  game  time  is  clock-driven, 
smallest  increment  of  time  being  one  hour,  and  the  participants  are  not  tight  to  make  decisions  in  specified  points 
of  time  but  can  make  decisions  whenever  during  the  game  they  choose.  Fourth,  all  the  companies  are  indirectly 
in  connection  to  each  other  through  the  market  server.  So,  the  market  server  with  the  contributions  and  actions 
made  by  each  participating  company  form  a kind  of  a virtual  market.  This  virtual  market  with  its  human 
component  is  much  richer  and  more  authentic  than  a pure  compute^based  model  of  the  functioning  of  the 
business  world  alone.  DBG  includes  all  the  major  business  decision  functions,  which  correspond  to  the  realistic 
contexts  found  in  business  world.  The  idea  is  that  students  can  use  any  of  the  tools  and  information  resources 
available  at  any  time.  Also  multiple  tools  can  be  used  simultaneously.  In  a way  the  working  with  DBG  resembles 
continuous  problem  solving.  Figure  2 represents  the  game  user  interface  (company  decision-making  application). 
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Figui-e  1:  Interactivity  in  DBG  Figui'e  2:  Literface  oftlie  decision-making  application 


However,  to  overcome  the  difficulties  in  simulation  environments  found  in  previous  studies  (e.g.  de  Jong  & van 
Joolingen,  1998)  it  is  also  possible  to  arrange  other  instructional  supporting  activities  besides  mere  simulation 
working.  Our  own  approach  was  home  assignments,  which  supported  and  focused  student  processing  towards 
the  relevant  contents  and  variables  in  business  simulation.  A typical  assignment  was,  e.g.,  to  carry  out  different 
business  analysis  and  designing  future  business  strategies  for  their  virtual  firm,  and  then  to  apply  these  in  the 
next  simulation  session.  We  also  assigned  them  to  create  an  Excel  spreadsheet  (which  they  could  use  afterwards 
during  working)  for  calculating  profit  and  other  key  figures.  These  additional  arrangements  were  to  engage  the 
students  in  a deeper  processing  of  the  substance  area.  Other  supporting  feature  which  was  build  in  this 
simulation  was  so  called  model  progression  (e.g.  White  & Frederiksen,  1990)  method,  which  means  that  the 
complexity  of  simulation  is  gradually  increased.  This  was  done  by  accelerating  the  game  clock  speed  and  adding 
more  market  areas  into  the  game  during  the  course.  We  hypothesize  that  this  would  help  students  in  the 
beginning  to  get  used  to  the  simulation  and  to  decrease  their  cognitive  load.  In  all,  we  assume  that  with  DBG  it  is 
possible  to  create  a learning  environment,  which  supports  constructive,  situated  and  social  processes  of  learning 
(Nurmi  & Jaakkola,  in  press). 

Overview  of  the  Research 

Our  first  research  of  a DBG  based  learning  environment  was  conducted  in  autumn  2001.  The  study  course  was 
arranged  in  higher  education  level,  and  19  students  of  information  systems  and  computer  science  participated  in 
the  course.  The  course  consisted  of  24  hours  of  wrking  with  DBG  (6  sessions  of  4 hours)  and  in  addition  to 
several  home  assignments.  The  DBG  was  used  in  the  groups  of  three  students,  so  there  were  seven  companies  in 
virtual  market  competition.  We  have  collected  large  amount  of  both  quantitative  (financial  figures,  attitudes 
towards  ICT,  teamwork  and  business  studies,  feedback  from  the  course,  log  data  from  simulation,  and  scores  of 
pre-  and  post-tests  of  substance  knowledge)  and  qualitative  data  (observation  data  of  group  working,  reflecting 
the  roles  of  group  members).  The  purpose  of  our  research  was  to  understand  the  actual  collaborative  learning  and 
working  processes  with  simulation,  and  to  find  out  what  are  the  effects  of  simulation  course  on  students 
substance  knowledge  and  attitudes  towards  ICT,  team  work  and  business  studies. 

Preliminary  Results  and  Discussion 

At  the  moment  only  part  of  the  data  is  analyzed.  Qualitative  process  data  from  the  observation  of  small  groups 
will  be  analyzed  in  the  near  future,  as  well  as  companies’  log  data  and  financial  progress  in  the  game.  These 
preliminary  results  are  based  only  on  the  effects  of  simulation  working  and  feedback  and  experiences  from  the 
students.  When  reviewing  students*  scores  on  pre-  and  posttest  questions,  it  can  be  seen  that  simulation  course 
had  statistically  significant  (T-test  p < .01;  Wilcoxon  test,  value  of  z = -2.672,  p < .01)  effects  on  scores  of 
substance  knowledge  (see  table  1).  The  effects  of  playing  to  scores  were  significant,  but  the  overall  increase  in 
score  was  quite  modest,  average  scores  increased  only  from  12.7  to  1 3.9 points  (max  21). 
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Repeated  measures  T-Test 


test  scores 

mean 

N 

St  dev. 

t- value 

df. 

sig*  (p-value) 

pretest  score 

12.69 

16 

3.301 

posttest  score 

13.88 

16 

3.243 

-3.230 

15 

0.006** 

♦♦  p<.01 

Table  1:  Effects  of  simulation  course  to  the  scores  on  pre-  and  posttest  of  substance  knowledge. 


We  were  not  quite  satisfied  with  our  test  questions,  which  were  concentrating  too  much  on  fact  knowledge,  and 
the  content  of  our  problem  solving  tasks  didn’t  match  too  well  with  the  issues  students  had  to  face  during 
simulation  working.  It  seems  to  be  the  case  that  by  its  very  nature  the  results  of  simulation  working  are  so 
qualitatively  different  from  just  acquisition  of  new  factual  knowledge  that  those  effects  cannot  be  detected  by 
traditional  knowledge  tests  (c.f.  Swaak  & de  Jong,  1996).  Swaak  et  al.  (1998)  concluded  that  it  is  not  clear  how 
the  effects  of  learning  from  simulation  are  to  be  measured.  Further  they  inferred  that  simulation  working 
produces  intuitive  (or  implicit  or  tacit)  knowledge,  which  tends  to  be  difficult  to  verbalize  and  to  measure.  As  a 
consequence  in  following  studies  we  have  changed  our  test  questions  much  more  towards  problem  solving  and 
knowledge  applying  tasks  as  well  as  concept  mapping  assignments,  because  simulations  are  more  valuable  in 
facilitating  deeper  understanding  and  complex  problem  solving  skills  than  in  teaching  factual  knowledge.  In 
future  it  would  also  be  interesting  to  compare  DBG  with  more  traditional  batch-processed  business  game. 

We  also  ask  participating  students  to  reflect  their  experiences  and  give  feedback  from  simulation  course  and 
especially  from  the  simulation  itself  According  to  these  comments  several  conclusion  can  be  made.  First  of  all, 
all  the  respondents  regarded  DBG  as  authentic.  They  thought  that  the  questions  and  problems  they  were  dealing 
with  during  the  game  can  also  be  faced  in  real  working-life  context  of  any  manufacturing  company.  Participating 
students  evaluated  that  DBG  represent  authentic  and  complex  business  processes  rather  realistic  way.  For 
example  one  claims  that  “Situations  and  problems  faced  while  playing  felt  very  realistic.  Also  game  business 
model  looked  like  real,  but  someway  it’s  little  simplified  (e.g.  customers  are  not  always  so  rational  etc.). 
However,  I Mnk  that  it  is  impossible  to  model  all  the  factors  in  business  world  in  computer  game”.  The 
complexity  of  business  world  was  also  seem  to  transmitted  to  students  , which  can  be  seen  from  these  comments  : 
“What  made  this  playing  to  felt  so  real,  was  that  you  had  to  take  care  so  many  things  simultaneously  to  get  the 
firm  do  well  ” and  “I  realized  that  enterprise  is  always  a risk.  You  can  7 be  sure  that  even  considered  decisions 
will  lead  to  anything  but  a loss.  ” 

Secondly,  the  simulation  was  valued  as  very  engaging  (or  immersive)  and  the  working  with  it  was  experienced 
as  meaningful  and  interesting.  Once  the  game  started  some  of  the  students  didn’t  want  to  stop  playing  when  the 
course  ended.  “I  could  say  that  I was  little  addicted  to  game,  it  would  be  fun  to  keep  on  playing  longer”  said  one 
participant.  Especially  real  time  element  of  the  game  was  seen  as  very  important  feature  which  affects  on 
authenticity  and  engagement,  because  real  timing  “makes  it  possible  to  see  the  consequences  of  own  decisions 
and  actions”,  “...shows  how  important  it  is  to  make  your  decisions  as  quick  as  possible  and  before  your 
competitors”,  and  “...forces  to  observe  the  market  situation  analyze  the  actions  of  other  companies  and  revise 
your  strategy”. 

Thirdly,  especially  participants  assessed  the  collaboration  around  simulation  as  very  fruitful  and  useful.  They 
said  that  simulation  can  function  as  a shared  frame  of  reference  which  allows  talking  about  difficult  issues  even 
without  correct  concepts.  This  can  be  seen  e.g.  from  this  response:  “Our  team  work  succeeded  well,  and  we  were 
all  aiming  to  get  our  company  to  show  a profit.  I could  say  that  the  working  was  very  intensive  during  the  whole 
course  and  the  game  inspired  our  discussion  ”.  Basically  one  can  claim  that  the  collaboration  in  small  groups 
brought  additional  value  to  wholeness  of  simulation  working.  Collaborative  processes  seems  to  be  much  more 
than  just  a sum  of  its  parts  (the  characteristics  and  abilities  of  group  members),  because  none  of  the  background 
variables  (attitudes  to  ICT  and  team  work,  learning  orientations,  scores  of  pretest)  couldn’t  clearly  explain  the 
successfulness  of  small  groups  in  the  virtual  market  of  game.  It  clearly  seems  that  in  order  to  understand  learning 
with  simulation,  we  have  to  concentrate  on  analyzing  the  collaborative  processes  around  computer  occurring  in 
small  groups. 

Students  also  saw  DBG  as  very  motivating  teaching  and  learning  method  for  business  education  - especially 
compared  to  more  traditional  instructional  methods , and  based  on  their  opinions  it  can  be  said  that  simulation 
could  maintain  task  related  orientation  during  the  whole  course.  For  instance  one  student  said  that  “...maybe  the 


most  motivating  aspect  of  the  game  was  the  competition  against  other  companies  ”,  while  some  other  regarded 
the  meaningful  problems  and  situations  as  the  real  source  of  motivation.  However  students  also  saw  the 
importance  of  traditional  teaching  methods:  "Although  playing  is  fun  and  it  gives  the  big  picture  of  business 
processes,  it’s  not  enough.  It  would  be  good  to  combine  theoretical  lectures  and  concrete  playing  in  the  same 
coursi\  One  suggested  mode  to  use  this  simulation  was  the  knowledge  applying  approach:  "...game  is  a perfect 
opportunity  of  testing  what  you  know  before  in  a safe  environment.  It  lets  you  simulate  different  approaches  and 
their  effects”.  When  asking  students  to  reflect,  ^\ilat  the  most  important  thing  they  have  learned  during  the 
simulation  course  was,  majority  stated  that  it  was  acquiring  the  consistent  conception  of  business  processes  as 
whole.  They  said  that  they  could  now  understand  how  many  different  factors  are  affecting  to  successfulness  of 
business  company.  For  example  one  stated  that  "...the  most  important  thing  that  I have  learned  is  how  different 
parts  of  a company  interact  and  what  kind  of  things  you  have  to  take  in  considerations  in  such  a company  as 
this”  which  he  explained,  was  because  "...you  could  see  the  flow  of  the  whole  process  of  the  product  from  the 
raw  materials  to  the  end  product  and  the  different  procedures  in  between  Unfortunately  this  development  of 
holistic  picture  about  business  processes  could  not  be  catch  in  our  test  questions.  In  all,  the  overall  responses 
about  the  whole  course  were  very  positive  without  exceptioa 

In  conclusion,  on  the  basis  of  this  research,  we  state  that  simulation  working  can  be  regarded  as  authentic  and 
very  engaging  as  well  as  meaningful  and  motivating  for  students,  and  it  could  facilitate  the  development  of 
deeper  understanding  about  realistic  business  processes  as  whole.  However,  our  qualitative  analyses  about  small 
groups’  working  processes  are  not  done  yet,  and  they  will  shed  new  light  on  many  of  our  research  questions  and 
conclusions  in  future. 

At  the  moment  we  have  conducted  a follow^lp  research  where  students  with  different  levels  of  exp  ertise  have 
been  compared.  We  examined  how  novices  and  intermediate  experts  are  working  with  simulation.  The  research 
focus  is  on  the  questions  such  as : does  the  students  with  different  level  of  substance  knowledge  and  expertise  in 
economics  work  and  use  DBG  differently,  does  they  use  various  business  strategies  (e.g.  trial-and -error 
experiments  or  planned  theoretical  strategies  based  on  the  theories  of  economics)  during  the  game , does  they 
succeed  differently  in  the  competition  of  virtual  market,  and  does  they  benefit  equally  in  tests  as  an  effect  of 
playing?  The  results  of  this  study  are  reported  in  future. 
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Abstract:  Retention  has  been  indicated  as  one  of  the  greatest  weaknesses  in  online 
instruction.  While  a preponderance  of  literature  about  online  instruction  is  available, 
concrete  ideas  about  how  to  retain  students  are  lacking.  A DHHS  grant  for  over  $1 
million  dollars  has  made  it  possible  for  us  to  provide  an  Internet  option  enabling 
registered  nurses  to  obtain  a bachelor's  degree.  Now  in  the  third  year  of  the  program, 
we  have  maintained  an  excellent  retention  rate  ranging  from  85  to  93%.  Factors 
found  to  positively  influence  retention  included:  enhancing  the  comfort  level  of 
students  with  the  technology,  creating  a sensitive  online  faculty  persona  generating 
trust  in  the  environment,  and  addressing  safety  and  security  needs  to  support  highly 
interactive  experiences.  These  strategies  promoted  the  development  of  relationships 
with  “classroom”  colleagues  and  fostered  linkages  necessary  for  students  to  remain 
connected  to  the  learning  experiences  and  continue  to  degree  completion. 


Student  Retention  in  the  Web  Environment 

Despite  great  attention  to  appropriate  design  for  online  courses,  high  student  attrition  and/or  lack  of  re- 
enrollment continues  to  be  the  dominant  concern  about  web-based  learning.  The  literature  addressing  web- 
based  learning  contains  few  practical  suggestions  for  connecting  with  students  online  to  retain  the  high-touch  in 
a high-tech  environment.  A study  conducted  at  one  small  liberal  arts  university  (Lynch,  2001),  reported  student 
dropout  rates  from  Internet  courses  were  as  high  as  35-50%,  compared  to  14%  for  traditional  classes.  Lack  of 
feeling  connected  to  other  students  has  been  cited  as  a major  reason  for  students  dropping  out  of  online 
education.  “This  feeling  of  isolation  is  credited  for  the  high  attrition  rates  of  this  type  of  instruction.”  (Link  & 
Scholtz,  2000,  p.  275)  Obviously,  poor  retention  would  preclude  the  viability  of  online  learning  unless 
strategies  to  increase  student  success  and  control  costs  related  to  student  attrition  can  be  reversed.  How  good 
can  the  method  be  if  students  do  not  to  continue  in  the  environment? 

A DHHS  grant  for  over  $1  million  dollars  provided  the  opportunity  to  develop  an  Internet  option  enabling 
registered  nurses  to  obtain  a bachelor's  degree  in  nursing.  Analysis  over  three  years  of  program  delivery  has 
demonstrated  a remarkable  retention  rate  of  85  to  93%.  What  factors  affect  student  retention?  We  need  to 
attend  to  preparing  students  for  this  brave  new  world  of  learning.  Greater  attention  needs  to  be  paid  to  directing 
the  students*  earliest  online  experiences.  In  redesigning  this  program  for  web-based  delivery,  it  was  rapidly 
discovered  that  student  comfort  levels  and  faculty  persona  made  a tremendous  difference  in  the  student*s 
perception  of  their  potential  for  success.  Students  must  feel  safe  and  secure  in  the  environment  before  they  can 
develop  the  level  of  trust  needed  for  positive  learning  and  growth  to  occur.  Trust  is  more  difficult,  but  not 
impossible,  to  achieve  when  the  learner  is  virtual.  Trust  is  also  essential  for  engaging  students  in  highly 
interactive  learning  experiences.  Students  must  feel  free  to  take  risks  and  challenge  assumptions.  Without  this 
level  of  comfort  between  the  student  and  faculty,  they  will  be  reticent  to  dispute  ideas  and  stretch  their  thinking. 
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They  will  also  find  the  learning  experience  very  isolating  and  many  will  retreat  to  the  comforts  of  a classroom 
setting  or  be  lost  to  continuing  education  entirely.  This  presentation  will  describe  the  strategies  found  useful  in 
establishing  trust  and  comfort  that  is  essential  for  student  retention. 


Comfort  Level  with  Technology 

One  challenge  of  online  learning  is  the  lack  of  learner  familiarity  with  the  delivery  system.  Therefore,  it  is 
important  to  remember  that  transitioning  courses  for  online  learning  requires  far  more  than  a simple  conversion 
to  a web-based  format.  Lack  of  face-to-face  contact  with  the  learner  requires  new  ways  of  thinking  about  how 
to  connect  with  students.  One  of  the  first  priorities  for  faculty  is  to  assure  that  the  technology  does  not  get  in  the 
way  of  establishing  this  connection.  "Comfort  with  technology  is  key  to  whether  collaboration  takes  place." 
(Boettcher  & Conrad,  1999,  p.90)  Students  need  to  focus  on  the  learning  and  not  be  distracted  by  the 
underlying  technology.  Therefore,  faculty  need  to  generate  ways  of  assisting  students  to  become  comfortable 
with  the  delivery  system.  An  important  and  useful  method  for  achieving  this  goal  was  the  development  of  a 
technical  training  manual.  This  manual  contained  step-by-step  instructions  for  activities  that  students  would 
perform  in  the  online  courses.  These  activities,  or  “boot  camp  challenges”,  gave  students  practice  taking  online 
quizzes,  using  the  e-mail  system  to  introduce  themselves  to  their  classmates,  posting  messages  on  a discussion 
board,  discussing  group  assignments  in  the  chat  room,  and  uploading  assignments  to  the  course  faculty,  etc. 
These  exercises  enabled  students  to  become  familiar  with  and  reasonably  comfortable  in  the  learning 
environment.  A second  method  was  the  early  inclusion  of  an  informatics  course  that  incorporated  the  essential 
computer  skills  that  students  would  need  to  be  successful  throughout  the  program.  Students  acquired  basic  skills 
in  computer  application  software  (Word,  Excel,  PowerPoint,  and  Access)  and  information  literacy  skills  (use  of 
electronic  resources  available  through  the  library  and  the  World  Wide  Web).  To  provide  relevance  and  to  help 
students  more  easily  acquire  these  skills,  exercises  using  technology  were  designed  especially  to  give  students 
the  opportunity  to  apply  their  newly  acquired  skills  to  their  discipline.  Facilitating  the  use  of  the  technology 
will  prevent  frustrating  technical  dilemmas  that  interfere  with  learning. 


Course  Design  and  Instruction 

Many  factors  related  to  good  online  course  design  and  instruction  apply  when  addressing  the  issue  of  student 
retention.  A well-designed  course  is  satisfying  to  the  learner  and  will  encourage  return.  Some  factors  not 
commonly  published  include  the  fact  that  not  all  faculty  are  well  suited  for  the  online  instruction.  Students  will 
not  hesitate  to  drop  a traditional  course  when  a favorable  impression  of  the  faculty  is  lacking.  Online  students 
look  for  several  factors  in  their  virtual  faculty  including  flexibility,  personal  credibility,  and  the  willingness  to 
consider  other  perspectives  without  injury  to  the  contributor.  Nowhere  is  an  instructor  more  vulnerable  with 
this  type  of  student  critique  than  in  the  online  environment.  As  such,  instructors  must  be  carefully  selected  for 
online  teaching  assignments.  Faculty  must  be  able  to  view  their  teaching  from  the  student's  perspective.  The 
ability  of  the  faculty  to  effectively  communicate  in  the  written  format  quickly  becomes  transparent  to  all. 
Attention  to  clarity,  grammar,  spelling,  and  punctuation  are  essential.  Therefore,  faculty  who  harshly  criticize 
students'  written  work,  when  their  own  skills  demand  attention,  may  unwittingly  establish  an  adversarial 
relationship  with  students.  In  addition,  the  communication  of  complex  concepts  necessary  for  critical  thinking 
requires  clear,  concise,  yet  innovative  teaching  strategies  because  faculty  are  not  able  to  evaluate  non-verbal 
clues  to  comprehension  online.  Those  who  consider  teaching  online  an  easy  contrast  to  the  dmdgery  of  the 
classroom,  should  carefully  be  evaluated.  Simply  posting  PowerPoint  content  outlines  used  in  traditional 
classroom  presentations  has  little  meaning  in  the  online  environment.  Conversely,  posting  extensive  lectures 
notes  that  do  not  contain  material  significantly  different  from  the  textbook  is  a time-consuming,  boring 
approach.  "A  well  designed  distance  course  that  has  a focus  on  interactivity  includes  many  topics  for 
discussion,  feedback  from  students  as  well  as  experts,  and  finally  links  to  sources  of  pertinent  information. 
Literature  citations,  journal  articles  and  URLs  are  a few  of  the  possibilities”  (Parker,  1999,  p.l6).  Developing 
content  to  foster  interactivity  is  essential.  Strategies  for  effective  interaction  online  are  varied  but  not  limited. 
Most  adult  students  desire  interactive  learning.  We  are  reminded  that  "The  web  may  be  a great  medium  for 
discovery  learning,  but  there  has  to  be  substance  for  the  student  to  discover!"  (Carlson  & Repman,  WebNet 
Journal,  2000,  p.l  1).  Without  the  credible  faculty,  retention  will  be  a moot  point. 
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Many  factors  influence  the  initiation  and  maintenance  of  trust  in  this  environment.  Assigning  "busy  work"  is 
inappropriate  in  any  learning  environment,  but  it  is  particularly  destructive  in  online  education.  The  product  of 
online  learning  is  written.  Students  submit  their  work  in  the  time  intensive  and  laborious  manner  of  a written 
format.  When  that  product  is  perceived  as  "busy  work",  trust  in  the  faculty  member’s  expertise  is  quickly 
eroded.  Students  are  less  likely  to  become  disgruntled  with  time  spent  in  high-level,  interactive  activities  that 
challenge  their  comprehension  of  materials  through  analysis,  application,  or  synthesis  strategies  than  with 
activities  that  require  them  to  regurgitate  what  they  have  read.  Therefore,  online  faculty  must  become  designers 
or  managers  of  learning  experiences,  shifting  the  work  of  learning  to  the  student.  “The  students  become  the 
center  as  interactive  collaborative  learners.”  (O’Leary,  2000,  p.  28)  Students  must  feel  confident  that  the 
faculty  will  not  humiliate  them  by  diminishing  their  contributions  in  the  face  of  their  peers.  This  degradation  is 
preventable,  but  is  devastating  to  continued  progress  once  the  student  has  experienced  it. 

Both  boredom  and  extraordinary  demands  (far  beyond  the  expectations  for  traditional  education)  are  primary 
concerns  for  students  considering  continuation  in  the  online  learning  environment.  Flexibility  should  be 
considered  the  hallmark  of  effective,  interpersonal  communication.  Rigidity  becomes  very  evident  in  online 
education.  Faculty  who  are  insensitive  to  subtle  cues  about  student  needs  will  struggle  to  establish  a trusting 
relationship  with  their  students.  Awareness  of  students'  work  at  the  beginning  of  a course  can  provide  clues  to 
changes  in  behavior  later  on.  These  behavior  changes  can  include  decreased  activity  level,  diminished  quality, 
and  delayed  responses.  Theses  are  critical  cues.  They  should  alert  the  faculty  member  to  solicit  student 
responses  regarding  the  meaning  of  the  new  online  behavior  pattern.  However,  when  students  provide  feedback 
about  the  instructional  methods,  faculty  must  be  flexible  and  willing  to  evaluate  the  validity  of  that  input  so  that 
they  can  make  meaningful  adjustments  in  the  course.  Results  of  a study  conducted  by  Indiana  University 
Bloomington  to  profile  the  innovative  teacher  using  technology  concluded  that  all  of  the  teachers  who  had 
changed  their  teaching  with  technology  shared  what  one  might  call  an  “innovative  personality”.  ‘They  were 
willing  to  remain  flexible  and  to  modify  their  innovations  as  they  received  feedback  from  their  students.”  In 
addition,  they  found  that  innovative  teachers  “liked  changing  things  mid -stream  to  make  them  work  better,  not 
having  to  wait  for  the  ne;d  time  they  teach  the  course.”  (Energizing  the  Innovative  Teacher:  Technology’s  Role, 
http://www.indiana.edu/~tltl/energy.html)  One  must  also  keep  in  mind  that  it  is  easy  to  become  so  enthusiastic 
about  the  endless  possibilities  of  activities  for  online  instruction  that  one  creates  an  overwhelming  workload  for 
the  students.  Identification  of  this  factor  demands  the  flexibility  to  adjust  the  assignments  appropriately. 
Teaching  online  can  be  far  more  demanding  than  classroom  instruction.  It  is  expected  that  the  faculty  will  gain 
^ close  association  with  each  student  and  must  be  responsive  to  their  individual  needs  or  risk  losing  the  student. 


Risk  Taking 

Online  learners  are  subjected  to  a different  set  of  stresses  than  those  in  the  classroom.  Writing  in  the  online 
environment  creates  feelings  of  vulnerability  in  both  the  students  and  faculty.  Written  communication  is 
perceived  as  a more  permanent  form  of  self-exposure  than  is  typical  in  the  conversational  environment  of  the 
traditional  classroom.  Thus,  faculty  need  to  take  the  lead  in  setting  the  example  for  risk-taking  online.  This  is 
particularly  true  if  your  subject  matter  involves  sensitive,  personal  expressions  or  examples  to  make  a point.  If 
faculty  want  students  to  apply  learning  to  personal  experiences  as  a demonstration  of  a concept  under 
discussion,  it  can  be  invaluable  for  the  faculty  to  provide  the  first  example.  Openness  on  the  part  of  the  faculty 
can  facilitate  honest  communication  from  the  students.  Faculty  leading  the  way  in  risk-taking  activities  enables 
the  braver  students  to  test  the  waters  and  others  will  follow.  If  risk-taking  is  an  expectation,  then  it  must  be 
handled  carefully  or  students  will  not  return  to  the  setting. 


Feedback 

Retention  is  related  to  the  development  of  a connected  relationship  between  the  faculty  and  the  student.  One 
mechanism  for  linking  to  the  student  is  feedback.  In  the  traditional  classroom,  faculty  are  able  to  respond  to 
students  immediately.  In  the  online  environment,  this  response  time  is  likely  to  be  delayed.  However,  the  delay 
should  be  minimized  as  much  as  possible  so  that  each  student  perceives  the  faculty  as  truly  committed  to  their 
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learning.  Timely  review  of  materials  supports  student  progress  in  a course.  Long  delays  translate  into  student 
concerns  about  whether  the  faculty  received  the  information  or,  if  received,  cared  enough  to  review  it.  The 
literature  suggests  that  the  feedback  time  to  online  learners  should  not  exceed  48  hours.  A student  who  logs  into 
a course  on  Sunday  and  has  questions  about  the  following  week’s  assignments,  will  become  very  frustrated  if 
the  faculty  does  not  respond  to  these  questions  very  early  in  the  week.  The  more  connected  the  adult  student 
feels  to  the  learning  situation  and  their  faculty,  the  more  likely  they  will  be  to  continue  in  the  course  and  to 
consider  enrolling  in  future  online  courses. 


Tone  of  Written  Communication 

Online  students  can  be  particularly  sensitive  to  the  tone  of  the  written  communication  that  they  perceive 
between  themselves  and  the  faculty.  Tone  can  be  an  elusive  factor  in  written  communication.  However,  it  is  a 
very  important  variable.  How  students  interpret  what  faculty  have  written  may  be  very  different  from  the 
message  that  was  intended.  Most  problems  occur  when  faculty  forget  that  it  is  imperative  to  be  able  to  stand 
back  from  the  message  and  view  it  from  the  students'  perspective.  This  view  relates  not  only  to  the  content,  but 
also,  perhaps  more  crucially,  to  the  nature  of  the  message.  Sensitive  critiques  including  attention  to  word 
choice,  understanding  student  perspectives,  openness  to  errors  in  thinking,  and  warmth  are  factors  that  reflect 
the  online  faculty  persona.  For  example,  an  authoritative  style  can  easily  be  communicated  to  and  resented  by 
adult  students  online  resulting  in  the  inhibition  of  trust.  When  this  occurs,  students  begin  to  produce  what  they 
believe  the  faculty  wants  rather  than  challenging  their  own  thought  processes.  On  the  other  hand,  an  investment 
in  the  creation  of  an  overall  trusting  atmosphere  will  allow  the  students  to  perceive  the  safety  net  required  to 
step -out-of-the -box  and  work  to  improve  their  critical  thinking  skills.  The  use  of  humor,  especially,  if  it  is  self- 
directed  and  not  sarcastic,  can  be  an  effective  method  for  engendering  trust  "Humor  can  lighten  the  burden  of 
the  learning  curve  for  both  the  student  and  faculty  and  can  generate  a felling  of  sincerity  among  those  in  the 
class.  (Parker,  1999,  p.  16)  But,  any  method  that  communicates  a non-judgmental,  open  approach  will  assist  the 
students  to  view  the  faculty  as  human  and  trustworthy.  In  any  case,  attention  to  the  tone  of  each  message 
requires  additional  time  and  effort  on  the  part  of  the  faculty.  Lack  of  attention  to  tone,  often  will  send  learners 
running  away  from  the  experience. 


Collaboration  in  Online  Learning 

Faculty  who  are  successful  in  establishing  an  online  learning  community  encourage  student  participation  and 
discourage  lurking  behavior.  Eisley  (1991)  and  Schmier  (1995)  identified  that  learning  is  best  accomplished 
when  the  learner  is  actively  engaged  in  the  process.  Knowlton,  Knowlton  and  Davis  (2000)  indicated  that  online 
discussions  help  students  understand  that  there  are  real  people  that  they  are  communicating  with  in  cyberspace 
and  that  discussions  facilitate  the  formation  of  an  educational  cyber-community  of  learners.  Harasim  (1989) 
pointed  out  the  benefit  of  knowledge  building  that  occurs  when  “students  explore  issues,  examine  one  another’s 
arguments,  agree,  disagree  and  question  positions.  Collaboration  contributes  to  higher  order  learning  through 
cognitive  restructuring  or  conflict  resolution,  in  which  new  ways  of  understanding  the  material  emerge  as  a 
result  of  contact  with  new  or  different  perspectives.  (Harasim,  1989,  p.5)  This  type  of  online  teaching  requires 
emphasis  on  quality  rather  than  efficiency.  A high  level  of  interactivity  allows  faculty  to  quickly  identify  when 
students  are  confused  or  need  additional  information  and  gain  a greater  understanding  of  individual  students’ 
performance  abilities.  As  noted  in  the  University  of  Illinois  Faculty  Seminar  Report  (2000),  "You  get  to  know 
your  students'  minds,  not  just  their  faces".  “Collaboration  is  an  integral  component  of  higher  education  not  only 
because  it  supports  active  learning,  but  also,  because  it  is  required  for  the  workplace  for  which  students  are 
prepared  (Ben-Jacob,  Levin,  & Ben-Jacob,  2000,  p.  8).  It  must  be  remembered  that  participation  is  not  likely  to 
happen  unless  it  is  deliberately  planned.  This  format  defines  a learner-centered  educational  model  where 
students  operate  in  a more  self-directed  mode  and  have  greater  control  of  their  own  learning.  It  encourages 
student  success  and  connectedness  that  in  turns  fosters  retention.  Again,  these  strategies  are  time  consuming 
and  require  committed  and  dedicated  faculty. 
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Conclusion 


Clearly,  many  factors  influence  student  retention.  Initiation  and  maintenance  of  trust  in  the  online  environment 
is  essential.  Collaboration  is  essential  for  active  learning  and  gives  students  valuable  practice  of  skills  that  can 
lead  to  advancement  in  the  workplace.  Faculty  contemplating  teaching  online  must  be  cognizant  of  the  need  to 
prepare  students  for  this  new  learning  environment.  Beyond  technology  preparation,  faculty  need  to  learn  how 
to  establish  credibility  with  the  students.  Major  factors  that  affect  credibility  include  written  communication 
skills,  use  of  higher  level  teaching  strategies,  flexible  approaches  to  learner  concerns  and  needs,  setting  an 
example  for  risk-taking,  timely  feedback,  and  setting  a tone  that  promotes  open  and  honest  interaction.  These 
factors  positively  influence  the  development  of  mutual  trust  between  the  faculty  and  their  students.  Online 
education  truly  demands  high-touch  and  requires  selection  of  faculty  who  are  able  to  demonstrate  the  qualities 
of  excellence  described  above  as  well  as  dedication  and  commitment  to  student  learning. 
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Abstract:  To  establish  standards  of  quality  in  online  courses  developed  through  the 
University  of  Hawaii's  Asynchronous  Learning  project  funded  by  the  Alfred  P.  Sloan 
Foundation,  a preliminary  study  of  courses  from  various  disciplines  was  conducted. 
The  aim  of  the  study  was  to  identify  teaching  activities  that  contributed  to  the  creation 
of  a high  quality  interactive  classroom  where  students  were  actively  engaged  in  the 
process  of  learning.  The  study  showed  that  a direct  relationship  exists  between  the 
teaching  activities  and  the  frequency  and  quality  of  collaborative  learning  in  the  online 
classroom.  Carefully  crafted  teaching  activities  contribute  toward  creating  the  social 
presence  of  the  teacher,  which  directly  or  indirectly  shapes  the  learning  environment 
and  thus  significantly  impacts  the  desired  learning  outcomes. 


Introduction 

A number  of  recent  studies  on  the  online  teaching-learning  process  have  specifically  dealt  with  the 
role  of  the  online  teacher  as  a facilitator  (Salmon,  00;  Palloff  et  al,  01;  Collison  et  al,  00).  Such  studies 
through  their  exclusive  focus  on  the  role  of  the  teacher  as  a facilitator  often  lead  to  the  misperception  that 
online  teaching  is  merely  facilitation.  The  role  of  the  online  instructor  is  also  to  create  a “teaching 
presence”  through  appropriate  instructional  activities  which  include  lecture  notes,  teaching  commentaries 
and  individual  or  collective  discussion  responses  to  guide  students  toward  critical  exploration  of  the  content 
(Anderson  et  al,  01). 

Studies  have  shown  that  the  constructivist  model  in  instructional  design  based  on  active  learning  is 
capable  of  providing  an  effective  theoretical  framework  within  which  to  analyze  the  teaching-learning 
experience  in  technology  mediated  instruction  (Bednar  et  al,  92;  Gold,  01).  Knowledge  construction  has 
been  described  as  a social  process  (Dewey,  96).  Learning  involves  construction  of  internal  representations 
of  knowledge  which  can  best  be  accomplished  in  a collaborative  situation  where  learners  encounter  other 
knowledge  representations  or  perspectives,  which  allow  them  to  review,  evaluate  and  assess  their  own 
perspective,  and  revise  it  if  the  revision  is  more  meaningful  (Bednar  et  al,  92).  In  an  effective  learning 
environment,  the  instructional  tasks  are  contextualized  in  authentic  situations  and  students  are  given 
opportunities  to  construct  knowledge  as  they  test  their  ideas  on  others  and  evaluate  other  perspectives.  The 
social  negotiation  of  meaning  is  enhanced  when  the  role  of  the  teacher  is  seen  as  a coach  and  a facilitator  in 
the  learning  process  (Vygotsky,  78). 

Current  research  on  online  learning  shows  that  new  technologies  allow  course  participants  to 
engage  in  meaningful  discussions  so  knowledge  is  not  transmitted  from  the  teacher  to  the  students,  but 
rather  discovered  as  individual  perspectives  are  shared  in  a collaborative  learning  environment  (Hiltz  97; 
Harasim,  90;  Joy  et  al,  00).  Students  acquire  knowledge  in  a social  context  where  they  are  given 
opportunities  to  articulate  and  express  their  views  as  they  construct  knowledge  that  is  personally 
meaningful  to  them.  The  teaching-learning  process  is  thus  seen  as  a shared  transaction  in  which  the  learners 
assume  responsibility  for  their  learning  and  the  teacher  shares  with  students  the  control  of  the  nature  and 
content  of  the  activities  in  the  classroom  as  s/he  ensures  that  any  new  knowledge  created  meets  the  shared 
disciplinary  standards  (Garrison  et  al,  00).  As  the  content  expert,  the  teacher  thus  guides  the  learners  as 
they  negotiate  the  exploratory  learning  environment.  The  direct  and  indirect  teaching  acts  performed  by  the 
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teacher  in  the  online  classroom  are  thus  critical  in  shaping  the  collaborative  learning  process  in  the 
classroom  to  achieve  the  desired  learning  outcomes. 


Research  Method 

In  order  to  determine  the  relationship  between  the  diverse  teaching  and  learning  activities  in 
completely  online  courses  offered  at  the  University  of  Hawaii  at  Manoa,  a detailed  study  was  conducted  of 
six  courses  from  a variety  of  disciplines:  information  and  computer  sciences,  music,  family  resource 
management,  anthropology,  political  science,  and  interdisciplinary  technology  and  culture.  The  six  online 
courses  were  taught  in  Fall  2001.  The  primary  aim  of  the  study  was  to  find  the  set  of  teaching-learning 
practices  that  were  conducive  to  creating  an  active  collaborative  learning  environment.  In  order  to  collect 
data  on  the  courses,  the  study  employed  a detailed  worksheet  listing  various  teaching  and  learning  activities 
possible  in  the  online  classroom.  Teaching  activities  which  were  examined  in  each  course  included  course 
overview  documents,  lecture  notes/content  overviews,  interpretive  teaching  commentaries,  discussion/ 
study  questions,  and  teacher’s  facilitative  discourse.  Learning  activities  that  were  examined  included 
weekly  responses  to  the  reading  assignments  or  individual  work  assignments,  online  discussion,  group 
projects,  and  students’  leadership  discourse.  After  collecting  data  on  the  teaching  and  learning  activities 
included  in  each  course,  the  relationship  between  the  two  types  of  activities  was  examined.  The  impact  of 
the  teaching  acts  on  the  learning  acts  was  studied  through  analyzing  the  quality  of  the  work  produced  by 
the  students  in  terms  of  online  class  discussion,  group  discussion  (if  included),  as  well  as  individual  work 
assignments  submitted  as  weekly  responses  to  the  readings.  This  study  did  not  include  the  work  students 
submitted  directly  to  the  instructor  in  the  form  of  short  or  long  research  papers  or  quizzes. 

Data  Gathering  and  Analysis 

Teaching  Activities 

The  courses  included  in  the  study  were  taught  using  a course  management  system,  either  WebCT 
or  Blackboard  which  provided  an  almost  complete  transcript  of  course  documents,  course  activities  as  well 
as  strategies  the  instructor  used  to  manage  the  course  on  a weekly  basis.  The  course  overview  documents 
along  with  the  weekly  introduction  of  materials  by  the  instructor  provided  a good  perspective  on  how  the 
course  was  designed.  Courses  studied  were  labeled  A,  B,  C,  D,  E,  and  F in  order  to  maintain  the  anonymity 
of  the  instructors  who  taught  the  courses. 

All  instructors  had  gone  through  a tutorial  on  online  teaching,  which  went  over  the  effective 
strategies  to  organize  and  manage  an  online  course.  With  the  result  all  courses  included  good  course 
overview  documents,  which  provided  a detailed  description  of  the  syllabus,  schedule,  course  procedures, 
requirements,  and  grading  system.  The  courses  A,  B,  and  C included  either  an  introductory  student  tutorial 
on  online  learning  or  detailed  technology  tips  for  students  to  access  the  course. 

In  courses  A,  B,  and  C the  instructors  rethought  the  traditional  lecture  presentation.  The  weekly 
reading  assignments  were  introduced  in  a non-linear  manner  through  lecture  notes,  content  overviews,  and 
web-based  resources.  The  purpose  of  lecture  notes  or  content  overviews  was  to  facilitate  students’  access  to 
weekly  readings  and  work  assignments. 

Whereas  lecture  notes  served  the  purpose  of  introducing  the  course  materials,  the  interpretive 
teaching  commentaries  used  in  the  courses  A,  B,  C,  D,  E,  and  F served  to  provide  a platform  through  which 
the  instructor  as  content  expert  responded  to  discussion  posts  and/or  individual  responses  to  the  readings  in 
the  form  of  weekly  collective  postings.  Different  instructors  used  teaching  commentaries  for  different 
purposes.  In  all  cases,  the  teaching  commentaries  became  for  the  instructor  one  of  the  ways  to  assert  his  or 
her  presence  in  the  electronic  classroom.  The  commentaries  served  to  contribute  to  the  dynamic  nature  of 
the  learning  environment  as  these  were  not  pre -scripted,  but  tailored  to  the  particular  learners  and  the 
learning  environment.  The  instructors  who  provided  interpretive  teaching  commentaries  thus  made  their 
presence  felt  in  ways  other  than  discussion  facilitators. 

In  order  to  make  the  course  materials  more  meaningful  to  students,  the  instructors  who  taught  the 
courses  A,  B,  and  C made  use  of  study  questions  or  discussion  questions  to  guide  students  to  think  of  the 
ideas  or  the  subject  matter  covered  in  a broader  context.  They  also  encouraged  students  to  apply  the  ideas 
under  discussion  to  real  life  situations.  The  discussion  questions  in  courses  A,  B and  C became  the  basis  for 
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the  instructor-led  component  of  class  discussion  that  also  had  a broader  student-led  discussion  component. 
No  discussion/study  questions  were  included  in  the  weekly  activities  of  the  courses  E and  F. 

The  success  different  instructors  had  with  the  efficient  management  of  the  classroom  was  studied 
through  the  examination  of  the  facilitative  discourse  which  was  present  in  all  courses  to  a greater  or  a lesser 
extent.  The  facilitative  discourse  included  any  direct  or  indirect  teacher  intervention  to  efficiently  manage 
the  course  and  make  it  run  smoothly  through  enforcing  deadlines  and  providing  guidance  when  needed.  It 
included  the  comments  to  generate  enthusiasm  or  to  promote  self-motivation  and  self-direction  among 
students.  The  examination  of  the  facilitative  discourse  was  restricted  to  what  was  revealed  in  the  course 
transcripts  and  not  to  any  individualized  feedback  that  the  instructors  might  have  provided  students  through 
private  e-mail. 


Learning  Activities 

All  courses  included  both  individual  as  well  as  collaborative  learning  activities,  which  were 
designed  differently  in  various  courses.  Students  were  required  to  submit  weekly  individual  responses  to 
readings  in  the  courses  A,  B,  D,  and  E,  whereas  in  the  course  F,  they  were  asked  to  submit  a biweekly 
response  to  the  readings.  The  courses  C and  F made  use  of  weekly  quizzes  to  assess  if  students  had  learned 
the  material. 

The  instructors  used  various  methods  to  collect  the  responses  submitted  by  the  students,  ti  the 
course  F,  students  were  divided  into  groups  and  the  responses  were  discussed  in  the  group  working  area.  In 
the  course  D,  students  were  asked  to  post  their  responses  on  the  class  discussion  board.  In  this  course,  the 
responses  had  to  include  discussion  questions  framed  by  students.  In  the  course  B,  students  were  required 
to  send  the  weekly  responses  privately  to  the  instructor.  After  students  had  submitted  their  responses  for  a 
particular  week,  the  instructor  of  this  course  made  the  responses  public  so  they  were  available  for  everyone 
to  read. 

Online  class  discussion  was  used  in  all  courses,  but  its  effectiveness  as  a collaborative  learning 
tool  varied  from  course  to  course.  In  the  courses  A,  B,  C and  D,  the  discussion  served  the  purpose  of 
helping  students  validate  each  other’s  learning.  Different  perspectives  helped  them  understand  ideas  that 
they  had  missed  or  not  understood  in  their  own  reading.  Students  in  these  courses  responded  thoughtfully  to 
each  other’s  comments.  Such  exchanges  created  a shared  context  for  learning.  In  the  courses  E and  F,  the 
discussions  were  very  superficial. 

As  far  as  the  incorporation  of  group  projects  was  concerned,  only  courses  A and  B incorporated 
group  projects.  The  group  project  served  as  the  focal  point  around  which  the  group  members  in  these  two 
courses  did  research,  gathered  information,  and  jointly  created  a product. 

Giving  students  a leadership  role  in  course  activities  was  used  by  most  of  the  instructors  as  an 
effective  way  to  get  actively  involved  in  course  activities.  Though  all  courses  had  this  feature  implicitly 
embedded  in  the  course  activities,  only  the  course  B had  explicitly  organized  the  groups  in  such  a way  that 
both  the  groups  and  the  individual  group  members  were  given  opportunities  to  engage  in  leadership 
discourse  as  they  initiated  weekly  discussions  and  managed  weekly  group  activities  on  a rotating  basis. 


Results 

The  detailed  analysis  of  the  course  transcripts  showed  that  selecting  a course  management  system 
and  creating  the  course  overview  documents  is  only  half  of  the  project  of  creating  an  effective  online 
course;  the  other  half  is  the  content  and  organization  of  weekly  multi-modal  teaching  and  learning 
activities.  The  study  indicated  that  the  teacher’s  implicit  or  explicit  teaching  philosophy  directly  impacted 
the  nature  and  the  number  of  teaching  acts  performed  on  a weekly  basis  in  the  online  classroom.  Different 
instructors  integrated  various  teaching  and  learning  activities  into  the  course  environment  to  different 
degrees.  It  was  obvious  that  the  presence  or  absence  of  teaching  activities  shaped  the  learning  environment 
in  each  course.  There  was  a direct  relationship  between  the  nature  of  direct  and  indirect  teaching  activities 
and  the  frequency  and  effectiveness  of  collaborative  learning  activities. 

Given  below  is  a chart  that  shows  the  relationship  between  the  teaching  and  learning  activities  and 
the  resulting  active  or  passive  collaborative  learning  environment.  Courses  labeled  A,  B,  and  C showed 
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maximum  multi-modal  teaching  and  collaborative  learning  activities  involving  discovery  or  active  learning. 
Courses  labeled  E and  F showed  minimum  teaching  activities  with  very  little  active  collaborative  learning. 
The  collaborative  learning  activities  in  these  courses  were  merely  a cosmetic  addition  to  the  class  activities. 
The  course  D fit  somewhere  between  the  two  as  far  as  the  effectiveness  of  collaborative  learning  activities 
was  concerned. 

Courses  A,  B,  & C:  Multiple  Multi-Modal  Teaching  Activities 

The  three  courses  labeled  A,  B,  and  C with  multi-modal  teaching  activities  produced  a learning 
environment  where  students  were  engaged  in  the  construction  of  knowledge  as  they  worked 
collaboratively.  The  more  the  teaching  acts,  the  greater  the  frequency  and  higher  the  quality  of 
collaborative  learning  acts  performed  by  students. 


Figure  1:  Relationship  between  Teaching  and  Learning  Activities 


Course  D:  (Constructivist  Model)  Reduced  Teaching  Activities 

The  course  D occupied  a position  somewhere  between  the  highest  and  the  lowest  in  terms  of  the 
quality  of  online  discussion.  Students  were  required  to  write  a weekly  response  to  the  readings  and 
formulate  two  to  three  discussion  questions  related  to  the  readings.  Both  the  response  and  the  questions 
were  posted  on  the  discussion  board.  Students  then  responded  to  each  other’s  discussion  questions  as  well 
as  responses.  The  quality  of  discussion  in  this  course  was  good.  The  instructor  provided  feedback  primarily 
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through  a collective  weekly  discussion  posting,  though  occasional  responses  to  individual  postings  were 
also  made.  Students  in  this  course,  however,  were  on  their  own  as  they  tackled  the  assigned  readings  on  a 
weekly  basis  unlike  in  the  other  three  courses  A,  B,  and  C where  the  instructors  had  provided  lecture  notes 
or  content  overviews  on  a weekly  basis.  Since  students  were  more  or  less  on  their  own  in  the  course  D as 
far  as  accessing  the  content  was  concerned,  it  is  unclear  if  they  could  engage  the  texts  as  intensively  as  they 
would  have  done  if  the  instructor  had  provided  more  guidance. 

rniir!«e!«  R and  F with  Minimum  Teaching  Activities 

In  courses  E and  F with  decreased  teaching  acts,  it  was  obvious  that  the  instructors  implicitly 
based  their  courses  on  the  traditional  instructional  model.  Even  though  the  instructors  used  the  suggested 
online  course  design  for  incorporating  collaborative  work,  it  was  against  the  background  of  the  assigned 
course  activities  where  students  were  primarily  asked  to  basically  read  the  materials,  complete  the 
assignments  and  send  the  completed  assignments  to  the  instructor.  In  these  two  courses,  no  specific 
instructions  were  given  to  students  as  to  how  they  should  conduct  their  discussion  activities.  In  the  first 
course,  students  were  simply  asked  to  discuss  the  readings  with  other  group  members.  In  the  second  course, 
each  student  had  to  complete  a biweekly  assignment.  This  assignment  was  in  the  form  of  a commentary  on 
the  readings,  a summarization  of  the  main  points  along  with  some  comments  on  what  transpired  in  the 
discussion  amongst  the  group  members.  Group  members  were  encouraged  to  discuss  the  work  in  the  chat 
room,  rather  than  through  asynchronous  discussion.  Both  the  chat  room  archives  and  the  asynchronous 
discussion,  when  examined,  reflected  very  superficial  interaction. 

In  the  above  two  courses  with  decreased  teaching  presence,  the  teaching  activity  was  thus 
confined  to  the  minimum  and  appeared  only  in  the  form  of  the  instructor’s  collective  response  to  discussion 
postings.  Students  found  themselves  in  a very  open  learning  environment,  as  they  had  to  rely  on  their  own 
resources  to  comprehend  the  materials  for  which  they  were  unequally  prepared.  The  discussion  transcripts 
of  the  two  courses  showed  if  the  instructor  is  not  clear  about  the  purpose  of  the  group  work  and  how  it  can 
be  used  to  encourage  students  in  constructing  knowledge  or  transferring  new  knowledge  to  other  contexts, 
online  discussion  is  bound  to  focus  on  the  reproduction  of  ideas.  In  the  absence  of  any  explicit  guidance 
from  the  instructor  through  discussion  questions  or  well -integrated  interpretive  teaching  commentaries  or 
lecture  notes  or  content  overviews,  students  were  very  much  on  their  own  as  they  went  through  the  course 
materials.  They  were  thus  more  intent  on  merely  learning  the  material  as  it  was  presented  to  them  in  the 
textbooks,  without  critically  evaluating  the  ideas  they  encountered  using  their  higher  order  thinking 
abilities.  The  only  collaborative  work,  the  online  synchronous  and/or  asynchronous  discussion,  seemed  to 
be  just  an  “add  on”  activity.  It  seemed  the  “real  thing”  students  were  asked  to  do  was  the  assignments 
(responses  or  quizzes)  they  worked  on  independently  which  they  sent  to  the  instructor.  Both  these  courses 
thus  retained  the  traditional  model  of  instruction  where  students  primarily  wrote  for  their  teachers  and 
reproduced  what  they  learned  in  textbooks.  It  was  also  clear  that  just  as  an  exclusively  teacher-centered 
model  is  not  educationally  effective  in  making  knowledge  meaningful  to  students,  so  is  the  other  extreme 
of  learner-centered  model  where  students  are  expected  to  work  in  a very  open  learning  environment  with 
little  guidance  from  the  teacher. 


Conclusion 

The  above  study  showed  that  diverse  multi-modal  teaching  activities  with  corresponding  learning 
activities  are  successful  in  creating  an  effective  learning  environment  where  students  are  engaged  in  the 
process  of  learning.  The  teacher’s  presence,  generated  through  multi-modal  teaching  acts,  promotes  self- 
motivation  and  self-direction  amongst  students  as  they  are  guided  to  actively  engage  in  collaborative 
learning  activities.  In  establishing  quality  standards  in  the  online  classroom,  therefore,  the  role  of  the 
instructor  in  creating  an  integrated  teaching-learning  environment  must  be  emphasized.  The  electronic 
medium  is  unique  in  that  it  can  be  easily  transformed  into  a dynamic  electronic  space  that  is  capable  of 
sustaining  the  social  presence  of  both  the  teacher  and  the  students.  The  instructor’s  expert  guidance 
manifested  through  multi-modal  teaching  activities  is  indispensable  in  creating  a shared  context  of  learning 
where  students  are  engaged  in  both  constructing  knowledge  and  applying  it  to  other  contexts.  The  diverse 
teaching-learning  activities  promote  a complex  mode  of  multi-level  interaction  amongst  the  learners,  the 
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instructor,  and  the  content  of  the  course,  which  creates  a social  context  for  constructing,  exchanging  and 
transforming  knowledge. 
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Abstract 

This  paper  provides  an  initial  examination  of  over  614  multimedia  courses  now  available  in  the  UK  with  discussion 


oeiiued  uii  leasons  for  recent  course  expansion.  Areas  of  specific  grevAh  and  issues  of  course  design  are  presented  In 
the  analysis  and  discussion.  Changes  in  the  application  of  multimedia  from  subjectspecific  to  second  subject 
contextualisation  are  also  discussed  with  reasoning  to  support  learning  benefits  to  be  derived  from  this  approach,  and 
implications  for  course  designers  and  providers  are  concluded. 


This  paper  will  be  of  interest  to  multimedia  and  online  course  designers,  managers  and  tutors,  instructional  designers, 
and  those  involved  in  institutional  curriculum  planning. 


Introduction  and  motivation  for  this  work 

The  motivation  for  this  study  arose  early  in  1999.  Our  study  centre,  CSALt,  had  successfully  run  an  MSc  in  Advanced  Learning  Technology 
since  1993  and  we  started  to  give  serious  consideration  to  a course  for  undergraduates  which  married  learning  technology  with  learning  theory . 
Our  objective  was  not  only  to  equip  a graduate  with  educational  knowledge  and  awareness,  but  to  support  this  with  a range  of  information  and 
communication  technology  (ICT)  and  multimedia  production  skills  in  order  to  prepare  them  to  make  better  use  of  educational  technology 
already  available  in  many  educational  settings. 

Our  first  step  towards  this  objective  was  to  review  existing  course  availability  in  this  field.  The  University  and  Colleges  Admissions  Service 
(UCAS)  in  the  UK  at  that  time  offered  53  courses  related  to  the  themes  of  'educational  technology',  'multimedia  technology'  and  'learning 
technology'.  We  also  noted  that  approximately  250  courses  were  then  running  which  included  the  term  multimedia  in  their  title.  Given  that 
competition  between  Higher  Education  Institutions  (HEIs)  in  the  UK  can  be  very  high,  the  relatively  low  provision  of  courses  in  the  educational 
technology  area  suggested  a yet-to-be  exploited  area  for  development.  Consequently  the  authoring  of  course  rationales  and  modules 
commenced. 

The  first  working  title  for  the  new  degree  was  'Multimedia  and  Online  Learning  Technologies'.  This  changed  to  'Multimedia  Learning 
Technologies'  within  a week  or  so  as  we  felt  the  first  title  to  be  too  long  and  we  also  embraced  the  idea  that "...  the  term  multimedia  is  used  to 
encompass  both  online  and  offline  projects"  (England  and  Finney,  1999)  from  one  of  the  identified  course  books. 

In  the  Spring  of  2001  the  degree  scheme  cleared  the  last  university  approval  committee,  though  several  changes  had  taken  place  by  this  point, 
not  least  to  the  title.  The  term  'Learning  Sciences*  now  replaced  Learning  Technologies  in  order  to  emphasise  that  theoretical  aspects  of 
learning  played  an  equal  role  to  those  of  technical  production  and  realisation.  The  second  title  change  was  the  removal  of  the  word  multimedia 
completely  to  leave  a new  title  of  'Learning  Sciences  & Technology'.  This  latter  change  caused  more  than  a little  concern  between  the  course 
authors  and  validators  over  what  potential  students-  and  tutors  - may  understand  as  the  focus  of  the  degree. 

With  this  latter  question  in  mind  we  decided  to  again  examine  the  availability  of  multimedia  and  learning  technology  related  courses  offered 
through  UCAS  commencing  in  2002 . To  our  great  surprise  614  courses  were  available  across  the  UK  which  included  the  term  multimedia  in 
their  title.  This  caused  pause  for  thought  and  fresh  questions  came  to  mind,  not  least: 

• What  were  the  causes  of  so  much  expansion  in  multimedia  course  provision  so  quickly  (around  160%  increase  in  2 years)? 

• Had  the  definition  of  multimedia  evolved  to  include  many  more  new  options? 

• What  aspects  of  multimedia  or  skills  were  students  learning  in  these  courses? 

• Though  we  had  removed  multimedia  from  the  title  of  our  own  scheme,  was  this  perhaps  the  most  appropriate  thing  to  do,  both  for 
marketing  and  pedagogic  reasons? 

Multimedia  course  analysis 

In  order  to  understand  this  change  it  was  necessary  to  review  multimedia  course  availabity  on  a year-ty-year  basis.  Had  this  change 
happened  gradually,  or  was  it  sudden  and  dramatic  due  to  improvements  in  access  to  and  in  the  performance  of  technologies  associated  with 
multimedia?  An  analysis  of  this  data  (table  1)  showed  a dramatic  increase  in  multimedia  course  provision  between  1998-99  with  179  new 
courses  running,  compared  to  95  new  courses  running  in  1997-98  and  56  in  1996-97.  The  data  also  showed  a decline  in  the  number  of 
multimedia  courses  running  in  for  2000-2001  (105  discontinued  courses  and  76  suspended  from  the  previous  year)  though  projections  had 
once  again  picked  up  for  2002. 


Year 

1996 

1997 

1998 

1999 

2000 

2001 

2002 

MM  courses 

Cancelled 

2 

31 

9 

7 

1 

Discontinued 

1 

23 

27 

46 

67 

105 

109 

^ Centre  for  Studies  in  Advanced  Learning  Technology,  Lancaster  University,  http://csalt.lancs.ac.uk/csalt/ 
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Running 

121 

177 

272 

451 

560 

547 

617* 

Suspended 

6 

3 

5 

9 

76 

2 

Grand  Total 

128 

200 

304 

533 

645 

735 

729 

Data  and  Analytical  services  (updated  January  2002). 


We  then  examined  the  courses  in  more  detail  in  order  to  identify  those  courses  relating  to  our  own  course  which  originally  combined 
multimedia  w ith  learning  and  technology.  This  inspection  highlighted  a number  of  variations  on  the  use  of  the  term  multimedia  itself.  Ten 
vanaticns  were  identified  ineluding  ’Creative  Multimedia  (M.M)',  ’Interactive  MM’,  ’MM  C'Cmmi miration'.  ’MM  Computing’.  ’MM  Electronic 
Technology’,  ’MM  Studies',  'MM  systems',  'MM  Technology’,  'MM  Visualisation*  and  'MM  web  production^.  Some  discussion  took  place  as  to 
what  these  terms  may  actually  mean,  what  the  courses  which  used  these  titles  focused  on,  and  whether  students  would  necessarily 
understand  differences  between  the  various  titles. 


The  next  stage  of  our  investigation  focused  on  the  placing  of  the  word  muitimedia  or  one  of  its  curriculum  variants  in  the  course  title;  some 
courses  offered  multimedia  or  a variant  alone,  some  offered  multimedia  or  a variant  with  a specific  topic  (e.g.  Multimedia  systems  and 
theology),  and  some  offered  a specific  topic  with  a multimedia  or  variant  (e.g.  Chemistry  and  Multimedia  Technology).  A classification  system 
was  drawn  up  to  structure  course  titles  in  order  to  identify  any  development  patterns  (table  2).  The  data  from  this  examination  indicated  that,  in 
the  vast  majority  of  cases,  the  term  multimedia  or  variant  had  been  coupled  with  another  subject  study  area  to  create  a combined  degree  (over 
478  courses  from  614  making  78%  MM  course  provision  coverage). 


MM  term  position: 

MM 

alone 

Qualified 

MM. 

MM  1" 

MM  2nd 

MM  3rd 

unsure 

Total 

HE  offered  degree ind.  FE/HE“  & FD° 

10 

42 

198 

235 

9 

7 

501 

HE  offered  HND 

14 

7 

3 

6 

0 

0 

30 

FE  offered  degree  ind.  FD. 

2 

0 

17 

2 

0 

0 

21 

FE  offered  HND 

45 

0 

9 

8 

0 

0 

62 

Total: 

71 

49 

227 

251 

9 

7 

614 

Table  2.  Course  titles  and  place  of ’multimedia’  i.e.  'Multimedia  and  topic.  Topic  and  multimedia',  or  Topic,  topic  and  multimedia'. 
HND  courses  usually  cover  a period  of  pre-university  study  up  to  and  sometimes  including  the  first  year  of  a degree  level  course. 


MM  term  position: 

MM 

Qual 

MM. 

MM  1" 

MM2™ 

MM  3^" 

unsure 

Total 

As% 

Applied  sciences 

31 

56 

1 

1 

89 

14.5 

Arts  & humanities 

77 

78 

1 

2 

158 

25.7 

Environment  & Natural  sciences 

17 

27 

0 

0 

44 

7.2 

Management 

47 

50 

7 

2 

106 

17.3 

Social  sciences 

29 

21 

0 

0 

50 

8.1 

Cross  depts 

18 

7 

0 

0 

25 

4.1 

Law 

2 

5 

0 

0 

7 

1.1 

Engineering 

1 

3 

0 

0 

4 

0.7 

Sports  science/Heath  care 

5 

4 

0 

0 

9 

1.5 

Undetermined 

71 

49 

0 

0 

0 

2 

122 

19.9 

Total: 

71 

49 

227 

251 

9 

7 

614 

100.0 

Table  3.  Multimedia  course  analysis  against  Faculty  provision.  The  Faculty  structure  used  was  that  of  the  authors’  own  university. 
The  grey  areas  for  MM  and  Qualified  Multimedia  degrees  are  left  blank  due  to  the  possibility  of  delivery  by  several  departments  in 
different  faculties. 


As  a next  step  we  made  a provisional  examination  of  these  multimedia  combined  courses  in  order  to  identify  any  specific  areas  of  growth  or 
development.  This  activity  proved  to  be  very  subjective  and  whilst  coupling  multimedia  with  some  subjects  seemed  to  make  some  rational 
sense  (e.g.  Multimedia  Communications  & Business  Studies,  or  Multimedia  and  Internet  Technologies)  others  proved  to  be  more  taxing  on  our 
immediate  understanding  (e.g.  Geratology  and  Multimedia  Systems).  We  were  particularly  curious  about  'Complimentary  therapies  and 
aromatherapy  and  Multimedia  systems’  insofar  as  whether  the  course  utilised  the  sense  of  smell  alongside  text,  images,  audio  and  graphics. 

In  order  to  limit  subjectivity  we  established  a Faculty  classification  system.  Taking  the  Faculty  and  Department  structure  at  Lancaster 
University  as  a framework,  each  course  was  placed  in  the  Faculty  which  would  best  provide  a home  for  its  study;  for  example.  Mathematics 
and  Multimedia  Communication  was  placed  in  Applied  Sciences  due  to  the  mathematical  and  possible  technical  components  of  what  may  be 
meant  by  Multimedia  Communication.  This  classification  system  resembled  the  hierarchical  classification  systems  used  in  phenomenographic 


^ The  figure  of  617  reported  here  shows  an  increase  on  our  earlier  reported  figure  of  614  due  to  3 new  courses  becoming  available  in  the 
period  between  our  initial  analysis  and  contacting  the  UCAS  for  a statistical  search. 

^ We  refer  to  these  as  Multimedia  offspring  or  Multimedia  variants 

^ Further  Education  (FE  or  tertiary  education)  is  usually  the  stage  before  Higher  Education  (HE  or  university)  for  many  UK  based  .students. 
^ Foundation  Degree  (FD)  is  sometimes  referred  to  as  a Year  0 course  and  can  be  taken  prior  to  University  (HE)  study. 
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analysis  (Saijo,  1988;  Marion  and  Saijo  1976a;  1976b).  In  support  of  this  method  we  applied  an  independent  co-judging  technique®  which 
provided  85.7%  classification  agreement,  within  Saijo 's  20%  error  tolerance.  The  resulting  classifications  were  then  tabulated  (table  3)  and 
spatially  examined  (diagram  1). 

The  analysis  of  this  data  highlighted  three  areas  of  interest  and  discussion: 

• That  over  a quarter  of  all  multimedia  related  degree  schemes  and  modules  were  related  to  Arts  and  Humanities 

• That  ICT  and  Applied  Sciences  (including  computing  and  networking)  accounted  for  less  than  15%  of  multimedia  degree  and  module 
related  delivery 

• That  expansion  of  multimedia  courses  related  to  Business  and  Management  and,  in  particular,  the  emergence  of  e-busIness  and  e- 
commerce  courses,  were  higher  than  expected. 


In  consideration  of  this  result  we  are  aware  of  the  subjectivity  involved  in  the  Faculty  placing  of  each  course,  even  allowing  forjudging  and  co- 
judging described.  In  order  to  refine  these  results  it  became  necessary  to  examine  the  rationale  and  structure  for  each  course  provided. 
However,  before  setting  out  to  complete  such  a detailed  analysis  for  all  614  multimedia  related  schemes  specified,  we  felt  an  examination  of  a 
sample  of  these  courses  would  help  us  to  best  determine  how  to  go  about  this  task  and  what  such  an  analysis  may  yield.  This  led  us  naturally 
into  the  second  of  our  initial  questions  as  to  whether  the  definition  of  multimedia  had  evolved  and  how  it  may  have  done  so. 

Initial  course  rationale  analysis 

In  our  previous  examination  of  course  titles  we  took  care  to  track  Higher  National  Diploma  (HND)  courses  separately  from  degree  courses. 
HND  courses  tend  to  be  delivered  by  colleges  with  a Further  Education  (FE)  focus  or  those  which  may  not  be  able  to  resource  and  staff  a 
complete  degree  course.  This  is  illustrated  In  the  data  (table  1)  where  62  of  90  HMDs  are  offered  by  FE  focused  establishments.  One  further 
difference  is  that  HMDs  have  tended  to  be  skill  or  vocational  orientated  qualifications,  compared  to  HE  awarded  degrees  which  may  comprise 
levels  of  reflection  or  meta-cognitve  components.  To  examine  this  situation  for  multimedia  related  qualifications  we  acquired  details  of  a 
number  of  schemes  for  analysis. 

The  first  HND  in  Multimedia  in  FE  in  the  United  Kingdom  was  produced  at  Halton  College  and  validated  by  the  Business  and  Technology 
Education  Council  (BTEC)  in  Summer  1996.  The  rationale  for  this  course  states: 


® With  Dr.  Michael  Pengelly,  CSALT,  Lancaster  University 
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'The  proposed  course  in  multimedia  has  been  developed  to  provide  students  seeking  a career  in  multimedia  with  experience, 
knowledge  and  awareness  of  production,  management  and  technical  aspects  of  the  relevant  environment.  It  is  expected  that 
students  following  the  course  will  leave  well  prepared  for  further  study  in  higher  education  or  for  working  life  in  a commercial  or 
business  setting,,,.  Construction  of  the  course  has  been  centered  around  multimedia  production,  authoring,  design  and 
management,  forming  the  main  themes  of  study  for  both  years." 

(O’Donoghue,  1996:  5) 


The  course  made  use  of  12  key  modules  which  covered  aspects  of  design  and  production  equally  and  was  delivered  by  a team  of  tutors 
drawn  from  the  cross-college  multimedia  department.  The  emphasis  on  "production,  authoring,  design  and  management"  provides  an 
indication  that  the  course  was  skills  focused.  This  can  be  compared  to  a degree  scheme  in  multimedia  produced  at  the  same  college  the 
following  year  whose  rationale  had  a wider  developmental  focus: 

” ? [to]  create  a core  of  multimedia  activities,  projects  and  studies  compulsory  for  all  students 
? [to]  create  a progressive  system  of  study  and  workshops  moving  from  the  skill  based  to  the  academic 
and  abstract . 

? [to]  create  a series  of  options  to  facilitate  general  awareness  of  the  changes  in  multimedia 
developments  (both  physical  and  in  thinking),  and  to  support  future  career  choices. " 


(O’Donoghue,  Potter  & Molyneux,  1997) 

The  scheme  also  called  for  "Project  deconstruction  and  critical  analysis  of  multimedia  work  [as]  essential  components  of  each  area  of  work",  a 
mechanism  for  meta-cognitive  activities  and  reflection  within  the  scheme  as  a whole. 

An  examination  of  an  HND  provided  by  the  University  of  Huddersfield  in  1997  stands  in  comparison  to  these  two  schemes  as  an  HE  rather 
than  FE  provider  of  this  qualification.  This  course  developed  and  taught  by  the  School  of  Computing  & Mathematics  offered  the  following 
rationale: 

" Information  (text,  images,  audio  and  video)  is  increasingly  being  presented  in  digital  form  for  delivery  by  computer  based 
systems.  The  pathway  is  concerned  with  applications  of  the  converging  technologies  of  computing,  communications  (including 
cable)  and  digital  media,  and  with  supporting  the  design  and  production  of  attractive  and  imaginative  vehicles  for  learning, 
entertaining,  marketing,  information  provision  and  cultural  exchange." 

(Topping  et  al,  1997) 

Documentation  suggests  that  design  activities  follow-on  in  year  two  from  earlier  computer  related  activities  in  year  one,  such  as  resource 
database  management.  However  two  specific  aims  of  this  course  indicate  a mixture  of  skills  training  coupled  with  reflective  practices: 

" ? To  provide  an  environment  for  students  to  develop  technical  and  design  skills  in  producing,  and  supporting  the  production 
of,  multimedia. 

? To  produce  diplomats  who  understand  the  importance  of  multimedia  in  today's  world,  and  have  the  technical  and  design 
skills  to  communicate  effectively  using  the  tools  and  structures  of  the  new  media". 

(Topping  et  al,  1997) 

Comparing  these  schemes  to  more  recent  activities  allows  us  to  investigate  the  possibility  of  a development  in  understanding  or  an  evolution 
in  what  is  understood  by  multimedia. 

One  related  module  to  an  existing  degree  programme  in  Educational  Studies  at  Keele  University,  UK,  is  called  'New  Learning  and  its 
Technologies’  and  is  designed  to: 

"...develop  the  skills  and  theoretical  knowledge  and  understanding  necessary  to  produce  interactive  multimedia  (IMM) 
resources  for  educational  purposes.  The  two  main  aims  of  the  module  are:  to  introduce  students  to  rationale  for  the  use,  design 
and  creation  of  interactive  multimedia  resources  intended  for  use  in  higher  education;  and  to  assist  students  to  create  an  IMM 
resource  that  exemplifies  good  practice  in  the  design  and  evaluation  of  such  materials". 

(McLean  and  Denning,  2000) 

Whilst  the  paper  reports  a variety  of  findings  on  how  students  progressed  with  the  module  and  took  on  a variety  of  roles  to  complete  their 
tasks,  from  the  course  design  viewpoint  it  is  interesting  to  note  that  the  multimedia  component  forms  part  of  the  confex/for  the  subject  matter, 
i.e.  multimedia  is  not  studied  alongside  other  s ubject  materials  independently,  but  is  used  to  produce  and  reflect  upon  course  materials  as  a 
means  of  providing  a deeper  insight  into  or  to  offer  a new  viewpoint  towards  subject  based  theoretical  knowledge. 

Further  evidence  of  multimedia  production,  design  and  evaluation  used  to  exemplify  subject  skills  has  also  been  identified  in  other  schemes, 
including  a degree  scheme  in  'Interactive  Multimedia  Arts  and  Animation'.  (Potter  et  al.,  2000). 

As  a result  of  these  reviews  there  are  indications  that  a number  of  multimedia  related  courses  may  have  moved  from  a position  of  relatively 
independent  study  of  technical  skills  and  creative  design  to  one  of  contextual  support  and  communication  embedded  in  other  subject  domains. 
The  extent  to  which  this  situation  is  supported  is,  as  yet,  undetermined,  and  It  seems  likely  that  some  HEIs  will  offer  multimedia  with  a subject 
option  with  little  thought  of  cross-over  of  context  or  reflective  practices  for  financial  reasons.  However,  these  glimpses  of  change  add  purpose 
to  a more  detailed  study  of  all  617  course  structures  and  rationales  available.  This  work  is  currently  in  progress^. 
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We  expect  to  be  able  to  report  on  this  work  during  our  presentation. 


Learning  outcomes  from  multimedia  course  provision 

With  such  a wide  array  of  multimedia  and  subject  topics  available  it  seems  appropriate  to  question  the  learning  outcomes  from  such  studies. 

As  many  of  these  courses  are  new  there  are  few  students  who  can  yet  be  questioned  on  their  experiences,  skills  and  concepts  of  multimedia 
as  a result  of  their  study.  However,  one  such  evaluation  on  students  perceptions  of  multimedia  allowed  us  to  make  an  initial  examination  of 
possibilities. 

The  evaluation  was  carried  out  with  12  students  participating  on  a HND  in  Multimedia  course  at  Knowsley  Community  College,  UK 
(O’Donoghue  and  Machell,  2000).  During  the  first  year  of  the  course  students  were  asked  to  define  multimedia.  Answered  tended  to  focus 
into  two  groups; 

• those  which  split  multimedia  into  components  for  production  and  integration,  e.g.  ’Visual,  sound,  text  and  video,  within  a digital  format 
that  can  be  accessed  and  navigated  by  the  user" 

• those  which  were  more  generic  by  nature,  e.g.  "Multimedia  is  the  ways  in  which  to  communicate,  changing  a situation  by  decision",  "A 
computer  course  with  a bit  of  everything". 

The  course  design  consisted  of  a six  month  period  of  intense  skills  development  with  a range  of  authoring  tools,  graphics  packages,  and 
audio-video  software,  followed  by  a period  of  project  work  which  included  industrial  work  placements  and  transnational  project  visits.  When  the 
students  were  interviewed  one  year  later  their  definitions  of  multimedia  were  now  related  to  their  project  work  and  their  experiences  which  they 
used  as  a frame  of  reference  to  explain  applications  rather  than  the  stand-alone  definitions  previously  reported. 

In  his  examination  of  learning  towards  a synthesis  for  networked  learning  environments  and  online  communities,  Goodyear  (2002)  provides  an 
account  of  three  kinds  of  knowledge  (academic,  generic  and  reflexive).  In  doing  this  he  introduces  the  term  Working  Knowledge’  to  which  he 
assigns  a number  of  attributes  including  "knowledge  which  is  relevant  to  one’s  own  work  (when  the  work  may  be  in  academia  or  in  what  other 
people  take  to  be  the  real  world)’’  and  "the  idea  of  knowledge  and  knowing  as  an  active  and  dynamic  rather  than  passive  and  static".  He  goes 
on  to  add: 

"Learning  in  higher  education  should  be  Imbued  with  a belief  in  the  particular  value  of  ’working  knowledge’.  Understanding  and 
engaging  with  different  ’ways  of  knowing’  is  key  to  effective  action  in  academia  and  in  the  workplaces  of  today’s  knowledge 
economy’.’ 

(Goodyear,  2002:55) 

Developing  Goodyear's  theme  to  contextualised  multimedia  teaming,  it  is  possible  to  suggest  that  the  skills  of  manipulation  and  production 
with  a variety  of  contexts  become  a Working  knowledge’  which  the  multimedia  graduate  can  build  on,  in  which  case,  the  study  of  multimedia 
within  a hybrid  course  or  alongside  a subject  base  may  have  many  benefits,  especially  in  meeting  employers’  expectations  (see  Harvey  and 
Mason,  1996).  How  ever,  there  are  also  other  themes  and  issues  to  consider,  not  least  those  of  Wild  and  Quinn  (1998)  who "...  advocate  the 
development  of  a coherent  model  or  models  of  instructional  design  in  multimedia",  and  those  of  McLean  and  Denning  (2000)  who  have 
adopted  a constructivist  approach  to  Interactive  Multimedia  Learning.  More  data  is  required  on  student’s  ideas  and  abilities  in  multimedia  as  a 
consequence  of  the  many  courses  now  available  in  order  to  determine  the  relationships  between  course  design,  rationale,  tuition,  and  skills 
and  concept  development. 

Summary 

In  this  paper  we  have  provided  data  and  analysis  on  617  courses  available  in  the  UK  which  make  reference  to  or  use  of  the  term  ’multimedia'. 

A large  number  of  such  courses  (78%)  offer  multimedia  or  one  of  its  curriculum  variants  with  a specific  topic.  Whilst  many  of  these  topics 
appear  to  compliment  the  use  and  application  of  multimedia  and  its  associated  technologies,  other  do  not  appear  to  do  so  immediately.  This 
gives  rise  to  speculation  that  some  HEls  may  be  offering  courses  with  a component  of  multimedia  to  attract  students  who  may  not  otherwise 
attend  specific  courses  or  institutions.  Analysis  of  course  provision  suggests  that  multimedia  related  courses  have  increased  in  Arts  and 
Humanties  subject  areas,  and  in  Management  and  Business  related  subject  areas,  more  than  in  computing  or  other  ICT  related  areas,  though 
the  authors  acknowledge  limitations  due  to  subjectivity  of  classification. 

A limited  examination  of  course  rationales  suggests  that  multimedia  course  design  may  have  undergone  a shift  of  focus,  from  an  almost 
independent  skills -based  and  technology-led  provision,  to  one  in  which  multimedia  is  used  within  a specific  subject  context.  The  benefits  of 
this  kind  of  course  provision  in  relation  to  learning  have  been  briefly  discussed,  though  it  is  recognised  that  this  area  is  worthy  of  more  detailed 
debate  and  was  not  intended  as  a principal  focus  for  this  study. 

Implication  for  multimedia  course  designers  and  tutors 

If  such  a shift  in  the  focus  of  multimedia  learning  has  taken  place,  perhaps  in  other  countries  as  may  be  happening  in  the  UK,  a series  of 
questions  arise  with  broader  implications  for  tutors,  course  managers  and  students,  not  least  with  assessrneni  procedures,  quality  of  tuition 
and  work  produced,  teaching  strategies,  and  whether  industries  dependent  on  students  emerging  with  a multimedia  qualification,  especially  in 
a subject  related  variant,  find  such  graduates  preferable  to  those  with  perhaps  a single  technical  or  design  focus.  There  are  also  implications 
for  resources  which  multimedia  course  managers  need  to  contend  with,  such  as  access  to  equipment,  actually  time  spent  in  multimedia 
production  activities,  and  staff  training  for  norvtechnical  tutors  asked  to  participate  in  fornns  of  multimedia  production  or  learning  activities®. 
Perhaps  the  area  with  the  greatest  implication  we  are  currently  examining  relates  to  what  we  may  think  of  as  the  essence  of  multimedia;  if  our 
indications  of  the  movement  of  multimedia  from  independent  subject  to  embedded  communication  and  reflective  medium  for  other  curriculum 
subjects  are  true,  then  perhaps  a rethink  of  multimedia  teaching  is  required  in  order  to  best  suit  the  needs  of  students,  employers,  and 
academic  non-multimedia  subject  specialists. 


® Some  of  these  issues  are  raised  by  McLean  and  Denning  (2000)  and  we  expect  to  report  on  this  situation  in  more  detail  during  our 
presentation. 
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Finally,  on  completion  of  this  study,  we  asked  ourselves  the  question  whether  it  was  appropriate  to  remove  the  term  multimedia  from  the  title 
of  our  own  new  degree  scheme.  Insofar  as  we  have  given  the  multimedia  components  much  thought  and  considered  constructive  and  re- 
constructive methods  for  employing  multimedia  in  the  communication  and  the  contextualisation  of  theories  of  learning,  replacing  multimedia  \n 
the  title  would  seem  to  be  a more  accurate  description  of  the  content  and  activities  of  the  course.  Additionally,  in  marketing  terms,  given  the 
relatively  few  courses  we  were  able  to  identify  within  Social  Sciences  with  a learning  component  (see  diagram  1 ),  the  inclusion  of  the  term, 
within  the  multimedia  learning  arena  as  a whole,  would  appear  to  represent  a purposeful  promotional  opportunity  for  the  department  and  the 
university,  but  we  consider  this  secondary  to  the  quality  of  course  provision  and  the  course  design. 
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Continuous  Zoom  applied  to  texts 


Abstract 

The  amount  of  text  that  we  are  required  to  read  in  the  present  days  is  amazing.  One  of  the  reasons 
for  a poor  performance  of  electronic  text  systems  may  result  from  the  fact  of  users  to  avoid  reading 
the  whole  text  and  make  decisions  mostly  based  on  their  headings  (Fox,  1992).  Aiming  to  minimize 
this  problem,  we  have  developed  a computer-based  tool  for  continuous  zoom  interfaces  (graphic 
interface  that  allows  a text  to  be  continuously  magnified  of  decreased).  Prior  to  displaying  the  text, 
this  tool  performs  a previous  analysis  of  the  text,  taking  into  account  its  grammatical  classification. 
Then,  the  text  is  displayed  in  its  most  condensed  format  and  lowest  zoom  level,  where  only  the 
most  important  words  are  shown.  For  that,  the  tool  uses  the  artificial  intelligence  technique  (pattern 
recognition).  The  text  is  then  displayed  in  its  most  condensed  format  and  lowest  zoom  level.  As 
zoom  increases,  more  elements  of  the  text  become  visible.  It  should  be  highlighted  that  fie  tool 
deals  with  language  “traps”  and  whenever  the  pattern  recognizer  fails  to  classify  a certain  word,  it 
makes  that  word  visible  in  its  most  reduced  zoom  format. 

Introduction 

Zoomable  User  Interfaces  are  a kind  of  information  visualization  [2].  Ken  Perlin  of  New  York 
University  defined  the  concept  of  Zoomable  surface  [1].  Ben  Benderson  designed  and  implemented 
PAD++  tool,  which  in  Maryland  University  gave  rise  to  Jazz  [2],  which  in  turn  provided  a Java  API  to 
develop  the  Zoomable  User  Interfaces  (ZUI).  When  working  with  version  0.6  of  Jazz,  we  had  to 
made  several  extensions  to  its  original  format. 

The  objective  of  this  work  is  to  contribute  to  reduce  the  effort  to  read  general  electronic  documents; 
to  produce  and  test  new  ways  to  analyze  texts  and  Identify  patterns  of  sentence  constructions,  as 
well  as  to  classify  grammatically  each  of  their  components.  One  of  the  most  important  questions 
about  electronic  documents  lies  in  the  research  of  their  usability,  which  advantages  will  be  provided 
and  how  they  will  be  [4]. 

The  idea  Is  that  the  tool  is  able  to  analyze  any  type  of  electronic  text  under  the  most  diverse 
formats,  such  as,  .doc,  .html,  etc,  and  then  handle  the  text  through  a continuous  zoom  interface. 
Each  text  would  be  submitted  to  both  a standard  dictionary  (an  inference  motor)  of  a specific  natural 
language,  and  another  zoom-related  dictionary.  This  latter  dictionary  would  be  specific  for  each 
style  of  text  and  would  serve  to  define  the  right  zoom  time  when  a certain  class  of  elements  (text) 
would  be  displayed  in  continuous  zoom  interface.  At  all  times,  text  integrity  would  be  preserved.  In 
other  words,  should  the  inference  motor  fail  to  “find  out”  the  class  of  the  element,  this  will  be  visible 
from  the  first  zoom  level. 

Pattern  Recognition 

Natural  languages  constitute  a number  of  traps  for  any  pattern  recognition  tool;  for  instance,  a Word 
may  have  several  meanings  under  diverse  contexts.  Such  meanings  could  lead  to  a different 
grammatical  class.  In  English,  the  word  Kiss  analyzed  separately  can  be  either  a verb  or  a noun. 
The  final  definition  will  depend  on  the  context  it  is  employed.  In  other  words,  almost  always  it  is 
necessary  to  analyze  a whole  sentence  to  be  able  to  define  the  grammatical  class  of  each  of  its 
components.  Sometimes,  the  definition  of  a particular  element  of  the  text  can  influence  that  of 
another  text.  This  problem  may  lead  the  inference  algorithm  to  an  endless  loop  condition.  In  this 
case,  after  a certain  number  of  attempts,  our  program  ends  by  classifying  the  word  as  “indefinite 
classification”.  This  special  class  is  given  a differentiated  treatment.  Should  a second  element,  in 
order  to  obtain  its  classification,  need  the  classification  of  another  element  in  the  same  sentence 
and  the  latter  is  defined  as  Indefinite  classification”,  the  former  also  assumes  this  condition. 
Therefore,  as  shown  below,  at  constructing  our  patterns,  we  should  be  more  economic  in  generic 
expressions,  that  is,  we  should  use  to  the  extent  possible  rules  that  should  not  depend  on  others. 
The  more  indefinite  dassification  we  have,  the  poorer  the  results  obtained.  There  is  also  the 
problem  of  verb  conjugations.  Here,  the  existence  of  a rule  to  deal  with  them  is  indispensable. 
Otherwise,  we  would  fall  into  the  Indefinite  classification”  condition.  Word  classifications  may  be 
constant,  such  as  adjective,  adverb,  definite  article,  verb,  etc.,  or  indefinite. 

Also,  there  is  a special  treatment  for  abbreviations.  Our  program  tries  to  identify  them  as  soon  as 
possible,  at  an  early  stage  as  we  separate  the  sentences.  This  process  is  important  because  the 
context  to  be  analyzed  is  always  that  of  a sentence.  Likewise,  punctuation  is  also  viewed  in  a 
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special  way.  There  are  marks  that  serve  to  end  sentences  and  other  do  not.  We  list  below  a 
punctuation  table  used  by  the  program. 

Finally,  there  is  the  dictionary  itself,  which  is  constituted  of  lines  of  a text  file.  Each  line  represents 
the  definition  of  a rule  for  word  classification.  Therefore,  each  dictionary  line  follows  the  following 
formation  law: 

<worc>:c/ass/f/caf/on1(context1)|classification2(context2)|  ....  | classificationN(contextN) 

"word"  is  the  term  that  we  are  trying  to  classify;  "'classification''  (shall  obligatorily  be  in  the 
classification  table);  and  "context”,  has  its  own  structure: 

{element  1 , element2 , . . . , element  N) 

Element  includes:  classification  (as  defined  above),  point,  word,  or  reserved  words  {**)  meaning  that 
it  does  not  matter  how  many  and  which  elements  appear  in  that  context  position;  {qq)  means  that  it 
does  not  matter  which  type  of  element  appears  in  that  position,  however  the  quantity  should  always 
be  match  the  number  of  occurrences  of  the  expression  {qq).  The  examples  below  illustrate  how 
such  rules  work: 

Example: 

KISS:  VERB(**,PRON,KlSS,**)  | NOUN(**,KlSS,**) 

The  word  KISS  appears  in  the  dictionary  under  two  possible  classifications,  depending  on  the 
context.  It  is  a verb  when  it  is  preceded  by  a pronoun,  regardless  the  rest  of  the  sentence  or  what 
comes  before  of  after  the  binomial  “PRON”,  “KISS”.  In  cases  where  this  rule  fails,  the  dictionary  will 
try  the  next  one,  which  will  classify  it  as  a noun,  under  any  context..  It  is  if  we  were  working  with  the 
structure:  Exceptioni,  Exception2,  ...,  ExceptionN,  General  Rule. 

Obviously,  the  more  and  better  the  dictionary  is  worked  out,  the  better  the  program  performance. 
We  find  that  the  construction  of  this  base  of  knowledge  is  a good  exercise  for  subjects  related  to  the 
teaching  of  natural  languages.  Upon  analyzing  a text,  the  program  produces  two  files  as  processing 
byproducts.  In  the  first  file,  each  line  represents  a word,  its  classification  and  the  sentence  of  the 
text  where  it  appears. 

This  means  that  the  word  He  was  classified  as  a pronoun  in  the  sentence  He  gave  a pie,  and  so  on. 
The  second  text  file  list  the  expressions  that  could  not  be  classified  and  need  be  defined  or  better 
worked. 

Finally,  after  all  this  analysis,  the  main  output  of  the  program  is  a list  of  sentences  of  the  text,  where 
all  their  components  are  classified  in  a way  or  another. 

Continuous  Zoom  Interface 

Of  all  extensions,  that  dealing  with  the  management  of  the  zoom  stands  out.  Since  we  are  working 
with  text,  each  element  will  be  assigned  its  particular  zoom  control.  This  is  due  to  the  fact  that  we 
will  control  the  exact  time  when  the  element  (text)  will  be  visible  to  the  final  user.  That  time  basically 
depends  on  both  its  classification  defined  as  above  and  the  text  style. 

To  address  this  issue,  the  program  will  be  provided  with  a zoom  dictionary  specific  for  each  style. 
Note  that  while  the  pattern  dictionary  defined  in  the  previous  section  is  generic  for  a given  natural 
language,  our  Zoom  dictionary  is  more  specific.  This  way,  the  program  can  work  with  any 
combination  of  both  the  dictionaries  predefined  by  the  final  user. 

Specifically,  zoom  dictionary  adopts  the  following  format  for  each  one  of  their  lines  in  the  text  file: 
<classification>:  magnitude  level1(Context1)|magnitude  level2  (Context2)|  ...|magnitude  levelN 
(ContextN). 

Where: 

Magnitude  level  is  of  java  float  type,  which  determines  the  time  when  elements  under  that 
classification  used  in  that  context  become  visible.  Zero  value  indicates  that  said  element  will 
become  visible  right  at  the  first  moment  and  is  therefore  an  essential  part  of  the  sentence.  Its 
suppression  will  make  the  sentence  senseless,  or  worse,  it  may  change  its  meaning. 

Concluding,  we  would  like  to  add  that,  for  aesthetic  reasons,  the  automatic  realignment  of  lines  as 
the  different  elements  of  text  appear  and  disappear  has  not  been  yet  implanted.  It  Is  worth  stressing 
that  the  user  can  magnify  the  text  (zoom  in)  or  reduce  it  (zoom  out)  at  any  time. 


1535 


Learning  management  Systems:  One  Size  Fits  All? 


Learning  management  systems  such  as  WebCT  and  Blackboard  are  rapidly  becoming  the 
principal  systems  behind  institutions’  moves  to  technology-based  learning  settings.  In  settings 
where  lecturers  have  had  no  or  limited  exposure  to  ICT  as  a teaching  tool,  LMSs  pave  the  way  for 
quick  introduction  and  eventual  adoption  of  technology-based  teaching  practices.  Many  lecturers 
quickly  progress  and  grow  in  the  field.  But  there  are  those  who  argue  that  the  use  of  LMS  tend  to 
be  counter-productive  and  that  in  their  current  forms,  they  cannot  be  used  effectively  as  a learning 
tool.  Critics  argue  that  they  support  only  traditional  forms  of  teaching  and  encourage  poor 
learning  designs  in  their  implementation.  This  panel  will  explore  LMSs  as  they  exist  today  and 
will  explore  whether  they  are  the  solution  to  a pressing  problem  or  in  fact  problems  within 
themselves. 


The  Panelists 

1.  Allard  Strijker  from  University  of  Twente  in  The  Netherlands  whose  interest  is  in  the  concept 
of  learning  objects  and  reusability. 

2.  Duan  van  der  Westhuizen  from  Rand  Afrikaans  University  in  South  Africa  with  an  interest  in 
staff  development  and  ICT  adoption  within  the  University. 

3.  Tom  Reeves  from  the  University  of  Georgia,,  USA,  has  expertise  in  the  evaluation  of 
technology-based  learning; 

4.  Barry  Harper  from  Wollongong  University  in  Australia  who  is  working  on  a large  project 
exploring  the  development  of  generic  learning  designs  for  Web-based  teaching  and  learning;  and 

5.  Ron  Oliver  from  Edith  Cowan  University  in  Australia  will  chair  the  panel  session. 

Session  Summary 

Duan  van  der  Westhuizen  (South  Africa):  LMS  as  successful  change  agents  for  HE 
The  use  of  LMS  with  their  seamless  and  intuitive  support  for  Web-based  learning  has  helped  to 
pave  the  way  for  institutions  and  organisations  throughout  the  world  to  adopt  ICT  as  a support  for 
teaching  and  learning.  They  necessarily  lack  some  flexibility  and  openness  in  their  design  so  that 
they  can  be  used  by  novices  and  experts  alike.  Duan  will  argue  their  strong  influence  and 
obvious  success  with  some  data  showing  the  level  of  use  and  uptake  in  the  short  time  they  have 
been  around. 

Allard  Strijker  (University  of  Twente):  "One  Size,  but  still  growing,  LMSs:  up  to  the  next 
generation" 

I think  the  current  forms  of  LMSs  are  a solution  that  will  not  last  long.  The  information  that  is 
retrieved  form  the  objects  are  most  of  the  time  not  useful  or  needed  for  standalone  bases.  LMSs 
will  be  incorporated  within  overall  management  systems  and  will  be  a feature  that  can  be  added 
to  the  existing  Human  Resource  Systems.  The  weakness  at  this  moment  is  that  LMSs  are  used  as 
a combination  between  portals  and  extended  logging  systems.  The  information  that  comes 
available  is  most  of  the  times  not  used  for  adaptive  learning  or  for  management  decisions. 

Barry  Harper  (University  of  Wollongong).  Designing  Learning  with  LMSs 
While  many  criticise  LMS  because  they  don't  provide  any  learning  designs,  in  many  cases  they 
don't  need  to.  With  the  development  of  generic  learning  designs,  the  various  LMS  become  very 
powerful  tools  and  their  flexibility  and  apparent  lack  of  structure  is  perhaps  their  most  powerful 
attribute  (not  the  weakness  that  many  claim).  LMS  serve  very  useful  purposes  if  they  are 
designed  to  be  able  to  deliver  resources  of  many  forms  to  the  users,  coupled  with  powerful 
communication  supports  and  learning  management  supports. 


Tom  Reeves  (University  of  Georgia) 

"So  You  Want  to  Know  What  They  Have  Learned:  The  Strengths  and  Limitations  of  LMS  for 
High  Quality  Assessment" 

High  quality  assessment  is  the  lynchpin  in  an  effective  learning  environment.  Although  LMS 
provide  opportunities  for  alternative  forms  of  assessment  that  faculty  may  fail  to  utilize,  they  also 
have  their  limitations.  Tom's  stance  will  be  both  critical  and  constructive. 
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Strategies  For  Beginning  Online  Designers 
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Abstract:  Much  of  the  conventional  development  of  Web-based  learning  environments  stems 
from  design  strategies  that  are  based  on  providing  delivery  of  the  course  content.  Contemporary 
courseware  delivery  systems  encourage  teachers  to  see  the  design  of  online  learning  as  settings  as 
a process  of  electronic  delivery  of  content  and  information.  As  a consequence  Web-based  courses 
have  tended  to  display  limited  evidence  of  an  underpinning  learning  design  and  varying  degrees 
of  use  of  the  opportunities  and  affordances  of  the  new  technologies.  This  paper  provides  an 
overview  of  instructional  design  principles  that  can  guide  beginning  designers  in  the  creation  of 
Web-based  learning  materials  that  support  learner  engagement  and  knowledge  construction.  The 
paper  suggests  strategies  for  professional  development  activities  that  can  help  beginning  designers 
to  reconsider  their  pedagogy  and  to  understand  the  processes  required  to  design  effective  settings 
for  online  learning. 


Introduction 

Despite  modest  gains  and  often  questionable  achievements,  there  is  still  unbridled  interest  and  enthusiasm 
among  institutions  and  organizations  for  online  learning  and  e-leaming  as  the  means  for  the  presentation  and 
delivery  of  higher  education.  Online  learning  has  been  mooted  as  the  solution  to  many  of  the  problems  that  face 
institutions  of  higher  education.  Many  writers  have  argued  the  use  of  o-leaming  to  attract  new  markets  and  new 
students.  Others  have  argued  its  capacity  to  increase  levels  of  equity  and  access  for  existing  students,  while  some 
see  the  new  technologies  as  capable  of  reducing  the  costs  of  delivery  of  programs  and  courses  (eg.  Holt  & 
Thompson,  1998;  Fraser  & Deane,  1997;  Nunan,  1996).  While  many  of  these  claims  and  potentials  have  yet  to 
be  verified  the  one  thing  we  have  learned  from  our  activities  in  this  domain  is  that  e-leaming,  when  done  well, 
can  improve  learning  and  deliver  enhanced  learning  outcomes  and  has  the  prospect  to  transform  the  way 
education  is  conducted  in  these  settings  (Biggs,  2001) 

The  criticisms  that  many  people  hold  for  much  of  the  current  effort  in  the  design  and  delivery  of  online  learning 
stem  from  the  narrow  instmctional  design  models  upon  which  much  of  the  material  is  based.  The  design  models 
in  use  appear  often  to  have  evolved  from  the  economic,  efficiency  and  marketing  imperatives  (eg.  Dehoney  & 
Reeves,  1999;  Mioduser,  Nachmias,  Oren,  & Lahav,1999;  Burbules  & Callister,  2000).  With  on-line  delivery  it 
is  possible  to  pursue  these  imperatives  and  at  the  same  time  to  pursue  such  goals  effective  teaching  and  learning. 

Staff  development  for  online  learning 

Much  of  the  development  of  online  learning  settings  is  currently  undertaken  by  academics  with  little  experience 
or  expertise  in  this  field.  The  skills  and  understandings  of  learning  that  many  teachers  develop  through  their 
face-to-face  teaching  are  often  insufficient  to  support  their  needs  in  online  learning  settings.  Those  academics 
who  feel  comfortable  working  with  technology  in  online  environments  are  rare,  and  there  is  generally  little 
transference  of  expertise  to  their  colleagues  (Bennett,  Priest,  & Macpherson,  1999),  a situation  compounded  by 
an  increasing  number  of  sessional  and  part-time  teachers  (Van  Dusen,  1997). 
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There  is  often  a resistance  to  technology  amongst  university  staff.  Factors  include  a lack  of  experience  or 
confidence  in  using  technology,  caution  about  methodologies  teachers  regard  as  unproven,  and  a belief  that 
computer-based  options  threaten  the  human  interaction  teachers  value  in  face  to  face  teaching  (Sparrow, 
Herrington,  & Herrington,  2000).  In  particular,  the  speed  of  adoption  of  the  Internet  in  higher  education  has 
caught  many  teachers  unaware  and  unprepared  to  face  the  challenges  required  to  succeed. 

A crucial  aspect  of  effective  online  learning  may  hold  the  key  to  the  changing  role  of  the  university  teacher. 
Wade  (1994)  argues  that  the  promotion  of  learner  autonomy  means  increased  responsibility  for  the  student 
which,  if  it  is  to  succeed,  requires  ‘a  strong  framework  of  support  and  guidance  for  the  student  from  the  outset’ 
(p.  13).  In  designing  effective  online  learning  materials,  perhaps  the  most  important  first  step  is  to  create  a role 
for  the  teacher  as  one  of  coaching  and  scaffolding  as  an  alternative  to  the  more  commonly  used  didactic  forms  of 
teaching.  Novices  frequently  fail  to  achieve  this  in  the  online  learning  designs. 

Changed  roles  for  the  teacher 

A traditional  approach  to  the  design  of  learning  environments  proposes  that  the  best  way  to  deal  with  complexity 
is  to  simplify  a topic  by  breaking  it  down  into  its  component  parts.  However,  Perkins  (1991)  suggests  that  the 
temptation  to  over-simplify  learning  environments  should  be  resisted,  and  instead  designers  and  teachers  should 
search  for  new  ways  to  provide  appropriate  scaffolding  and  support.  In  this  situation,  the  teacher  provides  the 
skills,  strategies  and  links  that  the  students  are  unable  to  provide  to  complete  the  task.  Gradually,  the  support  (the 
scaffolding)  is  removed  until  the  student  is  able  to  stand  alone. 

Many  designers  and  administrators  of  online  units  believe  that  they  should  be  self-contained  resources  that 
include  everything  the  student  needs  to  learn  a particular  topic.  However,  teachers  who  expect  students  to  work 
individually  on  online  units  are  not  only  denying  them  the  benefits  of  collaboration,  but  also  the  benefits  of 
expert  assistance — providing  hints,  suggestions,  critical  questions,  and  the  ‘scaffolding’  to  enable  them  to  solve 
more  complex  problems.  Collins,  Brown  and  Newman  (1989)  point  out  that  coaching  is  highly  situation-specific 
and  is  related  to  problems  that  arise  as  students  attenpt  to  integrate  skills  and  knowledge,  a role  that  is  still  best 
performed  by  the  teacher.  Instead  of  providing  and  delivering  information,  the  lecturer’s  principal  function  is  to 
create  collaborative,  challenging  and  supportive  learning  environments  within  which  the  learner  operates. 

Instructional  Design  for  Web-based  learning 

Conventional  approaches  with  their  content-based  approaches  can  be  difficult  molds  for  many  teachers  to  free 
themselves  from.  In  our  work  with  such  teachers  we  have  developed  strategies  that  we  have  found  quite 
successful  in  assisting  this  move.  In  previous  research,  we  have  described  a framework  for  designing  online 
learning  settings  (Table  1).  The  framework  comprises  three  interconnected  elements  which  are  presented  as 
critical  components  for  the  design  of  learning  settings.  In  particular,  the  framework  highlights  various 
distinctions  between  the  elements  involved  in  the  design  of  online  learning  settings  (Oliver,  1999). 


Table  1.  Framework  describing  critical  elements  of  online  learning  settings 


learning  design  elements 

description 

learning  tasks 

The  activities,  problems,  interactions  used  to  engage  the  learners  and  on 
which  learning  is  based 

learning  resources 

The  content,  information  and  resources  with  the  underpinning  knowledge 
and  with  which  the  learners  interact 

learning  supports 

The  schedules,  scaffolds,  structures,  encouragements,  motivations, 
assistances  and  connections  used  to  support  learning 

The  framework  provides  a means  to  identify  the  various  elements  within  learning  settings  and  suggests  emphases 
which  can  be  made  in  the  instructional  design  process.  Contemporary  learning  theories  posit  that  the  forms  of 
learning  design  most  appropriate  to  higher  education  are  those  based  on  constructivist  learning  principles.  The 
above  framework  takes  on  particular  and  discrete  forms  when  applied  this  way. 

a.  Learning  Tasks 

Most  beginning  online  learning  designers  commence  their  planning  by  considering  the  content  to  be  learned. 
Such  approaches  are  very  common  and  characterize  online  settings  based  on  pages  of  text  and  images  which 
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learners  are  required  to  read  and  follow.  The  learning  activities  in  technology-based  environments  play  a 
fundamental  role  in  determining  learning  outcomes  (Wild  & Quinn,  1997).  They  determine  how  the  learners 
will  engage  with  the  course  materials  and  the  forms  of  knowledge  construction  that  will  take  place. 
Contemporary  thinking  suggests  that  the  activities  must  be  active  and  engaging  (eg.  Wild  & Quinn,  1997). 

The  beginning  online  learning  designer  must  be  encouraged  to  think  not  about  content  but  about  how  learners 
will  be  required  to  use  the  content.  They  need  to  consider  ways  to  engender  cooperative  and  collaborative 
activities  among  learners  which  provide  opportunities  for  reflection  and  articulation.  The  selected  activities  need 
to  provide  the  purpose  and  the  context  for  learners  to  deal  with  the  content  and  information  (Duffy  & 
Cunningham,  1996).  The  designers  need  to  plan  activities  with  an  active  role  for  the  learners  in  which  there  is 
encouragement  and  provision  to  take  control,  make  decisions  and  act  in  a self-directed  fashion. 


b.  Learning  Supports 

While  considering  the  forms  of  learner  activity  the  beginning  online  designer  must  also  plan  ways  for  the  setting 
to  support  the  leaner.  This  critical  design  element  involves  the  provision  of  the  support  necessary  to  guide 
learners  and  to  provide  a feedback  mechanism  which  is  responsive  and  sensitive  to  their  individual  needs  (eg. 
McLoughlin  & Oliver,  1998).  A number  of  writers  have  developed  frameworks  to  describe  the  ideal  forms  of 
support  required  for  on-line  learning  environment  and  in  each  case,  there  is  usually  a strong  argument  made  for 
an  active  and  involved  teacher  (eg.  Laurillard,  1993).  The  role  of  the  teacher  needs  to  be  planned  as  that  of  a 
coach  and  facilitator  in  place  of  the  more  didactic  style  often  assumed.  In  contemporary  settings,  this  form  of 
learning  support  is  called  scaffolding  in  recognition  of  the  way  in  which  it  helps  to  build  knowledge  and  is  then 
removed  as  the  knowledge  construction  occurs. 

c.  Learning  Resources 

In  choosing  the  learning  activities  and  planning  the  supports  for  learning,  the  requirements  for  the  forms  of 
course  content  required  will  become  evident.  In  learning  environments  based  around  learning  tasks,  learners  will 
need  access  to  a variety  of  forms  of  content.  The  materials  need  not  all  be  on-line.  The  use  of  conventional 
materials  along  with  electronic  sources  can  provide  the  diversity  often  required.  Previously  designers  created 
course  materials  where  the  content  was  rigidly  organised  and  presented  to  the  learners  in  a strict  sequence. 
Today  it  is  recognised  that  learners  need  to  be  able  to  access  resources  in  a variety  of  ways  and  from  a variety  of 
perspectives  (Herrington  & Oliver,  1995). 

An  Approach  to  Web-based  Instructional  Design  for  Beginners 

Putting  these  elements  together  in  a meaningful  and  effective  fashion  is  aided  considerably  if  beginning  online 
designers  follow  the  6 stage  process  outlined  below.  This  process  provides  an  organizational  framework  that 
can  assists  novice  designers  and  teachers  to  focus  on  strategies  that  create  learner-centred  settings  which  can 
engage  and  support  student  learning.  The  framework  helps  to  remove  the  attention  away  from  the  content  as  the 
organising  element  towards  learning  tasks.  The  provision  of  such  a framework  for  the  novice  designer  provides 
a strong  scaffold  for  their  learning,  particularly  if  models  and  exemplars  are  also  provided  for  them  to  follow.  It 
is  often  a very  simple  process  to  model  the  form  of  professional  development  provided  around  this  framework  as 
well. 

Table  2:  Task-based  design  strategies 


Stage 

1 . Plan  the  mtcndcu  i^uming 

2.  Plan  the  learning  tasks 

3.  choose  assessment  strategies 


Stage  (cont.) 

4.  plan  resource  needs 

5.  plan  support  strategies 

6.  plan  the  organization  strategy  for  the  online  materials 


1.  Planning  the  intended  learning  outcomes 

The  best  place  to  commence  the  process  of  developing  the  online  learning  resources  is  by  considering  the 
intended  learning  outcomes.  The  beginning  designers  need  to  recognize  that  intended  learning  outcomes  need  to 
be  expressed  in  terms  that  describe  the  learning  that  will  be  achieved.  For  many  courses,  it  is  possible  to  express 
objectives  in  terms  of  the  learner  performance  or  competency. 
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It  is  important  that  the  learning  outcomes  are  not  expressed  solely  in  terms  of  learning  process  or  strategies.  The 
processes  and  strategies  are  the  means  by  which  the  learning  will  be  achieved.  It  is  important  for  the  beginning 
designers  not  to  confuse  the  activities,  which  are  a means  to  an  end,  with  the  intended  learning  outcomes. 

2.  Planning  the  learning  tasks 

Once  the  learning  objectives  have  been  stated,  beginning  designers  need  to  consider  and  plan  the  learning 
processes  that  will  provide  learners  with  the  experiences  that  will  help  them  to  learn.  The  learning  processes 
need  to  be  carefully  planned  with  the  intended  learning  outcomes  in  mind.  Planning  the  learning  experiences 
involves  cunsideralion  of  ways  to  engage  the  learners  in  ways  that  will  cause  the  required  forms  of  cognition  and 
thinking.  The  learning  activities  need  to  involve  a variety  of  processes  that  will  cause  the  learners  to  deal 
meaningfully  with  appropriate  resources  and  information. 

The  best  forms  of  learning  activity  to  consider  are  those  that  are  authentic.  Authentic  activities  have  been 
described  earlier  and  usually  involve  the  development  of  a product  or  an  artefact  in  much  the  same  way  as  the 
learner  will  need  to  demonstrate  and  practice  in  real  life.  Effective  learning  designs  often  involve  forms  of 
communication  and  collaboration,  activities  that  encourage  the  learners  to  reflect  on  their  learning  and  to 
articulate  their  ideas  in  ways  that  strengthen  and  assist  the  knowledge  construction  process.  Some  examples  of 
typical  learning  designs  include: 

• Having  learners  design  and  evaluate  a project  plan  or  proposal.  To  complete  the  design  the  learners 
need  to  develop  skills  and  competencies  in  the  underpinning  knowledge  and  concepts; 

• Having  learners  work  collaboratively  to  develop  a solution  to  an  ill -defined  problem.  The  learners  need 
to  plan  the  problem  solving  process,  gather  the  necessary  information,  decide  on  the  steps  to  be  taken, 
suggest  a solution  and  demonstrate  the  reasoning; 

• Having  learners  assume  the  roles  of  particular  players  in  a role-playing  setting.  This  involves  exploring 
the  person  whose  role  they  are  playing,  understanding  the  context  and  the  issues  and  acting  in  scenarios 
in  ways  that  use  this  knowledge  to  build  a further  understanding  of  the  person  and  the  setting. 

The  critical  factor  in  planning  the  learning  design  is  to  consider  activities  that  are  independent  of  the  learning 
resources.  It  is  important  when  planning  the  design  not  to  be  concerned  with  the  content  and  resources  and  to 
see  these  as  completely  independent  elements  in  the  design  process. 

3.  choosing  assessment  strategies 

At  the  same  time  as  the  learning  activities  are  being  planned,  it  is  important  to  decide  how  the  learning  outcomes 
will  be  assessed.  The  use  of  authentic  learning  activities  provides  a sound  basis  for  assessment  since  the 
outcomes  from  the  activities  often  demonstrate  the  extent  and  scope  of  the  learning  that  has  been  achieved.  The 
assessment  strategies  need  to  be  derived  from  the  stated  learning  outcomes.  It  is  important  that  the  forms  of 
assessment  not  only  provide  evidence  of  the  extent  of  learning  but  also  provide  guidance  to  the  learners  in  ways 
that  can  model  and  demonstrate  areas  where  further  activity  is  still  required. 

4.  planning  support  strategies 

Online  learning  is  best  designed  with  deliberate  and  intentional  support  mechanisms  for  learners.  There  are 
many  possible  support  strategies  that  can  be  implemented  and  these  include  roles  for  the  teacher,  other  learners, 
workplace  associates,  peers,  and  mentors.  It  is  also  possible  to  plan  the  learning  activities  so  that  they  have 
inbuilt  supports  and  to  provide  resources  which  can  assist  in  supporting  learners. 

The  purpose  of  any  support  in  an  online  setting  is  to  enable  learners  to  undertake  the  activities  that  have  been  set 
in  ways  which  assist  their  independence  and  self-regulation.  With  authentic  learning  settings,  students  will 
require  a range  of  supports  to  assist  them  as  they  develop  their  knowledge  and  skills.  Many  of  these  supports, 
scaffolds,  can  be  reduced  as  the  learning  process  progresses  to  the  point  where  the  learners  develop  the  capacity 
to  cater  for  themselves.  Beginning  designers  can  be  encouraged  to  plan  the  supports  as  they  plan  the  learning 
tasks.  Often  the  tasks  suggest  the  forms  of  support  that  will  be  required. 

The  planning  of  support  strategies  should  be  a part  of  the  planning  of  learning  designs  and  learning  resources, 
both  of  which  have  the  capacity  to  provide  supports  for  the  online  learner.  Learning  activities  that  provide 
learning  support  include  collaborative  settings,  settings  involving  workplace  activities,  involvement  of  buddies 
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and  mentors.  Learning  resources  that  provide  supports  include  such  items  as  templates,  checklists  learning 
pathways,  and  performance  support  systems. 

5.  planning  the  resource  needs 

In  order  to  complete  the  learning  activities  the  learners  will  need  access  to  a variety  of  resources  and  information 
and  part  of  the  design  process  is  to  decide  the  forms  and  extent  of  the  resources  that  will  be  needed.  Learning 
resources  can  take  many  forms  including  such  information  sources  as  textbooks,  newspapers,  journals  and  online 
information  sources.  The  beginning  designers  need  to  select  resources  from  a variety  of  settings.  They  should  be 
discouraged  from  Cuusidcring  writing  and  developing  the  resources  themselves.  1 hey  need  to  be  encouraged  to 
consider  reuse  and  choosing  from  among  those  that  already  exist. 

Depending  on  the  forms  of  learning  activity,  the  resources  might  also  include  documents  and  manuals  from 
industry  settings,  legislation,  rules  procedures  and  policy  manuals.  In  online  settings  there  is  considerable  scope 
for  the  use  of  existing  documents  and  resources.  Often  the  environment  will  include  resources  that  have  been 
specially  built  for  the  learning  setting.  For  example  examples  of  previous  students’  work,  interviews  with 
experts,  selected  readings,  compilations  of  resources.  It  is  not  uncommon  for  the  resource  sets  to  contain 
tutorials  and  learning  modules  to  assist  students  to  acquires  some  of  the  underpinning  skills  and  knowledge. 

Part  of  the  design  process  is  to  determine  what  resources  students  will  need  to  complete  the  activities  that  have 
been  set  and  to  collect  (and  make)  these  resources  for  the  learning  setting.  As  more  and  more  resources  are 
being  placed  on  the  Web,  the  need  for  developing  resources  is  diminishing  and  many  learning  settings  are  able  to 
be  resourced  almost  entirely  from  existing  materials.  The  majority  of  the  resources  in  online  settings  should 
have  no  instructional  elements.  It  is  best  to  separate  the  learning  activities  from  the  learning  resources  so  that 
both  the  resources  and  the  activities  can  be  modified  and  customised  and  reused.  When  the  resources  are  also 
instructional  elements,  the  capacity  for  customisation  and  reuse  is  significantly  hampered. 

6.  planning  the  organization  strategy  for  the  online  materials 

Once  the  learning  activities,  resources  and  supports  have  been  designed,  the  final  stage  is  to  consider  an 
organisational  and  contextual  strategy  that  will  provide  a natural  and  intuitive  setting  for  them.  There  are  many 
ways  to  organise  online  learning  settings.  Many  teachers  will  have  access  to  powerful  online  courseware 
delivery  systems  such  as  WebCT  and  Blackboard.  These  tools  provide  their  own  organizational  frameworks  for 
the  learning  setting.  In  some  cases,  it  is  possible  and  useful  to  design  a unique  interface  and  organizational 
structure,  for  example,  a virtual  setting  reflective  of  the  learning  context. 

Beginning  designers  with  access  to  courseware  delivery  systems  are  often  able  to  play  a significant  part  in  the 
process  of  developing  the  online  setting.  The  critical  factor  in  organising  the  setting  is  to  ensure  that  all  aspects 
of  the  learning  designs,  the  learning  resources  and  the  learning  supports  are  clearly  evident  to  the  learners  and 
freely  accessible.  The  site  needs  to  enable  learners  to  move  freely  within  and  to  know  at  all  times  where  they  are 
in  the  hyperspace  and  how  to  get  to  all  other  places. 

Summary  and  Conclusions 

We  have  been  working  with  beginning  online  designers  for  some  years  now  across  a variety  of  projects  and 
educational  sectors.  Many  of  those  looking  to  design  and  build  online  courses  for  the  first  time,  have 
preconceived  ideas  about  how  the  setting  will  be  formed  and  many  times  these  ideas  are  based  on  very 
conventional  views  of  teaching  and  learning.  The  strategies  we  have  presented  in  this  paper  provide  a strong 
framework  to  guide  the  professional  development  of  these  teachers.  The  framework  provides  a structure  which 
causes  the  teachers  to  rethink  their  pedagogies  and  to  plan  engaging  and  active  learning  settings  where  the  role 
of  the  teacher  is  more  a coach  and  facilitator  than  an  expert  delivering  content.  As  well  as  providing  a 
mechanism  to  assist  in  a pedagogical  rethink,  the  framework  also  provides  a form  of  blueprint  and  template  to 
guide  effective  design  strategies. 

The  biggest  challenge  we  have  found  in  providing  learning  support  for  beginning  designers  is  to  help  teachers 
reconsider  and  to  alter  the  pedagogical  viewpoints  they  hold.  Strategies  that  assist  in  this  endeavour  are  tasks 
and  activities  that  provide  the  learners  with  models  and  exemplars  to  follow.  It  is  very  important  for  the 
professional  development  exercise  to  practice  what  it  preaches  and  for  it  to  be  an  example  of  best  practice.  There 
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are  now  many  examples  and  cases  for  teachers  to  use  for  this  purpose  and  the  success  of  these  endeavours  is  now 
becoming  evident  in  the  growing  number  of  quality  online  learning  settings  that  are  beginning  to  emerge. 
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Abstract:  Maps  are  used  to  display  information  for  educational  and  scientific 
purposes  on  a daily  basis.  The  capability  of  generating  web-based  interactive 
maps  by  non-specialists  and  non-programmers  is  of  great  convenience.  The 
map  generating  system  described  in  this  paper  can  handle  any  arbitrary  map, 
including  those  with  embedded  and  disjoined  regions.  The  resulting  web-based 
interactive  maps  can  be  used  in  many  different  applications  requiring  color- 
coded  display  of  quantitative  data. 


1.  Introduction 

The  Digital  Flora  of  Texas  (DFT)  project  has  been  actively  pursuing  the  constmction  of  biodiversity 
informatics  tools  for  use  by  interested  biologists,  students,  scholars  and  researchers.  One  of  the  browsing 
tools  that  has  been  used  extensively  is  the  Herbarium  Specimen  Browser  (HSB),  originally  developed  in 
1996  and  modified  based  on  user  feedback  over  the  last  five  years.  The  HSB  allows  users  to  survey  the 
content  of  collections  from  various  herbaria  throughout  the  state  of  Texas.  It  supports  fast  access  and 
dynamic  filtering  of  data  on  the  fly.  Information  related  to  any  combination  of  counties,  herbaria  and 
specimens  can  be  displayed  in  real  time  by  merely  clicking  on  the  interface  shown  in  Figure  1 . 

The  HSB  also  provides  an  interactive  map  that  dynamically  displays  the  density  and  distribution  of 
herbarium  specimens  based  upon  user  selections  of  plant  families,  genera,  or  species  (Fig.  2).  Clicking  on  a 
county  displays  the  characteristics  of  the  plant  specimens  that  have  been  collected  from  the  county  by  the 
selected  herbaria.  The  HSB  supports  three  mapping  options:  counties  of  the  state  of  Texas;  counties  of  the 
states  of  Texas,  Oklahoma,  Louisiana,  New  Mexico  and  Arkansas  (shown);  and,  states/provinces  of  North 
America.  Lastly,  it  allows  users  to  query  the  collection  using  any  combination  of  search  terms  such  as  name 
of  herbaria,  specimen  accession  number  or  country. 

Of  course,  for  this  mapping  system  to  work  properly,  someone  must  generate  the  map  and  associate 
commands  that  make  it  respond  to  users’  requests.  In  the  original  HSB,  one  had  to  obtain  a map  in  PBM 
format,  convert  the  map  into  Run-Length  Encoding  (RLE)  format,  and  associate  region  names  with  the 
pixels  in  the  image  before  the  map  could  be  used.  The  entire  process  was  tedious  and  error  prone. 

The  objective  of  this  project  was  to  simplify  the  process  of  generating  a mapping  system  by  providing 
a web-based  automated  tool  that  accepts  some  of  the  widely  used  graphics  formats  for  the  PC.  In  other 
words,  given  files  in  GIF,  JPEG  or  BMP  format,  the  system  automatically  generates  the  RLE  file  needed 
for  the  HSB  or  other  run-length  encoded  mapping  system.  To  remove  the  burden  of  manually  associating 
region  naming  with  the  map  regions,  the  system  allows  users  to  name  map  regions  easily  and  generates  the 
required  files  that  allow  the  browser’s  mapping  system  to  recognize  the  regions  being  selected  by  the  user. 
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A verifier  is  also  provided  that  allows  users  to  check  and  correct  errors  in  their  naming  schemes  and  serves 
as  a prototype  for  interactive  mapping  routines. 
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Figure  1:  The  interface  of  the  Herbarium  Specimen  Browser 


Figure  2:  Distribution  of  specimens  from  counties  in  Texas, 
Oklahoma,  Louisiana,  New  Mexico  and  Arkansas 
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QIICC  - Using  Technology  to  Involve  Students  In  Assessment 


The  objective  for  this  session  is  to  show  participants  a new  and  innovative  way  of  involving  students  in  the 
assessment  process.  Participants  will  learn  how  to  incorporate  the  QIICC  Analysis  protocol  into  everyday 
assessment  strategies.  This  lecture  and  presentation  will  include  a demonstration  of  the  QIICC  analysis 
protocol  utilized  in  a web-based  quizzing  system  called  Telequiz.  During  this  presentation,  the  theory  of  the 
QIICC  Analysis  protocol  will  be  discussed.  This  theory  dictates  that  each  question  on  a quiz  be  rated  on 
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Qualities,  Interest,  Importance  are  measured  in  a straightforward  manner.  For  each  question  a student 
answers  on  a test  or  quiz,  they  are  asked  to  designate  the  quality  of  the  question,  their  interest  level  in  the 
content  of  the  question  and  their  impression  of  how  important  the  question  is  to  the  learning  of  the  course. 
Each  of  these  variables  is  designated  using  a three-level  scale:  High,  Medium  and  Low. 


The  next  variable  indicates  the  confidence  students  have  in  the  accuracy  of  their  answer.  This  is  gathered 
using  the  same  three-level  scale,  and  affects  the  score  that  the  student  receives  on  each  question.  Maximum 
points  are  awarded  for  a question  that  is  answered  correctly  with  high  confidence.  On  questions  that  have 
been  answered  accurately,  scores  fall  as  confidence  levels  fall.  For  those  questions  that  are  answered 
incorrectly,  question  score  rises  as  confidence  levels  fall.  This  is  due  to  the  fact  that  if  someone  states  high 
confidence  in  a wrong  answer,  or  low  confidence  in  a correct  answer,  there  is  a disconnect  between  what 
the  student  knows  and  what  they  expect  to  know.  If  a student  indicates  high  confidence  in  a correct  answer, 
or  low  confidence  in  an  incorrect  answer,  there  is  no  such  disconnect.  That  student  possesses  a realistic 
understanding  of  their  knowledge. 

The  final  variable  is  challenge.  In  the  QIICC  analysis  protocol,  each  student  is  allowed  to  challenge  a 
certain  number  of  questions  per  testing  cycle.  Any  question  that  is  challenged  does  not  count  against  the 
score.  This  allows  students  to  think  critically  about  the  form  and  content  of  a question,  and  take  a more 
active  role  in  the  outcome  of  the  quiz  or  test. 


The  QIICC  Analysis  protocol  is  designed  to  elicit  student  feedback  on  the  quality  of  items  found  on  a quiz 
or  test.  By  doing  so,  it  allows  the  instructional  staff  to  better  analyze  the  effectiveness  of  the  individual 
elements  that  are  included  in  the  assessment  process.  More  importantly,  the  QIICC  Analysis  helps  students 
analyze  their  own  knowledge  levels,  and  by  doing  so,  understand  “what  they  know,  and  what  they  do  not 
know.”  In  the  QIICC  protocol,  students  are  scored  on  not  only  how  accurately  they  answer  a question,  but 
also  on  how  confident  they  are  in  the  accuracy  of  their  answer.  This  capability  is  a powerful  tool  for 
assessing  student  learning.  Through  the  confidence  variable,  students  are  asked  to  take  a much  more  critical 
role  in  the  development  of  their  knowledge.  By  asking  for  a confidence  level  and  incorporating  it  into  the 
grading  rubric,  it  is  much  more  important  for  a student  to  “know  what  they  know”  than  is  seen  in  other 
assessment  strategies.  By  allowing  for  challenges,  QIICC  empowers  students  to  feel  comfortable  with  the 
assessment  process  and  take  control  of  the  knowledge  that  is  deemed  important  or  worthwhile  for  learning. 


Patrick  O'Shea  is  an  Instructional  Technology  Specialist  in  the  Brunswick  County  Public  School  system. 
Currently,  Mr.  O’Shea  is  working  on  his  Doctorate  in  Urban  Services  at  Old  Dominion  University.  Dr. 
Dwight  W.  Allen  is  the  Eminent  Professor  of  Educational  Reform  at  Old  Dominion  University.  Simon 
Richmond  is  a Doctoral  Candidate  at  Old  Dominion  University,  who  has  been  responsible  for  overseeing 
the  implementation  of  the  Telequiz  Assessment  Tool.  Dr.  Allen  and  Mr.  O’Shea  envisioned  and  designed 
the  Telequiz  and  QIICC  Analysis  protocols  together,  and  piloted  the  protocol  in  Dr.  Allen’s  Educational 
Curriculum  and  Instruction  Courses  for  pre -service  teachers  at  Old  Dominion  University. 


Virtual  Knowledge  Communities  in  an  Industrial  Environment 
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Introduction: 


Koneywell  CA  is  a worldwide  company  leader  in  industrial  automation.  As  a high  tech 
business,  needs  to  have  its  employees  on  the  top  of  the  technological  advancements  which 
means  permanent  training  and  development.  Also  it  is  going  through  a process  for  making 
digital  all  the  activities  including  employees  training.  The  strategy  being  used  is  the 
creation  of  a Knowledge  Communities  (KC)  portal  in  the  Intranet.  This  strategy  is  being 
supported  and  implemented  in  order  to  achieve  the  most  of  the  employees  learning  by  using 
the  many  opportunities  available  in  that  site.  In  this  paper,  the  emphasis  is  the  learning 
processes,  cost  reduction  and  goals.  There  will  be  also  analysed  the  pros  and  cons  of  that 
modality  of  training  for  one  Latin  American  affiliated. 

Knowledge  Communities  at  Honeywell  lA  Portal: 

A KC  is  a remote  virtual  space  for  communication,  problem  solving,  lesson  learned 
sharing,  training  and  knowledge  building  among  Honeywell  employees.  It  is  a very  rich 
learning  environment  represented  in  a portal  that  includes:  1)  Knowledge  Communities  on 
products,  markets,  disciplines,  roles  and  methodologies,  and  certifications.2)  templates, 
video  tutorials,  tips  and  tricks.  3)  Training  by  using  on  line  presentations,  video  tutorials 
ordered  and  schedule.  4)  Links.  5)  e-Library.  6)  Other  key  information  resources. 

The  Knowledge  Communities:  There  are  60  knowledge  communities  in  the  industrial 
portal  by  disciplines,  products,  markets,  roles  and  methodologies.  Each  KC  has  a similar 
structure:  members,  best  practices,  tools,  topics,  lesson  learned.  The  rationale  behind  each 
one  is:  sharing  knowledge,  problem  solving,  challenging  our  professionals  and 

collaborating  to  improve  and  innovate  for  the  generation  o new  knowledge.  Each  one  has  a 
Knowledge  Manager. 

Training:  The  Employee  Learning  Portal  includes  courses,  links  for  training,  video 
tutorials,  on  line  presentations,  technical  support  and  certifications.  This  is  one  the  portal 
most  important  features  for  the  digitization  of  the  company.  It  is  expected  to  reduce  in  a 
very  significant  amount  training  investment.  Benefits  expected:  maintain  people  up  dated 
on  products  and  technology  in  a more  effective  way  . On  line  training  sessions  are 
available  to  people  anytime.  Distance  learning  on  line  is  also  scheduled.  The  training 
opportunities  and  options  are  renewed  and  scheduled  maintaining  people  up  dated. 

E-Library:  people  can  find  documents,  resources  and  e-books  in  digital  format.  Key  to 
Honeywell  digitization  initiatives  as  well  as  efforts  to  reduce  paper  copies. 

Links : There  are  60  links  to  possible  sources  of  information,  tools  and  key  sites. 

* 


Honeywell  Andean  Region  Training  and  Development,  Knowledge  Community  Manager  in  LAR. 


Other  Key  Information  Resources:  there  is  available  a diversity  of  information  from 
airlines,  dictionaries,  people,  search  engines,  writing  guide,  business,  organizational  charts, 
news,  etc. 

Cost  Reduction  Measures: 

The  system  implemented  in  the  portal  permits  to  measure  cost  reduction  by  asking 
questions  regarding:  the  number  of  people  being  trained,  T&L  costs  if  the  training  was 
taken  in  another  place.  That  information  is  recorded  and  reports  presented  to  the  site 
administrator. 

Some  Advantages; 

Sharing  of  knowledge  at  most,  that  was  not  done  before. 

Lesson  Learned  sharing  between  world  wide  affiliates 
Improving  of  communication 
Cost  Reduction 

Challenges; 

Creation  of  the  distance  learning  culture  among  employees. 

Creation  of  the  knowledge  sharing  among  people 
Creation  of  incentives  for  training  on  the  web  achieved 
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Abstract:  A joint  Dutch-Scandinavian  distance  education  project,  part  of  the  DUSC  Programme,  was 
carried  out  from  November  1999  to  the  summer  2001.  The  aim  of  the  project  was  to  develop  a module 
to  train  lecturers  in  how  to  develop  a course  based  on  using  the  concept  of  collaborative  learning  in 
distance  education,  and  to  test  and  evaluate  the  module.  In  the  developing  process,  the  authors 
themselves,  from  two  different  countries  and  four  different  faculties,  went  through  a collaborative 
process,  exploring  different  aspects  of  communication  and  collaboration  at  a distance.  The  main 
communication  tool  was  the  asynchronous  German  BSCW  system,  but  also  the  synchronous  tools 
NetMeeting  and  videoconference,  and  face-to-face  meetings  were  used. 


Introduction 

The  DUSC  (Dutch  Scandinavian  Cooperation  in  Higher  Education)  Program  was  an  initiative  of  four  Dutch 
universities  of  professional  education  to  develop  expertise  with  Scandinavian  institutions  of  higher  education.  A 
number  of  projects  were  outlined,  and  Information  and  Communication  Technology  (ICT)  was  an  explicit  sub- 
component of  these. 

One  of  the  project  groups  was  called  CLiDE  (Collaborative  Learning  in  Distance  Education).  The  aim  of  the 
project  was  to  develop  a module  to  train  lecturers  in  how  to  develop  a course  based  on  the  concept  of  active 
learning  in  distance  education.  The  idea  was  that  up  to  now  lecturers  have  been  trained  to  develop  an  active 
learning  course  in  a classroom  situation,  but  often  do  not  have  the  skills  to  develop  such  a course  for  their 
students  at  a distance.  After  the  development  of  the  module  the  module  was  to  be  tested  and  evaluated  by 
running  it  with  a selection  of  interested  lecturers  within  the  educational  institutes  of  the  DUSC  network. 

The  objectives  stated  in  the  project  plan  clearly  divided  our  work  into  two  different  phases:  Firstly,  to  develop 
the  module,  and  secondly,  to  test  and  evaluate  the  module.  Accordingly,  the  first  few  months  we  were  supposed 
to  cooperate  and  collaborate  in  a creative  design  and  development  process,  and  then,  for  the  next  months,  try  out 
our  results  with  a group  of  students.  Most  of  the  collaboration  was  to  be  mediated  by  means  of  electronic 
networks.  This  way  of  organising  a project  work  seemed  very  useful  and  interesting,  because  by  developing  a 
course  in  a collaborative  way  (phase  1),  we  would  go  through  processes  similar  to  our  students  when  the 
developed  module  was  to  be  tested  and  evaluated  (phase  2).  We  believed  that  this  experience  would  make  us 
better  teachers  and  tutors  for  our  students.  For  distance  education  courses  in  general,  this  way  of  developing  a 
course  may  prove  to  be  successful. 


The  Project,  Phase  1 ^ 

This  was  in  fact  a distance  project  group,  with  members  in  two  different  countries  (The  Netherlands  and 
Norway).  We  had  to  rely  on  electronic  communication,  and  decided  to  do  the  greater  part  of  our  collaboration 
based  on  the  asynchronous  system  BSCW  (BSCW  2002).  BSCW  is  a computer-based  conferencing  tool, 
developed  at  German  National  Research  Center  for  Information  Technology  (GMD),  and  have  facilities  for  a 


great  range  of  functionalities  in  the  area  of  managing  discussions  and  collaborative  processes.  We  found  this  tool 
to  be  a good  place  for  our  discussions  and  contributions  toward  our  final  product:  The  module.  We  could  all 
reach  our  common  space  in  BSCW  from  wherever  we  were  in  the  world,  and  so  our  project  work,  as  far  as  this 
collaboration  is  concerned,  was  distance-independent. 

Our  Plan  of  Activity  outlined  four  different  elements  or  themes  to  be  worked  out  by  the  project  group:  A:  The 
concept  of  collaborative  learning,  B:  The  choice  of  the  digital  learning  space,  C:  The  practical  experience  and 
D:  Critical  success  factors.  Each  of  the  project  group  members  was  assigned  the  role  of  moderator  for  one  of  the 
themes,  and  thus  our  responsibilities  were  shared  by  all  of  us.  It  was  also  our  intention  that  being  in  charge  in  the 
moderating  role  would  provide  us,  the  project  group  members,  with  valuable  experience  in  this  aspect  of 
conferencing  and  collaboration  work.  By  doing  this  in  phase  1 of  the  project,  we  felt  we  would  be  better 
prepared  for  the  testing  and  evaluating  part  (phase  2). 

We  also  planned  other  activities,  other  ways  to  do  part  of  our  project.  In  theme  B,  The  choice  of  the  digital 
learning  space,  we  used  a videoconference  session,  in  addition  to  BSCW.  In  theme  C,  The  practical  experience, 
we  used  a synchronous,  computer-based  conferencing  tool  (Microsoft  NetMeeting),  and  after  theme  C,  we 
planned  and  carried  out  a face-to-face  meeting  in  Bergen.  See  the  complete  report  (Stofberg,  Otteraa,  Geers  van 
Gemeren,  Knudsen  2001)  for  experiences  with  these  different  sessions.  A web-site  (CLiDE  2000-01)  was  also 
established  for  the  project;  there  we  gathered  some  overall  information,  plans,  links  to  literature,  etc. 

Our  collaborations  in  phase  1 of  the  project  resulted  in  two  papers,  The  Concept  of  Collaborative  Learning  and 
The  Tools,  given  as  appendices  in  Stofberg,  Otteraa,  Geers  van  Gemeren,  Knudsen  (2001),  meant  as  background 
material  for  the  students  in  phase  2. 


The  Project,  Phase  2 

We  defined  four  main  tasks  for  our  students,  1)  Familiarising  with  the  BSCW  system  and  the  group,  2)  Problems 
and  possibilities  in  digital  learning  environments,  3)  About  different  tools  and  4)  Synthesizing  and  elaborating 
on  different  aspects  of  web-based  collaborative  learning  methods.  For  each  of  the  tasks,  we  specified  certain 
learning  objectives  and  a number  of  assignments.  It  was  decided  that  the  moderation  was  to  be  done  by  tutors  in 
pairs.  The  two  tutors  would  then  take  turns  in  taking  the  leading  role  in  the  moderation  of  the  course,  and  would 
turn  to  the  other  if  he/she  needed  advice  or  support  in  the  moderation  task. 

Studying  the  contributions  from  our  students,  there  are  definitely  a number  of  evaluative  remarks  worth 
mentioning.  The  tutors  would  corroborate  that  many  of  the  issues  were  also  the  issues  that  affected  the  process 
and  product  of  their  collaborative  work  while  designing  and  working  out  the  course  (phase  1).  For  the 
evaluation,  we  refer  to  the  report  (Stofberg,  Otteraa,  Geers  van  Gemeren,  Knudsen  2001). 


Conclusion 

Keeping  a long-term  international  relationship  or  collaborative  project  going  is  not  an  easy  matter.  Nevertheless, 
the  special  nature  of  this  project  made  it  possible.  Building  up  expertise  together,  developing  yourself  and  the 
group  as  a whole  by  going  through  a number  of  separate  and  clearly  structured  stages  with  each  other,  taking 
rotating  independent  responsibilities  (to  keep  each  one  of  us  alert  and  motivated),  pooling  resources  and 
knowledge,  making  joint  selections,  and  then  taking  these  personal  learning  experiences  even  further  by  putting 
them  to  use  in  the  same  project:  this  variety  of  elements  and  tasks  kept  the  project  alive  and  contributed  highly  to 
our  achievements.  All  in  all,  we  feel  we  have  greatly  benefited  from  this  unusual  experience  provided  for  us  by 
the  DUSC  programme. 
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Abstract:  Multimedia  elements  addressing  mathematical  problems  serve  as  an  alternative  or  addi- 
tion to  classical  forms  of  learning  and  teaching.  Within  the  project  math-kit  we  are  developing  a 
web-based  toolbox  to  provide  teachers  and  students  with  multimedia  support  to  existing  topics  in 
maths.  In  this  context,  the  integration  of  a computer  algebra  system  (CAS)  as  a 'mathematical  expert* 
offers  many  promising  opportunities  for  the  creation  of  internet -based  learning  units  for  undergradu- 
ate students.  In  this  paper  we  show  two  examples  of  how  the  CAS  MuPAD  is  used  to  create  web- 
based  mathematical  drills  with  instant  feedback. 


Introduction 

Mathematics  is  cmcial  for  the  understanding  of  many  scientific  fields  especially  the  technical  sciences. 
However,  the  level  of  abstraction  of  mathematics  poses  considerable  problems  for  many  undergraduate  students. 
math-kit\  a joint  project  of  the  four  German  universities  Bayreuth,  Hagen,  Hamburg  and  Paderbom,  as  well  as 
SciFace  Software,  is  intended  to  offer  support:  we  are  developing  a web-based  toolbox  to  provide  teachers  and 
students  with  multimedia  elements  addressing  mathematical  problems. 

The  aspects  of  math-kit  are  many  (see  Bungenstock  et  al.  2002,  Mertsching  et  al.  2002).  In  this  brief 
paper,  we  concentrate  on  a single  feature,  namely  the  integration  of  the  computer  algebra  system  (CAS)  Mu- 
PAD^  in  some  of  our  learning  units.  In  particular,  we  show  two  examples  of  how  the  CAS  can  be  used  to  create 
web-based  drills  with  instant  feedback. 


Using  a Computer  Algebra  System  on  the  Web 

MuPAD  is  a CAS  which  offers  several  powerful  libraries  for  symbolic  and  numerical  computing.  Fur- 
thermore, all  MuPAD  functions  can  be  used  in  internet-based  applications.  The  main  comp  onents  of  this  model 
of  web-based  computing  are  a Java  client  applet  (e.g.  Terminal  or  Session  Applet)  and  the  MuPAD  Confuting 
Server,  with  JavaScript  serving  as  the  means  of  communication  between  input/output  conponents  of  the  web 
page  and  the  client  applet  (Sorgatz  2001). 

The  Terminal  Applet  allows  an  online  MuPAD  session  where  the  user  can  enter  CAS  commands  as 
usual.  Alternatively,  the  Session  Applet  is  invisible  to  the  user  and  can  be  considered  merely  as  a means  of 
communicating  with  the  Computing  Server.  Here,  HTML  forms  serve  as  input  and  output  components  whereas 
the  CAS  and  its  syntax  remain  hidden.  In  this  setting,  user  input  is  processed  and  translated  into  MuPAD  lan- 
guage via  JavaScript;  the  computations  are  carried  out  in  a MuPAD  session  at  the  Computing  Server.  The  results 
are  returned  to  the  client  and  transformed  again  in  order  to  obtain  presentable  output. 


* http://www.math-kit.de  - Support  of  the  German  Federal  Ministry  of  Education  and  Research,  project  math-kit  (08NM084) 
is  gratefully  acknowledged 
^ http://www.mupad.de 
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Two  Examples  of  Web-Based  Drills  with  Instant  Feedback 


It  is  crucial  for  successful  learning  to  give  the  students  the  resources  to  practise  certain  mathematical 
methods  and  to  test  their  performance.  In  this  context,  the  necessity  of  constructive  feedback  has  often  been 
emphasized  (see  Musch  1999).  These  ideas  were  taken  into  account  when  developing  exercises  in  math-kit.  In 
this  section,  we  present  two  exa  mples  of  web-based  drills  with  instant  feedback  using  the  Session  Applet. 


Differentiator 

The  Differentiator  is  a tool  which  allows  students  to  practise  differentiation.  It  offers  a variety  of  func- 
tionalities: random  selection  of  exercises  or  user  input  of  problems,  results  checking,  tips  for  solving  the  differ- 
entiation problem,  detailed  outline  of  the  computation,  computation  of  a derivative  by  the  CAS. 

An  important  advantage  of  the  Differentiator  is  that  it  accepts  many  different  forms  of  the  same  answer 
with  MuPAD  checking  for  mathematical  equivalence.  This  'intelligence'  is  an  important  argument  for  using  a 
CAS.  Furthermore,  the  list  of  functions  presented  in  the  exercises  can  easily  be  modified  by  the  teacher  to  ensure 
that  the  student  will  practise  at  an  appropriate  level.  The  'tutorial  functions'  are  also  of  great  use  for  the  student  as 
they  help  them  to  understand  the  algorithm. 


Gausstrainer 

The  Gaussian  algorithm  is  the  standard  method  for  solving  systems  of  linear  equations,  by  which  a 
given  matrix  is  transformed  to  the  reduced  row  echelon  form.  The  Gausstrainer  supports  the  student  when  prac- 
tising this  transformation:  the  student  has  to  apply  elementary  row  operations  to  a given  matrix  using  input  forms 
and  buttons.  A protocol  of  the  steps  and  the  current  status  of  the  matrix  are  displayed  and  finally,  of  course,  the 
student  can  check  their  result. 

One  merit  of  the  Gausstrainer  is  that  the  student  can  concentrate  on  the  algorithm  rather  than  the  arith- 
metic or  MuPAD  commands.  Moreover,  the  number  of  attempts  is  unrestricted,  and  as  only  the  final  result  is 
checked  and  not  the  process  to  the  result,  the  student  can  build  their  own  strategies  as  long  as  the  operations  are 
valid.  The  user  can  also  decide  on  the  size  of  the  matrix  and  thus  on  the  complexity  of  the  problem. 


Conclusion 

We  have  shown  that  the  integration  of  a CAS  like  MuPAD  offers  new  promising  opportunities  for  cre- 
ating web-based  learning  elements  with  mathematical  content.  Not  only  does  it  ease  the  development  of  web- 
based  drills  as  all  necessary  mathematical  functionalities  are  provided  by  the  CAS,  but  it  also  allows  for  con- 
structive feedback.  Furthermore,  the  student  can  do  drills  independently  of  time,  location,  browser,  and  platform 
and  without  a local  installation  of  a CAS  let  alone  any  knowledge  of  how  to  use  it. 
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Abstract 

Technology  is  playing  an  increasingly  important  role  in  the  delivery  of  adult  education.  As  early 
as  1984,  Bates  suggested  that  new  technologies  promised  "a  wider  range  of  teaching  functions  and  a higher 
quality  of  learning,  lower  costs,  greater  student  control,  more  interaction  and  feedback  for  students”  (p. 

223).  This  paradigm  shift  has  ramifications  in  all  areas  of  society  from  the  major  metropolitan  areas  to  the 
rural  communities.  “Technology  has  contributed  to,  if  not  caused,  the  shift  to  an  information  society,  which 
is  creating  dramatic  changes  in  the  workforce... Adult  education  both  reflects  and  responds  to  the  forces 
prevalent  in  the  sociocultural  context”(Merriam  & Caffarella,  1999,  p.23).  That  shift  and  the  associated 
complexity  is  evident  in  distance  learning  technology  as  it  is  providing  some  adults  with  a new  and  exciting 
avenue  to  learn.  It  is  also  broadening  the  gap  both  educationally  and  economically  between  those  that  have 
access  to  computer  technology  and  those  that  do  not.  This  study  looks  at  the  cross  disciplinary  aspects 
within  fields  of  instructional  technology  and  adult  education.  Conceptually,  what  does  current  research  say 
about  adult  learners  and  their  perception  to  computer  mediated  communication  and  distributed  learning? 

Is  online  learning  a means  by  which  systemic  problems  can  be  addressed?  Does  distance  learning  provide 
the  instructional  design,  curriculum  and  support  for  the  rural  adult  learner?  While  there  has  been  much 
research  on  college  students  and  other  populations  of  learners  in  metropolitan  areas,  there  is  a considerable 
gap  in  the  literature  about  the  experiences  of  adults  in  rural  America  and  their  perceptions  of  online 
learning. 

This  study  adds  to  the  knowledge  base  of  adult  education  and  instructional  technology  because  it 
provides  conceptual  research  about  the  “digital  divide”  and  how  that  chasm  is  being  addressed.  We  are 
currently  living  in  the  Age  of  Information  and  whether  we  embrace  the  diffusion  of  technology  or  not  it  is 
affecting  our  lives  directly  and  indirectly.  Information  is  power.  This  research  also  helps  bridge  the  gap 
between  our  knowledge  of  how  adults  learn  in  new  electronic  educational  setting  and  the  factors  that  play  a 
crucial  role  in  this  domain. 


Andy’s  Perceived  Logical  Flow 

1.  Background  of  online  learning 

2.  Statistics  of  online  learning 

3.  Technology  Community  centers  and  their  problems 

4.  Digital  divide 

5.  Online  learning  for  everyone?  Has  the  instructional  technology  advanced  to  the  point  that  adult 
basic  education  learners  can  be  benefited?  I think  we  sometimes  forget  that  Instructional 
Technology  is  about  the  Technology  of  Instruction,  not  about  media,  or  computers  or  delivery 
systems  per  se,  and  that  there  is 

6.  Technology  and  methodology:  Research  and  theory  that  covers  the  practical  application  of  the 
science  of  instruction  and  how  it  is  dianging  to  meet  the  learners  need. 

7.  Systemic  Problem  with  staffing  community  centers  and  finding  relevant  content  for  adult  learners 
addressed  through  distance  learning.  Has  the  time  has  arrived? 


o 
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Comments:  I am  pondering  an  analysis  of  the  system  dynamics  of  online  learning  for  low -middle  income 
rural  Adults  at  community  centers. 

A subset  of  those  dynamics  is  the  emotions,  perceptions,  and  factors  contributing  to  or  inhibiting  education 
via  online  learning.  Only  the  adult  learner  can  tell  us  if  online  learning  and  the  interactivity  from  the 
technology  is  effective  or  not. 

Regarding  the  theoretical  framework/perspective,  you  could  say  it  is  post-Critical  Theory.  We  are  living  in 
an  information  age  so  we’d  better  learn  to  adapt  to  the  changes  with  technolo©^. 
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Abstract;  In  this  paper,  we  describe  the  architecture  of  an  authoring  tool  for  creating  web- 
based  presentations  that  embed  user  interface  agents  as  primary  actors.  The  expressive 
functions  of  the  medium  (UI  agents)  are  not  yet  revealed  and  we  expect  that  an  architecture 
that  enables  the  authoring  process  and  empowers  people  with  no  advanced  technical  skills, 
will  unfold  new  types  of  mediation  experiences  that  are  not  still  taken  under  consideration. 
We  discuss  the  desired  authoring  conditions  and  describe  ACT’s  architecture  that  tries  to 
realize  them.  ACT  is  comprised  by  two  primitive  modules,  a)  the  Authoring  module  which 
includes  the  Kinaesthetic  Space  component,  the  Cognitive  Artifact  Library,  the  Stages  Model 
and  the  Scenario  Authoring  Line  Authoring  component,  b)  the  Scenario  Processing  Engine 
module  which  is  responsible  for  the  execution  of  script  produced  by  the  Authoring  Module. 


Introduction 

The  computer  is  a different,  possibly  unique  medium,  in  its  ability  to  store,  deliver,  and  help 
manipulate  a variety  of  symbol  systems.  The  computer  affords  the  ability  to  work  in  a number  of  languages, 
visually,  acoustically,  textually,  numerically.  Embodied  animated  agents  are  a new  kind  of  metaphor  in  user 
interfaces,  which  assemble  a rich  unexplored  expressive  interactive  space.  Reasons  for  using  user  interface 
agents  are  (Andre,  1998,  Paiva,  2001): 

- an  animated  and  personified  agent  can  support  the  activities  of  the  user  in  a more  active  and  friendly  way, 

- an  animated  character  can  more  easily  provide  an  emotional  dimension  to  the  interaction, 

- an  animated  character  can  take  advantage  of  its  body  movements  and  expressions  to  convey  an  explanation  in 
a more  effective  way 

- an  animated  character  may  help  user  learn  procedural  tasks  by  demonstrating  their  execution  etc. 

Moreover,  lifelike  animated  agents  offer  great  promise  for  learning  environments.  Because  of  the 
immediate  and  deep  affinity  that  students  seem  to  develop  for  these  agents,  the  potential  pedagogical  benefits 
they  provide,  may  perhaps  evai  be  exceeded  by  their  motivational  benefits  (Lester,  1999).  Agent’s  multimodal 
communication  channels  via  gestures,  facial  expressions  and  speech  synthesizers  raise  a potential  of  different 
roles  and  goals  it  can  achieve.  If  agents  also  can  “sense”  their  world  of  living  and  they  are  enhanced  with  action 
capabilities  , their  presence  functions  as  a better  metaphor  of  human-to-human  communication. 

However,  the  question  of  who  is  capable  to  design,  develop  and  integrate  user  interface  agents  in 
computerized  learning  experiences  is  not  yet  answered.  The  research  community  is  mainly  focused  in  other 
aspects  of  design  and  implementation  issues  such  as  automatic  dialogue  generation,  behavior  modeling 
languages,  task  model  based  interaction  etc.  while  the  involvement  with  the  authoring  process  of  agent’s 
behavior  is  restricted  in  the  determination  of  general  variables  that  affect  agent’s  automatic  behavior.  We 
believe  that  authoring  tools  that  make  feasible  the  creation  of  agent  presentation  acts  by  teachers  with  no 
advanced  technical  skills  must  be  developed  in  order  to  figure  out  agent’s  multiple  usage  perspectives  in 
everyday  learning  experiences.  In  this  paper,  we  will  discuss  ACT,  an  agent-based  presentation  tool  for  creating 
web  presentations  that  tries  to  foimalize  the  authoring  process  in  a way  that  facilitates  teachers’  goals  and 
ensures  the  consistent  behavior  of  the  interface  agent  and  its  environmental  setting. 

Pedagogical  Interface  agents 

In  an  abstract  view,  user  interface  agents  have  four  modules  that  integrate  the  metaphor  of  face-to-face 
conversation  in  the  human-computer  interface  (Cassel,  2000).  Several  authors  address  the  need  for  models  of 
personality  in  designing  embodied  conversational  agents  (Andre,  1999).  Characters  with  personality  make  the 
information  easier  to  remember  because  the  narration  is  more  compelling.  Furthermore,  embodied 


conversational  agents  that  present  consistent  personality  cues  are  perceived  as  more  useful  (Cassel,  2000). 
Secondly,  models  of  emotions  that  can  inform  conversational  behaviour,  expand  the  communicative  capabilities 
of  the  agent  (Ball,  2000).  With  a richer  expressive  repertoire,  the  agent  can  easily  advise,  encourage  and 
empathize  with  students.  But  how  humans  are  communicating  a particular  message?  Performative  acts  are  goal 
oriented  structures  of  speech  acts  enhanced  with  non  verbal  behaviour.  They  create  the  semantics  of  the 
communication  message  (Pogii,  1998).  A conversation  has  also  a syntactic  framework.  This  consists  of  the 
interactional  conversational  functions  that  regulate  the  flow  of  the  discussion.  Any  modality  may  convey 
several  communicative  goals. 

User  interface  agents  that  exploit  the  totality  of  these  characteristics  are  already  in  use  in  educational 
environments.  “Herman  the  Bug”,  interactively  provides  contextualized  advice  to  learners  as  they  graphically 
assemble  plants  from  a library  of  plant  structures  in  a botanical  anatomy  and  physiology  lab  (Lester,  1999). 
STEVE  can  teach  students  how  to  operate  and  maintain  the  gas  turbine  engines  aboard  naval  ships,  including 
both  individual  and  team  tasks  (Rickel  2000).  Students  can  interact  with  another  agent,  named  Cosmo,  as  they 
learn  about  network  routing  mechanisms  by  navigating  through  a series  of  subnets.  Helpful,  encouraging  and 
with  a bit  of  attitude,  Cosmo  explains  how  computers  are  connected,  how  routing  is  performed,  etc. 

In  all  previous  examples,  agents  can  interact  proactively  and  reactively  with  the  student  because  they 
maintain  a representation  of  the  task  model  and  incorporate  dialogue  interaction  models  and  predefined 
heuristic  algorithms  for  manipulating  the  different  possible  perceptual  and  conceptual  states  of  the  student.  This 
kind  of  interactive  circumstances  are  particularly  critical  for  constructivist  learning  environments  in  which 
learners  participate  in  active  problem  solving  (Lester,  1999).  The  expressive  power  of  a computerized 
presentation  system  should  not  be  misvalued.  We  should  not  directly  correlate  presentation  systems  with 
instructionism.  Although  active  participation  in  learning  environments  can  motivate  a comprehensive 
knowledge  construction  in  students,  “displaying  acts”  may  assist  them  infer  coherence  and  meaning  on  the 
basis  of  the  semantic  structures  of  the  different  scenes.  Learning  in  this  occasion  is  the  reconstruct  of  a coherent 
semantic  world  starting  from  fragments  (Mancini,  2000).  The  main  actor  can  be  an  interface  agent  which 
performs  and  takes  action  in  a stage  which  is  informed  of  agent  behavior  and  it  can  also  serve  automatically  the 
trainer’s  goals . 

Authoring  and  designing 

Powerful  commercial  tools  provide  an  impressive  palette  of  media  authoring  techniques  to  designers. 
But  why  do  designers  prefer  pencil  and  paper,  to  make  design  specifications,  when  they  create  the  design  of  a 
commercial  title?  Research  has  indicated  that  when  the  authoring  procedure  starts  with  the  immediate  use  of 
authoring  systems,  critical  design  issues  are  implicitly  specified  while  defining  minor  perspectives  of  the 
development  environment.  The  final  products  are  usually  unstructured  and  problematic.  Causes  are  usually  the 
underestimation  of  the  potential  of  the  design  space,  the  “bounded  rationality”  of  human  beings,  and  the 
ambiguous  activity  unit.  Innovative  authoring  tools  should  include: 

- Conceptual  Modelling : conceptual  design  is  a top-down  strategy  for  the  confinement  of  the  design  space. 
Most  of  hypermedia  design  methodologies  as  HDM,  OOHDM,  contain  in  their  analysis  steps,  a conceptual 
modelling  procedure  that  not  only  constraints  the  design  space  but  also  enhances  it  with  a semantic  structure. 

- Cognitive  artifacts  : research  in  the  cognitive  field  has  shown  that  thinking  is  heavily  dependent  upon  the 
experiences  with  physical  and  intellectual  objects.  Decreasing  the  semantic  distance  between  the  computational 
objects  and  existing  everyday  working  objects,  inevitably  increases  the  number  of  the  cognitive  schemas  in 
which  these  object  can  participate  and  simultaneously  decreases  the  required  cognitive  effort. 

- Activity  Unit  : The  authoring  activity  is  a form  of  doing  directed  to  an  object.  The  relationship  between  the 
subject  and  the  object  of  activity  is  mediated  by  a tool  (cognitive  artifact)  into  which  the  historical  development 
of  the  relationship  is  condensed.  The  cognitive  artifact  is  transformed  into  an  external  scaffolding  that  simplifies 
our  cognitive  tasks.  Gradually,  people  form  a tightly  coupled  system  with  their  environment.  The  determination 
of  the  activity  unit  specifies  the  efforts  for  the  cognitive  artifact  transformation  that  essentially  will  define  the 
degree  of  complication  of  the  authoring  process . 

Agent-based  authoring  tools  usually  have  a semi -autonomous  execution  engine  which  frames  general 
agent  characteristics  as  personality  and  emotive  behaviour  into  certain  categorical  options  (Andre,  1999)  and  an 
authoring  tool  for  adding  dynamic  utterances  (Ball,  2000)  and  behavioural  templates.  These  conditions  narrow 
the  interpolative  actions  of  authors  and  presuppose  discrete  and  global  understanding  approaches  for  the 
kinaesthetic  behaviour  spaces  of  agents.  Teachers  need  to  be  in  control  of  their  products  and  always  want  to 
develop  a clear  understanding  of  the  underlying  assumptions  in  autonomous  behaviours. 
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act’s  architecture  goals 


The  main  goals  of  our  architecture  are  the  following  : 

• To  detect  the  appropriate  cognitive  design  artifacts  and  activity  units  for  scenario  procedures  that  facilitate 
the  mediation  experience  of  user  interface  agents  authoring. 

• To  provide  services  for  creating  multiple  composite  stages  on  which  agents  perform. 

• To  create  an  architecture  that  ensures  coherent  and  consistent  behaviour  sequences  during  execution  time . 

• To  establish  an  open  architecture  that  can  encapsulate  a variety  of  agent  technologies 

We  design  ACT  in  such  a way  that  it  follows  a theatrical  metaphor.  To  prepare  a story  presentation,  authors 
have  to 

• create  the  “theatrical”  stages  they  want  to  include  in  their  story 

• create  and  import  into  the  stages  artifacts  which  change  their  functionality  and  appearance  during  the 
presentation  and  comprise  essentially  world  update  calls 

• author  reusable  agent  performatives 

• instruct  in  an  abstract  level,  the  agent  to  perform  specific  tasks  or  actions  - scenario 
The  overall  architecture  can  be  seen  in  figure  1 . It  consists  of  two  modules. 

a)  the  authoring  module,  which  is  responsible  for  providing  the  necessary  facilities  in  the  authoring  process. 


Figure  1 ; ACT  architecture 

Authoring  Module 

The  authoring  module  consists  of  five  components: 

• the  Kinaesthetic  Space  Editor,  in  which  anyone  can  develop  different  categories  of  abstract  performatives 
with  an  internal  structure  and  declaration, 

• the  Kinaesthetic  Space  Library,  which  includes  pre -built  agent  performatives, 

• the  Stages  Model  component,  which  serves  as  the  stage  constructor  tool, 

• the  Cognitive  Artifacts  Library,  which  includes  pre -built  template-based  propositional  presentation  items 

• and  the  Scenario  Authoring  Line  which  uses  the  previous  four  in  order  to  make  available  to  the  user  a rich 
repertoire  of  activity  building  blocks  during  the  presentation  authoring. 

The  Kinaesthetic  Space  Editor  - Library. 

This  component  feeds  the  Scenario  Authoring  Line  component  with  alternative  agent  performatives. 
We  use  the  notion  of  performative  in  order  to  communicate  the  composite  structure  of  the  agent  actions  (Pogii, 
1998). 

The  performatives  that  we  use  here  are  based  on  a model  of  communication  in  terms  of  goals,  beliefs  and 
emotions.  A communicative  act  is  the  minimal  unit  of  communication  and  it  can  be  performed  via  linguistic 
devices  or  via  gestural,  bodily,  facial  devices.  The  meaning  of  a communicative  act  includes  performative  and 
propositional  content.  Different  classes  of  performatives  can  be  distinguished.  We  offer  three  categories  of 
performatives  : 

• emotive  performatives,  that  communicate  emotions. 
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• informative  performatives ^ which  we  can  classify  further  as  requests  (order,  command,  implore,  propose, 
offer  and  advise),  as  questions  and  as  informative  acts  (inform,  warn,  announce).  The  system  provides  the 
means  to  the  author  to  create  dynamic  utterances  by  using  a library  of  global  variables  ($time,  $date, 
Susemame,  etc.),  or  select  external  sources  of  information  that  the  system  supports  (e.g.  services  of  a news 
agency)  as  sources  of  utterances.  This  gives  the  environment  a dynamic  notion  and 

• believability  enhancing  performatives  (Lester,  1999),  that  can  give  the  « illusion  of  life  » to  our  character. 

The  whole  set  of  performatives  constitutes  the  kinaesthetic  space.  In  the  implementation  level,  the 
system  stores  sequences  of  gestures  and  utterances  that  correspond  to  different  discrete  performatives.  The  first 
two  categories  of  performatives  also  have  their  own  «distance»  axes  that  specify  the  difference  of  intensity  for 
each  performative  in  the  related  category  and  the  prerequisite  familiarity  the  agent  must  have  with  the  user. 
These  axes  are  used  later  by  the  scenario  processing  engine  in  order  to  provide  emotionally  and  informatively 
consistent  sequences  of  performatives  in  the  flow  of  the  presentation  (to  avoid  non-human  behaviour).  Also  its 
subcategory  of  informative  performatives  has  its  own  propositional  declaration  form  which  is  determined  by  its 
literary  operation.  For  example,  the  performative  order  has  the  definition:  <order,  whom*,  what*>.The  authors 
have  in  their  disposal  a rich  Kinaesthetic  Space  Library  which  can  be  enriched  by  the  Kinesthetic  Space 
Authoring  Tool.  The  Kinaesthetic  Space  Library  can  be  shared  between  different  users  of  the  web  application  or 
someone  can  request  to  have  his  own  instance  of  it. 


The  Stage  Model  component 

In  order  to  facilitate  the  scene  development  for  the  agent  acts  and  give  the  notion  of  real 
environmental  settings,  we  introduce  the  stage  model  - figure  3.  The  stage  model  consists  of  different  stages. 
Stages  are  the  various  theatrical  scenes  in  which  the  actor  can  perform  its  pedagogical  acts.  Every  stage  has  two 
layers: 

A)  A background  layer  which  can  be  built  by  primitive  HTML  objects  like  textboxes  and  images.  These  objects 
are  not  considered  as  active  and  they  are  not  taken  into  account  in  Scenario  Processing  Engine  sub-components. 

B)  An  active  layer  which  can  also  have  state-based  objects  (SB).  SB  objects  are  constituted  initially  by  two 
states,  a hidden  and  an  empty  state.  The  author  can  insert  any  object  in  the  empty  state  and  change  its 
appearance  properties,  he  can  insert  a new  empty  state  or  clone  one  of  the  existing.  Examples  of  SB  objects  are 


Figure  2;  Kinaesthetic  Space  Authoring 


Figure  3, 4TStag^ditor,  Scenario  Authoring  Line 
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an  URL  guide  SB  object  in  which  all  states  contain  an  HTML  “iframe”  which  displays  different  URLs  in 
different  states  and  another  example  is  a moving  SB  object  in  which  all  states  contain  the  same  object  with 
different  position  values.  The  author  can  insert  as  many  objects  he  wishes  in  each  state.  Every  primitive  object 
can  have  event-based  states  which  are  selected  when  the  corresponding  events  are  fired. 

The  Cognitive  Artifact  library 

The  Cognitive  Artifacts  library  should  provide  authors  with  authoring  elements  that  can  be  easily 
integrated  into  their  cognitive  schemes.  E.g.  while  commercial  presentation  programs  provide  generic 
computational  objects  (e.g.  bulleted  list),  we  differentiate  our  approach  by  providing  conceptual  objects  like 
summary,  comparison,  multiple  perspectives  object.  We  combine  the  function  of  editing  the  conceptual  objects 
with  templates  of  related  agent  performatives.  So,  while  in  the  first  place  the  author  should  edit  the  list,  insert 
the  items,  select  the  agent’s  performatives  and  specify  the  synchronization  conditions,  in  our  approach  the 
author  edits  a template  that  includes  the  structure  of  the  required  synergies.  For  each  conceptual  object  there  are 
many  behavioral  templates  with  characterization  attributes  similar  to  the  ones  in  the  Kinesthetic  space. 

The  Scenario  Authoring  Line  component 

The  Scenario  Authoring  Line  component  functions  as  the  central  unit  of  the  authoring  tool.  It  embeds  a 
conceptual  organizer  which  makes  feasible  the  hierarchical  organization  of  the  scenario  blocks.  The  conceptual 
organizer  functions  also  as  a navigation  tool  because  of  the  large  number  of  scenario  building  blocks  that  are 
usually  produced  in  such  environments.  The  basic  scenario  blocks  can  be 

a)  an  agent  performative,  (informative  or  emotive) 

b)  a cognitive  artifact  (summarization,  comparison  etc.), 

c)  a world  model  updating  (change  of  display  state  for  complex  objects). 

d)  a draw  attention  function  (only  items  in  active  layout  can  be  specified) 

e)  raw  actions  (very  close  to  the  kinaesthetic  space  authoring) 

f)  a branching  unit  . 

As  in  most  authoring  tools,  one  of  the  most  important  factors  for  the  design  complexity  manipulation  is 
the  provision  of  several  views  (Harada,  1996).  Multiple  views  are  provided  with  a reorganization  algorithm 
which  shows  the  building  blocks  by  type  (emotive  performatives,  informative  performatives,  etc.)  or  by 
conceptual  section.  Anytime,  there  exists  a focus  scene  for  which  the  properties  are  shown.  Since  the  meaning 
of  an  action  is  embedded  in  the  semantic  structure  of  the  multiple  sequential  building  blocks,  for  each  scene 
being  edited,  we  show  the  list  of  adjacent  utterances  and  gestures. 

Scenario  Processing  Engine 

The  products  of  the  authoring  process  are  stored  in  the  server  datastore.  In  order  to  be  executed,  a new 
page  is  spawned  containing  the  Scenario  Processing  Engine  with  the  scenario  data  structure  to  be  executed. 
The  main  architecture  of  the  engine  can  be  seen  in  figure  5.  Our  Scenario  Processing  Engine  is  predictive- 
reactive.  Before  the  actual  playback  commences,  it  passes  the  scenario  once,  and  computes  statistical 
information  about  the  flow  of  the  building  blocks.  These  statistics,  along  with  more  general  variables  that  are 
calculated  at  execution  time,  are  stored  in  the  Statistics  Datastore. 

The  Actions  Sequence  & Branching  Management  Module  is  responsible  for  user  input  and  branch 
processing.  Generally,  the  user  can  provide  feedback  in  a branching  situation,  can  select  the  conceptual  section 
that  wishes  to  attend,  can  change  the  speed  of  the  execution  engine  and  exit  the  presentation.  More  important 
role  for  this  module  is  to  analyze  each  building  block  and  activate  the  respective  Block  Management  Module 
(Attention,  Performative,  Stage).  (1)  The  Block  Management  Units  query  the  History  Tracing  Unit  and  the 
Statistics  Datastore  (2)  in  order  to  scan  the  previous  engine’s  choices  and  apply  heuristic  rules  for  selecting  (3) 
the  appropriate  instantiation  of  the  building  blocks  from  the  corresponding  libraries.  They  operate  in  a way  that 
ensures  their  self-consistency.  Each  one  maintains  its  own  history  of  actions  and  has  the  ability  to  communicate 
with  other  subsystems  in  order  to  exchange  required  data.  As  an  example,  while  executing  a scenario  data 
structure,  tfie  Performative  Management  Module  accepts  two  requests  for  suggestion  performatives  that  are 
close  in  time.  In  order  not  to  repeat  the  same  performative  twice,  it  searches  the  informative  «distance»  axis  in 
the  Kinaesthetic  Space  Library  for  an  adjacent  suggestion  performative.  The  Template  Instantiation  Module 
undertakes  the  presentation  of  the  selected  template  (4)  and  updates  the  History  Tracing  Unit  (5). 
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Abstract 

Educational  intermediaries  are  information  systems  that  support  the  exchange  of  learning 
resources  among  dispersed  users.  The  selection  of  the  appropriate  learning  resources  that 
cover  specific  educational  needs  requires  a concise  interaction  between  the  user  and  system. 

This  paper  describes  a data  mining  process  for  the  discovery  of  ontologies  from  learning 
resources  repositories.  Ontologies  express  the  associations  between  the  learning  resources 
metadata  and  provide  a controlled  vocabulary  of  concepts.  Ontologies  and  the  derived 
vocabularies  could  be  used  for  the  development  of  taxonomies  of  learning  resources  and  they 
contribute  to  the  sense  disambiguation  in  seeking  interesting  and  appropriate  knowledge. 

Introduction 

The  huge  volume  of  information  existing  in  the  World  Wide  Web,  the  complexity  of  its  structure, 
as  well  as  the  keyword-based  character  of  retrieval,  make  the  discovery  of  the  required 
information  resources  an  unfriendly,  time  consuming  and  inefficient  procedure.  A promising 
approach  to  tackle  the  existing  difficulties  and  word  sense  ambiguities  lies  in  the  development  of 
the  Semantic  Web  (Berners-Lee  et  al.,  2001),  i.e.  the  existence  of  machine-processable  and 
interoperable  semantics  in  Web-based  services  and  applications.  The  explicit  representation  of 
semantics  is  obtained  by  the  construction  and  usage  of  ontologies^  which  could  be  considered  as 
“metadata  schemas  providing  a controlled  vocabulary  of  concepts”  (Maedche  and  Staab,  2001). 

Educational  intermediaries,  also  referred  to  educational  e-markets  (Hamalainen  et  al.,  1996)  or 
learning  media  (Guth  et  al.,  2001),  host  a diversity  of  learning  resoxirces  (LR)  and  provide 
educational  services  to  their  users,  who  are  usually  Universities  or  organizations,  which  perform 
educational  programs  for  their  personnel.  Examples  of  educational  intermediaries  are 
ARIADNE’S  Knowledge  Pool,  EDUTELLA,  Gateway  to  Educational  Material  (GEM),  MERLOT 
and  UNIVERSAL.  Educational  intermediaries  store  metadata  descriptions  on  each  learning 
resource  providing  information  on  its  characteristics  (title,  subject,  type,  duration,  language, 
required  equipment  etc.).  In  order  to  ensure  the  concise  communications  with  their  users  these 
systems  should  provide  a meaningful  catalog  of  the  offered  LR.  This  requirement  leads  to  an 
automated  ontology  development  for  the  generation  of  flexible  and  dynamic  taxonomies  of  LR 
and  the  provision  of  a vocabulary  of  concepts  capable  to  express  explicitly  and  formally  (i.e. 
machine  understandably)  the  LR  relations. 

This  paper  proposes  a methodology  for  the  extraction  of  ontologies  fi-om  LR  repositories.  In 
particular  we  use  a data  mining  approach  in  order  to  discover  the  relations  of  the  LR  metadata 
files.  Similar  LR  are  grouped  into  clusters  and  the  cluster  processing  provides  a controlled 
vocabulary,  which  contributes  to:  (i)  the  efficient  and  meaningful  response  to  queries  and  (ii)  the 
dynamic  creation  of  LR  taxonomies,  without  the  manual  usage  of  static  classification  systems 
(e.g.  Dewey,  UDC).  Our  work  is  motivated  by  UNIVERSAL  (http://www.ist-universal.org/).  a 
European  Union  funded  project  (Information  Society  Technologies  Programme),  which  aims  to 
implement  a learning  resources  brokerage  platform  for  the  European  Higher  Education 
Institutions  (HEI).  The  platform  hosts  a variety  of  LR,  covering  many  scientific  domains  and 
different  educational  needs  (Vrabic  and  Simon,  2001).  The  following  section  presents  the 
ontologies  concept,  while  section  3 illustrates  the  steps  we  follow  for  the  ontology  discovery. 
Section  4 presents  the  experimental  setting  on  the  UNIVERSAL  repository  and  the 
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corresponding  results.  Finally,  section  5 summarizes  the  presented  work,  introducing  future 
plans. 

Ontologies 

Metadata  are  definitional  data  related  to  other  data  managed  within  an  application  or 
environment.  For  example,  metadata  would  document  data  about  data  elements  or  attributes 
(name,  size,  data  type,  etc.),  data  about  records  or  data  structures  (length,  fields,  column,  etc) 
and  data  about  data  (where  it  is  located,  how  it  is  associated,  ownership  etc).  Metadata  may 
include  descriptive  information  about  the  context,  quality  and  condition,  or  characteristics  of  the 
data  and  can  be  organized  in  ontologies. 

In  Artificial  Intelligence  ontology  is  defined  as  “an  explicit  formal  specification  of  how  to 
represent  the  objects,  concepts  and  other  entities  that  are  assumed  to  exist  in  some  area  of 
interest  and  the  relationships  that  hold  among  them”  (dictionary.com).  Noy  and  McGuiness 
(2001)  define  an  ontology  as  "a  formal  explicit  description  of  concepts  in  a domain  of  discourse, 
properties  of  each  concept  describing  various  features  and  attributes  of  the  concept  (slots),  and 
restrictions  on  slots”.  The  notion  of  ontology  is  becoming  very  useful  in  fields  such  as  intelligent 
information  integration,  cooperative  information  systems,  information  retrieval,  electronic 
commerce,  and  knowledge  management. 

The  vision  of  the  Semantic  Web,  aiming  at  the  conduct  of  automated  reasoning,  requires 
computers  to  have  access  to  structured  collections  of  information  and  sets  of  inference  rules  that 
they  can  use.  As  a first  step  for  the  development  of  this  technology,  called  knowledge 
representation,  XML  provides  a serialized  syntax  of  tree  structures.  At  the  same  time  a 
mechanism  for  encoding  and  transferring  metadata,  specified  by  the  Resource  Description 
Framework  (RDF),  is  being  developed  by  the  W3C  as  a foundation  for  processing  semantic 
information.  An  improved  language  has  appeared  recently  called  OIL  (Fensel  et  al.,  2001) 
describing  ontologies  and  offering  ontology  editors,  annotation  tools  and  tools  for  reasoning  with 
ontologies.  DAML  (DARPA  Agent  Markup  Language)  (Hendler,  2001)  is  also  being  developed 
with  the  aim  to  represent  semantic  relations  that  wiU  be  compatible  with  current  and  future 
technologies.  While  SHOE  (Simple  HTML  Ontology  Extension)  (Heflin  and  Hendler,  2000)  allows 
web  page  authors  to  annotate  their  web  documents  with  machine-readable  knowledge.  Another 
interesting  development  in  this  area  is  a generic  ontology  for  modeling  ill-structured  knowledge 
domains  within  educational  systems  in  F-logic  notation  (Papaterpos  et  al.,  2001). 

Ontology  engineering  deals  with  domain-specific  knowledge,  and  tries  to  develop  technology  for 
accumulating  knowledge  within  reasonable  size  of  stratified  domains  utilizing  ontologies 
(Mizoguchi,  1998).  The  product  of  such  a study  is  a catalog  of  the  types  of  things  that  are 
assvimed  to  exist  (Sowa,  2000).  Ontology  discovery  (Maedche  and  Staab,  2001)  extends  ontology- 
engineering environments  by  using  semiautomatic  ontology-construction  tools.  The  framework 
encompasses  ontology  import,  extraction,  pruning,  refinement,  and  evaluation. 

Requirements  on  the  ontology  design  are  manifold.  From  the  user’s  perspective  the  number  of 
categories  should  grow  with  the  number  of  resources  indexed,  otherwise  users  have  to  browse 
through  either  too  many  empty  or  overcrowded  categories.  In  the  first  case,  browsing  becomes 
ineffective  because  too  many  clicks  have  to  be  performed  to  enter  the  leaves  of  the  category  tree. 
In  the  later  case,  browsing  becomes  ineffective,  as  a long  list  requires  lots  of  scrolling,  usually 
favoring  the  first  listed  resources.  From  the  catalog  administrator’s  perspective,  the  effort  for 
categorizing  resources  should  be  the  least  possible.  A growing  category  system  is  not  desirable  as 
the  reclassification  of  resources  is  an  overcostly  process.  That  is  why  an  adaptive  category  system 
in  the  field  of  e-learning  has  not  been  experienced  yet.  In  this  paper  an  approach  for  building 
adaptive  ontologies  is  presented. 
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Methodology 


The  problem  of  ontology  discovery  could  be  considered  as  a data  mining  task,  since  the  fields  of 
the  metadata  indicating  the  title,  subject(s)  and  description  of  the  LR  are  associated.  The  stages 
inducing  fi*om  the  XML/RDF  metadata  syntax  to  the  desired  ontology  are  the  same  as  those  of 
any  other  data  mining  task:  data  collection  and  pre-processing,  pattern  discovery  and  knowledge 
post-processing. 

Data  Collection  and  pre-processing 

During  this  stage,  LR  metadata  are  gathered  and  the  title,  subject  and  description  fields  of  the 
metadata  XML/RDF  files  are  separated.  The  pre-processing  aims  to  enable  them  to  be  used  as 
input  to  the  next  stage  of  pattern  discovery.  The  main  objective  is  the  selection  of  appropriate 
keywords  from  the  metadata  files  that  allow  the  discovery  of  similarities  among  the  LR.  For  this 
reason  function  words  such  as  articles,  prepositions  and  conjunctions  are  dropped.  Then, 
language  engineering  tools  are  used,  such  as  the  Wordnet  dictionary  (Fellbaum,  1998)  for 
extracting  word  roots  (lemmatization)  and  the  Brill  tagger  algorithm  (Brill,  1994)  for  attaching 
tags  to  words  according  to  the  part  of  speech  they  represent.  The  outcome  of  this  processing 
transforms  the  remaining  words  into  a set  of  unique  nouns  in  singular  number,  which  represent 
the  keywords  set.  Finally  we  prepare  the  dataset,  i.e.  a matrix  for  which  each  column  (feature) 
corresponds  to  an  LR  and  each  row  (objects)  corresponds  to  a keyword.  The  matrix  consists  of 
binary  numbers  indicating  the  existence  or  not  of  a keyword  in  an  LR. 

Pattern  discovery 

Given  the  training  data  in  the  appropriate  form,  interesting  patterns  are  extracted  with  the  use 
of  machine  learning  techniques,  such  as  clustering,  classification,  association  rule  discovery  etc. 
The  choice  of  method  depends  largely  on  the  type  of  training  data  that  are  available.  The  main 
distinction  in  machine  learning  research  is  between  supervised  and  unsupervised  learning 
methods.  Supervised  learning  requires  the  training  data  to  be  pre-classified.  This  usually  means 
that  each  training  object  is  associated  with  a unique  label,  signifying  the  class  in  which  it 
belongs.  The  important  feature  of  this  approach  is  that  the  class  descriptions  are  built  relative  to 
the  pre-classification  of  the  objects  in  the  training  set.  In  contrast,  unsupervised  learning 
methods  do  not  require  pre-classification  of  the  training  objects.  These  methods  form  clusters  of 
objects,  which  share  common  characteristics.  When  the  cohesion  of  a cluster  is  high,  i.e.  the  items 
in  it  are  very  similar,  it  is  labeled  as  a class. 

The  metadata  file  structure  of  the  UNIVERSAL  project  provides  a taxonomy  field,  which  could  be 
used  for  the  data  pre-classification.  However  as  long  as  the  field  is  unused  we  have  opted  for  the 
use  of  unsupervised  learning.  In  order  to  provide  a conceptual  hierarchy  (taxonomy)  of  the  LR, 
we  could  use  the  conceptual  clustering  approach,  which  is  particularly  suitable  for  summarizing 
and  explaining  data  (Fisher,  1987).  Summarization  is  achieved  through  the  discovery  of 
appropriate  clusters  of  the  data  and  explanation  involves  concept  characterization,  i.e., 
determining  a useful  concept  description  for  each  class.  However,  most  conceptual  clustering 
algorithms,  produce  disjoint  groups.  In  our  case,  this  means  that  the  LR  groups  (concepts)  cannot 
be  overlapped,  claim  which  is  restrictive  in  educational  practice,  where  a LR  could  contribute  to 
several  courses. 

Due  to  the  mentioned  drawback,  we  have  selected  the  Cluster  Mining  approach  (Perkowitz  and 
Etzioni,  1998;  Paliouras  et  al.,  2000),  which  discovers  similar  LR  forming  a graph  and  looking  for 
all  cliques  in  it.  The  algorithm  starts  by  constructing  a weighted  graph  G(V,E).  The  set  of  vertices 
V corresponds  to  the  LR.  An  edge  eij  connecting  nodes  vi  and  vj  exists  if  a keyword  is  common  in 
LRi  and  LRj.  The  eij  weight  is  equal  to  the  number  of  the  common  keywords  in  these  two  LR.  The 
edge  weights  are  normalized  by  their  division  with  the  maximum  weight  in  the  graph.  The 
connectivity  of  the  resulting  graph  is  usually  very  high.  For  this  reason  we  make  use  of  a 
connectivity  threshold,  aiming  the  reduction  of  the  number  of  edges  in  the  graph.  The 


connectivity  threshold  represents  the  minimum  weight  allowed  for  the  edge  existence.  After  the 
edge  reduction  the  method  accepts  all  the  existing  cliques  as  clusters.  Despite  the  large 
complexity  of  the  clique-finding  problem,  the  implemented  algorithm  (Bron  and  Kerbosch,  1973) 
is  very  fast. 

Pattern  post-processing  and  evaluation 

In  order  to  examine  the  produced  clusters  descriptive  ness,  we  calculate  the  following  two 
properties  by  varying  the  connectivity  threshold: 

Coverage:  the  proportion  of  LR  participating  in  the  clustering,  since  due  to  the  connectivity 
threshold  not  all  the  LR  are  members  of  the  generated  clusters. 

Overlap:  the  extent  of  overlap  between  the  clusters.  This  is  measured  as  the  ratio  of  the  number 
of  the  common  LR  and  the  number  of  all  LR  in  the  resulting  clustering. 

Moreover  in  this  last  stage,  we  pay  substantial  attention  to  the  extraction  of  the  keywords  that 
characterize  the  derived  clusters.  These  representative  keywords  construct  a prototypical  model 
for  each  cluster  and  provide  a desired  vocabulary.  The  selection  of  the  descriptive  keywords  is 
done  with  the  aid  of  a simple  measure,  which  is  based  on  the  idea  that  a keyword  is 
representative  of  a cluster  if  its  frequency  within  the  cluster  is  significantly  higher  than  its 
frequency  in  the  dataset  (Paliouras  et  al.,  2000).  Given  a keyword  with  frequency  / in  the  entire 
dataset,  and  frequency  fi  in  a cluster  i the  difference  of  the  squares  of  the  two  frequencies  defines 
the  required  measure: 

FI=  fi2-  f2 

FI  stands  for  Frequency  Increase.  Clearly,  when  FI  is  negative  there  is  a decrease  in  frequency 
and  the  corresponding  keyword  is  not  candidate  to  the  vocabulary.  A keyword  is  representative  of 
a cluster,  if  FI  > a,  where  a is  a threshold  of  the  frequency  increase. 

Experimental  results 

In  Universal  LR  metadata^  is  described  by  using  RDF  (Brantner  et  al.,  2001),  which  is  serialized 
in  XML.  The  Universal  RDF  binding  is  based  on  the  RDF  binding  provided  by  the  IMS 
(http://www.imsproiect.org/rdf/).  which  combines  various  metadata  standardization  initiatives 
such  as  Dublin  Core,  IEEE  LOM,  and  vCard. 

For  our  experiments  we  used  69  LR  descriptions  stored  in  the  UNIVERSAL  repository  till 
September  2001.  The  LR  offered  by  twenty  European  Higher  Education  Institutes  covering  a 
variety  of  scientific  domains.  The  result  of  the  procedure  of  word  separation  from  the  fields 
indicating  the  title,  subject  and  description  of  the  XML/RDF  metadata  files  were  about  1,400 
words.  The  utilization  of  the  mentioned  language  engineering  tools  returned  678  nouns  in 
singular  number.  Thus  the  derived  dataset  was  a matrix  with  69  columns  and  678  rows. 

The  cluster  mining  algorithm  was  applied  to  the  dataset  and  constructed  sets  of  cliques  for 
various  values  of  the  connectivity  thresholds.  Depending  on  the  value  of  the  connectivity 
threshold  the  coverage  of  the  clusters  and  the  overlap  varied.  Figure  1 presents  the  results  along 
those  two  dimensions.  As  expected,  the  overlap  for  small  threshold  values  is  large  due  to  the 
large  number  of  very  large  cliques.  A similar  behavior  follows  the  coverage.  Around  the  threshold 
value  0.3  about  half  of  the  LR  appear  in  the  cliques  (coverage  equals  to  0.41),  while  there  is  little 
overlap  between  the  cliques  (overlap  equals  to  0.25).  This  observation  suggests  the  selection  of 
this  threshold  value  for  the  formation  of  the  desired  representative  vocabulary.  At  this 
connectivity  threshold  value  14  clusters  are  generated.  One  of  them  includes  6 LR,  three  include 
3 LR  and  the  other  ten  clusters  include  2 LR. 


The  Universal  metadata  model  is  available  at: 

http://universal.infonova.at/UNlVERSAL/servlet/Universal7pagel  D=aboutWebRDFmain 
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The  application  of  the  frequency  increase  measure  for  each  cluster  had  as  result  a matrix  with  14 
columns  and  678  rows.  Each  cell  kept  the  FI  value  of  each  keyword  for  each  cluster.  These  values 
ranged  from  —2  up  to  3,  while  the  value  for  most  keywords  in  all  clusters  was  zero.  Table  1 
presents  the  derived  vocabulary  per  cluster  for  a FI  level  greater  than  or  equal  to  2.  Overlapping 
clusters  share  common  keywords  and  thus  are  grouped  together.  In  the  parentheses  the  number 
of  LR  in  each  group  of  clusters  is  indicated. 


0 0,30  0,35  0,40  0,50  0,70  0,90 

connectivity  threshold 
Fig.  1.  The  coverage  and  overlap  curves 

Except  for  the  words  shown  in  the  table  the  method  has  selected  keywords,  which  could  be 
characterized  as  general  purpose  words  connecting  the  main  keywords.  These  words  are:  activity, 
aspect,  aim,  area,  background,  concept,  core,  course,  detail,  discussion,  introduction,  knowledge, 
layer,  lecture,  limit,  opportunity,  people,  possibility,  problem,  purpose,  reference,  responsibility. 


result,  right,  share,  skill,  subject,  syndicate,  today,  topic,  vision,  word,  work. 

Table  1.  The  derived  vocabulary  per  cluster,  connectivity  threshold  = 0.3,  FI  > 2 

Cluster  1 
(6  LR) 

Science 

Cluster  9-10 
(4  LR) 

commerce,  internet,  protocol,  overview, 
resource,  system,  unit,  use 

Cluster  3-5 
(SLR) 

blast,  coke,  datum,  development,  energy, 
furnace,  information,  iron,  material, 
reduction,  technique,  technology,  user 

Cluster  12 
(2  LR) 

lehrveranstaltung,  objekt,  ziele 

Cluster  6,  11 
(4LR) 

database,  entity,  relationship,  model, 
system 

Cluster  13 
(2  LR) 

application,  automaton,  content,  context, 
datum,  development,  demonstration, 
description,  display,  framework,  html, 
logic,  machine,  metadata,  ontology, 
protocol,  resource,  schema,  state,  system, 
technology,  tool,  use,  world,  web,  page, 
xml,  rdf 

Cluster  7 
(2LR) 

architecture,  control,  design,  engineer, 
model,  implementation,  software,  system, 
time,  treatment,  use 

Cluster  14 
(2  LR) 

business,  administration,  case,  study, 
company,  decision,  environment  levi, 
Strauss,  sourcing,  supplier 

Cluster  8 
(2  LR) 

design,  graphics 

From  the  results  of  Table  1 we  can  conclude  into  a three-level  taxonomy  starting  from  a general 
level  for  the  whole  UNIVERSAL  repository.  Four  main  categories  have  been  formulated 
consisting  of  Science  (cluster  1),  Engineering  (clusters  3-5),  Business  Administration  (cluster  14) 
and  Computer  Science  (clusters  6-10  and  11).  Furthermore  Computer  Science  category  could  be 
decomposed  in  three  subcategories:  Databases  & Software  Engineering  (clusters  6,  7,  8 and  11), 
E-commerce  (clusters  9,10)  and  Web  Technologies  (cluster  13).  Cluster  12  collects  LR  described  in 
the  German  language  and  is  therefore  not  integrated  in  the  taxonomy.  The  data  pre-processing 
phase  failed  for  that  as  the  language  engineering  tools  can  only  accept  input  from  the  English 
language. 

Conclusions  and  Outlook 

In  this  paper  we  described  a methodology  of  ontology  engineering  for  learning  resources 
repositories,  based  on  the  data  mining  approach.  The  main  conclusions  are  that  automated 
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discovery  of  adaptive  ontologies  is  essential  for  the  operation  of  educational  intermediaries  and 
constitutes  a powerful  tool  for  the  improvement  of  their  services. 

Critical  issues  for  extending  this  research  comprise  the  selection,  testing  and  evaluation  of 
appropriate  algorithms  for  the  construction  of  precise  and  meaningful  ontologies.  Specifically  we 
intend  to  experiment  using  statistical  clustering  methods  allowing  the  clusters  overlapping.  An 
important  issue  that  we  intent  to  explore  is  how  the  RDF  annotations  of  the  UNIVERSAL 
metadata  descriptions  could  be  used  for  the  creation  of  improved  ontological  descriptions  (Delteil, 
et  al.,  2001). 
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Abstract:  At  the  beginning  of  the  complex  2 1st  century,  education  is  no  longer  regarded  as  a 
means  of  teaching  or  certifying.  ...  Two  vital  areas  are  necessary  in  order  to  ensure  the 
improvement  and  equity  of  education:  first,  to  know  what  students  actually  learn,  and  second  how 
they  learn  in  the  present  context.  We  believe  that  the  concept  of  Human  Education  Interactive 
Network  (HEINET)  is  an  instrument  that  can  help  answering  these  questions  and  improving 
learning  conditions. 


Learning  versus  Constructivist  Approach 

The  complexity  of  educational  environments  is  one  of  the  main  factors  to  take  into  consideration  in  the 
teaching/leaming  process  nowadays.  The  stronger  the  development  is,  the  more  complex  it  becomes  to  learn  to  look 
beyond  the  immediate  to  the  structures  which  prevail  long  enough  to  be  “photographed”,  analyzed  and  modeled  by 
the  thing  we  conventionally  call  science.  In  this  environment,  learning  and  teaching  and  teaching  and  learning  are 
absolutely  independent  processes,  which  occur  at  moments  very  near  in  time. 

Never  as  today  was  mankind  so  much  culture  product  and  producer  in  a society,  which  interacts  at  an 
increasingly  interplanetary  level,  and  in  which  the  concept  of  a system  projects  the  level  of  data  and  information 
production  in  an  exponential  way.  Never  as  today  was  it  so  necessary  for  mankind  to  focus  its  limited  interaction 
resources  with  the  data  (world),  in  order  to  appreciate  the  underlying  structures  of  such  great  diversity.  Especially 
today,  facing  this  multi-stimulating  environment,  it  is  advis  able  to  make  use  of  some  of  our  sensitive  faculties  for 
cognitive  acquisition,  which  up  until  now  were  only  regarded  as  mere  means  of  balancing  and  interacting  with  the 
environment.  Nowadays  it  is  crucial  to  find  mechanisms  that  enable  the  increase  of  the  cognitive  performance,  by 
using  multi-modal  ways  of  sensitive  stimulation,  which  in  turn  can  activate  this  process  of  change.  This  issue  leads 
us  to  two  fundamental  questions:  one  of  ontological  nature  (in  view  of  knowledge)  and  another  one  of 
methodological  nature.  The  first  concerns  understanding  to  which  extent  and  where  the  limits  to  the  knowledge 
horizon  should  be  placed  in  the  traditionally  established  relationship  between  teacher  and  learner.  The  second  one 
concerns  understanding  the  frame,  freedom  and  flexibility  that  should  be  attributed  to  the  references,  to  the 
conceptual  frame  and  the  environmental  conditions  in  the  present  state  of  the  Society  of  Information. 

It  seems  to  us  that  the  constructivist  approach  to  knowledge  is  an  adequate  response  to  the  first  question,  for  it 
establishes  the  balance  between  the  components  of  instruction  and  construction  in  the  teaching/leaming  process. 
The  second  question,  of  a more  methodological  nature,  can  be  dealt  with  within  a perspective  of  flexible  orientation 
of  teaching/learhing  towards  solving  real  problems,  adequately  framed  by  the  different  contexts  in  which  they 
occur. 

Within  the  constructivist  approach,  the  teaching/leaming  processes  have  to  consider  measures  to  promote  the 
development  of  learners’  performance/cooperation  competences.  Therefore  it  will  be  necessary  that  learners  have 
the  possibility  to  apply  and  exercise  new  knowledge,  thus  developing  their  new  skills  and  differences  of 
thinking/acting.  For  this  purpose,  the  teacher  should  choose  to  use  different  approaches  to  the  lesson  plan: 
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• Creating  groups  and  assigning  tasks,  hereby  promoting  the  cooperative  side  of  learning,  through 
discussion  and  confrontation  of  ideas; 

• Supporting  contents  through  the  application  and  determination  of  real  cases; 

• Developing  simulations,  which  encourage  the  active  application  of  new  knowledge  and  skills; 

Supporting  this  kind  of  approaches,  it  is  up  to  the  teacher  to  decide  for  the  most  adequate  learning 
methodology  to  use.  This  decision  depends  essentially  of  the  contents  and  of  the  starting  conditions  of  the  learners. 
Some  of  the  methodologies  we  consider  most  interesting  are:  cooperative  learning,  learning  through  “anchorage”  and 
construction/simulation  sets. 


Figure  1:  Teaching/Leaming  Methodology 


Improving  Classroom  Teaching 

At  the  beginning  of  the  complex  21st  century,  education  is  no  longer  regarded  as  a means  of  teaching  or 
certifying.  The  bounds  of  education  lie  on  the  attitude  of  encouraging  the  acquisition  of  multidimensional 
knowledge,  as  well  as  on  the  development  of  corresponding  skills.  This  results  in  a set  of  knowledge,  attitudes  and 
behaviors  capable  of  providing  a responsible  and  enterprising  citizenship,  founded  on  innovation,  renovation  and 
the  leveling  by  the  highest  pattern  of  common  life.  Two  vital  areas  are  necessary  in  order  to  ensure  the 
improvement  and  equity  of  education:  first,  to  know  what  students  actually  learn,  and  second  how  they  learn  in 
the  present  context.  We  believe  that  the  concept  of  Human  Education  Interactive  Network  (HEINET)  is  an 
instrument  that  can  help  answering  these  questions  and  improving  learning  conditions.  Learning  environments, 
traditionally  called  classrooms,  are  currently  facing  processes  of  change,  which  are  the  essential  outcomes  of  two 
great  tendencies: 

• The  velocity  of  change  introduced  by  the  Information  Society  and  by  all  technical  means  that  operate 
within  it; 

• The  need  to  find  edification/installation  solutions,  which  in  the  short  term  do  not  acquire  characteristics  of 
obsolescence,  resulting  from  the  exponential  growing  of  maintenance  costs; 

The  development  of  HEINET  at  the  School  of  Health  Sciences,  in  the  Viseu  Centre  of  the  Portuguese  Catholic 
University,  enabled  us  to  fulfill  these  vectors,  as  well  as  to  increase  the  global  performance  of  the  system, 
concerning  the  following  aspects: 

• It  sends  on  solid  and  information  about  the  teaching  and  learning  processes  within  the  new  scientific - 
pedagogical  unit; 

• The  conditions  at  the  learning  space  provide  crucial  information  for  the  envelopment  of  the  new 
pedagogical  methodology  used  at  the  unit  at  stake  - Flexible  Learning  - Problem  Oriented  Learning; 

• Evaluation  of  the  pedagogical  methodology/leaming  process  and  factors  to  be  focused  on. 

From  these  three  vectors  emerges  the  need  for  the  use  of  HEINET  as  a first  step  towards  making  pedagogical 
processes  and  learning  contents  more  uniform.  However  the  improvement  of  learning  does  not  happen  only  by 
adopting  pedagogical  standards.  The  pedagogical  team  (teaching  staff,  learners  and  other  learning  actors)  should  as 
far  as  possible  analyze  the  processes  themselves,  in  order  to  attain  the  optimized  use  of  the  different  learning 
spaces.  The  inherent  ideas  to  the  creation  and  implementation  on  HEINET  reveal  a concern  for  the  continuous 
improvement  of  the  teaching/leaming  system  in  its  different  dimensions:  the  system,  the  process,  the  products  and 
finally  the  users.  HEINET,  as  a means  for  optimization,  allows  us  to  focus  on  teaching  in  the  real  learning  by 
students,  which  conveys  a new  vision/mission  to  the  teaching/leaming  process. 
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features  for  knowledge  generating  activities.  Now,  the  new  types’  smart  hypertext  is 
appearing  to  overcome  current  hypertext's  limitations.  They  seem  to  be  promising  since 
they  start  developing  recognizing  the  current  Internet  users’  needs.  The  study's  purpose 
is  to  investigate  the  new  hypertext’s  meaningfiilness  on  knowledge  activities  and  to 
propose  hypertext's  future  direction.  And  it  calls  researchers'  attentions  by  asking  its 
interrelationship  with  cognitive  development. 


The  Problems 

The  hypertext  is  well  known  as  a tool  to  represent  rich  semantic  relationships.  A learner  builds  up 
her  memory  structure  by  adding  a new  object  to  existing  memory  structure.  So  some  factors  such  as  proper 
visual  association,  meaningful  relationships  or  appropriate  category  are  very  significant  to  enhance 
learning.  For  this  reason,  hypertext  is  very  suitable  to  support  knowledge  activities. 

The  current  hypertext  is  built  up  only  by  links  created  by  a web  author.  It  is  incomplete  in  terms  of 
user-centered  information  environment  with  the  following  reasons.  Very  often  a web  page's  information 
can  be  interpreted  as  very  different  anchors  from  information  author's  original  intention.  It  could  mean  that 
it  does  not  provide  the  right  response  or  feedback  according  to  users’  inquiry.  Immediate  response  is  one  of 
very  important  factors  in  knowledge  activities  because  otherwise,  a user  might  lose  her  curiosity  or  even 
forget  soon  what  she  would  look  for.  Thus,  with  some  kinds  of  aids,  if  a user  is  able  to  construct  her  own 
hypertext -links  for  herself,  presumably,  she  is  able  to  follow  her  own  curiosity  and  to  facilitate  knowledge 
activities.  It  could  also  encourage  a user’s  flexible  and  creative  thinking  process.  On  the  basis  of  such  a 
point,  1 propose  that  the  current  hypertext  should  be  more  developed  to  be  more  user-centered,  context  - 
oriented  and  learning-focused.  On  the  basis  of  this  perspective,  I searched  the  Internet  and  found  out  three 
new  types’  hypertext  applications,  Atomica  (www.atomica.com),  Flyswat  (www.flyswat.com)  and 
Microcosm  fwww.microcosm.coml  reflecting  my  perspective. 


The  Example 

Among  them,  Atomica,  a free  shareware  is  an  exemplary  one  to  demonstrate  the  above  perspective. 
Its  key  point  is  the  simple  human  computer  interaction,  which  is  that  one  Just  clicks  on  the  word  you  want 
defined  or  explained.  When  a user  clicks  a word  on  where  it  is  available,  whether  you're  using,  for 
example,  an  e-mail  client,  a word  processor  or  a spreadsheet  as  well  as  Web,  Atomica  automatically 
searches  its  relevant  information  as  well  as  definition  and  then  opens  a small  window  to  show  the  list. 

There  is  an  example  for  understanding  its  features.  Suppose  that  a user  was  reading  an  article  on  the 
Web,  or  an  e-mail  and  she  came  across  this  sentence:  "The  abstemious  people  from  Woonsocket  invested 
aggressively  in  companies  like  Tecumseh  Products,  hoping  to  become  as  rich  as  their  idol,  John  Kluge." 
And  suppose  she  had  no  idea  what  "abstemious"  meant,  where  Woonsocket  was,  what  Tecumseh  Products 
did,  or  who  Mr.  Kluge  was.  To  find  them  out  quickly  using  the  Internet,  she  needs  to  look  up  each  term 
individually,  which  requires  her  to  leave  the  document  that  she  was  reading  and  laboriously  type  in  each 
term  for  multiple  searches,  probably  at  multiple  sites.  Instead,  she  could  use  Atomica  that  fetches  such 
answers  from  the  Internet,  without  leaving  the  text  she  was  reading.  Also,  Atomica  combines  the  ability  to 
search  multiple  information  sources  simultaneously  with  a natural  intelligence  that  helps  to  ensure  targeted 
results.  Users  don't  have  to  memorize  Boolean  logic  or  even  construct  an  English  language  query.  For  the 
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kinds  of  information  Atomica  is  designed  to  parse-word  definitions,  language  translations  and  company 
profiles -it  works  well. 

Finally,  one  of  the  most  distinguishable  elements  of  Atomica  is  trying  and  guessing  the  entire 
phrase  or  context  of  a word  that  a user  clicks  on  although  it  is  still  very  limited.  If,  for  example,  a user 
clicks  on  either  word  in  the  phrase  ’’Supreme  Court,”  the  program  pops  up  with  an  answer  about  the  highest 
court  in  the  U.S.,  not  a definition  of  either  word  on  its  own.  And  it  pays  attention  to  capitalization.  It  knows 
that  the  word  ’’polish”  is  something  a user  does  to  furniture,  but  Polish  is  a language  and  culture  (Wall 
Street  Journal,  2000). 

However,  the  service  is  still  a work  in  progress.  First,  it  doesn’t  work  with  text  inside  an  image. 
Second,  it  is  not  able  to  search  multimedia -based  information  such  as  mp3  files  or  quicktime  movie  files. 
Third,  while  it  draws  on  a host  of  reference  works  and  Web  databases,  it  doesn’t  use  premiere  sources,  such 
as  the  Encyclopedia  Britannica  and  major  newspapers.  It  tends  to  rely  on  almanacs,  single-volume 
encyclopedias  and  little-known  reference  sites,  though  it  does  use  major  dictionaries  (Wall  Street  Journal, 
2000). 


The  Suggestions  and  Discussion 

I conducted  the  small  pilot  study  with  five  graduate  students  in  Teachers  College,  Columbia 
University  in  order  to  explore  for  the  new  hypertext’s  future  direction.  Based  on  Atomica ’s  functions  and 
the  results  of  the  pilot  study,  I propose  the  following  directions. 

First,  hypertext  is  merging  with  a search  engine.  With  it,  hypertext  is  becoming  more  effective  and 
is  able  to  reduce  users’  information  seeking  efforts  considerably. 

Second,  the  future  hypertext’s  success  empowered  by  a search  engine  depends  on,  I believe,  how 
much  it  understands  users'  intention.  Namely,  they  should  be  smart  enough  to  understand  not  only  what  a 
user  is  looking  for  but  also  the  whole  context  where  a user  is.  The  findings  of  a pilot  study  support  it.  Its 
subjects  merely  used  very  simple  search  formulations.  What  they  wanted  to  obtain  was  simply  the  quick 
and  effective  search  results  and  not-so-complicated  human- computer  interaction.  Hence,  the  concept  of 
’agent  software’  is  inevitable  one  so  that  it  can  play  a role  like  a secretary  or  a task  manager  reducing  human 
users’  cognitive  overload  to  a certain  degree  that  they  feel  comfortable.  At  the  same  time,  such  agents  will 
restrain  information  overload  by  filtering  and  analyzing  information  on  web  pages. 

Third,  when  the  smart  agent,  hypertext  provides  advice  or  a hint  to  users  by  being  modeling  and 
doing  scaffolding  exactly  like  a teacher  in  a classroom,  I assume  that  it  could  reduce  users’  unnecessary 
search  behaviors  and  assist  developing  their’  meta-cognitive  process.  Atomica’s  'did  you  mean?'  description 
or  its  additional  key  words'  list  can  be  exemplary. 

Finally,  while  more  successful  information  seekers  seem  to  be  able  to  initiate  their  search 
behaviors  on  their  own,  less  successful  ones  need  a structure  to  help  them  search  and  do  knowledge 
activities.  Hence,  it  will  be  worth  to  consider  if  hypertext  can  provide  different  levels’  gateway  or  interface 
design  according  to  information  seeker's  search  capability  consisted  of  novice,  intermediate  and  expert. 

Such  a new  hypertext  seems  to  be  worthy  for  further  consideration  to  influence  users’  information 
search  behavior.  Although  I have  explored  mostly  the  positive  impacts  of  it  but  there  should  be  negative 
impacts  such  as  users'  dependence  on  its  smart  functions.  Thus,  it  should  be  pondered  critically  to  what 
degree  and  in  what  ways  it  would  support  human  beings’  Internet  search.  That  is,  we  should  investigate  its 
multi-dimensional  aspects  carefully.  I hope  that  this  study  could  draw  educational  researchers'  attention  to 
explore  its  roles  and  potential  to  build  up  the  enhanced  learning  environments. 
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Abstract:  This  paper  will  introduce  the  concept  of  a reusable  assessment  framework 
(RAF).  An  RAF  contains  a library  of  linked  assessment  design  objects  that  express  a) 
specific  set  of  proficiencies  (i.e.  the  knowledge,  skills,  and  abilities  of  students  for  a 
given  content  or  skill  area),  b)  the  types  of  evidence  that  can  be  used  to  estimate  those 
proficiencies,  and  c)  features  of  tasks  that  will  aid  in  the  design  of  activities  (e.g.  features 
that  need  to  be  present  in  order  for  students  to  produce  the  evidence,  features  that  affect 
task  difficulty,  etc.).  While  RAFs  can  speed  the  design  of  many  kinds  of  assessments,  in 
this  paper  we  focus  on  their  use  to  aid  instructional  designers  in  embedding  assessments 
within  computer-based  learning  environments.  The  RAF  concept  is  based  upon  the 
evidence-centered  design  methodology  described  in  Mislevy,  Steinberg,  Almond, 
Haertel,  & Penuel  (2001). 


The  Problem 

There  has  been  substantial  discussion  and  research  concerning  the  advantages  of  embedding 
assessments  within  instructional  activities  and  learning  environments  (e.g.  Shavelson,  1992;  Hopper,  1998; 
Treagust,  Jacobowitz,  Gallagher,  and  Parker,  2000;  Van  Lehn,  1988).  Van  Lehn  (1988),  for  example, 
describes  several  ways  in  which  embedded  assessments  can  help  people  learn  including  a)  determining 
appropriate  tasks  based  upon  estimates  of  students’  skill  level,  b)  determining  when  to  provide  feedback, 
and  c)  adapting  explanations  to  students’  level  of  understanding.  However,  designing  learning 
environments  that  embed  assessments  is  a labor-intensive  activity.  There  are  several  reasons  for  this.  First, 
creating  learning  environments  is  already  hard.  Given  the  time  and  budget  constraints  that  instructional 
designers  face,  it  is  hard  to  justify  the  added  time  effort,  time,  and  expense  of  incorporating  reliable,  valid 
assessments  as  part  of  a learning  environment.  Secondly,  designing  embedded  assessments  requires  an 
interdisciplinary  approach  involving  experts  in  such  diverse  areas  as  educational  measurement,  psychology, 
instructional  design,  statistics,  and  the  content  area.  The  opportunity  to  assemble  a team  that  has  the 
requisite  knowledge  of  both  assessment  design  and  instructional  design  is  rare.  Consequently,  while 
students  could  benefit  from  interacting  with  learning  environments  that  contain  well-designed  embedded 
assessments,  not  enough  are  produced. 


The  primary  problems  of  the  current  paper-pencil  tests 

Billions  of  dollars  are  spent  each  other  on  education,  yet  there  is  wide  spread  dissatisfaction  with 
our  educational  system  among  educators,  parents,  policymakers  and  the  business  community.  Efforts  to 
reform  and  restructure  schools  have  focused  attention  on  the  role  of  assessment  in  school  improvement. 
After  years  of  increases  in  the  quantity  of  formalized  testing  and  the  consequences  of  inappropriate  test 
scores,  many  educators  have  begun  to  doubt  the  measures  used  to  monitor  student  performance  and 
evaluate  programs.  They  claim  that  traditional  measures  fail  to  assess  significant  learning  outcomes  and 
thereby  undermine  curriculum,  instruction  and  policy  decisions.  The  timed  nature  of  the  tests  and  their 
format  of  one  right  answer  has  led  teachers  to  give  students  practice  in  responding  to  artificially  short  texts 
and  selecting  the  best  answer  rather  than  inventing  their  own  questions  or  answers.  When  teachers  teach  to 
traditional  tests  by  providing  daily  skill  instruction  in  formats  that  closely  resemble  tests,  their  instructional 


practices  are  both  ineffective  and  potentially  detrimental  due  to  their  reliance  on  outmoded  theories  of 
learning  and  instruction. 

Contrary  to  past  views  of  learning,  cognitive  psychology  suggests  that  learning  is  not  linear,  but 
that  it  proceeds  in  many  directions  at  once  and  at  an  uneven  pace.  Conceptual  learning  is  not  something  to 
be  delayed  until  a particular  age  or  until  all  the  basic  facts  have  been  mastered.  People  of  all  ages  and 
ability  levels  constantly  use  and  refine  concepts.  Furthermore,  there  is  tremendous  variety  in  the  modes  and 
speed  with  which  people  acquire  knowledge,  in  the  attention  and  memory  capabilities  they  can  apply  to 
knowledge  acquisition  and  performance. 

According  to  Dietel  et  al  (1991),  Current  evidence  about  the  nature  of  learning  makes  it  apparent 
that  instruction  which  strongly  emphasizes  structured  drill  and  practice  on  discrete,  factual  knowledge  does 
students  a major  disservice.  Learning  isolated  facts  and  skills  is  more  difficult  without  meaningful  ways  to 
organize  the  information  and  make  it  easy  to  remember.  Also,  applying  those  skills  later  to  solve  real-world 
problems  becomes  a separate  and  more  difficult  task.  Because  some  students  have  had  such  trouble 
mastering  decontextualized  "basics,"  they  are  rarely  given  the  opportunity  to  use  and  develop  higher-order 
thinking  skills. 

Recent  studies  of  the  integration  of  learning  and  motivation  also  have  highlighted  the  importance 
of  affective  and  metacognitive  skills  in  learning.  For  example,  poor  thinkers  and  problem  solvers  differ 
from  good  ones  not  so  much  in  the  particular  skills  they  possess  as  in  their  failure  to  use  them  in  certain 
tasks.  Acquisition  of  knowledge  skills  is  not  sufficient  to  make  one  into  a competent  thinker  or  problem 
solver.  People  also  need  to  acquire  the  disposition  to  use  the  skills  and  strategies,  as  well  as  the  knowledge 
of  when  and  how  to  apply  them  (Dietel  et  al,  1991).  These  are  appropriate  targets  of  assessment.  Thus, 
different  types'  alternative  assessments  are  being  considered  now. 


Alternative  Assessment  on  Computer-Based  Learning  Environments 

Alternative  assessment  based  on  cognitive  modeling  may  be  a more  valid  indicator  of  students' 
knowledge  and  abilities  because  they  require  students  to  actively  demonstrate  what  they  know.  The 
following  were  described  by  Leighton  et  al  (1999)  as  the  characteristics  of  alternative  assessment:  (a) 
assessment  should  be  integrated  with  instruction;  (b)  it  should  be  transparent,  that  is,  it  should  help  students 
learn  to  monitor,  self-evaluate,  and  reflect  on  their  own  performance;  (c)  assessment  tasks  should  be 
authentic,  extended  over  time,  meaningful  and  challenging;  (d)  students  should  have  access  to  tools, 
resources  and  coaching  support  during  assessment  of  their  performance  and  learning;  (e)  assessment  should 
be  diagnostic,  providing  information  about  students'  knowledge,  cognitive  processes,  misconceptions  and 
errors  during  performance  of  problem-solving  tasks;  (f)  students'  knowledge  and  their  ability  to  apply 
(transfer)  their  knowledge  to  new  or  novel  problems  should  be  assessed;  (g)  ability  to  collaborate 
effectively  with  others  in  solving  problems  should  be  assessed.  These  characteristics  of  alternative 
assessment  address  limitations  of  traditional  testing  approaches  and  promote  more  authentic  assessments  of 
knowledge,  learning,  cognitive  processing,  and  problems -solving  skill  in  complex  natural  domains  and 
contexts  of  task-oriented  action.  And  some  studies  show  that  the  integration  of  assessment  and  Intelligent 
Tutoring  System  (ITS)  could  play  a role  as  the  alternative  assessment  framework  as  a problem-solver  to 
cure  the  current  malfunction  ((Dietel  et  al,  1991).  However,  because  designing  and  confirming  validity  of  a 
cognitive  assessment  is  on  very  early  developing  stage,  needless  to  say,  embedding  such  one  on  ITS  that  is 
already  hard  to  develop  should  be  double -efforts.  Therefore,  as  I mentioned  earlier,  many  researchers  and 
practitioners  for  ITS  tend  to  assess  learners  temporarily  only  for  their  specific  purpose.  Consequently,  such 
assessment  methods  being  used  so  far  are  not  easy  to  transfer  to  another  ITS.  RAF  would  help  solving  such 
barriers. 


Deflning  of  Evidence-Centered  Design  (ECD)  Methodology 

The  RAF  concept  is  based  upon  the  evidence-centered  design  methodology  described  in  Mislevy, 
Steinberg,  Almond,  Haertel,  & Penuel  (2001).  To  grasp  RAF's  essential  mechanism  better,  it  will  be 
necessary  to  consider  what  ECD  is. 
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Its  essence  is  focused  on  reasoning  on  the  basis  of  evidence.  When  we  reason  from  masses  of  data 
of  different  kinds,  humans  interpret  complex  data  through  some  underlying  "story’*  - a narrative,  or  an 
organizing  theory.  We  attempt  to  weave  a sensible  and  defensible  story  around  the  specifics.  And  it  is  the 
story  that  builds  around  what  we  believe  to  be  the  generative  principles  and  patterns  in  the  domain.  Based 
on  their  story  and  principles,  we  can  use  them  as  building  blocks  when  we  tackle  some  novel  problems  that 
pose  new  questions  or  presents  new  kinds  of  evidence.  In  this  spirit,  RAF  applies  an  approach  structuring 
arguments  from  evidence  to  inference  manage  uncertainty  (Mislevy  et  al,  1999).  RAF  approach  is  an 
evidence-centered  perspective  on  assessment  design,  and  object  definitions  and  data  structures  for 
assessment  elements  and  their  interrelationships  on  the  b^sis  of  Student  model,  task  model  and 
evidence  model. 

Student  Model: 

The  student  mo  del  indicates,  "What  complex  of  knowledge,  skills  or  other  attributes  should  be 
assessed?"  The  student  model  variables  are  the  terms  in  which  we  want  to  talk  about  students  - the  level  at 
which  we  build  our  story,  to  determine  evaluations,  make  decisions  or  plan  instruction  - but  we  do  not  get 
to  see  the  values  directly  (Mislevy  et  al,  1999).  The  student  model  variables,  all  proficiencies  that  learners 
would  acquire  would  be  defined  through  cognitive  task  analysis.  And  then  we  just  see  what  the  students  say 
or  do  and  must  use  it  as  evidence  about  the  student  model  variables.  However,  we  encounter  the  evidentiary 
problem  of  constructing  this  inference  from  limited  evidence.  Hence,  The  inference  drawn  from  what  the 
learners  say  or  do  may  be  also  on  the  basis  of  probability  rules,  which  may  be  able  to  rule  out  its 
uncertainty. 

Evidence  Model: 

The  evidence  model  means,  "What  behaviors  or  performances  should  reveal  those  constructs  and 
what  is  the  connection?"  The  evidence  model  lays  out  our  argument  about  why  and  how  the  observations  in 
a given  task  situation  constitute  evidence  about  student  model  variables.  There  are  two  parts  of  evidence 
models,  the  evaluative  submodel  and  the  statistical  submodel.  The  evaluative  submodel  extracts  the  salient 
features  of  the  "work  product",  that  is,  whatever  the  student  says,  does,  or  creates  in  the  task  situation.  The 
statistical  submodel  updates  the  student  model  in  accordance  with  the  values  of  these  features,  effectively 
synthesizing  the  evidentiary  value  of  performances  over  tasks.  Defining  statistical  submodel  is  beyond  this 
study’s  purpose  so  I will  not  mention  it  any  further. 

Here,  I will  illustrate  more  clearly  what  work  product  means  since  it  is  a very  important  element 
mapping  between  the  concept  and  data  scoring.  According  to  Mislevy  et  al  (1999),  it  is  a unique  human 
production,  as  simple  as  a mark  on  an  answer  sheet,  as  complex  as  the  presentation  of  disconfirming 
evidence  or  a series  of  work  product  represent  "observable  variables".  It  represents  what  is  important  in  a 
performance  on  the  basis  of  our  belief.  And  these  mapping  between  learners  work  product  and  doing 
assessment  could  be  very  simple  or  could  be  very  complicated  to  require  several  expert’s  evaluation  of 
multiple  aspects.  In  the  same  context,  they  can  be  automatic,  or  they  can  require  human  judgment. 

Task  Model: 

Task  model  means,  "What  tasks  or  situations  should  elicit  those  behaviors?"  A task  model 
provides  a framework  for  constructing  and  describing  the  situations  in  which  learners  act  and  includes 
specifications  for  the  environment  in  which  the  learner  will  say  or  do,  or  produce  something.  The  examples 
could  be  characteristics  of  stimulus  material,  instructions,  help,  tools,  and  affordances.  It  also  includes 
specifications  for  the  work  product,  the  form  in  which  what  the  learner  says,  does  or  produces  will  be 
captured.  A particular  task  is  produced  by  assigning  specific  values  to  tasks  model  variables  and  providing 
materials  that  suit  the  specifications  there  given,  a task  thus  describes  particular  circumstances  meant  to 
provide  learners  an  opportunity  to  act  in  ways  that  produce  information  about  what  he  or  she  knows  or  can 
do  more  generally.  Accordingly,  the  task  controls  the  level  of  complexity  or  difficulty  of  circumstances  that 
learners  encounter.  The  task  itself  does  not  describe  how  we  should  evaluate  what  we  see.  This  is  specified 
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in  the  evidence  model.  The  advanced  ITS  is  a supreme  place  providing  the  means  for  producing  theses 
situations,  and  capturing,  evaluating  and  communicating  what  learners  do  there. 


A Solution 

One  solution  to  this  problem  is  to  develop  assessment  frameworks  that  compile  the  needed 
assessment  design  expertise  into  reusable  chunks  that  instructional  designers  can  instantiate  as  they  design 
learning  enviroiiiVients.  In  the  paper,  we  will  describe  the  strucuire  of  an  RAF  in  terms  of  an  assessincnl 
delivery  model  proposed  by  Almond,  Steinberg,  and  Mislevy,  2001.  The  figure  below  (adapted  firom 
Almond,  Steinberg,  and  Mislevy,  2001)  provides  a conceptual  representation  of  the  components  of  learning 
and  assessment  delivery  systems. 


Figure  1 : The  Four-process  model  of  Almond,  Steinberg,  and  Mislevy  (2001) 

The  activity  selection  process  is  responsible  for  determining  on  which  activity  a student  will  next 
work.  This  process  could  be  as  simple  as  allowing  a student  to  choose  the  next  activity  or  as  complicated 
as  an  automated  decision  based  upon  the  systems  assessment  of  a students  strengths  and  weaknesses.  The 
presentation  process  is  responsible  for  providing  all  direct  interactions  with  a student  including  presenting 
all  instructional  and  activity  related  material,  recording  all  student  interactions,  and  capturing  student  work. 
The  evidence  identification  process  scores  student  work,  extracting  evidence  from  the  student  work 
products  as  a set  of  “observables”  - variables  that  contain  evaluations  of  the  student  performance  along  a 
set  of  relevant  dimensions.  The  evidence  accumulation  process  uses  the  observables  to  update  a student 
model  that  consists  of  a set  of  estimates  of  students’  knowledge,  skills,  and  abilities. 

As  displayed  in  figure  2,  in  an  RAF  several  parts  of  this  framework  are  defined  by  assessment 
designers  before  instructional  designers  begin  their  work. 


Figure  2:  The  Four-process  model  with  the  RAF  components  highlighted 

In  the  figure,  these  predefined  components  are  represented  in  bold  text.  Specifically,  the  evidence 
accumulation  process  is  defined  and  implemented,  and  the  observables  and  the  work  products  are  defined. 

In  addition,  the  RAF  includes  task  models  as  defined  by  Mislevy,  Steinberg,  and  Almond  (1999).  The  task 
models  consist  of  collections  of  task  features  that  include  a)  features  that  need  to  be  present  so  that  students 
will  create  appropriate  work  products  (i.e.  those  that  contain  the  needed  evidence,  b)  features  that  affect 
task  difficulty,  and  c)  features  that  can  be  varied  to  produce  task  variants.  The  task  models  in  the  RAF 
define  requirements  for  tasks  that  instructional  designers  may  create.  However,  as  will  be  described  in  the 
paper,  considerable  flexibility  remains  for  instructional  designers  to  design  many  new  creative  tasks  as  long 
as  they  incorporate  selected  subsets  of  features  from  the  task  models. 


An  Example:  an  RAF  for  Science  Inquiry 

In  this  section,  we  will  describe  an  example  RAF  for  the  area  of  science  inquiry.  We  will  first 
present  the  general  structure  of  the  RAF  including  proficiencies,  evidence,  and  task  features.  We  will  then 
provide  an  example  of  how  an  instructional  designer  might  use  the  predefined  RAF  to  design  a computer- 
based  learning  environment  with  embedded  assessments  for  middle  school  children.  The  domain  of  the 
example  will  be  science  inquiry  focusing  on  the  physics  of  hot  air  balloons.  The  domain  of  the  knowledge 
is  provided  with  the  form  of  computer-based  simulation  program.  Whenever  learners  encounter  a challenge 
task,  they  solve  it;  otherwise  they  are  not  able  to  go  to  the  next  step.  During  such  an  itinerary,  their  learning 
progress  is  captured  by  systems,  their  learning  outcomes  are  collected  and  measured  through  RAF  method. 


Educational  Importance 

An  RAF  is  a way  to  represent  assessment  design  knowledge  in  reusable  structures.  Just  as  the 
object-oriented  programming  movement  in  computer  science  allowed  a new,  deeper  level  of  code 
reusability  for  programmers,  RAFs  offer  instructional  designers  access  to  reusable  assessment  design 
knowledge,  and  hence,  make  it  easier  to  for  them  incorporate  embedded  assessments  in  their  designs, 
without  overly  constraining  their  creativity. 

Implementation/Research  pe  rspective 
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A RAF  will  be  used  to  as  a tool  to  support  the  research  project  that  examines  the  correlation 
between  assessment  and  a learner’s  information  search  behavior  on  intelligent  tutoring  system  (ITS). 
Through  being  used  in  such  a project,  a RAF’s  validity  and  usefulness  will  be  examined  and  it  can  point  out 
a RAF’s  limitation  as  well  as  additional  usefulness  in  an  actual  performance, 
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Abstract:  This  paper  examines  the  decision-making  associated  with  the  design  and 
production  of  Hurricane  Strike!,  a CD-ROM  and  Web-based  instructional  multimedia 
module  that  teaches  about  hurricane  science  and  safety  for  fth-9th  graders.  The  module 
design  is  based  on  anchored  instruction  (Cognition  and  Technology  Group  at  Vanderbilt, 
1992)  that  engages  the  learner  in  a variety  of  activities  related  to  the  science  of  hurricane 
development,  movement,  and  structure,  and  to  home  and  personal  safety  practices  for 
when  hurricanes  threaten.  Our  process  of  defining  the  problem,  iteratively  creating 
solutions,  and  frequently  evaluating  those  solutions  illustrates  the  general  pattern  of 
Analysis,  Synthesis,  and  Evaluation  in  the  instmctional  design  process,  but  also 
demonstrates  how  the  process  is  often  unsystematic  and  how  the  phases  typically  overlap. 


New  instructional  development  projects  are  always  approached  with  a mixture  of  excitement  and 
uneasiness.  It  was  no  exception  in  our  case  when  the  Federal  Emergency  Management  Agency  (FEMA) 
asked  the  COMET®  Program  (Cooperative  Program  for  Operational  Meteorology,  Education  & Training, 
http://www.comet.ucar.edu)  to  create  an  instructional  multimedia  program  on  hurricane  saf^  for  an 
audience  of  children.  The  excitement  came  from  expanding  our  products  beyond  the  professional 
meteorologist  community  we  usually  serve,  and  from  being  able  to  contribute  an  educational  piece  that 
could  save  lives.  We  were  uneasy  because  of  several  unanswered  questions:  Could  we  develop  something 
suitable  for  kids,  making  it  both  appealing  and  highly  educational?  What  technologies  would  allow  us  to 
make  compelling  media  that  could  be  used  in  a wide  range  of  schools?  How  would  we  share  the  focus  on 
science  education  (our  specialty)  and  safety  education?  What  can  we  do  well  for  a very  modest  budget? 
And  finally,  what  processes  will  be  most  effective  and  efficient  for  completing  the  project? 

This  paper  examines  the  decision-making  associated  with  the  design  and  production  of  Hurricane 
Strike!,  a CD-ROM  and  Web-based  instructional  multimedia  module  (http://meted.ucar.edu/hurrican/strike) 
that  teaches  about  hurricane  science  and  preparedness  for  graders  who  live  in  regions  vulnerable  to 

hurricanes.  The  module  design,  modeled  on  anchored  instruction  (Cognition  and  Technology  Group  at 
Vanderbilt,  1992),  engages  the  learner  in  a variety  of  activities  related  to  the  science  of  hurricane 
development,  movement,  and  structure,  and  to  home  and  personal  safety  practices  for  when  hurricanes 
threaten.  Our  process  of  defining  the  problem,  iteratively  creating  solutions,  and  frequently  evaluating 
those  solutions  will  illustrate  the  general  pattern  of  Analysis,  Synthesis,  and  Evaluation  in  the  instructional 
design  process,  but  also  how  the  process  is  often  unsystematic  and  the  phases  typically  overlap. 
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Overview 


Background 

The  COMET  Program  is  a not-for-profit  organization  that  provides  classroom  and  distance 
education  and  training  for  weather  forecasters.  COMET  also  produces  materials  to  improve  the  use  of 
weather  products  by  users  of  weather  forecasts,  such  as  emergency  managers.  The  program  has  developed 
over  180  hours  of  computer-based  training  in  the  past  11  years,  including  many  hours  of  Web-based 
training.  Hurricane  Strike!  is  the  first  project  we  have  attempted  for  a K- 12  audience. 

This  project  required  contributions  from  a people  with  a wide  variety  of  skills,  including  part-time 
efforts  from  a lead  instructional  designer,  two  additional,  very  part-time  consulting  instructional  designers 
who  also  handled  project  coordination  (the  lead  designer  lived  out-of-state),  graphic  artists,  multimedia 
authors,  a digital  audio  technician,  a multimedia  quality  assurance  professional,  two  volunteer  middle 
school  teachers  to  review  the  design  and  content,  volunteer  subject  matter  experts  from  FEMA,  the  Red 
Cross,  and  the  National  Oceanic  and  Atmospheric  Administration  (NOAA),  contributed  recording  and 
voice  talent  from  The  Weather  Channel®,  and  our  own  kids  as  willing  subjects  for  formative  evaluation 
sessions.  Additional  important  resources  for  this  project  included  support  from  management  to  attempt  a 
new  type  of  project,  and  existing  content,  data,  and  graphical  images  we  had  already  developed  for  training 
forecasters  and  emergency  managers  on  similar  topics 


Analysis  (Deflning  the  Parameters  of  the  Problem) 

Sources  of  analysis  data  primarily  included  discussions  with  FEMA  professionals,  email  and 
conference  calls  with  teachers  in  school  systems  affected  by  hurricanes,  and  reviews  of  existing  content  on 
hurricanes,  hurricane  hazards,  and  public  safety  for  hurricane  preparedness.  While  most  of  the  analysis  was 
done  at  the  front  end,  new  analysis  information  continued  to  be  received  after  production  began.  This 
information  was  not  ignored,  and  was  considered  while  our  work  progressed. 

The  need  for  this  product  was  based  on  the  fact  that  increasing  populations  are  living  in  hurricane- 
prone  areas  in  the  U.S,  both  on  the  coast  and  inland.  Their  vulnerability  requires  that  the  public  become 
more  hurricane-aware.  We  believe  that  such  awareness  can  be  promoted  effectively  by  educating  students 
about  why/when/where  and  how  hurricanes  form,  the  difficulties  in  predicting  their  strength  and 
movement,  the  hazards  associated  with  them,  and  the  protective  measures  people  can  take.  Students  in 
middle  school  and  above  are  an  ideal  audience  for  this  effort  because  they  can  influence  their  parents  to 
take  appropriate  preparedness  measures  and  because  they  will  eventually  become  decision-making  adults. 

As  suggested  above,  the  project  had  two,  sometimes  complementary,  sometimes  competing,  goals. 
It  should  contribute  to  science  education  and  support  the  National  Science  Standards.  At  the  same  time,  it 
should  achieve  the  very  important  goal  of  teaching  hurricane  safety  and  preparedness,  a goal  that  could 
have  a direct  impact  on  saving  lives  and  reducing  property  losses  during  hurricanes.  Yet  these  distinct  goals 
should  not  be  allowed  to  make  the  final  product  seem  un -integrated. 

These  goals  helped  us  derive  many  objectives  regarding  knowledge  of  safety  measures  and 
hurricane  science.  However,  we  quickly  worked  to  establish  priorities,  knowing  that  the  effort,  unless  it 
received  further  funding,  would  have  to  be  relatively  narrow.  In  consultation  with  FEMA,  we  were  able  to 
establish  the  safety  and  preparedness  goal  as  primary.  This  goal  was  most  in  line  with  FEMA’s 
organizational  mission  and  would  have  the  most  direct  impact  on  saving  lives  and  property.  However,  the 
secondary  goal  of  enhancing  hurricane  science  education  contributed  to  the  first  by  providing  learners  the 
rationale  for  the  safety  measures.  Because  additional  funding  was  considered  a possibility,  we  created  a 
priority  table  instead  of  simply  dropping  objectives.  Many  of  the  objectives  that  were  considered  lower 
priority  were  ones  that  treated  the  science  in  more  depth.  Others  regarded  understanding  the  jobs  of 
hurricane  forecasters  and  emergency  managers.  We  also  reluctantly  dropped  the  priority  of  an  objective  to 
have  learners  understand  the  experience  of  surviving  a direct  hit  of  a forceful  hurricane.  While  this  would 
have  been  engaging,  it  presented  uncertainties  to  production  because  it  required  travel  to  interview 
hurricane  survivors. 
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Our  FEMA  representative  desired  a “whiz -bang”  treatment  to  grab  students’  attention.  The 
product  needed  to  stand  up  to  commercial  products  in  terms  of  a sophisticated  look  and  highly  interactive 
nature.  But  of  course,  this  desire  had  to  face  serious  constraints: 

• Available  funding  for  the  project  was  much  lower  than  is  typical  for  a highly  interactive 
multimedia  project.  We  needed  to  find  ways  to  leverage  off  existing  products  and  available  staff 
time. 

• The  project  could  not  interfere  with  our  core  prqiect  efforts  in  any  way. 

• The  product  needed  to  be  relatively  easy  for  teachers  to  use,  while  pushing  the  technology  to  a 
comfortable  limit. 

Other  constraints  also  affected  our  eventual  design  decisions: 

• The  product  needed  to  be  flexible  for  use  in  multple  contexts,  such  as  assigned  or  independent 
self-paced  instruction,  guided  group  instruction  within  a classroom  setting,  or  kiosk-based  delivery 
in  a traveling  exhibit. 

• It  needed  to  be  designed  in  sections  short  enough  to  hold  a student’s  attention  and  to  fit  within 
classroom  lesson  times. 

• It  was  meant  for  wide  and  inexpensive  distribution,  so  it  needed  to  be  Web-based,  but  also 
available  on  CD-ROM  for  those  with  limited  bandwidth  Internet  connections. 

• The  product  needed  to  be  fim,  not  didactic,  but  suitable  for  a rigorous  science  curriculum. 

• The  teacher  reviewers  on  the  team  wanted  it  to  be  relatively  self-contained  for  easy  use,  so  it 
needed  to  include  evaluation  tools. 

• As  mentioned  above,  it  had  to  link  to  National  Science  Standards  and  also  to  the  newly  released 
Red  Cross  “Masters  of  Disaster”  Curriculum. 

• Since  the  initial  budget  was  low,  but  the  aims  high,  it  needed  a modular  design  that  was 
expandable  in  case  new  funding  should  become  available. 


Synthesis  (Creating  a solution) 

Early  on,  when  only  a limited  analysis  of  the  needs  and  goals  of  the  project  had  been  done,  we  had 
already  begun  considering  solutions.  Because  we  had  to  decide  whether  we  even  wanted  to  pursue  the 
project,  early  consideration  was  necessary  about  how  we  might  approach  it  and  remain  within  the  given 
budget.  We  thought  that  the  safest  approach  was  to  take  an  existing  module,  “Community  Hurricane 
Preparedness,”  which  we  had  designed  for  emergency  managers,  and  simply  adapt  the  content  for  teenaged 
students.  Instead  of  the  complex  decision-making  simulation  it  contained,  we  would  substitute  several 
game-like  exercises  to  maintain  an  equal  level  of  engagement.  Had  we  carried  it  out  this  design,  the  product 
would  have  been  more  like  a textbook  with  multimedia  elements  to  help  maintain  interest.  Clearly,  this  was 
the  safest  route  to  ensure  the  project  could  be  done  within  budget. 

However,  while  we  felt  the  more  textbook -like  portions  of  the  adult  module  were  fine  for  the 
professionally  motivated  users  of  “Community  Hurricane  Preparedness,”  we  worried  that  we  would  quickly 
lose  our  younger  audience  without  a higher  degree  of  interaction.  Besides,  our  representative  from  FEMA 
who  funded  the  project  expressed  an  enthusiastic  desire  to  push  the  technological  envelope  and  create 
something  that  would  wow  students  (and  decision-makers  who  might  fund  additional  projects).  We  felt  this 
expectation  was  a bit  high  given  the  limited  funding,  not  to  mention  the  high  standards  of  today’s  teenagers 
when  it  comes  to  technology.  But  we  listened  and  pondered  our  options. 

Still  early  on,  and  before  our  first  extensive  meeting  with  FEMA,  we  agreed  that  a good  approach 
would  be  to  create  a learning  scenario  that  would  involve  students  in  decision-making  related  to  hurricane 
safety  and  preparedness,  while  also  supplying  motivating  experiences  with  science  content  through  games 
and  simulation.  The  exact  story  for  the  scenario  was  still  vague,  but  the  main  character  would  be  a teenager 
who  experiences  the  events  around  an  approaching,  and  then  landfalling,  hurricane.  As  this  treatment 
evolved,  we  realized  that  the  science  content  could  be  delivered  in  the  guise  of  information  provided  to  the 
main  character  to  help  her  understand  the  potential  impacts  of  the  hurricane  and  to  understand  the  limited 
predictability  that  we  face  when  making  decisions  about  preparedness  and  safety.  We  considered  the  use  of 


1579 


stories  to  be  an  effective  instructional  strategy  for  involving  and  engaging  learners,  as  well  as  providing  an 
effective  structure  for  encoding  and  recalling  information  (Schank,  1990).  The  story  would  allow  us  to 
model  good  safety  and  preparedness  behavior,  rather  than  simply  providing  information  about  good 
behavior.  In  addition,  the  scenario  structure  allowed  for  relatively  easy  future  content  additions.  If  budget 
allowed,  we  would  simply  add  supporting  activities  to  the  design.  We  were  sold  on  the  idea  and  the  FEMA 
representative  liked  it  as  well,  but  could  we  create  a learning  scenario  and  stay  within  budget? 

We  thought  we  could,  but  only  if  we  borrowed  as  much  content  from  existing  sources  as  possible, 
reducing  the  content  gathering  activity  that  is  typically  required.  We  already  had  expertise  in  hurricane 
science  and  safety,  had  contacts  who  would  be  willing  to  supply  the  expertise  we  did  not  have,  and 
possessed  many  visual  and  verbal  source  materials  from  the  “adult”  module.  Another  cost-saving  attribute 
of  the  project  was  that  the  sponsor  was  not  asking  for  a set  schedule  for  completion.  We  would  have  the 
luxury  of  fitting  the  project  into  the  small  lulls  within  our  more  schedule-sensitive  core  projects.  We 
decided  to  hire  an  instructional  designer  from  outside  the  organization,  but  one  who  was  a former 
employee,  highly  aware  of  our  sponsors,  the  content  area,  and  our  development  processes.  His  consultant 
status  allowed  the  project  to  be  non- disruptive  to  our  core  projects. 

In  addition,  we  saved  on  development  costs  by  “paying  it  forward.”  Macromedia  Flash  was  a Web 
tool  that  was  relatively  new  to  us,  but  we  knew  it  was  about  to  become  an  important  tool  for  us  because  of 
the  interactive  and  visually  compelling.  Web  applications  that  could  be  built  with  it.  We  would  use  this 
project  to  stimulate  an  R&D  effort  for  developing  Flash  skills  and  techniques  that  would  be  useful  in  the 
near  future.  Instead  of  risking  schedule  slippage  on  core  projects,  our  prototyping  with  Flash  would  happen 
on  a project  that  did  not  present  any  schedule  risk. 

Before  fleshing  out  the  scenario  any  further,  the  lead  instructional  designer  developed  a design 
matrix  that  depicted  how  and  where  each  of  the  primary  learning  objectives  would  be  treated  in  the 
scenario.  We  would  have  students: 

• Identify  the  main  ingredients  that  support  or  prevent  hurricane  formation.  This  is  done  within  a 
discovery  learning  game  where  students  experiment  by  changing  the  values  of  necessary 
ingredients  to  create  the  optimal  conditions  for  hurricane  formation. 

• Understand  the  key  characteristics,  life  cycles,  climatology,  and  classification  schemes  of  tropical 
storms  and  hurricanes.  This  is  presented  more  as  informat  ion,  but  treated  in  a visually  compelling 
way,  with  embedded  interactions  included.  Students  use  worksheets  to  encourage  engagement 
with  the  material.  Because  we  had  an  existing  3D  digital  model  of  a hurricane  developed  for  a 
previous  project,  we  include  an  activity  to  explore  this  model  via  “virtual  reality”  software. 
However,  the  model  was  reworked  to  illustrate  concepts  at  the  proper  level  for  this  new  audience. 
Discarded  methods  of  treatment  included  video  footage  of  a flight  through  a hurricane.  It  was 
decided  that  the  bandwidth  required  for  adequate  video  quality  was  prohibitive. 

• Identify  factors  that  can  contribute  to  uncertainty  in  hurricane  forecasts.  This  is  necessary  to  help 
the  public  understand  why  evacuation  can  sometimes  be  necessary  even  though  the  hurricane 
never  becomes  a local  threat.  This  is  treated  both  with  information  content  and  with  a 
simulation/game  that  allows  learners  to  change  the  positions  of  high-  and  low-pressure  systems 
and  see  the  resulting  hurricane  path.  This  exercise  was  developed  using  actual  numerical  model 
simulations  performed  by  a NOAA  researcher  at  Colorado  State  University,  although  the  game  is 
not  a fully  functioning  simulator  itself  The  interface  for  this  simulation  is  reproduced  below. 

• Identify  hazards  associated  with  landfalling  hurricanes.  Again,  this  content  is  treated  with 
information  presented  with  engaging  media  qualities  and  supported  with  embedded  interactions 
and  worksheet  questions.  Discarded  treatments  included  presenting  interviews  with  survivors  of 
hurricanes  and  others.  We  decided  that  this  would  likely  require  travel  and  would  increase 
uncertainty  in  the  schedule. 

• Choose  appropriate  safety  and  preparedness  measures  for  hurricane  events.  It  was  this  content 
that  would  primarily  drive  the  scenario.  This  is  discussed  more  fully  below. 

The  scenario  itself  involves  the  learner  making  a visit  to  Camille  Castillo  and  her  parents  in  the 
coastal  Florida  community  of  Ft.  Walton  Beach,  site  of  the  actual  Hurricane  Erin  strike  in  1995.  The 
scenario  unfolds  over  7 days  as  Hurricane  Erin  develops,  approaches,  and  then  makes  landfall  at  Ft.  Walton 
Beach.  Each  day  as  the  hurricane  nears  and  then  finally  arrives,  the  learner  participates  with  the  family  as 
they  prepare  their  home  for  the  threat  and  prepare  themselves  and  Aunt  Betsy  for  the  potential  of 
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evacuation.  Safety  activities  include  choosing  what  to  pack  in  the  evacuation  duffle  bag,  going  over 
emergency  plans,  shopping  for  appropriate  emergency  supplies,  preparing  the  house  for  high  winds,  and 
advising  Aunt  Betsy  what  to  do  as  she  evacuates  her  mobile  home  and  then  returns  to  a potentially 
damaged  home  after  the  storm  passes. 

Each  day  the  learner  has  access  to  an  informative  TV  weather  update  (courtesy  of  The  Weather 
Channel)  and  instructional  applications  stored  on  the  family’s  laptop  computer.  The  laptop  computer 
applications  primarily  treat  science  topics,  but  some  relate  to  the  hazards  and  safety  of  hurricanes  as  well. 
The  primary  interface  for  Hurricane  Strike!,  the  Castillo  Home,  is  displayed  below.  Learners  receive  a task 
list  each  day  to  remind  them  of  things  they  need  to  do,  but  they  can  also  just  explore  the  rooms  of  the  house 
and  click  on  the  active  items  that  lead  to  the  various  instructional  activities.  For  example,  Hugo  the  dog 
presents  a new  task  each  day  by  way  of  the  various  items  he  carries  in  his  mouth — clicking  on  Hugo  when 
he  carries  the  purse  instigates  a shopping  trip  for  emergency  supplies. 


Evaluation  (Fine-tuning  the  Solution) 

Evaluation  activities  for  the  project  began  almost  immediately.  After  our  initial  planning  meetings 
and  further  reflection,  the  instructional  designer  for  the  project  created  a Project  Plan  that  included  our 
analysis  assumptions  and  a detailed  list  of  proposed  objectives.  In  addition,  he  also  created  a Planning 
Matrix  that  described  the  potential  ways  we  would  treat  each  objective.  FEMA  representatives  and  two 
middle-school  teachers  reviewed  these  documents  to  validate  our  conclusions.  This  helped  us  clarify  our 
priorities,  but  fortunately,  they  did  not  observe  any  major  misconceptions  on  our  part. 

As  the  scripts  for  various  sections  were  written,  they  were  shared  with  the  middle  school  teachers 
and  content  experts  from  the  Red  Cross,  FEMA,  and  COMET.  This  helped  us  correct  misleading  or 
incorrect  pieces  of  content  and  also  helped  us  confirm  the  material  was  written  at  the  correct  level.  As  the 
pieces  were  developed,  we  performed  formative  evaluation  by  trying  them  out  ourselves,  but  also 
observing  how  our  own  middle  school-aged  children  used  them  (we  had  purposely  chosen  a modular 
design  so  that  we  could  develop  and  test  incrementally). 

The  scenario  structure  offered  tremendous  opportunities  for  learning  activities,  but  it  was  a bit 
daunting  to  think  through  all  the  usability  issues  in  advance.  As  we  recorded  audio  content  and  prototyped 
several  days  of  the  scenario,  the  difficulties  became  apparent.  For  example,  since  many  computer  games 
are  designed  to  be  exploratory  and  open-ended,  we  had  speculated  that  this  approach  would  be  engaging  to 
students.  But  since  we  also  had  specific  learning  objectives  we  wanted  learners  to  achieve,  we  weren’t 
willing  to  allow  them  to  bypass  or  miss  out  on  activities  (which  some  of  our  testers  did).  Worksheets  were 
designed  as  a solution  to  provide  the  structure  necessary  to  ensure  that  learners  used  all  the  parts  of  the 
module.  Similar  to  that  way  a list  in  a scavenger  hunt  works,  learners  would  have  to  explore  each  element 
of  the  module  in  order  to  complete  the  worksheets.  But  this  is  also  where  the  comparison  with  a home 
computer  game  broke  down.  Completing  the  worksheets  could  be  perceived  as  a typical  school  assignment 
as  opposed  to  part  of  a game.  Therefore,  some  of  us  felt  it  necessary  to  provide  more  guidance  within  the 
module  so  that  the  task  of  completing  worksheets  was  less  stressful.  There  were  a few  heated  debates 
within  the  team  about  the  amount  of  guidance  necessary.  In  the  end,  we  erred  on  the  side  of  clarity  by 
providing  task  lists  for  each  day  and  making  the  items  learners  needed  to  click  more  obvious. 

Another  design  decision  that  generated  lots  of  debate  centered  around  the  simulation/game  that 
allows  learners  to  move  high-  and  low-pressure  systems  to  see  their  effects  on  hurricane  paths.  This  was 
originally  proposed  as  a type  of  bowling  game,  including  pins  to  knock  down  and  a scoring  system.  Kids 
would  probably  have  loved  this,  but  some  of  us  were  concerned  with  the  ethical  dimensions  because 
hurricane  strikes  (and  spares)  obviously  can  take  lives  and  cause  serious  destruction.  Also,  we  had  to 
remind  ourselves  that  the  most  important  learning  objective  was  not  knowing  what  forces  move  hurricanes, 
but  that  hurricane  movements  are  difficult  to  predict.  In  the  end,  the  “game”  became  more  of  a scientific 
simulation  that  drives  this  point  home.  Some  kids  who  tested  this  application  ended  up  making  their  own 
game  out  of  the  exercise  anyway  when  they  tried  to  make  the  hurricane  strike  certain  cities.  At  least  they 
are  learning  about  the  causes  of  atmospheric  flow  patterns  and  why  hurricane  paths  are  hard  to  predict,  we 
told  ourselves. 

Usability  was  the  most  frequent  issue  we  encountered  during  formative  evaluations.  For  example, 
nearly  all  the  instructions  and  interfaces  for  the  various  exercises  needed  some  tweaking.  However,  as  the 
members  of  the  team  reviewed  each  other’s  work,  many  aesthetic  concerns  were  raised  as  well.  These  are 
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by  nature  subjective  and  sometimes  difficult  to  resolve  with  group  consensus.  We  did  the  best  we  could, 
giving  and  taking  to  keep  the  project  moving.  Subject  matter  experts  raised  a few  content  issues  during 
their  final  reviews,  but  early  reviews  of  the  scripts  meant  such  issues  were  few  and  did  not  require  any 
major  revisions. 


Summary 

As  this  paper  has  demonstrated,  the  instructional  design  process  is  often  non-linear.  We  may 
conveniently  think  of  the  process  as  sequential  Analysis,  Design,  Development,  Implementation  and 
Evaluation  phases,  but  such  linearity  in  reality  would  be  stifling.  Many  authors  have  recently  developed 
models  that  describe  a process  of  iterative  refinement  of  the  problem  definition  and  solutions  (Willis  & 
Wright,  2000)  or  depict  a dynamic  system  of  interacting  authoring  activities  that  lead  to  one  of  many 
possible  solutions  (Tennyson,  1997).  In  our  experience,  such  models  of  the  instructional  design  process 
more  accurately  describe  how  good  ID  is  done. 

Like  all  projects,  this  one  began  with  substantial  analysis  activities,  but  we  also  began  almost 
immediately  mentally  trying  out  solutions,  even  when  only  a limited  amount  of  analysis  data  had  been 
gathered.  These  initial  trial  solutions  were  necessary  to  consider  fully  our  production  needs  and 
implementation  constraints,  and  to  determine  how  best  to  use  the  available  budget.  The  early  potential 
solutions  essentially  became  additional  analysis  data  to  allow  us  to  consider  project  needs.  We  did  not 
allow  early  synthesis  of  potential  solutions  to  box  us  in,  however.  Only  a few  isolated  components  of  the 
product  saw  early  development  activity;  the  bulk  of  the  development  work  was  done  only  after  thorough 
analysis  and  design  review.  Evaluation  was  uii  essential  part  of  the  project  and  occurred  throughout. 
Designated  reviewers  and  fellow  team  members  reviewed  documents  and  prototypes  at  all  phases, 
including  the  initial  analysis.  Such  formative  evaluation,  performed  by  a range  of  staff  and  user 
representatives  with  a variety  of  skills  and  perspectives,  saved  many  hours  of  wasted  work.  Some  team 
members  may  become  frustrated  with  this  “committee”  approach  to  project  decision-making,  but,  if  done 
within  reason,  more  highly  participatory  design  and  evaluation  can  lead  to  important  improvements  and 
reduced  effort  by  identifying  more  numerous  and  diverse  design  options  and  issues  early  in  the  process. 
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Abstract;  Though  schools  report  an  increase  in  the  quantity  and  quality  of  use  of  technology  in  education, 
many  still  question  of  the  merit  of  use  of  technology  in  K-12.  Technology  simply  used  as  information 
distribution  deserves  such  scrutiny  and  criticism.  Using  technology  in  educational  projects  in  a variety  of 
ways  increases  novelty  and  student  interest  and  can  also  approach  higher  critical  thinking  levels. 
Government  has  recognized  a challenge  of  preparing  children  for  a lifelong  commitment  and  involvement 
in  the  court  and  identifies  the  schools  in  playing  a key  role  in  meeting  this  challenge.  This  project  calls  for 
direct  involvement  of  the  Court  in  a variety  of  programs  in  partnership  with  the  schools,  integrating 
education  on  the  court  into  the  curriciium  in  K-12.  This  project  between  the  courts  and  schools 
demonstrates  how  partnership  between  public  agencies  enhanced  by  technology  has  the  tremendous 
potential  to  multiply  the  impact  of  civic  education  for  students. 


Introduction 

There  is  a distinct  point  in  the  diffusion  of  innovations  that  I have  come  to  call  the  point  of  no  return,  not  to 
be  seen  as  detrimental,  but  to  indicate  that  a society  cannot  function  without  that  innovation  and  would 
indeed  be  deemed  primeval  with  its  absence.  Surely,  communication  technology  has  reached  that  point. 
One  indicator  of  this  placement  is  that  the  product  is  seen  as  essential  across  living  category  dimensions 
such  as  education,  religion,  entertainment  and  consumerism.  Another  indicator  in  reaching  this  pinnacle  is 
that  the  use  of  the  innovation  cuts  across  age  lines.  Despite  the  increase  in  the  quantity  and  quality  of  use 
of  technology  in  education,  there  is  still  the  persistent  questioning  of  the  merit  of  the  technology  in  K-12 
education  (Panel  on  Educational  Technology,  1997).  Technology  simply  used  as  information  distribution 
deserves  such  scrutiny  and  criticism.  The  research  in  technology  use  in  education  points  to  four  distinct 
ways  that  technology  can  be  used  as  media  (Bruce  and  Levin  (1997).  Media  as  inquiry  emphasizes  the  use 
of  the  Internet  and  other  databases  for  data  gathering  for  research  or  other  problem-centered,  fact-finding 
activities,  media  for  communication  underscores  functions  such  as  email  and  word-processing,  media  for 
construction  looks  at  computer-aided  design  and  media  for  expression  represents  more  interactive 
outcomes  of  technology  projects.  Using  technology  in  educational  projects  in  a variety  of  ways,  thus  a 
variety  of  levels,  not  only  increases  novelty  and  student  interest,  but  as  the  activity  moves  to  higher  levels 
of  interactivity  it  also  approaches  higher  critical  thinking  levels. 

At  the  same  time  education  is  assessing  its  use  of  technology  in  the  classroom,  government  is 
evaluating  its  role  in  the  community,  how  it  is  perceived  in  the  community  and  how  the  court  can  partner 
with  the  education  community  to  educate  K-12  students  on  the  Court’s  role  in  the  community  and  on  their 
future  role  and  responsibilities  in  the  administration  of  justice.  We  realize  that  the  court  aid  the  schools 
face  the  very  real  challenge  of  preparing  children  for  a lifelong  commitment  and  involvement  in  the  Court 
and  have  identified  the  schools  in  playing  a key  role  in  meeting  this  challenge. 

The  overarching  project  plan  calls  for  direct  involvement  of  the  Court  in  a variety  of  programs  in 
partnership  with  the  County  A Superintendent  of  Schools  integrating  education  on  the  court  into  the 
curriculum  in  K-12.  This  project  infuses  educational  objectives  with  the  goals  of  the  Court,  increases 
involvement  and  interest  on  the  part  of  our  educators,  and  places  the  learning  of  the  courts  into  a familiar 
and  credible  context  for  the  students  - the  educational  context.  This  model  reflects  the  ideas  of  the  Goals 
2000  legislation  in  using  authentic  tasks  in  teaching  with  technology 
http://www.cd.gOv/pubs/EdRcformStudics/EdTcch/ovcrview.html#authentic. 
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The  Project 


This  prototype  project  between  the  courts  and  schools  in  County  A demonstrates  how  partnership  between 
public  agencies  enhanced  by  technology  has  the  tremendous  potential  to  multiply  the  impact  of  civic 
education  for  young  students.  Schools  and  students  who  participate  in  this  project  will  develop  knowledge 
about  and  appreciation  for  the  justice  system  and  its  important  contribution  to  a free  society.  In  State  A’s 
schools  the  courts  are  first  studied  in  third  and  fourth  grade.  Very  little  material  is  available  to  make  this 
study  concrete  and  meaningful  for  young  students.  Most  teaching  is  done  from  generic  national  textbooks 
with  little  relationship  to  the  students'  own  community.  Television  reinforces  a concept  of  law  and  justice 
based  on  heroic  individualism  rather  than  Constitutionally  protected  procedure  and  thoughtful,  reflective 
deliberation.  This  is  reinforced  by  the  problem  that  teachers  have  little  direct  experience  with  the  justice 
system  beyond  jury  summons  or  a traffic  issue. 

The  Internet  offers  the  opportunity  for  many  classrooms  to  learn  about  and  participate  virtually  in  the 
justice  system.  Structured  creatively  and  at  the  proper  level  of  difficulty,  the  online  experience  can  be 
interactive  and  memorable.  In  addition,  classroom  support  materials  can  be  made  available  to  a wide 
audience  at  very  little  cost. 

Recognizing  the  enormity  of  such  a project,  the  Court  and  County  A Superintendent  of  Schools  have 
decided  that  the  first  step  in  enabling  this  collaboration  is  to  initiate  the  project  focusing  on  the  elementary 
school  grades  in  the  social  science  and  history  curriculum.  Initial  meetings  have  determined  that  the 
curriculum  has  to  be  innovative  and  exciting  for  students,  must  blend  educational  and  Court  goals,  and 
meet  the  Judicial  Counsel’s  long-range  goals  for  community  outreach  and  education. 

Specifically,  the  project  includes  1)  Multimedia  Curriculum.  This  package  includes  web-based  lessons 
and  non-web-based  activities  that  will  be  scheduled  throughout  the  year  that  promote  the  court  in  the 
community,  including  school  assemblies  that  feature  court -related  speakers,  a formal  integration  of  law  day 
into  the  schools  throughout  the  school  year,  while  still  identifying  May  1 as  the  “official”  Law  Day  as 
recognized  by  Congress.  Appreciating  the  research  that  indicates  that  online  materials  are  more  attractive 
to  teachers  when  included  in  a “curriculum  package”  we  will  link  these  activities  with  corresponding  web- 
based  curriculum  that  will  be  designed  for  student  preparation  or  post  study  to  tie  the  activity  to  further 
education  of  the  court,  2)  Virtual  Courthouse.  The  historic  courthouse  of  County  A houses  a wealth  of 
information,  stories,  and  legends  that  can  only  add  to  a student’s  understanding  and  appreciation  of  the  role 
of  the  court  and  law  in  the  community  and  county.  Our  plan  for  the  County  A Virtual  Courthouse  is  to  use 
actual  diagrams  and  pictures  of  the  County  A Courthouse,  not  cartoon  or  clip  art  as  is  typically  used.  The 
courthouse  will  be  interactive  and  hyperlinked,  using  pop-up  figures  that  will  respond  to  questions  that  a 
student  may  ask  from  a set  of  pre-preprogrammed  questions.  Various  paths  will  be  available  for  a teacher 
to  choose  to  focus  on  in  a lesson  (with  corresponding  online  curriculum  material)  that  teachers  help  in 
designing,  and  3)  Virtual  Fieldtrips.  County  A Courts  and  Superintendent  of  Schools  propose  to  develop  a 
virtual  fieldtrip  to  the  courts  for  3rd  and  4th  grade  students.  The  discussion  so  far  is  that  this  online  project 
will  utilize  the  hyper-linking  capability  of  the  Internet  by  allowing  the  user  to  follow  multiple  pathways 
through  the  site.  For  example,  a user  might  follow  the  day  of  a person  who  is  serving  on  a jury;  a typical 
day  in  court  of  an  attorney;  or  a day  in  the  life  of  a judge.  These  ’’day  in  the  life  of  Virtual  Field  Trips  will 
each  be  supported  by  classroom  materials.  The  materials  will  be  developed  by  outstanding  teachers  in  the 
region  and  be  specially  designed  to  meet  the  needs  of  County  A's  diverse  student  population  but  with  an 
eye  to  the  curriculum  of  schools  throughout  State  A.  As  part  of  this  Virtual  Tour,  various  times  will  be 
scheduled  for  students  to  interact  with  a court  official  via  omail  in  a type  of  "Ask  and  Expert"  experience. 
These  may  be  part  of  a response  to  video  interviews  of  a judge,  a bailiff,  an  attorney,  a court  reporter,  etc. 
posted  on  the  site. 
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Designing  for  a moving  target 


Designing  an  effective  learning  experience  on  a PDA  in  a wireless  environment  has 
forced  a reexamination  of  the  instructional  design  principles  normally  associated  with  the 
content  development  process,  a team  of  instructional  designers  in  Canada  has 
concluded.  The  typical  project  management  model  that  has  been  applied  to  face-to-face 
and  elearning  projects  can‘t  be  relied  on  for  the  emerging  miearning  environment. 
Instructional  design  strategies  have  to  be  modified  for  a mobile  learning  environment 
that  seems  to  be  changing  monthly. 

Several  significant  issues  are  affecting  the  instructional  designers  responsibilities  in  the 
new  miearning  world.  Constantly  changing  PDA  devices  means  that  the  potential  of  the 
technology  platform  is  largely  unknown  and  enormously  difficult  to  design  for. 
Uncertainties  surrounding  the  type(s)  of  content  that  may  or  may  not  work  on  the 
market-ready  PDAs  impact  on  the  choices  available  to  the  subject  matter  experts.  While 
many  of  the  learner  and  faculty  support  issues  are  similar,  the  complexities  of  these 
resources  are  increasing.  And,  of  course,  for  many  pilot  exercises,  the  costs  for 
miearning  production  activities  seem  to  be  larger  than  for  their  elearning  counterparts. 

At  the  end  of  the  day,  how  can  the  instructional  design  principles  developed  over  the 
past  few  years  be  applied  to  a moving  target?  Moreover,  how  can  student  and  faculty 
expectations  for  a satisfying  and,  ultimately,  successful  learning  experience  be  met? 
These  questions — along  with  the  increasing  importance  of  the  development  project  team 
that  involves  content  providers,  SMEs  and  instructional  designers — will  be  explored 
against  the  backdrop  of  a pioneering  miearning  educational  experience  at  two  leading 
Canadian  colleges. 
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Abstract:  Web-based  knowledge  assessment  with  on-line  quizzes  is  one  of  the  oldest  and  most  popular 
ways  of  using  the  Web  in  education.  Almost  all  major  Web  courseware  management  systems  support 
authoring  and  delivery  of  on-line  quizzes  made  from  static  questions.  Our  work  attempts  to  address 
several  known  shortcomings  of  static  questions.  We  suggest  an  approach  and  a system  to  author  and 
deliver  Web-based  dynamic  parameterized  quizzes  for  programming-related  subjects.  The  system  has 
been  used  in  a practical  class  for  two  semesters  with  both  desktop  and  mobile  computers  and  in  both 
aasessment  and  self-assessment  modes.  This  paper  briefly  describes  the  system  and  summarizes  the 
results  of  its  formal  classroom  evaluation. 

Introduction 

Objective  tests  and  quizzes  are  among  the  most  widely  used  and  well-developed  tools  in  higher  education.  Web-based 
knowledge  assessment  with  on-line  quizzes  was  the  first  to  be  implemented  and  currently  the  most  well  developed  and 
most  popular  interactive  component  in  Web-based  education  [Bmsilovsky  & Miller  1999;  Brusilovsky  & Miller  2001].  All 
major  Web  courseware  management  systems  support  authoring  and  delivery  of  on-line  quizzes  made  from  static 
questions.  A lot  of  efforts  have  been  put  by  the  vendors  of  these  systems  to  support  a larger  variety  of  question  types, 
make  question  authoring  easier  for  a teacher,  and  provide  support  in  managing  quizzes  and  pools  of  questions.  Yet,  with 
all  these  powerful  authoring  tools  can’t  solve  a few  inherent  problems  of  this  traditional  assessment  tool  - quizzes  made 
of  static  questions. 

All  these  problems  come  from  an  obvious  fact:  when  author  creates  one  static  question,  one  and  only  one 
question  is  created.  Modem  authoring  tools  made  creating  of  Web  quizzes  much  easier  then  it’s  used  to  be^  however, 
creating  every  question  is  still  a large  investment  of  time  because,  if  questions  are  taken  seriously,  creating  the  content 
takes  much  more  time  than  entering  this  content  into  a system.  Thus,  every  question  is  an  expensive  component  of 
teaching  material  and  the  number  of  questions  that  can  be  created  for  a particular  course  is  relatively  small.  In  this 
context,  it  is  quite  natural,  that  in  both  assessment  and  self-assessment  contexts  all  students  in  the  class  are  using  the 
same  set  of  static  questions^.  It  creates  at  least  two  well-known  problems: 

• In  an  assessment  context,  the  use  of  the  same  set  of  questions  for  the  whole  class  provokes  cheating.  Cheating  is 
quite  an  issue  even  for  the  classroom  quizzes.  For  take-home  and  Web-based  quizzes  cheating  becont  a major 
problem  that  actually  prevents  the  teacher  to  rely  on  results  of  this  kind  of  quizzes.  Cheating  is  an  issue  not  only 
within  the  class,  but  also  between  classes.  In  the  places  where  the  same  questions  are  being  used  in  several  classes 
(during  the  same  term  on  during  different  terms)  the  students  are  known  to  accumulate  and  pass  each  other  the 
correct  answers. 

• In  the  self-assessment  context,  where  the  cheating  is  not  a problem,  the  students  usually  suffer  from  the  lack  of 
material:  There  are  usually  too  few  questions  to  work  with  and  one  question  can  be  answered  only  once. 


^ A number  of  my  colleagues  who  were  creating  quizzes  with  the  top-of-the-line  Blackboard  5.1  system  have  claimed  that 
the  old  ’’paper  and  pencil"  way  is  still  much  easier  for  them.  Creating  quizzes  in  an  editor  such  as  Microsoft  Word  is  still 
faster  (especially  taking  into  account  slow  Web-based  authoring  tools).  What  can  really  save  time  is  automatic  grading 
(at  least  for  large  classes).  However,  in  many  cases,  quizzes  are  graded  by  Teaching  Assistants  and  thus  this  saving 
does  not  affect  the  professors  themselves. 

^ A more  progressive  approach  where  every  student  is  getting  a subset  of  a pool  of  questions  developed  for  a quiz  is 
used  very  rarely  exactly  because  questions  are  expensive  to  develop. 
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A known  remedy  for  this  approach  is  the  use  of  parameterized  questions.  With  this  approach,  an  author  creates  a 
pattern  of  a question.  At  the  presentation  time,  the  pattern  can  be  instantiated  with  parameters  taken  from  a particular 
set.  This,  every  question  pattern  is  able  to  produce  a large  or  even  unlimited  number  of  different  questions. 
Parameterized  questions  are  currently  a focus  of  one  of  the  promising  research  directions  in  the  field  of  Web-based 
education  (WBE).  A number  of  pioneer  systems  such  as  CAPA  [Kashy  et  al.  1997],  Web  Assign  [Titus,  1998  #1386], 
EEAP282  [Merat  & Chung  1997],  or  Mallard  [Graham,  Swafford  & Brown  1997]  have  has  explored  the  use  of 
parameterized  questions  in  different  settings.  The  early  results  are  very  encouraging.  For  example,  the  authors  of  the 
CAPA  systems  have  carefully  monitored  the  use  of  their  system  and  reported  that  the  use  of  parameterized  questions 
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Our  own  work  capitalizes  on  the  experience  of  early  systems  with  parameterized  questions  and  is  aimed  at 
extending  their  application  area  to  a non-traditional  and  challenging  subject  of  programming  languages.  We  have 
developed  an  approach  and  a system  to  author  and  deliver  Web-based  dynamic  parameterized  quizzes  for  programming- 
related  subjects.  The  system  has  been  used  in  a practical  class  for  two  semesters  with  both  desktop  and  mobile 
computers  and  in  both  assessment  and  self-assessment  modes.  This  paper  briefly  describes  the  system  and  summarizes 
the  results  of  its  formal  classroom  evaluation. 


Parameterized  Questions  for  Programming-related  Subjects 

Traditionally  parameterized  questions  are  used  in  math-based  courses  such  as  physics  and  various  math  subjects.  In 
these  application  areas,  a correct  answer  to  a parameterized  question  can  be  specified  by  a formula  that  includes  one  or 
more  question  parameters.  In  this  context,  creating  parameterized  questions  and  checking  the  correctness  of  student 
answers  is  relatively  easy:  the  question  author  provides  the  interval  for  the  possible  values  of  a randomized  parameter 
(or  parameters)  and  specifies  the  formula  to  calculate  the  correct  answer.  The  quiz  system  generates  a question  and 
stores  its  parameters  along  with  the  content  of  the  question.  To  check  the  student  answer,  the  correct  answer  for  the 
given  set  of  parameters  is  calculated  using  the  provided  formula  and  compared  with  the  given  answer.  The  formula - 
based  approach  is  simple  and  powerful.  It  even  enables  the  system  to  recognize  most  typical  misconceptions  if  an  author 
can  provide  a "wrong"  formula  that  corresponds  to  a misconception. 

Unfortunately,  the  formula-based  approach  can't  be  used  in  programming  except  for  very  simple  cases  of 
calculating  numerical  expressions.  Our  challenge  was  to  design  a different  approach  that  can  work  for  a large  variety  of 
questions  in  programming  courses.  While  analyzing  several  large  pools  of  questions  created  for  courses  based  on  such 
languages  as  Lisp,  C,  and  Java,  we  have  found  that  a large  portion  of  questions  in  these  pools  are  code  evaluation 
questions.  In  a code  evaluation  question,  the  user  is  provided  with  some  fragment  of  a program  and  is  asked  to  predict 
the  value  of  a particular  variable  or  a string  to  be  printed  at  the  end  of  execution  of  this  fragment.  This  kind  of  questions 
is  very  important  - it  enables  the  teacher  to  check  the  student's  understanding  of  the  semantics  of  programming  language 
constructs.  Semantics  of  a programming  language  is  an  important  body  of  knowledge  on  itself  and  also  a prerequisite  to 
a higher  level  programming  skills. 

In  traditional  Web-based  courses,  where  code  evaluation  questions  are  very  popular.  They  are  authored  much 
like  other  kind  questions  in  the  form  of  multiple  choice  or  blank-filling  using  provided  tools  for  developing  of  static 
questions.  They  are  evaluated  by  comparing  the  answer  given  by  a student  with  the  answer  pre -specified  by  a teacher. 
However,  it  is  known  that  code  evaluation  questions  allow  a different  approach  to  checking  the  correctness  of  student 
answers.  Using  a language  interpreter  or  compiler  the  system  can  ran  the  given  fragment  of  code  and  calculate  the 
correct  answer  for  a code  evaluation  questions  without  the  need  for  a teacher  to  provide  it.  Then  it  can  compare  the 
calculated  answer  with  student  answer.  We  have  used  this  approach  is  several  earlier  systems  such  as  ITEM/IP 
[Brusilovsky  1992],  ILEARN  [Brusilovsky  1994],  and  ELM-ART  [Weber  & Brusilovsky  2001]  to  save  some  sizeable 
amount  of  question  development  time  and  to  avoid  authoring  errors. 

When  developing  our  system  QuizPACK  (Quizzes  for  Parameterized  Assessment  of  C Knowledge)  we  have 
attempted  to  combine  our  earlier  work  on  code  evaluation  questions  with  the  ideas  of  parameterized  questions.  Our 
motivation  was  to  create  a system  that  would  allow  a teacher  to  create  parameterized  code  evaluation  questions  and 
quizzes  easier  than  creating  static  questions  with  existing  authoring  tools.  The  idea  of  QuizPACK  is  very  simple.  A 
teacher  provides  the  core  content  of  the  question:  a parameterized  fragment  of  code  to  be  evaluated  and  an  expression 
that  have  to  be  evaluated  by  the  student  at  the  end  of  the  fragment  execution.  The  system  does  the  rest:  randomly 
generates  a question  parameter,  creates  a presentation  of  the  parameterized  question  in  a Web-based  quiz,  gets  the 
student's  input,  compares  the  student  answer  with  the  result  of  running  the  parameterized  code  "behind  the  stage",  and 
records  the  results  into  a server-side  database. 

The  current  version  of  QuizPACK  can  support  a whole  programming-related  course  based  on  C or  C++ 
programming  language.  It  supports  simple  questions  based  on  a few  lines  of  code  as  well  as  very  advanced  questions 
with  the  code  that  includes  multiple  functions  and  header  files.  QuizPACK  allows  the  teacher  to  prepare  full  quizzes  of 
different  length  in  the  form  of  static  sequence  of  parameterized  questions.  It  also  allows  a teacher  to  display  various 
kinds  of  reports  about  the  student  performance  on  quizzes.  The  following  sections  of  the  paper  presents  the  student's 


and  the  teacher's  side  of  QuizPACK  and  reports  some  result  of  its  evaluation  in  a Data  Structure  course  based  on  C 
language. 

Students  Interface  of  QuizPACK 


The  students  are  accessing  the  quizzes  using  our  KnowledgeTree  learning  portal.  The  KnowledgeTree  portal  allows  a a 
teacher  to  create  a course  support  Web  site  that  can  use  course  materials  distributed  among  different  servers.  With 
knowledgeTree,  a teacher  is  able  to  specify  the  objectives,  readings,  and  other  critical  information  for  every  lecture  and 
to  specify  relevant  learning  activities  of  different  kinds,  in  particular,  a Icachei’  can  specify  a set  of  quizzes  to  suport  a 
lecture.  When  a student  selects  a quiz  in  KnowledgeTree,  the  portal  request  the  selected  quiz  from  the  quiz  server  and 
passes  the  student  parameters  to  the  server.  The  server  immediately  loads  the  first  question  of  the  quiz  in  a separate 
window. 
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Figure  1:  KnowledgeTree,  a portal  for  accessing  distributed  Web-based  course  support  material.  Left  side:  a list  of 
lectures  created  by  a teacher.  Right  side:  a view  of  a lecture  with  associated  learning  material.  Different  items  are  usually 

served  by  different  activity  servers. 

The  parameterized  question  is  presented  to  the  student  just  like  a normal  static  fill-in  question.  First,  the  program 
fragment  to  be  evaluated  is  presented  then  the  expression  to  be  evaluated,  then  form  box  to  write  down  the  answer 
(Figure  2,  left).  One  or  more  constants  that  the  student  can  see  in  the  body  of  the  question  are,  actually,  instantiated 
parameters.  They  may  be  different  for  different  students  taking  the  quiz  as  well  as  for  the  same  student  attempting  the 
question  several  times.  In  a properly  designed  question,  the  value  of  the  expression  to  be  evaluated  depends  on  the 
instantiated  parameters  and  thus  will  also  be  different  for  different  students.  Every  student  is  supposed  to  mentally 
execute  the  program,  evaluate  the  specified  expression,  enter  its  value  into  the  box,  and  hit  "submit"  button.  After  that 
the  system  generates  an  evaluation  screen  for  the  student.  The  evaluation  screen  repeats  the  content  of  the  question, 
evaluates  the  correctness  of  the  student's  answer,  provides  the  correct  answer  if  the  student's  answer  was  incorrect,  and 
generate  a link  to  the  next  question  (Figure  2,  right).  This  screen  lets  the  student  to  re-think  the  question  and  the  answer. 
In  particular,  the  student  may  want  to  attempt  the  same  question  again  by  using  the  Back  button  and  reloading  the 
question  screen.  The  student  can  attempt  the  same  question  many  times,  though  in  evaluation  context  only  the  results  of 
the  first  attempt  is  recorded  (Figure  2,  right).  Students  may  finish  working  with  the  quiz  anytime  simply  by  closing  the 
quiz  window  and  returning  to  the  KnolwedgeTree  portal.  Note  that  the  student  interface  does  not  sport  elaborated 
graphics  and  uses  the  minimal  required  space  to  present  the  question  and  the  feedback.  It  was  done  to  enable  working 
with  questions  on  multiple  handheld  wireless  devices.  As  a result,  the  QuizPACK  perfectly  works  on  small  Windows  CE 
computers  like  HP  Jornada  and  even  on  Palm-based  organizers. 
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We  welcome  the  readers  to  try  QuizPACK  system  by  logging  in  to  KnowledgeTree  system  at 
http://dbpc.sis.pitt.edu/sharad/kt/login.html.  The  system  allows  self-registration.  We  suggest  to  use  your  e-mail  address 
for  your  login  name  to  avoid  name  conflicts. 
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Figure  2.  The  user  interface  of  QuizPACK. 

QuizPACK  from  the  Teachers  Point  of  View 


Teachers  use  QuizPACK  to  publish  online  parameterized  quizzes  for  the  students.  There  are  three  major  steps  a teacher 

has  to  complete  in  preparing  a quiz. 

• First,  the  teacher  has  to  provide  the  programs  to  be  evaluated.  The  code  of  each  program  has  to  reside  in  a separate 
file.  The  place  of  a randomized  parameter  in  the  code  of  the  questions  is  denoted  by  a pseudo-variable  Z.  At  the 
question  generation  phase  QuizPACK  every  occurrence  of  Z variable  will  be  substituted  by  an  integer  randomized 
within  some  interval.  In  effect,  a teacher  defines  a pattern  of  a question.  A pattern  is  a simple  way  for  a teacher  to 
give  different  questions  of  the  same  difficulty  level  to  all  the  students  without  being  biased  with  the  choice  of 
questions  for  different  students. 

• Second,  the  teacher  has  to  specify  a quiz  in  a special  quiz  index  file.  Each  line  in  this  file  corresponds  to  one 
question.  For  each  question  the  teacher  has  to  specify  the  name  of  the  file  with  question  code,  the  lower  and  the 
upper  limits  for  randomized  variable,  an  expression  to  be  evaluated  by  the  students,  and  a type  of  this  expression 
(i.e.,  int  or  float).  With  this  approach,  the  code  of  the  question  in  QuizPACK  is  separated  from  the  pedagogic 
information  such  as  randomization  limits  and  expression.  It  allows  the  author  to  use  the  same  question  code  in 
different  quizzes  with  possibly  different  parameters. 

• The  last  step  is  to  compile  a quiz.  QuizPACK  system  is  developed  to  can  woik  on  any  Unix  system  with  a variety  of 
Web  servers.  The  system  itself  is  a collection  of  several  prototype  data  files  and  several  commands  (in  Unix  sense). 
The  commands  are  all  written  in  C-H-  and  can  be  recompiled  on  any  Unix  platform.  The  authoring  interface  for 
compiling  a quiz  consists  of  just  two  commands.  The  newquiz  command  generates  a folder  (catalog)  for  a new  quiz 
with  all  required  prototype  files.  The  name  of  this  folder  is  also  the  name  of  the  quiz.  This  is  the  folder  where  teacher 
have  to  place  the  question  files  and  the  quiz  index  file  for  the  quiz.  Naturally,  these  files  can  be  simply  created  inside 
the  quiz  folder  using  any  editor,  moved  from  other  folders  or  transported  by  FTP  from  another  computer  (including 
Macs  and  PCs).  After  all  files  are  added  to  the  quiz  folder,  the  author  has  to  execute  compile  command  that 
generates  and  compiles  a set  of  CGI  programs  for  the  quiz.  More  exactly,  two  CGI  programs  are  generated  for  each 
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question  - one  to  present  the  question  and  one  to  evaluate  the  answers.  The  code  of  the  original  question  is 
included  into  both  of  these  programs.  Thus,  the  same  code  provided  by  the  teacher  is  used  for  both  - presenting  the 
question  and  evaluating  the  answer.  If  the  author  will  decide  to  changed  or  update  the  quiz,  the  compile  command 
should  be  used  to  re -generate  the  quiz. 

QuizPACK  authoring  aproach  allows  the  authors  to  create  and  compile  quizzes  remotely  using  Telnet  and  FTP  tools. 
Note  that  the  authoring  side  of  QuizPACK  does  not  use  traditional  Web  form-based  authoring  approach,  but  relies  on 
generic  text  editing  and  communication  tools.  For  teachers  of  programming  subjects  it's  a benefit,  not  a shortcoming.  A 
lull-featured  randomized  quiz  can  be  produced  very  fast,  with  the  use  of  familiar  tools  on  their  preferred  platform  and  with 
minimal  authoring  efforts  over  creating  the  code  foi  questions.  Teachers  can  p ublidh  the  questions  frcnfi  their  hcnfies  and 
then  can  Just  e-mail  the  students  to  take  the  quiz.  They  don’t  need  to  come  to  the  class  and  distribute  the  papers  to  the 
students  to  take  the  quiz. 
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Figure  3:  The  structure  of  a database  table  with  the  quiz  results. 

QuizPACK  use  a specific  http-based  protocol  to  communicate  the  results  of  the  student  work  with  every  question 
to  a server  specified  by  a teacher.  We  have  successfully  used  this  http-based  approach  to  support  communication 
between  distributed  components  of  educational  software  in  our  previous  work  [Brusilovsky,  Ritter  & Schwarz  1997]. 
With  this  approach  any  http  server  can  be  used  to  log  or  store  the  results  of  the  student's  work.  For  the  needs  of  our 
own  course  we  have  implemented  a simple  storage  server  using  Microsoft  Internet  Server  and  Microsoft  Access 
database.  The  database  stores  results  of  the  student's  work  in  a very  simple  format  (Figure  3).  This  is  a convenient 
solution  since  allows  to  produce  easily  a variety  Active  Server  Pages  (ASP)  scripts  for  creating  student  progress  reports. 
With  these  reports,  teachers  can  evaluate  the  performance  of  the  a class  or  a particular  student  on  a quiz  or  question 
level  by  calling  the  URL  for  one  of  the  ASP  scripts. 

Evaluation  of  the  System  ' 

During  the  Fall  semester  of  2002  we  have  performed  a formative  evaluation  of  QuizPACK  in  a real  university  course 
"Data  Structures  and  Programming  Techniques"  at  the  University  of  Pittsburgh.  The  course  was  based  on  C 
programming  language.  During  the  first  part  of  the  course,  we  have  use  QuizPACK  in  in-class  assessment  mode  using 
wireless  handheld  computers.  During  the  second  part  of  the  course  we  have  used  the  system  in  out-of-class  self- 
assessment  mode  using  regular  desktop  computers.  The  analysis  of  the  in-class  use  of  the  system  is  beyond  the  scope 
of  this  paper  since  the  results  were  significantly  influenced  by  the  use  of  wireless  computers.  Below  we  discuss  some 
results  of  the  evaluation  of  QuizPACK  in  the  self-assessment  mode. 

QuizPACK  was  used  to  serve  self-assessment  quizzes  of  5 to  10  questions  each  for  almost  every  lecture  of  the 
second  part  of  the  course.  The  questions  were  prepared  to  let  the  student  practice  the  knowledge  of  programming 
constructs  and  data  structured  introduced  at  the  corresponding  lecture.  In  addition  to  self-assessment  computer  quizzes 
we  have  also  used  regular  paper-and-pencil  assessment  quizzes  at  the  beginning  of  every  lecture.  A few  weeks  before 
the  end  of  the  course  we  have  prepared  a short  9-question  assessment  tool  to  collect  the  students  feedback  about 
QuizPACK.  The  students  were  informed  about  the  questionnaire  and  were  encouraged  to  fill  it  in.  The  participation  in  the 
study  was  voluntary.  All  participants  were  rewarded  by  thee  extra  credit  points.  In  total,  27  students  of  40  choose  to 
participate.  It's  important  to  note  that  all  students  participated  in  the  study  have  taken  at  least  three  self-assessment 
quizzes  over  the  duration  of  the  course. 
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Overall,  the  students  have  got  a very  positive  feeling  about  the  self-assessment  questions.  48%  of  respondents 
though  that  it  "can  significantly  help",  37%  though  that  it  "can  help",  15%  thought  that  it  "can  sometimes  be  of  help" 
and  none  found  them  useless.  The  ability  to  take  a parameterized  quiz  several  times  was  also  highly  evaluated.  56% 
found  this  feature  "very  useful",  26%  found  it  useful,  19%  found  it  "sometimes  useful":  and  none  "useless". 

The  last  of  the  questions  in  the  questionnaire  allowed  the  students  to  provide  an  unstructured  feedback.  We 
have  specially  welcomed  critical  feedback  and  suggestions.  While  we  have  not  asked  about  praise,  many  students  has 
provided  a very  encouraging  feedback  from  unspecific:  "The  self-assessment  quizzes  were  of  great  help"  to  very 
specific:  "The  quizzes  helped  me  to  figure  out  link  list  implementation".  The  students  also  appreciated  the  parameterized 
nature  of  the  quizzes:  " You  can  also  take  the  self-quizzes  over  and  over  again  unlike  the  [paper-and-pencil]  in  class 
quizzes.  You  learn  more  for  taking  different  quizzes  over  and  you  din't  have  to  stress  yourself  over  getting  the  answer 
right  the  first  time".  The  students  have  also  pointed  out  several  problems  and  generated  a number  of  suggestions.  One 
of  the  most  often  mentioned  suggestion  was  to  provide  self-assessment  quizzes  for  every  single  course  lecture,  not  just 
for  a subset  of  them:  "There  were  a few  areas,  I had  trouble  with,  but  there  were  no  quizzes  to  go  along  with  them".  Many 
students  also  wished  the  tool  not  only  reported  the  write  answer,  but  also  provide  a detailed  explanation  of  how  this  was 
obtained.  We  think  that  such  an  explanation  can  be  generated  by  an  explanatory  visualization  tool  are  planning  to 
explore  this  direction.  Finally,  a number  of  students  commented  that  they  prefer  a quiz  format  where  all  questions  are 
shown  on  one  page  (as  in  paper-in-pencil  quizzes).  This  format  allows  them  to  see  the  whole  set  of  questions,  take  them 
in  arbitrary  order,  and  re -consider  the  answers  before  submitting  the  whole  set. 

Summary 

We  have  reported  an  approach  and  a tool  QuizPACK  that  enable  teachers  to  produce  easily  parameterized  quizzes  for 
programming  subjects.  Parameterized  quizzes  greatly  increase  the  productivity  of  a teacher  enabling  them  to  produce  a 
range  of  questions  where  regular  tool  allows  producing  only  one  question.  Parameterized  quizzes  prevent  cheating  in  an 
assessment  mode  and  allow  the  students  to  take  the  same  question  over  and  over  in  the  process  of  mastering  the 
subject.  The  tool  was  used  in  an  undergraduate  programming  course  and  was  very  positively  evaluated  by  the  students. 
We  are  planning  to  continue  the  work  in  this  direction  taking  into  account  students  suggestions  generated  during  the 
first  formative  evaluation. 
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Abstract:  In  the  evolving  Information  Society  (IS),  programming  and  computing  skills  are  becoming  important  in 
educational,  professional  and  personal  endeavors  and  constitute  an  integral  part  of  computer  science  education.  In 
the  current  programming  languages,  programming  constructs,  semantic  concepts,  and  syntactic  elements,  are  based 
on  English  and  implemented  using  ASCII  character  sets,  seriously  limiting  the  universal  access  to  programming 
and  computing  skills.  Especially,  native  users  (e.g.,  non-English  speaking  students,  adults,  linguistic  minorities  etc), 
encounter  enormous  difficulties  in  understanding  and  translating  their  programming  plans  into  the  syntax  and 
semantics  of  a programming  language.  Authors  have  designed  and  implemented  a universal  design  methodology 
and  experimental  programming  language  - EQUAL  and  its  compiler/interpreter  and  an  integrated  programming 
environment,  in  which,  native  users  can  write  and  execute  programs  in  their  native  ''customized'  programming 
languages.  EQUAL  provides  a unique  opportunity  to  native  users  for  learning  programming  and  may  be  used  as  a 
scaffolding  tool  for  teaching  basic  concepts  of  programming  and  computing. 

Introduction 

In  the  evolving  Information  Society  (IS),  programming  and  computing  skills  are  becoming 
important  in  educational,  professional  and  personal  endeavors  and  constitute  an  integral  part  of  computer 
science  education.  With  globalization  and  rapid  developments  in  Information  Technologies  (IT),  the 
majority  of  Internet  users  do  not  use  English,  native  users’  issues  may  become  serious  because  the  vast 
majority  of  the  world’s  population  who  do  not,  and  will  not  in  the  foreseeable  future  speak  English  will  be 
excluded  from  the  Information  Society.  Students  and  people  lacking  programming  and  computing  skills 
may  become  a serious  individual  and  social  problem  [Dyson,  1997]  and  in  the  worst  case,  may  lead  to  - 
Internet  Apartheid  [Shneiderman,  2000].  Native  users  referred  to  here  include  non-English  based 
educational  background  students  (e.g.,  primary  and  high  schools,  colleges  and  universities),  adults, 
indigenous  societies,  people  with  special  needs,  linguistic  minorities  from  developing  and  underdeveloped 
countries  from  Asian,  the  Middle  East,  Eastern  European,  African  and  South  American  countries  or 
geographical  regions.  For  these  native  users,  it  is  unrealistic  and  impractical  to  learn  and  adapt  to  English 
based  paradigm  of  HCI  interactions  and  communications.  In  particular,  millions  of  native  students,  for  some 
reasons  (e.g.,  political,  social,  etc.)  are  compelled  to  learn  in  native  medium,  have  no  alternative  solutions  to 
learn  the  IT  skills  and  programming  expertise. 

Policymakers,  educators  and  government  administrators  from  these  countries  have  realized  the 
potential  of  IT  and  their  impact  on  the  emerging  information  societies.  Even  with  special  initiatives,  free 
hardware  or  subsidized  access  to  Web,  Universal  Access  (UA)  to  appropriate  linguistic  and  cultural  content, 
textual  information  structures  and  programming  languages  has  been  a serious  impediment  to  the 
development  of  IT  skills  and  effective  E-applications  (e.g..  E-democracy,  E-Govemance,  E-Leaming  etc). 
Unfortunately,  simple  textual  information  structures  (e.g.,  email  addresses,  domain  names,  operating  system 
components),  novice  programming  languages  (e.g.,  LOGO,  BASIC  etc.)  and  textual  languages  (e.g., 
command,  query,  programming  etc.)  are  not  available  and  useful  to  the  native  users. 

In  general,  from  a technical  point  of  view,  there  is  a critical  need  of  Enabling^  Scaffolding,  and 
Assistive  mechanisms  to  cater  to  the  diverse  requirements  of  native  users.  Enabling  mechanisms  are  mainly 
concerned  to  the  enabling  the  native  users  to  interact  in  native  languages  and  socio-cultural  environments 
with  the  computers.  Scaffolding  mechanisms  are  necessary  to  help  and  assist  the  native  users  to  join  the 
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main  stream  of  English  based  paradigm  of  HCI  interactions  and  programming.  Special  assistive  mechanisms 
are  necessary  to  the  native  users  with  special  needs.  We  consider  universalization  textual  information 
structures  and  textual  languages  as  fundamental  strategy  to  support  enabling,  scaffolding  and  assistive 
technologies  for  the  diverse  needs,  preferences  and  requirements  of  native  users.  With  a long,  term  goal  of 
IT  Empowerment  and  Life  Long  Learning,  we  take  the  proactive  approach  of  universalization  of  textual 
information  structures  and  textual  languages,  and  as  a step  towards  a Inclusive  Society,  we  recommend  that, 
the  '"digital  divide""  has  to  be  minimized  by  developing  sustainable,  enabling  and  scaffolding  technologies  by 
adopting  universal  design  perspectives  towards  the  design  and  implementation  of  textual  information 
structures,  programming  languages  and  computing  environments. 


languages  and  computing  skills  with  English  as  "programming paradigm"  will  seriously  limit  the  universal 
access  to  IT  technologies  and  programming  skill.  Because  of  historical  reasons  the  majority  of  programming 
languages  and  programming  paradigms  (e.g.,  declarative,  procedural,  functional,  object  oriented  etc)  are 
mainly  based  on  English  semantics  and  syntax  and  implemented  using  ASCII  character  sets.  Especially,  the 
native  users  such  as  students  and  adult  users,  whose  medium  of  education  and  instruction  is  in  their  native, 
natural  languages  and  socio-cultural  environments,  encounter  enormous  difficulties  in  understanding  and 
translating  their  programming  plans  into  the  English-based  semantics  and  syntax  of  a programming 
language.  In  the  new  technology  adaptation  for  Educational  Computing  and  Learning,  in  particular,  with 
respect  to  programming  languages,  non-availability  and  non-accessibility  of  programming  languages  and 
computing  environments  (e.g.,  integrated  editors,  debuggers  etc)  and  language  processors  (e.g.,  interpreters, 
compilers,  translators  etc)  constitute  a major  stumbling  block  to  native  users  [Rogers,  2000],  Other  factors, 
such  as  lack  of  native  user  interfaces  and  interactions,  linguistic  barriers,  complex  social  and  cultural  factors 
will  continue  to  hinder  the  effective  learning  of  programming  and  computing  skills.  We  also  believe  that 
user  interfaces  to  systems  for  programming  and  computing  environment  can  critically  influence  the 
development  of  programming  skills.  Hence,  a wide  range  of  native  user  programming  languages  and 
computing  issues  must  be  included  in  the  universal  design  and  universal  access  to  programming  languages, 
language  processors  and  computing  environments. 

In  this  paper,  we  mainly  focus  on  on-going  work  research  work  related  to  the  universal  usability 
issues  of  programming  languages,  language  processors  and  computing  environments  from  native  users' 
perspectives  with  more  emphasis  on  semantics  and  syntax  i.e.,  notational  aspects  of  programming  languages 
and  programming  constructs.  We  also  discuss  a universal,  inclusive  design  methodology  of  programming 
languages,  language  processors  and  computing  environments.  Furthermore,  it  will  contribute  to  the  better 
understanding  of  native  user  requirements,  universal  design  of  programming  languages  and  computing 
environments,  and  implications  to  scaffolding  and  teaching  tools. 

Native  Users:  A Profile 

The  research  studies  on  needs  and  requirements  of  native  users  are  scarce  and  poorly  understood. 
Within  the  scope  of  our  research  work  concerned  with  universal  design  methodologies  and  universal  access 
to  textual  languages  (e.g.,  command,  query,  programming  etc)  with  a long  term  research  goal  of 
'Tnformation  Technology  for  All  (IT4ALL/\  we  define  and  focus  our  studies  to  a diverse,  global,  end-user 
populations  called  ""native  users'".  In-house  studies  on  usability  and  universal  accessibility  of  textual 
languages  and  novice  programming  issues  were  conducted  with  the  help  of  researchers,  HCI  experts,  native 
users  and  students  with  diverse  multilingual  and  multicultural  backgrounds.  From  these  preliminary  studies, 
in  general,  native  users  may  be  defined  as  end-users,  including  users  with  special  needs  with  one  or  more 
characteristics  such  as:  (a).  They  prefer  or  use  their  native,  natural  language-like  constructs  for  human- 
computer  interactions  and  programming,  (b).  They  have  minimal  or  no  English  knowledge,  (c).  Their 
medium  of  education  and  instruction  is  in  their  native,  natural  languages  and  socio-cultural  environments, 
(d).  They  are  novices  or  posses  minimal  computing  skills,  (e).  Native  student  communities  would  like  to 
acquire  not  only  information  manipulation  skills  but  also  skills  of  programming,  (f).  Native  users  may  also 
include  people  with  special  needs,  (e.g.,  visually  challenged  users  prefer  to  use  native  Braille), 

Native  User  HCI  Model  and  Programming  Language  Issues 

In  general,  programming  is  the  task  of  mapping  the  mental  plans  and  program  compositions  into 
language  constructs,  i.e.,  semantics  and  syntax  of  a computer  language  to  achieve  a particular  computing 
task.  For  historical  reasons,  the  majority  of  programming  languages  and  paradigms  are  derived  from  English 
semantics  and  syntax  and  implemented  using  ASCII  character  sets.  A native  user  HCI  model  of  interaction 


and  communication  between  the  native  user  and  the  computer  for  programming  tasks  can  occur  at 
conceptual,  semantic,  syntactic,  and  lexical  levels.  At  conceptual  level  the  "meaning"  of  the  task  model 
should  match  the  native  user's  world  model,  and  here  reside  the  concepts  of  programming,  programming 
language  features,  relationships,  and  operations.  At  the  conceptual  level,  the  technology,  programming 
paradigms,  metaphors  are  new  and  unfamiliar  to  the  majority  of  native  users.  At  the  semantic  level,  the  ideas 
from  the  conceptual  level  are  represented  by  means  of  a programming  language  placed  in  the  context  of  a 
computing  task.  The  abstract  operations  from  the  conceptual  level  are  fully  defined  in  terms  of  objects, 
functions  and  programming  constructs  of  a programming  language.  At  the  semantic  level,  the  semantics  of 
programming  do  not  match  well  with  the  natural  semantics  of  native  users'  languages.  At  the  syntactic  level. 
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elements  and  program  construction.  At  the  lexical  level,  the  primitive  elements  such  as  tokens  and  literals  of 
a programming  language  are  used  to  write  a program  to  achieve  a computing  task.  The  ''grammar"  at  the 
syntax  level  is  highly  inflexible  to  the  requirements  of  the  native  users  and  the  ASCII-based  lexical  tokens  at 
the  lexical  level  do  not  cover  the  majority  of  native  user  languages  and  cultural  conventions. 

In  other  words,  the  majority  of  programming  languages  "do  not  speak  the  language  " of  the  native 
users  and  seriously  hamper  understandability,  memorability,  leamability  and  ease  of  use  of  programming 
languages.  The  native  users  are  compelled  to  use  semantic  and  syntax  concepts  that  are  not  compatible  and 
comprehensible  to  the  majority  of  native  users.  Hence,  current  programming  languages  does  not  provide 
flexible  approaches  and  alternative  opportunities  to  native  users  to  learn  programming.  In  general, 
programming  is  difficult,  but  native  users  encounter  additional  difficulties  at  conceptual,  cognitive,  semantic 
and  syntactic  levels  in  translating  the  programming  plans  into  the  formal  syntax  of  a programming  language. 
Thus,  we  need  not  only  syntactic  but  also  semantic  and  cognitive  adaptations  in  programming  languages  and 
computing  environments. 

In  general,  from  the  native  users’  perspective  as  well  as  from  the  preliminary  usability  studies,  the 
major  issues  are: 


• Lack  of  Computing  Resources:  The  native  user  computing  resources  are  scare  and  poorly  understood. 
The  computing  resources  include  programming  languages,  language  processors,  computing 
environments,  teaching  and  learning  resources  and  materials.  There  is  a critical  need  of  production, 
publication  and  dissemination  of  computing  resources  for  the  native  users.  Other  serious  issues  are 
complex  computing  concepts,  unfamiliar  metaphors,  unnatural  programming  paradigms,  lack  of 
integration  of  pedagogical  concerns,  and  poor  training  and  teaching  resources. 

• Syntactic  Inflexibility:  The  majority  of  syntactic  elements  or  vocabularies,  punctuation,  programming 
conventions  and  cues  of  programming  languages  is  based  on  English  syntax,  and  conventions  and 
implemented  using  ASCII  character  sets.  Using  ASCII  character  set,  seriously  hampers  the  universal 
access  to  programming  at  syntactic  and  lexical  levels  for  non-Roman  based  scripts  and  complex  native 
language  diacritics. 

• Semantic  Mismatch:  The  semantics  of  programming  constructs  do  not  match  well  with  the  concepts  and 
semantics  of  native  user  natural  languages.  For  example,  many  programming  semantics  of  input/output 
statements  (e.g.,  READ,  WRITE,  INPUT,  OUTPUT,  PUT,  GET,  PRINT  etc)  have  a generic  form 
<VERB>  <objects>.  Many  native  users'  languages  (e.g.,  Asian,  Arabic,  Indian  etc)  have  the  generic 
form  <objects>  <VERB>  and  sometimes  freely  mixes  both  the  formats  in  general  communication. 
Similarly,  the  IF-THEN  conditional  statement  and  its  variants  have  a generic  structure  IF  <condition> 
THEN  (statements),  but  these  programming  constructs  have  diverse  native  interpretations.  In  novice 
programming  studies,  it  is  observed  in  [Bonar  and  Solo  way  1985,  Ebrahimi,  1994]  that,  the  majority  of 
semantic  errors  arise  due  to  the  semantic  mismatch  and  novice  users  relaying  on  preconceived 
knowledge  and  translating  the  programming  plans  using  their  natural  language  semantics. 


The  visual  programming  systems,  programming  by  demonstration  and  form-based  programming 
may  be  used  to  address  some  of  the  native  users  programming  issues  by  providing  native  ''customized"  GUI 
components.  Visual  languages  for  programming  may  help  and  highlight  only  a few  aspects  of  programming 
and  manipulation  of  information  structures.  But,  it  is  observed  in  [Gilmore  and  Green  1984,  Green  and 
Petre  1992]  that  the  textual  languages  are  more  useful  in  learning  the  skills  of  programming  such  as 
comprehension,  creation,  documentation,  modification  and  debugging  and  learning  programming 
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EQUAL  Design  Principles  and  Rationale 

EQUAL  {Easy,  Quick,  Universal,  Accessible  Language)  is  a universal,  human-centric,  inclusive 
design  methodology  for  the  design,  implementation  and  evaluation  of  textual  languages  for  command,  query, 
programming  tasks  and  computing  applications  for  native  users,  including  native  users  with  special  needs 
[Patil  et  al.,  2001a,  Patil  et  al.,  2001b]. 

Since,  native  users  are  novices,  the  research  studies  and  empirical  results  in  the  fields  of  Psychology 
of  Programming  /Hoc  and  Nguyen,  1990,  Dyck  and  Mayer,  1985,  Rogalski  and  Samurcay,  1994]  and 
Novice  Programming  [Mayer,  1981,  Dayman  and  Mayer,  1983,  Bonar  and  Soloway  1985,  Boulay  1989] 
provide  valuable  insights  and  may  be  extended  and  integrated  into  the  universal  design  of  novice 
programming  languages  and  computing  environments. 

It  is  observed  that  natural  language-like  constructs  and  commands  are  effective  in  the  design  of  user 
interactions  with  the  computer  systems  [Ledgard  et  al.,  1980,  Landauer  et  al.,  1983,  Biermann  & Ballard, 
1983].  Even  the  recent  studies  [Pane  et  al.,  2001]  demonstrate  the  concept  of  ""natural  programmin^\ 
Natural  programming  does  not  mean  that  the  native  users  should  use  their  natural  languages,  but  the 
languages  of  programming  and  interactions  should  be  as  ''naturaP  as  possible  and  able  to  ''mentalize*'  the 
interactions,  learn  quickly  and  use  effectively  - support  of  high  degree  of  naturalness. 

Cognitive  processes  and  structures,  program  constructs  play  a vital  role  in  problem  solving,  programming 
and  comprehension  [Hoc  and  Nguyen,  1990,  Rogalski  and  Samurcay  1994].  Such  an  understanding  would 
lead  to  better  programming  languages  whose  syntactic  structure  more  closely  reflects  internal  semantic 
structures  thereby  easing  the  process  of  programming  - support  closeness  of  mapping. 

The  process  of  translating  a programming  plan  into  one  that  is  compatible  with  the  computer  is  a highly 
cognitive  activity.  The  programming  language  should  minimize  the  difficulty  of  this  translation  by  providing 
high  level  primitives  that  match  the  operators  in  the  programming  plan  and  composition  - exploit  cognitive 
dimensions  of  notations  and  information  structures. 

The  characteristics  of  the  language  and  notational  aspects  play  a vital  role  in  acquiring  the  skills  of 
programming.  Avoid  unnecessary  and  confusing  operators,  cryptic  programming  constructs,  complex  data 
structures  and  control  statements  - keep  it  simple  and  effective.  From  the  basic  guidelines  and  principles  of 
usability  engineering,  the  programming  languages  and  computing  resources  should  satisfy  the  needs, 
preferences  and  requirements  of  native  users  - speak  native  users'  languages. 

Salient  Features  of  EQUAL  Language 

EQUAL  is  a simple,  experimental,  novice  programming  language  mainly  designed  by  taking  into 
consideration  the  special  needs  and  requirements  of  the  native  end-users,  including  visually  impaired  users. 
The  design  and  rationale  of  the  EQUAL  language  is  discussed  more  detail  in  [Patil  et  al.,  2001a].  Here  we 
highlight  the  salient  features: 

• EQUAL  language  features  are  deliberately  kept  simple  but  have  important  features  of  programming  to 
teach  novice  programming.  Syntax  is  simple,  readable,  understandable  and  consistent  with  native  user’s 
knowledge.  EQUAL  supports  simple  data  types,  commands  and  statements  and  control  constructs. 

• The  language  ''vocabularies"  and  programming  paradigms  are  highly  customizable  to  the  universal 
needs  and  preferences. 

• The  computing  environment  is  ""internationalized'  and  supports  divers  time,  date,  currency,  calendar, 
numeral  standards  and  formats. 

• The  visually  impaired  native  users  can  program  using  native  Braille  codes  using  suitable  transcription 
mechanisms.  The  programming  constructs  and  interactions  (e.g.,  messages,  errors,  results  etc)  may  be 
extended  and  enhanced  with  digitized  speech  and  text-to-speech  technologies  in  users’  native  languages. 

EQUAL  Programming  Constructs 
Variable  Names,  Values  and  Operators 

The  empirical  studies  in  [Carroll  1978,  Teasley  1994]  have  demonstrated  that  meaningful  variable 
names  and  numerical  values  improved  program  comprehension  and  facilitate  learning.  For  example,  in  a 
simple  statement  AreajofjCircle:  = 2 * PI  * Radius;  may  represented  in  EQUAL  for  Chinese  user  (Refer  Fig 
Id).  In  Arabic  (Refer  Fig.  lb),  programs  are  written  and  displayed  in  bi-directional  format  and  prefer  ’five 
pointed  star’  or  ’X’  for  multiplication  operator.  The  assignment  operator  ’-  may  be  represented  by  ’ ’.  The 
general  interpretation  of  statement  termination  symbols  ’;’  varies  considerably  (e.g.,  ’;’  is  used  for  question 
mark  in  Greek).  The  enclosing  the  string  constants,  also  vary  considerably  (e.g.,  «string  constant))  in  French, 


„string  constant^  in  German,  etc.).  The  relational  and  logical  operators  are  quite  confusing  and  native  users 
misinterpret  their  meaning  and  it  is  mainly  due  to  lack  of  knowledge  of  Boolean  algebra.  EQUAL  universal 
design  framework  supports  these  notational  flexibility  and  adaptations,  and  EQUAL  helps  in  easy 
comprehension  and  facilitates  effective  learning.  These  simple  adaptations  will  reduce  cognitive  loads,  avoid 
unnecessary  frustrations  and  motivate  native  students  to  learn  programming  skills. 


Command  Names  and  Simple  Statements 

Both  formal  and  informal  observational  analyses  repeatedly  identified  names,  naming  and  structural 
contexts  as  an  important  practical  problem  [Landauer  at  el.,  1983].  The  majority  of  assignment  statements, 
input/output  statements  are  simple  imperative  statements  or  commands  and  often  take  the  form  of  VERB 
<Gbjccts>  or  <Ghjccts>  VERB  or  mixed  format.  EQUAL  supports  the  above  formats  and  exploits  the 
natural patterns''  of  commands,  programming  constructs  and  information  structures  that  are  consistent  with 
native  languages  and  general  communication  conventions. 
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Figure  1:  Cropped  screen  shots  of  EQUAL  programs  in  German  (a),  Arabic  (b),  (c)  Zimbabwean-Shona 

and  (d)  Chinese  versions. 

Control  Statements 

Control  strategies  and  looping  constructs  are  very  difficult  concepts  and  demand  high  cognitive 
loads  [Soloway  et  al.,  1983].  It  may  be  observed  that,  just  by  providing  equivalent  control  and  looping 
constructs  in  native  programming  language  constructs  may  be  insufficient  and  inadequate,  some  time  quite 
misleading  and  confusing  from  the  preconceived  knowledge  [Bonar  & Solo  way,  1985].  In  IF-THEN 
statement  or  its  variants,  it  is  observed  that  [Pane  et  al.,  2001]  end-users  use  THEN  for  ’"sequencing”  which 
contradicts  the  meaning  of  ’’consequently”  in  the  formal  languages.  In  REPEAT  or  WHILE  control 
constructs,  native  users  fail  to  understand  the  terminating  or  exit  conditions.  One  effective  design  strategy  is 
to  ’’map  closely”  the  natural  semantics  of  native  users'  languages  and  concepts  onto  the  formal  semantics  of 
programming.  In  the  present  state  of  research,  explicit  training  is  the  best  way  to  circumvent  these  difficulties. 


EQUAL  Implementation  and  Computing  Environment 

The  basic  design  principle  of  the  EQUAL  language  design  and  development  environment  is  to 
separate  the  formal  grammar,  semantic  concepts,  syntactic  and  lexical  components  and  represent  them  in  a 
language  independent,  user  defined  profiles  and  culture  neutral  representations.  From  these  design  constrains 
and  specifications,  EQUAL  design  and  development  environment  provides  an  integrated  platform  to  generate 
automatically  or  semi-automatically  language  components,  scanners,  parsers  and  compilers/interpreters  [Patil 
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et  al,  2001b].  Special  morphological  and  linguistic  knowledge  is  necessary  to  process  complex  languages. 
EQUAL  uses  Unicode,  Java,  XML  and  Compiler  technologies  and  functional  components  may  be  deployed 
as  standalone  application  or  client-server  architecture. 

The  EQUAL  computing  environment  consists  of  Code  Window y Input  Window ^ Output  Window,  Error 
Window  and  Help  Window,  The  native  users  can  write  EQUAL  programs  using  Unicode  compatible  text 
editors  and  submit  them  for  the  compilation.  The  syntactic  and  semantic  errors  are  displayed  in  the  Error 
Window  with  necessary  information  to  facilitate  easy  correction.  In  case  of  an  error-free  EQUAL  program, 
the  interpreter/compiler  will  execute  the  program  on  Java  Virtual  Machine  (JVM).  The  Input  Window  expects 
the  inputs  to  the  program  and  generates  the  outputs  in  the  Output  Window,  On-line  help  information  and 
documentation  is  available  in  iht  Hel^  Window,  The  error  messages,  dialoguesj  help  information,  training  and 
learning  resources  are  hand-coded  and  stored  in  separate  files  in  native  user  preferred  languages.  EQUAL  is 
flexible  enough  to  incorporate  pedagogical  and  learning  modules. 

Conclusions  and  Future  Work 

In  this  paper,  we  discussed  the  universal  accessibility  and  usability  issues  of  textual  information 
structures,  programming  languages  and  computing  environments  from  native  users’  perspective  and 
developed  a universal  design  framework  and  methodology  for  the  design,  implementation  and  evaluation  of 
programming  languages  for  computing  tasks  and  applications.  We  highlighted  universalization  of  textual 
languages  as  a basic  strategy  for  the  enabling,  scaffolding  and  assistive  technologies  with  necessary 
conceptual,  semantic,  syntactic  and  lexical  adaptations  in  computer  languages  and  computing  environments. 

We  also  discussed  the  salient  features  of  the  EQUAL  language  in  a few  representative  native  user  languages 
such  as  Arabic,  Chinese,  Zimbabwean  and  German.  The  EQUAL  design  methodology  and  computing 
environment  is  also  flexible  to  incorporate  native  users'  versions  of  Braille  codes,  digitized  and  text-speech 
technologies.  We  believe  that,  the  EQUAL  language  and  EQUAL  computing  environment  provide  a unique 
opportunity  to  the  native  users  to  learn  programming  and  computing  skills.  We  highlighted  the  general 
principles,  guidelines  and  design  methodology  of  EQUAL,  but  they  are  also  applicable  in  different  textual 
languages,  programming  paradigms,  and  textual  applications.  The  experimental  work  is  in  progress  and  the 
implementation  of  complex  data  types,  control  constructs,  usability  studies  and  critical  evaluations  are 
planned  in  the  future  work. 
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Abstract:  This  paper  describes  the  pilot  portal  project,  involving  development  of  public  installations 
in  two  distant  locations,  which  will  act  as  portals  into  each  other.  The  current  state  of  the  world  is 
calling  for  new  means  of  world-wide  communication  and  understanding.  The  Portal  Project  is  a work 
in  progress.  Its  objectives  are  to  connect  people  and  places,  and  to  encourage  broad  cultural  and 
environmental  observation  and  reflection.  Portals  will  be  supported  by  a network  of  cameras  streaming 
live  video  which  is  received  and  projected  by  plasma  screens  in  other  locations  and  collectively 
displayed  at  a website.  The  video  installations  will  promote  awareness  and  communication  among 
participating  foreigners. 


Introduction 


The  Portal  Project  will  bring  together  people  and  places  in  a way  that  has  become  possible  only  through  the 
advent  of  modem  communication  technologies,  via  live  video  web  streams.  Unlike  more  traditional  art  projects  that 
usually  happen  at  a specific  place  and  time,  for  those  particular  observers  who  choose  to  participate,  the  Portal  Project 
installation  creates  a frame w>rk  of  experience  to  facilitate  cross-cultural  interaction  at  the  most  basic,  passive,  everyday 
level. 

New  York,  US  11:20  Vienna,  Austria  5:20 


Imagine  this:  While  taking  a break  from  work  at  1 1 :20  in  New  York,  right  next  to  the  fed  ex  truck,  you  will  be 
able  to  look  into  Vienna,  where  idle  Austrians  enjoy  an  afternoon  stroll.  You  may  even  make  a friend  and  arraign  to 
meet  on  sixty-first  and  Lexington  in  New  York  and  Stephansplatz  in  Vienna. 

Rationale 

Cities  are  complex  environments,  comprised  of  people,  social  interactions,  communities,  architecture, 
invention,  history,  etc,  offering  offer  rich  learning  opportunities.  However,  our  everyday  lives  make  the  meanings  of 
the  city  and  the  dilemmas  coexisting  in  it  remain  unseen.  We  become  accustomed  to  them  as  part  of  our  visual 
environment  and  they  no  longer  provoke  our  reflection  (Greene,  1978).  Art,  and  urban  installations,  have  the  potential 
to  make  these  resources  more  evident  for  us. 
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The  installation  inserts  itself  in  the  city  landscape  intercepting  people  in  their  daily  existence.  It  calls  for  being 
seen  and,  in  the  same  process,  makes  the  city  itself  visible  by  its  inhabitants.  It  provides  an  opportunity  to  bring  the 
background  to  the  stage.  By  including  the  mediational  device,  the  images  of  a spot  in  a city  get  the  status  and  attention 
of  a spectacle.  This  project,  inspired  by  Kit  Galloway  and  Sherrie  Rubinowitz’s  1974  installation  “Hole  in  Space,*’  allows  for 
taking  a look  at  the  lives,  faces,  spaces,  and  reactions  of  the  others;  people  from  another  city  and  culture.  This  is  a 
chance  for  seeing  but  also  a chance  for  awareness  of  being  seen.  And  being  seen  in  a way  that  we  don’t  know  ourselves 
- by  other  eyes,  by  other  culture,  as  others. 

New  York  and  Vienna  are  dissimilar  and  alike.  Almost  the  same  neutral  space  of  the  central  touristy  area 
exists  in  every  city,  an  equivalent  functional  place,  crossed  by  city  people  with  similar  enneems,  walking  on  avenues 
and  squares.  Big  cities’  spaces  are  common  to  all  big  cities  around  the  world  in  globalization,  where  people  are  always 
in  transit  with  their  bodies  and  sensibilities. 

Our  present  time  highlights  the  need  for  cultures  to  find  new  ways  of  communicating  and  of  seeing  each  other 
and  themselves.  American  people  and  all  cultures  are  subject  to  stereotyped  perceptions  of  the  others,  of  us,  and  of 
conflict--  in  which  the  media  plays  a mayor  role.  The  Portal  Project  intends  to  provide  a different,  less  censored  and  less 
edited  insight  opportunity  for  understanding.  It  intends  to  provoke  questioning,  confusion,  reflection,  awareness.  And 
in  this  process  of  identification  and  differentiation,  it  intends  to  mobilize  citizens  of  distinct  cultures  in  active 
explorations  of  the  complex  world  community  within  which  we  are  all  connected.  If  we  are  successful  with  this  pilot 
connection,  we  will  expand  to  other  connections  between  locations  of  similar  and  dissimilar  environments  and  cultures. 


Design  and  Technology 

Much  like  backlit  print  advertisements  at  bus  stops,  the  Portal  Project  uses  a metallic  frame  to  become  integral 
part  of  the  city  landscape  and  our  daily  visual  experience  as  inhabitants  of  our  cities.  Instead  of  print,  however,  this 
metallic  structure  contains  a 50”  plasma  television  screen.  The  moving  images  are  a live  video  feed,  streamed  over  the 
Internet,  from  an  identical  box  located  in  Vienna.  Both  boxes,  hidden  on  top  of  the  structure,  contain  a network  video 
camera  connected  to  the  Internet  via  a high-speed  connection.  These  boxes,  therefore,  become  windows  into  a different 
city  and  culture.  Like  a window,  they  provide  a view  into  a different  environment,  as  well  as  becoming  part  of  the 
familiar  one. 


Complementary  WWW  Site 

The  website  will  include  the  live  images  of  the  two  locations  one  next  to  the  other  on  the  screen.  The  two  sites 
and  the  reactions  of  people  looking  at  the  screens  may  be  seen  by  any  website  visitor.  The  site  will  include  a discussion 
forum  and  chat  room  for  the  observers  to  discuss  these  views.  Links  to  useful  resources,  such  as  online  translation 
dictionaries  and  cultural  historical  references,  will  be  provided.  An  archive  of  selected  video  clips  and  discussions  will 
create  a record  of  the  participation  of  the  users  and  viewers  in  the  installation.  This  archive,  as  well  as  the  experiences 
of  comparing  the  two  screens  and/or  of  being  in  the  location  of  one  city  will  be  the  basis  for  educational  activities 
designed  for  school  and  museum  groups. 

Future  Directions 

The  project  is  in  its  early  stages.  Our  near  future  goals  are  to  establish  corporate,  government,  school  and 
museum  partnerships.  We  are  seeking  partnerships  with  organizations  wishing  to  promote  their  products  by  donating 
their  use.  We  are  looking  for  the  provision  of  space  in  the  ideal  locations,  balancing  public  access  and  security,  in  each 
city.  We  are  also  seeking  partnerships  with  schools  and  museums  to  collaborate  in  the  development  of  supplementary 
educational  materials  provided  on  the  web  site.  This  methodology  can  be  exported  and  expanded  by  altering  the 
locations  and  connections  in  the  network.  For  example,  other  projects  nay  be  intended  to  remind  city  folk  of  non- 
human aspects  of  nature,  involving  live-video  feeds  from  cameras  in  remote  outdoor  locales  to  projections  in  cities,  or 
they  might  support/build/highlight  the  connection  between  people  of  similar  ethnic  origin  living  across  the  world. 
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Abstract  Standardization  initiatives  in  the  field  of  learning  technologies  have 
produced  standards  for  the  interoperability  of  learning  environments  and 
learning  management  systems.  Learning  resources  based  on  these  standards  can 
be  reused,  recombined,  and  adapted  to  the  user.  However,  these  standards  follow 
a content-oriented  approach.  The  process  of  applying  pedagogical  concepts  is 
not  covered  by  these  standards.  In  the  last  years,  several  approaches  for 
pedagogical  and  didactical  concepts  have  emerged,  but  their  use  within  a 
framework  of  standards  is  not  yet  consistently  solved.  Therefore,  a model  is 
presented  which  combines  pedagogical  and  content -oriented  design.  The  model, 
based  on  the  Essen  Learning  Model,  is  a basis  for  the  reuse  and  recombination 
of  pedagogical  expertise. 

Introduction 

Development  processes  of  learning  environments  have  changed  significantly  in  the  last  years. 

The  enormous  cost  for  multimedia  production  led  to  the  consequence  that  a high  amount  of  end- 
users  are  necessary  to  be  efficient.  Learning  resources  have  to  be  adapted  and  updated  in  shorter 
periods  of  time.  Furthermore,  learning  environments  must  be  individually  tailored  to  the  users 
needs  and  preferences  in  order  to  optimize  the  learner’s  performance. 

Standardization  is  a means  to  ensure  interoperability  between  systems.  Both,  learning 
management  systems  and  learning  environments  have  been  subject  of  standardization  initiatives. 
These  initiatives,  such  as  IEEE  Learning  Technology  Standards  Committee  (LTSC),  Instructional 
Management  Systems  (IMS)-Project,  or  the  Advanced  Distributed  Learning  Network  (ADLNET) 
have  developed  a broad  range  of  standards,  from  high-level  specifications  for  architectures  to 
bindings  for  certain  components.  However,  most  of  these  approaches  focus  on  the  reuse  of 
content.  These  specifications  do  not  provide  adequate  concepts  for  modeling  pedagogical 
concepts.  As  a consequence,  several  approaches  of  pedagogical  models,  such  as  Educational 
Modeling  Language  (EML)  or  Tutorial  Modeling  Language  (TML)  are  being  developed.  These 
models  cannot  be  seen  as  a substitute  for  content-oriented  standards.  Moreover,  a framework  for 
the  combination  of  content-oriented  standards  and  pedagogical  approaches  is  needed  to  model  a 
complex  learning  environments. 

In  this  paper,  an  approach  is  presented  providing  a framework  for  the  description  of  pedagogical 
and  didactical  concepts.  The  approach  is  based  on  the  Essen  Learning  Model  (ELM).  ELM  is  a 
development  model  for  the  development  of  learning  environments.  It  provides  support  for 
developers,  designers,  content  providers,  teachers,  and  learners  on  various  levels.  Processes  and 
activities  in  project  management,  quality  assurance,  curriculum  development,  course  design,  and 
implementation  are  supported.  Furthermore,  the  specification  of  learning  technology  standards  is 
integrated  in  this  model. 

First  of  all,  related  standards  are  presented.  It  is  shown  how  these  standards  provide  a framework 
for  the  interoperability  of  learning  environments.  Secondly,  related  approaches  for  pedagogical 
concepts  are  presented.  Furthermore,  an  introduction  to  relevant  aspects  of  the  Essen  Learning 
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Model  is  given.  We  focus  on  the  aspects  of  modeling  pedagogical  concepts.  As  a conclusion, 
further  research  activ  ities  are  suggested. 


Learning  Technology  Standards 

Learning  Technology  Standards  are  being  developed  by  several  initiatives.  In  this  section,  the 
relevant  standards  for  learning  environments  are  presented.  Currently,  Uvo  standards  have 
reached  a high  level  of  maturity  (Pawlowski,  Adelsberger  2001a,  Pawlowski  2001):  Learning 
Object  Metadata  (LOM)  and  the  Sharable  Content  Object  Reference  Model  (SCORM). 

Learning  Object  Metadata  (LOM)  is  a standard  by  the  IEEE  Learning  Technology  Standards 
Committee  (LTSC)  for  the  description  of  learning  resources  (LTSC  2001).  LOM  are  used  to 
describe  learning  resources,  such  as  learning  environments.  For  this  purpose,  nine  categories  are 
included:  General,  Lifecycle,  Meta-Metadata,  Technical,  Educational,  Rights,  Relation, 
Annotation,  and  Classification.  This  standard  is  useful  to  describe  general  characteristics  of 
learning  environments  to  facilitate  search,  retrieval,  and  reuse  of  learning  resources.  However,  the 
description  of  educational  aspects  is  limited  to  a generalized  summary  (e.g.,  interactivity  type, 
learning  resource  type,  interactivity  level).  This  classification  cannot  be  used  to  represent  detailed 
information  on  pe^gogical  and  didactical  concepts  used  in  a learning  environment. 

The  Sharable  Content  Object  Reference  Model  (SCORM)  (Dodds  2001a,  Dodds  2001b)  by  the 
Advanced  Distributed  Learning  Network  (ADLNET)  is  a standard  to  ensure  interoperability 
between  Learning  Management  Systems  (LMS)  and  learning  environments.  SCORM  integrates 
LOM  and  the  content  aggregation  model  which  is  a standard  to  represent  the  structure  of  a 
learning  environment  and  relations  of  learning  units.  The  run-time  environment  in  SCORM  is  the 
interface  between  LMS  and  learning  environments.  It  controls  the  sequence  of  a learning 
environment  by  integrating  the  Computer  Managed  Instruction  (CMI)  standard  (LTSC  2000). 

This  standard  can  be  used  to  represent  courses  and  learning  environments.  However,  the  focus  of 
this  standard  is  to  combine  and  control  learning  environments.  The  main  aspect  is  to  model 
content  and  its  structure.  Although  the  structure  can  implicitly  contain  a pedagogical  approach,  no 
pedagogical  concept  is  used  to  determine  the  structure,  navigation,  or  other  adaptations. 

Both  standards  serve  as  a base  for  interoperable  description  and  reuse  of  learning  environments 
and  learning  resources.  However,  these  standards  need  individual  extensions  in  order  to  apply 
pedagogical  concepts. 


Pedagogical  Models 

The  current  representation  of  metadata  such  as  LOM  does  not  provide  an  adequate  representation 
of  pedagogical  concepts  (Koper  2001 , Pawlowski  2001).  Additionally,  there  is  no  adequate 
mapping  of  content-oriented  representation  to  a pedagogy-oriented  representation.  A variety  of 
models  have  been  developed  in  order  to  close  this  gap. 

The  Tutorial  Markup  Language  (TML)  (Netquest  2000)  is  a markup  language  for  the 
development  of  tutorial  systems.  Questing  and  problem-solving  scenarios  can  be  specified 
through  questions,  answers,  rules,  and  help  functions.  It  is  possible  to  develop  simple  tutorial 
systems.  However,  only  certain  didactical  approaches  can  be  realized  using  TML. 

The  Instructional  Material  Description  Language  (IMDL)  (Gaede  2000)  represents  structure, 
content,  assessments,  metadata,  and  a learner  profile.  The  approach  strictly  follows  an 
instructional  design  approach.  Therefore,  it  restricts  the  pedagogical  design.  It  is  not  flexible 
enough  to  model  any  given  pedagogical  approach. 

A promising  approach  for  the  representation  of  pedagogical  concepts  is  the  Educational  Modeling 
Language  (EML)  (Koper  2001)  which  is  based  on  a meta -model  for  pedagogical  modeling.  It 
focuses  on  the  embedding  of  learning  resources  in  a pedagogical  context.  The  metamodel  consists 
of  four  components: 
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■ Theories  of  learning  and  instruction  describe  theories,  principles,  and  models  of  learning  and 
teaching.  The  model  distinguishes  between  enpiricist,  rationalist,  pragmatist-sociohistoric, 
and  eclectic  theories. 

■ The  describes  the  interactions  of  learners  in  specific  learning  situations. 

■ The  Domain  represents  the  domain  in  which  a learning  environment  is  used. 

■ The  Unit  of  Sudy  Model  describes  the  design  of  learning  units  depending  on  learning 
theories,  learner  models  and  domain  models.  The  following  categories  are  included  in  this 
model:  Unit-of-study,  Metadata,  Roles,  Learning -objectives.  Prerequisites,  Content,  Activity, 
Environment,  Method. 

EML  provides  a promising  representation  of  content  and  pedagogical  concepts.  Currently,  there  is 
no  mapping  to  other  standards  such  as  SCORM.  In  order  to  use  both  standards,  a common 
framework  must  be  specified. 

Essen  Learning  Model  (ELM) 

ELM  Development  Model 

The  Essen  Learning  Model  is  a modular  system  ^ig.  1)  supporting  development  processes  as 
well  as  the  system’s  use  on  different  levels:  the  support  of  curriculum  design  (C- level),  the 
development  of  learning  sequences  (D- level),  and  the  development  of  learning  imits  (E- level) 
(Adelsberger,  Bick,  Pawlowski  2000,  Pawlowski,  2000,  Pawlowski,  2001).  Three  abstraction 
levels  can  be  distinguished:  The  generic  development  model  provides  knowledge  for  a variety  of 
contexts.  This  generic  model  is  customized  dbpending  on  the  users’  needs  and  preferences,  and 
transformed  into  a specific  process  model  for  each  ivelopment  project.  The  process  model  is 
implemented  using  the  Architecture  of  Integrated  Information  Systems  (ARIS)  and  provides  a 
framework  for  educational  technology  projects.  ARIS  is  a frame  concept  for  a global  description 
(modeling)  of  computer  supported  information  systems,  covering  the  whole  life-cycle  range  - 
fi’om  business  process  design  to  information  technology  cbployment  (Scheer,  1998).  The  third 
level  is  the  result  of  the  development  process  in  the  form  of  certain  implementations  for  each 
module. 

Figure  2 represents  the  main  processes  of  the  Essen  Learning  Model.  The  result  of  ELM-C  is  a 
detailed  network  of  learning  objectives  and  goals,  determining  structure  and  relations  of  learning 
sequences  (e.g.  courses).  Based  on  these  results,  learning  sequences  are  developed  in  ELM-D. 
The  focus  of  this  phase  is  to  find  an  adequate  didactical  method  together  with  the  right 
technology  depending  on  learning  objectives  and  user  groups.  Finally,  single  learning  units  are 
designed  and  implemented  in  ELM-E,  using  the  Extensible  Markup  Language  (XML). 
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Fig.  1 : The  Essen  Learning  Model 
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Fig.  2:  Main  process  of  ELM 
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Modeling  Pedagogical  Concepts  in  ELM 

Modeling  pedagogical  concepts  mainly  consists  of  three  processes  in  ELM: 

Context  Modeling:  Learning  cannot  be  separated  from  the  context.  The  context  includes  the 
environment  of  the  learner  (e.g.,  institution,  organization,  company)  and  the  learner’s  experiences 
and  knowledge.  The  context  is  analyzed  on  two  levels:  A description  of  the  learner’s 
characteristics  and  preferences  and  the  organization  the  learner  is  involved  in. 

Content  Modeling:  The  content  of  a learning  environment  is  described  on  three  levels:  learning 
sequences,  composite  learning  units,  and  learning  units.  These  levels  correspond  to  the  structure 
of  SCORM  (Content,  Block,  Sharable  Content  Object). 

Didactical  Modeling:  Didactical  concepts  are  directly  related  to  the  leaner  (actor),  learning 
objectives,  learning  setting,  and  the  description  of  a method  (see  Fig.  3).  The  description  of  a 
method  consists  of  phases  (activities)  which  can  be  grouped  to  phase  blocks.  Furthermore, 
experiences  and  the  usability  of  a method  for  certain  content,  settings,  or  learning  objectives  is 
described  (see  Bick,  Pawlowski,  Veith  2001).  By  this  description,  a knowledge  base  for  didactical 
methods  is  created.  Designing  a learning  environment  means  that  the  context,  content,  and 
didactical  method  must  be  related.  As  an  example,  the  method  “simulation  game”  consists  of  the 
phases  introduction,  motivation,  activity/interaction,  reflection,  abstraction,  and 
analysis/feedback.  These  phases  are  mapped  to  learning  units.  The  mapping  of  learning  units  to 
phases  provides  the  connection  between  the  pedagogical  model  and  the  content  model  (e.g., 
SCORM). 


The  model  of  a pedagogical  concept  consists  of  the  three  main  components  (context,  content,  and 
method).  In  Table  1 the  information  model  for  methods  is  described. 


Table  1 : Information  Model  Method 


Category 

Description 

General  Data 

Dublin  Core 

Dublin  Core-Elements 

Reference 

Reference  to  external  information  model 

ERLC 


1 
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Category 

Description 

Description 

Name 

Name  of  the  method 

Source 

Source  of  the  method 

Recommendation 

Recommendation  for  context,  learning  objectives,  content,  actor 

Template 

Reference  to  a template,  template  format 

Setting 

Context 

Reference  to  context  specification 

Location 

Location,  Distribution 

Time 

Period,  temporal  restrictions 

Phases 

Phase 

Name  of  a phase  of  the  method,  aggregation 

Kind 

Kind  of  a phase  (grouping  to  pha  se  blocks) 

Sequence 

Relation  to  other  phases,  sequence,  schedule,  sequence  operator  (linear, 
parallel,  free) 

Runs 

Repetitions  of  a phase 

Interaction 

Description 

Description  of  intended  / possible  interactions 

Role 

Role  of  participating  actors,  description  of  role 

Type 

Type  of  interaction 

Topology 

Unidirectional,  bidirectional 

Kind 

Synchronous,  asynchronous 

Applications 

Communication  applications/systems 

Reference 

Reference  to  external  interaction  specification 

Presentation 

Type 

Type  of  presentation  object 

Applications 

Presentation  application 

Reference 

Reference  to  external  presentation  specification 

Evaluation 

Kind 

Kind  of  evaluation  (Test,  Scenario,  . . .) 

Applications 

Reference  to  evaluation  application 

Evaluation 

Reference  to  learning  object  evaluation 

Conclusion 

In  this  paper,  the  weaknesses  of  current  standards  have  been  identified.  Learning  technology 
standards  such  as  LOM  and  SCORM  focus  on  content-oriented  models  of  learning.  The 
representation  of  pedagogical  concepts  is  neglected  in  these  model.  It  has  been  shown  that  most 
existing  pedagogical  models  do  not  provide  a generic  representation  for  pedagogic  expertise.  The 
Educational  Modeling  Language  (EML)  is  a step  towards  a generic  representation.  However, 
pedagogical  specifications  and  other  existing  standards  need  to  be  mapped  into  a common 
framework.  The  Essen  Learning  Model  provides  this  framework  by  integrating  existing  standard 
with  a generic  pedagogical  model.  This  model  serves  as  a base  for  a high-quality  design  of 
learning  environments  from  a pedagogical  and  content  perspective. 
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Abstract 

This  paper  discusses  the  issues  that  we  consider  central  to  accessible  online  education. 
A recent  research  project  investigated  the  accessibility  of  online  courses  for  students 
with  disabilities  by  evaluating  existing  courses,  developed  by  academics  at  the 
University  of  New  South  Wales  (UNSW).  The  evaluation  revealed  evidence  that 
some  aspects  of  the  courses  were  inaccessible,  but  that  with  careful  design  guidance 
and  awareness  of  the  needs  of  students  with  disabilities,  those  courses  could  be  made 
more  accessible.  As  a result  a set  of  Guidelines  for  Accessible  Online  Courses  was 
produced  (Pearson  & Koppi  2001)  specifically  to  aid  academic  developers  of  online 
courses  with  limited  technical  ability. 

However,  these  guidelines  assume  that  the  academic  developer  understands  and 
accepts  the  issues  related  to  inclusive  design  and  development.  We  decided  therefore 
to  take  a step  back  and  use  our  experience  and  the  knowledge  gained  through 
researching  the  guidelines  to  identify  the  specific  issues  that  one  needs  to  consider, 
and  the  required  skills  to  understand  inclusion  and  accessible  design.  The  issues  were 
encapsulated  into  five  major  themes:  legal  or  quality  assurance  requirements; 
awareness  of  and  the  ability  to  use  the  available  guidelines  and  protocols;  some 
understanding  of  the  assistive  technologies  used  by  students  with  disabilities; 
awareness  of  designing  for  inclusion;  and  understand  and  apply  the  checking  tools 
and  mechanisms  that  are  available 

These  themes  have  been  incorporated  into  a series  of  face-to-face  workshops  and  an 
online  course  offered  in  flexible  mode,  aimed  at  assisting  the  academic  in 
understanding  both  the  broader  issues  of  accessibility,  and  in  developing  the  skills  and 
knowledge  for  accessible  course  design. 


Background 
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Our  own  research  (Pearson,  2001)  and  others  (Grimaldi  and  Goette,  1999)  have  noted 
the  liberating  effect  for  students  with  disabilities  being  able  to  access  learning 
materials  and  resources  on  the  web.  However,  the  same  research  concludes  that 
students  may  have  particular  needs  which  need  to  be  addressed  in  the  design  of  online 
course  and  learning  materials  and  which  can  in  fact  present  further  barriers  to 
inclusion.  Some  examples  include: 

• students  with  vision  impairments  may  use  screen  magiiiiicis  or  screen  readers 

• those  with  hearing  impairment  may  need  captioning  or  commentary  for  video 
and  other  multimedia 

• students  with  learning  disabilities  such  as  dyslexia  might  use  speech 
recognition  software  for  data  input 

• people  with  other  disabilities  including  aphasia  or  colour  blindness  need  clear 
access  to  structure,  careful  use  of  colour,  and  consistent  organization  of 
learning  materials 

• students  with  physical  disabilities  may  use  alternative  input  mechanisms  and 
need  easy  navigation 

All  of  these  requirements  have  an  impact  on  the  way  the  student  interacts  with  a 
learning  environment.  Furthermore,  some  students  might  not  be  fluent  in  the  language 
of  study  or  there  could  be  other  technical  barriers  e.g.,  a slow  modem  or  they  might  be 
using  a text-only  browser  - all  of  these  things  can  present  further  barriers  to  access. 
While  many  of  the  adjustments,  which  make  sites  accessible  for  use  with  screen 
readers,  also  provide  effective  access  for  others  with  disabilities;  that  is  not  always 
enough.  Careful  design  - particularly  in  the  context  of  online  learning  - can  mean 
more  effective  access  for  all  students  and  provide  a better  experience  for  all. 


Partners 

This  work  is  a collaborative  project  between  University  of  New  South  Wales 
(UNSW),  and  University  of  Teesside  (UoT).  UoT  provide  the  research  expertise  in 
special  needs  and  accessibility,  while  UNSW  contribute  the  technical  expertise  and 
resources.  The  courses  have  been  developed  for  staff  development  purposes  at  UNSW 
and  for  a Masters  level  module  for  Multimedia  students  at  UoT. 


Introduction 

As  a result  of  a research  project  (supported  by  the  Leverhulme  Trust,  UK)  which  set 
out  to  investigate  the  accessibility  of  online  courses  (in  this  case  courses  developed  in 
WebCT),  we  produced  a set  of  guidelines  for  accessible  online  courses  (Pearson  & 
Koppi,  2001).  The  guidelines  were  based  on  those  produced  by  the  Web  Accessibility 
Initiative  (http://www.w3. orgAVAI/)  but  have  been  designed  specifically  to  aid 
academic  developers  of  online  courses  with  limited  technical  ability.  The  guidelines 
constitute  a handbook  and  guide  for  academic  developers  for  the  design  and 
implementation  of  online  courses.  They  provide  advice,  practical  examples, 
references  and  links  for  the  designer.  The  guidelines  are  available  to  staff  at  UNSW, 
but  also  to  anyone  requesting  them  both  in  print  and  online  (in  an  accessible  format). 
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Following  on  from  this  research  and  the  resultant  guidelines,  we  set  out  to  explore  the 
extent  to  which  the  guidelines  can  be  used  as  a basis  for  staff  development  and  to 
encourage  prospective  online  course  designers  to  develop  courses  which  are 
accessible  and  inclusive  for  all  students.  We  used  the  Guidelines  first  as  a basis  for  the 
development  of  a face-to«face  workshop  concerned  with  accessibility.  The  workshop 
was  used  not  only  to  train  Teaching  Fellows  in  accessible  design,  but  also  as  a pilot 
for  the  creation  of  an  online  course  in  accessible  design.  Our  experience  of  the  face- 
to-face  workshop  provided  us  with  the  opportunily  lu  icucct  on  the  process,  the 
content  and  the  tasks  and  consider  how  they  could  be  translated  to  a flexible 
mode(Koppi  and  Pearson,  2002). 

Five  major  themes  were  identified  which  we  considered  desirable  for  the  academic  to 
understand,  appreciate  or  develop  skills  in  for  accessible  design.  These  five  themes 
encompass: 

• legal  or  quality  assurance  requirements; 

• awareness  of  and  the  ability  to  use  the  available  guidelines  and  protocols; 

• some  understanding  of  the  assistive  technologies  used  by  students  with 
disabilities; 

• awareness  of  designing  for  inclusion;  and 

• understand  and  apply  the  checking  tools  and  mechanisms  that  are  available. 

The  workshops  and  online  course  were  developed  to  cover  these  five  areas.  The 
remainder  of  this  paper  discusses  each  of  these  areas  and  how  they  relate  to  the 
development  of  accessible  online  courses  and  the  designer’s  role  in  meeting  the 
requirements  of  students  with  disabilities. 


Legal  and  Quality  Assurance  Considerations 

The  rights  and  needs  of  students  with  disabilities  are  now  recognised  in  the  Special 
Educational  Needs  and  Disability  Act  2001  and  through  changes  to  the  Disability 
Discrimination  Act  1995  in  the  UK,  the  Disability  Standards  for  Education  under  the 
Disability  Discrimination  Act  1992  in  Australia,  and  the  section  508  of  the  Americans 
with  Disabilities  Act  in  the  USA.  Those  developing  courseware  need  at  least  to  be 
aware  of  the  acts  and  take  this  into  consideration  in  the  provision  of  online  resources. 

However,  it  is  perhaps  more  relevant  for  academics  to  consider  accessibility  in  terms 
of  quality  assurance  and  the  Quality  Assurance  Agency’s  (QAA)  (in  the  UK) 
expectations  of  Higher  Education  Institutions  in  complying  with  legal  obligations. 

The  QAA  set  out  24  precepts  or  standards  that  institutions  are  expected  to  meet, 
including  treating  students  with  disabilities  as  part  of  the  academic  community. 
Possible  action  includes: 

• accessible  Web  and  Intranet  sites,  and  alternative  formats  for  programme 
details  and  other  information; 

• adaptation  of  course  material  (including  electronic  material)  and  course 
delivery  to  ensure  access;  and 
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• training  staff  to  use  relevant  technology  and  to  produce  accessible  electronic 
courseware,  (JISC,  2001). 

Guidelines  and  Protocols 

Web  designers  and  course  developers  need  to  be  aware  of  and  be  able  to  use  the 
guidelines  to  ensure  that  their  web  sites  and  courses  are  compliant  with  the  W3C 
thereby  meeting  basic  accessibility  requirements. 

The  W3C  guidelines  for  accessible  web  design  are,  at  present,  quite  technical  in 
nature  and  lack  examples.  Although  the  WAI  are  working  on  developing  a more  user 
friendly  set  of  guidelines,  the  Pearson  and  Koppi  (2001)  Guidelines  for  Accessible 
Online  Courses,  are  designed  to  assist  the  average  non-technical  academic  designer 
who  is  developing  online  learning  materials  and  resources.  They  consist  of  a set  of 
tips,  techniques,  examples,  references  and  links  to  other  sources  of  information  to 
help  the  developer  to  produce  accessible  courses.  There  are  also  other  guidelines, 
most  of  which  are  based  on  those  developed  by  the  WAI  at  W3C,  relating  to  the 
design  of  web  sites  in  general. 


Assistive  Technologies 

Assistive  or  enabling  technologies  are  devices,  hardware  or  software,  which  enable 
people  with  disabilities  to  use  the  computer.  Examples  of  assistive  technology  include 
screen  readers  (such  as  JAWS),  screen  magnifiers,  alternative  keyboards  or  input 
devices,  voice  recognition  software  and  text-only  browsers.  Some  experience  in  the 
use  of  these  devices  is  beneficial  for  the  courseware  developer.  It  helps  to  understand 
the  way  that  students  interact  with  the  online  environment  and  the  overheads  that  the 
use  of  such  technology  can  involve,  including  difficulty  in  navigation  and  the  extra 
time  that  may  be  required.  Experience  in  interacting  with  the  learning  environment 
using  assistive  technologies  can  also  highlight  their  limitations  when  used  with  certain 
features  e.g.,  chat  rooms.  Some  assistive  technologies  (e.g.,  screen  magnifiers,  high 
contrast  screen  settings)  can  also  be  used  for  simple  accessibility  checks. 


Designing  for  Inclusion 

Design  is  central  to  the  whole  issue  of  accessibility  and  in  this  context  we  are  really 
referring  to  learner-centred  design  (Pearson  & Green  1999),  which  means 
understanding  and  considering  who  the  user  is,  what  their  needs  are,  what  you  want 
them  to  learn,  how  they  are  going  to  learn  it,  and  how  you  are  going  to  support  them 
in  achieving  their  learning  objectives  (McLoughlin  and  Marshall,  2000;  and  Winnips, 
2001).  In  addition,  we  need  to  consider  structural,  navigational  and  interface  design, 
and  we  also  need  to  think  about  designing  accessible  resources  and  documents. 

Accessible  document  design  requires  some  forethought  and  skill  to  be  accessible  to 
those  using  assistive  technologies.  The  academic  designer  needs  to  consider  the 
structure  and  appropriateness  of  their  documents  as  well  as  the  format  in  which  they 
are  presented.  PDF  documents,  for  instance,  can  be  a particular  problem.  With  the 
correct  tools  and  skills,  however,  many  PDF  and  other  documents  can  be  made 
accessible. 
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Checking  tools  and  mechanisms 

Checking  tools  can  be  used  for  either  checking  documents  for  accessibility  before 
they  are  published  or  to  check  an  existing  web  site  online.  There  is  a number  of  tools, 
or  mechanisms,  available. 

BOBBY  (CAST  2001)  can  be  used  to  check  single  pages  online,  or  can  be 
downloaded  to  check  entire  web  sites  on  payment  of  a license  fee.  It  provides  checks 
on  the  accessibility  of  web  pages  in  accordance  with  the  W3C  guidelines  and  the 
requirements  of  ADA  section  508.  The  checks  are,  however,  largely  functional  and 
many  design  elements  require  a manual  check. 

A-Prompt  (Accessibility  Prompt  2001)  evaluates  web  pages  for  accessibility  barriers 
and  making  repairs  to  correct  those  problems.  A-Prompt  is  also  based  on  the  W3C 
guidelines  and  is  made  available  free  of  charge  through  the  University  of  Toronto  and 
the  University  of  Wisconsin. 

Authoring  tools  such  as  Macromedia  Dreamweaver  include  a number  of  plug-ins  or 
extensions  that  enable  accessibility  checks  of  web  pages  as  they  are  developed.  These 
extensions  include  the  ability  to  evaluate  and  fix  some  accessibility  problems  to  meet 
the  requirements  of  the  W3C  guidelines  and  the  ADA  508  requirements. 

There  are  tools  available  within  Internet  Explorer  or  as  Windows  utilities  that  might 
be  used  by  people  with  disabilities  and  that  are  available  to  any  PC  user  at  no 
additional  cost.  So  that,  for  instance,  you  can  use  the  screen  magnifier  or  switch  off 
graphics  to  check  what  the  site  would  look  like  to  someone  who  either  doesn’t  use 
graphics  or  can’t  see  them. 


Conclusion  and  Further  Developments 

Taking  all  of  these  themes  into  consideration,  the  legal  and  quality  assurance 
obligations;  the  guidelines  that  are  available  for  us  to  follow;  awareness  of  and  an 
ability  to  use  some  of  the  assistive  technologies  that  people  with  disabilities  use; 
particular  attention  to  the  design  of  learning  environments;  and  being  able  to  use  the 
checking  tools  that  are  available,  we  have  created  a course  which  will  give  students  of 
multimedia  and  staff  developing  online  courses  the  opportunity  to  investigate  and 
develop  skills  in  each  of  these  areas.  At  the  end  of  the  course  the  participants  will  be 
well  equipped  to  both  understand  and  undertake  the  development  of  accessible  online 
courses.  It  is  important  that  course  designers  both  understand  the  need  for  accessible 
courses  but  also  that  they  are  equipped  with  the  skills  and  resources  necessary  to  be 
able  to  do  that. 

Further  developments  will  include  adapting  the  course  to  cater  for  the  needs  of  the 
more  technical  web  developer  to  encompass  techniques  in  the  development  of 
accessible  multimedia,  such  as  video  streaming;  and  understanding  of  the  accessibility 
features,  claims  and  limitations  of  software  products  such  as  Adobe  and  Macromedia. 

The  next  stage  of  the  process  is  to  evaluate  and  compare  the  face-to-face  workshops 
and  the  online  course  in  practice  to  determine  its  effectiveness  and  requirements  for 
enhancement. 
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Abstract;  The  design  of  resource  software  intended  for  learning  raises  various  problems.  In 
this  paper  we  shall  deal  with  one  problem  in  particular  - the  guidance  for  the  learner.  To 
illustrate  our  ideas  we  shall  use  as  an  example  the  Lexical  Research  Module,  or  LRM,  of  the 
AMICAL  project. 

In  the  first  section  of  this  paper,  we  shall  introduce  the  AMICAL  PROJECT.  Then  we  shall 
describe  the  various  functions  of  the  LRM,  as  well  as  the  various  problems  that  it  raises. 
Finally,  in  the  third  section,  we  shall  look  in  more  detail  at  the  problems  of  such  guidance, 
concentrating  more  specifically  on  the  different  uses  this  guidance  can  have. 


1.  The  AMICAL  Project 

AMICAL[‘]  (Chambreuil,  & al.,  2001)  is  a theoretical  research  project,  and  one  of  its  objectives  is 
to  think  about  the  design  of  computer  environments  for  helping  learning  in  one  particular  area  - reading. 

In  the  computer  environment  of  the  AMICAL  project,  currently  intended  for  five -year- old  children 
beginning  to  learn  to  read,  belong  several  types  of  software,  which  we  call  ‘modules’.  These  modules  are 
knowledge- based  systems  and  correspond  to  specific  types  of  learning  situations  and  interactions  with  the 
student.  We  can  separate  these  actions  into  three  groups:  “tutorial”,  “explanation”  and  “resource”  modules. 
Modules  in  the  tutorial  group,  also  called  tutorial  modules,  lead,  in  a controlled  manner,  to  the  pupil  gaining 
knowledge.  Their  aim  is  to  propose  to  the  learner  training  courses  in  reading.  These  courses  are  the  result  of 
didactic  activities.  They  are  created  dynamically  by  the  system  and  adapted  to  a given  learner  at  a particular 
stage  of  his/her  training. 

Modules  in  the  ‘explanation’  group  are  interactive- learning  environments  in  which  a collection  of 
components  are  put  at  the  learners  disposal.  The  pupil  can  thus  combine  them  in  order  to  make  his  own, 
more  complex  components.  The  system  is  interactive  in  the  sense  that  it  is  capable  of  communicating,  of 
reacting  in  real  time  to  various  actions  carried  out  by  the  learner.  The  components  are  complex  in  the  sense 
that  they  correspond  to  a combination  of  simpler  elements  - those  provided  by  the  system. 

Modules  in  the  “resource”  group  are  environments  which  can  be  compared  to  resource-documents 
(CNDP,  1992)  such  as  dictionaries  or  encyclqjaedias  which  are  already  traditionally  present  in  classrooms. 
Resource  modules,  in  the  environment  of  the  AMICAL  project,  have  - in  comparison  to  resource- 
documents  - the  ability  to  provide  specific  information  for  the  initial  learning  of  reading,  and  offer 
guidance  to  the  learner. 

More  specifically,  we  are  interested  in  a particular  module  of  the  resource  type-  the  LRM. 

Z The  LRM  and  its  Problems 

The  goal  of  the  LRM  is  to  present,  for  each  lexical  element  present  in  the  system,  information 
relevant  to  leaming-to-read  from  scratch. 

At  the  beginning,  the  system  provides  the  learner,  through  various  access  methods,  the  possibility 
of  selecting  a lexical  element,  which  we  shall  subsequently  call  a ‘word’.  Among  these  possibilities  we 
distinguish  between  the  access  modes  by  text,  by  theme,  and  by  a list  of  the  words  of  the  module  in 


‘ AMICAL:  Architecture  Multi- agents  Interactive  Compagnon  pour  I'Apprentissage  de  la  Lecture 
(interactive  leaming-to-read  environment  with  a multi-agent  architecture) 
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alphabetical  order.  The  access  mode  by  text  allows  the  learner  to  select  a word  in  the  module,  taken  from  a 
text  that  s/he  has  already  seen  in  class.  The  system  proposes,  firstly,  that  the  learner  chooses  the  title  of  the 
text  he  wants.  Then,  to  allow  him/her  to  choose  a particular  word  of  the  text,  the  system  displays  on  the 
screen  the  entire  text,  and  the  name(s)  of  the  author(s).  The  access  mode  by  theme  allows  the  learner  to  first 
of  all  select  a theme  such  as  ‘animals’,  ‘places’,  or  even  ‘fruits’,  and  then  a word  belonging  to  this  theme 
such  as  ‘cat’,  ‘dog’,  or  even  ‘hen’,  given  that  the  learner  chose  the  theme  animals.  Finally,  the  access  mode 
by  the  list  of  words  in  alphabetical  order  allows  the  student  to  choose  a word  by  initially  selecting  its  first 
letter. 

Afterwards,  having  selected  a word,  the  learner  can  visit  the  various  interfaces  carrying 
information  related  to  this  word.  These  interfaces  have  the  feature  of  being  able  to  make  use  ot  knowledge 
linked  to  the  field  of  learning  (Pelissier,2001). 

A piece  of  knowledge  linked  to  the  learning  domain  with  regard  to  reading  is,  for  example,  ‘a  word 
is  a structure  of  letters’.  This  knowledge  consists  of  making  the  child  aware  that  a word  can  be  broken  up 
into  letters,  re-formed  from  these  letters,  and  that  these  letters  have  a particular  order  in  the  word.  For  the 
learner  to  acquire  this  knowledge,  many  interfaces  carrying  information  can  be  envisaged.  We  can  quote, 
for  example,  the  animation:  ‘The  presentation  of  the  splitting  up  of  a word  into  letters  using  flashing 
letters’.  In  this  animation,  the  word  chosen  by  the  learner  initially  appears  on  the  screen  [Screen  1].  Then, 
the  letters  making  up  this  word  flash  in  turn,  one  by  one  from  left  to  right  [Screen  2]. 
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Screen  1:  The  word  chosen  by  the  learner  initially  appears  on  the  screen 


Screen  2 : The  letters  making  up  this  word  flash  in  turn,  one  by  one  from  left  to  right 
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We  should  note  that  the  term  “knot”,  which  will  be  used  subsequently,  indicates  a collection  of 
selection  interfaces  which  allow  a student,  using  the  LRM,  to  select  a word  and  the  interfaces  carrying 
information  associated  with  this  word. 

To  allow  the  learner  to  move,  within  the  module,  from  one  knot  to  another  - i.e.  to  select  a word 
and  see  the  collection  of  interfaces  carrying  information  linked  to  the  word  - a navigation  and  guidance 
specific  to  the  learner  are  put  into  action. 

Finally,  in  order  to  respect  the  various  given  functions  of  this  type  of  module,  a specific 
computational  architecture  ought  to  be  envisaged. 

Therefore,  the  design  of  a module  of  the  ’resource'  type,  such  as  the  LRM,  raises  the  following 
problems: 

• the  identification  of  knowledge  in  the  area 

• the  design  of  the  interfaces  carrying  information 

• the  access  modes 

• the  navigation 

• the  guidance 

• the  computational  architecture  of  the  system 

Of  these  six  problems,  we  shall  focus  more  particularly  on  the  guidance. 


3.  The  Guidance 
3.1  The  Problem 

According  to  Tom  Murray  (Murray,  & al.,  2000),  the  disorientation  of  the  learner  is  one  of  the 
basic  problems  which  seem  to  arise  when  designing  a computer  environment.  This  disorientation  is,  for 
Aude  Dufresne  (Dufresne,  1991),  linked  to  the  incapacity  of  the  user,  “who  confuses  one  interface  with 
another,  unable  to  remember  where  he  came  from  or  what  he  wanted  to  do”;  to  be  disorientated  is  also  not 
to  know  where  you  are,  not  to  know  how  to  get  somewhere  you  think  you  know  exists  (Conklin,  1987),  to 
not  know  if  there  remain  pertinent  documents  in  the  system  which  you  would  have  forgotten  to  visit,  to 
forget  selections  which  you  have  previously  taken  or  finally  to  be  incapable  of  giving  an  overview  or  a 
summary  of  what  you  have  just  seen. 

To  prevent  the  user  from  being  disorientated,  guidance  would  seem  to  be  necessary,  even 
indispensable.  It  seems  to  facilitate  learning  at  the  same  time  as  using  a computer  system  (Bastien  & Scapin 
1993). 

Bastien  & Scapin  define  guidance  in  a technical  report  of  the  INRIA  as  being  “the  collection  of  the 
methods  used  to  advise,  orientate,  inform,  and  guide  the  user  when  s/he  is  using  the  computer 
(messages/alarms/labels )”. 

Two  problems  linked  to  the  guidance  can  therefore  be  distinguished: 

• The  functions  of  the  guidance 

• The  methods  implemented  by  the  system  to  make  use  of  these  functions. 

Of  these  two  problems,  we  choose  to  present  more  particularly  the  problems  with  the  various 
functions. 


3.2  The  various  functions 

Within  the  framework  of  designing  a computer  environment  to  help  learning,  we  can  cite  three 
main  functions  that  the  guidance  can  have:  the  function  to  inform,  the  function  to  advise,  and  the  function 
to  position.  These  three  functions  pose  three  main  types  of  problems,  namely,  the  elements  or  types  of 
elements  affected  by  the  function,  the  timing  of  their  being  presented  and  their  ways  of  being  presented  to 
the  learner. 

Peter  Brusilovsky  (Brusilovsky  2001)  points  out  two  main  items  on  which  the  guidance  operates: 

• The  informational  content  of  the  knot 

• The  navigation  links 
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We  shall  subsequently  see  that,  depending  on  the  functions,  these  two  items  can  be  regarded  in  a 
different  way,  and  that  other  items,  not  presented  by  this  author,  can  also  be  the  subject  of  a guidance. 

According  to  Kobsa  (Kobsa,  & al.,  2001),  three  groups  of  data  are  taken  into  account  when  writing 
a guidance: 

• the  data  related  to  the  user's  characteristics 

• the  data  related  to  the  interaction  between  the  user  and  the  machine 

• the  data  of  the  environment  which  brings  together  all  the  aspects  not  related  to  the  user. 

We  shall  see,  more  specifically,  how  the  first  two  groups  of  data  can  be  used  in  the  guidance.  We 
shall  also  present  other  information,  not  listed  by  the  author,  which  can  be  usefully  used. 


The  Function  to  Inform 

By  this  function,  the  ^stem  gives  the  learner  a range  of  information  linked  to  the  knot  that  he  is 
trying  to  visualise. 

This  function  firstly  raises  the  problem  of  the  different  types  of  information  which  the  system  can 
provide  for  the  learner.  These  types  of  information  can  be  different  according  to  the  knot.  However,  we  can 
cite  at  least  three  types  of  elements  about  which  the  system  can  inform  the  learner: 

• the  type  of  knot  involved 

• the  components  of  the  knot 

The  system  can  provide  the  learner  with  information  which  permits  him  to  identify  the  type  of  knot 
he  is  visualising,  i.e,  if  the  learner  is  before  a selection  interface  or  an  interface  presenting  information. 
Information  can  also  be  carried  by  the  components  of  the  knot  - which  can  be,  in  the  case  of  the  LRM, 
words,  texts,  images,  graphics  etc,  on  the  different  navigation  links  shown  on  the  screen,  as  well  as  events 
such  as  transfers,  colour  changes,  flashing  etc  - which  can  be  associated  with  the  information. 

Among  the  various  navigation  links,  we  can  distinguish  between  those  belonging  to  intra-knot 
navigation  and  those  belonging  to  inter-knot  navigation  (Pelissier  2002).  Intra-knot  navigation  occurs 
within  the  same  knot.  In  the  selection  interface  for  a theme,  intra-knot  navigation  allows  the  learner  to 
obtain  the  reading  of  each  theme  present  on  the  screen.  Inter- knot  navigation  refers  to  the  movement  from 
one  knot  to  another.  The  student,  from  the  selection  interface  for  a title  of  a text,  can  reach  the  selection 
interface  for  a word  within  a text. 

The  system  to  glide  the  learner  can  inform  him/her  of  the  existence  of  these  two  types  of 
navigation  and  their  consequences.  When  selecting  a link  related  to  intra-knot  navigation,  the  knot  on  the 
screen  stays  the  same,  whereas  if  the  learner  uses  an  inter-knot  navigation  link,  the  knot  on  the  screen 
disappears  and  a new  knot  appears. 

This  function  then  poses  the  probleni  of  the  nature  of  the  information  presented.  The  system  can 
inform  the  learner  exclusively  of  the  existence  and/or  position  of  the  components  on  the  screen.  For 
example,  in  the  LRM,  the  system  can  tell  the  learner  that  a reading  button  for  the  title  of  a text  is  at  his 
disposal  by  specifying,  or  not,  its  position  on  the  screen  (at  the  top  / at  the  bottom,  on  the  right  / on  the  left). 

Thirdly,  there  is  the  problem  of  choosing  the  information  and  the  timing  of  its  presentation.  In 
effect,  all  the  information  cannot  be  presented  at  the  same  time  for  fear  of  overloading  the  learner.  The 
guidance  should  be  able  to  provide  information  at  an  opportune  moment  during  the  session.  It  is  therefore 
necessary  to  select  the  information  to  be  presented  for  a given  knot,  and  to  choose  the  precise  moment  of  its 
presentation. 

The  choice  of  information  to  provide  can  be  made  according  to  the  type  of  knot,  of  its  components, 
but  also,  as  for  the  selection  of  the  timing,  partly  from  diverse  hypotheses  made  about  the  learner  - such  as 
hypotheses  regarding  his  level  of  knowledge,  his  behaviour,  his  habits,  his  motivation  - and  partly  from  a 
history  made  from  different  knots  seen  ty  the  learner  during  the  current  session  and/or  during  previous 
sessions.  For  example,  if  we  draw  the  hypothesis  that  a learner  has  acquired  one  or  several  particular  pieces 
of  knowledge  related  to  an  area,  we  can  chose  whether  or  not  to  let  the  computer  system  present  a 
navigation  link  allowing  the  user  to  access  a knot,  making  use  of  his  knowledge.  Similarly,  if  the  learner 
visits  for  the  n^  time  the  same  knot,  the  system  can,  for  example,  avoid  presenting  the  different 
components,  such  as  the  various  navigational  possibilities  available,  when  the  learner  opens  this  knot. 


The  function  to  Advise 
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To  advise  is  to  encourage  the  learner  to  do  one  thing  rather  than  another,  a particular  choice  from  a 
range  of  possibilities  available  to  him/her. 

First  of  all,  this  function  poses  the  problem  of  the  elements  on  which  it  operates.  For  example,  the 
system  can  advise  a learner,  confronted  with  a selection  interface  for  the  title  of  a text,  to  choose  one 
particular  title  such  as  the  last  title  he  has  seen  in  class.  The  system  can  also  advise  him/her  to  use  an  intra- 
knot  navigation  link  like  a loud-speaker  button,  which  allows  the  title  to  be  read  out  loud. 

Then  there  is  the  problem  regarding  the  choice  and  timing  of  the  presentation  of  different  types  of 
elements  for  a given  knot  during  a session.  For  example,  the  system  can  advise  the  learner  at  one  particular 
moment  to  choose  an  interface  carrying  specific  information,  and  then,  afterwards,  recommend  the  same 
learner  to  choose  another  information- carrying  interface  in  the  same  knot. 

This  choice  of  different  elements  and  the  timing  of  the  advice  can  be  made,  as  with  the  function  to 
inform,  according  to  hypotheses  about  the  pupil  and  the  history  of  the  different  knots  already  seen.  For 
example,  according  to  the  history,  if  the  learner  is  regularly  confronted  with  one  word  in  particular,  the 
system  can  advise  the  learner  to  visualise  one  or  several  other  word(s)  of  the  module. 

Advice  can  also  be  given  according  to  pedagogical  hypotheses.  For  example,  the  irterfaces 
carrying  information  are  characterised  by  the  different  areas  of  knowledge  they  utilise.  Such  knowledge  can 
sustain  a particular  line  of  succession  in  relation  to  learning.  In  this  case,  rules  can  allow  the  system  to 
encourage  the  learner  to  use  one  particular  information-carrying  interface  before  another. 


The  Function  to  Position 

To  position  the  learner  in  a computer  environment  is  to  allow  him  to  have  an  overview  of  the  items 
which  he  has  already  encountered  in  relation  to  the  set  of  items  left  at  his  disposal. 

This  function  can  work,  for  example,  on  the  level  of: 

• knots 

• navigation  links 

The  system  can  position  the  learner  in  relation  to  the  various  knots  he  has  encountered.  It  can,  for 
example,  sum  up  the  information- carrying  interfaces  associated  with  a specific  word,  seen  by  the  learner 
during  the  course  of  one  or  several  sessions,  in  relation  to  the  collection  of  knots  it  is  possible  to  access. 
The  system  can  also  show  the  student,  for  a particular  knot,  the  set  of  links  he  has  used  in  relation  to  the  set 
of  links  available  to  him. 

This  function  of  positioning  poses,  firstly,  problems  such  as  that  of  the  identification  of  knots 
which  could  be  the  subject  of  a positioning.  For  example,  in  the  LRM,  we  can  position  the  learner  solely  on 
the  level  of  selection  interfaces  and/or  interfaces  presenting  information. 

Then,  there  is  the  problem  of  the  determination  of  the  positioning.  For  example,  the  system  can 
take  into  account,  in  the  positioning,  the  different  knots  that  the  learner  has  encountered  since  his  entry  into 
the  environment,  or  since  his  first  session. 

Finally,  there  is  the  problem  of  choosing  the  elements  of  the  knot  from  which  the  learner  can  be 
positioned.  For  example,  the  learner  can  be  positioned  in  the  selection  interface  of  a title  of  text,  in  relation 
to  the  titles  he  has  chosen  and/or  in  relation  to  the  navigation  links  used.  The  selection  of  these  elements 
can  be  done  in  accordance  with  various  theoretical  and  experimental  hypotheses  (Dufresne,  1991),  but  also 
in  accordance  with  hypotheses  formed  about  the  learner.  From  the  interactions  between  the  learner  and  the 
machine,  the  computer  system  can  determine  the  elements  and  the  moment  where  a positioning  seems 
necessary.  For  example,  if  the  learner  takes  no  action  whatsoever,  no  longer  clicks  anywhere,  the  system 
can  inform  the  user  of  the  collection  of  interfaces  he  has  seen  since  the  beginning  of  the  session,  carrying 
information  associated  with  the  word,  in  relation  to  the  collection  of  information -carrying  interfaces  it  is 
possible  to  see. 


Conclusions 

We  have  just  presented  the  different  functions  associated  with  a guidance  in  a resource  module 
intended  for  learning,  such  as  the  LRM,  the  function  to  inform,  to  advise,  and  the  function  to  position. 
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These  different  functions  can  be  used  in  the  system  in  one  or  more  ways.  That  is  the  second 
problem  linked  to  a guidance.  We  can  distinguish  several  sorts  of  ways.  Firstly,  the  visual  methods  such  as 
the  changing  of  colours  and  the  flashing  on  the  screen  cf  the  components  of  the  interface.  These  methods 
can  have,  for  example,  the  function  of  informing  the  learner  of  their  presence.  Next,  we  have  audible 
methods  such  as  commentaries  which,  for  example,  are  able  to  advise  the  learner  when  making  his/her 
choice.  Finally,  we  have  guidance  knots  which  are  specific  interfaces,  for  example,  that  show  the  learner, 
on  the  screen,  the  collection  of  interfaces  carrying  information  associated  with  a particular  word;  among 
this  collection  the  interfaces  seen  by  the  learner  are  presented  in  a specific  way. 
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Abstract:  During  the  academic  year  2001-2002  the  Oral  Pathology  web-based  course  ran  its 
3*^*^  edition.  The  site  is  the  result  of  the  collaboration  between  the  Ctu  - Centre  of 
Technologies  for  Learning  of  the  University  of  Milan  and  the  University  Unit  of  Oral 
Pathology.  The  aim  of  the  site  is  to  support  traditional  in-class  teaching,  to  offer  students  an 
environment  that  enables  them  to  communicate  with  teachers  and  other  students,  to  give 
access  to  a wealth  of  teaching  materials,  to  carry  out  self-evaluation  of  their  learning  and  to 
favour  the  acquisition  of  a methodological  approach  to  solve  clinical  problems.  This  paper 
describes  the  three-year  development  of  the  web-site,  its  conception  and  the  basic  rationale  of 
the  course:  from  a focus  on  the  definition  of  the  site  itself  to  its  smooth  integration  with  face- 
to-face  lessons. 


Introduction 

During  the  last  three  years  problems  in  regulating  student  intake  numbers  in  the  Degree  program  in  Dentistry  at 
the  University  of  Milan  have  led  to  an  average  of  250  students  in  every  first  year  class  with  students’  age  ranging 
from  between  20  to  50  years.  With  such  a “scenario”  some  teaching'  difficulties  have  also  emerged:  life 
histories,  attitudes,  interest  in  the  subject  and  demands  vary  widely  within  the  class;  difficulties  in  establishing  a 
student-teacher  relationship  have  increased;  the  “lecture”  format  tends  to  be  preferred;  clinical  instruction  has 
become  more  or  less  impossible. 

In  an  attempt  to  solve  problems  due  to  student  numbers  and  support  the  face-to-face  classroom,  in  1999  a 
collaboration  between  the  Centre  of  Technologies  for  Learning  of  the  University  of  Milan  (CTU)  and  the 
University’s  Unit  of  Oral  Pathology^  was  set  up:  it  led  to  the  development  of  an  online  environment  which  could 
support  teaching  activity  and  give  students  the  possibility  to  develop  new  skills  in  the  use  of  technologies  for 
their  learning  too.  The  project  team  considered  it  very  important  that  students  face  their  future  professional 
activity  with  basic  competence  in  the  use  of  those  tools  that  would  allow  them  to  benefit  from  a continuing  self 
updating  (internet  search,  database  consultation,  evidence  based  approach  to  learning),  as  well  as  from 
communication  with  colleagues  and  access  to  patient  records.  Moreover,  multimedia  and  Information 
Technology  now  offer  plenty  of  opportunities  to  support  medical  education  effectively,  especially  for  image 


' The  traditional  course  of  Oral  Pathology  starts  in  October  and  ends  in  May  and  consists  of:  a two-hour  traditional  face  to 
face  lesson  once  a week;  weekly  professor  and  tutor  counselling;  first  official  exam  sessions  (May  - June  - July). 

^ The  Ctu  is  Milan  University’s  centre  devoted  to  educational  technologies  (www.ctu.unimi.itV  In  the  joint  project  it  was 
responsible  for  designing  the  software  and  the  site,  for  methodological  support  to  the  teachers  during  the  course,  for  technical 
help  and  the  final  evaluation  of  the  online  environment.  The  teaching  team  from  the  Unit  of  Oral  Pathology,  consisting  of  1 
professor  and  6 assistants,  was  responsible  for  the  site  contents  and  the  activities  carried  out  in  the  site  itself 


o 

ERIC 


1619 


intensive  subjects  such  as  Anatomy,  Histology  and  Pathology  (Dowker  et  al.  1997),  (Lehmann  et  al.  1999), 
(Grimes  2001),  (Hallgren  et  al.  2002),  (Greenhalgh  2001). 

Do  students  get  this  opportunity?  Does  the  learning  change  with  the  introduction  of  a blended  course?  Some 
criticism  has  been  reported  about  the  methods  that  have  been  and  are  used  to  evaluate  new  computer-aided  and 
web-based  learning  instruments,  comparing  their  teaching  efficacy  with  the  so  defined  “traditional”  methods 
(Keane  et  al  1991),  and  it  is  not  an  easy  task  to  separate  and  evaluate  the  parts  that  play  in  an  educational 
intervention  (Hutchinson  1999). 

Most  of  the  studies  that  have  been  conducted  to  evaluate  the  effects  of  computer-aided  packages  or  web-based 
courses  collate  information  regarding  the  subjective  rate  of  acceptance  of  the  programs,  of  their  usefulness  and 
of  the  skills  gained  after  their  use,  as  well  as  give  an  overview  of  the  informatic  background  of  the  users. 

When  more  objective  evaluations  are  conducted,  many  studies  comparing  online  versus  traditional  in-class 
learning  report  no  significant  difference  in  terms  of  students’  evaluation  of  performances  and  support  the 
argument  that  the  development  of  reasoning  and  positive  attitudes  toward  the  subjects  being  studied  can  take 
place  in  both  online  and  classroom  environments  and  that  online  instruction  can  be  designed  to  be  as  effective  as 
traditional  face-to-face  instruction  (http://teleeducation.nb.ca/nosignificantdifference/.) 

In  our  case,  the  website  was  designed  to  support  and  not  replace  the  in-class  learning  activity.  Moreover,  it 
offered  a wide  range  of  web  tools  to  complement  the  traditional  learning. 

Therefore,  for  the  first  two  years  of  experience,  we  chose  to  base  the  evaluation  of  the  quality  and  efficacy  of  the 
Oral  Pathology  web  course,  on  the  four  criteria  on  which,  according  to  the  Cerfad  classification  (Cerfad  2001), 
quality  can  be  measured: 

1)  What  did  the  students  learn? 

2)  Were  the  educational  demands  for  which  the  site  was  planned  satisfactorily  met? 

3)  The  learning  route:  who  takes  part,  how,  when  and  what  sort  is  it? 

4)  How  does  the  system  work? 

Data  were  recorded  through  two  kinds  of  questionnaires.  The  first,  distributed  at  the  beginning  of  the  academic 
year,  aimed  to  trace  students’  computer  literacy  and  the  availability  of  computer/Intemet  facilities,  the  second 
focused  on  the  attention  of  student  opinions  about  the  web  support,  and  was  distributed  to  students  in  the  middle 
as  well  as  at  the  end  of  the  course.  Interviews  with  both  a couple  of  randomly  selected  students  and  teacher  of 
the  course  were  recorded  each  year.  Moreover,  the  tracking  system  of  the  web  site  allowed  the  monitoring  of 
files  most  downloaded,  total  number  of  accesses  (per  user)  and  number  of  messages  s ent  per  user. 

In  addition,  during  the  still  on-going  third  edition  of  the  course,  a pilot  study  with  an  experimental  design  has 
bee  set  up  in  order  to  include  more  objective  information  on  the  subjective  data  already  existing. 

The  study  aims  to  assess  if  the  opportunities  and  learning  aids  offered  by  the  web-course  make  any  difference  in 
the  learning  gain  of  students  who  have  access  to  the  web-support.  Students  were  randomly  divided  into  two 
groups.  During  a one  month  period  the  test  group  had  access  to  the  web  site,  while  the  control  group  did  not.  The 
web  site  offered  material  to  integrate  lessons:  images  and  short  videos,  link  suggestions  and,  most  of  all, 
exercises  to  stimulate  students  to  search  the  web  and  try  to  develop  a method  to  observe  images  and  describe 
them.  Pre  and  post  tests  were  recorded  as  well  as  questionnaires  about  computer  skills,  attitudes  and  web  course 
usage  opinion  (just  for  the  test  group).  Moreover,  an  external  observer  (pedagogist)  attended  lectures  “randomly” 
in  order  to  record  the  teacher’s  behaviour  during  in-class  teaching  activities.  Data  analysis  is  currently  study. 

Structure 

First  and  Second  Editions  (99-00;  00-01) 

The  site  was  organized  into  four  main  sections  (classroom,  resources,  secretarial  area,  R & D)  and  15 
subsections.  Classroom  was  composed  of:  lessons  (text,  graphs,  tables,  images);  case  studies  (exercises  based  on 
“problem  solving”  or  “role  playing”  methodology);  syllabus]  material  archive]  web-boards  (one  dedicated  to 
counselling,  the  others  to  case  study  discussion);  self  assessment]  professor  and  students  (details  of  the  teaching 
team  and  a list  of  students  registered  with  the  site).  Resources  consisted  of:  bibliography  and  links. 


^ The  test  - generator  system,  named  “Mirror”,  is  a property  of  Ctu  software  f http://mitTonctu.imimi.itk  it  creates  tests 
containing  questions  selected  at  random  from  question  banks.  Tracking  is  available  for  each  student.  Each  question  can  be 
linked  to  a forum,  to  positive  or  negative  feedback,  and  to  teaching  material  to  help  answer  the  question. 
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In  the  Secretarial  Area  you  could  find:  notice  board;  the  explanation  of  the  exams;  help  online  (technical  aid 
web-board);  personal  profile;  e-mail.  The  R & D area,  a part  from  the  questionnaires,  was  visible  only  by  the 
project  team  (statistics  and  project  team  web-board) 

Third  Edition  2001-2002 

In  the  third  edition  of  the  course  the  site  structure  has  been  modified,  reducing  the  number  of  sections  from  15  to 
6 by  merging  some  aspects  and  eliminating  those  that  in  the  previous  versions  had  been  indicated  as  not  useful. 
According  to  the  results  of  the  previous  courses,  it  is  necessary  to  maintain  a site  structure  that  foresees  not  only 
a sequential  usage  supporting  the  logical  progress  of  the  learning  route  but  also  a free  usage  encouraging  critical 
independence  in  the  analysis  and  selection  of  sources.  The  web  site  includes:  the  home  page  of  the  course  (latest 
announcements,  important  deadlines,  the  latest  messages  in  the  forums);  syllabus;  table  of  contents'^;  study  area; 
self  assessment  and  web-boards. 


Evaluation  Results 

First  Edition  1999-2000 

What  did  the  students  leant?  Only  65  students  out  of  151  sat  the  final  examination  in  the  summer  session,  25 
failed  and  40  passed  with  an  average  mark  of  27/30.  Considering  the  answers  to  interviews  and  questionnaires  to 
the  question  “Has  the  site  made  your  learning  more  efficient?”:  63%  replied  YES,  11%  replied  NO  and  26%  did 
not  reply.  Comments  given  to  these  replies  were:  teaching  material  was  abundant,  precise,  corresponding  to  the 
lectures  and  easy  to  consult,  allows  a better  organisation  of  the  studying;  forum  allows  doubts  to  be  cleared  up, 
as  well  as  an  exchange  with  professors  and  fellow  students',  self-assessment  provides  practice,  the  wealth  of 
images  and  case  studies  bring  the  clinical  aspect  of  the  subject  closer. 

Even  if  clearly  of  a subjective  nature,  these  latter  data  show  appreciation  for  the  learning  experience  yeilding  a 
more  positive  stance  towards  the  experience  itself  and  thus  an  active  and  fertile  attitude  towards  learning 
(Ananda  2001). 

Were  the  educational  demands  for  which  the  site  was  planned  satisfactorily  met?  Students  found  it  difficult  to 
set  up  a correct  diagnostic  procedure;  they  frequently  faced  clinical  problems  in  a disorganised  manner.  The 
section  of  the  site  dedicated  to  this  skill  was  that  regarding  case  studies.  Only  a few  students  participated  in  it. 
Most  of  the  students  focused  on  the  self-assessment  part  (9000  tests  done!)  and  on  studying  the  lectures  | (the 
parts  in  which  the  examination  consisted  of). 

The  learning  route:  who  takes  part,  how,  when  and  what  sort  is  it?  Although  few  students  actually  took  part  in 
the  forums,  to  the  question  “Did  the  online  course  allow  for  more  direct  contact  with  the  teacher?”  3%  replied 
not  at  all,  26%  not  much,  36%  a fair  amount,  24%  a lot  (11%  did  not  reply).  To  the  question  “Did  the  online 
course  allow  for  more  dialogue  with  fellow  students?”  24%  replied  not  at  all,  33%  not  much,  24%  a fair  amount, 
8%  a lot  {\\%  did  not  reply). 

From  the  messages  it  is  possible  to  see  that  the  communication  within  the  forum  took  place  mostly  between 
teacher  and  students  and  came  in  the  “question  and  answer”  category  rather  than  being  an  open  debate. 

A clear  increase  in  the  use  of  the  site  was  registered  during  the  pre -examination  period. 

How  does  the  system  work?  Some  of  the  students  had  insufficient  computer  skills  to  allow  for  adequate  use  of 
the  site  activities;  the  answers  to  the  question  “How  much  experience  do  you  have  of  using  a PC?”  were:  15% 
none,  55%  little,  23%  quite  a lot,  4%  a lot  (2%  did  not  give  an  answer).  Not  all  students  had  Internet  access  at 
home  (only  31%)  and  the  university  did  not  have  laboratories  for  individual  use  of  the  Internet  for  learning;  the 
system  was  often  slow  in  loading  pages  and  downloading  material. 


^ The  table  of  contents  is  sequentially  organised,  displaying  together  all  the  lessons  in  chronological  order;  specific  materials 
for  further  study  and  practice  exercises  are  linked  to  each  lesson.  The  organization  according  to  units  of  content  was  adopted 
in  order  to  provide  students  with  preparatory  information,  both  for  the  understanding  of  the  lesson  itself  and  for  doing  the 
exercises.  In  the  study  area  the  student  can  access  and  use  the  materials  according  to  his/her  own  interests  or  requirements,  it 
contains  content  organized  by  type. 


Consideration  and  discussion 


The  results  of  this  first  version  of  the  course  were  quite  satisfactory.  The  objectives  established  were  not  fully 
achieved,  but,  both  students  and  teachers  proved  to  be  enthusiastic  about  the  online  environment.  This  aspect  has 
also  been  found  in  other  previous,  similar  studies  (Pilcher  2001,  Wegner  e/  al.  2002). 

Emphasis  should  be  placed  on  the  commitment  and  expertise  demonstrated  by  the  teaching  team  in  creating 
appropriate  teaching  materials.  Overall,  students  reported  that  the  communication  tool  increased  student-teacher 
contact  more  than  student-student  interaction.  This  is  quite  reasonable  considering  that  students  could  meet  every 
day  in  class.  It  could  also  be  a characteristic  of  the  blended  modality  of  the  course  since  lack  of  instructor’s 
contact  can  also  be  perceived  by  students  in  completely  on-line  courses  (*wegner  ei  ai.  2002,  Gagne  M.  and 
Shepherd  M.  2001),. 

On  conclusion  of  this  first  version  of  the  course,  we  faced  a dual  challenge: 

1)  solving  problems  relating  to  the  system  and  the  use  of  the  course:  speed  of  the  system,  computer  laboratories, 
teaching  students  basic  computer  literacy; 

2)  involving  the  students  more  actively:  the  site  should  be  seen  not  as  a sort  of  “super  textbook”  to  get  ready  for 
the  examination  but  as  a proper  learning  place  to  acquire  a methodological  approach  to  diagnosis. 


Second  Edition  2001-2002 

What  Did  the  Students  Learn?  Only  28  out  of  190  students  sat  the  examination  at  the  summer  session:  21  passed, 
7 failed  with  an  average  mark  of  26.7/30.  Considering  the  answers  to  interviews  and  questionnaires  to  the 
question  “Did  the  site  help  your  learning  efficiency?”:  nobody  replied  not  at  all,  5%  very  little,  9%  quite  a lot  in 
and  54%  a lot  (31%  did  not  answer  the  question).  To  the  question:  “Do  you  think  it  is  useful  to  have  both 
lectures  and  website?”:  60%  said  YES^  nobody  A/0  whilst  37%  did  not  answer  the  question. 

Were  the  Educational  Demands  for  which  the  Site  was  Planned  Satisfactorily  Met?  Students  in  the  second 
edition  stressed  the  value  of  the  whole  project  from  a methodological  point  of  view  (e.g.  clinical  cases, 
consultation  with  teacher  or  tutors,  range  of  sources). 

The  Learning  Route:  Who  Takes  Part,  How,  When  and  What  Sort  Is  It?  Fewer  mess^es  than  in  the  previous 
year  were  exchanged  through  the  forum  (104  instead  of  120),  but  the  quality  of  the  content  remained  high  (88 
content  messages  out  of  104):  the  same  results  were  obtained  for  the  case  studies  section.  Sections  most  widely 
used  were  the  lectures  (files  downloaded  4643  times)  and  the  self-assessment  (13082  tests  done),  while  the 
teaching  archive  was  little  used  (files  downloaded  1351  times). 

How  Does  the  System  Work?  The  difficulties  and  demands  encountered  by  the  students  in  the  second  version  of 
the  course  were  mainly  of  a didactic-organizational  nature:  too  many  demanding  courses  had  to  be  followed.  The 
majority  of  the  students  did  not  take  part  to  the  web-course  activity  probably  because  working  on  the  site  was 
left  to  the  student’s  desire  to  go  in  depth  with  the  material  and  not  mandatory  for  the  final  examination;  in 
addition,  it  seems  there  was  an  unreadiness  of  third-year  students  to  approach  the  on-line  exercises 
independently  and  critically:  this  was  the  first  clinical  course  they  had  face  and,  despite  having  studied,  the 
student  had  not  yet  fully  grasped  all  the  knowledge  required  to  take  part  in  discussion  on  case  studies. 

Consideration  and  discussion 

In  the  second  version  of  the  course  an  attempt  was  made  to  meet  the  challenges  raised  by  the  first  one:  four 
computer  laboratories  were  opened  for  the  students;  a seminar  in  basic  computer  skills  for  the  whole  class  was 
organised;  students’  critical-analytical  skills  and  their  ability  to  establish  correct  diagnostic  procedure  vas 
promoted. 

Whilst  during  its  first  experience  the  didactic  group  was  enthusiastic  about  educational  technologies,  after  the 
first  two  months  of  the  second  edition  of  the  course,  a somewhat  “cooler”  attitude  of  the  teaching  team  seemed 
to  be  recognizable.  A decrease  in  enthusiasm  and  participation  of  the  teachers  was  observed,  probably  because 
the  “novelty  effect”  (Clark  1992),  which  could  have  lasted  the  whole  of  the  first  year,  had  worn  off. 

Face-to-face  lectures  seemed  to  be  unrelated  to  the  site  and  there  was  a lack  of  participation  in  online  activities. 
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As  reported  in  other  iudies  (McNulty  et  al.  2000)  peak  in  the  access  of  web  learning  resources  have  been 
recorded  during  the  pre -examination  period.  The  site  was  mostly  used  as  a teaching  text  and  guide  to  the  learning 
route. 

From  this  second  experience,  reflections  arose  on  the  nature,  usefulness  and  function  of  a web  based-course 
supporting  classroom  lectures.  Both  students  and  teachers  should  be  taught  to  reach  beyond  the  concept  of  the 
site  exclusively  as  a collection  of  teaching  materials  and  the  lecture  room  as  the  place  where  the  real  teaching 
takes  place:  the  site  has  to  be  considered  an  environment  for  leaming/teaching  relations  and  activities  that  are 
complementary  to  the  classroom  course.  We  became  aware  that  in  it  is  far  more  difficult  to  “educate’’  students 
and  teachers  in  the  use  of  online  teaching/Ieaming  methodology  in  a blended  course  than  in  fully  online  courses. 
In  fact,  as  opposed  to  the  latter,  the  classroom  lectures  provide  a weekly  contact  during  lessons,  where  students 
can  exchange  opinions  and  compare  notes  directly  and  where  the  professor  teaches  according  to  his  own 
experience  and  customary  approach.  Thus,  often  the  site  is  relegated  to  the  status  of  a mere  extra  activity. 


Third  Edition  2001  -2002 

The  course  in  its  third  edition  is  only  at  an  initial  stage  and  it  is  not  yet  possible  to  give  an  overall  assessment. 
However,  on  the  basis  of  the  first  month’s  work,  some  considerations  are  emerging. 

In  this  edition  both  the  role  and  demands  of  the  students  and  of  the  teachers  have  become  the  object  of  attention. 
Participation  in  the  activities  offered  by  the  site  has  remained  voluntary  but  this  version  foresees  the  skills 
acquired  online  being  evaluated  during  the  examination. 

To  accommodate  any  difficulties  or  hesitation  relating  to  the  use  of  the  Internet  on  the  part  of  the  students,  the 
tutors  collect  the  considerations  of  the  online  debate  at  the  end  of  each  month  and  discuss  the  methodological 
organization  of  individual  contributions  in  the  classroom,  leading  students  towards  a correct  approach  to 
diagnostic  procedure.  Another  new  feature  is  that  it  is  not  only  is  the  course  online  evaluated,  but  also  the 
teacher’s  behaviour  and  the  teaching  activity  since  an  external  observer  attends  lectures  “randomly”  in  order  to 
record  data. 

Considerations  and  Discussion 

Teaching  in  a web  environment  requires  preparation  of  material  in  advance,  attention  to  students’  questions, 
facilitation  of  interaction  among  students  and  a continuous  guiding  of  learners  who  value  the  input  of  the 
teachers  (Binshan  and  Chang-tseh  H.  2001).  However,  an  initial  analysis  of  the  data  about  web  usage  of  the  third 
edition  shows  that  teachers  have  still  not  yet  grasped  the  concept  of  an  integrated  “classroom-website”  system 
and  concentrate  all  their  teaching  activity  in  the  classroom,  without  linking  classroom  activity  with  the  other  side 
of  the  system. 

Furthermore,  since  teaching  activities  offered  by  the  site,  such  as  the  discussion  of  case  studies  and  exercises, 
aiming  to  encourage  and  educate  students  to  participate  more  actively  in  their  learning,  are  not  emphasised  by 
the  teachers  who  do  not  mention  the  web-site  in  the  classroom,  students  mainly  access  the  site  to  download 
lessons.  One  of  the  values  of  web-base  support  most  reported  by  students  still  remains  access  to  material  and 
lectures.  Handouts  have  been  reported  to  be  the  most  helpful  features  als  o in  other  similar  studies  conducted  with 
dental  students  (Pilcher  2001),  with  only  a small  portion  of  them  which  take  part  in  the  case  studies  discussion. 


Conclusions  and  Future  Prospects 

Much  work  remains  to  be  done  in  working  out  the  correct  organization  for  a blended  course  in  order  to  make  it 
effective  and  efficient.  The  opportunity  of  access  to  computer  laboratories  and  to  a course  in  computer  literacy, 
the  efficiency  of  the  system,  technical  aid,  etc.  are  necessary  but  are  not  sufficient  conditions  for  the  success  and 
effectiveness  of  a course.  To  reach  success  in  running  a blended  web-course,  the  preparation  of  both  teachers 
and  students  for  this  new  type  of  educational  interaction  online  is  needed.  In  fact,  this  kind  of  environment 
requires  a shift  from  “lecture-style”  to  “teacher-mentor”  teaching,  which  guides  and  motivates  students.  It  is  a 
shift  from  passive  students  who  mainly  engage  in  listening  to  active  students  who  ask  questions,  make 
suggestions,  and  write  their  observation  and  comments. 

Our  experience  shows  that,  in  a context  of  a correct  and  active  learning  relationship  between  teachers  and 
students  and  amongst  students  themselves,  the  following  factors  are  essential:  careful  and  holistic  planning  of  the 
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curricula  for  classroom-website  interaction  and  the  training  and  education  of  both  students  and  teachers  in  the 
use  of  educational  technologies. 

Future  work  will  focus  on  training  teachers  to  use  educational  technologies.  The  following  projects  are  foreseen: 

1)  a survey  regarding  computer  literacy  and  present  knowledge  and  use  of  educational  technologies; 

2)  a seminar  to  explain  the  educational  technologies  available  at  the  Ctu  at  present; 

3)  a workshop  on  educational  technologies  for  those  teachers  who  prove  to  be  particularly  interested  in  the 
subject. 
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Abstract;  When  teachers  document  their  practice  of  teaching  using  technology,  and  share  what 
they’ve  learned  with  others  by  making  their  work  public,  they  can  effect  important  changes  in 
faculty  development,  student  learning,  and  pedagogical  innovation.  In  particular,  multimedia 
technologies  allow  teachers  to  document  interactions  and  processes  that  don’t  easily  lend 
themselves  to  text-only  representations;  they  provide  opportunities  for  faculty  to  relatively  easily 
share  one  another’s  materials,  assignments,  and  students’  work  in  order  to  build  on  one  another’s 
research  and  reflections;  faculty  may  critique  each  others’  work  to  provide  a peer  review  process 
akin  to  other  academic  disciplines.  The  production  of  more  and  more  sites  that  document  teaching 
will  also  make  it  possible  to  look  across  the  experiences  of  many  different  faculty  members 
working  in  many  different  disciplines  and  contexts,  and  to  generalize  from  those  experiences. 

The  Knowledge  Media  Lab  at  the  Carnegie  Foundation  has  been  collaborating  with  K-12  and  Higher  Education 
faculty  members  to  document  and  represent  inquiry  into  their  practice  of  teaching  using  multimedia  and  online 
technologies.  Over  the  past  three  years,  these  faculty  have  participated  in  various  cohorts  of  a 2-year  program 
(CASTL)  to  share  what  they  know  (and  do)  about  teaching.  Their  final  products  have  constituted  a diverse  array  of 
representations,  ranging  from  dramatic  presentations  to  more  traditional  academic  genres  such  as  books  and  journal 
articles.  The  diversity  of  the  representations  demonstrates  the  importance  of  using  existing  disciplinary  genres  and 
methods  to  invent  new  ones.  The  use  of  technology  enables  faculty  to  deepen  their  individual  knowledge  about 
previously  unrevealed  aspects  of  their  pedagogy  and  curriculum;  develop  collective  understandings  of  the  practice 
of  teaching  and  learning  through  sharing  what  they’ve  observed  with  colleagues;  and  to  enable  others  to  build  upon 
what  they’ve  learned  by  making  their  inquiry  into  teaching  and  learning  public. 

Deepening  Knowledge  about  One’s  Practice 

Once  documentation  is  complete,  the  effort  that  goes  into  creating  an  engaging  and  compelling,  and 
succinct  website  can  lead  faculty  to  deepen  their  knowledge  about  the  representation  of  their  practice  and  the  central 
questions  of  their  inquiry.  Roberto  Corrada,  a professor  of  law  at  University  of  Denver,  wanted  to  document  his 
active  learning  course  on  Labor  Law,  one  in  which  students  may  elect  a union  and  negotiate  a collective  bargaining 
agreement  with  Corrada,  their  “employer.”  Working  in  collaboration  with  the  Carnegie  KML,  Corrada  has  created 
an  interactive  timeline  to  illustrate  the  spread  of  union  activity  in  his  class,  and  has  begun  documenting  his  class  to 
enrich  the  timeline  with  clips  from  his  course,  student  work  samples,  and  email  communications  from  his  student 
“union  activists.”  This  documentation  process  has  transformed  his  perspective  on  his  teaching;  he  commented,  “Just 
thinking  about  how  to  represent  the  scholarship  on  the  web  has  made  me  think  differently  about  how  I teach  the 
class,  how  I present  the  class,  and  most  importantly,  how  I document  what  I do  in  the  class.” 

By  choosing  to  represent  his  scholarship  online,  Corrada  transformed  his  perspective  on  the  unfolding  of 
his  course  from  one  in  which  the  course  was  largely  unpredictable  to  one  in  which  he  was  attuned  to  look  for  and 
document  the  “explosive  moments,”  and  in  so  doing,  be  able  to  tell  the  story  of  the  “infestation”  in  his  class  by 
student-led  union  activity.  By  documenting  his  work  through  multimedia,  he  begins  to  be  able  to  make  his  work 
public  and,  therefore,  be  able  to  share  it  with  others. 

Developing  Collective  Understandings 

Some  scholars  have  found  that  sharing  what  they’ve  learned  about  their  teaching  leads  them  to  develop 
deeper  collective  understandings  of  their  practice,  transforming  artifacts  of  teaching  into  objects  of  knowledge  that 
can  be  discussed  and  debated.  Heidi  Lyne,  a teacher  at  the  Mission  Hill  School  in  Boston,  wanted  to  represent  her 
school’s  portfolio  graduation  process  through  documentary  film.  She  followed  several  students  over  the  course  of 
the  year  as  they  prepared  their  portfolios,  all  of  which  were  expected  to  reflect  the  school’s  “Habits  of  Mind,”  and 
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collected  student  work  samples,  notes  from  faculty  meetings,  her  own  reflections,  and  school  assessment  rubrics  to 
support  and  extend  the  video. 

As  she  began  to  select  clips  and  share  them  with  her  colleagues  at  Mission  Hill,  Lyne  commented  that  their 
conversations  began  to  transform:  “An  interesting  thing  happened  when  I or  other  people,  other  staff  members 
started  watching  [the  videos]...  the  information  was  gathered  in  one  place,  whereas  otherwise  it  would  have  been 
discrete  people  doing  discrete  pieces,  and  it  would  have  taken  years...  to  get  to  the  point  of  knowledge  about  what 
we  were  doing,  that  we  got  to  very  quickly.'' 

Lyne,  her  students,  and  her  colleagues  were  all  able  to  benefit  from  examining  their  collective 
practice — once  the  portfolio  preparation  process  was  transformed  from  ethereal,  moment  to  moment  interactions,  to 
a set  of  documents  and  data  that  could  be  examined,  debated,  and  reflected  upon.  Before  she  even  got  to  the  point  of 
analysis  (Lyne  2001),  Lyne's  multimedia  documentation  had  already  had  an  impact  upon  the  school's  plans  to 
transform  the  portfolio  graduation  process  for  the  coming  year,  when  Lyne  again  plans  to  document  their  collective 
work  through  video. 

Influencing  Others 

Particularly  for  novice  and  student  teachers,  professional  developers,  mentor  teachers  and  teacher 
educators,  having  models  and  examples  to  look  at  may  enable  them  to  talk  more  deeply  about  and  build  more  richly 
from  their  teaching  experiences.  For  example,  Deborah  Smith,  an  Assistant  Professor  of  Education  at  Michigan 
State  University,  has  created  a website  to  document  a large  course  on  the  teaching  of  science  for  pre-service 
elementary  teachers.  While  Smith  hopes  to  use  her  site  (Smith  2001)  to  discuss  her  inquiry-based  approach  with 
other  faculty  in  her  field,  she  also  plans  to  use  the  site  as  a key  means  of  support  for  the  many  teaching  assistants 
who  lead  sections  of  the  course.  In  her  work  with  a local  school  district,  she  also  planned  to  share  her  experiences 
with  documenting  her  practice  using  multimedia  with  novice  teachers,  encouraging  them  to  examine  their  own 
pedagogy,  curriculum,  and  classroom  documents,  as  well  as  sharing  them  with  others:  “I  think  the  power  of 
[multimedia  representations  of  teaching]  is  a way  to  help  pre-service  and  other  practicing  teachers  get  underneath 
and  inside  a teacher’s  practice  and  feel  like  it’s  accessible-and  that,  I think,  is  very  heartening  for  people  who  want 
to  learn  to  teach  or  change  their  practice." 

Smith  is  able  to  use  her  website  not  only  to  show  her  own  investigations  into  her  practice,  but  to  inspire 
others  to  do  the  same.  These  sites  also  create  learning  opportunities  for  the  students  themselves.  Students  taking 
Smith’s  course  can  see  examples  from  previous  years  of  students’  work,  and  they  can  read  the  instructors’  feedback 
to  determine,  before  they  begin  work  on  their  own  projects,  the  kinds  of  questions  and  problems  they  may 
encounter. 

Implications 

The  value  of  scholarship  is  determined  in  its  use  - in  discussions,  in  critiques,  replication  efforts,  responses, 
applications,  deviations  and  adaptations.  Thus,  the  evolution  of  scholarship  online  will  go  hand  in  hand  with  the 
development  of  the  means  for  reviewing  and  exchanging  scholarly  work  and  the  growth  of  the  communities  in 
which  it  is  discussed  and  debated.  Faculty  who  develop  web  galleries  of  “best  practices,"  or  interactive  portfolios  of 
teacher  and  student  learning,  can  contribute  to  an  advancement  of  the  profession  as  a whole,  particularly  in  the 
development  of  new  teachers.  However,  websites  that  collect  teaching  materials  and  document  teaching  practice 
cannot  advance  the  field  without  the  language,  criteria  and  forums  for  informal  and  formal  assessment  needed  to 
examine  them.  In  short,  we  know  that  we  can  put  scholarship  online,  but  it  remains  to  be  seen  how  online 
representations  can  be  more  effective  in  capturing  the  many  aspects  of  teaching  and  learning  that  are  often  too 
ephemeral,  too  subjective  and  too  fleeting  to  represent  productively  in  print.  We  believe  that  the  collective  effort  to 
produce,  examine  and  use  websites  and  other  means  of  representing  teaching  will  facilitate  both  the  development  of 
the  scholarship  of  teaching  and  the  growing  professional  and  public  understanding  that  can  serve  to  advance  the 
field. 
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Student  Roles  in  Online  Learning  Communities: 
Navigating  Threaded  Discussions 
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University  at  Albany 


Auslraci 

This  paper  discusses  navigation  strategies  affecting  the  interactivity  of  a graduate  level  course’s  asynchronous  online 
discussion  with  a focus  on  the  development  of  a learning  community  and  knowledge  building.  This  qualitative  analysis  reports  on  an 
exploratory  investigation  of  student  interviews  and  observations  while  interacting  with  the  online  discussion  board.  The  data  was 
collected  from  students  taking  different  online  courses  from  the  State  University  of  New  York  Learning  Network  (SLN)  in  the  2001- 
02  academic  year.  Observations  revealed  that  students  are  interacting  with  the  content  vicariously  where  the  integration  of  multiple 
perspectives  facilitate  the  construction  of  knowledge.  Routines  for  reading  and  responding  to  discussion  board  messages  also  suggest 
that  first  students  negotiate  through  the  threads  created,  then  based  on  personal  criteria  interact  directly  or  vicariously  with  the 
content  of  messages. 

Purpose 

Quantitative  researches  provide  valuable  information  for  online  teaching  and  learning  however,  the  emphasis 
on  grade/test  score  improvement  does  not  really  explore  what  is  really  going  on  in  an  online  learning  environment.  How 
can  we  identify  factors  that  increase  participation  and  the  quality  of  an  online  discussion?  How  can  we  evaluate  the 
navigation  patterns  and  their  influence  on  the  dynamics  of  the  online  community  only  assumed  in  an  online  discussion? 
This  study  presents  the  development  and  implementation  of  a research  and  evaluation  protocol  that  analyzes  online 
course  participation  for  community  building  indicators  and  perceptions  of  learning. 

Literature 

Many  believe  that  the  defining  characteristic  of  the  computing  medium  is  its  interactivity  (Bolter,  1991; 
Landow,  1992;  Murray,  1997;  Turkle,  1997).  Recent  research  has  focused  on  the  general  topic  of  student  interaction 
within  four  defined  categories:  (a)  learner-content,  (b)  learner-instructor,  (c)  learner-learner,  (d)  learner-interface 
(Hillman,  Willis,  and  Gunawardena,  1994;  Moore,  1989).  And  it  has  been  more  recently  reported  that  a fifth  interaction 
is  that  of  (e)  ‘vicarious’  interaction  (Sutton,  2001),  whereby  students  do  not  interact  directly  at  all,  but  observe  and 
process  both  sides  of  direct  interaction  between  two  other  students  or  between  another  student  and  the  instructor 
(Sutton,  2001). 

The  online  discussion  board  of  an  asynchronous  course  requires  students  to  navigate  and  interact  with  the 
discussion  board  thereby  manipulating  its  structure.  This  research  project  examines  these  interactions,  however,  further 
investigates  the  interaction  a learner  has  with  the  interface  of  the  discussion  board,  as  well  as  the  instructor,  course- 
mates,  and  content.  The  online  discussion  board  is  one  tool  students  use  to  communicate  with  each  other,  where  they 
are  working  together  to  reach  course  goals  while  providing  feedback  to  one  another.  Working  together  produces  the 
external izati on  of  the  thought  processes,  the  comparison  of  alternative  perspectives,  social  facilitation,  better  learning, 
high  self-esteem  and  more  positive  attitudes  toward  the  learning  experience  (Salomon  & Globerson,  1989). 

Methodology 

This  qualitative  analyst  involved  interviews  and  observations.  Students  who  were  enrolled  in  online  graduate 
courses  volunteered  to  take  part  in  this  research.  This  exploratory  study  included  a heterogeneous  mix  of  eight  students 
enrolled  in  online  courses  in  education  at  a large  research  university  in  the  Northeast.  Their  participation  included  an 
interview  and  an  observation.  Interviews  included  questions  about  their  feelings,  attitudes  and  beliefs  about  using  the 
discussion  board  to  build  knowledge  and  community  online.  Observations  were  made  using  a camcorder  and  picture  in 
picture  monitor,  where  the  student’s  image  and  computer  screen  were  recorded  onto  videotape.  The  students  were 
video  taped  to  have  a record  of  their  interaction  and  their  thoughts  when  participating  in  the  discussion  board.  During 
this  time,  students  were  asked  to  ‘think  aloud.*  The  think  aloud  method  was  used  to  elicit  any  emotions,  feeling  or 
perceptions  of  the  messages  posted  to  the  discussion  board,  from  the  perspective  of  the  user. 

Results 

In  the  interviews,  students  responded  that  they  felt  a part  of  a learning  community  and  described  their  routines 
for  navigating  the  online  discussion  board.  Examples  for  each  are  given: 


Learning  community 

Knowledge  building 

Routines  for  Navigation 

“I  feel  a part  of  the  online  community 
of  all  people  who’ve  taken  online 
courses  and  also  a part  of  the  smaller 
community  that  is  my  online  course 
I’m  taking  now.” 

1 

“I  am  excited  that  the  information  in 
this  module  correlates  with  an 
ETAP  course  I took  last  summer!  I 
have  the  opportunity  to  draw  on  prior 
knowledge.” 

1 

“First  I check  to  see  if 
anyone  has  responded  to  any  postings 
I have  made.  Then  I check  out  new 
postings.  I respond  to  several 

postings,  sometimes  I do  this 
randomly  as  to 

not  respond  to  the  same  person  all  the 

tUUV/. 

In  the  observations,  student  interactions  with  the  structure  of  the  discussion  board,  the  content,  their  peers  and 
instructor  were  a result  of  the  learning  community’s  dynamics.  As  a result,  an  online  discussion  provides  students  with 
the  opportunity  to  experience  the  interactions  of  others  based  on  their  own  personal  interests  and  goals.  Observations 
and  interviews  indicated  that  students  were  more  motivated  to  read  and  respond  to  a message  with  which  they  identified 
culturally,  professionally,  or  personally;  to  meet  course  requirements;  to  make  others  feel  a part  of  the  course;  to  anyone 
who  responded  to  them;  to  those  messages  that  integrated  multiple  perspectives. 

Similar  to  Salomon  & Globerson  (1989),  students  reported  that  learning  took  place  in  the  comparison  of 
alternative  perspectives.  Also,  as  indicated  by  Gunawardena  & Zittle  (1997)  and  others,  students  were  aware  of  other 
student’s  social  presence  based  on  immediacy  behaviors.  During  observations,  students  made  mention  of  other’s 
negotiation  patterns  (such  as  how  where,  when  and  to  whom  they  posted),  experiences  and  cultural  backgrounds.  In 
addition,  students  used  the  discussion  board  to  ‘prime’  their  thought  processes  when  composing  messages  to  others, 
where  vicarious  interactions  were  observed. 

Conclusion 

Bangert -Drowns  & Pyke  (1997)  defined  learning  engagement  as  the  strategic  mobilization  of  cognitive, 
volition  and  affective  processes  to  foster  personal  meaning  making.  For  the  context  of  an  online  learning  environment, 
we  will  continue  to  conduct  more  interviews  and  observations  to  investigate  the  possibilities  of  cognitive  processes  in 
terms  of  the  message  content,  volition  (decision-making)  as  the  student’s  rationale  for  reading  and  responding  (or  not), 
and  affect  as  the  integration  of  social  presence.  We  will  also  examine  strategies  for  facilitating  a discussion  that 
promotes  the  integration  of  multiple  perspectives,  as  well  as,  learning  engagement  where  community  and  knowledge  are 
built. 
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Abstract:  Multimedia  Technology  is  an  excellent  technology  for  training  students  with 
disabilities.  Because  multimedia  is  interactive  and  synthesizes  sound,  images,  and  text, 
it  supports  new  methods  of  communication  in  the  learning  environment.  Multimedia 
Technology,  applied  to  education,  typically  involves  the  use  of  computer-assisted 
instruction  (CAT)  in  conjunction  with  other  media  and  is  very  helpful  when  incorporated 
in  technology -based  training  (TBT).  Although  a computer  cannot  replace  a teacher  in 
the  classroom,  when  used  as  supplement,  multimedia  use  in  the  classroom  provides 
undeniable  benefits  for  both  teachers  and  students.  The  purpose  of  this  research  is  to 
examine  the  degree  of  effectiveness  of  multimedia  technology  in  training,  in 
comparison  to  traditional  print-based  training  methods.  The  course  content  is  the  same 
in  the  print-based  training  methods  as  in  the  multimedia  program;  it  is  the  method  of 
delivery  that  is  different.  Another  concern  of  this  research  is  to  measure  the  qualitative 
nature  of  learning  using  multimedia. 


INTRODUCTION 

This  study  will  discuss  knowledge  engineering  in  multimedia  design  for  training  purposes.  The  specific 
training  context  for  testing  is  the  training  of  finger  spelling  in  eighth  grade  high  school  student  at  the 
School  for  the  Deaf  in  Bangkok,  Thailand.  Finger  spelling  is  different  from  sign  language;  sign  language  is 
a complex  visual-spatial  language  that  is  used  by  the  deaf  in  which  meaning  is  conveyed  by  a system  of 
articulated  hand  gestures  and  their  placement  relative  to  the  upper  body  whereas  finger  spelling  means  a 
communication  by  signs  made  with  the  fingers  (Merriam  2001). 

A knowledge  engineer  is  one  who  participates  in  building  expert  systems  (Poulter,  Morris,  and  Dow  1994). 
Knowledge  engineers  are  specialists  in  eliciting  knowledge  from  experts  rather  than  necessarily  being 
experts  in  the  domain  itself.  A new  professional  specialty  has  emerged  from  the  development  of  expert 
systems.  The  term  knowledge  engineers  (Heinich  1999)  has  been  coined  to  describe  the  people  who  work 
with  experts  in  a field  to  assemble  and  organize  a body  of  knowledge  and  then  design  the  software  package 
that  makes  it  possible  to  train  someone  to  become  skilled  in  the  area  or  to  enable  anyone  to  call  upon  the 
skills  of  experts  to  solve  the  problem.  The  work  of  a knowledge  engineering  is  similar  to  that  done  by 
instructional  designers  in  task  analysis  and  module  design. 
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The  objectives  addressed  in  this  study  are:  (1)  to  examine  the  effectiveness  of  multimedia  technology  in 
the  training  of  hearing  impaired  students.  In  particular,  is  multimedia  technology  effective  in  training  those 
who  wish  to  learn  finger  spelling?  (2)  to  measure  the  qualitative  nature  of  learning  using  multimedia 
technology. 

EXPERIMENTS 

To  test  the  effectiveness  of  multimedia  technology  compared  to  traditional  training  methods,  the  testing 
involved  two  comparison  groups  of  learners  from  eighth  grade  high  school  students  from  the  School  for  the 
Deafm  Bangkok,  Thailand.  Group  one  (the  first  group),  used  the  multimedia  training  program  in  CD-ROM 
format.  The  program  called  Life  in  Saskatchewan,  Group  two  (the  second  group),  learned  from  print 
material  representing  traditional  methods.  There  were  nine  hearing-impaired  students  from  the  eighth  grade 
who  participated  in  this  program.  Four  hearing  impaired  students  were  in  the  first  comparison  group  and 
five  students  in  the  other  group.  All  students  in  both  groups  were  required  to  finger  spell  specific  30 
English  words. 

ANALYSIS  & RESULTS 

• The  results  of  the  test  have  shown  that  most  students  improved  their  learning  in  both  methods. 

• The  two  methods  of  teaching  and  learning  (multimedia  training  and  print-based  methods)  impacted 
students  about  the  same  since  the  average  scores  of  both  groups  of  students  were  not  much  different. 

• Most  students  (eighth  grade  high  school  students  at  the  School  for  the  Deaf  enjoyed  learning  from 
CD-ROM. 

• Even  though  the  result  has  shown  that  67%  of  the  students  preferred  computer,  there  were  22%  felt 
more  comfortable  to  learn  from  the  instructor. 

• In  learning  English  Finger  Spellings  the  students  preferred  computer  rather  than  instructor  whereas 
22%  preferred  both. 

CONCLUSIONS 

In  this  study  we  have  been  trying  to  examine  the  degree  of  effectiveness  and  potential  benefits  of 
multimedia  technology  and  to  measure  the  qualitative  nature  of  learning  using  multimedia.  The  result  of 
this  study  has  shown  that  there  were  similar  results  in  learning  between  multimedia  technology  and  print- 
based  methods.  However,  students  enjoy  learning  from  multimedia  training  more  than  from  traditional 
methods.  We  found  that  the  impact  of  multimedia  was  much  more  effective  than  print -based  methods  in 
motivating  the  students  to  want  to  learn  and  practice  the  lessons.  The  results  have  shown  that  almost  70% 
of  the  students  accepted  that  learning  with  computer  is  excited  and  fun.  In  addition  we  would  like  to  add 
that  multimedia  technology  can  be  used  as  a motivational  tool  and  as  a valuable  supplement  in  the  training. 
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Abstract 

In  its  third  year,  this  PT3  project  places  established  professional  educators  and  pre-service  teacher 
education  students  with  technology  skills  in  a mutually  supportive,  collaborative,  technology 
enhanced  environment  to  enhance  the  modeling  of  classroom  technology  integration.  Hear  about 
lessons  learned  and  how  project  activities  will  continue  after  the  end  of  the  grant  period. 


Description  of  Project 

A recent  national  survey  on  the  preparation  of  new  teachers  to  use  information  technology  (Moursund,  1 999)  reports 
that  most  college  and  university  faculty  do  not  model  the  use  of  information  technology  skills  and  most  student  teachers  do  not 
routinely  use  technology  during  field  experiences.  This  integration  factor,  composed  of  items  that  addressed  graduates’  classroom 
skills  and  the  actual  use  of  information  technology  during  college  training  was  the  best  predictor  of  other  scores  on  the  survey. 
The  “Teaching  to  Learn  Project”  offers  a model,  which  provides  a long-term  solution  to  the  “lack  of  modeling  and  use  of 
technology  in  pre-service  teacher  education”  problem  in  university  classrooms  and  in  field  placements.  The  project’s 
consultant/client  model  places  pre-service  teacher  education  students  with  technology  skills  and  established  educators  (university 
faculty  and  K-12  teachers)  in  a mutually  supportive,  collaborative  environment  which  itself  is  enhanced  through  information 
technology. 

The  central  focus  of  our  project  is  a 4 semester  hour  elective  course  entitled  “Consultation:  Technology  Applications  in 
Education”  which  is  open  to  pre-service  teacher  education  students  who  have  successfully  completed  a required  hands-on 
Educational  Applications  in  Technology  course.  During  the  Consultation  course  the  students  are  taught  consulting  skills  in  a real 
world,  project -based  learning  situation,  A required  component  of  the  course  is  an  80-100  hour  commitment  to  serve  as  a 
technology  consultant  to  a K-12  classroom  teacher  (client)  in  an  urban  or  suburban  district  which  serves  as  a field  placement  site 
for  our  education  program,  or  to  a university  professor  (client)  in  any  discipline  which  might  serve  teacher  education  students. 
This  consultant  experience  provides  “behind  the  scenes”  experience  with  the  actual  problems  classroom  teachers  face  when  they 
try  to  integrate  technology  into  their  daily  classroom  activities. 

The  primary  goal  of  each  student  consultant  is  to  help  the  client  (experienced  educator)  enhance  the  modeling  of 
technology  integration  in  his  or  her  classroom.  The  secondary  goal  is  to  learn  “classroom  wisdom”  from  the  experienced 
educator.  After  establishing  and  piloting  the  course  in  the  first  year  of  our  PT3  grant,  during  the  second  year  we  refined  the 
course,  determined  more  specifically  what  support  the  student  consultants  need  to  successfully  carry  out  their  charge,  developed 
enhanced  support  mechanisms  and  materials  in  the  form  of: 

• major  additions  to  the  project  website  (http ://pt3 .oakland.edul 

• major  additions  to  print  and  software  resources  available  in  our  Educational 
Resource  Laboratory 

• improvements  in  identified  areas  in  the  recruitment,  selection  and  preparation 
of  the  clients  (experienced  educators) 

We  have  also  begin  the  process  of  identifying  and  developing  alternative  systems  for  carrying  on  various  components  of  the 
project  after  the  end  of  the  grant. 

Major  findings 

Persistence  pays  off.  As  we  enter  the  third  and  final  year  of  our  grant  period  we  are  very  much  aware  of  one  of  our 
original  goals,  which  was  to  develop  a strong  following  for  the  consultant  course,  the  cornerstone  of  our  project.  Because  it  is  an 
elective  for  our  undergraduate  teacher  education  students  and  is  very  demanding,  if  the  course  is  not  considered  “worth  the 
effort”  students  will  not  continue  to  enroll  in  the  course.  We  have  now  offered  the  course  5 times,  including  the  pilot,  and  it 
appears  that  the  word  is  out  among  the  students  that  the  course  is  definitely  one  of  the  highlights  of  the  undergraduate  program. 
We  have  averaged  approximately  1 3 students  in  each  class  after  the  pilot  offering,  and  already  have  students  on  the  waiting  list 
for  the  Fall  2002  course  after  the  end  of  the  grant.  We  have  received  permission  and  have  begun  to  develop  a major  and  a minor 
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in  Instructional  Systems  Technology  for  which  the  Consultant  course  will  be  the  capstone  experience.  This  program  begin  going 
through  the  approval  process  in  Fall  of  2002. 

Quotes  from  Project  Evaluators  Report 

Qualitative  types  of  information  are  being  collected  during  the  evaluation  process.  A summative  evaluation  has  been 
conducted  at  the  end  of  each  semester  beginning  with  the  pilot  semester  and  will  continue  in  the  years  after  the  project  has  ended 
in  order  to  assess  the  project’s  outcomes  and  long-term  impact.  Using  both  individual  interviews  with  the  consultants  (students) 
and  the  clients  (faculty  members,  K-12  teachers),  observations,  and  surveys,  the  impact  of  the  project  will  be  measured. 

Client;  All  clients  highlighted  the  benefits  of  having  a consultant  who  can  cater  (one-to-one)  to  their  needs  and  can 
accommodate  their  own  personal  pace  of  learning.  Some  of  them  have  already  participated  in  technology  related  training  and 
described  how  difficult  it  was  to  keep  pace  with  all  the  other  participants  and  after  the  training  to  find  the  time  to  practice  what 
was  learned.  The  client -consultant  relationship  allowed  them  not  only  to  learn  at  their  own  pace,  but  also  to  practice  under  the 
guidance  of  their  consultant.  One  of  the  clients  described  her  experience  in  the  following  way: 

It  is  a wonderful  program.  To  me  it  was  a tremendous  support,  because  you 
see,  I love  the  one-to-one  thing,  and  the  fact  that  the  consultant  is  a 
student,  because  then  you  don’t  feel  intimidated.  Every  time,  I tried  taking 
classes,  but  most  of  the  people  who  go  there,  some  of  them  are  really  good, 
and  I was  always  three  steps  behind,  and  I would  feel  completely  stupid, 
because  I didn’t  know  what  they  were  doing,  and  I did  it  a couple  of  times, 
and  it  didn’t  work  for  me,  because  it  was  too  far,  or  I got  lost,  and  I was 
embarrassed  to  ask  questions.  This  way  is  great,  because  the  student  comes 
and  if  you  have  any  questions,  she  goes  and  asks  and  the  next  time  she  has 
the  answer,  so  it’s  perfect.  I think  it  is  a perfect  situation.  I love  it.  To 

me  it  was  great.  Whoever  invented  this  had  a great,  great  idea  and  I think  the  people  are  doing  a 
wonderful  job.  And  I commend  them,  and  I’m  very  thankful  that  I learned  about  this  [PT3 
program]  because  I could  never  have  done  this  by  myself.  So  to  me  it  was  very,  very  helpful. 

Consultant:  The  primary  reason  for  participation  in  the  project  by  the  pre-service  teacher  consultants  was  to  increase 
their  knowledge  of  how  to  integrate  technology  into  their  classroom  teaching.  A majority  of  the  consultants  did  not  consider 
themselves  to  be  highly  skilled  in  technology  as  they  began  the  Consultant  course.  For  this  reason,  a majority  of  consultants  felt 
discomfort  and  apprehension  at  the  beginning  of  the  project.  The  principal  source  of  this  apprehension  was  a perception  of  not 
being  an  expert  in  technology.  However,  as  the  project  proceeded,  eighty  percent  of  the  consultants  indicated  that  they  had 
crossed  a confidence  threshold  or  competency  threshold  with  regard  to  the  use  of  technology.  One  of  the  consultants  described 
her  experience  in  the  following  way: 

I think  of  all  the  things  that  I learned,  the  most  valuable  is  just  get 
out  there  and  play  around  and  just  give  it  a try,  rather  than  be  scared.  I 
used  to  always  want  somebody  over  my  shoulder  when  I did  it.  Now,  I am 
willing  to  get  out  there  and  try  it  and  fail  a little  bit.  But  I just  keep 
pushing  away  at  that  and  I don’t  think  I had  that  before.  Because  that’s,  to 
me,  that’s  a lot  more  important  because  I can  try  every  program  now  and  I 
can  get  my  way  through  it. 
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Understanding  Participation  in  Online  Courses:  A Triangulated  Study  of  Perceptions  of 
Interaction 


Abstract 


This  presentation  reports  the  results  of  a study  that  explored  instructor  and  learner 
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Participants  included 


faculty  and  learners  involved  with  an  online  course  as  well  as  other  faculty  and  learners 
who  have  expressed  reluctance  toward  participating  in  online  courses.  Analysis  of  data 
revealed  that  although  perceptions  regarding  interaction  varied  among  all  interviewed, 
upon  reflection  all  course  participants  agreed  that  the  interactivity  was  at  least  adequate 
in  learning. 


Purpose 

The  purpose  of  this  study  is  two- fold.  First,  our  goal  was  to  gain  a deeper  understanding 
of  the  perceptions  shared  by  learners  and  faculty  that  are  reluctant  to  enroll  in  online 
courses  or  those  who  withdraw  early  in  the  experience.  Second,  we  wanted  to  explore 
instructors’  and  learners’  attitudes  and  perceptions  of  interaction  after  they  had 
experienced  an  online  course. 

Research  Questions 

1 . How  do  learners  and  faculty  (both  participants  and  non-participants  of  the  course) 
perceive  the  value  of  interaction  in  online  courses? 

2.  What  is  the  nature  of  the  relationship  between  learner- learner  and  learner- 
instructor  in  the  online  class? 

3.  What  strategies  did  the  instructional  team  design  to  promote  interaction? 

4.  In  what  ways  did  the  instructional  team  address  learner- interface  interactions,  and 
how  did  the  learners  perceive  this  form  of  interaction? 

5.  What  pattern  of  interaction  developed? 


Methods 


This  study  was  based  on  data  gathered  from  interviews,  online  course  documents,  and 
computer  transcripts.  Participants  included  3 faculty  (1  course  instructor  and  2 non- 
course participants)  and  15  learners  (11  learners  who  were  enrolled  in  Engaging  with 
Technology  (EWT)  and  4 who  either  chose  not  to  enroll  or  withdrew  after  the  first 
session). 

The  online  course  documents  included  an  analysis  of  the  course  syllabus,  a 
participant’s  guide  and  other  materials  that  supported  each  of  the  learning  activities. 
Transcripts  of  online  communications  captured  threaded  and  chat  discussion  among 
learners  and  the  instructor. 

The  analysis  of  the  transcripts  used  a 5- step  discussion  analysis  technique  devised 
by  Henri  (1992). 
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Analysis  and  Preliminary  Results 

The  interviews  revealed  the  following  about  the  four  types  of  interactions:  Interaction 
between  the  learner  and  the  course  content — most  of  the  learners  found  that  the  course 
content  was  appropriate  for  the  stated  objectives  of  the  course  and  was  presented  in  a 
clear  and  concise  manner. 

Interaction  between  the  learner  and  the  instructor — most  of  the  learners  foimd 
their  online  interactions  with  the  instructor  (through  the  chat  room  and  discussion  board) 
to  be  very  helpful. 

Interaction  between  the  learner  and  other  learners — most  of  the  learners  thought 
that  interactions  with  their  fellow  learners  were  very  helpful  in  doing  assignments,  and 
clearing  up  questions  and  doubts. 

Interaction  between  the  learners  and  the  interface — most  of  the  respondents  said 
that  they  were  for  the  most  part  satisfied  with  the  interface  of  the  online  course. 

Computer  Transcripts  Analysis 

We  analyzed  the  average  length  of  a student’s  post.  For  example,  in  Week  2,  the  learners 
replied  to  the  instructor’s  messages  and  posted  on  average  55  words  or  about  5 sentences. 
For  later  weeks,  learners  seemed  to  post  more  than  the  average  of  the  first  week  because 
they  were  looking  for  help  in  terms  of  course  projects,  and  computer  and  technology- 
related  problems.  Also,  some  of  them  needed  help  in  order  to  implement  technology  in 
their  teaching  in  their  school  districts. 

We  found  that  most  of  the  messages  that  were  posted  involved  the  course  content, 
computer  and  technology  skills,  and  the  course  projects.  Not  only  did  the  learners  share 
knowledge,  but  content  analysis  indicated  that  the  learners  were  posting  messages  at  a 
very  high  cognitive  level.  Furthermore,  after  reviewing  the  messages  each  week  the 
researchers  were  able  to  distinguish  some  of  the  learners’  characteristics  in  both  the 
discussion  board  and  the  real-time  chat  room.  For  instance,  many  learners  who  are 
sociable  in  nature  posted  more  messages  and  demonstrated  high  cognitive  skills. 
Moreover,  they  added  the  comparison  of  the  messages  between  themselves  and  the  other 
classmates.  Some  learners,  however,  posted  quite  short  messages. 

Summary 

Analysis  of  data  revealed  that  although  perceptions  regarding  interaction  varied  among 
all  interviewed,  upon  reflection  all  course  participants  agreed  that  the  interactivity  was  at 
least  adequate.  Detailed  discussion  analysis  substantiated  these  perceptions,  showing  that 
deep  levels  of  processing  and  interactivity  were  achieved. 
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Abstract;  This  paper  is  a report  on  the  findings  of  an  exploratory  study  investigating  students’  use 
of  the  WWW  and  the  corresponding  learning  outcome  in  a school  related  setting.  45  students  aged 
around  17  were  asked  to  perform  research  on  a topic  in  physics  in  order  to  write  an  essay.  The  only 
help  provided  was  access  to  the  Internet  and  a word  processing  program.  A pre-post-testdesign 
captured  learning  outcome,  and  log  files  were  used  to  follow  user  activities. 

First  results  show  that  we  are  able  to  describe  and  distinguish  different  strategies  of  students  using 
the  Internet.  Beside  other  factors  like  prior  domain  specific  knowledge,  computer  literacy,  and 
writing  skills  navigation  strategies  do  have  a significant  impact  on  students’  posttest  scores:  there 
are  favorable  and  unsuitable  Internet -Work-Strategies. 

The  identification  and  description  of  effective  strategies  and  their  influencing  factors  will  lead  to  a 
better  understanding  of  students’  work  with  the  WWW  and  hence  can  improve  their  Internet 
performance  for  the  purpose  of  learning. 


The  Internet  plays  an  important  role  in  schools  nowadays  and  is  seen  as  an  essential  tool  to  support 
teaching  and  learning.  Much  credit  is  given  to  this  new  medium  pointing  out  that  the  WWW  opens  a set  of  new 
options  which  teachers  haven’t  had  before  the  times  of  the  Internet.  In  the  past,  much  euphoria  about  the  Internet 
itself  concealed  the  basic  idea  of  any  media,  to  provide  a tool  to  support  learning.  We  like  to  draw  the  attention  on 
aspects  of  learning  by  identifying  and  evaluating  Web-Work-Strategies  of  students.  By  analyzing  both  these 
strategies  of  students’  work  with  the  Internet  and  the  gained  knowledge,  we  investigate  the  dependence  between 
these  two  aspects  (see  Priemer  & Schon  2001). 


The  main  objectives  of  the  investigation  are: 

1.  an  identification  of  Web-Work-Strategies  and  factors  influencing  and  describing  these  strategies, 

2.  the  assessment  and  analysis  of  domain  specific  knowledge  gained  by  students  working  with  the  WWW,  and 

3.  an  approach  in  detecting  dependences  and  correlations  between  strategies  and  gained  knowledge. 

In  December  2301  and  January  2002  we  asked  45  students  of  German  high  schools  to  participate  in  a 
laboratory  pre-post-test  investigation.  The  students  had  to  write  an  essay  on  a topic  in  physics:  the  description  of 
the  generation  of  tides.  For  the  corresponding  research  and  text  editing  the  students  were  allowed  to  use  computers 
only,  which  provided  Internet  access  and  word  processing  programs.  The  students  were  selected  in  a way  that  they 
had  sufficient  prior  knowledge  in  general  physics  and  the  mechanical  laws  behind  the  complex  phenomenon  of  tides. 
About  tides  in  particular  the  students  had  almost  no  knowledge  at  all.  In  the  pre -evaluation  this  proved  to  be  ideal  to 
connect  the  Internet  work  with  students’  learning  outcome.  A data  mining  program  collected  all  activities  of  the 
users:  webpages  visited,  time  on  webpages,  words  typed,  local  actions  aso. 


On  a five  level  rating-scale  students  specified  in  a self-assessment  how  much  their  work  strategy  matches 
with  the  navigation  strategies  defined  by  Canter  et  al.  (1985). 


Introduction 
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Findings 


In  addition,  the  posttest  contained  questions  about  tides  to  assess  learning  outcome. 


Navigation  Strategy 

Correlation  with  Posttest 

Scanning:  users  seek  to  cover  a large  area  without  great  depth. 

-.066 

Browsing:  users  go  wherever  the  data  takes  them  until  their  interest  is  caught. 

- .367* 

Searching:  users  are  motivated  to  find  a particular  target. 

.357* 

Exploring:  users  are  seeking  the  extent  and  nature  of  the  field. 

.504** 

users  amble  along  and  inevilably  revisit  nodes  in  an  unstructured  journey. 

- .424** 

Tab.  1:  Navigation  strategies  (Canter  et  al.  1985)  and  their  relation  to  posttest  scores 

The  left  column  of  table  1 summarizes  the  strategies.  Students’  strategies  do  not  necessarily  exclude  each 
other,  e.g.  a person  can  see  himself  as  an  ’’Explorer”  and  as  a ’’Searcher”.  In  addition,  strategies  do  not  have  to  be 
stable  attributes  of  students  but  may  depend  on  the  task  they  are  working  on.  Table  1 shows  the  correlations 
between  navigation  strategies  and  posttest  scores.  We  can  conclude  on  one  hand  that  ’’Searching”  and  ’’Exploring” 
seem  to  be  favorable  strategies  since  they  correlate  significantly  with  the  posttest.  On  the  other  hand  ’’Browsing” 
and  ’’Wandering”  seem  to  hinder  learning  with  the  WWW.  Between  the  self-assessment  of  working  as  a ’’Scanner” 
and  the  posttest  scores  no  significant  correlations  were  found. 

Log  file  data  is  used  to  verify  these  results  by  objective  measurements  and  the  help  of  a mathematical  model 
(a  first  approach  is  outlined  in  Zajonc  & Priemer  2001)  that  differs  between  three  categories  (see  as  well  Choo,  Detlor 
& Turnbull  2000):  moves  (visits  of  single  webpages),  patterns  (clusters  of  moves  that  are  directly  connected  to  each 
other),  and  strategies  (clusters  of  patterns  that  sequentially  performed  describe  a complete  Internet-session).  This 
will  allow  us  to  collect  and  process  user  data  in  order  to  describe  and  evaluate  Internet-Work  activities  without 
having  to  rely  on  subjective  assessments. 

Conclusions  and  Outlook 

Our  investigation  shows  that  indeed  Internet  work  strategies  do  have  a veritably  influence  on  learning 
outcome.  ’’Searching”  and  ’’Exploring”  can  be  seen  as  helpful  strategies  while  ’’Browsing”  and  ’’Wandering”  hinders 
learning  with  the  WWW  at  least  as  far  as  a specific  given  learning  objective  is  concerned.  The  identification  and 
description  of  effective  Web-Work-Strategies  (or  combinations)  enables  teachers  to  support  students’  work  with  the 
WWW.  An  objective  and  automated  collection  of  user  data  with  log  files  can  identify  and  evaluate  Web-Work- 
Strategies  (in  addition  to  those  described  by  Canter  at  al.  1985).  In  a long  run  we  aim  to  set  the  base  for  an  adaptive 
help  tool  that  reads,  analyses,  and  evaluates  log  files  of  students’  who  work  with  the  Internet,  detects  weaknesses 
and  gives  suggestions  to  improve  the  working  style. 
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AiSSiKACi 

Recent  developments  in  the  WWW  have  brought  major  changes  in  distance  learning  practices.  With  the  connectivity  advantage,  dis- 
tance learning  through  the  web  can  now  be  conducted  on-line  to  support  both  instructional  aspects  and  technical  aspects.  The  De- 
partment of  Electrical  Engineering,  University  of  Indonesia  has  started  the  mplementation  of  such  distance  learning  mechanism 
called  The  Daskom  On-line  ”.  One  of  the  supporting  Junctions  the  the  Daskom  On-line  is  the  Online  Test  System. 

This  paper  will  discuss  the  implementation  of  the  computer-based  testing  in  the  way  the  Daskom  On-line  been  developed  with.  The 
type  of  computer-based  testing  adopted  in  the  system  is  a linear  testing  method.  The  system  used  for  computer-based  testing  is  called 
**Kuis  Dasar  Komputer  On-Line”  ("Daskom  On-Line  Quiz  System”).  By  implementing  the  software,  lecturers  can  better  manage 
evaluations  on  student’s  performance,  thus  resulting  more  objective  scores. 

Keyword:  Daskom  On-Line,  Quiz  On-Line,  Web  Based  Learning,  Distance  Learning. 


1.  INTRODUCTION 

The  Internet  with  all  its  infrastructures  spread  all  over  the  world  has  proven  to  be  a very  effective  and  efficient  tool  to  provide  any  requirements 
and  services.  Its  revolution  of  information  transfer  capacity  attracts  many  people  to  immediately  participate  in  its  utilization  [1].  Internet  is  the 
superhighway  of  information  network  also  known  as  the  Web  Based  System  (WWW)  [2].  Among  many  fields  utilizing  the  intomet,  the  education 
is  one  of  the  most  that  has  been  taking  advantage  of  the  internet  community  in  the  form  called  web-based  learning  or  distance  learning.  Some 
systems  developed  for  such  distance  learning  environment  were  EDLIN  [3]  and  PASSENGER  [4]. 

Whenever  there  is  a learning  process,  there  will  also  be  an  assessment  process  to  evaluate  the  level  of  student’s  understanding  and  capability.  In  a 
distance  learning  environment,  computer  based  testing  or  online  testing  is  an  environment  where  students  participating  in  distance  learning  can 
take  their  exam  or  quizzes  through  out  their  own  computers  technically  without  making  the  use  of  papers  or  any  other  writing  tools. 

This  paper  will  discuss  some  aspects  found  in  the  implementation  of  distance  learning  project  developed  in  the  Electrical  Engineering  Etep ail- 
ment, University  of  Indonesia  which  is  called  the  Daskom-Online.  The  focus  of  this  paper  will  be  on  the  learning  evaluatbn  process. 

2.  THE  DASKOM-ONLINE  |21 

The  Daskom  On-Line  is  using  a strategy  called  collaborative  learning  strategy.  Collaborative  learning  has  proven  to  be  a unique  and  important 
strategy  in  online  learning.  Collaborative  learning  is  the  learning  strategy  that  relies  strongly  on  the  concept  of  interactivity  between  teacher  and 
the  class  [5].  In  this  research,  interactivity  can  be  implemented  through  all  media  in  the  Internet. 

Another  consideration  of  using  collaborative  learning  strategy  is  that  it  provides  greater  chance  of  achieving  best  result  rather  than  other  strate- 
gies [6].  The  concept  of  distance  learning  and  Daskom  On-Line  incoiporates  the  collaborative  learning  strategy  in  this  research  through  the  utili- 
zation of  computer  network  media  to  access  the  material. 

The  course’s  material  is  developed  for  the  web  based  learning  impbmentation  based  on  materials  provided  by  the  teacher  of  basic  computer 
course.  Material  was  developed  using  HTML  and  was  placed  in  a web  server  that  is  connected  to  the  campus  wide  network.  This  type  of  material 
distribution  has  several  advantages  such  as  easy  to  update,  can  be  access  from  almost  every  places  through  Internet  connection  and  can  be  pro- 
vided with  interactive  basis.  Links  from  the  syllabus  to  a web  page  for  each  class  session  or  each  chapter  can  be  added  to  this  kind  of  material. 

Daskom  On-Line  use  several  types  of  media  to  communicate,  which  are  e-mail,  mailing  list,  bulletin  board,  and  discussion  board.  The  students 
can  use  these  communication  media  to  solve  problems  or  to  communicate  with  the  teacher  or  assistant. 

3.  THE  DESIGN  OF  DASKOM  ON-LINE  QUIZ  SYSTEM  |1| 

Daskom  On-Line  Quiz  System  is  a computer-based  testing  imptemented  to  support  the  Daskom  On-Line  Distance  Learning  environment.  It 
support  the  multiple  choice  questions  on  the  topics  related  to  the  Daskom  materials  (Introduction  to  Computers).  St  udents  participating  in  the 
Daskom  On-Line  can  take  quizzes  as  part  of  the  evaluation  process  required  for  them  to  pass  the  course.  The  participation  in  this  system  also 
benefits  the  analysis  of  the  computer-based  testing  implementation  that  being  developed. 

The  Daskom  Online  Quiz  Process 

The  Daskom  On-Line  Quiz  System  performs  several  steps  from  the  beginning  of  the  quiz  until  the  end.  All  the  processes  are  sequential  processes 
that  required  to  be  executed  exactly  the  same  way.  To  ensure  that  the  system  will  work  in  the  exact  sequential  manner,  pages  delivered  to  the 
student’s  browser  is  managed  by  a scripting  language.  This  scripting  language  will  also  enables  some  overrides  in  the  system  whenever  needed. 

4.  SYSTEM’S  IMPLEMEN  TATION  AND  TESTING 
Entering  Questions  and  Answer  Options 

The  Question’s  type  currently  supported  is  the  multiple  choice  questions.  A radio  button  is  used  to  provide  answer  selection  among  options  This 
will  prevent  duplicate  answers. 
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Quiz  Result’s  Analysis 

After  a quiz  has  ended,  the  results  in  forms  of  score  cards  of  all  the  participants  can  be  analyzed.  There  are  two  analysis  modes  provided  in  the 
system: 

• Simple  Mode 

Using  this  mode,  a lecturer  can  easily  observe  the  scores  each  students  got  in  their  quiz.  All  the  results  are  shown  in  pages. 

• Expert  Mode 

This  mode  support  advanced  filtering  technique  and  detailed  view  of  the  quiz  results.  It  is  intended  to  provide  in  depth  analysis  of  the  quiz  per- 
formance and  the  quality  of  questions. 

Question’s  Analysis 

The  purpose  of  question’s  analysis  is  to  determine  wl^ether  the  question  is  correctly  stated  and  whether  the  students  can  answer  it  correctly.  Ques- 
tions are  displayed  with  their  hit  scores.  The  hit  scores  are  the  number  of  right  answers  the  students  choose  on  that  specific  question. 

This  tool  is  a very  useful  tool  especially  in  detecting  faulty  questions  and  in  the  long  run  can  detect  the  depth  of  overall  student’s  achievement 
that  may  require  improvements  in  specific  part  of  the  course  material.  The  tool  can  also  rank  the  level  of  difficulty  of  topics  being  examined. 
Figure  1 shows  the  screen  capture  of  Question’s  Analysis  interface. 

Advantages  and  Weaknesses  of  theDaskom  On-Line  Quiz  System 

The  Daskom  On-Line  Quiz  System  has  several  advantages  and  weaknesses. 

The  Advantages  are: 

• Provide  isolated  environment 

The  scrambling  technique  can  minimize  cheating  especially  since  no  students  got  the  same  presentation  of  question 
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Figure  1.  Question’s  Analysis  in 
Daskom  On-Line  Quiz 


• User  Interface 

The  user  interface  is  designed  to  provide  simple  operation  with  interesting  displays  using  specific  colors  to  differentiate  parts  of  the  ques- 
tions presented. 

• Figures  and  graphics  support 

Figures  and  graphics  support  can  improve  the  understanding  of  the  question  presented. 

• Facilities 

There  are  three  facilities  provided  to  students  when  taking  their  quiz  which  are  question’s  navigation,  checking  answers  before  grading  oc- 
curred, and  retaking  the  quiz  whenever  permitted. 

The  Weaknesses  are: 

• The  radio  button  prevents  students  to  cancel  their  answers.  Any  selection  will  indicate  that  an  answer  has  been  performed  and  the  user  can 
only  change  the  answer  but  cannot  cancel  the  process. 

• Since  the  answers  cannot  be  cancelled,  the  only  grading  mechanism  supported  is  the  correct  answers  calculation  mechanism.  Implementing 
negative  scores  in  wrong  answers  will  not  be  fair  since  the  students  are  forced  to  choose  any  options  presented  whenever  they  have  comml- 
ted  the  answer. 

• The  current  database  design  can  only  support  a one  time  quiz  session.  Any  re-examination  will  result  overwriting  the  existing  data. 

Analysis  on  the  Online  Quiz  Execution 

Through  a field  observation  in  an  environment  where  the  quiz  takers  are  performing  their  exams,  there  are  several  problems  identified. 

The  problems  are  closely  related  to  the  behavior  changes  in  exam  environment.  Some  of  the  students  felt  inconvenient  taking  exams  Irom  the 
web  using  a very  limited  screen  size.  This  problem  leads  to  impatience  that  induces  them  to  become  careless  when  answering  the  questions. 
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Generally,  the  Daskom  On  Line  Quiz  system  satisfied  the  requirements.  Comparing  the  results  of  the  online  Quiz  with  the  conventional  ones  (in 

this  case  these  same  questions  have  been  im  plemented  in  different  environment),  it  shows  no  significant  difference  between  the  two.  Most  of  the 

students  acquired  similar  scores  and  statistically  the  results  are  identical. 

6.  CONCLUSION 

• The  Daskom  On-line  Quiz  System  has  proven  to  satisfy  the  standards  of  exam  requirement  and  worked  well  under  the  circumstances 
in  the  department  of  Electrical  Engineering,  Faculty  of  Engineering,  University  of  Indonesia. 

• The  evaluation  results  show  that  the  Online  Quiz  has  similar  results  when  compared  to  the  conventional  exam  environment. 

• There  are  some  weaknesses  in  the  system,  but  most  of  the  problem  can  be  overcome  when  the  students  got  enough  practice  with  the 
new  learning  environment  of  distance  learning  through  the  web  and  its  multimedia  network. 
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Using  Computer-Based  Learning  Environments  to  Offer  Collaborative 
Opportunities  to  Engage  Students: 

One  Case  Study  Using  the  Program  “Alien  Rescue” 

Ondrea  Quiros 

Department  of  Instructional  Technology 
The  University  of  Texas  at  Austin 
United  States 
quiros@cox-internet.com 

Abstract:  A project  was  initiated  to  study  computer-based  learning  environments  as  they 
relate  to  collaboration  and  engagement.  In  a sixth-grade  classroom,  five  pairs  of  students  were 
observed  while  they  worked  through  a computer-based  program  called  “Alien  Rescue”.  The 
main  research  questions  were:  1)  what  aspects  of  the  program  “Alien  Rescue”  in  a classroom 
environment  support  collaborative  efforts  to  engage  students?  and  2)  what  behaviors  are 
exhibited  by  the  students  that  could  indicate  engagement?  It  was  observed  that  time  pressures 
and  complexity  of  the  problem  were  two  factors  that  supported  full  engagement  of 
collaborative  efforts. 

Theoretical  Perspective 

The  CD-ROM  based  program,  “Alien  Rescue”,  developed  and  designed  under  the  supervision  of  Dr. 
Min  Liu  at  the  University  of  Texas  at  Austin,  is  designed  to  offer  a meaningful  learning  experience  by 
combining  a realistic,  authentic  environment  with  problem-based  learning  meant  to  incorporate  collaborative 
activities.  Students  must  learn  about  several  alien  species  as  well  as  the  planets  and  moons  in  our  universe  in 
order  to  find  suitable  homes  for  each  alien  species.  To  obtain  the  information,  the  students  have  access  to 
several  databases,  as  well  as  a simulated  base  from  wh  ich  a student  can  retrieve  information  from  probes. 

For  students  to  be  successful  in  completing  the  program  in  the  time  allotted,  the  students  must 
consistently  collaborate  with  each  other  and  stay  engaged  in  the  task  of  finding  the  aliens  homes.  Yet,  what 
does  it  mean  to  be  engaged?  One  way  to  look  at  engagement  is  to  break  it  down  into  a combination  of  two 
ideas:  user  involvement  and  user  participation  (Hwang  & Thom,  1999).  Csikszentmihalyi  (1997)  looks  at  user 
involvement  in  his  theory  of  “flow.”  One  becomes  so  mentally  involved  that  “the  sense  of  time  is  distorted, 
hours  seems  to  pass  by  in  minutes”  (p.29).  It  is  this  complete  psychological  immergence  within  the  experience 
that  characterizes  a state  of  flow  (Foshay,  1991).  Metros  (1999)  feels  that  such  involvement  depends  upon  the 
user’s  perception  of  the  information,  or  the  “value”  as  Metros  says  it,  that  keeps  a user  engaged.  One  could 
speculate  that  a problem-based  learning  activity,  in  which  students  found  value  of  information-gathering 
activities,  combined  with  interesting  interactive  tools  and  a collaborative  format,  would  lead  to  a high  level  of 
user  participation.  Furthermore,  the  instructional  design  of  the  activity  would  need  to  incorporate  a high  level 
of  concentration  as  well  as  collaboration  to  lead  to  an  engaging  experience.  Therefore,  the  questions  this  study 
hoped  to  answer  included:  1)  what  aspects  of  the  program  “Alien  Rescue”  support  collaborative  efforts  to 
engage  students?  and  2)  what  behaviors  exhibited  by  the  students  could  indicate  engagement? 

Method 

This  study  was  done  in  one  sixth-grade  classroom  in  a middle  school  in  the  Greater  Austin,  Texas  area. 
Students  were  tasked  to  work  in  pairs  to  decide,  based  on  evidence  they  collected  through  the  computer 
program  “Alien  Rescue”,  where  each  of  six  species  of  aliens  should  be  placed  in  our  solar  system.  Each  pair  of 
students  had  one  computer  to  use  for  45  minutes  a day  for  a total  of  three  weeks.  The  researcher  observed  five 
sets  of  students,  four  from  one  table  and  one  from  a separate  table  for  approximately  two  weeks.  The  pairs  of 
students  were  chosen  based  on  preliminary  observations  of  the  class  as  a whole,  and  the  decision  to  choose  both 
students  who  seemed  to  collaborate  well  together  and  students  who  did  not.  The  pairs  of  students  were  labeled 
A,  B,  C,  D and  E.  Pairs  A through  D were  at  one  table,  and  pair  E was  at  a separate  table.  Data  was  taken  in 
the  form  of  observations  of  the  class  as  a whole  and  the  five  pairs  of  students,  questions  asked  of  the  students, 
and  the  place  each  pair  stood  when  the  teacher  checked  the  databases. 
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Results 


At  first,  the  students  read  and  took  notes  about  the  fictitious  aliens,  a relatively  simple  task.  During 
this  time,  the  students  seemed  relaxed,  and  sometimes  made  jokes  among  them  that  were  not  topic-related.  Of 
the  five  pairs  chosen  for  study,  only  four  of  the  groups  seemed  to  work  well  together.  As  the  days  went  by, 
students  became  more  aware  that  they  would  soon  need  to  be  finished  with  the  program.  At  the  same  time, 
most  groups  progressed  to  the  point  of  creating  probes  to  find  out  information  about  the  Solar  System.  Creating 
a probe  incorporated  complex  thought  processes  that  included  matching  data  needed  to  appropriate  instruments 
to  probe  mechanics.  Groups  A,  B and  C frequently  asked  questions  of  each  other  concerning  probe  design. 
However,  group  D did  not  participate  in  the  table  discussion.  Group  E continued  a pattern  of  not  discussing  the 
program  with  each  other  and  often  not  exhibiting  behaviors  consistent  with  being  engaged  in  the  assigned  task. 

Four  days  before  the  end  of  the  project  was  due,  the  teacher  looked  at  each  pair’s  database  to  assess 
where  each  pair  was  in  relation  to  finishing  the  program.  Groups  A,  B,  and  C were  allowed  to  continue,  while 
Groups  D and  E were  told  to  stop.  During  the  last  four  days,  pairs  A,  B and  C worked  together  more  and  more 
often,  to  the  point  of  delegating  responsibilities  among  themselves  with  the  idea  of  sharing  what  each  had  found 
with  the  others.  Furthermore,  in  these  last  four  days,  the  students  of  pairs  A,  B and  C rarely  made  jokes  or 
talked  of  anything  besides  the  topics  at  hand.  They  seemed  to  lose  tack  of  the  time  in  class,  and  they 
commented  on  how  fast  the  classes  seemed  to  go  by.  These  comments  support  the  idea  that  these  pairs  of 
students  became  engaged  to  the  point  of  “flow”  (Csikszentmihalyi,  1997;  Foshay,  1991). 

Discussion 

If  Metros  (1999)  is  correct  about  how  perception  is  related  to  engagement,  then  the  activities  held  more 
value  as  the  deadline  neared  (a  point  of  extrinsic  motivation)  and  the  complexity  of  the  tasks  increased  (a  point 
of  intrinsic  motivation).  Furthermore,  the  more  successful  each  group  was  in  collaboration  within  the  group 
and  with  other  groups,  the  farther  the  group  progressed  in  the  program.  It  would  seem  that  the  collaborative 
aspect  of  the  activities  created  a more  engaging  environment,  as  such  an  environment  led  to  the  shared  value  of 
the  activities. 

However,  this  study  has  several  limitations.  The  scope  of  the  study  was  very  small,  and  it  may  not  be 
possible  to  reproduce  the  results.  Furthermore,  as  this  is  a newly-initiated  project,  more  observations  need  to  be 
done  to  better  determine  the  relevance  of  the  data  collected  during  this  study.  Factors  that  may  have  contributed 
include  student  perception  and  self-efficacy  as  they  related  to  the  task,  and  the  depth  of  teacher  facilitation. 
Further  studies  regarding  teacher  facilitation  and  motivational  factors,  such  as  time -limitations  and  complexity 
of  technology-enhanced  tasks,  need  to  be  implemented  so  as  to  better  understand  how  a constructivist  paradigm 
fits  into  a technology-enhanced  instructional  design.  However,  it  is  hoped  that  this  study  can  begin 
investigations  to  lead  instructional  designers  of  multimedia  programs,  as  well  as  those  who  use  such  programs, 
to  consider  factors  relating  to  the  collaborative  engagement  of  their  intended  users. 
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NEW  DEVICES 


CD-ROM  Drives  & Their  Working; 

CD-ROM  is  the  abbreviation  for  Compact  Disk  - Read  Only  Memory.  A CD-ROM  is  actually  a 
thin  piece  of  aluminum  oxide  with  a little  added  gold  or  silver.  This  disk  is  covered  both  on  the  top  and 
bottom  by  a transparent  piece  of  plastic.  Data  is  stored  on  to  these  disks  by  ^'burning'’  holes  on  the  disk. 
These  holes  are  called  *'pits'\  Depending  on  how  deep  they  are,  the  data  is  read  in  as  a or  a *0*.  To 

store  data  on  these  disks,  we  need  a CDR  drive.  There  are  also  disks  called  CD-RW  where  data  may  be 
burned  on  to  the  disk,  erased  and  re  burnt  later  on  with  new  data.  These  disks  are  however,  costly  and  are 
beyond  the  scope  of  our  discussion. 

To  read  from  a CD-ROM,  the  LASER  hits  the  area  to  be  read,  and  reflects  of  the  disk  to  a 
miniature  scanner,  which  translates  it  to  the  machine  language.  The  difference  that  actually  occurs  in  the 
reflected  light  is  that,  the  intensity  is  lower  due  to  diffraction  when  the  LASER  falls  on  a deeper  pit. 

Tape  Drives  and  their  Working: 

Tape  drives  are  very  much  similar  in  working  when  compared  to  that  we  use  on  our  music 
systems.  Both  have  a thin  plastic  strips  with  a magnetic  coating.  This  magnetic  coating  is  what  is 
responsible  for  the  data  storage.  Data  is  stored  by  actually  magnetizing  a few  parts  of  the  tape  to  indicate 
the  'on ' state  or  the  7 ' in  machine  language. 

As  you  can  see  that  the  winding  up  increases  the  access  time.  This  is  what  is  the  greatest 
disadvantage  of  the  tape  drives.  These  drives  are  still  existent  in  a few  companies.  Despite  a lot  of 
advantages,  the  tape  drives  are  in  the  verge  of  extinction  due  to  their  speeds. 

Our  Model: 

As  discussed  earlier,  our  model  uses  the  same  mechanism  of  the  CD-ROM  drives  and  the 
Tape  drives.  Each  of  those  technologies  contributes  a significant  part  in  our  device. 

DESCRIPTION: 

The  device  contains  a similar  casing  as  that  of  the  tapes  with  a small  modification.  The  casing  is 
transparent  on  the  top  and  the  bottom  near  the  middle.  A small  shutter,  like  the  ones  in  the  floppy  drive, 
covers  these  places.  They  can  be  opened  and  closed  at  will  inside  the  tape  drive.  The  rolling  mechanism  is 
the  same  as  the  magnetic  tapes.  The  tape  that  holds  the  data  is  a thin  strip  of  plastic  in  which  another  strip 
of  aluminum  oxide  is  sandwiched.  This  tape  is  rolled  up  and  placed  inside  the  casing.  Data  is  burned  into 
our  tape  by  the  use  of  a LASER  as  in  CD-ROM  drives.  This  data  is  stored  in  the  tapes  one  after  another  in 
the  data  strips,  as  we  shall  call  them.  These  strips  are  the  thin  layers  of  aluminum  oxide.  The  data  access 
time  may  be  a little  better  that  that  of  the  tape  drives  as  the  reading  of  our  device  user  a LASER  rather  that 
an  magnetic  head  which  takes  considerable  time  to  position  itself.  The  fine  LASER  should  be  capable  of 
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emitting  light  at  multiple  frequencies  and  should  be  able  to  focus  at  various  distances.  This  shall  enable  us 
to  store  data  on  both  sides  of  the  tape  thus  increasing  the  amount  of  storage  space  available.  As  in  a CD- 
ROM,  a deeper  pit  is  read  in  as  the  'off*  state  and  the  less  deep  pit  is  read  in  as  the  'on  * state. 

WORKING: 

The  tape  moves  along  the  LASER  beam  and  the  scanner  at  the  other  end  reads  the  data.  This  data 
is  then  decoded  to  machine  language.  The  data  may  be  stored  on  the  tape  in  the  same  way  as  given  below. 
The  ASCII  value  of  the  codes  are  converted  to  their  respective  8 bits  or  16-bit  binary  code  and  stored  in  a 
sequence.  Say  we  want  to  read  the  data  on  a particular  strip  at  a particular  position;  the  tape  drive 
mechanism  moves  the  tape  first  to  the  position  and  then  seeks  the  strip  on  that  position.  This  is  similar  to 
the  floppy  and  hard  disk  mechanism  where  the  data  is  retrieved  by  first  moving  to  the  sector  from  where  the 
data  is  to  be  retrieved  and  then  to  the  track  from  which  to  retrieve.  The  codes  that  may  be  used  for  storage, 
is  given  in  the  next  page.  The  shutter  of  the  casing  opens  automatically  due  to  the  use  of  some  mechanical 
device  like  the  ones  on  the  floppy  drive.  The  Laser  is  focused  on  the  tape  drive  directly  and  also  by  making 
it  reflecting  over  a mirror  place  inside  the  casing.  This  allows  the  LASER  to  scan  both  sides  of  the  tape. 
This  adds  to  more  data  storage.  The  LASER  should  be  capable  of  multiple  frequencies  for  the  sake  of 
security  purposes.  The  default  frequency  as  in  the  CD-ROM  disks  is  used  to  store  the  code  which  resembles 
the  frequency  at  which  to  read  the  rest  of  the  drive  if  the  data  to  be  stored  is  to  kept  universal  or  available 
to  all  else  the  code  for  the  frequency  has  to  be  entered  by  the  user  in  order  to  read  from  the  tape  as  the  data 
is  stored  at  that  frequency  and  reading  it  using  other  wavelength  of  light  will  return  meaningless  data 
called  Junk  in  computer  jargon. 

ADVANTAGES: 

Most  of  the  advantages  of  this  disk  have  been  discussed  earlier.  But  for  the  sake  of  summing  things  up, 
here  is  what  it  has  to  offer 

y No  loss  of  data  due  to  scratches 
y No  problem  of  absolute  data  loss  due  to  breakages 
y Larger  storage  space  than  other  storage  media 

y Better  access  time  than  tapes  though  inferior  to  CD-ROM* s in  that  aspect 
y Better  security  for  the  data 

y No  contact  between  the  head  and  the  tape  as  in  tape  drives  thus  no  problem  of  the  tape  getting 
rolled  up 

y Can  be  used  to  store  data,  video  or  sound  as  in  the  case  of  other  storage  media. 
REFERENCES:  ^This  is  our  idea  for  our  research  work 


SSST  COPT  AVAILABLE 


16'! '1 


Computer  based  investigative  laboratory 
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Abstract:  In  an  effort  to  fill  the  need  for  lab- like  experiences  for 
distance  learners,  we  have  developed  the  ‘BiologyOne’  CD-ROM 
series.  The  CD  series  provides  students  with  a series  of  open  ended 
computer  simulations  that  mimic  the  laboratory  investigations 
commonly  encountered  in  a traditional  course.  Assessments  of  the 
effectiveness  of  these  simulations  have  shown  student 
comprehension  comparable  to  students  in  traditional  courses. 


Offering  a general  biology  course  in  a distance  learning  format  has  presented  instructors 
with  the  problem  of  providing  the  students  a laboratory  experience.  To  include  this 
experience,  instructors  have  often  relied  on  compromising  the  course’s  outreach  by 
having  students  attend  an  on  site  lab  periodically,  by  investing  in  materials  to  be  sent 
individually  to  students  or  by  relying  on  the  numerous  computer  based  tools  which 
generally  offer  a ‘interactive’  narrative  of  the  text  material  but  fall  well  short  of 
investigative  lab  experiences.  In  an  effort  to  fill  the  need  for  lab- like  experiences  for 
distance  learners,  we  have  developed  the  ‘BiologyOne’  CD-ROM  series. 

Material  in  the  BiologyOne  CD  is  organized  into  four  primary  sections.  There  are  a 
Tutorial  section,  a Glossary  section,  a Laboratory  section,  and  a Testing  section.  In 
addition,  to  aid  and  or  speed  up  a users  navigation  through  the  CD,  a ‘Site  Map’  of  the 
material  is  also  available. 

Within  the  BiologyOne  CD,  Tutorial  Units  have  been  developed  which  provide  the 
student  with  dynamic  visual  and  descriptive  introductions  to  concepts  and  processes 
commonly  covered  in  an  introductory  biology  course.  Visual  resources  include 
illustrations,  animations,  photos,  and  video.  If  the  student  is  enrolled  in  a traditional 
course,  these  resources  provide  a means  of  reviewing  material  seen  in  lecture  or  lab.  For 
the  distance  learning  student,  these  units  allow  the  student  to  view  the  material  in  a 
format  that  resembles  the  experience  they  would  have  in  the  actual  laboratory. 
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For  many  of  the  topics  covered  in  a general  biology  course,  the  BiologyOne  CD  provides 
an  investigative  laboratory  simulation.  The  goal  of  these  laboratories  is  to  provide  the 
student  with  an  experience  that  mimics  the  laboratory  experience  they  might  have  in  a 
traditional  course.  For  many  of  these  simulations,  the  student  has  control  of  multiple 
variables  that  offers  the  opportunity  for  greater  experimentation  than  is  typically 
available  in  an  introductory  biology  laboratory.  Additionally,  as  we  have  developed  these 
simulations,  we  have  tried  to  expose  the  student  to  some  of  the  equipment  they  might 
encounter  in  a ‘live’  course  but  at  the  same  time  have  tried  to  keep  the  interface  with  the 
computer  straight  forward. 

In  addition  to  these  sections,  the  BiologyOne  CD  also  provides  the  student  a glossary  and 
a selftesting  section.  The  glossary  currently  contains  approximately  2,000  terms  that  the 
student  may  encounter  during  a general  biology  course.  The  student  may  access  this 
glossary  from  any  point  within  the  program  to  review  or  clarify  the  meaning  of  a term. 
The  Testing  Section  is  designed  as  for  the  student  to  use  to  self  assess  their  mastery  of  the 
material.  Questions  within  the  Testing  Units  include  crossword  puzzles,  multiple  choice 
and  matching  formats. 

Accompanying  the  CDs  are  web  based  resources  that  provide  the  student  a ‘laboratory 
manual’  as  well  as  a short  introductory  biology  textbook.  The  laboratory  manual  provides 
the  students  with  directed  activities  to  complete  using  the  simulations  and  a location  to 
record  their  observations. 

Feedback  from  students  that  had  used  the  BiologyOne  CD  in  a distance  learning  course 
has  been  quite  positive.  Comments  include:  “The  lab  simulations  were  fun  and 
educational.  I was  wondering  how  well  the  labs  would  be  for  this  online  class,  and  I was 
pleasantly  surprised.”;  “I  was  very  impressed  with  how  real  some  of  the  lab  simulations 
seemed.  I learned  a little  more  than  I thought  would  be  possible.”;  “I  think  the  lab 
simulations  are  a really  neat  way  to  get  the  point  and  experience  across  conveniently.  I 
wondered  before  I took  the  course  how  the  lab  was  going  to  be  done.  I think  it  is  great.”; 
“The  labs  were  my  favorite  part  of  the  course.” 

Of  primary  importance  is  the  effectiveness  of  the  BiologyOne  CD  in  allowing  the 
students  to  learn  the  course  material.  An  assessment  of  the  effectiveness  of  these 
simulations  was  performed  through  comparisons  with  students  in  tradition  courses. 
Student  groups  were  either  not  exposed  to  the  BiologyOne  CD,  used  the  BiologyOne  CD 
in  addition  to  their  traditional  course  work,  or  used  the  BiologyOne  CD  as  an  alternative 
to  their  traditional  course  work.  When  students  used  the  BiologyOne  CD  only,  their 
comprehension  of  the  course  material  was  found  to  be  comparable  to  those  students  in  the 
traditional  course.  When  students  received  both  the  computer  simulations  as  well  as  the 
traditional  laboratory,  comprehension  appears  to  increase  with  test  scores  increasing 
between  two  to  ten  percentage  points. 
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Abstract:  This  paper  identifies  three  rationales  for  closing  the  gender  gap  in  computing  - 
economic,  cultural  and  political  - in  the  relevant  literature.  Each  rationale  implies  a different 
set  of  indicators  of  present  inequalities,  disparate  goals  for  creating  equality,  and  distinct 
principles  for  software  and  web  site  design  that  aims  to  help  girls  overcome  the  gap  by 
increasing  their  interest  and  knowledge  about  computing.  It  is  argued  that  designers  should 
pay  greater  attention  to  the  political  rationale  for  equity,  conceiving  software  and  web  sites 
that  cultivate  girls’  civic  uses  of  computers,  so  that  women  can  exercise  equal  control  over  the 
architecture  and  policy  of  the  information  age. 


In  the  recent  literature  on  the  gender  gap  in  computing,  arguments  for  pursuing  greater  equity  among 
males  and  females  may  be  urged  for  economic,  for  cultural,  or  for  political  reasons.  Each  rationale  is 
significant,  and  often  they  are  entwined  in  practice,  but  they  are  worth  disentangling  because  each  suggests 
different  goals  for  achieving  equity,  distinct  measures  of  current  inequalities,  and  disparate  design  principles  for 
media  that  aim  to  help  girls  overcome  the  gap.  Focusing  exclusively  on  one  rationale  is  likely  to  obscure 
designers’  understanding  of  the  gender  gap  and  how  to  address  it.  This  paper  compares  each  strand  of  thinking, 
and  considers  their  implications  for  designing  educational  and  recreational  multimedia  and  hypermedia  aimed 
at  attracting  girls  to  computing.  Although  many  reasons  have  been  offered  for  the  gender  gap,  the  lack  of 
suitable  software  and  web  sites  for  girls  is  often  cited  as  a cause  (American  Association  of  University  Women 
2000;  Cottrell  1992;  Furger  1998;  Schofield  1995).  Finally,  it  is  argued  here  that  the  political  rationale  for 
equity  has  been  neglected  and  that  designers  should  devote  greater  attention  to  fostering  the  dvic  uses  of 
computers  by  girls. 


Economic  Rationales 

Traditional  liberal  feminist  concerns  about  gender  equity  in  schooling  and  the  workplace  underlie 
economic  arguments  for  ameliorating  inequalities  of  access  to,  uses  of,  and  attitudes  about  computers  (Fig.  1). 
Indeed,  there  is  continuing  cause  for  concern  about  girls’  preparation  for  technical  careers,  according  to  the 
measures  employed  in  this  approach.  Compared  to  boys,  girls  still  report  less  experience  with  computers 
(Schumacher  & Morahan-Martin  2001),  less  confidence  in  their  computing  abilities  (Young  2000),  and  less 
interest  in  the  technology  (American  Association  of  University  Women  1999).  In  higher  education,  women’s 
share  of  bachelor’s  degrees  in  computer  science  declined  from  a high  of  37  percent  in  1983  to  28  percent  in 
1996  (Camp,  Miller  & Davies  2000)  and  women  earned  just  17  percent  of  the  doctorates  in  the  field  in  1998 
(U.S.  Bureau  of  the  Census  2000).  In  2000,  UCLA’s  annual  survey  of  first  year  college  students  nationwide 
found  that  1.8  percent  of  women,  compared  to  9.3  percent  of  men,  said  they  planned  to  pursue  a career  in 
computer  programming,  the  biggest  gender  difference  since  the  survey  first  posed  the  question  in  1971  (Higher 
Education  Research  Institute  2001).  In  the  workforce,  women  hold  an  estimated  20  percent  of  information 
technology  (IT)  jobs,  and  are  especially  underrepresented  in  systems  analysis,  software  design,  programming, 
and  technological  entrepreneurship  (American  Association  of  University  Women  2000). 

Arguments  for  educational  and  economic  equity  often  lead  to  calls  for  software  and  web  sites  that 
feature  women  role  models  in  the  technical  professions  (e.g.,  Furger  1998),  such  as  Cascade  Pass’s  You  Can  Be 
A Woman  Engineer  CD-ROM  or  web  sites  for  girls  about  careers  in  the  field  (e.g.,  www.backyard.org).  It  is 
hoped  that  this  strategy  will  counteract  many  girls’  image  of  technical  work  as  masculine,  dull,  sedentary  and 
anti-social  (Garnett  Foundation  1997),  sucking  more  females  into  the  educational  “pipeline”  toward  higher 
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Argument 

Type  of  Equity 

Gender  Gap  Measures: 
Proportion  of  Females 

Design  Strategies 

Economic 

• Educational 

• Occupational 

• In  computer  clubs,  camps 

• In  computer  science  and 
engineering  classes,  degrees 

• In  technical  jobs  in  FT 

• As  equal  earners  in  field 

• In  entrepreneurial  leadership 

• Role  models:  corporate 
technical  professionals, 
managers 

Cultural 

• Expressive 

• Aesthetic 

• Relational 

• As  internet  users 

• As  software  consumers 

• As  software  entrepreneurs, 
designers 

• Gender-Traditional 

• Gender-Nontraditional 

• Gender-Neutral 

Political 

• Civic 

• As  decision-makers,  framers  of 
computer  design,  regulation 

• Ethics,  law 

• Constructivist,  feminist 
pedagogy 

• Critical  thinking  with  and 
about  technology 

Figure  1 : Arguments  for  Gender  Equity  in  Computing 

degrees  and  jobs  in  computer  science  and  engineering.  However,  sole  pursuit  of  the  economic  rationale  can 
narrow  designers’  vision  in  ways  that  may  be  counterproductive,  even  for  girls’  economic  interests.  Too  often, 
the  range  of  role  models  provided  is  limited  to  professional  and  managerial  roles  in  corporate  IT  settings,  where 
employment  is  notoriously  unstable.  Moreover,  as  computer  skills  become  central  to  many  fields,  from 
medicine  and  law  to  auto  mechanics,  these  fields  should  exert  more  influence  over  technology  design  and  uses 
(American  Association  of  University  Women  2000).  Even  if  gender  parity  in  IT  Jobs  were  achieved  tomorrow, 
the  vast  majority  of  girls  (and  boys)  would  not  work  in  this  sector.  Thus,  girls  would  be  better  served  by  media 
that  demonstrate  the  relevance  of  computing  skills  to  a broader  range  of  work  than  systems  analysis  or 
programming,  which  are  not  the  only  paths  to  molding  technology  and  may  be  rejected  by  many  girls. 

The  larger  problem  with  the  economic  argument  is  that  it  tends  to  reduce  the  goal  of  technological 
fluency,  indeed  of  education  and  play,  to  vocational  training.  The  rhetoric  of  “keeping  up”  or  “achieving 
equality”  with  males  in  technical  careers  rarely  encourages  girls  to  question  the  norms  of  the  high  technology 
workplace  or  the  purposes  of  technology  design.  Nor  does  this  approach  ask  girls  to  consider  how  women 
could  transform  those  norms  in  ways  that  might  benefit  everyone,  such  as  by  making  high  tech  careers  more 
compatible  with  family  life.  Focus  group  research  suggests  girls  are  not  necessarily  fearful  of  computers  but 
disenchanted  with  a masculine  computer  culture  that  they  associate  with  obsessive  work,  materialistic  motives, 
trivial  and  unethical  uses  of  technology,  and  a false  sense  of  control  over  the  world  (American  Association  of 
University  Women  2000).  For  these  girls,  appeals  to  join  such  a culture  that  do  not  acknowledge  how  it  might 
be  changed  and  why  it  would  be  worth  making  the  effort  are  likely  to  fall  on  deaf  ears. 


Cultural  Rationales 

Cultural  arguments  for  equity  focus  on  increasing  females’  opportunities  to  use  and  adapt  technology 
to  explore  and  express  their  identities,  create  and  communicate,  and  maintain  relationships.  Here,  the  gender 
gap  appears  to  be  closing  in  some  ways,  according  to  the  indicators  used  or  implied  by  this  approach.  Men  and 
women  entering  college  now  report  that  they  use  computers  frequently  in  almost  equal  numbers  (Higher 
Education  Research  Institute  2001)  and  women  have  gained  parity  with  men  in  using  the  Internet,  with 
teenagers  accounting  for  the  fastest  growing  segment  of  female  users  (Hamilton  2000).  Women  take  greater 
advantage  of  email  to  maintain  relationships  with  friends  and  kin  across  distance  than  do  men  (Boneva,  Kraut 
& Frohlich  2001).  In  accordance  with  females’  tendency  to  view  the  computer  as  a tool  to  accomplish  tasks 
rather  than  an  object  of  interest  to  be  explored  in  its  own  right,  females  are  more  likely  to  use  computer 
applications  for  word  processing,  graphic  design  and  communication  instead  of  play,  programming  or  systems 
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design  (American  Association  of  University  Women  2000).  There  are  more  multimedia  games  designed  by 
women-led  companies  for  girls,  although  girls  still  buy  only  12  percent  of  games  (Gorriz  & Medina  2000). 

Cultural  arguments  emerged  in  discourse  about  the  gender  gap  in  the  1990s,  especially  in  the  debate 
over  how  to  design  multimedia  games  for  girls  (see  Cassell  & Jenkins  1998).  One  camp  appealed  to  girls’ 
traditional  interests  in  cooperation,  collaboration,  glamour,  bright  colors,  multiple  paths  through  stories  and 
environments,  and  so  on.  This  “game  girls”  strategy  suggests  that  the  most  effective  way  to  increase  girls’ 
interest  in  computers  is  to  show  how  the  technology  can  be  used  to  explore  conventionally  “feminine” 
activities,  endorsing  traditional  aspects  of  girls’  culture.  It  is  indebted  to  cultural  feminism,  which  posits  a 
separate  women’s  culture  of  nurturing  and  knowing  (e.g.,  Gilligan  1982).  This  approach  extends  to  web  sites 
for  girls  that  primarily  feature  information  on  dating,  friendships,  shopping  and  communication. 

A second  strategy  urges  girls  to  become  more  comfortable  playing  aggressive  games  designed  for 
boys,  in  hopes  of  fostering  girls’  assertiveness.  The  “game  grrls,”  as  these  teens  and  young  women  call 
themselves,  appropriate  the  formerly  male  domain  of  computer  games,  embracing  its  images  of  feminine 
warriors  and  potent  female  sexuality  as  self-empowering  (Wakeford  2000).  Game  grrls  reject  constricting 
images  of  females  as  passive  victims  or  supportive  helpmates,  which  they  accuse  girl  games,  boy  games  and 
prior  feminist  thinking  of  perpetuating.  From  this  standpoint,  the  optimal  way  to  boost  girls’  interest  in 
computers  is  not  to  pursue  a separatist  design  strategy,  but  to  demonstrate  the  relevance  of  nontraditional 
gender  interests  to  girls’  lives,  appropriate  them  as  legitimately  feminine,  and  beat  boys  at  their  own  games. 
The  game  grrls  share  “third  wave”  or  postfeminist  rejections  of  earlier  feminisms,  seen  by  grrls  as  “constricting 
(politically  correct),  guilt  inducing,  essentialist,  anti-technology,  anti-sex,  and  not  relevant  to  women’s 
circumstances  in  the  new  technologies”  (Wilding  n.d.)  The  game  grrl  strategy  extends  to  the  web  on  sites  that 
offer  discussion  forums  and  articles  on  games,  sex,  and  culture. 

A third  approach  calls  for  creating  gender-neutral  content  based  on  non-violent  interests  common  to 
boys  and  girls.  This  software  may  include  mystery  games  (such  as  Myst)  and  puzzle  games  (such  as  Tetris) 
that  do  not  feature  recognizably  gendered  protagonists  and  villains,  or  that  employ  characters  (including 
animals)  of  indeterminate  sex.  Justine  Cassell  (1998),  one  of  the  foremost  advocates  of  this  strategy,  has 
designed  software  that  appeals  to  boys’  and  girls’  shared  interests  in  self-expression  and  self-construction  by 
embedding  computers  in  familiar  objects  such  as  blankets  and  stuffed  animals  that  children  can  use  to  record 
and  share  their  own  stories.  Her  games  and  toys  apply  feminist  pedagogy’s  call  for  sharing  power  more 
equitably  between  teacher  and  student  to  the  relationship  of  designer  and  user,  and  an  emphasis  on  validating 
and  learning  from  children’s  own  subjective  experience.  Under  this  approach,  the  best  way  to  interest  girls  in 
computers  is  to  show  their  relevance  to  all  children’s  interests  in  acting  as  detective  or  storyteller,  with  no 
intervening  protagonists  and  overt  gender  preferences  with  which  to  identify. 

One  danger  of  the  cultural  approach  is  its  exclusive  thinking  about  how  to  address  girls’  identities  and 
gender  socialization.  Thus  far,  the  debate  over  game  design  for  girls  seems  to  have  been  won  in  the 
marketplace  by  gender-traditional  advocates,  as  evidenced  by  the  commercial  dominance  of  Barbie  Fashion 
Designer  and  other  “pink  software”  (Gorriz  & Medina  2000),  and  in  academic  and  policy  circles  by  gender- 
neutral  proponents  (see  American  Association  of  University  Women  2000).  However,  recent  experimental 
research  suggests  designers  can  increase  girls’  interest  in  computers,  even  in  programming  techniques,  by 
demonstrating  their  relevance  both  to  girls’  non-traditional  and  traditional  interests  (Lynn,  Raphael,  Olefsky  & 
Bachen  forthcoming).  Approaches  that  offer  a single  way  of  constructing  girls’  identities,  including  those  that 
reflect  only  the  interests  they  share  with  boys,  are  bound  to  feel  limiting  for  some  girls. 

In  addition,  cultural  rationales  require  a great  leap  of  faith  that  girls’  use  of  current  multimedia  games, 
regardless  of  how  they  construe  users’  identities,  can  serve  as  a gateway  to  technical  skills  and  careers.  Almost 
none  of  the  current  multimedia  games  foster  an  understanding  of  programming,  systems  or  network  design, 
where  the  gender  gap  is  widest.  Rather,  they  treat  computers  as  a means  to  design  clothes,  disembowel  aliens, 
or  disclose  stories.  If  we  want  to  introduce  girls  to  what  is  compelling  about  working  with  computers,  none  of 
these  design  strategies  reveals  much  that  is  compelling  about  how  computers  work.  If  we  want  girls  to  have 
equal  opportunity  to  shape  the  hardware  and  software  of  the  future,  multimedia  design  needs  to  encourage  girls 
more  overtly  to  explore  how  they  might  design  or  adapt  the  technology  and  to  what  ends. 


Political  Rationales 

Political  arguments  for  equity  stress  the  need  for  women  to  be  fully  informed  and  enfranchised  citizens 
of  the  information  age.  This  approach  suggests  that  girls  and  women  need  to  know  how  technology  works  to 
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participate  in  democratic  processes  of  designing,  implementing  and  regulating  what  Lawrence  Lessig  (1999) 
has  called  “code.”  By  “code”  Lessig  means  both  the  design  of  computer  architecture  (software,  hardware  and 
systems)  but  also  relevant  codes  of  law,  policy  and  ethics.  Girls  need  hands-on  experience  writing  both  kinds 
of  code  according  to  this  rationale,  which  surfaces  occasionally  in  policy  discussions  about  technology  in 
education  (e.g.,  Committee  on  Information  Technology  Literacy  1999;  Alliance  for  Childhood  2000),  but 
almost  never  in  the  literature  on  gender  equity  and  computing. 

Is  there  a gender  gap  in  the  civic  aspects  of  computing?  We  know  far  less  about  this  gap  than  the 
economic  or  cultural  ones,  largely  because  we  have  not  developed  indicators  that  would  allow  us  to  measure  its 
dimensions.  Doing  so  requires  thinking  about  who  shapes  technology  design  and  regulation  within 
corporations,  government,  academia  and  civil  society.  We  could  begin  by  assessing  the  amount  and  quality  of 
women’s  participation  in  making  decisions  about  code  as  corporate  managers,  directors,  market  researchers  and 
designers;  as  members  of  relevant  congressional  committees  and  executive  branch  agencies  that  control 
research  and  development  funding,  marketing  subsidies,  military  procurement,  and  the  like;  as  managers  of 
government  IT  projects;  as  members  of  the  judiciary  that  frequently  hear  high  tech  law  cases;  as  academic 
researchers  and  publishers  on  technology  issues  (not  only  in  computer  science  and  engineering,  but  in  law, 
medicine  and  other  disciplines);  as  program  officers,  presidents  and  trustees  of  foundations  and  nonprofit 
organizations  involved  in  technology  policy  and  research;  as  organizers  of  technology-related  ballot  initiatives; 
and  so  on. 

From  the  civic  standpoint,  girls  need  to  know  how  computers  work,  both  their  capabilities  and  limits, 
primarily  to  develop  personal  and  public  codes  of  technology  ethics,  law,  and  policy.  Girls  need  confidence  in 
their  own  ability  to  effect  change  in  computerized  settings,  democratize  technology  design,  reflect  on  their  own 
choices  about  computers  and  get  feedback  on  their  social  impacts.  They  need  to  develop  critical  thinking  and 
ethical  reasoning  skills  both  with  computers  and  about  them.  A single  piece  of  educational  software  might 
allow  them  to  do  both,  for  example,  by  simulating  the  impact  of  different  methods  of  computer  donation, 
recycling  or  disposal  on  a town’s  schools,  environment  and  economy.  A game  might  be  built  around  the 
challenges  of  keeping  and  communicating  secrets  in  a school  where  web  and  email  monitoring  software  is 
deployed,  spurring  exploration  of  how  such  programs  work  and  their  legal  and  ethical  dimensions. 


Gender-Specific  Features 

Gender-Neutral  Features 

“Code”  Issues 

• Exploring  realistic  social 
relationships 

• Role  playing  in  familiar  settings 

• Nurturing 

• Negotiation  and  conflict 
resolution 

• Creativity  and  art 

• Exploring  worlds  rather  than 
conquering  them 

• Complex  characters 

• Rehearsing  teen  and  adult  issues 

• Customizability  (ability  to  create 
own  characters,  narratives, 
objects,  pace) 

• Strategy  and  skill  development 

• Multiple  levels,  areas,  narratives 

• Ability  to  use  collaboratively, 
socially 

• Puzzles  and  mysteries 

• Focus  on  a goal 

• Privacy  (especially  of  children 
online) 

• Access,  digital  divide 

• Security 

• Virtual  communities  (and  their 
relationship  to  physical  ones) 

• Educational  technology 
investment  and  uses 

• Commercial  pressures  on  the 
Internet 

• Open  and  closed  source 
software;  monopolistic 
practices 

• Environmental  impacts  of 
computing 

Figure  2:  Some  Engaging  Design  Features  for  Girls  and  Contemporary  Issues  of  “Code” 

Media  designers  concerned  with  boosting  female  participation  in  writing  the  krger  code  of  the 
information  age  would  be  well-served  by  design  principles  common  to  constructivist  (Papert  1993)  and  feminist 
pedagogy  (Cassell  1998).  Designers  would  foster  users’  active  construction  of  games  and  web  sites  rather  than 
laying  down  fixed  rules  and  rigid  structures.  Designers  could  develop  content  from  the  start  in  consultation 
with  girls  about  meaningful  problems  of  interest  to  them,  showing  girls  how  computing  is  relevant  to  the 
sources,  solutions  and  arenas  for  exploring  these  problems.  Media  could  encourage  users’  autonomous 
experimentation  and  discovery  with  the  computer  itself  rather  than  leading  them  down  narrow  paths  toward 
predetermined  conclusions  or  lessons.  Design  might  foster  greater  cooperation  and  interactivity  among  users. 
This  kind  of  approach  aims  to  inspire  ongoing,  reflexive  consideration  of  girls’  own  experience  with  computer 
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technology,  “incorporating  and  coordinating  considerations  of  self,  others,  and  society”  (Kahn,  Jr.  & Friedman 
1998,  p.  165).  It  asks  girls  to  write  not  only  programming  code,  but  regulatory  code. 

Fig.  2 presents  some  design  features  that  research  has  found  are  specifically  compelling  for  girls 
(Beato  1997;  Kafai  1996),  and  for  girls  and  boys  (American  Association  of  University  Women  2000),  as  well  as 
some  of  today’s  most  widely  debated  issues  of  code  in  technology  law,  policy  and  ethics.  To  serve  girls’  needs 
and  rights  as  citizens,  we  can  start  drawing  more  lines  between  the  columns,  employing  design  features  that 
work  for  girls  to  encourage  them  to  explore  questions  of  code  and  discover  how  they  might  shape  the  politics 
and  architecture  of  computing. 


Conclusion 

This  review  of  the  literature  suggests  three  distinct  rationales  for  closing  the  gender  gap  in  computing, 
indeed  three  different  gaps:  economic,  cultural  and  political.  It  argues  that  scholars  and  policymakers  have 
neglected  the  political  gap  and  urges  software  and  wbb  designers  to  devote  greater  attention  to  addressing  it. 
There  are  a number  of  good  reasons  for  doing  so.  First,  the  cultural  divide  in  computing  already  appears  to  be 
narrowing,  as  women  go  online  in  equal  numbers  and  use  the  internet  for  communication.  Second,  by  pursuing 
the  approach  suggested  here,  designers  would  more  overtly  foster  girls’  understanding  of  computing 
architecture,  contributing  to  closing  the  economic  gap  as  well  by  helping  to  familiarize  girls  with  how  the 
technology  works  as  well  as  the  social  choices  encoded  in  the  hardware  and  software  we  use.  Finally,  and 
perhaps  most  important,  this  kind  of  design  strategy  would  develop  the  kind  of  skills  that  all  girls,  and  boys, 
need  for  effective  citizenship  in  the  information  age:  critical  thinking  with  and  about  computing’s  impacts  on 
our  lives,  and  an  appreciation  of  the  relevance  of  information  technology  ethics  and  policy  to  gender-traditional 
and  nontraditional  interests. 
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Abstract:  Heuristic  evaluation  is  a methodology  for  investigating  the  usability  of  software  originally  devel- 
oped by  Nielsen  (1993,  2000).  Nielsen’s  protocol  was  modified  and  refined  for  evaluating  e-leaming  programs 
by  participants  in  a doctoral  seminar  held  at  The  University  of  Georgia  in  2001.  The  modifications  primarily 
involved  expanding  Nielsen’s  original  ten  heuristics  (developed  for  software  in  general)  to  fifteen  heuristics 
(designed  to  be  more  closely  focused  on  e-leaming  programs).  The  application  of  this  evaluation  protocol  to  a 
commercial  e-leaming  program  supported  enhancements  in  the  usability  of  the  program. 


Introduction 

Heuristic  evaluation  is  a methodology  for  investigating  the  usability  of  software  originally  developed  by  Jakob  Niel- 
sen (1993,  2000),  a widely  acknowledged  usability  expert.  According  to  Nielsen  (1994),  heuristic  evaluation  “involves 
having  a small  set  of  evaluators  examine  the  interface  and  judge  its  compliance  with  recognized  usability  principles 
(the  "heuristics”).”  Nielsen’s  original  protocol  for  heuristic  evaluation  can  be  found  on  the  Web  at: 
http://www.useit.com/papers/heuristic/. 

For  this  study,  Nielsen’s  protocol  was  modified  and  refined  for  evaluating  e-leaming  programs  by  the  participants  in 
a doctoral  seminar  held  at  The  University  of  Georgia  between  August  and  December  2001.  The  modifications  primar- 
ily involved  expanding  Nielsen’s  original  ten  heuristics  (developed  for  evaluating  software  in  general)  to  fifteen  heu- 
ristics (developed  for  evaluations  of  e-leaming  programs).  This  paper  describes  the  set  of  fifteen  e-leaming  heuristics 
as  well  as  the  protocol  that  guided  the  evaluation  process.  In  addition,  the  results  of  the  application  of  this  heuristic 
evaluation  protocol  to  a commercial  e-leaming  training  program  are  illustrated. 

Method 

Figure  1 presents  the  protocol  we  developed  for  the  heuristic  evaluation  of  e-leaming  programs.  Figure  2 presents  a 
set  of  fifteen  usability  and  instructional  design  heuristics  which  should  be  viewed  as  a work  in  progress.  Further  re- 
finements are  likely  to  be  made  based  upon  the  application  of  the  heuristics  to  various  e-leaming  products  as  well  as 
the  feedback  we  receive  from  others  who  use  this  tool  for  their  own  heuristic  evaluations. 

Seven  instructional  technology  doctoral  students  and  two  faculty  members  (one  from  cognitive  psychology  and  one 
from  instructional  technology)  were  involved  in  the  development  and  initial  application  of  these  heuristics  and  the 
accompanying  protocol.  There  were  twenty  heuristics  in  our  initial  set.  These  were  generated  through  a critical 
analysis  of  Nielsen’s  (1994)  heuristics  as  well  as  several  small  and  whole  group  “brainstorming”  sessions  aimed  at 
identifying  additional  heuristics  for  the  usability  evaluation  of  e-leaming  programs. 

The  initial  twenty  heuristics  and  the  protocol  were  first  applied  to  a commercial  e-leaming  program  called  “GMP  Ba- 
sics” designed  for  the  American  Red  Cross  by  LeamWright,  an  award-winning  company  located  in  Rockville,  Mary- 
land, USA  (http://www.leamwright.com).  LeamWright  specializes  in  the  development  of  e-leaming  programs  for 
regulated  industries  such  as  pharmaceuticals.  Training  for  these  industries  is  mandated  by  the  U.S.  Food  and  Drug 
Administration  (FDA).  From  the  perspective  of  the  FDA,  the  collection  of  blood  and  plasma  products  is  a regulated 
activity.  While  not  everyone  at  the  Red  Cross  is  involved  in  collecting,  processing,  or  distributing  blood  or  plasma 
products  (e.g.,  some  workers  are  involved  exclusively  in  disaster  relief),  all  workers  in  blood  services  must  learn  the 
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principles  encompassed  in  the  FDA’s  regulations  for  Good  Manufacturing  Practices  (GMP)  The  courseware  we 
evaluated  was  presented  on  a CD-ROM,  although  it  will  eventually  be  accessible  via  a Web-based  intranet. 


Protocol  for  ^Learning  Heuristic  Evaluation 

This  instrument  and  protocol  are  intended  for  use  by  instructional  designers  and  other  experts  engaged  in  heuristic  evaluations  of 
e-leaming  programs.  The  instrument  itself  lists  fifteen  heuristics  for  e-leaming  programs,  some  of  which  are  based  upon  Jakob 
Nielsen’s  widely  used  protocol  for  heuristic  evaluation  of  any  type  of  software  (http://useit.com/papers/heuristic/),  and  the  rest 
of  which  are  based  upon  factors  related  to  instructional  design.  Although  we  have  tried  to  be  comprehensive,  experts  may  decide 
to  add  new  heuristics  deemed  relevant  to  the  types  of  e-leaming  programs  being  evaluated  or  to  the  expert’s  specific  expertise. 

Steps: 

1 . An  expert  should  review  the  heuristics  and  accompanying  “Sample  questions  to  ask  yourself’  in  the  instrument  before  review- 
ing an  e-leaming  product.  The  expert  should  modify  the  instrument  if  needed,  by  adding,  deleting,  or  changing  heuristics. 

2.  It  is  important  that  the  expert  spend  substantial  time  exploring  the  e-leaming  program  before  beginning  the  actual  heuristic 
evaluation.  Ideally,  the  expert  will  assume  the  role  of  typical  learner  who  would  use  this  e-leaming  program.  Before  beginning  the 
review,  the  expert  should  be  given  (or  try  to  discover)  background  information  related  to  the  e-leaming  program  such  as: 

a.  Target  audience  and  learner  characteristics  : A thorough  description  of  the  intended  audience  and  their  learner  characteris- 
tics (e.g.,  education  level,  motivation,  incentive,  and  computer  expertise)  will  enable  the  expert  to  judge  the  appropriateness  of  the 
user  interface  and  other  aspects  of  the  program’s  usability  in  an  informed  manner. 

b.  Instructional  goals  and  objectives  : The  expert  should  know  as  much  as  possible  about  the  needs  that  the  e-leaming  pro- 
gram is  intended  to  address,  ideally  in  terms  of  clear  goals  and  objectives. 

c.  Typical  context  for  using  this  program:  Realistic  scenarios  for  when,  where,  and  how  the  e-leaming  program  will  be  used 
should  be  known  by  the  expert. 

d.  Instructional  design  strategies  used  in  the  program:  If  possible,  a description  of  the  design  specifications  used  in  develop- 
ing the  e-leaming  program  should  be  provided  to  the  expert  so  that  the  expert’s  judgment  of  the  appropriateness  of  the  instruc- 
tional design  strategies  are  informed  with  respect  to  the  instructional  designer’s  intentions. 

e.  The  status  of  the  program  s development  and  possibilities  for  change:  The  expert  should  be  informed  as  to  where  the  pro- 
gram is  in  the  development  cycle  (e.g.,  an  early  prototype,  beta  version,  or  older  version  under  consideration  for  redesign). 

3.  After  spending  enough  time  to  become  familiar  with  the  program,  the  expert  should  go  through  it  from  beginning  to  end  to  con- 
duct the  actual  heuristic  evaluation.  (With  lengthy  programs,  the  expert  may  only  review  a representative  sample  of  the  program.) 

4.  The  expert  should  make  note  of  every  usability  problem  found.  For  each  problem,  the  expert  should  identify  the  heuristic  it 
violates,  and  then  give  it  a severity  rating  using  the  severity  scale  below.  If  the  problem  cannot  be  attributed  to  a violation  of  a 
specific  heuristic,  the  expert  should  make  a note  of  this.  (If  a number  of  problems  are  found  that  cannot  be  associated  with  spe- 
cific heuristics,  this  may  suggest  the  need  for  the  development  of  new  heuristics.) 

Severity  Scale 

1 . cosmetic  problem  only;  need  not  be  fixed  unless  extra  time  is  available 

2.  minor  usability  problem;  fixing  this  should  be  given  low  priority 

3.  major  usability  problem;  important  to  fix;  so  should  be  given  a high  priority 

4.  usability  catastrophe;  imperative  to  fix  before  this  product  is  released 

5.  After  all  the  usability  problems  arc  found,  the  expert  should  go  back  though  them  and  give  each  one  an  extensiveness  rating  us- 
ing the  extensiveness  scale  below. 

Extensiveness  Scale 

1.  this  is  a single  case 

2.  problem  occurs  in  several  places  in  the  program 

3.  this  problem  is  widespread  throughout  the  program 

6.  Most  heuristic  evaluations  involve  4 to  5 experts.  Once  all  experts  have  completed  their  evaluations,  they  may  be  brought  to- 
gether for  a debriefing  led  by  a moderator.  The  discussion  of  the  usability  problems  may  be  videotaped  for  further  analysis.  If  ma- 
jor differences  appear  in  the  problems  found  or  the  ratings  given,  the  moderator  may  try  to  get  the  experts  to  resolve  their 
differences  and  reach  consensus.  The  experts  may  also  be  asked  to  suggest  strategies  for  resolving  the  major  usability  problems 
found. 
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7.  A heuristic  evaluation  report  should  then  be  compiled.  Bar  charts,  tables,  and  other  illustrations  should  be  used  to  display  the 
results.  If  feasible,  screen  captures  should  also  be  incorporated  into  the  report  to  illustrate  major  problems  as  well  as  suggested 
enhancements. 

8.  The  most  important  component  of  the  heuristic  report  is  a set  of  recommendations  for  improving  the  usability  of  the  e-leaming 

program.  These  should  be  as  specific  as  possible  to  provide  the  designers  with  the  information  they  need  to  eliminate  the  prob- 
lems  and  improve  the  e-leaming  program. 


Figuie  1 . Fiutucul  fui  iicunslic  cvaluaiiuii  ufc-icaming  programs. 

1 . Visibility  of  system  status:  The  e-leaming  program  keeps  the  learner  informed  about  what  is  happening,  through  appropriate 
feedback  within  reasonable  time. 

Sample  questions  to  ask  yourself: 

a.  When  modules  and  other  components  of  the  e-leaming  (e.g.,  streaming  video)  are  downloading,  is  the  status  of  the  download 
communicated  clearly? 

b.  Is  the  user  provided  with  information  that  indicates  that  the  e-leaming  program  is  operating  correctly? 

2.  Match  between  system  and  the  real  world:  The  e-leaming  program’s  interface  employs  words,  phrases  and  concepts  familiar 
to  the  learner  or  appropriate  to  the  content,  as  opposed  to  system-oriented  terms.  Wherever  possible,  the  e-leaming  program  util- 
izes real-world  conventions  that  make  information  appear  in  a natural  and  logical  order. 

Sample  questions  to  ask  yourself: 

a.  Does  the  e-leaming  program’s  interactive  design  utilize  metaphors  that  are  familiar  to  the  learner  or  related  to  the  specific  con- 
tent of  the  program? 

b.  Is  the  interface  “user  friendly,”  given  the  content  of  the  program  and  its  target  audience? 

3.  Error  recovery  and  exiting:  The  e-leaming  program  allows  the  learner  to  recover  from  input  mistakes  and  provides  a clearly 
marked  “exit”  to  leave  the  program  without  requiring  the  user  to  go  through  an  extended  dialogue. 

Sample  questions  to  ask  yourself: 

a.  Does  the  e-leaming  program  distinguish  between  input  errors  and  cognitive  errors,  allowing  easy  recovery  from  the  former  al- 
ways, and  from  the  latter  when  it  is  pedagogically  appropriate? 

b.  Does  the  program  allow  the  learner  to  leave  whenever  desired,  but  easily  return  to  the  closest  logical  point  in  the  program? 

4.  Consistency  and  standards:  When  appropriate  to  the  content  and  target  audience,  the  e-leaming  program  adheres  to  general 
software  conventions  and  is  consistent  in  its  use  of  different  words,  situations,  or  actions. 

Sample  questions  to  ask  yourself: 

a.  If  appropriate  to  the  content  and  target  audience,  does  the  e-leaming  product  adhere  to  widely  recognized  standards  for  soft- 
ware interactions  (e.g.,  going  back  in  a Web  browser)? 

b.  If  the  e-leaming  program  does  not  utilize  common  software  conventions  for  interactions,  are  the  novel  interactions  appropriate 
for  the  content  and  target  audience? 

c.  Does  the  program  maintain  an  appropriate  level  of  consistency  in  its  design  from  one  part  of  the  program  to  another? 

5.  Error  prevention:  The  e-leaming  program  is  designed  to  prevent  common  problems  from  occurring  in  the  first  place. 

Sample  questions  to  ask  yourself: 

a.  Is  the  program  designed  so  that  the  learner  recognizes  when  he/she  has  made  a mistake  related  to  input  rather  than  content? 

b.  Is  the  e-leaming  program  designed  to  provide  a second  chance  when  unexpected  input  is  received  (e.g.,  “You  typed  “bat”  in  re- 
sponse to  the  question.  Did  you  mean  “tab?”)? 

6.  Navigation  support:  The  e-leaming  program  makes  objects,  actions,  and  options  visible  so  that  the  user  does  not  have  to  re- 
member information  when  navigating  from  one  part  of  the  program  to  another.  Instmctions  for  use  of  the  program  are  always 
visible  or  easily  retrievable. 

Sample  questions  to  ask  yourself: 

a.  Does  the  interface  of  the  e-leaming  program  speak  for  itself  so  that  extensive  consultation  of  a manual  or  other  documentation 
does  not  interfere  with  learning? 

b.  Does  the  e-leaming  program  provide  user-friendly  hints  and/or  clear  directions  when  the  learner  requests  assistance? 

c.  Does  the  e-leaming  program  include  a map  or  table  of  contents  that  allows  the  learner  to  see  what  has  been  seen  and  not  seen? 
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7.  Aesthetics:  Screen  displays  do  not  contain  information  that  is  irrelevant,  and  “bells  and  whistles”  are  not  gratuitously  added  to 
the  e-leaming  program. 

Sample  questions  to  ask  yourself: 

a.  Are  the  font  choices,  colors,  and  sizes  consistent  with  good  screen  design  recommendations  for  e-leaming  programs? 

b.  Does  the  e-leaming  program  utilize  white  space  and  other  screen  design  conventions  appropriately? 


S,  KeiiJ  and  uucuiiit;iuai.iuii;  The  c-icaiiiiiig  prograi'ii  piOviucS  iicip  and  doCuiVicritatiOii  tiiat  iS  readily  aCCcSSiulc  lO  tiic  UScr  wllcii 
necessary.  The  help  provides  specific  concrete  steps  for  the  user  to  follow.  All  documentation  is  written  clearly  and  succinctly. 
Sample  questions  to  ask  yourself: 

a.  Is  help  provided  that  is  screen  or  context  specific? 

b.  Is  help  or  documentation  available  from  any  logical  part  of  the  e-leaming  program? 

c.  Is  help  or  documentation  written  clearly? 


Figure  2.  Usability  and  instructional  design  heuristics  for  evaluation  of  e-leaming  programs. 


9.  Interactivity:  The  e-leaming  program  provides  content -related  interactions  and  tasks  that  support  meaningful  learning. 

Sample  questions  to  ask  yourself: 

a.  Does  the  e-leaming  program  provide  meaningful  interactions  for  the  user,  rather  than  simply  presenting  long  sections  of  text? 

b.  Does  the  e-leaming  program  engage  the  learner  in  content -specific  tasks  to  complete  and  problems  to  solve  that  take  advantage 
of  the  state-of-the-art  of  e-leaming  capabilities? 

10.  Message  Design:  The  e-leaming  program  presents  information  in  accord  with  sound  information-processing  principles. 
Sample  questions  to  ask  yourself: 

a.  Is  the  most  important  information  on  the  screen  placed  in  the  areas  most  likely  to  attract  attention? 

b.  Does  the  e-leaming  program  follow  good  information  presentation  guidelines  for  organization  and  layout? 

11.  Learning  Design:  The  interactions  in  the  e-leaming  program  have  been  designed  in  accord  with  sound  principles  of  learning 
theory. 

Sample  questions  to  ask  yourself: 

a.  Does  the  e-leaming  program  follow  an  appropriate  learning  design  to  achieve  its  stated  objectives? 

b.  Does  the  e-leaming  program  engage  learners  in  tasks  that  are  closely  aligned  with  the  learning  goals  and  objectives? 

12.  Media  Integration:  The  inclusion  of  media  in  the  e-leaming  program  serves  clear  pedagogical  and/or  motivational  purposes. 
Sample  questions  to  ask  yourself: 

a.  Is  media  included  that  is  obviously  superfluous,  i.e.,  lacking  a strong  connection  to  the  objectives  and  design  of  the  program? 

b.  Is  the  most  appropriate  media  selected  to  match  message  design  guidelines  or  to  support  instructional  design  principles? 

c.  If  appropriate  to  the  content,  are  various  forms  media  included  for  remediation  and/or  enrichment? 

13.  Instructional  Assessment:  The  e-leaming  program  provides  assessment  opportunities  that  arc  aligned  with  the  program  ob- 
jectives and  content. 

Sample  questions  to  ask  yourself: 

a.  If  appropriate  to  the  content,  does  the  e-leaming  program  provide  opportunities  for  self-assessments  that  advance  learner 
achievement? 

b.  If  appropriate  to  the  content,  do  assessments  provide  sufficient  feedback  to  the  learner  to  provide  remedial  directions? 

c.  Wherever  appropriate,  are  higher  order  assessments  (e,g.,  analysis,  synthesis,  and  evaluation)  provided  rather  than  lower  order 
assessments  (e.g.,  recall  and  recognition)? 

14.  Resources:  The  e-leaming  program  provides  access  to  all  the  resources  necessary  to  support  effective  learning. 

Sample  questions  to  ask  yourself: 

a.  Does  the  e-leaming  program  provide  access  to  a range  of  resources  (e.g.,  examples  or  real  data  archives)  appropriate  to  the 
learning  context? 
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b.  If  the  e-leaming  program  includes  links  to  external  World  Wide  Web  or  Intranet  resources,  are  the  links  kept  up-to-date? 

c.  Are  resources  provided  in  a manner  that  replicates  as  closely  as  possible  their  availability  and  use  in  the  real  world? 


15.  Feedback:  The  e-leaming  program  provides  feedback  that  is  contextual  and  relevant  to  the  problem  or  task  in  which  the 
learner  is  engaged. 

Sample  questions  to  ask  yourself: 

a.  Is  the  feedback  given  at  any  specific  time  tailored  to  the  content  being  studied,  problem  being  solved,  or  task  being  completed 

1.11V  iviiiiivi* 

b.  Does  feedback  provide  the  learner  with  information  concerning  his/her  current  level  of  achievement  within  the  program? 

c.  Does  the  e-leaming  program  provide  learners  with  opportunities  to  access  extended  feedback  from  instmctors,  experts,  peers, 
or  others  through  e-mail  or  other  Internet  communications? 


Figure  2.  Usability  and  instructional  design  heuristics  for  evaluation  of  e-leaming  programs  (continued). 

All  seven  doctoral  students  and  the  cognitive  psychology  faculty  member  served  as  the  experts  for  this  heuristic 
evaluation  and  followed  the  protocol  during  a one-week  period  in  October  2001.  After  an  initial  group  review  of  the 
findings,  another  week  was  allowed  to  refine  the  individual  reports  that  each  expert  compiled.  Next,  the  instructional 
technology  (IT)  professor  led  a debriefing  session  during  which  copies  of  all  the  reports  were  made  available  to  all 
experts.  During  this  session,  an  acceptable  level  of  consensus  was  reached  concerning  the  major  usability  and  in- 
structional design  problems,  their  severity,  and  their  extensiveness.  After  the  discussion,  the  IT  professor  synthe- 
sized the  findings  of  the  individual  heuristic  evaluations  into  a succinct  report  that  was  provided  to  the  client. 
Findings 

The  results  of  the  heuristic  evaluation  of  the  GMP  Basics  courseware  identified  a number  of  important,  but  resolv- 
able, problems  with  the  e-leaming  program.  For  example,  one  of  the  heuristics  violated  in  the  initial  version  of  the 
GMP  program  was  Number  3,  Error  Recover  and  Exiting.  In  Figure  3,  the  screen  capture  on  the  left  is  from  a timeline 
game  within  the  GMP  Basics  course  that  did  not  allow  the  learner  to  exit  easily,  short  of  quitting  the  program  alto- 
gether. The  screen  capture  on  the  right  illustrates  the  same  game  in  the  new  version  of  the  e-leaming  program  after 
being  redesigned  based  upon  the  results  of  the  heuristic  evaluation  described  in  this  paper  as  well  as  the  findings  of 
a field  evaluation  conducted  by  the  first  author  in  December  2001.  In  the  revised  version  of  the  program,  the  learner 
can  easily  go  back  to  the  previous  screen  or  exit  this  part  of  the  program. 


Figure  3.  GMP  Basics  timeline  game  screen  captures  before  (left)  and  after  (right)  evaluation. 

Another  heuristic  violated  in  the  early  version  of  GMP  Basics  was  Number  7,  Aesthetics.  In  Figure  4,  the  screen  cap- 
ture on  the  left  is  from  a quiz  show  game  within  first  version  of  the  GMP  Basics  course.  The  screen  capture  on  the 
right  illustrates  the  same  game  within  the  revised  program.  The  aesthetics  as  well  as  the  professional  look-and-feel  of 
the  e-leaming  program  have  been  enhanced  in  the  new  version. 
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Figure  4.  GMP  Basics  quiz  show  game  screen  captures  before  (left)  and  after  (right)  evaluation. 


As  a final  illustration  of  the  results  of  this  heuristic  evaluation,  the  Message  Design  (Heuristic  Number  10)  of  the  e- 
leaming  program  was  enhanced.  In  Figure  5,  the  screen  capture  on  the  left  is  from  a typical  instructional  segment 
within  the  GMP  Basics  course  that  some  learners  found  difficult  to  read  because  of  the  extensive  use  of  light  text  on 
darker  backgrounds.  The  screen  capture  on  the  right  illustrates  the  same  screen  with  the  primary  text  appearing  in  a 
dark  font  on  a lighter  background.  The  screen  on  the  right  also  illustrates  improvements  in  the  Navigation  Support 
(Heuristic  Number  6)  that  were  made  after  this  evaluation. 


Figure  5.  GMP  Basics  instructional  screen  captures  before  (left)  and  after  (right)  evaluation. 

Interestingly,  both  the  heuristic  evaluation  and  the  field  evaluation  revealed  relatively  few  instructional  design  prob- 
lems of  the  kind  that  would  be  represented  by  Heuristics  Numbers  9 ~ 15.  In  other  words,  the  original  GMP  program 
represented  sound  principles  of  instructional  design,  but  the  usability  of  that  version  of  the  program  needed  to  be 
strengthened.  Of  course,  usability  is  critical  to  a learner’s  experience  with  an  e-leaming  program,  and  the  effective- 
ness of  even  the  most  instructional ly  sound  programs  will  be  decreased  if  the  learner’s  experience  suffers  from  prob- 
lems related  to  navigation,  orientation,  visual  appeal,  and  other  usability  criteria  (Reeves  & Carter,  2001). 

Discussion 

There  are  limitations  to  this  study  that  must  be  clarified.  This  heuristic  evaluation  took  place  over  more  than  two 
weeks  whereas  most  heuristic  evaluations  generally  last  only  one  or  two  days.  There  were  eight  experts  involved  in- 
stead of  the  usual  four  or  five  recommended  by  Nielsen  (1994).  In  addition,  changes  made  in  the  GMP  Basics  program 
were  informed  by  a field  evaluation  conducted  by  the  first  author  at  a pharmaceutical  manufacturing  plant  with  real 
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workers,  not  by  the  heuristic  evaluation  alone.  It  is  impossible  to  attribute  exactly  which  enhancements  were  based 
upon  which  of  the  two  evaluations.  Nonetheless,  LeamWright  expressed  great  satisfaction  with  the  results  of  the 
heuristic  evaluation  protocol  and  has  requested  that  the  protocol  be  applied  to  other  programs  being  developed. 


Although  heuristic  evaluation  is  fast,  convenient,  and  economical,  it  is  not  sufficient  for  evaluating  e-leaming  pro- 
grams, and  usability  testing  is  advised  (Nielsen,  2000).  Heuristic  evaluation  and  usability  testing  usually  detect  dif- 
ferent types  of  usability  problems  (Nielsen,  1994).  Heuristic  evaluation  is  often  conducted  as  a supplement  to 
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usability  heuristics  were  insufficient  for  e-leaming  programs,  and  we  developed  and  refined  the  fifteen  heuristics  pre- 
sented in  Figure  2.  We  encourage  others  to  adapt  our  e-leaming  heuristics  to  their  own  evaluation  needs.  We  only 
request  that,  if  possible,  the  results  of  subsequent  evaluations  be  shared  so  that  we  can  further  refine  this  tool. 
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The  Role  of  the  Facilitator  in  Active  Learner  Support 


Franz  Reichl,  Ursula  E,  Vierlinger 

University  Extension  Centre,  Vienna  University  of  Technology 


Background 

The  SOCRATES/ODL  project  FACILE  (FACIlitated  open  distance  Learning  for  continuing  Engineering  education, 
1998-2000)  was  a starting  point  for  universities’  centres  for  continuing  engineering  education  to  complement 
continuing  education  by  e -Learning  provisions  with  active  learner  support.  The  experiences  made  during  the 
project  are  currently  applied  for  e-Leaming  in  university  continuing  education  and  in  training  for  companies. 

e-Learning  for  Practitioners  and  Professionals 

In  continuing  education,  learners  acquire  knowledge  for  a variety  of  different  reasons.  Practitioners  and 
professionals  come  to  a course  as  learners  with  already  a high  level  of  expertise.  Motivation  to  learn  is  generally 
high,  but  does  not  regard  ’’formal”  completion  of  the  course.  Instead,  participation  in  such  a course  generally  is 
driven  by  the  need  to  solve  a specific  problem  at  the  work  place.  This  specific  need  varies  widely  among 
individual  learners,  as  does  the  familiarity  with  the  subject  and  the  level  of  mastery  of  the  skills  necessary 
[Payr/Reichl/CsanyWierlinger2001]. 

The  learner’s  motivation  and  needs  or  demands  to  learn  determines  the  contents  and  type  of  learning.  Following 
Bloom’s  taxonomy  [Bloom  1954],  Sparkes  distinguishes  four  different  types  of  learning: 

• learning  factual  knowledge; 

♦ learning  mental  or  physical  skills; 

♦ learning  understanding  or  comprehension; 

• learning  attitudes  and  values  [Sparkes  1985,  pp.4-10;  Rowntree  1992,  p.46]. 

Sabine  Payr  complements  these  learning  and  knowledge  types  with  ’’mastery”,  i.e.  transfer  of  knowledge  and 
skills  to  personal  practice  [Payr/Reichl/CsanyiA^ierlinger  200 1 ; Baumgartner/Payr  1999;  Payr  1999]. 

Learner  Support  by  Facilitation 

Individual  learners  in  continuing  education  learn  for  different  reasons,  want  to  achieve  different  objectives,  have 
different  previous  knowledge,  have  developed  different  strategies  to  acquire  knowledge,  etc.  It  is  neither 
appropriate  nor  economically  feasible  to  deliver  completely  different  learning  packages  to  meet  all  these  different 
learning  needs.  Thus,  an  appropriate  educational  method  and  delivery  system  has  to  provide  the  flexibility 
necessary  to  apply  general  learning  material  in  heterogeneous  learning  situations. 

Reflection,  group  learning,  consultation  and  guidance,  as  well  as  interaction  are  of  utmost  importance  for  the 
success  of  any  learning  activity.  Especially  in  such  a heterogeneous  learning  situation,  the  effectiveness  of 
learning  significantly  profits  from  interaction  between  individual  learners  who  may  gain  different  viewpoints  from 
discussing  the  subject  with  their  colleagues.  Learner  support  services  thus  constitute  an  indispensable 
complement  to  packaged  distance  learning  materials  to  ensure  interactivity  and  feedback. 

Especially  in  continuing  education,  self-directed  learning  is  most  effective.  Significant  learning  is  much  improved 
if  learners  can  be  given  the  opportunity  to  discover  laws,  concepts  and  theories  - they  should  not  be  tought  but 
can  be  coached  [Sparkes  1 985,  p.9;  Schon  1 987,  p.  1 7,  quoting  Dewey  1 974,  p.  1 5 1 ].  A facilitator  should  be  willing 
and  interested  in  helping  the  learners;  facilitators  should  not  repeat  the  contents  or  immediately  answer  the 
learners’  questions;  they  shall  help  the  group  of  learners  to  discover  the  answers  by  themselves  [Gibbons  1977, 
pp.18-19;  Rowntree  1992,pp.79-80]. 

Experiences  with  Tutored  Video  Instruction  [Gibbons/Kincheloe/Down  1977]  and  with  facilitated  e-Leaming  show 
that  people  do  not  learn  directly  from  learning  material  - the  learning  material  rather  acts  as  an  ’’organiser”  of  the 
learning  process.  For  the  success  of  the  learning  process,  the  best  results  occur  when  the  participants  can  bring 
problems,  examples  and  experience  from  their  own  work  into  the  learning  process.  Thus,  it  is  important  to 
integrate  real  life  experience  and  problems  into  the  process  [Reichl  1996]. 

Experiments  with  different  Models  for  Facilitation 

Typically,  ’’facilitation”  in  ODL  (Open&Distance  Learning)  is  understood  as  the  support  of  learner  activities  at  a 
remote  learning  site  [Sherry  1996].  The  facilitator  is  the  one  and  often  only  person  involved  in  a course  with  whom 
the  learners  are  in  face-to-face  contact.  The  role  of  a facilitator  is  to  ensure  communication  between  faculty  and 
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students,  to  ensure  smooth  running  of  technical  equipment,  to  respond  to  students’  questions  concerning  course 
administration,  learning  facilities,  but  rarely  on  the  subject  itself  [Pay r/Reichl/CsanyWierlinger  2001], 

In  the  FACILE  project,  on-site  facilitation  was  replaced  by  distance  facilitation.  While  in  most  ODL  courses  the 
role  of  support,  tutoring,  coordination  and  contact  at  a distance  is  taken  over  by  the  teacher/trainer  him/herself, 
this  model  did  not  work  out  satisfactorily  for  this  new  type  of  networked  ODL.  The  first  approach  taken  in  the 
project  was  to  make  the  author  of  each  module  responsible  for  its  delivery.  Different  content  areas  would  be 
facilitated  by  different  persons.  Learners  experienced  problems  with  this  approach,  for  several  reasons: 

• Each  author  brought  his/her  own  concepts  and  methods  of  ODL  to  the  undertaking. 

• Learners  did  not  know  any  of  the  authors  personally  and  did  not  have  time  to  get  to  know  (and  to  adapt  to) 
their  differing  methods  and  pedagogic  backgrounds. 

® The  individual  authors  of  the  modules  were  ’’bound”  to  the  specific  contents  they  wanted  to  deliver,  so  that 
no  continuous  course  activities  could  develop. 

For  the  following  attempts,  the  project  partners  thus  separated  the  role  of  the  facilitator  from  the  role  of  the 
content  providers  and  experts  and  to  assign  one  single  person  as  the  facilitator.  During  the  pilot  run  of  the 
FACILE  course,  the  role  of  the  facilitator  was  a ’’passive”  one,  being  available  for  questions  and  problems  when 
needed,  but  the  facilitator  did  not  intervene  actively  in  their  learning  process.  The  drop  out  rate  of  this  pilot  run 
was  around  80%. 

Experiences  showed  that  learners  need  to  be  guided  actively  through  the  course.  In  consequence,  the  project 
partnership  developed  a didactical  and  pedagogical  concept  to  provide  effective  online  facilitation.  The 
(tele-)presence  of  a single  facilitator  in  the  FACILE  course  as  delivered  in  its  final  form  turned  out  to  be  a 
substantial  factor  in  providing  stability  and  continuity  in  this  networked  ODL  environment. 

The  facilitator  did  not  only  co-ordinate  learners’  activities,  but  also  became  a mediator  between  the  learners  and 
the  different  authors  and  experts  in  the  field.  While  the  modules  still  were  authored  by  different  people,  learner 
activities  were  designed  and  monitored  centrally  and  over  the  whole  duration  of  the  course.  This  active 
facilitation  has  turned  out  to  be  the  key  to  success  in  the  FACILE  courses.  The  drop  out  rate  of  the  second 
course  run  was  only  15%  [Reichl/Payr/Csanyi/Vierlinger2001]. 

Roles  and  Tasks  in  Facilitated  ODL 

In  our  approach  for  learner  support  through  active  facilitation,  it  is  important  to  clearly  distinguish  between  the 
different  roles  of  actors  involved,  the  most  important  being: 

• Learners; 

• content  providers  or  authors  of  learning  material; 

• experts  in  the  subject  area; 

• facilitators. 

It  is  not  necessary  that  these  different  roles  be  ’’played”  by  different  persons  - the  roles  of  expert  and  content 
provider  can  be  played  by  one  and  the  same  person,  as  can  be  the  roles  of  expert  and  facilitator,  and  as  in  peer 
tutoring  even  one  of  the  learners  could  take  over  the  facilitator’s  role-,  but  for  the  success  of  our  approach  to  e- 
Leaming,  it  is  important  to  have  these  different  roles  represented. 

The  central  and  most  important  role  in  our  approach  is  the  one  of  the  facilitator.  The  facilitator’s  role  could  best 
be  compared  to  a ’’spider  in  the  web”,  tying  together  the  most  divers  e persons,  interests  and  activities.  The  most 
important  task  of  a facilitator  is  to  take  ’’motivating”  action,  not  by  re-enforcing  extrinsic  motivation  (e.g.  by 
sanctions),  but  by  building  a learning  group  that  develops  its  own  social  dynamics  and  commitments: 

• the  facilitator  is  the  primary  contact  person  for  all  the  participants  and  contacts  authors  and  experts; 

• the  facilitator  has  to  take  the  initiative  e.g.  in  soliciting  questions,  contributions  or  personal  information  from 

participants;  (s)he  elicits  questions  from  participants  and  acts  as  moderator  between  the  learning  group  and 
the  authors/experts; 

• the  facilitator  keeps  track  of  and  monitors  each  participants’  activities,  e.g.  by  reminding  them  of  their  tasks; 

• the  facilitator  creates  a social  learning  environment  within  a virtual  learning  group,  using  the  available 

technology. 

In  short,  the  facilitator  has  to  react  timely  to  every  input  by  a participant,  triggers  self  help  and  group  support 
instead  of  answers  by  facilitator  or  expert,  and  keeps  the  team  learning  going  by  setting  tasks  and  deadlines,  by 
ensuring  information  flow  among  participants,  by  being  supportive  and  communicative  \Payr  2000]. 

While  a content  provider  or  expert  has  to  be  qualified  in  his/her  subject  area,  the  facilitator  - unless  (s)he  is  the 
same  person  - does  not  have  to  be  overly  qualified  in  the  subject  area  (but  should  have  enough  insight  into  the 
content  to  decide  whether  a question  from  a learner  can  be  solved  within  the  learning  group  or  needs  the  advice 
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of  an  expert);  the  facilitator  has  to  be  technically  and  socially  competent  in  communication  technologies  and 
communication  processes.  If  this  role  is  played  by  one  of  the  experts,  this  person  also  has  to  hold  this  necessary 
qualification. 


Since  the  most  important  type  of  learning  (as  described  above)  that  can  take  place  in  an  ODL  course  for 
practitioners  is  learning  mental  or  physical  skills,  intensive  assistance  for  the  learners  by  active  facilitation  from  a 
socially  competent  person  has  turned  out  to  be  the  key  to  success  in  e-Leaming  courses. 
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Abstract:This  paper  discusses  the  design  and  pedagogy  underpinning  the  use  of  online 
quiz  items  in  which  students  are  presented  with  a range  of  questions  designed  to 
enhance  understanding  of  complex  linguistic  constructions.  It  explores  the  design  of 
different  types  of  quiz  question  from  the  perspective  of  pedagogy  and  cognitive 
demand.  The  particular  types  of  questions  presented  in  this  online  learning  setting 
demonstrate  a formative  approach  to  assessment,  closely  integrated  with  learning 
processes.  A matrix  of  questions  is  presented  using  Bloom’s  taxonomy  showing  the 
type  of  question,  pedagogical  underpinnings  and  cognitive  skills  required.  The 
implication  of  the  p^er  is  that  automated  quiz  type  questions  do  not  necessarily  imply  a 
narrow  focus  on  recall,  but  can  be  designed  to  assess  a range  of  learning  processes. 


Theoretical  background:  CAA 

Educators  can  be  in  no  doubt  of  the  demands  of  society  for  lifelong  capable  learners  who  are  able  to  perform 
cognitive,  metacognitive  and  metacognitive  tasks  and  demonstrate  competencies  such  as  problem  solving, 
critical  thinking,  questioning,  searching  for  information,  making  judgments  and  evaluating  information 
(Reeves,  2000;  Oliver  & McLoughlin,  2001).  Figure  I represents  the  changing  nature  of  assessment,  showing 
a transition  from  a focus  on  testing  to  a focus  on  learning  and  transfer  of  understanding.  The  traditional 
approach  to  assessment  was  largely  a form  of  objective  testing  which  valued  students'  capacity  to  memorize 
facts  and  then  recall  them  during  a test  situation.  Testing  was  concerned  with  measuring  a range  of  cognitive 
skills,  though  many  of  these  tests  relied  on  quantifiable  approaches  rather  than  qualitative  displays  of  skills 
and  knowledge.  Magone  et  al  (1994)  called  this  the  one  right  answer  mentality. 

testing  measuring  competence  transfer/leaming 

Figure  1:  Continuum  of  assessment 

The  next  form  of  assessment  depicted  in  the  continuum  is  the  measurement  of  competencies,  or  what  we  call 
'sequestered  problem  solving'  (Schwartz  et  al,  2000).  In  these  contexts  students  are  asked  to  solve  problems  is 
isolation  and  without  the  resources  that  are  typically  available  in  the  real  world  such  as  texts.  Web-resources 
and  peers.  Often  these  tests  of  aptitude  are  single  shot,  and  summative  rather  than  formative.  In  contrast, 
assessment  that  supports  learning  and  knowledge  transfer  provides  the  basis  for  future  learning,  and 
continuing  motivation  to  learn  (far  right  of  the  continuum  in  Figure  2).  This  approach  is  sometimes  called  the 
alternative  assessment  movement,  as  it  is  concerned  with  authentic  performance  (Gumming  & Maxwell, 

1999).  Both  testing  and  measuring  competence  as  forms  of  assessment  have  been  critiqued  as  being 
controlling,  limiting  and  contrary  to  student  centered  teaching  and  learning.  Morgan  and  O'Reilly  (1 999)  add 
the  following  criticisms  of  traditional  assessment  practices: 

• a lack  of  variety  and  autonomy  and  student  choice; 

• lack  of  applied  work-based  and  project  based  learning; 
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• overuse  of  summative  forms  of  assessment; 

• limited  use  of  peer  and  self-assessment  strategies. 

Other  indicators  of  the  need  to  rethink  online  assessment  have  come  from  Bull  & McKenna  (2000)  who 
argue  that  "the  development  and  integration  of  computer-aided  assessment  has  been  done  in  an  ad  hoc 
manner".  In  a similar  vein,  Angelo  (1999)  maintains  that  we  need  a more  compelling  vision  of  assessment, 
research-based  guidelines  for  learner-centered  assessment,  and  a new  mental  model  of  assessment. 


Online  assessment:  Are  multiple  choice  items  appropriate? 

Computer-based  assessment  using  a range  of  question  types  has  been  shown  to  have  positive  effects  on 
student  learning,  as  students  appreciate  the  speed  of  marking  and  the  feedback  provided  (Thelwall,  2000). 
Multiple  choice  items  can  be  designed  to  assess  a range  of  competencies  relating  to  student  performance 
rather  than  concentrate  on  ‘objective’  questions  that  text  recall  of  discrete  facts  (Brown  et  al,  1999). 
Multiple  choice  items  can  be  used  for  both  formative  and  summative  assessment,  and  thus  to  broaden  the 
range  of  skills  that  is  assessed.  This  paper  describes  the  use  of  a system  of  computerized  assessment 
utilising  a variety  of  question  types  to  assess  higher  order  skills.  Despite  the  skepticism  surrounding  the  use 
of  multiple  choice  items  as  a means  to  assess  the  skills  of  analysis,  synthesis  and  evaluation,  carefully 
crafted  questions  can  overcome  this  limitation  (Haladyna,  1997). 


Context:  Extending  the  use  of  multiple  choice  items 

The  context  of  the  study  was  a year  level  university  course  of  study  in  Linguistics  delivered  online. 

In  this  unit  students  are  required  to  develop  and  apply  skills  in  analyzing  the  grammar  of  a language  which 
they  have  not  encountered  before.  Intended  learning  outcomes  are  analytical  skills,  the  capacity  to 
synthesise  conceptual  knowledge  and 

It  was  decided  to  use  the  quiz  feature  of  WebCT  and  to  create  a database  driven  item  bank  of  questions  to 
assess  student  learning  and  provide  feedback.  The  learning  objective  was  for  students  to  develop  skills  in 
linguistic  analysis  and  synthesis.  Students  are  given  a database  of  84  sentences  in  language  X,  beginning 
with  simple  sentences  (like  ‘The  woman  is  going  home’),  and  increasing  in  complexity  until  they  involve 
quite  complex  sentences  involving  constructions  like  relatives  and  passives  (as  in  ‘The  woman  who  cooked 
the  meat  is  sitting  down.’)  Students  are  supplied  with  English  translations  for  each  sentence,  but  they  do 
not  receive  any  indication  of  which  words  mean  what,  and  which  of  the  meaningful  subparts  of  words 
(morphemes)  are  represented.  Using  prior  knowledge  of  linguistics,  students  have  to  make  inferences  about 
words  parts  and  combine  these  in  order  to  interpret  the  language.  Students  were  required  to  satisfy 
assessment  requirements  by  completing  two  separate  activities;  the  construction  of  a dictionary  of  this 
language  and  the  completion  of  a grammatical  description  of  this  language.  This  required  higher  order 
thinking  skills  and  the  capacity  to  analyze  and  synthesize  language  elements. 

Bloom’s  taxonomy  was  used  to  categorise  questions  according  to  cognitive  skills  required.  See  Table  1 for 
the  matrix  summarizing  question  types. 


Question  type 

Link  with  learning  ontcome 

Level  in  Bloom*s 
taxonomy 

Multiple  choice 

Learn  about  and  understand  how 
to  use  the  lexicon 

Knowledge,  Analysis 

Matching  items 

Apply  new  knowledge 

Comprehension,  Analysis 

Short  answer 

Synthesise  new  knowledge  in 
order  to  create  new  sentences 

Application,  Synthesis 

Table  1:  Matrix  showing  question  type  and  cognitive  skill  required 
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Design  of  the  quiz  questions 


The  quiz  items  were  constructed  with  three  different  question  styles,  designed  to  assess  a range  of  learning 
outcomes. 


Multiple  Choice  Questions  were  used  to  test  skills  in  the  cognitive  domain,  focusing  on  comprehension 
and  analysis  of  the  grammatical  patterns  evident  in  the  data.  For  example  in  the  sample  question  below, 
students  are  asked  to  select  variables  true  of  a word  class  type. 


Question  5 (3  points) 

Which  of  the  following  characteristics  are  true  of  the  class  of  nouns  in  this  language?  [You  need  to  select  all  the 
characteristics  which  are  true  to  score  full  marks,  and  there  may  be  more  than  a single  correct  answer] 

Nouns  can  occur; 

1.  as  a bare  stem  (unaffixed  root) 

2.  with  the  prefix  ’ma?-’ 

3.  with  the  possessive  suffixes 

4.  with  the  definite  suffix  ’-e’ 

5.  with  the  suffix-ka? 


Multiple  choice  items  were  extended  in  several  ways; 
multiple  variables  could  be  true 
all  true  variables  had  to  be  selected  for  full  marks 

all  untrue  variables  were  negatively  weighted,  so  that  selection  of  all  variables  gave  0%. 

Matching  Questions  were  also  used  to  test  the  skills,  focusing  on  comprehension  and  analysis  of  the 
grammatical  patterns  evident  in  the  data.  For  example  in  the  sample  question  below,  students  are  required 
to  demonstrate  their  understanding  of  word  class  characteristics  by  matching  each  word  class  type  with  a 
sample  sentence  which  only  that  wordclass  could  fill.  Note  that  the  sample  sentences  are  not  necessarily  in 
the  database,  and  are  untranslated.  In  WebCT  Quiz  mode,  students  see  all  four  matching  variables 
randomly  listed  for  each  subpart  of  the  question. 


Question  2 

Match  each  word  class  category  listed  on  the  left  below  with  the  Lg  X structural  frame  that  could  only  be  filled  by  a word  of 
that  class. 

noun:  u-ita-i -e 

verb  : -I  tau-e 

adjective:  tau  -e  pole-i 

preposition:  lao-ka? marege 


Short  Answer  Questions  were  used  to  test  students’  ability  to  synthesise  and  apply  concepts.  Application 
and  synthesis,  as  cognitive  objectives,  are  demonstrated  by  students’  ability  to  apply  general  grammatical 
rules  they  have  learned  to  the  creation  of  new  sentences.  In  other  words,  students  were  required  to  go 
beyond  the  database  given  to  them,  and  make  up  new  sentences  \n Language  X which  were  fully 
grammatical  in  terms  of  the  grammatical  rules  they  had  already  worked  out. 

The  Short  Answer  style  of  question  allows  students  to  translate  a language  item  and  to  enter  a response,  as 
in  this  sample  question. 


Question  36 

How  would  you  say  the  following  sentence  in  Language  X? 

[Use  lower  case  only,  use  no  punctuation,  leave  only  a single  space  between  words,  leave  no  spaces  between 
morphemes  within  a word]. 

The  woman  who  cooked  the  meat  is  hit  by  her  father. 

Answer: 
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The  marking  of  questions  of  this  type  is  also  automatic  using  the  Regular  Expression  syntax  feature  built, 
into  WebCT’s  Short  Answer  shell.  This  compares  students’  responses  against  a specified  string,  but  unlike 
the  Equals  and  Contains  functions,  Regular  Expression  allows  predictable  variables  (is  the  first  letter 
capitalized  or  not?,  has  the  students  put  a double  space  between  two  word?)  to  be  catered  for.  It 
additionally  allows,  still  in  automated  mode,  for  a range  of  less  than  perfect  responses  to  be  awarded  partial 
marks,  for  example* 

100%=  [M|m]akunrai/s+tunungengngi/s+jukkue/s+ripeddiriwi/s+ri/s+ambo?na 

50%=  [M|m]akunrai/s+tunungengngi/s+jukkue/s+peddiriwi/s+ri/s+ambo?na 

50%=  [M|m]akunrai/s+tunungengngi/s+jukkue/s+ripeddiri/s+ri/s+ambo?na 

40%=  [M|m]akunrai/s+tunungi/s+jukkue/s+ripeddiriwi/s+ri/s+ambo?na 

20%=  [M|m]akunrai/s+tunungengngi/s+jukkue/s+peddiri/s+ri/s+ambo?na 


Task  1:  Building  the  dictionary 

Firstly  students  had  to  work  out  the  meanings  of  each  of  the  words  and  word  parts  in  the  data  provided  to 
them  on  language  X.  This  was  achieved  as  a collaborative  task  whereby  students  contributed  their  analyses 
of  word  meanings  to  an  online  dictionary  available  on  the  unit  homepage.  This  dictionary  begins  as  an 
empty  shell  with  4 fields  of  information  for  each  entry.  Students  are  free  to  add  any  text  to  the  ‘headword’ 
and  ‘meaning’  fields.  The  ‘word  class’  field  offers  a fixed  menu  of  choices.  The  ‘name  of  contributor’  field 
is  generated  automatically.  The  dictionary  required  the  development  of  a specialized  tool  which  does  not 
come  as  a standard  feature  of  WebCT.  This  tool  enables  an  online  class  of  students  to  contribute  entries  to  a 
database,  and  those  entries  are  viewed  on  screen  as  though  they  were  a single  document.  An  extract  of  the 
dictionary  looks  like  Table  2: 


Word 

Class 

Definition 

Person 

buaja 

Noun 

crocodile 

Janice  L [edit  | delete] 

c@mme 

Verb  (tr) 

wash 

Sharon  E [edit  delete] 

de? 

Particle 

negative 

Sharon  E [edit  delete] 

galung 

Noun 

field 

Ruth  D [edit  | delete] 

Table  2:  Dictionary  extract 

The  unit  co-ordinator  has  the  capacity  to  edit  and  delete  any  entry,  while  each  student  has  the  capacity  to 
edit  and  delete  only  the  entries  that  they  themselves  have  contributed.  The  class  as  a whole  is  responsible 
for  the  accuracy  of  all  entries,  so  students  must  negotiate  changes  to  other  students’  contributions  through 
discussion  on  the  Bulletin  Board.  This  form  of  collaborative  negotiated  learning  is  highly  conducive  to  co- 
construction of  meaning  and  self-directed  learning  (Coomey  & Stephenson,  2001). 

During  the  semester  there  were  hundreds  of  postings  to  the  Bulletin  Board  renegotiating  such  changes.  A 
sample  exchange  is  included  below; 

•Hi  Joanne  et  all  I have  been  mulling  over  all  the  *ng*s  that  are  placed  in  differing  amounts  on 
various  words!  When,earlierJ  had  analysed  the  verb  'drink*,  I had  thought  of  it  as  'menung* 
rather  than  'menungng'.  This  is  becausefrom  the  available  data,  both  verbs  and  nouns  that  end 
in  'ng'  seem  to  only  have  it  as  a single  sound.  Where  further  instances  of'ng'  appear  on  a word 
they  seem  to  be  added  as  suffixes  (e.g.  'tedong*  - 'tedongnge).  However,  / may  be  entirely 
wrong!  What  do  you  all  think? 
cheers,  Sally. 

•Sally,  I think  the  verb  is  definitely  menung,  but  all  words  ending  in  -ng  reduplicate  this  cluster 
before  certain  suffixes  for  instance  the  suffix  -i  and  the  suffix  -e.  (Anna) 
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•Dear  Sally,  Anna,  Joanne  & all, 

I agree  with  Anna  about  the  verb  menung.  The  *ng*  is  reduplication  before  a suffix.  (Eileen) 

•Hi  guys, 

Yes  1 can  see  now  that  you’re  right.  , IMl  change  this  straight  away  in  the 
lexicon. (Joanne) 

The  construction  of  the  dictionary  was  not  formally  assessed.  Each  student  was  asked  to  contribute,  and  a 
cap  of  5 entries  per  student  was  specified  to  encourage  involvement  by  all  students.  Students  were  highly 
motivated  to  assemble  the  dictionary  as  soon  as  they  could,  in  order  to  facilitate  their  capacity  to  tackle  the 
higher  level  grammatical  analysis.  All  words  were  entered  within  the  first  3 weeks  of  the  semester,  with 
subsequent  tinkering,  as  negotiated  on  the  Bulletin  Board,  over  several  more  weeks.  The  lexicon  was 
essentially  completed  by  week  6 of  the  semester. 

Task  2:  Completing  the  grammar 

After  analyzing  the  grammatical  structure  of  this  language,  students  were  required  to  demonstrate  their 
understanding  of  it  by  completing  an  online  Quiz  consisting  of  39  questions.  Students  had  the  option  of 
submitting  their  Quiz  twice.  On  the  first  submission,  they  received  back  only  a grade,  but  no  indication  of 
which  answers  were  right  or  wrong.  This  mechanism  had  several  advantages; 

it  was  reassuring  to  those  students  who  were  apprehensive  about  the  technology,  and  who  felt 
uncomfortable  with  opening  a new  tool  and  submitting  a Quiz  in  a single  go.  These  students  took 
comfort  from  a practice  run. 

it  gave  students  some  means  to  check  that  they  were  on  the  right  track,  but  without  explicitly 
highlighting  their  weak  spots.  Most  students  reported  that  they  found  this  spurred  them  on  to  do 
better.  So  that  on  learning  that  they  scored  72%  on  their  first  submission,  they  were  motivated  to 
do  for  some  further  study  before  the  second  submission  in  order  to  better  their  score. 


Results 

Analysis  showed  that  the  short  answer  items  were  the  most  demanding  as  it  was  these  questions  that 
required  students  to  transfer  their  knowledge  of  the  lexicon  into  more  abstract  forms  of  understanding.  The 
items  also  required  students  to  synthesise  and  apply  knowledge  to  create  new  lexical  items,  and  there  was  a 
greater  range  of  responses  and  grades  than  for  the  other  items.  Without  a doubt,  students  who  successfully 
completed  task  2,  (creating  the  dictionary)  demonstrated  the  desired  learning  outcomes  of  analysis  and 
synthesis.  In  all  three  types  of  questions,  the  automated  quiz  tool  and  feedback  mechanisms  provided 
learning  support.  The  pedagogic  rationale  for  the  use  of  these  forms  of  assessment  was  to  align  learning 
objectives  with  assessment  tasks,  while  providing  feedback  and  performance  data  to  learners.  The  quiz 
questions  provided  individual  and  group  feedback,  allowed  students  to  check  their  responses  and  enter  an 
alternative  answer,  and  compare  their  scores  with  others.  In  addition,  it  fostered  self-regulated  learning  as 
learners  would  access  the  tests  at  a time  and  place  convenient  to  them. 


Conclusions  and  further  research 

Computer-based  assessment  may  suffer  an  'image  problem'  as  some  assume  it  is  capable  only  of  summative 
testing  using  multiple  choice  tests  derived  from  item-banks.  Increasingly,  computer-based  assessment  is 
enabling  innovative  approaches  to  formative  assessment  that  close  the  gap  between  actual  and  desired 
levels  of  performance'.  A number  of  factors  can  be  cited  concerning  the  potential  value  of  online  learning 
environments  and  computer-based  assessment.  Among  these  are  its  potential  to  cater  for  individual  needs, 
foster  self-regulation,  increase  motivation  for  learning  by  giving  student  more  control  over  when  and  how 
often  they  test  their  own  knowledge.  Current  software  development  and  the  interactive  capabilities  of  the 
Web  enable  the  creation  of  procedural,  conceptual,  cognitive  and  collaborative  assessment  tasks  and  can 
tap  students’  prior  knowledge,  and  assess  learning  in  personally  meaningful  ways  that  are  flexible  and 
learner-centered. 
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Using  computers  to  evaluate  teachers'  understanding  of  physics  concepts 
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Introduction 

The  video  "A  Private  Universe",  a production  of  Harvard- Smithsonian  Center  for  Astrophysics, 
Science  Education  Department,  Science  Media  Group,  was  considered  an  important  eye  opener  in  the 
history  of  Science  Education.  The  interviews  with  Harvard  graduates  revealed  students'  misconceptions 
in  science  among  the  best  students  of  the  country.  According  to  Mazur  (1997)  another  eye  opener  was 
the  release  of  the  Force  Concept  Inventory  (FCI)  developed  by  Hestenes,  Wells,  & Swackhamer  (1992). 
It  was  called  an  eye  opener  because  he  realized  that  students'  misconceptions  could  not  be  diagnosed 
through  the  conventional  standard  or  teacher-  made  tests  or  even  problem  solving  strategies  that  he  used 
to  use  for  many  years  of  his  teaching  at  Harvard  University.  Research  in  the  area  of  science  education 
has  repeatedly  shown  that  only  certain  types  of  questions  can  evaluate  the  ability  of  students  to  resolve 
concepts  from  one  another  and  to  apply  them  to  real  situations.  However,  successful  construction  of 
such  conceptual  tests  is  hardly  ever  reported  in  the  literature.  So  far  FCI  and  a few  other  conceptual 
tests  have  been  used  exclusively  to  evaluate  students'  misconceptions  rather  than  teachers' 
misconceptions. 

Misconceptions  are  not  limited  to  students.  They  are  also  very  common  among  teachers  (Mestre,  1994). 
For  example,  after  a comprehensive  study  on  159  science  teachers’  misconceptions,  Kruger,  Palacio, 
and  Summers  (1992,  p.  344)  concluded  that  “ virtually  none  of  the  primary  school  teachers  had  a correct 
(in  the  Newtonian  sense)  view  of  the  instances  involving  forces  and  motion  with  which  they  were 
presented”.  According  to  Mestre  (1994)  some  significant  changes  in  teacher  education  programs  are 
necessary.  He  suggests  that  the  science  content  knowledge  of  elementary  and  middle  school  teachers  has 
to  be  upgraded,  but  this  does  not  mean  that  in-service  and  pre- service  teachers  need  to  take  an  inordinate 
number  of  science  courses.  Therefore,  there  is  an  urgent  need  to  investigate  teachers'  misconceptions  in 
science  targeting  the  concepts  that  they  are  supposed  to  teach  in  the  future.  The  authors  believe  that  the 
best  way  to  evaluate  teachers'  misconception  is  to  develop  more  conceptual  tests  similar  to  FCI. 
However,  there  are  many  scientific  conceptions  that  could  not  easily  be  evaluated  with  paper-pencil  tests 
such  as  FCI.  In  an  attempt  to  make  a more  sophisticated  evaluation  and  instructional  tool  the  authors 
have  developed  an  instructional  model  (Inventive  Model)  and  a physics  CD  and  an  on-line  physics 
course  based  on  the  model.  Using  such  multimedia  and/or  Internet  tool  with  sophisticated  interactive 
simulations  to  investigate  teachers'  misconceptions  is  considered  to  be  very  important  and  ultimately 
could  be  a third  eye  opener  in  the  area  of  science  education. 

Goals  and  objectives 

In  this  paper  the  authors  introduce  two  educational  software  that  have  been  used  in  conceptual  physics 
instruction.  First,  a multimedia  CD  targeting  common  misconceptions  in  physics  based  on  a new 
instructional  model  (Inventive  Model)  developed  by  the  authors.  Second,  an  ongoing  collaborative 
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project  among  several  universities  in  Canada  using  the  Inventive  Model.  The  project  is  called  Modular 
Approach  to  Physics  (MAP).  The  goal  of  this  paper  is  to  propose  a project  in  which  the  physics  CD  and 
the  omline  physics  course  will  be  used  to  investigate  teachers'  misconceptions  in  physics.  Although  this 
investigation  will  deal  specifically  with  examples  from  physics,  the  authors  believe  that  our  model  and 
conclusions  would  have  enough  validity  to  be  used  in  other  areas  of  science  and  perhaps  beyond.  The 
focus  will  be  on  concepts  that  teachers  are  supposed  to  teach  in  their  future  teaching  career. 

Cognitive  learning  theories  such  as  discovery  learning  or  constructivism  have  focused  on  students' 
learning  and  students'  cognitive  processes.  Generally  in  student-centered  science  education  the  focus  is 
on  student  rather  than  teachers  (Roblyer,  & Edwards,  2000).  In  these  approaches  the  teachers  play  a role 
as  facilitator  rather  than  instructor  or  lecturer.  However,  the  authors  believe  constructivist  teachers 
should  have  a much  deeper  understanding  of  scientific  concepts  than  traditional  teachers.  Therefore, 
evaluation  of  teachers'  misconceptions  could  have  significant  implications  for  the  quality  of  science 
teacher  training  programs. 


Theoretical  Background 

The  quality  of  current  educational  software 

The  quality  of  educational  software  particularly  for  the  purpose  of  conceptual  learning  has  been  the 
subject  of  so  many  studies  since  early  1980s.  Komoski  (1989)  estimated  that  1500-2400  new  packages 
are  published  in  the  United  States  each  year  and  that  the  proportion  of  good  quality  software  is 
something  between  5 to  10  per  cent.  Even  today  U.S.  Department  of  education  reports  that  a significant 
proportion  of  computer  software  in  schools  is  obsolete  by  today's  standards.  Results  of  a survey  of 
educational  software  shows  that  regarding  the  criteria  they  use  to  guide  software  development  67 
percent  have  no  formal  guidelines  ( Roth  & Petty,  1988).  According  to  the  literature,  low  quality  of 
educational  software  has  many  reasons  including  lack  of  a theoretical  model  for  the  instructional  design. 
After  reviewing  hundreds  of  research  papers  in  the  area  of  instructional  design,  and  dozens  of 
educational  software,  the  author  developed  the  "Inventive  Model"  as  a theoretical  framework  for  science 
instruction  and  particularly  for  software  development  in  science  education.  (Rezaei  & Katz,  1 998)  The 
Inventive  Model  is  developed  to  pave  the  way  from  edutainment  or  infotainment  toward  high  quality 
educational  software. 

Even  if  teachers  find  good  quality  software,  the  problem  is  they  often  find  it  difficult  to  integrate  it  into 
their  lesson  plan.  In  other  words  some  parts  of  the  software  does  not  match  with  the  goals  and  objectives 
of  the  course.  The  Modular  Approach  to  Physics  has  been  developed  to  overcome  this  difficulty.  This 
approach  provides  customized  instruction  for  teachers.  Teachers  usually  prefer  to  make  their  own 
instruction  rather  than  fitting  themselves  into  somebody  else's  design.  This  on-line  navigation  and 
arrangement  tool  provides  modules  instead  of  units  of  instruction.  Therefore,  teachers  can  easily  select 
the  modules  that  they  want  to  use  and  arrange  the  modules  in  their  own  way.  All  modules  are  designed 
based  on  the  "Inventive  Model"  (i.e.,  each  single  module  is  designed  based  on  research  findings  on 
students'  difficulties  in  understanding  scientific  concepts).  Lack  of  interest  and  perhaps  lack  of  a good 
background  in  science,  has  caused  a great  decline  in  North  American  students’  enrollment  in  science 
related  fields  at  the  undergraduate  and  especially  graduate  levels  (Halpem,  1992).  With  the  integration 
of  technology,  however,  it  is  possible  to  make  it  easier  for  teachers  to  teach  and  for  students  to  learn. 
(Katz,  Rezaei,  1999) 


Previous  research  on  the  Inventive  Model 


The  Inventive  Model  has  been  developed  as  a theoretical  model  for  educational  software  development 
and  also  as  a conceptual  change  model  to  improve  conceptual  learning.  Research  findings  consistently 
show  that  students'  misconceptions  are  deeply  seated  and  likely  to  remain  after  instruction,  or  even  to 
resurface  some  weeks  after  students  have  displayed  some  initial  understanding  immediately  following 
instruction  (Halloun  & Hestenes,  1985;  Mazur,  1997;  Mestre,  1994).  Research  also  shows  that 
instructional  approaches  that  facilitate  conceptual  change  are  more  effective  than  other  approaches  that 
disregard  students'  cognitive  structure  (Mazur,  1 997).  However,  teachers  and  educational  software 
developers  rarely  build  their  instruction  on  a valid  measure  of  students'  misconceptions.  The 
instructional  design  used  in  the  Inventive  Model  is  based  mainly  on  such  a longitudinal  and  systematic 
evaluation  of  students'  misconceptions. 

Theoretically,  the  inventive  model  has  four  phases.  The  first  phase  starts  with  a systematic  analysis  of 
students'  preconceptions.  Although  students’  conceptions  are  unique  and  private,  most  of  their  concepts 
are  similar  and  public.  Therefore,  in  the  instructional  design  of  the  inventive  model  both  kinds  of 
misconceptions  are  addressed.  The  simulations  and  the  videotaped  experiments  are  developed  based  on 
literature  on  students'  common  misconceptions.  However,  the  feedback  offered  by  the  teacher  or  the 
software  is  based  on  the  individual  student's  answers  to  the  conceptual  questions.  It  should  be  noted  here 
that  the  piece  of  software  based  on  this  model  could  be  used  either  by  the  teacher  in  the  class  as  an 
instructional  tool  or  by  students  after  class  individually  or  collaboratively. 

In  the  second  phase,  advance  organizers  or  other  cognitive  strategies  such  as,  concept  maps  and/or 
analogies,  are  used  to  activate  the  students'  prior  knowledge  and  bridge  it  to  the  new  concepts  to  be 
learned.  How  these  cognitive  strategies  are  integrated  into  the  inventive  model  is  explained  in  an  earlier 
paper.  (Rezaei,  Katz  1998)  Both  in  this  second  phase  and  in  the  last  phase,  students'  acceptable  concepts 
are  refined  and  reinforced  in  a guided  discovery  maimer.  This  means  that,  if  the  majority  of  students 
have  a deep  misconception,  the  teacher  will  move  to  the  third  phase  processes  to  rectify  the 
misconception.  However,  if  the  answers  are  close  enough  to  the  scientific  conceptions,  the  teachers  will 
move  directly  to  the  forth  phase  were  they  refine  students’  conceptions. 

The  third  phase  includes  different  activities,  such  as  having  students: 

a.  test  their  preconceptions  through  hands-on  activities  or  computer-based  simulations. 

b.  compare  their  preconceptions  with  natural  phenomena  and  related  scientific  theories  and 
identify  any  conflicts  between  their  misconceptions  and  the  scientific  theories. 

c.  become  dissatisfied  with  their  misconceptions  through  a multi-perspective  demonstration  and 
problem-solving  situation  and  feel  the  need  for  a new  concept;  or,  explore  plausible 
alternatives  by  themselves  or  as  suggested  by  their  classmates  or  the  teacher,  and  choose  the 
more  convincing  one. 

In  the  fourth  phase,  the  teacher  explains  the  correct  answer  and  demonstrates  the  advantages  of  the 
conceptions  currently  accepted  by  the  science  community  through  a multi-perspective  demonstration. 
The  teacher  may  also  help  students  to  summarize  what  they  have  learned. 
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The  basic  rationale  of  the  Inventive  Model  is  the  belief  that  conceptual  change  does  not  occur  simply 
because  students  see  a conflict  between  their  preconceptions  and  the  scientific  realities;  rather,  students, 
after  testing  their  preconceptions,  will  gradually  realize  the  advantages  of  scientific  explanations.  This 
will  happen  only  if  the  teacher  and  technology  supply  the  required  cognitive  tools,  and  provide  a clear 
contrast  between  the  student's  conceptions  and  scientific  conceptions  through  a variety  of 
demonstrations. 

The  key  factor  in  the  Inventive  Model  is  the  multi-perspective  presentation.  The  author  believes  that 
generalizations  based  on  a single  experimental  design  might  be  misleading.  However,  considerable  time 
is  required  to  probe  students'  conceptions  and  to  present  new  concepts  from  different  perspectives  via  a 
variety  of  demonstrations.  An  effective  way  of  dealing  with  the  time  limitations  in  formal  science 
classrooms  and  many  inquiry  approaches  is  to  empower  the  instruction  and  the  inquiry  processes  by 
using  computers.  Rezaei  (1999)  developed  the  multimedia  physics  CD  based  on  the  Inventive  Model  as 
a practical  way  to  achieve  this  goal.  Videotaped  or  simulated  science  experiments  were  designed, 
developed,  and  presented  based  on  the  best  scripts  of  teachers'  probes  in  longitudinal  studies.  Dozens  of 
science  experiments,  animations,  sound  clips,  simulations,  pictures,  graphs,  tables,  concept  maps, 
analogies,  metaphors,  and  advance  organizers  were  saved  in  a multimedia  program,  available  on  a single 
CD-ROM. 

Modular  Approach  to  Physics 

The  Modular  Approach  to  Physics  (MAP)  project  is  a multi- institution  venture  ftmded  by  the  Province 
of  Alberta  in  Canada  under  the  Learning  Enhancement  Envelope  (LEE).  The  project  is  currently  in  its 
3rd  year  of  ftmding  and  coordinates  the  efforts  of  groups  at  6 college  and  universities  in  Alberta, 

Canada.  There  is  a need  for  being  able  to  efficiently  and  easily  collect  and  organize  Web-based  content 
into  a package  such  that  others  can  easily  find  in  the  package  what  they  are  looking  for.  A program  with 
this  capability  has  been  developed  at  the  University  of  Calgary.  CANU  (Content  Arranging  and 
Navigating  Utility)  is  a Web-based  Java  program  with  which  one  can  arrange  and  deliver  structured 
content.  The  program  has  been  designed  at  the  University  of  Calgary  as  part  of  a LEE- funded  MAP 
project.  Although  currently  CANU  is  being  used  to  arrange  and  deliver  physics  content,  it  could  just  as 
easily  be  used  to  arrange  and  deliver  many  other  kinds  of  content,  e.g.,  materials  for  other  courses  or 
descriptions  of  the  programs  of  a faculty  or  an  entire  university.  Minor  changes  only  would  be  required, 
e.g.,  a change  to  a new  title  page.  The  principle  investigator  of  MAP,  Dr.  Hans  Laue,  has  a strong 
interest  in  conceptual  difficulties  in  learning  physics  and  has  been  involved  since  1987  in  the  creation  of 
tutorial  software  at  the  introductory  physics  level  aimed  at  helping  overcome  such  difficulties.  Hicks,  R. 
B.,  & Laue,  H.  (1992)  have  developed  an  extensive  computer  tutorial  system  to  support  some  first  and 
second  year  university  level  courses.  Their  Computer  Assisted  Learning  in  Physics  (CALiPH)  provides 
40  tutorial  modules.  Dr.  Laue,  together  with  Dr.  B.  Martin,  the  present  author,  and  several  other  Alberta 
physicists,  are  currently  involved  in  the  evaluation  process  of  MAP. 

Finally,  as  mentioned  before,  in  the  MAP  project,  two  versions  of  the  software  have  been  developed 
both  based  on  the  Inventive  Model.  One  version  is  the  customized  version  and  the  other  one  is  the 
sequential  version.  Although  both  versions  are  based  on  the  Inventive  Model  and  both  focus  on 
conceptual  learning  they  are  not  quite  similar.  One  purpose  of  the  ongoing  project  is  to  compare  the 
effectiveness  of  these  two  versions.  The  results  of  such  a comparison  could  be  very  important  for  the 
future  of  on-line  course  development.  The  Modular  Approach  is  currently  under  its  evaluation  process 
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and  the  results  will  be  published  later.  Two  versions  of  the  course  are  provided.  One  is  pre  organized  or 
sequential  and  the  other  is  the  customized  version.  Teachers  and/or  students  could  choose  if  they  want  to 
follow  up  the  sequential  version  or  they  want  to  customize  their  own  units.  The  on-line  courses  based  on 
this  model  are  available  at 
http  ://www  .ucalgary . ca/M  AP/ 
http://ugl-gw.phys.ualberta.ca/~alphy/meltp/ 

The  proposed  project 

The  Inventive  Method  was  used  in  a two-year  experimental  research  by  the  first  author.  One 
hundred  and  forty  three  “grade  10- 12”  high  school  students  from  three  high  schools  were  randomly 
assigned  to  three  groups.  The  Inventive  Model  was  compared  with  a radical  constructivist  approach  and 
the  conventional  physics  instruction.  The  results  showed  that  the  Inventive  Model  group  scored 
significantly  higher  than  the  Radical  Constructive  group  on  the  conceptual  posttest.  It  was  also 
concluded  that  only  the  Inventive  Model  led  to  conceptual  change  in  the  students'  understanding  of 
Newton's  laws  of  motion.  Finally  it  was  obserwd  that  3 hours  of  working  on  the  Inventive  Model  CD 
was  as  effective  as  16  hours  of  conventional  physics  instruction.  The  qualitative  analysis  indicated  that 
most  students  actively  interacted  with  the  software  during  the  whole  program.  In  terms  of  rectilying 
students'  misconceptions  it  was  found  that  the  model  was  effective  for  about  more  than  50%  of  the 
students.  This  should  be  noted  that  successful  report  of  conceptual  change  is  very  rare  in  the  literature. 

The  author  surprisingly  observed  some  misconceptions  among  teachers  while  they  were  previewing  the 
CD  and  while  they  were  teaching  their  students.  Therefore,  it  was  decided  to  perform  a new  research  to 
investigate  teachers'  misconceptions  in  physics.  A detailed  analysis  of  teachers  interactions  with  the 
computer  (e.g.,  keeping  log  files  of  which  items  are  chosen,  which  buttons  are  clicked,  the  amount  of 
time  spent  on  a question)  could  be  an  invaluable  source  of  information  on  teachers'  cognitive  processes. 

To  start  the  research  the  authors  will  contact  American  Association  of  Physics  to  locate  and  call  high 
school  physics  teachers  in  Los  Angeles,  California.  Only  volunteer  physics  teachers  will  be  called  to 
participate  in  this  project.  Either  the  physics  CD  or  the  On-Line  physics  course  will  be  used  to  evaluate 
teachers'  misconceptions.  The  CD  version  is  limited  to  Newton's  laws  of  motion  and  will  be  used  by 
teachers  who  do  not  have  access  to  Internet.  The  On-line  course  is  a comprehensive  advance  physics 
course  for  high  school  teachers.  Although  both  software  have  been  developed  for  conceptual  physics 
instruction  the  investigators  plan  to  use  them  as  a means  to  evaluate  teachers'  misconceptions.  As  an 
Incentive  to  encourage  teachers  to  participate  in  the  project  they  will  be  paid  to  evaluate  free  edicational 
software  (i.e.,  the  physics  CD  or  the  on-line  physics  course).  The  plan  is  to  ask  teachers  to  preview  the 
software  to  see  if  they  would  like  to  use  it  in  their  teaching.  However,  teachers'  interaction  with 
software  will  be  saved  in  a log- file  that  later  will  be  used  by  the  investigators  to  evaluate  teachers' 
misconception.  The  teachers  will  be  informed  that  their  interaction  with  the  software  will  be  saved.  They 
will  also  be  asked  not  to  input  their  real  name  or  other  kinds  of  identifying  information.  Therefore,  the 
anonymity  of  the  participants  will  be  guaranteed. 
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Assessing  out-of-the-box  thinkers:  Professional  development  for  IT 
support  staff 

Background 

Since  its  inception  in  1996,  the  Center  for  Instructional  Technologies  (CIT)  has 
offered  leadership  and  assistance  to  educators  in  integrating  technology  into 
instruction  throughout  the  UT  Austin  campus  and  beyond.  It  serves  as  a training, 
consulting,  and  resource  center,  and  an  incubator  for  projects  using  promising 
new  technologies — all  aimed  at  enriching  the  teaching  and  learning  experience. 

Internal  knowledge  and  skills  gap  analysis 

The  CIT  has  traditionally  focused  on  technology  implementation  and 
consultation.  Recently,  The  CIT  has  been  engaged  in  activities  designed  to  move 
the  center  from  a production  shop  toward  a systems-approach  to  support.  As  a 
necessary  first  step,  an  internal  team  of  three,  supplemented  with  feedback  from 
the  rest  of  the  CIT  staff,  developed  a tool  to  help  staff  members  self-assess  their 
knowledge  and  skills  in  teaching,  learning,  and  technology.  When  the  team 
analyzed  the  results,  they  found  a large  section  of  technically  cross-trained  staff 
members  throughout  each  unit.  However,  only  three  staff  members  out  of  18 
believed  themselves  to  be  expert  in  instructional  design  and  only  two  assessed 
themselves  as  experts  in  learning  theory.  After  the  results  of  the  self  assessment 
were  presented,  the  CIT  challenged  itself  to  increase  it’s  staffs  comfort  and  use 
of  instructional  design  and  learning  theory  with  minimal  financial  cost  and  without 
sacrificing  needed  staff  production  time. 

Training  goais 

Based  on  findings  from  the  skills  gap  analysis,  Joe  Sanchez  and  Suzanne 
Rhodes  developed  a curriculum  based  on  engaged  constructivist  learning 
principles  to  meet  the  following  goals: 

• Provide  ongoing  training  to  increase  internal  staff  awareness  and  comfort 
level  regarding  use  of  instructional  design  techniques  and  strategies  to 
improve  faculty  support 

• Increased  faculty  client  satisfaction  with  CIT  support  services 

• Motivate  staff  to  create  new  workshops  with  creative  and  solid  engaged- 
learning  principles  at  the  foundation. 

Training  strategy 

We  are  employing  a blended  approach  of  classroom  computer  lab-based 
instruction  and  online  resources  and  collaborative  tools.  The  training  curriculum 
components  include: 

• Classroom-based  presentation  and  collaborative  activities 
The  concepts  were  first  presented,  using  multimedia  where  appropriate  as 
well  as  examples  of  best  practices.  Questions  and  whole-group  discussion 
are  integrated  with  the  presentation.  Second,  using  a problem-based 
learning-by-doing  approach,  participants  work  in  small  groups  with 
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facilitation  from  the  two  trainers.  The  groups  select  a real-life  scenario  and 
develop  solutions  by  answering  guiding  questions  about  instructional 
goals  and  approach  and  the  best  application  of  a variety  of  technologies. 
The  small  groups  then  present  their  solutions  to  the  whole  group,  leading 
to  whole-group  discussion  targeted  around  current  faculty  consultation 
opportunities.  This  format  varies  depending  on  the  content  and  the  current 
needs  of  the  participants,  but  always  employs  a hands-on  approach  to 
learning  that  is  personal  and  relevant  to  the  participants. 

Topics  have  varied  from  Bloom’s  Taxonomy  to  John  Keller’s  ARCS  Model. 
Each  of  the  classroom-based  presentations  not  only  exposed  the  users  to 
instructional  design  principles,  they  also  served  as  a demonstration  of 
different  teaching  techniques  and  styles.  Gaming,  simulation,  discussion, 
group  activity,  experiential  learning,  and  the  Socratic  method  were  all 
teaching  styles  used  as  examples  of  effective  teaching.  Each  session 
included  a debriefing  explaining  the  reasons  for  selecting  a particular 
teaching  style  followed  by  a discussion  of  the  planning  process. 

• Online  resources 

Using  Blackboard  courseware,  we  created  an  online  course  site  to  provide 
digital  versions  of  the  presentations  and  handouts,  additional  links  and 
resources  related  to  the  training  for  extending  learning,  and  a message 
board  to  extend  the  session  discussions.  In  addition,  online  surveys 
allowed  participants  to  provide  feedback  for  each  session. 

Project  plan  and  timeline 

Month  1 -begin  curriculum  and  content  development  for  training  sessions 
Month  2-  conduct  and  evaluate  in-house  training 
Month  4-modify  and  expand  training  to  include  UT  faculty  and  TAs 
Month  6-CIT  staff  to  participate  in  skills  gap  analysis  #2 

Assessment  approach 

This  project  is  ongoing  and  currently  under  deployment  with  short-term  goals 
completed  in  November  2001 . We  plan  to  use  CIT  internal  training  as  a pilot  that, 
with  modifications,  will  expand  to  reach  other  audiences  such  as  other  IT  staff, 
teaching  assistants,  UT  professors,  and  possibly  even  students.  Another  goal  of 
the  project  is  for  CIT  members  to  use  the  instructional  design  models  they  have 
learned  to  create  new  or  improved  workshops.  The  workshops  have  also  served 
as  a testing  ground  for  new  technologies  such  as  Horizon  Live,  a synchronous 
streaming  video  presentation  tool,  and  the  department’s  new  smart  board. 

This  project  employs  an  iterative  design,  development,  and  assessment  process. 
Data  will  be  collected  before  and  after  each  training  session  and  by  online  means 
through  tracking,  observation,  and  surveys.  Past  training  activities  will  be  revised 
and  presented  to  new  audiences  as  well  as  converted  to  become  completely 
online  for  those  interested  in  learning  the  material  on  a self-paced,  just-in-time 
basis.  New  activities  will  be  designed  and  developed  each  month  as  needed. 
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We  hope  that  the  materials  and  strategies  used  for  internal  CIT  training  in 
instructional  technology  will  result  in  the  following  outcomes: 

• Increased  comfort-level  and  awareness  of  instructional  design  techniques 

Staff  members,  while  unable  to  draw  directly  from  instructional  design 
theory,  have  been  using  systematic  design  process  with  each  project  they 
develop.  The  workshops  will  identify  the  processes  and  provide  a broader 
range  of  design  principles  to  draw  from. 

• Increased  use  of  instructional  design  principles  during  faculty  consultations 

CIT  staff  members  will  be  able  to  quickly  identify  the  needs  of  faculty 
members  during  consultations  and  place  their  needs  in  an  instructional 
design  framework. 

• New  innovative  workshops  created  by  staff  with  sound  ID  principles 

Using  working  knowledge  of  instructional  design  principles  will  allow  for 
creation  of  hands-on,  user-centric  based  workshops  for  faculty  clients. 
Previous  workshops  maybe  re-evaluated  for  instructional  soundness. 

• Enhanced  student  learning  and  engagement 

Application  of  sound  instructional  design  principles  will  lead  to  hands-on, 
interactive  workshops  that  will  enhance  student  learning. 


Conference  attendees  interested  in  the  following  concepts  would  find  this 
demonstration  to  be  of  interest: 

• Training  strategies  for  constructivist  learning  with  technology 

• Building  community  within  an  organization 

• Integrating  new  or  existing  IT  support  organizations 

• Assessment  and  evaluation  techniques 

• Skills  gap  analysis  techniques 

• Moving  IT  support  to  a systems  approach 


ERIC 
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Abstract;  Many  high  school  science  teachers  will  retire  in  the  ten  years  and 
unfortunately  so  many  new  ones  will  not  become  graduated  science 
teachers  in  Finland.  In  addition,  the  amount  of  high  school  and  university 
students  who  are  interested  in  physics  has  decreased  dramatically  during 
the  last  decades  but  now,  this  is  turning  slowly  to  the  better  direction. 
Anyway,  the  situation  can  lead  to  serious  lack  of  physics  researchers, 
physics  teachers.  Web  University,  an  international  research  pilot  of  the 
Open  Learning  Environment  project  in  Finland,  has  created  a new  model 
how  the  research  discoveries  of  the  European  Organisation  for  Nuclear 
Research,  CERN,  can  be  integrated  into  high  school  curriculum  in  Finland. 
The  activities  are  targeted  for  both  the  students  and  the  teachers  of  Finnish 
high  schools.  A great  part  of  the  activities  is  based  on  exploiting  the 
capabilities  of  information  and  communications  technology  and  the 
Internet.  We  apply,  for  example,  web-based  learning  for  high  school 
students  and  we  have  created  an  Internet  community  for  the  network  of 
high  school  teachers.  In  addition,  we  organise  intensive  physics  training  for 
high  schools  students  and  in-service  training  for  high  school  teachers.  Both 
of  these  activities  take  place  at  CERN,  where  the  visitors  are  guided  and 
instructed  by  particle  physics  researchers.  The  long-term  goal  is  to 
permanently  raise  the  status  of  physics  studies  and  physics-related 
occupation  in  the  society. 


The  European  Organisation  for  Nuclear  Research,  CERN  [8],  situated  in  Geneva,  Switzerland,  is  one  of 
the  largest  particle  physics  research  organisations  in  the  world.  CERN  tries  to  find  out  what  happened 
in  the  big  bang  when  the  universe  was  bom  by  studying  the  origin  and  the  structure  of  the  matter  and 
antimatter.  As  a physics  research  organisation,  CERN  supports  initiatives  of  the  member  states  that  are 
aiming  at  improving  the  level  of  knowledge  in  natural  sciences. 

Web  University  [9]  was  launched  already  in  1995.  In  the  beginning,  Web  University  co-operated  with 
Finnish  universities  and  promoted  CERN  and  physics  studies  by  enabling  remote  audiences  to 
participate  in  lectures  and  seminars  held  at  CERN  [1],  [2].  Only  recently,  the  activities  of  Web 
University  have  been  extent  to  cover  also  the  high  schools  [3],  [4],  [5],  [6],  [7]. 

Inspired  by  positive  experiences  of  the  first  visits  of  Finnish  high  school  student  groups  at  CERN  in 
autumn  2000,  Web  University  has  initiated  a model  aiming  to  integrate  the  research  discoveries  of 
CERN  into  the  high  school  physics  curriculum.  The  objective  is  to  increase  the  amount  of  talented  high 
school  students  who  are  interested  in  physics.  The  flexible  learning  consists  learning  periods  both  in 
the  own  school  in  Finland  and  at  CERN  in  Switzerland. 

Learning  in  the  own  high  school 

Learning  takes  place  in  modem  physics  period  in  the  physics  curriculum.  Students  must  have 
completed  their  modem  physics  course  or,  at  least,  they  are  on  the  course  in  parallel  with  the  intensive 
training  period  at  CERN.  It  is  recommendable  that  the  students  have  familiarised  themselves  with  the 
CERN  research  topics  already  before  coming  to  CERN.  There  are  several  options  how  to  prepare 
before  travelling  to  CERN.  The  students  can,  for  example,  study  CERN-related  digital  material  on  the 
web  produced  by  Web  University  or  they  can  attend  an  introductory  lecture  on  CERN  activities  given 
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by  a local  university  professor.  The  students  can  also  given  preparative  research  tasks  or  project  works, 
which  they  prepare  beforehand  and  continue  at  CERN.  However,  the  science  teachers,  who  are 
responsible  for  the  class,  carry  also  the  responsibility  of  the  students’  preparation  for  the  training  period 
at  CERN.  Web  University  provides  them  with  electronic  material  and  instructions  how  it  is 
recommended  to  prepare.  In  addition,  Web  University  has  created  an  Internet  community  for  science 
teachers  to  help  them  when  they  are  teaching  the  students  about  CERN  research  topics. 

After  training  at  CERN,  the  students  publish  their  personal  research  work,  for  example,  on  the  net  or  in 
newspapers.  Some  high  schools  have  organised  events,  where  students  can  present  the  results  to 
journalists,  school  board,  parents  and  other  students. 

Intensive  physics  training  period  at  CERN 

The  intensive  three-day  physics  training  has  been  planned  in  collaboration  with  Finnish  physics 
researchers.  The  researchers  give  lectures,  guide  trips  to  physics  experiments  and  discuss  their  research 
topics  with  the  students.  At  the  same  time,  the  researchers  can  share  their  enthusiasm  and  knowledge 
with  the  students. 

The  objective  of  the  first  day  is  to  give  a general  outlook  of  CERN  and  its  most  important  experiments 
and  research  goals  of  the  future.  On  the  second  day,  the  students  get  familiar  with  the  structure  of  the 
matter  and  nuclear  physics.  On  the  last  day,  they  get  acquainted  with  particle  physics  and  antimatter 
research.  The  researchers  guide  the  students  at  different  experiment  sites.  Before  each  site  visit,  the 
students  are  informed  about  the  theoretical  background  of  the  experiment.  At  the  end  of  the  training, 
the  students  give  feedback  about  their  visit  CERN.  The  results  are  used  for  refining  the  intensive 
physics  training  for  the  next  group. 


Conclusions 

Altogether,  five  training  periods  for  Finnish  high  school  students  were  organised  during  the  school  year 
2000-2001.  The  school  year  2001-2002,  eight  student  groups  of  about  20  students  will  visit  CERN. 

Both  the  parents  and  the  teachers  of  the  students,  who  have  visited  CERN,  have  given  us  positive 
comments  on  the  influence  of  the  flexible  physics  training  on  the  students.  However,  because  our 
activities  are  relatively  recent,  we  cannot  yet  report  how  they  have  affected  the  status  of  physics  studies 
among  the  high  school  students  or  the  amount  of  candidates  for  physics  studies  in  universities. 

At  the  same  time,  the  teachers  have  been  able  to  improve  their  knowledge  on  modem  particle  physics. 
This  should  have  a positive  impact  on  the  education.  The  teachers  have  also  networked  with  other 
teachers  and  university  professors  in  order  to  get  more  information  how  their  school  classes  can 
prepare  for  the  training  period  at  CERN.  For  this  reason,  during  the  school  year  2001-2002,  it  has  been 
started  the  in-service  training  for  high  school  physics  teachers  at  CERN.  Furthermore,  we  are 
developing  net-based  applications  and  tools  to  advance  communication  and  networking  between  those 
who  are  interested  in  developing  physics  education  in  high  schools. 
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Abstract:  The  Technology  Serving  Learning  Institutes  have  provided  instructional  technology  training  to 
73%  of  Southeast  Missouri  State  University  faculty  since  1997.  The  Institutes  were  perceived  to  be  at  the 
center  of  a major  transformation  of  campus  culture,  in  which  both  faculty  and  students  embraced  technology 
in  teaching  and  learning.  In  an  effort  to  assess  the  role  of  the  Institutes  in  this  transformation,  and  to  validate 
the  transformation  itself,  we  defined  "transformation"  and  developed  instruments  to  probe  the  role  of 
technology  in  teaching  and  learning  on  campus.  We  will  briefly  outline  our  assessment  effort,  describe  our 
instruments,  and  show  how  the  results  support  our  claim  that  the  Institutes  have  contributed  to  campus 
transformation.  We  conclude  by  summarizing  ways  to  use  assessment  to  guide  future  faculty  development 
efforts  and  to  influence  campus  policies. 


The  instructional  technology  revolution  has  attracted  attention  and  funding  for  several  years. 
Many  programs,  products,  and  experiences  have  been  generated  to  facilitate  both  teaching  and  learning. 
But  educators,  administrators,  and  funding  bodies  are  not  sustained  forever  by  excitement;  calls  for 
objective  assessment  of  the  effectiveness  of  technology  efforts  have  increased.  On  the  campus  of  Southeast 
Missouri  State  University,  assessment  of  our  faculty  development  program  for  instructional  technology 
began  in  earnest  when  we  applied  for  a Hesburgh  award  for  excellent  faculty  development  programs.  Our 
application  focused  on  evidence  of  campus  transformation  resulting  from  our  “Technology  Serving 
Learning”  (TSL)  faculty  development  Institutes. 

The  TSL  Institutes  have  been  offered  to  faculty  at  Southeast  Missouri  State  University  since 
Summer,  1997.  In  that  time,  73%  of  faculty  have  attended  at  least  one  Institute  session.  The  Institutes  were 
developed  in  response  to  the  University’s  1995  Strategic  Plan,  which  placed  strong  emphasis  on 
implementation  of  information  technology  in  teaching  and  learning.  Prior  to  1997,  the  University 
considered  itself  to  be  at  the  trailing  edge  of  technology  adoption,  with  no  online  courses,  incomplete 
infrastructure,  and  a faculty  quite  deficient  in  the  skills  and  understanding  necessary  to  successfully 
implement  technology  in  courses.  The  Institutes  provided  needed  technology  training  in  a context  of  sound 
pedagogy,  targeted  primarily  to  faculty  who  were  technology  novices. 

Faculty  used  the  training  to  develop  80  online  courses  in  only  three  years,  and  in  so  doing,  began 
to  address  a critical  need  for  enhanced  student  access  to  University  courses  in  our  large,  rural  service  area. 
Faculty  use  of  instructional  servers  greatly  increased,  so  that  today  275  faculty  maintain  an  instructional 
server  account,  and  500  courses  have  a Web  presence.  Faculty  interest  in  technology  also  increased,  as 
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indicated  by  faculty  attendance  at  technology  conferences  and  greater  willingness  to  take  part  in 
camp  us  wide  technology  planning. 

The  rapid  embracing  of  technology  by  the  faculty  led  us  to  believe  that  we  had  achieved  a 
fundamental  transformation  of  the  campus  culture.  We  set  about  to  define  "transformation”,  as  it  would  be 
understood  by  faculty,  administrators,  students,  and  the  public.  We  then  investigated  the  drive  for 
assessment  on  our  campus:  Who  is  interested  in  knowing  if  transformation  has  occurred,  and  what 
difference  does  it  make  if  it  has  occurred?  We  then  developed  survey  instruments  for  both  students  and 
faculty  to  collect  data  from  both  groups  on  the  role  of  technology  in  teaching  and  learning.  We  designed 
questions  to  probe  the  effectiveness  of  online  instruction,  especially  as  compared  with  face-to-face  courses. 
The  survey  results  became  an  integral  part  of  the  award  application,  but  they  have  also  found  application  in 
other  venues  on-  and  off-campus. 

In  this  session,  we  present  our  definition  of  “transformation”  as  we  briefly  outline  the  history  of 
our  assessment  effort.  We  then  describe  the  instruments  that  we  developed,  and  show  results  for  our 
campus.  A total  of  1250  students  enrolled  in  Southeast  Online  courses  over  the  past  two  academic  years 
were  asked  to  complete  the  student  instrument;  265  students  (21.2%)  responded.  A total  of  52  faculty  who 
taught  online  in  the  same  time  period  were  asked  to  complete  the  faculty  instrument;  35  faculty  (67.3%) 
responded.  Results  from  both  instruments  indicated  widespread  acceptance  of  online  courses,  and  a 
perception  that  online  courses  were  effective  vehicles  for  teaching  and  learning.  For  example,  we  found 
that  75%  of  student  respondents  felt  that  the  online  instructor  was  knowledgeable,  encouraged 
participation,  provided  helpful  feedback,  and  was  responsive  to  student  concerns.  Sixty-five  percent  of 
these  students  rated  the  instructor’s  overall  performance  as  high  or  very  high  as  compared  to  on-campus 
courses,  and  70%  would  recommend  the  instructor  to  other  students.  Clear  course  objectives,  well- 
organized  materials,  and  effective  communication  were  seen  by  70-80%  of  students.  Sixty  percent  of 
students  said  that  interest  in  the  course  subject  had  been  increased,  85%  rated  their  level  of  learning  as 
medium  to  very  high,  70%  felt  the  academic  quality  was  on  par  with  traditional  courses,  70%  agreed  that 
they  have  learned  as  much  as  in  a face-to-face  course,  and  80%  said  that  the  online  course  was  as  good  as 
or  better  than  a face-to-face  course.  Ninety-three  percent  of  surveyed  students  in  face-to-face  courses  felt 
that  course  Web  pages  improved  their  learning  experience  in  the  course.  In  the  faculty  survey,  80%  felt  that 
the  TSL  Institutes  prepared  them  well  to  develop  an  online  course,  with  70%  saying  the  Institutes  prepared 
them  well  to  teach  online.  Also,  90%  felt  that  online  instruction  increases  student  access,  65%  thought  that 
students  find  online  courses  as  good  as  or  better  than  face-to-face  courses,  and  90%  would  teach  online 
again. 


We  will  show  how  the  results  support  the  claim  that  our  Institutes,  offered  since  1997,  have  indeed 
made  a major,  measurable  contribution  to  campus  instructional  technology  transformation.  We  will 
conclude  with  a summary  of  some  of  ways  that  the  assessment  information  has  been  and  might  be  used  to 
guide  future  faculty  development  efforts,  and  to  influence  policy  on  campus. 
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Abstract:  This  paper  reports  on  two  key  elements  of  successful  online  teaching:  learner-centric 
design  and  harmonization  of  content  and  facilities.  We  show  how  these  two  crucial 
requirements  could  be  fulfilled  with  great  success  in  a project  that  involves  a non -computing 
related  subject  matter  (civil  engineering)  and  thirty -one  educators  from  five  European 
countries.  We  explain  the  context  of  the  project,  called  WiB A-Net,  and  introduce  the  system 
chosen  for  providing  content  access  and  navigation.  We  also  outline  key  decisions  in  designing 
and  building  Wi BA-Net,  emphasizing  the  two  crucial  aspects  mentioned  above.  This  outline 
can  serve  as  a guideline  for  projects  in  other  subject  matters  and  context,  and  is  thus  regarded 
as  valuable  for  a general  audience. 


Introduction 

Many  online  learning  materials  suffer  from  two  common  drawbacks:  the  lack  of  learner-centric  design  and 
display  disparities  in  content  by  different  authors.  The  first  drawback  especially  touches  the  offered  learning  path 
which  often  comes  in  either  of  two  extremes:  statically  fixed  or  nearly  absent.  Fixed  learning  paths  are 
encountered  for  example  in  lecture  slides  which  are  placed  in  a certain  order  and  may  be  restricted  to  forward 
and  backward  navigation.  Mostly  unstructured  HTML  documents  with  cross-references,  on  the  other  hand,  do 
not  provide  any  clear  path  by  giving  too  much  leeway  to  the  user. 

Neither  approach  is  helpful  for  the  student.  As  has  been  shown  in  various  research  projects,  not  all  learners  are 
the  same!  Providing  a single  type  of  access  for  all  students  is  therefore  unlikely  to  result  in  optimum  learning  and 
retention.  Learner-centric  navigation  requires  guiding  the  learner  to  a learning  path  tailored  to  his  or  her 
individual  preferences  and  learning  style.  However,  the  learning  path  should  not  be  enforced.  Thus,  both 
extremes  are  ill  suited  for  learner-centric  teaching. 

Within  our  ongoing  research  project,  we  are  looking  for  a middle  way  solution.  To  be  usable  outside  computer 
science,  the  system  must  not  place  high  expectation  of  computing  familiarity  on  users  or  authors.  Finally, 
standard  content  as  commonly  found  on  Web  pages  should  be  embeddable,  for  example  QuickTime  or  MPEG 
movies  that  illustrate  special  subtopics. 

Our  research  is  part  of  the  WiB  A-Net  project  funded  by  the  German  Ministry  of  Education  and  Research.  The 
project  shall  provide  a common  platform  for  online  teaching.  Our  current  topic  focus  is  on  civil  engineering.  The 
finished  system  shall  be  embedded  into  the  teaching  at  the  various  participating  universities,  to  the  benefit  of 
both  learners  and  educators. 

In  this  paper,  we  present  the  chosen  approach  for  supporting  learner-centric  online  teaching.  The  paper  is 
organized  as  follows.  We  first  provide  a short  overview  of  related  online  teaching  systems,  focusing  on  their 
support  for  individual  students.  We  then  give  a short  overview  of  the  1^  system  used  for  content  modeling, 
followed  by  the  technical  implementation.  Finally  we  summarize  the  paper  and  outline  areas  of  future  research. 


Related  Work 

Although  e-leaming  has  become  very  popular  lately,  there  is  still  no  “ideal  environment”  which  attracts  masses 
of  learners  and  fulfills  all  their  needs.  Many  “traditional”  educators  are  therefore  skeptic  about  massive  use  of 
the  Internet  as  a medium  for  learning  processes.  Virtual  School  or  University  is  still  more  “vision”  than  reality. 
However,  most  universities  are  trying  to  implement  an  online  form  for  at  least  some  courses.  At  foe  moment 
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probably  the  best  individual  access  is  given  in  WebCT  (WebCT  2002),  which  allows  the  tutor  to  distribute  the 
learning  materials  and  assessments  to  each  student  separately  - we  could  say  “manual  individualization  of 
study”.  Element  K KnowfedgeHub’’’*^  (Element  K 2002)  also  offers  this  feature.  Macromedia  Authorware  6.0 
(Macromedia  2002)  can  be  considered  as  the  most  flexible  environment  in  which  a high  level  of  interactivity  can 
be  reached.  For  course  and  test  building,  it  provides  an  iconographical  language  with  decision  step  elements  so 
the  author  can  in  fact  “program”  some  individualization  for  the  students.  However,  this  is  relatively  difficult  for 
authors  without  programming  experience. 

Teachers  are  of  course  aware  that  really  effective  learning  requires  more  than  the  individualization  in  the  above- 
mentioned  meaning.  It  is  only  logical  that  we  recently  observe  a boom  of  many  different  projects,  mainly  at 
uiiiveisities,  fucused  on  building  learning  environments,  which  would  provide  epistemological  pluralism  and 
hence  a better  transfer  to  the  learner.  Following  the  learner  way  of  thinking  has  been  shown  to  increase  the 
efficiency  of  education  (training)  and  retention.  In  these  environments  intelligent  tutoring  and  adaptability  to 
learner  needs  are  present  at  different  levels  and  are  also  interpreted  differently.  A brief  overview  with  a short 
characteristic  of  the  most  important  projects  follows. 

TANGOW  {Cdiuo  et  al.  2001)  (Task-based  .Adaptive  Learner  Guidance  on  the  ^fWW)  lets  students  choose  their 
learning  strategy,  for  example  placing  theory  before  exercises.  Suggestions  for  changing  the  current  strategy  can 
be  made  by  the  system  if  the  student  does  not  obtain  good  results  with  a given  strategy. 

ELM-ART  II  {ELM-PiXX  2002)  is  an  intelligent  interactive  textbook  for  programming  in  LISP.  Adaptability  is 
implemented  by  selecting  the  next  best  step  in  the  curriculum  on  demand.  Links  are  annotated  according  to  a 
traffic  lights  metaphor,  indicating  if  a section  is  ready  to  be  learned,  ready  but  not  recommended  or  not  ready 
yet.  The  annotation  is  updated  after  a learning  unit  is  finished,  reviewing  all  prerequisite  concepts  to  the  current 
unit  (Carro  et  al.  2001).  ELM- ART  II  enables  direct  interactivity  by  providing  live  examples  and  intelligent 
diagnoses  of  problem  solutions.  All  function  call  examples  can  be  evaluated.  When  the  learner  clicks  on  a live 
example  link,  an  evaluator  window  shows  the  evaluation  of  the  function  call.  Users  can  type  solutions  to  a 
programming  problem  into  an  editable  window  and  then  send  them  to  the  server  (Weber  et  al.  2002). 

SKILLS  (Neumann  et  al.  2002)  (scalable  Internet-based  teaching  and  learning  system)  organizes  the  course 
material  according  to  its  prerequisites.  Adaptability  is  implemented  by  taking  the  student’s  previous  knowledge 
of  the  subject  into  account  (Carro  et  al.  2001).  The  teacher  prepares  a default  configuration  for  every  course. 
Concepts  addressed  by  the  system  are  marked  in  the  user’s  profile.  If  the  user  is  familiar  with  some  concepts 
from  external  sources,  control  questions  are  given  from  the  tutoring  component.  If  these  questions  are  answered, 
the  known  concepts  are  removed  (hidden)  from  the  course.  If  the  student  does  not  want  to  answer  the  control 
questions,  the  concepts  to  be  removed  are  marked  until  the  student  solves  the  exercises  at  a later  time  (Neumann 
et  al.  2002).  Annotations,  notes  and  additional  files  can  be  stored  or  published  to  other  students. 

WINDS  (W^eb-based  Melligent  Design  and  Tutoring  system)  (WINDS  2002)  produces  individualized 
courseware  for  the  students  according  to  their  current  state  of  knowledge,  their  preferences  and  learning  styles. 
Each  learning  element  has  a didactical  goal.  Authors  can  create  new  content  using  a set  of  predefined  paragraph 
templates.  Examples  paragraphs  are  Cover  Story  Simple  Explanationor  Picture  Compare.  Complex  Paragraphs 
combine  several  elements  (“content  blocks”)  with  different  pedagogical  functions  to  fulfill  a pedagogical  goal, 
for  example  introduction , definition  or  example.  Each  such  chunk  of  information  has  a predefined  order, 
pedagogical  role  and  other  metadata.  Based  on  the  user  model  and  the  metadata,  the  system  can  adapt  the  content 
sequence  according  to  the  chosen  learning  strategy.  For  instance,  a concrete  example  can  precede  or  complement 
an  abstract  statement  if  needed.  To  prevent  or  at  least  reduce  cognitive  overload,  the  pedagogical  roles  of  the 
learning  elements  as  well  as  those  of  the  content  blocks  can  also  be  expressed  graphically,  for  example  by 
different  background  colors  (Specht  et  al.  2001). 

The  Learning  Environment 

L?  (Leidig  2001)  (“/ife-/ong  /earning “)  is  a learning  environment  by  SAP  CEC  at  Karlsruhe,  Germany.  £ 
structures  course  materials  into  four  different  types  of  containers.  Learning  networks  (LN)  are  the  topmost 
element  of  the  hierarchy  and  may  contain  a set  of  LN,  learning  objects  (LO)  and  instructional  elements  (IE).  The 
actual  content  is  represented  by  instructional  elements  (IE),  each  covering  roughly  one  page  of  paper.  The 
different  types  of  IE  include  example,  action,  introductions  or  resource  links.  Learning  objects  combine  a set  of 
lEs  addressing  a common  topic.  Additionally,  the  relation  between  given  elements  can  have  different  types, 
specified  by  the  classification  of  the  edge  connecting  the  elements,  as  shown  in  (Fig.  1). 
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Figure  1:  I?  course  editor,  showing  the  course  structure,  metadata  and  connection  types 

Each  element  has  a title,  an  author  and  a set  of  metadata.  This  metadata  includes  the  knowledge  and  media  type, 
approximate  time  required  for  finishing  the  unit,  the  types  of  competencies  that  can  be  achieved  by  working  on 
the  unit  and  a classification  of  the  content  according  to  an  underlying  taxonomy. 

The  learning  path  is  generated  based  on  the  relations  between  the  elements  and  the  content  type  of  lEs.  The  user 
can  choose  a learning  strategy  on  two  different  levels.  The  macro- strategy  covers  the  order  of  the  higher-level 
elements  such  as  LNs  and  LOs.  The  following  macro -strategies  are  offered:  table  of  contents,  deductive  and 
inductive,  as  well  as  one  linear  version  each  of  these  strategies.  The  ordering  of  the  lEs  within  a given  learning 
object  is  controlled  by  a miem -strategy.  Currently  available  are  orientation  only,  orientation  first,  explanation 
oriented,  action  oriented  and  example  oriented  microstrategies.  The  learning  material  can  also  encode  temporal 
or  causal  dependencies  between  lEs  as  prerequisites,  if  they  cannot  be  avoided. 

The  learner  may  submit  offered  tests  at  any  time.  Failed  tests  may  influence  the  course  process,  for  example  by 
asking  the  learner  to  review  parts  of  the  course.  Any  SCORM  -compatible  test  (SCORM  2002)  can  be  used,  for 
example  generated  using  Authorware  (Macromedia  2002).  Additional  techniques  that  may  be  integrated  into 
page  content  are  beyond  the  scope  of  this  paper.  Basically,  all  content  types  acceptable  in  HTML  4.0  can  be 
integrated  easily,  for  example  Flash,  MPEG,  RealMedia  or  QuickTime  media. 

L?  courses  are  executed  on  a standard  Apache  Web  server  with  an  Enhydra  Application  Server  (Enhydra  2002). 
Enhydra  performs  the  rendering  of  the  content.  The  end-user  requires  only  a current  web  browser.  Additional 
plug-ins  may  be  required  for  special  embedded  content,  such  as  MPEG  or  QuickTime  video  material. 


The  WiBA-Net  Approach 

WiBA-Net  is  a course  network  that  contains  learning  units  for  civil  engineers  and  architects.  The  content  is  a 
central  part  of  civil  engineering  studies.  We  have  to  address  the  following  user  groups,  each  with  its  own  unique 
interest  in  the  system: 

• Undergraduate  students  use  the  network  to  prepare  for  the  mandatory  exam  at  the  end  of  the  course, 

• Graduate  students  use  WiBA-Net  to  access  the  basics  of  their  study  and  additional  courses, 

• Professionals  in  related  fields  use  the  network  for  research  and  learning  more  about  covered  materials, 

• High  school  students  can  experiment  with  WiBA-Net  to  gain  or  increase  their  interest  in  the  topic  area 
and  possibly  induce  a desire  for  studying  in  this  field. 
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The  target  audience  also  contains  the  educators  who  can  use  the  WiBA-Net  for  publishing  content  for  lectures 
and  sharing  it  with  other  educators.  Currently,  31  educators  have  agreed  on  a content  memorandum  for  WiBA- 
Net,  coming  from  universities  in  Germany,  Austria,  Switzerland,  Denmark  and  Hungary. 

We  have  to  consider  not  only  different  needs  of  user  but  also  a different  (moreover  non-computer  science) 
backgrounds  of  the  authors.  Therefore,  our  system  has  to  respect  the  learner’s  individual  needs  and  be  easy  to 
use.  At  the  same  time,  it  must  also  be  comfortable  for  authors  who  want  to  share  their  materials  and  combine 
them  in  different  way  to  course  contents.  We  have  decided  to  adopt  as  the  tool  for  managing  and  displaying 
the  content  for  WiBA-Net  due  to  the  features  offered.  Key  aspects  in  the  decision  were  the  following: 

1.  learner-centric  approach  - for  example,  the  “orientation  only”  micro  -strategy  fits  high  school  students, 

2.  a user-friendly  interface  which  can  be  used  intuitively, 

3.  metadata  support  should  allow  authors  to  provide  essential  information  about  materials  for  efficient 
sharing  of  resources,  even  among  different  universities, 

4.  online  courses  require  high  interactivity  and  lots  of  multimedia  materials.  Therefore,  we  need  good 
support  for  tests  and  multimedia  files. 
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Figure  2:  Example  learning  path  in 

As  shown  in  (Fig.  2),  the  course  elements  on  the  learning  path  are  marked  with  different  colors  according  to  their 
degree  of  recommendation,  including  not  recommended,  recommended  and  visited  elements,  as  well  as  the 
current  element  and  test  entities.  The  elements  in  the  right  row  from  top  to  bottom  contain  the  current  element, 
two  recommended  elements  followed  by  two  tests  and  two  elements  which  are  not  yet  recommended.  The 
second  element  in  the  middle  row  is  a finished  learning  object. 

The  educator  specifying  the  course  ©ntents  does  not  directly  define  the  learning  path.  Rather,  the  path  is 
assembled  dynamically  from  the  learner’s  chosen  strategy.  Thus,  WiBA-Net  offers  a very  good  compromise 
between  cognitive  and  constructive  learning,  both  for  educators  and  students.  The  learner  can  also  pursue  a 
different  path  than  the  offered  learning  path  by  clicking  on  the  element  in  the  navigation  structure  at  any  time. 


Figure  3:  Example  content  as  shown  in  WiBA-Net 

An  example  content  provided  by  WiBA-Net  is  shown  in  (Fig.  3).  The  left  frame  contains  elements  for 
customizing  the  environment,  selecting  a course,  and  entering  communication  via  mail,  forums  or  direct  contact. 
The  actual  content  of  the  page  is  shown  in  the  center  of  the  window.  The  standard  navigation  elements  for 
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leaving  the  current  course  or  accessing  the  index,  as  well  as  stepping  forward  and  backwards,  are  placed  below 
the  content  area.  The  bottom  of  the  page  shows  the  current  leaning  path. 

Similar  to  the  common  navigational  elements  provided  by  L?,  the  course  con  tents  are  formatted  using  a common 
underlying  style  sheet,  providing  a corporate  identity  for  the  whole  set  of  courses.  By  assuming  the  same 
underlying  layout,  we  achieve  a harmonization  of  both  the  content  and  the  facilities,  making  the  finished  product 
“smoother’’  to  both  look  at  and  use. 

A key  concern  within  the  project  is  that  learners  shall  not  be  restricted  by  the  tight  focus  of  the  single  course  they 
currently  are  in.  For  this  end,  our  project  partners  have  coined  the  concept  of  knowledge  clusters.  A cluster 
contains  a set  of  pages  addressing  a common  topic  in  a well-structured  manner.  They  can  be  referenced  from  any 
course  as  “i  elated  tuples”,  offering  a more  in-depth  treatment  for  the  selected  topic  than  would  be  possible 
within  a given  course.  The  page  order  of  clusters  may  also  adapt  dynamically  based  on  the  strategies. 

Since  the  project  shall  create  a platform  for  online  learning  at  the  field  of  civil  engineering  for  the  whole  German 
academic  community,  metadata  are  one  of  the  key  points.  The  metadata  must  provide  sufficient  reusability  of  the 
learning  materials,  as  well  as  a basis  for  an  effective  searching  function  for  the  authors  and  for  the  students.  L? 
itself  supports  the  definition  of  various  LOM-based  metadata  (LOM  2002),  as  illustrated  in  (Fig.  4).  For  the 
reasons  mentioned  above  we  have  decided  to  adapt  this  model  as  follows: 

• all  the  metadata  are  fully  SCORM  compatible  (SCORM  2002), 

• those  optional  data  in  SCORM  model  which  are  relevant  to  the  project;  most  of  them  will  actually  be 
mandatory  for  the  authors  within  the  project. 

• as  much  as  possible  is  generated  automatically  by  the  system  (e.g.  metadata  scheme,  size  of  the  file,...), 

• many  categories  have  already  predefined  content  but  authors  are  allowed  to  change  it  anytime  (e.g. 
intended  end  user  role,  copyrights,. . .). 
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his  Introduction  should  be  read  first  as  It  outlines  the  main  concepts 
if  the  course  contents.  It  also  provides  and  overview  of  what  the  teamer 
an  expect  to  be  asked  to  perform  In  the  final  test 
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Figure  4:  Example  screen  shot  of  the  LOM  -based  Metadata  editor 

We  are  working  on  realizing  automatic  links  to  the  appropriate  knowledge  clusters  for  the  current  content.  This 
requires  both  a thorough  classification  of  the  content  with  metadata  and  a clever  server  component  that  can 
automatically  deduce  “related”  entities.  Additionally,  a precise  taxonomy  is  needed  for  building  a meaningful  set 
of  terms  for  classifying  the  contents. 


Conclusions  and  Further  Research 

Effective  Web-based  learning  requires  two  key  features:  learner-centric  design  and  a harmonization  of  both 
content  and  facilities.  The  former  point  mainly  touches  upon  the  degree  of  guidance  given  to  learners  for 
navigating  the  content. 

The  WiBA-Net  project  builds  a prototypical  system  for  learner-centric  online  teaching  that  incorporates  the 
SCORM  (SCORM  2002)  standard.  WiBA-Net  has  to  address  different  target  user  groups  and  accommodate  a 
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large  set  of  educators  from  different  countries.  The  main  focus  of  our  prototype  system  thus  lies  in  learner- 
centric  navigation  support  and  the  harmonization  of  both  content  and  facilities. 

The  content  is  presented  within  the  I?  environment.  accepts  standard  HTMLKJon formant  input  for  its 
components  and  thus  supports  many  different  input  types.  We  employ  a default  style  sheets  that  provides  a 
common  look  and  feel  that  is  not  restricted  to  a single  course.  The  large  amounts  of  participating  parties  that 
provide  content  necessitate  such  measures  to  keep  the  system’s  harmonious  look. 

The  project  presents  several  challenging  aspects  for  all  project  partners,  including  different  views  of  software 
solutions  and  slight  but  important  differences  in  the  usage  of  key  terms.  More  seriously,  some  key  definitions 
needed  for  the  software  solution  are  currently  lacking.  This  is  especially  the  case  for  the  support  of  related 
clciiicnts  to  be  linked  to  the  euirenliy  displayed  content,  termed  ’’knowledge  clusters"  within  the  project. 
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Abstract:  A number  of  university  campuses  have  undertaken  the  development  of  digital 
classrooms  that  enable  presentation  of  digital  media  and  digital  lecture  recording.  Deploying 
the  infrastructure  for  a digital  classroom  is  difficult  at  best  even  for  a technically  savvy  person. 
As  people  from  many  disciplines  become  interested  in  building  similar  digital  classroom 
spaces,  there  is  a need  to  produce  a useful  set  of  design  and  implementation  guidelines  to 
reduce  the  project  risk  and  steepness  of  the  deployment  curve.  The  goal  of  this  paper  is  to 
report  on  the  experiences  we  have  had  in  deploying  the  UCSB  digital  classroom.  The  two 
main  contributions  of  this  paper  are:  (1)  a phased  deployment  model;  and  (2)  a discussion  of 
how  the  proposed  technology  enables  new  educational  models  and  techniques. 


Introduction 

Advances  in  technology  coupled  with  increased  familiarity  with  technical  tools  have  paved  the  way  for 
new  paradigms  in  teaching  and  learning.  Instructors  are  now  using  media  such  as  PowerPoint  slides  and  digital 
video  in  their  lectures.  Students  can  take  digital  notes  on  laptop  computers  or  Personal  Digital  Assistants 
(PDAs).  These  types  of  technologies  allow  students  and  instructors  to  communicate  digitally  across  time  and 
space.  However,  while  these  tools  are  readily  available,  using  them  in  a coherent  manner  is  still  a challenge.  A 
number  of  university  campuses  have  undertaken  the  goal  of  developing  digital  classrooms  that  enable 
presentation  of  information  using  cutting  edge  multimedia  tools  as  well  the  capability  to  digitally  record  an 
account  of  the  classroom  activity.  The  account  can  be  used  in  realtime  to  enable  distance  learning  or  realtime 
collaboration,  or  can  be  archived  and  reviewed  at  a later  time. 

A number  of  universities  have  deployed  digital  classrooms  for  both  teaching  and  research  on  new 
learning  methodologies  and  tools.  One  of  the  earliest  experiments  with  this  kind  of  technology  was  the  AT&T 
Leaming/Teaching  Theater  at  the  University  of  Maryland  (Schneiderman  et  al.  95).  More  recent  examples 
include  405  Soda  at  UC  Berkeley  (Wu,  Swan,  & Rowe  99)  and  Georgia  Tech's  eClass  (Abowd  99).  While  the 
research  that  has  come  out  of  these  projects  has  focused  largely  on  user-level  issues,  the  piece  of  the  puzzle  that 
has  yet  to  be  solved  is  the  question  of  what  functionality  these  classroom  spaces  should  support,  and  more 
importantly,  how  that  can  be  achieved.  Without  a useful  model  to  draw  from,  there  is  an  enormous  learning 
curve  involved  in  determining  first,  what  functionality  a classroom  should  support,  and  second,  what 
technology  exists  to  realize  the  design.  A huge  number  of  tradeoffs  need  to  be  considered.  It  is  difficult  at  best 
for  a technically  savvy  person  to  undertake  the  challenge  of  deploying  a classroom.  As  people  from  across 
disciplines  become  interested  in  building  similar  digital  classroom  spaces,  there  is  a need  to  produce  a useful  set 
of  design  and  implementation  guidelines  for  ease  of  deployment. 

The  goal  of  this  paper  is  to  report  on  the  experiences  we  have  had  deploying  a digital  classroom  By 
drawing  from  our  experiences,  future  classroom  architects  can  reduce  project  risk  as  well  as  the  steepness  of  the 
deployment  curve.  We  identify  four  classroom  functions,  and  suggest  that  a classroom  should  be  deployed  in 
four  phases  corresponding  to  those  functions.  In  June  of  2000  we  took  on  the  challenge  of  deploying  a digital 
classroom  at  UC  Santa  Barbara.  To  date,  we  have  spent  approximately  $70,000  on  our  classroom  setup  broken 
down  as  roughly  $14,000  for  phase  1,  $42,000  for  phase  2,  and  $12,000  for  phase  3.  To  date,  our  phase  4 
deployment  has  leveraged  technology  purchased  in  the  prior  phases.  It  is  difficult  if  not  impossible  to  deploy  a 
fully  functional  digital  classroom  infrastructure  before  testing  or  using  it.  Therefore,  it  is  imperative  to  support 
incremental  development,  deployment,  use,  evaluation,  and  modification. 
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Presentation  Facilities 


The  first  phase  of  classroom  infrastructure  deployment  focuses  on  providing  technology  to  allow  an 
instructor  to  give  a multimedia  presentation  in  a digital  classroom.  It  is  impossible  to  develop  an  infrastructure 
that  will  accommodate  every  lecturer  or  class  ever  held  in  a digital  classroom.  Some  professors  will  use 
PowerPoint  slides  while  others  prefer  to  use  transparencies  while  still  others  stick  to  the  standard  chalkboard 
method.  In  addition,  instructors  using  digital  presentation  media  may  require  a variety  of  software.  Managing  a 
few  pieces  of  software  is  tractable,  however  a system  to  manage  lots  of  software  is  not.  Fortunately,  a large 
percentage  of  cases  can  be  accommodated  with  a standard  collection  of  hardware  and  software.  Minimally,  a 
classroom  should  include  a data  projector  to  show  PowerPoint  slides  or  other  computer  video  in  addition  to 
providing  an  Internet  connection  for  a presentation  laptop  or  desktop  machine. 

Selecting  a presentation  computer  and  data  projector  for  purchase  requires  some  thought  about  the 
specific  classroom  and  the  complete  functionality  that  will  eventually  be  supported  by  the  classroom.  We 
identify  three  main  concerns  that  need  to  be  addressed  when  selecting  equipment.  The  first  is  compatibility.  A 
major  concern  is  whether  or  not  each  piece  of  equipment  will  be  compatible  with  the  remaining  infrastructure. 
For  example,  if  the  classroom  will  eventually  have  a room  control  system  to  control  various  components  (e.g., 
power  on/off,  input  device  switching,  etc),  does  the  data  projector  support  that  type  of  control?  An  additional 
concern  with  data  projectors  is  how  to  install  our  mount  them.  One  option  is  to  simply  purchase  a media  cart 
where  a projector  can  be  stored.  However,  a media  cart  is  not  a scalable  or  permanent  solution.  The  preferred 
solution  is  to  mount  the  data  projectors  into  the  ceiling.  This  requires  the  purchase  of  a ceiling-mount  kit  for 
each  projector.  Additional  concerns  include  providing  a power  source  as  well  as  ensuring  that  the  ceiling  is 
high  enough  to  mount  the  projectors  out  of  the  way  of  sight  for  students  and  other  equipment.  Once  a data 
projector  and  presentation  computer  have  been  selected,  the  next  concern  is  connecting  them  together.  The 
standard  solution  is  to  run  a VGA  cable  from  the  computer  to  the  projector.  However,  in  a digital  classroom  the 
distance  might  be  too  great  thus  the  quality  of  the  video  signal  degrades.  The  solution  is  to  purchase  a signal 
interface  --  a device  to  boost  a computer  video  signal  such  that  it  can  travel  greater  distances. 


Figure  1:  The  UCSB  Digital  Classroom. 


Figure  2:  The  UCSB  digital  classroom 
infrastructure. 


The  UCSB  digital  classroom  shown  in  Figure  1 has  three  ceiling  mounted  data  projectors  that  project 
on  standard  projection  screens,  two  presentation  laptop  computers  with  signal  interfaces  at  the  front  of  the 
room,  and  one  presentation  desktop  with  signal  interface  at  the  back  of  the  room.  In  addition,  we  provide  a 
VCR  for  showing  standard  VHS  videotapes.  Each  computer  has  a DVD  player  and  an  Internet  connection  and 
can  be  used  for  showing  DVDs,  presenting  PowerPoint  slides,  and/or  web  browsing.  In  addition,  speakers  may 
bring  their  own  laptop  computers  with  specialized  hardware  and/or  software  and  use  the  data  projectors  we 
provide.  Finally,  students  may  connect  to  the  Internet  using  their  own  laptops  or  PDAs  though  a 10Mbps 
wireless  network. 

We  are  currently  in  the  process  of  identifying  other  extensions  to  the  infrastructure.  Short-term 
extensions  include  integrating  existing  technology  such  as  a document  camera.  Longer-term  extensions  include 
upgrading  the  display  technology  to  be  more  sophisticated.  Instead  of  three  separate  screens  we  could  have  a 
single  wall-sized  display  (Fox  et  al.  00). 
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A Webcasting  Studio 


Once  the  basic  infrastructure  is  in  place  to  support  lectures  in  a classroom,  the  next  step  is  expanding 
the  infrastructure  to  include  support  for  webcasting.  A webcasting  environment  captures  audio  and  video  feeds 
generated  in  the  classroom  and  sends  them,  potentially  accompanied  by  other  materials  such  as  slides,  over  the 
Internet  to  a remote  location.  There  are  two  pieces  involved  in  making  this  happen.  The  first  is  to  provide 
support  for  capturing  video  and  audio  of  the  presentation.  The  remote  audience  should  be  able  to  see  and  hear 
the  instructor.  Second,  multimedia  material  (e.g.,  slides)  presented  to  the  local  audience  should  also  be 
presented  to  the  remote  audience. 

The  minimal  requirements  for  a webcasting  environment  are  one  camera,  a microphone  for  the 
instructor,  and  an  encoding  computer.  However,  producing  a reasonable  quality  webcast  requires  a great  deal 
more  effort.  First,  a single  camera  can  be  limiting  if  you  hope  to  capture  all  of  the  activity  that  occurs  in  a 
classroom.  To  capture  all  of  the  classroom  activity  including  instructor  and  student  movement,  multiple 
cameras  must  be  mounted  in  various  locations  in  the  room.  In  addition,  capturing  slides  or  a web  page  can  be 
done  simply  by  focusing  a camera  on  the  projection  screen.  However,  capturing  the  computer  video  feed 
straight  from  the  computer  can  produce  a higher  quality  image.  Determining  which  streams  are  encoded  and 
webcast  at  any  given  time  adds  a non-trivial  bit  of  complexity  to  the  system.  Generally,  a producer  must  be 
available  to  manually  select  the  stream  for  webcast.  A producer  is  generally  a student  or  staff  member  who 
produces  the  webcast  by  controlling  the  encoding  tools  and  selecting  the  appropriate  video  and  audio  streams. 

Managing  multiple  streams  simultaneously  introduces  a host  of  complexities.  First,  devices  such  as 
cameras  have  an  associated  control  interface  and  may  be  controlled  (e.g.,  zoomed)  from  a remote  control  or 
computer  interface.  However,  when  numerous,  heterogeneous  devices  are  installed  in  a room,  it  becomes 
difficult  to  control  all  of  the  individual  elements.  Rather  than  having  multiple  interfaces  such  as  computers  and 
remote  controls,  it  is  desirable  to  support  a single,  integrated  interface  that  supports  control  of  many  or  all  of  the 
devices  in  the  room  (Yu  et  al.  01).  Also,  it  is  unlikely  that  any  hardware  configuration  would  support  encoding 
of  all  possible  streams  simultaneously.  The  general  protocol  is  to  select  a subset  of  all  available  streams  for 
encoding.  In  order  to  accomplish  this,  the  infrastructure  must  include  a video  matrix  switch.  A video  matrix 
switch  is  a device  that  takes  as  input  a set  of  video  signals  and  allows  routing  of  the  video  signals  to  one  or 
more  of  the  switch  output  channels.  By  routing  all  video  through  a video  switch,  the  architecture  becomes 
much  more  modular.  Audio  signal  capture  poses  many  of  the  same  problems  encountered  by  video  stream 
capture.  If  an  infrastructure  supports  only  a single  microphone  used  by  the  instructor,  the  signal  can  be  directly 
connected  to  a sound  card.  However,  this  model  begins  to  break  down  relatively  quickly.  Capturing  other 
audio  sources  such  as  audience  discussion  or  the  audio  track  from  a VHS  video  is  imperative.  The  ultimate 
solution  is  to  install  a professional  quality  audio  system  that  is  capable  of  mixing  audio  signals  from  the 
instructor,  microphones  placed  to  capture  ubiquitous  audience  discussion,  audio  streams  from  remote  sites,  and 
other  sources  such  as  video.  Finally,  video  format  compatibility  is  a concern.  Video  capture  hardware 
generally  expects  a composite  video  signal.  Therefore,  the  computer  video  signal  sent  from  a presentation 
computer  cannot  be  directly  encoded  as  part  of  a video  stream.  The  solution  is  to  use  a scan  converter  to 
convert  the  high-quality  computer  video  signal  to  a composite  video  signal. 

Developing  an  infrastructure  to  manage  and  select  multiple  media  streams  in  various  formats  is 
extremely  complex  and  requires  much  thought.  However,  once  the  architecture  to  capture  a selected  stream  is 
in  place,  the  next  step  is  to  determine  how  to  encode  the  audio  and  video  streams  into  a format  that  can  be  easily 
distributed  and  viewed  by  remote  participants.  There  are  two  primary  concerns  that  need  to  be  addressed.  The 
first  is  expense.  Hardware -based  encoding  solutions  provide  efficient,  high-quality  encoding.  However,  while 
cheaper  solutions  are  on  the  way  (e.g.,  NCast-www.ncast.com  and  VBrick  - www.vbrick.com),  current 
solutions  can  be  very  expensive.  Software -based  encoding  solutions  can  also  be  expensive,  but  a range  of 
lower-cost  solutions  exist  as  well.  While  encoding  formats  such  as  MPEG-4  may  seem  to  be  the  highest 
quality  solution,  the  chosen  encoding  format  should  have  a widely  available,  cross-platform  decoder/viewer. 
Ideally,  students  would  have  the  viewing  tools  already  installed  on  their  desktop  thus  avoiding  having  to 
download  or  purchase  them.  The  most  common  tools  currently  on  the  market  are  RealPlayer  and  Windows 
Media  Player  both  of  which  have  freely  available  and  easy-to-install  viewers  and  encoders. 

The  UCSB  digital  classroom  infrastructure  supports  webcast  of  a single  video  stream  selected  from  a 
set  cameras  or  other  video  input.  In  addition,  we  have  implemented  the  audio  setup  suggested  by  the  Access 
Grid  specification  (www.accessgrid.org ^ which  includes  a number  of  microphones  to  pick  up  ambient  sound 
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and  a high  quality  sound  mixer*.  The  heart  of  the  UCSB  classroom  infrastructure  is  a 12  input,  8 output  video 
matrix  switch.  The  switch  supports  both  composite  and  computer  video.  The  first  two  switch  inputs  (see 
Figure  2)  are  the  composite  video  signals  generated  from  the  classroom  cameras  mounted  to  capture  both 
instructor  and  audience  views.  In  addition,  composite  video  from  the  VCR  is  also  routed  through  the  switch. 
All  computer  video  sources  from  the  presentation  machines,  as  well  as  remote  sources,  are  also  inputs  to  the 
switch.  However,  in  order  to  capture  and  encode  any  of  these  feeds,  the  feed  must  be  routed  through  a scan 
converter  and  converted  from  computer  video  to  composite  video.  The  resulting  stream  is  then  fed  back  into 
the  switch  and  may  be  selected  for  encoding.  The  streams  selected  for  encoding  are  fed  into  a video  capture 
card  installed  on  a standai  d FC.  in  addition,  the  mixed  audio  stream  is  fed  into  the  sound  card  of  the  PC.  The 
audio  and  video  streams  are  then  encoded  using  either  RealMedia  format  or  Windows  Media  format  and 
webcast  to  a remote  audience. 


Remote  Collaboration 

The  one-way  distance  learning  scenario  supported  by  a webcasting  studio  does  not  capture  the  true 
learning  experience.  For  distance  learning  to  truly  be  effective,  we  have  to  enable  remote  students  to  ask 
questions,  participate  in  discussion,  and  otherwise  appear  to  be  at  the  local  site.  This  requires  two  pieces.  First, 
the  remote  site  must  have  facilities  similar  to  that  of  the  local  site.  Second,  the  technology  to  communicate  in 
realtime  between  the  two  sites  must  be  in  place. 

Ideally,  a remote  site  would  be  an  exact  replica  of  the  primary  site.  In  reality  though,  the  infrastructure 
of  a remote  site  is  generally  a subset  of  the  infrastructure  deployed  at  a primary  site.  Minimally,  a remote  site 
must  include  a camera,  a microphone,  an  encoding  machine,  and  a decoding  machine.  This  could  take  the  form 
of  a standard  webcam  and  microphone  connected  to  a student’s  home  PC  where  the  PC  is  both  the  encoding  and 
decoding  machine.  However,  if  a remote  site  is  designed  to  support  multiple  students  (e.g.,  an  extension 
campus)  a more  complex  infrastructure  is  necessary.  For  example,  displaying  video  of  the  instructor  on  a PC 
screen  is  probably  not  sufficient.  In  addition,  the  camera  should  be  able  to  capture  an  audience  larger  than  a 
single  person.  Fortunately,  the  problems  encountered  when  developing  the  remote  site  infrastructure  are  the 
same  problems  encountered  when  deploying  the  primary  classroom  and  thus  we  can  apply  the  same  solutions. 

While  deploying  the  remote  site  infrastructure  itself  is  relatively  straightforward  and  follows  directly 
from  the  experiences  we  have  already  described,  deploying  a communication  layer  on  top  of  the  infrastructure 
is  more  difficult.  There  are  two  main  topics  to  be  addressed.  The  first  is  realtime  encoding.  Our  first 
inclination  was  to  use  the  same  software  encoding  solutions  for  realtime  communication  that  we  used  for  one- 
way webcasting.  However,  off-the-shelf  encoding  software  such  as  RealMedia  and  Windows  Media  introduce 
intolerable  delays  from  7 to  15  seconds  one-way  due  to  buffering  requirements.  As  with  many  of  the  problems 
we  have  encountered,  extremely  e>pensive  encoding  solutions  exist.  But,  deploying  these  solutions  without 
knowing  whether  or  not  they  are  going  to  work  is  a risky  venture.  The  alternate  solution  is  to  use  encoding 
software  designed  for  video  conferencing  such  as  vie  or  Microsoft  NetMeeting.  While  off-the-shelf  video 
conferencing  software  is  generally  easy  to  use,  quality  is  sacrificed  to  meet  realtime  requirements.  Also,  in 
addition  to  watching  video  generated  at  a primary  site,  students  at  a remote  should  be  able  to  ask  questions 
(Malpani  & Rowe  97)  and  access  shared  components  such  as  whiteboards.  Additionally,  as  the  number  of 
remote  sites  grows,  it  becomes  necessary  to  manage  the  sites  so  that  only  one  remote  site  is  asking  a question  or 
sourcing  video  at  a given  time.  Some  of  these  problems  may  be  solved  with  standard  videoconferencing 
software.  However,  the  requirements  are  different  for  any  given  infrastructure  and  differences  require 
specialized  solutions. 

We  have  deployed  a test  remote  site  we  call  a kiosk.  The  kiosk  has  one  camera,  a single  microphone 
(to  be  passed  from  participant  to  participant),  a single  encoding  machine,  and  two  decoding  laptop  computers 
connected  to  data  projectors  for  display  (see  Figure  3).  We  use  Microsoft  NetMeeting  to  communicate  between 
the  sites.  The  primary  classroom  sends  slides  (using  the  NetMeeting  screen  sharing  facility),  a video  feed  of  the 
speaker,  and  an  audio  stream  from  the  speaker  microphone  to  the  kiosk.  The  kiosk  sends  a single  video  stream 
and  a single  audio  stream  back  to  the  primary  classroom  where  it  is  displayed  on  a side  mounted  projection 
screen.  Figure  4 shows  the  flow  of  streams  between  sites.  In  building  the  kiosk,  we  have  made  a number  of 
simplifying  assumptions.  First,  to  avoid  the  problems  of  floor  control  and  remote  stream  selection,  we  assume 

' The  Access  Grid  is  an  initiative  to  enable  research  labs  and  universities  to  conduct  large  scale,  distributed 
meetings  over  the  Internet  through  an  always  on  infrastructure. 


that  there  are  only  two  sites  participating  at  any  given  time.  Second,  we  assume  that  the  only  audio  stream  for 
either  site  is  generated  from  the  single  microphone  available  at  that  site  thus  we  do  not  have  to  mix  the  audio 
signal.  Extensions  to  our  infrastructure  would  include  enabling  multiple  sites  to  participate  at  any  given  time. 


Lecture  Replay 

One  of  the  primary  advantages  of  recording  a lecture  or  course  is  that  it  may  be  reviewed  after  the  fact. 
Students  may  review  material  at  the  end  of  a course  or  before  an  examination.  In  addition,  lectures  given  by 
guest  speakers  or  experts  in  their  field  may  be  archived  and  watched  by  students  for  years  to  come  (Tsichritzis 
99).  There  are  a number  of  issues  involved  with  recording  lectures  for  replay.  However,  most  of  the  difficultly 
lies  in  providing  more  functionality  over  straightforward,  sequential  playback.  The  goal  of  the  infrastructure  is 
simply  to  provide  a content  base  that  may  be  accessed  and  used  to  research  new  methods  of  access.  The 
challenge  is  to  record  and  encode  the  content  such  that  it  may  be  accessed  later  in  different  ways  using  a variety 
of  tools.  Support  of  VCR-style  interactivity  as  well  as  integration  of  multiple  media  types  provides  a more 
effective  replay  experience. 

There  are  two  main  issues  in  the  deployment  of  an  infrastructure  for  lecture  replay.  The  first  is  the 
deployment  of  a mediaserver.  An  hour  of  lecture  can  be  SOOMbytes  or  more  depending  on  the  encoding 
scheme.  Therefore,  the  first  concern  is  to  deploy  a media  server  with  enough  disk  space  to  hold  the  recorded 
lectures.  The  second  issue  is  to  determine  which  encoding  standard  to  use.  The  simplest  solution  is  to 
simultaneously  save  the  stream  already  being  encoded  for  webcast.  However,  it  may  be  desirable  to  support 
postprocessing  of  the  stream  such  as  including  synchronization  between  video  and  slides  (Mukopadhyay  99). 

The  UCSB  digital  classroom  primarily  focuses  on  replay  of  the  webcast  stream.  During  webcast,  the 
encoded  Real  or  Windows  Media  stream  is  saved  to  a file  and  may  be  streamed  from  the  server  later.  We  are 
still  in  the  early  stages  of  generating  a content  base  and  thus  have  not  yet  deployed  a server  to  support  large 
amounts  of  data.  We  have  also  built  a tool  to  support  synchronization  between  notes  written  by  participants  and 
the  video  of  the  lecture. 


Potential  Impact  on  Education 

Multimedia  in  the  classroom  presents  a number  of  opportunities  for  students  and  educators  alike.  First, 
using  technology,  learning  can  become  a more  interactive  process.  Teachers  can  use  a variety  of  media  to  teach 
students  in  new  and  different  ways  while  students  can  use  technology  such  as  laptop  computers  or  PDAs  to 
share  information  and  communicate  with  one  another.  Moreover,  using  webcasting  and  remote  collaboration 
facilities,  we  can  remove  the  physical  barrier  imposed  by  a classroom  environment.  Enrolled  students  can  “web 
commute”  rather  than  miss  lectures,  and  students  who  may  have  otherwise  been  unable  to  take  a class  at  all 
have  more  flexibility  to  choose  to  attend  lecture  from  their  home.  Additionally,  the  number  of  people  who  can 
fit  in  a room  or  be  physically  present  no  longer  limits  audience  size.  For  example,  if  an  expert  speaker  visits  a 
university,  the  number  of  people  interested  in  attending  her  lecture  may  exceed  the  capacity  of  the  largest 
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Figure  3:  Remote  site  infrastructure. 


Figure  4:  Classroom  and  remote  site  communication. 


lecture  hall  on  campus.  Finally,  as  webcasting  enables  a limitless  number  of  people  to  watch  a lecture  given  by 
an  expert,  lecture  archival  and  replay  can  make  the  same  lecture  persistent.  Students  for  years  to  come  can 
watch  and  learn  from  experts  in  their  field 

Conclusion 

Deploying  the  infrastructure  for  a digital  classroom  is  a long  and  often  tedious  process.  In  theory,  it 
involves  technical  staff,  facilities  staff,  as  well  as  researchers.  In  the  past  year,  we  have  brought  the  UCSB 
digital  classroom  online  to  support  the  four  functions  defined  by  our  model  to  varying  degrees  of  completion. 
While  the  process  has  been  slower  than  we  originally  anticipated,  delays  in  deployment  can  be  attributed  to  lack 
of  staff  as  well  as  to  factors  such  as  back-ordered  equipment  and  equipment  incompatibility.  However,  we  are 
pleased  with  the  resulting  infrastructure  and  its  use  so  far. 

To  date,  we  have  used  the  classroom  for  three  different  types  of  events:  (1)  we  have  hosted  and 
recorded  a standard  graduate  course;  (2)  we  have  conducted  a graduate  student  seminar  with  participants 
distributed  between  the  classroom  and  our  remote  kiosk;  and  (3)  we  webcasted  a talk  given  by  a Nobel  laureate 
to  an  elementary  school  classroom  located  in  a nearby  town.  The  Nobel  laureate's  talk  was  also  viewed  from 
other  locations  in  the  US,  as  well  as  in  the  UK.  Overall,  our  experiences  have  been  successful.  While  we  are 
working  toward  automating  the  process  of  set  up  and  configuration  as  well  as  lecture  production,  our 
infrastructure  supports  nearly  all  of  the  intended  functionality. 

Throughout  the  process  of  designing  and  implementing  our  classroom  model,  we  have  identified  a 
number  of  considerations  that  may  not  be  immediately  obvious  to  the  designer  or  implementer.  These 
considerations  range  from  high-level  decisions  such  as  supported  functionality  to  low  level  choices  such  as 
required  equipment.  We  would  like  to  acknowledge  many  helpful  discussions  with  Larry  Rowe  and  other 
classroom  architects  that  have  helped  us  to  determine  the  common  properties  of  most  digital  classrooms.  We 
believe  that  most  digital  classroom  implementations  support  similar  functionality.  Thus,  it  is  unnecessary  for 
each  design  team  to  start  from  ground  zero.  As  campuses  around  the  world  begin  to  embrace  technology  in 
their  curriculums,  it  is  essential  to  be  able  to  quickly  and  easily  deploy  technologically  enhanced  meeting 
spaces. 
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Abstract:  Team  collaboration  is  an  important  part  of  working  in  business  today.  We 
produced  an  experiment  in  which  students  were  given  the  opportunity  to  use  many 
different  collaborative  tools.  We  used  a pre-test  survey  and  post-test  survey  to  evaluate 
the  student’s  willingness  to  use  collaborative  tools.  The  paper  describes  the  test,  findings 
and  ramifications. 

Introduction: 

Team  collaboration  is  an  important  part  of  working  in  business  today.  Because  teams  are 
used  often  in  business  settings,  universities  require  prospective  business  candidates  to  become 
exposed  to  collaboration  through  team  projects.  One  type  of  collaboration  that  can  be  used  in  team 
settings  is  electronic.  The  purpose  of  this  study  is  to  show  that  electronic  collaboration  can  impact 
students’  frequency  of  technology  usage  as  well  as  their  acceptance  to  using  collaborative 
technologies.  Our  first  goal  was  to  verify  that  students  will  use  technology  more  often  over  time  in 
project  team  settings  after  they  have  been  exposed  to  both  the  technology  and  the  actual  team 
collaboration.  Our  second  goal  is  to  verify  that  students  will  find  the  technologies  useful,  thus 
increasing  their  acceptance  of  it,  with  certain  features  for  group  projects.  Therefore,  the  student 
acceptance  of  using  technology  in  team  settings  will  increase  as  they  become  more  acquainted 
with  the  tool  and  its  features. 

Several  new  tools  have  been  designed  to  make  teamwork  and  collaboration  more  efficient. 
These  groupware  programs  can  allow  users  to  exchange  files,  draw  collaborative  sketches,  and 
use  auditory  voicing  along  with  providing  many  other  features.  These  features  are  attractive  to 
users  who  are  highly  Involved  in  teams  and  collaboration.  Furthermore,  electronic  collaboration  is 
becoming  more  Important  In  both  academic  and  business  settings.  The  benefits  these  technologies 
can  offer  to  users  will  hopefully  solve  problems  of  distance,  communication,  sharing  information, 
and  costs. 

This  study  in  electronic  collaboration  usage  and  implications  which  extends  what  has 
already  been  done  In  groupware  and  group  collaboration  systems.  Electronic  collaboration  holds 
promise  for  future  studies  In  a variety  of  settings  as  technology  continues  to  grow  and  becomes  an 
even  more  integral  part  of  culture  throughout  the  world.  The  structure  of  this  paper  is  presented  in 
the  following  format:  I.  Introduction,  II.  Literature  Review,  III.  Hypothesis  Development,  IV. 
Methodology,  V.  Data  Analysis,  VI.  Research  Results,  and  VII.  Recommendations,  and  VIII.  Further 
Studies. 

Literature  Review: 

Information  technology  has  had  a revolutionary  Impact  on  society  within  the  last  several 
years.  The  recent  growth  of  networks,  cellular  communications,  and  fiber  optic  cables  have 
spawned  the  Internet  boom  and  have  provided  opportunities  for  convergence  with  existing  media 
such  as  television  (Halal,  1992).  This  shows  the  trend  of  technology  being  converged  into  current 
social  norms.  Other  examples  Include  the  Internet  and  teleconferencing  as  means  to  improving 
communication  In  office,  school,  and  home  settings  (Halal,  1992).  These  instances  further  show 
the  fact  that  technology  is  changing  the  way  we  live  and  work. 
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The  result  of  these  trends  shows  that  technology  is  a catalyst  for  defining  our  social 
domain.  Consequently,  the  term  “virtual  community”  formed  to  reflect  how  our  culture  acts  toward 
technology;  a virtual  community  is  one  where  the  people  are  able  to  communicate  and  interact  in  an 
online  environment  (Halal,  1992).  Because  of  the  heavy  reliance  on  technology  in  our  global 
society,  people  have  a strong  need  for  communication.  Collaboration  is  one  aspect  which 
technology  can  advance  in  order  to  bring  citizens  in  virtual  communities  closer  together. 

Collaboration  is  a term  for  actively  communicating  with  other  people.  The  benefits  of 
effective  collaboration  can  be  highly  favorable  in  work,  academic,  and  personal  settings  (Chandra, 

1996) .  This  current  term  for  communicating  with  the  aid  of  technology  is  called  electronic 
collaburaiiuri.  There  is  a irernenduus  synergy  ihai  exists  ifi  an  effective  e-cuilaboration 
environment.  The  benefits  from  these  virtual  teams  can  solve  many  problems  such  as  distance  and 
schedule  conflicts. 

With  the  enhancements  of  team  communication  in  business  and  academic  environments, 
electronic  collaboration  can  revolutionize  the  way  people  exchange  information.  For  example, 
many  businesses  have  integrated  collaborative  designs  and  opted  toward  a paperless  environment. 
In  addition  to  e-collaboration  being  an  advantage,  there  are  several  types  of  electronic  tools  that 
offer  unique  features. 

Electronic  collaboration  tools,  also  known  as  groupware  contains  a certain  outline  of 
functionality,  which  includes  messaging,  scheduling,  and  audio  communication.  There  are  also 
special  capabilities  of  e-mail,  including  beeper  triggering,  which  can  be  useful  to  Individuals  who 
carry  pagers.  Another  type  of  e-collaboration  is  instant  messaging.  Three  popular  services,  which 
provide  instant  messaging,  are  Yahoo,  AOL,  and  MSN  (Agnew,  2000).  Instant  messaging  is  a 
forum  for  real-time  communication  that  can  include  text  chat  and  audio  transmission  over  the 
Internet.  This  real-time  synergy  allows  individuals  to  communicate  instantaneously  with  other 
users,  unlike  e-mail,  which  stores  a record.  Furthermore,  text  chat  simulates  person-to-person 
chatting  while  e-mail  represents  letter  writing. 

Additionally,  there  are  other  groupware  products  on  the  market  that  incorporate  several 
features  into  one  package.  Among  these  tools  are  Blackboard,  Stuffincommon,  Centranow, 

Webex,  Placeware,  Blink,  Backflip,  Groove,  and  Lotus  Notes.  These  e-collaboration  tools  each 
have  their  own  unique  features.  One  feature  is  known  as  an  electronic  calendar  (Agnew,  2000). 
This  can  allow  one  to  schedule  and  plan  activities  online.  Other  important  features  of  e- 
collaboration  tools  include  notification  of  users  online  and  file  download  capabilities.  These  abilities 
can  let  someone  know  when  a team  member  is  online.  Furthermore,  the  ability  to  download  and 
upload  files  is  a tremendous  advantage  to  exchanging  information. 

Several  popular  groupware  packages,  such  as  Groove,  Lotus  Notes,  and  E-Room,  allow 
users  to  utilize  one  program  for  many  features  (Fontana,  2001).  Some  of  these  features  include 
threaded  discussion  and  file  sharing  across  the  Internet.  Furthermore,  bookmarks,  post-it  notes, 
and  white  boards  also  allow  users  to  post  and  store  information  online.  Bookmarks  are  useful  for 
saving  links  to  other  websites  while  post-its  can  act  as  reminders  for  information.  Also,  whiteboards 
are  a popular  feature  that  allows  users  to  interact  real-time  and  draw  (Fontana,  2001).  Though  e- 
collaboration  features  are  very  important  for  user-specific  tasks,  the  benefits  from  those  features 
are  far  more  significant.  Another  important  benefit  is  the  improved  flow  of  communication  between 
users  (McDougall,  2000).  This  peer-to-peer  advantage  can  increase  the  effectiveness  of  online 
collaboration  by  minimizing  interruptions  and  system  lock-downs. 

Flexibility,  navigation,  and  control  are  very  important  when  using  software  packages  (Borck, 
2000).  These  benefits  allow  the  system  to  adapt  to  the  user  and  promote  usability.  Other 
advantages  of  using  groupware  include  convenience,  accessibility  and  user-friendly  interfaces.  If  a 
collaboration  tool  is  fast,  easy  to  use,  easy  to  learn,  and  visually  appealing,  individuals  will  more 
likely  be  willing  to  use  the  program.  Also,  timing  and  costs  are  important  considerations  (Darrow, 

1997) .  Collaboration  tools  that  save  users  time,  allow  them  to  work  over  distant  geographic 
boundaries,  and  save  costs,  can  be  tremendous  benefits  to  users  in  work  as  well  as  academic 
settings  for  team  projects. 

Another  important  aspect  of  electronic  collaboration  is  its  usage  by  individuals  in  team 
settings.  The  term  usage  refers  to  the  frequency  of  an  e-collaboration  tool  or  how  often  it  was 
used.  The  frequency  of  use  indicates  the  number  of  times  a user  employs  or  activates  an 
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electronic  collaboration  tool  or  one  of  its  features.  If  a user  increases  his  frequency  of  usage,  he  is 
more  likely  to  become  more  dependent  on  the  collaboration  tool. 

In  addition  to  frequency,  usage  can  also  be  referred  to  by  effectiveness.  For  instance,  if  a 
user  sees  a purpose  in  using  an  e-collaboration  tool,  he  is  more  likely  to  use  it.  This  shows  that  the 
tool  has  been  effective  in  providing  a feature  to  the  user  for  intended  use.  The  variables  that  mainly 
affect  this  acceptance  in  use  include  level  of  experience  using  the  product,  level  of  tool  exposure, 
and  ease  of  use  (Agarwal,  2000).  If  a user  has  had  more  experience  using  a product  or  has  had  a 
significant  level  of  exposure  to  it,  he  may  be  more  willing  to  use  it,  thus,  increasing  his  acceptance 
of  the  product.  Acceptance  refers  to  the  opposite  of  resistance,  as  it  is  used  in  an  organizational  or 
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new  one.  If  a user  sees  a purpose  for  using  an  e-collaboration  tool  or  feature,  he  may  increase  his 
level  of  usage. 


Hypothesis  Development: 

Within  this  study,  all  subject  matter  is  written  in  terms  of  class  or  group  project  teams.  With 
this  in  mind,  we  will  proceed  to  the  hypothesis  development.  There  are  several  variables  in  this 
study  that  require  hypothesis  testing  in  order  to  generate  conclusions. 

In  this  study,  an  analysis  of  the  electronic  collaboration  affects  on  students  in  work  teams 
and  their  implications  was  provided.  The  first  objective  was  to  measure  how  often  students  use  a 
particular  technology  over  time  in  the  context  of  a team  setting.  The  variables  measured  for  this 
objective  include  the  number  of  occurrences  or  times  using  a particular  technology  as  well  as  the 
average  time  spent  using  a technology  in  one  session.  The  arguments  presented  above  form  the 
following  hypothesis: 

H1 : The  frequency  of  technology  usage  by  experienced  students  after  two  weeks 

using  the  tool  in  a team  setting  will  increase 

The  second  objective  in  this  study  was  to  measure  student  preferences  on  collaboration 
tool  features  as  well  as  the  benefits  that  can  be  derived  from  them.  The  variables  measured  for  this 
objective  include  the  level  of  importance  of  a particular  technology  feature  as  well  as  the  level  of 
usage  while  working  on  team  projects  over  a two-week  time  frame.  These  arguments  present  the 
following  hypothesis: 

H2:  The  usefulness  and  acceptance  of  technologies  by  experienced  students  after 

two  weeks  using  the  tool  in  a team  setting  will  increase 


Methodology: 


The  methodology  of  this  study  is  a progressive  test  experiment  that  compares  sets  of  data 
over  a period  of  time.  The  design  of  the  experiment  began  with  a pilot  test  consisting  of  a survey 
targeting  college  students.  The  pilot  was  given  to  approximately  10  candidates  who  were  upper 
level  information  systems  students. 

The  next  step  of  the  study  was  to  survey  a set  of  students.  The  set  consisted  of  two 
classes;  these  were  upper  level  and  graduate  students.  Although  there  were  75  original  subjects, 
only  38  subjects  were  usable.  There  were  actually  three  separate  surveys  given  to  each  student 
throughout  the  semester.  The  first  survey  obtained  general  background  information  on  each 
subject  and  was  given  in  the  middle  of  the  semester.  The  second  and  third  surveys  were  given  two 
weeks  apart  toward  the  end  of  the  semester. 

Throughout  the  experiment,  the  students  were  given  the  opportunity  to  use  several  e- 
collaboration  tools  including  e-mail,  instant  messaging,  and  peer-to-peer  tools  including  Blackboard 
and  Groove;  they  were  encouraged  to  use  these  tools  for  their  team  projects  throughout  the 
semester.  The  surveys  were  posted  online  through  a university  server.  This  allowed  the  users  to 
complete  the  surveys  at  convenient  times. 

Data  Analysis: 
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After  the  data  was  collected,  we  performed  data  analysis  to  determine  if  there  was  a 
statistical  significance  in  the  responses  sent  via  electronic  surveys. 

For  this  survey  there  were  several  variables  analyzed  primarily  to  describe  background 
data  about  the  subjects.  This  included  the  variables  gender,  age,  year  in  school,  and  location  of 


address. 

The  first  hypothesis  to  measure  frequency  of  technology  usage  was  tested  using  several 
variables  for  different  collaboration  types:  email,  instant  messaging.  Blackboard,  and  peer-to-peer 
groupware.  These  variables  were  compared  from  survey  two  to  survey  three  after  two  weeks  of 
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two  weeks  was  also  included. 

Finally,  the  second  hypothesis  to  measure  the  usefulness  and  increase  in  importance  or 
acceptance  for  online  collaboration  was  tested.  Comparing  survey  two  and  survey  three  in  terms  of 
the  features  and  benefits  did  this.  There  were  a total  of  11  feature  variables  and  14  benefit 
variables  measured  in  this  study. 


Research  Results: 


After  the  data  was  analyzed,  the  results  of  the  research  project  were  reported  in  order  to 
deduce  outcomes  and  generate  conclusions.  In  terms  of  experience  in  team  environments,  all 
users  stated  they  had  opportunities  to  work  in  those  settings.  The  average  amount  of  team  projects 
students  have  had  in  the  last  year  was  between  five  and  seven.  Also,  teams  mostly  met  at  the 
campus  library  on  evenings  in  person.  The  majority  of  subjects  claimed  to  exchange  information 
online  through  email,  as  opposed  to  instant  messaging  and  other  more  complex  groupware 
packages.  Instead  of  meeting  online  for  team  project  collaboration,  they  normally  shared 
information  electronically. 

H1:  The  frequency  of  technology  usage  by  experienced  students  after  two  weeks 
using  the  tool  in  a team  setting  will  increase 

The  usage  of  Instant  messaging  slightly  declined  over  two  weeks.  This  could  be  a result 
of  students  meeting  physically  in  their  teams  more  often  than  electronically. 

There  is  no  significant  difference  in  the  amount  of  time  spent  per  session  using  e-mail  as 
opposed  to  instant  messaging.  Hypothesis  1 is  not  supported  due  to  the  decline  in  the  means.  The 
decline  in  email  usage  over  the  two-week  period  could  be  a result  of  team  members  exchanging 
more  files  in  one  setting.  Also,  only  a few  members  of  each  team  may  have  been  responsible  for 
updating  the  other  members  and  sending  files  through  email. 

Third,  results  for  the  groupware.  Blackboard,  demonstrated  a decline  in  frequency  of 
usage.  The  means  for  this  variable  are  distinct:  Hypothesis  1 is  not  supported  due  to  the  decline  in 
the  means.  This  shows  that  there  was  actually  a decrease  in  the  frequency  of  technology  usage  for 
Blackboard.  This  decrease  Is  mostly  a result  of  a disruption  in  class  meeting  times  from 
Thanksgiving  break  (see  Further  Study).  Also,  the  number  of  class  assignments  requiring  students 
to  access  Blackboard  had  decreased  before  the  Thanksgiving  break. 

Finally,  students  were  tested  on  their  usage  of  their  two  most  preferred  groupware 
programs.  However,  the  majority  of  students  did  not  have  a preference  for  using  a groupware 
collaboration  tool,  nor  did  they  spend  time  using  them  during  their  group  activities.  This  was  the 
case  before  the  study  as  well  as  after  the  two-week  test. 

H2:  The  usefulness  and  acceptance  of  technologies  by  experienced  students  after 
two  weeks  using  the  tool  in  a team  setting  will  increase 

First,  the  variable  for  the  collaboration  feature  real-time  voice  was  tested.  Initially,  students 
believed  that  real-time  voice  was  an  average  concern.  It  was  rated  3 on  a scale  of  1 to  7,  where  7 
Is  the  highest  rating.  However,  after  the  experiment,  preference  for  voice  changed  to  a rating  of 
two.  This  shows  that  students  did  not  see  much  of  a purpose  for  using  real-time  voice  with  in  their 
teams.  In  addition  to  voice,  real-time  text  messaging  had  also  slightly  decreased  in  level  of 
importance.  Initially,  students  rated  real-time  messaging  as  highly  important  at  level  six.  However, 
after  the  study,  students  rated  real-time  text  chat  at  level  of  5.6.  Although  this  change  represents  a 
slight  decline  for  text  chat,  students  generally  use  this  feature  and  find  it  important.  Students’  initial 
perceptions  of  these  features  were  slightly  higher  than  their  actual  level  of  use  during  the  two-week 
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experiment.  The  drop  in  voice  usage  could  have  been  a result  of  restrictions  requiring  students  to 
keep  a low  voice  volume  when  working  in  the  computer  labs. 

The  next  set  of  features  that  represented  a significant  decline  in  preference,  were  file 
download  capabilities  and  Internet  file  sharing.  Initially,  file  download  capabilities  and  file  sharing 
were  rated  at  6.5  and  5.7  respectively.  However,  after  two  weeks,  the  two  features  were  rated  at  5 
and  4.6  respectively.  This  shows  that  within  the  two-week  experiment,  most  users  did  not  use 
these  capabilities  as  much  as  they  initially  did. 

Another  less  commonly  used  feature  was  the  online  calendar.  This  variable  was  initially 
given  a rating  of  5.1  out  of  7.  However,  after  two  weeks,  users  decreased  their  level  of  importance 
rating  to  2.4.  This  shows  that  students  did  not  have  a need  for  coiiaburaiion  tool  caiendars  through 
out  the  span  of  their  project. 

Also,  one  last  feature  was  the  ability  to  notify  others  online.  Before  the  experiment, 
students  had  a preference  of  5.8  out  of  7 for  this  feature.  After  the  two-week  time  period,  students 
rated  online  notification  at  the  range  of  5.5.  This  shows  that  there  was  no  significant  change  in  this 
variable.  Therefore,  online  notification  is  still  considered  moderately  Important  to  students. 

In  addition  to  feature  ratings,  students  were  also  tested  on  whether  they  saw  a benefit  from 
using  electronic  collaboration  tools.  Initially,  students  rated  the  importance  of  control,  access,  and 
convenience  in  the  range  of  6.4.  After  two  weeks  of  working  with  collaboration  tools,  their 
preferences  for  these  three  benefits  changed  to  a range  of  6.2  out  of  7.  Though  there  was  a slight 
drop  in  the  means  between  the  beginning  and  end  of  the  experiment,  the  ratings  were  very  high  at 
both  times,  which  supports  Hypothesis  2.  Because  users  rated  these  benefits  above  6 both  before 
and  after  the  study,  they  believed  the  benefits  were  useful,  thus  increasing  their  acceptance  of  the 
tools. 

The  next  set  of  benefits  that  had  a slightly  higher  decline  in  preferences  included  flexibility, 
information  flow,  navigation,  and  security.  Originally,  students  rated  these  benefits  in  the  range  of 

6.2  out  of  7.  After  two  weeks,  these  students  changed  their  preferences  to  approximately  5.8. 
Although  there  was  a slight  drop,  users  generally  find  these  benefits  important.  However,  they  may 
not  be  as  important  as  they  were  before  the  study. 

The  next  two  benefits  with  similar  initial  preferences  were  the  speed  and  the  user-friendly 
interface  of  the  collaboration  tool,  which  were  initially  rated  at  6.3.  After  the  short  two-week  time 
period  ended,  students  rated  these  features  at  approximately  6.1.  This  shows  that  students  still  find 
system  speed  and  ease  of  use  to  be  very  important  despite  the  fact  that  preferences  dropped 
slightly.  The  student  rankings  were  above  6 on  the  7-point  scale  both  before  and  after  the 
experiment.  This  supports  Hypothesis  2,  because  students  believed  the  benefits  from  fast  and 
easy  usage  were  useful,  which  increased  their  acceptance  of  the  tools. 

In  terms  of  features  available,  users  do  not  have  much  of  a preference  for  the  number  of 
options  that  one  groupware  package  can  offer.  This  was  shown  by  the  decline  in  the  means  from 

5.2  to  4.7  for  the  importance  of  feature  quantity  in  e-collaboration  tools. 

One  final  set  of  benefits  that  users  rated  included  the  ability  to  save  time,  travel,  and  costs. 
Despite  a slight  decline,  students  believe  that  the  benefit  from  time  savings  when  using  e- 
collaboration  tools  are  very  high,  which  supports  Hypothesis  2;  the  benefit  was  found  useful  for 
students  and  has  Increased  their  awareness  and  acceptance  of  the  technologies. 

In  summary,  students  generally  rate  certain  features  of  collaboration  tools  more  important 
then  others.  For  example,  real  text  messaging,  online  notification,  and  file  download  capabilities 
were  the  most  used  over  the  span  of  two  weeks  within  their  team  projects.  This  shows  that 
students  see  a purpose  or  importance  to  using  electronic  collaboration  tools,  such  as  e-mail  and 
instant  messaging,  which  contain  these  features.  Users  ranked  the  importance  for  convenience, 
easy  access,  control,  user-friendly  environment,  speed,  and  time  and  distance-savings  as  the  most 
Important.  Although  the  Importance  of  these  benefits  has  not  significantly  changed  over  the  two- 
week  time  period,  students  have  continued  to  rank  their  usefulness  high.  This  supports  Hypothesis 
2,  because  receiving  a benefit  from  technology  is  considered  very  important  to  students;  therefore, 
their  acceptance  of  the  technologies  has  Increased  as  a result  of  them  using  the  tools  to 
collaborate. 

Recommendation: 
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After  analyzing  the  results  of  this  study,  we  have  proposed  a few  recommendations. 
Hypothesis  1 was  not  supported;  It  is  suggested  that  facilitators  increase  the  level  of  e-tool 
exposure  and  requirements  for  their  students  in  order  for  them  to  gain  experience  using  a multitude 
of  electronic  collaboration  features.  By  making  it  a requirement  for  students  to  study  and  learn  the 
wide  variety  of  collaboration  features  available,  students  would  then  be  more  educated  and 
informed  about  using  an  e-collaboration  tool. 

In  contrast  to  the  first  hypothesis.  Hypothesis  2 was  supported;  students  found  several 
electronic  collaboration  features  necessary  for  communicating  in  virtual  environments.  Thus,  they 
saw  a purpose  for  using  e-collaboration  tools  and  should  be  willing  to  use  these  tools  for  projects  in 
the  future.  I suggest  that  students  shouid  have  many  teauhers  promote  electronic  collaboration 
environments  for  classroom  use  in  order  to  positively  reinforce  the  benefits  from  these  tools. 

Further  Study: 

In  addition  to  this  project,  there  are  several  possible  extensions  for  consideration  when 
doing  further  research.  For  example,  time  was  a very  important  factor  in  the  completion  of  this 
experiment.  If  more  time  can  be  allocated  between  survey  instruments,  then  the  accuracy  of  the 
comparisons  would  improve.  Also,  the  time  restriction  in  this  study  brought  several  constraints  to 
the  subjects  of  the  experiment  who  actually  used  technology.  In  addition  to  timing,  another 
consideration  for  further  research  was  related  to  the  actual  subjects.  For  instance,  in  this  study, 
many  of  the  student  subjects  were  majoring  in  academic  curriculums  that  focused  on  technology, 
such  as  information  systems.  By  expanding  the  subjects  by  area  of  expertise  and  experience,  the 
external  validity  of  this  research  could  proxy  for  a wider  distribution  of  student  academic 
curriculums. 
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Abstract 


We  present  an  HTTP-based  discussion  tool  that  allows  in-context  asynchronous  interaction.  Our 
prototypal  system  can  be  integrated  in  any  HTTP-based  e-Ieaming  platform.  It  allows  to  annotate  web 
pages,  and  to  share  the  notes  with  the  teacher,  with  (a  selected  group)  of  other  students,  or  to  keep  them 
as  personal  notes. 


Introduction 

E-leaming  platforms  aim  non  only  at  providing  content  to  the  student:  in  order  to  be  effective  they 
must  support  the  learning  process.  For  this  reason  virtually  every  ^learning  suite  provides  at  least 
some  tool  for  asynchronous  collaboration,  like  bulletin  boards  etc.  Very  often  these  tool  offer  a poor 
degree  of  integration  with  the  content  of  the  e-courses.  In  many  cases  discussion  among  the  “actors” 
(students  and  teachers)  is  carried  on  a forum:  the  forum  though  is  a different  tool,  so  it  is  separated 
from  the  presentation  material.  While  sometimes  this  is  perfectly  acceptable,  in  many  cases  it  is  rather 
inconvenient  since  the  discussion  happens  out  of  context,  and  many  external  references  are  necessary, 
making  the  whole  process  rather  cumbersome. 

Our  system  allows  to  discuss  in  a context:  the  (e.g.)  lecture  content  can  be  annotated  by  (e.g.)  students 
for  their  exclusive  use,  or  they  can  share  the  notes  with  their  classmates  or  with  the  teacher.  Readers 
can  respond  to  the  notes  in  place,  for  instance  clarifying  the  original  content  of  the  page.  The  system 
allows  to  annotate  web  pages,  since  most  platforms  today  use  HTML  as  a lingua  franca  for  content 
delivery. 

WebNotes  use 

In  order  to  use  WebNotes,  existing  HTML  pages  need  to  be  preprocessed  to  conform  to  the  standard 
expected  by  it.  An  automatic  tool  allows  to  perform  the  preprocessing  of  the  page.  During  the 
conversion  process  the  granularity  of  the  maikable  context  that  can  be  annotated  is  chosen:  it  can  be 
each  word,  each  phrase  or  each  paragraph.  The  process  is  necessary  in  order  to  guarantee  that  the 
resulting  pages  are  XHTML  compliant,  and  that  the  contexts  are  suitably  defined.  The  resulting  page  is 
still  normal  HTML,  although  it  conforms  the  more  restrictive  rules  that  make  it  XML  compliant. 

The  pages  requested  by  the  user  are  filtered  through  a servlet  enabled  Web  server,  that  maintains  a set 
of  notes  and  their  properties  (like  ownership,  visibility  etc).  Notes  are  shown  in  the  text  as  tiny  icons 
embedded  in  the  page.  Public  notes  are  visible  to  everybody.  After  the  user  identifies  him/herself 
through  a login  process,  s/he  can  see  all  notes  allowed  by  her/his  privileges. 

The  user  can  toggle  between  two  modes:  browsing  and  editing.  While  browsing,  when  s/he  moves  the 
mouse  pointer  over  an  icon  that  shows  the  presence  of  a note,  the  note  pops  up  on  place.  When  moving 
the  mouse  away  the  note  disappears,  unless  it  was  stuck  by  the  reader,  in  which  case  it  disappears  only 
when  explicitly  closed. 

In  order  to  edit  the  page  (adding  or  modifying  a note)  the  user  enables  editing  mode  and  simply  clicks 
on  the  location  where  s/he  wants  to  add  the  note.  A popup  window  appears,  and  the  user  can 
edit/modify  the  note.  In  the  basic  version  the  note  contains  only  text,  without  any  formatting.  In  an 
enhanced  version  of  WebNotes,  the  window  contains  an  applet  that  allows  writing  HTML  in  a simple 
“What-You-See-Is-What-You-Get”  fashion.  We  kept  the  applet  separated  in  order  to  minimize  the 
requirements  we  make  on  the  browser,  and  in  order  to  avoid  delays  related  to  its  downloading. 

The  system  is  currently  in  the  final  stages  of  development,  and  already  underwent  some  extensive 
testing  with  different  browsers. 

Although  WebNotes  can  be  used  in  various  contexts,  it  was  explicitly  designed  for  deployment  in  an  e- 
leaming  environment.  We  envision  its  use  for  performing  some  of  the  discussions  that  presently 
happen  on  electronic  forums.  In  particular,  we  target  at  those  (many)  cases  where  the  emphasis  is  on 
clarifying  a point  in  a lecture.  For  instance,  we  had  many  cases  of  students  discussing  programming 


code  presented  in  an  electronic  lecture:  in  those  circumstances,  continuous  references  to  “line  X of 
code  Y on  page  Z”  make  the  writing  and  reading  of  the  notes  an  exhausting  task.  Our  system  allows 
simplifying  these  discussions,  and  by  diminishing  the  needed  effort  to  start  or  join  to  a discussion 
thread  it  will  increase  the  student  participation  rate.  The  increased  interaction  should  give  a better 
learning  process  that  is  the  final  goal  of  every  e-learning  environment. 

At  the  moment  of  writing,  we  plan  to  use  its  preliminary  version  on  the  field  in  some  e-courses  at  the 
University  of  Trento  during  the  last  bimester.  By  the  time  of  the  conference  we  expect  to  have  gathered 
statistics  on  its  use  and  feedback  by  the  users. 

WebNotes  requirements 

WebNotes  is  made  by  a series  of  Java  Servlets,  and  can  therefore  be  deployed  on  any  Servlet-enhanced 
Web  Server  (nowadays  virtually  eveiy  serious  Web  Server  can  has  a servlet-compliant  engine).  We  do 
not  assume  any  special  requirement  on  the  client  side,  the  only  request  being  the  support  of  (the 
standard  subset  of)  JavaScript  (that  is  available  on  all  recent  browsers).  We  tested  our  annotated  pages 
with  all  the  major  players  (i.e.  Opera,  Mozilla,  Netscape,  Internet  Explorer)  on  several  platforms 
without  encountering  problems.  The  notes  are  simple  text,  so  their  writing  and  the  editing  is  performed 
in  a normal  web  page.  Also  the  administration  of  the  system  (e.g.  group  creation  and  maintenance) 
occurs  in  a web  page.  An  enriched  version  of  our  tool  also  includes  a Java  Applet  that  is  a HTML 
composer:  this  allows  formatting  the  notes  in  a reach  way,  using  the  full  power  of  HTML.  Since  the 
Applet  adds  functionality  but  is  not  needed  to  make  the  system  usable,  WebNotes  can  be  configured 
according  to  the  preference  of  the  deployer.  Thanks  to  these  choices  WebNotes  can  be  integrated  in 
most  e -learning  platforms,  enriching  them  with  a new  functionality. 


Related  Work 

Although  much  emphasis  has  been  given  to  the  support  of  collaboration  via  web  since  the  early  WWW 
years  [COL],  a convincing  technology  has  not  yet  emerged.  Interesting  approaches  run  from  an  almost 
uncontrolled  interactive  editing  of  web  pages  (wiki’s  [CUN])  to  more  controlled  annotations.  In 
particular,  adding  notes  to  a Web  page  is  not  a new  idea  per  se.  Tools  comparable  to  what  we  present 
have  been  proposed  in  the  past,  but  in  most  cases  their  architecture  was  dissimilar  from  the  one  we’ve 
chosen.  The  main  architectural  choices  were  either  based  on  browser-specific  tools,  or  on  external 
mediator  servers.  A comparison  of  some  available  tools  can  be  found  in  [GAR]. 

Brower  specific  tools  (e.g.  [COM])  require  the  use  of  a customized  browser  which  is  certainly  an 
inconvenient  choice. 

In  the  proxy  -based  approach,  the  annotation  process  is  taken  care  of  by  an  intermediate  server,  that 
collects  the  notes,  gets  the  original  page  and  present  to  the  user  a page  decorated  it  with  the 
annotations.  This  allows  taking  control  of  the  annotation  process,  and  in  some  cases  is  offered  by 
commercial  enterprises  [ANN]  [THI].  A non-commercial  instance  of  this  class  of  tools  is  CritLink 
[CRI]. 

Our  tool  is  rather  based  on  an  automatic  pre-processing  of  an  HTML  page  to  include  the  fields  that 
(with  the  chosen  granularity)  define  the  markable  locations,  and  rely  then  on  a basic  browser  and  on  a 
rather  standard  web-server  architecture  to  deliver  the  content.  This  promises  to  be  a winning  point, 
since  it  allows  an  easier  integration  with  existing  e- learning  platforms. 

Bibliography 

[ANN]  http://www.annotate.net/  Annotate 

[COL]  “Collaboration,  Knowledge  Representation  and  Automatability” 

http://www.w3.org/Collaboration/ 

[CUN]  Cunnigham,  W.:  wiki  (wiki  wiki  web)  http://c2.com/cgi/wiki7WikiHistorv 
[CRI]  CritLink  http://crit.orp^ttp://crit.org/index.html 

[COM]  ComMentor  http://hci.stanford.edu/commentor/.  by  Martin  Roscheisen,  Christian  Mogensen, 

and  Terry  Winograd  at  the  Stanford  Digital  Library  Project 
[GAR]  Garfunkel  J.,  “Web  Annotation  Technologies**, 

http://look.boston.ma.us/garf/webdev/annote/software.html 
[THI]  ThirdVoice  http://www.thirdvoice.com/  Following  the  link,  one  discovers  that:  “As  of  April  2, 
2001,  we  are  no  longer  supporting  the  consumer  version  of  the  our  search  and  annotation 
tools.’’ 


The  Good,  The  Bad,  & The  Ugly:  lessons  learned  from  electronic  portfolio 

implementation 

At  the  AACE  Conferences  during  2001,  we  were  inspired  by  the  work  being  done  with 
electronic  portfolios.  Returning  to  campus,  we  knew  that  this  was  something  we  wanted 
to  do.  It  fit  so  nicely  with  what  we  had  heard  about  the  new  NCATE  guidelines.  So, 
during  the  past  year.  West  Liberty  State  College,  a small  4-year  state  institution  in 
northern  West  Virginia,  undertook  the  challenge  to  move  to  performance-based 
assessment  and  electronic  portfolios  within  the  Professional  Education  Department.  As  a 
result,  over  900  of  the  institutions  2500  students  are  now  creating  electronic  portfolios  for 
student  and  program  assessments. 

This  decision  was  not  made  quickly  or  lightly  and  reflects  our  commitment  to  remain 
aligned  with  the  new  NCATE  guidelines  and  the  NETStandards.  Through  the 
development  and  implementation  of  our  plans,  we  learned  many  lessons  that  should  be 
valuable  to  any  other  institution  considering  a similar  program.  Our  faculty  required 
education  on  portfolios  and  performance-based  assessment  before  we  could  begin 
discussing  portfolios,  electronic  portfolios,  or  artifacts  and  structures.  This  lead  to  an 
enlightenment  that  perhaps  some  very  tried,  if  not  true,  teaching  methods  might  need  to 
change.  We  conducted  workshops  and  in-services  in  addition  to  one-to-one  mentoring 
sessions.  In  addition  to  motivation  for  the  common  good  of  our  students,  we  are 
approaching  an  NCATE  review  next  year. 

Once  a commitment  to  portfolios  was  formalized,  albeit  some  reluctance,  a committee 
was  formed  to  make  a proposal.  The  format  was  determined  to  be  electronic  with  a 
meshing  of  our  NCATE  Conceptual  Framework  and  the  INTASC  Standards.  Students 
would  be  required  to  select  two  artifacts  to  support  each  standard  and  provide  a written 
reflection  offering  (1)  a description  of  the  artifact,  (2)  a statement  relating  how  the 
artifact  demonstrates  work  related  to  that  standard,  and  (3)  a personal  reflection  of  future 
growth  goals  in  this  area.  Additionally,  each  portfolio  would  include  a personal 
statement/resume,  a philosophy  of  education,  and  any  additional  information  the  student 
wished  to  include.  Finally,  it  was  decided  that  the  portfolio  would  be  conceptually 
introduced  in  the  first  education  course.  Introduction  to  Education  and  the  technological 
mechanics  would  be  addressed  in  the  first  technology  course. 

The  second  hurdle  was  the  decision  of  how  to  “use”  the  portfolios.  The  department 
agreed  that  this  should  be  an  assessment  portfolio  demonstrating  breadth,  depth,  and 
growth  on  the  part  of  student.  A comprehensive  rubric  was  developed  and  a decision 
made  to  assess  the  student  reflections  rather  than  the  specific  content  of  each  artifact.  As 
a result,  we  back  peddled  and  decided  that  each  and  every  EDUC,  READ  or  SPED 
course  would  include  multiple  assignments  that  would  be  designated  as  “potential” 
portfolio  items.  Each  assignment  should  be  performance-based  and  assessed  within  the 
course  where  it  was  assigned.  The  instructor  of  each  course  would  be  required  to  assess 
the  assignment  and  give  a course  grade.  From  this  collection  of  “potential”  portfolio 
assignments,  the  student  then  selected  the  items  for  reflection  and  inclusion  within  the 
portfolio.  The  initial  evaluation  of  the  portfolio  would  be  at  the  time  of  formal 


application  to  the  Education  Program.  This  usually  occurs  after  approximately  60  credit 
hours.  A committee  of  two  faculty  members  would  review  each  portfolio  in  conjunction 
with  other  admittance  criteria. 

The  second  assessment  would  occur  midway  through  the  professional/student  teaching 
semester.  At  this  time,  the  portfolio  would  be  assessed  to  determine  how  the  student 
demonstrated  mastery  of  each  of  the  standards.  Again,  a rubric  was  developed  to  guide  a 
committee  of  three  faculty  members.  Scores  for  each  standard  were  noted  and  used  to 
determine  weaknesses  in  our  program  and  courses.  Students  received  an  unacceptable, 
developing,  acceptable,  or  exemplary  rating.  Feedback  from  this  assessment  is  used  for 
program  modifications  and  department  decisions. 

One  of  the  most  challenging  decisions  was  to  use  an  electronic  format.  Several  faculty 
were  not  comfortable  with  this  idea.  Unfortunately,  these  people  are  not  comfortable 
with  much  of  the  technology  available  today  and  reflect  many  of  the  skills  still  found  in 
our  local  schools.  For  this  purpose,  it  became  necessary  to  select  a platform  that  was 
advanced  enough  to  provide  a comprehensive  picture  of  the  candidate’s  knowledge,  skill 
and  attitude  yet  still  be  consistent  with  what  might  be  useful  for  future  purposes  (i.e. 
interviewing).  We  chose  to  build  web-based  portfolios  with  electronic  documents, 
scanned  documents  and  images,  digital  photos,  video  clips,  and  audio  clips.  Students 
were  encouraged  to  link  PowerPoint  presentations,  Web  Quests  and  other  projects  as 
artifacts. 

We  have  worked  on  this  for  one  year  and  now  know  what  we  did  right  and  what  we  did 
wrong.  Our  first  set  of  portfolios  has  helped  us  to  see  what  we  should  have  considered. 
The  largest  problems  we  are  facing  are  reluctant  students  and  overworked  faculty 
members.  The  students  don’t  see  the  value/need  for  these.  They  are  very  comfortable 
with  the  tried  and  true  lecture  to  me  and  then  give  me  two  tests  model.  Many  are  kicking 
and  screaming  about  a shift  to  performance-based  assessment.  Yes,  it  is  more  work  for 
them;  and  for  us,  the  faculty,  too.  The  faculty  found  more  work  than  expected  in 
designing,  developing  and  grading  these  portfolio  assignments.  Faculty  had  the 
additional  burden  of  assessing  the  portfolios.  Let’s  see...  90  student  teachers  with  3 
evaluators  each  means  270  assessments  divided  among  12  faculty  members  who  don’t  all 
do  share.  Sound  familiar?  This  was  the  ugly  part! 

The  actual  portfolio  results  have  been  both  good  and  bad.  Some  are  disappointing. 

Others  made  it  all  worthwhile.  The  elementary  students  have  more  motivation  but 
sometimes  lacked  the  depth  of  reflection  or  technical  skills.  Some  of  the  secondary 
students  really  showed  their  expertise  and  have  portfolios  on  CDs  that  will  certainly  open 
many  job  doors.  Some  others,  often  by  major,  are  less  satisfactory.  For  us,  we  will 
continue  and  hope  that  we  can  grow  from  our  lessons  learned.  Perhaps,  our  lessons 
learned  will  help  you  too. 


Abstract 


Teacher  roles  in  synchronous  chat  classes 


Synchronous  classes  are  mushrooming  in  post-graduate  education  and  new  roles  have  been 
heralded  as  necessary  for  effective  class  instruction.  This  case  study  looks  into  transcripts  of  chat 
classes  from  Newport  Asia  Pacific  University  Master  Program  and  analyses  its  discourse  analysis 
to  see  how  true  is  this  assertion  of  paradigmatic  role  change.  A coding  scheme  was  developed  for 
this  purpose  and  data  were  then  compared  against  the  result  of  online  interviews  with  tutors  and 
students.  Results  point  to  caution  in  assigning  dramatic  changes  in  online  instruction  because  of 
new  technology. 


This  work  will  be  a case  study  of  online  classes  belonging  to  Newport  Asia  Pacific  University  and  it 
will  look  specifically  into  roles  played  by  tutors  in  the  classrooms,  whether  they  are  radically 
different  from  face  to  face  classrooms  and  what  patterns  of  class  organization  may  appear  as 
distinctive. 

Newport  Asia  Pacific  University  (NAPU)  fwww.napu.edu')  is  an  Internet  university  dedicated  to 
providing  post  graduate  courses  in  TESOL  (Teaching  English  to  Speakers  of  Other  Languages) 
through  its  Master  of  Science  program.  Students  from  around  the  world  participate  in  real-time 
online  classes  with  world  renowned  professors  on  a weekly  basis.  All  subject  content  is  provided 
via  a Web  site  and  all  communication,  including  assignment  submission,  takes  place  via  the 
Internet.  Faculty  and  tutors  are  Dr.  David  Nunan,  Dr.  Rod  Ellis  and  Dr.  Ruth  Wynryb.  These 
professionals  moderate  classes  and  discussion  lists  and  hold  their  classes  through  chat  rooms  in  a 
synchronous  mode. 

in  order  to  analyze  these  classes  a coding  scheme  was  developed  for  discourse  analysis  and 
qualitative  interviews  with  both  tutors  and  students  of  the  courses  delivered  were  implemented. 

The  criteria  used  to  select  sample  lessons  were: 

a.  classes  with  a beginning/ending  difference  with  2/3  students  only. 

b.  classes  with  one  student  only. 

c.  classes  with  4+  students 

d.  classes  with  same  tutor,  same  content  and  number  of  students,  from  beginning  stages  of  the 
program  and  in  the  last  few  months. 

The  coding  scheme  used 

The  Online  Instruction/Moderator/Facilitator's  roles  have  been  categorized  by  Berge  (1995)  into  the 
following  four  categories:  pedagogical,  social,  managerial,  and  technical. 


Following  coding  into  Berge’s  four  role  categories,  I developed  a coding  scheme  to  capture  the 
essence  of  the  instructor’s  actions  within  each  category  focusing  on  abstracting  what  the  instructor 
actually  did  in  each  posting  and  categorizing  these  pedagogic  acts.  This  coding  scheme  was  drawn 
following  Speech  Act  Theory  which  identifies  both  locutory  (literal  meaning  of  words)  and 
illocutory  (motivating  intention  of  utterance)  acts  as  “markers”  encoded  in  the  language  of  the 
postings.  (Searle,  1976) 
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Berge’s  categories  were  further  categorized  as: 

A.  Pedagogical:  Encompasses  all  attempts  to  reach  a particular  learning  objective  relevant  to  the 
course.  Includes: 

1 . Feedback 

2.  Setting  instructions 

3.  Giving  information 

4.  Advice/suggestions/opinions 

5.  Framing  questions 

6.  Summarizing  comments 

7.  Referring  to  other  sources 

B.  Social:  Encompasses  attempts  to  make  students  comfortable  and/or  promote  feelings  of 
participation. 

1 . Empathy 

2.  Personal  commimication 

3.  Meta-communication 

4.  Humour 

C.  Managerial:  Encompasses  attempts  to  coordinate  course  activities  and  maintain  a sense  of 
coherence  and  structure  in  the  course.  Instructor  acts  include: 

1 . Coordinating  assignments 

2.  Coordinating  Discussion 

3.  Coordinating  course  and  program  information 

D.  Technical:  Includes  assistance  to  students  in  mastering  and  troubleshooting  course  delivery 
software. 


1.  User 

2.  System  issues 

3 . Problem  not  easily  detected 

Twenty- five  classes  by  the  three  different  tutors  of  the  program  were  analyzed  in  all.  The  analysis 
of  raw  discourse  used  the  participant-defined  paragraph  since  it  provided  a practical  answer  to  the 
need  to  get  a code  that  reflected  both  the  essence  and  context  of  tutor  contributions  to  the  online 
class. 

Conclusion 

What  the  data  clearly  show  is  that  the  majority  of  interventions  have  been  the  tutor's  while  students’ 
interventions  have  been  secondary.  The  tutor's  most  important  role  has  been  management  of  the 
online  class  while  pedagogic  roles  ran  second. 

The  analysis  of  these  classes  also  point  out  to  highly  structured  classes,  where  agendas  are  set  out  at 
start.  In  general  no  classes  present  evidence  of  cooperative  work  among  students,  reflection  among 
students  and  group  production,  or  class  objectives  suggested  by  student  input  or  questions.  Role 
shifting  (the  teacher- as- learner,  the  learner- as-teacher)  did  not  show  up  in  any  of  the  samples.  They 


typically  show  a teacher- centered  classroom  delivery  where  the  “sage  teacher  as  expert”  role  is 
predominant.  On  the  contrary,  the  I-R-F  (Initiation-Response-Feedback)  scheme  of  discourse 
characteristic  of  teacher-led  classrooms  is  recurrent  in  most  classes.  (Sinclair  & Coulthard  , 1975; 
Sinclair  & Brazil,  1982) 

Whatever  the  reason  why  chat  lessons  are  so  characterized,  what  the  y certainly  point  to  is  the  need 
to  provide  a pedagogic  scenario  that  can  complement,  supplement,  and  also  offer  the  possibility  to 
carry  out  all  the  cooperative  work  and  social  network  necessary  to  make  learning  a wholesome 
experience.  In  the  case  of  NAPU,  this  is  already  set  up  by  an  ongoing  discussion  list  and  a face-to- 
face  residential.  These  two  elements  provide  the  rest  of  the  architecture  that  supports  the  feeling  of 
online  presence  acknowledged  by  all  the  actors. 
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Online  Mastery 

A Certification  Curriculum  for  Faculty 
ABSTRACT 

Faculty  certification  increases  the  chances  that  online  instruction  in  higher  education  will 
successfiilly  meet  the  needs  of  students.  This  paper  introduces  a new  approach  to 
designing  a program  of  faculty  certification  for  online  instruction.  The  paper  also 
proposes  that  the  national  and  state  departments  of  education  as  well  as  accreditation 
institutions  establish  certification  standards  that  reflect  the  quality  of  preparation  defined 
in  the  proposed  program  of  certification  or  a comparable  program. 

Introduction 

Status  of  the  Work:  Beginning 

Partners  Involved:  The  Academy  for  Teaching  Excellence,  the  School  of  Letters,  Arts, 
and  Sciences,  and  the  Division  of  Information  Technology  at  Metropolitan  State  College 
of  Denver. 

Major  Goals:  The  project  described  in  this  paper/presentation  has  two  goals.  First,  the 
system  goal  involves  the  introduction  of  a competency  model  of  certified  training  of 
faculty  diat  will  demonstrate  how  faculty  should  be  educated  to  develop  and  instruct 
online  courses.  Second,  the  outcomes  goal  involves  training  faculty  for  certification  of 
quality  online  instruction. 

Basic  Approach:  With  the  participation  of  the  Director  of  the  Academy  for  Teaching 
Excellence,  the  Director  of  Training  for  the  Division  of  Information  Technology,  and  key 
faculty  and  staff  from  the  School  of  Letters,  Arts,  and  Sciences,  develop  and  validate  a 
sequenced  and  articulate  curriculum  of  mastery  certification  in  online  instruction. 

The  Educational  Problem:  What  sort  of  faculty  preparation  does  it  take  to  equip  an 
institution  of  higher  education  for  success  in  online  education?  The  authors  believe  that 
certification  of  faculty  is  essential  yet  very  rare  in  the  traditional  colleges  and 
universities.  Few  offer  a sequenced,  articulated  program  that  leads  to  officially 
sanctioned  certification  for  all  faculty  members  who  would  teach  online  (POD  Network, 
2001).  Barring  such  certification,  online  instruction  may  be  devoid  of  any  institutional 
standards  of  quality  (Willis,  1994;  Ridley  & Husband,  1998;  and  Palloff 
& Pratt,  1999).  The  lack  of  standards  rapidly  translates  into  highly  uneven  quality  as  seen 
by  students  and  hence  into  lost  enrollment,  particularly  given  the  competition  for  student 
tuition  dollars  that  now  prevails  in  the  domain  of  post-secondary  online  education  (Berge, 
1997  and  Dominquez  & Ridley,  1999).  Therefore,  for  the  sake  of  student  learning  and 
institutional  success,  the  authors  propose  a new  approach  for  preparing  faculty  for  the 
online  educational  enterprise.  Institutions  should  require  certification  of  faculty  for 
online  educational  instruction. 

Background:  The  increase  in  online  instruction  in  the  past  five  years  and  its  projected 
acceleration  in  the  next  twenty  leave  no  doubt  about  the  need  for  faculty  professional 
development  in  online  instruction  both  in  terms  of  technology  and  pedagogical  ' 

competencies  (Ridley,  1996). 

In  the  U.S.  very  few  faculty  development  programs  exist  that  train  faculty  in  higher 
education  to  enhance  their  online  instructional  efforts  (Schrum,  1995).  Those  that  do 
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exist  consist  of  fragmented  workshop  programs  or  one-shot  conferences  on  technology 
(Williams  & Peters,  1997). 

Description:  A specific  concept  of  a mastery  certification  curriculum  has  been  developed. 
This  curriculum  includes  preparation  in  the  core  competencies  as  well  as  in  enabling  and 
sustaining  competencies.  See  below  for  a full  description  of  the  concept. 

Validation:  Two  actions  will  assist  the  researchers  in  validating  the  certification  model 
for  this  faculty  online  development  training  prior  to  the  date  of  the  Ed-Media  Conference. 
First,  the  researchers  will  validate  the  concept  of  faculty  certification  training  for  online 
instruction  in  a conference  of  peers  entitled  “Teaching  Online  in  Higher  Education 
Online  Conference”  on  November  12,  2001  in  which  the  model  for  faculty  development 
training  will  be  discussed  and  assessed  by  an  online  set  of  peer  reviewers  at  the 
conference.  Second,  a survey  of  the  top  ten  institutions  with  the  largest  enrollments 
where  the  majority  of  the  program  majors  and  minors  are  taught  online  will  be  instituted 
on  current  faculty  development  programs  and  the  data  reported  in  the  presentation  at  the 
Ed-Media  Conference.  In  addition  to  these  two  actions,  a pilot  test  of  the  model  will  be 
initiated  during  the  summer  months  at  Metropolitan  State  College  of  Denver  through  a 
U.S.  Department  of  Education,  Title  III,  Activity  Two  grant,  the  funding  of  which  the 
college  received  for  the  professional  development  of  its  faculty  at  the  institution. 

Future  Work  and  Implications  for  Others:  The  paper/presentation  proposes  that  national 
and  state  departments  of  education  as  well  as  accreditation  institutions  establish 
certification  standards  that  reflect  the  quality  of  preparation  defined  in  the  proposed 
program  of  certification  or  a comparable  program. 

Core  Competencies 

A program  of  certification  in  online  education  operates  on  the  basis  of  six  functions  that 
Duchastel  (1997)  describes  as:  1)  specifying  the  goals  to  be  pursued;  2)  accepting 
diversity  of  outcomes;  3)  requesting  production  of  knowledge;  4)  evaluating  at  the  task 
level;  5)  building  learning  teams;  and  6)  encouraging  global  communities.  These 
functions  influence  the  development  of  five  core  competencies  learned  by  online 
instructors: 


• Selecting: 

• Preparing: 

• Delivering: 

• Managing: 

• Assessing: 


Selecting  materials,  activities  and  courses  that  are  suited  for 
online  delivery 

Preparing  content  for  online  delivery 
Delivering  instruction  online 
Managing  the  online  course 

Assessing  student  learning  in  the  online  environment,  providing 
praise,  encouragement,  and  corrective  feedback  as  needed. 


Enabling  Competencies 

Many  faculty  members  who  are  excellent  instructors  and  who  wish  to  teach  online  do  not 
have  the  computer  skills  that  constitute  the  essential  threshold  to  online  education.  For 
these  educators,  the  certification  curriculum  should  offer  instruction  in  fundamental 
enabling  competencies  including: 


Selecting  a computer  system  and  or  essential  components 

Setting  up  a computer  system  and  its  components 

Creating  a healthy  workspace 

Acquiring  ergonomically  sound  work  habits 

Using  input  devices 

Using  the  operating  system 
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Developing  word  processing  and  spread  sheet  competencies 
Using  voice  input 

Selecting  and  using  ADA  input  and  output  options 

Operating  the  electronic  communication  system  in  the  college/university 


Sustaining  Competencies 

To  ensure  that  a program  of  online  education  does  not  fall  victim  to  the  entropy  of 
complacency,  the  certification  curriculum  must  include  instruction  in  the  competencies 
that  are  essential  for  creating  and  perpetuating  excellence.  These  include; 


• Evaluating:  Evaluating  the  effectiveness  of  the  online  course 

• Revising:  Altering  the  online  course  when  and  where  appropriate 

A Closer  Examination  of  Core  Competencies 

The  five  core  competencies  constitute  the  heart  of  the  curriculum  because  they  are  what 
faculty  do: 

• From  a vast  body  of  domain-specific  knowledge,  faculty  select  the  concepts, 
information,  and  skills  to  include  in  their  courses. 

• They  prepare  this  material  for  instruction. 

• They  deliver  it  in  such  a way  that  learners  can  make  it  their  own. 

• They  manage  the  learning  experience,  taking  into  account  the  mix  of  personality 
and  ability  among  their  students  as  well  as  the  technology  and  logistics  of  the 
instructional  environment. 

• Throughout  the  course,  they  assess  the  learning  of  their  students,  providing  praise, 
encouragement,  and  corrective  feedback  as  needed. 

Competency  in  these  five  steps  is  essential  for  success  in  moving  from  purely  on-site 
instruction  to  instruction  that  is  either  partially  or  totally  online.  The  following  sections 
expand  on  the  concept  of  each  core  competency. 


1.  Selecting  What  to  Put  Online 

Faculty  will  inventory  the  courses  they  teach  and  identify  materials,  activities  and 
possibly  entire  courses  that  could  work  well  online.  This  component  of  the  curriculum 
will  provide  faculty  with  specific  guidelines  for  accomplishing  this  inventory.  Faculty 
will  be  encouraged  to  work  closely  with  their  department  chairs  for  validation  of  their 
selection  of  materials  for  online  delivery. 

2.  Preparing  the  Learning  Experience 

With  specific  instructional  content  in  mind,  faculty  will  explore  the  options  for  preparing 
instruction  for  online  delivery.  They  will  learn  pedagogical  principles  and  techniques  for 
making  the  materials  they  put  online  more  effective  for  their  students.  They  will  learn  to 
make  accurate  estimates  of  how  long  it  will  take  to  implement  their  plans  for  preparing 
their  materials  for  online  delivery.  They  will  learn  how  to  use  the  tools  of  technology  to 
actually  prepare  course  materials  for  online  delivery.  They  also  will  learn  how  to  work 
with  specialists  in  multimedia  development  to  produce  advanced  media  components  or 
even  entire  online  courses. 

3.  Delivering  the  Learning  Experience 

Studies  of  what  makes  for  successful  classroom  teaching  indicate  that  personal 
dynamism,  energy,  and  the  ability  to  make  contact  with  learners  are  major  components  of 
success.  Dynamism,  energy,  and  ability  to  make  contact  are  also  major  factors  in  the 
success  of  online  delivery,  but  the  effective  application  of  these  traits  requires  a set  of 
very  different  skills  compared  to  classroom  instruction.  Workshops  in  this  section  of  the 
curriculum  will  present  methods  of  online  instruction  and  help  faculty  identify  those 
methods  that  are  most  appropriate  for  their  teaching  style  and  various  types  of  course 
content. 

4.  Managing  the  Online  Class 

One  of  the  most  frequent  comments  from  faculty  new  to  online  teaching  is  that  they  find 
it  extremely  time  consuming.  Can  one  avoid  being  overwhelmed  and  still  provide  a 
quality  learning  experience  for  students?  What  about  decorum  in  the  online 
environment?  How  does  the  online  teacher  keep  the  tone  of  the  entire  experience  on  high 
plane?  And  what  does  one  do  if  students  start  “flaming”  one  another  or  the  instructor? 
This  component  of  the  curriculum  will  present  methods  of  managing  the  online  class  and 
suggest  techniques  for  success. 

3.  Assessing  Student  Performance 

The  online  learning  environment  puts  in  question  many  of  the  standard  practices  of  on- 
site assessment  of  student  performance.  What  kinds  of  tests  can  one  offer  online? 

Is  there  any  way  to  guard  against  cheating?  Are  proctored  tests  a good  solution?  Can 
one  assess  students  in  other  ways  than  just  testing?  Can  one  require  graded  group 
activities  in  an  online  course?  Classes  in  this  section  of  the  workshop  curriculum  will 
help  answer  these  questions  by  presenting  options  for  assessing  student  performance  in 
online  courses  with  an  emphasis  on  alternatives  to  the  proctored  written  exam. 


1711 


A Closer  Look  at  Enabling  Competencies 


For  faculty  members,  equipping  for  the  online  endeavor  means  expanding  their  store  of 
knowledge  and  skill  to  include  use  and  ultimately  mastery  of  the  essential  tools  of 
technology.  It  means  acquiring  and  installing  the  essential  computer  system  components. 
It  means  learning  how  to  use  those  components.  Finally,  it  means  creating  an 
ergonomically  appropriate  workspace  and  adopting  healthy  computer  work  practices. 

All  these  factors  become  particularly  important  when  the  faculty  member  plans  to  handle 
online  courses  from  home. 

Most  programs  of  faculty  preparation  overlook  these  foundational  considerations.  The 
negative  results  of  such  omission  affect  the  full  spectrum  of  faculty,  from  computer 
novices  to  long-time  veterans  of  computing.  Some  novices  drop  out  because  they  never 
grasp  the  basics  essential  for  being  at  ease  in  using  computers.  Others  survive  but 
fossilize  in  inefficient  uses  of  their  computer  systems.  Finally,  many  expert  users  find 
themselves  out  of  action  or  hindered  in  their  work  due  to  unhealthy  computing  practices 
or  poor  ergonomics. 

A Closer  Look  at  Sustaining  Competencies 

The  two  sustaining  competencies— evaluate  and  revise— are  essential  to  ensure  that  flaws 
and  weaknesses  are  identified  and  eliminated  as  soon  as  possible.  This  is  a particularly 
problematic  area  of  endeavor.  Some  extensive  online  education  enterprises  that  have 
been  in  operation  for  a number  of  years  still  have  not  determined  how  to  realize  peer 
evaluation  of  online  instruction.  As  to  student  evaluation  of  online  instruction,  it  may  be 
simple  to  implement  but  quite  difficult  to  validate.  Students  come  to  the  online  learning 
experience  with  widely  varying  expectations  and  levels  of  competency  in  the  computer 
essentials.  Under  such  conditions,  students  may  be  incapable  of  differentiating  between 
peripheral  frustrations  and  the  quality  of  instruction  as  they  evaluate  a specific  online 
course  and  instructor.  Finally,  where  institutional  evaluation  exists,  it  may  be  purely 
summative  and  generalized,  serving  as  a poor  tool  for  identifying  what  specific  changes 
should  be  made  to  improve  instruction  in  specific  course  offerings.  Thus,  instruction  in 
the  sustaining  competencies  will  provide  instructors  the  tools  for  formative  evaluation 
and  revision  of  their  own  courses  as  they  are  teaching  them. 

Summary 

This  curriculum  will  offer  learning  experiences  that  lead  to  certification  in  each  of  the 
core,  enabling,  and  sustaining  competencies.  These  learning  experiences  may  be  of 
various  sorts,  e.g.  peer  tutoring/coaching,  team  teaching  with  certified  faculty,  computer 
or  Web-based  instruction,  workshops,  classes,  and  seminars.  Faculty  who  accumulate 
certification  or  validation  in  all  competencies  will  earn  “Online  Master”  certification. 
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Abstract.  This  article  presents  the  Helsinki-Lappi  project  (HelLa)  co-run  by  two  Finnish  uni- 
versities: the  University  of  Helsinki,  and  the  University  of  Lapland.  The  general  aim  of  the 
project  is  to  study,  develop,  and  assess  how  different  training  programs  relate  to  educational 
use  of  information  and  communication  technologies  (ICT).  The  project  is  expected  to 
contribute  substantively  to  the  national  Virtual  University  Project  of  the  Faculties  of 
Education  (KasVi).  The  specific  aims  of  the  project  are  to  analyze  (i)  how  the  pedagogical 
models  of  network -based  education  (NBE)  can  be  seen  in  educational  use  of  ICT  programs; 
(ii)  what  kind  of  design,  quality,  and  assessment  principles  can  be  elaborated  in  NBE  (iii)  how 
a didactic  network  environment  and  NBE  support  media  skills  and  the  development  of  the 
individual  and  the  community;  and  (iv)  how  national  train  ing  programs  in  media  education 
and  educational  use  of  ICT  can  be  streamlined  and  made  accessible  to  the  international 
audience.  The  outcomes  of  the  project  will  be  models  and  didactic  principles,  which  can  be 
used  in  design  and  assessing  NBE. 

Keywords:  Network-based  education  (NBE);  teachingstudying-leaming  (TSL)  process; 
learning  and  studying  environments;  pedagogical  models;  designing  and  assessing  principles; 
meaningful  learning;  information  and  communication  technologies  (ICT). 


Introduction 

The  Finnish  virtual  university  project  (KasVi)  aims  at  developing:  1)  teacher  education  curricula  in  technological 
environments,  2)  flexible  training  systems  based  on  distance  and  flexi-mode  teaching,  3)  continuing  teacher 
education,  4)  qualitative  ICT  training  programs,  and  5)  innovative  pedagogy  for  NBE.  The  Universities  of 
Helsinki  and  Lapland  initiated  the  joint  HelLa  project  in  2001  as  part  of  the  national  KasVi  project.  The 
objective  of  the  HelLa  project  is  to  study  the  following  areas  in  2001-2003:  i)  the  pedagogical  models  of  NBE  in 
training  programs  of  media  education  and  educational  use  of  ICT,  ii)  the  further  development  of  the  principles 
and  characteristics  of  designing  and  assessing  NBE,  iii)  the  support  of  NBE  for  the  media  skills  of  the  individual 
and  the  community,  and  iv)  the  standardization  and  internationalization  of  national  ICT  training  programs.  This 
article  examines  the  pedagogical  models  for  NBE  and  the  principles  of  design  and  assessment. 


The  Challenges  of  Assessment 

/ 

Quality  can  be  defined  as  a relation  between  the  object  and  the  aim  of  the  study.  The  assessment  of  quality  is 
always  related  to  the  situation;  it  is  dependent  on  time,  space  and  context.  (Raivola  2000,  19-22.)  In  this  article, 
the  quality  of  NBE  is  primarily  studied  as  the  realization  of  the  objectives  of  education.  The  quality  of  university 
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education  is  always  closely  related  to  research.  Research  creates  knowledge,  and  assessment,  again,  is  deeply 
rooted  in  this  knowledge  obtahed.  Teaching  based  on  research  is  seen  as  quality  teaching.  In  this  respect,  quality 
is  also  understood  as  a corresponding  principle.  (Raivola  2000,  38-51.)  In  this  study,  quality  is  also  defined  as 
fundamental,  goal-oriented,  curriculum -oriented,  meaningful,  and  profiting  from  the  didactic  principles  of  NBE. 

When  assessing  NBE  certain  issues  should  be  raised  to  fore:  the  operational  context  of  NBE, 
communication  and  communalism,  research  into  the  role  of  the  teachers  and  students,  and  modes  of  studying  and 
teaching.  (Ruokamo  & Pohjolainen  1999;  Telia  et  al.  2001.)  According  to  Duffy  and  Jonassen  (1992),  learning 
and  studying  environments  that  have  focused  on  users  and  their  needs  and  a problem- oriented  approach  have 
been  successful.  (Vahtivuori  2001).  It  seems  that  NBE  should  be  assessed  at  all  levels;  a study  restricted  to  the 
suifacc  level  is  insufficient.  Thus,  social  perspectives,  media  culture,  communication  and  mediation  should  be 
taken  into  account.  (Telia  et  al.  2001.) 

Pedagogical  Models  of  Network-Based  Education  and  Learning  Theory 

Based  on  pedagogical  models  of  NBE  and  theoretical  models  of  learning  (e.g.,  Jonassen  1995;  Ruokamo  & 
Pohjolainen  1999;  Telia  et  al.  2001),  the  HelLa  project  will  study  the  didactic  thinking  involved  in  the  teaching- 
study  ing-leaming  (TSL)  process  (Uljens  1997).  We  aim  at  presenting  different  pedagogical  models  that  can  be 
used  to  define  the  principles  for  designing  and  assessing  NBE.  Our  study  focuses  especially  on  those 
pedagogical  models  that  can  be  used  to  realize  cooperative  and  communal  modes  of  learning  and  studying.  In 
this  article,  cooperative  means  that  students  work  together  and  build  their  knowledge  in  cooperation  with  others, 
using  their  knowledge  and  skills  beneficially  (Jonassen  1995;  Ruokamo  & Pohjolainen  1998;  1999;  2000). 
Cooperative  learning  is  based  on  the  socio- constructivist  perspective.  By  communal  learning  and  studying,  we 
refer  to  the  philosophy  of  interaction  and  to  the  individual  way  of  life.  In  communal  studying,  a common  culture 
of  activity  and  the  process  of  work  are  strongly  emphasized  (Vygotsky  1934/1962;  Panitz  1996;  Telia  1998; 
Passi  & Vahtivuori  1998;  Hakkarainen  et  al.  1998). 

Uljens  (1997)  emphasizes  the  teaching-studying-leaming  process  (TSL)  as  central  for  didactics. 
According  to  hhi,  both  teaching  and  learning  are  important  but  equally  important  is  studying.  (Cf.  Kansanen  et 
al.  2000.)  Uljens  (1997)  argues  that  teaching  cannot  directly  influence  learning;  rather,  teaching  [ajfects] 
learning  indirectly  through  the  individual's  study  activity  (Uljens  1997,  39).  For  this  reason,  we  also  need  to 
examine  teaching,  studying,  and  learning  as  equally  important  components.  (Kynaslahti  2000,  25;  Telia  et  al. 
2001.)  We  argue  that  these  concepts  are  the  most  beneficial  way  to  organize  and  assess  NBE. 

The  multi -dimensional  model  of  media  education  is  a conceptual  framework  for  NBE,  in  which  designing 
and  assessing  NBE  are  handled  in  an  integrative  way.  In  the  model,  the  concepts  of  media  education  and  the 
didactic  TSL  process  are  incorporated  to  communication  and  mediation.  This  model  attempts  to  avoid  the 
problems  contained  within  earlier  analytical  criteria  and  the  simplification  of  the  complex  realities  involved  in 
NBE.  (Telia  & Mononen-Aaltonen  2000.)  Another  model  used  is  that  of  different  uses  of  ICT,  partly  based  on 
Goldsworthy’s  (1999)  lenses  of  learning.  It  deals  with  four  categories  that  examine  the  relationships  between 
studying  and  technology.  The  four  categories  of  the  model  are  pedagogic,  instrumental,  communal,  and 
communicative  use  of  ICT.  NBE  profits  most  from  several  or  all  four  different  ways  of  using  ICT  being  linked 
to  the  same  educational  program  or  to  a TSL  situation.  (Vahtivuori  & Masalin  2000;  Vahtivuori  2001) 

Our  study  also  focuses  on  the  shared  and  divided  expertise  (Oatley  1990;  Brown  et  al.  1993),  which  is 
based  on  research  into  the  theory  of  learning,  and  the  reciprocal  teaching  method  (Palincsar  & Brown  1984)  in 
NBE  (see  also  Hakkarainen  et  al.  2000).  We  have  organized  some  experiments  in  this  field  in  media  education 
and  NBE  courses  (Ruokamo  & Syij^ari  2001).  In  addition,  we  shall  examine  problem-based  learning  (Norman 
& Schmidt  1992)  and  meaningful  learning  (Ausubel  1968;  Jonassen  1995).  Experiences  were  promising  for 
increasing  quality  of  NBE.  Research-focused  teaching  has  also  been  used  to  study  interaction  and  dialog  in  NBE. 
One  of  the  key  observations  points  to  the  significant  role  of  the  teacher  in  creating  research-focused  dialog.  The 
meaning  of  social  interaction  in  studying  was  also  emphasized  strongly  alongside  interaction  which  handled 
substance-specific  issues.  (Mononen-Aaltonen  1999;  Lintula  1999).  Observations  of  the  NBE  courses  showed 
that  groupwork  software  supports  communal  modes  of  learning  and  studying,  different  styles  of  studying  and 
students’  developing  skills.  (Cf.  Sharan  & Sharan  1992;  Vahtivuori,  Wager  & Passi  1999.) 

Some  Principles  for  Designing  and  Assessing  NBE 

In  the  following,  a few  principles  for  designing  and  assessing  NBE  will  be  presented,  based  on  the  pedagogic 
models  above.  In  terms  of  the  TSL  process,  our  key  observation  is  that,  as  far  as  design  and  assessment  go,  the 


same  kinds  of  principles  and  characteristics  apply  to  all  the  three  components  of  TSL.  (Ruokamo  & Pohjolainen 
1999;  Telia  et  al.  2001.)  In  designing  learning  and  studying  environments,  NBE  and  educational  use  of  ICT  can 
be  understood  as  broad  trends  in  which  different  tools  and  environments  are  used  in  many  ways  to  support  the 
TSL  process.  Networks  are  not  to  be  understood  simply  as  tools;  rather,  they  can  also  serve  as  intelligent 
partners  and  new  contexts  (Jonassen  1995;  Telia  1998).  The  entire  context  of  NBE  should  be  examined.  The 
materials,  the  users’  activity  and  needs  and  general  didactic  planning  are  crucial  for  the  design  and  assessment  of 
NBE  (see  also  Bonk  & Reynolds  1997).  NBE  also  calls  for  user -oriented  applications  and  customized  models  at 
the  local  level.  In  designing  network-based  environments,  attention  should  be  paid  to  their  functionality  to  enable 
multi-level  teaching  and  guidance.  For  meaningful  and  purposive  learning  and  studying,  we  need,  for  instance,  a 
fully  operational  tutoring  and  mentoring  system  in  addition  to  the  activity  of  the  teacher.  We  will  now  present 
eleven  characteristics  of  meaningful  learning  and  studying  and  describe  the  corresponding  didactic  and  learning 
theoretical  network-based  environment  together  with  the  desired  teacher  orientation. 

1.  Constructive  and  Cumulative.  Students  build  new  knowledge  upon  the  basis  of  their  earlier  knowledge  (de 
Corte  1995;  Jonassen  1995;  Lehtinen  1997;  Ruokamo  & Pohjolainen  1999;  Mannisenmaki  2000;  Nevgi  & Tirri 
2001).  Well -organized,  complicated  skills  for  constructing  and  justifying  information  structures  are  emphasized 
in  a learning  and  studying  environment  (Sinko  & Lehtinen  1998;  Manninen  et  aL  2000).  The  students’  earlier 
knowledge  controls  the  acquisition  and  interpretation  of  information  (Lehtinen  1997).  The  teacher  should 
construct  learning  and  studying  situations  and  contexts  (Telia  et  al.  2001)  in  such  a way  that  a student  can  build 
on  his  or  her  prior  knowledge.  Initial  mapping  can  clarify  the  students’  preparedness  to  receive  information  and 
skills.  The  learning  and  studying  environment  can  be  adapted  to  suit  students’  activities  best  and  to  fulfil  their 
needs  through  the  support  of  the  teacher,  tutor,  and  the  peer  group.  (Manninen  et  al.  1999.) 

2.  Active  and  Self-Directed.  The  roles  of  the  students  and  other  members  of  the  learning  community  are  active. 
The  students  commit  themselves  to  objective-oriented  (Uljens  1997)  and  sensible  processing,  for  which  they  are 
responsible.  (Jonassen  1995;  de  Corte  1995;  Lehtinen  1997;  Ruokamo  & Pohjolainen  1999;  Mannisenmaki 
2000;  Nevgi  & Tirri  2001.)  A learning  and  studying  environment  enables  a new  kind  of  teacher-student 
relationship,  which  emphasizes  communalism  and  the  personal  expertise  of  the  students.  (Sinko  & Lehtinen 
1998;  Manninen  et  al.  2000.)  Students  can  ask  practical,  informative  questions,  acquire  information,  and 
critically  evaluate  that  information.  The  teacher  creates  and  maintains  students’  activities  with  challenging  and 
interesting  settings  for  tasks  shared  with  the  students  themselves.  Students  are  encouraged  to  express  new  ideas 
and  models  of  thinking.  This  way  the  entire  community  can  see  and  profit  from  the  “media  traces”  bom  during 
the  process  of  learning  (Sharan  & Sharan  1992;  Vahtivuori,  Wager  & Passi  1999;  Mannisenmaki  2000;  Telia  et 
al.  2001). 

3.  Cooperative  and  Communal.  Students  work  together  and  build  new  knowledge  in  cooperation  with  one 
another  while  benefiting  fron  the  knowledge  and  skills  of  others  (de  Corte  1995;  Jonassen  1995;  Lehtinen  1997; 
Ruokamo  & Pohjolainen  1999;  Mannisenmaki  2000;  Nevgi  & Tirri  2001).  Communalism  is  embedded  in 
dialogic  thought  (cf.  Vygotsky  1934/1962;  Vahtivuori,  Wager  & Passi  1999;  Telia  et  al.  2001).  Based  on  the 
theory  of  shared  expertise,  social  interaction  and  communal  modes  of  learning  and  studying  are  emphasized. 
The  teacher  is  expected  to  maintain  dialog  and  contact  with  the  students  in  NBE  (Passi  & Vahtivuori  1998; 
Sinko  & Lehtinen  1998;  Ruokamo  & Pohjolainen  1999;  Manninen  et  al.  1999;  2000.) 

4.  Conversational  and  Interactive.  A central  element  of  the  TSL  process  on  the  net  is  dialog  (Jonassen  1995; 
Telia  & Mononen-Aaltonen  1998;  Mannisenmaki  2000).  Dialog  and  dialogic  communication,  i.e.,  compre- 
hensive understanding  of  and  respect  for  one  another  as  well  as  interaction  and  interactivity  are  at  the  core  of 
NBE  (Telia  & Mononen-Aaltonen  1998).  The  quality  of  interaction  has  to  be  considered  when  an  environment 
for  NBE  is  being  designed  and  assessed.  Many  different  channels  of  communication  should  be  used  in  learning 
and  studying  environments  (Lintula  1999).  The  well-designed  environment  includes,  among  other  things, 
common  and  shared  conversation  and  working  spaces  in  which  documents  can  be  worked  on  together. 

5.  Contextual  and  Situational.  There  are  some  problematic  issues  related  to  the  physical  environment  the 
situation  as  well  as  to  time  and  place.  On  a network,  the  place  and  the  situation  gain  new  forms.  When  teaching 
and  studying  on  a net,  it  is  necessary  to  consider  the  changes  in  these  phenomena  and  how  they  affect  designing 
and  assessing  NBE.  (Vahtivuori  2001.)  According  to  the  features  of  meaningful  learning  and  studying,  learning 
tasks  support  meaningful  solutions  to  the  problems  of  the  real  world,  or  are  simulated  through  certain  case^ 
specific  or  problem-based  examples  of  the  real  world  (Sharan  & Sharan  1992;  de  Corte  1995;  Jonassen  1995; 
Lehtinen  1997;  Ruokamo  & Pohjolainen  1998;  Mannisenmaki  2000).  Gaining  experiences  in  NBE  is  at  least  as 


important  as  in  the  faco-to-face  TSL  process  (Ackermann  1994;  Bond  & Feletti  1999;  Vahtivuori  2001).  The 
links  with  school  are  essential  in  the  professional  practices  of  working  life  and  the  rest  of  society  (Sinko  & 
Lehtinen  1998;  Manninen  et  al.  2000).  Among  other  things,  simulations,  videos,  Internet  links,  implementations 
in  the  microworlds  and  applied  problem -based  situations  are  used  in  the  learning  and  studying  environment 
(Mannisenmaki  2000).  Especially  promising  are  on-line  strategic  and  role  games  that  include  the  principles  and 
criteria  linked  to  the  community  and  experience  (Vahtivuori  2001).  In  teaching  we  should  find  examples  and 
connections  to  the  real  world  and  to  the  personal  world  of  the  students  through  questions  touching  on  their 
interests.  Through  joint  discussions,  the  teacher  can  perceive  the  external  world  and  widen  his  or  her  own 
perspective  substantively.  (Sharan  & Sharan  1992;  Manninen  et  al.  1999.) 

6.  Transferable.  Students  know  how  to  use  their  knowledge  and  skills  in  other  situations  and  how  to  learn, 
adopt  and  benefit  from  them  when  learning  new  topics  (Ruokamo  & Pohjolainen  1999;  Mannisenmaki  2000; 
Nevgi  & Tirri  2001).  A learning  and  studying  environment  supports  emerging  learning  skills,  problem  solving 
skills  and  the  skills  of  self-directed  learning.  (Sinko  & Lehtinen  1998;  Manninen  et  al.  2000.)  A learning  and 
studying  environment  includes  cognitive  tools,  hypertext,  professional  systems,  and  databanks  that  underpin 
meaningful  learning  (Mannisenmaki  2000).  The  deep -going  discussions  between  the  teacher  or  the  tutor  and  the 
different  groups  of  students  are  fruitfully  conducive  to  enabling  the  transfer  of  learnt  knowledge  to  future 
situations  and  contexts  (Manninen  et  al.  1999). 

7.  Goal -Oriented  and  Purposive.  Students  achieve  a cognitive  goal  proactively.  They  can  define  and  set 
objectives  of  their  own  (de  Corte  1995;  Jonassen  1995;  Lehtinen  1997;  Ruokamo  & Pohjolainen  1999; 
Mannisenmaki  2000).  The  guidance  and  support  given  by  the  teacher  are  related  to  goal- oriented  and  purposive 
studying  (Uljens  1997;  Mononen- Aaltonen  1999).  Self- guidance  and  a genuine  attempt  to  learn  also  contribute 
to  creating  a learning  and  studying  environment  (Sinko  & Lehtinen  1998;  Manninen  et  al.  2000).  A learning 
environment  includes  various  tools  for  planning  and  monitoring  one's  learning  process.  Here,  a teacher  helps  to 
support  the  materialization  of  his  or  her  own  curriculum  (Manninen  et  ai.  1999;  Mannisenmaki  2000). 

8.  Guided.  Learning  itself  as  well  as  the  learner's  assessment  of  his  or  her  own  skill  are  promoted  by  the  teacher 
and  by  the  feedback  and  support  from  other  students  and  actors  (cf  Vygotsky  1934/1962;  Nevgi  & Tirri  2001). 
The  need  for  guidance  increases  rather  than  decreases  in  NBE.  On  the  other  hand,  the  rdes  of  the  teacher  and  the 
student  gain  new  dimensions  and  emphases  in  that  in  NBE  they  alternate  and,  at  the  same  time,  they  enrich  the 
TSL  process.  Students  and  teachers  take  turns  in  acting  as  guides  and  experts.  (Telia  et  al.  2001.)  The  learning 
and  studying  environment  has  such  tools  for  cooperation  and  discussion  that  enable  feedback  and  support  for 
both  the  individual  and  the  studying  community.  The  most  important  characteristics  of  the  learning  and  studying 
environment  are  functional  interaction,  the  opportunity  to  communicate  informally,  and  to  get  immediate 
(synchronous)  and  delayed  (asynchronous)  feedback. 

9.  Individual.  Humans  have  individual  learning  styles  and  strategies.  Learning  and  studying  are  always 
influenced  by  the  students'  prior  knowledge,  concepts  of  learning,  foci  of  interest  and  motivation  (de  Corte 
1995;  Lehtinen  1997).  A learning  and  studying  environment  is  basically  individual  (Nevgi  & Tirri  2001),  that  is, 
it  is  never  the  same  to  all  students.  Learners  can  construct  their  own  individualized  interpretation  of  the 
challenges  and  opportunities  posed  by  the  environment.  Each  learner  constructs  his  or  her  own  learning  and 
studying  environment,  where  the  immediate  feedback  given  by  the  teacher  supports  the  learner’s  individual 
process  of  handling  and  coping  with  information.  (Lehtinen  1997,  20.)  Here,  the  individualized  guidance  given 
by  a teacher  is  of  primary  importance. 

10.  Reflective.  Learners  express  what  they  have  learnt  and  they  examine  the  thinking  processes  and  decisions 
required  by  the  learning  process  (Jonassen  1995;  Ruokamo  & Pohjolainen  1999,  Mannisenmaki  2000;  Nevgi  & 
Tirri  2001).  Information  retrieval  and  processing  as  well  as  the  skills  of  critical  assessment  are  emphasized  in  the 
modem  learning  environment  (Sinko  & Lehtinen  1998;  Manninen  et  al.  2000).  A learning  environment  includes 
the  tools,  such  as  diaries  and  the  tools  for  a portfolio,  necessary  to  support  the  assessment  of  one's  learning  and 
active  output  (Mannisenmaki  2000).  The  environment  should  also  include  the  tools  that  the  teacher  can  use  when 
guiding  students  to  assess  their  own  studying  process. 

11.  Abstract.  Learning  can  be  defined  as  the  constmction  of  new  ideas  at  an  abstract  level;  the  development  of 
theoretical  ideas  reaches  from  practical  experience  to  the  deeper  level  (Lehtinen  1997).  The  learning  and 
studying  environment  enables  students  to  review  the  socio-constmctive  process  of  abstract  scientific  theories  and 
ideals  (Lehtinen  1997).  Here,  the  support  given  by  the  teacher  is  the  more  important  the  younger  the  students  are. 
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Conclusions 


The  above-mentioned  characteristics  of  NBE  should  be  seen  as  flexible  in  their  nature.  Through  them,  it  is 
possible  to  examine  how  different  pedagogical  models  realize  the  principles  of  design  and  assessment  and  how 
pedagogical  models  can  be  further  developed.  New  learning  environments  apparently  demand  new  cultural  skills 
and  media  skills  that  call  for  further  practice.  Teacher  educators  have  the  key  position  to  develop  the  media  skills 
of  future  teachers  in  educational  use  of  ICT  and  in  designing  goal-oriented  and  purposive  NBE.  The  design  and 
assessment  of  NBE  presuppose  that  the  teacher  and  all  users  have  sufficient  media  skills  and  an  ability  to  use 
their  skills  for  the  benefit  of  the  community  and  themselves.  (Telia  et  al.  2001.) 

This  article  has  examined  the  objectives  and  pedagogical  models  of  the  HelLa  project,  the  principles  of 
didactics  and  the  teaching-studying-leaming  process  as  applied  to  NBE,  and,  most  importantly,  the  development 
of  the  characteristics  of  meaningful  learning  and  studying  as  principles  of  designing  and  assessing  learning 
environments.  The  aim  has  been  to  analyze  certain  didactic  and  learning-theoretical  issues  in  order  to  develop 
NBE  and  educational  use  of  ICT  in  the  context  of  network  environments. 

To  sum  it  up,  the  principles  arising  from  the  didactic  models  of  NBE  are  in  harmony  with  the  principles 
emerging  from  the  learning-theoretical  models,  e.g.,  the  model  of  meaningful  learning.  In  particular,  both 
approaches  underline  the  significance  of  contextuality,  the  importance  of  guidance,  working  together,  dialog  and 
social  interaction.  The  training  programs  in  educational  use  of  ICT  being  implemented  in  Finland  provide  a 
versatile  opportunity  to  experiment  with  different  concretizstions  of  NBE  that  can  be  tested  against  the  principles 
of  design  and  assessment.  In  the  HelLa  project,  the  aim  is  to  summarize  and  further  develop  pedagogical  models, 
on  the  one  hand,  and  the  principles  of  design  and  assessment,  on  the  other. 
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Abstract  This  article  presents  the  solution  developed  at  the  University  of  Lapland  for  organizing  local 
activities  to  support  the  Finnish  Virtual  University.  The  Centre  for  Media  Pedagogy  (CMP)  was  established 
in  February  2001  and  its  fields  of  operation  include  research,  support  services,  and  training.  The  focus  of 
research  is  based  on  network-based  learning  environments.  Support  is  provided  to  meet  the  needs  of 
pedagogy,  technology,  and  content  production.  The  objective  of  the  support  service  is  to  help  university 
personnel  and  students  to  face  the  challenges  of  the  educational  use  of  information  and  communications 
technology  (ICT).  The  objective  of  training  is  to  support  and  produce  the  preparedness  for  those  things  the 
teaching  staff  need  in  educational  use  ICT.  The  University  of  Lapland  is  participating  and  coordinating 
several  Virtual  University  network  and  internal  projects.  The  CMP  has  also  other  national  and  international 
cooperative  projects  with  businesses  and  other  universities. 


Background 

The  establishment  of  the  Virtual  University  was  taken  as  an  objective  in  the  Information  Strategy  for 
Training  and  Research  for  the  years  2000-2004  by  the  Ministry  of  Education  (Ministry  of  Education  1999).  The 
Finnish  Virtual  University  was  established  in  February  2001,  when  the  representatives  of  all  universities  sgned 
an  agreement.  The  Virtual  University  is  not  a new  or  independent  university;  rather,  it  is  a new  means  for  the 
universities  to  act  as  a network,  [http://www.virtuaaliyliopisto.fi] 

An  objective  of  the  Virtual  University  is  to  provide  students  the  opportunities  for  flexible  and  open  study, 
where  information  and  communications  technology  is  utilized  in  teaching.  The  Centre  for  Media  Pedagogy 
(CMP)  project  started  in  the  University  of  Lapland  at  the  beginning  of  2001.  The  research  and  product 
development,  support  services  of  the  Virtual  University  and  training  are  centred  in  the  Centre  for  Media 
Pedagogy  [http://www.urova.fi/home/mpk]  as  a synergistic  entirety.  The  objective  of  the  CMP  is  to  support  and 
coordinate  the  development  and  use  of  new  teaching  methods  and  the  activities  of  the  Virtual  University  in  the 
University  of  Lapland.  (Ruokamo  & Syijakari  2001a.)  The  organization  and  fields  of  operation  for  the  CMP  are 
presented  in  Figure  1 . 


Figure  L The  organization  and  fields  of  operation  for  the  CMP 

The  development  group  of  CMP  consists  of  representatives  from  different  faculties  and  departments  and 
its  support  group  was  formed  of  the  representatives  of  the  projects  by  the  Virtual  University  working  in 
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cooperation  with  a team  consisting  of  researchers  and  staff  from  the  CMP.  The  team,  with  its  pedagogical  and 
technical  expertise,  is  responsible  for  the  operation  and  realization  of  support  services  and  realizes  the  training 
provided  to  university  personnel  and  students  in  cooperation  with  professionals  from  the  faculties.  The 
responsibility  for  directing  research  rests  with  the  faculties. 


Research  and  Cooperative  Projects 

The  activities  of  the  CMP  include  research  (e.g,  Ruokamo  2001a,  b,  c)  and  product  development,  which 
in  turn  support  the  activities  of  the  Virtual  University  and  the  design,  realization,  and  assessment  of  network- 
based  education  (see  Ruokamo  et.  al.).  The  Faculty  of  Education  is  responsible  for  directing  research.  It  initiated 
a seminar  on  network-based  learning  environments  in  the  autumn  2000,  the  realization  of  which  utilized  the 
BSCW  group  work  program.  The  research  topics  relate  to,  among  other  things,  online  interactive  curriculum 
representation  (see  Lehtonen),  interaction  in  cognitive  science  network  studies  (see  Johansson  & Ruokamo)  and 
the  e ffectiveness  of  training. 

The  CMP  has  underway  and  in  progress  some  national  and  international  cooperative  projects  with 
universities  and  the  business  world.  The  Centre  for  Media  Pedagogy  has  operated  as  a cooperative  actor  with  the 
Spanish  University  of  Gitalonia  in  the  Scottish  University  of  Highlands  and  Islands  (UHI)  project.  The  aim  of 
the  UHI  project  was  to  create  a decentralized  university  that  is  based  on  cooperation  between  local  institutions  of 
education.  The  UHI  is  part  of  the  EU- funded  ADAPT- LINC  project  that  was  realized  from  April  2000  to  June 
2001.  (Ruokamo  & Syijakari  2001b,)  In  2002,  the  CMP  initiated  the  IMPEL  FIN_2002  project.  The  objective  of 
the  project  is  to  design,  realize,  and  pilot  a web-based  period  of  study  as  part  of  the  subject  sphere  of  the 
Ministry  of  the  Environment,  in  which  students  will  be  from  several  European  countries. 

Support  Services  - Support  for  Personnel  and  Students  in  Multiform  Education 

An  objective  of  the  Virtual  University  is  to  provide  university  students  wth  opportunities  for  open  and 
flexible  studies.  For  example,  through  flexibility  and  openness,  the  barriers  to  study  will  be  removed,  in  that 
university  students  can  participate  in  the  education  provided  by  another  university  without  the  need  to  travel  to 
another  locality.  The  opportunities  for  flexible  and  open  study  require  education  to  be  multiform,  which  often 
means  that  teaching  utilizes  video-conferencing  and  web-based  learning  environments.  In  some  cases,  an  entire 
course  will  be  fully  realized  by  studying  on  the  net.  The  development  of  education  to  become  multiform  sets  new 
challenges  in  front  of  teachers:  it  is  necessary  to  know  how  to  utilize  information  and  communications 
technology  in  an  appropriate  pedagogical  manner.  Even  today,  the  development  and  realization  of  network -based 
education  are  the  activities  of  a small  group  of  pioneers,  and  neither  has  it  yet  become  established  as  normal 
practice.  A network-based  learning  environment  is  a student -centred  mode  of  action  that  requires  the 
development  of  a new  form  of  study  skills  and  self  direction  (Syrjakari  & Abrahamsson  2000),  For  this  work  of 
development,  teachers  need  various  forms  of  preparedness  (technical,  pedagogical,  and  content  production)  as 
well  as  a ring  of  support  around  them.  Besides  support  activities,  the  follow-up  and  continuing  education  of 
teaching  staff  is  one  of  the  most  central  means  to  meet  these  challenges.  (Ruokamo  & Pohjolainen  2000,) 

The  CMP  organizes  developmental  projects  for  university  personnel  for  the  pedagogical,  tchnical, 
content  production,  and  material  support  of  teaching  utilizing  information  and  communications  technology.  The 
support  team  provides  advice,  guidance,  and  training.  The  support  is  provided  either  personally  or  as  guidance 
for  a small  group  of  university  personnel,  and  it  is  also  provided  to  students.  With  respect  network -based 
learning  environments,  teachers  are  offered  either  the  WebCT  [http ://www, webct.com/],  or  the  Discendum 
Optima  [http://www.discendum.com/]  platforms.  Experience  is  also  being  gained  with  other  platforms,  such  as 
Humap  [http://www.humap.com/humap/default.htm]  and  Future  Learning  Environment  [http://fle3.uiah,fi/]. 

The  CMP  organizes  an  extensive  range  of  support  in  the  educational  use  of  information  and 
communications  technology,  which  corresponds  to  the  needs  of  the  representatives  of  Virtual  University  projects 
and  other  personnel.  The  training  needs,  that  relate  to  net  pedagogy  and  the  multiform  nature  of  education  as 
well  as  to  the  content  production  of  network-based  education,  of  university  teachers  and  the  teaching  support 
staff  were  surveyed  in  the  spring  2001.  Thirty-five  people  responded  to  the  questionnaire.  Figure  2 below  shows 
the  earlier  experience  of  teachers  in  multiform  education. 


Tutoring 


Producing  web-based 
learning  material 

Network-based 

education 

Video-conferencing 


Audio-conferencing 


0 5 10  15  20  25 


® No 
O Yes 


Figure  2.  The  earlier  experience  of  teachers  in  multiform  education 

As  can  be  seen  in  the  diagram,  over  half  the  respondents  were  unfamiliar  with  the  production  of  web- 
based  learning  materials  and  almost  half  stated  that  they  had  no  personal  experience  of  web -based  education. 
The  utilization  of  video-conferencing  in  teaching  was  familiar  for  many  either  in  the  practice  of  a meeting  or  in 
the  role  of  a student  during  a video  lecture.  Based  on  the  survey  of  training  needs,  the  training  of  personnel  was 
planned  for  the  2001-2002  semester  and  it  began  in  the  autumn  2001.  Its  contents  include  the  development  of  a 
web-based  course  as  a process,  net  pedagogy,  the  WebCT,  Sonera  experience  Training,  Sonera  experience 
Learning  and  Discendum  Optima  platforms,  the  ABC  of  video-conferencing,  the  production  of  web -based 
learning  materials,  and  the  strategic  preparation  for  the  educational  use  of  information  and  communications 
technology.  In  addition,  the  training  is  organized  through  linking  sound,  picture,  and  video  to  web -based 
learning  material. 

At  the  beginning  of  2002,  the  support  training  was  redesigned.  The  objective  is  to  support  the 
diversification  of  staff  teaching  projects  even  more  effectively.  The  provision  of  training  was  planned  to 
originate  from  the  idea  of  the  project  and  supports  the  teacher  through  to  its  realization.  Teachers  have  the 
opportunity  to  participate  in  content  courses  so  that  they  can  if  wished  also  complete  media  education  studies  as 
credits.  In  addition,  teachers  can  select  courses  in  skills  according  to  their  own  needs,  which  include  handling 
sound,  picture,  and  video  and  attaching  them  to  the  web-based  learning  materials.  In  addition  to  these,  teachers 
can  participate  in  courses  on  using  different  media  such  as  the  use  of  video-conferencing  in  teaching. 

In  allocating  support  for  2001—2002,  the  projects  aimed  at  what  the  Virtual  University  has  to  offer  are  in 
a very  special  position.  The  University  of  Lapland  is  participating  in  the  following  8 network  projects  by  the 
Finnish  Virtual  University.  SOSNET  is  a national  university  network  for  social  work  coordhated  by  the 
University  of  Lapland  [http://www.urova.fi/home/mpk/Sosnet.htm].  KasVi  - The  Virtual  University  of 
Educational  Sciences  is  a joint  project  by  the  Finnish  Faculties  of  Education 

[http://www.urova.fi/home/mpk/Kasvin%kuvaus.htm]  and  its  sub-project  HelLa  is  a research  project  for  the 
design  and  assessment  of  pedagogical  models  in  network-based  education 
[http://www.urova.fi/home/mpk/HelLa.htm]  (see  Ruokamo  et.  al.).  COA/A/£T- Cognitive  Science  and  Cognitive 
Engineering  National  Education  Programme  is  realized  through  the  cooperation  of  eight  universities  and  the 
objective  is  to  trail  multi-disciplinary  professionals  to  meet  the  needs  of  the  information  society 
[http://www.urova.fi/home/mpk/Connet.htm]  (see  Johansson  & Ruokamo).  Network  projects  include  also 
Postgraduate  Studies  of  Law,  Finland  Futures  Academy  (fVA),  a network-based  teaching  project  for  the 
acquisition  of  scientific  knowledge  (J'ieDot)  and  the  Virtual  University  of  Women’s  Studies  {Naisverkko). 

The  University  of  Lapland  is  also  participating  in  2 national  projects  by  the  Virtual  University;  IT-Peda  is 
a network  for  enhancing  knowledge  on  online  teaching  and  learning  in  Finnish  universities  and  TieVie  is  a 
national  teacher-training  programme  for  the  use  of  information  and  communications  technology  in  higher 
education.  The  University  of  Lapland  has  its  own  Virtual  University  projects  in  content  production  including  a 
course  on  the  community,  art  and  the  environment,  subject  studies  in  quantitative  research,  a Swedish,  English, 
and  German  language  module  for  students  of  economics,  the  use  of  mobile  services  in  teaching,  and  developing 
constitutional  rights. 
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The  activity  of  the  Virtual  University  realizes  the  existing  information  administration  strategy  for  the 
years  1998-2001  and  especially  its  focus  of  development  **New  Learning  Environments  - the  Diversification  of 
Teaching’',  In  cooperation  with  the  Faculty  of  Education,  the  Centre  for  Media  Pedagogy  participates  in  the 
information  strategy  process  initiated  at  the  university,  especially  in  preparing  the  strategy  for  the  educational 
use  of  information  and  communications  technology. 


Training 

The  CMP  arranges  and  acquires  funding  in  order  to  organize  long-term  training.  An  example  of  such  is 
the  15  credit  study  program  New  Learning  Environments  in  the  Work  of  the  Teacher  (NET),  which  is  directed  by 
the  CMP  [http://www.urova.fi/home/mpk/Koulutus.htm].  NET,  which  is  financed  and  realized  with  funding 
from  the  European  Union’s  European  Social  Fund  (ESF),  is  a development  project  of  university-level  teaching 
methods  that  promotes  the  development  of  employment  by  supporting  employability,  entrepreneurship, 
adaptability,  and  equality  as  well  as  by  investing  in  human  resources.  It  has  been  aimed  at  the  university  teachers 
and  teaching  support  staff  in  the  University  of  Lapland,  in  the  Rovaniemi  Polytechnic  and  in  the  Kemi-Tomio 
Polytechnic.  The  study  program  is  realized  2 times  in  2000—2001.  At  the  beginning  of  2000  there  were  initially 
74  participants  and  72  participants  in  the  autumn  2001.  As  of  the  autumn  2001,  the  studies  are  in  accordance 
with  the  requirements  set  by  the  Faculty  of  Education  at  the  University  of  Lapland.  Research  into  effectiveness 
will  be  performed  in  relation  to  the  NET  studies. 

The  CMP  functions  as  the  local  actor  responsible  for  supporting  the  T/eK/e  training  project  initiated  by 
the  Virtual  University.  The  objective  of  the  TieVie  training  is  to  familiarize  participants  with  information  and 
communications  technology  from  the  perspective  of  pedagogic,  technological,  and  organizational  change  as  well 
as  to  research  the  challenges  and  problems  of  networked  education.  Representatives  of  the  personnel  at  all 
Finnish  universities  are  participating  in  the  training.  The  CMP  is  responsible  locally  for  providing  information, 
support  services,  and  mentoring. 

In  the  University  of  Lapland,  the  CMP  coordinates  the  Virtual  University’s  CONNET  - Cognitive 
Science  and  Cognitive  Engineering  National  Education  Programme.  This  project  was  coordinated  by  the 
University  of  Helsinki  in  2001  and  will  be  coordinated  by  the  University  of  Jyvaskyla  in  2002-2003.  The 
curriculum,  the  purpose  of  which  is  to  train  multi-disciplinary  professionals  to  meet  the  needs  of  the  information 
society,  consists  of  eight  different  subjects.  Each  of  the  eight  universities  coordinates  the  practical  organization 
of  the  information,  registration,  and  teaching  at  its  own  university.  Basic  studies  in  cognitive  science  began  in 
the  autumn  semester  2001  and  research  related  to  the  interaction  for  their  realization  is  underway.  (See 
Johansson  & Ruokamo.)  Subject  studies  in  cognitive  science  will  begin  in  the  autumn  semester  2002. 

The  University  of  Lapland  has  responsibility  for  a course  on  Organisations  and  Modern  Communication, 
which  is  held  in  cooperation  with  the  University  of  Helsinki.  The  course  was  realized  for  the  first  time  in  the 
spring  2002  and  the  Faculty  of  Education  at  the  University  of  Lapland  was  responsible  for  the  professionalism  of 
its  content.  Students  from  six  different  universities  participated  in  the  section  of  the  course  prepared  by  the 
University  of  Lapland.  They  were  divided  into  small  groiqDs  both  according  to  university  and  between 
universities.  The  course  consisted  of  an  introductory  lecture  realized  through  video-conferencing  and  two 
separate  parts:  a web-based  reading  group  and  study  tasks.  In  the  web-based  reading  group,  each  student  wrote 
an  introduction  based  on  course  literature  and  these  were  discussed  in  small  groups  utilizing  WebCT.  In  the 
study  tasks,  the  student  groups  wrote  cooperatively  an  essay  based  on  course  literature,  which  they  also 
presented  through  video*conferencing  with  other  universities.  The  small  groups  also  commented  on  each  other's 
presentations  in  video*conferences.  Overall,  59  % of  the  students  that  had  registered  for  sections  of  the  course 
completed  it.  The  experiences  of  the  first  realization  of  the  course  were  encouraging  and  motivated  the 
continuing  development  of  the  course.  The  learning  tasks  for  the  next  realization  of  the  course  in  the  autumn 
2002  will  be  developed  to  become  problem  based  and  the  guidance  for  students  will  be  further  increased. 


Conclusions 

The  operations  of  the  CMP  are  in  their  initial  phases  and  the  modes  and  means  of  action  are  still  being 
created.  The  objective  in  the  future  is  to  firmly  establish  its  activities  and  the  status  of  the  unit.  Information  and 
communications  technology  is  developing  quickly  and  it  enables  new  forms  of  realization  for  the  development  of 
teaching  and  study.  Finland,  and  especially  Lapland,  is  a sparsely  populated  region  with  long  distances  between 
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places,  so  flexible  opportunities  for  study  are  necessary  in  order  to  expand  the  possibilities  for  participation  in 
training.  The  preparedness  of  teaching  staff  for  the  educational  use  of  information  and  communications 
technology  is  growing  and  the  attitudes  that  accompany  it  are  becoming  more  positive.  The  increas  e in  research 
knowledge  for  the  educational  use  of  information  and  communications  technology  supports  the  development  of 
teaching  more  than  ever  before  to  correspond  to  the  needs  of  the  student. 
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A Case  Study  on  the  Use  of  Video  Teleconferencing  to  Study  Geography  at  the 
United  States  Air  Force  Academy:  Lessons  Learned 

Barbara  Rusnak 
Introduction 

As  some  instructors  have  discovered,  guest  speakers  can  greatly  enhance  courses  (Davis, 
1993).  Guest  speakers  can  provide  expertise  and  special  skills  that  courses’  instructors  may  lack. 
However,  it  is  not  always  feasible  to  get  a guest  speaker  to  the  classroom,  given  geographical 
separation,  travel  expenses,  time  constraints,  and  conflicting  schedules.  Video  teleconferencing 
(VTC)  helps  bring  speakers  to  classes  as  moneysaving  and  timesaving  alternatives. 

VTCs  link  geographically  separated  groups  with  two-way  audio  and  visual  displays, 
providing  synchronous  connectivity.  With  synchronous  connectivity,  both  ends  of  the  connection 
have  television  monitors  with  speakers,  enabling  them  to  see  and  hear  each  other  nearly 
simultaneously.  This  connectivity  enables  interactivity  and  provides  both  ends  with  immediate 
response  and  feedback. 

My  interest  in  VTCs  began  when  I developed  a new  course  at  the  United  States  Air  Force 
Academy  (USAFA)  on  the  geography  of  the  British  Isles.  Believing  it  was  important  for  cadets  to 
experience  the  cultures  and  other  geographical  aspects  of  the  region,  I decided  a VTC  would  be  an 
interesting,  educational  learning  experience. 

I felt  a VTC  would  be  a valuable  supplement  for  a cultural  geography  lesson  because 
cadets  could  interact  with  people  from  the  distant  culture.  Davis  (1993)  recommends  enhancing 
courses  by  augmenting  them  with  speakers  from  diverse  ethnic  communities.  My  intention  was  for 
the  cadets  to  observe  the  linguistic,  ethnic,  and  sociological  differences  during  the  interaction. 
Conducting  a VTC  was  more  cost  effective  than  sponsoring  a guest  speaker  to  fly  over  from  the 
British  Isles.  The  results  of  the  VTC  were  promising  and  cadet  interest  was  high.  This  paper 
focuses  on  the  lessons  learned  from  the  use  of  this  technology  in  the  course  and  provides  some 
suggested  guidelines  for  its  effective  use. 

Conducting  the  VTC 

The  VTC  was  performed  with  Dr.  Gerald  Mills  at  University  College  of  Dublin  (UCD).  The 
focus  of  the  session  was  on  Irish  culture  and  politics.  We  devised  a lesson  that  would  give  cadets 
an  opportunity  to  talk  with  and  ask  questions  of  himself,  a geography  instructor,  and  two 
undergraduate  students  in  their  final  year  at  UCD.  The  objectives  of  the  lesson  were  for  the  cadets 
to  understand  the  cultural  differences  between  them  and  the  Irish  subjects,  know  the  current 
events  that  shape  the  lives  of  the  Irish,  and  understand  the  role  that  politics  has  in  Irish  culture. 
Cadets  were  tasked  to  read  The  Irish  Times  prior  to  the  VTC  to  get  a flavor  of  Irish  culture  through 
the  current  events  and  local  interest  articles,  editorials,  and  advertisements.  In  addition,  the  cadets 
were  instructed  to  prepare  questions  based  on  both  the  readings  and  their  intellectual  curiosity. 

Dr.  Mills  prepared  a 10-minute  presentation  to  begin  the  session,  which  centered  on  his  view  of  the 
readings  in  the  newspaper.  The  remainder  of  the  session  was  an  open  question  and  answer 
session,  with  both  cadets  and  UCD  students  asking  questions  of  each  side. 

USAFA  is  fortunate  to  have  a high-tech  VTC  lab  with  an  excellent  staff  to  maintain  and 
operate  it.  We  held  the  session  in  the  lab,  connecting  our  students  with  the  Irish  panel.  Each  side 
of  the  connection  had  two  monitors — one  with  a view  of  themselves  and  the  other  with  a view  of 
the  other  end  of  the  connection.  Microphones  were  on  desks,  making  audio  communication  simple 
and  clear.  As  this  session  was  the  first  VTC  for  all  involved,  I asked  a colleague  experienced  with 
VTCs  to  begin  the  session  by  providing  a four-minute  PowerPoint  lesson  on  VTC  protocol 
(communication  techniques).  My  colleague  discussed  time  delays,  background  noise,  speech 
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delivery,  and  visual  limitations.  Participants  on  both  sides  then  introduced  themselves  and  the 
session  proceeded  as  planned. 


Lessons  Learned 

I went  into  the  session  blind,  having  neither  previously  conducted  a VTC  nor  studied  the 
background  on  procedures  and  techniques.  I thought  it  important,  however,  to  evaluate  this 
attempt  with  the  VTC  process  and  prepared  a questionnaire  based  upon  one  created  and  used  by 
Heckman,  Whitford,  & Lawyer  (2000).  The  cadets  each  completed  a questionnaire  anonymously. 
In  the  future,  I will  administer  the  questionnaire  to  all  participants,  not  solely  students,  to  capture  all 
perspectives.  The  cadet  comments  revealed  that  some  aspects  of  the  session  went  well,  while 
others  areas  needed  improvement.  Following  is  a synopsis  of  the  results  of  the  questionnaire  in 
which  a score  of  “1”  is  “strongly  disagree/no  value”  and  a score  of  “5”  is  “strongly  agree/great 
value."  The  data  is  limited,  as  there  were  only  six  cadets  in  the  class.  However,  there  was 
consensus  among  most  of  the  cadets,  as  many  provided  similar  answers  to  questions. 

One  of  the  major  benefits  of  using  the  VTC  as  a learning  tool  is  the  visual  picture  that  helps 
create  a human  connection.  Extolling  the  virtues  of  the  use  of  video  in  geography  classes.  Gold 
(as  cited  in  Gold  et  al.,  1991)  states  that  “sight  is  of  prime  importance  in  human  sensory 
perception”  (p.  15).  VTCs  especially  aid  those  who  are  both  aural  and  visual  learners,  as  long  as 
effective  visual  aids  or  gestures  are  used.  For  example,  talking  heads  do  not  enhance  the  visual 
portion  of  the  session. 

When  asked  if  they  thought  the  session  was  valuable  in  gaining  insight  into  the  geography 
of  the  region,  the  cadets  gave  an  86.67  percent  approval  rating  (4.33  mean  score).  One  cadet 
commented  that  linking  to  the  panel  at  UCD  was  “better  than  just  reading  a book”.  I could  not  have 
provided  the  same  learning  experience  for  the  cadets  had  I been  the  sole  instructor  of  the  lesson 
because  I have  never  lived  in  Ireland  and  could  not  share  the  same  thoughts  and  feelings  as  did 
the  Irish  subjects.  I could  have  accomplished  the  objectives  through  means  other  than  a VTC — 
assigned  readings,  in-class  discussions,  presentations,  or  group  activities,  for  example.  However, 
the  real-time  audio  and  visual  exchange  provided  more  insight  and  left  us  with  a more  lasting 
impression  of  Irish  culture.  Online  chats,  email  exchanges,  and  telephone  conferences  can  be 
useful  in  exchanging  information,  but  they  lack  the  ability  of  the  VTC  to  present  a nearly  face-to- 
face  lesson  and  provide  the  two-way  visual  interface.  The  VTC  provides  the  nuances  (language 
and  mannerisms)  that  are  particularly  important  in  the  realm  of  cultural  geography. 

There  are  some  limitations  in  conducting  VTCs.  There  are  constraints  on  days  and  times 
you  can  connect.  With  the  popularity  of  VTCs,  instructors  are  increasingly  using  the  facilities. 
There  are  also  limitations  to  the  times  that  the  lab  is  open  and  support  staff  are  available.  Due  to 
the  time  difference  between  USAFA  and  UCD,  we  were  unable  to  conduct  the  VTC  during  our 
class  time  and  had  to  deconflict  student  schedules  to  find  a good  time. 

Thanks  to  my  colleague’s  guidance  and  the  assistance  of  the  lab  manager,  there  were  no 
glitches  in  the  session.  Once  the  connection  was  made  with  UCD,  we  found  the  VTC  to  be  very 
user-friendly  and  maintenance-free.  Cadets  commented  that  the  session  “seemed  quite  natural,” 
“ran  smoothly,”  and  was  “relaxed  and  flowing.”  Cadets  gave  the  format  a 73.33  percent  approval 
rating  (3.67  mean  score). 

All  of  the  cadets  noted  that  the  key  factors  in  making  a VTC  successful  are  the 
preparedness  of  the  students  and  the  quality  of  the  speaker(s).  For  example,  those  who  had 
studied  the  material  and  prepared  possible  questions  before  class  enjoyed  the  session  much  more 
than  those  who  had  not.  The  two  who  had  not  prepared  questions  commented  that  they  wished 
they  had.  They  also  claimed  that  an  instructor  who  is  uninteresting  in  one  delivery  format  will  most 
likely  be  just  as  uninteresting  in  another  setting. 

One  of  the  most  significant  ideas  that  I developed  as  a result  of  our  VTC  was  that  a VTC 
can  only  be  as  effective  as  the  instructor  or  facilitator  can  make  it.  Many  of  the  same  methods  that 
are  used  effectively  in  a VTC  (e.g.,  active  or  problem-based  learning)  should  be  similarly  effective 
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in  face-to-face  classes.  Mishra  (1999)  states  that  “most  learning  theories  suggest[s]  that  for 
learning  to  be  effective,  it  should  be  active”  (p.  243).  Conversely,  many  of  the  same  methods  that 
are  not  as  effective  in  a VTC  (e.g.,  traditional  lecture)  should  be  less  effective  in  face-to-face 
classes.  If  the  VTC  had  consisted  solely  of  a lecture  by  Dr.  Mills,  the  cadets  would  not  have 
appreciated  or  learned  as  much  from  the  session.  The  same  can  be  said  of  a face-to-face  class. 
Therefore,  the  major  benefit  of  a VTC  versus  a face-to-face  class,  in  this  scenario,  is  the  ability  to 
reach  out  to  a native  and  an  expert.  The  cadets  would  have  experienced  nearly  the  same  learning 
experience  if  the  panel  had  appeared  in  person  in  our  class,  however,  that  luxury  wasn’t  financially 
or  logistically  possible.  In  this  case,  the  VTC  was  the  technology  that  allowed  us  to  connect  and 
create  a positive  learning  experience. 

Although  cadets  emphasized  the  benefits  of  seeing  visual  images  of  the  other  participants, 
visual  aids  could  have  been  used  to  make  the  session  even  more  valuable.  The  visual  aspect  of 
VTCs  is  one  of  the  key  areas  that  should  be  exploited.  Photos  of  Ireland  would  have  further 
enriched  our  discussion  of  culture  through  examples  of  the  architecture,  landscape,  and  human 
subjects  from  the  area. 

For  a course  focusing  on  a region  so  geographically  separated,  the  VTC  was  particularly 
effective  in  that  it  allowed  cadets  to  ask  questions  of  and  exchange  information  with  students  of 
their  own  age  and  to  observe  a few  of  their  cultural  traits  first-hand.  In  that  50-minute  session,  the 
cadets  observed  the  Irish  subjects’  language,  dialect,  appearance,  dress,  and  mannerisms,  which 
are  all  aspects  of  cultural  geography.  The  Irish  subjects  also  shared  some  of  their  thoughts  and 
feelings  on  subjects  ranging  from  college  life  to  terrorism,  which  helped  paint  a picture  of  Irish 
culture.  When  questioned  about  the  value  of  interacting  with  the  Irish  panel,  there  was  a 98.33 
percent  approval  rating  (4.83  mean  score).  Five  of  the  cadets  wrote  comments  similar  to  one,  who 
wrote  that  it  was  “great  to  get  a current  perspective  from  native  Irish  students”  on  current  events 
affecting  Irish  culture.  The  sixth  cadet  wrote  that  the  interaction  with  the  instructor  was  good,  but 
that  “the  students  seemed  indifferent  and  unprepared”.  A better  approach  may  have  been  to  pass 
along  prepared  questions  to  the  Dubliners  prior  to  the  session  to  better  prepare  them.  Cadets  also 
noted  that  listening  to  the  panel  was  “much  more  interesting  than  [listening  to]  speculation  of  non- 
natives”. Most  cadets  were  well  prepared  and  asked  questions  that  could  be  best  answered  by  the 
panel.  For  example,  one  cadet  asked  the  Irish  students  how  the  conflict  in  Northern  Ireland 
impacted  their  daily  lives  and  if  the  conflict  was  a daily  topic  discussed  in  coffeehouses  or  pubs. 
Their  USAFA  instructors  could  not  have  adequately  answered  this  question. 

Cadets  agreed  we  should  pursue  using  VTC  technology  in  this  and  other  courses  at 
USAFA,  with  a 96.67  percent  approval  rating.  This  information  was  forwarded  to  USAFA’s 
instructor  education  office  to  possibly  encourage  other  faculty  members  to  use  the  technology  in 
the  future. 

One  cadet  went  so  far  as  to  suggest  a specific  course,  comparative  politics,  which  could 
benefit  from  the  technology.  Another  cadet  suggested  that  we  use  VTC  sessions  to  interview 
subject  area  experts  in  performing  research  for  papers  and  presentations.  Given  the  expenditure 
of  resources  (time,  personnel,  and  money)  to  conduct  a VTC,  it  might  be  difficult  to  justify  using  the 
technology  for  such  a limited  audience  and  objective.  Another  cadet  suggested  that  VTCs  are 
more  effective  with  small  class  sizes. 

During  his  discussion  of  VTC  protocol,  my  colleague  informed  us  that  one  limitation  of  our 
connection  would  be  that  UCD  had  a slower  baud  (transmission)  rate.  Because  of  the  slower  rate, 
there  would  be  a slight  delay  in  transmission.  He  instructed  us  to  begin  a response  to  the  other 
side  by  predicting  and  interrupting  at  the  end  of  their  question  or  statement.  I found  this  to  be  as 
awkward  as  it  sounds,  although  the  briefing  helped  me  mentally  prepare  for  the  limitation.  There 
were  pregnant  pauses  and  interruptions  on  both  sides.  The  situation  improved,  however,  as  the 
session  progressed  and  both  sides  became  better  accustomed  to  the  procedures.  Although  the 
delay  was  a bit  awkward,  the  students  thought  the  VTC  provided  them  a valuable,  interactive 
learning  experience. 
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Many  cadets  did  not  find  the  introductory  protocol  briefing  as  useful  as  did  I.  They  gave  the 
briefing  a 66.67  percent  approval  rating  (3.33  mean  score).  Most  commented  that  the  briefing  was 
useful,  but  based  on  common  sense.  They  followed  the  guidelines  fairly  well  throughout  the 
session,  providing  comments  such  as  “it  ran  smoothly,  relaxed,  and  free  flowing”.  One  cadet 
commented  that  the  briefing  started  the  session  out  with  a formal  atmosphere  and  suggested 
instructors  discuss  the  guidelines  on  their  respective  sides  prior  to  the  session.  Given  the  lack  of 
enthusiasm  over  the  briefing,  the  suggestion  is  a good  one,  especially  since  it  adds  more  time  for 
achievement  of  objectives. 


Suggested  Guidelines 

Although  VTC  technology  has  been  used  extensively  for  quite  some  time,  it  is  not 
necessarily  being  used  appropriately.  Yes,  quality  equipment  is  an  important  component  of  VTC 
sessions,  but  “any  successful  program  must  focus  on  the  instructional  needs  of  the  parties,  rather 
than  on  the  technology  itself  (Muller,  2000,  p.  225).  Following  are  a few  aspects  of  both 
technology  and  instructional  methodology  to  consider  when  preparing  for  a VTC. 

Instructors  should  become  familiar  with  equipment  operation.  “Learners  get  more  from  the 
courses  when  the  instructor  seems  comfortable  with  the  technology”  (Willis,  2001,  Guide  #1,  H 6). 
Communication  checks  should  be  performed  prior  to  the  day  of  the  session  and  just  prior  to  the 
session.  Once  participants  are  in  place,  microphone  and  camera  placement  should  be  checked, 
ensuring  all  participants  are  close  enough  to  a microphone  and  camera  angles  are  such  that  they 
center  on  the  group.  Some  cameras  can  zoom  and  pan  the  room  during  the  session  (Video 
Development  Institute  (ViDi),  2000,  Guide  #2,  H 3). 

Heckman  et  al.  (2000)  suggest  that  time  and  experience  are  necessary  for  instructors  to 
become  comfortable  with  this  technology.  They  note  that  VTCs  require  more  preparation  time  than 
face-to-face  classes  and  that  instructors  must  be  more  vigilant  in  monitoring  the  various  aspects 
(equipment,  students,  and  activities  at  each  end  of  the  connection)  of  the  session.  Hearnshaw’s 
(2000)  VTC  discourse  content  analysis  shows  a marked  increase  in  student  interaction  and,  thus, 
comfort  with  the  technology  as  a course  progresses. 

Visual  aids  are  important  materials  to  incorporate  into  VTCs,  although  they  are  not  as  easy 
to  use  as  they  are  in  face-to-face  classrooms.  Images  are  sometimes  obscured  or  blemished 
through  transmission.  VTC  equipment  is  available  that  allows  users  to  incorporate  high  quality 
visual  aids  into  sessions.  The  New  England  Medical  Center  has  a camera  that  allows  them  to  view 
x-rays  in  real  time.  Using  this  type  of  equipment  can  greatly  increase  the  overall  quality  of  the 
session,  as  it  can  provide  clear,  professional  images.  Demonstrations,  experiments,  and  videos 
can  help  enhance  the  VTC  as  well.  Regardless  of  the  tools  and  methods  used,  they  should  be 
tested  in  the  lab  prior  to  the  session  to  ensure  connectivity  and  quality.  As  with  face-to-face 
classes,  images  and  other  products  should  be  accurate  and  large  enough  for  participants  on  both 
sides  to  see  clearly. 

At  the  beginning  of  the  session,  all  parties  should  be  given  a short  briefing  on  VTC  protocol, 
unless  they  have  all  participated  in  and  are  familiar  with  this  type  of  session.  ViDi  (2000)  highlights 
several  protocol  items  to  help  ensure  a smooth-flowing  session.  Unnecessary  movement  should  be 
avoided,  as  it  can  adversely  affect  (break  up)  both  audio  and  video,  particularly  on  lower  quality 
systems.  Proper  etiquette  used  in  face-to-face  classes,  such  as  maintaining  eye  contact  with  the 
speaker  and  avoiding  personal  conversations,  is  even  more  important  in  VTCs.  “Side 
conversations  at  remote  sites  seem  to  spring  up  more  readily  that  they  would  if  everyone  were  in 
the  same  actual  room”  (ViDi,  2000,  Etiquette  section,  H 5).  These  side  conversations  are 
disrespectful  in  any  setting.  In  a VTC,  however,  these  distractions  add  unnecessary  noise  to  the 
session,  causing  disruption  and  moments  where  sometimes  no  one  can  hear  the  conversation  at 
hand.  To  compound  the  problem,  open  microphones  placed  around  the  room  allow  these 
conversations  to  be  readily  picked  up  by  the  other  side.  Instructors  must  constantly  monitor  the 
session  and  discretely  put  a stop  to  all  personal  conversations. 
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Cultural  differences  are  of  particular  concern  in  VTCs  connecting  people  from  different 
countries.  Cultural  sensitivities  should  be  discussed  and  stressed  prior  to  the  session  to  avoid 
offensive  comments  or  gestures.  Willis  (2001)  states  “that  students  may  have  different  language 
skills,  and  that  humor  is  culturally  specific  and  won’t  be  perceived  the  same  way  by  all”  (U  5).  We 
experienced  this  in  our  VTC,  as  a couple  of  the  jokes  that  the  cadets  made  were  not  understood  by 
the  Irish  panel.  Although  there  was  no  offense  by  either  side,  the  jokes  created  uncomfortable 
pauses  while  the  panel  attempted,  unsuccessfully,  to  comprehend  the  jokes. 

Finally,  participants  should  be  briefed  on  the  limitations  of  the  equipment,  such  as 
transmission  delays.  This  can  help  reduce  awkward  silences.  It  is  no  easy  task,  however,  and 
does  take  practice.  In  our  VTC,  participants  began  to  better  anticipate  the  other  side  and  silent 
periods  were  reduced  toward  the  end  of  the  session. 

Our  geography  class  consisted  of  only  six  cadets.  VTC  sessions  should  limit  class  size  in 
accordance  with  seating  and  equipment  limitations.  If  we  had  more  than  14,  there  would  have 
been  difficulty  getting  all  cadets  within  an  effective  range  of  the  speakers  and  camera.  As  with 
face-to-face  classes,  successful  discussions  are  much  easier  to  conduct  in  relatively  small  groups 
of  approximately  six  to  twelve  each.  Bates  (1995)  claims  that  “the  opportunity  for  any  individual  to 
participate  in  questioning  or  discussion  decreases  in  proportion  to  the  number  of  students  viewing 
a live  interactive  programme”  (p.  101).  Small  class  numbers  are  essential  when  conducting 
interactive  VTC  sessions.  The  classes  should  not  be  too  small,  however  (much  less  than  six),  or 
the  opportunity  to  share  multiple  perspectives  is  reduced. 

Instructor  presence  during  VTC  sessions  is  necessary.  VTCs  are  not  television  programs 
or  films  for  students  to  sit  and  watch  inactively.  Without  guidance  from  the  instructor,  students  can 
stray  off  of  the  topic  and  are  more  easily  refined  in  their  participation.  Students  are  more 
compelled  to  participate  under  the  watchful  eye  of  their  instructor.  In  addition,  an  observant 
instructor  can  analyze  participants’  non-verbal  cues  and  refocus  or  redirect  the  session.  Because 
of  transmission  delays  and  varied  levels  of  participant  comfort  in  conducting  VTCs,  instructors 
should  plan  to  cover  fewer  objectives  in  a VTC  than  in  a face-to-face  class.  (Willis,  2001). 

Many  VTC  guidelines  stress  an  avoidance  of  pure  lecture.  Mishra  (1999)  concludes, 
“teleconference  is  a medium  of  sharing  rather  than  lecturing”  (p.  247).  A sharing  environment  is 
one  that  promotes  interaction  between  all  participants  rather  than  unidirectional  instruction.  Willis 
(2001)  suggests  an  instructional  design  for  VTC  lessons  to  stimulate  student  learning  through 
diversity  of  activities.  McKeachie  (1994)  recommends  that  students  prepare  critical  questions 
when  a guest  speaker  is  featured.  Questions  help  encourage  human  interaction  and  help  students 
with  the  relevance  of  covered  topics. 

Evaluations  should  be  administered  immediately  following  any  VTC  session.  There  is  not 
much  literature  on  VTC  evaluations.  However,  Hearnshaw  (2000)  suggests  an  objective  method 
for  evaluating  the  learning  processes  of  VTCs.  In  his  evaluation,  he  assigns  codes  to  types  of 
questions  and  comments  made  by  students.  These  codes  are  based  on  the  depth  of  discourse 
and  the  learning  types  (cognitive  vs.  affective)  involved.  The  results  are  then  used  to  determine 
the  progress  of  student  learning  throughout  a series  of  VTCs.  The  difficulty  in  applying  his 
approach  is  that  it  was  designed  for  a distance  education  course  using  desktop  videoconferencing 
where  all  participants  (students  and  instructors)  are  geographically  separated.  Applying  the 
approach  to  two-way  VTCs  that  take  place  in  classroom  settings  must  be  evaluated.  The  contrast 
between  the  desktop  and  classroom  settings  could  produce  dissimilar  results,  as  students  in 
desktop  videoconferencing  may  perform  and  react  differently  than  they  would  in  classroom  VTCs. 
The  perception  of  anonymity  may  enhance  their  readiness  to  participate  more  freely  and  critically 
in  the  desktop  setting. 

Finally,  instructors  should  have  all  participants,  not  solely  students,  complete  a 
questionnaire.  The  lessons  learned  can  be  extremely  beneficial  for  future  VTC  attempts,  as  I 
discovered  in  our  session. 
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Conclusion 


In  courses  such  as  geography,  where  the  experts  (native,  empirical  sources)  are  at  such  a 
great  distance,  it  is  not  often  possible  to  physically  connect  students  with  guest  speakers.  The 
current  global  crisis  surrounding  the  threat  of  terrorism  may  even  reduce  the  opportunities  to 
physically  connect,  as  people  fear  public  transportation.  The  VTC  is  a fitting  alternative  in  such 
cases.  Instructors  need  to  provide  clear,  structured  guidance  to  both  students  and  speakers  prior 
to  the  session  and  should  follow  the  aforementioned  guidelines.  In  order  to  promote  the  effective 
use  of  this  technology,  institutions  must  support  faculty  by  providing  training  and  development 
programs  and  quality  services.  VTC  sessions  are  not  the  answer  to  every  situation  and  cannot 
replace  the  use  of  in-person  instruction,  but  they  are  wonderful  alternatives  when  needed  and  used 
appropriately. 
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Higher  education,  including  teacher  preparation  programs,  is  rapidly  becoming  technology-based. 
Educational  Technology  (ET)  is  expanding  opportunities  for  effective  learning,  penetrating  into  both 
campus-based  and  web-based  curricula.  ET  offers  the  means  to  improve  the  quality  of  teaching  and 
enhance  learning.  However,  along  with  the  numerous  potential  benefits,  it  brings  new  problems  and 
complications.  One  of  the  counter  effects  of  increasing  technology  dependence  is  a growing  distance 
divide,  both  physical  and  social,  between  students  and  instructors  in  today's  education  that  is  grounded  in 
the  paradox  of  contemporary  social  life  and  technology. 

A similar  picture  can  be  seen  in  strictly  pedagogical  terms.  Nowadays  education  is  (or  at  least, 
should  be)  unequivocally  learner-centered.  Learning,  we  believe,  in  order  to  be  efficient,  needs  to  provide 
the  learner  with  a multiplicity  of  options  of  customized  reality-based  curricula.  Nevertheless,  due  to  some 
social  and  technological  factors,  among  them  the  rapidly  spreading  use  of  information  technology 
(electronic  media  in  general  and  online  education  in  particular),  learners  are  becoming  increasingly 
distanced  from  their  schools  and  instructors.  Reality-based  experiences  seem  to  disappear  from  the 
instructional  process  as  the  learner  becomes  more  and  more  divorced  from  hands-on  practices.  This  leads 
not  only  to  the  growing  divide  between  the  key  participants  in  the  educational  interaction  but  to  the 
disappearance  of  the  teacher  as  a role  model  from  the  student's  perspective  as  well.  Moreover,  it  is  not  only 
the  teacher's  personality  that  may  well  become  obsolete  in  online  education  (as  classes  are  taught  mostly  by 
merely  sharing  text  files),  but  also  life  and  professional  experiences  that  the  teacher  brings  to  class  may 
become  lost  through  text  dominated  technology.  It  becomes  vital  to  preserve  "human  dimension"  in  a 
technology -based  learning  environment  (Serdyukov  1999).  In  our  research  we  address  important 
pedagogical  issues  attempting  to  bridge  the  physical  and  social  distance  emerging  in  online  education  by 
capturing  and  bringing  reality-based  experiences  straight  to  the  student's  computer. 

In  various  methodology  classes  comprising,  together  with  foundational  courses,  undergraduate  as 
well  as  postgraduate  preparation  teacher  preparation,  e.g.  "Methodology  for  Second  Language  Instruction", 
there  is  an  inherent  need  for  reality-based  visual  approaches  to  classroom  practices.  It  is  only  through  visual 
representation  of  reality-based  situations  that  a student  can  attain  a truly  comprehensive  teaching  model  as 
a premise  to  develop  their  own  instructional  techniques.  In  terms  of  online  classes  which  purport  to  teach  a 
methodological  approach  we  posit  this  solution  set;  visual  presentation  of  the  instructional  content  through 
Streaming  Video  (SV).  The  rationale  is  straightforward.  You  cannot  fairly  ask  a student  to  efficiently 
demonstrate  an  approach  without  the  antecedent  of  a competent  presentation.  In  other  words,  for 
methodology  classes  visual  expert  presentation  precedes  actual  student  demonstration.  The  pedagogic  focus 
of  online  teacher  preparation  is,  after  all,  on  how  to  adapt  and  transform  traditional  teaching  practices  and 
the  content  of  courses  to  include  students  for  whom  the  traditional  onsite  practices  and  interactions  are 
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restricted  due  to  the  nature  of  their  distance  learning.  In  teacher  preparation  it  is  imperative  to  find  ways 
and  means  to  compensate  for  the  lack  of  reaHife  instructional  situations. 

The  infusion  of  SV  in  online  education  is  based  on  the  situated  learning  model  (Lave  and  Wenger 
1991).  It  is  not  so  much  that  learners  acquire  structures  or  models  to  understand  the  world,  but  they 
participate  in  frameworks  that  that  have  structure.  Learning  involves  participation  in  a community  of 
practice.  (Cit.  from  Smith  2001)  In  other  words,  the  complexity  of  the  online  course  should  not  be  on  the 
screen  but  in  the  interaction  between  a student  and  the  instructor  (Ryan  2002). 

To  create  authentic  experiential  and  reflective  environments  for  learning,  we  can  use  a number  of 
professionally  made  video  materials,  including  special  instructional  videos,  documentaries  and  even  feature 
films.  "Best  practices"  videos  are  one  of  the  best  resources  for  teacher  preparation  programs.  However, 
often  we  cannot  find  the  video  that  suits  our  topic  or  objectives.  Then  we  can  create  one  within  our  means, 
e.g.  record  our  own  video  introduction  to  the  course,  videotape  the  lecture  we  are  going  to  teach,  or  present 
a tape  with  a demonstration  of  some  situation  we  want  to  use  in  classroom  activities. 

The  advent  of  professor’s  video  presentations  leads  us  to  a student-produced  demonstration  of  a 
given  methodological  approach.  The  value  of  having  the  student  send  a video  demonstration  (now  via 
mailed  VHS,  soon  via  video  streaming)  with  age  appropriate  learners  (after  viewing  a professor’s  chosen 
video  presentation)  of  a discreet  methodological  approach  cannot  be  underestimated.  This  kind  of  student 
produced  video  assignment  is  one  that  can  almost  never  be  replicated  in  the  traditional  higher  education 
classroom  where  age  appropriate  students  (5  to  17  years  old)  are  simply  not  available.  Thus,  in  a very 
special  way,  this  highly  documented  approach  takes  us  one  step  further  in  reality-based  education.  Clearly, 
online  education,  especially  in  a methodology  class,  engages  the  human  element  via  video,  which  enhances 
the  student’s  range  of  social  and  pedagogic  skills. 

There  are  several  goals  in  using  video.  They  are  as  follows: 

-To  deliver  the  course  content  (videotaped  lectures  and  instructional  video) 

-To  illustrate  some  points  of  the  lecture  presented  orally  or  in  a text  format  (video  clips) 

-To  use  videotaped  materials  for  analysis  and  discussion  ("Best  practices"  documentary  and 
demonstrations  video,  students’  videos) 

-To  offer  visual  reality-based  models  for  a comprehensive  view  of  practices  which  work  best  in  today’s 
classroom. 

- To  use  videotaped  material  as  an  instrument  for  self-evaluation  or  assessment  (students'  self-video) 

-To  incorporate  a more  holistic  (i.e.  intellectual,  social,  emotional)  dimension  to  classroom  learning, 
spurring  both  cognitive  and  affective  growth  (feature  film,  videotaped  demonstrations). 

-To  humanize  the  online  class  by  presenting  both  the  instructor  and  students  in  video  format. 

The  advantages  of  a SV-based  approach  are  not  exclusive  to  online  instruction.  In  fact,  any  traditional 
onsite  class  may  avail  itself  of  the  SV  reality-based  benefits  (e.g.  working  with  age  appropriate  learners). 

This  is  to  say  that  SV  is  as  a hybrid  tool  which  can  be  used  in  literally  any  instructional  format.  It  is  so 
because  it  adds  a reality-based  human  dimension  to  learning.  Students  are  best  prepared  for  teaching  when 
instruction  brings  the  real  world  into  the  classroom. 

SV  is  opening  new  opportunities  for  online  education.  It  brings  the  human  dimension  back  into 
students'  learning  - in  a specific,  technology-mediated  format:  students  can  now  see  their  instructor  on  the 
screen,  watch  real-life  situations  like  teaching  practices  that  are  used  to  prepare  them  for  their  own 
teaching.  It  enhances  new  content  acquisition  by  combining  the  multiple  modalities,  text  and  image.  It 
creates  the  platform  for  situated  learning  based  on  observing  reality-based  classroom  situations.  Finally,  it 
permits  students  to  perform  actual  demonstrations  in  real  classroom  with  live  students,  allowing 
professorial  feedback  on  a “hands-on”  approach. 
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Abstract:  This  case  study  was  to  explore  how  a learner  transforms  knowledge  structure  in 
hypermedia  authoring.  As  a data  collection  method,  think-aloud  protocol,  stimulated  recall 
interview,  and  open-ended  interview  were  used  to  examine  the  transformation  process.  The 
study  revealed  that  the  participant  mainly  used  two  distinctive  cognitive  processes  to 
transform  his  knowledge:  1)  top-down  approach  and  2)  chunking  strategy.  Regarding  top- 
down  approach,  the  participant  searched  his  knowledge  structure  to  identify  highest 
knowledge  component;  then  it  was  followed  from  the  highest  knowledge  component  to 
organize  it  into  hypermedia.  Also,  he  used  a chunking  strategy  in  considering  whether  or  not 
each  piece  of  knowledge  is  appropriate  for  the  chunk.  Additionally  the  participant’s  thinking 
flow  might  be  constrained  with  the  authoring  tools’  function. 


Knowledge  Transformation  in  Hypermedia  Authoring 

Hypermedia  Authoring 

There  have  been  some  studies  to  explore  the  benefits  of  implementing  hypermedia  authoring,  also 
referred  to  as  students -as -multimedia-author,  with  two  main  premises.  First,  it  is  assumed  that  authoring 
hypermedia  can  provide  learners  high  authenticity  of  a given  task.  Beichner  (1994)  insisted  that  learners 
regarded  the  work  they  were  doing  had  importance  outside  of  the  classroom.  Furthermore,  learners  showed  that 
they  were  highly  involved  in  the  given  tasks.  High  degree  of  learner  engagement  was  found  throughout  the 
project  (Lehrer,  1993;  Lehrer,  Erickson,  & Connell,  1994). 

Second,  hypermedia  authoring  is  assumed  that  it  provides  more  opportunities  of  having  learners 
develop  sophisticated  knowledge  structure  because  they  have  to  use  multiple  and  linked  mental  representation 
to  author  hypermedia  as  a final  product.  Learners  working  with  authoring  environments  outperformed  than 
control  group  in  terms  of  articulating  and  defending  essay  topics  (Spoehr,  1993).  Learners’  knowledge 
structures  became  more  sophisticated,  their  design  skills  emulated  a cognitive  framework  and  a problem  solving 
process.  As  a result,  their  problem-solving  skills  increased  and  became  more  complex  (Oughton  & Reed,  1998). 
Similar  result  was  reported  that  students  in  hypermedia  authoring  showed  the  increased  cognitive  process  such 
as  finding,  interpreting,  and  articulating  information  and  knowledge  (Lehrer  et  al.,  1994).  These  cognitive 
processes  seem  a promising  advantage  of  hypermedia  authoring. 

In  an  effort  to  design  an  instructional  material,  a learner  also  has  to  reflect  on  his/her  own  knowledge 
in  terms  of  transformation  of  an  author’s  knowledge  into  a hypermedia  product.  In  other  words,  learners  need  to 
refer  to  their  structural  knowledge  in  order  to  produce  the  given  task.  Along  with  this  process,  the  learners  in 
hypermedia  authoring  demonstrated  high  creative  thinking  (Liu,  1998).  Additionally  nonlinear  structure  of 
hypermedia  enabled  learners  to  be  more  reflective  (Koehler  & Lehrer,  1998;  Lehrer,  1993). 


Cognitive  Process  in  Hypermedia  Authoring 

There  are  various  frameworks  representing  author’s  cognitive  process  during  hypermedia  authoring. 
For  instance,  it  consists  of  collecting,  organizing,  and  displaying  information  in  a multimedia  format  (Nicaise  & 
Crane,  1999).  However  they  did  not  provide  in-depth  descriptions  about  what  cognitive  process  should  be 
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implemented  in  each  phase.  Moreover,  there  was  no  phase  for  revising  or  assessing  the  author’s  knowledge 
structure. 

Unlike  Nicaise  and  Crane’s  cognitive  framework,  Lehrer  (1991,  1993)  and  Lehrer,  Erickson,  and 
Connell  (1994)  modeled  a cognitive  framework  including  evaluation  and  revision  as  (a)  planning,  (b) 
transforming,  (c)  evaluating,  and  (d)  revision.  Brief  description  was  listed  in  the  below. 

• Planning  the  design  of  a hypermedia  document  involves  deciding  upon  the  nature  of  the  problem  and 
developing  ideas  about  the  overall  structure  of  the  document. 

• Transforming  is  to  modify  information  into  knowledge.  Knowledge  is  distinguished  from  information 
by  its  systematicity-  a network  of  relationships  that  sustain  one  or  more  themes. 

• Evaluation  relies  on  designers’  abilities  to  solicit  feedback,  which  in  turn  relies  upon  their  skills  in 
articulating  their  goals  and  intentions  to  others. 

• Revision  requires  multiple  cycles  of  drafting  and  revision  in  order  to  make  a successful  product. 

Moreover  Oughton  and  Reed  (1998)  suggested  a cognitive  framework  very  similar  to  Lehrefs  though 
they  proposed  3 stages  as  1)  planning,  2)  transforming,  and  3)  evaluation  and  revising. 

Summary  of  Hypermedia  Authoring 

Planning  plays  a role  as  an  initial  step  to  locate  required  information;  however,  the  identified 
information  can  be  discarded  depending  on  the  author’s  decision  when  he/she  transform  and/or  revise  a 
hypermedia.  Among  planning,  transformation,  and  revision,  transformation  and  revision  are  more  critical  to 
produce  an  artifact  as  a final  product  in  hypermedia  authoring  process.  In  this  sense,  transformation  and 
revision  will  be  a main  activity,  which  learners  involve  in  authoring  hypermedia. 

Therefore,  main  benefits  of  implementing  hypermedia  authoring  may  come  from  transformation  and 
revision.  By  performing  transformation  and  revision,  learners  modify  and  adopt  their  knowledge  structure  as  an 
artifact,  hypermedia.  However,  little  is  known  about  specifically  focusing  on  cognitive  strategies  of 
transformation  and  revision. 

Indeed  the  previous  studies  have  loosely  authenticated  how  learners  use  the  authoring  tool  for  their 
project.  For  instance  what  functions  would  be  effective  on  learners’  authoring  is  not  identified.  More 
specifically  this  question  is  to  identify  key  components  of  developing  ideal  authoring  tools.  It  is  assumed  that  if 
there  are  distinctive  learning  process  or  pattern  they  can  be  applied  for  developing  an  authoring  tool. 


Research  Questions 

This  study  was  focusing  on  examining  how  the  learner  transforms  own  knowledge  into  hypermedia 
authoring.  The  purpose  of  this  research  was  to  identify  cognitive  strategies  with  hypermedia  authoring  and  to 
ejqjlain  how  he/she  transforms  knowledge  to  develop  hypermedia.  In  particular  tfiis  study  will  examine  3 
questions;  I)  How  did  the  participant  plan  for  the  authoring  tool,  2)  What  features  of  the  authoring  tool  did  the 
participant  use,  and  3)  How  did  the  participant  use  the  authoring  tool. 


Method 

Selection  of  Study  Participant 

In  order  to  select  an  appropriate  participant  of  this  study  the  following  criteria  were  used:  a participant 
of  this  study  should  have  (a)  a familiarity  with  a given  subject  matter  and  (b)  knowledge  of  instructional 
designing  or  experience  of  developing  instructional  materials.  The  first  requirement  is  strictly  related  to  the 
purpose  of  this  study.  It  was  to  explore  knowledge  transformation  process  in  hypermedia  authoring  process 
rather  than  knowledge  acquisition.  Knowledge  transformation  process  is  a manipulation  of  already  acquired 
knowledge.  In  order  to  meet  this  purpose,  it  was  necessary  that  a participant  was  knowledgeable  to  a given 
content.  Second,  knowledge  of  instructional  designing  or  experience  of  developing  instructional  materials  was 
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also  required  for  a study  participant  because  hypermedia  authoring  requires  designing  skills  of  developing 
instructional  material. 

The  Participant 

One  participant  was  selected  with  fulfillment  of  tow  criteria  described  above.  The  participant,  Chris  (a 
pseudonym),  is  a first  year  doctoral  student  who  has  enrolled  in  a doctoral  program  of  instructional  design  at  a 
southeast  public  university.  Chris  took  two  courses  related  to  instructional  design  theories.  He  was  regarded  as 
an  intermediate  instructional  designer  with  satisfying  the  second  criteria  of  selecting  a participant  of  the  study. 
Regarding  the  first  criteria  of  selecting  a participant,  one  of  the  classes,  which  Chris  was  taking,  was  selected  as 
a subject  matter.  The  selected  class  was  about  theoretical  studies  of  learning  and  cognition  for  graduate  students. 

Materials  and  Tasks 

The  selected  material  is  about  Ausubel’s  meaningful  reception  learning  (Driscoll,  1999).  Thirty 
concepts  were  selected,  and  they  were  developed  in  a printout  material  as  well  as  a digitalized  format  as  a 
package.  Each  concept  was  called  as  a slide,  and  one  slide  contained  only  one  concept.  All  slides  were  text- 
based  format  without  any  graphics  or  charts.  The  slides  were  sorted  in  an  alphabetical  order.  The  following  6 
sequential  tasks  were  assigned  to  Chris. 

(1)  Rating  Slides  with  5 Points  Likert  Scale:  It  was  given  to  verify  whether  Chris  was  familiar  with 
the  given  material.  5 points  Likert  scale  was  used  from  1 point  with  very-well-unfamiliar  to  5 point  with  very- 
well-familiar. 

(2)  Practice  of  the  Authoring  Tool:  After  rating  the  given  package,  Chris  was  asked  to  practice  using 
the  authoring  tool.  He  was  able  to  use  the  authoring  tool  without  any  problems  in  a given  two  trial 

(3)  Practice  of  Think-Aloud  Protocol:  It  was  the  first  time  of  performing  think-aloud  protocol  for 
Chris;  hence,  practice  of  performing  think-aloud  protocol  was  needed.  This  task  was  conducted  during  the 
second  trial  of  practicing  the  authoring  tool.  The  researcher  asked  what  he  was  thinking  to  prompt  his  think- 
aloud.  During  this  practice,  the  following  sample  questions  were  used.  Why  are  you  selecting  the  slide?  Are  you 
looking  at  that  slide?  What  are  you  thinking  the  relationship  between  the  slides?  What  does  the  line  mean? 

(4)  Authoring  Hypermedia:  Prior  to  authoring  hypermedia,  planning  session  was  given  as  proposed  in 
the  previous  studies.  In  order  to  visualize  Chris  ’ planning  process,  a blank  paper  proved  to  draw  a planning  map 
of  how  each  slide  will  be  connected  and  what  would  be  master  or  non-master  slides.  It  was  also  asked  to  draw 
links.  After  completing  the  planning  session,  Chris  rearranged  and  linked  the  given  set  of  slides  by  using  the 
authoring  tool. 

(5)  Evaluating  and  Revising  the  Developed  Hypermedia:  The  final  product  was  created;  then,  Chris 
was  told  to  evaluate  and  revise  the  developed  hypermedia.  He  looked  through  all  master  slides  and  linked  slides 
to  revise  whether  there  are  non-logical  sequences.  This  phase  was  given  to  Chris  to  verify  his  initial  plan  for 
authoring.  Then  he  revised  the  first  planning  map  and  re -authored  the  authoring  tool. 

(6)  Concept  Mapping  : At  the  end  of  all  tasks,  Chris  also  was  asked  to  make  his  concept  map  about  the 
content  without  consideration  of  implementing  the  content  to  the  authoring  tool. 

Authoring  Tool 

An  authoring  tool  was  developed  for  this  study  by  using  Toolbook  version  5.0.  The  authoring  tool  had 
a function  of  rearranging  the  given  materials.  Composing,  editing,  and  nnnaging  multimedia  objects  were  not 
implemented  to  this  authoring  tool.  The  authoring  tool  contained  slides,  and  each  slide  contained  one  concept  of 
the  subject  matter.  The  user  was  able  to  rearrange  the  order  of  slides  by  selecting  slides. 

Data  Collection 

This  study  employed  three  ways  of  collecting  data;  an  observation  with  think-aloud,  stimulated  recall 
interview,  and  open-ended  interviews.  These  methods  were  to  gain  unbiased  observation,  to  increase  credence 
in  the  interpretation,  and  to  demonstrate  commonality  of  an  assertion  for  data  source  triangulation  (Stake, 
1995).  For  the  observation  the  researcher  videotaped  each  reviser’s  utterances,  and  interrupted  as  little  as 
possible,  except  to  answer  direct  questions  from  the  participant,  to  offer  expressions  of  support,  and  to  give 
Chris  prompts  to  keep  continuing  thinking  aloud.  With  videotaping,  the  researcher  took  notes  for  stimulated 
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recall  interview,  which  was  conducted  at  the  end  of  all  assigned  tasks.  The  stimulated  recall  interview  was  to 
investigate  particular  utterance  or  behavior  observed  in  his  authoring  process  during  think-aloud  protocol.  The 
notes  were  used  to  stimulate  Chris  ’ recall.  Finally  all  examines  were  ended  with  the  open-ended  interview. 


The  participant’s  rating  of  the  given  slides  with  5 points  Likert  scale  showed  that  17  (57%)  slides  were 
rated  as  very  familiar,  12  (40%)  were  familiar,  and  1 (3%)  was  just  familiar.  It  indicated  that  Chris  was  so 
familiar  with  all  given  thirty  slides. 

How  did  the  participant  plan  for  the  authoring  tool? 

Observation  with  think-aloud  protocol  and  stimulated  recall  interview  showed  that  Chris  was  trying  to 
follow  his  knowledge  structural  of  the  given  content  in  order  to  establish  main  structure.  In  this  process  he 
mainly  used  two  strategies:  1)  top-down  approach  and 2)  chunking  strategy. 

Regarding  top-down  approach,  he  tried  to  identify  which  slide  could  be  the  highest  concept.  After 
setting  several  highest  concepts,  Chris  narrowed  down  from  them.  During  identifying  the  highest  concepts,  he 
compared  slides  in  terms  of  hierarchical  structure.  The  following  excerpts  represent  protocols  from  think-aloud 
and  stimulated  recall  interview. 

Let’s  see  this  slide  is  conceptually  higher  than  number  5 (each  slide  had  its  unique  identification 
number)... Oh!  It  is  right.  That  slide  (number  5)  is  definitely  lower  than  this  slide.  Then  which  one  can  be 
equivalent  with  this  slide?  (Chris  is  shufflingslides  to  find  another  one.)  I got  it  (picking  up  another  slide).  I 
need  to  make  sure  if  these  two  slides  are  equally  highest  one. 

Top-down  approach  is  a conceptually  driven  processing  guided  by  information  already  stored  in 
memory.  This  process  screens  structural  knowledge  from  higher  level  to  generate  expectations  and  hypotheses 
relating  to  the  interpretation  and  evaluation  (Eysenck,  1990).  It  supports  that  Chris  was  already  familiar  with  the 
contents;  hence,  he  did  not  need  to  figure  out  the  concepts.  Rather,  as  an  author,  he  tried  to  establish  key 
structure  that  can  be  easily  conveyed  to  target  audiences.  To  accomplish  the  cognitive  task,  he  compared  the 
given  slides  with  considering  how  the  slides  were  related  to  each  other.  In  other  words  what  he  was  seeking 
were  procedural  knowledge  in  terms  of  reflecting  on  each  slide’s  relation.  It  can  be  supported  from  the  result  of 
Nicaise  and  Crane  (1999)  showing  that  all  students  reported  having  learned  procedural  skills. 

Another  strategy  mostly  used  in  planning  stage  was  chunking  strategy.  Chris  spent  more  time  to 
determine  whether  which  slide  could  be  chunked  in  a particular  highest  concept.  More  cognitive  efforts  were 
made  to  chunk  with  the  same  conceptual  structure.  As  a result,  he  created  four  key  chunks,  and  each  key  chunk 
consisted  of  from  4 to  9 slides,  and  it  was  well-structured  one.  Regarding  the  total  number  of  slides  of  the  given 
package,  each  chunk  contained  from  13%  to  30%  of  the  total  slides.  The  percentages  of  each  chunk  were 
relatively  high  in  terms  of  total  numbers  of  slide.  This  feature  is  consistent  with  the  study  supporting  that 
hypermedia  authoring  increased  number  links  within  the  same  concept  (Reed  & Rosenbluth,  1995).  During 
using  chunking  strategy,  Chris  applied  two  criteria:  1)  degree  of  contiguity  and  2)  sequence  within  the  same 
chunk.  The  following  from  think-aloud  and  stimulated  recall  interview  proves  what  criteria  he  used. 

I think  that  number  10  (slide’s  number)  is  closer  to  that  slide  rather  than  this  one.  Is  it  correct?  (Picking  up 
another  slide).  This  one  should  belong  to  that  category  and  it  won’t  be  a problem.  Still  this  slide  is  unclear. 

Urn. . . Although  this  slide  has  different  title  (for  that  reason  he  is  confused),  I bet  it  is  closer  to  this  category. 

[From  think-aloud  protocol] 

Actually  I got  confused  because  that  slide  could  belong  to  another  category.  However,  I thought  that  I should 
decide  one  category.  Sometimes  deciding  only  category  was  not  easy  decision,  but  it  was  very  important.  If  I 
made  a wrong  category  (chunk),  my  students  might  have  misunderstandings.  Just  I wanted  to  prevent  that 
kind  of  situation.  [From  stimulated  recall  interview] 
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Um...  Which  one  should  be  a first  one?  This  one  or  that  slide?  Let 's  see.. . This  should  be  place  in  advance, 
then  that  slide  should  be  second  one  (Two  slides  were  already  categorized  into  the  same  chunk).  [From 
think-aloud  protocol] 

What  features  did  the  participant  use  the  authoring  tool? 

Regarding  the  difficulties  of  using  the  authoring,  there  was  no  problem  and  any  confusion  for  Chris  to 
rearrange  and  link  the  given  slides.  He  could  finish  authoring  itself  very  quickly.  In  fact,  using  the  authoring 
tool  would  be  easier  to  use  than  the  other  authoring  tools  due  to  the  restricted  function  of  the  authoring  tool  of 
this  study. 

Although  the  authoring  tool  was  so  easy  to  sue,  the  think-aloud  protocol  and  open-ended  interview 
disclosed  potential  problems.  Chris  was  in  troubles  of  presenting  some  slides  that  did  not  have  a hierarchical 
structure.  This  problem  might  come  from  the  navigation  methods  of  the  authoring  tool  based  on  linear  ways  of 
rearranging  the  given  slides. 

It  may  not  be  a right  way  of  putting  this  slide.  However,  this  authoring  tool  just  has  linear- way  of  presenting 
information.  How  can  I compromise  this  problem?  [From  think-aloud  protocol] 

Sometimes  it  was  hard  to  prioritize  slides  when  they  do  not  have  concrete  hierarchy.  In  addition,  the 
authoring  tool  was  based  on  linear  navigation  though  it  had  a function  to  link  related  information  to  any 
master  slide.  I think  that  if  you  add  some  function  of  showing  non-linear  relationship  between  more  than  two 
slides  it  would  be  much  better  way  to  convey  my  concept  mapping  on  the  hypermedia  product.  [From  open- 
ended  interview] 

The  discrepancy  between  the  picture  of  planning  step  and  concept  mapping  supported  that  the 
observation  and  suggestions  from  think-aloud  protocol  and  open-ended  interview.  In  the  planning  step  he  draw 
a very  linear  fashioned  picture  in  both  of  initial  and  revised  plan.  However,  his  concept  mapping  showed  non- 
linear directions  and  dynamic  relationships  between  any  single  concepts.  It  may  come  from  difficulty  of  making 
appropriate  technique  representing  learners’  knowledge  structure.  Although  representing  learners’  knowledge 
structure  is  very  important,  finding  the  technique  is  very  challenging  (Jonassen,  Beissner,  & Yacci,  1993). 

It  may  reflect  that  functions  of  authoring  tool  can  be  a constraint  of  organizing  knowledge.  In  this 
study,  as  mentioned  early,  the  authoring  tool  was  designed  to  produce  linear  based  hypermedia  by  rearranging 
the  given  slides.  For  this  reason  Chris  needed  to  fit  the  authoring  environment.  It  suggests  that  various  ways  to 
represent  different  characteristics  of  knowledge  should  be  considered  in  designing  authoring  tools. 

How  did  the  participant  use  the  authoring  tool? 

Revision  process  was  pretty  simple  because  of  the  simplicity  of  the  given  authoring  tool.  Chris  still 
employed  the  same  criteria  (degree  of  contiguity  and  sequence)  used  in  the  planning  step.  Most  processes  he 
performed  were  very  similar  to  those  of  planning  step.  However,  it  may  not  reflect  cognitive  process  associated 
with  actual  revision  because  this  study  did  not  provide  any  feedback  to  Chris.  In  other  words,  his  revision  was 
based  on  self-evaluation.  In  general  revision  requires  interactions  with  audiences  in  authoring  hypermedia 
method  with  specific  feedback. 


Implications 

There  are  two  implications  from  this  study.  Firstly,  Chris  evaluated  the  information  in  terms  of 
hierarchical  order.  It  reflects  if  an  authoring  tool  has  a function  to  evaluate  value  of  information  the  user  can 
easily  organize  information  into  knowledge  structure.  In  addition,  this  function  will  be  helpful  to  chunk  the 
information  in  terms  of  the  degree  of  importance  within  the  same  chunk. 

Secondly,  this  study  suggests  that  an  authoring  tool  should  indicate  multiple  relationships  between 
information.  Due  to  the  linearity  of  the  authoring  tool,  Chris  was  not  able  to  connect  certain  information  with 
another  chunk  if  the  knowledge  did  not  belong  to  the  chunk.  It  was  a hard  decision  for  Chris  to  ma  ke  a decision 
to  link  information  with  a single  indicator.  For  instance,  some  information  may  be  connected  into  another 
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information  in  multiple  ways  such  as  prior,  currently  related,  or  advance  knowledge.  These  multiple  layers  of 
indicators  showing  types  of  knowledge  can  be  represented  with  different  colors  within  an  authoring  tool. 
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Abstract.  Today’s  children  are  discerning  consumers  of  learning  media  in  an 
interactive  world.  They  explore  computer  programs  and  on-line  environments  at  home 
and  at  school  for  different  purposes,  use  many  different  learning  strategies  and  develop 
perceptions  about  the  usefulness  of  various  types  of  interactive  media.  In  this  study,  we 
consulted  with  five  children  in  Grade  5 to  explore  their  experiences  learning  with 
interactive  reference  media,  and  to  document  their  participation  in  the  development  of  a 
rapid  prototype  for  an  interactive  encyclopedia.  We  explored  qualitative  methods 
synthesized  from  the  disciplines  of  educational  psychology  and  information  design  to 
study  how  and  what  we  can  learn  from  children’s  educational  experiences  with 
interactive  media. 


Introduction 

Children  are  active  learners  who  interact  with  teachers,  fellow  learners  and  educational  media,  to 
actively  construct  their  knowledge  of  the  world.  Their  opinions,  insights  and  interpretations  of  their 
experiences  are  invaluable  to  the  development  of  effective  learning  tools. 

Approaches  to  studying  children’s  learning  with  interactive  media  and  their  participation  in  the 
design  of  interactive  media  bridge  disciplines,  encompassing  ideas,  theories  and  practices  embraced  by 
constructivism,  constructionism,  user-centred  design,  participatory  design,  cooperative  design,  design- 
centred  learning,  collaborative  learning  and  cooperative  inquiry.  In  bridging  the  disciplines  of  educational 
psychology  and  information  design,  we  may  coordinate,  synthesize  and  strengthen  our  approaches  to  the 
design  of  effective  educational  media.  Educational  psychology  brings  knowledge  of  how  children  learn  and 
how  to  apply  this  knowledge  effectively  in  learning  situations.  Phenomenology  offers  a relevant,  practical 
approach  to  explore  the  experiences  of  learners  with  interactive  media.  Data  analysis  consists  of  deriving 
general  or  universal  meanings  from  individual  descriptions,  to  gain  the  essence  of  the  experience. 
Researchers  must  bracket  their  own  biases  (Creswell,  1998).  Information  design  stresses  a socially 
responsible,  user-centred  focus  to  the  design  of  educational  technology  and  media.  In  the  emerging  culture 
of  participatory  design,  the  user  becomes  a proactive  participant  in  the  design  process  (Sanders,  1999). 

In  this  study,  we  explored  appropriate  methods  to  study  children's  participation  in  the  design  of 
interactive  educational  media  and  how  can  these  experiences  inform  the  design  of  interactive  learning 
media.  Cooperative  inquiry  (Druin,  1999)  is  a design  approach  to  creating  new  technologies  for  children, 
with  children,  and  involves  discussion  and  hands-on  working  groups  in  the  rapid  iterative  creation  of 
prototypes.  This  approach  has  been  adapted  from  the  methodologies  of  cooperative  design,  participatory 
design  and  contextual  inquiry.  Participatory  design  processes  have  generally  involved  adults,  but 
cooperative  inquiry  employs  intergenerational  teams.  We  consulted  only  with  children  as  learners,  users, 
experts,  researchers  and  designers  to  find  out  what  kind  of  interactive  encyclopedias  would  be  useful  and 
usable  for  them.  We  focused  on  the  process  of  the  cooperative  inquiry,  rather  than  the  outcome  (a  prototype 
of  an  interactive  encyclopedia). 


Method  and  Results 


Five  Grade  5 children,  aged  about  10  or  11  years,  were  diosen  at  random  by  their  teacher,  and 
participated  in  three  sessions  for  a total  of  two  hours  over  three  days.  During  the  first,  30 -minute  individual 
session,  participants  searched  for  lemmings  and  ptarmigans,  using  a think-aloud  protocol,  and  responded  to 
structured  interview  questions  about  the  design  and  use  of  interactive  encyclopedias.  During  the  second, 
30-minute  group  session,  five  participants  responded  to  semi -structured  interview  questions  about  their 
experiences  in  the  first  session.  They  were  also  asked  to  comment  nn  hnw  CD-ROM  encyclopedias  should 
be  designed  to  help  the  user  find  information.  Finally,  during  a 60-minute  session,  the  five  participants 
worked  collaboratively  to  design  a rapid  paper  prototype  of  what  they  considered  to  be  an  effective  CD- 
ROM  encyclopedia  in  the  form  of  sample  ‘pages’.  Comments  were  noted,  and  the  working  process  was 
documented  with  sketches  and  photography. 

What  are  appropriate  methods  to  study  children's  participation  in  the  design  of  interactive 
educational  media?  Any  methods  that  can  directly  and  actively  involve  children  as  consultants  are 
valuable.  These  methods  may  need  to  be  adapted  from  bridging  disciplines,  and  to  determine  their 
usefulness.  How  can  these  experiences  inform  the  design  of  interactive  learning  media?  These  experiences 
can  be  analysed  to  find  common  themes  that  would  support  the  development  of  guidelines  for  design.  For 
example,  participants  were  quite  aware  of  how  information  can  be  presented  effectively,  and  wanted  to  see 
all  images  next  to  dated  text.  They  were  able  to  judge  features  that  provide  easy  navigation  to  get  at 
desired  information.  The  students  were  quite  concerned  with  appropriateness  of  features  and  their  functions 
("no  weirdo  stuff,"  must  be  "easy  to  understand  if  you  don't  speak  English,"  must  have  "access  for  all  ages" 
and  we  should  "add  voice  button,  in  case  people  can't  read").  They  felt  that  the  search  function  should  be 
easier  to  use,  and  they  surprised  to  note  that  they  had  trouble  using  the  help  function.  The  children  found  it 
"frustrating  to  work  this  quickly"  during  the  design  activity,  and  wanted  to  continue  the  task  to  use 
authoring  tools  to  design  an  improved  interactive  encyclopedia.  They  incorporated  into  the  design  of  the 
prototype  what  they  considered  to  be  essential  functions:  search  function,  back  button,  bookmark  and  a 
place  to  store  notes  and  documents.  This  group  was  highly  effective  at  working  together  to  organize  their 
materials  and  process  in  a short  time  period.  They  demonstrated  flexibility  and  innovation  in  thought  and 
action,  possibly  learned  from  experience  in  problem-solving  and  group  work. 


Conclusion 

Children’s  insights  regarding  the  effectiveness  of  such  media,  and  their  participation  in  its  design, 
are  essential  to  developing  learning  materials  that  are  best  suited  to  their  needs  and  interests.  Synthesizing 
research  methods  promises  to  be  beneficial  in  designing  studies  that  involve  children  in  cooperative 
inquiry,  as  consultants,  researchers  and  designers  in  developing  interactive  learning  tools.  Being  involved 
in  this  type  of  research-based,  design-centred  learning  was  enjoyable  and  beneficial  for  the  participants.  We 
would  like  to  continue  to  study  the  design  process  with  children,  and  would  like  to  move  to  the  next  step  of 
authoring  technology  so  the  participants  can  see  some  of  their  ideas  in  action.  Further  research  could 
involve  the  development  and  evaluation  of  an  interactive  encyclopedia  incorporating  the  guidelines 
outlined  by  the  child  designers,  in  comparison  to  a commercial  product. 
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Abstract.  In  many  jurisdictions,  the  use  of  computer-based  learning  tools  is  endorsed  at 
a curricular  level.  Looking  at  how  children  search,  select,  organise  and  visually 
represent  information  can  challenge  our  assumptions  about  learning  and  technology,  to 
guide  the  design  of  effective  learning  tools.  We  report  on  the  preliminary  findings  from 
a study  involving  43  Grade  5 students  designed  to  explore  the  effect  of  information 
presentation  in  print  and  CD-ROM  encyclopedias  on  children’s  retrieval  of  information 
and  design  of  research  posters.  We  outline  the  effects  of  prior  media  experience,  topic 
knowledge  and  attitude  towards  using  interactive  reference  media  to  do  research. 
Children  may  come  to  a research  task  with  flexible  strategies  and  mental  models.  Ease, 
convenience  and  speediness  of  use,  and  the  availability  and  relevance  of  different  types 
of  information  appear  to  be  at  least  as  important  as  the  presentation  of  information. 


Introduction 

In  our  visually -oriented  world,  children  learn  with  computer  technology  and  interactive  media, 
encountering  different  types  of  information  presentation  and  using  different  strategies  to  respond  to 
research  tasks.  Just  providing  children  with  technology  does  not  necessarily  increase  learning.  Well- 
designed,  user-centred  interfaces  facilitate  children’s  information-seeking  (Zammit,  2000;  Bilal,  2000).  The 
presentation  of  information  may  influence  children’s  use  of  these  learning  tools.  Even  the  way  a computer 
program  is  constructed  influences  the  search  strategies  chosen  by  the  user  (Zammit,  2000).  Due  to 
"shortcomings  in  design  and  research  the  potential  of  hypertext/hypermedia  for  enhancing  learning  may 
have  been  underestimated"  (Tergan,  1997,  p.  209).  Crook  suggested  that  studies  that  evaluate  academic 
performance  may  fail  to  investigate  how  media  is  used  for  research,  synthesis  and  investigation.  In  our 
learner-centred  study,  we  integrated  methods  in  developmental  science,  instructional  technology  and 
information  design  to  study  how  information  presentation  in  print  and  CD-ROM  encyclopedias  affects 
children's  retrieval  and  visual  representation  of  information. 


Study  Method  and  Results 

Forty-three  Grade  5 children,  aged  about  10  or  11  years,  participated  in  this  study  using  the  Junior 
print  version  of  the  Canadian  Encyclopedia,  and  a Macintosh  PowerBook  laptop  computer  displaying  the 
starting  interface  for  Student  CD-ROM  version.  In  individual  sessions  of  30  minutes,  using  a think-aloud 
protocol,  randomly -assigned  participants  searched  for  information  about  lemmings  and  ptarmigans  using 
both  encyclopedias,  and  sketched  a small  poster  to  show  the  life  cycle  of  each  animal.  Participants  then 
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responded  to  semi -structured  questions  about  their  experience  using  encyclopedias,  topic  knowledge  and 
attitudes  towards  searching  for  and  presenting  information. 


Results  and  Discussion 

All  participants  reported  some  prior  experience  with  encyclopedias  (print,  88%;  CD-ROM,  77%; 
on-line,  21%  and  with  the  Internet  (95%).  Over  half  (56%)  of  the  participants  preferred  the  CD-ROM 
version  (print,  28%;  no  preference,  16%).  The  print  encyclopedia  was  ’’more  hands-on”  and  "gives  more 
information.”  The  CD-ROM  version  was  ’’easier  to  read"  and  "easier  to  find  [the  topic]"  since  "you  can 
type  in  what  you  want  and  it  takes  you  there  with  no  page-flipping." 

On  average,  participants  spent  almost  twice  the  amount  of  time  to  search  the  CD-ROM 
encyclopedia  (7.0  min)  as  the  print  version  (3.3  min).  Some  participants  suggested  that  the  CD-ROM 
encyclopedia  was  not  well  designed,  as  it  wasn't  clear  where  to  type  in  a search  term.  Even  children  with 
previous  media  experience  were  temporarily  confused.  Difficulty  with  navigation,  increased  options  or 
motivation  to  explore  may  explain  the  difference  in  time. 

On  average,  participants  spent  about  the  same  amount  of  time  to  create  posters  after  the  print 
version  (9.1  min)  as  after  the  CD-ROM  (8.7  min).  This  may  support  children's  having  prior  mental  models 
of  how  to  present  information  in  research  reports.  Comparing  features  found  in  posters  and  in 
encyclopedias  yielded  only  two  features  of  significance:  the  use  of  comfortable  margins  and  the  inclusion 
of  paragraphs  or  paragraph-like  chunks  of  textual  information.  Comfortable  margins  are  found  only  in  the 
print  encyclopedia,  and  paragraphs  were  found  in  both  encyclopedias,  so  it  difficult  to  find  a direct  effect  of 
information  presentation.  Throughout  the  study,  participants  focused  on  the  need  to  have  easily  accessed, 
relevant,  readable  information,  with  images  close  to  related  text. 

Almost  half  (42%)  of  the  participants  included  "life  cycle"  through  a variety  of  layouts  using 
diagrams  or  words  in  their  at  least  one  of  their  posters,  although  there  were  no  life -cycle  diagrams  or 
textual  descriptions  for  either  topic  in  either  encyclopedia.  This  suggests  that  they  may  have  had  a prior 
mental  model  about  how  life  cycles  could  be  visually  represented.  Indeed,  students  commented  that  they 
had  learned  about  animal  life  cycles  in  science  classes  in  earlier  grades. 

Conclusion 

Children  may  have  flexible  learning  strategies  and  may  fit  new  information  to  pre-existing  mental  models. 
Ease,  convenience  and  speediness  of  use,  and  the  availability  and  relevance  of  different  types  of 
information  appear  to  be  at  least  as  important  as  the  specific  arrangement  of  the  information.  These 
findings  have  implications  for  research-based  design.  We  must  challenge  some  of  our  assumptions  about 
the  design  of  learning  tools.  Studying  the  learner  provides  a powerful  opportunity  to  focus  not  only  on 
information  presentation  and  other  aspects  of  design,  but  on  learners’  motivations,  preferences  and  learning 
strategies,  which  are  critical  factors  affecting  the  optimal  design  and  use  of  learning  tools. 
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Abstract  Learners  taking  an  online  course  often  develop  implicit  expectations  as  to 
what  they  can  expect  from  traditional  dassrooms.  The  purpose  of  this  study  was  to 
determine  student  expectations  of  an  online  collaborative  learning  environment,  as  well 
as  to  examine  how  students  adjust  their  expectations  when  the  course  ends.  The  results 
from  an  online  survey  indicated  that  student  expectations  before  taking  the  course  and 
after  the  course  are  different.  Overall,  the  students  raised  their  expectations  at  the  end  of 
the  course,  except  for  their  expectations  of  the  instructor  and  the  level  of  effort  and 
convenience.  In  addition,  students  significantly  gained  more  understanding  of  an  online 
learning  environment  after  the  course  was  completed. 


Introduction 

Learners  construct  their  own  meaning  based  on  the  beliefs,  understanding,  prior  knowledge,  and 
cultural  practices  they  bring  to  class  (Bransford,  Brown,  & Cocking,  2000).  With  prior  knowledge 
formulated  in  traditional  classroom  settings,  learners  taking  an  online  course  often  develop  implicit 
expectations  and  faulty  expectations  as  to  what  they  can  expect  based  on  their  prior  experience  in 
traditional  face-to-face  classroom  instruction  (Bigelow,  1999).  It  is  believed  that  a deeper  understanding  of 
students  initial  expectations  of  online  learning  may  be  helpful  in  addressing  the  erroneous  expectations 
leading  to  a higher  success  rate  with  novice  online  learners.  As  a result,  the  goals  of  this  study  are  to 
determine  student  expectations  of  an  online  collaborative  learning  environment,  as  well  as  to  examine  how 
students  adjust  their  expectations  based  on  a one-semester  experience. 


The  Study 

The  participants  of  this  study  consisted  of  seven  voluntary  graduate  students,  six  females  and  one 
male,  enrolled  in  a graduate  Web-based  course  on  Computer  Supported  Collaborative  Learning 
(CSCL2001).  The  CSCL  course  was  offered  to  both  on-campus  students  and  the  University  of  Texas 
System  TeleCampus  students.  The  course  was  previously  offered  in  a face-to-face  format  for  four  years. 
The  course  emphasized  collaborative  learning  with  students  working  together  in  virtual  learning  teams  in 
completing  learning  tasks  and  projects.  After  the  course  had  been  completed,  the  online  survey  was 
administered.  The  Student  Expectations  Questionnaire  consists  of  45  questions.  Of  the  45  questions,  43 
used  a Likert -type  response  scale  (strongly  disagree  to  strongly  agree).  The  other  two  questions  also  have 
five  options.  The  questionnaire  contains  two  response  columns:  Initial  Expectations  and  Present 
Expectations.  Each  column  provides  the  same  five-  point  Likert  scale  (strongly  agree  to  strongly  disagree). 
The  data  was  processed  using  the  SPSS.  To  obtain  an  insight  of  student  expectations,  the  entries  from 
students’  introduction  and  reflections  posted  on  Teach  net,  a primary  communication  tool  based  on 
FirstClass  Groupware  used  in  this  course,  were  also  exa  mined. 
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The  Findings 


According  to  students’  entries  posted  on  Teachnet,  students  came  to  the  course  with  an 
expectation  that  the  course  would  provide  them  with  in-depth  knowledge  of  the  course  subject  matter 
(CSCL)  with  hands  on  experience  in  the  online  learning  environment.  At  the  end  of  the  course,  several 
students  showed  their  satisfaction  with  the  course  organization.  In  their  final  reflection  papers,  most 
students  expressed  tbeii*  feelings  and  positive  impressions  towards  working  collaboratively  with  their 
online  team  members.  This  could  imply  that  their  expectations  from  the  course  activities/  course  content, 
course  design,  and  the  online  community  had  been  met,  or  have  even  been  exceeded.  Besides  reflecting  on 
their  expectations  and  perceptions  toward  the  course  and  the  online  community,  some  students  indicated 
the  need  for  immediate  feedback,  especially  when  they  had  problems  with  a task. 

The  results  revealed  that  students’  initial  expectations  and  their  expectations  when  the  course 
finished  were  different.  At  the  end  of  the  course  student  expectations  were  higher  in  most  aspects,  except 
for  their  expectations  of  the  instructor  and  the  level  of  effort  and  convenience.  The  three  expectation 
aspects  that  significantly  increased  at  the  end  of  the  course  were  expectations  of  the  online  community, 
learning  outcomes,  and  understanding.  In  contrast,  students  lowered  their  expectations  for  the  instructor  at 
the  end  of  the  course.  It  is  possible  that,  before  taking  the  course,  some  students  might  never  have  had 
experience  with  Web-based  collaborative  learning.  Thus,  they  came  to  class  with  high  expectations  based 
on  prior  knowledge  from  traditional  classrooms  and  expected  that  the  instructor  would  be  continuously 
available  and  give  them  immediate  feedback  all  the  time.  In  fact,  the  communication  patterns  in  this  course 
had  been  shifted  from  vertical  to  horizontal  communication.  That  is,  students  are  supposed  to  communicate 
with  and  learn  from  other  team  members  as  well  as  the  instructor,  rather  than  focusing  only  on  the 
instructor.  In  other  words,  the  instructor  takes  a supportive  but  less  directive  role  (Wegner  et  al.,  1999).  As 
a result,  online  students  might  feel  a sense  of  increased  distance  on  the  part  of  the  instructor. 


Conclusions 

The  findings  of  this  study  reflect  the  views  of  one  group  of  students  taking  a one  semester  Web- 
based  course.  Based  on  the  very  limited  sample,  it  is  not  possible  to  generalize  the  results  of  the  study  to 
the  broad  array  of  online  courses.  However,  the  survey  results  and  the  entries  from  student  reflections 
posted  on  the  course  bulletin  board  system  do  give  some  insights  into  what  expectations  students  brought  to 
the  online  course,  emphasizing  collaborative  learning,  and  the  differences  between  the  expectations  of 
students  before  and  after  the  course.  The  results  suggest  that,  the  students  had  higher  expectations  about 
online  learning  when  the  course  was  completed.  With  experience  in  online  learning,- students  also  gained  a 
greater  understanding  of  the  online  collaborative  learning  environment. 
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Abstract:  This  presentation  reports  on  our  current  and  on-going  R&D  efforts 
towards  the  design  and  development  of  an  instructional  system  that  supports 
individual  learning  preferences  through  an  adaptive  web-based  learning 
environment.  The  aim  of  the  system  is  to  fecilitate  individualized  instruction  by 
providing  both  content-level  as  well  as  link-level  adaptations  through  the 
exploitation  of  the  user's  particular  learning  style.  In  order  to  accomplish  this, 
different  learning  style  models  are  explored  and  compared.  These  learning  style 
models  have  been  formulated  and  used  in  classifying  students  according  to  their 
preferences.  The  implementation  of  the  adaptive  instructional  system  is  based  on  the 
use  of  learning  object  technology  on  the  one  hand  and  an  inference  engine  (JESS)  on 
the  other.  The  former  provides  the  framework  for  the  structuring  of  the  domain 
model  while  the  latter  renders  the  rules  upon  which  the  adaptation  is  defined. 


Introduction 

In  the  present  study  we  seek  and  analyze  state  of  the  art  learning  prescriptions  for  designing  an 
adaptive  web-based  environment,  which  is  a)  pedagogically  sound  through  the  exploitation  of  the 
student's  particular  learning  style  and  b)  economically  feasible  based  on  emerging  technologies  and 
notions  of  the  semantic  web. 

Research  has  shown  that  learning  is  more  likely  to  occur  if  instruction  is  matched  to  the  student's 
learning  style.  Examining  the  instructional  system  in  this  perspective,  the  notion  of  adaptiveness 
presupposes  that  the  system  will  be  able  to  initially  determine  and  subsequently  accommodate  the 
specific  learning  characteristics  of  each  learner. 

The  keywords  here  are  "determination"  and  "accommodation". 

• Which  are  the  elements  that  define  these  learning  styles  and  how  can  they  be  determined? 
Researchers  have  proposed  different  approaches  without,  however,  having  provided  statistically 
significant  evidence  to  date  as  to  how  effective  these  models  are. 

• How  should  the  word  "accommodation"  be  defined?  Do  students  learn  better  when  instruction  is 
adapted  to  their  learning  style  preferences  or  can  and  should  they,  at  certain  times,  be  trained  to 
adopt  a particular  learning  style?  Do  learning  styles  remain  stable  over  time  or  are  they  dynamic? 
Are  there  particular  learning  style  dimensions  that  remain  stable  and  others  that  are  modified  over 
time?  Which  are  these  and  how  should  the  system  deal  with  them? 

This  study  reviews  and  compares  prevalent  learning  style  models.  These  models  have  been  taken  into 
consideration  in  designing  the  system  prototype  of  the  web-based  instructional  system. 

Thus,  this  paper  discusses  our  efforts  towards  the  design  and  development  of  an  adaptive  instructional 
system  that  supports  individual  learning  preferences  through  a web-based  learning  environment.  The 
primary  characteristics  of  the  system  are  the  following: 

1 . It  adapts  instruction  to  user  learning  needs  according  the  particular  learning  style. 

2.  It  provides  adaptation  of  instruction  both  in  terms  of  the  content  presented  (content-level 
adaptation)  as  well  as  in  terms  of  the  navigation  of  the  instructional  material  (link-level 
adaptation). 

3.  The  key  architectural  components  of  the  system  are  its  domain  model,  its  learner  model  and  its 
inference  engine. 

4.  These  components  are  implemented  through  the  use  of  learning  object  technology  and  JESS  rules. 


The  role  of  learning  styles 


Learners  are  characterized  by  different  learning  styles.  Learning  styles  are  distinctive  strengths  and 
preferences  for  dealing  with  intellectual  tasks  in  a specific  way.  This  means  that  learners  receive  and 
process  information  in  different  ways,  preferentially  focus  on  different  types  of  information,  tend  to 
operate  on  perceived  information  in  different  ways,  and  achieve  understanding  at  different  rates  (Pask, 
1988;  Schmeck,  1988).  In  fact,  according  to  the  particular  learning  style  mode,  some  learners  may 
prefer  to  think  about  the  nature  of  a task  aud  detemiine  a detailed  plan  before  mo\dng  on  to  a specific 
course  of  action,  while  others  may  prefer  to  act  on  the  first  idea  they  have  and  see  where  it  leads  them. 
Some  students  prefer  focusing  on  data,  facts,  and  algorithms,  while  others  rely  more  on  theories  and 
mathematical  models.  Some  perceive  information  more  effectively  when  it  is  in  visual  form  (pictures, 
diagrams,  graphs,  demonstrations),  while  others  respond  better  to  verbal  information  (sounds  and 
spoken  words  or  formulas). 

Educational  researchers  have  been  concerned  with  the  implications  of  learning  styles  for  instruction  for 
many  decades  (Tobias,  1990;  Kolb,  1984;  Bruner,  1966;  Gagne,  1967).  Research  to  date  has  been 
attempting  to  identify  robust  relationships  between  learning  styles  and  instructional  methods  (Tobias, 
1990;  Yin,  2001;  Frick,  1990).  The  lack  of  statistically  significant  evidence  to  support  the  robustness  of 
these  relationships  may  mean  that  traditional  learning  style  models  need  to  be  revised  and  modified 
taking  into  consideration  new  learning  approaches  and  theories.  Once  this  is  achieved,  learning  style 
models  could  be  used  to  provide  a framework  for  designing  learning  environments  (whether  this  means 
designing  a course  curriculum,  creating  class  activities  or  assessment  methods,  writing  a textbook, 
developing  instructional  software,  or  designing  more  adaptive  instructional  systems)  that  will  be 
compatible  with  particular  learning  preferences,  strengths,  weaknesses,  and  behaviors  of  students. 

Learning  style  models 

Different  learning  style  models  have  been  formulated  and  have  been  used  in  classifying  students 
according  to  their  preferences  and  in  helping  educators  to  provide  students  with  instruction  that 
addresses  their  learning  styles.  The  authors  have  been  exploring  the  strengths  and  limitations  of  these 
models  in  order  to  exploit  those  particular  learning  style  dimensions  that  would  be  more  effective  in 
serving  as  the  basis  of  the  user  model  module  for  an  adaptive  instructional  system. 

One  of  the  first  learning  style  models  proposed  during  the  early  1960s  by  Jerome  Kagan  was  based  on 
the  dimension  of  reflectivity/impulsivity.  Kagan  suggested  that  some  students  seem  to  be 
characteristically  reflective  while  others  are  characteristically  impulsive.  Reflective  students  prefer  to 
collect  and  analyze  information  before  acting  whereas  impulsive  students  prefer  to  act  briskly. 

Herbert  Witkin  proposed  the  dimension  of  field-dependence/independence  for  classifying  students, 
referring  to  the  extent  to  which  an  individual’s  perception  and  thinking  about  a specific  piece  of 
information  is  influenced  by  the  surrounding  context.  Field-dependent  students  prefer  to  work  within 
existing  structure  whereas  field-independent  students  prefer  their  own  structure  and  they  are  more 
capable  of  isolating  target  information,  despite  the  fact  that  it  is  embedded  within  a larger  and  more 
complex  context.  The  importance  of  this  difference  in  approach  becomes  clear  when  the  student  is 
dealing  with  ill-structured  tasks.  Field-independent  students  usually  perform  better  in  these  situations 
because  of  their  willingness  to  create  a more  meaningful  structure. 

According  to  another  such  model,  formulated  by  Robert  Sternberg  (2000),  there  are  five  dimensions 
(functions,  forms,  levels,  scope,  and  leaning)  which  compose  the  learning  style  of  each  learner.  There 
are  legislative,  executive,  and  judicial  functions;  monarchic,  hierarchic,  oligarchic,  and  anarchic  forms; 
global  and  local  levels;  internal  and  external  scopes  and  liberal  and  conservative  leanings.  Most  people 
have  a preference  for  one  style  within  each  dimension.  For  example,  legislative  style  learners  prefer  to 
create  and  plan,  executive  style  learners  like  to  follow  explicit  rules,  while  judicial  style  learners  are 
keen  on  evaluating  and  judging. 

A learning  style  model  that  has  been  widely  used  is  Kolb’s  learning  style  model  (1984).  Kolb 
proposes  the  classification  of  students  in  terms  of  how  they  choose  to  take  in  information  (in  which 
case  they  may  prefer  either:  a)  concrete  experience  or  b)  abstract  conceptualization)  and  in  terms  of 
how  they  prefer  to  internalize  that  information  (in  which  case  they  may  have  a preference  for:  a)  active 


experimentation  or  b)  reflective  observation).  This  establishes  four  types  of  learners.  Type  A learners 
are  concrete  and  reflective  and  respond  well  to  explanations  of  how  course  material  relates  to  their 
experience,  their  interests,  and  their  future  careers.  To  be  effective  with  Type  A students,  the  instructor 
or  the  instructional  system  should  function  as  a motivator.  Type  B learners  are  abstract  and  reflective. 
These  learners  respond  well  to  information  presented  in  a logical,  organized  fashion  and  benefit  if  they 
have  time  for  reflection.  To  be  effective  with  this  group  of  learners  the  instructor  or  instructional 
system  should  fake  on  the  role  of  an  expert.  Type  C learners  who  are  abstract  and  active,  like  to  be 
given  opportunities  to  work  actively  on  well-defined  tasks  and  to  learn  by  trial-and-error  in  an 
environment  that  allows  safety  in  failing.  To  be  effective,  the  instructor  or  instructional  system  needs  to 
function  as  a coach,  providing  guided  practice  and  feedback.  Finally,  type  D learners  are  concrete, 
active  learners  and  like  applying  course  material  in  new  situations  to  solve  real  problems.  To  be 
effective,  these  students  need  to  be  given  opportunities  to  find  things  out  for  themselves,  maximizing 
their  time  and  freedom  for  exploration. 

The  Myers-Briggs  model  (Lawrence,  1994;  McCaulley,  1990)  based  on  psychologist  Carl  Jung's 
theory  of  psychological  types  classifies  students  according  to  their  preferences  into:  a)  extraverts  (who 
try  things  out  while  focussing  on  the  outer  world  of  people)  or  introverts  (who  think  things  through 
while  focussing  on  the  inner  world  of  ideas);  b)  sensors  (who  are  practical,  detail-oriented,  and  focus 
on  facts  and  procedures)  or  intuitors  (who  are  imaginative,  concept-oriented,  and  focus  on  meanings 
and  possibilities);  c)  thinkers  (who  are  skeptical,  tending  to  make  decisions  based  on  logic  and  rules)  or 
feelers  (who  are  appreciative,  tending  to  make  decisions  based  on  personal  and  humanistic 
considerations);  d)  judgers  (who  set  and  follow  agendas  and  seek  closure  even  with  incomplete  data)  or 
perceivers  (who  adapt  to  changing  circumstances  and  resist  closure  to  obtain  more  data).  The  Myers- 
Briggs  type  preferences  can  be  combined  to  form  16  different  learning  style  types.  For  example,  one 
student  may  be  an  EITJ  (extravert,  intuitor,  thinker,  judger)  and  another  may  be  an  ISFP  (introvert, 
sensor,  feeler,  perceiver). 

Another  learning  style  model,  accredited  to  Felder  and  Silverman  (1988),  the  Felder-Silverman 
learning  style  model  classifies  students  according  to  the  following  dimensions:  a)  the  type  of 
information  that  the  student  preferentially  perceives:  sensory — sights,  sounds,  physical  sensations, 
concrete,  facts  and  procedures  or  intuitive — memories,  ideas,  insights,  conceptual,  theories  and 
meanings;  b)  the  modality  through  which  the  sensory  information  is  most  effectively  perceived:  visual- 
—pictures,  diagrams,  graphs,  demonstrations,  or  verbal — sounds,  written  and  spoken  explanations  and 
formulas;  c)  the  type  of  information  organization  that  the  student  is  most  comfortable  with:  inductive — 
presentations  that  go  from  the  specific  to  the  general,  facts  and  observations  are  given,  underlying 
principles  are  inferred,  or  deductive — presentations  that  proceed  from  the  general  to  the  specific, 
principles  are  given,  consequences  and  applications  are  deduced;  d)  the  way  in  which  the  student 
prefers  to  process  information:  actively — by  trying  things  out,  by  becoming  engaged  in  physical 
activity  or  discussion,  or  reflectively — by  thinking  things  through,  by  working  alone;  e)  the  type  of 
progress  toward  understanding  that  the  student  is  more  capable  of  achieving:  sequentially — linear, 
orderly,  by  learning  in  small  incremental  steps  or  globally — in  large  leaps,  holistically.  The 
dichotomous  learning  style  dimensions  of  this  model  (sensing/intuitive,  visual/verbal, 

inductive/deductive,  active/reflective,  and  sequential/global)  should  not  be  considered  as  either/or 
categories  but  should  be  regarded  as  continua.  A student's  preference  on  a given  scale  (e.g.  for  visual  or 
verbal  presentation)  may  be  strong,  moderate,  or  almost  nonexistent,  may  vary  over  time,  and  may 
change  from  one  subject  or  learning  environment  to  another. 

The  Herrmann  Brain  Dominance  model  (1990)  classifies  students  in  terms  of  their  relative 
preferences  for  thinking  in  four  different  modes  based  on  the  task-specialized  functioning  of  the 
physical  brain.  The  four  modes  or  quadrants  in  this  classification  scheme  are:  a)  Quadrant  A (left  brain, 
cerebral):  logical,  analjdical,  quantitative,  factual,  critical;  b)  Quadrant  B (left  brain,  limbic): 
sequential,  organized,  planned,  detailed,  structured;  c)  Quadrant  C:  (right  brain,  limbic):  emotional, 
interpersonal,  sensory,  kinesthetic,  symbolic;  d)  Quadrant  D (right  brain,  cerebral):  visual,  holistic, 
innovative.  The  needs  of  students  with  strong  preferences  for  C and  D quadrant  thinking  are  often  not 
met  by  traditional  forms  of  classroom  instruction  or  by  instructional  systems  that  are  not  oriented 
toward  taking  these  particular  preferences  into  account. 

Discussion 
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The  above  learning  style  instruments  use  lists  of  verbal  statements  or  different  visual  forms  prepared 
for  people  to  choose  or  rank.  Our  initial  research  questions  related  to  the  determination  and 
accommodation  of  the  learner's  specific  style  coincides  with  some  of  the  issues  raised  by  Sewall 
(1986).  Sewall  compared  several  of  the  above  popular  instruments  and  found  that  their  scores  have 
limited  validity,  are  difficult  to  interpret,  and  that  their  reliability  estimates  are  unstable.  He  also  raised 
many  questions  such  as  the  issue  of  whether  learning  styles  are  constantly  modified  by  the  educational 
environment  and  social  context  or  are  too  dynamic  to  be  measured  on  a predetermined  and  predictable 
scale,  the  issue  of  whether  it  is  possible  for  learners  to  be  trained  to  adopt  a particular  learning  style, 
and  the  issue  of  whether  a significant  change  in  life  situations  result  in  changes  in  learning  styles. 
Researchers  dealing  with  learning  styles  will  need  to  explore  these  issues  in  order  to  assert  that  these 
learning  models  are  effective  tools  and  not  merely  artificial  instruments,  in  which  case  other  tools  that 
will  be  able  to  appropriately  survey  the  dynamic  nature  of  learning  styles  will  have  to  be  proposed. 
Yin  (2001),  suggests  that  these  predefined  assessment  tools  require  people  to  fit  in  the  limited  and 
selective  choices  with  “I  think  this  is  who  I am”  or  “I  think  this  is  what  I will  do  or  how  I will  react,” 
which  may  not  predict  how  one  actually  reacts  to,  or  interacts  with,  a problem  in  a specific  learning 
situation.  He  proposes  that  we  observe  and  faithfully  record  the  event  changes  of  a learner- 
demonstrated  process  over  time. 


Learning  Style  Model 

Dimensions 

Number  of  L.S 

Jerome  Kagan 

Process_info  (reflectivity,  impulsivity) 

2 

Herbert  Witkin 

Field  (dependence,  independence) 

2 

Robert  Sternberg 

Functions  (legislative,  executive,  judicial). 

Forms  (monarchic,  hierarchic,  oligarchic,  anarchic). 

Levels  (global,  local). 

Scope  (internal,  external). 
Leaning  (liberal,  conservative) 

96 

Kolb*s 

InpuUnfo  (concrete  experience,  abstract  conceptualization). 
Internal ize_info  (active  experimentation,  reflective  observation) 

4 

Myers-Briggs 

Dimension  1 (extraverts,  introverts), 
Dimension2  (sensors,  intuitors), 
Dimension3  (thinkers,  feelers), 
Dimension4  (judgers,  perceivers) 

16 

Felder-Silverman 

Perce ive_Info  (sensory,  intuitive^ 
Modality  (visual,  verbal) 

Organizejnfo  (inductive,  deductive) 
Process_Info  (actively,  reflectively) 
Understand_Info  (sequentially,  globally) 

32 

Herrmann 

Brain_mode  (QuadrantA,  QuadrantB,  QuadrantC,  QuadrantD) 

4 

Table  1 : Learning  Style  Models  and  Dimensions 


Learning  Orientations 

We  would  consider  the  previous  discussion  on  learning  styles  incomplete  without  touching  upon  the 
idea  of  learning  orientations,  a different  classification  scheme  for  students  introduced  by  Margaret 
Martinez  (2001).  Without  disregarding  the  importance  of  learning  styles,  Martinez  considers  them  to 
be  of  secondary  importance  since  they  focus  only  on  cognitive  factors  of  learning  (i.e.  on  how  learners 
receive,  process,  build  and  store  knowledge)  and  overlook  emotions  and  intentions  as  being  equally 
important  factors  in  the  learning  process.  For  Martinez  the  basis  for  providing  individualized 
instruction  lies  in  what  she  states  as  “a  whole-person  understanding  about  key  psychological  sources 
that  influence  how  individuals  want  and  intend  to  learn  online.” 

The  primary  difference  between  learning  styles  and  learning  orientations  is  that  whereas  learning  styles 
recognize  the  dominant  power  of  cognitive  factors  (and  degrade  other  factors  to  a secondary  or  no 
role),  learning  orientations  recognize  the  dominant  influence  of  emotions  and  intentions.  As  Martinez 
states,  this  perspective  reflects  recent  neurological  research  that  provides  evidence  for  the  dominant 
influence  of  the  brain’s  emotional  center  on  learning  and  memory  (Ledoux,  1996). 

According  to  the  Learning  Orientations  model,  learners  can  be  classified  into  four  types: 

Type  A:  Transforming  learners,  who  are  generally  highly  motivated,  passionate,  and  highly  committed 
learners.  These  learners  respond  better  to  discovery-oriented,  ensequenced,  and  mentoring 
environments.  Type  B:  Performing  Learners,  who  are  generally  self-motivated  in  learning  situations 
(task-oriented,  project-oriented,  hands-on  applications)  that  interest  them.  These  learners  respond  well 
in  task-  or  project-oriented,  competitive,  and  interactive  (hands-on)  environments.  Type  C:  Conforming 
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Learners,  who  are  generally  more  compliant  and  will  passively  accept  knowledge,  store  it,  and 
reproduce  it  to  conform,  complete  routine  or  assigned  tasks  (if  they  can),  and  please  others.  They  need 
scaffolded,  structured  solutions,  guiding  direction,  simple  problems,  linear  sequencing,  and  explicit 
feedback.  Type  D:  Resistant  Learners,  who  lack  a fundamental  belief  that  academic  learning  and 
achievement  can  help  them  achieve  personal  goals  or  initiate  positive  change. 


Prototype  and  implementation  issues 

The  above  analysis  has  been  used  as  the  basis  for  designing  the  adaptive  elements  of  the  system 
functioning  within  a Semantic  Web  environment.  The  Semantic  Web  is  an  ideal  environment  for 
personalized  learning  especially  when  combined  with  the  use  of  learning  objects.  Learning  objects 
were  used  for  the  design  of  the  present  adaptive  web-based  instructional  system. 

A learning  object  is  defined  as  an  entity  that  can  be  used,  reused,  or  referred  to  as  a learning  resource 
by  a computer-based  environment.  Learning  objects  (LTSC,  2000a)  are  considered  to  be  the 
predominant  choice  in  terms  of  the  pedagogical  design,  implementation  and  distribution  of  educational 
content  due  to  their  ability  to  provide  reusability,  productivity,  adaptivity,  and  scalability. 

The  personalization  of  the  system  takes  many  forms  as  it  adapts  content,  tasks,  feedback,  or  navigation 
to  match  individual  progress  and  performance.  For  example,  two  individuals  using  the  same  course 
simultaneously  may  see  two  completely  different  sets  of  learning  objects.  The  greatest  benefit  of 
learning  personalization  is  the  ability  of  the  system  to  make  complex  instruction  easier  by  presenting 
only  suitable  information  that  a particular  learner  wants  or  needs  according  to  specific  preferences 
expressed  by  the  learning  style. 


The  adaptive  instructional  system  consists  of  the  following  modules: 

(a)  The  knowledge  of  the  system  is  stored  in  a database  (DB)  and  is: 

(al)  Knowledge  regarding  the  learner.  This  knowledge  comprises  the  user  model  and  includes 
knowledge  corresponding  to  the  learner’s  knowledge  level  as  well  as  knowledge  related  to  the 
characteristics  of  the  learner’s  learning  style. 

(a2)  Knowledge  related  to  the  domain  that  the  learner  is  learning  about  (i.e.  the  domain  model). 

(b)  The  monitoring  and  evaluation  modules.  During  the  course  of  a lesson  the  monitoring  and 
evaluation  modules  examine  the  learner  responses  and  behavior  and  renew  the  DB  data  (user 
model)  in  terms  of  the  learner  knowledge  level  and  learning  style.  It  then  gives  the  output  variables 
“learning  style”  and  “knowledge  level”  to  the  lesson  creation  module. 

(c)  The  instructional  strategies  module.  This  contains  a set  of  rules  corresponding  to  the  instructional 
method  that  will  be  applied.  The  rules  have  been  derived  from  the  fields  of  didactics  and  cognitive 
psychology. 

(d)  The  lesson  creation  module.  This  module  receives  the  input  variables  “learning  style”  and 
“knowledge  level”,  and  composes  the  lesson  that  will  be  presented  to  the  student  by  selecting  the 
appropriate  educational  content  fi*om  the  DB  and  a suitable  strategy  fi*om  the  instructional 
strategies  module.  The  lesson  creation  module  determines  the  form  and  arranges  the  sequencing  of 
the  educational  content. 

The  intelligence  of  the  instructional  system  is  derived  fi*om  its  capability  to: 

(a)  Diagnose  the  knowledge  level  and  learning  style  of  the  student,  based  on  the  data  in  the  user  model 
and  according  to  the  learner’s  responses  and 

(b)  Adapt  the  lesson  appropriately  to  the  student’s  needs  by  drawing  suitable  educational  content  from 
the  domain  model,  by  using  the  proper  instructional  strategy,  and  by  determining  the  form  and 
sequencing  of  the  educational  content. 

The  underlying  technology  is  based  mainly  on  freely  available  software  components:  an  Apache 

Webserver,  an  Access  (Microsoft  Corp.)  database  management  system,  an  Apache  Java  Servlet  engine 

(Tomcat  version  3.2.1)  and  the  Jess  scripting  environment,  which  is  a rule  engine  written  entirely  in 

Sun’s  Java  language. 


Server 

- Learning  Objects 

Servlet  Engine 

- Jess  (inference 
engine) 

- Knowledge  Base 


Figure  1 : The  modules  of  the  system  and  the  way  they  communicate 
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Future  research 


Our  priorities  for  future  studies  are  to  attempt  to  reconsider  and  redefine  the  concept  of  learning  styles. 
In  order  to  extend  the  presently  implemented  system,  which  is  based  on  existing  theories  and  models, 
we  should  consider  future  research  (both  on  a theoretical  as  well  as  an  experimental  level)  regarding: 

1 . The  suitability  of  the  questionnaires  corresponding  to  the  determination  of  the  learning  styles.  Are 
these  questionnaires  appropriate  and  to  what  degree  can  they  be  interpreted  reliably  by  the  system? 

2.  The  particular  implementation  of  the  system  (currently  based  on  the  use  of  servlets)  and  extending 
ihe  way  that  it  presently  allows  for  the  progress  of  the  .students  to  be  recorded  correctly.  How 
could  the  recording  of  these  events  lead  to  the  definition  of  learning  styles? 

3.  The  stability  of  learning  styles  over  time.  What  is  the  stability  of  each  learning  style  dimension 
over  time? 

4.  The  issue  of  adapting  the  educational  environment  according  to  the  learning  style  of  the  student.  Is 
this  always  necessary  or  would  it  be  beneficial  if,  at  certain  times,  the  system  trained  the  learner  to 
adopt  a different  learning  style  imposed  by  the  nature  of  the  educational  content  or  the  particular 
learning  goals?  How  would  it  be  possible  to  achieve  this? 

Obviously,  many  research  studies  to  date  have  attempted  to  deal  with  issues  related  to  learning  styles. 
However,  on  the  one  hand  these  studies  have  not  yielded  statistically  significant  results  and  on  the 
other  the  web  has  emerged  as  a new  environment  with  unique  characteristics,  which  need  to  be 
explored,  since  it  is  expected  to  play  a particularly  important  role  in  education. 
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Utilizing  SMARTBoards  to  Enhance  Technology  Integration  in  University 

Classrooms 


Abstract 

This  paper  presents  the  preliminary  results  of  a university-wide  effort  to  enhance  technology  integration  in  the  classroom. 
The  College  of  Education  at  Idaho  State  University  has  received  a grant  to  disseminate  SMARTBoards,  computers  and  projectors  to 
all  seven  colleges  within  the  university,  to  train  faculty  in  the  use  of  the  equipment,  to  provide  support  and  ideas  for  implementing 
the  new  technology,  and  to  collect  information  on  faculty  and  student  attitudes  towards,  and  use  of,  the  technology.  It  is  projected 
that  the  use  of  a non-threatening  but  extremely  flexible  digital  technology,  like  the  SMARTBoard,  will  encourage  faculty  to  develop 
their  own  original  digital  classroom  materials,  to  incorporate  computer  technology  on  a regular  basis,  and  to  encourage  pedagogical 
changes  in  higher  education  classrooms. 

Introduction 

The  College  of  Education,  Idaho  State  University,  received  a State  Board  of  Education  Technology  Incentive  Grant  to  place 
SMARTBoards  in  classrooms  in  each  of  the  seven  colleges.  The  purpose  of  the  project  is  to  facilitate  the  integration  of  technology 
into  higher  education  curricula,  to  enhance  faculty  productivity,  to  amplify  the  rate  and  quality  of  student  learning,  and  to  increase 
classroom  access  to  educational  programs.  With  a user-friendly  delivery  medium  such  as  the  SMARTBoard,  it  is  hoped  that 
university  faculty  will  become  more  motivated  to  create  their  own  applications,  including  web  courses,  original  digital  graphics  and 
movies,  electronic  presentations  and  CD-ROMs.  The  project  was  begun  in  the  fall  of  2001  and,  although  not  yet  complete,  has 
garnered  preliminary  indications  of  faculty  skills  and  predisposition  toward  computer  technology. 

The  SMARTBoard  is  an  interactive  surface,  which  acts  as  both  a large  computer  touch  screen  and  whiteboard.  When  the  computer 
desktop  is  projected  onto  the  SMARTBoard,  the  Board  itself  becomes  the  desktop  and  the  user  is  able  to  open  programs,  type,  select, 
and  move  objects  by  touching  the  board.  The  project  is  based  on  the  premise  that  interactive  digital  boards  are  a convenient  and 
comfortable  way  to  reach  instructors  who  may  have  resisted  integrating  technology  in  the  past.  Since  the  SMARTBoard  requires 
minimal  training  and  appears  non-threatening,  it  is  not  as  intimidating  as  other  forms  of  technology.  It  is  one  of  the  few  innovations 
that  do  not  place  a barrier  of  equipment  between  the  instructor  and  the  student.  The  Smart  Board  enhances  the  instructor’s  role  in  the 
classroom. 

Bates  (2000)  argued  that  since  faculty  is  critical  to  the  instructional  praxis  of  the  university,  any  change  in  the  core  activity  of 
teaching  has  to  have  faculty  approval  in  order  to  succeed.  Therefore,  the  integration  of  technology  in  the  university  classroom  cannot 
be  imposed  on  faculty  if  it  is  to  succeed.  However,  adapting  computer  technology  into  the  classroom  may  suggest  a reversal  of 
teacher-centered  instruction,  and  this  change  process  is  critical  to  the  successful  adoption  of  technology  by  faculty.  Bates  identified 
several  factors,  which  inhibited  faculty’s  adoption  of  computer  technology  in  the  classroom,  including  anger,  resistance,  grieving  for 
the  old,  cautious  adoption,  total  disbelief,  and  fear.  Dusick  (1998)  also  identified  factors  that  influence  faculty  members’  decision  not 
to  use  computers  for  teaching,  including  environmental  factors  outside  the  faculty’s  control  and  anxiety.  Other  environmental  factors 
encourage  the  use  of  technology  for  teaching:  a supportive  administration,  availability  of  the  requisite  hardware  and  software,  a 
supportive  technical  staff,  and  positive  personal  and  social  elements.  Both  Dusick  and  Bates  argue  that  training  in  technology 
integration  is  one  way  to  overcome  faculty  fear. 

This  project  sought  to  address  some  of  these  factors  in  order  to  facilitate  faculty  technology  integration.  SMARTBoards  were 
chosen  in  order  to  introduce  faculty  to  technology  that  was  not  only  user  friendly,  but  which  also  lends  itself  well  to  most  classroom 
uses  without  drastically  altering  the  instructor’s  role.  Faculty  training  on  the  SMARTBoard  was  carried  out  in  a non-threatening  way 
and  with  plenty  of  opportunities  for  practical  hands-on  experiences.  The  training  sought  to  convince  faculty  that  using  the 
SMARTBoard  could  enhance  both  teaching  and  student  learning.  The  principle  investigators  argued  that,  if  faculty  members  believed 
that  computer  technologies  did  indeed  make  a difference  to  classroom  teaching,  then  they  were  more  likely  to  integrate  them  on  a 
more  permanent  basis  (Woodrow  1991).  If,  on  the  other  hand,  faculty  saw  technology  in  teaching  as  a fad,  they  were  more  likely  to 
adopt  it  just  to  conform  and  not  take  it  seriously. 

The  project  also  seeks  to  determine  if  faculty  members’  attitudes  and  perceptions  toward  computer  technology  in  teaching  will 
change  as  a result  of  using  SMARTBoards. . Straub  and  Karahana  (1999)  argue  that  knowing  the  distinction  between  pre-  and  post- 
adoption  beliefs  and  attitudes  is  important  to  understanding  and  managing  the  process  of  technology  integration.  To  fulfill  this  goal, 
faculty  and  students  are  completing  four  separate  evaluations.  The  first  two  instruments  are  an  evaluation  of  the  faculty  training 
workshops  and  an  attitudinal  and  skills  survey  (reported  here). 

Preliminary  Results 

As  of  this  writing,  75  members  of  the  ISU  professional  community  have  participated  in  the  SMARTBoard  training, 
preparatory  to  using  the  technology  in  the  classroom.  Of  these,  61  identify  themselves  by  faculty  rank  (19  Full,  1 1 Associate,  18 
Assistant  Professors,  and  13  Instructors);  the  other  14  are  staff  positions,  graduate  students,  or  teaching  assistants.  The  initial 
attitudinal  survey  indicates  that  many  of  these  trainees  already  exhibit  a moderate  to  strong  disposition  toward  integrating  technology 
in  the  classroom.  Most  respondents  (more  than  75%)  indicated  that  they  were  comfortable  using  computers  in  instruction,  that  they 
believed  technology  could  enhance  learning,  and  that  they  wanted  to  learn  more  about  integrating  technology  into  their  instructional 
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methods.  A slightly  smaller  majority  disagreed  that  traditional  lecture  methods  of  teaching  were  preferable,  or  that  computer 
technology  had  no  place  in  their  discipline.  Almost  all  the  participants  indicated  that  they  used  both  the  Internet  and  email  in 
teaching,  but  less  than  half  were  using  WebCT  (the  on-line  curriculum  program  used  at  ISU). 

In  terms  of  their  skills,  this  faculty  considered  themselves  as  intermediate  or  advanced  users  of  the  Internet  and  word 
processing.  A sizable  minority  considered  their  skill  level  as  low  on  electronic  presentations.  Approximately  half  considered 
themselves  as  novices  (or  less)  in  terms  of  statistical  software,  spreadsheets,  databases,  and  subject  specific  software.  The  majority 
rated  their  skill  level  as  none  or  novice  on  the  SMARTBoard  technology,  multimedia  authoring  software,  and  WebCT.  Thus  it 
appears  that,  if  these  faculty  are  indeed  comfortable  integrating  technology  into  their  teaching  methods,  the  technology  they  use  is 
primarily  confined  lu  wuiu  piocessing  and  the  Internet,  with  PowerPoint  skill  levels  still  being  developed.  However,  more  “difficult” 
applications,  or  applications  which  have  not  been  traditionally  presented  in  direct-instruction  environments,  are  not  usually  part  of  the 
faculty’s  technology  repertoire. 

In  order  to  understand  how  faculty  determines  whether  or  not  they  can  commit  to  learning  a new  technology,  they  were 
queried  as  to  which  factors  they  considered  important.  Although  all  of  the  factors  were  rated  as  important  or  very  important  by  most 
participants,  there  was  a pattern  as  to  which  of  these  factors  were  rated  as  very  important  by  increasing  numbers  of  participants. 
Figure  1 illustrates  those  factors,  which  were  rated  as  very  important.  Surprisingly,  faculty  rated  the  simple,  practical  factor  of 
Availability  as  Very  Important,  more  than  any  other  factor.  Time  to  devote  to  the  technology  and  Ease  of  Use  were  also  rated  as  Very 
Important  by  a large  number  of  respondents. 


Figure  1.  Number  of faculty  rating  specific  factors  as  Very  Important  to  their  integration  of  technology  into  classroom  teaching.  1. 
Learning  Curve;  2.  Personal  Comfort  Level  with  Software;  3.  Advantage  of  Technology  over  Traditional  Teaching  Methods;  4. 
Compatibility  with  Discipline;  5.  Administrative  Support;  6.  Time;  7.  Ease  of  Use;  and  8.  Availability. 


Conclusion: 

Although  this  project  is  on  going,  some  interesting  trends  are  already  identified  in  the  data  collected.  The  most  important 
appears  to  be  that,  although  faculty  are  well  disposed  toward  using  computer  technology  in  their  communication  with  students  and  in 
teaching,  they  actually  have  limited  skills  and  confine  their  use  to  email,  the  Internet,  and  word  processing.  Only  a few  produce 
electronic  presentations  for  the  classroom,  and  fewer  still  use  the  capabilities  of  the  web  to  post  presentations  or  other  material  for 
their  students. 

It  is  hoped  that  the  SMARTBoard  project  will  encourage  faculty  to  expand  their  technological  skills,  producing  original 
web  sites,  multimedia  projects,  digital  video  and  audio,  and  other  electronic  media  to  incorporate  into  their  instruction.  Whether  the 
project  succeeds  will  be  determined  by  faculty  and  student  evaluation  of  courses,  which  are  held  in  SMARTBoard-equipped 
classrooms  by  project-trained  faculty.  These  final  evaluations  and  surveys  will  be  conducted  at  the  end  of  spring  semester  2002  and 
again  in  fall  semester  2003. 
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Abstract.  This  paper  investigates  the  accommodation  of  learning  styles  research  in  personalised  learning  (PL) 
systems.  The  paper  outlines  some  of  the  most  well-known  learning  styles  theories  and  models,  as  well  as  some 
criteria  for  selecting  among  them.  It  also  outlines  some  PL  systems  which  are  utilising  learning  styles  research, 
with  emphasis  on  the  system  which  is  being  developed  h the  context  of  the  KOD  “Knowledge  on  Demand” 
European  Project. 


Personalised  learning  (PL)  systems  are  attracting  increasing  interest,  since  they  bare  the  potential  to 
meet  the  requirements  of  the  knowledge  society  and  knowledge-based  economy  for  high-quality  education  and 
training.  PL  systems  can  be  defined  by  their  capability  to  automatically  and  continuously  adapt  to  the  changing 
attributes  of  the  “learning  context”,  which  can,  in  turn,  be  defined  by  the  individual  learner  characteristics,  the 
type  of  the  educational  material,  etc. 

In  the  context  of  this  paper,  PL  systems  are  categorised  and  differentiated  in  terms  of  their  adaptation 
logic,  which  is  defined  by: 

• PL  constituents:  the  aspects  of  the  learning  context  which  are  subject  to  adaptations;  that  is,  is  the 
educational  content  being  adapted?  and  if  so,  how  do  we  categorise  educational  content  so  that  we  can 
select  it? 

• PL  determinants:  the  aspects  of  the  learning  content  which  “drive”  adaptations;  that  is,  are  adaptations 
based  on  the  learner’s  profile?  and  if  so,  how  is  the  learner  profile  defined? 

• PL  rules:  the  rules  which  define  which  PL  constituents  are  selected  for  different  PL  determinants 
(Sampson,  Karagiannidis,  & Kinshuk,  2002). 

PL  systems  can  be  quite  diversified  according  to  their  adaptation  logics,  depending  on  the 
requirements  of  the  specific  learning  context.  For  example,  PL  determinants  can  include  learners’ 
characteristics,  which  can,  in  turn,  include  learner’s  background,  expertise,  prior  knowledge,  skills, 
requirements,  preferences,  etc. 

This  paper  addresses  the  incorporation  of  learning  styles  research  in  the  adaptation  logic  of  PL 
systems.  That  is,  the  definition  of  new  PL  determinants,  constituents  and  rules  which  are  based  on,  and  reflect 
specific  learning  styles  theories  and  models.  The  next  section  provides  a short  overview  of  the  most  well-known 
learning  styles  theories  and  models,  as  well  as  some  criteria  for  selecting  among  them  when  developing  a PL 
system.  Finally,  the  paper  outlines  some  existing  PL  systems  vhich  utilise  learning  styles  research,  with 
emphasis  on  the  PL  system  which  is  being  developed  in  the  context  of  the  KOD  “Knowledge  on  Demand” 
European  project  (see  acknowledgements  section). 

Learning  Styles  Research:  A Brief  Overview 

There  is  no  single  way  to  describe  learning  styles,  as  a number  of  definitions  appear  in  the  literature. 
Learning  styles  can  be  generally  described  as  “an  individual’s  preferred  approach  to  organising  and  presenting 
information”  (Riding  & Rayner,  1998);  “the  way  in  which  learners  perceive,  process,  store  and  recall  attempts 
of  learning”  (James  & Gardner,  1995);  “distinctive  behaviours  which  serve  as  indicators  of  how  a person  learns 
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from,  and  adapts  to  his/her  environment,  and  provide  clues  as  to  how  a person’s  mind  operates”  (Gregorc, 
1979);  “a  gestalt  combining  internal  and  external  operations  derived  from  the  individual’s  neurobiology, 
personality  and  development,  and  reflected  in  learner  behaviour”  (Keefe  & Ferrell,  1990). 

Existing  learning  styles  models  can  be  presented  through  an  onion  metaphor  (proposed  in  Curry, 
1983),  consisting  of  three  basic  layers  which  categorise  learners  in  terms  of  instructional  preferences  (outermost 
layer),  information  processing  (middle  layer)  and  personality  (innermost  layer).  Social  interaction,  a fourth 
layer  placed  between  Curry’s  two  outer  layers,  was  proposed  in  (Claxton  & Murrell,  1987). 

The  mo.st  well-known  and  used  learning  styles  theories  and  models  are  presented  in  Table  1.  For  each 
model,  the  presentation  includes 

• the  learners  categorisations  proposed  by  each  model, 

• the  existence  of  an  assessment  instrument  for  categorising  each  learner  in  the  above  categories,  and 

• indicative  references  for  each  model. 


Name 

Learners*  Categorisation 

Assessment  Instrument 

References 

Kolb  Learning 
Style  Inventory 

Divergers  (concrete,  reflective), 
Assimilators  (abstract,  reflective). 
Convergers  (abstract/active), 
Accommodators  (concrete/active) 

Learning  Style  Inventory  (LSI),  consisting 
of  12  items  in  which  subjects  are  asked  to 
rank  12  sentences  describing  how  they 
best  learn. 

Kolb,  1984;  Kolb, 
1985 

Dunn  and  Dunn 
- Learning  Style 
Assessment 
Instrument 

Environmental,  Emotional, 
Sociological,  Physical  factors. 

(i)  Learning  Style  Inventory  (LSI) 
designed  for  children  grade  3-12;  (ii) 
Productivity  Environmental  Preference 
Survey  (PEPS)  - adult  version  of  the  LSI 
containing  100  items 

Dunn  & Dunn, 
1978;  Dunn  & 
Dunn,  1999 

Felder-Silverman 
- Index  of 
Learning  Styles 

Sensing-intuitive,  Visual-verbal, 
Indicative-deductive,  Active- 
reflective,  Sequential-global 

Soloman  and  Felder  questionnaire, 
consisting  of  44  questions 

Felder,  1996; 
Felder  & 
Silverman,  1988 

Riding  - 
Cognitive  Style 
Analysis 

Wholists-Analytics,  Verbalisers- 
Imagers 

CSA  (Cognitive  Styles  Analysis)  test, 
consisting  of  three  sub  tests  based  on  the 
comparison  of  the  response  time  to 
different  items 

Riding  & Cheema, 
1991 ; Riding,  1994 

Honey  and 
Mumford  - 
Learning  Styles 
Questionnaire 

Theorist,  Activist,  Reflector, 
Pragmatist 

Honey  & Mumford’ s Learning  Styles 
Questionnaire  (LSQ),  consisting  of  80 
items  with  true/false  answers 

Honey  & 
Mumford,  1992 

Gregoric  - Mind 
Styles  and 
Gregoric  Style 
Delineator 

Abstract  Sequential,  Abstract 
Random,  Concrete  Sequential, 
Concrete  Random 

Gregoric  Style  Delineator  containing  40 
words  arranged  in  10  columns  with  4 
items  each;  the  leaner  is  asked  to  rank  the 
words  in  terms  of  personal  preference 

Gregoric,  1979; 
Gregoric,  1982 

McCarthy  - 4 
Mat  System 

Innovative,  Analytic,  Common 
sense.  Dynamic 

- 

McCarthy,  1980; 
McCarthy,  1997 

Gardner  - 
Multiple 
Intelligence 
Inventory 

Linguistic,  Logical-  mathematical. 
Musical,  Bodily-kinesthetic, 
Spatial,  Interpersonal, 
Intrapersonal 

an  instrument  consisting  of  8 questions 

Gardner,  1993a; 
Gardner,  1993b 

Grasha- 
Riechmann  - 
Student  Learning 
Style  Scale 

Competitive-Collaborative, 
Avoidant -Participant,  Dependent- 
Independent. 

90  items  self-report  inventory  measuring 
the  preferences  of  both  high  school  and 
college  students 

Hruska-Riechmann 
& Grasha,  1982; 
Grasha,  1996 

Hermann  - Brain 

Dominance 

Model 

Quadrant  A (left  brain,  cerebral). 
Quadrant  B (left  brain,  limbic). 
Quadrant  C (right  brain,  limbic). 
Quadrant  D (right  brain,  cerebral) 

120  questions  that  refer  to  four  profile 
preferences  codes  corresponding  to  each 
quadrant 

Herrmann,  1982; 
Herrmann,  1995 

Mayers-Briggs  - 
Type  Indicator 

Extroversion,  Introversion, 
Sensing,  Intuition,  Thinking, 
Feeling,  Judgement,  Perception 

(i)  MBTI  (Myers-Briggs  Type  Indicator), 

(ii)  Kiersey  Temperament  Sorter  I,  and 

(iii)  Kiersey  Character  Sorter  II 

Myers  & Kirby, 
1994;  Myers,  et  al, 
1998 

Table  1:  Overview  of  Learning  Styles 
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Some  Criteria  for  Selecting  Among  Different  Learning  Style  Models  in  PL  Systems 

Given  the  variety  of  learning  styles  theories  and  models  that  are  available  in  the  related  literature,  we 
need  to  define  a set  of  selection  criteria  for  selecting  the  most  appropriate  learning  style  model  to  be 
accommodated  in  a specific  PL  system.  Of  course,  the  most  important  criteria  are  the  theoretical  and  empirical 
justification  of  the  model.  In  addition,  the  learning  style  model  should  be  suitable  for  the  specific  learning 
context.  For  example,  if  all  learners  of  a specific  learning  context  are  “experts”  in  the  domain  (e.g.  an 
educational  application  for  aircraft  pilots),  then  it  might  not  be  reasonable  to  select  a learning  style  model  which 
categorises  learners  according  to  their  expertise  in  the  domain.  Similarly,  if  all  the  educational  material  that  is 
available  for  a specific  case  is  in  textual  form,  then  it  is  not  reasonable  to  select  a model  which  differentiates 
content  according  to  its  medium. 

A set  of  additional  important  selection  criteria  are  briefly  summarised  below. 

Measurability 

We  need  to  be  able  to  “measure”  whether  learners  belong  to  the  categories  defined  by  each  model.  For 
example,  one  model  may  differentiate  learners  according  to  their  emotions.  While  this  may  be  reasonable  from 
a theoretical  point  of  view,  since  emotions  can  affect  learning,  it  may  not  be  reasonable  to  select  such  a model 
for  a PL  system,  since  it  may  be  difficult  to  measure  learners*  emotions.  Felder  and  Silberman  model,  fir 
instance,  is  supported  by  the  Felder  and  Solomon  questionnaire,  which  easily  determines  how  a learner  is 
categorised  according  to  the  dimensions  proposed  by  the  model.  The  lack  of  such  an  assessment  instrument 
(questionnaire)  can  be  a reason  fornot  selecting  one  model. 

Time  effectiveness 

The  assessment  instrument  related  to  each  learning  style  model  needs  to  include  a reasonable  number 
of  questions  in  order  to  be  time  effective.  For  example,  if  an  assessment  instrument  consists  of  200  questions, 
then  the  instrument  may  not  be  effective  time  wise.  The  user  may  not  be  willing  to  dedicate  his/her  time  in 
order  to  complete  a large  questionnaire  before  starting  using  the  system. 

Descriptiveness  and  Prescriptiveness 

From  a practical  point  of  view,  it  is  important  that  the  model  describes  not  only  how  learners  are 
categorised,  but  also  how  instruction  should  be  adapted  for  each  learner  category;  that  it,  apart  from  the 
descriptive  information  (e.g.  learners  are  categorised  into  “active**  and  “reflective**),  the  model  should  provide 
prescriptive  guidelines,  which  can  lead  to  specific  rules  for  designing  instruction  and  adaptation  (e.g.  what 
types  of  educational  content  should  be  selected  for  active  and  reflective  learners). 

Cost 


The  cost  of  a learning  style  model  along  with  its  assessment  instrument  is  another  parameter  that 
system  designers  may  need  to  consider.  The  situation  here  varies,  as  some  assessment  instruments  are  only 
available  for  use  after  payment,  while  others  are  available  to  be  used  free-of-charge.  In  this  case  designers  need 
to  consider  the  cost  of  the  model  and  its  assessment  instrument.  Then,  the  availability  of  the  test  in  relation  to 
the  number  of  users  needs  to  be  considered. 


Some  Examples  of  Accommodating  Learning  Styles  Research  in  PLSystems 

Learning  styles  research  has  formed  the  basis  for  the  development  of  a number  of  PL  systems. 
TrainingPlace.com  is  a notable  example  of  a commercial  PL  system  which  is  based  on  learning  styles  research. 
The  system  is  based  on  Learning  Orientation  Theory,  which  categorises  learners  as  transforming,  performing, 
conforming  and  resistant.  Based  on  this  categorisation,  the  system  presents  different  “learning  experiences**  to 
each  learner.  For  example,  the  system  selects  “loosely  structured  environments  that  promote  challenging  goals, 
discovery  and  self-managed  learning**  for  transforming  learners,  and  “semi-complex,  semi  -structured,  coaching 


1755 


environments  that  stimulate  personal  value  and  provide  creative  interaction”  for  performing  learners  (Martinez 
& Bunderson,  2000). 

SMILE,  a web-based  knowledge  support  system  aiming  at  promoting  intelligent  support  for  dealing 
with  open-ended  problem  situations,  utilises  a learner  profile  which  takes  into  consideration  the  learner’s 
learning  style,  following  Honey  and  Minford’s  categorisation  (Stoyanov  & Kommers,  1999).  The  same 
learning  style  model  is  also  used  by  the  INSPIRE  system,  which  aims  to  generate  different  lessons  for  each 
individual  learner,  for  meeting  his/her  learning  goals  (Grigoriadou  et  al,  2001).  The  3DE  European  Project 
(www.3deproject.com)  categorises  learners  into  activists,  reflectors,  theorists  and  pragmatists,  in  order  to  create 
courses  customized  to  their  needs. 

The  KOD  European  project  aims  to  deliver  an  adaptive  learning  environment  for  personalised  learning 
(Karagiannidis,  Sampson,  & Cardinali,  2001).  In  this  context,  the  aim  of  the  project  is  to  facilitate  the 
development  of  adaptive  educational  content  which  can  be  easily  interchanged  and  re-used  across  different  e- 
learning  applications  and  services.  In  particular,  the  KOD  project  is  working  on  the  knowledge  packaging 
format  (an  extension  of  the  existing  IMS  Content  Packaging  Specification  (IMS,  2001a),  for  the  description,  in 
a common  format,  of  knowledge  packages  (i.e.  collections  of  learning  objects),  together  with  the  rules  which 
determine  which  learning  objects  should  be  selected  for  different  learner  profiles.  As  a result,  the  KOD  o 
learning  system  (or  any  system  compliant  with  the  knowledge  packaging  format),  can  import  a knowledge 
package  (a  collection  of  learning  objects  described  through  the  knowledge  packaging  format),  interpret  the  rules 
included  in  it,  and  present  different  knowledge  routes  to  each  individual  learner,  according  to  his/her  profile, 
thus  facilitating  personalised  learning. 

The  KOD  project  includes  an  authoring  environment  (the  KOD  Packager)  for  describing  adaptive 
educational  content  through  the  knowledge  packaging  format.  Through  the  KOD  Packager,  the  user  (learning 
material  author,  tutors,  publisher,  etc)  can  define  the  PL  logic  (determinants,  constituents  and  rules)  which  drive 
the  personalisation  of  the  knowledge  package. 

In  order  to  assist  the  developer  of  knowledge  packages,  the  KOD  Packager  includes  parts  of  different 
PL  logics,  which  can  be  easily  imported  as  ‘templates”.  For  example,  the  KOD  Packager  includes  all  the 
elements  which  are  proposed  by  the  IMS  Learner  Information  Profile  Specification  (IMS,  2001b)  for  describing 
learner  profiles.  The  designer  can  easily  select  which  of  these  determinants  are  suitable  for  the  specific  learning 
context,  and  include  them  in  a new  knowledge  package.  Similarly,  the  KOD  Packager  includes  the  elements 
which  are  proposed  by  the  IEEE  Learning  Objects  Meta-Data  Specification  (IEEE,  2001)  for  describing 
learning  objects  characteristics;  as  well  as  a set  of  PL  rules,  which  select  different  PL  constituents  (learning 
objects  characteristics)  for  different  PL  determinants  (learner  characteristics). 

Parts  of  the  PL  logics  which  are  built-in  in  the  KOD  Packager  are  based  on  specific  learning  styles 
models,  for  assisting  the  user  to  easily  accommodate  these  learning  styles  models  into  the  development  of 
adaptive  educational  e-content.  For  example,  the  user  can  select  to  accommodate  the  Felder  and  Silberman 
learning  styles  model;  in  this  case,  the  learner  profile  which  is  created  by  the  KOD  system  for  each  learner, 
includes  an  element  for  representing  whether  each  individual  learner  is  visual/verbal;  similarly,  the  educational 
meta-data  file  which  describes  each  learning  object  includes  a specific  element  for  representing  whether  the 
learning  object  is  visual  or  verbal.  Finally,  the  PL  rule  “IF  Learner  is  Visual  THEM  learning  object  is  Visual”  is 
ready  to  be  included  in  a knowledge  package. 


Conclusions 

This  paper  has  investigated  the  accommodation  of  learning  styles  research  in  PL  systems.  It  has  briefly 
reviewed  the  most  well-known  learning  styles  theories  and  models,  as  well  as  some  criteria  for  selecting  among 
them  when  developing  a PL  system.  The  paper  has  also  outlined  some  PL  systems  which  utilise  this  line  of 
research  for  delivering  personalised  learning,  with  emphasis  on  the  PL  system  which  is  being  developed  in  the 
context  of  the  KOD  European  project. 

Our  current  and  future  work  in  this  context  includes  the  development  of  different  knowledge  packages 
(through  the  KOD  tools),  which  are  based  on  different  learning  theories  and  models.  This  will  provide  to  us  a 
testbed  for  further  investigating  the  use  of  learning  styles  research  in  PL  systems. 
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Abstract;  In  this  paper  we  present  the  way  a multi-criteria  decision  making  methodology  is  applied  in 
the  case  of  agent-based  selection  of  offered  learning  objects.  The  problem  of  selection  is  modeled  as  a 
decision  making  one,  with  the  decision  variables  being  the  learner  model  and  the  learning  objects’ 
educational  description.  In  this  way,  selection  of  educational  content  is  based  on  dynamic  data  input 
collected  at  the  time  of  the  decision.  This  methodology  is  studied  in  the  context  of  an  agent-based  e- 
market  for  educational  content  brokering,  and  is  engaged  by  the  broker  agents  recommending  learning 
objects  to  learners,  according  to  their  cognitive  style. 


introduction 

In  agent-mediated  educational  content  brokering,  artificial  agents  are  responsible  of  locating  offered 
learning  objects,  negotiating  with  the  content  providers  on  the  terms  of  the  service  provision,  and  for  presenting  the 
learner  with  those  offers  that  best  match  his/her  needs  and  preferences.  Successful  completion  of  this  task  is  based 
on  two  parameters:  the  capability  of  the  mediating  agent  to  understand  and  model  the  user  needs,  and  its  capability 
to  evaluate  all  available  learning  objects  and  recommend  the  most  suitable.  The  most  common  ways  to  introduce 
intelligent  brokering  for  educational  content  are  based  on  techniques  from  Artificial  Intelligence  (AI):  knowledge 
representation,  reasoning,  and  expert  systems,  often  offer  tools  and  techniques  in  the  developers  of  such  systems 
since  they  allow  the  agent  to  behave  intelligently  in  two  ways;  first,  with  the  ability  to  store  the  knowledge  of  the 
experts  (for  example  using  a knowledge  base);  second,  using  the  previous  knowledge  to  infer  rational  decisions. 

In  the  last  years  though,  most  agent-based  e-commerce  systems  engage  methodologies  and  tools  which 
originally  come  from  sciences  other  than  AI;  these  include  methods  and  techniques  from  Game  Theory,  Optimization 
and  Decision  Making  (Guttman  et  al,  1998).  In  this  paper,  we  are  going  to  approach  the  problem  of  content  selection 
(know  as  the  recommendation  problem  in  e-commerce  systems)  from  a decision-making  point  of  view.  We  are  going 
to  introduce  how  a multi-criteria  decision  making  (MCDM)  methodology  is  applied  in  the  case  of  agent-based 
educational  content  brokering,  and  we  are  going  to  discuss  the  benefits  and  drawbacks  of  this  approach. 

There  is  an  interesting  feature  in  this  approach:  it  replaces  searching  knowledge  inside  a knowledge  base, 
using  constraints  posed  by  logical  expressions  and  logic  rules,  with  an  evaluation  of  knowledge  using  mathematical 
formulations.  It  can  therefore  provide  measures  of  “how  much  better  one  decision  is  compared  to  another”,  even  if 
both  decisions  are  rational  and  solve  the  constraints  problem.  This  usually  sounds  rather  awkward  though  - how  can 
we  represent  the  expert  knowledge  in  terms  of  mathematical  expressions?  Logic  variables  (CategoryOfUser=’This’) 
and  logic  rules  (IF  CategoryOfUser=’This’  THEN  TypeOfContent=’That’)  are  simple,  and  easily  comprehended;  how 
is  the  learner  going  to  be  modeled  using  mathematical  variables,  and  how  is  the  expert  going  to  express  the 
experience  in  a mathematical  form? 

We  will  try  to  give  answers  to  these  questions  by  introducing  the  example  of  modeling  the  learners  with  the 
classic  Honey  and  Mumford  cognitive  styles  model  (1992).  We  will  introduce  the  basic  principles  of  a multi-criteria 
decision  making  methodology,  and  study  which  parameters  of  the  Honey  & Mumford  model  serve  as  criteria  for  the 
selection  of  the  available  content  (learning  objects);  we  will  also  study  a method  that  can  be  used  to  carry  out  this 
selection.  Finally,  we  will  present  how  this  decision-making  procedure  is  incorporated  into  an  agent-based  system, 
and  how  the  broker  agents  operate  it. 
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The  Honey  & Mumford  Model 


Tennant  (1988)  defined  cognitive  style  as  "an  individuals  characteristic  and  consistent  approach  to 
organizing  and  processing  information".  There  are  several  learning  and  cognitive  styles  theories  and  models,  which 
categorize  learners  in  terms  of  instructional  preferences,  information  processing  and  personality  styles,  and  are 
usually  employed  for  the  realization  of  individualized  instruction  (for  a survxy,  see  Sampson  et  al,  2002).  In  order  to 
demonstrate  how  a cognitive  style  model  can  be  used  in  order  to  create  the  multi-criteria  selection  model,  we  will 
engage  the  classic  Honey  and  Mumford  model.  The  Honey  and  Mumford  model  (1992)  is  a cognitive  style  model, 
developed  for  use  in  commerce,  management  and  training  situations.  It  categorizes  learners  according  to  the 
following  dimensions  of  a person’s  learning  style:  Theorist,  Activist,  Reflector  and  Pragmatist.  In  order  to  rate 
learners  on  each  one  of  the  categories,  the  model  uses  the  answers  in  a specially  designed  Learning  Styles 
Questionnaire  (LSQ)  of  80  questions,  with  binary  answers  of ‘‘Correct”  and  “False”,  and  after  an  internal  processing 
of  the  results,  it  provides  a percentage  weight  for  each  category.  Therefore,  the  learner’s  style  is  defined  depending 
on  the  weight  at  each  learning  style  category. 


Figure  1:  An  example  learner’s  model  after  a LSQ  is  processed.  It  provides  a measure  of  how  the  learner  ‘scored’  in 
every  category  (source:  PSI-Press,  http://www.psi-press.co.uk/) 

From  a practical  point  of  view,  it  is  important  that  a learner’s  model  describes  not  only  how  learners  are 
categorised,  but  also  how  the  instruction  method  should  be  adapted  for  each  learner  category  (Spector,  2001).  The 
real  complexity  for  the  designers  of  e-leaming  systems  arises  when  they  try  to  match  subject  matter  with  learner 
characteristics  and  appropriate  instructional  methods.  Such  a process  includes  both  learner  modeling  (using  the 
Honey  & Mumford  model  results  - see  Figure  1)  and  description  of  the  educational  properties  of  the  learning  content 
(that  is  e.g.  a description  of  how  suitable  a course  is  for  the  learners  having  a specific  cognitive  style).  The  matching 
procedure  involves  all  these  parameters  as  input  - and  still  a proper  matching  mechanism  has  to  be  found.  We 
address  this  problem  as  a decision  making  one. 


Multi-Criteria  Decision  Making  Methodology 

According  to  Roy  (1996)  the  general  methodology  of  decision  making  problems  includes  four  steps:  (i) 
defining  the  object  of  the  decision,  that  is  the  set  of  potential  actions,  and  the  problematic  of  the  decision;  (ii) 
studying  the  parameters  influencing  decision  and  defining  so  a set  of  criteria;  (iii)  choosing  an  appropriate  multi- 
criteria aggregation  method;  (iv)  proceeding  at  the  activity  of  decision  aid. 

Step  One;  The  Object  of  Decision 

The  first  step  includes  definition  of  the  decision  variables  in  a form  of  a consideration  set^.  In  the  case  of 
educational  content  brokering,  diis  set  includes  all  the  available  learning  objects,  which  will  be  evaluated  by  the 
decision  maker  (in  our  case,  the  brokering  agent).  What  is  called  ‘the  problematic  of  the  decision’  is  the  definition  of 
what  kind  of  evaluation  or  choice  does  the  decision  maker  want  to  make  upon  the  different  objects  available  in  set^; 
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in  the  case  of  broker  agents  recommending  a learning  object,  the  decision  problematic  is  “selection  of  one”.  That  is 
the  selection  of  one  action  from  the  consideration  set  of  the  form  [qj,  Q2,  a„]. 

Step  Two:  The  Criteria 

The  next  step  is  deciding  what  will  be  the  criteria  upon  which  the  learning  object  will  be  evaluated.  This 
means  defining  a consistent  family  of  criteria,  assuming  that  these  criteria  are  non-decreasing  value  functions, 
exhaustive  and  non-redundant.  Each  criterion  is  defined  on  A as  it  follows: 
g,  : ^ ^ [g,.  ,g*  ] C 91  / a ^ g(a)  € , where  [g.,  is  the  criterion  evaluation  scale,  with  the  worst 

level  of  the  /th  criterion,  g-  the  best  level  of  the  ith  criterion,  g.  (a)  the  evaluation  or  performance  of  action  a on 

the  /th  criterion  and  g{ci)  the  vector  of  performances  of  actions  on  the  n criteria  (Jacquet-Lagreze  & Siskos,  2001). 

In  the  case  of  educational  content  brokering,  there  are  two  ways  to  deal  with  the  selection  of  the  criteria. 
Recommendation  is  usually  based  on  criteria  as  price,  time  of  delivery,  form  of  delivery,  etc.  - a common  practice  in  e- 
commerce  systems.  We  would  like  to  introduce  another  aspect  of  recommendation  in  agent-based  learning 
environments:  based  on  the  pedagogical  profile  of  the  learner.  That  is  using  the  categories  of  the  learner  model  as 
criteria;  each  learning  object  will  be  evaluated  on  how  suitable  it  is  for  each  category  of  learners.  This  is  exactly  the 
same  as  making  an  expression  as  “ TypeOfContent=’That’  ”.  To  be  more  specific,  in  a similar  case  an  expert  system 
would  need  a definition  as:  TypeOfContent="SuitableForActivists".  In  the  case  multiple  criteria  are  used  for 
the  description  of  the  content,  this  definition  is  transformed  (remaining  fully  equivalent)  to  something  similar  to  this: 

• SuitabilityOfContentForTheorists=  "NotSuitableAtAll" ; 

• SuitabilityOf Content ForActivists=  "Perfectly  suitable"; 

• SuitabilityOfContentForRef lectors=  "NotSuitableAtAll"; 

• SuitabilityOfContentForPragmatists=  "NotSuitableAtAll"; 

Therefore,  as  criteria  we  will  use  the  four  categories  of  the  Honey  & Mumford,  which  can  take  their  values  from  a 
5-scaled  climax  of  qualitative  descriptions  [“Not  suitable  at  all”,  “Not  very  suitable”,  “Moderately  suitable”,  “Very 
suitable”,  “Perfectly  suitable”]  showing  the  evaluation  of  each  learning  object  upon  each  categories.  It  is  obvious 
that  a learning  object  (e.g.  an  on-line  course)  is  never  “Perfectly  Suitable”  for  a learner  category  and  “Not  Suitable  At 
AH”  for  the  rest  -it  rather  addresses  some  of  the  needs  of  other  categories  of  learners  too.  Therefore,  such  an 
evaluation  provides  the  expert  with  a way  to  precisely  describe  the  pedagogical  ‘profile’  of  a learning  object.  This 
meta-data  description  is  a criteria  definition  that  fully  complies  with  the  definition  of  the  criteria  according  to  Roy 
(1996). 

Step  Three:  The  Utility  Model 

First  of  all,  we  have  to  define  the  preference  model  that  will  be  used  from  the  broker  agent  to  make  the 
decision;  as  we  have  already  stated,  in  our  case  we  are  using  the  Honey  & Mumford  learner  model  as  a preference 
model.  The  expression  of  the  model  is  then  pf=\p] , P2 , Ps  > where  /?/,  Pi  > and p4  are  the  weights  inferred  after 
the  learner  has  completed  the  Honey  & Mumford  LSQ  and  the  results  have  been  processed. 

This  is  the  point  where  the  analyst  has  to  define  which  evaluation  method  will  be  used;  that  is,  which  multi- 
criteria decision  making  method  will  be  engaged  in  order  for  the  decision  to  be  best  modeled  and  simulated.  In  this 
paper  we  will  use  one  of  the  most  traditional  approaches,  that  leads  to  a functional  representation  g that  can  be 
formed  directly  from  the  criteria  that  constitute  A.  The  goal  of  using  this  approach  is  to  present  how  the 

methodology  works  - depending  on  the  application,  other  more  complex  methods  can  be  used.  Thus,  the 
comprehensive  preference  model  is  characterized  by  a unique  synthesizing  criterion  g.‘  g(d)=V\gi(a),...,g„(a)],  where 
Fis  an  aggregation  function.  The  function  will  be  in  the  form: 

y(a)  = ^Pigi{a) 

where  gi(a)  are  the  evaluations  of  each  learning  object  a regarding  the  suitability  for  each  category  of  learners.  The 
weights  Pi  consist  the  preference  model,  which  are  the  parameters  calculated  by  the  Honey  and  Mumford  LSQ.  The 
final  value  of  Kis  the  total  utility  of  each  learning  object  for  the  learner  under  study. 
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Step  Four:  The  Activity  of  Decision  Aid 

Let  us  now  introduce  the  way  the  activity  of  decision  making  will  be  carried  out,  in  the  context  of  an  agent- 
based  system  We  define  a generic  architecture,  with  agents  capable  of  handling  and  communicating  descriptions  of 
learning  objects,  as  defined  in  the  previous  paragraph.  The  three  basic  roles  are: 

The  Assistant.  The  assistant  agents  are  responsible  for  user  needs  and  requirements  elicitation,  and  formulation  of 
requests  into  messages  understandable  by  the  broker  repository  system.  The  LAssistant  is  the  agent  that  interacts 
with  the  user  and  can  identify  the  request  for  a learning  object.  The  CPAssistant  is  the  agent  that  gets  the  learning 
object  descriptions  from  the  Content  Providers,  and  publishes  them  as  offers  in  the  e-market.  The  TAssistant  serves 
as  the  pedagogical  counselor  of  the  learner:  it  creates  the  learner  model  of  the  user  and  provides  this  model  to  the 
LBroker.  The  TAssistant  can  also  be  the  expert  that  describes  a learning  object  using  the  cognitive  styles;  this  is  a 
task  that  can  be  directly  carried  out  by  the  Content  Author  too,  so  we  will  not  focus  more  on  this  point. 

The  Broker.  The  broker  agents  represent  each  human  user  in  the  e-market  and  facilitate  educational  content  seeking 
or  advertising.  The  Broker  agents  interact  among  each  other  in  the  Brokerage  Pool  and  either  advertise  published 
learning  objects,  or  make  requests  when  looking  for  them.  The  LBroker  evaluates  the  available  offers  from  the 
CPBrokers  in  the  way  described  in  the  previous  paragraphs.  The  LBroker  then  recommends  the  content  selection  to 
the  LAssistant.  The  Brokers  are  also  responsible  for  negotiation  among  the  participating  parties  upon  the  terms  of 
the  service  provisioning  (beyond  the  scope  of  this  paper). 

The  Matchmaker.  This  facilitator  agent  provides  mediating  services,  by  informing  agents  about  other  agents  of  the 
broker  repository  and  their  availability.  The  Matchmaker  provides  all  necessary  administrative  support  information  to 
the  human  administrator  of  the  agent  pool. 

We  can  see  that  the  Broker  agent  engaging  the  multi-criteria  methodology  previously  described  is  the 
LBroker.  In  order  to  pedagogically  evaluate  the  available  learning  objects  and  recommend  one  to  the  learner,  the 
LBroker  needs  the  learner  model  (provided  by  the  user)  and  the  meta-description  of  the  available  learning  objects 
(provided  by  the  CPBroker  or  the  Tutor).  Other  kinds  of  evaluation  of  the  offers  (e.g.  based  on  the  price,  time,  and 
other  terms  of  delivery)  are  beyond  the  scope  of  this  paper. 


Usage  Scenario 


Let  us  demonstrate  how  the  recommendation  procedure  works  in  this  agent-based  system.  A content  author 
creates  different  o-learning  courses  concerning  the  same  subject.  The  author  wishes  those  courses  to  address 
different  learner  needs,  depending  on  the  cognitive  style  of  the  learner.  When  a learning  object  (course)  is  created, 
the  content  author  also  provides  an  evaluation  of  its  suitability  for  each  different  category  of  the  Honey  & Mumford 
model.  The  author  creates  thus  five  different  learning  objects,  with  different  pedagogical  descriptions  (as  depicted  in 
Table  1). 


LOl 

L02 

L03 

L04 

LOS 

Activist 

3 

2 

5 

3 

5 

Reflector 

1 , 

3 

1 

2 

2 

Theorist 

4 

5 

2 

1 

1 

Pragmatist 

3 

2 

1 

5 

2 

Table  1:  The  content  author  provides  an  description  of  each  learning  object  (LO)  on  a scale  from  ‘1-  Not  Suitable  At 
Air  to  ‘5-Perfectly  Suitable’,  regarding  each  one  of  the  cognitive  categories. 
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Activist 

Reflector 

Theorist 

Pragmatist 

LSQ  values 

90% 

55% 

25% 

85% 

Normalized 

0,35 

0,22 

0,10 

0,33 

Table  2:  The  cognitive  styles  preference  model,  as  derived  from  the  Honey  and  Mumford  Learning  Styles 
Questionnaire  (values  are  properly  normalized  in  order  to  provide  a sum  of  1). 


LOl 

L02 

L03 

L04 

LOS 

Utility 

2,67 

2,51 

2,51 

3,25 

2,96 

Table  3:  The  total  utilities  of  the  available  learning  objects  (a  score  upon  the  ’1-Not  Suitable  At  All’  to  ’5-  Perfectly 
Suitable’  climax). 

Introducing  the  multi-criteria  methodology  presented  in  the  previous  section  the  Broker  will  select  a learning  object 
according  to  these  steps: 

1.  The  Tutor  provides  a multi-attribute  cognitive  model  of  the  user,  according  to  the  results  of  the  Honey  and 
Mumford  model  (Table  2). 

.2.  When  the  Learner’s  Broker  receives  the  five  offers  for  learning  objects,  it  calculates  the  total  utility  (that  is 
suitability)  of  each  one,  using  the  methodology  described  in  the  previous  section. 

3.  The  Learner’s  Broker  selects  the  most  appropriate  learning  object  to  be  presented  to  the  user  (Table  3). 

It  is  interesting  to  note  in  this  example  the  difference  between  the  application  of  a simple  rule -based  selection  of  the 
learning  object  (in  this  case  the  agent  would  classify  the  learner  as  ‘Activist’  and  would  propose  the  learning  object 
L03)  and  the  proposed  methodology.  Using  the  multi-attribute  cognitive  model  and  the  multi-criteria  evaluation 
model,  we  observe  that  the  most  appropriate  learning  object  seems  to  be  L04,  being  the  only  one  suitable  for  the 
combination  of  categories  that  characterize  the  specific  learner’s  cognitive  style. 


Conclusions  and  Future  Work 

In  this  paper  we  presented  an  agent-based  recommendation  system  for  educational  content  brokering  that 
implements  a multi-criteria  decision  making  methodology.  Due  to  the  differences  of  learners’  cognitive  models, 
recommendation  can  be  either  carried  out  by  classifying  learners  into  broader  categories  that  then  are  directly  linked 
to  learning  objects  ^.g.  with  rules  like  IF  ‘thistypeofleamer’  THEN  ‘thistypeofmaterial’)  or  by  enhancing  the 
mediating  agent  with  the  appropriate  intelligence  to  propose  the  most  suitable  learning  object  available  at  the  market. 
By  engaging  a multi-attribute  utility  (MAUT)  model,  the  brokering  agents  do  not  need  to  posses  intelligence  in  the 
form  of  a knowledge  base  or  hard-coded  rules  but  only  to  include  the  multi-attribute  evaluation  logic  in  their  decision 
making  module.  In  this  way  learners  are  modeled  in  a multi-attribute  way,  instead  of  being  simply  classified  into  a 
general  learning  category,  learning  objects  are  described  according  to  their  educational  scopes,  brokering  agents 
dynamically  evaluate  all  available  proposals  of  content,  and  select  the  learning  objects  most  suitable  for  the  specific 
learner’s  profile,  and  finally  the  knowledge  the  broker  agents  have  to  contain  in  permanent  storage  is  minimized  since 
only  the  MAUT  evaluation  formula  is  needed;  all  other  input  is  provided  at  run-time. 

In  this  paper,  the  decis  ion  problem  was  modeled  using  the  Honey  and  Mumford  LSQ  parameters  as  the 
preference  model,  but  we  believe  that  the  proposed  methodology  can  be  similarly  applied  in  the  case  of  other 
cognitive  style  models  too.  Future  research  will  be  focused  on  applying  of  several  other  LS  models,  either  separately 
or  combined.  An  issue  to  be  also  studied  is  the  application  of  more  complex  multi-criteria  decision  making 
methodologies  for  modeling  the  user  preferences;  such  methodologies  provide  easier  and  more  realistic  means  of 
expressing  preferences,  but  require  more  complicated  MCDM  methods  to  conclude  to  the  best  proposal  for  each 
learner,  as  one  of  the  famous  family  of  the  ELECTRE  methods  (Roy,  1996). 

Current  international  standardization  efforts  regarding  syntax  and  semantics  of  a learner  model  that 
characterize  a learner  and  his  or  her  knowledge/abilities;  i.e.  IMS  LIP  (IMS,  2001),  IEEE  LTSC  PAPI  (IEEE,  2000)  do 
not  explicitly  define  information  about  the  cognitive  style  of  the  learner.  However,  both  the  IMS  LIP  and  IEEE  PAPI 
Learner  preference/performance  data  types  provide  data  elements  that  allow  cognitive  style  related  information  to  be 
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stored  in  an  appropriate  form.  These  data  types  could  be  used  in  order  to  represent  a multi-attribute  representation  of 
the  cognitive  style  model  of  the  learner,  by  an  appropriate  extension  of  the  corresponding  XML  schema. 

The  1ST  Project  KOD  “Knowledge-On-Demand”  (KOD,  2001)  engages  a multi-agent  brokering  system  for 
selection  of  learning  objects  and  dynamic  synthesis  of  on-line  courses.  In  the  case  of  KOD,  intelligence  is  included 
into  the  learning  objects  in  the  form  of  rules  that  express  the  suitability  of  the  object  for  each  type  of  learners,  so  that 
courses  can  be  constructed  based  on  how  the  learning  objects  are  characterized.  This  is  a case  very  close  to  the 
problem  we  introduced,  and  we  are  currently  studying  ways  to  introduce  the  multi-criteria  descriptions  in  the  learning 
objects  of  KOD,  in  order  for  the  KOD  agents  to  be  able  to  dynamically  assess  and  synthesize  courses  based  on  the 
introduced  decision  making  methodology  and  not  on  prescribed  rules. 
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Internet-based  education  and  training  offer  many  potential  benefits  specific  to  adult  learners  with  emphasis 
given  to  learner-centered  and  self-directed  instruction  models  empowered  by  web-based  educational  resources. 
Indeed,  the  recent  growth  of  the  World  Wide  Web  (WWW)  has  greatly  increased  the  amount  of  information 
and  educational  resources  available  to  the  education  community. 

The  full  exploitation  of  this  mass  body  of  knowledge  resources  available  on  the  Web,  can  be,  however, 
compromised,  by  the  difficulty  in  describing,  classifying  and  maintaining  those  resources  in  such  a way  that 
they  can  be  retrieved  in  an  ""educationally  efficient  and  effective  waff.  Today,  the  web  community  has 
embraced  the  collection  and  use  of  metadata  to  characterise  and  index  educational  resources,  which  lead  to 
semantically  more  accurate  retrieval  of  information  than  search  engines.  In  general  sense,  metadata  is 
information  about  data.  In  the  context  of  resource  discovery,  descriptive  metadata  is  a characterisation  that  aims 
to  represent  the  intellectual  content  of  the  resource.  The  most  popular  technology  for  representing  metadata  is 
XML  (extensible  Markup  Language). 

Typically,  the  educational  content  providers  of  an  e-learning  scenario  need  an  educationally  oriented 
application  facilitating  editorial  and  management  features  for  manipulating  educational  metadata  documents. 
An  efficient  and  effective  educational  metadata  management  tool  should  satisfy  a number  of  requirements 
derived  from  different  dimensions  such  as: 

• The  need  of  addressing  user  categories  (i.e.  learners,  content  providers) 

• The  availability  of  different  educational  metadata  specifications/  standards  (i.e.  IEEE  LOM,  DC) 

• The  emerge  of  metadata  enabling  representation  technologies  (i.e.  XML,  DTD,  XMLSchema). 

The  basic  requirements  towards  an  integrated  and  generic  educational  metadata  management  tool  are: 

• creation  of  new  and  modification  of  existing  educational  metadata  documents 

• data  and  structure  validation  of  the  educational  metadata  document 

• support  of  any  educational  metadata  standard  / specification 

• creation  of  new  metadata  specification  that  applies  to  user’s  requirements  for  each  case  study  (e.g. 
extension  of  existing  metadata  standard) 

• support  enabling  metadata  technologies  (XML,  DTD,  XMLSchema) 

• mapping  of  educational  metadata  files  between  different  specifications/standards 

• facilitate  content  management  of  the  educational  metadata  documents  (e.g.  search,  retrieve, 
update  data) 

The  proposed  architecture  of  the  EM^  tool  (Sampson,  2002),  combines  all  the  required  components  towards 
the  implementation  of  an  educational  metadata  management  tool  (editors,  repositories,  associations, 
management,  validation  and  mapping  components)  and  fulfills  the  above  requirements  (figure  1). 

This  paper  proposes  an  educational  metadata  management  system  using  a deductive,  object-oriented  database 
approach.  The  EM^  tool  can  provide  storage  and  retrieval  of  data  from  the  XML  metadata  files  in  addition  to 
its  features  (i.e.  authoring,  editing,  etc)  by  integrating  the  X-DEVICE  system  (Bassi Hades,  2000).  In  figure  2,  a 
graphical  representation  of  the  proposed  architecture  for  the  cooperation  between  the  two  systems  is  presented. 
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Figure  1 : Architecture  of  an  Educational  metadata 
management  tool 


integration 


EM^  provides  the  graphical  interface  for  the  interaction  with  the  user.  The  user  can  create  a new 
educational  metadata  file  based  on  metadata  specifications,  open  or  edit/update  data  on  an  existing  one,  convert 
metadata  files  between  specifications  as  well  as  create  maps  for  these  conversions.  The  created  or  modified 
XML  documents  after  structure  and  data  validation  (whenever  it  is  possible)  are  stored  to  the  associated  XML 
repository  of  EM^.  The  XML  Repository  is  a system  folder  storing  the  XML  documents  as  files,  and  it  is  not  a 
database  system.  In  addition  the  DTD  and  XMLSchema  files  are  stored  in  their  associated  repositories 
respectively.  By  integrating  ? -DEVICE  to  the  EM^  tool,  database  storage  and  data  retrieval  can  be  provided 
based  on  the  user’s  queries.  Each  stored  XML  document,  together  with  its  associated  DTD  is  passed  to  the  X- 
DEVICE  system  where  the  mapping  to  OODB  system  takes  place.  The  EM^  tool  can  generate  a DTD  in  case 
there  is  no  DTD  or  XML  Schema  that  defines  the  XML  document.  In  addition,  if  only  an  XML  Schema  exists 
for  a specific  XML  document  then  transformation  from  XML  Schema  to  DTD  is  also  an  available  feature.  The 
transformation  of  XvlLSchema  generates  a sample  DTD,  which  may  lack  information  compared  to  the 
XMLSchema.  The  OODB  holds  the  data  of  every  educational  metadata  file  that  has  been  created,  updated  or 
stored  in  the  EM^  XML  repository.  In  addition,  the  EM^  will  provide  an  interface  to  the  user,  for  query 
submission.  When  specific  data  is  required  from  the  XML  documents,  the  user  should  compose  his  query 
following  the  syntax  of  the  X-DEVICE  system  query  language.  The  query  is  then  passed  to  the  X-DEVICE 
system,  which  will  obtain  the  required  information  from  the  OODB.  The  information  is  in  XML  format  and  it 
will  be  presented  to  the  user  through  the  user  interface  of  EM^. 
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Abstract.  GAIA  is  a microworld-based  learning  environment  aiming  to  provide  means  for 
exploratory  learning  to  secondary  school  students  in  the  fields  of  physics,  mathematics  and 
geosciences.  This  paper  presents  Gilbert,  one  of  GAIA’s  microworlds  aiming  to  familiarize  students 
with  contemporary  geosciences. 


Introduction 

Unlike  in  other  fields  of  studies,  such  as  physics  and  mathematics  (Thornton  and  Sokolo,  1990),  most  of 
the  available  commercial  educational  software  related  to  Earth  School  Studies,  is  either  drill-and-practice 
oriented,  where  students  are  presented  with  problems  to  solve  and  they  receive  feedback  about  the  accuracy  of 
their  responses,  or  IVLiondA-based  oriented,  following  the  archetypal  form  of  text,  graphics  or  multimedia 
information  associated  with  relevant  self-evaluation  tests.  -However,  such  environments  (Kali  et  al.  1997)  do 
not  strongly  facilitate  students  towards  constructing  their  own  knowledge  through  open-ended  experimentation 
(Jonassen,  2000).  On  the  other  hand,  microworld -based  educational  software  can  provide  the  means  for 
exploratory  learning,  by  providing  the  environment  for  students  to  learn -by-doing,  rather  than  just  watch  or 
listen  to  a description  of  how  something  operates.  Briefly,  a microworld  is  a computer-assisted  problem 
exploration  and  experimentation  space  in  which  one  can  explore  and  construct.  Typically,  they  are  constrained 
versions  of  reality  that  enable  learners  to  manipulate  variables  and  experiments  within  parameters  of  some 
system.  Furthermore,  the  most  advanced  microworld-based  educational  software  provides  the  means  to 
educators  and  learners  to  construct  their  own  exploration  spaces.  This  is  this  the  case  of  GAIA  which  allows 
learners  to  represent  their  own  thinking  in  the  ways  that  they  explore,  manipulate  and  experiment  with  the 
environment. 


Figure  1.  Overview  of  the  GAIA  Project  Design  and  Development  Process  (Papageorgiou  et  al,  2001) 
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GAIA  is  a microworld -based  exploratory  learning  environment  in  the  field  of  geosciences  education, 
physics  and  mathematics.  This  is  accomplished  through  constrained  simulations  of  real-world  phenomena  in 
which  students  can  navigate,  manipulate  or  even  create  objects,  and  experiment  with  their  effect  on  each  other. 
In  the  current  version  of  GAIA,  seven  microworlds  -named  after  great  scientists  - are  designed  and  developed  to 
provide  the  environment  for  replicating  the  essential  functionalities  needed  to  explore  phenomena  about  the 
Earth  and  the  planets,  regarding  their  interior,  atmosphere,  morphology,  magnetic  field,  their  orbits  in  space  as 
well  as  their  astronomical  environment.  Figure  1 gives  an  overview  of  the  GAIA  project  design  and 
development  process  (Papageorgiou  et  al,  2001).  The  development  of  GAIA  is  based  on  the  component-based 
educational  authoring  environment  “E-Slate”  (Birbilis  G.  et  al,  2000),  which  provides  a workbench  for  creating 
highly  dynamic  software  with  rich  functionality  (Roschelle  et  al,  1999). 

The  Gilbert  microworld,  presented  in  this  paper,  deals  with  Earth’s  magnetic  field,  within  the  frame  of  the 
school  curriculum.  This  microworld  aims  to  the  comprehension  of  Brth’s  magnetic  field  concept  and  how 
contemporary  science  studies  and  explores  this  field.  The  Earth’s  magnetic  field  is  one  of  the  most  attractive 
concepts  to  students’  imagination,  but  its  teaching  becomes  difficult  as  it  is  hard  for  them  to  resolve  its  vector 
within  the  3D  space  in  relation  to  geographical  coordinates,  and  to  precisely  comprehend  its  existence  as  they 
do  for  instance  with  the  concept  of  temperature.  The  Gilbert  microworld  contributes  to  all  the  aforementioned 
and  familiarizes  students  with  the  existence  and  nature  of  Earth’s  magnetic  field. 


Earth’s  magnetic  field  study  through  GAIA  software 

The  Earth’s  magnetic  field  can  be  directly  noticed  by  the  students  with  the  use  of  a magnetic  needle  and 
that  is  the  reason  why  they  must  be  familiarized  with  the  compass  use  in  school  laboratory.  This  is  usually  done 
through  the  measurement  of  the  field’s  direction.  Although  laboratory  measurement  is  useful,  it  limits  students 
to  identify  and  measure  earth’s  magnetic  field,  only  at  the  place  they  are. 

The  GAIA  interactive  software  presented  in  this  paper  allows  students  to  virtually  travel  everywhere  on 
Earth,  and  measure  earth’s  magnetic  field  elements  (Figure  2).  Students  have  also  the  possibility,  besides  the 
compass,  to  use  a total  magnetic  fie  Id  magnetometer.  The  selection  of  the  virtual  site  for  the  experiment  is 
accomplished  either  on  a 3D  rotating  globe  (see  Figure  2:  top  left  window)  or  on  a plane  geophysical  map.  The 
altitude  where  the  measurement  is  taken  place  is  set  in  another  window  (see  Figure  2:  top  right  window),  by 
pointing  the  mouse  on  a magnetometer  sketch  on  the  screen,  or  alternatively  by  regulating  a properly  designed 
“slider”  (see  Figure  2:  bottom  right  window).  The  perspective  of  earth’s  projection  is  done  in  such  a way,  as  to 
show  the  magnetometer’s  sketch  on  the  screen.  In  another  window  (see  Figure  2:  bottom  left  window)  students 
can  observe  and  use  three  instruments:  a horizontally  positioned  compass  that  measures  declination,  a compass 
placed  in  the  magnetic  meridian  plane  that  measures  magnetic  inclination  and  a total  field  magnetometer. 

In  parallel  to  the  virtual  measuring  of  the  field,  students  can  also  observe  the  resolved  components  of  the 
magnetic  field  vector  at  any  point  around  the  Earth  (see  Figure  2:  top  middle  window).  In  this  resolution  the 
quantities  of  field’s  intensity,  declination  and  inclination  are  clearly  noticeable.  The  students  have  also  the 
possibility  to  visualize  and  resolve  the  vector  within  the  Cartesian  system  coordinates.  In  addition,  they  can 
rotate  the  projection  of  the  vector  representation  for  more  convenient  observation  of  the  relative  angles  and 
vectors. 

At  a higher  knowledge  level,  students  can  also  change  the  value  of  the  measurement  date  parameter, 
provided  that  they  have  been  introduced  in  the  field’s  variation  with  time.  Of  course,  it  is  assumed  that  the 
students  have  also  learned  some  basic  notions  about  the  origin  of  the  Earth’s  magnetic  field.  Measurement 
values  are  simulated  to  real  ones  as  they  based  on  the  IGRF  (International  Geomagnetic  Reference  Model) 
geomagnetic  field  model  of  International  Association  of  Geomagnetism  and  Aeronomy  (lAGA  1995).  This 
model  is  designed  to  approximately  calculate  Earth’s  magnetic  field  from  Earth’s  interior  upwards  into  space 
without  taking  into  account  external  magnetic  field  sources.  It  is  based  on  the  usual  expansion  of  the  spherical 
harmonics  of  the  potential  in  geocentric  coordinates  (Campbell,  1997;  Lowrie,  1997)  and  describes  the  field’s 
variation  within  time.  At  regular  times,  new  spherical  harmonics  coefficients  supplement  it  in  order  to  renew 
and  fix  its  anticipation  in  the  future.  In  view  of  the  educational  purposes  for  which  the  model  was  used,  the 
magnetic  field  values  had  no  higher  accuracy  than  that  of  the  order  of  1 p?. 


Figure  2.  In  a special  configured  environment,  students  can  select 
whichever  place  of  the  world  they  like  on  a 3D  rotating  globe  or  on  a 
plane  map,  and  to  measure  the  magnetic  field*s  elements  at  a certain 
height,  meanwhile  they  can  see  the  field*s  vector  analysis. 


Figure  3.  A window  of  Gilbert 
microworld  that  presents  Earth *s 
magnetic  field  main  dipole 
component.  Students  experiment  with 
placing  and  moving  a number  of 
compasses  and  finally  become 
accustomed  to  the  lines  of  force 
notion. 


The  dipole  component  of  the  Earth *s  magnetic  field,  which  consists  the  95%  of  the  total  field,  is  presented 
in  another  window  of  the  Gilbert  microworld  (Figure  3).  In  that  window,  students  can  test  the  field*s  existence 
around  the  whole  planet  Earth  by  a number  of  compasses  that  can  freely  move  into  space.  In  this  environment, 
students  can  easily  learn  the  lines  of  force  as  well  as  to  magnetic  poles  natural  position.  Field^s  lines  of  force 
may  be  displayed  at  student*s  choice. 

Displaying  the  three  elements  of  the  magnetic  field  (intensity,  declination,  inclination)  on  maps  (Figure  4), 
it  helps  students  to  realize  that  these  three  elements  coexist  in  every  point  and  are  modified  at  the  same  time. 
For  instance,  students  can  observe  the  magnetic  poles  in  the  three  maps,  to  compare  their  observations  and  to 
realize  that  in  all  of  these  three  plots  the  poles  have  characteristic  features.  Students  also  realize  that  in  many 
places  on  earth,  compass  points  out  to  a direction  deviating  considerably  from  the  North.  The  coincident  display 
of  these  three  maps  reveals  the  value  of  the  “multiple  representations**  as  well  as  the  “direct  manipulation**  of 
the  objects  (Teodoro,  1990;  Shama  and  Layman  1997). 

The  introduction  of  examples  from  an  Applied  Geophysics  geomagnetic  survey  was  one  of  the  essential 
innovations  of  GAIA  in  the  teaching  of  the  magnetic  field  in  secondary  education  level.  Beyond  the 
comprehension  of  notions  as  the  induced  magnetization,  the  addition  of  the  induced  field  vector  to  the  Earth *s 
field  and  the  magnetic  permeability  of  the  materials,  the  microworld  further  aimed  to  present  a simplified 
example  of  a practical  application  of  Earth*s  magnetic  field  measurements,  such  as  the  detection  of  mineral 
deposits  and  the  antiquities. 

In  the  environment  shown  in  Figure  5,  students  experiment  with  a buried  plate  of  square  or  rectangular 
section.  The  plate  is  considered  infinitely  elongated  in  order  to  simplify  the  problem  from  the  existence  of  edge 
anomalies.  The  plate  is  oriented  E*W  while  the  magnetometer  is  moving  and  measuring  vertically  to  the  plate, 
to  N-S  (Figure  6).  The  plate  *s  magnetization  is  considered  only  induced  and  its  material  can  be  selected  out  of  a 
large  variety  of  geological  rocks  with  different  values  of  magnetic  permeability.  For  each  rock,  typical  values  of 
permeability  and  susceptibility  are  given,  in  companion  with  the  range  of  these  values  in  the  real  state. 
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Figure  4.  The  coincident  display  of  the  magnetic  maps 
facilitates  the  observation  and  modification  of  all  Earth’s 
magnetic  field  elements  at  the  same  time. 


Figure  5.  In  the  magnetic  survey  environment, 
students  can  expertise  with  the  buried  plate 
detection  by  the  implementation  of  a 
magnetometer.  They  can  also  select  the  material  of 
the  plate,  and  the  location  where  the  experiment 
takes  place.  Here  the  experiment  takes  place  with  a 
magnetite  plate  of  square  section,  at  an  area  close 
to  the  North  Pole. 


Although  the  virtual  experiment  can  take  place  everywhere  on  Earth,  we  suggest  the  choice  of  three  places 
with  characteristic  inclination  (?)  values.  These  are:  the  North  magnetic  Pole  (?  = 90\  the  magnetic  equator  (?  = 
0)  and  an  intermediate  place  like  North  Africa  with  inclination  equal  to  4S".  Anomalies  observed  in 
measurements  of  the  total  field  above  the  buried  plate  in  these  three  areas  are  advisable  for  the  induced 
magnetization  presentation  to  the  high  school  students. 


Figure  6.  The  orientation  of  the  buried  plate  is  E-W 
while  this  of  the  survey  traverse  is  N-S 


Figure  7.  Earth’s  magnetic  field  anomaly  caused  by 
the  induced  magnetization  of  the  square  section  buried 
plate  in  the  North  magnetic  Pole. 
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At  the  North  magnetic  Pole  the  Earth’s  magnetic  field  vector  is  perpendicular  to  the  plane  of 
measurements.  Since  the  particle  magnets  of  a paramagnetic  material  are  polarized  according  to  the  field  (a 
diamagnetic  material  causes  the  opposite  effect)  the  induced  magnetization  will  be  also  perpendicular  to  the 
plane  of  measurements.  In  the  case  of  paramagnetic  materials,  which  are  the  most  common  in  nature,  the  plate’s 
induced  magnetization  is  shown  in  Figure  7.  The  lines  of  force  of  the  induced  magnetization  field  of  the  body 
are  symmetrical  relative  to  the  vertical  axis  that  passes  through  the  centre  of  the  body’s  section.  The  values  of 
the  measurements  are  increased  at  the  points  where  the  body’s  field  has  the  same  direction  with  Earth’s  field, 
while  are  reduced  where  the  two  vectors  are  opposed.  At  the  points  where  body’s  field  is  horizontal  (like  on 
body’s  sides),  no  essential  variation  is  resulted. 

In  magnetic  equator  (?  = 0‘ ) Earth’s  magnetic  field  vector  is  horizontal.  Following  the  aforementioned 
technique,  it  is  easy  to  find  out  that  the  diagram  of  the  measurements  is  also  symmetric  to  the  body  but  inversed 
in  sign  in  relation  to  the  one  of  the  North  magnetic  Pole  (Figure  8).  In  body’s  centre,  the  lines  of  force  of  the 
body’s  induced  magnetization  are  opposed  to  the  Earth’s  magnetic  field  and  thus  the  field’s  anomaly  is 
negative.  In  the  case  that  inclination  is  45",  the  lines  of  force  are  symmetrical  to  body’s  diagonal  (Figure  9). 
The  magnetic  field’s  anomaly  has  an  asymmetrical  pattern  and  therefore  its  interpretation  regarding  body’s 
position  becomes  more  complicated.  The  point  where  Earth’s  field  vector  is  vertical  to  that  of  dynamic  lines  is 
just  above  the  body,  and  there  we  observe  that  field’s  anomaly  is  practically  zero.  With  exception  to  the 
magnetic  poles  and  equator  the  diagram  is  asymmetric  at  all  the  other  positions  (More  details  in  Dobrin  and 
Savit,  1988). 

By  the  final  stage  of  the  microworld  Gilbert  students  can  comprehend  the  notion  of  induced  magnetization 
and  how  it  is  depending  on  the  magnetic  properties  of  materials.  From  the  example  of  the  magnetic  survey  the 
students  are  introduced  to  the  field  of  Applied  Geophysics  and  its  contribution  to  the  detection  of  oil  deposits, 
minerals,  antiquities,  etc.  Contemporary  science  can  provide  students  with  many  such  examples,  possibly 
exciting  their  interest  for  acquiring  new  knowledge. 


Figure  8.  Earth’s  magnetic  field  anomaly  caused  by  Figure  9.  Earth’s  magnetic  field  anomaly  caused  by 
the  induced  magnetization  of  the  square  section  buried  the  induced  magnetization  of  the  square  section  buried 
plate  in  the  magnetic  Equator.  plate  in  position  with  magnetic  inclination  equal  to  45° 

(e.g.  North  Africa) 
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Conclusions 


The  GAIA  software  was  designed  and  developed  to  allow  to  operate  a virtual  educational  laboratory 
addressed  in  secondary  school  students.  Students  may  comprehend  basic  notions  of  Earth’s  magnetic  field  and 
become  accustomed  to  them  as  well  as  with  other  more  fundamental  notions  of  magnetic  field  theory,  such  as 
induced  magnetization.  A number  of  activities  based  on  Gilbert  microworld  have  been  already  created. 
Students,  working  in  teams  under  teacher’s  guidance,  can  enhance  their  knowledge  on  Earth’s  magnetic  field  by 
experimenting  with  Gilbert.  Initial  evaluation  of  students  performance  indicate  a significant  advantage  in  their 
understanding  on  basic  Geoscience  concepts  when  using  GAIA  as  compared  to  their  traditional  school  teaching 
methods. 
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Abstract;  In  this  communication  we  present  how  to  design  and  to  develop  a course  in  an 
asynchronous  on-line  university.  It  is  based  on  three  key  aspects:  a detailed  working  plan, 
interactive  and  multimedia  learning  resources  and  the  teacher  coaching  activity  during  the  course. 
As  an  example  of  a Mathematical  course  implementation  we  discuss  some  aspects  of  a Statistics 
course  of  the  Computer  Science  curricula  at  the  UOC. 


Introduction 

La  Universitat  Oberta  de  Catalunya  -UOC  [The  Open  University  of  Catalunya]-  was  bom  at  the  end  of  the  1990's  to 
offer  university  undergraduate  and  postgraduate  degrees  in  Spain.  The  UOC  is  an  on-line  university  based  on  an 
asynchronous  model,  both  in  time  and  space.  The  relationship  between  the  student  and  the  UOC  is  established 
through  a Virtual  Campus,  an  Internet-based  communications  environment.  The  Virtual  Campus  allows  students  and 
lecturers  to  interact,  without  having  to  occupy  the  same  time  or  space;  it  also  facilitates  communication  with  the 
other  areas  within  the  university  itself  (the  Secretary’s  office,  library,  I.T.  helpdesk,  etc.)  through  a PC  connected  to 
the  Internet. 

In  general,  mathematics’  subjects  suppose  a stumbling  block  for  all  the  students  who  have  to  overcome  them,  more 
in  particular;  in  distance  learning  environments  the  difficulties  that  are  detected  are  even  bigger.  In  fact,  the  students 
who  attend  a kind  of  subject  like  this  one  only  can  make  use  of  a text  and  the  possibility  of  getting  in  contact  with 
the  teacher  by  writing  a message  to  him.  In  these  conditions,  the  difficulties  for  the  learning  process  are,  in  general, 
considerable.  With  the  purpose  of  facilitating  the  learning  process  of  this  type  of  subjects,  a detailed  planning  of  the 
activities  and  the  use  of  technological  resources  compatible  with  the  environment  are  proposed  and  also  the 
redefinition  of  the  teacher’s  role. 

The  teacher  becomes  more  a learning  facilitator  rather  than  an  information  or  content  transmitter. 

On-line  learning  strategy 

Every  semester  the  student  enrols  the  different  courses  selecting  those  ones  of  interest  from  the  curricula.  Each  one 
of  them  has  a detailed  planning.  This  planning  consists  on  a set  of  weekly  working  packages.  A working  package 
includes  general  objectives,  theoretical  contents,  examples,  practical  activities,  and  assignment  criteria.  In  an  on-line 
learning  environment  the  student  has  a computer  connected  to  the  net.  So,  the  planning  presentation  and  the  most  of 
the  learning  resources  take  advantage  of  the  information  and  communication  technology.  The  learning  resources  are 
any  kind  of  multimedia  document  (even  printed  text),  interactive  simulations  and  self  evaluation  systems  [Mor 
2000].  This  kind  of  resources  are  fundamental  to  overcome  the  limitation  of  learning  material  based  on  printed  text 
[Vallverdu  1999].  On  the  other  hand,  the  planning  presentation  evolves  depending  on  the  student  activity  and  his  or 
her  interaction  with  the  teacher.  The  role  of  the  teacher  is  no  more  focused  on  the  information  transmission  but  the 


learning  facilitation.  This  means  that  the  teacher  coaches  the  student,  promoting  activities,  moderating  debates  and 
reflection,  and,  of  course,  has  to  make  the  assignment. 

An  example:  a course  of  statistics 

In  almost  all  areas  of  learning,  from  engineering  and  sociology,  through  to  biology  and  economy,  certain  issues  arise 
and  they  are  hard  to  describe  using  mathematical  models  which  are  often  too  complex  or,  conversely,  too  simple.  In 
practice,  results  are  often  obtained  from  experience  and  from  a series  of  well-organised  and  appropriately-treated 
data.  Statistics  studies  how  to  obtain  relevant  information  from  a series  of  data  and  can  become  a fundamental  tool 
in  the  decision-making  process.  In  view  of  this,  we  had  a special  interest  in  design  and  develop  a course  of  statistics, 
which  would  help  the  student  attain  the  objectives  established  for  the  subject  for  which  he/she  has  enrolled.  We. 
present  a course  of  Statistics  of  the  Computer  Science  curricula  at  the  UOC  [Sancho  2000]. 

By  following  a 'Learning  by  doing’  methodology  [Smith  98],  the  chance  to  perform  a statistics-related  task  is 
offered  to  the  students,  the  interest  of  which  lies  both  in  the  conclusions  he/she  can  draw  from  it  and  the  processes 
and  tools  used.  This  task  is  made  up  of  several  parts:  collection  and  organisation  of  data,  descriptive  analysis  of  the 
data,  interpretation  and  meaning  in  relation  to  populations  and  conclusions  and  decision-making.  Each  student's 
learning  process  is  built  up  adaptively,  based  on  how  he/she  resolves  these  practical  tasks,  requiring  the  student  to 
apply  the  various  concepts  he/she  has  assimilated.  The  teaching  strategy  for  the  student  to  assimilate  the  necessary 
content  is  provided  by  the  scheme  of  work  in  which  targets,  content  and  the  activities  necessary  to  attain  these 
targets  are  determined  in  a personalised  fashion. 

With  the  purpose  of  assimilating  by  the  students  the  basic  contents  which  have  to  allow  them  to  accomplish  the 
statistic  works,  the  consultant  will  propose  a pace  of  work  and  some  weekly  activities  where  the  most  important 
aspects  to  discuss,  according  to  the  timing  of  the  subject,  will  be  pointed  out.  The  consultant  will  also  comment  the 
most  relevant  video  units  for  the  weekly  theme  and  the  convenience  of  visiting  certain  web  pages  to  strengthen  the 
ideas,  which  have  been  introduced  to  the  study  notes. 

It  is,  then,  important  to  highlight  the  fact  that  this  learning  process  takes  place  on  a network  where  the  different 
agents  involved  can  interact:  students,  lecturer  and  content.  Interaction  takes  place  through  the  resolution  of 
exercises  and  experimentation  using  simulation  tools  via  the  Internet  (XML  standards,  Java  applets,  J2EE)  that  helps 
to  set  the  rhythm  of  the  student's  progress. 

Conclusions 

The  work  we  have  done  has  given  us  a new  perspective  on  both  the  creation  process  and  the  design  of  teaching 
materials  required  for  a virtual  teaching  and  learning  system.  This  perspective  is  based  on  the  atomisation  of 
contents  and  on  the  creation  of  all  the  possible  links  between  them.  In  this  way,  we  can  achieve  an  adaptive 
environment,  which  will  help  personalise  the  teaching  provided  and  ensure  that  the  course  objectives  are  met. 

In  terms  of  the  common,  adaptive  learning  environment  for  different  university  degrees,  which  include  "Statistics” 
within  their  syllabus,  we  have  explored  different  didactic  materials  for  the  teaching  of  quantitative  subjects.  By 
paying  particular  attention  to  the  methodology,  the  content  and  the  technology,  the  basic  pillars  which  support  the 
following  have  been  established:  the  basics  of  knowledge,  the  interface,  specialised  strategies  and  the  necessary 
technology  to  make  everything  work  correctly. 
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Abstract;  With  the  Web  now  providing  the  infrastructure  for  cooperation,  the  opportunity  is  ripe  to 
create  virtual  community  of  problem  solvers.  We  are  working  to  create  WebSaber,  a computer- 
supported  cooperative  learning  (CSCL)  environment  in  which  teachers  have  a set  of 
communicating  and  coordinating  tools,  for  promoting  cooperative  activities  of  problem  solving.  To 
explain  WebSaber,  we  offer  an  overview  of  cooperative  learning,  and  CSCL,  describe  WebSaber, 
detailing  its  features,  and,  finally,  point  out  the  conclusions  and  future  works. 


Introduction 

Cooperation  is  acting  together,  in  the  pursuit  of  shared  goals,  the  enjoyment  of  the  joint  activity,  or  simply 
furthering  the  relationship  (Argyle,  1991).  The  educational  system  may  not  be  concerned  with  fostering  cooperation 
in  learning  but  learners*  work  together,  depending  on  the  particular  context.  Cooperative  learning  is  a teaching 
technique  that  develops  problem  solving  and  critical  thinking  skills  through  student  teamwork.  It  consists  of  small 
groups  working  together  to  complete  academic  tasks,  which  involve  a range  of  objectives,  such  as  searching  for 
facts,  applying  concepts  and  principles,  problem  solving  and  creative  thinking,  and  teacher  as  facilitator  and 
coordinator,  setting  the  guidelines,  encouraging  cooperation,  reviewing  performance  and  joining  to  the  groups. 

Cooperative  learning  can  be  sustained  by  learning  theories  based  on  the  same  underlying  assumptions  that 
individuals  are  active  agents  that  purposefully  seek  and  construct  knowledge  within  meaningful  context. 
Sociocultural  and  Constructivism  theories  can  play  this  role.  In  the  other  hand,  the  Internet  can  provide  the 
infrastructure  to  foster  cooperative  learning  activities. 

Despite  the  potential,  CSCL  environments  met  serious  problems.  Frequently,  cooperative  learning  is 
assumed  as  a natural  output  of  teamwork.  It  is  also  assumed  that  the  availability  of  CSCL  tools  is  enough  to 
motivate  peers  cooperation.  However,  cooperation  cannot  occur  naturally  and  needs  to  be  articulated  by  the  teacher. 
We  believe  that  CSCL  environments  should  provide  teachers  with  a comprehensive  set  of  cooperative  tools,  able  to 
be  customizable  according  to  teachers’  pedagogical  goals.  Face  the  expressive  number  of  CSCL  environments, 
WebSaber  was  created  to  offer  to  the  teacher  safe,  flexible  and  customizable  cooperative  learning  setting. 


WebSaber  - an  Environment  for  Cooperative  Problem  Solving 

WebSaber  was  developed  following  the  four  phases  of  Software  Engineering  classic  life  cycle:  analysis, 
design,  implementation  and  evaluation  and  tests.  The  analysis  phase  aims  to  define  and  clarify  the  problem  to  be 
solved  with  the  proposed  software.  In  Brazilian  software  market,  CSCL  environments  available  in  Portuguese,  are 
lacking.  Addressed  solution  was  the  development  of  an  open  CSCL  environment  for  the  domain  of  problem  solving 
to  be  publish  in  the  Internet  and  oriented  to  elementary  school  teachers  and  learners.  The  design  phase  requires  the 
understanding  of  the  nature  of  what  is  to  solve  problems  in  a cooperative  way.  In  WebSaber,  to  solve  problems  is  a 
social  and  technical  step-by-step  process.  Five  steps  of  a simplified  approach  of  classic  scientific  method  was 
adopted:  the  analysis  of  the  problem,  the  building  of  preliminary  ideas,  the  choice  of  good  ideas,  the  planning  of 
solution  and  its  implementation.  Current  online  system  basically  contains  coordinating  tools,  with  which  teachers 
pose  problems  to  be  solved,  choose  steps  and  tools  and  control  tasks,  goals,  dates,  and  cooperative  tools,  with  which 
the  learners  cooperate  and  communicate  among  them,  and  built  shared  solution.  The  prototype  main  interface 
provides  links  to  system  components:  Available  Problems  library.  Selected  Texts,  the  Tutor  Mode  and  the 
Environment  itself  Available  Problems  is  a library  that  lists  existent  problems. 
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The  Tutor  Mode  is  the  teacher  working  space.  Accessing  system  at  the  first  time,  teachers  must  to  fill  a 
form  for  obtaining  free  access.  They  can  choose  one  of  the  available  problems  to  be  solved,  in  the  problem  library, 
or  register  a new  problem.  The  system  automatically  includes  the  new  problem  into  the  problem  library.  Two 
distributed  tools  help  the  coordination  of  the  work:  a schedule  form  allows  the  teacher  to  include,  manage  and 
publish  activities,  tools  and  dates;  and  an  distributed  agenda  informs  the  learners  about  synchronous  meeting  dates 
and  schedule  adjustments.  Teachers  can  also  register  URLs  to  be  used  as  online  complementary  bibliography.  The 
system  automatically  displays  and  distributes  the  information.  Planned  the  tasks,  the  system  opens  its  infrastructure 
to  support  cooperative  problem  solving  activities.  For  that,  it  displays  selected  steps,  allocates  tools,  creates  a 
mailing  list  and  a forum  room  and  opens  a room  chat,  allowing  users  to  access  the  Environment  it  self 

The  Environment  it  self  is  the  learners  working  space.  The  system  metaphor  is  a meeting  room,  where  users 
can  interact  and  work  in.  WebSaber  provides  three  main  locales:  a Hall,  a SittingRoom  and  a WorkRoom.  Each 
locale  has  a clear  purpose,  its  own  activities,  communicating  and  coordinating  tools  and  a temporal  dimension.  The 
Hall  displays  available  problems  in  a virtual  wall.  The  Sitting  Room  is  the  space  for  social  changes,  where  users  can 
chat,  send  messages  and  write  and  show  off  personal  annotations.  The  WorkRoom  is  the  work  place  and  it  supports 
the  problem  solving  dynamics.  The  dynamics  follows  enchained  steps,  and  these  steps  are  related  to  specific  tools 
and  to  the  teacher  coordination  support.  Some  steps  of  problem  solving  demand  a shared  whiteboard.  The  WebSaber 
central  component,  that  makes  it  different  from  similar  environments,  is  precisely  the  whiteboard.  Cardoso,  Lima  e 
Silva  (2000),  in  the  context  of  a Computer  Science  bachelor  degree  project,  advised  by  the  author  of  this  paper, 
developed  a whiteboard,  that  can  be  accessed  from  WebSaber  environment.  The  tool  is  composed  of  the  whiteboard 
it  self,  a chat  software,  an  individual  notepad  and  mechanisms  for  awareness,  brainstorming  and  voting.  The 
whiteboard  allows  file  downloading  and  importing.  The  multi-user  interface  is  based  on  the  tightly  cooperation 
mode,  using  the  WYSIWIS  (what  you  see  is  what  I see)  approach.  It  assures  the  sharing  of  the  work  context  for  all 
logged  users.  System  scalability  does  not  restrict  the  number  of  participants,  whose  access  the  whiteboard  by  typing 
valid  username  and  password.  It  is  the  security  mechanism.  WebSaber  also  has  its  own  e-mail  and  chat  servers. 

The  evaluation  and  tests  phases  met  problems.  A critical  point  in  the  trajectory  of  WebSaber  I^^  version  was 
the  lack  of  a community  of  users,  whose  could  evaluate  the  system  in  real  use,  and  give  feedback  to  enhance  future 
versions.  Maybe,  the  system  security  politics  had  inhibited  its  use.  Tests  indicated  that  WebSaber  fulfils  technical 
requirements.  Over  100  Computer  Science  undergraduate  students  tested  the  prototype  performance,  attesting  its 
scalability,  extensibility  and  security.  In  these  cases,  the  protocols,  the  priorities  and  restrictions  mechanisms  have 
performed  well.  These  users  related  that  it  is  easy  to  work  in  the  Tutor  Mode,  but  they  are  not  the  intended  audience. 


Conclusions  and  Future  Works 

The  Internet  client-server  architecture  allows  the  WebSaber  gradual  extension  and  upgrading.  We  received 
Governmental  grants  to  rethink  and  improve  the  system.  We  are,  at  the  mo  ment,  refining  coordinating  mechanisms, 
designing  a new  interface  and  developing  a set  of  tools  (a  forum,  a graphics  editor,  a new  whiteboard)  and  adding 
two  software  agent  to  improve  cooperation.  WebSaber  is  primarily  a work-in-progress  and  an  academic  project.  Our 
research  goal  is  to  put  available  an  experimental  space,  where  teachers,  undergraduate  students  and  researchers  can 
think  and  rethink  about  cooperative  learning,  new  ideas  and  innovative  features  for  CSCL  environments. 
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Abstract:  The  way  to  obtain  information  has  exceedingly  evolved  thank  to  the  development 
of  new  technologies  and  the  possibilities  that  multimedia  offers.  The  project  presented  in  this 
paper  aims  to  create  a Community  of  Digital  Libraries  that  allow  euskaltegiak  (schools  to 
learn  Basque),  and  more  concretely,  their  students,  study  Basque  language  by  using  multi- 
media  applications.  CyberBiblio,  our  particular  Digital  Library,  is  a local  solution  with  con- 
tents of  general  interest.  It  is  based  on  a simple  scalable  architecture  and  uses  lesource- 
sharing  mechanisms  to  achieve  its  objectives.  This  paper  describes  in  detail  aspects  that 
should  be  considered  if  one  wanted  to  implement  such  a service,  highlighting  some  problems 
one  must  resolve. 


Introduction 

The  explosion  of  educational  multimedia  applications  is  a reality,  and  some  euskaltegiak  (schools  to 
learn  Basque)  have  embraced  this  option.  These  euskaltegiak  have  added  some  workspaces  where  people  can 
develop  their  Basque  language  skills  on  their  own  through  books  and  computer  programs.  The  first  step  begun 
in  2000  with  HEZINET  (P6rez,  T.  A.  et  al.  1999),  a system  for  distance  language  learning  called  Hezinet.  This 
product  uses  multimedia  and  artificial  intelligence  techniques  to  offer  adaptation  to  the  user.  At  the  moment  the 
system  is  being  used  in  about  2500  workstations  in  the  Basque  country  and  Latin  America. 

In  order  to  complete  the  system  more  reference  material  is  demanded.  Not  only  books,  but  also  interac- 
tive programs  to  complete  the  workspaces  already  mentioned.  The  use  of  one  of  such  programs  is  simple,  but 
when  there  are  several  ones  to  manage  the  use  becomes  not  so  easy.  For  example,  some  of  them  need  to  have 
the  original  CD  in  the  drive  to  work  properly.  This  would  be  fine  if  we  used  only  one  of  such  CDs,  but  becomes 
unmanageable  when  the  number  of  these  systems  increases.  If  we  also  wanted  to  optimize  the  nurrber  of  CDs, 
we  had  to  restrict  the  use  or  install  and  uninstall  such  systems  based  on  the  users’  demand. 

To  overcome  these  drawbacks  we  have  developed  a digital  library  (DL)  to  be  used  in  the  environment 
previously  described.  We  call  this  DL  CyberBiblio.  It  contains  interactive  multimedia  programs.  Users  have  the 
opportunity  to  access  every  material  included  in  the  system  from  their  workstations  without  being  worried 
about  which  CD  is  in  a drive,  or  which  requirements  the  software  has  or  without  having  to  install  or  uninstall 
any  of  the  programs  in  their  workstations. 

The  challenge  has  been  to  build  such  a system  having  in  mind  the  idea  that  the  system  would  grow  in 
the  near  future  and  the  architecture  for  the  system  would  have  to  be  stable  to  support  that  problem.  Besides  the 
response  time  would  have  to  be  acceptable  in  such  cases.  In  other  words,  the  system  was  required  to  be  scal- 
able: if  we  wanted  to  increase  the  number  of  workstations  or  the  quantity  of  material  included  in  the  system  the 
architecture  have  to  adapt  itself  to  those  changes  without  [major]  changes  in  the  system.  Even  to  add  new 
services  the  system  has  to  be  able  to  include  them.  Also,  the  architecture  has  had  to  be  simple:  easy  to  under- 
stand and  build.  The  users  of  the  system  would  be  non-specialized  people.  The  administrators  of  the  system  will 
be  people  with  a limited  experience  on  computers.  So  a complicated  organization  can  guarantee  the  failure  of 
the  system.  Finally,  the  system  has  to  be  fault  tolerant  and  economically  affordable,  that  is  to  say  that  some  of 
the  requirements  can  vary  slightly  in  order  to  get  a cheaper  (let  us  say,  not  so  expensive)  system. 

In  this  paper  we  present  the  major  problems  we  have  faced  to  build  a system  like  the  one  described  be- 
fore, and  the  way  we  have  solved  them.  To  do  this  we  first  introduce  a taxonomy  for  digital  libraries,  then  we 
describe  with  some  detail  the  library  and  then  the  architecture  of  CyberBiblio.  Afterwards,  we  discuss  some 
problems  overcome  during  the  implementations  of  our  particular  DL  and  some  conclusions  and  future  lines. 


A Taxonomy  for  Digital  Libraries 


There  are  several  criteria  to  classify  DLs  These  include  (1)  the  media  used  on  their  documents,  (2)  the 
kind  of  documents  they  store,  (3)  the  geographic  scale  of  the  network  they  use,  (3)  deployment  of  the  informa- 
tion and  (4)  user-access  policies. 

According  to  the  media  used  on  their  documents,  DLs  may  be  classified  in  text  file,  audio,  image  libraries  or 
multimedia  libraries.  The  library  created  by  the  Gutenberg  Project  (http://www.Dro mo. net/p^  contains  basically 
texts.  Texts  are  usually  formatted  or  digitized  form  a printed  book,  a map  or  even  from  an  Egyptian  papyrus 
such  as  the  documents  in  Tehtunis,  one  of  the  sections  of  the  Berkeley  Digital  Library 
(http://sunsite.berkelev.edu).  Texts  usually  also  include  some  pictures  or  graphics.  The  Printed  and  Photographs 
Online  Catalog  (PPOC)  from  The  Library  of  Congress  (LOC)  (http://www.loc.gov)  or  the  Microsoft  Design 
Gallery  Live  (http://dgl.microsoft.com)  are  exa  mples  of  image  libraries.  The  LOC  also  contains  a catalog  of  its 
audio  section  of  the  library,  called  SONIC.  Another  audio  library  can  be  the  'bihlioteca  de  voces''  (library  of 
voices)  included  in  the  Miguel  de  Cervantes  Digital  Library  (http://cervantesvirtual.com).  which  is  a result  of  a 
project  to  make  available  best-known  books  (such  as  Don  Quixote)  to  visually  challenged  or  illiterate  people. 
Even  the  Napster  Community  (http://www.napster.com/ ) can  be  considered  an  audio  library  which  requires  a 
specific  software  to  browse  it.  Although  a library  can  contain  several  of  these  types  of  documents,  we  cannot 
say  it  is  multimedia  unless  it  works  and  relates  with  all  those  media  altogether  (or,  at  least,  some  of  them).  We 
could  describe  some  of  the  libraries  as  ''multiple  libraries  of  one  specific  media 

Considering  the  contents  of  the  documents,  there  are  different  kinds  of  libraries.  There  are  general 
purpose  DL  such  as  the  LOC.  This  DL  is  open  to  many  topics,  whilst  other  are  devoted  to  a more  specific 
purpose^  such  as  the  ACM  Digital  Library  (http://www.acm.org/dl).  which  is  about  computer  science. 

If  we  look  at  the  geographic  dimensions  of  the  network,  two  types  of  DLs  can  be  found.  On  the  one 
hand,  those  that  cover  the  worldwide  area,  based  on  the  use  of  Internet.  On  the  other  hand,  the  ones  that  consti- 
tute a local  solution,  based  on  an  intranet.  The  former  ones  are  accessible  from  any  computer  connected  to 
Internet  while  the  latter  ones  offer  their  services  to  a more  concrete  audience,  restricting  access  to  a local  area 
network. 

Another  criteria  are  the  number  of  places  used  to  store  the  documents.  In  that  sense,  centralized  librar- 
ies store  the  documents  in  one  place  and  distributed  ones  have  several  document  containers  spread  out  accord- 
ing to  the  geographic  dimensions  of  the  network  they  use. 

Finally  we  include  the  user-policy  as  another  criteria  to  consider.  Free-access  libraries  allow  users  to 
browse  through  their  virtual  premises  and  read  or  consult  anything  they  want.  Some  other  libraries  only  allow 
browsing  and  charge  users  for  reading.  Others  charge  even  for  browsing.  The  latter  ones  are  restricted\\hx2iX\QS. 
Of  course,  we  have  to  take  into  account  their  geographic  dimensions.  An  example  of  the  former  can  be  a library 
in  a town  that  allows  users  to  access  to  the  e-books  in  their  computers,  and  the  only  thing  you  have  to  do  to 
access  that  books  is  to  go  in  and  use  the  computer.  We  would  consider  that  library  as  an  example  of  the  latter  if 
that  library  would  only  let  use  the  computers  to  women,  people  over  18,  those  who  had  paid  an  amount,  etc. 
Usually,  free-access  library  are  owned  by  public  entities  such  a city  hall,  a University,  a non-profit  foundation, 
etc.  Restricted  ones  are  owned  by  groups  of  people  with  some  common  interest. 


CyberBiblio 

The  project  CyberBiblio  has  been  carried  out  between  the  Hypermedia  and  Multimedia  Group  of  the 
University  of  the  Basque  Country  and  with  the  help  of  a private  foundation  dedicated  to  the  development  of 
Basque  Language  called  Aurten  Bai.  The  principal  tasks  have  been  done  during  2000.  In  2001  we  have  started 
to  install  some  units  of  these  community  of  libraries  through  the  Basque  Country  (in  either  euskaltegiak  or  city 
libraries). 

CyberBiblio  is  a DL  of  interactive  programs  published  in  the  Basque  Country.  The  catalog  shows  a list 
of  the  available  titles  at  that  time  (Fig.  1 ).  The  main  characteristic  of  the  shared  material  is  that  use  multimedia 
presentations  like  animations  made  with  Macromedia  Director,  Flash,  ToolBook,  etcetera.  Also,  users  could 
find  other  documents,  like  mp3  audio  files,  pdf  or  word  documents,  and  so  on.  In  general,  all  kinds  of  material 
available  on  CD-ROM.  Usually,  it  contains  atlas,  encyclopedias,  learning  material  for  children,  games,  mono- 
graphs, Basque  music...  everything  one  can  use  to  learn  Basque  or  increase  his/her  knowledge  about  Basque 
culture. 
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Figure  1:  CyberBiblio  Catalog 


Corresponding  to  the  criteria  given  in  section  2,  we  could  say  that  CyberBiblio  is  to  be  used  in  an 
Intranet.  Since  we  do  not  hold  the  copyrights  of  all  the  programs  we  use  in  the  DL,  we  are  not  entitled  to  share 
them  with  all  users  in  Internet.  There  would  be  no  problem  to  show  the  catalog,  but  anything  beyond  that  point 
would  violate  the  rights  of  some  of  the  programs  the  DL  includes.  It  also  has  been  designed  as  a distributed 
library  since  the  programs  can  be  stored  in  different  computers.  The  catalog  is  in  charge  of  indicating  where  to 
find  the  desired  resource.  The  target  environment  would  decide  the  user  policy.  In  the  case  of  CyberBiblio  we 
have  both  policies.  When  it  is  installed  in  a euskaltegia  the  only  users  allowed  to  use  it  are  the  students  of  that 
school.  However,  we  have  also  installed  the  same  DL  in  public  libraries,  in  which  every  user  of  the  library  (or  a 
visitor)  can  access  to  the  conputers  (of  course,  if  there  is  some  space  available). 

In  order  to  classify  the  purpose  of  CyberBiblio,  we  should  say  that  the  main  purpose  was  to  provide 
contents  to  learn  Basque.  Nevertheless,  these  contents  have  grown  and  the  purpose  is  to  spread  Basque  culture, 
as  a traditional  library  could  do.  So  we  can  say  that  it  has  a specific  purpose  for  a group  of  population  (Basque 
students)  and  a general  purpose  for  those  who  already  know  the  language.  Currently  the  contents  are  limited  to 
Basque  Country  issues.  The  implementation  is  always  in  a library  either  in  the  Basque  school  or  a public 
library. 


Architecture  of  CyberBiblio 

Since  CyberBiblio  has  been  developed  to  share  multimedia  applications  distributed  on  CD-ROM,  it  is 
necessary  to  find  a way  to  do  it.  During  de  design  phase  of  the  system  we  consider  two  main  ideas:  install  on- 
demand  and  resource  sharing. 


Install  on-demand 

This  configuration  emulates  the  behavior  of  a jukebox.  When  users  browse  the  catalog,  they  can  select 
(ask  for)  any  application  included.  The  system  first  controls  whether  the  application  has  been  already  installed 
in  the  system,  if  not,  it  searches  through  the  CD-ROM  base  and  installs  the  application  on  the  corresponding 
PC.  When  he  stops  using  the  software  required,  the  catalog  would  be  shown  again. 
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BEST  COPY  AVAIMBLE 


This  option  assumes  than  periodically  the  system  would  clean  the  workstations,  because  the  installa- 
tion, uninstallation  and  reinstallation  process  leaves  sometimes  files  on  disks  and  the  registry  is  not  left  as  it  was 
found  at  the  beginning.  Also,  the  system  has  to  be  able  to  uninstall  some  software  due  to  space  requirements 
(there  is  not  enough  disk  space  for  another  application).  Finally,  the  process  of  install  and  uninstall  applications 
add  a delay  (usually  several  minutes)  to  the  system  that  may  not  be  assumed  by  the  users. 

To  accomplish  this  alternative,  there  is  some  software  available.  For  example,  Winlnstall  from  Micio- 
mouse  (www.micromouse.com)  or  Microsoft  Windows  Installer  (http://msdn.microsoft.com/librarv/ 
default.asp?URL-/librarv/deVDrods/vs6/vstudio/vsinstal/veconmicrosoftwindowsinstallertcchnology.htm)  from  the 
Microsoft’s  Zero  Administration  Kit  (ZAK)  or  Microsoft  System  Management  Server 
(http://www.microsoft.com/catalog/displav.asp?subid=22&site=650&x=22&v=l  1 ). 

Resource  Sharing 

In  this  approach,  the  resources  are  shared  from  the  place(s)  they  are  stored.  This  is  not  a new  concept. 
In  fact,  every  operating  system  includes  this  functionality.  Microsoft  Windows  2000  includes,  for  example 
terminal  server  services  and  sharing  of  file  system. 

This  alternative  presents  some  advantages  when  compared  to  the  previous  solution.  First  of  all,  it  is  not 
necessary  to  wait  for  the  applications  to  be  installed.  Although  they  have  to  be  installed,  this  process  is  done 
only  once.  Besides,  there  is  no  need  to  use  the  network  to  do  that.  However,  in  this  case  the  network  is  more 
frequently  used. 

The  terminal  server  let  each  terminal  establish  a session  in  one  of  the  computers  that  contains  the  re- 
quired resource.  The  user  works  directly  against  the  application  container,  but  the  execution  seems  to  be  hap- 
pening on  the  local  computer.  This  is  a cheap  and  simple  solution  unless  you  have  to  use  local  peripherals  from 
the  server  execution  (for  instance,  if  you  want  to  listen  to  an  audio  file  on  the  speakers  of  your  computer  instead 
of  the  ones  on  the  server;  or  local  disk  drives;  etc.).  This  problem  can  be  overcome  using  the  Independent 
Computer  Architecture  (ICA)  from  Citrix  httD://www.citrix.com/products/clients/ica/technology.asD)  at  a not  so 
cheap  price  as  the  operating  system  software  like  Windows  2000. 

An  alternative  is  to  use  the  file-sharing  feature  of  the  operating  system.  Using  this  approach,  each 
workstation  would  have  to  have  access  to  the  files  installed  in  one  common  place  and  there  will  be  necessary  to 
do  several  small  actions  in  each  workstation  (modify  the  register  when  appropriate,  add  some  dll’s,  etc.) 

Decision  taken 

Our  choice  has  been  resource  sharing  using  the  file  system  sharing  of  the  operating  system.  The  as- 
pects we  have  taken  into  account  to  make  a decision  include  the  quality  of  the  solution,  the  difficulty  of  its 
implementation,  and  its  cost  (one  of  our  main  handicaps).  In  our  case,  the  price  avoided  the  election  of  ICA 
protocols,  one  of  our  best  choices.  We  also  consider  using  Windows  2K,  but  its  price  push  us  to  choose  to  use  it 
only  in  a server  machine  and  use  other  operating  systems  like  Windows  Millenium,  also  from  Microsoft.  This 
choice  implies  a more  difficult  way  of  implementing  the  security  in  the  system  but  accomplishable  (although 
less  transparent  to  users). 


Problems  Overcome 

Most  of  the  problems  of  the  implementation  of  CyberBiblio  arose  when  adding  applications  to  the  li- 
brary (at  installation  level)  and  others  to  maintain  system  security  (avoid  users  misuse  the  system).  In  the 
following  paragraphs  we  will  detail  both. 

Problems  at  application  level 

Multimedia  and  interactive  applications  shared  in  a system  usually  try  not  to  overload  the  computers  in 
which  they  are  executed.  So  only  a small  part  of  data  is  copied  in  the  user  disk,  being  necessary  the  CD-ROM 
to  access  the  rest  of  information.  Based  on  our  experience  with  the  installation  of  multimedia  applications,  we 
can  distinguish  four  types  of  installation: 

1.  Direct  copy.  In  fact  the  applications  in  this  category  do  not  need  to  be  installed.  There  are  some 
executable  programs  and  data  files.  The  installation  process  is  as  easy  as  copying  the  files  to  a 
shared  device  (commonly  a shared  hard  disk). 


2.  Files  with  a specific  reader^  In  this  case,  the  installation  process  is  similar  to  the  one  above.  How- 
ever, there  are  common  (freeware)  reader  programs  to  access  to  these  types  of  documents.  In  this 
category  we  could  find  pdf,  word  or  postscript  documents;  audio  files  in  mp3  or  wav;  video 
documents  in  quicktime  or  mpeg  format.  Or  even  navigable  documents  in  HTML.  In  this  case  it  is 
necessary  to  install  the  specific  reader  at  each  workstation. 

3.  Simple  install  applications.  Our  experience  tells  that  most  of  the  applications  belong  to  this  type. 
In  this  group  we  include  those  applications  that  include  an  installation  program  responsible  of  put- 
ting the  files  in  the  appropriated  location.  This  program  is  generally  called  install  or  setup.  What 
makes  this  process  simple  is  that  the  program  does  not  need  to  modify  the  workstation  computer 
registry.  Sometimes  it  is  necessary  to  include  some  library  files  (DLL)  on  the  workstations  to 
achieve  a correct  execution.  Sometimes  there  are  incompatibilities  between  requirements  of  one 
application  and  another.  For  example,  two  applications  use  the  same  DLL.  But  one  of  their  instal- 
lation programs  do  not  care  about  which  one  is  available  in  the  system  at  one  moment.  So,  they  al- 
low the  substitution  of  newer  DLLs  with  older  ones  interfering  with  the  other  application  execu- 
tion. 

4.  Complex  install  applications.  The  applications  in  this  group  include  installation  program  as  the 
previous  ones,  but  during  this  process  or  during  executions  some  problems  arise.  For  example, 
some  of  the  applications  check  if  the  CD-ROM  is  present  on  the  machine  during  their  execution. 
These  types  of  applications  have  being  the  most  difficult  ones  to  install  in  order  to  achieve  sharing 
access.  Though  we  installed  most  of  them,  there  are  only  two  that  we  couldn’t  share  through  the 
network.  In  these  cases  we  should  establish  an  alternative  method  to  use  them.  For  example,  tradi- 
tional CD-ROM  borrowing. 

Problems  at  security  level 

We  have  considered  very  important  to  prevent  the  workstations  from  potential  malicious  use.  Some- 
times users  know  more  about  computers  than  librarians.  So  we  have  spent  some  time  determining  a security 
policy.  This  alternative  could  prevent  from  actions  that  do  not  let  the  system  remain  configured.  We  have 
considered  that  the  best  option  is  to  prevent  people  to  access  the  workstations  with  other  purposes  than  access- 
ing the  catalog.  The  operating  system  can  help  to  reach  the  desired  security  level.  For  example,  in  Windows 
2000  Server  in  combination  with  Windows  2000  Professional  it  is  easy  to  establish  a security  policy  and  to 
spread  it  to  the  workstations. 

However,  if  we  want  to  do  it  in  simpler  operating  systems  (like  Windows  9x  or  Me)  it  is  necessary  to 
work  directly  with  the  registry.  Also,  we  must  define  which  resources  we  want  to  show  in  the  desktop  (none)  or 
which  ones  at  the  start  menu  (only  applications  from  the  catalog).  And  disable  some  common  functionalities 
(for  example,  right  mouse  button  at  desktop). 

In  our  case,  we  decided  to  limit  the  access  to  the  resources.  Users  only  have  access  to  the  catalog  and 
the  applications  it  shows.  So,  if  the  identification  is  correct,  he/she  will  be  able  to  access  the  information,  but 
only  to  CyberBiblio  (Fig.  2).  If  identification  fails,  the  only  available  option  is  to  close  de  session  and  back  to 
the  user  identification  screen. 


Another  problem  we  had  to  solve  during  CyberBiblio  implementation  process  has  been  the  need  to 
write  some  own  data  of  several  applications.  Some  applications  need  write  permission  in  some  folders.  For 


Figure  2:  Operating  system  interface  (limited  to  CyberBiblio) 


example  in  those  that  contain  configuration  files.  Other  folders,  also,  offer  some  free  space  where  user  can  write 
information.  This  has  been  a very  important  aspect  considered,  because  it  is  not  recommendable  to  let  users 
modify  system  files  and  folders  in  order  to  prevent  an  stable  behavior  of  the  system. 


Conclusions  and  Future  Work 

In  this  paper  we  have  presented  CyberBiblio,  a DLs  community  centered  on  Basque  culture.  CyberBib- 
lio  represents  a significant  contribution  to  scientific  community,  because,  as  far  as  we  know,  there  is  no  other 
DL  within  CyberBiblio  characteristics.  Also  to  Basque  society,  because  its  installation  will  notably  improve  the 
Basque  language  learning  and  normalization.  And  by  extension,  to  society  in  general,  because  it  is  very  simple 
to  implementing  DLs  with  similar  characteristics  in  other  sites  and  in  other  contexts. 

Introduction  of  CyberBiblio,  the  first  DLs  community  in  the  Basque  Country,  has  being  a revolution  in 
divulgation  of  multimedia  material.  This  is  because  it  guarantees  almost  all  population  access  to  a large  amount 
of  multimedia  information  published  on  CD-ROM.  And  everything  in  a quickly  simple  way. 

As  a result  of  this  project  there  is  a commercial  product  now  implemented  in  different  places  of  the 
Basque  Country.  At  the  moment  there  are  three  DL  in  the  local  libraries  at  Amorebieta,  Balmaseda  and  Bara- 
caldo.  There  is  another  DL  located  at  Zornotza-Amorebieta  Barnetegi,  a school  dedicated  to  the  learning  of 
Basque  language.  Now  we  are  in  contact  with  other  town  or  city  halls  of  the  Basque  Country  to  show  them  the 
product,  its  possibilities  and  characteristics.  Our  main  objective  is  creating  the  major  number  of  public  places 
dedicated  to  Basque  language  learning  and  practice.  Also  we  pretend  to  include  there  other  computing  tools  like 
HEZINET. 

Apparently,  this  is  only  the  beginning  of  a progressive  and  constant  settlement  of  Multimedia  DLs  in 
the  Basque  Country.  Librarians’  experience  by  now  has  been  very  positive.  As  well  as  users’  opinion,  which 
has  flooded  all  our  perspectives,  and  give  us  enough  courage  to  improve  the  library  and  introduce  new  services. 
In  fact,  we  are  now  working  on  a version  of  CyberBiblio  to  Spanish  culture. 
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Abstract:  In  most  universities,  the  required  undergraduate  courses  have  high  enrollment  (multiple  sections)  and 
are  frequently  taught  by  part-time  instructors  or  graduate  students.  Part-time  faculty  are  an  important  asset  to 
any  university,  but  this  group  can  also  present  a wide  range  of  professional  qualifications  and  teaching 
.experience  which  can  result  in  significant  variations  in  the  depth  and  quality  of  instruction  This  paper  reports  on 
the  collaborative  effort  by  full  and  part-time  faculty  in  an  urban  Midwestern  College  of  Education  to  improve 
the  consistency  of  course  content  and  quality  of  instruction  between  multiple  sections  of  a core  undergraduate 
course.  One  strategy  used  to  reduce  the  threat  of  “course  drift”  was  the  development  of  web-based  resources  to 
support  both  the  instructor  and  student. 


In  most  universities,  the  required  undergraduate  courses  have  high  enrollment  (multiple 
sections)  and  are  frequently  taught  by  part-time  instructors  or  graduate  students.  The 
National  Center  for  Education  Statistics  reported  that  as  of  Fall  1998,  43  percent  of 
postsecondary  instructional  faculty  and  staff  were  employed  part  time  as  defined  by  their 
institution  (2001,  p.  2).  An  additional  survey  by  the  Coalition  on  the  Academic 
Workforce  of  humanities  and  social  science  disciplines  showed  that  less  than  half  of 
introductory  undergraduate  courses  were  taught  by  full-time  tenured  faculty,  and 
approximately  43  percent  of  all  undergraduate  courses  in  those  disciplines  are  taught  by 
part-time  faculty  or  graduate  students  (Townsend  2000,  p.  2). 

Part-time  faculty  are  an  important  asset  to  any  university,  but  this  group  can  also  present 
a wide  range  of  professional  qualifications  and  teaching  experience  which  can  result  in 
significant  variations  in  the  depth  and  quality  of  instruction  (Lankard  1993,  p.  1).  In  the 
case  of  large  classes  with  multiple  sections,  the  use  of  part-time  instructors  or  graduate 
students  contributes  to  significant  problems  with  consistency  and  coherence  to  learning 
objectives  (Roberts  2001,  p.  2). 

This  paper  reports  on  the  collaborative  effort  by  full  and  part-time  faculty  in  an  urban 
Midwestern  College  of  Education  to  improve  the  consistency  of  course  content  and 
quality  of  instruction  between  multiple  sections  of  a core  undergraduate  course.  One 
strategy  used  to  reduce  the  threat  of  “course  drift”  was  the  development  of  web-based 


resources  to  support  both  the  instructor  and  student.  The  web  site  of  shared  course 
materials  was  intentionally  designed  to: 

1 . Improve  consistency  and  quality  of  instruction  throughout  all  sections  of  the 
course. 

2.  Reduce  the  preparation  time  for  faculty. 

3.  Provide  class  materials  such  as  PowerPoint  presentations,  web  links,  assignments, 
and  other  related  materials  in  one  easy  to  find  location. 

4.  Model  Internet  resource  use 

The  web  site  was  developed  using  Microsoft  Word,  PowerPoint,  and  FrontPage,  and  was 
made  accessible  to  students  and  instructors  approximately  one  month  after  the  start  of 
classes  in  the  Fall  2001  semester.  The  web  site  has  been  enhanced  and  modified  during 
its  use  in  the  Spring  2002  semester.  A survey  and  set  of  interview  questions  have  been 
developed  to  collect  data  from  students  and  instructors  on  the  utility  of  the  shared  web 
resources.  Student  work  will  be  collected  from  sections  that  used  the  web  resources  and 
from  sections  that  did  not  to  compare  the  quality  of  the  products.  Additional  sources  of 
information  include  grades,  course  evaluations,  and  student  comments  on  those 
evaluation  forms. 
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Abstract:  Are  there  online  activities  that  can  encourage  exploration,  risk-taking,  cooperation  and  experimentation  that 

both  teacher  and  learner  can  benefit  from?  This  brief  paper  will  include  a discussion  of  two  effective  discussion 
techniques  for  teacher  trainers  and  second  language  learners  that  have  been  used  in  an  online  format.  Kiva,  a discussion 
technique  adapted  from  the  Native  Americans  of  the  America  Southwest,  can  be  used  as  a forum  for  discussing  issues  of 
professional  interest  such  as  the  role  of  the  teacher  trainer  and  contemporary  topics  of  interest  to  language  learners.  Six 
Colored  Hats  is  a technique  that  promotes  whole  brain  thinking  and  problem  solving  from  varied  points  of  view.  Both 
techniques  can  be  used  in  an  online  forum  to  invite  reflection  upon  issues  related  to  teaching,  teacher  training,  and 
learning.  Both  Kiva  and  Six  Colored  Hats  are  carried  out  in  a CALL  lab  using  the  LAN  to  post  questions. 


Introduction 

Techniques  for  creating  successful  online  discussion  are  similar  to  techniques  used  to  facilitate  face-to-face 
discussion.  Both  are  characterized  by  the  following: 

• They  emphasize  active  participation  and  interaction 

• They  have  a task,  theme  or  goal 

• They  gather  a range  of  ideas,  opinions  and  concerns 

• They  are  usually  run  by  a group  leader  or  facilitator 

There  are  many  discussion  activities  that  encourage  exploration,  risk-taking,  cooperation  and  experimentation  that  both 
teacher  and  learner  can  benefit  from.  “Talk  is  a major  means  by  which  learners  explore  the  relationship  between  what 
they  already  know  and  new  observations  or  interpretations  which  they  meet.”  (Barnes,  1976;  in  Cullinan,  1993) 

Computer  Mediated  Communication  (CMC)  and  Discussion 

Computer  mediated  communication  permits  the  learners  to  benefit  from  the  shared  experience  of  a group 
engaged  in  the  same  study  and  the  opportunity  to  measure  his  or  her  ideas  against  those  of  others  in  the 
group.  (Tam,  2000)  Many  group  discussion  techniques  can  be  facilitated  through  the  use  of  a Computer 
Assisted  Language  Lab  (CALL).  Two  such  techniques,  Kiva  and  Six  Colored  Hats  have  been  used  in  a 
CALL  lab  to  great  advantage. 

KIVA 

Kiva  is  a discussion  technique  that  has  been  adapted  from  the  Native  Americans  of  the  American  Southwest. 
It  develops  critical  thinking  skills  by  causing  participants  to  identify  and  cite  reasons  for  their  decisions 
making  within  the  group.  Kiva  also  encourages  creative  language  learning  as  second  language  learners 
combine  (orally)  ideas  and  responses  in  novel  ways  to  produce  complex  thinking  patterns. 

How  does  it  work  in  the  CALL  lab? 

Traditionally,  participants  in  the  Kiva  are  placed  in  two  concentric  circles,  the  inner  circle  consisting  of  four 
to  five  members.  Only  the  inner  circle  speaks,  discussing  the  chosen  topic.  Members  of  the  outer  circle 


listen.  After  a set  time  has  been  called,  inner  and  outer  circle  members  exchange  places  with  each  other  and 
the  discussion  continues.  In  the  CALL  lab,  the  format  is  basically  the  same.  The  teacher/facilitator  posts  a 
question  to  4 - 5 students4)articipants.  These  members  of  the  “inner  circle”  discuss  the  discussion  topic  in  an 
asychronchrous  discussion.  After  three  days,  the  “inner  circle”  changes,  teacher  appointed  or  participant 
volunteers,  and  the  discussion  continues.  After  a week  or  10  days,  the  discussion  ends  with  the 
teacher/facilitator  summarizing  the  salient  points  of  discussion.  If  the  topic  was  a problem  solving  one,  then 
a course  of  action  is  recommended  and  posted  to  the  group.  An  example  of  an  appropriate  question  for 
teacher  trainers  might  be:  What  is  the  role  of  the  teacher  trainer?  A student  topic  might  be:  English  should  be 
the  official  language  of  the  United  States. 

Six  Colored  Hats 

Six  Colored  Hats  is  a management  technique  developed  by  Edward  DeBono  (1985).  It  has  been  adapted  to 
the  classroom  in  order  to  promote  whole  brain  thinking  and  problem  solving.  It  also  incorporates  lateral 
thinking  as  well  as  critical  thinking. 

How  does  it  work  in  the  CALL  lab? 

Six  Colored  Hats  six  modes  of  thinking,  represented  by  six  colors,  which  are  considered  directions 

to  think  in  rather  than  labels  for  thinking.  For  example,  when  given  a topic  to  discuss,  the  white  hat 
represents  facts  and  figures,  the  red  hat:  feelings  and  emotions,  black  hat:  judgment  and  caution,  yellow  hat: 
positive  aspects,  green  hat:  creative  ideas  (related  to  topic)  and  the  blue  hat:  a summary  of  topic  discussion. 

In  the  online  environment,  the  topic  for  discussion  is  posted  and  the  group  is  divided  evenly  to  one  color 
apiece.  Each  group  is  therefore  responsible  for  discussing  the  topic  according  to  the  aspect  of  the  hat  color 
they  are  assigned.  The  blue  hat  group  will  provide  a summary  at  the  end  of  the  discussion.  This  online 
discussion  activity  requires  advance  in-class  preparation  before  it  is  attempted  online.  An  example  of  an 
appropriate  question  for  teacher  trainers  might  be:  Egyptian  university  graduates  are  often  assigned  to  teach 
English  whether  they  want  to  or  not.  Is  this  an  appropriate  policy  and  what  can  be  done  to  solve  it?  (actual 
question  used  in  teacher  training  session  in  Egypt  by  author).  An  appropriate  problem  solving  task  for 
students  might  be:  Students  are  talking  while  others  are  talking  or  teaching.  What  can  be  done? 

Conclusion 

Kiva  and  Six  Colored  Hats  both  provide  a means  of  initiating  online  asynchronous  discussion  between 
teacher/facilitator  and  student/participant.  With  effective  moderation  and  preparation  by  the 
teacher/facilitator,  participants  will  be  able  to  achieve  their  educational  and  professional  goals.  The  potential 
of  telecommunication  technology  lies  in  its  ability  to  function  as  a gateway;  a gateway  to  resources, 
collaborative  learning  and  individual  achievement.  (Tam,  2000) 
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Abstract  In  this  presentation  we  will  demonstrate  an  instructional  CD-ROM, 

Investigating  Community  Environmental  Health  Problems,  and  describe  how  it  was 
incorporated  into  the  cooperative  learning  technique  called  structured  controversy  to 
teach  public  health  students  in  a Masters  level  course  in  Environmental  Health.  The 
program  was  designed  as  a resource  for  students  regarding  the  various  lessons  that  were 
learned  in  a well-known  case.  The  CD-ROM  was  produced  to  help  users  determine  the 
environmental  health  of  a community,  particularly  as  it  relates  to  the  presence  of 
hazardous  chemicals.  The  CD-ROM  relies  heavily  on  the  experiences  of  citizens  and 
public  health  professionals  working  in  the  community  of  Woburn,  Massachusetts  around 
the  contamination  episode  publicized  in  the  book  and  movie  "A  Civil  Action". 


Background  of  the  Course 

The  Woburn  Case  Study  was  used  in  a survey  course  in  Environmental  Health,  EH708  Introduction  to  Environmental 
Health.  EH708  is  a required  course  enrolling  approximately  100  students  not  concentrating  in  Environmental  Health. 
Although  student  interest  in  the  course  content  is  highly  variable,  the  course  must  introduce  students  to  the  widest 
range  of  environmental  health  problems.  This  may  be  the  their  only  exposure  to  an  environmental  health  course. 
However,  it  is  highly  likely  that  these  students  will  work  in  communities  facing  environmental  health  challenges,  which 
have  implications  for  other  dimensions  of  public  health  within  that  community. 

The  used  of  the  traditional  case  method  and  a classroom  strategy  called  structured  controversy  was  enhanced  through 
the  use  of  multimedia  CD-ROM  technology.  The  Woburn  CD  provides  a rich  context  for  students  to  identity 
environmental  health  data,  to  better  understand  the  implications  of  epidemiological  principles  and  how  to  apply  these 
principles  in  community  settings.  We  tell  the  Woburn  story  through  audio  and  video  segments  taken  from  local  cable 
documentaries  and  expert  panels,  as  well  as  documents,  photographs  and  other  media  used  in  or  produced  in  connection 
with  the  trial. 

Using  the  Case  Study  and  Structured  Controversy 

Several  weeks  before  the  scheduled  class  during  which  the  case  was  discussed  students  were  asked  to  explore  the  CD- 
ROM,  Investigating  Community  Environmental  Health  Problems^  along  with  several  printed  study  aids,  study 
questions  and  websites.  One  week  in  advance  of  the  scheduled  class  meeting,  the  class  was  divided  into  six  groups, 
each  group  taking  the  role  of  one  of  the  key  players  in  the  Woburn  story.  The  six  groups  were  organized  into  three  pairs, 
forming  the  basis  for  the  structured  controversy  exercise.  During  the  first  30  minutes  of  the  two -hour  class  period  each 
group  met  separately.  Each  group  was  given  a controversial  question  and  was  asked  to  prepare  a response  directed  to 
the  other  group  in  their  pairing.  During  the  second  part  of  the  class  period  a facilitated,  spirited  discussion  ensued  with 
students  speaking  in  their  roles  as  lawyers,  scientists,  etc. 

Pairings  and  Discussion  Questions 


i 


Question  for  citizens  group  and  Department  of  Public  Health  group: 

Like  many  environmental  health  problems,  the  possible  link  between  drinking  contaminated  water  and  risk  of  childhood 
leukemia  in  Woburn  was  laden  with  uncertainty.  Which  kind  of  error  do  you  think  would  be  worse  in  assessing  a 
situation  like  that  in  Woburn:  incorrectly  concluding  that  there  was  an  association  between  the  water  and  leukemia  if  in 
fact  there  wasn*t:  or  incorrectly  concluding  that  there  wasn*t  an  association  if  there  was? 

Question  for  scientists  group  and  press  group: 

In  Woburn,  government  agencies,  scientists,  local  residents,  and  the  legal  system  all  tried  in  their  own  ways  to  figure  out 
whether  industrial  pollution  of  drinking  water  was  causing  excess  leukemia.  What  role  do  you  think  community  members 
should  play,  if  any,  in  scientific  studies  of  pollution  and  health  in  their  neighborhoods? 

Question  for  WR  Grace  (the  industry  named  in  the  civil  lawsuit)  group  and  plaintiffs’  lawyers  group: 

Local  activists  who  took  on  WR  Grace  and  Beatrice  Foods  had  a lot  of  community  support  from  residents  who  were 
concerned  about  their  health.  On  the  other  hand,  some  residents  accused  the  activists  of  giving  Woburn  a bad  name, 
pushing  property  values  down,  and  driving  industry  out  of  town.  How  do  you  think  a community  can  best  balance  its 
concerns  for  environmental  health  and  economic  development? 

Session  Objectives 

1.  At  the  end  of  this  session,  students  will  be  familiar  with  some  of  the  details  of  the  Woburn  case,  such  as  the  scope  of 
the  childhood  leukemia  cluster,  the  source  of  drinking  water  contamination,  the  various  health  data  and  follow-up 
studies,  the  lawsuit,  and  the  clean-up  activities. 

2.  At  the  end  of  this  session,  students  will  be  able  to  articulate  problems  faced  by  various  stakeholders  in  Woburn, 
including  the  difficulties  citizens  faced  in  getting  health  studies  done  and  clean-up  begun,  the  concerns  of  health 
officials  not  to  create  false  alarm  but  also  not  give  false  reassurance,  and  the  conflict  between  economic  development  of 
the  community  and  health  and  well-being  of  its  citizens. 

3.  At  the  end  of  this  session,  students  will  have  an  understanding  of  the  role  that  citizens  and  citizen  organizations  can 
play  in  creating  pressure  on  government  agencies  to  act,  in  engaging  and  working  with  scientists  in  conducting  health 
studies,  and  in  following  through  on  efforts  to  reduce  harmful  exposures. 


Observed  outcomes  from  using  the  case  study 

The  case  study,  as  presented  via  the  CD-ROM,  met  the  objectives  of  familiarizing  students  with  many  details  of  the 
Woburn  case.  More  importantly,  when  combined  with  the  structured  controversy  class  activity,  it  allowed  students  the 
opportunity  to  "walk  a mile  in  another  person's  moccasins,"  or  understand  another  point  of  view  about  the  meaning  of 
the  toxic  contamination  problem  for  a community.  Because  the  format  of  the  class  activity  required  students  to  consider 
and  articulate  their  answers  to  specific  questions,  several  commented  that  they  had  learned  something  in  a new  way. 

The  level  of  energy  and  interaction  in  the  small  groups  and  the  class  wide  discussion  combined  with  student  feedback 
on  course  evaluations  confirmed  that  this  was  an  active  learning  process  that  largely  succeeded  in  meeting  our 
objectives. 

Summary 

During  our  presentation  we  will  demonstrate  this  highly  engaging  CD-ROM  and  share  video  clips  of  these  paired 
dialogues  and  subsequent  class  wide  discussion.  We  will  also  discuss  our  lessons  learned  from  producing  the  CD-ROM 
and  facilitating  its  use  in  a public  health  curriculum. 
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Abstract:  This  paper  provides  a short  description  of  the  theoretical  foundation,  accessibility 
criteria,  and  the  research  approach  related  to  the  project  "Accessibility  and  learning  in  higher 
education"  conducted  by  Malmoe  University  in  Sweden.  The  criteria  of  accessibility  are 
developed  in  connection  with  a pilot  course  under  the  projects’  management,  and  the  research 
approach  is  related  to  course  accessibility  and  how  disabled  students  participate  in  the 
collaborative  work  within  a course. 


Introduction 

Many  disabled  people  do  not  enter  universities  even  though  they  have  the  intellectual  ability.  The  situation  in 
Sweden  and  US  is  similar  (Horn  & Berktold,  1999;  National  Agency  for  higher  education,  2000).  New 
technology  has  given  disabled  people  better  opportunities  to  get  information  and  to  communicate  with  people 
from  different  places  all  over  the  world.  Distance  learning  can  be  a good  studying  alternative  for  disabled 
students,  just  as  it  is  for  non-disabled  students.  Malmoe  University  in  Sweden  is  conducting  a project  entitled 
“Accessibility  and  learning  and  higher  education”,  which  is  an  integrated  research,  development  and  training 
project.  The  theoretical  goal  of  the  project  is  to  combine  recent  contributions  in  collaborative  learning  theory 
with  advances  in  information  technology.  The  programme  focuses  on  making  online  university  education  more 
accessible  to  students  by  intelligent  use  of  the  potentials  of  computer  support.  In  November  2001  a pilot  course 
started  under  the  projects  management.  The  course  design  is  bas  ed  on  a theoretical  foundation  and  on  criteria  of 
accessibility  that  have  been  formulated  within  the  project.  To  receive  more  information  about  the  project,  read 
Schenker’s  and  Scadden’s  paper  “The  design  of  accessible  distance  education  environments  that  use 
collaborative  learning”  (Schenker  & Scadden,  2002).  The  theoretical  foundation,  the  criteria  of  accessibility 
and  a research  focus  of  the  project  will  shortly  be  presented  below. 

Theoretical  foundation 

In  the  social  oriented  learning  theory  the  context  for  learning  is  central.  The  existing  situation  drives  the 
learning  process  (Lave  & Wenger,  1991;  Vygotsky,  1978).  The  theoretical  foundation  is  partly  based  on 
Bakhtin’s  and  Vygotsky’s  theories  which  have  been  developed  by  Dysthe  (1996).  According  to  these  theories, 
dialogue  promotes  learning,  and  understanding  is  seen  as  an  active  process.  Learning  through  dialogue  means 
that  it  is  important  to  give  and  to  receive  responses.  Trying  to  understand  the  other  person’s  thoughts  and 
intentions  is  necessary,  when  giving  a response.  Dysthe  outlines  also  that  the  differences  between  thoughts 
presented  orally  or  in  written  text,  are  a learning-potential.  Furthermore,  there  must  be  a confrontation  between 
voices  to  make  learning  occur  - people  must  meet  in  a dialogue  (Bakhtin,  1981;  Dysthe,  1996).  The  viewpoint 
is  thus  that  learning  is  a social  process.  Even  if  learning  is  a social  process,  the  process  is  also  individual.  It  is 
important  to  help  students  to  be  aware  of  and  reflect  upon  their  learning  approaches.  Bruner  (1996)  states  that  a 
teacher  who  supports  and  encourages  students’  learning  processes  is  more  important  than  a lecturer. 

Criteria  of  accessibility 

The  degree  of  course-accessibility  for  disabled  students  depends  on  several  issues.  The  section  below  identifies 
some  of  these  issues.  For  every  issue,  one  or  more  criteria  are  formulated.  The  criteria  are  a synthesis  of  the 
theoretical  foundation,  experiences  carried  out  by  project  members,  and  their  recommendations.  The  criteria 
relate  the  entire  spectra  from  course  providers,  students,  educational  design,  and  technological  issues. 

1.  Educational  design:  (A)  Course  assignments  help  students  to  explore  their  personal  learning  preferences. 
(B)  Flexibility  in  the  conduct  of  course  assignments  and  activities  is  provided  to  ensure  that  students  can 
select  the  most  appropriate  learning  environment  and  style. 


2.  Presentation  of  course  content:  (A)  Course  materials  that  are  obligatory  are  accessible  for  all  students.  (B) 
The  design  of  the  course  content  and  its  presentation  in  formats  allow  students  to  select  the  most 
appropriate  and  preferred  mode  to  meet  their  learning  needs  and  preferences.  (C)  The  guidelines  for  the 
design  of  accessible  educational  software  developed  by  the  CPB/WGBH  National  Center  on  Accessible 
Media  [1]  are  followed. 

3.  The  courseware  system  (a  web-based  environment  for  distance  education  courses):  (A)  It  is  possible  to 
enroll  in  the  course  without  installing  additional  software  beyond  that  which  is  normally  needed  to  use  and 
communicate  via  the  Internet.  (B)  The  courseware  system  is  built  using  guidelines  for  accessible  web 
content  developed  by  the  World  Wide  Web  Consortium  (W3C)  [2]. 

4.  The  approach  used  by  the  course  providers:  (A)  The  course  providers  are  service  minded  and  give 
individual  support  when  it  is  needed.  (B)  The  course  providers  are  flexible  and  prepared  to  make  decisions 
that  promote  course  accessibility. 

5.  The  capacity  of  the  student’s  hardware,  browser,  and  adaptive  technology:  (A)  Students  with  disabilities 
have  current  computer  hardware,  modems  with  speed  adequate  to  handle  the  kind  of  presentations  required 
in  the  course,  and  the  most  appropriate  adaptive  technology  and  display  parameters.  (B)  Course  providers 
assist  students  by  providing  guidance  and  hints  on  improving  accessibility  through  optimal  use  of  students* 
hardware  and  software. 

6.  The  student’s  disability  and  degree  of  functional  limitation:  (A)  No  student  is  disconnected  from  any  part 
of  the  course  due  to  his  or  her  functional  impairment. 

Research  focus 

The  research  focus  relates  to  disabled  students  who  are  participating  in  a distance-learning  course  that  has  a 
collaborative  net-based  environment.  The  first  sub-focus  is  to  test  the  criteria  of  accessibility  in  the  course.  The 
students  will  be  asked  to  rate  the  accuracy  of  the  criteria.  The  students  will  also  be  asked  to  give  feed  back  on 
how  the  criteria  can  be  developed  further.  Another  sub-focus  is  to  interview  the  students  about  their  written 
contributions  in  the  courseware  system.  Are  the  written  contributions  comparable  with  the  student’s  actual 
participation  in  the  dialogue?  The  purpose  is  also  to  learn  more  about  the  context  in  which  collaboration  on  the 
web  are  experienced  as  engaging  and  touching.  Research  will  attempt  to  assess  whether  Bakhtin’s  and  Dysthe’s 
theories  appropriate  in  this  context. 
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Abstract:  A recent  article  in  Educational  Technology  Review  describes  3 types  of 
learning.  The  tendency  today  is  towards  the  lesser-planned  associative  methods  seen  in 
electronic  learning.  Teachers  would  like  to  combine  such  ‘surfing’  methods  with  their 
more  traditional  Information  Retrieval  techniques  (IR)  tailored  to  their  own  acquisition 
needs,  and  habits.  Metaphor  is  the  concept  we  use  to  construct  a way  of  envisioning  the 
tools  of  the  future  for  exploring  specialized  information  through  multi -link  media.  The 
proper  manipulation  and  communication  of  facts  will  depend  on  such  tools. 


What  KING  et  al  define  as  “Exploration  without  Objectives”  or  “Serendipity”  in  their  article  on  e- 
learning  is  the  type  of  activity  the  scholar  of  literature  loves  to  delve  into.  Learning  ‘within  the  framework 
of  the  total  freedom  of  thought  can  be  quite  exciting.  One  of  the  greatest  tools  for  knowledge  transfer  of  all 
time  offers  such  liberty:  the  Internet.  E-learning  simply  through  looking  at  various  web  pages  and 
discovering  the  links  they  offer— and  then  the  links  they  offer— is  a leisure  that  lends  itself  very  nicely  to 
general  long-term  knowledge  acquisition.  There  is  room  for  improvement  though. 

Personal  goals -driven  e-leaming  (exploration)  or  e -learning  by  chance  (serendipity)  when 
“stumbling  across  seductive  details  on  a page  irrevelant  to  the  individual’s  initial  purpose  for  browsing” 
(King  et  al  2001)  breaks  away  from  the  traditional  setting — good  oV  in -school,  in-person  education.  It  is 
thus  totally  different  from  performance-related  information  accumulation  through  programmed 
“Instruction”  ‘shoved’  at  us  by  the  Other  (the  teacher).  As  many  teachers  and  researchers  get  in  the  know 
using  the  rather  unstructured  approaches,  we  will  follow-up  on  this  trend  to  learn  on  one’s  own  and 
endeavour  to  build  an  approach  for  improving  it. 

Ad  hoc  information  research  devices  on  the  World  Wide  Web  impede  writers  of  web  pages  from 
effectively  communicating  their  knowledge  to  us. 

The  problem  with  having  too  much  freedom  in  learning  is  that  the  links  one  follows — say  in  an 
hour’s  time — will  most  likely  yield  information  pieces  that  have  little  in  common  with  each  other.  This  will 
not  do  if  e-leaming  is  to  be  taken  up  by  specialists  of  any  sort,  whether  in  literature,  medicine,  technology, 
sports,  business  or  handicrafts.  In  order  to  succeed,  the  structure  of  e-leaming  must  show  a few  constraints. 

We  take  professors  as  our  example;  we  however  could  have  used  top-notch  students,  children  or  the 
inquisitive  retired  as  our  example  of  a user.  Our  idea  is  that  professors,  say  of  Philosophy,  are  quite  apt  of 
refusing  to  read  texts  on  the  screen  because  they  cannot  circle  important  passages,  they  cannot  write  their 
comments  in  the  margin,  and  the  pleasure  of  fondling  a book  written  by  an  inspiring  writer  or  just  flipping 
through  its  pages  is  lost.  Indeed,  targeting  finicky  users  challenges  the  interface  designer.  Professors  tend  to 
opt  for  the  more  traditional  setting,  i.e.  having  7 or  more  books  open  on  the  desk  in  front  of  them,  on  their 
lap  and  on  the  floor,  and  of  course  having  the  gall  to  earmark  pages,  sitting  pen-in-hand  to  question  their 
earlier  annotations...  some  professors  in  this  situation  are  even  inclined  to  ask  a loyal  wife/husband  or  a 
mediocre  student  to  bring  them  another  book!  Why  can’t  they  just  use  computers  like  everyone  else? 

Their  behaviour  is  only  suitable  to  have  at  home  with  one’s  own  books.  So  can’t  the  Internet  be  the 
professor’s  big  in-home  book?  Quite  often,  his  computer  is  in  his  home. 

Just  as  “serendipity”  is  a little  too  unconstrained  a framework  for  a professor  to  learn  in,  being 
limited  to  one  screen  will  not  do.  But  isn’t  reading  out  of  9 books  at  once  a little  too  ‘loose’  of  an  approach 
to  constructively  learning  something?!  No.  The  reader  must  not  lose  track  of  the  fact  that  a professor  is 
bright  and  always  chooses  books  and  articles  from  within  the  domain  he  is  very  at  ease  with.  In  fact,  few 
professors  truly  like  the  Internet  because  they  never  know  which  section  they  are  in  (in  a regular  library, 
you  can  always  find  your  professor  next  to  the  same  rack  of  books  unless  they  change  it  around). 


In  the  end,  o-leaming  and  constructing  technological  responses  for  others  who  desire  to  learn  is 
about  the  business  of  correctly  communicating  information  over  the  Internet.  In  communication,  since  the 
linear  image  of  the  telegraph  proposed  by  SHANNON  & WEAVER  (1949),  we  have  seen  metaphors  like  the 
orchestra  conceived  at  Palo  Alto  by  BATESON  (1981),  and  the  desk-top  metaphor  from  Apple  Inc.  just  to 
mention  a few.  The  latter  is  the  least  important,  as  it  is  used  simply  to  communicate  to  users  how  a new 
work  environment  should  be  perceived  in  order  to  better  use  it.  The  other  two  endeavour  to  explain  ex  post 
facto  how  human  communication  functions.  What  we  are  trying  to  do  is  define  a metaphor  to  be  used 
before  hand,  that  is  in  communicating  to  designers  and  programmers  of  software — and  eventually  hardware 
builders — the  functionality  required  by  professors  who  like  to  ‘hog’  a small  bit  of  the  library  in  order  to 
read  and  quote  related  passages  and  “wear  out”  books.  This  would  require  an  original  interface  integrating 
the  activities  of  more  than  one  person.  Perhaps  we  should  modify  the  activity. 

Today  we  have  the  capability  to  surpass  the  present  library  work  environment  taking  into  account 
the  goals  of  specialists.  Until  now,  the  friendly  librarian  would  simply  help  a philosopher  put  Physics,  Math 
and  Cooking,  etc.  to  find  just  Philosophical  texts,  eventually  pinpointing  certain  authors.  But  can  a librarian 
out-perform  a specialist  on,  say  Descartes,  when  it  come  to  IR  on  Descartes  texts?  Such  a user  has  half  of 
the  information  he  will  use  in  his  head  already,  though  he  is  not  sure  of  the  rest  yet.  He  will  only  know 
what  he  is  looking  for  once  he  has  it.  Here  the  problem  of  reference  JACQUES  1990;  SCHMIDT  1997) 
surfaces;  the  specific  information-seeking  context  of  the  problem  is  nicely  formulated  by  Le  Coadic  (1994): 
“A  user  tries  to  describe  something  he  wants  to  another  person  (librarian)  without  being  sure  of  what  it  is, 
and  that  person  will  quite  likely  not  know  what  it  is  eithef’V  Our  specialist  is  calling  out  for  a more 
individualized  activity. 

Apart  from  putting  the  total  works  of  Descartes  on-line,  we  propose  to  offer,  thanks  to  indexing 
techniques,  the  specialist  simultaneous  reading  of  his  favourite  passages.  Displaying  a number  of 
documents  all  at  once  (possibly  using  depth  as  a further  dimension)  would  enable  a parallel  reading  of  this 
precious  material.  The  main  question  is  twofold:  7./just  how  much  complexity  shoul4  he  have  at  his  finger 
tips?  ...should  he  be  required  to  take  a course?  ; and2./in  what  form  exactly  should  he  see  it?  He  is  so  used 
to  doing  this  at  his  desk,  on  the  floor  or  the  kitchen  counter.  Obviously,  his  efficiency  as  a teacher  or 
researcher  is  at  stake.  Once  these  questions  are  answered,  we  can  implement  an  on-line  version  suitable  to 
our  enthusiastic  scholar  or  the  mad  scientist  metaphor  outlined  here  (to  be  developed  to  guide  design). 

We  are  therefore  entering  into  the  era  of  the  personal  ©-library  in  which  one  may  ‘treat  books 
mischievously’,  scribbling  notes  on  then  so  as  to  establish  a multi-linked  environment  suitable  to  being 
accepted  as  a new  on-line  professors’  workbench.  Professors  always  try  to  find  their  way  back  to  the 
bookrack  where  they  read  a related  passage  3 years  before,  so  keeping  track  of  milestones  and  the  lesser 
important  stops  on  their  voyage  through  a large  corpus  of  literature  could  be  useful  too.  Scholars  do  have 
difficulty  convincing  themselves  to  use  two -dimensioned  vertical  glass  screens.  And  in  a domain -bound 
multi-link  Hypertext  system  it  is  important  to  be  able  to  set  one’s  own  “home”  or  anchor  text  (i.e. 
Discourses).  Weeding  out  the  non-pertinent  is  a challenge  to  e-leaming. 
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Abstract:  While  there  are  dozens  of  “e-leaming”  platforms,  not  many  systems  support  the 
“orchestration”  of  rich  socio-constructivistic  learning  scenarios.  We  suggest  to  explore  the 
pedagogical  potential  of  the  increasingly  popular  portals  we  call  “Community,  Content  and 
Collaboration  Management  SysteWis”  and  we  present  our  work  strategy. 


The  Problem 

Currently,  there  exist  many  variants  of  web-supported  pedagogies,  e.g.  transmission  of  contents,  web-based 
instruction,  learning  by  apprenticeship  in  virtual  environments,  pedagogical  work  flow  scenarios.  Constructivist 
(e.g.  project  or  problem-based)  scenarios  are  quite  popular  (Wilson  & Lowry,  2001)  but  supporting  technology  is 
hard  to  find  and  there  is  a particular  need  for  tools  that  support  socio-constructivist  learning  at  the  activities  level. 
Empirical  research  (e.g.  Dillenbourg  1 999)  reveals  that  collaborative  or  collective  constructivist  learning  is  not  per 
se  an  effective  learning  method.  It  is  more  effective  if  individuals  and  groups  have  to  evolve  within  well-specified 
scenarios,  i.e.  sequences  of  phases  within  which  group  members  do  tasks  and  play  specific  roles.  While  teachers 
can  orchestrate  complex  scenarios  with  very  little  technology,  the  effort  can  soon  become  cumbersome. 

Remember  1 993 ’s  slogan  of  “shifting  the  focus  from  teaching  to  facilitating”?  Todays  “E-leaming”  systems 
are  mostly  anchored  in  the  behaviorist  CBT  tradition.  They  focus  on  content  delivery  and  the  teacher’s  “facilitator” 
role  is  degraded  to  deal  with  web  contents,  quizzes  and  grading.  They  fail  to  support  rich  socio-constructivist 
scenarios  engaging  students  in  active  project-based  learning.  Therefore  we  argue  that  R&D  in  educational  software 
should  not  just  focus  on  improving  passive  “interactive”  courseware  but  on  tools  supporting  students  to  solve  more 
complex  and  open-ended  tasks. 

The  Work  Plan  and  Strategy 

Implementation  strategy  #1:  Imitation  of  “Internet  culture”.  Web  pages,  forums,  e-mail  and  FTP  are 
successful  because  they  support  the  basic  needs  for  exchange,  communication  and  collaboration.  While  simple  web 
technology  does  enable  creative  educational  scenarios  it  has  2 drawbacks:  (1)  Maintaining  static  web-sites  is  time- 
consuming  and  simple  discussion  systems  do  bad  knowledge  management.  (2)  More  sophisticated  scenarios  (like 
co-authoring  or  work-flow)  are  badly  supported.  Now,  community  web-sites  facing  the  same  problems  found  an 
answer.  Within  the  last  two  years  have  spmng  up  what  we  coin  C^MS  (Community,  Content  and  Collaboration 
Management  Systems).  Inspired  by  personal  weblogs,  news  systems,  simple  CMS  and  various  groupware  like  file- 
sharing  or  calendars,  C^MS  are  modular  tools  for  configuring  interactive  community  web-sites.  Systems  like 
PostNuke  or  PhPWebSite  offer  a good  set  of  core  portal  functionalities,  such  as  user  administration,  a news/joumal 
system,  web-links  sharing,  search,  FAQs  and  polls.  Extension  mechanisms  allow  third  parties  to  contribute 
additional  functionalities  like  collaborative  hypertexts,  picture  galleries,  simple  content  management  systems, 
event  calendars,  chats,  project  managers,  file-upload,  and  glossary  management.  Since  we  believe  that  a large 
number  of  rich  educational  scenarios  can  be  supported  by  modular  C^MS  systems  at  reasonable  cost,  we  started 
deploying  a few  systems  with  teachers  to  investigate. 

Implementation  strategy  #2:  Adapt  to  teachers.  Success  stories  of  new  technologies  in  education  are  often 
related  to  the  teachers’  ability  to  insert  it  into  existing  knowledge.  It  is  easier  to  promote  change  when  technology 
is  simple  and  when  teachers  can  relate  to  “models”  they  know,  even  if  they  are  not  necessarily  related  to  teaching. 
Teachers  must  have  an  operational  awareness  (vonGlasersfeld)  in  addition  to  operational  control.  Inspired  by 


c 


Guzdial’s  (2002)  work  with  CoWeb,  we  start  by  presenting  simple  activities  that  can  be  enhanced  with  C^MS.  At 
the  same  time,  we  also  introduce  these  portals  as  community  tools  (for  teacher  support)  and  we  hope  that  the 
perception  of  their  usefulness  for  “professional  life”  will  help  introducing  them  to  education  in  the  narrow  sense. 
A first  version  of  the  catalog  of  educational  scenarios  and  C^MS  bricks  is  available  (http://tecfa.unige.ch/proj/seed/ 
catalog/^  and  we  hope  to  observe  and  report  interesting  experiments  within  the  next  2 years. 

Implementation  strategy  #3:  Support  different  user  categories:  Such  a system  in  order  to  be  acceptable 
by  the  teacher  community  should  appeal  to  different  levels  technical  competence  and  “activeness”.  We  discrim- 
inate four  levels  of  use  with  respect  to  how  teachers  appropriate  learning  technologies:  (1)  Reusing:  teachers  who 
appreciate  ready-to-use  material.  In  our  case,  this  is  a scenario  that  has  been  instantiated  with  content.  (2)  Editing: 
teachers  who  feel  the  need  to  modify  the  content  of  a scenario  they  appreciate.  (3)  Designing:  this  means  in  our 
case  to  compose  completely  new  scenarios  by  re-assembling  basic  components.  (4)  Programming:  some  teachers 
like  to  program  and  we  can  expect  them  to  develop  modules.  The  originality  of  our  approach  is  to  enable  teachers 
to  work  according  to  their  technical  skills,  to  the  personal  investment,  to  what  is  available. 

Implementation  strategy  #4:  Use  synergies.  C^MS  are  by  definition  a space  for  communities.  This  first 
means  that  we  could  integrate  educational  activities  into  existing  community  portals,  e.g.  create  campus  portals 
that  are  actually  useful  to  education  and  not  just  an  underused  presentation/information  tool  designed  by  some 
central  service.  Conversely,  since  people  learn  a lot  from  informal  exchange  within  tightly  or  loosely  defined 
communities  (fellow  learners,  professors,  experts),  we  could  open  educational  portals  to  other  activities  (news, 
sharing  of  links  and  contents,  forums  for  professional  activities).  Additional  synergies  can  be  gained  from  making 
contents  available  (as  MIT’s  OpenCourseWare  or  Berkeley’s  lU  Project)  and  allowing  the  outside  world  (other 
classes,  teachers,  parents,  experts)  to  annotate.  Lastly,  while  each  small  community  may  want  to  run  its  own  portal, 
contents  can  be  syndicated.  Building  information  and  communication  networks  has  become  easier  than  ever. 

Implementation  strategy  #5:  Don’t  overdo  it.  We  do  not  know  yet  the  boundaries  of  C^MS  portals.  One 
major  drawback  seems  to  be  the  lack  of  provision  for  integration  (e.g.  data-flow)  between  applications  which  are 
required  for  more  complex  CSCL  scenarios.  Another  drawback  is  the  management  of  contents  over  time.  Handling 
these  issues  require  the  same  sort  of  planning  that  a traditional  user-driven  educational  web  site  does  and  future 
systems  (on  which  we  started  design  work)  should  address  these  issues. 

Conclusion 

This  research  on  the  use  of  “Community,  Content  and  Collaboration  Management  Systems”  for  socio- 
constructivist  scenarios  is  at  its  beginning  stage.  So  far,  we  initiated  a few  field  experiments  and  we  produced  an 
initial  catalog  of  socio-constructivist  activities  with  C^MS  bricks.  We  plan  to  support  further  sites,  will  prepare 
more  dissemination  materials  and  hope  to  report  results  within  the  next  2 years.  We  are  aware  that  C^MS  portals 
are  not  fit  for  complex  CSCL  workflow  scenarios,  but  we  believe  that  there  is  an  important  need  to  actively  support 
educational  scenarios  with  simple  technology  under  the  control  of  teachers.  Finally,  C^MS  may  be  a chance  to 
maintain  the  Internet  Spirit  in  education  which  is  threatened  by  the  philosophy  of  many  e-leaming  systems. 
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Abstract  The  aim  of  this  project  is  to  design  a prototype  for  a web-based  tool  to  provide  direct 
feedback  regarding  clinical  content  and  outcomes  experienced  in  residency  training  to  both  medical 
residents  and  their  educators.  This  tool,  the  Resident  Assessment  Performance  System  (RAPS), 
will  provide  residents,  program  directors,  and,  eventually,  accrediting  organizations  with  a 
comprehensive  record  of  residents’  patient  care  experiences.  To  ensure  the  tool’s  usefulness  and 
before  any  actual  programming  is  done,  it  is  essential  to  have  an  understanding  of  the  stakeholders’ 
needs  and  desires.  Thus,  our  initial  effort  was  a needs  analysis,  structured  around  specific 
individuals  and  themes.  The  results  were  combined  with  pilot  data  to  develop  draft  screens  that 
were  reviewed  by  residents  and  educators  to  illustrate  how  resident  profiling  might  improve  the 
quality  of  resident  training. 


RATIONALE 

Because  of  the  greater  variety  of  medical  treatments  available  today,  more  emphasis  is  being  placed  on 
making  evidence-based  medical  decisions  and  documenting  the  patient  outcomes  achieved  from  treatment.  Also, 
changing  patterns  of  medical  practice,  including  reduced  length  of  hospital  stays  and  increased  use  of  ambulatory 
services,  necessitate  increased  experience  in  ambulatory  settings  for  internal  medicine  residents.  These  changes  are 
increasing  the  need  for  appropriate  accountability  measures  that  will  assure  the  profession  and  the  public  that  the 
medical  education  system  is  both  monitoring  the  content  of  training  and  documenting  how  well  it  prepares  graduates 


for  future  practice.  As  such,  residency  program  directors  need  to  assess  and  continually  monitor  the  content, 
balance,  and  outcomes  of  both  the  inpatient  and  ambulatory  care  clinic  experiences  of  their  residents. 

Methods  often  used  to  assess  resident  competency,  such  as  surveys  and  knowledge  tests,  are  limited  in 
scope  and  do  not  provide  a complete  assessment  of  the  residency  experience.  The  Accreditation  Council  for 
Graduate  Medical  Education  (ACGME)  Outcome  Project  outlines  criteria  for  evaluating  residents  and  residency 
programs,  including  use  of  dependable  measures  to  assess  residents’  competence  in  patient  care  and  mechanisms  for 
providing  timely  feedback.  Residency  programs  are  encouraged  to  use  these  outcome  assessment  results  in  their 
evaluation  of  educational  effectiveness  (ACGME  2000). 

The  RAPS  project  aims  to  address  these  needs  and  requirements  through  the  design  of  a new  user-specified 
and  user-friendly  web-based  tool  that  would  record  and  evaluate  a resident’s  patient  care  experience  and  outcome 
performance,  track  hospital  resource  utilization,  and  compare  performance  to  local  and  to  national  peer  groups.  Our 
review  of  the  literature  did  not  reveal  any  projects  similar  to  RAPS.  The  development  and  evaluation  of  the  RAPS 
tool  will  be  an  ongoing,  iterative  process.  In  keeping  with  our  user-centered  design  methodology,  tryout  and 
revision  cycles  will  continue  as  usability  testing  is  done  and  end-user  stakeholders  will  be  actively  involved  to 
provide  feedback  on  the  content  and  usability  of  RAPS  to  meet  their  requirements. 


METHODS 

RESIDENT  RECRUITMENT 

We  chose  the  University  Medical  Associates  (UMA)  Clinic  as  our  study  site.  UMA  is  the  primary  site  for 
general  medical  care  of  adults  at  the  University  of  Virginia  Health  System.  It  is  also  where  Internal  Medicine 
residents  receive  most  of  their  training  in  primary  care.  We  recruited  a cohort  of  fourteen  internal  medicine 
residents  from  the  University  of  Virginia  Department  of  Internal  Medicine  to  participate  in  the  design  of  the  RAPS 
tool.  From  these  residents,  we  solicited  specific  content  and  display  recommendations  by  conducting  a formal  needs 
assessment  using  individual  interviews  and  focus  groups.  We  also  conducted  interviews  with  the  residency  program 
directors  and  faculty. 


SYSTEM  ARCHITECTURE 

The  RAPS  project  uses  routinely  collected  electronic  data  contained  in  the  University  of  Virginia’s  Health 
System’s  Clinical  Data  Repository  (CDR)  (Scully  et  al.  1997).  The  CDR  is  a relational  data  warehouse  that  was 
developed  beginning  in  1985  to  support  research  and  education  at  our  institution.  It  integrates  data  from  disparate 
systems  and  now  includes  eight  years  of  inpatient  data  (approximately  30,000  admissions  per  year),  eight  years  of 
physician  billing  data,  five  years  of  outpatient  data  (450,000  outpatient  visits  per  year),  six  years  of  clinical 
laboratory  results  (750,000  lab  results  recorded  per  month),  and  eight  years  of  surgical  outcomes  data  from  the 
University  of  Virginia  Heart  Center  Thoracic  Cardiovascular  (TCV)  departmental  database.  Last  year,  death 
registry  data  from  the  Virginia  Department  of  Health,  Division  of  Health  Statistics  was  added.  Currently,  the 
database  contains  over  40GB  of  information  representing  approximately  six  million  patient  visits. 

The  CDR  resides  on  a Dell  PowerEdge  1300  (Dual  Intel  400MHz  processors,  512  MB  RAM)  running  the 
Linux  4.3  operating  system  and  Sybase  1 1.9.1  relational  database  management  system  (RDBMS)  (Sybase  Inc., 
Emeryville,  CA).  It  uses  a custom-built  Web-based  front  end  that  allows  local  users  to  formulate  ad  hoc  queries  and 
download  data  to  their  own  PC’s.  Alternatively,  experienced  users  may  submit  their  own  Structured  Query 
Language  (SQL)  statements  directly  to  the  Sybase  Server  or,  with  the  installation  of  Sybase  drivers,  may  access  the 
Sybase  tables  directly  with  Microsoft  Access  or  Excel. 


NEEDS  ASSESSMENT 

The  RAPS  tool  is  designed  to  meet  the  needs  of  both  internal  medicine  residents  practicing  in  a general 
medicine  clinic  and  their  program  faculty  directors  and  advisors.  As  such,  the  results  of  the  needs  assessment  were 
analyzed  to  determine  the  goals  for  the  RAPS  interface  tool.  A consensus  list  of  priority  conditions,  patient 
information,  and  measurement  end-points  were  assembled  from  the  interviews  and  focus  group  feedback. 


Overwhelmingly,  residents  and  faculty  expressed  interest  in  information  to  help  them  manage  specific  populations, 
such  as  diabetes  and  hypertension  patients,  and  health  screening  practices  compared  with  published  guidelines. 
Residents  are  interested  in  detailed  information  about  their  assigned  patient  panels,  such  as  most  common  diagnoses, 
procedures,  and  demographic  information.  In  addition  to  this  information,  faculty  desire  information  describing  the 
continuity  of  care,  such  as  whether  the  same  resident  is  seeing  the  patient  on  an  ongoing  basis.  Measures  of 
resource  utilization  are  of  interest  to  both  residents  and  faculty.  Information  desired  includes  medication 
compliance,  laboratory  tests,  procedures  completed,  costs  and  reimbursement,  the  amount  of  time  physicians  spend 
with  patients,  and  follow-up  appointment  time.  Other  patient  care  outcomes  of  interest  include  hospital  length-of* 
stay,  emergency  room  visits,  mortality,  hospital  re -admissions,  patient  and  nurse  satisfaction  data,  and  missed  and 
canceled  appointments.  To  be  meaningful,  information  used  for  resident  physician  profiling  must  associate 
variation  in  resource  utilization  with  differences  in  the  disease  burden  of  the  residents’  patient  panels. 


CONTENT  SELECTION 

Our  content  selection  was  based  on  the  following  decision  criteria: 

■ RAPS  project  objectives  and  scope 

■ Data  readily  available 

■ Potential  sources  of  new  data 

■ Patient- identified  data  and  surrounding  confidentiality  issues 


OBJECTIVE  AND  SCOPE 

The  RAPS  project  is  intended  to  provide  aggregate  retrospective  patient  data  to  evaluate  a resident’s  patient 
care  experience  and  outcome  performance.  It  is  not  intended  for  direct  patient  care.  A pilot  study  conducted  by  the 
Association  of  Program  Directors  in  Internal  Medicine  of  its  membership  identified  desired  components  of  quality  in 
residency  training.  Assessment  outcomes  of  training  indicators  that  were  rated  highly  include  appropriate  amount  of 
evaluation  and  feedback  to  residents,  adequacy  of  patient  mix,  and  applicability  of  the  overall  curriculum  to  future 
practice  as  general  internists  (Klessig  et  al.  2000).  In  keeping  with  the  original  intent  and  scope  of  the  RAPS 
project,  we  focused  on  data  ele  ments  that  provide  a broad  view  of  residents’  patient  care  experience  such  as 
diagnoses  most  commonly  seen,  aggregate  measures  of  hospital  resource  utilization,  and  disease  screening  with 
appropriate  guidelines  and  comparisons. 


DATA  SOURCES 

The  core  data  source  for  the  RAPS  tool  is  the  Clinical  Data  Repository  (CDR),  a relational  data  warehouse 
with  a Web  browser  interface  that  provides  direct  access  to  data  for  education,  clinical  research,  and  quality 
improvement  initiatives  at  the  UVA  Health  System.  The  CDR  contains  inpatient  billing  and  administrative  data, 
physician  billing  data,  clinical  laboratory  data,  the  state  death  index,  and  various  departmental  databases.  Much  of 
the  data  desired  by  the  residents  and  program  directors  is  currently  contained  in  the  CDR.  Additional  data  feeds  are 
planned,  including  outpatient  clinic  pharmacy  data  and  clinic  scheduling  data.  Both  will  provide  important  data 
elements  for  our  project. 


CONFIDENTIALITY  ISSUES 

The  Clinical  Data  Repository  has  the  functionality  to  provide  detailed  information  at  the  patient  level. 

Users  can  “click”  on  a patient  identification  number  (an  encrypted  version  of  the  medical  record  number)  or  account 
number  and  “drill-down”  to  detailed  patient  information  such  as  the  results  of  individual  laboratory  tests.  We 
decided  not  to  provide  this  feature  in  RAPS  because  the  intent  of  the  RAPS  tool  is  to  provide  aggregate, 
retrospective  data  for  assessment.  Also,  patient  identifiable  data  for  use  in  direct  patient  care  is  not  within  the 
defined  scope  of  either  the  CDR  or  the  RAPS  project.  Second,  in  light  of  increased  emphasis  on  security  and 
privacy  standards  brought  about  by  HIPAA  legislation  (The  Health  Insurance  Portability  and  Accountability  Act  of 
1996),  further  assessment  is  needed  before  we  make  patient  identified  data  available  via  the  Web. 


RESULTS 

RAPS  PROTOTYPE  SAMPLE  SCREENS 


The  screens  contain  actual  sample  data  collected  during  a pilot  data  collection  period,  January -March, 

200 1 . The  screens  have  not  yet  been  integrated  into  the  CDR;  rather  they  are  mock-up  design  prototypes.  (Figure  1 
depicts  one  of  the  sample  screens.) 

Screenshots  of  the  RAPS  web-based  tool  screens  may  be  accessed  at  the  following  site: 
http://www.people.virginia.edu/~jrw5d/RAPS/ 


EVALUATION 

We  made  substantial  revisions  (mostly  additions)  to  the  content  of  the  RAPS  tool  following  the  focus  group 
conducted  with  the  participating  residents  and  presented  the  revised  screens  to  three  of  the  faculty  program  directors. 
In  individual  sessions,  faculty  navigated  through  the  RAPS  screens,  asking  questions  and  making  comments.  We 
recorded  their  suggestions  in  order  to  make  additional  revisions  to  the  content  and  clarity  of  the  presentations.  In 
June  2001,  we  sent  an  e-mail  survey  to  a representative  group  of  residency  program  directors  soliciting  feedback 
about  the  draft  formats  and  reports  for  the  RAPS  tool.  Questions  were  asked  about  the  usefulness  of  the  data  and  the 
usability  of  format.  Respondents  were  all  program  directors  of  internal  medicine  residency  programs  and  were  all 
very  positive. 


DISCUSSION 

DATA  ISSUES 

RAPS  is  designed  to  view  information  in  two  basic  ways:  patient-level  and  admission-  or  visit-level. 
Patient-level  data  includes  information  about  all  of  a patient’s  encounters  with  the  University  of  Virginia  Health 
System,  both  hospital  admissions  and  outpatient  clinic  visits.  Admission-  or  visit-level  data  includes  only  data  about 
a particular  encounter,  either  a single  hospital  admission  or  clinic  visit.  Both  views  are  important  and  desired  by 
residents  and  faculty.  In  some  instances,  it  is  important  to  have  access  to  information  about  a patient’s  entire 
encounter  with  the  Health  System,  regardless  of  whether  the  assigned  resident  saw  the  patient  or  not.  Patient-level 
information  is  most  useful  for  disease  screening  and  management.  In  other  instances,  it  is  more  meaningful  to  look 
at  admission-  or  visit-level  information.  For  instance,  evaluating  the  breadth  of  a resident’s  exposure  to  various 
diagnoses  would  require  only  information  particular  to  encounters  where  the  resident  of  interest  saw  the  patient. 


MEANINGFUL  PRESENTATION 
CLUSTERING  METHODOLOGIES 

An  assessment  of  the  content  of  residents’  ambulatory  care  experiences  necessitates  a method  to  describe 
the  illnesses  seen  by  individual  residents  in  various  outpatient  settings  (Bordley  et  al.  1987).  We  used  the  Clinical 
Classification  Software  (CCS),  a 1999  update  to  the  former  Clinical  Classifications  for  Health  Policy  Research 
(CCHPR)  Version  2.  This  classification  scheme,  which  is  in  the  pubic  domain,  was  developed  by  the  Agency  for 
Healthcare  Research  and  Quality  (AHRQ)  and,  in  previous  studies,  has  shown  promise  for  comparing  resource 
utilization  among  physicians  and  managed  care  organizations  (Cowen  et  al.  1998).  Diagnoses  and  procedures  for 
inpatient  hospital  stays  are  coded  using  the  International  Classification  of  Diseases,  9*^  Revision,  Clinical 
Modification  (ICD-9-CM),  Fifth  Edition  (Public  Health  Service  and  Health  Care  Financing  Administration,  1994). 
ICD-9-CM  consists  of  about  12,000  diagnosis  codes  and  3,500  procedure  codes.  The  CCS  methodology  aggregates 
these  codes  into  clinically  meaningful  categories,  providing  a reasonable  method  for  cataloging  the  illnesses  seen  by 
residents.  This  methodology  will  accommodate  evaluation  of  individual  residents  and  comparison  of  the  UMA 
Clinic  residency  experience  with  national  norms. 


COMPARATIVE  DATA:  WHAVS  USEFUL,  *TAIR** 

Because  variation  in  utilization  of  health  care  resources  is  associated  with  differences  in  disease  burden, 
adjustments  for  comorbid  illnesses  are  needed  before  accurate  comparisons  can  be  made  between  patient  groups.  In 
addition,  comparative  data  should  be  useful  and  “fair”.  The  RAPS  report  screens  are  designed  to  permit  each 
resident  to  view  his  or  her  own  patient  data  and  comparative  data  that  will  include  national  guidelines  where 
available  (i.e.,  disease  screening)  and  aggregate  data  for  meaningful  comparative  groups  such  as  year  of  residency 
training,  all  UMA  residents,  all  internal  medicine  residents,  and  total  hospital.  System  logic  will  be  applied  to  adjust 
the  RAPS  screens  when  comparative  data  are  not  meaningful.  For  example,  messages  will  alert  the  user  when 
sample  sizes  are  insufficient  to  yield  valid  comparisons. 


CONTINUITY  OF  CARE 

The  value  of  physicians  caring  for  patients  over  time  includes  opportunities  to  observe  the  course  of  illness 
and  treatment  and  stronger  physician-patient  relationships.  Of  the  120  medical  school  respondents  to  a 1998  survey, 
96%  with  longitudinal  programs  indicated  that  teaching  continuity  is  valuable,  yet  only  73%  were  designed  to  foster 
such  continuity.  The  logistics  of  accomplishing  continuity  of  care  are  often  the  most  difficult  obstacle  (Vogt  et  al. 
2000).  We  developed  a measure  of  continuity  of  care  for  the  RAPS  tool  to  enhance  tracking.  This  measure 
provides  an  index  of  how  many  of  a resident’s  patients’  visits  have  been  with  that  resident. 


CONTENT  ORGANIZATION 

RAPS  is  designed  for  both  residents  and  their  educators.  At  the  logon  screen,  users  select  either  the 
resident  or  faculty  screens  to  view.  The  logon  identification  determines  which  screens  the  user  will  be  able  to 
access.  The  screens  are  grouped  into  two  main  categories:  1)  the  resident’s  outpatient  panel,  and  2)  the  resident’s 
patient  care  experience.  The  outpatient  panel  selection  provides  information  related  to  a resident’s  assigned  panel  of 
patients  from  the  UMA  Clinic.  This  information  will  provide  patient-level  data  whether  or  not  the  resident  actually 
saw  the  patient.  The  second  grouping  of  screens  provides  information  about  only  those  patients  that  a resident 
actually  saw,  whether  or  not  the  patient  belongs  on  the  resident’s  panel,  in  the  case  of  the  UMA  Clinic  choices. 
Faculty  will  have  access  to  all  resident  data,  individually  and  compared  to  various  desired  groups. 


CONCLUSION 

We  have  completed  the  initial  design  of  the  RAPS  web-based  tool,  both  its  content  and  technical 
specifications,  and  have  developed  prototype  screens.  Our  user-centered  design  and  iterative  tryout  and  revision 
strategy  is  an  ongoing  process  that  will  continue  as  we  operationalize  the  tool  over  the  next  year.  Internal  and 
external  reviews  have  been  extremely  positive. 

The  value  of  RAPS  as  an  assessment  method  to  evaluate  individual  residents  and  the  overall  residency 
program  will  require  selection  and  application  of  appropriate  techniques.  We  will  address  the  question  of  whether 
residents  who  use  RAPS  perform  better.  Event  tracking  and  other  administrative  tools  will  be  implemented  within 
the  RAPS  tool  to  calculate  usage  statistics.  Surveys  and  interviews  will  be  used  to  collect  feedback.  Resident 
evaluations,  in-service  exam  scores,  and  other  metrics  will  be  used  to  measure  differences  in  performance  by 
residents  who  use  RAPS  to  those  who  do  not. 

Historically,  medical  residency  programs  have  been  designed  using  a rotation-based  format,  based  on  the 
premise  that  by  rotating  through  a series  of  educational  experiences  residents  will  assimilate  the  skills  needed  to 
become  competent  physicians.  An  alternative  educational  approach  focuses  on  attainment  of  competency,  rather 
than  completion  of  a set  of  educational  experiences.  Within  internal  medicine  there  is  interest  in  the  application  of 
competency-based  education,  also  known  as  mastery  learning  or  outcomes-based  assessment  (Bell  et  al.  1997). 
RAPS  is  designed  to  encourage  the  transition  within  graduate  medical  education  from  experience-based  to 
competency-based  training  by  providing  aggregate  information  to  evaluate  a resident’s  patient  care  experience  and 
outcome  performance.  It  is  our  hope  that  RAPS  will  also  help  clinicians  develop  the  independent  quality  reporting 
and  accountability  skills  needed  for  future  practice. 


FIGURE  1.  RAPS  MENUSCREEN 
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Virtually  There:  Online  Case  Studies 
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Abstract:  The  candidates  enrolled  in  one  of  three  degree  programs  offered  through  the  Greenfield 
Coalition  (GC)  have  a unique  learning  situation  where  classroom  exercises  link  with  their  job 
experience.  Situated  at  the  Focus:  HOPE  Center  for  Advanced  Technologies,  candidates  work  in  a 
real  manufacturing  production  environment  by  day  and  attend  courses  by  night.  In  order  to  export 
these  experiences  to  students  enrolled  in  other  manufacturing  engineering  programs,  GC  developed 
case  studies  accessible  online.  Cases  currently  available  support  courses  in  engineering  economics, 
statistics  in  manufacturing  product  launch,  forming  technologies,  and  tool  and  fixture  design.  A 
wealth  of  resources  including  interview  transcripts,  WWW  links,  raw  data  and  expert  advice  are 
accessible  online  for  learners  to  investigate  each  case  and  complete  the  required  tasks. 


The  Need  for  Online  Case  Studies  at  Greenfield  Coalition 

The  candidates  (Greenfield  Coalition  (GC)  students)  at  the  Focus:  HOPE  Center  for  Advanced  Technologies 
(CAT)  have  a unique  learning  environment.  They  have  an  advantage  over  students  enrolled  in  traditional 
manufacturing  engineering  curricula  because  they  have  the  daily  opportunity  to  apply  new  concepts  learned  in  the 
classroom  to  real  situations  on  the  manufacturing  shop  floor.  This  characteristic  of  the  curricula  at  the  Greenfield 
Coalition  is  not  only  unique  but  also  provides  a natural  contextual  environment  for  the  application  and  transfer  of 
new  knowledge  and  skills.  In  terms  of  teaching  and  learning,  a better  environment  could  not  be  simulated.  Therefore, 
it  became  a critical  component  of  the  teaching  and  learning  strategies  at  GC. 

However,  in  order  to  share  this  advantage  and  paradigm  with  other  students  enrolled  in  similar  engineering 
programs  so  that  they  might  learn  from  the  experiences  of  the  CAT  candidates,  GC  had  to  somehow  export  these 
experiences.  Presenting  case  studies  from  the  CAT  in  an  online  environment  seemed  the  obvious  choice. 


The  Value  of  Case  Studies 

Case  studies  are  ideal  for  illustrating  complex  concepts,  especially  common  in  engineering.  Horton  (2000) 
suggests  the  use  of  case  studies  as  an  excellent  way  for  learners  to  practice  judgment  skills  necessary  in  real  life 
situations  that  are  not  as  simple  as  textbook  problems.  As  instructional  strategies  are  concerned,  engaging  critical 
thinking  skills  through  case  studies  is  among  a recommended  set  of  activities  (Bonk  & Reynolds,  1998). 

GC  has  developed  several  case  studies  that  challenge  learners  in  ways  a manufacturing  engineer  would.  The 
majority  of  the  case  examinations  come  from  real  situations  faced  by  Focus:  HOPE  and  other  industry  partners  of  GC 
(such  as  Society  for  Manufacturing  Engineering,  Ford  Motor  Company,  General  Motors,  and  DiamlerChrysler).  One 
case  developed  as  a capstone  experience  in  engineering  economics  involves  identifying  the  source  of  lost  dollars  for 
a tier  one  supplier  to  the  automotive  industry.  Another  case  involves  the  redesign  of  a tie -rod  end  component  used  in 
the  HMMWV  (commonly  known  as  the  Hummer),  whose  primary  customer  is  the  U.S.  Department  of  Defense.  As 
part  of  a course  in  Integrated  Product  Realization,  a case  requires  learners  to  apply  processes  and  procedures  for  the 
launch  of  a new  product.  Yet  another  case  supporting  a course  on  statistics  in  manufacturing  requires  the  analysis  of 
variances  of  post-production  part  specifications  in  order  to  utilize  statistical  methodologies  to  explain  scrap  lost 
dollars.  Many  other  cases  have  been  developed  by  GC  to  support  courses  in  tool  and  fixture  design,  forming 
technologies,  operations  management  and  facilities  design.  These  cases  require  students  to  identify  appropriate 
processes  to  apply  in  various  situations,  generate  cost/benefit  information  and  make  appropriate  decisions  on  the 
basis  of  that  information,  respond  to  requests  for  bid  for  a formed  part,  investigate  and  recommend  solutions  to 
eliminate  tolerance  irregularities  during  machining  of  parts  as  well  as  many  other  tasks  typically  found  in 
manufacturing  engineering  environments. 

Through  these  case  studies,  learners  get  practice  applying  newly  learned  skills  and  knowledge  to  authentic 
situations  not  found  textbooks. 


Online  Environment  for  Case  Studies 


In  the  online  environment,  GC  has  captured  a wealth  of  resources  related  to  each  case  and  provides  it  for 
learners  to  examine  and  contemplate  much  like  a student  could  do  if  s/he  were  enrolled  in  an  onsite  course.  These 
resources  are  organized  by  typical  activities  that  would  be  performed,  to  prevent  the  learner  from  feeling 
overwhelmed  at  any  given  time  during  the  case  analysis. 

Case  Study  Orientation 

The  home  page  for  the  case  serves  as  the  learner  guide.  From  this  page,  students  are  able  to  view  the 
recommended  set  of  investigation  activities,  expectations  for  assessment  of  their  investigation  as  well  as  the  full  list 
of  resources  provided  for  their  perusal.  Basic  navigation,  similar  to  GC  courses,  is  accessible  at  any  time,  from  the 
left-hand  side  and  across  the  top  of  the  interface,  providing  constant  availability  of  broad  level  course  information. 
Further,  the  web  interface  is  organized  to  keep  learners  from  becoming  frustrated  or  lost.  Figure  1 shows  that  once  a 
case  is  selected,  the  specific  scenario  is  depicted,  and  instructions  and  activities  are  outlined.  One  level  up,  at  the 
module  level,  it  is  possible  to  link  to  several  case  studies  that  all  meet  the  same  objective  and,  therefore,  would  be 
grouped  into  the  same  learning  module. 
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Figure  1:  Engineering  Economics  case  analysis  home  page  with  Situation  pop-up 


Learner  Responsibility 


Given  the  situation  or  scenario  description,  it  is  the  responsibility  of  the  learner  (or  team  of  learners)  to 
investigate  and  uncover  what  they  believe  to  be  the  real  problem.  Students  are  challenged  to  determine  what  is 
relevant  to  the  case  and  process  its  importance.  Exploring  the  resources  helps  them  achieve  this  . While  each  case  has 
a different  set  of  resources,  many  media  formats  are  used  throughout  all  of  the  GC  cases.  In  a case  examining  the 
scrap  rate  through  the  balancing  operation  for  pulleys,  a process  map  and  product  path  are  depicted  using  still  images 
with  mouse-overs  for  additional  information  (see  Figure  2 & 3).  Further,  MSWord  and  Excel  documents  such  as  part 
price  data,  scrap  rates  and  subsequent  lost  dollar  reports  are  viewable  in  the  web  browser  (see  Figure  4). 


BEST  COPY  AVAIIAIBIUS 


hUp://dev.greenfield-coalilion.oig/gfcvl /cs/conlenlT empiate.asp?dbtable=BalanceiDiagr...  IQ 


Process  Flow  Overview  of  the  Balancing  Area 


Bulancrr 
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Figure  3:  Photographic  product  path  for  pulleys  from  arrival  through  shipping 
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Figure  4:  Sample  data  document  for  Boring  Dimensional  Irregularities  case  study 


Students  can  even  simulate  a conversation  with  important  personnel  by  reviewing  the  interview  section  of  the 
resources.  Here,  streaming  video  clips  and  transcripts  of  real  interviews  conducted  by  CAT  candidates  can  be  viewed 
(see  Figure  5).  WWW  links  to  vendors  of  balancing  machines  and/or  information  about  new  balancing  technology 
make  it  possible  for  learners  to  research  new  technology  or  uncover  how  other  cortpanies  resolved  similar  situations 
(see  Figure  6). 
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Figure  5:  Interview  of  customer  for  the  Redesign  of  a Tie  -Rod  End  case  study 
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Figure  6:  WWW  links  for  the  Automotive  Drive  Shaft  bulk  metal  forming  case 


Recommendation  for  Resolution 


Once  the  leamer(s)  has  investigated  the  case  thoroughly,  s/he  must  prepare  a report  aligned  with  the 
guidelines  for  that  specific  case.  For  Engineering  Economics,  this  report  requires  a comprehensive  economic  analysis 
of  the  situation  and  related  data,  aid  must  include  the  student’s  recommendation  for  resolution  to  the  problem. 
Supporting  documentation  such  as  a decision  tree  and  sensitivity  analysis  also  complement  the  report. 

Although  not  explicitly  required  for  each  subsequent  delivery  of  this  case,  a presentation  to  Focus:  HOPE 
management  was  a component  of  the  pilot  of  the  economics  case  study.  The  experience  of  presenting  the  alternatives 
to  the  stakeholders  made  the  needs  of  the  client  perfectly  clear  to  the  candidates  and  had  a greater  learning  impact 
than  initially  expected.  They  were  able  to  better  understand  the  urgency  for  resolution  and  the  impact  in  terms  of 
time  and  money  that  implementing  any  one  of  the  alternatives  would  cost.  At  the  pleasure  of  the  instructor  using 
these  materials,  this  experience  could  be  simulated  for  other  cases  by  simply  asking  classmates  or  industry  experts  to 
pose  as  the  customer. 


Supporting  Teaching  and  Learning  In  The  Online  Environment 

Learner  Support 

If,  at  any  time,  the  learner  feels  overwhelmed  by  the  rich  set  of  information  provided  and  feels  that  the 
exploration  through  the  resources  seems  unproductive  or  inefficient,  they  may  consult  the  mentor  notes.  Mentor 
notes,  a series  of  tasks  and  issues  related  to  the  case  and  its  objectives,  serve  as  a tour  glide  and  give  necessary 
learner  support  when  they  need  and  want  it.  The  learner  is  not  required  to  follow  the  tasks  and/or  consider  the  issues 
in  a lock-step  fashion.  Rather,  Mentor  Notes  help  to  orient  the  learner  to  the  items  necessary  for  consideration  in  a 
thorough  investigation  of  the  situation.  Filipczak  (1996)  refers  to  this  as  scaffolding  or  “guided  discovery”,  by 
retaining  learners  opportunity  to  explore  while  still  making  certain  that  the  established  objectives  are  attainable. 
Candidates  may  choose  to  execute  all,  some,  or  none  of  the  suggested  tasks.  Additionally,  this  compilation  of  expert 
notes  can  be  applied  to  many  situations.  As  a result,  it  functions  as  a tool  for  learning  as  well  as  a job  aid  for  real 
manufacturing  engineering  situations. 

Often  case  studies  at  GC  serve  as  a capstone  for  the  courses,  however,  they  are  not  tests.  Therefore,  learners 
may  refer  to  the  mentor  notes  as  often  as  necessary  without  negative  consequence.  The  most  important  purpose  is 


that  students  can  apply  previously  learned  concepts  and  principles,  and  practice  decision-making  and  problem- 
solving processes  in  a non-threatening  but  very  real  situation. 

Instructor  Support 

Scaffolding  is  also  provided  for  the  course  instructor  in  an  area  entitled  Faculty  Interface,  in  the  event  that 
the  instructor  using  the  case  study  materials  is  not  as  knowledgeable  about  a particular  topic  as  the  case  study 
designer  and/or  developer.  These  notes,  similar  to  the  mentor  notes,  were  recorded  during  the  pilot  offering  of  the 
case  study  to  assist  subsequent  instructors  adjusting  to  their  new  roles  as  a facilitator  of  learning  and  a manufacturing 
plant  supervisor  posing  a variety  of  questions  and  concerns  common  in  a real  engineering  setting. 


Integration  with  Other  Instructional  Methods 

It  is  critically  important  to  be  aware  that  the  use  of  the  web-enabled  case  studies  does  not  occur  without  the 
leadership  of  an  instructor.  The  instructor  at  GC  plays  several  roles  simultaneously:  traditional  instruction  of  course 
concepts,  mentoring  and  coaching  during  the  case  investigation,  and  finally  that  of  a supervisor  challenging  the 
recommendations  from  a manufacturing  enterprise  perspective  (Schuch-Miller  & Plonka,  2001).  Moreover,  follow- 
up classroom  discussions,  an  integral  component  of  the  case  design,  allow  learners  to  reflect,  summarize  and  solidify 
their  own  learning  and  structure  it  in  a way  that  is  meaningful  to  them  (Hidi  & Anderson,  1986). 

At  GC,  we  strongly  believe  in  a blended  approach  to  teaching  and  learning,  involving  the  integration  of 
instructor-facilitated  classroom  activities,  sharing  of  manufacturing  experiences  of  the  candidates  at  the  CAT, 
bestowing  course  credit  for  experiences  on  the  shop  floor  as  well  as  utilizing  the  web-enabled  technology  wherever 
possible  in  order  to  enhance  manufacturing  engineering  education.  The  result  is  a very  fulfilling  learning  experience 
that  includes  learner  collaboration,  learner  support,  and  innovation.  By  replicating  these  experiences  in  an  online 
environment,  it  is  hoped  that  other  instructional  programs  can  be  enriched  by  using  these  resources  as  well. 

For  more  information  about  Greenfield  Coalition  or  the  case  studies  discussed  in  this  paper,  access  the 
following  website:  www.greenfield-coalition.org 
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Abstract:  In  the  three  years  since  their  introduction,  Internet -capable  cellular  telephones  are 
used  by  over  47  million  Japanese  (37%  of  the  population)  which  nearly  equals  the  number  of 
people  using  personal  computers  to  access  the  Internet.  If  this  trend  continues,  the  cellular 
telephone  will  overtake  the  personal  computer  as  the  most  widely  used  Internet  access  device 
in  Japan.  However,  this  development  has  gone  largely  unnoticed  by  researchers.  Such  a trend 
deserves  greater  attention,  both  to  understand  the  nature  of  Japanese  use  of  communication 
technologies  and  to  assess  these  technologies’  potential  for  educational  application.  This 
paper  examines  the  impediments  that  slowed  Japan’s  early  acceptance  of  the  Internet, 
describes  how  these  barriers  were  overcome  by  Internet -capable  cellular  telephones,  considers 
the  strengths  and  weaknesses  of  these  emerging  technologies,  and  assesses  the  potential  for 
using  portable  wireless  communication  devices  in  educational  settings. 


Introduction 

Despite  nearly  a decade  of  general  economic  stagnation,  Japan’s  telecommunications  industry  has  continued  to 
grow  by  developing  innovative  and  sophisticated  wireless  communication  devices.  These  products,  most 
notably  Internet-capable  cellular  telephones,  have  been  readily  accepted  by  Japanese  consumers  and  the 
increasingly  widespread  use  of  these  devices  has  both  changed  popular  culture  and,  more  importantly,  greatly 
expanded  Internet  use  in  Japan.  Indeed,  the  number  of  Internet-capable  cell  phones  in  use  has  grown 
dramatically  since  they  first  appeared  in  1999,  and  the  cell  phone  may  soon  overtake  the  personal  computer  as 
Japan’s  Internet-access  medium  of  choice. 

Despite  this  trend,  few,  if  any,  scholars  have  formally  asked  what  role  Internet-capable  cellular  telephones  may 
play  in  educational  telecommunications.  Certainly,  the  limitations  of  current  cell  phone  technology  make  its 
near-term  use  in  educational  contexts  doubtful.  However,  Japan’s  rapid  and  widespread  adoption  of  cellular 
telephones  as  an  Internet  access  medium  make  this  an  important  area  of  study  to  determine  if  these  technologies 
can  be  used  in  educational  settings.  Examining  Japan’s  use  of  these  technologies  is  appropriate  as  Japanese 
cellular  technology  is  estimated  to  be  several  years  ahead  of  America  (Despeignes  2000).  Thus,  understanding 
the  Japanese  case  could  be  a strong  indicator  of  the  kinds  of  communication  tools  and  opportunities  that  may 
become  available  in  America. 

This  paper  begins  with  an  description  of  the  conditions  that  restrained  Japan’s  early  use  of  the  Internet  relative 
to  other  countries.  The  next  section  describes  Japan’s  rapid  acceptance  of  Internet-capable  cellular  telephones 
and  shows  how  these  devices  overcame  the  barriers  to  Internet  use.  The  following  section  assesses  the  strengths 
and  weaknesses  of  current  cell  phone  technologies  and  the  concluding  sections  considers  how  these  devices 
might  be  used  in  educational  s'tettings. 


1308 


Impediments  to  Internet  Acceptance  in  Japan 

Personal  computer  and  Internet  use  in  Japan  have  lagged  behind  the  United  States  and  other  industrialized 
countries  according  to  a Japanese  Ministry  of  Public  Management,  Home  Affairs,  Posts  and 
Telecommunications’  report  (MPHPT  1999).  Scholarly  research  on  the  nature  of  Japanese  computer  and 
Internet  use  is  limited,  but  Aoki’s  assessment  of  barriers  to  wider  Internet  access  in  Japan  reflects  the 
conventional  thinking  behind  Japan’s  slow  acceptance  (Aoki  1995): 

1.  Businesspeople  prefer  face-to-face  communications,  and  access  to  e-mail  in  the  office  is  limited. 

2.  Phone  rates  are  calculated  by  the  minute  which  inhibits  extended  online  time. 

3.  Computer  use  is  considered  as  unfashionable  and  computer  users  are  often  characterized  as  nerds. 

4.  Typing  is  still  a relatively  uncommon  skill  among  Japanese  people,  and  involves  using  four  types  of 
written  Japanese  characters. 

It  should  be  noted  that  since  Aoki’s  report  was  written,  flat-rate  ISDN  Internet  access  (among  others)  has 
flourished  in  Japan  bringing  online  costs  down  to  more  accessible  levels.  However,  lingering  negative  images 
of  computer  users  and  the  QWERTY  keyboard  interface  continue  to  be  strong  disincentives  to  increased 
computer-based  Internet  access  in  Japan.  For  comparison,  one  estimate  indicates  that  only  14%  of  Japanese 
people  use  computers  to  access  the  Internet  versus  37%  in  the  United  States  (Schmetzer  2000). 

In  contrast  to  the  slow  pace  of  computer-based  Internet  use,  using  cellular  telephones  (or  keitai,  pronounced 
“kay-tie”)  for  Internet  access  has  increased  dramatically  in  Japan  since  its  introduction  in  February,  1999,  and  is 
rapidly  becoming  a,  if  not  the  most,  popular  way  to  access  the  Internet.  Computer  access  of  the  Internet  has 
shown  a large  increase  over  the  past  couple  of  years,  but  cellular  phone  access  rose  at  an  even  greater  rate; 
MPHPT  statistics  show  (MPHPT  2001)  that  between  1999  and  2000,  households  using  personal  computers  to 
access  the  Internet  almost  doubled  (19.1%  to  34.0%),  but  the  percentage  of  households  using  cellular  phones  to 
access  the  Internet  tripled  (8.9%  to  26.7%). 


Figure  1 : Sample  keitai  mail  message  in  English 

Additional  quantification  can  be  seen  by  looking  at  the  example  of  NTT  DoCoMo,  Japan’s  largest 
telecommunications  company.  Internet-capable  cellular  phones  were  introduced  by  the  DoCoMo  in  Japan  in 
February,  1999,  under  the  brand  name  ”i-mode.”  Starting  in  mid-2000,  all  DoCoMo  cell  phones  were 
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“compatible  with  Internet  standards  such  as  HTML,  http,  gif,  and  Java,’*  so  users  can  surf  the  Web  in  addition  to 
checking  their  mail  (Legace  2001).  In  less  than  three  years,  nearly  29  million  people  subscribe  to  DoCoMo’s 
Internet  cell  phone  service  at  the  end  of  2001  (TCA  2001).  The  total  number  of  Internet-capable  cell  phones  in 
Japan  is  just  over  47  million  out  of  an  installed  base  of  over  66  million  cell  phone  service  subscribers  according 
to  the  Japanese  Telecommunications  Carriers  Association’s  monthly  data  (TCA  2001).  Based  on  these 
numbers,  nearly  40%  of  all  Japanese  people  carry  Internet-capable  keitai. 


Keitai  Use  Increases  Japanese  Internet  Access 

Clearly  keitai,  especially  those  with  Internet  capability,  have  been  rapidly  and  widely  accepted  by  the  Japanese 
consumer.  This  high  rate  of  acceptance  indicates  that  keitai  have  successfully  overcome  the  barriers  to  Internet 
access-which  personal  computers  failed  to  do-despite  having  been  on  the  market  for  only  three  years. 

Internet  access  from  the  office:  The  first  barrier  to  wider  Internet  access  posed  by  Aoki  was  in  the  business 
realm  where  face-to-face  meetings  and  limited  in -office  e-mail  access  limited  greater  use.  Certainly,  face-to- 
face  communications  will  edge  out  most  technology-mediated  communications  in  certain  business  situations. 
However,  business  people  can  use  their  own  keitai  to  be  able  to  access  the  Internet  despite  restrictions  on  using 
company  computers  for  personal  e-mail  or  WWW  use  (see,  for  example,  Schmetzer  2000). 

Low  cost:  As  noted  above,  computer  online  time  has  become  less  expensive  since  Aoki  released  her  paper. 
However,  since  that  time,  increased  competition  between  keitai  service  providers  has  made  monthly  calling 
packages  more  affordable  and  inexpensive  e-mail  messages  are  among  the  most  popular  services  Indeed,  my 
students  emphatically  state  (see  the  next  section  for  additional  details  of  these  empirical  data^  that  low  cost  is 
the  primary  motivation  for  using  written  communications  rather  than  making  a voice  call.[  ] Perhaps  more 
importantly,  the  initial  cost  of  buying  the  necessary  hardware  (in  some  cases  the  phone  is  free),  makes  a keitai 
an  attractive  choice  over  even  the  most  affordable  computer. 

Socially  acceptable:  While  it  is  rare  to  see  a commuter  on  a train  in  Tokyo  using  a laptop  computer  (besides 
himself,  the  author  has  seen  only  one  other  person  using  a computer  on  the  train  in  the  last  1 2 months),  it  is  rare 
not  to  see  several  commuters  engrossed  in  reading  and  writing  keitai  mail.  Indeed,  keitai  mail  users  are  so 
common  at  coffee  shops,  on  public  transportation,  and  in  parks  that  they  would  only  be  conspicuous  in  their 
absence.  Keitai  use  has  become  an  acceptable  means  of  communicating  with  others  in  public  without  the 
negative  images  still  assigned  to  computer  mail  users.  Indeed,  we  may  have  already  entered  a situation  in  Japan 
where  not  carrying  a cell  phone  is  seen  as  socially  unacceptable.  A straw  poll  of  104  of  my  students  in  an  large, 
private,  Japanese  university  indicates  that  all  104  of  them  have  a cell  phone,  and  that  68  of  the  104  used  their 
cell  phone  to  access  the  Internet  before  using  a personal  computer.  Another  indication  of  how  indispensable 
these  devices  have  become,  over  half  (18)  the  students  in  a class  of  32  do  not  wear  a watch  but  rather  use  their 
cell  phone  clock  to  tell  time. 

Unique  Input  Method:  The  most  common  input  method  is  using  one  or  both  thumbs  on  the  phone’s  keypad  (see 
Fig.  2).  Typing  is  possible  (using  an  attachable  keyboard)  but  few  people  use  them  in  public.  While  the  author, 
a novice  at  keitai  mail,  taps  out  only  a few  words  per  minute,  experienced  users  can  achieve  remarkable  speeds 
using  only  their  thumb  to  input  text  and  select  graphic  emoticons  (French,  2000).  French  also  notes  that  one 
heavy  keitai  user  can  reportedly  type  out  messages  solely  by  touch;  this  is  not  as  rare  a skill  as  it  might  sound: 
Several  of  my  students  claim  to  be  able  to  type  keitai  mail  without  looking  at  the  keypad. 

Taken  together,  these  points  indicate  that  the  barriers  to  Internet  use  cited  by  Aoki  concerning  Internet  access 
seem  to  have  been  largely,  if  not  completely,  overcome  by  the  Internet-capable  cellular  telephone.  However, 
while  its  ability  to  enhance  Japanese  Internet  use  may  be  established,  the  keitai  faces  greater  challenges  if  it  is  to 
become  a widely  used  Internet  appliance  for  educational  telecommunications. 


[1]  Each  keitai  e-mail  message  costs  about  2.5  cents  to  send  and  is  free  to  receive  compared  with  voice 
messages  which  run  about  45  cents  per  minute  to  send  but  are  also  free  to  receive.  This  difference  can  add  up 
when  one  realizes  that  it  is  not  uncommon  for  young  people  to  send  10  or  more  messages  per  day. 


Figure  2:  Input  Keypad  Showing  Japanese  and  English  Characters 


Limitations  and  Challenges 

Despite  the  many  advantages  to  the  keitai,  several  obvious  limitations  will  need  to  be  overcome  to  enhance  this 
device’s  potential  as  a general-use  Internet  application.  Initially,  the  small  screen,  especially  when  typing  in 
English,  is  a clear  limiting  factor.  Generally,  the  screen  can  only  display  64  characters  at  one  time  (see  Fig.  1), 
although  total  message  length  depends  on  the  options  allowed  by  the  telephone  and  the  service  provider.  For 
instance,  the  author’s  phone  (shown  in  all  the  images  in  this  paper)  is  capable  of  creating  messages  up  to  128 
characters,  or  “long  mail”  messages  of  up  to  5910  characters.  However,  even  with  a good  resolution  color 
screen,  reading  (let  alone  keying  in!)  a 5000+  character  message  would  strain  the  recipient’s  eyes  to  the  point  of 
exhaustion.  Using  a keitai  for  educational  purposes  would  require  tasks  that  can  be  easily  completed  within  a 
few  hundred  characters  or  modifying  the  keitai's  output  ability  to  enable  the  user  to  read  or  hear  longer 
messages. 

As  indicated  d)ove,  inputting  a lengthy  keitai  mail  message  is  impractical  for  the  average  user.  External 
keyboards  are  available -ranging  from  roughly  the  size  of  a credit  card  to  approximately  the  size  of  a postcard- 
but  the  author  has  never  seen  one  being  used  in  public.  It  is  unclear  if  the  difficulty  of  inputting  text  can  be 
overcome  with  practice,  but  empirical  reports  from  Japanese  college  students  indicates  that  their  typing  speed 
increased  the  longer  they  used  their  keitai.  However,  after  several  weeks  of  inputting  English-language 
messages  using  my  keitai 's  keypad,  my  typing  speed  is  still  painfully  slow  and  I prefer  leaving  voice  messages 
despite  the  increased  cost.  Emoticons  and  preset  phrases  are  one  answer  (see,  for  example,  French  2000). 
However,  I would  welcome  a personal  digital  assistant-like  character  recognition  system  using  an  input  stylus. 
This  is  still  slower  than  typing  on  a keyboard,  but  retains  much  of  the  keitai 's  portability. 


The  Potential  for  keitai  in  Educational  Settings 

The  Japanese  case  shows  that  a large  number  of  people  are  using  their  cellular  phone  to  exchange  e-mail  and 
receive  information  from  the  World  Wide  Web.  Such  widespread  use  should  not  be  ignored  as  such  devices 
may  have  a place  in  educational  telecommunications.  Despite  its  potential,  it  seems  clear  that  keitai  technology 
will  require  additional  development  if  it  is  to  receive  widespread  use  in  educational  settings. 
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Attitudinal  Challenges:  Many  American  schools  had  bans  on  bringing  and  using  cellular  telephones.  After  the 
terrorist  attack  on  September  11,  2001,  however,  some  schools  are  rethinking  this  prohibition  (“Schools 
Rethink”  2001).  While  greater  access  to  cell  phone  use  in  schools  may  have  started  with  a national  tragedy, 
additional  access  will  increase  only  if  the  potential  to  provide  meaningful  educational  opportunities  can  be 
shown. 

Despite  gains  in  many  areas,  actual  computer  and  Internet  use  in  Japanese  and  American  classrooms  is  still  in 
its  early  stages.  Access  issues  in  both  countries  have  largely  been  addressed  and  more  attention  is  being  paid  to 
inservice  education  and  curriculum  development.  However,  computer  use  in  schools  is  still  far  from  established 
and  commonplace  and  there  is  a strong  sense  that  educators  in  both  countries  are  still  in  the  experimental  stage. 
The  prospect  of  purchasing  new  hardware,  educating  teachers,  and  devising  curricula  to  incorporate  a new  type 
of  technology  may  get  a cold  reception  at  many  schools.  It  seems  reasonable  that  any  new  communications 
technology  would  have  to  prove  its  utility  up  front,  and  may  need  to  start  with  a supplementary  role  alongside 
existing  equipment  and  projects,  enhancing  rather  than  replacing  what  is  already  in  place.  One  can  imagine,  for 
instance,  a cell  phone-like  device  being  used  by  one  group  in  the  field  to  communicate  sounds,  text,  and  images 
back  to  their  home  station  computer  where  these  data  are  stored  and  manipulated. 

Hardware  Challenges:  The  expense  of  personal  computers  has  usually  been  borne  by  schools  and  made 
available  to  students  in  the  form  of  a classroom  computer(s)  or  a*  computer  lab.  Cellular  telephones  are  usually 
personal  communication  devices,  purchased  and  maintained  by  an  individual,  and  are  rarely  shared.  Certainly 
allowing  students  to  use  their  own  telephones  as  part  of  a school  exercise  has  some  appeal,  as  the  cost  of  the 
phone  and  connection  service  is  borne  by  the  student.  But  unlike  my  economically  comfortable  university 
students  all  of  whom  carry  cell  phones,  many  college  and  high  school  students  in  Japan  and  America  are  unable 
to  afford  this  service.  If  schools  are  to  provide  wireless  communications  devices  to  students,  they  must  be  able 
to  exert  some  control  over  how  these  devices  are  used.  It  may  be  that  this  need  can  be  filled  by  a portable 
wireless  communication  device  similar  to  a personal  digital  assistant-cell  phone  hybrid  that  enables  the 
exchange  of  text,  sound,  and  graphics  with  a restricted  number  of  other  units.  In  any  event,  current  cell  phone 
technology  shows  some  promise,  but  may  require  modification  and  development  before  it  can  be  comfortably 
used  in  K-12  classrooms. 


Conclusion 

The  greatest  benefit  of  Internet-capable  cell  phones  in  Japan  has  been  their  ability  to  bring  large  numbers  of 
Japanese  people  onto  the  Internet  in  a way  that  has  been  useful  and  entertaining  for  them.  The  learning  curve  is 
negligible  compared  to  personal  computer-based  access,  and  the  cost  of  setting  up  and  using  this  service 
continues  to  attract  large  numbers  of  new  users.  Cellular  phones  have  largely  overcome  Internet-access  barriers 
where  personal  computers  failed,  and  future  developments  should  be  monitored  to  see  if  these  devices  can  be 
adapted  to  educational  use. 

Despite  the  limitations  cited  above,  the  continued  development  of  cellular  phone  technology,  and  the  potential 
to  blend  it  with  personal  digital  assistants  or  other  palm-top  devices,  may  allow  fourth-  or  fifth-generation 
wireless  devices  to  be  incorporated  as  one  part  of  a larger  information  and  telecommunications  strategy  for  use 
in  schools.  As  schools,  in  Japan  and  elsewhere,  become  more  accustomed  to  using  technological  tools  in  the 
classroom,  and  the  cost  of  hardware  continues  to  decline,  we  may  see  a division  of  labor  where  cellular 
telephones  are  primarily  used  for  transmitting  voice,  text,  and  images  while  the  personal  computer  is  used 
primarily  for  more  extensive  communications  and  production  work. 

When  NTT  launched  its  i-mode  service  three  short  years  ago,  few  observers  could  have  guessed  that  millions  of 
users  would  be  exchanging  text  messages  on  their  cell  phones,  performing  online  banking  services,  or  reading 
restaurant  reviews  on  the  Web.  There  is  a sense  that  attempts  to  speculate  where  these  technologies  will  be  in 
three  more  years  will  fall  woefully  short.  However,  I firmly  believe  that  this  trend  deserves  our  attention,  and 
we  should  follow  the  progress  of  portable  wireless  communication  devices  like  keitai  with  an  eye  to 
incorporating  them  into  our  arsenal  or  educational  telecommunications  technologies. 
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Abstract;  Visual  abstractions  play  an  important  role  in  many  learning  situations.  They  refer  to 
visual  structures  that  represent  abstract  ideas  and  encode  the  meaning  of  these  ideas  in 
themselves.  Interaction  with  these  visuals,  when  well-designed,  enables  the  decoding  of  their 
meaning.  When  using  visuals  in  educational  software,  designers  need  to  know  the  different 
forms  of  interaction  available  to  facilitate  learning.  This  paper  briefly  discusses  some  points  to 
be  considered  when  designing  interactive  visual  abstractions. 


Introduction  and  Background 

Visual,  image-based  representations  play  an  important  role  in  learning.  Sometimes,  they  facilitate 
reasoning  and  thinking  in  ways  that  may  be  impossible  to  achieve  using  sententi ally-based  representations 
[Goldstein,  1996].  Visual  abstractions  (VAs)  refer  to  visual  structures  that  represent  abstract  ideas  and  encode 
the  meaning  of  these  ideas  in  themselves  [Pimm,  1995].  An  example  of  a VA  is  the  representation  of  the  concept 
of  rotation  in  two-dimensional  transformation  geometry  as  an  arc  of  rotation  found  in  mathematics  textbooks. 
This  VA  not  only  represents  the  concept  of  rotation,  but  also  its  embedded  sub-concepts  of  angle  and  center  of 
rotation.  Another  example  of  a VA  is  the  representation  of  the  concept  of  force  in  physics  as  an  arrow. 
Researchers  have  studied  the  different  forms  of  representation  and  how  people  reason  using  visuals  [Peterson, 
1996],  most  of  which  have  focused  on  the  static  form  of  these  representations.  With  the  advent  of  powerful 
graphics  systems,  there  is  an  ever  greater  use  of  visuals  in  interactive  contexts.  Interaction  with  VAs  can  play  an 
important  role  in  sense-making  and  active  learning  [Card  et  al.,  1999;  Sedig  et  al.,  2001]. 

It  is  important  for  designers  of  leamware  to  know  about  different  ways  of  designing  interaction  with 
VAs  to  facilitate  active  learning.  However,  literature  on  the  subject  is  very  limited.  Instructional  technology 
books,  research  handbooks,  and  encyclopedias  rarely  discuss  interaction  design  techniques,  much  less 
interaction  design  for  VAs. 

Our  research  group  has  been  designing  a number  of  systems  to  investigate  different  forms  of  interaction 
with  VAs.  This  paper  briefly  discusses  a few  factors  we  feel  need  to  be  considered  when  designing  interaction 
that  facilitates  learning. 


Some  Design  Considerations 

One  of  the  first  considerations  is  whether  the  learner  should  interact  with  a VA  directly  or  indirectly. 
The  former  refers  to  acting  on  and  interacting  with  a VA  directly  via  a pointing  device.  The  latter  refers  to 
interacting  with  a VA  through  an  intermediary  representation.  The  choice  depends  on  the  educational  goal  to  be 
achieved.  In  general,  indirect  interaction  requires  more  mental  effort  [Holst,  1996],  which  can  be  an  important 
factor  in  deep  learning.  However,  direct  interaction,  combined  with  visual  scaffolding,  can  also  support  the 
learning  of  concepts  very  effectively  [Sedig  et  al.,  2001]. 

Another  important  consideration  is  whether  the  learner  should  interact  with  a VA  by  manipulating 
(“handling”)  it,  navigating  (“walking  on”)  it,  or  issuing  commands  to  (“conversing  with”)  it.  These  three  modes 
of  interaction  can  serve  different  purposes.  Manipulation  can  be  used  to  decode  the  internal  meaning  of  a VA. 
For  instance,  in  the  case  of  the  arc  of  rotation,  mentioned  above,  performing  an  operation  on  its  visual 
representation — e.g.,  dragging  the  center  of  rotation — can  help  the  learner  understand  the  structure  and  its 
constituent  elements  [Sedig  et  al.,  2001].  Navigation  can  be  used  to  help  the  learner  construct  a cognitive  map  of 


the  visual  abstraction  [Chen,  1999] — that  is,  a mental  map  of  the  VA’s  structure  which  is  internally  available  for 
inspection  and  sense-making.  Rather  than  manipulating  the  VA  and  affecting  changes  in  its  form,  navigation 
allows  the  learner  to  “walk”  on  the  VA  to  internalize  the  structure’s  landmarks,  geometry,  and  topology.  For 
instance,  given  a semantic  network  as  a VA,  by  navigating  the  network,  the  learner  can  discover  its  landmark 
nodes  and  essential  paths.  However,  if  the  goal  of  learning  was  to  have  the  learner  construct  the  network,  then 
the  preferred  mode  of  interaction  would  be  manipulation.  Conversation  can  be  used  when  the  goal  is  to  allow 
the  learner  to  ask  for  the  properties  of  a VA  in  a linguistic,  sentential  manner.  Conversation  usually  requires 
more  mental  effort  than  manipulation  or  navigation  since  linguistic  commands  and  visual  representations  are  not 
cognitively  congruent. 

An  example  of  an  application  that  uses  two  modes  of  interaction  is  a cognitive  tool  we  have  designed 
intended  to  facilitate  learning  of  Platonic  and  Archimedean  solids,  shown  in  Figure  1.  The  learner  can 
manipulate  the  geometric  structure  on  the  right  to  change  its  form  and  observe  how  it  morphs  to  another 
geometric  structure  [Morey  et  al.,  2001].  The  learner  can  navigate  the  map  on  the  left  to  understand  the 
relationship  among  the  solids.^  The  learner  can  also  click  on  the  linguistic  buttons  to  select  a new  visual 
representation.  Although  this  is  not  a purely  conversational  mode  of  interaction,  one  can  design  the  interface 
such  that  the  learner  could  type  a command  to  query  a given  geometric  structure  about  its  properties. 


Figure  1 


This  paper  has  presented  some  points  to  be  considered  when  designing  interactive  VAs  for  leamware. 
The  understanding  of  many  educational  concepts  can  be  facilitated  using  VAs  [Glasgow  et  al.,  1995];  therefore, 
it  is  important  to  further  investigate  different  forms  of  interaction  with  VAs  so  that  they  can  support  active 
learning. 
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Abstract:  Many  children  find  mathematics  uninteresting  and  irrelevant,  never  appreciating  its 
role  in  artistic  endeavors.  Computer-based  interactive  tools  can  help  introduce  children  to  the 
artistic  aspects  of  mathematics.  This  paper  presents  TileLand,  a multi-faceted  constructionist 
microworld  intended  to  engage  children  in  the  creation  of  mathematical  art.  TileLand  employs 
the  metaphor  of  a Magic  Tile  to  introduce  children  to  the  world  of  regular  polygons  and 
colorful  visual  patterns  and  tilings.  It  integrates  several  types  of  pedagogical  interactions: 
open-ended  exploration,  goal-directed  game  play,  experiential  and  reflective  thinking,  and 
visual  and  linguistic  information  processing. 


Introduction 

Many  children  find  mathematics  uninteresting  and  irrelevant.  This  is  partly  due  to  the  fact  that  most  of  their 
encounter  with  mathematics  is  algebraic  [Kinsey  and  Moore,  2002].  As  a result  these  children  never  develop  a 
mathematical  eye.  Without  a mathematical  eye,  they  never  realize  the  role  of  mathematics  in  artistic  expressions 
such  as  painting,  sculpturing,  and  designing  architectural  artifacts.  It  is  important  for  non-mathematicians  to 
experience  the  beauty  of  mathematics  by  venturing  into  its  art  form,  as  visualization  of  patterns  and  relationships 
[Peterson,  2001].  Tilings,  tessellations,  and  visual  patterns  are  examples  of  mathematical  art.  The  National 
Council  of  Teachers  of  Mathematics  [1989]  emphasizes  the  importance  of  young  learners  experiencing  the 
deeper  aspects  of  geometry.  Novel  computer  tools  can  help  introduce  children  to  the  visual  aspects  of 
mathematics  (for  instance,  see  [MECC,  1995]). 

However,  designing  technology  that  is  motivating,  supports  children’s  deep  learning,  and  is  not  gender- 
biased  is  not  an  easy  task  [Alessi  and  Trollip,  2001;  Druin,  1999;  Sedig,  1998;  Sedig  et  al.,  2001].  This  is 
especially  true  if  the  design  is  aimed  at  a broad  spectrum  of  children,  ranging  from  kindergarten  to  the  6^  grade. 
This  paper  presents  TileLand,  a microworld  designed  to  engage  children  in  geometric  reasoning  and  analysis 
through  the  creation  of  mathematical  art  in  the  form  of  tiling  patterns.  As  a microworld,  TileLand  is  a self- 
contained  conceptual  system  embodying  underlying  geometric  principles  related  to  five  regular  polygonal 
shapes.  Through  interaction  with  the  system,  it  is  intended  that  children  develop  a sense  of  spatial  visualization, 
notice  the  relationship  among  polygonal  figures,  and  discover  the  role  of  different  polygons  when  creating  tiling 
patterns  - thereby  constructing  their  own  knowledge  of  the  embedded  concepts  and  ideas  in  the  system  [Forman 
andPufall,  1988]. 


TileLand 

TileLand  is  a multi-faceted  microworld.  It  allows  children  to  construct  colorful  tiling  patterns  using  regular 
polygons,  engaging  them  in  creative,  mathematical  art.  TileLand  is  also  a logic-based,  problem-solving  game,  in 
which  children  need  to  re-produce  a series  of  given  patterns  in  order  to  advance  through  the  game.  It  is  also  a 
programming  microworld,  where  using  a simple  command-based  language,  children  can  describe  how  to  create 
a tiling  pattern,  test  the  program,  and  observe  its  behavior. 


TileLand  is  aimed  at  children  who  are  five  years  and  older.  Consequently,  it  has  simple  interaction  rules. 
There  are  only  a few  commands  that  children  need  to  learn.  Nevertheless,  TileLand  allows  children  to  construct 
fairly  complex  geometrical  patterns.  Figure  1 shows  a screen  capture  of  TileLand.  The  screen  is  divided  into 
three  sections:  Action  Panel , Sample  Tilings^  and  Pattern  Area.  The  Action  Panel  contains  the  command  buttons 
used  to  create  tiling  patterns.  The  Sample  Tilings  contains  a number  of  pre-created  patterns,  with  varying 
degrees  of  complexity.  These  patterns  are  intended  to  introduce  children  to  a range  of  possible  visualizations 
that  can  be  created  using  the  environment.  Finally,  the  Pattern  Area  is  where  children  construct  their  patterns. 
Figure  1 shows  a constructed  pattern  made  of  four  types  of  polygons:  triangles,  squares,  pentagons,  and  an 
octagon. 

There  are  five  types  of  tiles  with  which  children  can  construct  patterns.  All  five  tiles  are  regular  polygons: 
triangle,  square,  pentagon,  hexagon,  and  octagon.  To  provide  an  easy-to-understand  conceptual  model  for  how 
tilings  are  created,  TileLand  uses  the  metaphor  of  a Magic  Tile.  The  Magic  Tile  is  represented  in  the  Pattern 
Area  by  any  one  of  the  five  polygons.  One  of  the  sides  of  the  Magic  Tile  is  designated  as  a Connecting  Side. 
This  side  is  highlighted  to  differentiate  it  from  the  other  sides  of  the  Magic  Tile.  The  Connecting  Side  is  where  a 
new  polygon  attaches.  The  magic  of  the  tile  is  derived  from  its  transformational  movement.  This  movement 
occurs  when  a new  polygon  is  attached  to  the  connecting  side  of  the  polygon  currently  in  the  Pattern  Area  and 
the  current  polygon  magically  disappears.  Clicking  on  one  of  the  five  polygonal  buttons  in  the  Action  Panel 
causes  the  Magic  Tile  in  the  Pattern  Area  to  transform  to  the  new  polygon.  Clicking  on  the  Turn  button  will 
change  the  Magic  Tile’s  Connecting  Side  one  step  at  a time  in  the  clockwise  direction.  A central  location  in  the 
Pattern  Area  represents  the  Home  position  for  starting  a pattern*.  At  the  start  of  a session,  when  the  user  clicks 
on  a polygonal  button,  the  Magic  Tile  appears  at  the  Home  position. 


The  Magic  Tile  can  be  filled  with  a color  or  left  blank.  For  instance,  if  the  user  clicks  on  the  red  button,  the 
Magic  Tile’s  polygonal  frame  will  be  filled  with  the  color  red — creating  a red  footprint.  The  user  can  thus 
construct  a pattern  by  leaving  colorful  footprints  as  the  Magic  Tile  ‘walks  around’  the  Pattern  Area.  If  the  Magic 
Tile  is  filled  with  color  while  it  overlaps  existing  footprints,  the  underlying  footprints  disappear  and  are  replaced 
by  the  new  footprint.  This  behavior  ensures  that  the  resulting  pattern  is  solely  comprised  of  non-overlapping 
regular  polygons. 

The  magical  movement  of  the  tile  and  understanding  how  footprints  appear  and  disappear  constitute  all  the 
basic  rules  of  the  environment  which  children  need  to  know  in  order  to  work  with  TileLand.  Once  children  know 
how  to  operate  the  mouse,  this  conceptual  simplicity  of  TileLand  is  intended  to  make  it  easy  for  them  to  interact 
with  the  environment. 

In  addition  to  the  above  basic  rules,  there  are  a few  other  features  to  enhance  the  environment.  An  UNDO 
feature  allows  children  to  undo  their  previous  actions.  They  can  thus  explore  and  backtrack  different 
visualization  paths.  A Clear  feature  allows  children  to  clear  the  screen.  A homing  feature  {Home  button)  allows 
children  to  reset  the  location  of  the  Magic  Tile  and  return  to  the  center  of  the  Pattern  Area.  Children  can  also 
turn  a background  grid  on  or  off.  A Look  Ahead  feature  allows  children  to  see  the  next  position  of  a Magic  Tile 


* Our  formative  usability  evaluations  suggested  that  it  is  easier  for  very  young  children  to  be  given  a starting  position  so 
they  do  not  have  to  worry  about  where  to  begin. 
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if  they  were  to  click  on  it.  Children  can  also  save  or  print  a tiling  pattern  that  they  have  constructed  and  open  an 
existing  one. 

TileLand  is  an  open-ended  environment  intended  to  allow  children  to  progress  through  it  at  their  own  pace, 
gradually  developing  more  sophisticated  2-dimensional  visualization  abilities.  Currently,  TileLand  has  four 
levels.  Different  levels  have  different  learning  features  and  can  be  regarded  as  different  types  of  learning 
environments. 


Level  1:  Open-Ended  Construction  Tool 

Level  1 provides  children  with  a construction  tool  that  has  a specific  focus — that  of  creating  mathematically- 
based  patterns  and  tilings.  Therefore  it  is  based  on  the  constructionist  approach  to  learning  [Papert,  1980;  Harel 
and  Papert,  1991].  This  level  is  purely  exploratory.  Children  are  free  to  pursue  their  interests  and  create  any  type 
of  pattern  they  desire.  Although  these  patterns  may  have  little  mathematical  properties,  they  help  children 
develop  an  understanding  of  the  metaphoric  language  of  interaction  used  in  the  environment.  Additionally, 
Level  1 provides  a motivating  environment  in  which  children  can  construct  colorful  designs.  Figure  1 shows  a 
sample  design  created  by  a seven-year-old  child.  Although  this  module  is  referred  to  as  a level,  it  is  a self- 
contained  environment — that  is,  children  need  not  progress  to  or  explore  any  other  levels. 


Level  2:  Logic-Based,  Problem-Solving  Game 

Level  2 is  a typical  sequential  game-based,  problem-solving  activity  [Alessi  and  Trollip,  2001].  It  is  a goal- 
driven  version  of  Level  1.  At  this  level,  children  need  to  solve  a variety  of  geometrical  pattern  problems  using 
the  basic  rules  of  TileLand  which  they  learn  in  Level  1.  Children  are  given  explicit,  pre-created  patterns  to  re- 
produce. Whereas  Level  1 is  more  experiential.  Level  2 is  more  about  making  decisions,  comparing  the  merits 
of  different  sequences  and  polygons,  and  reflecting  on  their  relationships  to  determine  how  an  existing  pattern 
can  be  created  [Norman,  1993;  Sedig,  1998].  For  instance.  Figure  2 shows  a simple  pattern  that  children  can  re- 
produce. They  can  produce  the  pattern  in  several  ways.  One  strategy  is  to  produce  the  pattern  in  a radial 
manner — i.e.,  start  from  the  center,  move  out,  return  to  the  center,  turn,  move  out  again,  return  to  the  center,  and 
so  on.  Another  strategy  is  to  produce  the  pattern  by  walking  around  the  central  hexagon — i.e.,  start  from  the 
center,  move  out,  turn,  walk  around,  placing  other  hexagons  around  the  central  one  (see  Figure  5). 


Level  3:  Open-Ended  Programming  Microworld 

Sedig  et  al.  [2001]  suggest  that  interfaces  that  gradually  move  learners  from  a visual  thinking  strategy  to  a 
linguistic  (algebraic  or  command-based)  one  can  be  effective  in  engaging  learners  in  reflective  thought  and 
deeper  processing  of  educational  concepts.  In  Level  3,  children’s  interactions  with  the  Magic  Tile  are 
simultaneously  translated  into  linguistic  commands  appearing  in  a Programming  Panel  on  the  right-hand  side  of 
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the  screen  (see  Figure  3).  For  instance,  clicking  on  the  square  button  produces  the  command  ‘square’,  and  other 
buttons  produce  commands  such  as  ‘pentagon’,  ‘red’,  and  ‘turn’.  There  are  sixteen  commands  in  total  which  can 
be  categorized  into  two  main  conceptual  groups — five  polygon  commands  and  nine  color  commands — plus 
Turn,  Home,  and  Clear.  The  sequence  of  commands  in  the  panel  corresponds  to  all  the  actions  the  user  has 
performed.  This  sequence  serves  at  least  three  goals:  1)  it  can  act  as  a bridge  between  the  visual  and  linguistic 
description  of  a pattern,  2)  it  can  act  as  a mapping  between  the  spatial  and  temporal  representations  of  a pattern, 
and  3)  it  can  act  as  an  external  representation  that  can  be  examined  and  modified. 

Visual-to-Linguistic  Description:  All  visual  mathematical  concepts  have  a corresponding  linguistic 
description  [Devlin,  2000].  Additionally,  an  important  feature  of  educational  environments  is  to  provide 
multiple  representations  of  content  [Tabachneck-Schijf  and  Simon,  1996;  Lajoie,  2000].  In  Level  3 children 
learn  that  visual  patterns  can  be  described  linguistically.  One  of  the  benefits  of  this  multiplicity  of 
representations  is  that  visual  representations  are  processed  differently  from  linguistic  representations.  The 
former  are  processed  in  parallel,  whereas  the  latter  are  processed  sequentially,  requiring  more  cognitive 
processing  and  investment  of  mental  effort  [Sternberg,  1999].  As  such  the  two  forms  of  representation 
complement  each  other  (see  Figure  5). 

Temporal-to-Spatial  Mapping  of  Interaction:  When  children  interact  with  the  buttons  in  TileLand,  their 
interactions  produce  a spatial  image.  This  image  implicitly  contains  the  history  of  the  steps  that  led  to  its 
creation.  However,  the  only  way  that  children  can  access  this  history  is  to  examine  their  own  memory  which  may 
contain  cognitive  residues  of  their  interaction  with  the  system.  Instead  of  relying  on  children’s  short-term 
memory,  TileLand  maps  their  temporal  interactions  into  a spatial  representation — i.e.,  a linearized  history  of 
their  actions  in  the  Programming  Panel. 

Externalization  as  an  Object  of  Reflection  and  Interaction:  The  extemalization  of  the  visual  description  and 
interaction — that  is,  its  conversion  to  ‘code’ — affords  children  to  both  reflect  upon  the  code  as  well  as  interact 
with  it.  Children  can  use  the  programming  panel  to  modify  and  augment  the  sequence  of  actions  (i.e,  the  code) 
by  copying,  cutting,  pasting,  and  typing  code.  By  copying  and  pasting  blocks  of  code,  children  are  performing 
rudimentary  procedural  routines^.  Children  can  bypass  the  use  of  buttons  entirely  and  use  the  Programming 
Panel  as  an  editor  as  well  as  a dynamic  interpreter  to  write  code  from  scratch  to  produce  their  patterns.  Code 
written  in  the  programming  panel  can  be  ‘run’,  and  a visual  simulation  of  its  corresponding  pattern  is  generated 
step  by  step.  This  provides  children  with  a very  simple  programming  microworld — a self-contained  software 
environment  which  embodies  a reduced  set  of  principles  and  concepts  from  a domain  of  knowledge  [Ginsburg  & 
Zelman,  1988].  Children  can  augment  the  history  of  their  actions  by  inserting  comments  in  the  code.  This 
feature  helps  the  reflective  practice  of  coding  allowing  children  to  chunk  blocks  of  code  into  meaningful  units. 
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Figure  5:  A simple  pattern,  a possible  program  for  its  creation,  and  its  corresponding  strategy 


^ Indeed  using  cutting  and  pasting,  the  user  may  find  it  easier  to  produce  more  elaborate  patterns  (see  Figure  4). 


Level  4:  Programming  Game 


Level  4 is  a problem-solving  game,  similar  to  Level  2.  However,  the  Action  Panel  is  no  longer  present  (see 
Figure  4).  It  is  replaced  by  the  Programming  Panel.  In  order  to  advance  through  the  game,  children  need  to 
produce  a purely  linguistic  description  of  the  given  patterns.  Children  do  this  in  a coding-testing-reflecting- 
modifying  cycle.  They  can  write  a few  lines  of  code  or  an  entire  program,  test  it,  reflect  on  the  resulting  pattern 
and  its  code  (i.e.,  discover  why  it  did  or  did  not  work),  and  modify  the  code  as  needed.  Unlike  Level  2 where 
children  can  see  the  immediate  result  of  every  action  while  clicking  on  action  buttons  trying  to  re-produce  a 
pattern,  in  Level  4 there  is  a temporal  disconnect  between  devising  a solution  and  testing  it.  This  disconnect 
implies  that  children  may  need  to  mentally  ‘walk  through’  the  visualization  before  they  produce  and  test  the 
code  and  observe  its  extemalization.  This  mental  simulation  is  one  of  the  most  important  skills  in  visualization 
that  a learner  can  acquire  [NCTM,  1989]. 


Usability  Evaluation 

TileLand  has  gone  through  a series  of  formative  evaluations.  These  evaluations  have  greatly  influenced  its 
design.  Formative  evaluations  were  conducted  with  half  a dozen  children,  ages  ranging  from  five  to  eight.  In  the 
early  stages  of  its  design,  TileLand  contained  such  actions  as  Forward,  Back,  TumLeft,  TumRight,  and  Mark. 
However,  we  discovered  that  children  had  a difficult  time  constructing  a proper  conceptual  model  of  the 
environment.  For  instance,  the  Current  Tile  had  a Current  Color  while  it  moved.  Children  had  to  click  on  a Mark 
button  for  the  tile  to  leave  a footprint.  They  found  it  difficult  to  understand  why  a tile  that  had  a current  color 
would  not  leave  a footprint  while  moving  and  that  they  needed  to  click  on  a ‘mark’  button  for  the  footprint  to  be 
created.  Additionally,  they  could  not  understand  the  difference  between  a tile  attaching  to  a current  one  and 
moving  a tile  forward  and  backward. 

After  an  iterative  process  of  testing  and  modifying  the  environment,  we  minimized  the  number  of 
conceptual  actions  and  introduced  the  notion  of  the  Magic  Tile.  In  our  latest  usability  evaluations,  several 
children,  as  young  as  five  years  old,  used  the  system  with  ease  and  found  it  engaging.  They  produced  many 
printouts  and  were  proud  of  their  designs. 


Summary  and  Future  Work 

This  paper  has  presented  TileLand,  a multi-faceted  microworld.  TileLand  allows  children  to  construct  colorful 
tiling  patterns  using  regular  polygons,  engaging  them  in  creative,  mathematical  art.  TileLand  is  also  a logic- 
based,  problem-solving  game  which  encourages  children  to  explore  the  underlying  geometric  principles 
involved  in  constructing  tiling  patterns.  TileLand  can  also  be  considered  a programming  microworld.  It  allows 
children  to  use  simple  descriptive  language  to  express  how  a tiling  pattern  should  be  created  while  dynamically 
testing  their  description. 

TileLand  integrates  several  types  of  pedagogical  interaction  styles.  It  involves  open-ended  exploration  as 
well  as  goal-directed  play;  it  can  be  considered  an  experiential,  fun  tool  as  well  as  a reflective,  challenging  one; 
and  it  requires  visual  as  well  as  linguistic  information  processing  skills. 

Future  work  involves  conducting  formal  empirical  evaluations  of  TileLand  to  assess  its  educational 
effectiveness.  Additionally,  the  order  of  the  presentation  of  the  tiling  puzzles  in  the  game  levels  need  to  be 
examined,  as  each  tiling  pattern  embodies  different  mathematical  concepts  [Peterson,  2001;  Kinsey  and  Moore, 
2002].  This  organization  can  affect  the  way  in  which  children  conceptualize  and  understand  the  patterns  and 
their  underlying  principles. 
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Abstract 

This  presentation  describes  a project  to  educate  a team  of  college  students  in  designing  and  developing  an 
online  course  for  the  National  Security  Agency  (NSA)  utilizing  constructivist  methods  of  learning. 

Constructivist  Approaches  To  Learning  And  Research 

This  project  involved  converting  a face-to-face  NSA  course  to  a multimedia  web-based  course.  The  conversion 
included  a team  of  undergraduate  and  graduate  students  from  Instructional  Technology,  Art  Education,  Computer 
Science,  Special  Education,  and  Graphic  Design.  Representation  of  various  cultures  was  taken  into  consideration 
when  selecting  the  students.  In  addition,  two  students  were  already  teachers  in  the  public  schools  and  had  experience 
in  curriculum  development.  Two  professors  directed  the  project  and  were  part  of  the  team  — one  from  Instructional 
Technology  and  the  other  from  Art  Education.  The  team’s  focus  was  on  collaborative  group  work  and  a 
constructivist  approach  to  learning.  Constructivist  approaches  served  as  the  basic  foundation  for  organizing  and 
developing  the  project  “...where  students  develop  their  knowledge  through  team  collaboration,  discuss  different 
interpretations  of  a problem,  and  negotiate  and  synthesize  ideas  drawing  from  various  disciplines”  (Boyer  & 
Semrau,  1995,  14).  The  constructivist  goal  was  for  the  students  to  collaborate  and  the  faculty  to  facilitate. 
Technology  allowed  the  students  to  have  more  control  over  their  own  learning  and  become  more  independent  in 
their  search  for  knowledge.  Jonassen  noted  that  significant  knowledge  construction  could  be  facilitated  by: 

1.  Learning  environments,  which  include  working  with  authentic  tasks, 

2.  Providing  real  world  learning  environments, 

3.  Fostering  reflective  practices, 

4.  And,  supporting  collaborative  construction  of  knowledge  through  social  negotiation  rather  than  competition 
among  learners  for  recognition. 

The  students  were  trained  in  new  skills  and  how  to  research  and  work  collaboratively  with  both  faculty 
members  and  each  other.  Training  was  provided  on  html,  Photoshop,  Premiere,  MediaCleaner,  Pagemill  and  the 
designing  of  web  pages.  WebCT  was  used  for  managing  the  website,  communication  and  collaboration,  and 
uploading  pages. 

A major  course  goal  was  to  keep  the  web  course  interactive  emphasizing  learner  control.  Useful  tools  for 
designing  constructivist  approaches  to  learning  are  hypertext  and  hypermedia  because  they  allow  for  a branched 
instructional  design  rather  than  a linear  top-down  approach.  It  was  also  important  in  the  distance  education  course 
that  there  be  a balance  between  the  text  and  the  use  of  graphics,  diagrams,  video  and  audio  to  reinforce  the  learning. 

For  a foundation  in  working  collaboratively  in  building  a website,  the  students  first  designed  a demonstration 
site  on  streaming  media  with  RealAudio  and  RealVideo  examples.  The  demonstration  site  was  developed  for 
sharing  our  research  and  for  communication  among  the  students  and  project  faculty.  Emphasis  was  placed  on 


demonstrating  educational  uses  of  streaming  video  and  audio  in  education  as  well  as  being  a source  of  research  data 
and  instruction  for  creating  streaming  video  and  audio  for  web-based  training.  Roberts  et  al  state,  “...students  need  to 
be  able  to  find  and  use  relevant  information,  share  and  discuss  data  and  ideas,  and  collaborate  on  problem  solving” 
(1990,  116). 

In  this  presentation  five  stages  are  described,  which  are  involved  in  creating  a distance  education  course. 
Specific  strategies  are  detailed  and  actual  solutions  are  illustrated  for  each  of  the  following  5 stages: 

1 . Planning  and  Development  of  the  Content 

2.  Layout  and  Design  of  the  Website 

3.  Implementation 

4.  Editing  the  Content 

5.  Assessing  Learning  and  Evaluation 


Conclusion 

The  new  web-based  version  of  this  course  will  make  the  content  accessible  to  more  NSA  employees  located  in 
various  places  throughout  the  world.  Plus,  the  online  version  will  include  interactive  pop  quizzes  with  immediate 
scoring  and  feedback  as  well  as  streaming  media  lectures  to  complement  the  course  content,  which  is  presented  as 
text  on  the  web  pages. 

In  addition  as  Boettcher  and  Cartwright  (1997)  emphasized,  web  course  designers  must  realize  that  the 
“individual  learner  must  be  viewed  as  the  key  design  element.”  We  need  to  design  instruction  so  that  each  individual 
learner  can  effectively  build  on  what  he  or  she  knows  and  have  the  resources  and  assistance  to  learn  (p.  2). 

Through  this  project  the  students  became  constructively  involved  in  their  own  learning  and  acquired  in-depth 
experiences  in  collaborative  learning  and  team  approaches.  New  roles  were  created  for  both  the  teachers  and  the 
students.  The  students  became  empowered  to  be  creators  of  their  own  curricular  materials  and  web  pages  instead  of 
being  passive  viewers  of  others’.  In  this  project  all  of  Bloom’s  higher-level  taxonomies  were  implemented: 

• Students  analyzed  websites, 

• Synthesized  criteria  that  they  researched, 

• Applied  their  book  readings, 

• Compared  and  contrasted  their  criteria, 

• Designed  and  produced  their  own  web  pages  focused  on  sound  educational  practices. 

The  faculty  worked  side-by-side  with  the  students,  setting  new  goals  together,  and  evaluating  and  problem- 
solving as  a team.  As  Gibson  (1998)  noted  “...we,  as  distance  educators,  need  to  be  learner-centered  reflective 
practitioners”  (p.  143). 
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Abstract:  DePaul  University  has  developed  the  Instruction  Builder,  an  application  for  creating,  delivering  and 
administering  online  tutorials.  This  paper  discusses  how  online  library  instruction  differs  from  online  instruction  for  a 
semester  or  quarter  long  course.  It  also  examines  the  roles  of  peer-to-peer,  student/'instructor,  and  ongoing 
communication  in  an  online  learning  environment. 


Courses  with  a strong  research  component  will  often  turn  to  the  library  to  teach  a *one-shot’  session  on 
effective  library  research.  The  librarian's  goal  is  to  foster  information  literacy:  to  teach  the  students  the  skills 
necessary  to  find  the  information  they  need.  The  quantity  and  complexity  of  research  sources  has  grown 
tremendously  in  recent  years,  and  it  can  be  a challenge  to  cover  anything  beyond  the  basics  in  these 
workshops.  There  is  also  little  chance  for  the  librarian  to  follow  up  with  individual  students,  and  some 
students  may  see  the  session  as  busy  work  if  it  there  is  not  a visible  degree  of  collaboration  between  the 
librarian  and  the  instructor. 

Faced  with  higher  enrollments,  shrinking  resources,  more  content  to  teach,  and  a growing  population  of 
distance  ed.  students,  librarians  have  turned  to  the  web  to  teach  information  literacy  skills.  One  of  the  major 
challenges  librarians  face  is  that  while  they  now  teach  their  skills  online,  the  integration  of  these  skills  into 
the  course  itself  has  not  changed.  Most  web-based  research  tutorials  are  still  a one-shot  session,  an 
assignment  that  students  must  complete  on  their  own  time.  Librarians  are  creating  online  workshops  as 
opposed  to  online  courses. 

The  distinction  between  online  workshop  and  online  course  is  a crucial  one  in  selecting  the  right  tools. 
Most  Learning  Management  Systems  (LMS),  such  as  Blackboard  or  WebCT  are  built  with  the  assumption 
that  online  instruction  will  'mimic'  a traditional  quarter  or  semester  long  course.  In  his  ED-MEDIA  2001 
keynote  address,  David  Jonassen  argued  that  these  assumptions  reinforce  some  of  the  worst  aspects  of 
traditional  in-class  education.^  For  librarians,  they  simply  don't  work.  These  systems  provide  an  array  of 
features  that  cannot  be  used  in  a one-shot  session  and  don't  provide  the  tools  needed  to  effectively  teach 
the  skills  in  question. 

The  DePaul  Libraries  began  offering  online  library  instruction  in  the  Winter  of  1998  with  the  Library 
Research  Workshop.  This  workshop  teaches  up  to  40  sections  of  freshman  English  students  per  quarter 
how  to  search  online  databases  to  find  journal  articles.  The  library's  first  version  of  the  workshop  used 
WebCT  to  create  the  class  navigation.  The  problem?  We  were  unable  integrate  teaching  materials, 
demonstrations,  quizzes  and  assignments  on  the  student's  own  topic.  The  first  version  of  the  workshop 
forced  students  to  wade  through  a great  deal  of  material  before  reaching  anything  directly  relevant  to  their 
own  interests.  In  the  fall  of  2000,  the  Library  needed  to  bring  a more  extensive  research  program  for  adult 
learners  online.  We  saw  that  WebCT  and  the  other  LMS  systems  would  not  meet  our  needs,  so  we  began 
work  on  the  Instruction  Builder,  an  in-house  Learning  Content  Management  System  (LCMS). 


* David  Jonassen.  (June,  2001).  E-Leaming  to  Solve  Problems.  Keynote  Address:  ED-MEDIA  World  Conference  on 
Educational  Multimedia,  Hypermedia  & Telecommunications.  Tampere,  Finland 


The  distinction  between  the  LMS  and  the  LCMS  is  derived  from  an  IDC  white  paper.^  LCMS  software  is 
more  focused  on  teaching  skills  than  on  managing  classrooms.  Consequently,  rather  than  offering 
discussion  boards,  course  syllabi,  automatic  grading  and  other  common  LMS  features,  it  focuses  on  helping 
teachers  break  down  their  content  into  small  sections,  and  teaching  that  content  in  more  interesting  ways.  It 
allows  both  the  teaching  material  and  the  students’  responses  to  be  stored  in  a database,  to  better  track 
which  skills  are  taught  well  and  which  skills  students  are  frequently  missing. 

The  Instruction  Builder  organizes  content  into  skill-based  modules.  Each  module  ends  with  an  assignment 
that  applies  the  skill  taught  to  the  student’s  own  interests  using  open-ended  questions.  Because  the  system 
is  divided  into  segments,  authors  are  deterred  from  creating  long  pages  that  attempt  to  teach  too  many  skills 
at  once.  Students  are  tested  on  skills  based  on  their  own  interests  as  they  leam  them,  rather  than  having  to 
plow  through  a long  workshop,  then  take  a long  test.  The  student’s  answers  are  reviewed  by  a librarian, 
who  then  responds  to  the  student. 

When  librarians  taught  these  workshops  in  person,  a great  deal  of  emphasis  was  placed  on  peer-to-peer 
communication.  Librarians  had  students  work  in  groups  to  construct  search  strategies  for  one  another’s 
topics.  This  taught  the  creative  process  of  word  choice  in  database  searching.  Some  students  were  likely 
to  come  up  with  terms  that  wouldn’t  have  occurred  to  others,  and  it  was  clear  that  talking  about  the  topic 
helped  them  search  it  more  effectively.  There  was,  however,  very  little  one  to  one  or  ongoing  communication 
between  the  librarian  and  any  individual  student.  When  students  were  in  the  lab  working  on  the 
assignment,  there  was  one  librarian  attempting  to  help  over  20  students  at  once! 

The  move  to  an  online  environment  has  eliminated  peer-to-peer  communication,  but  has  greatly  enhanced 
one  on  one  and  ongoing  communication  between  the  librarian  and  student.  As  of  December  2001, 
approximately  1,700  students  have  taken  workshops  created  with  Instruction  Builder.  Each  student’s 
assignment  is  individually  reviewed,  and  the  librarian  is  able  to  recommend  strategies  and  alternatives  when 
appropriate.  In  terms  of  communication  with  a librarian,  the  library  research  portion  of  these  courses  is  now 
more  fully  integrated  into  the  course  itself 

Both  the  loss  of  peer-to-peer  communication  and  the  enhancement  of  student/librarian  communication  are 
crucial.  Authors  such  as  Curtis  Bonk  increasingly  point  to  communication  and  interaction  as  the  key  to 
successful  online  learning.  In  ”A  Dozen  Recommendations  for  Placing  the  Student  at  the  Center  of  Web- 
Based  Instruction”,  six  of  the  recommendations  pertain  to  peer-to-peer  or  ongoing  communication.  ^ 

The  Library  has  already  outgrown  the  system  developed  in-house.  While  there  are  a number  of  promising 
LCMS  systems  appearing  on  the  scene,  including  Avaltus  and  Knowledge  Mechanics,  these  systems  don't 
provide  the  tools  for  fostering  ongoing  communication  between  student  and  instructor  which  we  see  as  the 
core  of  our  workshops’  value.  DePaul’s  Instructional  Technology  Department  has  begun  work  on  a more 
robust  Instruction  Builder  system,  to  be  implemented  throughout  the  University. 

The  fact  that  the  one-shot  research  workshop  model  has  been  carried  over  from  the  traditional  setting  to  the 
online  environment  has  more  to  do  with  the  assumptions  & habits  of  librarians  and  instructors  than  the 
limitations  of  technology.  This  can  (and  should)  be  changed  by  breaking  up  the  one-long-workshop  model 
into  smaller  segments,  which  are  taken  during  selected  weeks  of  the  quarter.  It  was  assumed,  for  instance 
that  since  we  had  an  hour  and  a half  in  class,  that  we  would  also  have  an  hour  and  half  online.  But  an  hour 
and  a half  sitting  at  the  computer  taking  an  online  course  is  a very  different  experience,  and  can  be  fairly 
numbing.  Breaking  the  workshop  up  would  integrate  it  more  completely  into  the  course  content,  foster  more 
exchange  between  the  student  and  librarian,  and  would  ease  the  fatigue  of  online  instruction. 


^ Brennan,  Michael,  et  al.  (May,  2001).  The  Learning  Content  Management  System:  A New  E-Leaming  Market 
Segment  Emerges.  http://www.intemettime.eom/itimegroup/lcmsADCLCMSWhitePaper.pdf 

^ Bonk,  C.  J.,  & Cummings,  J.  A.  (1998).  A dozen  recommendations  for  placing  the  student  at  the  center  of  Web-based 
instruction.  Educational  Media  International,  ii(2),  82-89. 
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Abstract:  This  conceptual  essay  examines  theoretical  and  methodological  problems  associated  with  teaching 
the  Social  Foundations  of  Education  course  and  provides  a framework  on  how  to  overcome  these  difficulties. 
Foundations  of  Education  courses  are  notable  for  both  their  volume  and  the  ambivalence  with  which  students 
and  instructors  regard  their  role  in  preparation  for  teaching.  The  ambivalence  is  held  because  foundations 
courses  generally  suffer  from  instructors  unknowingly  devising  incompatible  goals  for  the  course  while  the 
multidisciplinary  nature  of  the  course  frequently  leads  to  intellectual  incoherence.  One  way  to  overcome 
these  deficiencies  is  to  have  students  acquire  an  “ironic”  understanding  using  the  case  method.  Case  studies 
that  use  digital  technology  make  this  approach  even  more  powerful  because  digital  technology  provides 
students  with  a cognitive  tool  that  encourages  the  profound  knowledge  necessary  to  attain  ironic 
understanding. 


This  conceptual  essay  examines  theoretical  and  methodological  problems  associated  with  teaching 
the  Social  Foundations  of  Education  course  and  provides  a framework  on  how  to  overcome  these 
difficulties.  A staple  of  teacher  education  since  the  1930's  (Cohen  1999;  Lagemann  2000),  Foundations  of 
Education  courses  are  notable  for  both  their  volume  and  the  ambivalence  with  which  students  and 
instructors  regard  their  role  in  preparation  for  teaching.  The  ambivalence  is  rightfully  held  by  both  because 
foundations  courses,  from  their  inception,  have  suffered  two  significant  shortconings.  First,  instructors 
unknowingly  have  devised  incompatible  goals  for  the  course.  Second,  the  multidisciplinary  nature  of  the 
course  frequently  leads  to  intellectual  incoherence  for  all  concerned. 

As  a teacher  of  Foundations  courses,  I have  struggled  with  overcoming  these  embedded  course 
deficiencies.  Over  time,  I came  to  a realization  that  the  course  needed  a new  goal.  I borrowed  from  Kieran 
Egan  (1997)  the  idea  of  having  students  acquire  an  “ironic”  understanding.  Due  to  the  multidimensional 
representation  possible  with  cases,  attaining  ironic  understanding  is  enhanced  when  case  methods  are 
employed  (Merseth,  1996).  Most  recently,  I have  begun  to  supplement  paper  copies  of  case  studies  with 
digital  technology  and  believe  that  the  enriched  learning  experience  provided  by  such  an  approach  makes 
the  case  study  method  even  more  powerful.  Digital  technology  provides  students  with  a cognitive  tool  that 
encourages  the  profound  knowledge  necessary  to  attain  ironic  understanding. 

Many  instructors  of  foundations  of  education  courses  have  grappled  with  ways  to  overcome  the 
incompatible  goals  and  the  intellectual  muddle  of  the  course.  An  effective  way  to  resolve  the  paradox  is  to 
create  a course  that  seeks  a new  goal  using  new  intellectual  tools.  Vygotsky  (1978)  argued  that  people 
make  sense  of  the  world  using  mediating  intellectual  tools.  Intellectual  development,  then,  is  dependent  not 
only  on  psychological  stages,  like  those  described  by  Piaget,  or  on  the  knowledge  we  accumulate,  but  also 
relies  on  the  intellectual  tools  that  society  makes  available.  While  Vygotsky  focused  on  oral  language,  or 
sign  systems,  in  his  formulation,  other  intellectual  tools  are  available.  Using  these  tools  in  innovative  ways 
can  refashion  the  social  foundations  of  education  course. 

For  me,  the  way  out  of  the  teaching  dilemma  presented  with  a course  that  has  at  its  core 
incompatible  goals  is  to  radically  change  the  student  outcome  desired.  Guided  by  the  insights  of  Kieran 
Egan  (1997)  on  the  larger  purposes  of  schooling,  I seek  to  assist  students  in  acquiring  an  ironic 
understanding  of  schools  and  teaching.  Briefly,  irony,  as  we  know  it  today,  is  tied  to  a belief  that  the  truth 
long  sought  in  the  western  intellectual  tradition  is  illusionary.  In  place  of  certainty,  irony  provides  for  the 
centrality  of  reflexiveness  in  thinking.  Ironic  understanding,  then,  leads  to  the  removal  of  a commitment  to 


simple  truth  of  general  schemes.  The  sophisticated  ironist,  instead,  enjoys  extensive  ways  of  understanding 
and  recognizes  the  value  of  a varied  spectrum  of  perspectives. 

Fostering  multiple  perspectives  is  at  the  heart  of  the  articulate  ironist.  Gliding  from  perspective  to 
perspective  is  valued  and  becomes  second  nature  in  such  a person.  The  ironist  applies  the  varying 
perspectives  to  make  sense  of  their  experiences.  A teacher  might  ironically  note,  for  example,  that  a new 
school  policy  designed  to  improve  math  performance  in  fact  creates  a wider  gap  between  high  math 
achievers  and  low  achieving  math  students.  A better  identification  of  the  problem  and  a clearer  solution  is 
then  possible.  One  must  take  care,  ironically  enough,  to  make  sure  that  the  disciplinary  incoherence  of  the 
social  foundations  course  is  not  replaced  with  incoherence  arising  from  seeing  events  with  different 
perspectives.  The  value  of  a case  study  approach  to  teacher  education  is  that  while  cases  offer  multiple 
perspectives,  they  are  at  the  same  time  grounded  in  a specific  narrative  context.  General  theoretical 
principles  flow  from  an  array  of  possibilities  that  build  on  one  another.  Out  of  what  seems  to  be  chaos 
comes  not  only  insight  but  also  orderliness. 

Emerging  digital  technology  is  allowing  the  case  study  method  to  develop  into  an  even  more 
powerful  tool  by  permitting  cases  to  become  more  contextually  grounded.  More  specifically,  an  interactive 
course  web  site  that  supplements  the  text  of  a case  with  multimedia  presentations  and  links  to  other  data 
sources  allows  great  flexibility  in  student  construction  of  knowledge.  To  illustrate  this,  I shall  borrow  a 
metaphor  from  the  world  of  art  museums.  Elizabeth  Valance  (1995)  refers  to  a “public  curriculum” 
inherent  in  the  orderly  images  found  in  art  museums.  This  public  curriculum  consists  of  the  images 
displayed;  the  order  in  which  the  images  are  placed;  the  information  provided  for  visitors;  the  tours, 
lectures  and  workshops  and  other  programs  offered  to  the  public;  and  the  publications  produced.  The 
curriculum  is  randomly  accessed  by  visitors  and  is  readily  available  to  anyone  who  comes  to  the  museum. 
All  visitors,  however,  study”  the  curriculum  on  their  own  and  make  sense  of  the  images  in  endless  varieties 
of  ways.  While  visitors  may  understand  the  curriculum  in  a variety  of  ways,  the  curriculum  is  deliberate. 
Images  are  arranged  in  some  order  (usually  by  culture  chronology  but  such  need  not  be  case)  with  or 
without  helpful  labels.  Some  images  are  quite  accessible  to  all  but  others  are  difficult,  and  visitors  may 
even  view  an  image  with  hostility.  Visitors  will  frequently  confront  a museum  worker  with  some  sort  of 
refrain  along  the  lines  of,  “Is  that  really  art?” 

Similarly,  a well-designed  course  website  will  be  accessible  and  challenging.  Depending  on  how 
images,  text,  video  and  audio  are  arranged,  one  can  create  different  learning  environments  and  attempt  to 
frame  relevant  issues  in  key  ways.  Arranging  course  materials  using  digital  technology  into  generative 
topics  allows  for  students  to  respond  to  the  realities  of  classroom  practice  and  educational  policy-making  in 
multiple  ways.  What  cannot  be  controlled,  of  course,  is  what  students  learn.  The  goal,  instead,  is  to  provide 
as  many  avenues  as  possible  to  allow  students  to  make  their  own  connections  and  to  form  their  own 
categories.  Like  a well  run  art  museum  that  helps  novice  visitors  find  accessible  entry  points  into  the  world 
of  art,  an  interactive  website  can  assist  the  novice  teacher  into  gaining  access  to  the  profession  of  teaching. 
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Abstract  Recently,  attention  has  focused  on  developing  methods  for  facilitating  critical  thinking  in 
students.  Scaffolding  is  a method  that  has  been  successfully  used  in  many  settings  to  support  different 
learning  goals.  This  study  focused  on  the  influence  of  scaffolding  on  critical  thinking  skills  in  a 
technology-mediated  environment.  The  main  research  questions  explored  changes  in  participant  use 
of  scaffolding  and  influences  on  the  evolution  of  critical  thinking.  A qualitative  design  guided  data 
collection  and  analysis.  Five  graduate  participants  were  purposefully  selected  from  an  online 
instructional  design  class  and  interviewed  repeatedly  over  one  semester.  Major  findings  indicated  that 
participant  use  of  scaffolding  moved  from  externally  directed  to  internally  relevant  assimilation. 
Influences  on  the  evolution  of  critical  thinking  included  prior  knowledge,  reflection,  feedback,  project 
context,  and  perception  of  self  as  learner.  Implications  for  research  and  practice  are  outlined. 


Introduction 

This  paper  describes  a study  conducted  to  explore  the  influence  of  scaffolding  on  critical  thinking  skills  in  a 
technology -mediated  environment.  With  the  recent  increase  in  online  education  and  Internet-based  content,  the 
development  of  critical  thinking  skills  becomes  more  important,  along  with  the  development  of  methods  for  supporting 
critical  thinking.  Much  attention  has  been  focused  on  identifying  and  developing  methods  for  supporting  and  facilitating 
learning  in  non-traditional  environments.  Scaffolding  is  one  method  that  has  been  used  in  a variety  of  settings,  ranging 
from  one-on-one  tutoring  to  traditional  classrooms  for  supporting  very  different  learning  goals  (Bliss,  Askew,  & 

Macrae,  1996;  Graves  & et  al.,  1996;  Hogan  & Pressley,  1997;  Lepper,  Drake,  & O'Donnell -Johnson,  1997;  Palincsar, 
1986;  Saxe,  Gearhart,  & Guberman,  1984;  Wood,  Bruner,  & Ross,  1976). 

Scaffolding  was  first  defined  as  a process  by  which  an  expert  supports  a learner  in  the  accomplishment  of  a 
task  beyond  the  learner's  individual  capabilities,  and  then  gradually  fades  that  support  as  the  learner  becomes  more 
competent  in  task  accomplishment  (Wood  et  al.,  1976).  Scaffolding  has  been  used  effectively  in  classroom 
environments  for  supporting  the  learning  of  reading  (Palincsar,  1986),  language  (Roehler  & Cantlon,  1997), 
mathematics  (Schoenfeld,  1991),  and  scientific  inquiry  (Hogan  & Pressley,  1997).  In  these  environments,  teachers 
interact  with  groups  of  students,  providing  them  with  appropriate  support  through  modeling,  questioning,  or  cued 
reflection  and  extemalization  of  metacognition.  Studies  have  shown  that  through  such  support  and  scaffolding,  students 
gradually  internalize  cognitive  processes  and  assume  responsibility  for  and  control  of  their  own  learning  (Roehler  & 
Cantlon,  1997). 

Researchers  have  tried  to  broaden  the  availability  and  application  of  scaffolding  by  developing  technology- 
mediated  and  technology-based  scaffolding.  Despite  the  limitations  of  computer-based  intelligent  systems  in  emulating 
the  sensitivity  and  responsiveness  of  the  human  expert,  computers  were  able  to  mediate  and  emulate  certain  facets  of  the 
scaffolding  process  (Chee,  1995;  Guzdial,  1994;  Kao  & et  al.,  1996;  Soloway  et  al.,  1993).  In  most  cases,  these 
applications  functioned  as  on-demand  help  systems  and  provided  the  learner  with  procedural  and  directive  instructions 
for  task  completion. 

Recently,  a few  technology-based  scaffolding  applications  have  been  developed  that  address  the  internalization 
of  cognitive  strategies  and  higher  order  thinking  skills.  Reading  Partner  (Salomon,  Globerson,  & Guterman,  1989) 
focused  on  supporting  the  internalization  of  metacognitive  strategies  by  providing  intellectual  and  pedagogical 
scaffolding  for  seventh  graders'  reading  skills.  Despite  the  lack  of  fading,  the  continuous  introduction  of  metacognitive 
questions  and  hints  resulted  in  students'  internalization  of  strategies  and  improvements  in  a near  transfer  task— writing. 


The  Web-based  Inquiry  Science  Environment  (WISE)  (University  of  California  at  Berkeley,  1999),  another  prominent 
application,  focused  on  scaffolding  students’  scientific  inquiry  and  higher  order  thinking  skills  on  the  Internet.  With  the 
help  of  the  software  tools  and  scaffolding  in  the  form  of  questions,  students  evaluated  web  sites  and  asked  critical 
questions  aimed  at  evaluating  the  utility  of  the  site’s  content  for  a specific  project.  Empirical  evidence  indicated  that 
scaffolding  tools  positively  influenced  students’  critiques  and  their  ability  to  ask  critical  questions  (Slotta  & Linn, 

2000). 

Research  indicates  that  scaffolding  of  higher  order  thinking  skills  is  best  achieved  through  metacognitive  hints 
and  questions — directive  guidance  seems  to  engender  an  over-dependence  on  scaffolding  and  reduces  the  capability  of 
the  learner  to  function  independently  (see  for  example,  Kao  & Lehman,  1997).  Additionally,  scaffolding  must  be 
overtly  faded  to  encourage  and  identify  the  transfer  of  metacognitive  strategies  to  related  future  tasks.  Apart  from 
techniques  and  implementations  of  scaffolding,  internalization  of  cognitive  strategies  is  also  affected  by  context  and 
task  structure.  Research  indicates  that  students’  ability  to  assimilate  technology-based  scaffolding  is  increased  by 
providing  goals  and  orientation  (Oliver  & Hannafm,  2000;  Slotta  & Linn,  2000). 

Students’  epistemological  readiness  and  interests  mediate  both  the  use  of  scaffolding  (Oliver  & Hannafm, 
2000;  Sherman,  1994)  and  the  ability  to  think  reflectively  and  critically  (King  & Kitchener,  1994;  Kuhn,  1999).  For 
younger  students,  directive  and  instructive  scaffolding  may  prove  more  effective  because  of  students’  naive 
epistemologies  and  willingness  to  accept  authoritative  knowledge  (Kuhn,  1999).  For  slightly  older  students,  however, 
scaffolding  in  the  form  of  Socratic  and  open-ended  questions  might  prove  more  effective  due  to  students’ 
epistemological  readiness  to  question  and  seek  knowledge. 


Research  Questions  and  Design 

Based  on  reviewed  literature  and  studies  and  the  purpose  of  the  current  study,  three  major  research  questions 
were  addressed: 

• How  does  student  perception  and  use  of  scaffolding  evolve? 

• What  processes  and  sources  influence  the  development  of  critical  thinking  skills? 

A qualitative  research  design  was  used  to  meet  the  purpose  of  the  study  and  to  address  the  research  questions. 
The  study  was  conducted  in  the  context  of  a primarily  online  Instructional  Design  (ID)  offered  in  the  spring  semester 
2001  in  a large  Southeastern  university.  Within  this  online  environment,  scaffolding  supported  the  performance  of 
critical  thinking  tasks  in  ID.  Data  were  mainly  gathered  by  interviewing  five  purposefully  selected  graduate  students 
within  the  course.  Participants  were  selected  through  administration  of  the  Cornell  Critical  Thinking  Test  (CCTT)  Level 
Z test  (Fisher  & Scriven,  1 997)  and  their  willingness  and  ability  to  articulate  their  thinking  and  learning.  Participant 
generated  documents  were  used  to  cue  recall  during  the  interviews.  Interview  data  were  analyzed  using  the  constant 
comparison  method  (Glaser  & Strauss,  1 967). 

During  the  course,  the  students  used  a textbook  as  a primary  source  of  content  and  instruction.  In  addition, 
systematic,  deliberate,  and  static  scaffolding  was  provided  via  course  materials  at  five  discrete  intervals  to  coincide  with 
the  completion  of  specific  class-based  tasks.  The  scaffolding,  designed  collaboratively  by  the  researcher  and  course 
instructor,  was  designed  to  facilitate  deeper  reflection,  metacognition,  and  critical  thinking  for  the  execution  of  six 
class-related  documents.  Decisions  about  level  and  type  of  scaffolding  were  negotiated  throughout  the  course,  and 
scaffolding  was  gradually  faded. 

Students  accessed  the  static  scaffolding  directly  through  the  web  page  that  described  the  assignment.  Thus,  the 
task  was  presented  concurrently  with  the  appropriate  mode  of  scaffolding — Socratic  questioning,  modeling,  or 
extemalization  of  metacognition  and  reflection.  The  scaffolding  provided  for  critical  thinking  in  relation  to  Instructional 
Design  was  based  on  relevant  literature  in  ID  (Briggs,  1977;  Dick  & Carey,  1985;  Smith  & Ragan,  1993),  critical 
thinking  (Beyer,  1997;  Ennis,  1987;  Paul,  1990),  and  scaffolding  (Hogan  & Pressley,  1997;  Rogoff  & Wertsch,  1984; 
Vygotsky,  1978;  Wertsch,  1984;  Wood  & Wood,  1996). 

Data  were  gathered  through  two  main  sources:  interviews  and  participant  document  artifacts,  which  included 
six  assignments  that  were  directly  related  to  participants’  individual  instructional  projects,  and  four  concept  maps 
generated  at  various  stages  by  each  participant.  Participants  were  interviewed  five  times  during  the  semester,  at  an 
interval  of  2-3  weeks  and  during  interviews,  the  concept  maps  and  document  assignments  were  used  as  visual  triggers 
for  participants  to  reflect  on  their  meaning  making  process.  Some  participants  also  generated  bi-weekly  journals  that 
detailed  their  thinking  and  experiences  in  ID.  Also,  participants  posted  their  reflections  about  changes  in  their  individual 
views  of  ID  on  the  class  bulletin  board  at  the  middle  and  end  of  the  semester.  Transcripts  of  the  reflective  bulletin  board 
activities  were  used  as  additional  document  data,  in  conjunction  with  the  reflective  journals  generated  by  some 
participants.  The  multiple  sources  of  data  also  provided  a source  of  triangulation  and  validity. 
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Major  Findings 

Use  of  Scaffolding 


Initially,  some  participants  perceived  the  scaffolding  as  directive  and  used  the  questions  and  examples  as 
directions  for  task  completion.  For  example,  one  participant  said,  “...it  kinda  coached  me  through  the  activity  ...  and  I 
thought  about  what  it  was  she  [the  instructor]  really  wanted  you  to  put  in  there. “ A similar  attitude  was  adopted  by 
another  participant  who  tried  to  “parrot  the  examples”  in  the  beginning  of  the  semester.  Towards  the  middle  of  the 
semester,  as  participants  became  more  involved  in  their  instructional  project,  most  began  to  perceive  scaffolding  as  a 
guide  to  help  them  achieve  their  own  special  needs.  Participants  began  to  speak  about  the  advantages  of  having  a 
“framework  ...  to  help  you  pinpoint  where  you  need  to  focus  on”  and  “prompts  that  [thinking]  and  that  definitely 
helps.”  In  most  cases,  the  change  in  participants’  perception  and  use  of  scaffolding  was  an  acknowledgement  of  the 
their  individual  needs  and  contexts.  One  participant  admitted  that  she  began  to  view  the  scaffolding  in  terms  of  its 
applicability  to  her  project,  and  by  extension,  her  life:  “I’m  beginning  to  say  you  know,  how  would  this  fit  into  my  real 
life  and  what  should  I be  concerned  about  getting  from  it  and  whatever.” 

Use  and  interpretation  of  scaffolding  moved  through  three  distinct  phases — in  the  first,  scaffolding  was 
considered  as  directive  instructions.  In  the  second  phase,  scaffolding  was  used  as  a guide  within  participants’  individual 
context.  Scaffolding  in  the  third  phase  was  marked  by  interpretation  and  selection  to  suit  individual  needs.  In  the 
externally  dictated  mode,  participants  chose  to  implement  the  provided  scaffolding  literally  and  “answer  the  questions” 
or  react  in  accordance  to  their  perceptions  of  what  the  “instructor  wanted.”  However,  as  participants  became 
increasingly  conscious  of  their  individual  contexts  and  goals,  they  began  to  perceive  the  scaffolding  as  a guide  that 
allowed  them  to  direct  their  thinking  along  appropriate  and  personally  meaningful  direction.  Some  participants  used 
scaffolding  in  a customized  manner  from  the  very  beginning  and  indicated  that  the  customization  was  guided  by 
relevance  to  their  project  goal.  The  other  participants  also  indicated  that  they  eventually  changed  their  perception  and 
use  of  scaffolding  based  on  their  individual  project  goals  and  needs.  The  findings  of  the  study  indicate  that  it  is 
important  to  identify  methods  for  alerting  students  to  the  function  of  scaffolding — that  is,  to  assist  them  in  realizing 
their  individual  goals.  In  the  current  study,  goal  realization  was  an  individual  process  occurring  at  different  times.  The 
importance  of  continuously  defining  and  refining  the  learning  goal  during  the  scaffolding  interaction  is  a key 
component  of  Wertsch’s(1984)  explication  of  scaffolding  processes.  Goal  clarification  has  been  identified  as  a 
keystone  for  success  in  the  assimilation  and  use  of  scaffolding  by  students  of  different  ages  and  in  a variety  of  settings 
(see  for  example,  Rogoff  & Wertsch,  1984;  Saxe  et  al.,  1984;  Slotta  & Linn,  2000). 


Influences  on  Critical  Thinking 

Five  elements  seemed  to  influence  the  evolution  of  critical  thinking:  reflection,  feedback,  problem  context, 
perception  of  self  as  learner,  and  prior  knowledge.  Reflection  emerged  as  a very  important  influence  in  the  development 
of  critical  thinking.  One  participant  often  mentioned  the  value  of  reflection  in  changing  her  thinking,  specifically  with 
regard  to  the  assignments  and  the  necessity  of  looking  beyond  “predefined  notions  of  how  to  do”  things.  Other 
participants  acknowledged  the  novel  role  and  the  value  of  reflection  in  their  activities  as  exemplified  in  the  quote  by  this 
participant:  “Basically  I would  say  that  reflecting  is  a new  element  that’s  coming  into  play  and  looking  at  details  is  not 
something  I’ve  had  to  consciously  do  ...and  it’s  not  something  that  comes  naturally  to  me.”  Authors  such  as  Kuhn 
(1 999)  emphasize  that  fostering  metacognitive  and  reflective  skills  is  of  utmost  importance  in  promoting  critical 
thinking.  Brookfield  (1986)  identifies  reflection  as  one  of  the  most  “useful”  tasks  in  helping  people  think  critically. 
Consequently,  those  participants  who  engaged  in  reflection  often  and  who  valued  reflection  as  a tool  to  promote  their 
thinking  were  most  likely  to  engage  in  critical  thinking. 

Seeking  and  reacting  to  feedback  was  another  influence  on  shaping  participants’  critical  thinking  in  ID. 
Participants  often  mentioned  their  higher  regard  for  seeking  feedback,  or  as  one  participant  said:  “...maybe  I’m  just 
more  comfortable  hearing  the  feedback.”  Feedback  was  rated  highly  by  participants  as  a means  to  gather  different 
viewpoints  and  as  a way  of  becoming  more  “open  to  just  different  ideas  or  different  approaches.”  McPeck  (McPeck, 
1990)cites  feedback  or  “rational  disagreement”  (p.  52)  as  one  of  the  primary  methods  for  stimulating  the  examination  of 
ideas.  McPeck  also  recommends  that  teachers  who  attempt  to  facilitate  critical  thinking  in  their  students  must  learn  to 
move  their  teaching  from  more  “didactic”  to  “discursive”  modes  of  presentation.  This  approach  is  consistent  with  the 
findings  of  the  study,  which  indicate  that  feedback  served  to  refine  participants’  understanding. 

For  many  participants,  the  exigencies  of  their  individual  project  context  appeared  to  influence  the  development 
and  evolution  of  critical  thinking.  The  more  authentic  and  unstructured  the  problem  context,  the  more  likely  that 


individuals  engaged  in  critical  thinking.  The  two  participants  who  addressed  authentic,  ill-structured  problems  for  their 
instructional  project  acknowledged  the  importance  of  the  integral  nature  of  the  project  in  their  lives,  and  thereby  the 
additional  meaning  and  reflection  gained  from  the  experience.  Research  and  theory  offers  two  perspectives  on  these 
differences.  King,  Wood  and  Mines’  (1990)  research  suggests  that  reflective,  critical  thinking  is  most  often  triggered  by 
ill-structured  problems  rather  than  “well-structured  problems”  (Churchman,  1971).  Two  other  participants  who  engaged 
in  relatively  well -structured  projects  that  involved  very  little  ambiguity  indicated  fewer  instances  of  critical  thinking. 

Participants’  perceptions  of  themselves  as  learners,  as  well  as  their  perceptions  of  their  role  as  learners 
significantly  influenced  changes  in  their  thinking.  For  example,  one  of  the  older  participants  indicated  that  she  preferred 
to  “mimic”  the  processes  being  presented  and  she  consistently  referred  to  her  novice  status  as  a hindrance  for  critical 
thinking.  Another  participant  who  characterized  herself  as  a “creative”  and  “alternative”  thinker,  said  that  the  analytical 
nature  of  thinking  required  within  the  course  was  not  to  her  liking  or  style.  These  self-imposed  conceptions  of  self  as 
learner  hindered  critical  thinking  in  the  instance  of  these  two  participants.  Brookfield  (1986)  suggests  that  adults’ 
understanding  of  their  learning  styles,  selves  as  learners,  and  patterns  of  learning  is  an  important  prerequisite  for  critical 
thinking.  Learning  styles  are  defined  as  “the  characteristic  and  preferred  way  in  which  an  adult  engages  in  learning 
activities”  (Knox,  1986,  p.  20).  Reflecting  consciously  on  and  understanding  one’s  learning  has  two  implications  for 
critical  thinking.  First,  an  awareness  of  one’s  learning  style  allows  an  individual  to  select  appropriate  strategies  for 
effective  learning  (Brookfield,  1986).  In  addition,  such  understanding  can  promote  appropriate  adaptation  of  learning 
styles  to  suit  changed  circumstances.  For  example,  some  of  the  other  participants,  while  acknowledging  distinct 
learning  styles,  mentioned  that  they  tried  to  integrate  their  styles  with  the  class  requirements,  and  indicated  higher  levels 
of  critical  thinking. 

The  role  of  prior  knowledge  in  shaping  participants’  critical  thinking  in  ID  was  apparent  in  a number  of 
instances.  In  the  case  of  three  participants,  prior  knowledge  served  as  a basis  from  which  to  compare  their  current 
learning  and  experiences  in  the  class.  These  three  participants  incorporated  their  prior  knowledge  and  experiences  into 
their  understanding  of  ID  almost  from  the  very  beginning.  For  one  participant,  prior  knowledge  served  as  a source  of 
data  for  guiding  and  refining  his  thinking;  for  another,  prior  knowledge  stimulated  an  acknowledgement  of  existing 
biases  and  the  need  to  make  a “conscious  effort  to  displace  yourself.”  In  these  instances,  prior  knowledge  helped  to 
activate  a comparison  of  prior  and  current  experiences  and  to  stimulate  additional  thinking  and  reflection  when  an 
inconsistency  was  encountered.  This  finding  is  consistent  with  research  on  the  role  of  prior  knowledge  in  triggering 
additional  reflection  and  the  search  for  additional  information  (see  for  example.  Baker,  1979;  Pitts,  1994).  In  other 
instances,  especially  in  the  case  of  the  youngest  participant,  prior  knowledge  conferred  an  authority  to  her  thinking  that 
impeded  critical  thinking  and  she  said  in  explanation:  “I  know  every  aspect  of  it....”  This  attitude  is  consistent  with 
research  indicating  that  prior  knowledge  can  give  rise  to  counterproductive  strategies  such  as  assuming  the  validity  of 
default  knowledge  (Phillips,  1992)  or  preserving  information  that  supports  one’s  beliefs  (Klayman  & Ha,  1987) 

Relevance  and  Significance 

This  study  has  implications  for  different  groups.  First,  practitioners  and  educators  interested  in  developing 
critical  thinking  skills  in  their  students  might  benefit  from  the  results  of  this  study  in  being  able  to  identify  how  and  why 
students  choose  to  assimilate  and  use  critical  thinking  skills  when  adequately  supported  by  scaffolding.  Second,  using 
the  strategies  recommended  in  this  study  as  a launching  point,  designers  and  developers  of  online  or  technology- 
mediated  instruction  might  adapt  and  introduce  more  effective  strategies  for  implementing  scaffolding  for  critical 
thinking  in  such  environments.  Specifically,  this  study  provides  a starting  point  for  those  interested  in  designing  and 
implementing  scaffolding  for  critical  thinking  related  to  Instructional  Design,  and  encouraging  further  research  and 
design  based  on  the  results  of  this  study. 

Apart  from  practical  implications,  this  study  contributes  some  interesting  theoretical  perspectives  on 
scaffolding  and  critical  thinking.  In  literature  on  scaffolding,  attention  has  been  focused  in  two  main  areas:  (1) 
identifying  characteristics  of  an  expert  “scaffold”  (see  for  example,  Lepper  etal.,  1997;  Saxe  et  al.,  1984;  Wood  et  al., 
1976),  and  (2)  identifying  outcomes  of  the  scaffolding  interaction  between  expert  and  novice  (see  for  example,  Hogan 
& Pressley,  1997;  Kao  & Lehman,  1997;  Salomon  et  al.,  1989;  Slotta  & Linn,  2000).  This  study  contributes  to  a missing 
perspective — the  role  of  student  characteristics  in  assimilating  and  using  scaffolding.  By  identifying  a few  of  the 
cognitive  characteristics  and  beliefs  that  affect  student  use  of  scaffolding,  this  study  offers  a starting  point  for  other 
researchers  who  are  interested  in  further  exploring  the  interaction  of  student  characteristics  and  scaffolding. 

A second  area  of  theoretical  contribution  is  offered  in  this  study’s  description  of  the  influences  on  development 
of  critical  thinking.  Although  students  may  be  eminently  capable  of  thinking  critically,  a variety  of  influences  preclude 
the  development  and  demonstration  of  critical  thinking  skills.  The  documentation  and  description  offered  in  this  study 
also  offer  a starting  point  for  exploring  and  deriving  theoretical  frameworks  that  define  the  impact  of  student  and 
context  characteristics  on  the  development  of  critical  thinking. 
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Abstract:  This  work  in  progress,  called  Icicle,  is  a computing  environment  for  children  that  is 
easy  to  learn  and  use.  It  is  based  on  two  principles:  use  interaction  and  feedback  to  elicit 
program  descriptions  from  the  users  and  maintain  a close  mapping  between  user  actions, 
program  commands  and  program  representations.  Children  construct  performers  for  tlie  Icicle 
world  and  then  train  the  performers  to  behave  in  particular  ways.  The  training  takes  the  form 
of  a dialogue  between  the  children  and  the  system  and  uses  direct-manipulation  and 
programming-by-demonstration  techniques  to  encode  programs  as  animations. 


The  Icicle  system  came  about  following  work  on  a programming-by-demonstration  (Cypher  1993)  turtle- 
graphics  environment  called  ICE  (Sheehan  1999).  ICE  was  a successful  experiment  in  that  it  was  easy  to 
produce  programs  within  the  environment  that  generated  pictures,  many  of  them  with  sophisticated  effects. 
However  children  want  to  make  games  (Sheehan  2001),  not  just  fancy  pictures,  and  there  were  some  things 
missing  from  ICE  if  it  was  going  to  be  used  to  produce  games  and  other  forms  of  interactive  behaviour. 

Programming  by  Interaction 

The  ability  to  cope  with  interactions  was  missing  from  ICE.  The  only  object  to  control  was  the  turtle.  In  Icicle, 
interactions  are  at  the  heart  of  the  system;  this  includes  interactions  between  a child  developer  and  the  system, 
between  different  objects  in  an  Icicle  program  and  between  a child  player  and  an  Icicle  program. 

The  first  thing  children  have  to  do  when  presented  with  a blank  world  in  Icicle  is  to  make  a performer. 
Performers  are  the  object  actors  in  the  Icicle  world.  They  have  shapes,  behaviours  and  state  information. 
Programs  in  Icicle  are  represented  by  the  rules  associated  with  performers. 

When  designing  a performer*s  appearance  children  can  initially  choose  from  three  differently  coloured  simple 
shapes:  an  octagon,  a square  or  a triangle.  Rather  than  relying  on  professional  clip-art,  children  must  construct 
their  own  shapes  using  a simple  drawing  environment  by  modifying  these  initial  shapes.  Each  performer  can 
have  as  many  different  shapes  as  the  children  want.  As  Icicle  programs  run,  performers  can  morph  from  one 
shape  to  another.  This  provides  a remarkably  flexible  system  for  smoothly  animating  behaviour. 

Once  a performer  has  an  appearance  it  can  be  placed  in  the  Icicle  world.  A drag  and  drop  of  a shape  from  a 
performer  window  places  a new  instance  of  that  performer  in  the  world.  The  system  immediately  checks  to  see 
if  there  is  a default  behaviour  for  this  new  performer  and  if  there  isn’t  it  asks  the  child,  “What  should  this 
performer  normally  do?”. 

The  most  common  behaviours  for  performers  can  be  demonstrated  using  direct-manipulation.  In  particular  the 
user  can  drag  the  performer  around  the  screen.  This  results  in  turn  and  move  behaviours  being  learnt.  Similarly 
a different  shape  can  be  dragged  onto  the  performer  in  the  world  - this  action  is  interpreted  as  requesting  a 
morph  from  the  present  shape  to  the  new  one.  Handles  on  a performer  can  also  be  dragged  indicating  a zoom 
instruction. 

When  constructing  a rule  the  user  can  undo  any  mistakes  and  demonstrate  new  instructions.  As  well  as 
removing  instructions  from  the  rule,  the  undo  command  restores  the  performer  to  the  state  it  was  in  before  it 
was  moved  or  morphed.  When  the  user  is  satisfied  with  a sequence  of  commands  he  or  she  closes  the  dialog 
box.  Turning  the  world  on  makes  the  performer  repeatedly  execute  its  default  behaviour.  If  the  user  drags  more 
copies  of  the  performer  on  to  the  world  they  will  automatically  begin  executing  this  default  behaviour. 

When  a new  interaction,  such  as  a collision  between  performers  occurs,  and  there  is  no  existing  rule 
corresponding  to  this  situation  the  system  once  again  asks  the  child  to  demonstrate  the  required  behaviour.  The 
user  is  asked  to  provide  instructions  for  each  performer  involved  in  the  interaction. 

Users  can  be  the  source  of  interactions  as  well.  If  a key  is  pressed  when  the  world  is  running  the  system  checks 
to  see  if  any  performers  have  a rule  corresponding  to  the  key  press.  If  so  the  appropriate  instructions  are 
executed,  if  not  the  user  is  asked  to  choose  a performer  and  define  a rule  for  the  key  press.  Thus  users  can 
influence  the  behaviour  of  performers  as  the  world  runs,  obviously  necessary  when  implementing  games. 


Representation  of  the  Rules 


One  of  the  problems  in  programming-by-demonstration  systems  is  providing  a listing  of  the  generated  program 
that  is  useful  to  the  user  (Myers  & McDaniel  2001).  Supplying  the  listing  of  a conventional  textual  program  is 
not  acceptable  for  novice  users,  in  this  case  children.  The  connection  between  a textual  program  listing  and  the 
actions  of  objects  on  the  screen  is  not  clear  enough.  There  is  a gap  in  the  novice  user’s  understanding  between 
the  textual  representation  and  the  effect  produced. 

The  Icicle  system  minimizes  this  gap  by  using  animations  of  the  action’s  results  as  the  program  listing.  In  terms 
of  the  cognitive  dimensions  (Green  & Petre  1996)  there  is  a true  closeness  of  mapping  between  the  program 
code  and  the  desired  behaviour  - they  are  virtually  identical.  The  conditions  which  cause  the  rules  to  activate  are 
also  animated,  e.g.  two  performers  colliding.  Even  for  children  unable  to  read  the  descriptions  beneath  each 
animation,  it  is  clear  what  is  being  represented  here. 

In  order  to  increase  the  perceived  connection  between  the  program  code  and  the  results  of  that  code,  changes  to 
the  code  are  reflected  immediately  in  all  the  performers  active  in  the  world.  Similarly  changes  to  the  shapes  and 
colours  of  the  performers  in  the  drawing  editor  are  shown  simultaneously.  The  world  can  keep  running  while 
new  performers  are  created  and  added  to  it  or  while  old  performers  are  modified.  This  produces  a sense  of 
liveliness  that  hopefully  makes  the  performers  appear  more  like  active  agents  in  the  world  (Travers  1996). 

The  fact  that  performers  can  only  change  their  shapes  by  morphing  is  also  an  attempt  to  convey  a sense  of 
concreteness  about  each  performer.  A performer’s  shape  is  part  of  what  the  performer  is  and  even  though  it  can 
change  shape  it  does  it  gradually  in  a way  that  preserves  its  continuity.  When  a child  single  steps  the  world 
forwards  or  backwards,  it  is  easy  to  see  the  gradual  change  from  one  shape  to  another. 

Parallel  Processing 

All  performers  execute  their  rules  in  parallel.  When  an  interaction  occurs  the  performers  involved  in  the 
interaction  change  to  run  the  interaction  code.  After  the  interaction  is  dealt  with  the  performers  return  to  their 
default  behaviours.  This  parallel  processing  is  not  the  only  parallelism  the  system  provides. 

When  a rule  is  made  that  comprises  several  instructions,  such  as  a move,  a turn  and  a morph,  it  is  possible  to 
make  two  or  more  of  the  instructions  occur  in  parallel.  This  is  useful  if  the  user  wants  the  performer  to  move 
along  a curved  path  or  to  change  shape  as  it  moves. 

Changing  from  sequential  to  parallel  is  simple;  the  user  drags  one  instruction  on  top  of  another.  The  rule 
animations  merge  to  show  the  result  of  the  parallel  instruction  e.g.  moving  and  changing  shape  simultaneously. 
Preliminary  testing  shows  that  this  is  easy  for  children  to  use,  how  much  they  comprehend  is  as  yet  unknown. 

Similar  Work 

Different  aspects  of  Icicle  can  be  seen  in  a variety  of  children’s  programming  environments  or  programming- 
by-demonstration  systems.  Klik  <Sc  Play  was  a commercial  product  which  turned  into  The  Games  Factory  and 
uses  interactions  to  ask  for  rules.  Stagecast  Creator  (Smith  & Cypher  1999)  and  Agentsheets  (Repenning  1993) 
maintain  rules  which  are  fired  when  preconditions  are  met. 
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Abstract:  The  majority  of  today’s  college  students  enter  their  first  computer  science  course 
with  a working  knowledge  of  computers,  primarily  PCAVindows  based.  Their  experience 
with  PC  systems  as  their  “native  language”  is  then  reinforced  in  the  many  universities  that 
require  laptop  PCs  or  distribute  them  to  incoming  freshmen.  Teaching  students  the  Unix 
environment  thus  has  become  more  difficult  as  students  cling  to  the  operating  system  that  they 
have  grown  comfortable  with.  This  paper  describes  a tutorial  program  intended  to  move 
students  from  PCAVindows  to  Unix.  The  self-paced  tutorial  lessons  are  written  in 
Macromedia  Director  and  can  be  accessed  from  the  students’  personal  computers  on  the  Web. 
The  lessons  begin  with  basic  Unix  commands  executed  through  a simulated  terminal  window, 
provide  remote  access  to  an  actual  Unix  terminal  window  for  “real”  feedback,  direct  students 
to  a Unix  lab  at  our  university,  proceed  through  more  advanced  Unix  utilities,  and  offer 
feedback  through  graded  quizzes.* 


The  Problem 


The  growing  number  of  laptop-equipped  students  in  today’s  universities  is  a mixed  blessing  for  the 
computer  science  discipline.  Students  entering  our  courses  believe  themselves  to  be  quite  computer-sawy,  but 
most  have  been  brought  up  on  PCsAVindows  and  have  a natural  reluctance  to  move  out  of  the  environment  that 
has  grown  so  familiar  to  them.  Their  portable  computers  also  spoil  them  on  working  in  their  dorms  and 
apartments,  and  Unix  labs  that  used  to  bustle  with  student  activity  are  now  mostly  quiet.  Most  academic 
computer  scientists,  however,  continue  to  see  the  value  in  teaching  students  the  Unix  operating  system,  and  for 
many  good  reasons  - its  robustness;  its  groundbreaking  position  as  a system  for  multitasking,  networked,  and 
heterogeneous  computing  environments;  and  the  source-code  transparency  that  lends  itself  to  learning  operating 
system  concepts  and  implementation.  A second  problem  in  introducing  students  to  the  Unix  environment  is  that 
user-level  skills  don’t  seem  worthy  of  classroom  time,  yet  we  want  our  students  to  come  out  of  our  courses  with 
a certain  level  of  proficiency  in  using  the  operating  system.  We  need  a way  to  ensure  minimum  competency  in 
Unix  lab  skills  without  sacrificing  classroom  instruction  time  to  recite  simple  Unix  commands  that  students 
could  learn  on  their  own.  Thus,  the  purpose  of  our  tutorial  system  is  to  wean  PC-devoted  students  to  the  Unix 
environment  by  letting  them  get  their  first  glimpse  of  it  through  their  own  personal  computers,  and  to  do  this  in 
a self-paced  tutorial  system  with  student  accountability  via  quizzes  and  a database  that  records  student  activity. 

The  Tutorial 

Our  tutorial  system  is  accessible  from  the  Web  at  http  ://www.  cs . wfu . edu/--bur  g/ 
UnixTutorials/Tutorial.htm.  We  have  so  far  implemented  and  tested  six  tutorial  lessons:  Lesson  1 . Unix  and 
the  Unix  Lab;  Lesson  2.  Getting  Started;  Lesson  3.  Basic  Unix  Commands  ; Lesson  4.  Creating  Text  Files; 
Lesson  5.  Using  Unix  from  Your  PC;  and  Lesson  6.  Compiling  and  Running  C++  Programs 

To  clarify  the  difference  between  working  directly  on  Unix-based  computers  and  remotely  from  PCs, 
the  first  tutorial  gives  the  students  a visual  tour  of  the  Unix  lab  at  their  university.  (This  is  the  only  lesson 
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specific  to  our  university  and  its  labs.)  Students  learn  where  the  lab  is  located,  how  to  get  access,  and  how  to 
get  an  account  on  our  system.  This  lesson  also  touches  on  what  an  operating  system  is  and  how  Unix  relates  to 
Windows  in  the  computing  world,  along  with  a quick  explanation  of  the  hardware  and  software  that  we  have  for 
them  to  use. 


[i  Lesson  3:  Basic  Unbi  Commands 


Command:  mv 

Use:  Renames  a fils  or  moves  It  to  a now 
location. 

Try  It  bslowl  Typs 
mv  otdname  newnams 
to  rename  a file.  Replace  oWhamo  with  the 
name  of  the  file  you  ere  renaming,  and  replace 
newnamo  with  Itf  e new  name. 


yi^dDW  Edit  Options 

|bobbige|  'aumwun:  mUIr  IcssonJ 
(bobbigej  'oumwvn:  Is 
ttssen3/ 

{bobbogc)  'oumwun:  cd  IsssenJ 

(bobbogej  "sumwun/lcsssnJ:  cp  fhomc/lcmpffllc.bct  new.M 

[babbagej  '’smnwun/lcsssn3:  la 

new.M 

(babbage)  'smnwun/lcason3: 


Lesson  2.  »<e  »wnt  over  tne  mv 
cominano.  LWs  review  it. 


Howwouio  you  rename  me  file 
new.M  to  tmic  ta? 


Oivo  It  o tryl 


Figure  1.  Basic  Unix  Commands 


The  second  and  third  lessons,  covering  basic  Unix  commands,  are  presented  by  means  of  a simulated  Unix 
terminal  window.  The  students  are  given  practice  with  simple  Unix  commands  in  a controlled  environment 
where  errors  can  be  checked  and  feedback  can  direct  their  learning.  A character  called  “Floppy”  (reminiscent 
of  “Clippy©”  from  Microsoft  Office)  guides  the  students  through  the  lessons.  Lesson  4 discusses  the  use  of  text 
editors  in  a Unix  terminal  window.  We  chose  to  use  the  vi  text  editor.  While  this  may  not  be  the  easiest  to 
learn,  it  has  the  advantage  of  not  requiring  X-Windows.  The  students  are  guided  step-by-step  throughout  the 
process  of  using  the  editor.  In  Lesson  5,  we  begin  to  wean  the  students  from  their  PCs  by  showing  them  how  to 
get  remote  access  to  an  actual  Unix  terminal  window.  They  are  given  directions  on  downloading  and  installing 
a remote  login  program,  which  then  opens  a terminal  window  alongside  the  tutorial.  At  the  end  of  Lesson  5, 
students  are  given  a series  of  exercises  to  try  in  the  Unix  terminal  window,  some  of  which  will  have  them 
investigate  new  commands,  command  options,  and  error  conditions  more  closely.  For  example,  they  are  asked 
to  remove  a directory  that  still  contains  files,  or  they  are  asked  to  find  an  option  for  the  Is  command  that  lists 
even  “hidden  files.”  Lesson  6 introduces  the  use  of  compilers.  C/C-H-  is  chosen  as  their  introductory  language 
since  it  is  the  language  most  closely  related  to  the  Unix  operating  system. 


WAKEFOl 


Quiz  3;  Basic  Unix  Contmands 


bastion  6 

What  is  the  dlferencs  bstwean  rm  end  rmdirT 

OA.  one  Changes  directories,  the  other 
makes  new  ones 

OB.  one  deletes  files,  the  other  deletes 
directories 

Oc.  one  lists  flies,  the  other  ilsts  directories 

O D.  one  logs  you  In,  the  other  logs  you  out 
Current  Answer 

[ Prevtom  QuesOon  [ rioi  Question  "1 


You  have  now  reached  the  end 
of  the  quiz.  TochecKyour 
answers  or  to  make  correctiorts. 
please  use  the  Previous 
Question  button  to  go  back 
through  the  quiz.  Once  you  are 
satisfied  with  your  answers,  click 
Submit  Quiz  here. 


Click  above  to  continue 


Figure  2.  Quiz  3 


Each  lesson  ends  in  a 10-item  quiz.  At  the  end  of  the  quiz,  the  students  can,  if  they  choose,  step 
through  the  items  they  got  wrong  and  see  the  correct  answers.  The  quiz  scores  are  written  back  to  a database 
for  the  professor’s  reference. 
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Project  TALENT:  Creating  Change  in  Teaching  Activities 
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Abstract:  TALENT  (Teaching  And  leading  for  Educational  Needs  with  Technology)  is  a 2000 
PT3  Implementation  grant.  Besides  faculty  and  K-12  teacher  training  through  focused  workshops 
and  weeklong  institutes,  TALENT  has  taken  an  innovative  approach  in  creating  'triads*  consisting 
of  a preservice  teacher,  a K-12  master  teacher,  and  a university  supervisor,  who  function  as  a 
learning  community.  This  learning  community  spans  two  parallel  activity  systems:  the  university 
and  the  school  in  which  the  preservice  teacher  is  placed.  The  project  is  discussed  using  the 
framework  of  Activity  Theory,  which  can  clarify  activities  and  add  predictive  validity  to 
talent’s  grant  efforts. 


Background 

TALENT  (Teaching  And  Leading  for  Educational  Needs  with  Technology)  is  a 2000  Preparing  Tomorrow’s  Teachers  to  Use 
Technology  (PT3)  Implementation  grant  funded  by  the  U.S.  Department  of  Education.  Co-directed  by  Bohlin,  Chiero,  and  Harris, 
talent’s  purpose  is  to  infuse  instructional  technology  into  the  teacher  preparation  program  at  California  State  University, 
Fresno  (CSUF).  The  project’s  overarching  aim  is  for  teacher- candidates  to  complete  their  credential  programs  with  the 
knowledge  and  understanding  to  deal  with  the  digital  needs  of  diverse,  limited  English-speaking,  high  poverty,  and  rural  students 
in  the  K-12  schools  in  which  they  intend  to  teach. 

In  order  to  empower  all  educators  in  the  teacher  preparation  program  - including  university  faculty  both  in  Education  and  in 
other  departments  where  teacher  candidates  take  their  undergraduate  and  subject  area  courses,  K-12  master  teachers,  and 
university  supervisors  of  student  teachers  - TALENT  designed,  developed,  and  delivered  weeklong  institutes  and  full  day 
workshops.  Topics  were  tailored  to  the  participants’  particular  knowledge,  skill,  and  interest  level,  and  to  their  expressed  needs. 
Typical  tools  taught  ranged  from  PowerPoint,  Web  research,  and  Inspiration  to  digital  video  and  virtual  reality.  K-12  teachers 
were  trained  along  with  university  faculty  in  the  institutes,  and  with  university  supervisors  in  the  workshops.  Thus,  there  was 
cross-fertilization  of  ideas  among  these  three  populations  in  the  training  programs  (Fig.  1). 


Figure  1:  Project  TALENT  Training  Activities 

In  the  next  phase  of  its  activities,  TALENT  began  to  forge  links  between  the  university  supervisors,  the  K-12  master  teachers, 
and  the  teacher  candidates  using  a “triad”  learning  community  model,  in  which  each  participant  could  share  strategies  and  ideas 
about  teaching  with  technology  with  the  other  participants.  The  triad  structure  already  existed,  but  the  concept  of  using  the 
structure  as  the  basis  of  a learning  community  focused  on  technology  infusion  was  unique  to  TALENT.  In  the  TALENT  model,  a 
learning  community  could  also  consist  of  a larger  team  of  master  teachers,  supervisors,  and  teacher  candidates  at  a K-12  school. 
The  purpose  of  these  learning  communities  would  be  to  increase  comfort,  confidence,  and  competence  in  technology  infusion 
among  all  members  and  to  begin  to  develop  technology  leadership  abilities  among  the  newly  credentialed  teachers. 


TALENT  as  an  Activity  System 

In  both  of  these  sets  of  professional  development  activities,  the  project  directors  envision  learning  as  social  construction  of 
knowledge  within  a community  of  practice.  Learning  is  situated  in  authentic  activities,  and  there  is  plenty  of  time  for  group 
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discussion  and  reflection  to  apply  newly  acquired  knowledge  and  skills  to  both  the  university  and  the  K-12  classroom.  Moreover, 
the  presenters  explicitly  model  student-centered  learning  and  student  creation  of  projects. 

There  is  a rich  base  of  social  learning  theory,  beginning  with  Vygotsky  (1978)  in  the  Soviet  Union.  For  example.  Lave  (1991) 
envisioned  knowing,  learning,  and  cognition  as  social  constructions,  expressed  in  actions  of  people  interacting  within 
communities  of  practice.  Koschmann  (1996)  coined  the  term  “computer-supported  collaborative  learning”  to  describe  learning 
communities  that  engaged  in  authentic,  knowledge-building  tasks  supported  by  electronic  interactions.  Tavalin  and  Gibson 
(Sherry,  Billig,  Tavalin,  & Gibson,  2000a,  2000b;  Sherry  & Tavalin,  2000)  facilitated  the  use  of  online  sharing  of  digital  art  and 
music  files  within  a community  of  experts,  had  students  post  their  products  to  a public  audience,  and  then  had  the  students 
selectively  filter  the  ensuing  feedback  for  the  purpose  of  revising  and  refining  their  work.  Carroll  (2001)  extended  this  notion  to 
the  entire  PT3  grant  initiative,  in  which  electronic  interactions  among  teachers  and  learners  help  to  foster  online  communities  of 
learners.  He  considered  schools  — including  schools  of  education  as  well  as  K-12  schools  — as  connected  learning  communities  of 
teachers  and  students,  in  which  some  participants  were  expert  learners  and  others  were  novice  learners,  sharing  information, 
supporting  one  another’s  learning  processes,  and  constructing  a common  base  of  understanding  and  shared  skills. 

Engestrom  (1996,  1998)  pushed  the  limits  of  these  viewpoints  by  envisioning  communities  of  practice  as  Activity  Systems  in 
which  individuals  intentionally  used  tools  (or  technologies)  to  create  or  transform  objects  or  concepts  in  order  to  bring  about  a 
desired  outcome.  In  an  Activity  System,  any  given  individual’s  actions  take  place  within  a sociocultural  framework  that  includes 
the  community  of  which  the  individual  is  a part,  together  with  the  norms  and  conventions  of  use  of  those  tools,  and  the  social 
roles  or  division  of  labor  that  characterize  individual  actions  within  local  collective  activities.  The  basic  difference  between 
traditional  social  learning  theory  as  characterized  by  Lave  and  Wenger  (1991)  and  Engestrom  is  Activity  Theory’s  emphasis  on 
the  transformational  nature  of  collaborative  endeavors.  “Activity  theory  suggests  that  collective  developments  occur  when, 
through  their  actions,  people  reinterpret  their  environment,  rebuild  their  activities,  and  reconceive  of  themselves  (Blackler, 
Crump,  & McDonald,  2000,  p.  296).” 

It  is  interesting  to  note  the  striking  parallel  between  Engestrom’s  depiction  of  an  Activity  System  and  the  process  that  may  be 
occurring  in  the  TALENT  project  regarding  the  adoption  and  use  of  instructional  technologies  to  support  teaching  and  learning. 
Moreover,  the  current  work  of  Engestrom  (2001)  and  his  colleagues  is  now  focusing  on  parallel  Activity  Systems,  such  as  HMOs 
and  hospitals,  that  share  a common  goal:  to  preserve  and  improve  the  health  of  individual  patients.  In  TALENT,  the  university 
teacher  preparation  faculty  and  the  teachers  in  the  cooperating  K-12  schools  can  be  considered  as  a pair  of  parallel  Activity 
Systems  with  a common  goal:  to  foster  good  educational  practice  so  that  all  children  can  learn.  The  learning  community  model 
links  these  two  Activity  Systems  through  the  university  supervisors  and  teacher  candidates,  (Fig,  2). 


Figure  2:  Parallel  Activity  Systems 

As  typically  takes  place  within  an  Activity  System,  TALENT  introduces  new  tools  to  the  university  faculty  and  to  the  K-12 
master  teachers  to  accomplish  two  planned  outcomes:  restructured,  technology  -infused  courses  within  the  university,  and 
technology -infused  lessons  within  the  K-12  schools.  For  the  faculty  member  or  teacher  who  becomes  interested  enough  in  the 
new  tools  to  acquire  more  knowledge  about  their  potential  uses  to  enhance  instruction,  to  develop  a positive  attitude  toward 
incorporating  them  into  their  own  practices,  and  to  actually  make  the  decision  to  implement  technology  in  the  classroom,  this  is 
only  the  beginning.  Each  trainee  must  go  through  a leaming/adoption  trajectory  (Sherry,  1998)  to  gain  the  necessary  comfort, 
confidence,  and  competence  to  do  so.  This  is  the  purpose  of  the  institutes  and  workshops  and  of  the  continuing  technical  and 
collegial  support  provided  by  the  TALENT  leadership,  staff,  and  teaching  community. 

Besides  gaining  the  expertise  and  knowledge  to  use  new  tools  to  enhance  instruction,  there  is  a second  order  effect  that  TALENT 
hopes  to  bring  about  - increased  modeling  of  promising  piactices  for  teaching  with  technology,  by  both  university  faculty  and  K- 
12  master  teachers,  for  their  respective  teacher  candidates.  For  TALENT  to  succeed,  the  project  directors  may  wish  to  consider 
fostering  the  coaching  and  modeling  process  among  its  trainees,  just  as  student -centered  learning  is  explicitly  modeled  within  the 
institutes.  However,  since  the  master  teachers  are  not  paid  for  their  efforts,  and  since  the  Office  of  Field  Placement  does  not  wish 
to  discourage  an  otherwise  qualified  person  from  being  a master  teacher,  this  is  a mediating  factor  that  could  serve  as  an  inhibitor 
for  K-12  teacher  modeling.  It  may  be  fruitful  to  contemplate  discussing  this  issue  with  master  teachers  because  modeling  could 
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help  to  empower  teacher  candidates.  Moreover,  the  extent  of  modeling  the  use  of  technology  for  teacher  candidates  at  field 
placement  sites  represents  one  of  the  research  areas  in  the  national  evaluation  of  the  PT3  grant  initiative.  Since  these  activities 
take  place  within  a sociocultural  context,  the  outcomes-  technology -infused  instruction-  generate  a “ripple  effect”  within  each 
Activity  System  (Fig.  3 & 4). 


Figure  3.  K- 12  Activity  System 


Figure  4:  University  Activity  System 


Changing  Norms 

Rules,  norms,  and  beliefs  define  the  culture  of  an  Activity  System.  Let  us  focus  on  the  cultural  norms  within  the  university  and 
the  K-12  environment.  Wilson  and  Myers  (2000)  state,  “Using  a tool  in  a certain  manner  implies  adoption  of  a cultural  belief 
system  about  how  the  tool  is  to  be  used.”  For  example,  if  the  norm  in  a specific  academic  department  at  the  university  is  to  use 
closed-answer  tests  that  can  be  scored  using  scantrons,  and  if  this  continues  to  make  a professor’s  day  to  day  tasks  relatively 
easy,  then  new  tools  and  assessment  strategies  that  make  his/her  job  harder  will  not  be  used.  They  have  no  relative  advantage 
over  the  status  quo.  Moreover,  they  require  extra  learning  and  extra  work  that  could  be  better  devoted  to  teaching  and  research. 
On  the  other  hand,  if  the  norm  within  that  department,  say,  anthropology,  is  to  collect  data  from  primary  sources  via  observations 
and  interviews,  then  the  use  of  digital  video  could  make  his/her  work  more  fruitful.  Additionally,  video  case  studies  could  be 
shared  and  discussed  with  university  students,  and  could  become  part  of  a living  archive  of  the  community’s  culture.  In  that  case, 
chances  are  that  the  professor  would  continue  to  explore  the  use  of  digital  video  because  it  adds  value  to  his/her  instruction. 
Moreover,  if  other  educators  within  the  department  can  observe  the  improvement  in  the  professor’s  instruction,  and  its  effect 
upon  the  university  students,  then  the  use  of  digital  video  could  very  well  become  the  norm  within  the  department. 

Besides  observability,  Rogers  (1995)  lists  compatibility  with  potential  adopters’  needs  and  values,  relative  simplicity,  the  ability 
to  experiment  with  an  innovation  on  a limited,  risk-free  basis,  and  relative  advantage  over  the  status  quo  as  facilitators  for 
innovative  concepts,  processes,  and  products.  In  a discussion  following  a training  session  in  the  January  2001  institute,  three 
faculty  members  engaged  in  the  following  conversation,  discussing  the  relative  advantage  of  using  digital  photography  for 
university  students’  products: 

/'m  thinking  about  maybe  teaching  some  of  my  history  students  about  how  to  make  a little  film, 

I think  that's  a good  thing  there.  You  're  mentioning  how  you  would  attempt  to  use  it.  How  would  any  of  the  people  in 
the  room  here  intend  to  use  it.  even  what  you  have  learned  within  the  last  three  days  here? 
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Well,  this  just  adds  a whole  other  dimension  to  [student  projects].  There  are  all  kinds  of  cultural  events  going  on  all 
over  Fresno,  Send  people  out  with  a camera  and  equipment,  and  a video  camera  might  be  a little  easier.  Go  to  the 
events  and  put  together,.,  a total  essay  on  PowerPoint  using  the  digital  images,  or  a short  introductory,  maybe  5 to  JO 
minute  short  film  on  cultural  events.  That  would  be  a no  brainer! 

The  same  holds  true  in  the  K-12  school  situation.  National  surveys  (CEO  Forum  on  Education  and  Technology,  1999;  NCES 
1999a,  199b)  indicate  that  students  are  more  adept  at  using  interactive  technologies  than  their  teachers.  Technology  is  becoming 
the  norm  for  school  children.  As  the  use  of  Internet  searching  and  PowerPoint  presentations  by  children  increase,  there  is  a 
chance  that  that  those  tools  will  become  part  of  “the  way  we  do  things  around  here”  within  the  school  setting.  Technology -savvy 
children  will  tend  to  drive  learning  in  their  teachers  as  well.  This  is  especially  true  if  greater  depth  of  research  and  more 
sophisticated,  interactive  presentations  will  result  in  greater  achievement  for  the  children.  RMC  Research  Corporation’s 
evaluation  of  several  PT3  and  Technology  Innovation  Challenge  grants  provides  evidence  from  observations,  interviews,  and 
focus  groups  with  faculty,  K-12  teachers,  and  students,  which  have  shown  that  both  of  these  outcomes  are  common. 

In  the  case  of  TALENT,  activities  are  focused  on  changing  two  sets  of  norms.  At  the  university,  the  new  norm  would  be  to  focus 
instruction  on  the  new  California  state  standards  for  teacher  credentialing,  which  are  echoed  in  the  ISTE/NETS  standards  for 
educators.  At  the  K-12  schools,  the  new  norm  would  be  a greater  focus  on  technology -infused,  student -centered  learning. 
However,  this  is  moderated  by  the  schools’  focus  on  measuring  student  achievement  by  their  performance  on  the  Stanford  9 test 
- an  external  factor  over  which  TALENT  has  no  control. 


Changing  Roles 

Its  division  of  labor  - the  generally  accepted  roles  of  its  members  - defines  the  social  structure  of  an  Activity  System.  Generally, 
teachers  are  considered  experts  while  students  are  considered  learners.  However,  this  role  structure  is  called  into  question  when 
students  have  greater  comfort,  confidence,  and  competence  with  technology  than  their  teachers  do,  and  are  able  to  use  tools  such 
as  word  processing,  concept  mapping,  database  searching,  graphing  calculators,  interactive  mathematics  applets,  Web  search 
engines,  graphic  art,  and  digital  video  effectively  to  increase  their  academic  performance. 

Consider  the  self-perceived  role  of  the  teacher.  If  that  role  is  to  transmit  information,  then  word  processing  and  PowerPoint  will 
suffice,  and  other  tools  are  considered  unnecessary.  Moreover,  if  a teacher  is  concerned  about  the  validity  of  information  on  the 
Web  sites  that  his/her  students  are  citing  in  research  papers,  and  considers  that  open-ended  searching  could  jeopardize  his/her 
role  as  the  expert  in  the  classroom,  then  there  is  no  reason  for  a teacher  to  support  such  online  activities.  But  if  the  role  of  the 
teacher  focuses  on  creating  a rich  environment  for  active  learning  in  the  classroom  (Grabinger,  1996),  then  he/she  would 
encourage  the  students  to  use  the  full  range  of  current  technologies.  Such  activities,  in  turn,  transform  the  role  of  the  teacher  to  a 
co-leamer  and  co- explorer.  This  is  indeed  the  case  in  an  elementary  school  where  one  of  TALENT’S  K-12  master  teachers  has 
his  second  through  fourth  grade  classes  use  digital  video  to  capture  learning  experiences  and  present  them  to  other  teachers, 
parents,  and  the  entire  school  community. 

A sudden  reversal  of  traditional  roles  can  cause  discomfort  and  a disturbance  in  an  Activity  System.  Brown  and  Palincsar  (1989) 
considered  cognitive  dissonance  and  its  ensuing  discomfort  to  be  the  driver  of  learning.  If  the  disturbance  caused  by  a role 
reversal  creates  learning  and  makes  life  better,  then  it  will  be  sustained;  else  the  system  will  reject  it  and  return  to  stasis.  For 
example,  at  a high  school  in  Vermont  that  p articipated  in  The  WEB  Project,  one  student  became  the  school’s  acknowledged 
expert  in  digital  editing.  His  portfolio  gained  for  him  a four-year,  all-expense  paid,  scholarship  to  a leading  university. 
Subsequently,  more  students  became  interested  in  digital  editing,  and  similar  activities  still  continue  at  the  school.  As  a counter- 
example, consider  a tenured  professor  who  considers  him/herself  as  a “stand-up  lecturer”,  as  mentioned  by  one  of  the  TALENT 
trainees  during  a discussion  in  one  of  the  training  institutes.  This  individual  was  interested  in  using  PowerPoint  to  enrich  lectures 
with  slides  that  illustrated  historical  and  artistic  trends.  True,  this  is  using  technology,  but  it  does  not  create  a student -centered 
environment,  nor  will  it  change  the  role  of  the  professor.  As  a result,  our  fear  is  that  his/her  students  will  leave  the  university 
teaching  as  they  have  been  taught  - as  transmitters  of  information,  rather  than  as  creators  of  technology -rich  environments  for 
active  student  learning. 

In  the  case  of  TALENT,  there  was  a wide  range  of  potential  changes  in  roles,  with  several  professors  writing  mini- grant 
proposals  to  restructure  their  courses  to  include  more  university  student  use  of  technology.  As  a whole,  the  university  supervisors 
perceived  technology  as  useful  tools  for  their  preservice  teachers  more  often  than  did  the  technology -savvy  K-12  master  teachers, 
who  primarily  saw  technology  as  a teacher- centered  tool.  However,  the  master  teachers  who  co-taught  several  of  the  TALENT 
training  sessions  countered  this  trend,  often  presenting  and  discussing  products  that  their  K-12  students  had  created. 

The  use  of  technology -focused  learning  communities  would  also  result  in  some  role  changes.  In  a learning  community,  each  of 
the  individuals  would  be  a co-leamer  and  a co- contributor,  constructing  knowledge  as  a coherent  team,  and  serving  as  an 
interface  between  the  university  and  the  K-12  environment.  A potential  outcome  of  the  community  might  be  a few  lessons  that 
were  relevant  and  appropriate  for  the  teacher  candidate’s  classroom,  together  with  the  necessary  resources  and  documentation 


that  constitute  part  of  a teaching  unit.  Another  possibility  would  be  a videotape  of  the  teacher  candidate  demonstrating  a 
promising  practice  for  teaching  with  technology  in  the  K-12  classroom.  These  videotapes  could  then  be  used  to  spark  discussions 
within  the  triad.  They  could  also  be  archived  and  used  as  exemplars  within  the  teacher  education  program. 


Changing  the  Community 

Besides  norms,  rules,  and  roles,  every  Activity  System  has  structures  and  boundaries  that  self-identify  it  as  a system.  At  the 
university,  an  important  structure  is  the  criterion  used  for  review,  tenure  and  promotion.  If  a university’s  structure  supports 
technology  innovation  with  incentives  like  release  time,  extra  pay,  or  some  other  form  of  recognition  for  a faculty  member’s 
effort,  then  the  chances  that  more  tools  will  be  used  increase. 

At  CSUF,  there  is  a Center  for  the  Enhancement  of  Teaching  and  Learning  (CETL).  This  organization  provides  mini  grants  for 
faculty  members  who  restructure  curriculum  in  some  innovative  way  that  enhances  instruction  or  assessment.  At  the  end  of  the 
January  2001  TALENT  institute  from  faculty  and  teachers  in  the  social  sciences,  five  professors  wrote  mini-grant  proposals  for 
restructured  syllabi;  all  five  proposals  were  funded.  Based  on  this  record  of  success,  six  faculty  members  in  the  June  2001 
institute  for  faculty  and  teachers  in  the  arts,  music,  and  humanities  also  expressed  an  interest  in  doing  likewise.  Likewise,  faculty 
members  wrote  two  mini-grant  proposals  following  the  mathematics  and  science  institute  in  July  2001. 

Another  distinctive  feature  of  CSUF  is  its  tenure  and  promotion  process,  which  rewards  innovative  teaching  practices  as  an 
element  of  professional  scholarship.  Moreover,  the  Provost  has  an  annual  award  related  to  outstanding  teaching  with  technology, 
which  recognizes  and  supports  the  work  of  TALENT.  If  such  efforts  continue  to  be  supported  by  the  university,  then  the  vision  of 
the  PT3  project  at  CSUF  may  well  be  realized. 


Driving  Change  in  Activity  Systems 

Two  interesting  research  questions  emerge  from  studying  TALENT:  By  what  process  does  change  in  an  Activity  System  take 
place?  And  how  does  this  differ  for  connected,  parallel  Activity  Systems?  Engestrom’s  reply  would  be  something  like  this: 
‘‘Disturbances  within  and  between  Activity  Systems  are  the  driving  force  for  growth  and  change.”  When  these  questions  were 
posed  at  the  2001  AERA  conference,  one  of  Engestrom’s  peers  (Barab)  replied,  “The  direction  of  change  depends  on  who  holds 
the  power.”  Thus,  further  investigation  into  these  two  questions  revolves  around  empowering  the  agents  within  each  Activity 
System  to  drive  change,  and  empowering  those  who  reside  at  the  intersection  of  two  parallel  Activity  Systems  to  moderate 
change  between  the  two  connected  systems.  Shifting  the  balance  of  power  created  disturbances,  and  "As  disturbances  become 
evident  within  and  between  Activity  Systems,  participants  may  begin  to  address  underlying  issues  and  to  create  new  learning." 
(Blackler,  Crump,  & McDonald,  2000,  p.  281) 

In  conducting  case  studies  of  PT3  grantees  across  the  U.S.,  we  have  found  that  there  is  no  a priori  direction  for  bringing  about 
change  in  either  a university’s  teacher  preparation  program  or  the  field  experiences  of  its  teacher  candidates  at  associated  K-12 
schools.  For  example,  in  teacher  preparation  programs  where  strong  leadership  resides  within  the  school,  college,  or  department 
of  education  (SCDE),  it  is  the  SCDE  that  drives  change  within  the  university  and  at  the  partner  K-12  schools.  In  SCDEs  with  a 
Professional  Development  School  (PDS)  structure,  or  with  schools  that  have  some  of  the  characteristics  of  a PDS,  it  is  the  K-12 
teachers  who  often  drive  the  change  - whether  inservice  or  preservice. 

In  the  case  of  TALENT,  it  is  the  project  directors,  technical  support  people,  and  K-12  master  teachers  who  drive  change,  while 
the  university  system  is  the  recipient  of  the  change,  as  intended  by  the  project  directors.  This  quotation  from  a veteran  professor 
illustrates  this  trend: 

Quite  frankly,  the  issue  is,  are  we  doing  anything  for  [the  students  in  the  teacher  preparation  program]?  I’m  a 
traditional  lecturer,  and  we  look  at  what  they’re  doing  in  the  high  school.  Sometimes  I have  to  wonder,  is  what  I’m 
doing-  even  though  I’m  good  at  it  - is  it  really  pertinent  to  what  they’re  teaching?  I’m  wondering  what  their  students 
think,  who  are  doing  some  of  these  really  creative  things  such  as  PowerPoint... what  do  they  think  of  what  they  learned 
in  our  classes?  How  useful  that  is?  I’ve  given  them  all  sorts  of  information.  ..maybe  the  younger  people  know  how  to  do 
things  that  are  innovative,  people  doing  things  out  there  in  the  high  school...!  mean,  I feel  like  I’m  in  the  dark  ages! 

However,  the  direction  of  change  could  easily  be  reversed,  once  the  university  faculty  members  begin  teaching  their  restructured 
courses  throughout  the  teacher  preparation  program,  and  the  teacher  candidates  begin  to  carry  their  newfound  expertise  and  skills 
into  their  student  teaching  experiences.  Moreover,  learning  communities  that  link  the  university  Activity  System  with  the  K-12 
Activity  System  could  change  this  whole  dynamic  by  sharing  information  between  the  K-12  schools  and  the  SCDE.  This  is  a 
fruitful  area  for  further  investigation  that  could  both  clarify  and  add  predictive  validity  to  TALENT’S  grant  efforts. 


Project  TALENT  learning  communities  are  currently  engaged  in  joint  activities  that  will  be  completed  by  the  end  of  the  current 
academic  term.  Preliminary  findings  will  be  presented  at  Ed-Media. 
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Abstract:  This  exploratory  study  investigates  the  effects  of  advance  organizers  and  learner- 
controlled  sequencing  in  a Web-based  learning  environment.  This  study  used  a 2x2  factorial, 
posttest  only  design.  The  two  independent  variables  were  learner-controlled  sequencing  and 
advance  organizers.  The  two  dependent  variables  were  students’  ability  as  measured  by  scores  on 
the  posttest  and  time  spent  completing  (a)  the  instructional  materials  and  (b)  the  performance  test. 
The  treatments  were  four  Web-based  tutorials  designed  to  teach  students  how  to  produce  simple 
Web  pages  using  Hypertext  Markup  Language  (HTML).  The  analyses  showed  that  neither  learner- 
controlled  sequencing  nor  advance  organizers  affected  student  test  scores.  However,  the  results 
revealed  an  unexpected  effect  on  the  time  students  spent  learning  and  applying  new  material.  The 
study  revealed  that  those  who  consistently  took  the  least  amount  of  time  were  students  using 
program-controlled  sequencing  with  advance  organizers. 


The  Study 


This  study  investigates  the  effects  of  advance  organizers  and  learner-controlled  sequencing  in  a Web-based 
learning  environment.  An  advance  organizer  is  defined  by  Ausubel  (1963,  pp.  81-82)  as  “appropriately  relevant  and 
inclusive  materials”  introduced  in  advance  of  learning  which  functions  to  provide  “ideational  scaffolding”  for  the 
stable  incorporation  and  retention  of  the  more  detailed  and  differentiated  material  to  be  learned.  For  this  study,  the 
advance  organizers  are  in  the  form  of  short  paragraphs  serving  as  metaphors  that  liken  the  components  of  a web 
page  to  that  of  a written  paper  or  report. 

This  study  used  a 2 x 2 factorial,  posttest  only  design.  The  two  independent  variables  were  learner- 
controlled  sequencing  and  advance  organizers.  The  two  dependent  variables  were  students’  ability  as  measured  by 
scores  on  the  posttest  and  time  spent  completing  (a)  the  instructional  materials  and  (b)  the  performance  test.  The 
treatments  were  four  Web-based  tutorials  designed  to  teach  students  how  to  produce  simple  Web  pages  using 
Hypertext  Markup  Language  (HTML). 

The  topic  of  instruction  was  chosen  on  the  basis  of  its  unfamiliarity  to  the  sample  population.  Only  one  out 
of  63  students  in  the  same  class  surveyed  in  a previous  semester  had  ever  created  a Web  page.  Since  one  of  the 
independent  variables  of  the  study  was  the  use  of  advance  organizers,  unfamiliarity  of  the  material  was  an  essential 
criterion.  According  to  Ausubel  (1960,  p.  268),  advance  organizers  are  designed  to  “facilitate  retention  in  areas  of 
knowledge  new  to  learners.”  Using  an  unfamiliar  topic  helped  ensure  that  all  students  started  from  approximately 
the  same  baseline  in  learning  the  material. 

The  subjects  for  this  study  were  74  pre -service  elementary  aid  secondary  teachers,  71  of  whom  were 
female  between  the  ages  of  19  and  23.  All  74  students  were  enrolled  in  a technology  course  at  a large  university  in 
the  intermountain  west.  The  study  was  introduced  and  described  to  students  during  regular  classroom  meeting  time. 
Students  were  asked  to  volunteer  to  participate,  and  all  those  who  consented  were  directed  to  go  to  the  computer  lab 
where  they  were  randomly  assigned  to  one  of  the  four  experimental  groups  approximately  equal  in  size. 

The  posttest  was  a performance-based  activity  where  the  students  were  required  to  produce  a Web  page 
according  to  a set  of  criteria.  A lab  assistant  recorded  a rating  for  each  student  based  on  the  specified  criteria.  This 
rating  was  used  to  determine  student  performance.  In  addition,  each  student  completed  a time  log  that  indicated  the 
time  spent  working  on  each  assignment  and  the  posttest.  The  time  log  was  designed  to  determine  if  there  was  a 
significant  difference  in  the  amount  of  time  spent  between  groups.  Two-way  ANOVAs  were  conducted  for  each 
dependent  variable  to  analyze  the  data. 
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The  Findings 


The  analyses  showed  that  neither  learner-controlled  sequencing  nor  advance  organizers  affected  student 
test  scores.  Whether  analyzed  separately  or  together,  learner-controlled  sequencing  and  advance  organizers  failed  to 
produce  significant  improvements  in  posttest  scores.  Average  mean  scores  were  nearly  identical  for  the  four 
experimental  groups.  However,  students  who  were  given  program-controlled  sequencing  took  significantly  less 
time  (13  percent  less)  to  complete  the  instructional  materials  than  students  who  were  given  learner-controlled 
sequencing.  These  results  suggest  that  program-controlled  sequencing  significantly  reduces  the  time  students  spend 
learning  new  instructional  material.  In  addition,  students  using  advance  organizers  completed  the  performance  test 
in  15  percent  less  time  than  students  not  using  advance  organizers.  These  results  suggest  that  although  performance 
scores  are  not  affected,  advance  organizers  have  a positive  effect  in  reducing  the  time  students  spend  applying  newly 
gained  knowledge. 


Conclusions 

Although  learner-controlled  sequencing  and  advance  organizers  failed  to  produce  more  effective  learning 
(i.e.,  higher  test  scores),  these  variables  yielded  an  unexpected  effect  by  producing  more  efficient  learning.  Program 
control  produced  more  efficient  learning  by  reducing  the  time  students  spent  learning  new  material.  Advance 
organizers  produced  more  efficient  learning  by  reducing  the  time  students  spent  applying  the  newly  learned  material 
in  a performance  test.  Students  who  consistently  took  the  least  amount  of  time  were  those  using  program-controlled 
sequencing  with  advance  organizers.  When  compared  to  the  group  taking  the  most  time,  this  group  was  20  percent 
faster  learning  the  instructional  materials  and  25  percent  faster  completing  the  performance  test. 

What  are  the  implications  of  these  results  in  educational  practices?  The  author  postulates  that  the  combined 
use  of  program  control  and  advance  organizers  can  lead  to  significantly  reduced  instructional  and  testing  time  in  a 
Web-based  learning  environment.  The  combined  effect  of  program-controlled  sequencing  with  advance  organizers 
produced  the  same  performance  test  results  as  any  other  combination  in  the  2x2  factorial  design  used  in  this  study. 
The  benefit  is  in  the  time  it  takes  to  obtain  those  results,  which  in  this  study  was  significantly  less.  When 
considering  the  proliferation  of  Web-based  instruction  and  training  costs  increasing  in  the  future,  the  implications 
are  significant  in  educational  and  business  settings  or  anywhere  that  time  is  a highly  valued  resource. 
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Abstract:  This  paper  describes  the  way  teachers  at  Delft  University  of  Technology  make 
use  of  the  new  online  assessment  tool,  called  Etude.  In  order  to  judge  the  extent  to  which 
the  benefits  of  online  assessment  will  be  achieved,  a checklist  was  developed.  Results 
show  that  lecturers  as  well  as  students  appreciate  the  possibilities  of  online  assessment, 
but  an  educational  guidance  program  will  be  necessary  in  order  to  benefit  optimally  from 
the  educational  opportunities  of  online  assessment. 


1.  Introduction 

Delft  University  of  Technology  (DUT)  started  the  development  of  a new  online  assessment 
system,  named  Etude,  in  1 996.  The  system  became  available  to  the  teaching  staff  in  September  2000. 
The  objective  for  developing  the  Etude  system  was  to  offer  teachers  one  system  that  is  supported  and 
maintained  centrally.  Etude  is  an  item -banking  program  that  provides  for  item  entry  and  storage , test 
creation,  online  testing,  scoring,  reporting  and  analysis  of  tests.  In  earlier  papers  we  described  the 
completion  and  functionality  of  Etude  (Van  de  Ven,  et  al,  1999),  and  reported  about  the  use  of  the 
system  in  two  pilot  projects  (Dopper  and  Van  de  Ven,  2001).  At  the  moment  of  writing  this  paper. 
Etude  is  used  at  17  different  courses  at  DUT. 

This  paper  offers  an  outline  for  evaluating  the  use  of  online  assessment.  We  will  identify  the 
potential  benefits  of  online  assessment  with  Etude  and  describe  the  conditions  for  achieving  these.  This 
will  result  in  a checklist  used  to  determine  whether  teachers  will  realize  the  benefits  of  an  online 
assessment  system  or  not.  We  analyzed  the  way  the  present  users  at  DUT  have  implemented  Etude  in 
their  (online)  course,  based  on  interviews.  One  of  these  cases  will  be  described  in  more  detail  and  the 
results  of  the  appreciation  of  students  of  two  of  the  identified  benefits  will  be  presented.  Finally,  we 
will  answer  the  research  question:  is  the  way  in  which  Etude  is  used  by  the  present  users  appropriate  to 
realize  the  benefits  of  the  system? 

2,  Potential  benefits  of  Etude  and  conditions  to  reach  them 

We  have  divided  the  potential  benefits  in  three  sections,  i.e.  storage  of  items  (2.1),  item  entry 
(2.2)  and  taking  a test  online  (2.3).  This  division  is  based  on  two  main  functions  of  online  assessment 
systems:  building  and  maintaining  an  item  bank,  and  students  taking  a test  online. 

2.1  Storage  of  items 

An  item  bank  is  a collection  of  test  items  that  may  be  easily  accessed  for  use  in  preparing 
examinations.  Good  item  bank  management  enables  efficient  and  effective  generation  of  tests  and 
efficient  re-use  of  items.  To  achieve  these  benefits,  it  is  important  that  items  are  not  stored  in  the  item 
bank  in  an  arbitrary  order.  In  order  to  retrieve  items  as  necessary  to  make  revisions  or  to  create  a test, 
items  have  to  be  coded  (Ward  and  Murray -Ward,  1994).  In  Etude  the  following  codes  can  be  used:  key 
word,  subject  matter,  difficulty  level,  literature,  and  objectives.  Content  codes  are  preferably  based  on 
objectives.  Once  a coding  scheme  is  set  up,  there  should  be  no  modification  in  the  coding  scheme 
unless  the  entire  bank  is  recoded  (Ward  and  Murray  -Ward  1994). 

2.2  Item  entry 

Most  of  the  online  assessment  systems  on  the  market  have  a great  variety  of  possibilities  to 
design  items,  so  does  Etude.  Five  important  options  are  described  in  this  section. 

Type  of  questions 

The  Etude  system  provides  for  four  types  of  questions:  multiple-choice  questions  (1  out  of  m and  n out 
of  m),  q>en  numerical  questions  with  the  use  of  variables,  short  answer  questions  and  hotspot 
questions.  Care  should  be  taken  that  items  stored  in  the  bank  are  of  high  quality.  An  important 


condition  for  this  is  that  the  chosen  question  type  fits  the  learning  objectives  to  be  tested.  For  example 
an  open  numerical  question  is  more  appropriate  than  a multiple-choice  question  when  students  have  to 
calculate.  This  implies  that  teachers  have  to  rethink  their  old  questions:  Is  this  the  best  way  to  present 
the  question  online  in  order  to  achieve  this  learning  objective? 

Multi-media  elements 

Etude  enables  adding  several  multimedia- elements  to  items,  such  as  pictures,  sound,  animations  and 
video.  The  use  of  multimedia  elements  makes  it  possible  to  assess  other  learning  objectives,  for 
example  asking  students  to  identify  the  cavitation  type  by  showing  a video  of  a rotating  propeller.  An 
important  condition  for  the  use  of  multi-media  in  a test  is  that  they  are  functional,  i.e.  that  students  need 
them  to  answer  the  question  coffectly. 

Feedback 

Etude  offers  different  feedback  options  in  order  to  provide  students  with  immediate  feedback. 
Feedback  assists  the  learner  in  constructing  his  or  her  understanding.  Ultimately,  in  the  absence  of 
feedback,  learners  will  feel  isolated,  motivation  to  learn  will  drop,  and  ultimately  learning  will  suffer 
(Rovai,  2001).  It  seems  that  students  not  only  want  to  know  if  their  answer  was  right  or  wrong,  but  also 
why  it  was  right  or  wrong.  So  the  feedback  requires  containing  explanatory  remarks.  A simple  "sorry, 
this  answer  is  wrong,  try  again",  is  not  sufficient. 

Hints 

Hints  can  be  added  to  a question,  with  or  without  score  discount.  The  difference  with  feedback  is  that  a 
hint  appears  before  the  question  has  been  answered  and  that  the  student  can  decide  whether  opening  the 
hint  or  not.  Hints  have  great  potentials,  but  also  high  risks.  Hints  can  help  students  to  get  on  the  right 
track,  when  they  don't  know  the  protocol  to  solve  the  problem.  But  when  a student  uses  a hint  and  the 
hint  turns  out  to  be  useless,  motivation  to  learn  will  drop,  especially  when  it  means  a score  discount. 
Consequently,  a hint  should  contain  useful  information  for  the  student,  for  example  the  line  of 
reasoning  to  solve  the  problem  or  a reference  to  resources  where  the  answer  can  be  found.  After 
reading  a hint  students  should  have  the  feeling  that  it  was  worth  using  it. 

Main  and  sub  question  mechanism 

The  main  and  sub  question  mechanism  offers  students  a difficult  main  question,  which  give  students 
the  chance  to  earn  the  total  score  for  the  item.  If  the  main-question  is  answered  correctly,  the  student 
can  go  to  the  next  main- question,  if  it  is  answered  incorrectly,  the  student  is  offered  a few  sub- 
questions, which  can  still  result  in  a part  of  the  total  score  for  the  item.  Main*  and  sub-questions  can  be 
used  to  distinguish  between  excellent  students  and  average  students.  Excellent  students  are  able  to 
generate  the  protocol  to  solve  the  problem,  whereas  average  students  need  help  to  solve  the  problem  in 
little  steps.  A condit  ion  for  a functional  use  of  the  main  and  sub  question  mechanism  is  that  teachers 
know  exactly  what  difficulties  students  have  when  solving  certain  problems. 

2.3  Taking  a test  online 
Just  in  time  testing 

Students  can  take  an  online  test  whenever  and  whereever  they  want.  Especially  when  the  aim  of  the  test 
is  to  practice  and  not  to  grade  students,  this  can  have  great  benefits.  When  the  aim  of  the  test  is  to 
practice,  online  testing  can  have  the  benefit  of  just  in  time  learning.  In  a constructivist-learning 
environment  students  have  to  plan  their  own  learning  activities.  A lecturer  can  make  the  test  available 
for  some  weeks  during  the  course,  and  students  can  decide  themselves  when  the  online  test  fits  best  in 
their  learning  process.  This  can  differ  from  student  to  student.  Also  the  place  where  students  take  the 
online  test  is  flexible.  They  can  take  the  test  on  any  computer  with  an  Internet  Connection.  An 
important  condition  for  just  in  time  testing  is  that  students  know  the  best  moment  for  taking  the  test. 

Immediate  feedback  on  scores 

When  students  take  a test  online,  it  is  possible  to  give  them  information  about  their  score  on  a question 
immediately  after  answering  the  question.  This  is  a benefit  of  online  testing  compared  to  written  tests. 
Moreover,  after  finishing  the  test,  students  immediately  know  their  total  score  for  the  test. 

Adaptive  testing 

A test  is  called  adaptive  if  it  is  able  to  select  an  item  on  the  basis  of  the  student's  performance  on  earlier 
items  (Essenius,  1995).  The  advantage  of  adaptive  testing  is  that  students  receive  a set  of  questions 
appropriate  to  their  abilities,  so  the  test  remains  challenging  for  each  individual  student.  Adaptive  tests 
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can  be  used  in  training  situations  as  well  as  in  examinations.  In  a training  situation  the  aim  is  to 
improve  the  performance  of  the  students.  An  adaptive  exam  aims  to  assess  the  student's  performance- 
level.  The  Etude  system  offers  two  adaptive  test  methods,  one  is  adaptive  for  the  difficulty  level,  the 
other  is  pre-planned  by  the  teacher.  Using  the  first  method  students  get  one  question  with  a certain 
difficulty  level,  and  depending  on  the  correctness  of  their  answer,  they  will  get  a more  or  less  difficult 
question.  Using  the  second  method  the  test  also  starts  with  one  item,  and  depending  on  the  answer,  the 
student  will  go  into  another  pre-defined  path  of  items.  A necessary  condition  for  the  first  method  is  a 
large  item  bank  and  calibrated  items.  For  the  second  method  one  needs  fewer  items  in  the  item  bank, 
but  the  teacher  has  to  think  ahead  carefully  about  the  paths  to  define. 


Conditions  for  reaching  the  benefits: 

Do  teachers  reach 
the  benefits?  Yes/no 

A.  Storage  of  items 

• Is  a coding  scheme  available? 

• Is  the  coding  scheme  based  on  objectives? 

B.  Item  entry 

B1 . Type  of  questions 

• Does  the  chosen  question  type  fit  the  learning  objective? 

B2.  Multimedia 
elements 

• Are  the  multimedia  elements  functional? 

• Are  the  multimedia  elements  appropriate  to  the  objectives? 

B3.  Feedback 

• Does  the  feedback  contain  explanatory  remarks? 

B4.  Hints 

• Does  the  hint  contain  useful  information  for  the  student? 

B5.  Main  and  sub 
questions 

• Does  the  main  question  discriminate  between  excellent  and 
average  students? 

• Do  the  sub  questions  lead  to  an  answer  to  the  main  question? 

C.  Taking  a test  online 
Cl . Just  in  time  testing 

• Do  the  students  know  the  best  moment  for  taking  the  test? 

C2.  Immediate 
feedback  about  scores 

• Is  the  effect  of  knowing  the  score  on  the  behavior  of  the 
student  beneficial  for  the  objective  of  the  test? 

C3.  Adaptive  testing 

• Are  a large  item  bank  and  calibrated  items  available  (difficulty 
levels)? 

• If  students  can  do  the  difficult  paths,  does  that  mean  that  they 
can  do  the  easy  paths  as  well?  (pre-defined  paths)? 

Figure  1:  Checklist  to  determine  whether  the  benefits  of  an  online  assessment  tool  will  be  achieved 

3.  How  is  Etude  used? 

We  interviewed  eleven  of  the  present  users  of  Etude  at  DUT.  We  developed  an  interview 
scheme  based  on  the  identified  benefits,  in  order  to  find  out  what  functionality  they  used  and  what 
reasons  they  had  to  use  it  or  not.  The  results  are  presented  in  this  section. 

3.1  Storage  of  items 

In  order  to  achieve  the  benefit  of  re-using  items,  a coding  scheme  is  necessary  (see  figure  I). 
However,  none  of  the  respondents  use  codes  to  classify  their  items.  They  organize  their  items  in 
folders,  using  chapters  of  the  book  as  the  organizing  principle.  Teachers  explained  that  the  reason  for 
not  having  a coding  scheme  is  lack  of  time.  Furthermore,  they  were  not  convinced  that  having  a coding 
scheme  is  necessary,  because  of  the  so  far  relatively  small  number  of  items  in  the  item  bank  (varying 
from  10  to  250).  However,  retrieving  items  in  order  to  make  revisions  or  to  create  a test,  turned  out  to 
be  harder  when  item  banks  expand.  Consequently,  benefits  like  efficient  re-use  of  items  will  probably 
not  be  reached.  Moreover,  ordering  items  according  to  the  chapters  of  a book  has  the  risk  that  when  the 
book  is  replaced,  the  item  bank  becomes  useless. 

3.2  Item  entry 
Type  of  questions 

Teachers  do  not  tend  to  rethink  their  old  questions  and  think  about  what  question  type  fits  the  learning 
objectives  best.  They  start  to  fill  the  item  bank  with  items  they  already  have.  The  existing  questions  are 
mostly  multiple-choice  or  open  numerical.  The  interviews  showed  that  only  two  respondents  use 
hotspot  questions  and  one  respondent  uses  short -answer  questions. 

Multi -media  elements 

The  only  multi- media  elements  used  by  (half  of)  the  present  Etude  users  are  pictures.  Pictures  are 
applied  by  the  teachers  in  a functionally  way.  Reasons  for  not  using  sound,  animations  and  video  are 
"too  busy  with  other  things",  "no  material  available",  or  technical  reasons  like  "no  sound  card  or 
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headphones  available  in  the  computer  room"  or  "slow  connections  in  student  houses".  None  of  the 
respondents  related  the  use  of  multi- media  to  other  learning  objectives. 

Feedback 

Eight  of  the  interviewed  teachers  make  use  of  feedback.  Most  of  the  teachers  use  feedback  for  both 
correct  and  incorrect  answers.  Some  of  them  also  add  specific  feedback  to  different  possble  answers. 
They  understand  the  importance  of  feedback,  especially  when  the  aim  of  the  test  is  to  practice.  Those 
who  don't  use  feedback  yet,  plan  to  do  it  in  the  near  future.  However,  not  all  of  the  teachers  use 
explanatory  remarks. 

Hints 

Only  three  respondents  add  hints  to  their  questions.  This  is  because  constructing  hints  is  labour- 
intensive.  Also,  not  all  teachers  recognize  the  added  value  of  hints.  It  appears  that  the  use  of  feedback 
is  more  natural  than  using  hints. 

Main  and  sub  question  mechanism 

Only  one  of  the  interviewed  teachers  uses  the  main  and  sub  question  mechanism.  It  seems  that  the 
present  users  of  Etude  were  not  acquainted  with  this  mechanism.  Once  it  was  explained,  most  of  them 
were  interested.  ^ 

3.3  Taking  a test  online 
Just  in  time  testing 

Nearly  half  of  the  teachers  use  Etude  for  both  exams  and  practice  tests.  Two  teachers  use  Etude  just  for 
exams  and  four  use  Etude  only  for  practice  purposes.  In  the  case  of  exams,  all  tests  are  taken  in  a 
controlled  assessment  setting.  The  practice  tests  were  available  for  students  for  some  weeks,  and  they 
could  decide  themselves  when  the  test  fitted  best  in  their  learning  process. 

Immediate  feedback  about  scores 

Opinions  differ  on  whether  students  should  be  shown  their  scores  immediately.  In  practice  test 
situations  all  teachers,  except  one,  gave  immediate  feedback  about  the  scores.  They  assumed  that  the 
scores  could  stimulate  students  to  learn  the  subject  matter  again,  or  that  the  scores  stimulated  students 
to  do  the  test  more  seriously.  In  case  of  an  exam,  half  of  the  teachers  offer  students  their  scores 
immediately.  A reason  they  gave  for  this  was:  "a  benefit  of  online  assessment  is  that  students  know 
their  test  result  right  after  finishing  the  exam".  Reasons  for  not  showing  the  scores  were:  "fear  that 
students  would  quit  the  exam  once  they  know  they  will  pass",  "fear  for  failures  in  the  calculating 
system  of  the  scores",  and  "fear  for  a psychological  effect,  when  students  know  during  the  test  that  they 
are  doing  poorly". 

Adaptive  testing 

None  of  the  present  users  make  use  of  adaptive  testing.  They  are  simply  not  familiar  with  the 
possibility,  or  did  not  find  the  time  yet  to  think  about  it.  Half  of  the  teachers  is  interested  in  applying 
adaptive  testing  for  practice  purposes  in  the  future.  Most  of  them  consider  the  main  and  sub  question 
mechanism  to  be  more  attainable. 

4.  How  do  students  appreciate  two  of  the  promises  of  online  assessment? 

In  the  previous  section,  we  predicted  to  what  extent  the  teachers  will  reach  the  benefits  of  the 
online  assessment  tool  Etude.  In  this  section,  we  will  address  and  evaluate  the  implications  of  using 
such  a tool  for  a course  on  collaborative  report  writing.  The  course  on  collaborative  report  writing  is 
part  of  the  first-year  module  Systematic  Problem  Solving  at  the  faculty  of  Technology,  Policy  and 
Management  at  DUT.  Since  2000,  students  and  tutors  no  longer  meet  face  to  face.  Instead,  all 
instructions,  course  material,  exercises,  etc.  are  provided  through  an  electronic  learning  environment, 
called  Blackboard  Courseinfo  (http://www.blackboard.eom/y  This  platform  has  been  chosen  as  part  of 
a faculty  wide  project  on  ICT  in  education  (Brakels  et  al.  2002).  Our  experience  with  using  this 
platform  for  electronic  peer  review  in  collaborative  writing  has  been  presented  in  Sjoer  and  Brakels 
(2001).  In  2001  the  online  assessment  tool  Etude  was  added  to  the  ICT  learning  environment.  The 
introduction  of  online  assessment  was  meant  to  stimulate  the  students  to  study  writing  techniques. 

4.1  Evaluating  the  use  of  Etude:  research  questions  and  design 

The  application  of  the  online  assessment  tool  Etude  was  evaluated  during  the  autumn  course 
of  2001.  As  part  of  the  course,  a three-week  project  has  to  be  carried  out.  Part  of  this  project  consisted 
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of  writing  a report.  During  the  three-week  period,  at  a self -chosen  moment,  each  student  had  to  do  an 
online  assessment  on  report  writing.  The  main  goal  of  this  assessment  is  that  students  can  apply  their 
knowledge  on  report  writing  to  cases  similar  to  those  they  have  to  write  themselves.  The  assessment 
was  set  up  in  such  a way  that  the  students  could  do  the  assessment  at  the  time  when  case-based 
exercises  are  useful  to  them.  Moreover,  the  tutors  have  taken  the  effort  to  develop  specific 
computerized  feedback  for  every  possible  answer. 

To  evaluate  the  application  of  Etude,  we  formulated  and  investigated  the  following  research  questions. 

At  what  moment  in  the  educational  process  do  the  students  carry  out  the  assessment  and  are  they 

satisfied  afterwards  with  the  moment  they  selected? 

Did  the  students  appreciate  the  immediate  feedback,  and  did  they  find  it  useful? 

To  answer  these  questions,  a questionnaire  was  developed.  The  questionnaire  was  split  into  four 
sections  and  consisted  of  closed  questions.  With  respect  to  the  research  questions  we  asked  the  student 
the  point  in  the  writing  process  where  they  did  the  assessment  and  their  appreciation  of  the  feedback 
provided.  138  students  performed  the  assessment.  84  students  completed  the  questionnaire,  which 
implies  a response  of  61%.  The  questionnaire  was  answered  anonymously,  directly  after  the  course  was 
completed. 

4.2  Learning  ‘Just-in-time’ 

In  the  assessment  instruction,  the  students  had  been  advised  to  take  the  test  after  they  had  read 
certain  parts  of  the  manual  about  report  writing,  but  before  they  actually  started  composing,  and 
revising  texts  of  their  own.  Only  1 1 students  followed  the  assessment  instructions  to  the  letter.  82%  of 
the  students  indicated  that  they  studied  the  course  book  and  54%  of  the  students  studied  the  course 
book  before  doing  the  assessment  (figure  2).  If  we  look  at  which  moment  in  the  educational  process  the 
student  carried  out  the  assessment,  we  see  that  more  than  half  of  them  did  the  assessment  during  the 
writing  process  (as  shown  in  figure  3).  Yet,  nearly  a third  of  them  would  have  preferred  to  do  it  earlier. 
Moreover,  although  20%  of  the  students  indicate  that  they  did  the  assessment  after  finishing  the  report, 
more  than  half  of  them  concluded  that  this  was  a mistake  i.e.  they  should  have  performed  the 
assessment  earlier  on.  To  conclude,  nearly  half  of  the  students  either  followed  the  assessment 
instructions  (relative  to  writing)  or  indicated  that  they  should  have  followed  them. 


When  did  the  student  study  the 
prescribed  book  on  report  writing? 


not  at  all  before  during  in  after 
between 
tests 


Study  relative  to  assesment 


When  did  the  student  perform  the  assessment? 


before  during  after 

Assessment  relative  to  writing 


B earlier  BOK  □ later 


Figure  2:  Moment  selected  for  studying  book  Figure  3:  Moment  selected  for  performing  test 

4.3  Computerized  Feedback 

The  students  reacted  positively  to  questions  that  are  related  to  the  computerized  feedback. 
82%  of  the  students  indicate  that  they  appreciate  getting  feedback  on  every  answer,  which  probably 
accounts  for  the  fact  that  nearly  85%  took  the  effort  to  read  most  or  all  of  the  provided  feedback.  The 
students  had  a higher  appreciation  for  feedback  on  wrong  answers  than  for  correct  answers,  scoring 
70%  for  wrong  answers  and  64%  for  correct  answers  on  ‘useful*  or  ‘very  useful*.  Despite  the  high 
appreciation  for  the  feedback,  42  % indicated  that  they  had  learned  ‘pretty  much*  from  the  feedback, 
while  36%  indicated  that  they  learned  only  a little.  This  suggests  that  giving  feedback  is  very  positive, 
but  that  even  more  effort  should  be  spent  on  the  content  of  the  feedback.  Especially  when  half  of  the 
students  do  not  always  find  the  feedback  clear  enough. 
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To  sum  up,  students  should  be  stimulated  more  to  start  the  assessment  before  they  actually 
start  writing  the  report.  The  feedback  option  was  appreciated  greatly.  It  shows  great  promise  for 
distance  education,  provided  that  enough  effort  is  spent  on  the  content  and  readability  of  the  feedback. 

5.  Conclusions  and  recommendations 

In  this  section  we  answer  the  research  question:  is  the  way  Etude  is  used  by  the  present  users 
suitable  to  realize  the  benefits  of  the  system?  In  general  we  can  conclude  that  offering  an  assessment 
tool  like  Etude  doesn’t  mean  automatically  that  the  users  will  achieve  the  potential  benefits  of  the  tool. 
It  appears  that  when  a new  ICT  tool  is  applied,  people  tend  to  start  with  the  substitution  of  what  they 
were  used  to  doing.  In  other  words,  they  don’t  adapt  their  method  to  the  tool.  This  conclusion  is 
supported  by  our  interview  results  in  several  ways,  as  shown  in  the  following  sections. 

5.1  Storage  of  items 

Teachers  use  chapters  of  the  book  to  organize  their  items,  because  this  classification  is 
common  to  them  and  immediately  available.  We  recommend  to  support  teachers  to  think  of  a coding 
scheme  which  is  based  on  the  learning  objectives,  because  this  will  improve  the  chance  that  they  will 
fully  profit  from  the  benefit  of  efficient  re-use  of  items  and  effective  generation  of  tests. 

5.2  Item  entry 

Teachers  start  filling  their  item  bank  with  existing  items,  instead  of  considering  what  type  of 
question  fits  best  to  the  learning  objectives.  This  also  applies  the  use  of  multi-media,  especially  sound, 
video  and  animations.  It  seems  to  take  a lot  of  effort  and  educational  insight  to  create  items  that  assess 
the  learning  objectives  most  favorably.  Part  of  the  potential  of  the  system  is  not  used  at  all,  because 
teachers  are  not  familiar  with  it.  Students  as  well  as  teachers  are  positive  about  computerized  feedback. 
Teachers  understand  the  added  value  of  feedback  for  the  learning  process  of  the  students.  Students 
appreciate  feedback  on  wrong  answers  as  well  as  correct  answers.  The  case  on  report  writing  shows 
that  feedback  has  to  be  clear  in  order  to  reach  the  benefits.  Educational  assistance  should  focus  on  the 
quality  of  the  feedback.  Teachers  don’t  recognize  the  possible  benefits  of  hints.  Consequently,  they 
don’t  know  how  to  apply  a hint  functionally.  Educational  assistance  is  necessary  in  order  to  make  clear 
in  what  way  hints  can  support  the  students’  learning  process.  Teachers  are  not  acquainted  with  the 
main-  and  sub  question  mechanism  in  Etude.  The  same  counts  for  the  adaptive  test  methods.  Teachers 
should  be  informed  about  these  possibilities. 

5.3  Taking  a test  online 

Teachers  should  consider  how  to  guide  students  in  their  learning  process.  In  the  case  on  report 
writing  the  appropriate  time  to  take  the  test  was  included  in  the  test  instructions.  This  was  not  sufficient 
for  all  the  students  to  take  the  test  at  the  right  moment.  As  a result  of  this  case  study  an  educiional 
guidance  program  is  being  developed  in  order  to  integrate  the  new  assessment  tool  successfully.  It 
includes  support  on  different  organizational  levels  in  order  to  set  up  well-founded  code  schemes,  to  fill 
item  banks  with  high  -quality  items,  and  to  implement  the  tool  in  the  educational  process. 
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Beyond  Technology  Training: 

PT3  Grantee  Fills  the  Technical  Support  Void,  Enabling  Schools  to 
Make  the  Most  of  Their  Available  Technology-related  Resources, 
Helping  to  Bring  Technology  to  Life  in  the  Classroom 

( 

Susan  Skuda,  Project  ImPACT,  The  University  of  Tennessee 
Knoxville,  TN,  USA  sskuda@utk.edu 

Abstract: 

Project  ImPACT  (Implementing  Partnerships  Across  the  Curriculum  with  Technology)  is 
a first-year  PT3  implementation  grant  awarded  to  the  University  of  Tennessee  in  2001 . A 
primary  goal  of  the  project  is  to  work  in  partnership  with  preservice  teachers,  mentoring 
teachers,  local  area  schools  and  school  systems  to  effectively  infuse  technology  into  the 
curriculum  to  enhance  student  learning.  At  the  local  school  level,  Project  ImPACT 
provides  training  to  mentoring  teachers,  interns,  the  university  faculty  supervisor,  and  the 
school  technology  coordinator  on  computer  applications  and  techniques  as  well  as 
methods  of  integrating  technology  into  the  curriculum.  Early  in  the  initial  training  phase 
of  the  project,  local  schools  in  one  participating  school  district  received  wireless  mobile 
computer  labs  accompanied  by  limited  or  no  technical  support  for  setup  and  integration. 

This  paper  will  focus  on  the  tremendous  change  that  occurred  in  the  overall  dynamics  of 
one  specific  university-school  partnership  as  Project  ImPACT  stepped  in  to  fill  the 
technical  support  void  and  worked  together  with  the  participants,  the  school  and  the  ' 

school  system  to  bring  this  newly  purchased  technology  out  of  the  carton  and  into  the 
classroom,  and  will  share  the  tremendous  explosion  in  technology  -enhanced  learning  that 
occurred  as  a result. 


Project  ImPACT  Q.mplementing  Partnerships  Across  the  Curriculum  with  Technology)  is  a first-year  PT3 
implementation  grant  awarded  to  the  University  of  Tennessee  in  2001.  A primary  goal  of  the  project  is  to 
work  in  partnership  with  preservice  teachers,  mentoring  teachers,  local  area  schools  and  school  systems  to 
effectively  infuse  technology  into  the  curriculum  to  enhance  student  learning.  At  the  local  school  level. 
Project  ImPACT  provides  training  to  mentoring  teachers,  preservice  teachers,  university  faculty 
supervisors,  and  school  technology  coordinators  on  computer  applications  and  techniques  as  well  as 
methods  of  integrating  technology  into  the  curriculum. 

Prior  to  the  beginning  of  project  training,  teachers  at  one  participating  middle  school  expressed  tremendous 
frustration  with  the  very  limited  technological  resources  available  to  them.  Most  classrooms  had  only  one 
current  computer  system,  which  primarily  served  as  a teacher  station  to  record  grades  and  perform  basic 
word -processing  tasks.  The  library,  which  was  to  serve  as  the  training  center  for  the  project,  was  equipped 
with  several  computers,  but  of  varying  ages  and  capabilities.  As  a group,  teachers  at  this  school  agreed  that 
there  was  simply  no  way  to  integrate  technology  into  the  curriculum  with  their  limited  and  often  unreliable 
hardware.  The  frustration  was  so  great,  and  their  outlook  so  grim,  one  group  member  stated  that  none  of 
them  would  even  be  participating  in  the  project  were  it  not  for  the  stipend  they  could  look  forward  to 
receiving  upon  completion  of  project  activities. 

Before  training  was  to  begin,  however,  each  school  in  this  particular  school  district  learned  that  they  were 
to  receive  wireless  mobile  computer  labs  as  a result  of  a long-term  district  initiative  to  create  a world-class 
school  system.  After  learning  of  the  impending  arrival  of  the  new  equipment,  the  group  became  somewhat 
energized,  but  still  expressed  doubt  as  to  the  practicality  and  logistics  of  using  the  labs  in  their  classrooms. 
While  the  purchase  and  distribution  of  the  computer  hardware  represented  the  achievement  of  an  extremely 
admirable  goal,  the  school  district  provided  limited  or  no  technical  support  for  its  setup  and  integration. 


When  ImPACT  staff  arrived  to  begin  the  training  phase  of  the  project,  participating  teachers  expressed 
great  frustration  that  this  wonderful  new  resource  remained  packed  in  boxes  and  stacked  in  the  hallway. 


Immediately  upon  becoming  aware  of  the  situation,  Project  ImPACT  trainers  met  with  the  administration 
of  the  school  to  offer  their  help  and  time  to  prepare  the  equipment  for  use  at  the  school.  Working  together 
with  school  personnel  and  PTSA  volunteers.  Project  ImPACT  staff  quickly  set  up  the  first  of  the  mobile 
computer  labs.  Once  operational,  ImPACT  trainers  began  to  integrate  the  wireless  lab  into  project  training 
sessions. 

Initial  reticence  soon  gave  way  to  growing  enthusiasm  as  project  participants  became  familiar  with  the  new 
equipment  and  the  myriad  possibilities  that  it  made  available.  Once  the  first  mobile  computer  lab  was  set 
up  and  put  into  use,  the  other  labs  were  quickly  assembled  as  well.  Project  ImPACT  mentoring  and 
preservice  teachers  soon  “adopted”  the  labs,  taking  ownership  and  pride  in  their  new  equipment. 

Project  staff  and  participants  weathered  the  storm  together  as  they  worked  through  the  initial  bugs  and 
difficulties  that  so  often  accompany  the  deployment  of  new  equipment.  Mentoring  teachers  and  interns 
learned  quickly  to  take  things  in  stride  when  the  new  hardware  did  not  always  work  as  planned,  even  for 
instructional  technology  professionals.  Working  through  the  idiosyncrasies  of  the  hardware  and  software 
together  with  the  ImPACT  staff,  school  personnel  became  far  more  comfortable  with  the  equipment  than 
they  might  have  been  if  exposed  in  a more  controlled  laboratory  setting. 

The  partnership  among  school  personnel,  volunteers,  and  Project  Staff  blossomed  as  the  local  PTSA  unit 
hosted  a reception  for  the  superintendent  of  the  school  district,  featuring  a project  training  session  using  the 
wireless  mobile  computer  labs  so  recently  purchased  by  the  district.  Guests  at  the  reception  included 
School  Board  Members,  the  mayor  of  the  town,  members  of  the  local  Chamber  of  Commerce,  as  well  as 
many  volunteers  and  community  activists.  This  presented  the  perfect  opportunity  for  the  project  and  the 
school  to  showcase  their  growing  and  mutually  beneficial  partnership.  Delighted  with  the  chance  to  see 
his  initiative  in  action,  the  superintendent  spent  a great  deal  of  time  with  grant  participants,  discussing 
potential  use  of  the  new  equipment  and  the  possibilities  for  use  in  the  classroom.  The  community  was  once 
again  treated  to  a demonstration  of  the  newly  purchased  equipment  as  the  PTSA  subsequently  invited 
Project  ImPACT  to  participate  in  their  annual  Founder’s  Day  celebration  by  demonstrating  the  use  of  the 
wireless  labs  and  explaining  project  training. 

Stressing  teamwork,  partnership,  and  collaboration  throughout  the  training  process.  Project  ImPACT  staff 
continued  to  work  with  participants  as  they  began  to  plan  for  the  integration  of  technology  into  the 
curriculum.  Project  ImPACT  staff  facilitated  regular  team  meetings,  at  which  teachers  and  interns  began  to 
share  their  technology  integration  plans.  As  one  eighth  grade  math  teacher  discussed  some  of  her  ideas  to 
incorporate  technology  into  her  classroom  lessons,  a seventh  grade  math  teacher  realized  that  she,  too,  was 
working  on  activities  involving  the  very  same  curriculum  content  for  her  advanced  class.  Inspiration 
finally  struck,  and  these  two  teachers,  who  had  never  considered  working  together  before,  began  to  share 
ideas  and  lesson  plans.  Collaboration  grew  from  there  as  other  teachers  who  were  accustomed  to  working 
in  the  isolation  of  their  pods  and  grade  levels  began  to  collaborate  as  well.  As  Project  activities  continued. 
Project  ImPACT  participants  began  to  be  sought  out  by  their  colleagues  as  a valuable  resource  on  how  to 
best  use  the  new  labs.  Today  the  labs  are  in  high  demand  all  over  the  school.  Technology-enhanced 
lessons,  once  thought  to  be  impractical  at  best,  have  started  to  become  the  norm,  rather  than  the  exception. 

Acting  as  school  technology  coordinator  frr  Project  ImPACT  activities,  the  long-time  school  librarian 
recently  wrote,  “Thanks  for  all  your  hard  work  on  Project  Impact.  It  was  a lot  more  fun  than  I had  thought 
it  would  be  and  I feel  like  I developed  some  close  relationships  with  our  faculty  that  I hadn't  had  before. 
That's  an  aspect  of  the  program  I think  gets  overlooked~the  relationship  part.”  What  a far  cry  from  the 
disgruntled  comments  made  by  a begrudging  participant  just  a few  months  earlier. 


Using  Video-  tape  Language  Samples  and  HyperStudio  Applications  to 
develop  literacy  (reading)  skills  in  young  children 
Ages  4-8  identified  as  developmentally  delayed  in  oral  language  and 

literacy  skills 

Denise  M.  Smith  Ed.D.,  CCCSLP 
Fidel  M.  Salinas  Ed.D. 

Children  who  are  at  risk  for  developmental  delays  in  the  area  of  oral 
language  are  also  at  risk  for  developmental  delays  in  reading.  Oral  language  skills 
provide  an  essential  foundation  for  the  emergence  of  literacy  competence.  As 
school-based  language  intervention  emerged,  combined  with  an  emphasis  on  a 
more  naturalistic  approach  to  literacy  instruction,  the  importance  of  strategies  such 
as  language  samples  have  increased  in  importance  and  utility  in  identifying  areas  of 
concern  and  devising  daily  lessons  for  improving  oral  language  skills. 

Advances  in  video  technology  have  improved  the  accuracy  of  language 
samples.  In  addition,  a video  provides  additional  information  such  as  body  language 
and  attentiveness,  not  available  in  the  more  traditional  approach  of  audio-tape 
language  samples.  Technological  advances  in  software  applications  such  as  Hyper- 
Studio have  the  capacity  to  utilize  pictures,  sound,  voice,  animation,  and  text  in  a 
combination  of  lessons  to  build  oral  language  skills  and  literacy  skills. 

The  children  who  participated  in  this  study  were  enrolled  in  a cross- 
categorical  non- graded  program  at  a local  regular  education  elementary  school. 
Their  age  range  was  four  to  eight  years.  All  the  children  were  ambulatory  and  age 
appropriate  in  the  area  of  self-help  and  gross/fme  motor  skills.  Cognitive  and 
language  skills  were  at  least  six  months  to  one  year  delayed  as  measured  by 
Brigance  Early  Screening  Inventory  of  Basic  Skills. 

University  students  enrolled  in  an  upper  division  (junior/senior  level) 
special  education  course  addressing  communication  challenges  in  young  children 
(pre-school  through  third  grade)  were  the  primary  investigators.  The  course 
instructor  and  media  specialist  at  the  local  elementary  provided  guidance  in  the  area 
of  analysis  of  language  samples  and  developmeit  of  instructional  sequence  of 
curricular  lessons  on  Hyper-studio. 

This  investigation  provides  information  in  the  following  areas : 

1)  information  relevant  to  normal  and  atypical  development  of  oral 
language  and  literacy  skills 

2)  strategies  to  analyze  language  samples 

2)  examples  of  young  children’s  language  sample  videos  and  Hyper- 
Studio lessons  and  provide  rationale  for  their  content 

3)  feedback  from  university  students  regarding  the  relevance  of  the 
assignment  and  its  practical  application  to  their  future  employment  as 
special  education  teachers. 
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Abstract:  This  paper  discusses  the  formative  evaluation  of  computer-based  research  tools  and 
methodology  used  in  a study  investigating  how  different  levels  of  interaction  in  a computer 
game-like  situations  affect  the  learning  of  spatial  visualization  skills.  The  current  study,  as  yet 
not  complete,  uses  Java  Applets  to  implement  interactive  puzzles  which  write  data  back  to  a 
server. 


There  are  three  research  questions  in  the  current  study:  How  does  interaction  with  a computer  versus 
observation  affect:  1)  mental  rotation,  2)  path  memory  and  3)  mental  imagery?  For  each  research  question,  we 
constructed  an  online  experiment. 

The  experiments  make  use  of  Java  applets  running  over  the  internet  that  send  data  back  to  a server 
machine  on  the  Stony  Brook  campus.  We  set  up  this  infrastructure  to  achieve  the  goals  of  conducting 
experiments  remotely  with  minimal  loss  of  data  and  to  provide  a quick  turn-around  time  for  developing  new 
experiments.  All  programming  has  been  done  by  undergraduate  computer  science  students  for  credit.  In  the 
treatments,  participants  were  randomly  assigned  to  either  1)  an  interactive  or  2)  an  observational  condition,  the 
posttests,  all  participants  performed  the  same  task  regardless  of  previous  treatment  condition. 

Experiment  1 was  designed  to  investigate  the  effect  of  interaction  on  mental  rotation.  The  scenario 
experiment  one  is  similar  to  the  classic  mental  rotation  experiments  of  Shepard  & Cooper  (1982).  Two 
polygons  are  shown.  The  polygon  on  the  right  is  either  a rotated  version  of  the  one  on  the  left,  or  it  is  reflected 
as  well  as  rotated.  The  participant  is  asked  to  specify  (by  pressing  a button)  whether  the  polygon  on  the  right  is 
1)  just  rotated  or  2)  reflected  as  well  as  rotated.  The  polygons  are  randomly  generated  shapes.  In  the  treatment 
condition,  the  interactive  participant  is  allowed  to  interactively  rotate  (with  mouse)  the  polygons  so  that  they  are 
the  same  orientation  and  it  is  easier  to  verify  whether  the  polygons  are  reflected.  In  the  observational  condition, 
the  polygon  on  the  right  rotates  by  itself  (animation)  until  both  polygons  align.  In  both  conditions  participants 
must  specify  whether  the  polygon  on  the  right  is  a rotation  only  or  reflected.  The  posttest  is  essentially  identical 
to  the  classic  mental  rotations  of  Shepard  & Cooper  (1982).  There  is  no  movement  or  animation.  The  polygons 
are  static.  The  participant  has  to  perform  mental  rotation  in  his/her  head  and  specify  whether  the  polygons  are 
rotated  only  or  reflections  of  each  other.  The  program  automatically  records  elapsed  time  and  whether  they  got 
the  correct  answer.  For  each  trial,  the  same  random  polygon  shape  is  used  in  both  treatment  and  posttest. 
However  in  both  treatment  and  posttest,  it  was  random  whether  the  polygon  on  the  right  was  a reflected  version 
of  the  polygon  on  the  left,  or  just  rotated.  Each  participant  did  ten  trials.  In  both  treatment  and  posttest, 
participants  received  feedback  on  their  success. 

Experiment  2 was  designed  to  investigate  the  effect  of  interaction  on  path  memory,  remembering  a 
path  through  a maze.  The  experiment  had  a computer  game  type  fantasy  of  finding  your  way  through  a maze  in 
the  darkened  interior  of  a pyramid.  It  had  a treatment  posttest  design.  In  the  treatment,  participants  are  shown  in 
top  view  a maze  in  the  dark,  i.e.,  they  can  see  only  their  starting  point.  There  were  two  treatment  groups,  an 
interactive  group  and  an  observational  group.  The  interactive  group  used  the  arrow  keys  to  maneuver  a 
Pacman-like  character  through  the  darkened  maze.  When  they  tried  to  go  in  a wrong  direction  into  a wall,  they 
were  alerted  by  a beeping  noise.  When  they  go  in  the  correct  direction,  the  square  they  move  to  is  lit  up, 
gradually  revealing  the  maze.  The  observational  group  views  an  animation  of  a Pacman-like  character  traveling 
through  the  darkened  maze,  squares  lit  up  as  the  character  enters  them.  Both  treatment  groups  received  the  same 
posttest.  They  are  asked  to  remember  the  maze.  Then  they  are  shown  a darkened  square  the  same  dimensions  as 
the  maze  and  asked  to  recall,  by  clicking  in  the  squares,  the  path  through  the  previous  maze.  They  receive 
feedback  on  whether  and  to  what  degree  they  remember  it  correctly. 
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Experiment  3 was  designed  to  investigate  the  effect  of  an  interactively  rotating  a shape  on  a mental 
imagery  task.  During  the  treatment,  both  treatment  groups  (interactive  and  non-interactive)  were  asked  to 
memorize  a random  polygon,  with  the  additional  hint  of  ”What  does  it  look  like?'*.  The  interactive  group  were 
allowed  to  interactively  rotate  the  shape  to  help  them  remember  it.  The  presumption  was  that  they  would  rotate 
it  find  an  orientation  that  looked  like  something  they  recognized.  The  non-interactive  group  were  shown  the 
polygon,  and  asked  to  remember  it,  without  being  able  to  rotate  it.  The  posttest  mental  imagery  task  was  to 
specify  whether  a "hole  in  the  ground",  seen  in  profile,  could  accommodate  the  memorized  polygon,  either  a)  a 
rotation  of  the  memorized  polygon,  b)  a reflection  of  the  memorized  polygon,  or  c)  not  at  all,  i.e.,  the  "hole  in 
the  ground"  was  the  shape  of  a completely  different  polygon. 

We  have  been  conducting  a series  of  formative  evaluations  of  the  research  infrastructure  with  approximate 
80  third  through  fifth  graders  (average  age  8.5  years)  from  public  elementary  schools  on  Long  Island,  New 
York.  The  formative  evaluations  have  taken  place  in  computer  labs  of  participating  elementary  schools  on  Long 
Island,  New  York,  and  also  in  computer  laboratories  on  the  Stony  Brook  Campus. 

Younger  children  (such  as  third  graders)  had  difficulty  understanding  terms  like  "polygons",  "rotated",  and 
"reflected".  We  therefore  changed  the  terms  to  "shapes",  "turned"  and  "flipped". 

One  of  the  problems  we  encountered  early  on  was  that  it  was  difficult  for  the  younger  students,  such  as 
third  graders,  to  read  the  instructions  and  understand  the  task.  We  therefore  decided  to  add  voice-over 
instructions.  Scripts  were  developed  for  the  voice-over  instructions,  and  recorded  with  a non-threatening  female 
voice  to  wave  file  formats  and  then  incorporated  into  the  Java  applets.  The  voice-over  instructions  clearly 
improved  participant  comprehension  of  the  tasks.  Some  participants  had  particular  trouble  understanding  what 
was  required  for  the  path-memory  experiment.  We  added  a fantasy  story  line  with  voice  over,  "you  are  lost 
inside  a dark  pyramid.",  and  visuals  of  the  inside  of  a pyramid.  The  fantasy  story  line  help  users  understand  the 
task  and  added  to  their  engagement. 

Participants  expressed  dissatisfaction  at  not  knowing  how  or  why  their  answers  were  correct  or  incorrect. 
We  changed  the  programs  to  include  audio,  written,  and  animations  of  shapes  to  provide  feedback  as  to  why 
answers  were  correct  or  incorrect. 

The  largest  problems  so  far  have  been  with  downloading  and  uploading  data  to  K12  computer  labs.  We 
programmed  the  communication  between  client  machines  and  server  using  Remote  Method  Invocation  (RMI). 
RMI  allows  the  Java  applet  operating  on  the  client  machine  (the  computer  of  the  user)  to  invoke  methods 
(programs)  on  the  server  machine.  This  implies  a "higher  level"  more  abstract  level  of  communication  than  the 
"http"  communication  used  by  most  web  pages.  Unfortunately,  after  running  a number  of  tests  at  different 
school  districts,  we  discovered  that  the  "firewalls",  proxy  servers,  and  other  filtering  software  in  place  in  almost 
all  K12  computer  lab  settings  simply  do  not  allow  RMI  communication  between  their  computers  and  a remote 
server.  We  converted  all  our  code  over  to  a "http"  communication,  the  method  of  communication  between  client 
machine  and  servers  that  the  majority  of  web  pages  use.  From  the  point  of  view  of  computers  in  K12  computer 
labs  (and  their  firewalls,  filtering  programs  and  proxy  servers),  it  appears  that  our  programs  are  just  calling  for 
and  downloading  ordinary  web  pages,  however  we  are  downloading  java  programs  and  uploading  data  from 
experiments.  During  a number  of  "dry-run"  tests  from  K12  computer  labs,  data  was  able  to  get  through. 

Next  during  the  first  major  period  of  experiments  in  school  computer  labs,  over  the  course  of  a week, 
approximately  100  fifth  grade  students  attempted  the  puzzles.  There  were  major  technical  difficulties.  With 
groups  of  20-25  students  on  the  running  the  java  applets  simultaneously,  the  download  of  the  applets 
(containing  a significant  amount  of  multimedia  content)  was  agonizingly  slow. 

With  20  to  25  students  running  the  Applets  at  once,  the  programs  froze  part-way  through  in  the  vast 
majority  of  cases.  We  only  able  to  run  the  programs  with  maximum  of  three  students  simultaneously.  And  even 
then,  very  few  were  able  to  proceed  in  through  to  the  third  experiment.  The  result  was  that  we  were  able  to 
gather  a significant  amount  of  data  on  the  first  experiment  only.  Mental  Rotation.  And  that  data  was  suspect 
because  of  the  high  attrition  rate.  One  cause  of  the  problem,  we  later  discovered  was  the  limited  capacity  of  the 
database  program.  Access,  we  are  using  on  our  server. 

We  have  not  yet  achieved  our  goal  of  setting  up  a web-based  infrastructure  for  developing  and  conducting 
experiments.  Various  technical  issues  at  K12  computer  labs  present  difficulties.  These  include  firewalls,  proxy 
servers,  filtering  programs  as  well  as  the  lower  quality  of  computer  hardware  at  schools,  compared  to 
universities.  There  are  other  problems  associated  with  undergraduates  computer  science  students  developing  the 
code,  such  as  continuity  between  semesters  and  incomplete  knowledge  of  some  of  the  technical  requirements. 
We  have  been  now  working  on  the  infrastructure  for  a year.  We  project  that  we  will  be  able  to  set  up  our 
experiment  infrastructure  within  a few  months. 
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Abstract:  Distance  learning  web-environments  do  not  provide  the  most  basic  communication 
tools  for  discussion  of  math  problem  solving  vital  to  math  courses.  A needs  assessment 
indicates  that  online  instructors  and  students  need  to  be  able  to  view,  edit,  and  post  diagrams 
and  formulas  directly  in  online  postings,  without  going  through  laborious  intermediary  stages. 
MathThread,  an  nsf  funded  project,  is  dedicated  to  providing  these  tools  to  online  instructors 
and  students,  supporting  instructors  in  adapting  their  classes  to  use  these  tools  and  researching 
the  effects  of  more  natural  math-oriented  threaded  discussion  tools  on  mathematics  learning. 
After  reviewing  the  existing  possible  tools,  we  have  formed  a partnership  with  NetTutor  who 
makes  the  Whiteboard  math  discussion  tools.  We  have  conducted  a pilot  study  evaluating 
undergraduate  students*  ease  of  using  the  Whiteboard  for  calculus  discussion.  We  found  that 
most  participants  had  little  trouble  understanding  and  using  the  tools. 


Current  distance  learning  web-environments  emphasize  the  written  word  and  threaded  discussions,  but 
do  not  provide  the  most  basic  communication  tools  for  discussion  of  math  problem  solving  vital  to  math 
courses.  In  face-to-face  math  classes,  instructors  work  problems,  write  formulas  and  d*aw  diagrams  on  the 
chalkboard  while  students  ask  questions,  providing  a feedback  communication  loop.  When  students  hand  in 
homework,  instructors  and  TA's  can  cross  things  out  and  write  other  formulas.  This  ability  to  demonstrate 
problem  solving,  represent  problems  through  diagrams,  and  provide  a communication  feedback  loop  are 
fundamental  to  math  learning,  yet  are  awkward  in  web-based  distance  learning  environments.  Currently  math 
notation,  as  well  as  diagrams,  can  not  be  directly  included  in  online  distance  ed.  documents.  Because  math 
notation  and  diagrams  are  NOT  integral  with  the  online  distance  education  environments,  instructors 
laboriously  go  through  several  steps  to  attach  an  image  file  with  formulas  and  diagrams.  Since  students  can  not 
respond  by  writing  math  notation,  this  disrupts  the  communication  loop.  In  a Sept.  2000  needs  assessment  of 
web-based  math  instructors  in  N.Y.  State  using  a dominant  web-based  environment,  instructors  declared,  "It 
has  been  extremely  frustrating  teaching  math  online...",  and  "Your  project  sounds  very  timely  and  useful ." 
Instructors  consistently  wished  they  could  put  equations,  math  notation,  conceptual  diagrams  and  graphs 
directly  into  online  postings  (i.e.,  a math  notation  editor,  a drawing  tool  for  creating  conceptual  sketches,  and  a 
graphing  calculator). 


Literature  search 

Current  web-based  distance  learning  environments  do  not  directly  support  formulas  and  diagrams,  in 
their  postings.  Logically  this  can  have  at  least  three  possible  effects  on  math  students  and  instructors.  Firstly,  it 
can  force  instructors  and  students  to  work  through  a number  of  intervening  digital  representations  in  order  to 
communicate  with  formulas  and  diagrams.  Online  instructors  must  do  this  extra  work  if  they  are  to  deliver 
mathematical  content.  Secondly,  students  and  instructors  may  avoid  this  extra  work  and  force  ordinary  written 
prose  to  carry  the  extra  burden  of  mathematical  meaning,  without  the  support  of  formulas  and  diagrams. 


Need 


Thirdly,  students  may  drop  or  shy  away  from  web-based  distance  mathematics  courses.  Instructors  may  avoid 
teaching  them. 

It  seems  intuitive  that  these  issues  interfere  with  student  mathematical  learning  outcomes  and  with 
mathematics  instructor  teaching  effectiveness.  Since  a plethora  of  intuitions  that  turn  out  to  be  scientifically  and 
empirically  incorrect,  here  are  some  social  science  concepts  which  lend  support. 

The  saying  "a  diagram  is  sometime  worth  ten  thousand  words"  is  supported  by  information  processing 
theory,  cognitive  psychology  and  cognitive  neuroscience.  A diagram  "preserves  explicitly  the  information  about 
the  topological  and  geometric  relations  among  the  components  of  the  problem"  in  a way  that  supports  parallel 
processing  by  human  beings.  Purely  verbal  and  symbolic  descriptions  force  humans  to  sequentially  parse  the 
relationships  between  elements  of  a problem.  So  while  verbal  descriptions  of  math  problems  may  be 
informationally  equivalent  to  diagrams  (all  relations  in  one  representation  are  inferable  in  the  other  and  vice 
versa),  there  are  many  cases  where  diagrams  are  computationally  more  efficient  than  verbal  descriptions 
(Larkin  & Simon,  1995).  The  eye,  retina,  visual  perceptual  system  and  visual  short  term  memory  are  object- 
oriented,  i.e.,  designed  to  efficiently,  and  in  parallel,  process  the  relations  between  different  parts  of  objects.  At 
a neurological  level,  human  visual  processing  has  two  well  developed  "streams"  which  process  "what"  (object 
discrimination)  and  "where"  (spatial  relationships  between  objects  and  features  of  objects)  (Ungerleider  & 
Haxby,  1998).  Both  streams  are  likely  used  in  parsing  diagrams.  Similarly,  while  the  capacity  of  Visual  Short 
Term  Memory  (VSTM)  is  commonly  held  to  be  four  items,  if  items  are  features  of  an  object,  instead  of  separate 
objects,  capacity  Jumps  to  at  least  16  (Jiang,  Olson,  and  Chun,  2000;  Olson  & Gettner,  1998)  i.e.,  if  elements  of 
a problem  are  presented  diagrammatically  as  features  of  an  object,  significant  learning  benefits  may  accrue. 

In  current  web  environments,  students  (and  instructors)  may  be  able  introduce  formulas/diagrams  in 
the  their  postings  by  going  through  several  intermediary  steps,  such  as  using  MathType  within  word,  saving  the 
file  as  a image  file  (GIF),  and  then  uploading  or  attaching  the  image  file  to  a posting.  But  such  extra  steps  in  the 
communication  process  cannot  help  but  distract  student's  limited  attend onal  resources  away  from  the  vital 
elements  of  new  schema's  they  are  learning  and  impose  an  extraneous  "cognitive  load"  on  the  students*  short- 
term memory  (Sweller,  1994). 


Previous  work 

There  are  a number  of  companies,  consortiums  and  organizations  working  on  supporting  online  math 
threaded  discussion.  WebEQ  provides  an  extremely  well-designed  interface  for  creating  mathematical  formulas 
that  go  into  documents  which  can  be  posted  to  the  web.  WebEQ  is  an  interface  that  has  grown  out  of 
MathType,  which  provides  an  add-on  to  Microsoft  Word,  to  enable  the  inclusion  of  formulas.  It  is  notable  that 
WebEQ  is  in  being  picked  up  by  some  distance  education  providers,  such  as  Blackboard.  Although  WebEQ  is  a 
well  designed  interface,  it  does  not  provide  for  the  inclusion  of  diagrams  or  the  graphing  of  equations. 

Another  approach  is  the  Imath  Communicator  (www.imath.org),  a math-enabled  browser,  which  displays, 
creates  and  edits  math  notation  and  some  diagrams,  by  using  Mathematical  Markup  Language  (MML)  an  XML 
extension  to  HTML  that  provides  for  math  notation.  Imath  has  modified  the  open  source  code  of  Amaya  (of  the 
Worldwide  Web  Consortium  (W3C))  to  create  Imath  Communicator.  Imath  has  been  used  somewhat 
successfully  as  a web  component  in  distance  beginning  algebra  courses.  The  advantages  of  Imath/ Amaya 
approach  are:  1)  it  is  based  on  an  extension  to  html,  so  all  browsers  may  support  this  solution  in  the  future,  and 
2)  it  is  open  source,  allowing  a diversity  of  programmers  from  around  the  world  to  potentially  improve  its 
source  code.  The  disadvantage  of  the  Imath  approach  is  that  it  does  not  perform  quickly  with  large  html  pages. 

LiveMath  (ivww.livemath.com),  is  a wonderful  way  to  display  interactive  three  dimensional  graphs  of 
mathematical  equations  over  the  internet.  Distance  educators  can  download  and  purchase  the  LiveMath  Maker, 
a program  that  allows  them  to  input  mathematical  equations  and  create  a variety  of  two  and  three  dimensional 
graphs  of  the  equations.  These  graphs  can  be  posted  to  web  pages,  where  users  can  view  them  after 
downloading  a free  plug-in.  The  user  can  interactively  rotate  and  view  these  graphs.  However  LiveMath  does 
not  provide  a means  of  putting  formulas,  diagrams  and  graphs  directly  into  online  threaded  discussion  postings. 
The  user  must  create  the  files  with  LiveMath  Maker,  upload  the  file  created,  and  then  link  to  the  file  from 
within  a threaded  discussion  environment.  The  interface  of  LiveMath  Maker  is  complicated  and  would  likely  be 
difficult  for  remotely  connected  undergraduates  to  learn.  It  is  perhaps  best  used  as  a valuable  supplementary 
resource. 

Probably  the  best  and  most  complete  solution  is  the  Whiteboard  by  NetTutor  (^w.nettutor.comJ . 
Whiteboard,  designed  as  a chat  system  for  math  and  predominantly  used  for  round-the-clock  chat-based  math 
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tutoring,  it  can  also  be  used  in  asynchronous  mode  for  math  oriented  threaded  discussions.  So  far  Whiteboard  is 
the  only  working  tool  that  allows  users  to  put  directly  into  asynchronous  threaded  discussions  the  most  vital 
elements  of  math  communicatio:  formulas,  graphs,  and  diagrams.  There  are  no  intermediary  stages,  just  an 
interface  that  allows  users  to  view,  edit  and  post  discussions  with  these  elements.  The  NetTutor/ Whiteboard 
could  use  some  improvements  on  its  interface.  The  company  that  owns  it.  Link  Systems,  is  relatively  small  and 
seems  amenable  to  making  improvements.  Since  Whiteboard  is  the  only  reasonably  complete  and  working 
product  supplying  the  needs  of  math  asynchronous  threaded  discussion,  we  have  chosen  Whiteboard  as  the  tool 
we  will  use  and  they  have  agreed  to  a partnership  with  us. 


Work  is  beginning  and  on-going 

The  Math  Thread  project  has  recently  been  funded  by  the  National  Science  Foundation,  through  the 
program  Course,  Curriculum,  and  Laboratory  Improvement  - Adaptation  and  Implementation  (NSF  CCLI 
A&I).  The  funding  starts  officially  January  2002  and  runs  for  three  years. 


Major  goals  and  approach 

1)  Provide  distance  ed.  math  instructors  with  basic  math  communication  tools  to  fluently  teach  math, 
including  direct  inclusion  of  math  notation,  conceptual  drawings  and  graphs  into  distance  ed.  postings  by  both 
instructors  and  students.  Instructors  will  directly  put  math  symbols  into  distance  ed.  documents,  without 
converting  documents  into  unmodifiable  formats  (such  as  image  files,  gifs,  LaTex  etc.)  and  adding  them  as 
attachments.  Our  goal  is  to  have  modifiable  math  notation  in  distance  ed.  documents,  so  instructors  can  easily 
copy,  modify  and  annotate  students'  work  and  thereby  achieve  fluent  of  two-way  math  communication.  Achieve 
by:  Unite  an  existing  distance  ed.  Environment  with  a math-friendly  interface,  NetTutor's  Whiteboard,  which 
creates,  views  and  edits  math  notation  and  diagrams.  2)  Adapt  existing  web-based  and  face-to-face  college 
math  courses  to  include  these  basic  communication  tools  for  math.  Achieve  by:  Through  the  SUNY  Learning 
Network,  a non-profit  infrastructure  for  distance  ed.  at  the  State  Universities  of  New  York,  redesign/deliver  two 
Intro.  Calculus  and  two  Pre -Calculus  courses  using  the  math  friendly  communications  tools.  3)  Provide  faculty 
development/support  for  instructors  to  re-design  their  math  courses.  Achieve  by:  During  redesign  of  adapted 
courses,  instructors  meet  with  the  project  personnel  and  receive  feedback  on  designing  online  math  classes  from 
other  instructors  at  SLN,  through  online  threaded  discussion.  4)  Research  the  impact  of  more  natural  math  and 
diagrammatic  communication  on  math  learning.  Achieve  by:  Once  adapted  courses  are  running  in  the  SLN 
environment,  research  students'  use  of  different  modalities  (e.g.,  diagrams,  graphs,  formulas)  in  their  learning, 
for  example,  the  correlation  between  diagrams  use  and  comprehensbn/achievement,  particularly  for 
developmental  students.  5)  Facilitate  the  quick  spread  of  basic  math  distance  ed.  communication  tools  to  other 
math  instructors  a)  in  the  Dept,  of  Applied  Math  & Statistics  at  STONY  BROOK,  b)  within  SLN  through-out 
the  state  of  New  York  and  c)  nationally.  Achieve  by:  During  redesign  of  adapted  math  courses,  approach  other 
math  instructors  at  SLN  for  suggestions.  Offer  on-campus  workshops  through  the  Center  for  Excellence  in 
Learning  and  Teaching  (CELT)  at  STONY  BROOK.  Provide  other  math  instructors  with  a simulation  of  a 
distance  ed.  math  course  using  easy  creation/viewing/editing  of  math  notation,  diagrams  and  graphs  within 
assignments  and  discussions.  During  teaching  of  adapted  courses,  allow  other  math  instructors  to  unobtrusively 
observe.  Suggest  that  other  math  instructors  use  MathThread  in  one  experimental  unit.  Present  at  the  yearly 
SUNY  Conference  on  Instructional  Technology  (CIT).  Approach  other  distance  ed.  providers  about  using  the 
basic  math  communication  tools.  Write  papers  for  peer-reviewed  journals  and  present  workshops  and  papers  at 
national  conferences. 


Validation 

In  a needs-assessment  of  40  web-based  distance  education  math  and  math-related  instructors  across  the 
nation,  the  instructors  declared,  'It  has  been  an  extremely  frustrating  experience  teaching  math  online...",  and 
"This  aspect  of  my  internet  course  is  the  most  discouraging.  I find  myself  spending  an  exorbitant  amount  of  time 
responding  to  students  mathematically  [because  the  infrastructure  does  not  support  it]  and  not  enough  time 
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teaching.  If  this  doesn't  change,  it  will  eventually  be  the  reason  why  I give  up  teaching  mathematics  on  the 
internet. " 

In  response  to  our  mentioning  our  plans  to  address  the  problems,  the  following  comments  were  made: 
"Your  project  sounds  very  timely  and  useful  ",  and  " Yes!  I am  so  glad  that  you  are  working  on  this  project. " 

The  instructors  consistently  asked  for  a common  set  of  tools,  i.e.,  the  ability  to  put  equations  and 
mathematical  notation  directly  into  online  pstings  and  threaded  discussions,  a drawing  tool  for  creating 
conceptual  sketches  and  a graphing  calculator. 


Math  Thread  Pilot  Study  of  NetTutor 

Prior  to  introducing  the  Net  Tutor  software  into  experimental  classroom  settings  it  is  important  to 
identify  human  and  other  factors  (such  interface  design  and  technical  usability)  that  might  influence  how  well 
the  software  would  function  as  part  of  a course.  Early  identification  of  such  factors  provides  opportunities  to 
modify  or  eliminate  them  prior  to  actual  implementation,  or  to  control  for  the  identified  factors  in  the 
experimental  setting  and  account  for  them  in  subsequent  project  assessment. 

A series  of  six  pilot  sessions  were  completed.  Students  who  had  taken  or  were  currently  taking  pre- 
calculus  or  calculus  were  recruited  to  participate  in  single  sessions  to  evaluate  the  use  of  the  Net  Tutor  software 
in  a naturalistic  setting.  Sessions  were  held  in  a computer  lab,  and  students  received  course  credit  for  their 
participation. 

In  order  to  decrease  any  anxiety  resulting  from  stereotype  threat  (Spencer,  Steele,  & Quinn,  1999; 
Steele,  1997)  and/or  from  concerns  about  self-presentation  or  mathematical  ability,  students  were  repeatedly 
assured  that  the  session  goal  was  to  evaluate  the  software  and  not  their  math  or  computer  knowledge  and 
expertise.  The  two-hour  pilot  sessions  had  four  parts.  First,  participants  were  provided  with  a facilitated 
software  tutorial,  during  which  they  could  practice  using  the  tools  and  ask  questions.  Second,  participants  were 
each  assigned  to  a small  group,  provided  with  a set  of  calculus-related  problems/tasks,  and  directed  to  work 
through  the  tasks  using  the  Net  Tutor  tools  and  online  threaded  discussion  with  the  other  members  of  their 
assigned  group.  Third,  participants  were  asked  to  complete  a brief  open-ended  questionnaire  regarding  their 
experience  using  the  software.  Last,  participants  took  part  in  a group  interview  regarding  the  use  of  the  Net 
Tutor  software  and  the  pilot  session  experience. 


Results  of  the  Pilot  Sessions 

In  general,  responses  to  Net  Tutor  were  relatively  positive.  Most,  though  not  all,  participants  had  little 
or  no  difficulty  understanding  how  to  use  the  tools  (“It  wasn’t  hard  at  all.”),  found  most  of  them  easy  to  work 
with,  felt  they  had  the  right  tools  to  do  what  they  wanted  to  do,  and  felt  there  ^\ere  enough  tools  provided. 
Several  participants  especially  liked  the  graphing  feature,  although  several  would  have  liked  a built-in  graphing 
calculator.  There  also  appeared  to  be  a consensus  that  there  was  a certain  awkwardness  and  tediousness  to  using 
Net  Tutor,  and  participants  suggested  that  they  would  feel  more  positive  about  the  software  if  it  functioned  in 
simpler  and  more  familiar  ways  (i.e.,  more  like  the  MS  Windows  applications  to  which  they  are  accustomed:  “I 
think  I would  also  like  key  commands.”).  Most  participants  indicated  that  they  believed  they  could  become 
comfortable  and  proficient  with  the  tools  as  part  of  a class  in  a fairly  short  time. 

Participants’  responses  to  questionnaires  and  during  interviews  indicated  a number  of  moderating 
factors  clustered  into  several  common  themes.  First,  participants’  perceptions  of  the  ease  of  use  of  Net  Tutor 
were  clearly  moderated  by  their  level  of  computer  expertise  and  experience  with  online  environments.  Those 
who  indicated  a higher  level  of  technical  expertise,  especially  in  the  use  of  graphics  software,  reported  less 
difficulty  in  learning  to  use  the  Net  Tutor  tools  and  in  navigating  the  software.  Also,  prior  experience  with 
online  threaded  class  discussions  (e.g..  Blackboard)  appeared  to  moderate  participants’  comfort  and  satisfaction 
with  the  group  problem-solving  tasks,  although  some  dissatisfaction  with  the  basic  attributes  of  this  function 
(e.g.,  read  messages  do  not  change  color  and  there  is  not  an  automatic  refresh)  appeared  to  moderate  even  the 
responses  of  computer-savvy  participants.  We  suspect  that  variability  in  students’  computer  expertise  will 
continue  to  be  a major  issue  in  computer-based  classroom  approaches  for  some  time  to  come.  Instructors 
implementing  such  approaches  will  need  to  build  in  adequate  time  for  training  and  tutorials,  especially  for  less 
technically  able  students. 

Second,  participants  who  perceived  themselves  to  have  less  math  expertise  expressed  some 


dissatisfaction  with  the  amount  of  information  provided  on  how  to  use  the  tools.  They  suggested  that  pop-up 
windows  that  provided  some  definition  and  description  of  the  tools  and  how  to  use  them  would  be  helpful,  as 
well  as  a glossary  of  relevant  mathematical  terms. 

A third  group  of  factors  clustered  around  personal  communication  preferences.  There  was  a fair 
amount  of  variation  in  whether  or  not  participants  thought  they  would  like  using  Net  Tutor  for  group  projects  or 
to  communicate  with  professors  and  teaching  assistants.  Some  expressed  a strong  desire  to  ask  questions  face- 
to-face  and  to  use  pencil  and  paper  to  draw  diagrams.  Others  felt  Net  Tutor  would  be  a useful  means  to 
communicate  questions  to  professors  and  teaching  assistants  outside  of  office  hours.  Instructors  implementing 
Net  Tutor  in  classes  would  be  advised  to  provide  adequate  opportunities  for  face-to-face  meetings  with 
students,  in  order  to  accommodate  a variety  of  communication  preferences. 

In  addition,  observations  by  the  project  staff  suggest  some  concerns  to  be  addressed  in  training 
sessions  with  instructors  for  the  experimental  classrooms.  Despite  repeated  reassurances,  some  participants  still 
suffered  from  self-presentation  concerns  which  might  be  expected  to  negatively  affect  their  experiences  with 
Net  Tutor  and  their  performance.  Instructors  need  to  be  aware  of  this  and  find  some  means  of  monitoring 
students’  participation  and  interactions  to  mediate  the  impact  of  self-presentation  issues  on  performance. 
Another  observation  involves  the  recognition  of  a learning  curve  for  instructors  as  well  as  students.  As  the  skills 
of  the  pilot  sessions  instructor/facilitator  improved,  she  noted  that  it  appeared  she  was  able  to  accelerate 
participants’  learning  because  her  increasing  knowledge  meant  she  could  more  readily  answer  questions. 
Instructors  should  be  aware  of  their  own  learning  curve,  and  strive  to  be  well  ahead  of  students  prior  to 
implementing  Net  Tutor  into  classes. 

Although  the  asynchronous  component  of  the  NetTutor  Whiteboard  seemed  fairly  usable,  there  were 
certain  technical  shortcomings  and  some  features  that  students  suggested  would  make  the  WhiteBoard  more 
usable.  In  using  NetTutor  for  a ascynchronous  threaded  discussion,  it  could  be  more  robust.  It  often  happened 
that  students  opened  up  two  windows  with  the  interface,  causing  the  browser  to  freeze.  Additionally  students 
called  for  a number  of  additional  features,  some  of  which  are  present  in  common  distance  learning 
environments  and/or  in  Windows -based  applications.  The  people  at  NetTutor  have  agreed  to  address  these 
issues  immediately.  We  feel  that  when  these  problems  are  fixed,  NetTutor's  Whiteboard  will  be  a very 
serviceable  tool  for  mathematics-oriented  asynchronous  threaded  discussion. 

These  pilot  sessions  yielded  useful  information  for  the  next  phase  of  Math  Thread,  identifying  and 
training  instructors  for  the  experimental  classrooms. 

Upcoming  work 

The  next  phase  in  our  project  is  to  train  and  support  several  online  mathematics  instructors  to 
incorporate  the  Whiteboard  tools  into  their  existing  courses.  We  have  made  a partnership  with  a community 
college  in  up-state  New  York  to  convert  a pre -calculus  and  a calculus  class  to  take  full  advantage  of  the 
Whiteboard  tools.  We  will  conduct  seminars  over  the  summer  of  2002  to  assist  them  with  incorporating 
Whiteboard  into  the  fabric  of  their  courses.  The  adapted  courses  will  be  offered  Fall  of  2002. 
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ABSTRACT 


Teacher  preparation  has  emerged  as  a critical  factor  in  the  effective  use  of  new  technologies  in 
education.  Information  technology  investments  may  not  pay  off  unless  future  teachers  are  given 
the  opportunity  to  become  technology-proficient  educators.  This  study  presents  a theoretical 
framework  for  implementing  technology  innovation  through  a cohort  model.  Results  from 
interviews  of  the  first  year’s  participants,  and  the  lessons  learned  from  their  efforts  are  reported. 
Findings  clearly  indicate  that  structure  and  ownership  are  essential  ingredients  to  producing  a 
viable  and  collaborative  cohort  model  and  that  the  power  of  technology  integration  can  be 
realized  through  collaboration. 

Introduction 

Rapid  and  widespread  technological  growth  over  the  past  decade  has  had  a significant  effect  on 
our  society;  technology  is  changing  the  way  we  live.  Changes  are  most  noticeable  in  the  way  we 
do  business  and  the  way  we  communicate.  Many  teacher  education  institutions  are  studying  the 
way  in  which  technological  innovations  can  significantly  influence  the  way  we  teach  and  learn. 
Today,  more  than  at  any  other  time  in  recent  history,  teacher  preparation  has  emerged  as  a 
critical  factor  in  determining  the  effective  use  of  innovative  technologies  in  education.  Federal, 
state,  and  local  agencies  are  investing  billions  of  dollars  to  equip  schools  with  current 
technologies  and  telecommunications  networks.  However,  these  information  technology 
investments  may  not  pay  off  unless  future  teachers  are  given  the  opportunity  to  become 
technology- proficient  educators  who  know  how  to  take  advantage  of  these  new  learning  tools  to 
enhance  and  improve  the  education  of  all  students. 

In  recognition  of  the  urgent  need  for  technology-proficient  educators,  the  U.S.  Department  of 
Education,  with  the  support  of  Congress,  has  begun  a new  initiative  to  prepare  tomorrow's 
teachers  to  use  technology  for  improved  teaching  and  learning.  In  its  first  year,  the  Preparing 
Tomorrow’s  Teachers  to  Use  Technology  (PT3)  program  awarded  $75  million  to  225  grantees  in 
every  region  of  the  country.  One  of  the  PT3  project  grantees,  the  Learning  Generation 
(LeamGen)  Project  at  the  University  of  Kansas,  focuses  on  the  abilities  of  teacher  education 


students  to  assist  faculty  in  generating  technological  innovations  and  to  transfer  this  expertise  to 
their  teaching  experiences. 

The  LeamGen  Project  has  sought  to  accomplish  this  goal  through  the  creation  of  Technology 
Innovation  Cohorts  (TIC).  The  role  of  each  TIC  is  to  design,  implement,  and  assess  a 
“technology  iimovation”  that  is  intended  to  integrate  technology  into  teacher  education.  This 
paper  presents  the  theoretical  framework  for  the  project’s  efforts  to  implement  a TIC  model  and 
the  lessons  learned  from  the  first  year’s  efforts. 

Literature  Review 

Over  the  past  decade,  reports  have  indicated  that  the  importance  of  technology  in  teacher 
education  is  not  central  to  the  teacher  preparation  experience  in  most  colleges  of  education  in  the 
United  States  today  (Office  of  Technology  Assessment  (OTA),  1995;  American  Council  on 
Education  Report,  1999).  Typically,  teacher  educators  are  not  prepared  to  model  appropriate  uses 
of  information  technology  or  to  incorporate  the  use  of  technology  across  the  curriculum.  Instead, 
the  extent  of  their  ability  to  use  technology  centers  on  accessible  applications  (e.g.,  drill  and 
practice,  word  processing,  use  of  the  Internet).  Teacher  educators  tend  to  lack  more  sophisticated 
skills  that  enable  them  to  integrate  realistic  problem-solving  activities  into  teacher  training 
programs  (AbdakHaqq,  1995;  Office  of  Technology  Assessment  (OTA),  1995).  If  technology  is 
to  be  successfully  integrated  across  teacher  education  programs,  preservice  training  that  is 
focused  on  technology  integration  must  become  a priority,  especially  if  preservice  teachers  are  to 
become  comfortable  and  competent  with  the  use  of  technology  in  their  teaching  (Hasselbring, 
1989;  Langone,  Malone,  Stecker,  & Greene,  1998;  Office  of  Technology  Assessment  (OTA), 
1995;  Sheingold  & Hadley,  1993;  Wetzel,  1993). 

When  technology  is  introduced  during  the  preservice  years,  the  focus  is  generally  on  “how  to” 
components  and  skill  acquisition  of  technology  rather  than  on  effective  strategies  for  using 
technology  to  enhance  learning.  Preservice  teacher  education  students  need  to  experience 
technology  as  modeled  by  their  instructor(s)  rather  than  simply  being  informed  of  its  “potential 
benefits”  (Barker,  Helm,  & Taylor,  1995;  Bryant,  Erin,  Lock,  Allan,  & Resta,  1998;  Good,  1996; 
National  Council  for  Accreditation  of  Teacher  Education  (NCATE),  1997;  Smith,  Houston,  & 
Robin,  1995). 

There  is  acknowledgement  among  researchers  that  mentoring  relationships  are  paramount  to 
successfully  assisting  beginning  teachers  in  adjusting  to  teaching  requirements.  Through 
supportive  national,  state  and  local  efforts  seeking  to  support  mentoring  relationships  between 
experienced  teachers  and  preservice  teachers,  entry-level  teachers  have  a chance  to  become 
better  prepared  at  facing  the  day-to-day  challenges  of  the  classroom.  The  trained 
mentor/observer  is  able  to  observe  and  analyze  teaching  and  provide  the  preservice  teacher  with 
substantive  and  specific  feedback  as  part  of  the  mentoring  process.  The  mentoring  relationship  is 
generally  organized  around  such  supportive  activities  as:  (a)  observation  (visiting  each  other's 


classroom);  (b)  demonstration  (coaching);  (c)  conferencing  (feedback);  and,  (d)  joint  preparation 
(Alderman  & Milne,  1998;  Gratch,  1998;  Hawkley,  1998).  Reports  indicate  that  effective 
mentoring  is  based  upon  interpersonal  communication.  Through  communication,  mentors  and 
mentees  can  devebp  along  the  three  phases  of  the  mentor/mentee  relationship:  (a)  establishing 
the  relationship;  (b)  getting  to  work;  and  (c)  evaluation  and  follow-up  (Alderman  & Milne,  1998; 
Gratch  ,1998;  Hawkley,  1998;  Hiemstra  & Brockett,  1998). 

Method 

Semi- structured  interviews  were  used  to  collect  data  about:  (a)  the  training  program,  (b)  the 
mentoring  experience  and  (c)  the  technology  integration  efforts  from  TIC  participants.  Audio 
taped  interviews  were  conducted  individually  for  all  participants  and  were  approximately  30 
minutes  in  duration.  The  twenty  interview  questions  were  broadly  grouped  into  three  categories 
that  included:  (a)  TIC  collaboration;  (b)  TIC  support  and  interaction;  and  (c)  attitudes,  comfort, 
and  ability  to  use  technology.  For  this  study,  the  data  gathered  included  only  the  personal 
experiences  and  opinions  of  the  TIC  participants.  Using  the  process  of  constant  comparison, 
responses  were  coded  and  sorted  according  to  themes  that  developed.  All  interviews  were  audio 
taped  and  transcribed  for  content  analysis.  To  check  the  reliability  of  the  interpretations,  an 
independent  evaluator  reviewed  all  the  recordings  of  interviews  to  confirm  quotes  and 
organization  of  patterns  of  participant  responses.  A second  evaluator  checked  and  coded  the 
transcribed  responses  to  ensure  reliability.  A further  comparison  was  made  of  the  two  reviewers' 
codings  and  organizations.  Member  checking  was  also  performed  to  ensure  the  credibility  and 
trustworthiness  of  the  data.  Participants  unanimously  perceived  the  presented  results  as  accurate 
reflections  of  the  TIC  experience  and  concerns  specific  to  integration.  The  interview  responses 
were  examined  and  partitioned  into  data  units  by  the  reviewers.  These  data  units  were  organized 
into  categories  established  from  specific  themes  that  developed  out  of  the  interviews.  These 
categories  were  grouped  directly  from  the  themes  to  organize  the  findings. 

Each  TIC  was  responsible  for  producing  products  or  resources  that  exemplify  their  innovations. 
These  might  include  detailed  descriptions  of  teaching  strategies,  lessons,  media,  technology 
competency  tutorials,  or  student  portfolios  for  example.  TIC  developed  products  provided 
evidence  of  the  TICs  application  of  instructional  technology.  Throughout  their  involvement  in 
LeamGen,  progress  reports  were  submitted  by  participating  teacher  education  students.  These 
reports  reflected  the  progress  and  growth  of  each  TIC  as  they  developed  their  innovations.  In 
addition,  each  TIC  provided  relevant  information  about  their  cohort  for  their  cohort  website  as 
well  as  submitting  one  or  more  examples  of  their  innovation(s)  to  be  disseminated. 

The  LeamGen  project’s  purpose  was  to  integrate  instmctional  technology  into  a teacher 
education  program  through  a collaborative  model.  Therefore,  the  work  produced  by  the  TICs 
was  reviewed  for  evidence  of  the  following  attributes:  collaboration;  use  of  technology;  the 
integration  of  instmctional  technology;  and,  emphasis  on  teacher  education. 


Discussion 

Emerging  from  the  analysis  of  the  TIC  products  and  interview  results  were  several  informative 
themes  from  which  we  were  able  to  identify  lessons  to  apply  to  future  TICs.  Generally,  the  use 
of  technology  within  TICs  occurred  in  one  of  two  ways.  The  first  was  to  use  technology  as  a 
means  to  facilitate  the  work  of  the  TIC.  In  this  example,  technology  was  secondary  to  the 
mission  of  the  TIC  and  would  be  used  to  record  or  produce  their  final  product(s).  A second  way 
in  which  technology  was  used  was  as  a component  of  what  the  TIC  was  developing.  In  this  way, 
technology  was  a primary  focus  of  the  TIC  and  the  work  of  the  TIC  was  centered  around  its 
integration  into  educational  settings.  This  delineation  of  roles  was  also  seen  in  the  way  in  which 
each  TIC  viewed  the  integration  of  technology.  Some  TICs  viewed  technology  integration  as  a 
key  component  to  laying  the  foundation  for  fUrther  technology  integration.  Other  TICs  viewed 
technology  integration  as  an  opportunity  to  provide  a direct  integration  of  technology.  These 
TICs  approached  their  innovation  as  a way  to  learn  new  technology  and  integrate  this  technology 
directly  into  the  educational  setting. 

Individual  progress  reports  indicated  that  the  TICs  typically  consisted  of  a teacher  education 
faculty  member,  two  teacher  education  students  and  one  or  more  K-12  educators.  Findings  from 
the  TIC  final  reports  (reflecting  their  efforts  and  providing  a description  of  their  innovations) 
suggest  that  no  two  cohorts  functioned  in  exactly  the  same  manner.  In  fact,  their  individual 
accomplishments  were  attributable  to  their  unique  relationships  brought  about  through 
collaboration.  However,  simply  considering  what  the  TICs’  “produced”  or  reported  did  not  prove 
sufficient.  Those  involved  and  the  type  of  interactions  that  occurred  varied  greatly  from  group  to 
group. 

TICs  approached  the  integration  of  instructional  technology  in  one  of  two  ways.  First,  on  a 
curricular  level,  several  TICs  explored  how  to  integrate  instructional  technology  into  specific 
courses,  such  as  what  technology  skills  were  needed  by  prospective  teachers  or  what  special 
educators  expected  of  teachers  that  were  to  use  technology  to  aid  with  adaptive  instruction. 
Second,  on  a technology  level,  several  TICs  sought  to  develop  technology-based  products 
suitable  for  use  in  a course  or  a K-12  classroom,  such  as  the  legislative  tracking  assignment  or 
the  review  and  application  of  educational  software  within  three  professional  development 
schools.  Since  the  TICs  work  typically  represented  a snapshot  of  the  K-12  or  teacher  education 
course  or  program,  it  was  difficult  to  determine  the  reach  or  quality  of  the  integration.  With  the 
review  of  the  software,  for  example,  while  this  was  requested  by  the  K-12  educators,  there  is  no 
record  if  teachers  in  the  three  schools  took  advantage  of  the  information. 

The  original  expectation  for  TIC  members  was  to  gather  and  decide  amongst  thernselves  what 
the  direction  of  the  TIC  would  be  and  how  they  would  accomplish  it.  This  presented  a challenge 
for  some  TICs.  A teacher  education  faculty  member  observed,  “I  don’t  think  that  we  had  enough 
information  about  the  expectations  of  a faculty  member  or  what  a TIC  should  accomplish. 
Contrary  to  our  original  expectations,  we  discovered  that  some  clear  definition  was  essential  for 


the  process  to  emerge.  TICs  that  were  formed  around  an  idea  generated  by  the  teacher  education 
faculty  member  proved  more  successful  than  those  that  sought  to  use  the  entire  TIC  to  generate 
the  idea.  In  addition,  TICs  that  based  their  work  on  the  initiative  of  the  teacher  education  faculty 
member  were  up  and  miming  much  quicker  than  other  TICs.  This  is  not  to  suggest  that  the  other 
TIC  members  were  peripheral  to  the  process,  but  rather  that  the  innovation  was  directed  both 
toward  the  teacher  education  faculty  member  and  a portion  of  their  responsibility  in  the  teacher 
education  program. 

Unanimously  supported  by  TIC  members  interviewed  was  the  need  for  an  informal,  yet 
systematic  process  of  forming  TICs.  As  a result,  several  changes  were  made  to  the  cohort 
development  process  for  the  second  round  of  cohorts.  In  order  to  accommodate  faculty  needs  and 
offer  faculty  an  opportunity  to  explore  their  interests,  one  of  the  co-principal  investigators  of 
LeamGen  took  on  the  role  of  faculty  liaison.  As  faculty  liaison,  he  is  able  to  offer  additional 
opportunities  for  open-ended  discussions  and  exploration  of  faculty  ideas,  needs,  and  interests  as 
they  relate  to  technology  integration.  The  results  have  been  promising  to  date. 

An  intended  outcome  of  the  LeamGen  project  was  to  develop  and  encourage  future  collaborative 
efforts  among  faculty,  students,  and  the  K- 12  community.  The  strongest  evidence  to  support  this 
was  seen  in  the  work  and  activities  of  the  teacher  education  students  following  their  involvement 
in  the  TIC.  While  unfortunate  for  the  education  profession,  two  of  the  TIC  student  members  have 
since  chosen  technology-related  occupations  in  the  business  world.  Other  students  have  taken  on 
additional  responsibilities  related  to  technology  either  through  their  involvement  within  the 
school  of  education,  or  as  members  of  technology-based  projects  throughout  the  school  of 
education.  Some  students  have  reassessed  their  careers  and  have  chosen  to  pursue  terminal 
degrees.  These  students  are  considering  careers  in  the  field  of  higher  education  and  technology 
leadership  (i.e.,  district,  administrative,  or  site-based  technology  integration  specialists, 
technology  support,  and  technology  training). 

Faculty  mentoring  was  evident  throughout  the  study,  however,  reverse  mentoring  was  much 
more  common.  As  one  teacher  education  faculty  member  put  it,  “It  was  nice  to  be  working  with 
someone  who  had  a technical  expertise  to  be  able  to  help  me  figure  out  how  to  take  ideas  that  I 
had  and... make  th[em]  a reality.  I think  if  I’d  had  the  same  level  of  technical  expertise  as  them 
[the  student].  I’d  question  whether  or  not  our  TIC  would  have  been  as  successful  as  it  was 
because  basically  I’d  talked  about  doing  the  programming  and  setting  this  thing  up  and  I was 
lost.”  Preservice  teachers  found  themselves  in  “enviable”  positions.  They  soon  found  validation 
among  their  peers  and  among  faculty  for  the  technical  expertise  they  possessed.  “In  some  way  I 
actually  became  a mentor  as  far  as  instruction  was  concerned.”  The  role  of  the  student  in  the 
TIC  process  was  unique.  This  is  seen  as  a strong  indicator  of  the  success  of  the  TICs  in  general. 
As  one  student  discovered,  “I  didn’t  understand  it  until  [faculty  TIC  member]  led  a conference  in 
Dallas.  He  had  mentioned  while  giving  a presentation  for  the  first  time  that  I was  “his”  mentor. 


He  totally  took  me  off  guard.  I hadn’t  really  thought  about  it  that  way.  I realized  that  I kind  of 
was  because  I was  the  one  that  was  teaching  technology  and  trying  to  integrate  it.” 

The  impact  of  the  incentives  for  student  participation  was  significant.  Incentives  took  the  form  of 
laptop  computers  that  were  assigned  to  each  student  to  use  as  needed  throughout  their 
involvement  in  the  project;  a small  hourly  wage  for  their  time  commitment;  and  exposure  to  a 
wide  variety  of  technologies  and  opportunities. 

Lessons  Learned 

1.  Faculty  ownership  is  critical.  Given  the  experiences  of  the  first  year,  those  cohorts  that  were 
formed  around  an  idea  generated  by  the  teacher  education  faculty  member  were  more 
successful  than  those  that  sought  to  use  the  entire  cohort  to  generate  the  idea. 

2.  An  mformah  systematic  process  of  forming  and  reviewing  cohorts  is  needed.  Cohorts  now 
are  required  to  complete  a seven-stage  process. 

3.  Mentoring  plays  a key  roll.  TIC  members  adopted  a mentoring  approach  to  contributing  to 
the  "supportive  environment"  of  the  TIC. 

4.  Technology  integration  increased.  Participants  reported  an  enhanced  understanding  of 
instructional  technology,  an  improved  capability  to  integrate  technology  into  higher 
education  and  K-12  classroom  instruction,  and  general  understanding  of  how  to  use 
technobgy  as  an  instructional  tool  to  meet  all  learners’  needs. 

Conclusion 

Throughout  the  study,  participants  reported  an  enhanced  understanding  of  instructional 
technology,  an  improved  capability  to  integrate  technology  into  higher  education  and  K-12 
classroom  instruction,  and  general  understanding  of  how  to  use  technology  as  an  instructional 
tool  to  meet  the  needs  of  all  learners.  Clearly,  this  is  evidence  that  technology  integration  across 
preservice  course  work  is  on  the  increase,  and  the  potential  for  widespread  integration  is  a 
possibility. 

While  the  long-term  success  remains  undetermined,  those  involved  have  come  to  understand  that 
the  power  of  technology  integration  is  best  realized  through  collaborative  efforts.  The  strength  of 
this  project's  efforts  rests  in  the  inclusive  and  collaborative  nature  of  the  TICs,  the  support 
system  provided  to  them,  the  level  of  faculty  ownership  and  the  design,  testing,  and  reporting  of 
their  technology  iimovation  efforts. 
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MERLOT:  Reaping  the  On-line  Vineyard 
Karen  Smith-Gratto,  David  Wicks,  and  Carl  Berger 

“Once  upon  a time  there  was  a faculty  member  in  a school  of  education  who 
wanted  to  involve  students  in  activities  incorporating  the  World  Wide  Web.  This  faculty 
member  wandered  the  on-line  vineyards  and  found  K- 1 2 vines  that  produced  lesson  plans 
and  even  content-related  activities  for  use  with  elementary  and  high  school  students.  The 
faculty  member  wandered  for  days  and  finally  found  a few  vines  that  contained 
information  appropriate  for  use  in  an  undergraduate  teacher  education  class  and  a few 
vines  appropriate  for  use  in  graduate  classes  in  education.  Only  a few  of  these  vines  were 
useful  for  what  the  faculty  member  was  planning  to  teach.  The  yield  from  the  harvest 
was  small  for  the  time  spent  in  the  search.  The  faculty  member  was  discouraged  and 
wondered  whether  the  incorporation  of  web-based  activities  was  such  a good  idea.  If 
only  the  faculty  member  had  know  about  MERLOT,  the  search  would  have  been  quicker 
because  the  types  of  vines  are  identified  and  would  have  often  been  reviewed  by  peers.” 

MERLOT  (Multimedia  Educational  Resource  for  Learning  and  On-line  Teaching) 
is  a database  that  can  assist  university  faculty  to  quickly  locate  sites  that  would  contribute 
to  the  instructional  process.  The  MERLOT  database  is  a collection  of  links  to  web  sites 
that  present  content  specific  information  within  a variety  of  content  areas.  The  objective 
of  the  database  is  to  provide  a collection  of  materials  that  are  peer  reviewed  for  the 
educational  efficacy  within  specific  fields.  In  this  paper,  we  will  specifically  address  the 
Teacher  Education  portion  of  the  database. 

The  database  serves  two  functions.  The  first  function  is  the  collection  of  sites 
useful  for  teaching  at  the  university  level.  As  anyone  who  has  searched  the  web  for 
materials  can  attest,  the  time  spent  searching  the  web  can  be  daunting.  By  collecting  sites 
submitted  by  the  developers  or  by  professors  who  have  used  the  sites  with  classes,  the 
database  provides  a timesaver  for  professors  engaged  in  instructional  planning  in  which 
the  incorporation  of  the  web  is  desired.  This  is  extremely  important  in  teacher  education 
because  professors  must  model  those  behaviors  and  skills  that  teacher  education  students 
must  learn  and  later  exhibit  in  their  classrooms.  As  with  any  group  of  professors,  teacher 
education  professors  teach  classes,  attend  committees,  engage  in  service,  and  produce 
scholarly  works  and  time  is  at  a premium.  Time  spent  surfing  the  web  for  appropriate 
sites  to  integrate  into  classes  can  take  time  away  from  these  other  activities. 

The  database  provides  not  just  the  name  and  a link  to  the  site  but  a brief 
description  of  the  site.  Sites  that  have  a Peer  Review  are  identified  with  a rating  of  up  to 
five  stars.  A compilation  of  the  reviews  is  developed  and  can  be  accessed  by  users  to  see 
not  just  specific  details  about  how  the  site  fared  but  also  information  about  learning  goals 
that  can  be  addressed  by  the  site;  the  intended  audience  and  other  possible  audiences  for 
the  site;  the  prerequisite  knowledge  and  skills  needed  to  get  the  most  out  of  the  site;  the 
type  of  material,  and  a note  about  special  technical  requirements.  The  inclusion  of  this 
information  can  help  the  professor  quickly  eliminate, sites  and  cut  down  the  number  of 
sites  he  or  she  needs  to  directly  preview  for  use.  The  database  also  provides  the 
opportunity  for  users  to  make  comments  and  to  describe  activities  they  have  developed  to 
use  in  conjunction  with  the  sites. 


The  second  function  is  to  provide  professors  who  have  developed  materials  to  use 
in  their  own  teaching  may  get  those  materials  reviewed,  in  a manner  similar  to  how 
journal  articles  are  reviewed. 

The  review  process  for  the  Teacher  Education  portion  of  the  MERLOT  database 
begins  with  submission.  The  MERLOT  database  is  an  open  database  in  which  both  users 
and  developers  of  sites  may  submit  sites  for  inclusion.  Teams  of  reviewers  analyze  the 
site  -for  it  quality  in  three  areas:  Quality  of  the  Content;  Ease  of  Use;  and  Effectiveness  as 
a Teaching  Tool.  The  quality  of  content  includes  not  just  accuracy  but  the  clarity  of 
presentation  and  relevance.  Ease  of  use  covers  such  characteristics  as  how  easy  the 
interface  design  is  to  negotiate  for  both  faculty  and  students,  how  engaging  the 
presentation  is,  and  how  interactive  the  site  is.  The  last  aspect  that  is  analyzed  is  the 
effectiveness  of  the  site  for  use  in  an  instructional  setting.  The  site  is  examined  for  the 
presence  of  objectives,  potential  for  integration  into  teacher  education  classes,  and 
instructional  flexibility.  The  usefulness  of  the  database  would  be  limited  if  the 
organization  only  viewed  the  collected  sites  through  “Teacher  Education”  without 
breaking  the  major  category  down  into  areas  within  the  field. 

The  Teacher  Education  portion  of  the  database  is  divided  into  subcategories  that 
were  hammered  out  and  agreed  upon  by  educators  from  around  the  United  States  and 
Canada.  The  subcategories  are:  Educational  Foundations;  Educational  Psychology; 
Classroom  Management;  Diversity  and  Multicultural  Education;  Educational  Research; 
Instructional  Technology;  Special  Education;  Student  Assessment;  and  Teaching 
Methods.  The  database  can  be  searched  through  the  use  of  these  categories  or  through 
the  use  of  descriptors  using  the  search  function  within  MERLOT. 

MERLOT  can  be  a great  asset  to  busy  faculty  from  two  standpoints.  First  when 
faculty  develop  web-based  materials  for  student  use,  those  materials  can  be  reviewed  and 
provide  the  faculty  member  with  documentation  that  parallels  the  peer  reviewed  journal 
article.  This  can  help  faculty  gain  recognition  for  the  scholarly  aspects  of  teaching.  In 
addition,  the  availability  of  the  materials  in  the  database  can  help  save  time  in  the  task  of 
incorporating  web-based  materials  into  our  classes. 
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Abstract:  This  paper  describes  a prototype  of  a development  project  for  practicum 
classes  in  the  Hong  Kong  Institute  of  Education.  An  in-house  development  of  a streaming 
video  system  is  used  for  improving  student  teaching  by  building  a digitized  video 
database  from  a variety  of  sources.  The  main  objective  is  to  develop  an  interactive  web- 
based  video  platform  for  critical  analysis  of  significant  classroom  activities  and  events 
related  to  IT  teaching  skills  and  strategies.  In  designing  our  video  system,  Synchronized 
Multimedia  Integration  Language  (SMIL)  is  used.  Video  clips  can  be  synchronized  and 
displayed  in  parallel  with  text  clips  showing  the  teaching  sequences  and  key  ideas. 
Viewing  and  critiquing  the  videos  side  by  side  in  an  on-line  setting  is  the  key  learning 
strategy  that  provides  opportunities  to  develop  student  teachers'  reflective  thinking  skills 
and  interactive  reflections  with  their  classmates  through  on-line  forum  discussions. 


Introduction 

In  the  practicum  and  methods  classes,  the  student  teachers  usually  conduct  trial  or  experimental 
lessons  that  are  prepared,  taught,  and  then  collectively  critiqued.  Each  period  can  only  highlight  a different 
teaching  skill  and  edited  video  tapes  are  sometimes  used  to  supplement  for  any  classroom  practice. 
Recently,  there  are  trends  of  reducing  the  contact  hours  and  increasing  of  class  size  in  IT  practicum  and 
methods  classes.  Within  these  classroom  constraints,  teaching  skills  taught,  variety  of  classroom  situations 
observed  as  well  as  number  of  participating  student  teachers  are  limited.  The  proposed  interactive  web- 
based  video  platform  for  critical  analysis  of  significant  classroom  activities  will  provide  repertoires  of 
specific  teaching  skills  as  well  as  diverse  classroom  situations.  Because  of  the  24-hour  availability  and 
accessibility  of  these  videos,  student  teachers  can  achieve  extensive  field  experiences  prior  to  teaching 
practices.  Researches  (Risko,  1991;  Stephens  & Leavell,  1999)  suggest  that  viewing  video  cases  student 


teachers  are  able  to  examine  the  dynamics  of  school  classrooms  and  benefit  from  multiple  perspectives 
when  viewing  is  done  collectively  with  classmates  or  other  colleagues  in  the  faculty.  Stephens  & Leavell 
(1999)  also  reported  that  video  minicases  which  are  short  vignettes  selected  as  examples  of  teaching 
practice  can  greatly  enhance  student  teachers  instruction  ability  since  the  vignettes  are  purposefully  selected. 
Accordingly,  student  teachers  are  able  to  compare  their  interpretations  with  those  of  others  when  viewing 
the  same  video  case  or  similar  vignettes. 


System  Overviews 

Traditionally,  student  teachers  can  reflect  on  their  own  teaching  performance  or  analyse  other 
practicing  teachers’  performance  through  videotapes  with  paper-based  observation  forms.  Preset  questions 
are  usually  provided  for  the  analysis  of  the  teaching  sequences  against  the  intended  objectives,  designated 
activities  and  learning  outcomes.  The  disadvantages  of  this  approach  include  the  following: 

1 . Physical  videotapes  are  bulky  in  handling  and  viewing; 

2.  It  is  difficult  to  annotate  important  teaching  characteristics  onto  the  videotapes; 

3.  Searching  similar  characteristics  from  the  repository  of  related  videotapes  are  not  easy;  and 

4.  Critical  reflection  by  student  teachers  cannot  be  easily  done  visually  and  interactively; 
Furthermore,  video  vignettes  can  only  be  produced  in  video  segments  which  cannot  be  retrieved  or 
managed  easily.  Therefore,  a digital  solution  with  online  viewing,  searching  and  critiquing  is  highly 
desirable.  Figure  1 illustrates  a schematic  representation  of  our  desirable  system.  A video  case  can  be 
viewed  alongside  with  key  ideas  explained.  Hyperlinks  for  related  video  clips  or  websites  to  support  the 
topics  of  interests  are  timely  provided.  Critical  reflections  related  to  the  video  segments  can  be  collected. 
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Figure  1.  A Schematic  Representation  of  the  Video  System 
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In  designing  our  video  system,  Synchronized  Multimedia  Integration  Language  (SMIL)  is  used 
(Alesso,  2000;  Kennedy,  2002;  Williamson,  2001).  SMIL  is  ideal  to  our  application.  Video  clips  can  be 
synchronized  and  displayed  in  parallel  with  other  media  information.  In  conjunction  with  a streaming 
server,  video  cases  can  be  retrieved  over  the  Internet.  Figure  2 shows  our  prototype  using  SMIL  and  its 
related  technologies.  A student  teacher  conducts  teaching  in  a practicum  class.  The  four  teaching  sequences, 
namely  introduction^  motivation^  development  and  consolidation ^ can  be  viewed  serially  or  in  any  order  by 
clicking  the  links.  Key  points  are  further  explained  and  annotated.  Also,  if  there  are  existed  other  video 
segments  related  to  the  key  points,  hyperlinks  are  provided  for  easy  access.  Figure  3 shows  the  related  clip 
with  the  student  teacher,  Chan  TS,  after  clicking  on  the  item  on  Figure  2.  Student  teachers  can  also  enter 
into  the  forum  for  critical  reflection  at  any  time  as  shown  in  Figure  4. 
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Figure  2.  A Sample  Screen  Shot  for  our  Prototype 
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Figure  3.  A Sample  Screen  shot  from  Selecting  "Chan  TS"  on  Figure  2 
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Using  streaming  technology  to  build  the  video  cases,  the  advantages  of  our  system  are  the  following: 

1 . The  video  cases  can  be  viewed  online; 

2.  We  can  directly  and  effectively  index  into  any  portion  of  the  video  clips  in  our  repository; 

3.  One  can  easily  control  the  timeline  and  important  points  can  be  re-visited  at  ease; 

4.  We  do  not  need  to  split  and/or  merge  clips  into  a single  clip  for  the  teaching  and  learning 
purpose;  and 

5.  Video  stream  can  retrieve  without  long  delay  over  the  Internet. 

Furthermore,  by  responding  to  the  on-line  reflective  forum,  the  student  teachers  can  thoughtfully  examine 
effective  teaching  principles  and  practices  of  each  video  case.  Viewing  and  critiquing  the  videos  side  by 
side  in  an  on-line  setting  is  the  key  learning  strategy  that  provides  opportunities  to  develop  student 
teachers’  reflective  thinking  skills  and  interactive  reflections  with  their  classmates.  Unlike  any  class 
discussions,  viewing  videos  in  a detachment  setting,  student  teachers  can  independently  develop  their  own 
reflections  about  a specific  teaching  skill  and  then  argue  with  their  peers  in  the  web-based  discussion  forum. 

Conclusion  and  Future  Work 

The  proposed  system  allows  student  teachers  to  reflect  on  any  key  events  posed  in  the  video  cases  as 
well  as  to  link  the  theory  into  the  practice  more  coherently.  The  flexibility  of  our  system  is  apparent  in 
comparison  to  the  conventional  approach  of  analysing  videotapes  for  teaching  practice. 

Our  future  work  on  this  system  is  to  build  another  "meta"  database  which  can  assist  us  to  generate 
the  content  of  video  cases  dynamically.  By  using  structured  information  of  the  database,  we  can 
automatically  compose  SMIL  scripts  and  related  files  on-demand. 
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Abstract:  This  paper  describes  an  ongoing  study  of  the  learning  quality  of  online  dialogues.  It 
suggests  the  unit  of  a language  game  as  suited  for  a learning  perspective,  which  captures 
netbased  collaborative  learning  as  a process  evolving  in  online  communities  of  practice.  The 
paper  suggests  some  possible  modifications  to  the  Global  Change  course  that  will  allow  us  to 
examine  in  a systematic  way  the  use  of  an  alternative  analytical  unit. 

Introduction 

Development  of  an  instructional  design  that  stimulates  and  supports  collaborative  dialogue  on  the  Web  may 
require  that  we  start  from  scratch  and  create  a design  that  emerges  ontologically  from  a true  integration  of 
technology  and  pedagogy.  Design  and  assessment  of  learning  processes  in  such  an  approach  should  be  based  on 
the  true  premises  of  the  virtual  environment  (Sorensen  1999).  If  we  think  beyond  simply  creating  an  online 
mechanism  for  dialog  to  creating  a framework  that  will  promote  high-quality  interaction  and  allow  for  relevant 
evaluation/assessment,  we  must  search  for  the  appropriate  analytical  unit.  In  order  to  do  so,  we  need  to  start  fro  m a 
clarification  of  the  learning  perspective  behind  the  design  and  construct  the  analytical  unit  from  this  perspective. 
We  present  a theoretically  based  conceptual  framework  based  on  the  notion  of  “collaborative  learning  in  online 
communities  of  practice”  for  understanding  and  identifying  collaborative  knowledge  building  dialogue.  This 
includes  identification  of  an  alternative  analytical  and  evaluative  unit  in  distributed  collaborative  knowledge 
building  on  the  Web,  inspired  by  the  concept  of  “language  games”.  We  also  discuss  implications  of  this  for  design 
learning  processes  that  allow  students  to  collaboratively  develop  “knowledge  tapestries”  through  meta  awareness 
of  how  such  language  game  structure  is  developed. 


Learning  Collaboratively  in  Online  Communities  of  Practice 

Our  previous  pilot  study  (Sorensen  and  Takle  2001)  focused  on  analysis  of  individual  comments  posted  by 
students  in  the  Global  Change  course  and  the  influence  of  setting  requirements  on  higher-level  thinking  skills  in 
advance  of  submitting  the  comment  on  the  web.  To  examine  this  issue  further,  we  posed  the  notion  that  our 
previous  focus  of  analysis  was  misdirected  and  that  we  should  focus  on  the  thread  or  collections  of  threads  rather 
than  the  individual  comment  as  the  analytical  elem  ent  in  evaluating  collaborative  learning.  Acknowledging  this 
shortcoming  of  our  previous  attempts  revealed  that  we  had  inadvertently  suppressed  the  social  element  of  dialog: 
our  over-emphasis  on  contributions  that  can  be  created  individually  (i.e.  without  collaboration)  was  suppressing 
meaningful  interchange  among  students  and  suppressing  opportunities  for  true  collaboration.  The  specific 
learning  perspective  we  want  to  promote,  “collaboration  in  online  communities  of  practice”,  is  rooted  in  both  the 
principles  of  Computer  Supported  Collaborative  Learning  pillenbourg  et  al.  1995),  and  in  a social  theory  of 
learning  taking  place  in  communities  of  practice  (Wenger  1998).  We  base  our  search  for  an  analytical  unit  on 
these  theories.  In  Wenger*s  perspective,  a social  theory  of  learning  must  necessarily  encompass  the  elements  that 
denote  or  characterize  social  participation  as  a process  of  learning.  Learning  takes  place  through  engagement  in 
actions  and  interactions  in  communities  of  practice.  Collaborative  learn  ing  principles  call  for  the  perspective  and 
functioning  of  group  learning,  while  Wenger*s  theory  points  to  the  learning  process  as  an  aspect  of  the 
functioning  of  a community  of  practice.  They  both  emphasize  learning  as  an  individual  and  a social  phenomenon, 
and  they  both  argue  for  shared,  collaborative  and  democratic  learning  efforts,  stimulated  through  participation, 
engagement,  motivation,  and  ownership. 


^^Language  Gaines”  - An  Alternative  Unit? 


Most  attempts  to  analyze  asynchronous  electronic  discourse  seem  to  lack  a theoretical  base.  The  results  of  such 
analysis  are  usual  pictorial  maps  describing  the  interaction  over  time.  Some  have  attempted  to  map  message- 
interconnection  through  analyzing  the  “threading”  to  make  expose  linkages  among  interactions  (Levin  et  al. 
1990;  Ellis  & McCreary  1985).  Such  studies  demonstrate  important  aspects  of  the  “threading  facility”  of  most 
computer -mediated  communication  systems  (CMC),  but  they  fail  to  go  beyond  the  simple  characteristics  of 
message-to- message  referencing.  Wittgenstein’s  (1974)  notion  of  language  games  offers  a holistic,  social,  and 
constructive  view  in  which  structural  coherence  forms  a key  point  for  establishing  any  meaningful  dialogue.  The 
theory  appears  suitable  for  capturing  essentially  the  interactive  and  dynamic  aspects  of  human  communication. 
One  of  the  attractive  features  of  the  language  game  theory  is  that  it  is  very  general  and  comprises  central  aspects 
and  properties  of  collaborative  actions  in  practice.  In  the  concept  of  language  games,  an  action  must  be 
interpreted  in  inextricable  association  with  the  activity  in  which  it  is  used.  We  build  up  expectations  during  the 
course  of  interaction,  which  are  based  on  our  knowledge  of  various  kind  of  activities  and  their  rules.  Viewing  the 
concept  of  games  in  relation  to  social  interaction  it  becomes  clear  that  a language  game  is  realized  as  a sequence 
of  actions  by  participants.  This  means  that  in  some  sense  a "player"  in  the  game  knows  what  to  do  when  another 
player  has  acted.  Actions  are  subject  to  specific  rules,  and  the  players  in  the  game  possess  knowledge  of  rules  of 
the  game.  Knowledge  of  the  rules  is  not  explicit  in  the  sense  of  awareness  in  the  players.  Describing  a sequence 
of  utterances  in  terms  of  games  is  equivalent  to  assigning  it  a structure.  In  particular  the  linguistic  interaction  (the 
dialogue)  process  can  be  understood  as  successive  openings  and  closings  of  language  games  at  different  levels, 
where  some  games,  subgames,  are  "embedded"  in  and  controlled  by  others.  Each  game  opened  creates  a set  of 
expectations  about  the  continuation  of  the  game.  The  important  role  of  these  expectations  is  that  each  comment  is 
interpreted  according  to  the  expectations.  The  framework  suggested  here  includes  the  important  elements  of 
dialogue  and  discourse.  It  is  in  agreement  with  a general  theory  of  social  action  and  comprehension.  It  supports 
the  view  that  human  interaction  can  only  be  interpreted  as  part  of  a social  context,  and  that  “negotiation  of 
meaning”  is  not  a “defect”  of  interaction,  but  is  rather  constitutive  of  it,  to  the  extent  that  specific  interactive 
mechanisms  exist  that  allow  mutual  understanding  to  emerge  (Dillenbourg  et  al.  1995,  p.  204). 


Language  Gaines:  Implications  for  Design 

Our  previous  attempts  to  establish  dialog  within  the  Global  Change  course  created  threads  that  had  related  ideas 
but  which  often  lacked  substantive  interconnections.  Students  rarely  joined  together  to  mutually  explore  an  issue 
in  detail.  In  some  cases  there  was  no  evidence  that  an  initiator  of  a game  returned  to  the  game  even  though  it  was 
clearly  his/her  “turn”.  And  rarely,  if  ever,  was  there  any  weaving  of  different  threads  to  form  “tapestries  of 
dialog”.  Threads  of  dialog  seem  to  have  a natural  tendency  (perhaps  exacerbated  by  our  demands  on  dialog 
quantity)  to  diverge.  We  seek  a solid  foundation  in  learning  theory  for  the  development  of  an  instructional  design, 
since  paradigms  designed  for  face-to-face  learning  environments  might  not  apply  or  might  overlook  some  unique 
opportunities  of  the  online  virtual  environment.  We  have  suggested  the  concept  of  language  games  as  a basis  for 
considering  the  issue  of  closure  as  well  as  development  of  dialog  threads.  We  offer  some  tentative  suggestions 
that  will  form  the  basis  for  some  of  our  experiments  with  the  next  offering  of  the  Global  Change  course. 
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Abstract:  This  paper  addresses  the  problems  of  collaboration  in  distributed  web-based 
learning.  It  reviews,  treats  and  discusses  these  problems  from  the  learning  theoretical 
perspective  of ’’communities  of  practice”  as  presented  by  Etienne  Wenger  (1998),  with 
reference  to  past  and  future  web-based  designs.  The  paper  suggests  the  concept  and 
design  of  virtual  portfolio  as  a pedagogical  tool  to  be  implemented  in  the  virtual 
environment  for  the  enhancement  of  distributed  collaboration  in  web-based  learning. 


Introduction 

It  is  widely  accepted  that  \\eb-based  learning  environments,  on  the  one  hand,  offer  promising  and  qualitative 
potential  for  distributed  CSCL  (Computer  Supported  Collaborative  Learning).  It  is,  however,  equally  shared  that 
they  present  us  with  complex  challenges  in  terms  of  student  engagement,  interaction  and  collaboration.  The 
problems  of  engagement  and  interaction  in  collaborative  processes  in  distributed  Web-based  learning  appear  to  be 
recurring  phenomena,  often  stated  by  the  literature  (Kaye  1992;  Mason  1993;  Sorensen  1997;  Dirckinck- 
Holmfeld  1990;  Fjuk  1998).  It  seems  to  be  a stubborn  feature  in  the  delivery  of  distributed  CSCL  on  the  web  that 
collaborative  initiatives  and  processes  - in  their  broadest  sense  - have  a hard  time  coming  into  existence 
(Sorensen  1997). 

The  searching  for  reasons  to  these  problems  and  ways  of  solving  them  have  produced  a variety  of  foci 
and  initiatives,  all  of  which  are  resting  on  different  assumptions.  The  role  and  communicative  behavior  of  the 
instructor  as  a unit  of  analysis  has  received  a considerable  amount  of  attention,  based  on  the  assumption  that  the 
teacher’s  behavior  in  the  virtual  learning  process  to  a large  degree  is  responsible  for  a lacking  engagement  and 
interaction  among  students  (Davie  1989;  Feenberg  1989).  Another  concern  has  addressed  the  nature  of  the 
collaborative  activities  implemented  in  the  learning  process,  hsed  on  the  assumption  that  a main  reason  for 
lacking  interaction  and  collaboration  most  likely  were  to  be  found  in  the  way  the  distributed  collaborative  tasks 
and  activities  were  designed  (Collis  1996).  There  is  no  doubt  that  both  of  these  research  perspectives  deal  with 
very  pertinent  aspects  of  virtual  learning  designs  and  deliveries,  but  it  is  equally  unlikely  that  they  constitute  the 
whole  story,  so  to  speak.  What  characterizes  these  perspectives,  however,  is  that  both  assume  the  main  cause  of 
the  problem  concerning  interaction  and  collaboration  to  be  situated  in  the  instructional  and  pedagogical  aspects  of 
the  learning  process.  There  are  alternative  studies  concerned  with  the  nature  and  quality  of  the  virtual 
environment  and  its  ability  to  support  collaboration.  One  illustration  of  this  is  the  design  of  the  Virtual-U 
environment,  which  can  be  viewed  as  a result  of  design  efforts  especially  directed  towards  facilitating  and 
scaffolding  collaborative  interaction  in  web-based  learning  (Sorensen  1999).  Other  perspectives  concerned  with 
collaboration,  e.g.  as  a broader  concept  than  linguistic  interactions,  most  often  are  to  be  found  among  software 
designers,  developing  shared  tools  like  e.g.  shared  documents  or  shared  whiteboards  to  facilitate  ’’tangible”  (non- 
verbal) collaborative  (inter)actions  in  web -based  learning. 

Discussions  around  collaborative  learning  designs  often  mirror  the  latent  perspective  that  collaborative 
learning  is  more  or  less  synonymous  to  collaborative  (linguistic)  interactions.  Only  few  studies  seem  to  address, 
both  the  linguistic  interaction  and  the  non -linguistic  collaboration  among  students  as  two  sides  of  the  same  coin. 
This  paper  argies  that  linguistic  interaction  and  the  more  tangible  carrying-out  part  of  the  interaction  cannot. be 
separated  from  each  other.  We  must  build  shared  communities  of  practice,  shared  frameworks,  and  shared 
histories  in  order  to  support  distributed  CSCL. 


o 
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This  paper  presents  the  hypothesis,  that  implementing  a shared  virtual  (group)  portfolio,  in  which 
linguistic  and  non -linguistic  collaboration  among  students  are  combined  and  situated  in  time  and  space,  will 
support  student  engagement,  interaction  and  collaboration  and,  thus,  enhance  genuine  collaborative  learning 
(Wenger  1998;  Salomon  1995;  Sorensen  1999).  Structures  and  frames  for  managing  shared  experiences  and 
shared  histories  must  be  built  into  a virtual  learning  environment  if  engagement,  sharedness  and  cdlaboration  in 
learning  are  to  flourish  and  unfold.  The  implemented  virtual  portfolio  structures  from  two  web-based  courses 
constitute  the  investigated  data.  In  one  of  the  courses  attempts  were  made  to  provide  structures  to  support 
especially  the  linguistic  interactive  part  of  the  group  collaboration  and  engagement.  In  the  other  course,  it  was  the 
non-verbal  part  of  the  group  collaboration  that  received  the  structural  support.  Section  2 of  the  paper  briefly 
introduces  the  special  conditions  of  a web-based  learning  process  using  some  central  concepts  from  the 
theoretical  perspective  of  "Communities  of  Practice,  Learning,  Meaning,  and  Identity”  (Wenger  1998),  Section  3 
provides  a review  of  potentials  of  using  the  virtual  portfolio  concept.  While  section  4 provides  a discussion  of  two 
different  types  of  virtual  portfolio  (VP)  for  collaboration,  section  5 forms  the  forum  for  our  conclusions  and 
future  perspectives. 

Distributed  CSCL  as  Virtual  Communities  of  Practice 

Collaborative  learning  between  students  is  widely  recognized  as  a fruitful  way  of  learning.  It  is  through 
collaboration  that  negotiation  of  perspectives  between  peers  takes  place,  a process  through  which  students 
reconsider  and  reflect  on  the  perspectives  of  their  fellow  students  and  accommodate  those  to  their  own  knowledge 
and  beliefs  (Fjuk  & Dirckinck-Holmfeld  1997).  Collaboration  in  learning  creates  a positive  commitment  that 
motivates  participation  and  drives  the  learning  process  (Illeris  1981).  But  collaboration  is  not  something  that  can 
easily  be  implemented  in  a learning  situation.  As  argued  by  Salomon,  collaborative  learning  demands  that  the 
whole  learning  environment  being  designed  as  an  orchestrated  whole  (Salomon  1995),  meaning  that  all  the 
entities  and  processes  in  the  learning  environment  (including  curriculum,  activities,  and  roles)  must  be  taken  into 
account.  Salomon  argues  that  the  design  of  the  technology  itself  is  the  least  important  factor  in  facilitation  of 
distributed  CSCL  and  that  computers  alone  are  not  likely  to  produce  a genuine  interdependency  that  creates  need 
for  sharing,  for  joint  endeavor  and  for  a pooling  together  of  minds  (Salomon  1995), 

In  principle,  we  share  Salomon’s  view.  Nonetheless,  one  should  not  eliminate  the  role  of  technology, 
especially  in  the  context  of  distributed  CSCL,  as  there  are  reasons  to  assume  that  the  less  people  meet  face  to 
face,  the  more  dependent  they  become  of  co  mputer  tools  and  their  quality  in  order  to  succeed  in  collaboration. 
The  establishment  of  interdependency  between  peers,  which  (according  to  Salomon)  is  a fundament  for 
’Collaboration,  is  also  dependent  on  the  availability  of  appropriate  tools.  Virtual  collaboration  gets  established 
through  technological  tools  allowing  minds  to  focus  on  the  same  problem.  This  means  that  the  medium,  on  the 
one  hand,  enables  a group  of  people  to  communicate,  and  on  the  one  hand,  allows  them  to  carry  out  non-verbal 
collaborative  actions  and  produce  shared  reifications  during  their  collaborative  work  process.  These  tools  are 
important  for  collaboration  in  general,  but  for  distributed  CSCL  they  are  essential. 

The  design  of  these  tools  demands  an  understanding  of  how  people  learn  in  social  contexts.  For  that 
purpose  we  turn  to  Etienne  Wenger  and  his  concept  of  “communities  of  practice”  in  which  he  conceptualizes  a 
social  theory  of  learning  (Wenger  1998),  According  to  Wenger  the  core  of  social  learning  is  the  continuous 
negotiation  of  meaning  between  participants  in  a practice.  This  negotiation  process  is  an  inseparable  part  of 
practice.  If  there  is  no  negotiation  of  meaning,  there  is  no  practice  to  be  part  of  The  negotiation  of  meaning  is  an 
intricate  process.  It  is  not  limited  to  linguistic  behavior.  It  also  includes  our  social  non-verbal  interactions  and 
relations.  Wenger  explains  the  negotiation  of  meaning  as  involving  two  constituent  processes:  participation  and 
reification.  These  two  processes  exist  in  duality,  affecting  each  other  and  being  the  source  of  development  to  each 
other: 

I will  use  the  term  participation  to  describe  the  social  experience  of  living  in  the  world  in  terms  of  membership  in  serial 
communities  and  active  involvement  in  social  enterprises.  Participatiai  in  this  sense  is  both  personal  and  social.  It  is  a 
complex  process  that  combines  doing,  talking,  thinking,  feeling,  and  belonging.  It  involves  our  whole  person  including  our 
bodies,  minds  emotions,  and  social  relations  (Wenger,  1998,  p,  56) 

Reification  ,,,  refers  to  the  process  of  giving  form  to  our  experience  by  producing  objects  that  congeal  this  experience  into 
“thingness”  , , .Reification  can  refer  to  both  to  a process  and  its  product  (Wenger,  1 998,  p,  58) 

But  since  production  of  meaning  is  distributed  xross  reification  and  participation,  a dynamic  relationship 
between  the  two  must  be  established  in  our  design  and  facilitation  of  learning.  If  not,  the  negotiation  and 
construction  of  meaning  may  become  problematic. 


If  participation  prevails  - if  most  of  what  matters  is  left  unreified  then  there  may  not  be  enough  material  to  anchor  the 
specificities  of  coordination  and  to  uncover  diverging  assumptions. (...)  If  reification  prevails  - if  everything  is  reified,  but 
with  little  opportunity  for  shared  experience  ad  interactive  negotiation  - then  there  may  not  be  enough  overlap  in 
participation  to  recover  a coordinated,  relevant,  or  generative  meaning  (Wenger,  1998,  p.  65) 

In  a community  of  practice  the  reificative  and  participatory  aspects  form  what  Wenger  denotes  as  a shared 
repertoire  A shared  repertoire  is  the  fundamental  resource  for  negotiating  of  meaning  in  a community.  It  is  a 
product  of  a community  of  practice  over  time,  including  routines,  words,  tools,  ways  of  doing  things,  stories, 
gestures,  symbols,  genres,  actions  or  concepts  that  the  community  has  produced  or  adopted  in  the  course  of  its 
existence,  and  which  have  become  part  of  its  practice.  Viewing  CSCL  as  a virtual  community  of  practice,  a 
challenge  for  design  is  to  apply  technological  tools  that  constitute  a shared  repertoire.  The  shared  repertoire  (the 
duality  between  participation  and  reification)  is  open  and  directly  available  for  negotiation  in  face-to-face 
learning  communities.  But  in  a distributed,  symbolic;  asynchronous  virtuality,  which  is  fundamentally  established 
through  technology,  the  processes  of  both  participation  and  reification  must  be  built  into  the  technological  tools. 

In  our  view,  the  lack  of  shared  repertoire  is  one  of  the  most  serious  shortcomings  of  most  CSCL- 
environments.  Later  in  this  paper  we  suggest  the  virtual  portfolio  as  a promising  concept  for  visualizing  the 
shared  repertoire.  The  virtual  portfolio  is  interesting  and  valuable  because  it  makes  explicit , both  the  negotiation 
(i.e.  the  participatory  aspects)  and  reificationof  meaning 

Virtual  portfolios-  Up  Until  Now 

Portfolio  has  become  a popular  educational  tool.  It  is  being  used  in  courses  reaching  from  basic  to  higher 
education  (Niguidula  1993;  Leeman  1997/98),  and  it  has  been  adopted  as  a tool  for  professional  development  and 
lifelong  learning  (Tenhula  1996).  Historically,  the  portfolio  concept  has  developed  from  artistic  professions  (e.g. 
architects,  designers,  models,  etc.).  In  artistic  contexts  portfolios  are  used  as  collectors  of  the  products  of  the 
artists.  Likewise,  a student  portfolio  is  a purposeful  collection  of  samples  documenting  the  work  of  a student, 
exhibiting  quality  and  progress. 

From  a pedagogical  point  of  view  there  are  two  basic  reasons  for  using  portfolios  (Alter  1995).  It  is  a 
powerful  tool  for  assessment,  and  it  is  a supporting  tool  for  structiring  and  giving  momentum  to  the  learning 
trajectory.  In  any  case,  both  of  these  aspects  are  meaningless  to  separate,  since  they  appear  to  be  two  sides  of  the 
same  coin.  Our  motivation  for  using  virtual  portfolios  (VP's)  in  design  of  distributed  CSCL  is  their  potential  for 
structuring  the  learning  trajectory  in  a way  that  highlights  learning  as  a constructivist,  experiential,  reflective  and 
social  matter.  As  the  concept  of  VP  is  a very  open  concept,  many  of  the  constructivist  approaches  and  netbased 
learning  environments  are  included  VPs  provide  useful  frameworks  for  discussing  and  designing  student 
centered  learning  environments  on  the  net. 

Advantages  of  using  virtual  portfolios 

From  the  pres  ented  perspective  of  learning  as  a phenomenon,  taking  place  socially  through  negotiation  of 
meaning  between  participants  in  communities  of  practice,  VP’s  represent  the  following  attractive  potential  for 
supporting  learning: 

• VP  is  activity  oriented,  as  activities  aiming  at  fulfilling  learning  goals  may  be  implemented 

• VP  may  support  student -centeredness,  if  the  VP  is  owne  d and  controlled  by  the  student 

• The  student  may  be  involved  in  the  assessment  process,  as  the  VP  supports  self  reflection  as  a learning 
activity.  By  letting  students  assess  their  own  progress,  students  become  shareholders  of  their  own  destiny,  so 
to  speak  (Sorensen  & Takle  1999) 

• Assessment  can  be  based  on  samples  demonstrating  authentic  work  focusing  on  the  student  as  a problem 
solver 

• By  focusing  at  different  entities  during  a time  period,  VP’s  make  it  possible  to  observe  individual  student 
progression  (Sorensen  & Takle  1999) 

• VP’s  are  artifacts  especially  suited  for  reification  of  meaning.  They  provide  the  learner  with  a ‘‘picture”  of 
the  accumulated  experiences  and  process  of  development.  Meaning  is  created,  reified  through  visualization, 
and  laid  open  for  individual  and  collaborative  reflection 

• VP’s  provide  views  and  structure  to  the  future  learning  process  by  prescribing  activities  ahead.  It  can 
visualize  participation  as  history  but  also  as  future  events.  Structuring  within  course  parts  helps  to  maintain  a 
more  detailed,  structured  overview  of  course  elements  and  supports  the  cognitive  processes  of  perception. 
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categorization,  classification,  recognition,  and  integration  of  details  of  a theme  or  phenomenon  into  a whole 
(Sorensen  1996). 

Nevertheless,  implementing  VP’s  do  not  automatically  guaranty  successful  learning.  As  default,  a VP  provides  a 
structure  of  a process  over  time,  but  the  quality  of  the  VP  for  supporting  collaborative  learning  in  online 
communities  of  practice  is  entirely  dependent  on  the  pedagogical  perspective  and  techniques  that  get 
implemented  in  the  VP  (Sorensen  & Takle  2001). 

Disadvantages  of  using  virtual  portfolio 
VP’s  also  have  some  fundamental  limitations: 

• VP’s  are,  traditionally  viewed,  individual  tools.  The  focus  is  on  the  individual  learner  as  a constructor  of 
knowledge.  One  may  argue  that  publishing  a virtual  portfolio  on  the  Web  will  enhance  collaboration,  but  it 
does  not  happen  automatically  that  sharing  of  knowledge  provides  collaboration.  If  we  want  VP  owners  ty 
get  engaged  in  each  other’s  work,  we  must  design  for  that  to  be  a VP  activity. 

• There  is  only  aminor  distance  from  supporting  to  controlling  the  learner.  VP’s  contain  the  potential  of  being 
tools  for  control  and  surveillance.  VP’s  may  be  used  for  controlling  what  the  student  should  lean\  and  how 
he  should  learn.  An  instrumental  pedagogical  approach  may  be  implemented  in  which  the  VP  activities  are 
shaped  as  tasks  with  predefined  answers,  instead  of  problems  to  be  solved.  Viewed  from  such  a perspective, 
there  is  some  risk  that  the  prtfolio  may  be  a tool  for  reproduction  of  knowledge,  instead  of  a tool  for 
supporting  democratic  learning,  marked  by  ownership,  personal  engagement  and  lack  of  authoritative  power 
mechanisms. 


Two  Different  Types  of  Virtual  Portfolio  for  Collaboration 

In  order  to  illustrate  the  relation  between  participation  and  reification  and  their  significance  in  design  of  a virtual 
community,  which  supports  collaboration,  we  present  two  different  implementations  of  VP’s,  both  inspired  by  the 
portfolio  idea.  The  first  example  is  an  implementation  that  strongly  supports  participation,  but  support  for 
reification  of  meaning  is  limited  by  the  environment.  The  second  example  is  designed  with  reification  as  the 
primary  goal.  In  this  case,  students  are  using  portfolios  for  constructing  their  individual  curriculum,  but  not 
seeking  participation  and  knowledge  construction  in  a social  practice. 


Participation  but  limited  reification 

The  first  example  of  a VP  is  provided  by  a one-year  distance  education  course  on  how  to  employ  ICT  in  learning, 
offered  at  Aalborg  University  in  Denmark.  The  course  was  delivered  mainly  as  distributed  CSCL  and  attracted  36 
participants  from  all  over  Denmark.  The  main  part  of  the  course  was  carried  out  over  the  Web,  using  the  virtual 
learning  environment,  Virtual-U.  An  additional  four  face-to-face  seminars  were  held  during  this  one-year  period. 
Project-based  group  work  constituted  the  main  activity  of  the  course.  The  supervision  of  the  project  work  took 
place,  both  at  the  face-to-face  seminars,  but  mainly  asynchronously  in  the  Virtual-U  environment.  Inthe  problem- 
based  project  work,  the  students  specialized  in  a particular  area  within  the  main  subject.  They  departed  from  a 
research  problem  that  they  themselves  identified  and  formulated,  a problem  that  was  related  to  their  concrete 
daily  work  and  work  interests.  The  problem  of  their  project  work  was  elucidated  through  study  of  relevant 
literature  and  different  types  of  data  collection  within  the  problem  area. 

The  students  were  provided  with  a communicative  forum  as  the  collaborative  space  for  their  project 
work.  No  other  structuring  “environmental”  facilities  were  offered.  As  argued  above,  a virtual  environment  is  not 
necessarily  a rich  environment,  and  in  order  to  get  more  overview  and  meaning  out  of  their  dialog  the  students 
were  encouraged  to  create  sub -conferences  as  their  need  arose(Fig.  1). 
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Figure  1:  The  VP  structured  by  means  of  conferences  (were  used) 
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Although  the  names  in  figure  1 are  all  in  Danish,  it  is  clear  that  project  group  2 created  five  sub -conferences:  one 
for  literature,  one  for  announcements,  one  for  project  guidance,  one  for  project  steering,  and  one  for  discussing 
cases.  Sub-conferencing  may  be  attractive  for  structuring  the  dialog.  But  it  is  not  an  ideal  resource  for  reifying  a 
meaning,  which  is  not  obvious  to  everyone. 

In  fact,  several  project  groups  demonstrated  severe  difficulties  in  using  sub  conferences  for  both 
participation  and  reification.  E,g,  the  example  below  (Fig,  2)  mirrors  the  collaboration  of  project  group  1.  They 
created  several  sub  conferences,  but  they  never  used  them. 

HumInf:Gruppe KonfefencerPrpjel^^  (0  new  of  1 09)- 
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Humlnf:GruppeKonferencer:ProjektGruppe_l:SparNord  (O  new  oH) 
HumInf:Gruppekonferencer:ProjektCjruppe_;l::Htterati4r  (0  new^of  G| 

Figure  2:  The  VP  structured  by  means  of  conferences  (were  not  used) 

Many  virtual  learning  environments  reify  meaning,  but  mainly  as  sequences  of  written  text  adapted  to  a dialogical 
structure.  A virtual  conference  may  mirror  the  negotiation  process  chronologically  or  sorted  by  several  criteria  as 
e.g.  date,  participants,  thread,  etc.  But,  as  argued  by  Wenger  (1998),  although  linguistic  interaction  is  essential,  it 
is  only  one  of  several  aspects  in  the  negotiating  meaning.  In  principle,  virtual  learning  environments  have 
provided  limited  functionality  and  support  for  genuine  group  coll  aboration  in  terms  of  reificatioa  In  general,  they 
provide  good  support  for  the  involved  participatory  processes,  but  not  necessarily  for  the  reificative  part  of  the 
learning  process. 
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Reification  but  limited  participation 

The  second  example  stems  from  a course  in  computer  science  from  Ostfold  University  College  in  Norway.  The 
subject  was  LAN  (Local  Area  Network)  and  Intranet.  I was  an  open  and  flexible  course,  which  was  attended  by 
both  local  and  remote  students,  in  total  120 participants.  The  course  was  carried  out  without  ordinary  lectures,  but 
was  organized  around  the  structure  provided  by  the  student  VP.  In  the  course  the  metaphor  of  a workbook  was 
used  (instead  of  the  term  “VP”),  as  we  wanted  to  use  the  metaphor  to  indicate  the  expected  work  style  (a  book  is 
supposed  to  be  thoroughly  prepared). 


LAN  and  Intranet 


0stfold  College 


1 M * U 


.--Si  Hi) 


Workbook 


Part  0:  Preparation 


Read  about  the  course  model  and  what  is  eKpected  from  you  as  a student. 

Doschbo  your  looming  goals  and  you  oxpectations  for  taking  this  course,  this  is  tho 
first  chapter  in  you  wortcbook. 


Implement  a draft  of  your  workbook  on  www  and  post  the  UAL  to  Hakon  Tolsby. 
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Below  follows  an  overview  of  the  structure  of  the  workbook.  It  is  divided  into  three 
parts,  which  in  turn  is  divided  into  throe  thomos.  This  provides  a structure  of  what  you 
are  expected  to  deliver,  but  it  is  up  to  you  to  give  the  workbook  its  final  shape  and 
content. 


Part  1:  LAN 

Themes  for  the  workbook 

Study-resources 

Overview:  Present  an  survey  of  the 
technology  of  can 

• Infrastructure  of  LAN 

• TCP/IP 

• LAN  based  on  NT  server 

Case:  Describe  a case  that  you  will  use  as 
an  example  for  implementing  a LAN 

Problem:  Choose  a particidar  problem 
within  LAN  that  you  will  discuss. 

Cab-eKerdse:  Describe  your  experiences 
from  the  lab-exercise 

• Preparations  to  the  iab-oxorcise 

* lab-exercise  i LAN 

Figure  3:  The  VP  structured  as  a workbook 
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The  workbook  was  divided  into  four  chapters,  one  preliminary  and  three  man  chapters  covering  different  aspects 
of  the  subject  matter.  In  the  preliminary  chapter,  the  student  was  supposed  to  describe  his/her  learning  goals  and 
motives  for  attending  this  course.  The  three  main  chapters  covered  important  aspects  of  the  subject  mater.  Each 
of  them  consisted  of  a set  of  themes,  which  structured  the  work  of  the  student.  Our  experience  from  the  course 
was  that  the  workbook  motivated  the  students  to  work  in  an  experiential  manner  (Kolb  1983).  They  used  the 
workbook  as  an  artifact  where  they  reified  their  understanding.  They  were  continuously  returning  to  the 
workbook  as  they  got  more  insight  and  elaborated  on  the  problems,  and  they  created  workbooks  based  on 
personal  experiences  and  interests.  They  also  browsed  each  other’s  workbook.  Thsy  compared  their  own 
workbook  with  others.  They  borrowed  ideas  from  each  other,  but  they  did  not  become  engaged  in  each  other 
projects.  There  were  no  real  interaction  or  social  participation.  They  were  sharing  experiences,  but  they  were  not 
contributing  b a shared  knowledge  building  experience.  There  were  ho  common  goals  for  sharing,  no  joint 
enterprises,  no  mutual  engagements.  In  other  words,  there  was  no  construction  of  a community  of  practice. 

Conclusion  and  Future  Perspectives 

In  this  paper  we  have  argued  for  the  need  of  tools,  sudi  as  VP’s,  to  support  the  processes  of  both  reification  and 
participation  in  the  negotiation  of  meaning  in  distributed  CSCL.  This,  however,  appears  not  to  be  a simple  task. 

Participation  and  reification  must  be  in  such  proportion  and  relation  as  to  compensate  for  their  respective  shortcomings. 

When  too  much  reliance  is  placed  on  one  at  the  expenses  of  the  other,  the  continuity  of  meaning  is  likely  to  become 
problematic  in  practice  (Wenger  1 998,  p.  65) 

Although  VP  has  been  developed  as  an  individual  tool  for  learning,  we  envision  that  the  concept  can  be  expanded 
to  comprise  also  the  idea  of  a “shared  VP”  (i.e.  for  a group).  A shared  experience  consists  of  both  individual  and 
collaborative  contributions,  and  by  collecting  them  and  making  them  visible  and  accessible  for  the  community  of 
the  group,  a shared  repertoire  for  collaboration  may  be  established.  The  shared  VP  includes  collaborat  ive 
constructions  and  collaborative  reflections.  A shared  VP  should  not  be  viewed  as  a substitute  for  the  individual 
VP.  On  the  contrary,  a shared  portfolio  is  dependent  on  the  personal  engagement  that  the  individual  portfolio 
provides.  You  cannot  share  and  you  cannot  collaborate  without  having  something  to  contribute  - and  without 
reflecting  on  what  you  contribute  And  in  a digital  environment  you  are  not  equipped  for  collaboration  without  a 
set  of  digital  resources  that  can  present  your  knowledge  and  experience.  However  individual  portfolios  have 
limited  value  to  a learning  community  if  they  are  not  shared  and  used  for  collective  reflection  and  development. 
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Abstract:  This  paper  describes  a new  methodological  approach  to  design  education  on 
the  web  employed  in  the  system  called  ALE.  The  system  will  be  used  to  build  a large 
knowledge  base  supporting  Architecture  and  Civil  Engineering  Design  Courses  and  to 
experiment  a comprehensive  Virtual  University  of  Architecture  and  Engineering  Design. 
The  ALE  system  integrates  the  functionality  of  a complex  ©Learning  system  with 
adaptive  educational  hypermedia  on  the  Web.  In  this  paper  we  outline  the  system 
architecture  and  focus  on  the  design  and  functionality  of  its  adaptive  learning 
environment. 


Introduction 

WINDS  (WINDS  2002)  is  an  ongoing  European  project  with  the  objective  to  implement  an  o-Leaming 
environment  called  ALE  integrating  an  intelligent  tutoring  system,  a computer  instruction  management 
system  and  a set  of  cooperative  tools.  We  have  developed  the  current  version  of  the  system  and  our  project 
partners  from  universities  prepare  more  than  20  on-line  courses  that  should  be  used  by  students  from 
autumn  2002. 

Today  there  are  relatively  many  web  based  educational  systems  (e.g.  Hyperwave  eLearning  Suite) 
including  commercial  ones  available  for  teachers  and  students.  On  the  other  hand  several  tools  for 
developing  adaptive  courseware  (e.g.  InterBook)  exist,  mainly  as  academic  developments.  But  it  is  not  easy 
to  find  a solution  that  would  integrate  the  functionality  of  a complex  ^Learning  system  with  adaptive 
educational  hypermedia  on  the  Web.  And  this  is  the  objective  we  try  to  address  developing  the  ALE 
system. 

For  reaching  our  goals  we  have  taken  into  account  several  of  the  existing  standards  and  semi 
standards  on  learning  objects,  learner  models  and  their  storage  and  exchange  (LOM  2001,  AICC  2001).  We 
build  on  our  experience  with  development  of  adaptive  hypermedia  environments  like  ISIS -Tutor 
(Brusilovsky  & Pesin  1994),  ELM -ART  (ELM -ART  1998),  and  ACE  (Specht  & Oppermann  1998). 

The  ALE  system  produces  individualized  courseware  for  students  depending  on  their  current  state 
of  knowledge,  their  preferences  and  learning  styles.  The  author  specifies  the  metadata  according  to  which 
such  adaptation  methods  like  additional  explanation,  prerequisite  explanation,  comparative  explanation, 
explanation  variants  or  sorting  can  be  implemented,  taking  into  account  the  user  model.  In  this  way,  the 
system  can  adapt  the  sequence  of  content  blocks  according  to  the  chosen  learning  strategy.  For  instance,  a 
concrete  example  can  precede  or  complement  an  abstract  statement  if  the  student  needs  it.  To  reduce  the 
cognitive  overload  of  the  learners  various  annotation  techniques  are  implemented. 

ALE  is  designed  to  support  various  learning  strategies  and  their  combinations.  Paragraphs  contain 
materials  for  expository  (explanatory)  education.  Discovery  learning  is  encouraged  not  only  by  hyperlinks 
but  also  by  index  terms  and  their  interconnection  with  learning  objects  and  external  documents. 
Collaboration  facilities  promote  constructivistic  learning  approaches. 

This  paper  describes  the  ALE  integrated  environment.  First  we  introduce  the  basic  modules  of  the 
system.  The  main  focus  is  on  the  adaptive  learning  environment,  especially  its  domain  model,  user  model, 
learning  object  states,  annotation  techniques,  as  well  as  the  interconnection  with  the  course  index  and 
external  documents.  Finally  we  summarize  the  paper  and  outline  future  perspectives. 


ALE  Modules 


A user  of  the  ALE  system  can  have  one  or  more  of  the  following  three  roles  assigned: 

• Student 

• Tutor 

• Author 

• Administrator 

Each  role  has  access  to  one  special  environment  - Learning,  Tutoring,  Authoring  and 
Administration.  Additionally  each  user  can  access  Workspace.,  Tools^News  and  Help. 

The  learners  are  not  only  passive  receivers  of  knowledge  but  can  themselves  contribute  to  the 
corpus  writing  private  or  public  annotations,  providing  feedback  to  the  author,  or  discussing  course  related 
issues  with  colleagues  and  tutors  in  a discussion  forum.  In  this  way  students  can  answer  some  questions 
themselves  and  can  cultivate  their  ability  to  formulate  problems  and  give  argumentations. 

The  main  tools  provided  by  the  system  for  collaboration  purposes  are: 

• Workspace  based  on  the  BSCW  system  (Bentley  et  al.  1997)  to  store  and  exchange  documents, 

support  common  projects  and  submit  homework. 

• Discussion  forums  created  by  authors  for  each  course. 

• Annotation  facilities  to  add  private  or  public  notes  to  study  materials. 

• Chat  facilities  (external)  for  synchronous  communication. 

An  important  feature  of  the  system  is  the  awareness  support  - the  user  can  see  which  other 
students  have  registered  for  a certain  course  and  who  is  currently  working  in  it.  The  underlying  principle  is 
that  the  system  presents  all  collaboration  tools  in  a thematic  context  of  the  current  learning  unit  or  index 
term,  i.e.  the  students  and  tutors  can  start  a discussion  about  a single  picture  or  a paragraph  in  a course  and 
create  a cooperation  folder  in  the  shared  workspace  for  solving  a special  exercise. 

Students  can  be  assessed  when  they  solve  tests  and  exercises.  Tests  are  evaluated  immediately  and 
automatically.  Various  types  of  tests  like  single-  and  multiple -choice,  gap  fillings  and  matching  pairs  are 
provided  and  some  others  are  planned. 

More  complicated  exercises  (e.g.  design  tasks  or  Bayesian  network  exercises)  are  carried  out  with 
special  external  tools,  which  should  be  installed  locally  on  the  student’s  computer  or  downloaded  as  an 
applet.  The  result  should  be  submitted  as  homework  to  the  student’s  folder.  Additionally  the  system 
provides  interface  for  external  learning  components  to  send  messages  to  the  learner  model  to  synchronize 
external  and  internal  learning  support  by  a consistent  learner  model.  These  interfaces  can  be  used  to  request 
information  about  a single  learner  and  statistics  about  the  class  progress  in  a course  and  to  submit 
information  about  users  from  peer  reviewing  procedures  or  tutor  reviewing  forms  or  automatic  exercise 
evaluations. 

The  types  of  learning  objects  in  ALE  as  well  as  detailed  descriptions  of  the  authoring  environment 
can  be  found  in  (Specht  2001a,  Specht  2001b).  Functionality  of  the  learning  environment  is  described  in 
(Specht  2002a,  Specht  2002b). 

Adaptive  Learning  Environment 

The  ALE  learning  environment  (Fig.  1)  is  both  adaptive  and  adaptable  (Brusilovsky  & Maybury  2002).  It 
means  the  system  can  automatically  adapt  to  the  user  given  a user  model  and  the  user  can  influence  the 
adaptation  by  means  of  the  preferences.  The  user  can  always  enter  a configuration  dialogue  to  specify  such 
parameters  like  the  preferred  language,  learning  style  (access  to  related  questionnaires  is  provided),  and 
media  type.  Based  on  this  information  the  system  selects  the  appropriate  learning  environment  components 
and  a suitable  way  of  presentation.  To  support  mobile  users  we  plan  to  enhance  the  system  to  take  into 
account  additional  parameters  like  computing  platform,  bandwidth,  location,  and  time  to  specify  the 
context  of  the  user’s  work. 

Adaptation  of  the  content  presented  to  a particular  user  depends  on  the  domain  model  and  the  user 
model  (Kobsa  2001).  Students  can  be  clustered  mainly  according  to  their  learning  styles  to  provide  certain 
types  of  learning  objects  with  higher  priority. 
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Figure  1:  ALE  displaying  a learning  element  with  highlighted  index  terms  and  the  first  of  them  explained. 

The  learning  environment  in  ALE  consists  of  the  following  parts  (Fig.  1): 

• Learning  object  display 

• Annotated  course  structure 

• Index  terms 

• External  documents 

• Student’s  progress 


Domain  Model 

The  domain  model  includes  the  learning  objects,  their  metadata,  and  relations  between  them.  The  learning 
objects  in  ALE  are  designed  to  support  reusability  and  flexibility  in  production  of  individualized 
courseware.  The  following  basic  types  of  learning  objects  are  provided:  course  units,  learning  units, 
learning  elements  (paragraphs,  tests),  and  index  terms. 

All  the  learning  objects  can  be  associated  with  a subset  of  the  Learning  Object  Metadata  defined  in 
the  IEEE  LOM  specification  (LOM  2001).  The  course  authors  can  also  specify  basic  relations  between 
learning  objects  based  on  a subset  of  the  Dublin  core  recommendations  (DCMI  2001).  These  will  include 
prerequisite  relations,  relations  between  course  units,  learning  units,  and  learning  elements.  Additionally  all 
ALE  learning  objects  in  a course  unit  will  be  linked  dynamically  by  the  underlying  index  defined  by  the 
course  authors.  Each  learning  element  has  its  own  didactical  goal  and  the  authors  can  select  from  a set  of 
pre-defined  templates  according  to  the  Cisco  strategy  for  reusable  learning  objects  (Cisco  2000). 


User  Model 

The  system  records  all  the  user’s  educational  events,  called  user  episodes.  These  episodes  include  events  of 
different  types: 

• Student  actions 

• System  evaluation  (tests) 


• Tutor  evaluation  (homework) 

• Student  evaluation  (self  assessment) 

Based  on  the  episodes  the  user  model  is  constructed.  It  includes  the  user  preferences  and  the  user 
knowledge  that  can  be  assumed  (seen  learning  objects)  and  verified  (by  the  system,  tutor,  or  student). 
Concerning  the  verified  knowledge  the  last  record  is  valid,  but  the  user  (student  or  tutor)  can  specify  the 
priorities  of  the  resources  for  knowledge  assessment.  Additionally  the  user  model  contains  also  the  user 
goals  specified  either  as  learning  objects  or  index  terms.  Later  on  as  we  adjust  the  application  for  mobile 
users  the  context  model  will  be  added  too. 

The  user  rrodel  always  reflects  the  current  state  of  the  user’s  progress.  The  information  is 
available  both  for  the  tutor  to  control  the  student’s  study  process  and  for  the  system  to  adapt  the  course 
presentation  and  navigation  for  the  student.  Recommendations  are  generated  according  to  the  user’s 
learning  style  and  previous  behaviour  of  similar  users  but  only  if  they  were  successful. 

The  states  of  the  learning  objects  in  ALE  can  be  considered  from  several  points  of  view  or  in 
several  dimensions.  The  system  distinguishes  three  of  them  that  are  relatively  independent:  user*s 
readiness,  interaction  history , and  tested  knowledge.  Additionally  some  other  characteristics  can  be  defined 
depending  on  the  followed  objectives,  e.g.  current  task  or  context.  Interaction  history  and  tested  knowledge 
can  be  expressed  by  quantitative  information  defining  the  extent  of  the  seen  learning  object  or  the  tested 
knowledge,  user’s  readiness  is  nominal  information  and  depends  on  the  prerequisites  specified  by  the 
author  and  on  the  interaction  history  of  the  user. 


Interaction  History 

Depending  on  the  history  of  the  user  interaction  with  the  system  a learning  object  can  be  seen^  partially 
seen^  or  not  seen. 

A learning  element  is  not  seen  if  it  has  not  been  accessed  yet.  If  the  element  has  been  accessed  it  is 
considered  2isseen. 

A learning  unit  is  not  seen  if  the  unit  has  not  been  accessed  yet.  It  means  none  of  the  learning 
elements  under  this  unit  has  been  seen.  A unit  is  seen  if  all  the  learning  elements  recursively  contained  in 
this  unit  are  seen.  A unit  can  be  also  partially  seen,  where  the  ratio  of  the  seen  part  can  be  specified  as 
SLE/LE,  where  SLE  stands  for  the  number  of  the  seen  learning  elements  and  LE  means  the  number  of  all 
the  contained  learning  elements  in  the  learning  unit.  In  both  cases  the  considered  learning  elements  are  not 
just  directly  but  also  recursively  contained  in  the  learning  unit.  The  student  can  see  both  numbers  to  have  a 
clue  about  the  size  and  state  of  the  learning  unit. 

Thus  this  state  of  a learning  object  can  be  generally  characterized  by  quantitative  information.  It  is 
the  most  general  information  about  the  learning  object  that  should  be  always  applicable. 


Tested  Knowledge 

A student  can  check  his  or  her  knowledge  by  means  of  special  learning  objects  called  tests.  Each  test 
checks  the  student’s  knowledge  that  can  be  related  to  one  or  more  learning  objects  specified  by  the  author 
of  the  test  (currently  each  test  item  can  be  associated  with  one  or  no  learning  object  - learning  element  or 
learning  unit). 

Generally  the  student’s  knowledge  of  a learning  object  (learning  unit,  learning  element  - 
paragraph  or  test)  can  be  expressed  by  quantitative  information.  This  state  of  learning  objects  is  applicable 
only  if  the  author  provides  corresponding  tests. 


User*s  Readiness 

If  the  author  has  specified  the  learning  object  prerequisites  the  system  can  distinguish  the  learning  objects 
for  which  the  student  is  ready  from  those  for  which  the  student  is  not  ready  according  to  the  interaction 
history  or  alternatively  tested  knowledge. 

It  is  nominal  information  that  defines  this  state  of  the  learning  object. 
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Adaptive  Techniques 

To  represent  various  states  or  dimensions  of  an  information  space  various  visual  emphasis  techniques  can 
be  used.  Some  studies  (Bertin  1967/1999)  of  visual  perception  recommend  the  following  visual  attributes 
for  different  kind  of  information: 

• Nominal  information:  texture,  color  (hue),  position 

• Quantitative  information:  length,  position 

Adaptive  navigation  support  techniques  are  employed  to  display  the  annotated  course  structure  in 
ALE.  The  different  states  of  learning  objects  are  represented  by  adaptive  link  annotations  using  icons  and 
alternative  texts.  Another  solution  we  use  illustrates  the  level  of  the  interaction  history  for  a learning  object 
by  color  (hue)  and  the  tested  knowledge  by  text  size.  It  can  be  considered  as  adaptive  text  presentation 
based  on  dimming.  Beside  the  selection  of  preferences  of  students  we  give  the  authors  and  the  tutors  of  a 
course  the  possibility  to  adapt  the  applied  annotation  techniques  depending  on  the  target  group  and  user 
features.  Tutors  can  specify  that  the  course  annotation  and  navigational  guidance  should  be  more  restrictive 
for  novice  users  (hiding  advanced  materials  with  missing  prerequisites)  and  less  restrictive  for  advanced 
users.  This  is  based  on  empirical  findings  and  post-hoc  studies  preformed  by  (Specht  & Kobsa  1999). 


Course  Index 

With  each  learning  object  corresponding  index  terms  are  displayed.  Index  terms  provide  means  to 
interrelate  heterogeneous  course  contents  and  to  find  individualised  paths  through  the  learning  materials. 
The  ALE  course  index  component  maintains  the  index  terms  together  with  their  respective  descriptions, 
synonyms,  different  types  of  relations  (e.g.  is_a,  part_of,  relatedjto,  is_similar_to)  between  terms,  their 
occurrences  in  the  course  materials  as  well  as  in  external  documents.  The  student  can  access  all  the 
information  about  a specific  term  by  choosing  it  in  the  Index  frame. 

The  index  component  can  retrieve  and  highlight  occurrences  of  index  terms  within  the  course 
materials  as  well  as  within  registered  external  documents.  We  consider  it  as  a step  towards  open  corpus 
adaptive  hypermedia  (Brusilovsky  2001)  that  is  a modem  trend  in  the  field. 


External  Documents 

External  documents  relevant  to  the  course  domain  explain  in  more  detail  some  specific  issues  or  provide 
up-to-date  information  like  specialized  portals  do.  Such  external  documents  serve  also  as  resources  for 
homework  and  projects.  These  materials  can  go  into  more  details  than  the  course  or  give  alternative  views 
of  the  domain.  In  the  Documents  frame  the  student  can  choose  an  external  document  related  to  the  current 
learning  object  and  view  this  document  either  with  or  without  emphasized  index  terms. 


Conclusions  and  Further  Research 

In  this  paper  we  gave  an  overview  of  the  current  state  of  the  ALE  system,  especially  its  adaptive  learning 
environment.  It  presents  learning  objects  as  adaptive  educational  hypermedia  courseware.  One  of  the 
primary  objectives  of  the  ALE  system  is  to  make  the  authoring  process  relatively  simple  and  to  support 
sharing  of  resources.  Currently  several  tens  of  authors  test  the  usability  of  the  system  and  their  comments 
and  suggestions  help  us  to  make  the  system  more  advanced  and  reliable.  The  experience  shows  that 
creating  study  materials  of  high  quality  with  proper  metadata,  relationships,  index,  and  library  is  a very 
complex  process  even  for  teams  of  skilled  authors. 

But  the  potential  of  the  system  is  high.  The  created  learning  objects  can  be  delivered  in  a variety 
of  ways,  from  classical  web-based  training  to  combinations  of  classroom  events  and  online  seminars,  or 
even  as  personalized  books.  Beside  the  possibility  for  cross  media  publishing  and  the  flexible  combination 
of  learning  objects  in  individualized  curricula  the  indexing  system  and  the  connection  of  the  learning 
objects  allows  for  personalized  coaching  of  students. 
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Abstract:  Librarians  and  faculty  working  together  through  Utah’s  Academic  Library  Consortium  (UALC)  are 
developing  a Web  site  to  share  library  instruction  and  information  literacy  competency  support  materials  statewide. 
These  materials  include  handouts,  guides,  tutorials,  interactive  exercises,  glossaries,  lists  of  Web  links,  and 
assessment  instruments.  Through  Web  based  sharing,  each  institution  saves  a great  deal  of  time  and  effort  by 
reducing  the  need  for  duplication  of  resources  and  increasing  the  quality  of  resources. 


Background: 

The  Utah  Academic  Library  Consortium  was  established  in  1973  and  has  developed  many  cooperative  programs  for 
collection  development  and  resource  sharing.  UALC  includes  14  academic  libraries  at  11  colleges  and  universities; 
nine  of  these  are  public  and  two  are  private  institutions.  The  mission  of  UALC  is  “to  cooperate  in  continually 
improving  the  availability  and  delivery  of  library  and  information  services  to  the  higher  education  community  and  to 
the  State  of  Utah.”  There  were  over  126,000  students  enrolled  in  UALC  institutions  in  Fall  2000.  Previous  UALC 
collaborations  have  involved  statewide  collection  management  for  serials  and  online  databases  (Pioneer  Utah’s 
Online  Library,  http://pioneer.lib.ut.us/l.  reciprocal  borrowing,  and  electronic  article  delivery  services.  One  of  the 
latest  programs  being  developed  is  a Web  site  to  share  library  instruction  and  information  literacy  competency 
support  materials  across  Utah.  This  project  grew  out  of  the  need  for  a statewide  clearinghouse  for  library  instruction 
materials  and  from  the  redesign  of  UALC’s  online  information  literacy  course.  The  Internet  Navigator 
(http://wwwnavigator.utah.eduL  The  Faculty  Resource  Web  site  also  builds  on  the  national  instruction  material 
clearing  house  model  developed  by  LOEX  (Library  Orientation  Exchange)  in  Michigan  in  the  1970’s.  As  more 
shared  databases  are  made  available,  library  and  Internet  instruction  needs  are  becoming  more  similar  across 
institutional  and  geographic  boundaries. 

Many  of  the  resources  on  the  new  Faculty  Resource  Web  site  were  originally  developed  for  the  Internet  Navigator 
online  course.  The  UALC  online  course  consists  of  four  modules,  each  with  its  own  exercises,  assignments,  quiz, 
and  glossary.  This  course  has  been  used  successfully  by  thousands  of  distance  and  online  learners  throughout  Utah 
and  beyond  since  1995.  A team  of  librarians  and  faculty  from  throughout  the  state  of  Utah  worked  during  Spring 
and  Summer  of  2000  to  completely  redesign  the  course  in  order  to  address  the  dynamic  changes  in  library 
information  environments  and  to  meet  the  latest  information  needs  of  students.  As  the  new  content  was  being 
developed,  the  UALC  team  members  realized  the  value  of  having  access  to  much  of  the  content  in  a variety  of 
formats  for  multiple  uses,  independent  of  the  larger  course,  as  needed.  A decision  was  made  to  develop  a generic 
header  and  provide  the  same  content  in  a “stand-alone”  format  for  use  in  any  other  traditional  face-to-face  or  online 
instruction  sessions.  This  was  the  beginning  of  the  Faculty  Resource  Web  site. 


UALC  Faculty  Resource  Web  Site: 

The  Web  site  idea  grew  from  a few  handouts  to  a more  comprehensive  clearinghouse  of  instruction  materials. 
Although  librarians  and  faculty  have  shared  materials  in  the  past,  the  Web  site  makes  this  much  easier  and  more 
effective.  A graphic  designer  was  hired  to  design  a generic  layout  for  the  materials.  The  Web  site  contains  three 
main  sections,  plus  a suggestion  form.  These  sections  are: 
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Instructors  Pages:  These  pages  are  designed  to  be  used  by  new  Internet  Navigator  instructors.  This  section 
provides  guidelines  for  teaching  the  course,  including  suggested  grading  rubrics,  links  to  the  grade  database,  and 
standardized  email  replies.  Most  of  the  links  in  this  section  are  password-protected  and  accessible  only  to  current 
course  instructors. 

Instruction  Handouts:  This  section  is  a clearinghouse  of  instructional  aids  designed  for  librarians  and  faculty  from 
all  institutions.  All  are  printer  friendly,  available  in  html  and/or  PDF  format,  and  can  be  downloaded  and  customized 
for  each  institution’s  needs.  As  this  page  matures,  guidelines  will  be  developed  to  catalog  information,  either  by 
developing  institution,  by  subject  area,  or  a combination  of  the  two.  Handouts  or  Web  readings  with  broad  usage 
generated  from  the  Internet  Navigator  course  materials  include:  Outline  of  the  Research  Process;  Using  Reference 
Tools;  Using  Internet  Search  Engines;  Using  Article  Databases;  Using  Library  Catalogs;  Creating  Search  Statements 
Using  Keywords  or  Controlled  Vocabulary;  Phtroduction  to  HTML;  Using  Boolean  Logic  and  Other  Techniques  to 
Enhance  Search  Statements;  Publishing  and  Types  of  Publications;  Critically  Evaluating  Information;  Scholarly 
versus  Popular  Information;  Note  Taking  Tips;  Documenting  Sources;  Creating  Annotations;  and  Phformation  Ethics 
for  Students.  In  addition,  several  interactive  exercises  have  been  developed  that  can  be  used  independently  of  the 
Navigator  course.  Most  of  these  are  developed  with  Flash  programming  and  require  student  interaction  to  complete. 
These  include:  Finding  Call  Numbers;  Citation  Builder  (APA  and  MLA);  and  a Web  Site  Evaluator. 

Instruction  Web  Sites:  This  section  includes  a collection  of  links  on  various  instructional  topics:  Phformation 
Literacy  Competencies;  Copyright  and  Fair  Use;  Resources  for  Distance  Learners;  Teaching  Tips;  Phstructional 
Technology  Sites;  Related  Professional  Associations,  Guidelines  & Standards;  Assessment  Materials  & Information; 
Teaching  & Learning  Styles;  Examples  of  Syllabi  for  Library  Instruction;  Assignment  Tips  & Examp  les;  and  UALC 
Library  Instruction  Pages.  Current  plans  are  to  limit  the  number  of  sites  listed  to  the  best  10  to  15. 

In  addition  to  the  collaboratively  developed  segments  of  the  Internet  Navigator  course  listed  above,  the  UALC 
Reference  andinformation  Literacy  Committee  has  solicited  a variety  of  additional  materials  to  be  included  on  the 
site  from  all  of  the  UALC  institutions.  These  include  handouts,  brochures,  database  guides,  and  other  instructional 
materials.  Some  of  these  materials  are  institution  specific,  but  are  included  as  useful  models.  Most  items,  with  the 
exception  of  the  password  protected  “Instructors  Pages”,  are  publicly  accessible  and  may  be  copied,  printed, 
downloaded  and  customized  to  meet  the  needs  of  any  particular  individual,  class,  course,  or  institution. 


Future  Plans: 

This  Web  based  collection  of  resources  will  continue  to  grow  and  expand  as  more  quality  materials  are  added. 

Future  plans  may  include  the  development  of  more  interactive  tutorials  using  dynamic  Web  programming,  a 
searchable  interface,  more  extensive  development  policies  and  guidelines,  usage  statistics,  publication  of  assessment 
data  and  other  studies  done  on  the  UALC  Internet  Navigator  course  and  this  Web  site. 


Conclusion: 

Information  literacy  instruction  in  Utah  is  becoming  more  similar  across  institutional  boundaries  as  we  continue  to 
purchase  more  shared  online  databases  and  as  students  access  other  shared  resources  such  as  online  article  delivery 
and  full  text  journals.  As  electronic  resources  and  instructional  pedagogies  change,  each  institution  spends  much 
time  and  effort  designing  new  learning  resources  and  updating  existing  ones.  Because  Utah  libraries  share  so  many 
of  these  resources,  there  is  also  quite  a bit  of  duplication.  Utah  is  fortunate  to  have  a history  of  cooperative  programs 
linking  librarians  and  library  faculty.  Sharing  instruction  materials  statewide  via  the  Web  is  a natural  and  cost 
effective  outcome  of  UALC’s  ongoing  cooperative  efforts.  The  UALC  Faculty  Resource  Web  site  will  serve  as  a 
central  location  for  updated  versions  of  electronic  resource  guides,  offer  variations  in  teaching  and  learning 
strategies,  and  serve  as  a center  for  new  instructional  ideas.  In  addition,  institute-specific  information  may  be  posted 
as  a model  for  others.  These  materials  will  continue  to  improve  and  expand  as  more  inter-institutional  Web  based 
sharing  occurs.  This  site  enhances  support  for  information  literacy  competency  across  the  state  and  region. 
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Hierarchy  of  Navigation  and  its  Effect  on  Content 
A Designer’s  Perspective 


Staples,  Cary.,  University  of  Tennessee,  Knoxville,  TN  37996 


Abstract: 

Over  the  last  3 years  the  author  has  worked  with  a team  of  Physics  faculty  + student 
designers  on  interdisciplinary,  web  development  projects.  Regardless  of  the  specific 
visualization  objectives,  ease  of  navigation  was  paramount  in  all  of  the  projects.  This, 
presentation  will  analyze  projects  that  contain  the  similar  attributes  of:  1.  large  data  set, 
and  2.  varied  presentation  requirements,  as  similar  design  attributes  were  manifested  in 
all  three  cases.  Even  though  the  authors  were  working  with  several  different  publishers 
and  different  design  teams,  the  analysis  of  the  information  to  create  an  intuitive 
navigation  system  yielded  similar  visualizations  and  hierarchic  organizations. 


Paper: 

In  this  case  study  the  author  will  present  three  design  projects:  Biology  + Genetics,  Web  Design  and 
Astronomy.  In  all  cases  the  intent  was  similar:  to  design  an  interface  to  facilitate  the  acquisition  of 
information  by  beginning  level  college  students  or  elementary  school  teachers.  The  interface  had  to 
accommodate  text,  imagery,  video,  animations  and  interactive  laboratory  tutorials.  In  short,  the  interface 
design  problem  was  how  to  create  navigation  that  left  enough  room  for  content. 

Different  groups  of  design  students  were  included  in  the  investigation.  All  three  sets  of  students  were  asked 
to  analyze  the  problem  without  the  benefit  of  viewing  the  other  teams  work.  All  three  teams  came  to  similar 
conclusions  regarding  placement  and  organization  of  content,  chapter,  unit  and  site  navigation, 
differentiation  between  tools  and  navigation,  use  of  a site  map,  use  of  “bread-crumbing”  as  a navigational 
tool  and  as  a design  element  as  well  as  typeface  use  and  hierarchy. 

Navigation  vs.  Tools 

For  the  purpose  of  this  experiment.  Navigation  + Tools  were  differentiated  and  defined  as  follows. 
“Navigation”  refers  to  the  resources  available  to  the  user  to  move  through  the  information:  either  from  level 
to  level  or  through  individual  pages  in  a specific  level.  “Tools”  refer  to  the  electronic  resources  available  to 
the  viewer  through  out  the  experience:  this  includes  calculators,  web  links,  glossaries,  etc. 

True  vs.  Evaluative  Audiences 

In  all  cases  navigation  and  tools  were  differentiated.  Navigation  was  considered  primary,  however,  in  all 
cases,  every  effort  was  made  to  make  the  navigation  “intuitive”.  This  posed  a design  problem  in  that  there 
was  a “true”  audience,  the  students  who  would  need  to  access  the  information,  and  an  “evaluative” 
audience,  faculty  and  publishers  who  had  to  approve  the  project  and  who  had  the  ability  to  make  global 
changes.  In  several  instances,  the  “evaluative”  audience  showed  itself  not  to  be  as  computer  literate  as  the 
“true”  audience.  In  terms  of  interface  design,  the  “true”  audience  was  much  more  experienced  in  non-linear 
navigation  than  the  “evaluative”  audience. 

In  figure  1 the  template  for  the  Genetics  project  is  outlined.  While  the  author  does  not  support  the  notion 
that  all  sites  should  have  the  same  gestalt,  the  author  does  present  a template  that  provides  a good  starting 
point  for  projects  with  large,  complex  information  sets. 
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Fig.l  I http^/csep  10.phys.utk.edu/klug/ 


Fig. 2 I http://sunsite.utk.edu/verbal 

Fig. 3 I http://csepl O.phys. utk.edu/webteacher2002/winexp/flashmenu/intro4.html 


Figures  2 + 3 illustrate  different  uses  of  the  same  organizational  principles.  While  text  placement  has 
moved  from  the  right  hand  side  to  the  left  hand  side,  approximately  2/3  of  the  field  is  utilized  for  primary 
information  and  1/3  is  used  for  support  information.  Imagery  is  considered  primary  information  in  Figure  2 
while  text  is  considered  primary  information  in  Figure  3. 


These  three  interfaces  also  illustrate  different  approaches  to  programming.  The  Genetics  site  was 
programmed  in  Flash,  the  Verbal/NonVerbal  site  uses  Java  and  the  WebTeacher  site  uses  html. 
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Over-hyping  the  Web:  Directories,  Gateways,  Resources  and  Higher 

Education  Pedagogy. 


Dr  Stuart  D Stein  (Stuart.Stein@uwe.ac.uk) 

Senior  Lecturer  in  Comparative  Genocide 
University  of  the  West  of  England 

Abstract 

The  paper  focuses  on  the  adequacy  and  role  of  freely  available  Web  resources  in  higher  education  courses  in 
the  social  sciences.  In  the  United  Kingdom  various  higher  education  related  funding  bodies  have  been 
responsible  for  channelling  resources  into  the  provision  of  electronic  information  system  resources  for  use  by 
academic  staff  and  students.  Part  of  these  has  been  directed  at  funding  the  development  and  maintenance  of 
subject  gateways.  This  paper  explores  some  issues  relating  to  the  quality  of  the  resources  provided  and  their 
likely  use  by  those  that  they  are  targeted  at. 

The  report  of  the  UK  National  Committee  of  Inquiry  into  Higher  Education,  the  Bearing  Report,  1997, 
noted  that: 

the  innovative  exploitation  of  Communications  and  Information  Technology  (C&IT)  holds  out 
much  promise  for  improving  the  quality,  flexibility  and  effectiveness  of  higher  education.  The 
potential  benefits  will  extend  to,  and  affect  the  practice  of,  learning  and  teaching  and  research. 
....  We  believe  that,  for  the  majority  of  students,  over  the  next  ten  years  the  delivery  of  some 
course  materials  and  much  of  the  organisation  and  communication  of  course  arrangements 
will  be  conducted  by  computer.  Just  as  most  people  will  come  to  expect  to  be  connected  to, 
and  to  make  use  o f,  world  communications  networks  in  their  daily  lives,  all  students  will 
expect  continuous  access  to  the  network  of  the  institution(s)  at  which  they  are  studying,  as  a 
crucial  link  into  the  learning  environment.  § 13.1,  13.3  (available  at 
http://www.leeds.ac.uk/educol/ncihe/.  accessed  26  April  2002) 

Long  before  this  report  was  published,  the  Higher  Education  Councils  had  been  investing 
significantly  in  the  development  of  broadband-networked  communications,  linking  universities  and, 
subsequently.  Colleges  of  Further  Education  to  each  other,  and  to  Internet  hubs.  Universities 
themselves,  partly  through  the  influence  of  the  funding  councils,  have  sunk  substantial  resources  into 
providing  various  electronic  information  system  resources  (EIS),  and  in  training  staff  and  students  in 
their  use.  The  European  Union  has  also  been  heavily  involved  in  providing  monies  for  research  into 
the  uses  and  application  of  EIS  in  institutions  of  higher  education,  under  its  telematics  and  other 
programmes. 

Access  for  UK  staff  and  students  to  EIS  resources  is  provided  principally  through  the  Joint 
Information  Services  Committee  (JISC),  which  was  established  in  1992  and  which  has  a budget  of 
approximately  £80  million  per  annum.  Much  of  the  online  content,  in  the  form  of  access  to  electronic 
publications,  and  electronic  databases/datasets,  is  provided  through  the  Distributed  National  Electronic 
Resource,  DNER,  which  serves  some  6.3  million  staff  and  students  in  the  UK.  In  the  JISC  five  year 
strategy  plan,  2001-05,  particular  emphasis  is  placed  on  the  role  of  the  Internet  in  enhancing  learning 
and  access  to  resources:  “The  Internet  has  become  vital  to  further  and  higher  education,  not  just  to 
enhance  distance  and  flexible  learning  but  as  a useful  tool  in  traditional  learning  and  teaching, 
administration  and  management  and,  of  course,  in  research  ...  teaching  staff,  managers,  administrators 
and  students  are  all  increasingly  using  the  Internet  from  home  as  well  as  the  office.  Use  of  the  Internet 
is  nearly  ubiquitous  within  further  and  higher  education,  as  it  has  been  for  some  years  in  the  research 
community.’*  (http://wwwjisc.ac.uk/pubQl/strat  Q1  Q5/exec.html  Accessed  1 1 March  2002) 

Another  type  of  resource  that  is  made  available  through  JISC  funding,  organised  by  the  Resource 
Discover  Network  (RDN),  are  subject  gateways,  or  hubs.  Subject  gateways  are  online  catalogues  of 
Internet  resources.  For  the  most  part  these  resources  are  freely  available  to  all  those  who  access  them, 
not  only  to  members  of  the  UK  higher  education  community.  These  gateways  provides  access  to  a 
series  of  Internet  resource  catalogues  that  list  what  are  considered  to  be  high  quality  Internet  sites  and 
materials,  selected  and  described  by  specialists  from  within  UK  academia  and  affiliated  organisations. 
Five  hubs  are  currently  running,  covering  the  Health  and  Life  Sciences  (BIOME),  Engineering, 
Mathematics  and  Computing  (EEVL),  Humanities  (Humb  ul).  Physical  Sciences  (Pslgate),  and  the 
Social  Sciences  (SOSIG). 


The  rationales  for  providing  these  services,  leaving  aside  the  vested  interests  of  those  funded  from 
public  sources,  were  stated  in  a paper  focusing  specifically  on  the  social  science  gateway,  SOSIG,  and 
are  twofold.  First,  that  “the  information  needs  of  social  science  researchers  can  increasingly  be  met  via 
the  Internet.  Electronic  information  resources  relevant  to  the  social  sciences,  including  data  and 
datasets  are  being  made  publicly  available  world-wide  and  the  Internet  offers  the  potential  for 
researchers  to  access  these  from  their  desktops.”*  Secondly,  that  these  resources,  as  they  can  be 
uploaded  by  anyone,  are  of  variable  quality,  and  that  staff,  and  particularly  students,  are  not  always  in  a 
position  to  evaluate  their  worth:  “Without  publishers,  editors,  peer-review  and  other  ‘filters’,  much  of  it 
may  be  invalid,  inaccurate  or  irrelevant  to  academics.”  Accordingly,  “the  gateway  provides  access  to 
Internet  resources  via  an  online  catalogue  where  each  resource  has  been  classified  and  described  by  an 
information  professional.  The  SOSIG  team  locate,  assess  and  describe  high  quality  networked 
resources  from  around  the  world,  adding  value  and  saving  time  and  effort  for  researchers  and  users  of 
their  research  by  providing  the  facility  to  browse  and  search  resource  descriptions  and  connect  directly 
to  resources  of  interest.”^ 

There  are  a number  of  problems  with  this  analysis  and  the  solution  that  is  provided  through  the 
medium  of  subject  gateways.  First,  when  Internet  resources  are  contrasted  with  printed  materials,  the 
underlying  assumption  is  always  that  the  latter  are  necessarily  superior  because  of  the  factors 
mentioned  above.  This,  of  course,  is  far  from  always  being  the  case;  libraries  and  journals  are  full  of 
materials  whose  worth  is  problematic.  Second,  the  cataloguing  is  largely  done  by  librarians,  or,  as  they 
are  now  more  commonly  known.  Library  Information  Specialists.  Such  specialists  are  unlikely  to  be 
sufficiently  familiar  with  the  detailed  teaching  or  research  requirements  of  academic  staff  in  particular 
fields,  or  to  posses  the  requisite  mastery  of  the  discipline  that  the  resources  being  evaluated  relate  to,  so 
as  to  be  in  a position  to  assess  their  adequacy,  other  than  on  the  basis  of  largely  superficial  surface 
characteristics.  A substantial  proportion  of  the  items  catalogued  by  subject  gateways,  SOSIG  and 
Humbul,  for  instance,  are  not  likely  to  be  of  much  use  to  either  research  students  or  academic  staff 
Third,  it  is  often  easier  and  faster  to  locate  useful  resources  in  a particular  subject  area  by  means  of 
effective  use  and  knowledge  of  search  engines,  and  by  links  from  key  sites,  than  by  way  of  trawls 
through  subject  directories.  Fourth,  the  surface  characteristics  relied  upon  in  the  process  of 
classification,  institutional  affiliation,  for  instance,  fail  to  take  into  account  that  many  of  the  resources 
uploaded  by  academic  staff  are  of  exceptionally  poor  quality,  and  should,  in  fact,  be  uploaded  on 
Intranets,  rather  than  the  Internet.  Fifth,  two  annual  reports  by  the  JISC  Scientific  Adviser  indicate  that 
neither  staff  nor  students,  in  significant  numbers,  are  using  any  of  the  EIS  resources  provided  by  the 
JISC,  including  subject  gateways,  despite  considerable  resourc  es  being  expended  on  publicity,  and  the 
fact  that  most  library  pages  of  institutions  of  higher  education  in  the  UK  reference  them. ^ The  reason 
for  this,  although  not  adequately  covered  by  the  monitoring  research  funded  by  the  JISC,  is  probably 
related  to  the  fact  that  neither  academic  staff  nor  students  have  the  requisite  time  to  do  so,  which  is 
reinforced  by  the  poor  quality  of  many  resources  freely  available  when  they  do.  Staff  and  students 
would  in  all  probability  be  better  served,  in  both  the  short  and  longer  term,  by  more  extensive  training 
in  the  effective  use  of  search  engines,  and  the  evaluation  of  online  resources,  than  by  the  funding  of 
subject  gateways.  These,  in  any  event,  cannot  conceivably  keep  up  with  the  volume  of  resources  being 
uploaded. 


* Debra  Hiom,  “The  Social  Science  Information  Gateway:  Putting  Theory  into  Practice.” ’98: 
Conference  Papers,  International  Conference,  25-27  March,  1998.  Available  at 
http://www.sosig.ac.uk/iriss/papers/paper54.htm 
^ Ibid 

^ Jennifer  Rowley,  JISC  User  Behaviour  Monitoring  and  Evaluation  Framework . First  Annual  Report, 
August  2000.  Available  on  Web  from  http://www.iisc.as.ulc  Jennifer  Rowley,  JISC  User  Behaviour 
Monitoring  and  Evaluation  Framework.  Second  Annual  Report,  August  2001.  Available  on  Web  from 
http://www.jisc.as.uk 
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Abstract:  A lot  of  software  products  currently  promise  to  support  eLearning 
but  only  few  efforts  are  done  to  detach  eLearning  from  wireline  desktop 
devices  toward  small  mobile  and  portable  ones.  This  paper  investigates  how 
this  kind  of  mobility  can  broaden  the  mind  of  eLearning.  It  is  pointed  out  that 
current  eLearning  products  can  be  classified  towards  a relatively  low  maturity 
level  of  eLearning.  Based  on  special  constraints  and  additional  benefits  of 
ubiquitous  learning  processes  are  emphasized  which  are  well  suited  to  be 
supported  pervasively.  The  Model-View-ControIIer  (MVC)  architecture  is 
recommended  as  a design  pattern  for  integrated  learning  solutions. 


Introduction  and  Motivation 

Economy  is  evolving  to  a knowledge  based  economy.  The  life  of  knowledge  and  human  skills  is  shorter 
than  ever.  Well  educated  and  skilled  stuff  and  as  a result  intellectual  capital  is  becoming  the  keystone  for 
organizations  to  get  and  stay  competitive  in  dynamic  markets.  For  employees  this  fact  increases  the 
pressure  to  remain  at  the  forefront  of  education  and  training  throughout  their  whole  career.  Four  or  five 
year  degrees  are  just  the  beginning  of  a life  long  continuing  education  (see  also  Urdan  et  al.  00).  For 
organizations  life  long  learning  of  their  staff  members  is  as  well  a competition  weapon  as  a cost  factor. 
Learning  has  to  happen  therefor  quickly,  situative  and  as  effective  as  possible. 

Technology  has  changed  the  way  we  live,  think,  work  and  learn.  Technology  has  revolutionized  business 
and  now  it  must  revolutionize  learning.  Life  classroom  based  training  is  becoming  too  costly  and 
cumbersome.  80  % already  use  computers  on  the  job.  So  eLearning  can  be  seen  as  a means  supporting  life 
long  learning  with  a lot  of  benefits  (see  also  Rosenberg  01,  30ff).  The  worldwide  web,  high-capacity 
networks  and  high  speed  computers  make  learning  available  to  people  24  hours  a day  in  their  office,  at 
home  or  also  in  hotel  rooms  during  a business  trip  around  the  globe.  ELearning  enables  it  to  access  learning 
when  it  is  convenient.  A lot  of  web  based  eLearning  solutions  are  available  on  the  market  jet.  As  we  will 
see  later  on  they  can  be  assigned  to  different  maturity  levels  of  eLearning.  But  one  thing  they  have  in 
common:  the  extent  of  their  ‘anywhere  and  anytime*  is  limited  mostly  to  resident  desktop  computers  with 
very  comfortable  user  interfaces.  Through  the  rapid  development  of  wireless  technologies  and  pervasive 
computing  (see  also  Burkhardt  et  al.  01)  also  the  topic  of ‘mobile  learning’  (mLeaming)  will  become  more 
and  more  significant.  Currently  eLearning  applications  running  on  small  and  handsome  devices  supporting 
wireless  technologies  or  integrating  wireline  and  wireless  eLearning  approaches  are  almost  not  existent  in 
the  marketplace.  This  paper  treats  features  learners  will  be  expecting  from  mobile  learning  environments 
and  gives  so  hints  for  the  further  development  of  appropriate  applications.  The  following  key  questions  are 
investigated : 

- What  are  the  maturity-levels  of  eLearning,  what  achieve  current  products  and  how  can  mobility  be 
anchored  in  this  context  ? 

- What  are  the  constraints  and  additional  benefits  for  wireless  learning  ? 

- What  learning  processes  are  well  suited  to  be  supported  wirelessly  ? 

- What  drives  wireless  eLearning  and  how  can  it  be  integrated  best  into  wireline  eLearning  ? 

- How  can  wireless  and  wireline  solutions  be  integrated  best  ? 

A conclusion  summarizes  the  key  statements  and  gives  some  future  perspectives. 
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The  Nature  of  eLearning 


For  the  most  people  learning  is  symbolized  through  the  classroom.  This  is  the  reason  to  see  eLearning 
mainly  as  the  online-version  of  classroom  learning  in  the  sense  of  eTraining.  Instruction  and  draining  is 
one  perspective  of  eLearning,  drilling  knowledge  in  form  of  structured  multimedial  content  presentations 
accompanied  by  active  elements  like  self  tests,  quizzes  and  various  communication  possibilities. 
Constraints  for  successful  eTraining  are  discussed  in  (Steinberger  01)  and  are  not  detailed  further  in  this 
paper.  Despite  of  its  powerfulness  eTraining  is  not  always  the  accurate  way  to  convey  knowledge. 
Learning  is  not  teaching  and  eLearning  must  not  be  viewed  only  from  the  point  of  view  of  eTeaching.  It 
must  be  thought  in  terms  of  what  the  learner  needs.  Often  there  exist  situations  in  which  learning  happens 
best  and  in  a very  efficient  way  through  selective  access  to  collected  knowledge.  Training  is  not  necessary 
in  this  situation  what  makes  eKnowledge  an  as  important  and  successful  aspect  of  eLearning  (see  also 
Rosenberg  01  and  Maurer  00).  Table  1 mentions  the  typical  characteristics  of  this  eLearning  approaches. 


Etraining 

EKnowledge 

Drill  of  knowledge 

Delivery  of  information 

Structured  nature,  less  flexibility 

More  flexibility,  random  access  nature 

Long  lead-time  for  development,  developed  by  special 
authors 

Created  also  by  expert  users 

Focused  on  a specific  learning  outcome 

Focused  on  a specific  access  and  view  of 
content 

Purpose  is  defined  by  designer  or  coach 

Purpose  is  defined  by  user 

Based  on  the  user  characteristics  and  needs 

Based  on  the  knowledge  discipline 

Sequenced  for  optimum  memory  retention 

Sequenced  for  optimum  reference 

Contains  presentation,  practice,  feedback,  communication 
and  assessment  components 

Primarily  centered  on  effective  presentation 

Table  1:  Characteristics  of  eTraining  and  eKnowledge 

There  exist  a lot  of  eLearning  products  on  the  market  supporting  eTraining.  Some  of  them  also  support 
knowledge  management  mechanisms  but  only  few  focus  on  user  knowledge  collection  and  access  in 
particular  (e.g.  Future  Learning  Environment,  Muukkonen  et  al.99)  but  mostly  knowledge  management 
systems  are  seen  as  systems  separate  from  eLearning.  Only  very  few  prototypical  eLearning  products  add 
the  possibility  to  augment  learning  also  to  small  mobile  devices.  But  in  most  cases  this  is  done  only  in 
excerpt  form  (e.g.  downloading  of  the  course  calendar  to  a PDA  in  WebCT). 

Depending  on  their  properties  eLearning  products  can  be  assigned  to  one  of  five  maturity  levels  of 
eLearning.  Table  2 shows  some  characteristics  of  these  maturity  levels  (see  also  Lytras  et  al.Ol).  The 
lowest  level  covers  pure  eTraining,  drilling  in  an  inflexible  teacher-defined  process  manner  locking 
knowledge  in  ‘course-boxes’  with  no  course  independent  random  access.  Level  2 enhances  level  1 through 
eKnowledge  functionality  and  makes  knowledge  available  independent  from  its  pertinence  to  a special 
course.  Level  3 makes  it  possible  for  the  learner  to  select  his  favorite  learning  process  which  makes  the 
content  delivery  more  flexible.  Moreover  level  4 and  5 enhance  the  learning  possibilities  trough 
customization  and  integration  aspects. 


Level 

Some  Characteristics 

1 

ETraining  with  static  content 
delivery 

Pure  course  orientation  that  supports  a predefined,  static 
and  sequential  way  of  learning;  the  poor  flexibility  is 
inadequate  to  support  dynamic  nature  of  learning;  Often  no 
interoperability  of  learning  objects;  No  random  access  to 
learning  objects; 

Content  producing  authors,  content  consuming  learners; 

2 

Enhanced  eTraining  coupled  with 
Knowledge  Management 

Utilization  of  learning  objects  databases;  Reusability, 
Metadata  , Interoperability;  Communication  Tools; 
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mechanisms 

Random  access  to  eLearning  objects;  Acquisition  of  user 
knowledge; 

3 

ETraining  based  on  Learning 
Process  Specifications 

Pool  of  available  Learning  scenarios  for  eTraining; 
Orientation  to  learning  processes  and  learning  stiles; 
Makes  content  delivery  more  flexible; 

4 

Customized  Learning  Scenes 

Learning  needs  recognition,  learners  profiling;  high 
flexibility,  extensive  parameterization  of  learning 
platforms 

5 

Integrated  Learning 

Integrate  eLearning  systems  with  other  enterprise  solutions 

Table  2;  Maturity  Levels  of  eLearning 

Most  of  today’s  eLearning  Products  are  settled  on  maturity  levels  from  one  to  two.  If  they  support 
knowledge  management  at  all  it  is  mostly  fixed  to  inputs  form  special  context  authors  and  does  not  provide 
user  centered  knowledge  production  and  distribution.  The  user  is  seen  almost  ever  as  the  knowledge 
consuming  partner  in  a learning  management  environment.  Learning  process  specification  and  customized 
learning  (levels  3 and  4)  are  still  research  topics  of  eLearning.  Anyhow  some  products  try  to  integrate 
learning  and  administrative  systems  via  a learning  portal  and  to  bridge  the  gap  between  level  one/two  and 
level  five. 

Borders  between  wireline  and  wireless  eLearning  are  basically  independent  from  the  maturity  level  of  a 
solution.  Mobility  could  also  be  realized  on  level  1 and  2 but  it  is  rarely  done.  Constraints  for  wireless 
eLearning  and  what  eLearning  processes  are  best  supported  in  a wireless  manner  is  discussed  in  the 
following  sections. 


Constraints  for  wireless  Learning 

In  future  we  will  see  a tremendous  increase  of  wireless  multimedia  network  terminals  that  are  similar  to 
today’s  mobile  phones  or  PDAs  (handhelds).  Wireless  technologies  like  wireless  LAN,  Bluetooth  and 
GPRS  will  multiply  potential  handheld  applications.  According  to  IDC  in  2000  12.9  millions  handhelds 
were  sold.  For  2004  they  forecast  63  millions  handhelds  to  be  sold.  Whereas  handhelds  today  are  primary 
sold  to  private  persons  IDC  believes  that  in  2004  about  30%  will  be  sold  to  organizations.  About  38%  will 
be  smartphones,  integrating  PDA  functionality  with  features  for  communication.  Also  the  possibility  to  use 
Java  (J2ME)  on  mobile  clients  will  contribute  to  the  success  story  of  the  new  mobile  devices  also  in  the 
area  of  mobile  wireless  learning. 

This  mobile  eLearning  devices  will  be  characterized  by  the  following  properties: 

- small,  portable  device  - has  an  integrated  camera 

- is  a full  fledged  PC  - ,Everynet‘  with  high  performance  is  always  available 

* has  a global  positioning  system  - serves  as  electronically  wallet 

* has  speech  in-  and  output  - has  an  integrated  graphic  pad  to  enable  simple  notices 

- serves  as  portable  player  - can  operate  without  socket-power  for  a long  time 

Because  of  the  nature  of  their  user  interface  these  powerful  mobile  devices  cannot  compete  with  the  user 
interface  of  standard  desktop  or  laptop  devices.  The  user  interface  is  one  of  the  key  constraints  for  wireless 
augmented  eLearning.  This  is  the  main  reason  why  mobile  learning  must  not  be  seen  as  a simple  wireless 
prolongation  of  eLearning  as  it  happens  on  desktop  or  laptop  machines.  Differences  regarding  requirements 
and  utilization  between  learners  using  small,  portable  devices  and  learners  using  desktop  devices  have  to  be 
seen  making  eLearning  mobile  (see  also  table  3). 

The  most  important  criteria  for  the  efficiency  and  acceptance  of  mobile  eLearning  are  (see  alsoZobel 
01,166ff): 


5 minute  value:  to  achieve  a benefit  for  learners  using  mobile  learning  devices  within  a short  time;  mobile 
devices  are  mostly  used  in  niche  times  to  communicate  or  get  information  as  quickly  and  simultaneously  as 
possible  (e.g.  use  a discussion  forum ).  For  a longer  session  the  desktop  device  is  mainly  preferred; 
Simplicity:  to  be  simple  and  adapted  to  the  limited  display  and  input  possibilities  of  the  mobile  device.  E.g. 
although  it  is  technically  possible  nobody  would  enjoy  to  scroll  complex  and  multimedial  content  files 
using  a PDA-like  interface; 

Additional  value:  to  produce  an  additional  benefit  as  compared  with  desktop  solutions;  one  important 
additional  benefit  of  mobile  solutions  could  be  the  usage  of  context  dependent  information.  There  can  be 
distinguished  between  four  kinds  of  context  in  which  a learner  can  be  located  (see  also  Zobel,  50ff): 

- location  context:  the  system  knows  the  location  the  learner  resides; 

- behavioral  context:  there  are  some  activities  related  with  the  location  the  learner  resides; 

- temporal  context:  there  are  dynamic  and  time  dependent  data  related  to  the  location  the  learner  resides; 

- interest  specific  context:  the  preferences  of  the  learner  are  addressed  in  a specific  manner; 

A pure  mapping  of  desktop  eLearning  functionality  onto  mobile  devices  will  fail  therefore.  It  makes  more 
sense  to  support  special  learning  processes  according  to  the  learner  needs  using  mobile  devices  and 
boosting  the  strengthens  of  the  mobile  approach  this  way.  Processes  that  need  a sophisticated  user  interface 
should  rather  be  kept  on  the  desktop  integrating  wireless  activities  of  the  learning  community. 


Criteria 

Wireline  eLearning 

Wireless  eLearning 

Access 

Tedious 

One  touch 

Usage 

Time  > 30  minutes 

Access  time  < 5 minutes 

Navigation 

Browsing 

Aimed  random  access 

Functionality 

Abundant,  complex 

Simple,  fimny,  added  value 

Content 

Deep,  multimedia,  complex  user  interface,  lot  of  data 

Area  of  specialization 

Utility  value 

Over  a longer  period  of  time 

Immediate 

Table  3:  Differences  in  the  way  to  use  wireline  and  wireless  eLearning  Technology 


What  drives  wireless  eLearning 

The  willingness  of  learners  to  use  eLearning  is  relatively  high  but  they  will  never  tolerate  bad  solutions 
only  because  they  are  web  based  or  wireless.  Therefore  it  is  necessary  to  support  learning  learner-centered 
with  high  usability  fulfilling  users  needs.  Table  4 mentions  the  most  important  learning  processes  that  have 
to  be  supported  in  an  eLearning  environment  of  maturity  level  two  from  an  learners  point  of  view. 

Mobile  technologies  could  significantly  augment  some  of  these  processes  and  support  processes  wireline 
solutions  can  not  support.  But  also  some  processes  are  not  well  suited  for  mobility  at  all.  Table  4 expresses 
the  process  capability  to  get  wireless  evaluating  the  5-minutes  value  (A),  simplicity  (B)  and  additional 
value  (C). 


Prozess 

Description 

A 

B 

C 

Consume  training  events  focused  on  a 
specific  learning  outcome  (WebCasts) 

Conventional  training  events  are  transmitted  so 
that  many  distributed  learners  can  participate 

- 

+ 

+ 

Consume  topic  presentations  focused  on 

Read,  listen  to,  watch  carefully  crafted 

- 

- 

- 

a specific  learning  outcome 

multimedial  explanations;  Make  annotations 

Retrieve  special  knowledge 

Consume  specific  access  and  view  of  content 

-H- 

+ 

-H- 

Do  guided  research 

Gather,  analyze  and  report  on  information 

- 

- 

- 

Do  teamwork 

Work  in  a coordinated  team  to  solve  a problem 

- 

- 

- 

Use  virtual  labs 

Experiments  with  simulated  equipment 

- 

■f 

-H 

Participate  in  a brainstorming  process 

Discussion  in  a distributed  group 

-H 

-H- 

+ 

Do  Case  studies,  Role-playing  scenarios 

Study  a detailed  example  of  a real  world  event 

- 

- 

- 

Take  hands  on  activities 

Perform  a real  task  outside  the  lesson 

- 

- 

- 

Participate  in  discussions,  ask  and 
answer  questions,  give  feedback 

-H- 

-H- 

+ 

Generate  reports 

Writing  down  results,  upload  reports 

- 

+ 

- 

Play  learning  games 

Learning  by  playing 

+ 

+ 

-H- 

Self  evaluation  and  testing 

Apply  the  learned  concepts,  skills  and  attitudes 

+ 

-H- 

- 

Drill  and  practice  activities 

Repeatedly  practice  applying  special 
knowledge 

— 

“ 

“ 

Group  critiques 

Receive  and  react  to  criticisms 

-H- 

-H- 

+ 

Make  available  knowledge  of  ones  own 
to  the  others 

“ 

-H 

“ 

Use  context  dependent  knowledge 

-H- 

+ 

-H- 

Get  general  course  information 

Get  news,  dates,  activity  indicators  in  the 
learning  community,  grades,,. 

-H- 

++ 

-H- 

Table  4:  Mobility  of  eLearning  Processes  (from  ++  well  suited  to  --  badly  suited) 

To  summarize  table  4 and  the  evaluation  performed  there  it  can  be  fixed  that  processes  with  a high  mobility 
potential  typically  show  some  of  the  following  typical  properties: 

- they  focus  on  communication  (e,g,  mail  or  discussion  forums  to  be  supported  in  a SMS  like  manner); 

- if  they  have  to  present  content  mobile  this  has  to  be  done  in  a modest  form  (this  is  also  valid  for  special 
knowledge  retrieval)  - smaller  chunks  of  information  are  retrieved  (with  adjusted  presentation  templates); 

- they  assess  eICnowledge; 

- they  use  the  mobile  device  as  portable  TVs  or  mepia-players  to  become  multimedial; 

- they  entertain  the  learner  in  niche  times  (‘gameboy  effect’) 

- they  are  context  driven 

To  make  sure  an  adequate  mobile  process  support  language  input  and  integrated  graphic  tablets  will  have  to 
replace  the  desktop  keyboard  and  mouse  functionality.  Appropriate  bandwidth  and  global  positioning 
technique  are  assumed. 

To  integrate  resident  and  mobile  process  support  eLearning  an  orientation  on  the  MVC  (Model-View- 
Controller)  architecture  seems  to  be  adequate  to  realize  integrated  solutions  (see  also  Burbeck  92),  In  the 
MVC  paradigm  the  user  input,  the  modeling  of  the  external  world,  and  the  visual  feedback  to  the  user  are 
explicitly  separated  and  handled  by  three  types  of  object,  each  specialized  for  its  task.  The  view  manages 
the  graphical  and  textual  output  to  the  portion  of  the  display  that  is  allocated  to  its  application.  The 


controller  interprets  the  inputs  from  the  user,  commanding  the  model  and/or  the  view  to  change  as 
appropriate.  Finally,  the  model  manages  the  behavior  and  data  of  the  application  domain,  responds  to 
requests  for  information  about  its  state  (usually  from  the  view),  and  responds  to  instructions  to  change  state 
(usually  from  the  controller).  This  design  pattern  makes  it  possible  to  present  a eLearning  model  on  small 
ubiquitous  devices  in  a different  view  and  with  a different  controller  than  on  desktop  solutions.  The 
common  model  guarantees  by  this  way  a global  synchronized  learning  environment. 


Conclusions 

The  most  eLearning  solutions  focus  desktop  devices  and  so  complex  views  on  eLearning  models.  Not  all 
eLearning  processes  are  well  suited  for  supporting  them  ubiquitous.  But  desktop  eLearning  functionality 
can  be  approved  adding  additional  benefits  for  the  learner  through  context  information,  ad  hoc  access, 
random  access  to  very  special  information  chunks,  having  fun  and  filling  niche  times.  The  best  mobile 
process  areas  can  be  found  in  communication  and  special  knowledge  retrieval. 

The  MVC  design  pattern  seems  to  be  the  right  architecture  for  integrated  eLearning  solutions.  To  reach  this 
goal  mobile  applications  views  will  have  to  be  designed  in  a much  simpler  way  compared  to  desktop  views 
guaranteeing  a quick  and  simple  access.  In  future  ‘Information  in  places*  could  highlight  context  benefits 
and  make  learning  really  ubiquitous.  Communication  tools  like  chats,  discussion  forums  or  instant 
messaging  with  language  interfaces  could  be  used.  Global  position  technology  could  make  it  possible  to 
identify  persons  with  similar  interests  in  the  neighborhood  and  to  start  virtual  initiated  communication  in 
real  space  at  the  end.  ELearning  systems  have  been  seen  as  a ‘pull-systems’  until  today,  expecting  the  user 
to  browse  for  information.  Using  mobile  technology  the  significance  of  ‘push-systems’  will  grow  and 
proactive  applications  could  be  developed. 
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Abstract:  The  networking  of  small  schools  between  rural  communities  has  led  to  the  development  of  new  electronic 
educational  structures  (school  district  digital  intranets)  and  processes  (virtual  classes).  As  teachers  integrate  on-site  with  on- 
line instruction  they  are,  increasingly,  teaching  not  just  in  classrooms  but  between  classrooms.  New  professionals  are  being 
appointed  to  networks  of  schools  in  Atlantic  Canada:  E-Teachers  (electronic  teachers)  and  M-Teachers  (Mediating  teachers 
who  facilitate  interaction  (on-site)  between  on-line  students  and  e-teachers). Six  pedagogical  issues  have  emerged  in  this 
new  electronic  environment  in  which  open  classes  are  constructed  between  schools  that  have  traditionally  been  closed  to 
one  another. 


E-Leaming  in  rural  schools  in  the  Canadian  province  of  Newfoundland  and  Labrador  has  moved  through  three  phases 
over  the  last  five  years.  The  first  phase  began  with  E-leaming  in  a single  high  school  in  which  research  was  undertaken 
in  science  and  mathematics  education.  In  the  second  phase,  Eleaming  was  undertaken  in  a networked,  on-line 
environment,  through  the  creation  of  the  Vista  School  District  Digital  Intranet  (VSDDI).  This  provided  an  innovative 
structure  in  which  teachers,  administrators  and  researchers  were  able  to  collaborate  in  exploring  issues  in  E-leaming,  E- 
teaching  and  the  management  of  virtual  educational  environments.  Instead  of  engaging  in  teaching  and  learning 
exclusively  within  a conventional  school,  teachers  and  students  began  to  consider  ways  of  teaching  and  learning 
between  schools  (Stevens,  2001).  The  third  phase  in  the  development  of  Eleaming  in  this  part  of  Canada  began  in 
September  2001  in  a move  away  from  research  in  a single  school  district  digital  intranet  to  a stmcture  that  spans  the 
province.  In  the  new  ELearning  environment,  ten  different  subjects  were  introduced  to  students  across  a range  of 
grade-levels  in  ten  school  districts. 

In  the  current  school  year  two  new  teacher  categories  have  emerged:  e-teachers  (electronic  teachers)  and  m-teachers 
(mediating  teachers  who  facilitate  interaction,  on-site,  between  on-line  students  and  e-teachers).  The  development  of  E 
Learning  in  High  Schools  in  Newfoundland  and  Labrador  has  highlighted  six  pedagogical  issues  for  teachers  as  they 
seek  to  integrate  traditional  on-site  instmction  with  on-line  learning  in  classrooms  that  have  become,  in  effect,  sites 
within  teaching  and  learning  networks. 


Pedagogical  Issue  One:  What  is  the  pedagogical  justification  for  providing  instruction  in  virtual 
educational  environments? 

Increasing  numbers  of  teachers  in  Newfoundland  and  Labrador  High  Schools  now  have  to  make  professional  decisions 
whether  instmction  is  more  appropriately  provided  in  traditional  classrooms  or  in  virtual  classes  (Eleaming).  As  many 
students  in  small  schools  in  Newfoundland  and  Labrador  access  instmction  in  these  dual  environments,  teachers  have  to 
be  able  to  provide  pedagogical  justification  for  on-site  or  on-line  learning. 


Pedagogical  Issue  Two:  To  what  extent  should  schools  that  are  horizontally-integrated  across 
dispersed  sites,  seek  to  develop  vertical  integration  in  their  local  communities? 

Schools  in  Newfoundland  and  Labrador  have,  to  an  increasing  extent,  become  academically  and  administratively 
integrated.  Teaching  and  learning  can,  and  to  an  increasing  extent  is,  being  provided  in  the  emerging  electronic  system 
between  classrooms  across  dispersed  sites.  This  horizontal  integration  of  schools  across  dispersed  sites  has  provided 
new  pathways  for  learners  by  extending  curriculum  offerings  in  small  schools  in  remote  locations  which,  until  recently, 
was  not  possible  because  of  economies  of  scale.  At  the  present  time  schools  are  able  to  consider  vertical  integration 
within  their  local  communities  so  that  they  can  access  such  local  resources  as  parents,  businesses,  libraries  and  a range 


of  services.  By  vertically  integrating  to  complement  horizontal  pathways,  education  can  be  extended  to  adults  and 
“second  chance”  students  at  the  community  level,  thereby  promoting  equity  through  the  provision  of  opportunities  to 
engage  in  a society  in  which  learning  is  increasingly  important.  Without  vertical  integration,  it  is  possible  that  small 
schools  in  rural  communities  could  promote  social  divisions  between  those  with  access  to  digitized  learning,  and  those 
without  access. 

Pedagogical  Issue  Three:  Should  pedagogy  for  e-learning  reflect  horizontal  and  vertical 
integration? 

Schools  that  are  horizontally  integrated  are  able  to  move  resources  between  participating  sites  (eg  specialist  teachers 
and  curriculum  materials).  In  this  new,  electronic  educational  environment,  can  collaborating  schools  promote 
knowledge-building  through  student  collaboration?  In  the  process  of  knowledge  building,  what  is  the  role  of  peer- 
tutoring? 

Pedagogical  Issue  Four:  How  should  e-teachers  choose  between  synchronous  and  asynchronous 
instruction? 

The  integration  of  on-site  and  on-line  learning  in  horizontally-linked  schools  across  dispersed  sites  enables  teachers  to 
choose  between  synchronous  and  asynchronous  instruction.  The  choice  of  synchronous  and  asynchronous  instruction  by 
educational  professionals,  will,  increasingly,  have  to  be  defended  on  pedagogical  grounds  and  in  terms  of  potential 
learning  outcomes. 

Pedagogical  Issue  Five:  In  classrooms  in  which  on-site  and  on-line  instruction  is  available,  to 
what  extent  should  asymmetric  knowledge  advancement  be  encouraged? 

On  site  and  online  learning  raises  new  possibilities  of  classrooms  becoming  places  in  which  the  development  of 
knowledge  can  be  both  symmetric  as  well  as  asymmetric,  thereby  accommodating  diverse  approaches  to  learning. 

Pedagogical  Issue  Six:  To  what  extent  should  edearning  promote  the  development  of  personal 
intranets? 

On-site  and  on-line  teaching  and  learning  is  now  at  a stage  when  moves  from  structural  intranets  (eg  as  defined  by 
specific  schools  academically  and  administratively  interfacing  in  a designated  region)  to  personal  intranets  can  be 
contemplated.  In  a personal  intranet  students  have  the  opportunity  to  develop  understandings  within  customized 
learning  environments,  with  access  to  a range  of  teachers.  This  pedagogical  development  has  the  potential  to  be 
modified  to  accommodate  and  coordinate  teaching  and  learning  styles. 

Conclusion 

Education  at  the  high  school  level  in  rural  Newfoundland  and  Labrador  is  being  extended  in  terms  of  curriculum 
opportunities  at  a time  when  many  small  community  schools  are  being  closed.  The  integration  of  on-site  and  on-line 
education  has  promoted  a situation  in  which  there  are  now  classes  that  are  open  to  one  another  across  dispersed  sites, 
linking  schools  that  otherwise  remain  autonomous  and  closed  to  one  another.  In  this  changing  educational  environment, 
a number  of  pedagogical  issues  have  emerged.  Engagement  in  the  emerging  pedagogical  issues  of  e-leaming  will  to  a 
large  extent  define  new  professionals  who  teach  between  as  well  as  in  schools.  As  Keegan  (2000)  recently  noted,  it  is 
time  to  take  stock  of  where  we  are  as  a profession  at  a time  of  change. 
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Abstract 

All  too  often,  a need  is  seen  as  e-need.  Higher  education  and  learning  however  calls  for  a human  process,  an 
interaction  between  facilitator  and  learner.  A customised  approach  to  individualised  problems  is  more  than  just 
electronic  delivery.  The  crucial  issue  is  not  the  mode  of  delivery,  but  the  skills  required  to  facilitate  learning. 
Unfortunately,  evidence  of  a subtle  re -definition  of  education  and  instruction  seems  to  emerge  in  some  quarters, 
a re-definition  that  leans  toward  information  delivery  rather  than  education. 

Facilitators  live  under  an  impression  that  the  necessary  information  is  available  and  yet  learners  do  not  utilise 
opportunities  to  expectation.  The  problem  can  be  that  the  message  received  is  not  an  exact  replica  of  the  intended 
message  originally  sent.  The  'noise'  accumulated  during  'transmission'  obscures  the  true  value  of  the  'signal'. 

Fortunately  an  increased  awareness  is  growing  to  identify  best  practice  in  terms  of  learning  facilitation  in  the 
new  technologically  enriched  environment.  To  address  this  issue  requires  more  than  technical  competence.  This 
paper  acknowledges  that  a combinational  approach  is  called  for  and  proposes  a framework  within  which  the  role 
of  the  e-options  can  find  its  rightful  niche  within  the  realities  of  modem  educational  practice  once  the  current 
'signal  to  noise  ratio'  has  been  modified  to  send  clear  messages. 

Visual  overview 


Miscomunication 


> 


Background 


Lacking  skills 
Underutilisation  ^ 


Problem  statement 


> 


1.  Introduction 

1.1  Background 

There  are  no  quick  fixes  in  education.  Arguments  given  here  are  not  intended  to  be  universal  recipes  for  success. 
Neither  are  they  aimed  at  defining  simplistic  solutions.  This  paper  aims  to: 

^ highlight  contributing  factors; 

^ identify  methods  by  which  behaviour  can  be  understood;  and 

^ elude  to  key  concepts  to  be  aware  of  when  approaching  technologically  enhanced  education. 
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1.2  Problem  statement 


The  problem  is  discussed  under  the  following  headings: 

^ e Drive:  technology  chasing  . 

^ Miscommunication:  One-way  decontextualised  information  flow 
^ Lacking  skills : Faculty  not  trained  educators 

Under  utilisation:  Tools  and  opportunities  available  not  used 

2.  Noise  factors 

2.1  ’Receptivity’ 

Receptivity  is  a term  used  to  indicate  the  degree  to  which  individuals  would  be  willing  to  participate  out  of  free 
will.  Some  may  see  this  as  another  opportunity  for  technology  to  provide  solutions  (Shih  & Sorcinelli,  2000),  but 
increased  time  commitment  and  lack  of  money  were  the  top  barriers  to  electronic  delivery  of  education  as 
identified  by  Berge  & Muilenberg  (2000).  Simplistic  solutions  of  throwing  technology  at  problems  do  not  have 
a realistic  chance  of  succeeding. 

To  explore  factors  impacting  on  the  likelihood  of  succeeding,  one  needs  to  first  determine  to  what  extent  your 
audience  would  be  susceptible  to  change  interventions. 

2.2  Factors  influencing  receptivity 

The  first  dimension  to  receptivity  is  the  locus  of  apparent  need.  Should  change  agents  offer  interventions  on 
issues  that  perceived  as  valuable  in  order  to  address  an  immediate  need,  individuals  would  be  more  likely  to 
participate.  Another  factor  impacting  on  the  receptivity  deals  with  the  effect  of  change  on  individuals  and  their 
ability  to  deal  with  change  in  a manner  that  is  constructive.  Brock  & Salerno’s  1993  change  cycle  is  valuable 
reading  in  this  regard. 

People  who  should  find  themselves  in  the  early  stages  of  the  change  cycle,  experience  feelings  that  will  prohibit 
them  from  utilising  the  opportunities  to  work  through  those  changes.  The  feelings  of  people  who  are 
experiencing  the  first  three  stages  of  change  namely  loss,  doubt  or  discomfort,  are  not  the  type  of  feelings  that 
foster  thoughts  conducive  to  the  types  of  actions  that  will  bring  people  to  volunteer  for  developmental  activities. 

Receptive  attitudes  are  fertile  ground  and  add  value  to  co-operative  learning  environments.  This  correlates  with 
the  high  value  rating  respondents  gave  to  networks  and  peer  support. 

Change  per  se  can  therefore  not  be  identified  as  the  sole  determinant  of  whether  people  will  utilise  the 
opportunities  for  development  or  not.  The  key  factor  is  the  reaction  to  the  change  and  the  resulting  attitudinal 
position  within  the  change  cycle. 

Unless  learners  are  made  aware  of  this  impact  of  change  on  their  attitudes,  their  natural  reactions  to  change 
therefore  would  indicate  an  inverse  proportionality  between  the  likelihood  of  volunteering  participation  and  their 
exposure  to  change. 
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2.4.1 : Availability 

Some  pieces  of  information  are  just  not  available.  Non-existence  is  one  form  of  not  being  available,  but  there  is  a 
magnitude  of  information  that  is  available  in  the  sense  of  existence,  but  possibly  not  available  for  lack  of  access. 
Factors  like  fire  walls,  privacy  and  language  come  to  mind  together  with  instances  where  information  was  not 
properly  recorded  in  the  first  place  like  the  techniques  used  to  create  many  of  the  Seven  Wonders  of  the  World. 
Breivik  moves  beyond  the  issue  of  media  as  prime  determinant  of  access.  ”...But  where  most  politicians  see  the 
problem  as  one  of  access  to  computers  and  the  Internet,  a growing  number  of  leaders  in  higher  education  see  it 
more  as  an  issue  of  literacy  — information  literacy  (Breivik,  2000).” 


For  the  purpose  of  this  paper,  availability  is  representative  of  the  combination  of  three  interdependent  elements 
of  availability  namely: 

^ Ability  of  the  individual  to  utilise  opportunities; 

^ Practical  accessibility  of  the  information;  and 
^ Existence  of  applicable  information. 

2.4.2:  Clarity 


Unless  instructions  and  or  guidelines  are  understood  properly,  their  impact  would  be  limited  proportionally  to 
the  lack  of  clarity. 

The  problem  of  clarity  is  not  limited  to  everyday  language  issues.  Subject  specific  terms  and  layout  can  also 
reduce  clarity.  See  http://lme.mankato.msus.edu/class/629/dhmo.html  for  an  example.  The  fact  that  information 
is  available  does  not  guarantee  that  the  true  meaning  will  be  clear  to  all. 


2.4.3:  Context 


The  University  of  Pretoria  currently  has  a (recently  improved)  maximum  Internet  bandwidth  of  5Mbps.  Compare 
this  to  the  155Mbps  that  Dutch  institutions  have  for  international  Internet  access  and  the  staggering  622Mbps 
link  available  between  some  universities!  (Steyn,  2001). 

While  the  merits  of  global  competition  are  not  the  issue  here,  the  need  for  adoption  of  theorems  for  local 
conditions  and  context  becomes  apparent. 

2.5.  Dependency 

Professor  Robert  Kraut  from  Carnegie  Melon  University  caused  a stir  some  three  years  ago  when  he  reported  his 
findings.  His  studies  revealed  that  the  social  networks  of  heavy  Internet  users  declined  and  that  they  reported 
feeling  of  loneliness  (Kraut,  1999).  Now  he  seems  to  have  results  from  a follow  up  study  that  contradict  his 
earlier  findings  (Guernsey,  2001). 

When  it  comes  to  group  dependence  and  the  adoption  or  rejection  of  technological  options  individual 
characteristics  like  learning  style  preferences  will  come  into  play. 

2.5.1:  Learning  style  preference 

All  individuals  do  not  work  in  groups  as  a result  of  choice.  Social  learning  style  preferences  differ.  Kneak 
(2001)  distinguishes  between  individual  & group  learners  The  intention  is  not  to  say  that  individual  learners 
cannot  function  well  in  a group,  but  their  natural  choice  would  not  be  to  be  dependant  on  others. 


2.5.2:  Dependency  impact 


The  preference  of  the  target  audience  for  either  individually  or  group  centred  approaches  are  not  crucial  to  the 
success  of  technology  as  a factor  in  facilitating  learning.  This  is  because  there  are  various  technical  options 
available  that  can  accommodate  either  preference.  What  is  important  though  is  that  facilitators  take  note  of  these 
factors,  provide  alternatives  and  accommodate  the  individual  differences. 

Dependency  is  not  sufficiently  crucial  to  singularly  determine  success.  Important  though,  to  keep  its  implications 
in  mind.  A metaphorical  way  of  thinking  about  dependence  could  be  to  compare  it  to  a flat  surface.  Nowhere 
higher  or  ‘better’  than  anywhere  else,  yet  representative  of  a whole  open  area  of  positions  and  possibilities. 


3.  Filtration 


In  order  to  create  a visual  model  and  enhance  understanding,  the  quality  of  communication  can  be  improved  or 
'filtered’  by  interrelating  these  issues. 

3.1  Receptivity  and  Lucidity 

Plotting  these  two  factors  against  each  other  creates  four  quadrants. 


Receptivity  Low 


Lucidity  High 
1 
3 

Lucidity  Low 


Receptivity  High 


This  first  quadrant  is  the  one  where  barriers  need  to  be  broken.  To  only  make  information  available  would  not  be 
likely  to  add  major  value,  as  individuals  low  on  receptivity,  would  not  be  likely  to  conscientiously  take  on-line 
courses  or  participate  in  discussion  groups. 


The  barrier  of  low  receptivity  can  be  broken  by  first  moving  to  quadrant  # 2.  In  this  instance  the  information  is 
still  readily  available,  but  this  time  the  receptivity  of  the  audience  stems  from  their  perception  of  need. 


Where  quadrant  2 conditions  apply,  people  may  be  receptive  enough  to  browse  through  web  page  content,  since 
they  could  get  access  to  information  in  their  own  time  and  according  to  own  priority.  Clearly  a position  of 
choice. 


The  final  quadrant  where  there  is  a common  receptivity,  but  lacking  lucidity.  The  reason  for  the  lacking  clarity 
needs  to  be  explored  in  order  to  address  the  problem.  If  the  problem  resides  in  lacking  skills  to  access  the 
relevant  information,  interventions  to  provide  the  necessary  skills  may  be  received  positively. 


A different  approach  would  be  apparent  when  the  information  is  not  available  or  consensus  not  reached.  Where 
competence  and  access  allows  for  electronic  discussion,  there  are  various  virtual  communities  that  will  be  able  to 
assist  in  identifying  solutions  or  options. 


3.2:  Receptivity  and  dependency 


Plotting  these  two  factors  against  each  other  again  creates  four  quadrants. 

Dependency  High 


Receptivity  Low 


Receptivity  High 


Dependency  Low 


Quadrant  A is  where  learners  typically  are  not  receptive,  but  highly  dependent.  These  people  like  to  work  in 
groups,  prefer  synchronous  communication  and  contact. 

Technological  options  are  not  likely  to  prove  the  ideal  tool  for  the  situation. 

Quadrant  B holds  the  double  high.  The  high  receptivity  provides  the  opportunity  for  peer  interaction  and 
reflection  activities.  While  this  traditionally  is  the  domain  of  physical  contact,  some  technological  innovations 
like  chat  rooms  or  video  conferencing  could  also  play  a part. 

Individual  learners  with  low  receptivity  typify  quadrant  C people.  Again  something  needs  to  be  done  to  break  the 
non-receptivity.  Mettal  recorded  such  a case. 

ignored  our  services.  Word  soon  came  around  that  he  was  horrible  ...,  but  he  still 
avoided  us.  One  of  his  friends  ...  suggested  that  he  talk  to  me  about  it.  ...He  came 
over  and  we  talked  for  several  hours.  I gave  him  "new  perspectives"  on  what  was 
happening  between  him  and  [his  peers]...,  etc.  (Basically,  I talked  to  him  about  to  be 
a nice  person.)  The  next  semester  went  much  better.  I was  shocked  that  such  a short 
intervention  helped,  but  he  has  thanked  me  several  times  since  (Indiana,  2001^ 

The  last  quadrant  (D)  in  this  interrelation  deals  with  highly  receptive  individuals.  Technology  has  much  to  offer 
these  people.  The  search  capabilities  of  computers  make  information  available  to  work  with  at  leisure.  Some 
individual  interventions 

may  be  necessary  in  order  to  equip  the  individual  with  the  skills  to  utilise  the  technological  options. 

"A  colleague  that  would  not  touch  the  computer  for  his  life  now  uses  computers  in  his 
instruction  (Louisiana,  2001). 

3.3:  Receptivity,  Lucidity  and  dependency 

All  three  these  factors  can  be  combined  in  one  three  dimensional  interrelation  with  x,  y and  z axis . 

The  front  octants  containing  the  low  dependency  quadrants  and  the  back  octants  containing  the  High  dependency 
quadrants.  This  relation  may  assist  facilitators  to  sufficiently  analyse  each  particular  situation  in  order  to  class  it 
accordingly. 

Attempting  to  assign  particular  technological  solutions  to  specific  combinations  would  limit  the  use  of  the 
classification  to  the  items  specifically  measured  and  mentioned.  A more  inclusive  and  flexible  understanding 
comes  from  evaluating  the  factors  in  terms  of  their  impact  on  technology  use. 

The  impact  of  the  dependency  dimension  is  not  as  important  to  determine  whether  technological  solutions  could 
be  applied,  but  rather  to  inform  on  different  options  which  need  to  be  considered. 

4.  Conclusion: 

Facilitators  need  to  adjust  to  the  new  environment.  Strategies  should  allow  for  differences  in  personality  types 
and  their  reactions  to  change.  The  variety  of  new  possibilities  and  technological  options  requires  more  than 
technical  competence.  This  paper  acknowledges  that  instructional  design  should  be  a combinational  approach 
and  calls  for  and  proposes  a framework  within  which  the  role  of  receptivity,  lucidity  and  dependency  could  be 
interrelated  to  inform  on  issues  that  will  impact  on  acceptance  of  new  strategies.  Technological  options  will  find 
its  rightful  niche  within  the  realities  of  modem  educational  practice  once  the  current  'signal  to  noise  ratio'  has 
been  addressed  to  send  clear  messages. 
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Abstract:  This  paper  discusses  the  development  of  a multimedia  case  study  which  will 
provide  the  opportunity  to  develop  the  cognitive  and  communicative  skills  in  the  management 
of  the  neurological  patient.  The  authors  argue  that  the  substantive  structure  of  the  discipline  is 
constructed  when  the  methodology  (syntactical  structure)  of  the  dscipline  is  implemented. 
The  paper  considers  elements  of  an  authentic  learning  environment  and  discusses  the 
challenges  for  the  instructional  designer  when  designing  for  authentic  learning  environments. 
The  paper  furthermore  provides  a brief  description  of  the  program  and  documents  the  findings 
of  the  first  formative  user  evaluation. 


Introduction 

The  implementation  of  a problem-based  curriculum  and  problem-based  learning  in  the  Department  of 
Physiotherapy  in  the  Faculty  of  Health  Sciences  at  the  University  of  Pretoria  has  resulted  in  the  development  of 
new  learning  environments  which  move  away  from  traditional  approaches  to  physical  therapy  (physiotherapy) 
education. 

The  focus  of  problem-based  learning  (PBL)  is  the  integration  of  facts,  concepts,  principles,  skills  and  techniques 
directed  at  solving  a unique  clinical  problem  and  the  mastery  of  cognitive,  psychomotor  and  communicative 
skills  inherent  to  the  subject  being  studied.  Problem-based  learning  aims  to  reduce  the  shortcomings  of  the  more 
traditional  approaches  to  physical  therapy  education,  in  which  learners  focus  on  the  memorisation  of  facts, 
concepts,  skills  and  techniques  which  are  then  applied  to  clinical  practice. 

Eksteen  and  Slabbert  (2001)  distinguish  between  the  substantive  and  syntactical  structure  of  physical  therapy  as  a 
discipline.  The  substantive  structure  refers  to  the  conceptual  structure  of  the  body  of  knowledge  of  a discipline. 
The  syntactical  structure  refers  to  the  way  in  which  a body  of  knowledge  about  a discipline  comes  into  being  or 
the  process  implemented  to  gain  new  knowledge  in  a discipline. 

Implementation  of  problem-based  curricula  and  the  concomitant  adoption  of  problem-based  learning  (PBL) 
approaches  to  physical  therapy  education  do  not  implicitly  guarantee  the  successful  mastery  of  the  relevant  skills. 
The  implementation  of  problem  based  learning  in  the  Department  of  Physiotherapy,  University  of  Pretoria,  has 
been  problematic  due  to  factors  such  as  a lack  of  basic  knowledge  about  the  problem-based  learning  approach 
and  the  fact  that  it  is  very  time  consuming  to  implement  this  approach.  Another  concern  is  that  even  though  the 
learner  collects,  analyses,  organises  and  critically  evaluates  content  or  knowledge  with  the  primary  purpose  to  use 
this  knowledge  in  practice,  assessment  practices  focus  heavily  on  the  content  base  of  the  curriculum.  (Eksteen  & 
Slabbert,  2001). 

Physical  therapy  practice  always  constitutes  a problem  to  be  solved  as  a point  of  departure,  therefore,  despite 
these  shortcomings  and  the  concern  that  problem-based  learning  as  it  is  currently  being  implemented  is  not  based 
on  the  substantive  and  syntactical  structure  of  the  discipline,  physical  therapy  education  should  continue  to  have 
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as  its  focal  point  the  construction  of  new  meaning  through  solving  relevant  problems.  Construction  of  meaning 
and  preparation  for  the  demands  of  practice  can  best  be  achieved  within  the  context  of  problem  solving  because 
problem-based  learning  is  an  apprenticeship  for  reaHife  problem  solving. 


Foundations  of  problem-based  learning 

Problem-based  learning  (PBL)  has  many  applications.  It  can  briefly  be  described  as  a learning  experience  during 
which  the  learners  use  an  iterative  process  of  assessing  prior  learning,  identifying  what  they  further  need  to 
master,  gathering  information;  and  collaborating  on  the  evaluation  of  hypotheses  in  light  of  the  data  collected, 
in  order  to  solve  the  problem  (Stepien  & Gallagher,  1993). 

In  the  process  of  problem-based  learning,  learners  crisscross  a variety  of  disciplines.  They  build  substantial 
knowledge  bases  through  increasingly  self-directed  study.  Through  collaboration  with  their  peers  they  refine  and 
extend  their  skills  and  knowledge  base,  within  a conceptual  framework  (of  the  discipline),  in  such  a way  that  will 
promote  transfer  of  knowledge  and  skills  to  solve  new  problems.  The  learner  is  continually  constructing 
meaning.  The  role  of  the  teacher  / educator  also  changes  in  the  problem-based  learning  approach.  Teachers  take 
on  new  roles  in  problem-based  learning,  acting  as  facilitators,  coaches  and  tutors,  probing  findings,  hypotheses 
and  conclusions,  sharing  their  thinking  when  students  need  a model,  and  encouraging  the  development  of 
metacognitive  skills. 

There  are  three  factors  which  facilitate  learning  within  a problem-based  approach: 

• prior-knowledge  is  activated; 

• the  greater  the  similarity  between  the  formulated  ‘problem’  and  reality,  the  better  the  learner’s  success  in 
acquiring  the  relevant  skills; 

® new  knowledge  is  better  understood,  processed  and  recalled  if  the  learners  have  the  opportunity  to  elaborate 
on  the  knowledge  they  have  mastered  within  the  conceptual  framework  of  the  discipline  (Schmidt,  1983). 

A study  which  investigated  the  processes  used  by  students  in  developing  conceptual  knowledge  in  physical 
therapy  found  that  the  students  in  the  study  used  an  active,  experiential  approach  to  learning  that  included  a 
reliance  on  the  senses  during  the  learning  process.  The  study  concluded  that  the  type  of  learning  processes  used 
by  students  may  be  enhanced  by  educational  methods  such  as  collaboration  and  group  learning,  situated  cognition 
and  use  of  authentic  contexts,  cognitive  apprenticeship  and  whole -part-whole  learning  (Graham,  1996). 


Instructional  design  for  an  authentic  learning  environment:  a case  for  interactive 
multimedia 

The  need  for  authentic  contexts  within  which  to  design  learning  environments  and  experiences  in  a physical 
therapy  program  has  many  challenges  for  the  educator.  While  there  may  be  differences  in  the  exact 
implementation,  all  physical  therapy  education  programs  have  both  an  academic  and  clinical  component. 
Learners  are  required  to  complete  a certain  number  of  hours  in  the  clinical  setting.  The  amount  of  time  spent  in 
the  clinical  setting  does  not  guarantee  that  the  learners  will  be  exposed  to  the  entire  range  of  problems  - both  in 
quantity  and  complexity  - within  a specific  specialty.  There  are  also  many  ethical  issues  which  arise  when 
designing  learning  environments  in  the  clinical  setting,  which  require  learners  to  have  mastered  the  relevant 
cognitive,  psychomotor  and  communicative  skills  before  entering  their  clinical  blocks. 

These  constraints  could  prompt  the  physical  therapy  educator  to  look  at  alternative  methods  for  providing 
problem-based  learning  experiences.  Alternatives  include  practice  in  a simulated  setting  in  a laboratory  and, 
more  recently,  the  use  of  technologies  such  as  the  world  wide  web  and  interactive  multimedia.  The  principal 
theorists  of  situated  learning  have  rejected  the  idea  that  the  computer  could  provide  an  authentic  learning  context, 
arguing  that  the  courseware  becomes  the  learning  environment  and  not  the  authentic  situation.  However, 
researchers  and  teachers  have  continued  to  explore  the  model  of  situated  learning  using  the  computer  and  there  is 
increasing  agreement  that  such  technology  can  be  used  effectively  without  sacrificing  the  authentic  context  of  the 
learning  experience  (Herrington  & Oliver,  2000). 


It  was  within  this  context  that  the  Department  of  Physiotherapy  at  the  University  of  Pretoria  requested  funding 
for  the  development  of  a series  of  interactive  multimedia  programs,  based  on  the  problem-based  learning 
approach,  which  would  engage  the  learner  in  an  authentic  learning  environment.  The  author  of  this  paper  is 
responsible  for  the  instructional  design  of  the  program. 


Instructional  design  challenges 

Instructional  design  of  interactive  multimedia  learning  environments  grounded  in  a problem-based  learning 
approach  needs  to  consider  both  the  requirements  for  authentic,  problem-based  learning  and  instructional  design 
theory.  Development  of  educational  multimedia  programs  has  in  recent  years  largely  been  driven  by 
constructivist  learning  theory  and  traditional  instructional  systems  design. 

The  task  of  the  designer  is  to  incorporate  contextual  factors  appropriately  into  the  design  of  interactive 
multimedia  learning  environments  (Boyle,  2001).  The  central  challenge  for  educational  multimedia  designers  is 
to  create  contexts  that  promote  effective  learning.  As  designers,  we  need  to  engage  learners  in  hteractive 
scenarios.  We  impact  on  the  learner  through  our  skill  in  creating  these  contexts.  The  skilled  designer  who  can 
design  / create  the  context  and  interaction  in  such  a way  that  the  actual  courseware  becomes  seamless  will  have 
made  progress  in  overcoming  the  criticism  of  the  situated  learning  theorists,  namely  that  technology  becomes  the 
focus  for  learning  instead  of  the  authentic  context. 


Description  of  the  program 

The  content  specialist  had  been  afforded  the  opportunity  to  capture  on  video  the  complete  evaluation  of  patients 
with  selected  neurological  deficits.  The  range  of  problems  these  patients  presented  was  both  unique  and 
complex.  Learners  in  the  clinical  setting  will  seldom  have  the  opportunity  to  perform  such  comprehensive 
evaluations.  The  instructional  designer  was  requested  to  use  all  the  video  footage  and  the  information  provided 
by  the  content  specialist  to  create  an  interactive  multimedia  learning  environment  with  a case  study  approach  as 
the  methodology  of  choice.  The  specific  module  (theme)  in  the  first  series  of  case  studies  is  the  management  of 
the  patient  with  an  upper  motor  neuron  neurological  diagnosis.  The  content  specialist  also  intends  that  the 
program  enable  the  learner  to  acquire  the  way  in  which  new  knowledge  is  generated  in  the  discipline  - the 
methodology  of  the  discipline. 

The  management  of  the  patient  is  divided  into  several  phases: 

• The  comprehensive  evaluation  of  the  patient  - subjective,  objective  and  biomechanical  evaluation; 

• Identification  of  the  specific  problems  the  patient  has  (problem  formulation); 

• Formulating  the  goals  for  the  management  of  the  patient  (goal  formulation);  and 

• Planning  the  treatment. 

Each  section  has  several  questions  which  guide  (facilitate)  deep  learning  (construction  of  meaning).  The  learner 
must,  in  most  cases,  study  the  video  clip  available  at  that  point  in  the  program  and  then  attempt  to  answer  the 
question.  Answers  are  entered  electronically  and  saved  for  later  retrieval.  Once  the  learner  has  attempted  the 
answer  he/she  may  viewthe  expert  answer.  There  are  a total  of  22  questions  in  the  program. 

Exercises  which  require  the  learners  to  reflect  on  their  success  in  terms  of  their  competence  in  the  completion  of 
the  tasks  / questions  are  included  at  strategic  points  in  the  case  study.  The  aim  with  the  reflection  exercise  is  to 
facilitate  the  construction  of  the  conceptual  framework  (body  of  knowledge)  of  physical  therapy  as  learners  work 
through  the  case  study.  The  expert  answers  play  a key  role  in  the  self  evaluation  and  reflection  of  learners  on 
their  process  of  acquiring  the  relevant  competencies  required  from  them  in  the  clinical  field.  The  process,  as  well 
as  the  competencies  learners  have  to  implement  while  working  through  the  case  studies,  simulate  the  syntactical 
structure  of  physical  therapy  as  a discipline.  The  instructional  designer  designed  an  interactive  approach  - the  use 
of  game  techniques,  question  and  answers  and  multiple  choice  questions  - to  facilitate  learner  interaction  with  the 
content. 


The  techniques,  tools  and  strategies  used  by  the  designer,  together  with  this  valuable  material,  greatly  facilitated 
the  creation  of  a unique  learning  environment  which  could  be  extended  far  beyond  the  clinical  sessions  which  a 
learner  could,  at  most,  hope  to  experience.  The  latter  together  with  the  reflection  exercises  facilitate  self 
evaluation,  the  development  of  metacognitive  skills  and  independent,  self-directed  learning  The  option  to  re- 
visit such  an  authentic  context,  using  the  electronic  medium,  also  transcends  the  many  ethical  problems  which 
arise  when  working  with  patients  in  the  clinical  environment.  Table  1 summarises  the  design  elements  which 
contribute  to  the  problem-based  learning  approach  and  the  creation  of  an  authentic  learning  environment. 


Foundations  of  problem- based 
learning 

Operationalisation  in  this  multimedia/learning  environment 

An  iterative  process  of  assessment 

Opportunity  to  review  expert  answer  once  learner  has  tried  to  answer 
question. 

Acquiring  relevant  knowledge  / recognition  of  prior  learn  ing  while 
observing  the  physiotherapist  interact  with  the  patient. 

Assessment  of  contribution  in  group  discussion  classes. 

Collection  of  information 

Review  of  knowledge  base  at  appropriate  points  in  lesson. 

Collaboration 

If  learners  use  the  multimedia  in  groups,  otherwise  the  educator  plans 
to  have  group  discussions  after  the  multimedia  has  been  used. 

Integration  of  various  disciplines 

Knowledge  base  from  various  disciplines  embedded  into  program. 

Modeling  by  educator 

Provision  of  expert  answer. 

Application  of  metacognitive 
strategies  / Reflection 

Interaction  designed  specifically  to  enable  reflection  of  progress  - at 
least  three  points  in  program  where  this  occurs. 

Elements  of  authentic  learning 

(Herrrington  & Oliver,  2000) 

Operationalisation  in  this  multimedia/learning  environment 

Provide  authentic  contexts  that 
reflect  how  the  knowledge  will  be 
used  in  real  life. 

Case  study  approach  using  video  footage  of  an  actual  case. 

Provide  authentic  activities. 

Management  of  a current  case  (not  constructed  for  teaching  / learning 
purposes).. 

Provide  access  to  expert 
performances  and  the  modeling  of 
processes. 

Can  always  view  expert  answer. 

Provide  multiple  roles  and 
perspectives. 

Learner  observes  both  physiotherapist  evaluating  the  patient  and  the 
patient,  although  cannot  hear  patient  communicate. 

Support  collaborative  construction 
of  knowledge. 

Not  explicitly  embedded  into  design  of  multimedia.  The  case  study  is 
also  discussed  in  a contact  session  during  which  collaborative 
construction  of  knowledge  could  occur 

Promote  reflection  to  enable 
abstractions  to  be  performed. 

Interaction  designed  specifically  to  enable  reflection  of  progress  - at 
least  three  points  in  program  where  this  occurs. 

Promote  articulation  to  enable  tacit 
knowledge  to  be  made  explicit. 

Learner  required  to  enter  own  answer  and  submit  this  / use  it  as  basis 
for  class  discussions. 

Provide  scaffolding,  learning 
facilitation  and  coaching  at  critical 
times. 

Use  of  expert  answers,  availability  of  theoretical  knowledge  at  selected 
points  in  the  program. 

Provide  for  authentic  assessment  of 
learning  within  the  tasks 

Entire  case  study  approach  to  the  learning  environment. 

Table  1 : Operationalisation  of  design  elements  to  meet  criteria  for  problem-based  and  authentic  learning 

Evaluation  of  the  program 


Due  to  the  fact  that  the  design  and  use  of  multimedia  for  problem-based  learning  is  a new  experience  for  the 
instructional  designer,  content  specialist  and  learners  it  was  decided  to  commence  with  formative  evaluation  very 
early  in  the  development  cycle.  Development  commenced  in  March  2001  and  the  first  evaluation  was  conducted 


in  September  2001.  Thirty  seven  final  year  students,  who  were  due  to  write  their  final  examination  in  October 
2002,  participated  in  the  evaluation.  Both  the  instructional  designer  and  the  content  specialist  designed 
questionnaires  for  the  learners  to  complete  at  the  end  of  the  evaluation  session.  The  instructional  designer  also 
observed  the  learners  using  the  program.  The  content  specialist  focused  on  the  learner’s  perceptions  of  the 
contribution  of  the  program  to  the  learning  process.  The  instructional  designer  looked  at  elements  of  the 
instructional  design  which  could  both  positively  and  negatively  affect  their  learning.  Particular  attention  was 
paid  to  navigation  issues. 

At  the  time  of  evaluation  some  of  the  expert  answers  had  not  yet  been  finalised,  the  knowledge  base  was  only 
available  in  a dull,  non  interactive  text  -based  format  and  no  help  screens  had  been  designed.  The  learner  also  had 
limited  control  over  certain  features  of  the  program.  This  included  control  over  the  video  clips  and  the  ability  to 
navigate  backwards  and  forwards  in  the  program.  It  had  been  decided  that  the  learners  had  to  work  through  the 
questions  in  the  exact  order  determined  by  the  content  specialist.  Learners  could  only  return  to  previous  screens 
by  following  a linear  path  through  the  program.  The  programming  required  when  using  the  multimedia  in  a 
computer  laboratory  rather  than  a stand  alone  PC  was  also  not  complete.  The  first  evaluation  session  aimed  to 
investigate  some  of  the  functionality  of  this  programming.  The  instructional  designer  anticipated  that  there  might 
be  some  problems  with  the  manner  in  which  learner  input  was  stored  and  displayed.  These  potential  problems 
were  communicated  to  the  students  at  the  beginning  of  the  evaluation  session. 

Evaluation  of  the  instructional  design  and  learning  environment 

The  open-ended  questions  in  the  questionnaire  generated  a host  of  responses.  The  responses  were  categorized 
and  the  number  of  responses  in  each  category  quantified.  Tab.2  summarises  the  design  elements  which  the 
learners  liked  and  which  contribute  to  an  effective  learning  environment.  Tab  3 summarise  the  design  elements 
which  the  learners  disliked  and  which  could  negatively  influence  effective  learning. 


Instructional  and  interface  design  elements 

Strategies  / techniques  which  create  an  authentic 
learning  environment 

Program  is  easy  to  use. 

There  was  control  over  the  pace  at  which  learning 
occurs. 

Learners  had  control  over  the  number  of  times  they 
could  return  to  previous  video  clips,  expert  answers 
and  content  screens. 

‘Good’  colour. 

Well  organized  and  ‘fluent’. 

Audiovisual  presentation  supports  the  information. 
Legible. 

Nice  graphics. 

Liked  the  display  of  own  answer  next  to  the  expert 
answer 

The  expert  opinion  which  was  always  available 
after  learner  had  completed  a question  (n=19). 

Good  quality  of  the  video. 

Expert  answers  are  very  comprehensive. 

Content  of  video  clips  relevant. 

Program  simulates  the  way  in  which  the  physical 
therapist  thinks  when  evaluating  a patient. 

Clips  and  sound  clear  and  very  descriptive. 

Aspects  which  the  learner  needs  to  focus  on  are 
highlighted  by  the  very  nature  of  the  question 
Informative. 

Opportunity  to  attempt  to  answer  the  question 
before  viewing  the  expert  answer. 

Table  2:  Design  elements  which  the  learners  liked  and  which  contribute  to  an  effective  learning  environment 


Design  elements  which  could  negatively  influence 
the  learning  environment 

Elements  which  are  part  of  an  authentic 
learning  design  and  environment 

Lack  of  control  over  the  video  clips  - cannot  pause, 
rewind. 

Lack  of  control  over  navigation  in  program  - bound  by 
the  linear  navigation  path 

Lack  of  information  about  how  long  the  video  clip  is. 
Did  not  like  the  voice  over  - would  have  liked  to  hear 
the  patient  speak. 

Were  not  able  to  see  how  many  questions  still  had  to  be 
completed. 

Expert  answers  were  too  long  - would  have  liked  them 

Have  to  remember  a lot  of  detail  in  the  clips  and 
some  of  the  clips  provide  an  answer  to  other 
questions  in  the  program 
Have  to  concentrate  really  hard 
Some  of  the  answers  are  complex 

to  be  more  concise  and  to  the  point,  presented  as  a 
bulleted  list 

Lack  of  information  about  how  to  use  the  program. 

Program  was  very  long  and  time  consuming 
Parts  were  very  monotonous 

Content  screens  contained  a lot  of  information 

Table  3;  Design  elements  which  the  learners  disliked  and  which  could  negatively  influence  effective  learning 

Some  of  the  problems  experienced  during  the  evaluation  related  to  the  incompleteness  of  the  program  and  the 
fact  that  some  of  the  programming  appeared  to  be  incorrect.  A few  students  commented  on  the  inappropriateness 
of  the  content  of  the  expert  answer  to  the  question. 


Future  direction  for  the  development 

Despite  the  problems  and  many  suggestions  for  improvement,  the  overall  impression  was  that  learners  perceived 
the  program  as  a useful  tool  which  is  both  comprehensive  and  thorough.  During  the  evaluation  learners  were 
requested  to  provide  suggestions  for  improving  the  program.  The  most  frequent  request  was  for  better  control 
over  the  video  clips  and  navigation  in  the  program.  Some  of  the  comments  seemed  to  indicate  that  the  interface 
design  and  navigation  were  not  yet  seamless.  Learners  should  not  have  to  waste  time  struggling  to  use  the 
technology,  but  should  rather  be  able  to  interact  effectively  with  the  learning  environment. 

Further  development  will  focus  on  implementing  the  suggestions  and  improving  and  refining  the  design.  The 
content  specialist  will  also  be  looking  at  the  formulation  and  layout  of  some  of  the  expert  answers  as  well  as 
limiting  the  questions  to  those  which  facilitate  learning  most  effectively  - avoiding  questions  which  make  the 
duration  of  the  programme  too  long. 


Conclusion 

The  development  of  a multimedia  program  using  a case  study  approach  to  problem  solving  is  in  itself  an 
authentic  learning  experience,  especially  when  the  case  study  is  based  on  a real  patient.  The  challenge  for  the 
instructional  designer  is  to  consciously  design  the  learning  elements  in  such  a way  that  the  nine  criteria  for  an 
authentic  learning  environment  are  operationalised  and  that  the  focus  remains  on  the  learning  environment  and 
not  the  technology. 
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Abstract:  This  paper  offers  a new  perspective  in  teachers’  education  regarding  the 
effective  technology  integration.  The  core  theoretical  background  is  Case  based 
Reasoning.  We  develop  a knowledge  repository  filled  by  indexed  stories  collected 
directly  from  teachers  who  use  technology  in  their  classrooms.  The  index  is  developed 
and  under  continuous  revision  according  to  the  experience  we  gather  through  the 
interviews.  The  stories  are  collected  and  the  system  developed  by  a consortium  of  eight 
universities.  The  stories  can  be  retrieved  in  a Web-based  interface,  that  is  powered  by  a 
custom  made  engine,  that  uses  a near-neighbor  algorithm.  Future  research  will  be  around 
the  use  of  this  system  by  teachers  and  teacher  educators. 


Rationale 

A 1998  National  Center  for  Education  Statistics  (NCES)  study.  Teacher  Quality:  A Report  on  the 
Preparation  and  Qualifications  of  Public  School  Teachers,  that  dthough  many  educators  consider 
technology  as  a vehicle  for  transforming  education,  only  20  % feel  well  prepared  to  integrate  technology 
into  the  classroom.  To  improve  the  level  of  technology  integration,  a 1999  study  entitled  Will  New 
Teachers  be  Prepared  to  Teach  in  a Digital  Age?  recommended  that  researchers,  professional  societies, 
and  education  agencies  should  identify,  study,  and  disseminate  examples  of  effective  technology 
integration  on  an  ongoing  basis. 

Mission 

In  order  to  meet  these  challenges  to  educate  tomorrow’s  teachers  to  effectively  integrate  technology  in  their 
classroom  instruction,  we  at  the  University  of  Missouri  have  begun  the  Knowledge  Development  for  the 
Technology  Integration  Community  Project.  This  project  is  a Catalyst  grant  from  the  U.S.  Office  of 
education  that  involves  a consortium  of  eight  partners  institutions  collaborating  to  address  issues  in  creating 
and  diffusing  technology  integration  knowledge. 

The  project's  mission  is  to  build  a K-16  community  of  practice  through  a case  library  of  teacher  stories 
about  technology  integration.  In  order  to  fulfill  this  primary  goal,  we  have  designed  a case  library  for 
teachers  and  teacher  educators  and  are  currently  collecting  and  indexing  cases  to  be  added  to  our  case 
library.  Once  completed,  teachers  and  teachers’  educators  will  be  able  to  access  successful  and 
unsuccessful  attempts  to  integrate  technology  from  the  case  library.  The  case  library  is  being  built 
according  to  principles  of  case-based  reasoning. 

Theoretical  Background  - Case  based  Reasoning  (CBR) 

In  order  to  educate  teachers  and  teacher  educators  to  effectively  integrate  technology  into  instruction,  we 
should  expose  them  to  stories  generated  by  other  teachers.  One  way  to  do  this  is  by  exposing  them  to 
narratives  or  stories  or  cases  that  have  been  compiled  into  a case  library  (database  of  stories  made  available 
to  learners  as  a form  of  instructional  support).  Why  should  we  do  this? 


Supporting  learning  with  stories  can  help  new  teachers  and  teacher  educators  to  vicariously  gain 
experience.  Some  researchers  believe  that  hearing  stories  is  tantamount  to  experiencing  the  phenomenon 
oneself  (Ferguson,  Bareiss,  Bimbaum,  & Osgood,  1991).  This  kind  of  reasoning  (reasoning  from  stories  or 
cases)  provides  support  for  “inferences  necessary  for  addressing  the  kinds  of  ill -defined  or  complex 
problems  that  come  our  way  in  the  workplace,  at  school,  and  at  home”  (Kolodner,  1997,  p.  58).  The 
process  of  understanding  and  solving  new  problems  using  case  libraries  has  three  parts:  recalling  old 
experiences,  interpreting  the  new  situation  in  terms  of  the  old  experience  based  on  the  lessons  that  we 
learned  from  the  old  experience,  or  adapting  the  old  solution  to  meet  the  needs  of  the  new  situation 
(Kolodner,  1993). 

This  whole  process  is  described  by  Aamodt  and  Plaza  (1996)  as  the  CBR  cycle.  An  encountered  problem 
(the  new  case)  prompts  the  reasoner  to  retrieve  cases  from  memory,  to  reuse  the  old  case  (i.e.  interpret  the 
new  in  terms  of  the  old),  which  suggests  a solution.  If  the  suggested  solution  will  not  work,  then  the  old 
and  or  new  cases  are  revised.  When  the  effectiveness  is  confirmed,  then  the  learned  case  is  retained  for 
later  use.  The  main  process  in  developing  a Case  Based  Reasoning  tool  for  problem  solving  is  as  presented 
by  Jonassen  et  al  (1999). 

Collecting  and  Indexing  Stories 

Currently,  we  are  collecting  stories  to  be  included  in  our  case  libraries.  The  story  collection  process  is  being 
carried  out  by  Knowledge  Scouts  at  eight  different  institutions.  These  scouts  are  making  appointments  with 
teachers  in  schools  and  interviewing  them.  We  applied  elements  of  the  critical  incident  method  into  our 
interview  strategies.  The  interviewers  then  transcribe  the  interviews,  and  the  team  at  Missouri  will  index 
the  interview  and  build  narrative  sequences  out  of  the  interview. 

Building  the  Retrieval  Engine 

For  the  retrieval,  we  use  a near-neighbor  algorithm  to  match  a query -case  (given  by  any  user  at  our  web- 
based  interface)  with  existing  cases.  For  this  purpose  the  query  case  will  be  converted  into  a feature  vector, 
and  therefore  can  be  used  in  the  knowledge  repository.  The  user-friendly  interface  is  a web- 
based,dynamically  created  out  of  our  Oracle  database  and  allows  the  user  to  search  and  browse. 

Future  research 

When  we  have  aidded  and  indexed  a sufficient  number  of  cases  into  the  library,  we  plan  to  begin  studying 
how  teachers  and  teacher  educators  will  use  the  case  library.  We  hope  to  be  able  to  present  research 
findings  on  these  questions  at  the  next  conference. 
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Arguably  the  biggest  ‘buzz  word*  of  the  current  year  has  been 
‘learning  or  knowledge  object*.  To  understand  the  learning  object 
and  why  it  should  be  such  a highly  desirable  commodity,  we  need  to 
unpack  not  only  this  concept  but  more  importantly  revisit  some 
contributing  concepts  and  constructs  (more  buzz  words)  that  support 
the  building  of  truly  pedagogically  informed  reusable  objects  (Boyle 
& Cook  2001).  The  words  and  relationships  explored  in  this  paper 
are: 

• Learning  or  knowledge  objects  - the  desirable  construct  in 
today*s  E- learning  environment. 

• Ontologies  and  ways  of  expressing  them  through  topic  maps 
as  they  allow  us  to  define  and  describe  the  components  of  an 
entity 

• Metadata  and  XML  used  to  create  categorise,  label  and 
communicate  the  value  of  these  objects 

• Hermeneutics  and  phenomenology  as  they  refer  to  the 
interpretation  of  experience  and  events  and  evaluation  of 
learning  events 


Learning  or  knowledge  objects 

Learning  objects  (also  known  as  knowledge  or  instructional  objects)  are  digital  learning  events  that  are 
discrete,  reusable,  able  to  be  aggregated  and  are  tagged  with  defining  Metadata.  The  impetus  for  the 
development  of  learning  objects  comes  as  much  from  not  wanting  to  reinvent  the  wheel  as  wanting  to  find 


ways  to  adequately  define  and  categorise  online  instructional  events.  The  form,  size  and  shape  of  these 
learning  objects  can  be  as  varied  as  the  respective  face-to-fece  events  from  lecture,  course  or  discussion  to 
simulation  and  role-play. 

Downes  (2001)  succinctly  describes  the  programming  roots  of  learning  objects.  To  delve  more  deeply  into 
the  construction  and  organization  of  learning  objects,  it  is  necessary  to  introduce  another  concept  from 
computer  programming,  object-oriented  design  (e,g,,  Mont  lick,  1999),  The  idea  behind  object-oriented 
design  is  that  prototypical  entities,  once  defined,  are  then  cloned  and  used  by  a piece  of  software  as 
needed,  (Downes  2001  pp  7) 

Speaking  from  an  educational  perspective,  Reigleuth  and  Nelson  (1997)  propose  that  the  breaking  down 
into  instructional  components  (learning  objects)  is  akin  to  the  typical  practice  of  instructors  when  they 
develop  a course  or  learning  sequence.  The  content  and  relationships  are  broken  down  and  the  learning 
strategy  and  evaluation  is  then  selected  for  each  component.  However  it  is  the  technological  not  the 
educational  agenda  that  dominated  the  earliest  discussions  of  development,  categorisation  and  location  of 
learning  objects.  We  cannot  deny  the  importance  of  this  aspect,  for  reusable  objects  would  not  find  users 
if  the  content  and  form  were  not  able  to  be  effectively  defined,  described  and  delivered.  More  recently 
and  very  importantly  there  has  been  a cry  for  pedagogical  aspects  of  learning  objects  to  drive  the  process 
of  development  and  definition.  (Boyle  & Cook  2001,  Ip  & Morrison  2001)  Indeed  Ip  & Morrison  would 
draw  us  back  to  the  programming  roots  of  the  term  learning  objects  and  propose  that  unt  il  the  contextual 
rendering  of  these  objects  occurs  they  are  not  learning  objects  at  tall  but  digital  resources. 

The  issues  that  surround  the  value  of  the  seemingly  simple  concept  of  learning  objects  appear  manifold 
and  go  directly  to  the  heart  of  the  nature  of  instruction.  The  actual  objects,  their  granularity, 
interoperability  and  reusability  and  how  the  learning  event/s  or  exchanges  they  involve  can  be  fully 
described,  identified  and  evaluated  are  topics  for  hot  debate.  How  small  do  components  need  to  become 
to  be  discrete  and  reusable?  What  exactly  does  it  mean  for  a learning  object  to  be  reusable?  For  whom  is 
it  reusable  and  can  it  be  rendered  for  all  learning  contexts  and  paradigms?  How  will  instructors  be  able  to 
identify  and  select  the  appropriate  learning  objects  for  their  instructional  context? 

Can  these  learning  objects  be  seen  as  dynamic  and  iterative  in  an  evaluation  cycle  involving  their  users? 
We  hope  to  touch  on  some  of  these  issues  and  highlight  some  of  the  key  researchers  working  through 
these  issues. 

Epistemological  Ontologies 

Traditionally  ontology  dealt  with  the  area  of  metaphysics,  which  defined  the  fundamental  distinct  entities 
to  which  existence  could  be  ascribed.  Within  a particular  area  of  interest,  such  as  that  of  Teaching  and 
Learning  Systems,  we  need  to  qualify  this  to  allow  a plurality  of  ontologies  of  which  a specific  one 
constitutes  a valid  domain  in  its  own  right.  Smith  (Smith,  1998)  thus  distinguishes  R- ontology  (reality- 
based  ontology)  and  B ontology  (epistemological  ontology).  While  the  first  deals  in  the  general  meaning 
of  ‘being’,  the  second  provides  the  basis  of  a conceptualisation  of  a particular  domain. 

Ontology  construction  of  this  kind  is  now  an  accepted  tool  in  a number  of  disciplines.  In  the  field  of 
Geographic  Information  Systems,  for  instance,  Fonseca  and  Egenhofer  describe  one  such  system  thus. 
Ontology  plays  a central  role  in  the  definition  of  all  aspects  and  components  of  an  information  system  in 
the  so-called  ontology -driven  information  systems.  The  system  presented  here  uses  a container  of 
interoperable  geographic  objects  ...  This  approach  provides  a great  level  of  interoperability  and  allows 
partial  integration  of  information  when  completeness  is  impossible  (Fonseca  & Egenhofer,  1999  pp.l4). 
Researchers  have  described  the  value  of  this  process,  even  at  an  intermediate  or  tentative  stage.  Gaede 
and  Stoyan  state.  Supposing  that  universally  valid  terminology  is  neither  near  nor  realistic,  partial 
domain  as  well  as  pedagogical  ontologies  ...  have  already  proven  fruitful,  (Gaede,  & Stoyan,  2001  pp. 


Epistemological  ontologies  form  a suitable  area  for  the  use  of  these  methodologies.  Of  particular 
relevance  to  learning  systems,  constructivism  offers  one  such  radical  approach,  which  has  been 
characterised  as  an  epistemology  of  learning  rather  than  a framework  of  teaching  (Nunes,  McPherson, 
Rico,  2001).  Thus  it  has  been  argued  that  A constructivist  epistemology  views  knowledge  as  a construct  of 
individual  *s  understanding.  When  individuals  come  together,  such  as  in  a learning  environment,  their 
construction  undergoes  continuous  revision  due  to  cognitive  conflict  which  occurs  as  a result  of  dijferent 
constructions  among  individuals,  (Hendriks  & Maor,  2001  pp.  732). 
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Nevertheless,  even  this  social  constructivist  viewport  still  poses  the  question  of  how  methodologically  a 
constructivist  epistemology  can  be  incorporated  in  an  objective  model  of  learning.  One  approach  to  this 
we  could  postulate  would  be  to  attempt  to  derive  a constructivist  epistemological  ontology.  This  w)uld 
allow  the  interfacing  and  intern-elating  of  different  constructivist  models  into  a single  coherent  working 
framework.  Would  this  aim  violate  the  underlying  principles  of  constructivism?  This  need  not  be  the  case 
as  the  process  of  ontology  derivation  itself,  as  dynamic  and  subject  to  progressive  development,  could  be 
considered  within  the  constructivist  framework,  as  an  activity  at  a meta  level,  carried  out  by  constructivist 
educational  practitioners  rather  than  the  learners  themselves. 

Ontology  construction  can  also  employ  techniques,  such  as  conceptual  graphs  and  topic  maps. 
Futhermore,  development  environments  exist  that  will  allow  these  in  turn  to  be  mapped  to  HTML/XML 
representations.  For  instance,  the  collaborative  environment,  Veda  (Prata,  Paraguacu  & Reis,  2001 
pp.l512),  is  described  in  the  following  manner.  The  conceptual  graphs  ...  provide  a form  to  represent 
knowledge  through  the  ontology  (application  domain)  and  logic  (logical  first -class).  Following  its 
formalism,  we  can  represent  graphical  form  in  linear  form  and  create  a computational  model.  This 
computational  model  together  with  the  features  of  hypertext,  allows  indexing  the  knowledge  in  HTML 
format.  In  addition  to  the  techniques  discussed  earlier  for  ontology  development  generally,  the  process  of 
constructivist  ontology  construction  can  draw  on  a variety  of  other  techniques  from  various  fields.  Stutt, 
for  instance,  suggests  the  use  of  “ontological  and  other  (problem  solving)  categories  from  knowledge 
engineering,  a basically  constructivist  rather  than  mirroring  approach  to  knowledge  and,  finally, 
computational  models  of  and  computer  support  for  argumentational  structures”  (Stutt,  1997).  It  is  an 
approach  similar  in  principle  to  this,  which  could  utilise  topic  map  representations  to  derive  learning 
objects  implemented  in  XML,  that  we  intend  to  investigate  further. 


Conceptual  Graphs  and  Topic  Maps 

Topic  Maps:  ISO  standard  (ISO  13250,  httD://www.vl2.doe.gov/sgml/sc34/document/0 129.pdf)  in  its 
SGML/HyTime -implementation.  Final  1 .0  version  of  the  XML  http://www.topicmaps.org/xtm/ 1 .0/  . 

The  technology  of  topic  maps  is  nothing  but  formalising  relationships  of  concepts  using  topics, 
associations  and  occurrences,  in  a machine-readable  and  platform  independent  format.  The  connection  to 
the  real  world  and  its  learning  material  is  maintained  by  links  called  “occurrences”,  which  allow  for 
linking  out  of  the  abstract  topic  cloud  to  files,  URLs,  or  learning  objects  in  a learning  environment. 

From  a technical  standpoint,  topic  maps  can  be  represented  using  valid  XML  documents  with  basically 
three  different  sorts  of  elements:  topics,  associations  and  occurrences.  A topic  is  anything  the  designer  of 
the  map  wants  it  to  be,  mostly  the  abstract  representation  of  a “thing”  or  a concept,  which  is  connected  to 
other  things  via  associations.  These  associations  are  themselves  topics,  and  an  association  has  roles  for 
the  two  topics  it  connects,  defining  the  role  each  connected  topic  is  playing  in  the  association.  With  this 
set  of  elements  it  is  possible  to  structure  and  represent  an  ©ontology  in  an  abstract  and  technologically 
independent  manner.  The  topic  map  itself  is  created  while  parsing  (“reading”)  the  topic  map  exchange 
document  and  building  the  n-dimensional  mesh  of  topics  and  associations  in  the  memory  of  the  computer. 

When  two  topics  are  merged,  the  result  is  a single  topic  whose  characteristics  are  the  union  of  the 
characteristics  of  the  original  topics,  with  duplicates  removed.  It  is  an  approach  similar  in  principle  to 
this,  which  could  utilise  topic  map  representations  to  derive  learning  objects  implemented  in  XML  that 
system  designers  need  to  investigate  further.  Consider  if  we  chose  a given  a domain,  for  instance  civil 
engineering,  where  it  is  rather  easy  to  divide  the  domain  into  basic  concepts  (such  a concept  would  be 
“Newton”  - apple  falling  from  tree...).  Then  it  should  be  possible  to  create  a topic  map  made  out  of  these 
concepts  and  linking  associations,  thus  covering  the  domain  knowledge  on  an  abstract  level.  The  next  step 
is  to  mediate  these  concepts  through  learning  objects  such  as  course  material  or  resources  (text, 
animation,  movie  and  audio)  which  itself  is  built  so  that  it  is  basically  context  -free,  modular,  reusable  in 
different  places.  Given  those  two  conditions  a learner  might  ‘walk’  his/her  way  not  only  through 
predefined  paths  (defined  by  teacher  for  a certain  course  or  lesson),  but  also  leave  the  path  and  look  via 
the  topic  map  to  adjacent  topics,  and  from  there  to  other  learning  objects. 

XML  and  Metadata  Standards 

XML,  Extensible  Markup  Language  is  a W3C  recommendation  (not  yet  a standard),  derived  from  SGML 
(ISO  standard),  somewhat  similar  to  HTML,  as  it  is  markup,  but  with  freely  definable  tags. 
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Metadata  is  commonly  described  as  being  "'Data  about  data",  which  provides  a means  by  which  the 
multimedia 's  semantics  can  be  described  in  a structured fashion  for  use  by  various  applications.  More 
technically  Metadata  is  standardized  information  to  describe  digital  information  resources  (Domer,  2000 
pp.  74).  Metadata  can  be  seen  as  sets  of  rules  and  meaning/usage  of  tags  lead  to  the  following 
specifications:  IMS,  Dublin  Core,  LOM,  SCORM,  and  EML.  On  a higher  level  Metadata  can  be  viewed 
as  technical/administrative  Metadata  vs.  didactical  Metadata  http://eml.ou.n1/forum/faq/index.htm#Al_0. 

It  is  possible  to  define  one*s  own  tags,  which  allow  a parser  to  know  what  text  between  tags  (element)  is. 
This  gives  not  only  the  freedom  to  separate  the  content  from  the  formatting  (big  advantage,  as  we  can 
now  reformat  according  to  the  meaning  or  being  of  an  element),  but  also  the  freedom  to  define  things 
according  to  our  needs.  This  includes  the  possibility  to  set  up  rules  allowing  or  forbidding  the  existence  of 
an  element  at  a certain  position  (e.g.  within  other  elements)  using  DTDs  or  schemata  a parser  can 
determine  the  validity  of  a given  XML  document  against  a given  set  of  rules.  With  this,  we  have  a first 
step  towards  the  computational  model  mentioned;  machine  readable  content  with  mark-up  allowing  for 
meta information  and  meaning. 

Holzinger  et  al  (2001),  Kassanke  et  al  (2001)  and  Bick  et  al  (2001)  very  ably  unpack  the  development  and 
need  for  educational  Metadata  and  the  instructional  uses  of  XML.  Current  approaches  more  or  less  focus 
on  the  representation  and  structuring  of  learning  contents,  e.g.  the  Learning  Material  Markup  Language 
(LMML)  (Suess,  00).  Another  good  example  for  the  increased  significance  of  XML  in  CAL  is  the  XML 
binding  for  the  Instructional  Management  Systems  Project  (IMS)  Learning  Resource  Meta-data 
Information  Model,  which  is  based  on  the  IEEE  Learning  Technology  Standards  Commitee  (LTSC) 
Learning  Object  Metadata  (LOM)  base  document  (Anderson,  Wason,  00).  (1Bicketal2001  pp.  145) 

The  indexing  could  occur  in  a broader  Metadata  framework  using  such  emerging  tools  as  the  Open 
Archives  Metadata  Harvesting  Protocol,  an  important  new  infrastructure  component  for  supporting 
distributed  networked  resources.  The  Protocol  is  a mechanism  that  enables  data  providers  to  reveal  their 
chosen  Metadata,  whether  it  be  XML  or  some  other.  This  approach  brings  ontological  construction  into  a 
more  comprehensive  systems  and  services  architecture  (Lynch,  August  2001). 

What  kinds  of  issues  need  to  be  considered  pedagogically?  The  current  IMS  specifications  contain 
implicit  pedagogical  models.  The  absence  of  a generalized  representation  of  pedagogical  models  has  the 
potential  to  lock  out  other  models,  and  indeed  ones  that  have  yet  to  be  developed.  This  could  lead  to  all 
kinds  of  practical  problems,  especially  in  terms  of  interoperability.  The  EML  philosophy  is  that  EML 
should  be  capable  of  a complete  description  of  a course,  including  the  content,  the  work  processes 
between  the  actors  (learners  and  staff),  and  that  this  description  should  be  independent  of  the  LMS 
(learning  management  system)  is  order  to  ensure  long-term  interoperability,  sustainability  and 
opportunities  for  re-use. 


Hermeneutic  Analysis 

Hermeneutics  and  phenomenology  are  closely  linked  and  although  they  share  common  concerns,  they 
offer  different  perspectives.  Phenomenologists  focus  on  the  lived  experience  of  people,  and  seek  out  the 
commonalties  and  shared  meanings,  whilst  hermeneutics  refers  to  an  interpretation  of  language. 
Hermeneutics  derives  from  the  Greek  noun  'hermeneia’  which  translates  as  'interpretation'.  Hermeneutics 
has  a dual  value  in  helping  educators  to  understand  how  learners  experience  both  conceptual  chan  ge  and 
how  they  use  the  new  e-leaming  environments,  and  hence  it  has  insights  to  offer  in  terms  of  usability 
testing  and  evaluation.  In  this  sense  hermeneutics  can  offer  a model  for  pedagogy,  to  promote  active 
learning,  whilst  at  the  same  time  acting  as  a research  tool  for  exploring  the  individual  and  collaborative 
learning  experience  (Rogoff,  1990). 

As  a research  methodology,  hermeneutics  assumes  that  meaning  making  is  embedded  in  the  process  of 
dialogue  between  interpreter  and  narrator.  The  hermeneutic  circle  is  a way  of  articulating  and  interpreting 
discourse.  Hermeneutic  phenomenology  is  based  on  the  philosophy  of  Heidegger  and  Gadamer  (van 
Manen,  1 990)  Hermeneutic  phenomenology  is  both  descriptive  and  interpretative.  It  is  an  attempt  to  do 
the  impossible,  to  construct  a full  interpretation  of  some  aspect  of  the  life-world,  and  yet  to  remain  aware 
that  life  is  always  more  complex  than  any  explication  of  meaning  can  reveal,  (van  Manen,  1990  pp.l8) 

A conception  always  has  two  dhiensions,  one  focusing  on  what  people  experience,  i.e.  the  content  or  the 
referential  aspect,  and  the  other  focusing  on  how  someone  thinks  about  the  specific  phenomenon,  i.e.  the 


structural (Marton,  1996)  According  to  phenomenography,  knowledge  is  always  linked  to  specific 
content  and  can  only  be  described  as  knowledge  about  something.  This  means  that  knowledge  is  always 
provisional  and  qualitative.  To  become  more  knowledgeable  in  a subject  implies  a qualitative  change  to  a 
deeper  and  more  complex  understanding  of  a phenomenon.  Marton’s  work  is  a key  influence  in 
establishing  a phenomenographic  approach  to  researching  learning  and  teaching.  Knowledge  objects  can 
be  designed  to  engage  students  in  deep  learning,  which  promotes  real  understanding.  By  students 
engaging  in  critiques  of  existing  theories  and  ideas  new  knowledge  and  understandings  can  arise.  (Marton 
& Booth,  1997) 

Phenomenography  offers  a unique  approach  to  understanding  student  experiences.  The  process  can 
uncover  the  essence  of  a phenomenon  by  gathering  stories  from  students  engaged  with  learning  objects  in 
an  e-leaming  environment,  interpreting  those  stories  and  offering  implications  for  practice.  It  is  akin  to 
action  research  in  that  the  students  are  not  simply  subjects  of  the  study,  but  active  participants.  The 
research  process  creates  opportunities  for  reciprocal  learning,  metacognitive  learning  for  the  student  and 
an  awareness  of  how  the  e-leaming  environment  is  experienced  by  the  researcher. 

Hermeneutic  analysis  can  show  the  development  of  varying  perspectives  from  which  individual  learners 
understand  a task.  It  can  also  track  the  development  of  a joint  approach  to  a collaborative  task  and  can 
illuminate  the  process  of  learning.  A fine-grained  hermeneutic  analysis  can  provided  a framework  for 
'listening  to'  what  learners  can  tell  us  about  the  appearance  of  the  shared  space  to  individual  contributors. 
It  can  tell  us  how  individual  perspectives  are  different  from  one  another,  and  from  our  own  as  researchers 
or  as  instructional  designers.  This  insight  will  prove  to  be  vital  when  evaluating  the  effectiveness  of 
electronic  learning  materials  designed  to  foster  a social  construction  of  knowledge. 

Conclusion 

Can  hermeneutic  analysis  be  used  to  evaluate  pedagogically  based  learning  objects  designed  from 
constructivist  epistemological  ontologies  defined  in  XML  Metadata?  It  really  isnT  just  a string  of 
highfalutin  buzzwords  but  a question  to  stimulate  thinking  about  the  full  cycle  of  development  of  learning 
objects.  This  paper  set  out  to  explore  is  the  nexus  of  conceptual  and  theoretical  foundatbns,  systems 
design  and  current  pedagogical  frameworks  found  to  be  relevant  on  the  road  to  developing  true  learning 
objects.  If  we  can  begin  by  investing  the  time  to  construct  thick  epistemological  ontologies  and  make 
explicit  ontology/ies  through  representational  tools  like  topic  maps,  we  can  begin  to  answer  the  detractors 
who  see  learning  object  development  agenda  as  being  stolen  by  programming  considerations.  These  topic 
maps  in  turn  can  be  translated  into  the  rich  data  carriers  and  renderings  possible  with  XML  and  Metadata. 
The  representation  of  the  domain  in  related  and  merged  topic  maps  has  the  potential  to  allow  learners  to 
articulate,  direct,  share,  relate  and  critique  the  learning  experience  from  their  personal  perspective.  The 
hermeneutic  analysis  discussed  here  gives  designers  and  instructors  the  opportunity  to  do  that  listening  to 
learners.  In  terms  of  usability  testing  this  approach  offers  insights  into  how  learning  is  experienced,  not 
simply  how  the  instructional  designer  conceptualises  it.  The  learning  process  is  not  separate  from  specific 
learning  content  and  context.  The  learning  experience  is  a composite  of  the  learner,  the  domain  and  the  e- 
leaming  environment. 
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Abstract:  Gutman’s  means-end  theory,  widely  used  in  market  research,  identifies  three  levels 
of  abstraction  - attributes,  consequences,  and  values  - associated  with  the  use  of  products, 
representing  the  process  by  which  physical  attributes  of  products  gain  personal  meaning  for 
users.  The  primary  methodological  manifestation  of  means-end  theory  is  the  laddering 
interview,  which  is  claimed  to  generate  better  insights  than  other  qualitative/  quantitative 
methods.  This  study  asked:  Can  means-end  theory,  and  its  concomitant  laddering 
methodology,  be  successfully  applied  in  the  context  of  human-computer  interaction  research, 
specifically  to  help  understand  the  relationships  between  Web  sites  and  their  users?  The  study 
employed  laddering  interviews  to  elicit  data  concerning  Web  site  attributes,  their 
consequences,  and  user  end -values.  This  data  was  duly  processed  and  the  results  were 
subsequently  appraised.  Examination  determined  that  means-end  chains  indeed  characterize 
the  process  by  which  physical  attributes  of  Web  sites  gain  personal  meaning  for  their  users, 
thus  proving  the  theory’s  applicability. 


Introduction 

Gutman’s  means-end  theory  (see  Gutman  1982),  which  discusses  the  core  underlying  values  that 
motivate  consumers’  purchase  decisions,  is  immensely  popular  in  the  marketing  research  community  (Peter  & 
Olson  1994),  and  enjoys  an  impressive  track  record  in  the  commercial  sector  where  it  is  often  used  to  generate 
message  strategy  themes  for  promotional  campaigns  (Gengler  et  al.  1999).  The  most  common  methodological 
manifestation  of  means-end  theory  in  the  field  of  marketing  research  is  laddering,  a method  by  which  the  core 
attributes  and  values  that  motivate  product  users  are  identified  through  a special  form  of  in-depth,  one-on-one 
interviews  that  force  the  respondent  up  a ladder  of  abstraction,  thereby  bridging  relatively  concrete  product 
meanings  at  the  product  attribute  level  with  more  abstract  meanings  at  the  consequence  and  personal  value 
levels  (Gengler  et  al.  1999). 

In  support  of  laddering,  Fortin i- Campbell  (1990)  claims  that  other  quantitative/  qualitative  social 
scientific  research  methods  - such  as  focus  groups,  surveys,  and  demographic  information  - often  fall  short  of 
generating  key  insights  about  a product  and  of  identifying  a target  audience  in  concise  and  well-defined  terms, 
since  common  responses  to  the  question  of  why  a person  chooses  a particular  product  do  not  even  begin  to 
describe  the  deep  underlying  psychological  and  emotional  reasons  that  influence  people’s  choices. 

Meanwhile,  the  celebrated  consumer  psychologist  and  University  of  Minnesota  Professor  Emeritus  Dr. 
William  D.  Wells  suggested  to  this  researcher  that  means-end  theory  could  very  well  prove  to  be  an  excellent 
point  of  reference  for  unearthing  the  underlying  core  values  that  motivate  Web  site  users  as  well,  and  that  he 
did  not  know  of  any  prior  studies  that  have  used  this  theory  in  the  substantive  context  of  Web  usage,  or 
anywhere  else  within  the  broader  field  of  human-computer  interaction  or  HCI  (W.  D.  Wells,  personal 
communication,  July  2000). 

This  study  could  therefore  be  a potentially  valuable  introduction  of  this  new  theoretical  and 
methodological  framework  to  the  field  of  Web  usage  research  in  particular,  and  HCI  in  general.  It  is  hoped  that 
the  insights  uncovered  by  this  study  - along  with  those  generated  by  subsequent  studies  of  a similar  nature  - 
could  be  subsequently  drawn  upon  to  describe  and  delineate  the  characteristics  of  distinct  Web  user  segments, 
through  the  use  of  techniques  such  as  consumer  prototyping,  at  an  appropriate  future  occasion.  Information  of 
this  nature  would  be  of  great  potential  significance  to  academic  researchers,  as  well  as  to  professionals  in  the 
Web  industry. 
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Research  Question 


This  research  study  essentially  poses  the  question:  Can  means-end  theory,  and  its  concomitant 
laddering  methodology,  be  successfully  applied  in  the  context  of  human-computer  interaction  research, 
specifically  to  help  understand  the  relationship  between  Web  sites  and  their  users?  The  study  attempts  to 
explore  this  question  by  employing  a laddering  methodology  with  the  aim  of  eliciting  data  concerning  the  site- 
attributes,  consequences,  and  end-values  that  motivate  Web  users  to  choose  particular  Web  sites  over  others. 
With  the  resultant  data  at  hand,  detailed  analysis  can  be  done  to  conclude  if  means-end  theory  is  indeed  useful 
in  the  context  of  Web  usage  in  particular,  and  HCI  in  general. 


Review  of  Literature 

Means-End  Theory 

Means-end  theory  examines  the  important  meanings  that  users  associate  with  the  products  they  choose 
and  consume,  by  distinguishing  three  levels  of  abstraction  associated  with  the  use  of  a product  (Olson  & 
Reynolds  1983):  product  attributes;  consequences  of  product  consumption;  and  personal  values  of  the  user. 
Product  attributes  are  concrete  meanings  that  represent  the  physical  or  observable  characteristics  of  a product. 
Consequences  of  consumption  are  more  abstract  meanings  that  reflect  the  perceived  benefits/costs  associated 
with  specific  product  attributes.  Finally,  personal  values  are  highly  abstract  meanings  that  refer  to  centrally 
held,  enduring  beliefs,  or  end  states  of  existence,  that  people  seek  to  fulfill  through  their  choice  and 
consumption  behavior  (Rokeach  1973).  Taken  together,  this  pattern  of  associations  from  attributes  to 
consequences  and  from  consequences  to  personal  values  represents  a special  type  of  knowledge  structure  called 
a means-end  chain  (Gutman  1982,  Howard  1977,  Olson  et  al.  1983).  The  means-end  chain  model  provides  a 
simple  way  of  characterizing  the  basic  pattern  of  relationships  by  which  the  physical  features  or  attributes  of 
products  gain  personal  relevance  or  meaning  for  users  (Gengler  et  al.  1999). 


Laddering 

Laddering  is  the  principal  methodological  manifestation  of  means-end  theory.  It  is  a technique  by 
which  the  core  attributes  and  values  that  drive  product  users  are  identified  through  a special  form  of  in-depth, 
one-on-one  interviews  (Wansink  2000).  This  procedure  is  called  laddering  because  it  forces  the  respondent  up  a 
ladder  of  abstraction,  and  thus  bridges  relatively  concrete  product  meanings  at  the  attribute  level  with  more 
abstract  meanings  at  the  consequence  and  personal  value  levels  (Gengler  et  al.  1999).  This  is  done  by 
repeatedly  asking  “Why  is  that  important  to  you?”  questions.  Data  gathered  by  laddering  interviews  is  analyzed 
to  produce  an  accurate  depiction  of  the  concepts  germane  to  the  decision  and  the  relationships  between 
associated  concepts,  facilitating  inferences  and  enabling  the  researcher  to  identify  patterns  that  might  not  be 
evident  in  the  raw  data  (Gengler  et  al.  1995,  1999). 

Reynolds  & Gutman,  in  their  seminal  (1988)  article,  describe  the  processing  of  laddering  data  as 
follows:  The  initial  task  of  the  data  processing  is  to  content-analyze  all  of  the  elements  from  the  ladders, 
recording  the  entire  set  of  ladders  across  respondents  on  a separate  coding  form,  and  developing  a set  of 
summary  codes  that  reflect  everything  that  was  mentioned.  Once  the  summary  codes  are  finalized,  numbers  are 
assigned  to  each.  These  numbers  are  then  used  to  score  each  element  in  each  individual  respondent’s  ladder. 
The  next  step  involves  the  construction  of  an  implication  matrix,  which  displays  the  number  of  times  each 
element  leads  to  each  other  element.  Once  the  implication  matrix  has  been  prepared,  a hierarchical  value  map 
(HVM  ) can  be  constructed,  which  graphically  represents  all  the  respondents’  ladders  in  the  aggregate.  Once  a 
HVM  is  constructed,  one  typically  considers  any  pathway  from  bottom  to  top  as  a potential  chain  representing  a 
perceptual  orientation  that  warrants  consideration. 

Reynolds  & Gutman  (1988)  claim  that  that  such  quantitative  processing  of  qualitative  interview  data  is 
one  of  the  unique  aspects  of  laddering.  They  cite  several  particularly  valuable  types  of  information  afforded  by 
HVMs  obtained  through  laddering  procedures,  which  can  serve  as  a basis  for  segmenting  users  with  respect  to 
their  value  orientations  for  a product  class  or  brand;  assessing  brands  or  products  in  a fashion  similar  to  the  use 
of  more  traditional  ratings;  evaluating  competitive  advertising;  and  developing  advertising  strategies. 
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Method 


Interviewing 

As  part  of  the  study,  50  in-depth  laddering  interviews  were  conducted.  Through  these  interviews,  an 
attempt  was  made  to  examine  two  different  kinds  of  Web  site  use  by  respondents:  use  for  primarily 
entertainment  purposes;  and  use  for  primarily  information  gathering  purposes.  These  distinctions  were  made  on 
the  basis  of  several  preliminary  interviews  with  random  Web  users,  which  suggested  that  they  generally 
browsed  the  Web  either  to  gather  information  relevant  to  their  professional  lives,  or  to  access  content  that 
contributed  to  their  personal  lives  by  entertaining  them.  To  operationalize  the  distinctions  for  the  purpose  of  this 
study,  a particular  Web  site  was  deemed  an  “entertainment”  site  if  the  respondent  described  his/her  use  of  the 
site  as  primarily  entertainment-oriented;  and  was  deemed  an  “information”  site,  if  the  respondent  described 
his/her  use  of  the  site  as  for  primarily  information  gathering  purposes.  Of  the  50  laddering  interviews,  25 
concentrated  on  entertainment  Web  sites  and  the  other  25  on  information  Web  sites. 

The  laddering  interviews  were  conducted  over  the  first  four  months  of  2001.  Respondents  for  the 
interviews  were  of  diverse  ages  and  ethnicities,  came  from  varied  walks  of  life,  and  were  distributed  across 
three  U.S.  states  - Minnesota,  Illinois  and  Connecticut.  In  each  interview,  the  respondent  was  first  asked  to 
name  his/her  most  favorite  Web  site,  and  to  subsequently  define  whether  s/he  considered  it  to  be  an 
entertainment  Web  site  or  an  information  site.  Once  the  particular  genre  of  the  Web  site  (within  the  broader 
entertainment/information  category)  was  established  (e.g.  News,  Astrology,  Sport,  Cooking,  Music, 
Motorcycles,  Chat,  Search  Engine,  etc.),  Reynolds  & Gutman’s  (1988)  “preference-consumption  difference” 
technique  was  used  to  elicit  attribute  distinctions.  The  respondent  was  asked  to  name  two  other  competing  Web 
sites  of  the  same  genre  as  his/her  favorite  site,  and  was  then  asked  why  s/he  favored  this  particular  site  over  the 
other  two.  In  the  interests  of  consistency,  only  the  highest-rated  attribute  was  used  as  the  basis  for  building  an 
attribute-consequence-value  ladder  during  the  remaining  part  of  the  interview. 


Data  Processing 

Strictly  following  the  pioneering  methodological  procedure  established  by  Reynolds  & Gutman 
(1988),  once  all  50  laddering  interviews  were  concluded,  the  first  step  in  analyzing  the  data  gathered  using  the 
interviews  was  to  develop  a set  of  summary  codes  that  reflected  every  attribute/  consequence/  value  mentioned 
by  the  respondents.  Then  followed  the  assignment  of  numbers  to  each  code.  Once  the  codes  were  established 
and  numbered,  their  numbers  were  then  used  to  score  each  element  in  each  ladder  producing  a “raw”  matrix 
(not  shown  here)  with  rows  representing  an  individual  respondent’s  ladder  and  columns  corresponding  to  the 
sequential  elements  within  the  ladder.  The  number  of  columns  corresponded  to  the  number  of  elements  in  the 
longest  ladder.  With  these  coded  raw  matrices  compiled,  the  final  set  of  matrices,  i.e.  the  implication  matrices, 
were  constructed  in  order  to  display  the  number  of  times  each  element  leads  to  each  other  element,  both  directly 
and  indirectly.  In  these  matrices,  the  numbers  are  expressed  in  fractional  form,  with  direct  relations  to  the  left  of 
the  decimal  point  and  indirect  relations  to  the  right.  Once  the  implication  matrices  were  prepared,  work  started 
on  the  construction  of  the  respective  hierarchical  value  maps  or  HVMs.  The  HVMs  were  gradually  built  up  by 
connecting  all  the  chains  formed  by  considering  the  linkages  in  the  implication  matrices,  with  a cut-off  level  of 
two  direct  relations  established  for  the  purpose  of  plotting  the  HVMs  (Fig.  1-2).  In  these  HVMs,  any  pathway 
from  bottom  to  top  can  be  considered  as  a chain  representing  a significant  perceptual  orientation. 


Findings 

Both  sets  of  interviews  raised  six  attribute  codes  each.  It  is  noteworthy  that  10  out  of  12  attribute  codes 
(barring  “Better  Site  Design”  and  “Familiarity  of  Site”)  deal  with  content  issues.  From  this  data,  it  appears  that 
content  has  been  a very  important  factor  in  the  choice  of  Web  sites  for  the  respondents  in  this  study. 
Meanwhile,  it  is  also  interesting  to  note  that  both  sets  of  ladders  elicited  exactly  the  same  value  codes  - 
Satisfaction,  Relaxation,  Happiness,  and  Emotional  Security.  This  indicates  that  both  sets  of  respondents  were 
essentially  motivated  by  the  same  set  of  values  in  their  choice  of  Web  sites,  whether  for  entertainment  or  for 
information. 
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Figure  1 : Hierarchical  Value  Map  for  Entertainment  Web  Sites 

Furthermore,  among  the  consequence  codes,  some  are  reasonably  similar  to  each  other  (e.g.  Save 
Time/Energy,  Find/Interact  with  Other  People,  More  Time  for  Other  Activities,  Success),  while  others  are 
unique  to  entertainment  Web  Sites  or  to  information  Web  sites . There  could  possibly  be  some  valuable  aspects 
of  Web  site  usage  indicated  by  the  above  differences  also. 

Meanwhile,  looking  at  the  HVMs  can  reveal  certain  interesting  segmentations  in  the  manner  by  which 
attributes  are  linked  to  particular  consequences  and  values.  For  instance,  in  Fig.  2,  a clear  division  can  be  made 
down  the  center  of  the  map,  between  the  linkages  leading  to  Positive  States  of  Being  and  on  to  Satisfaction/ 
Emotional  Security  on  the  one  hand,  and  the  linkages  leading  to  Get  Information  Quickly/  Save  Time  and  on  to 
Relaxation/  Happiness  on  the  other.  Similarly,  in  Fig.  1,  the  path  leading  from  Location/Community-Specific 
Content  to  Emotional  Security/  Happiness  is  fundamentally  different  from  that  leading  from,  say.  Larger 
Quantity  of  Content  to  Satisfaction/  Relaxation. 
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Figure  2:  Hierarchical  Value  Map  for  Information  Web  Sites 


Thus,  a marketer  attempting  to  segment  users  of  his/her  information  Web  site  could  reason  thus: 
“There  are  basically  two  types  of  users  of  such  sites.  There  are  those  that  visit  these  sites  in  order  to  achieve 
certain  positive  states  of  being,  which  leads  to  improved  performance/  recognition,  which  in  turn  helps  them  to 
succeed  and  to  avoid  negative  states.  This  helps  them  contribute  to  other  people’s  lives,  thus  achieving 
emotional  security  and  satisfaction.  These  users  seem  to  be  attracted  to  location  and  subject-specific  content. 
Meanwhile,  there  are  those  busy  people  that  visit  solely  to  get  information  quickly,  which  helps  them  save  their 
precious  time.  This  gives  them  more  time  to  devote  to  other  people/activities,  which  brings  them  relaxation  and 
happiness.  Such  users  look  for  site  attributes  such  as  familiarity,  high-quality  content,  and  good  design.” 

Similarly,  the  marketer  for  an  entertainment  Web  site  could  reason  thus:  “There  are  two  types  of  users 
for  sites  like  mine.  There  are  those  that  use  the  sites  to  interface/network  with  other  users,  relate,  build 
relationships,  foster  community  ties,  and  thus  achieve  emotional  security,  happiness,  and  avoid  unwanted 
mental  states.  These  “people -minded”  users  are  attracted  primarily  to  location-specific  content.  Meanwhile, 
there  are  those  who  basically  use  these  sites  as  a means  of  avoiding  unnecessary  efforts.  With  the  time  they 
save,  they  perform  activities  that  helps  them  achieve  positive  goals.  This  helps  them  to  relax,  and  to  succeed, 
which  in  turn  brings  them  satisfaction.  It  looks  like  these  “success-minded”  users  desire  access  to  a large,  varied 
quantum  of  content  through  the  site.” 


Implications 

Therefore,  can  means-end  theory,  and  its  concomitant  laddering  methodology,  be  successfully  applied 
in  the  context  of  HCI  research,  specifically  to  help  understand  the  relationship  between  Web  sites  and  their 
users?  To  begin  with,  the  very  fact  that  all  of  the  50  respondents  who  participated  in  the  laddering  interviews 
were  able  to  name  distinct  site-attributes,  articulate  the  consequences  of  these  attributes,  and  identify  the  end- 
values  served  by  these  consequences,  suggests  that  means-end  chains  do  undeniably  characterize  the  basic 
pattern  of  relationships  by  which  the  physical  attributes  of  Web  sites  gain  personal  relevance  or  meaning  for 


their  users.  It  can  thus  be  proposed  that  means-end  theory  can  indeed  be  applied  in  the  context  of  new  media 
research,  to  help  understand  the  relationships  between  Web  sites  and  their  users. 

Meanwhile,  it  was  also  possible  to  successfully  perform  all  the  analysis  techniques  detailed  by 
Reynolds  & Gutman  (1988)  on  the  data  gathered  by  the  laddering  interviews  conducted  as  part  of  this  study. 
This  included  formulation  of  meaningful  content  codes,  compiling  the  raw  matrices  with  coded  rows 
representing  individual  respondents’  ladders,  preparing  the  implication  matrices  from  the  above  coded  data,  and 
finally  plotting  the  hierarchical  value  maps.  If  means-end  chains  could  not  be  used  to  describe  the  relationships 
between  Web  sites  and  their  users,  it  can  be  argued  that  it  might  not  have  been  possible  to  perform  all  of  the 
above  operations  and  obtain  consistently  meaningful  results.  For  instance,  the  fact  that  all  30  potential 
perceptual  orientation  chains  in  the  HVM  for  entertainment  Web  sites  (Fig.  1)  and  all  31  chains  in  the  HVM  for 
information  Web  sites  (Fig.  2)  make  logical  sense  is  an  example  of  the  results’  meaningfulness. 

Besides,  the  fact  that  a majority  of  the  respondents  named  content-related  site-attributes  as  most 
relevant  to  them  is  in  congruence  with  the  opinion  of  many  leading  webmasters  that  good  content  is  what 
motivates  users  to  visit  a Web  site  more  than  any  other  site  attribute.  To  quote  Silverio  (1995  section  4 para.  2), 
“You  can  make  your  pages  look  absolutely  fabulous  but  if  they  don't  say  anything,  nobody's  going  to  care.” 
Meanwhile,  according  to  Nelson  (2000  para.  7),  “Content  is  everything,”  and  encourages  repeat  visits  by  users. 


Final  Remarks 

The  most  important  contribution  of  this  study,  in  the  researcher’s  opinion,  is  its  successful 
demonstration  of  the  applicability  of  Gutman’s  means-end  theory  - an  important  part  of  marketing  research 
literature  - in  the  context  of  human-computer  interaction  research  (or  more  specifically,  Web  usage  research). 
By  describing  the  relationships  of  50  Web  users  with  their  favorite  Web  sites  in  terms  of  attribute-consequence- 
value  linkages,  the  study  indicates  that  means-end  chains  definitely  characterize  the  basic  pattern  of 
relationships  by  which  the  physical  attributes  of  Web  sites  gain  personal  relevance  or  meaning  for  their  users. 
The  chief  limitation  of  this  study  is  that  the  current  findings  may  not  be  generalizable  to  the  whole  universe  of 
Web  users,  since  the  study  only  involves  50  respondents  selected  on  a non-random  basis.  But  then,  the  object  of 
this  study  was  simply  to  be  an  introductory  demonstration  of  the  use  of  a new  theoretical  and  methodological 
framework  within  the  substantive  context  of  Web  usage  research,  and  not  the  formulation  of  meaningful 
hypotheses  about  the  characteristics  of  Web  users.  The  latter  could  be  the  goal  of  a future  study,  or  meta- 
analysis, that  enjoys  access  to  a much  more  extensive  set  of  interview  data.  This  might  actually  permit  the 
drawing  of  generalizable  inferences,  which  would  be  of  great  significance  to  academic  researchers  of  HCI  as 
well  as  to  professionals  in  the  Web  industry. 
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Abstract:  The  growth  of  a communication  network  technology  enables  us  to  take  part  in  a 
distant  lecture.  Although  many  lectures  are  held  by  using  Web  contents  in  universities,  normal 
lectures  using  a blackboard  are  still  held.  The  later  style  lecture  is  good  for  teacher’s  dynamic 
explanation.  All  the  way  to  modify  it  for  a distant  lecture  is  to  capture  by  a video  camera.  When 
we  video  lecture  scenes  for  the  distant  lecture,  a camera-person  usually  controls  a camera  to 
take  suitable  shots;  alternatively,  the  camera  is  static  and  captures  the  same  location  all  the  time. 
Both  of  them,  however,  have  some  defects.  It  is  necessary  to  control  a camera  automatically.  We 
are  developing  ACE  (Automatic  Camera  control  system  for  Education)  with  computer  vision 
techniques.  This  paper  describes  our  system,  our  camera  control  strategy  and  an  experiment  of 
applying  it  to  a real  lecture. 


1 Introduction 

The  growth  of  a communication  network  technology  enables  people  to  take  part  in  distant  lectures.  There  are 
various  methods  to  held  such  a lecture.  For  example,  it  could  be  held  by  an  web  page-based  method  and  a method 
of  sending  visual  and  audio  of  lecture  scenes.  One  of  the  authors  have  been  studying  some  supporting  systems 
for  a distant  lecture.  In  the  recent  years,  Suganuma  et  al.  [Mine  2000,  Suganuma  2000]  have  been  developing  a 
Computer  Aided  Cooperative  Classroom  Environment  (CACCE)  and  an  Automatic  Exercise  Generator  based  on 
the  Intelligence  of  Students  (AEGIS)  for  the  web  page-based  lecture. 

Indeed  some  lectures,  for  example  the  information  technology  or  the  programming,  are  frequently  held  by  using 
visual  facilities  or  computers  in  many  universities  but  many  lectures  are  held  by  the  traditional  style.  It  seems  that 
such  lectures  will  not  disappear  in  the  future  although  they  will  hold  by  combining  the  blackboard  and  a visual 
facility  such  as  an  OHP  or  Power  Point  software.  We  are,  consequently,  also  developing  another  supporting  system 
for  the  distant  lecture  with  videoing  the  traditional  lecture.  This  paper  describes  an  Automatic  Camera  control 
system  for  Education  (ACE). 

We  envisage  that  scenes  of  a lecture  held  in  a normal  classroom  are  recorded  by  a video  camera  and  students 
in  remote  classroom  take  part  in  the  lecture  by  watching  the  scenes  projected  on  a screen.  Figure  1 illustrates  a 
form  of  the  distant  lecture  by  videoing  the  traditional  classroom.  A teacher  teaches  his  students  in  an  ordinary 
classroom.  There  is  a blackboard  in  the  room.  He  writes  and  explains  something  on  it.  Watching  it  and  listening  to 
his  talk,  students  in  the  room  take  part  in  the  lecture.  Some  cameras  are  setting  in  the  room  and  take  a lecture  scene 
in  order  that  the  captured  scene  is  sent  to  distant  classrooms.  On  the  other  hand,  students  in  the  distant  rooms  take 
part  in  the  lecture  by  watching  a scene  reflected  on  a screen. 

When  we  video  lecture  scenes  for  the  distant  lecture,  a camera-person  usually  controls  a camera  to  take  suitable 
shots;  alternatively,  the  camera  is  static  and  captures  the  same  location  all  the  time.  It  is  not  easy,  however,  to 
employ  a camera-person  for  every  occasion,  and  the  scenes  captured  by  a steady  camera  hardly  give  us  a feeling  of 
the  live  lecture.  It  is  necessary  to  control  a camera  automatically.  ACE  enables  people  to  do  it  for  taking  suitable 
shots  for  a distant  lecture.  Receiving  a scene  from  a camera,  ACE  analyses  it  and  recognizes  the  complexion  on 
the  lecture.  ACE  judges  what  is  important  in  the  scene  and  controls  the  camera  to  focus  on  it.  If  the  sent  scenes 
are  most  suitable  and  effective,  the  educational  effect  on  the  students  in  the  distant  rooms  is  as  good  as  that  of  the 
students  in  the  room. 
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Figure  1 : A form  of  the  distant  lecture  by  videoing  the  Figure  2:  An  Overview  of  ACE 

normal  classroom 


In  this  paper,  section  2 presents  design  of  ACE  and  our  strategy  of  camera  control.  Section  3 describes  the 
algorithm  to  extract  the  latest  object  written  by  a teacher  on  a blackboard,  and  section  4 describes  an  experiment 
to  apply  ACE  to  a real  lecture.  Finally,  concluding  remarks  are  given  in  section  5. 

2 Overview  of  ACE 

2.1  Design 

We  have  designed  and  implemented  ACE,  which  is  an  application  based  on  Computer  Vision  Technique.  When 
we  designed  it,  we  assumed  the  following: 

• A teacher  teaches  his  students  by  using  only  a blackboard. 

• Students  aren’t  reflected  in  the  scenes  captured  by  the  camera. 

• A teacher  isn’t  required  to  give  the  system  a special  cue. 

The  first  assumption  means  that  the  lecture  captured  by  ACE  is  a traditional  one.  The  teacher  writes  something 
on  the  blackboard,  and  explains  them.  Indeed  he  teaches  his  students  using  OHP  and/or  other  visual  facilities  in 
the  resent  years,  but  many  traditional  ones  are  held  in  many  schools. 

The  second  assumption  is  made  to  decrease  processing  costs.  If  students  are  reflected  in  the  scenes,  ACE  always 
has  to  distinguish  a teacher  and  them.  This  processing  is  complex  and  take  much  time.  It  is  easy  to  satisfy  this 
assumption  if  we  take  a scene  from  the  ceiling. 

Finally,  the  third  assumption  is  very  important  for  a teacher.  If  a teacher  gives  ACE  his  special  cue  such  as  to 
press  a button  of  a remote  controller,  ACE  may  control  a camera  more  easily.  ACE  has  only  to  keep  waiting  his 
cue  in  that  situation.  If  the  teacher,  furthermore,  put  on  a special  cloth,  on  which  some  color  markers  are  attached, 
it  is  easier  to  detect  his  position  and/or  action.  The  special  cue  and  the  special  cloth,  however,  increase  the  load 
on  the  teacher.  He  may  omit  to  give  ACE  his  cue.  He  ought  to  concentrate  his  attention  on  his  explaining.  We 
decided,  consequently,  we  didn’t  require  him  to  give  ACE  his  cue. 

The  overview  of  ACE  is  shown  in  Figure  2.  ACE  requires  two  cameras.  One  is  a steady  camera  and  the  other 
is  a active  one.  The  steady  camera  captures  a whole  blackboard  at  a constant  angle  for  image  processing.  The 
captured  image  is  sent  to  ACE  system  running  on  a PC  over  an  IEEE- 1394  protocol.  ACE  analyzes  the  image  and 
decide  how  to  control  the  active  camera  according  to  a camera  control  strategy  shown  in  section  2.2.  The  control 
signals  are  sent  to  the  active  camera  over  an  RS-232C.  The  active  camera,  hereby,  takes  suitable  shots.  The  visual 
and  the  audio  are  sent  to  the  distant  room.  Students  in  the  room  watch  and  listen  to  them,  and  take  part  in  the 
lecture.  In  our  study,  we  are  interested  in  how  to  video  the  lecture  held  in  normal  classroom.  We  are  using  a known 
method  or  product  as  a way  sending  the  video  via  the  network. 
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(a)  An  ordinary  shot 


(b)  A key  shot 


Figure  3:  Sample  shots  of  a lecture  scene  captured  by  ACE 


2.2  Camera  control  strategy 

What  does  ACE  capture?  It  is  a very  important  thing 
for  the  system  such  as  ACE.  One  solution  is  to  take 
the  scenes  that  students  want  to  watch,  but  in  this  case 
many  scenes  are  probably  requested  by  many  students 
at  the  same  time.  Although  this  solution  needs  the  con- 
sensus of  all  students,  it  is  very  difficult  to  make  it. 

We  decide,  therefore,  that  ACE  captures  the  most  im- 
portant things  from  a point  of  view  of  a teacher.  The 
most  important  thing  from  teacher’s  point  of  view  is 
also  difficult.  We  guessed  the  objects  that  teacher  was 
explaining  were  the  most  important  things  for  all  stu- 
dents. When  he  explains  something,  he  probably  wants 
his  students  to  watch  it.  He  frequently  explains  the  lat- 
est object  that  he  have  written  on  the  blackboard.  We 
decided  consequently  that  ACE  captured  the  latest  ob- 
ject written  on  the  blackboard. 

When  the  lecturing  scenes  are  videoed,  both  constantly  changing  shots  and  over-rendering  shots  are  not  suitable. 
A changeless  shots  are,  if  anything,  more  appropriate  than  those  shots.  It  is  important  that  students  can  easily  read 
contents  on  the  blackboard.  The  shots  captured  by  ACE  is  shown  in  Figure  3.  ACE  usually  takes  a shot  containing 
the  latest  object  and  a region  near  it  in  a discernible  size.  The  blackboard  often  consist  of  four  or  six  small  boards 
like  a picture  in  Figure  4.  A teacher  frequently  writes  relational  objects  within  one  board  in  this  case.  Now,  ACE 
takes  a shot  by  the  small  board  such  as  figure  4-(a).  On  the  other  hand,  ACE  takes  a shot  zoomed  in  on  the  latest 
object  after  the  teacher  has  written  it  on  the  blackboard  such  as  figure  4-(b).  After  a- few-second  zooming,  ACE 
takes  an  ordinary  shot  again. 

If  we  take  the  scene  by  a steady  camera,  the  shot  may  be  like  a shot  in  Figure  4.  In  this  case  the  camera  must 
capture  the  whole  blackboard,  because  the  teacher  writes  something  anywhere.  The  characters  in  this  shot  are  too 
small  for  students  to  read.  The  shot  of  ACE  is  superior  to  that  of  the  steady  camera. 


Figure  4:  A sample  shot  captured  by  a steady  camera 


3 Extracting  the  latest  object 

3.1  Background  subtraction 

We  use  a background  subtraction  technique  to  detect  objects  on  the  blackboard.  The  background  subtraction 
technique  is  a method  to  detach  the  foreground  image  from  the  background  image.  In  the  method,  the  background 
image  is  captured  before  opening  the  lecture.  The  image  contains  only  the  blackboard  on  which  written  no  object. 
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It  cannot  contain  a teacher.  We  can  get  some  objects  on  the  blackboard  and  in  front  of  blackboard  when  we  subtract 
the  background  from  the  image  captured  by  the  same  camera  during  the  lecture. 

We  adopted  a background  model[Haritaoglu  1998]  in  our  system.  This  model  is  robust  against  a noise  such  as  a 
flickered  noise  and  so  on.  The  platform  is  lightened  by  fluorescent  lamps  in  a normal  classroom.  There  are  usually 
many  noises  such  as  flickered  ones  in  a shot  when  a video  camera  captures  objects  lightened  by  them.  ACE  needs 
a robust  method  against  noises  for  this  reason. 

In  the  original  background  model,  the  background  image  is  modeled  by  representing  each  pixel  by  the  following 
values;  its  maximum  (Max(p))  and  minimum  (Mm(p))  intensity  values  and  the  maximum  intensity  {D(p))  differ- 
ence between  two  pieces  of  successive  Irames  observed  during  capturing  the  background  image.  Pixel  p is  a back- 
ground pixel  if  the  intensity  of  pixel  p satisfies  the  following:  Min{p)  - I(p)  < D{p)  or  7(p)  - Max{p)  < D(p). 
The  first  inequality  represents  the  lower-bound  in  order  that  the  pixel  belongs  to  the  background,  the  other  repre- 
sents the  upper-bound. 

We  specialized  the  method  for  the  lecturing  scene.  The  foreground  objects  segmented  by  the  technique  are 
something  to  write  on  the  blackboard,  something  to  erase  on  it,  the  teacher  and  so  on.  We  need  only  the  written 
object.  Their  pixels  appear  only  above  the  upper-bound  because  the  object  written  on  the  blackboard  is  brighter 
than  the  blackboard.  Our  method  takes,  therefore,  no  longer  the  pixel  whose  brightness  is  less  than  the  lower- 
bound.  ACE  segments  the  object  by  using  only  the  above  second  inequality. 

The  foreground  objects  are  extracted  by  thresholding  and  noise  clearing.  The  objects  represent  highlighted 
pixels  in  the  background  subtraction  image. 

3.2  Separating  an  object  from  the  foreground  image 

The  foreground  image  almost  always  includes  a teacher.  We  would  like  to  detect  only  the  wntten  object.  If  we 
mask  teacher’s  region,  we  can  get  the  region  correctly.  We  have  to  detect,  therefore,  teacher’s  region.  We  assumed 
that  all  the  moving  object  is  a teacher.  A method  using  a subtraction  between  a frame  and  a frame  that  captured 
after  a short  interval  is  usually  used  when  we  want  to  detect  a moving  object.  ACE  calculates  the  subtraction 
image  and  makes  moving  objects  highlight.  ACE  makes  a rectangle  circumscribed  highlight  pixels  the  temporal 
teacher’s  region.  After  all  pixels  in  teacher’s  rectangle  in  the  foreground  image  are  changed  to  dark  ones,  the 
remnant  highlight  pixels  are  the  written  objects  if  teacher’s  region  is  segmented  correctly  enough.  ACE  makes  a 
rectangle  circumscribed  the  highlight  pixels,  and  deals  with  it  after  this  processing. 

3.3  Remake  the  background  model 

We  have  to  distinguish  the  latest  object  and  others.  ACE  keeps  tracing  the  latest  object  written  by  a teacher  from 
our  camera  control  strategy  described  by  section  2.2.  Once  the  object  has  been  detected  as  the  written  object,  it 
doesn’t  need  to  be  detected  more  than  twice.  After  detecting  the  latest  object,  ACE  re-calculates  the  values  of  the 
background  model  for  each  pixel  in  the  region  of  the  object.  ACE  always  detects,  therefore,  only  the  latest  one. 


3.4  Timing  of  zooming  in 

We  cannot  control  a camera  even  if  we  get  the  region  of  the  latest  object.  We  have  to  find  the  timing  of  zooming 
in.  If  ACE  zooms  in  on  the  written  object  before  a teacher  has  written,  ACE  must  take  a scene  occluding  the  object 
behind  the  teacher  body.  After  guessing  whether  the  teacher  finished  writing,  consequently,  ACE  zooms  in  on  the 
object  written  by  him  at  that  moment. 

The  rectangle  circumscribed  the  latest  object  usually  change  frame  by  frame.  This  main  reason  is  the  following: 

• The  rectangle  increases  or  decreases  because  the  teacher  wrote  something  new  or  erased  something. 

• The  masked  region  changes  because  the  teacher  moved  to  write  something  new.  Then  the  rectangle  increases 
or  decreases. 

Shortly,  the  rectangle  changes  when  the  teacher  is  writing  something.  On  the  other  hand,  he  usually  clears  the 
object  to  make  his  students  watch  it  after  he  has  written.  ACE  take  advantage  of  this  feature  to  guess  whether  he 
finished.  The  rectangle  does  not  change  when  he  cleared  the  object.  ACE  counts  the  number  of  frames  in  which 
the  rectangle  does  not  change.  If  the  number  is  over  a threshold,  then  ACE  judges  the  teacher  finished  writing,  and 
control  a camera  to  zoom  in  on  the  written  object. 
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4 Applying  ACE  to  a real  lecture 

4.1  Condition 

We  have  developed  ACE  and  done  an  experiment  of  apply- 
ing it  to  a real  lecture.  We  held  two  25-minutes  lectures  on 
Mathematics  for  85  undergraduates.  A teacher  taught  them 
by  only  using  a blackboard.  One  of  the  authors  played  the 
role  of  the  teacher.  Although  he  knows  the  detecting  algo- 
rithm of  ACE,  he  taught  by  usual  style.  Sort  of  thing,  he 
behaved  as  disadvantage  for  ACE.  We  videoed  the  lecture 
scene  by  ACE  and  by  a steady  camera  in  order  to  compare 
their  shots.  These  shots  were  recorded  on  video  cassettes 
and  played  them  in  classrooms.  We  divided  85  students  into 
two  groups,  each  group  discretely  watched  the  video  lectures 
projected  on  a screen.  We  evaluated  ACE  in  terms  of  the  fol- 
lowing: 

1 . How  correctly  did  the  student  copy  contents  from  the 
screen  to  his  note? 

2.  How  good  were  the  shots  of  ACE  by  comparison  with 
that  of  the  steady  camera? 

4.2  How  correctly  did  the  student  copy  contents? 

We  designed  ACE  to  capture  the  latest  object  because  a teacher  often  explains  it.  Is  this  strategy  adequate  enough 
to  video  a lecture  scene?  We  evaluated  the  videoing  strategy  of  ACE  from  a point  of  view  that  students  could  make 
a note.  We  made  the  students  make  a note  for  this  evaluation.  We  told  the  students  before  the  lecture  that  we  would 
collect  their  notes  after  the  video  lecture  and  check  that  they  ware  made  properly.  We  precomposed  a master  note 
and  wrote  the  all  contents  of  it  on  the  blackboard  all  through  the  lecture. 

After  the  lecture  we  collected  their  notes  and  compared  them  with  the  master  note.  We  counted  their  missing  in 
each  note.  The  result  is  shown  in  Figure  5.  36%  of  all  the  students  can  perfectly  copy  the  contents  from  the  shots 
captured  by  ACE.  45%  of  the  students  make  only  one  mistake.  There  are  some  small  characters  such  as  the  index 
of  a formula  in  this  lecture  scene.  Many  mistakes  are  caused  by  them.  Almost  all  students  make  not  more  than 
three  mistakes.  Although  there  was  a few  students  that  hardly  copied  the  contents  from  the  screen  to  their  note, 
this  reason  was  that  they  sat  dawn  far  from  the  screen  when  he  took  part  in  the  video  lecture.  As  the  results  of  this 
estimation,  almost  all  students  can  make  a note  correctly  enough  to, learn  it.  The  shots  seem  good  enough. 

4.3  How  good  were  the  shots  of  ACE? 

We  asked  five  questions  to  the  students  after  each  video  lecture.  They  scored  each  scene  captured  by  ACE  and 
the  steady  camera,  from  1 to  5.  The  average  scores  of  each  question  are  shown  in  Table  1.  We  omitted  the  third 
question  for  the  scene  captured  by  the  steady  camera  because  it  always  contains  all  objects  on  the  blackboard.  We 
used  the  t-test  to  compare  these  scores.  Our  null  hypothesis  is  “The  scene  captured  by  ACE  is  as  good  as  the  scene 
captured  by  the  steady  camera.  ” 

The  score  of  the  scene  captured  by  ACE  is  better  then  that  of  the  scene  by  the  steady  camera,  except  the 
first  question.  Since  ACE  focus  on  the  latest  object,  the  scene  does  not  always  contain  a teacher.  Then  the  first 
evaluation  of  ACE  is  lower  than  that  of  the  steady  camera.  On  the  other  hand,  a teacher  is  relatively  small  because 
the  scene  captured  by  the  steady  camera  contains  the  whole  blackboard.  Teacher’s  action  is  hard  to  interpret  in  the 
scene.  This  is  why  the  score  of  the  steady  camera  is  also  low.  Our  null  hypothesis,  therefore,  is  not  rejected  with 
both  1%  and  5%  level  of  significance  in  this  question.  The  point  of  the  third  question  is,  furthermore,  what  we 
abandoned  at  design  of  ACE.  We  were  afraid  that  many  students  could  not  watch  something  to  want  because  ACE 
zoomed  in  on  the  latest  object.  When  they  copy  the  contents,  they  often  miss  watching  what  and  when  they  need, 
if  timings  of  their  watching  do  not  synchronize  with  that  of  ACE’s  zooming  in.  Sure  enough,  the  score  is  lowest 
of  all  scores  of  ACE. 


Figure  5:  How  many  things  did  the  student  miss 
copying? 


Table  1 : Questionnaire 


Questions 

Steady 

ACE 

Null  hypothesis 

1. 

Could  you  watch  the  teacher’s  action  well? 

3.27 

3.19 

Accept 

2. 

Could  you  watch  the  objects  on  the  blackboard  well? 

1.58 

2.75 

Reject 

3. 

Could  you  watch  the  object  you  wanted? 

— 

2.49 

— 

4. 

Were  you  given  a feeling  of  the  live  lecture? 

2.71 

2.86 

Accept 

5. 

Could  you  give  the  scene  a overall  score  as  a lecture  One? 

2.12 

2.49 

Reject 

(1%  level  of  significance) 


Our  null  hypothesis  is  rejected  with  1%  level  of  significance  in  the  second  question  and  the  fifth  question.  In 
both  of  them  the  scores  of  ACE  are  better  than  that  of  the  steady  camera.  The  evaluation  of  ACE  is,  therefore, 
superior  to  that  of  the  steady  camera.  Neither  of  the  scores,  however,  is  high.  In  this  experiment,  the  quality  of  the 
audio  of  our  videos  was  not  so  good.  Teacher’s  voice  was,  especially,  hard  to  hear  because  we  recorded  it  not  by  a 
wireless  microphone  at  his  breast  but  by  a standard  one  on  a desk.  This  is  why  the  magnitudes  of  the  evaluations 
are  not  large.  If  the  quality  had  been  better,  they  would  have  been  larger.  Because  there  were  many  feedbacks  that 
described  the  low-quality  audio.  The  shots  captured  by  ACE  is,  consequently,  good  enough  to  video  a lecture. 


5 Conclusion 

We  have  designed  a camera  control  strategy  for  videoing  a lecture  and  developed  a prototype  of  ACE.  We  evaluated, 
moreover,  it  with  applying  it  to  a real  lecture.  As  a consequence,  we  make  sure  that  ACE  is  a useful  tool  for 
videoing  a traditional  lecture. 

ACE  takes  a suitable  shot  if  the  teacher  explains  the  object  as  soon  as  he  writes  on  the  board.  It  cannot  take, 
however,  a suitable  shot  when  he  explains  something  written  in  past.  He  usually  teaches  his  students  pointing  the 
objects  which  he  wants  them  to  look  at.  Interpreting  teacher’s  action  and/or  posture,  ACE  could  capture  more 
suitable  scene.  We  will  make  ACE  interpret  it.  We  assume  that  a teacher  teaches  his  students  with  a blackboard. 
But  he  sometimes  also  uses  with  an  OHP.  We  will  also  make  ACE  be  applied  to  such  a situation. 
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Abstract:  Popularization  of  computers  and  the  Internet  enable  people  to  hold  lectures  using 
Web  contents  as  a teaching  material.  Although  teachers  have  prepared  a lot  of  Web  contents, 
most  of  them  are  used  so  as  only  to  be  browsed  by  students.  If  we  arrange  some  exercises  ac- 
cording to  lecture  notes  and  prepare  an  answering  mechanism  for  the  exercises  via  the  Internet, 
students  can  attempt  the  exercises  any  time.  This  paper  proposes  AEGIS  (Automatic  Exercise 
Generator  based  on  the  Intelligence  of  Students)  that  generates  exercises  of  various  difficulty 
levels  according  to  each  student’s  achievement  level,  marks  his/her  answers  and  returns  them 
to  him/her.  It  is  necessary  for  AEGIS  to  evaluate  dynamically  both  the  levels  in  order  that  it 
selects  a suitable  question  for  each  student.  This  paper  also  declares  the  validity  of  the  method 
with  a simulator. 


1 Introduction 

As  the  Internet  has  come  into  wide  use,  WWW  environments  provide  lots  of  opportunities  to  various  fields.  In  the 
educational  domain,  many  Internet  technologies  enable  people  to  hold  lectures  using  Web  contents  as  a teaching 
material  and  even  develop  new  lecture  methods  using  the  technologies.  Web  data  are,  therefore,  being  expanded 
rapidly  as  useful  materials. 

We  have  devoted  ourself  to  develop  a Web-based  self-teaching  system  and  to  build  the  tools  for  helping  stu- 
dents understand  their  subjects  [Sato  1997,  Mine  1998,  Fujimoto  1999,  Suganuma  2000].  Through  our  experiences 
teaching  in  classes  and  developing  such  systems,  we  recognize  the  necessity  of  both  a method  evaluating  students’ 
achievement  levels  and  generating  exercises  suitable  for  the  students  automatically. 

For  example,  in  trying  to  make  some  exercises  for  the  students  in  a class,  we  have  to  take  at  least  their  achieve- 
ment level  into  considerations.  The  well-considered  exercises  are  useful  not  only  to  measure  the  achievement  level 
of  the  students,  but  also  to  improve  their  performance.  Unfortunately,  it  is  not  an  easy  task  for  any  teacher  to  make 
exercises  with  the  difficulties  suited  to  their  achievement  levels.  Besides,  it  is  very  important  to  mark  their  answers 
to  the  exercises  and  return  the  marked  results  to  them  for  keeping  their  learning  enthusiasm.  These  tasks  become 
harder  in  proportion  to  the  number  of  the  students  in  a class[Hirokawa  1996]. 

In  this  paper,  we  present  the  automatic  student’s  achievement  level  evaluator  that  generates  exercises  from 
tagged  documents,  presents  them  to  students  and  marks  their  answers  automatically.  We  call  the  system  AEGIS 
(Automatic  Exercise  Generator  based  on  the  Intelligence  of  Students) [Shoudai  2000,  Mine  2000].  AEGIS  gen- 
erates the  three  question-types  from  the  same  tagged  data.  Guessing  the  achievement  level  of  each  student  from 
his/her  trial  history,  AEGIS  selects  the  most  suitable  question-type  and  exercise  for  him/her  according  to  not  only 
his/her  achievement  level  but  also  the  difficulty  level  of  the  exercises.  It  is  necessary  for  AEGIS  to  evaluate  dy- 
namically both  the  levels  in  order  that  it  gives  each  student  a suitable  question.  Although  many  CAI  systems  have 
been  proposed,  our  system  is  different  from  them  in  the  points  of  re-usability  of  pre-existing  electronic  materials 
and  re-estimation  of  both  the  levels.  We  have  already  proposed  the  method  to  re-estimate  them[Mine  2000]  but 
have  not  yet  validated  it  competently.  This  paper  declares  the  validity  with  a simulator. 

In  what  follows  below,  the  remainder  of  this  section  discusses  related  works.  Section  2 describes  the  exercise 
generating  process  by  AEGIS  and  the  specification  of  the  tags  designed  to  generate  an  exercise.  Section  3 describes 
the  algorithm  to  re-estimate  the  achievement  level  of  a student  and  the  difficulty  level  of  a question.  Section  4 
discusses  experiments  and  their  results  to  show  the  validity  of  the  algorithm  described  in  Section  3,  and  Section  5 
shows  the  overview  of  AEGIS.  Finally,  concluding  remarks  are  given  in  section  6. 


1.  multiple-choice  question 

Complete  the  sentence.  Choose  your  answer  from  the 
following  list. 

Data  structures  need  to  be  studied order  to 

understand  the  algorithms. 

(1)  an  (2)  in  (3)  on  (4)  at  (5)  by 

2.  fill-the-gap  question 

Fill  in  the  blank  with  the  right  word. 

Data  structures  need  to  be  studied order  to 

understand  the  algorithms. 

3.  error-correcting  question 

Right  or  wrong?  Correct  the  sentence  if  it  is  wrong. 
Data  structures  need  to  be  studied  an  order  to  un- 
derstand the  algorithms. 


(QUESTION  SUBJECT=”idioms”) 
Data  structures  need  to  be  studied 
(DEL  CAND=”an,on,at,by”)  in  (/DEL) 
order  to  understand  the  algorithms. 
(/QUESTION) 


Figure  2:  The  tagged  data  to  generate  the 
examples  of  three  question-types  shown  in 
Fig.  1 


Figure  1 : Examples  of  three  question-types 


Related  Works  A lot  of  automatic  quiz  generators  have  been  proposed.  Browning  et  al.  proposed  Tutorial  Mark- 
up Language(TML  for  short)  to  generate  questions  automatically  [Browning  1997,  Browning  1998].  TML  has  a 
couple  of  tags  to  specify  a question,  a multiple-choice  and  a message.  It  requires  a correct  answer  in  a multiple- 
choice  tag  to  mark  a student’s  answer  to  the  question.  Carbone  et  al.  proposed  CADAL  Quiz[Carbone  1997], 
which  generates  a multiple-choice  quiz  from  a question  database.  After  marking  a student’s  answer,  CADAL  Quiz 
returns  the  result  to  him/her  and  to  tutors.  Both  of  them  restrict  the  question  type  only  to  a multiple-choice  quiz. 
On  the  other  hand,  ClassBuilder[ClassBuilder]  generates  many  kinds  of  quizzes  and  grades  a student’s  answer. 
However,  all  of  them  do  not  mention  any  effect  of  making  the  difficulty  level  of  question-type  change  according 
to  the  students’  achievement  level.  In  order  to  improve  their  performance  and  keep  their  enthusiasm  to  attempt  the 
quiz  for  a long  time,  it  is  indispensable  to  consider  their  performance  level  for  generating  their  exercise.  This  point 
is  the  difference  from  other  systems.  AEGIS  makes  use  of  pre-existing  electronic  documents  so  as  to  embed  tags 
in  them,  generates  exercises  automatically  with  tagged  documents  according  to  students’  achievement  levels,  and 
re-estimates  both  their  levels  and  the  difficulty  level  of  the  generated  question  through  marking  their  answers. 

2 Automatic  Exercise  Generating 

There  can  be  several  types  of  a question  in  every  subject.  Since  our  aim  is  to  make  a computer  generate  an  exercise 
and  mark  a student’s  answer  to  it,  we  thus  restrict  the  question-types  to  the  following  three:  multiple-choice 
question^  fill-the-gap  question^  and  error-correcting  question.  Figure  1 shows  examples  of  these  question-types. 
All  of  these  question-types  can  be  constructed  from  a sentence  by  replacing  one  or  more  consecutive  words  with  a 
blank  or  a wrong  expression.  We  call  the  region  to  be  replaced,  hidden  region.  We  note  that  these  three  question- 
types  have  different  difficulties  even  though  they  are  constructed  from  the  same  hidden  region. 

The  exercise  generating  process  from  teaching  documents  is  summarized  as  follows:  (1)  Setting  a hidden  region., 
(2)  Selecting  a paragraph  or  sentence(s)  from  teaching  documents,  and  (3)  Constructing  a candidate  list.  These 
three  steps  are  deeply  related  to  the  teachers’  intentions.  It  is  not  easy  to  extract  such  intentions  automatically 
from  the  teaching  documents.  AEGIS  system  thus  deals  with  tagged  documents  including  the  information  such  as 
hidden  regions  and  candidate  lists. 

In  order  to  embed  the  above  three  kinds  of  information  in  teaching  documents,  we  define  the  following  three 
tags:  QUESTION  tag  that  surrounds  a question  region,  DEL  tag  that  indicates  a hidden  region,  and  LABEL  tag 
that  surrounds  the  relevant  sentence/s  to  the  DEL  tag.  Figure  2 shows  the  tagged  data  to  be  used  for  generating 
the  examples  in  Fig.  1.  Replacing  the  word  “in”  which  is  located  between  DEL  tags  with  a blank  generates  the 
example  of  the  fill-the-gap  question.  In  addition,  the  value  of  CAND  “an,on,at,by”  constructs  the  candidate  list  of 
the  multiple -choice  question.  On  the  other  hand,  replacing  the  hidden  region  with  an  element  in  the  value  of  the 
CAND  generates  the  error-correcting  question.  Thus  we  can  generate  three  question-types  from  only  one  question 
region  where  the  above  tags  are  embedded.  The  additional  three  attributes  of  DEL,  which  contain  the  information 
of  the  difficulty  to  solve  an  exercise,  are  LEVEL,  GROUP,  and  REF.  They  specify  the  difficulty  of  each  hidden 


{QUESTION  SUBJECT=”W_^S”)  question  region  (/QUESTION) 

W_S  ::=  word  or  symbol,  where  a backslash  (\)  must  be  added  just  before  the  symbol  if  it  is  a 

comma(,),  double  quOtes(”),  or  a backslash(\). 


(DEL  CAND=”CANDIDATE’ 

’ LEVEL=”PAIR”  GROUP=”ID”  REF=”ID”)  hidden  region  (/DEL) 

CANDIDATE 

::= 

W_S  1 W_S  CANDIDATE 

W_S 

word  or  symbol,  where  a backslash  (\)  must  be  added  just  before  the  symbol  if  it  is  a 
comma  (,),  double  quotes  (”),  or  a backslash  (\). 

PAIR 

::= 

LOW  ’ ’ HIGH 

LOW 

::= 

an  integer  between  1 and  1 0 

HIGH 

::= 

an  integer  between  1 and  1 0 

ID 

keyword 

(LABEL  NAME- TD”)  sentences  (/LABEL) 
ID  ::=  keyword 


Figure  3:  Definition  of  tags  for  exercise  generation 


region,  and  the  connections  to  other  hidden  region. 

Figure  3 shows  the  definition  of  the  tags  described  above  in  BNF.  The  LEVEL  attribute  must  be  provided 
because  AEGIS  initializes  the  difficulty  level  of  the  hidden  region  with  its  value.  On  the  other  hand,  the  others 
are  not  essential  attributes.  When  a value  of  CAND  attribute  is  utilized  as  a distractor,  AEGIS  generates  only  a 
fill-the-gap  question. 


3 Re-estimation  of  Achievement  Level  and  Difficulty  Level 

It  is  very  important  for  AEGIS  to  estimate  an  achievement  level  of  a student  because  it  generates  exercises  ac- 
cording to  the  level.  Such  a student  level  fluctuates  constantly  because  AEGIS  measures  it  whenever  the  student 
answers  a question.  The  achievement  level  of  student  i at  time  t is  calculated  with  the  following  formula: 


Si,t  = 


-f 


jeQ 


if  ^ 6ij  ^ 0 
J6Q 


Otherwise 


where  Q is  a set  of  questions  that  he/she  answered  in  the  recent  30  trials  and  qj^t  stands  for  the  difficulty  level 
of  question  j at  the  time  when  the  achievement  level  Si^t  is  calculated.  The  weight  value,  is  1 if  student 
i correctly  answered  question  j whose  difficulty  level  is  more  than  his/her  achievement  level  Si^t-i  or  he/she 
incorrectly  answered  question  j whose  level  is  less  than  Si^t-i,  or  0 otherwise.  The  achievement  level  of  student 
i is  initialized  to  1 when  he/she  tries  a question  at  first  time.  The  student  who  has  continuously  answered  a lot  of 
questions  correctly,  obtains  a higher  achievement  level,  and  can  attempt  more  difficult  one.  He/she  can  try  more 
difficult  ones  if  he/she  answers  them  correctly  again.  On  the  other  hand,  the  achievement  level  of  the  student 
who  answered  most  of  the  questions  wrongly,  becomes  lower,  he/she  comes  to  attempt  easier  ones.  Continuously 
answering  them  correctly,  he/she  can  gradually  attempt  more  difficult  questions. 

The  difficulty  level  of  a question  is  carefully  configured,  because  AEGIS  uses  it  to  calculate  the  students’ 
achievement  level  and  refers  it  to  generate  a suitable  question  for  a student.  Since  the  teachers  set  it  up  with  the 
attribute  LEVEL  of  DEL  tag,  they  can  assign  the  upper  and  lower  limits  of  the  difficulty  level  of  a hidden  region. 
However,  there  may  be  a gap  between  the  difficulty  level  evaluated  by  the  teachers  and  that  found  by  the  students. 
A question  evaluated  by  the  teachers  as  an  easy  one,  may  not  always  be  answered  correctly  by  lots  of  students,  and 
vice  versa.  AEGIS,  therefore,  utilizes  the  value  of  the  attribute  LEVEL  as  an  initial  value  of  the  difficulty  level  of 


Figure  4:  Variance  of  students’  achievement  level 
evaluated  by  AEGIS  in  our  simulation 


Figure  5:  Variance  of  questions’  difficulty  level  eval 
uated  by  AEGIS  in  our  simulation 


the  hidden  region,  and  calculates  the  level  dynamically  at  regular  intervals  with  the  following  formula: 


= 


ies 


ieS 


Otherwise 


where  S is  a set  of  students  who  answered  the  question  j between  time  (t—1)  and  t,  Si^r  is  a student’s  achievement 
level  at  time  r{t-l  <r  <t).  The  weight  value,  stands  for  1 if  students  whose  achievement  level  is  more  than 
the  difficulty  level  Qj,t-i  of  question  j answered  it  wrongly  or  students  whose  level  is  less  than  qj,t-i  answered 
it  correctly,  or  0 otherwise.  This  equation  is  a recurrence  formula,  and  Qj,o,  which  is  the  initial  difficulty  level  of 
question  j,  is  given  with  the  attribute  LEVEL  of  DEL  tag  by  teachers. 

The  above  formula  calculates  a new  difficulty  level  of  the  question  for  each  question-type.  Since  the  teachers 
assign  the  upper  and  lower  limits  of  the  difficulty  level  of  a hidden  region  with  the  attribute  LEVEL  of  DEL 
tag,  AEGIS  sets  the  upper  (lower)  limit  as  the  initial  difficulty  level  of  the  question  to  be  generated  in  the  error- 
correcting  question  (multiple-choice  question).  That  of  the  fill-the-gap  question  is  assigned  the  mean  of  the  upper 
and  lower  limits.  After  calculating  the  new  difficulty  levels,  AEGIS  sets  the  maximum  (minimum)  value  among 
the  three-question  types  as  the  upper  (lower)  limit  of  the  difficulty  level  of  the  hidden  region. 


4 Evaluation  with  Simulator 

AEGIS  estimates  dynamically  both  the  achievement  level  of  each  student  and  the  difficulty  level  of  each  question 
with  the  equations  defined  in  Section  3.  In  order  to  examine  their  validity,  we  investigated  the  following  three 
things:  1.  How  does  AEGIS  estimate  the  inherent  achievement  level  of  a student?  2.  How  does  AEGIS  estimate 
the  inherent  difficulty  level  of  a question?  3.  Can  AEGIS  provide  only  questions  suitable  for  a student? 

We  assumed  that  the  probability  that  a student  answers  a question  follows  the  function  of  both  the  inherent 
difficulty  level  of  the  question(q(^^^^^)  and  his/her  inherent  achievement  level  (s^^^^^^),  and  that  he/she  can 
correctly  answer  the  question  with  50%  probability  if  is  equal  to 

1 . Re-estimation  of  the  achievement  level  of  a student 

Based  on  the  student’s  achievement  level  calculating  formula  defined  in  Section  3,  we  made  AEGIS  calculate 
the  achievement  level  of  three  students  whose  inherent  achievement  levels  are  high  = 

8),  middle  = 5)  and  low  = 2).  We  used  1 00  questions  whose  difficulty  levels  are 

distributed  at  the  equal  interval  from  0 to  10. 

Figure  4 shows  the  variance  of  Since  all  of  their  levels,  are  set  to  1 at  first,  the  three 

curves  in  the  graph  start  from  1.  They  were  gradually  separated  each  other  because  they  answered  questions 


o 

ERIC 


1$39 


Figure  6:  Distribution  of  the  questions  that  AEGIS 
gave  to  students  in  our  simulation 


Teachers  Students 


correctly  or  incorrectly  based  on  their  inherent  achievement  level.  Each  curve  of  in  our  simulation 

approximates  closely  the  value  of  that  we  presupposed,  AEGIS  can  consequently  distinguish 

between  them, 

2,  Re-estimation  of  the  difficulty  level  of  a question 

Based  on  the  question’s  difficulty  level  calculating  formula  defined  in  Section  3,  we  made  AEGIS  calculate 
the  difficulty  level  of  three  questions  whose  inherent  difficulty  levels  are  high  = 8), 

middle  = 5)  and  low  = 2).  We  at  first  set  to  a different  value  far  from 

g{TRUE)^  that  is,  questions  whose  inherent  levels  were  5 or  8 was  set  to  1,  and  that  of 

question  whose  inherent  level  was  2,  was  set  to  8. 

In  order  to  estimate  the  movement  of  the  difficulty  level  of  these  questions,  we  used  the  two  real  distributions, 
called  Class  A and  Class  B,  of  achievement  level  of  students  who  were  given  a lecture  by  one  of  the  authors. 
Class  A is  superior  to  Class  B,  We  assumed  that  the  distributions  did  not  change  during  our  experiment. 

The  curve  of  is  shown  in  Fig.  5 for  the  data  of  Class  A.  Each  curve  of  approximately 

converges  to  its  inherent  difficulty  level  The  students  in  Class  A tried  to  solve  a lot  of  questions 

of  higher  difficulty  level.  Although  we  would  have  expected  that  the  current  difficulty  level  of  a question 
tried  by  many  students  would  influence  the  change  of  the  new  difficulty  level  of  a question,  the  result  of  our 
experiment  on  Class  B becomes  similar  to  the  curve  in  Fig,  5.  We  can  conclude  that  our  method  can  well 
estimate  the  difficulty  level  of  each  question, 

3.  The  difficulty  level  of  questions  generated  by  AEGIS 

We  prepared  three  trial  histories  each  of  which  belonged  to  high,  middle,  and  low  achievement  level  stu- 
dents, We  applied  1 ,000  questions  of  various  difficulty  levels  to  these  students  so  as  to  confirm  that  AEGIS 
generates  good  questions  suitable  for  the  students’  achievement  level.  We  assume  that  each  100  questions 
are  uniformly  distributed  between  difficulty  levels  0 and  10, 

Figure  6 shows  the  distributions  of  the  difficulty  levels  of  questions  which  each  student  tried  to  solve,  A 
student  of  high  (resp.  middle,  low)  achievement  level  tried  a lot  of  questions  of  high  (resp.  middle,  low) 
difficulty  level.  Let  (x,  y)  be  a pair  of  the  mean  value  x and  the  standard  deviation  y of  each  distribution  of 
the  curve  in  Figure  6,  (x,y)  of  each  curve  is  (1.6,  0.78),  (4.6,  0.87)  and  (7.6,  0.96),  respectively.  The  result 
shows  that  AEGIS  generates  questions  suitable  for  the  students’  achievement  level. 


5 Overview  of  AEGIS 

The  AEGIS  system  consists  of  three  databases:  Exercise  DB  (EDB  for  short).  User  Profile  DB  (UPDB  for  short) 
and  Level  Management  DB  (LMDB  for  short),  and  three  main  database  managers:  Exercise  Generator ^ Answer 
Evaluator  and  Level  Manager\fi/L\nt  2000].  The  overview  of  AEGIS  is  shown  in  Fig.  7. 
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Teaching  documents  with  the  tags  are  compiled  into  the  EDB  and  LMDB.  All  of  the  question  regions  are 
indexed  sequentially  and  each  hidden  region  is  labeled  with  its  own  subindex  of  the  index  of  each  question  region. 
The  level  of  a hidden  region,  which  is  deeply  related  to  the  level  of  the  question  to  be  generated  from  the  hidden 
region,  is  stored  in  the  LMDB  together  with  the  index  of  the  hidden  region.  The  level  of  each  hidden  region  in 
LMDB  is  reexamined  regularly.  UPDB  keeps  students’  trial  histories  with  their  current  achievement  levels. 


6 Conclusion  and  Future  Work 

We  discussed  our  new  Web-aided  system  AEGIS.  The  system  is  currently  implemented  in  Perl  scripts,  PostgreSQL 
and  CGI.  AEGIS  measures  the  student’s  achievement  level  every  time  he/she  answers  a question,  and  regularly  re- 
estimates  the  difficulty  level  of  questions  in  order  to  generate  suitable  exercises  according  to  his/her  achievement 
level.  AEGIS  is  consequently  utilized  as  not  only  a system  generating  exercises  but  also  a tool  classifying  questions 
because  the  re-estimated  level  keeps  close  to  their  real  difficulty  level. 

Experimental  results  with  the  simulator  showed  the  effectiveness  of  the  algorithm  estimating  both  the  achieve- 
ment level  of  a student  and  the  difficulty  level  of  a question  as  were  expected.  We  have  a plan  to  evaluate  this 
system  by  applying  it  to  the  real  courses  of  Computer  Literacy,  which  are  taken  by  more  than  2300  students  at 
Kyushu  University.  We  hope  it  will  work  fine  as  an  educational  tool  for  every  student  and  help  him/her  to  un- 
derstand his/her  subjects.  Also,  we  will  implement  a tagging  tool  and  an  algorithm  to  generate  another  kind  of 
exercise  that  allows  more  than  one  correct  answers. 
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Abstract:  This  paper  describes  the  theoretical  basis  for  a constructivist  design  which 

can  help  students  develop  a set  of  mental  models  that  increasingly  correspond  to 
scientific  models  that  explain  or  predict  scientific  phenomena.  These  student -constructed 
models  can  be  linked  to  form  perceptual -tb-conceptual  bridges  via  two  complementary 
instructional  pathways:  (1)  interactive  miltimedia  simulations  can  enhance  the  student 
decision-making  process  in  a learning  environment  by  providing  guidance  through  trail- 
and-error  experiences  that  culminate  in  productive  experimental  outcomes;  and  (2) 
student-generated  modeling  software  can  allow  visualization  of  a simple  mechanism, 
which  is  shown  as  perceptual  icons  which  can  be  combined  in  a meaningful  manner  that 
shows  conceptual  entities  and  their  interactions.  The  design  of  both  types  of  software  can 
support  scaffolded  student  understanding  of  the  connection  between  a scientific 
phenomenon  (studied  in  the  laboratory)  and  its  underlying  scientific  principle  (studied  in 
lecture). 


Introduction 

With  the  help  of  scaffolded  support  provided  by  instructional  design,  students  can  begin  to  generate 
mental  models  based  upon  their  interactive  experiences  with  simulated  phenomena  that  allow  them  to  visualize 
scientific  principles.  Furthermore,  modeling  software  can  be  designed  to  help  them  construct  their  own 
scientific  models  which  are  composed  of  simpler  models  that  provide  a concrete  mechanism  for  them  to  build 
more  complex  models  (White  1993,  Frederiksen  et  al.  1999).  In  other  words,  simulations  and  modeling 
software  can  serve  complementary  functions  when  the  former  provides  a decision-making  task  that  allows 
student  prediction  of  a phenomenon  (i.e.,  the  simulation  outcome),  while  the  latter  provides  an  explanation  of  a 
phenomenon  in  terms  of  its  underlying  model  that  drives  the  simulation. 

In  the  domain  of  chemistry,  several  researchers  have  found  that  student  understanding  of  chemical 
principles  and  phenomena  are  linked  to  their  ability  to  visualize  the  particulate  level  of  matter  (Gabel  1993)  and 
then  to  use  this  level  to  connect  the  more  abstract  principles  to  the  more  concrete  phenomena.  Thus,  it  follows 
that  if  students  use  the  particulate  level  to  construct  their  own  mental  model,  then  they  could  use  that  model  to 
predict  the  outcome  of  a simulated  phenomenon.  This  paper  features  the  pedagogic  content  knowledge  needed 
to  begin  instructional  design  for  these  two  forms  of  learning  technology,  i.e.,  simulations  and  modeling 
software,  that  serve  complementary  functions. 

Traditional  instruction  usually  does  nothing  to  help  students  (Kim  et  al.  2000)  because  phenomena  and 
principles  are  studied  in  different  places  (laboratory  vs.  lecture  room)  and  usually  at  different  times  during  the 
semester.  Also,  it  focuses  on  a passive,  information-communicating  method  of  instruction.  Conversely,  this 
paper  emphasizes  the  use  of  technology  to  provide  an  active,  constructivist  approach  that  is  designed  to  engage 
students  in  authentic  experiences  in  scientific  thinking  and  activity  (White  1993). 
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Goal 


The  major  goal  of  this  paper  is  to  describe  how  two  related  types  of  software  can  be  designed  to 
complement  each  other:  (1)  interactive  multimedia  simulations  (IMS^  can  provide  opportunities  for  student 
decision-making  regarding  the  experimental  conditions  (parameters)  and  observation  of  the  consequences  of 
these  decisions  when  their  simulation  is  run;  and  (2)  student-generated  modeling  software  (SGM)  can  allow  student 
construction  of  their  own  model  within  the  constraints  set  by  the  appropriate  scientific  models.  Thus,  the 
students  are  generating  their  own  model,  which  can  serve  to  describe  a simple  mechanism  that  explains  the 
underlying  abstract  features  of  a scientific  phenomenon. 


Concrete  Level  Versus  Abstract  Level 

The  abstract  level  of  a scientific  principle  represents  the  apex  of  a hierarchical  pyramid  that  is  based  on 
a foundation  of  concrete-level  phenomena.  Experts  can  use  these  abstractions  to  explain  a wide  range  of 
phenomena  and  to  apply  them  appropriately  when  solving  complex  problems.  Also,  they  can  use  these 
abstractions  to  organize  information  into  larger  chunks  of  information,  which  can  reduce  the  load  on  working 
memory.  In  sharp  contrast,  novices  tend  to  view  these  abstractions  as  information  to  be  memorized  or  as 
algorithms  to  be  applied  to  problems  in  a very  inflexible  manner.  Thus,  they  can  only  engage  in  constraint- 
based  reasoning  that  uses  abstract  principles  (e.g.  P V = n R T)  as  variables  in  an  equation  which  they 
manipulate  to  find  a quantitative  answer  (Frederiksen  et  al.  1999).  On  the  other  hand,  when  novices  use  SGM 
software  they  can  benefit  from  a causal-based  reasoning  strategy  that  they  have  learned  through  their  own 
construction  of  scientific  models.  A laboratory  setting  can  give  students  “hands  on”  experience  with  a domain - 
specific  phenomenon;  however,  they  may  be  so  busy  manipulating  the  apparatus  that  they  fail  to  observe  the 
phenomenon  as  it  occurs  in  real  time.  Conversely,  an  IMS  of  the  same  phenomenon  replaces  the  physical 
manipulation  of  experimental  apparatus  with  the  mental  manipulation  of  the  experimental  parameters  that 
determine  the  experimental  outcome.  Thus  students  participate  in  experimental  design  rather  than  mimicking  a 
pre -determined  procedure  that  may  have  little  meaning  to  them. 


Intermediate  Level  of  Abstraction 

Both  IMS  and  SGM  software  emphasize  an  intermediate  level  of  abstraction  that  connects  and  gives 
meaning  to  concrete-level  phenomena  and  abstract-level  symbolic  notation  systems  (mathematical  equations  or 
verbal  generalizations).  This  intermediate  level  usually  displays  icons  (dots  for  molecules  or  arrows  for  vectors) 
that  help  students  visualize  abstract  objects  (atoms,  electrical  charges,  or  biomolecules)  and/or  processes 
(chemical  reactions,  flow  through  electrical  circuits,  or  cellular  metabolism).  The  icons  serve  as  cybernetic 
manipulatives  (Kaput  1995)  that  are  similar  to  physical  manipulatives  (molecular  model  kits,  or  wooden 
counting  blocks). 


Interactive  Multimedia  Simulations 

An  IMS  can  fully  engage  the  decision-making  processes  of  students  while  extending  the  range  of 
phenomena  that  can  be  explored.  IMS’s  use  models  of  scientific  phenomena  and  data  from  real  experiments  to 
allow  students  to  determine  the  experimental  parameters  and  hence  to  design  experiments.  Upon  running  the 
experiment,  they  receive  feedback  about  the  experimental  outcome.  If  the  outcome  does  not  meet  their 
expectations,  they  can  change  the  experimental  conditions  (parameter(s)  or  their  values)  and  rerun  the 
experiment.  In  addition,  an  IMS  enables  the  student  to  create  idealized  simplifications  for  scientific  systems  and 
to  transform  abstractions  into  concrete  manipulative  representations  (White  1993).  This  transformation  allows 
them  to  see  dynamic  representations  of  phenomena  that  are  connected  to  more  abstract  representations.  Thus, 
they  can  relate  and  apply  their  formal  knowledge  to  their  everyday  situations  (White  1993).  This  “intuitive 
knowledge”  is  a flexible  and  generative  type  of  knowledge  (diSessa  2000)  that  can  be  used  as  the  building 
blocks  to  construct  conceptual  knowledge. 


We  plan  to  illustrate  the  effectiveness  of  an  IMS  with  an  example  that  can  be  found  at 
WWW.  ri verdeep.net  under  the  “High  School  Science  Gateways’*  section  and  the  “Understanding  Chemical 
Equilibrium”  subsection.  Specifically,  an  acid^ase  titration  allows  the  student  to  explore  an  animated  sequence 
that  shows  how  acids  and  bases  react  during  the  titration  process.  The  Logal  Express  simulation  engine  used  in 
this  IMS  was  selected  for  the  following  reasons:  (1)  it  is  interactive  and  it  provides  students  with  a microscopic 
visualization  of  chemical  concepts  and  related  graphic  representations;  (2)  it  provides  a comprehensive  content- 
rich  chemistry  curriculum,  correlated  to  National  Standards^  and  (3)  it  features  a user-friendly  interface,  a 
scripting  language,  and  a simulation  engine  thatallows  the  authors  (and  their  graduate  students)  the 
opportunities  to  customize  and  modify  existing  simulation  activities  to  suit  our  specific  curricular  and 
instructional  needs. 

Abstract 
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Figure  1:  Student  interactions  with  models,  simulations,  and  phenomena  (Diack  & Suits  2001,  Suits  & 

Hypolite  2002;  Originally  adapted  from  Raghavan  & Glaser  1995). 
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Student-Generated  Modeling  Software 

Properly  designed  SGM  software  allows  student  construction  of  the  underlying  scientific  model  that 
drives  the  IMS  explorations  described  above.  For  example,  in  an  SGM -based  titration,  icons  can  represent  the 
active  part  (H+)  of  an  acid  molecule  (HA,  A is  a negative  ion)  and  the  active  part  of  a base  (OH-)  from  an  ionic 
compound  (MOH).  The  student  can  select  the  amount  of  acid  and  base  to  represent  four  reaction  conditions 
{Frames):  (1)  the  starting  condition  in  which  only  acid  are  present  in  the  receiving  flask,  (2)  the  “buffered” 
condition  in  which  H+  predominates  over  OH-,  (3)  the  equivalence  point  at  which  the  amount  (moles)  of  H+  is 
equal  to  OH-,  and  (4)  the  final  condition  in  which  OH-  predominates  over  H+.  If  a molecular  model  kit  is  used, 
then  the  student  must  disassemble  each  condition  {Frames  1 to  4)  before  reassembling  the  next  condition.  Thus, 
it  is  difficult  for  novices  to  understand  the  causal  relationships  between  the  conditions  because  working  memory 
is  limited  (Sweller  et  al.  1998).  Conversely,  a computer  can  “run”  each  condition  in  the  proper  order  {Frames  1 
to  4)  to  show  the  progression  of  interactions  among  chemical  species  (H+,  OH-,  and  the  product  H2O).  Thus, 
the  student  has  created  a situation  model  that  serves  as  an  “external  memory  aid”  (Newton  1996)  for  each 
condition  (/  to  4).  In  addition,  the  computer  can  present  an  “animated  filmstrip”  (Williamson  & Abraham  1995) 
which  can  serve  as  a sequence  of  discrete  causal  events  that  follow  a simple  mechanism  (White  1993). 


OH- 

OH- 

HOH 

HOH 

HOH 

OH- 

HOH 

OH- 

OH- 

HOH 

OH- 

HOH 

HOH 

HOH 

HOH 

HOH 

HOH 

HOH 

HOH 

HOH 

HOH 

HOH 

HOH 

HOH 

H+ 

H+ 

H+ 

H+ 

H+ 

H+ 

H+ 

H+ 

H+ 

H+ 

H+ 

H+ 
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Figure  2:  Student  construction  of  four  situational  models  (frames)  and  their  links  in  an  animated  sequence 

In  our  initial  trial  with  this  SGM  example,  we  audio-taped  a student  as  he  built  the  four  reaction 
conditions  (jFrames  1 to  4)  described  above.  The  student  had  just  finished  his  second  semester  of  college 
general  chemistry  lecture  and  a 2-credit  hour  laboratory  techniques  course.  Prior  to  his  modeling  experience,  he 
had  done  countless  calculations  involving  titration  data  for  his  lecture  courses,  plus  physical  manipulation  of  the 
titration  glassware  during  an  acid-base  titration  experiment  in  the  laboratory.  However,  it  was  only  during  the 
building  of  these  models  {Frames  1 to  4)  that  he  “suddenly  realized”  that  mixing  acid  (H+)  and  base  (OH-)  “just 
made  water.”  In  other  words,  he  was  able  to  visualize  the  essence  of  an  acid-base  chemical  reaction  for  the  first 
time  despite  his  more  concrete  experiences  with  acids  and  bases  in  the  laboratory  and  his  more 
abstract/mathematical  calculations  performed  during  his  lecture  courses.  He  recommended  that  this  “model 
building”  exercise  be  used  as  a pre-laboratory  activity  for  students  so  that  they  could  understand  the  titration 
process  before  they  actually  performed  the  experiment.  Currently,  we  are  working  to  carry  out  this  student’s 
recommendation  by  designing  an  appropriate  SGM  software  program  on  this  chemistry  topic. 

This  causal  model  enables  students  to  envision  causal  mechanisms  that  are  implicit  in  an  abstraction 
(e.g.,  calculatbns  used  to  solve  titration  problems).  The  abstraction,  in  turn,  allows  the  student  to  predict  and 
explain  a wide  range  of  domain  phenomena  (White  1993).  For  example,  the  students  could  use  the  same  iconic 
manipulatives  to  represent  the  process  of  precipitation  of  a solid  when  to  aqueous  solutions  of  ions  are  mixed 
together.  Also,  this  visualization  process  helps  students  encode  a complex  set  of  information  as  a simpler 
knowledge  structure  (situation  model)  that  can  be  stored  and  recalled  as  needed.  They  can  literally  “see” 
linkages  between  a set  of  these  structures  (Frederiksen  et  al.  1999)  because  the  “emergent  properties”  of  the 
lower  level  model  (e.g.,  each  individual  condition,  1 to  4)  become  the  primitive  elements  of  the  more  abstract, 
derived  model  (e.g.,  the  sequence  of  conditions).  Overall,  two  distinctive  but  complementary  mental  pathways 


can  provide  an  intermediate  level  of  abstraction:  (a)  intuitive  knowledge  developed  from  experiences  (diSessa 
2000),  and  (b)  causal  reasoning  models  (White  1993). 

Implications  for  Instructional  Design 

The  design  of  IMS  and  SGM  technologies  should  be  most  effective  when  three  levels  of  design  are 
used  to  make  instructional  decisions  (Lavoie  1995,  Suits  & Courville  1999,  Suits  & Lagowski  1981).  Briefly, 
the  goal  fcr  the  primary  level  of  design  is  to  help  students  solve  a scientific  problem.  Thus,  instructional 
features  must  be  constrained  by  the  structure  of  the  scientific  discipline  (Suits  & Courville  1999).  In  other 
words,  the  nature  of  the  learner’s  interaction  with  the  simulated  phenomena  must  be  authentic  with  respect  to 
scientific  principles.  On  the  secondary  level,  the  goal  should  be  to  use  instructional  design  principles  as 
external  constraints  designed  to  keep  learners  “on  task”  such  that  the  level  of  task  demands  roughly  matches  the 
student’s  level  of  knowledge  and  skills  (Suits  & Courville  1999).  Finally,  on  the  tertiary  design  level,  the 
student  should  be  challenged  to  solve  a complex  scientific  problem  (primary  level)  while  being  provided  with 
just  enough  guidance  to  sustain  their  progress  in  solving  the  problem  (secondary  level).  Guided  inquiry  (Suits 
& Lagowski  1981,  Zuckerman  et  al  1998)  allows  the  integration  of  multiple  sources  of  information  without 
overwhelming  the  learner’s  cognitive  load  capacities  (Sweller  et  al.  1998).  This  form  of  instruction  is 
consistent  with  the  constructivist  view  (Collins  1996)  in  which  students  can  invent  and  adapt  cultural  tools  for 
thinking  about  scientific  ideas  (Zuckerman  et  al  1998).  The  instructional  design  of  IMS  and  SGM  technologies 
should  help  students  use  sketches  and  mental  pictures  (icons)  as  visual  representations  (Kozma  et  al  2000)  to 
build  a bridge  between  the  perceptions  of  the  simulated  phenomenon  (Kleinman  et  al  1987)  and  their 
conceptions  of  abstract  scientific  principles  (Connell  1998).  Thus  the  purpose  of  this  paper  is  to  illustrate  how 
technologies  can  scaffold  higher-order  learning  tasks  that  allow  student  freedom  to  explore  a phenomenon  while 
constraining  student  expression  towards  correct  scientific  representations. 

IMS  and  SGM  technologies  can  provide  the  scaffolded  support  that  students  need  to  connect  their 
perceptions  of  phenomena  to  the  conceptions  of  abstract  scientific  principles.  Technology,  based  upon  good 
tutoring  practices,  should  be  designed  (Roehler  & Cantlon  1997)  to:  (1)  offer  explanations,  (2)  elicit  active 
student  participation,  (3)  verify  and  clarify  student  understandings,  (4)  model  objects  and  actions  to  make 
thinking  visible  (Linn  1995),  and  (5)  encourage  students  to  be  autonomous  learners  (Linn  1995)  who  are 
responsible  for  their  own  learning.  Furthermore,  scaffolded  guidance  should  be  faded  as  the  learner  gains  the 
ability  to  construct  and  integrate  knowledge  within  a scientific  domain.  This  paper  describes  the  basis  for  some 
of  the  instructional  design  questions  that  should  be  addressed  when  IMS  and  SGM  are  used  to  provide 
perceptual-to-conceptual  bridges  to  help  students  connect  phenomena  to  scientific  principles.  Instructional 
design  can  produce  software  that  initially  models  a domain  at  a low  level  of  complexity  and  employs  causal 
reasoning,  which  exhibits  a simple  mechanism  at  an  intermediate  level  of  abstraction  (White  1993).  This 
process  should  supply  scaffolded  guidance  (Roehler  & Cantlon  1997)  that  is  faded  as  the  student  gains  domain- 
specific  competence  and  confidence  in  his/her  performance.  This  guidance  can  help  make  “thinking  visible” 
when  students  are  aware  of  their  selection  and  use  of  problem-solving  strategies  (Linn  1995). 
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LEARNING  THROUGH  ONLINE  DISCUSSION: 
WHERE'S  THE  BEEF? 


MODERATOR:  Karen  Swan,  University  at  Albany 

Asynchronous  online  discussion,  while  significantly  different  from  face-to-face  communication, 
seems  a significant  factor  in  the  success  of  online  courses.  In  online  discussion,  all  students  have  a voice 
and  no  students  can  dominate  the  conversation.  Accordingly,  many  researchers  note  that  students  perceive 
online  discussion  as  more  equitable  and  more  democratic  than  traditional  classroom  discussions  (Harasim, 
1990;  Levin,  Kim  & Riel,  1990).  Ruberg,  Moore  and  Taylor  (1996),  for  example,  found  that  computer- 
mediated  communication  encouraged  experimentation,  sharing  of  ideas,  and  increased  and  more  distributed 
participation. 

Because  it  is  asynchronous,  online  discussion  also  affords  participants  the  opportunity  to  reflect  on 
their  classmates’  contributions  while  creating  their  own,  and  to  reflect  on  their  own  writing  before  posting 
it.  Researchers  argue  that  this  creates  a certain  mindfulness  among  students  and  a culture  of  reflection  in  an 
online  course  (Hiltz,  1994;  Poole,  2000).  In  addition,  many  researchers  familiar  with  computer-mediated 
communication  have  noted  what  Walther  (1994)  refers  to  as  the  "hyperpersonalness”  of  the  medium. 
Participants  in  online  discussion  seem  to  project  their  personalities  into  it,  creating  feelings  of  "immediacy" 
that  build  online  discourse  communities  (Gunawardena  & Zittle,  1997;  Poole,  2000;  Rourke,  Anderson, 
Garrison  & Archer,  2001). 

The  question  remains,  however,  as  to  whether  students  actually  learn  anything  in  online 
discussion.  Hewitt,  for  example,  provides  evidence  that  most  threaded  online  discussions  are 
overwhelmingly  divergent  and  argues  that  convergence  is  an  important  part  of  knowledge  building. 
Likewise,  McLoughlin  and  Luca  (1999)  distinguish  between  knowledge  construction  and  knowledge 
reproduction,  and  suggest  that  we  need  to  develop  frameworks  for  analyzing  the  negotiation  and  revision  of 
knowledge. 

This  panel  will  explore  issues  of  learning  through  online  discussion.  It  will  ask  the  questions  “Can 
students  learn  from  online  discussions,  and  if  so  under  what  conditions?”  and  “What  might  count  as 
evidence  of  learning  through  online  discussion?”  Specific  topics  to  be  covered  are: 

Perceived  Learning  and  Social  Presence  in  Asynchronous  Online  Courses,  Jennifer  Richardson, 
Purdue  University. 

Research  has  demonstrated  that  social  presence  affects  student  outcomes  as  well  as  student,  and 
possibly  instructor,  satisfaction  with  a course  (Moore,  Masterson,  Christophel,  and  Shea,  1996).  Teacher 
immediacy  behaviors  and  the  presence  of  others  are  especially  important  issues  for  those  involved  in 
delivering  online  education.  This  presentation  will  report  on  a study  which  used  survey  data  to  explore  the 


relationship  between  students*  perceptions  of  social  presence  and  their  perceived  learning  in  online  course 
environments.  Implications  of  the  study  effect  the  design  and  instruction  of  online  courses  in  terms  of 
students*  perceived  learning  and  satisfaction  with  the  instructor.  Instructors  need  to  be  aware  of  the  impact 
that  their  immediacy  behaviors  and  social  presence  or  lack  thereof  may  have  on  their  students*  satisfaction, 
motivation,  and  learning. 

Using  the  Discussion  Board  as  a Learning  Tool:  Facilitating  Multiple  Perspectives.  Linda  Polhemus, 
University  at  Albany 

Students  learn  from  online  discussions  that  promote  the  integration  of  multiple  perspectives. 
Online  learning  communities  are  created  when  discussion  threads  are  generated.  Discussion  threads  are 
generated  when  values  and  experiences  are  integrated  into  messages  that  promote  alternative  perspectives. 
Students  from  SUNY  Learning  Network  (SLN)  participated  in  interviews,  observations  and  responded  to  a 
questionnaire  related  to  knowledge  building  communities  in  online  learning.  Interviews  addressed  issues 
of  navigation,  expectations  and  beliefs  about  online  learning.  Selected  students  were  also  video  taped  when 
participating  in  the  online  discussion  using  a picture-in-picture  device  to  capture  student  reactions  when 
‘thinking  aloud.*  In  addition,  questionnaires  were  distributed  to  students  enrolled  in  graduate  level  courses 
and  discussion  threads  were  analyzed.  Results  revealed  that  when  students  read  and  respond  in  the  online 
discussion,  their  exposure  to  multiple  perspectives  enhanced  their  perception  of  knowledge  building. 
Evidence  of  learning  through  the  online  discussion  is  apparent  in  the  observations  where  students 
summarize  the  messages  of  others,  build  upon  their  ideas  and  make  connections  to  their  own  personal 
experiences.  The  ability  for  students  to  reflect  upon  the  messages  of  others,  navigate  the  discussion  board  *s 
interface  at  a high  level,  and  read  and  compose  messages  which  are  of  high  quality  result  in  learning 
engagement  at  the  critical  or  literate  level. 

Embedded  Formative  Evaluation  of  Student  Learning.  Michael  Danchak,  RPI 

Current  web  toolkits  such  as  WebCT  and  BlackBoard  provide  tools  for  quizzing  and  discussion. 
However,  in  order  to  use  these  tools  one  must  leave  the  content.  Quizzing  and  surveys  really  do  not  address 
recitation  well.  In  addition,  the  bulletin  board  does  not  “force**  a contribution  before  the  students  can  see 
all  the  postings.  I will  describe  a Java  applet  based  tool  we  built  that  allows  the  instructor  to  ask  either 
multiple  choice  or  discussion  questions.  Once  answered,  the  student  sees  the  instructor*s  answer  and  may 
see  other  students*  responses,  at  the  instructor*s  discretion.  The  applet  can  be  embedded  directly  in  any 
html  content  page  and  can  be  used  for  either  synchronous  or  asynchronous  feedback  to  the  student. 

Higher-Order  Learning  Through  Computer  Conferencing.  Liam  Rourke,  University  of  Alberta 

The  educational  claims  made  for  computer  mediated  discussion  are  largely  cognitive  in  nature — 
discussion  promotes  critical  thinking,  self- reflective  thought,  higher-order  learning — the  list  is  impressive. 
Yet,  researchers  in  this  domain  have  rarely  used  techniques  developed  specifically  to  investigate  cognitive 
processes.  Thus,  many  of  the  claims  remain  speculative,  anecdotal,  or  hyperbolic.  Protocol  analysis  is  one 


technique  that  has  proven  to  be  useful  for  revealing  the  mental  processes  of  subjects  engaged  in  a wide 
variety  of  tasks.  An  analysis  of  students’  concurrent  verbalizations  as  they  read  and  respond  to  messages  in 
a computer  conference  provides  direct  evidence  of  the  mental  processes  in  which  they  are  engaged.  This 
presentation  summarizes  the  provisional  results  of  a protocol  analysis  study  involving  upper-level, 
undergraduate  humanities  students  engaged  in  a computer  conference.  Eight  students  were  recorded 
individually  as  they  read  and  responded  to  messages  that  appeared  in  the  computer  conference  supplement 
to  their  face-to-face  course.  Two  distinct  sets  of  cognitive  processes  appeared  to  be  occurring  among  the 
participants — sociocognitive  conflict  (Doise  & Mugny,  1985)  and  narrative  cognition  (Bruner,  1990).  In 
the  former  case,  students  composed  coherent  and  rhetorical  interpretations  of  course  readings,  received 
critical  feedback  from  others,  and  re-evaluated  their  original  interpretations.  In  the  latter,  students 
embedded  their  interpretations  of  course  material  in  autobiographical  narratives,  and  immersed  themselves 
empathetically  in  the  narratives  of  others.  Within  these  two  sets  of  cognitive  processes,  there  was  evidence 
of  many  potentially  valuable  sub-processes  including  metacognition,  internalization  of  the  discourse 
process,  cognitive  flexibility,  and  naturalistic  generalization. 

Can  We  Assess  the  Quality  of  Learning  in  Online  Conferences?  Catherine  McLoughlin.  Australian 
Catholic  University 

While  computer-based  tools  to  support  asynchronous  dialogue  and  discussion  are  now  part  of  many  on-line 
units  of  study,  there  is  a lack  of  research  on  how  to  assess  the  quality  of  such  interactions,  and  in  particular, 
how  they  support  learning.  Mason  (1991:  161)  posed  the  question:  ‘Are  conference  interchanges  more  than 
merely  the  outpourings  of  lonely  or  loquacious  students?’  Since  then  several  theorists  have  provided 
interaction  analysis/content  analysis  techniques  to  examine  the  construction  of  knowledge  in  computer 
conferencing.  These  are  reviewed  in  the  light  of  constructivist  theories  of  knowledge  and  the  adoption  of  a 
socio-cultural  model  of  learning,  emphasizing  social  interaction  and  dialogue  as  central  to  learning.  If 
knowledge  construction  is  to  be  evaluated  rather  than  knowledge  reproduction,  we  need  to  assess  the 
processes  that  support  such  construction,  and  consider  negotiation  and  revision  of  knowledge.  For  tertiary 
students,  working  with  others  in  teams,  evaluating  others’  perspectives  and  the  ability  to  communicate 
abstract  ideas  are  recognized  and  valued  educational  outcomes.  A valid  framework  for  analysis  would 
therefore  require  a focus  on  process  variables,  such  as  how  learners  negotiate  and  interact,  and  product 
variables,  such  as  the  content  and  outcomes  of  learner  interactions.  Educators  need  practical  assessment 
tools  to  evaluate  on-line  discussion  and  show  the  potential  of  on-line  forums  to  support  collaborative 
learning  and  construction  of  knowledge. 


PARTICIPANTS 

Michael  Danchak  is  Professor  of  Computer  Science  at  Rensselaer  Polytechnic  Institute  in  Troy,  NY.  He 
has  been  teaching  at  a distance  for  more  than  10  years  and  is  currently  researching  tools  and  techniques  for 
making  online  courses  more  interactive. 
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Jennifer  Richardson  is  an  Assistant.  Professor  of  Educational  Technology  at  Purdue  University.  Jennifer 
has  extensive  experience  both  doing  research  in  the  schools  in  terms  of  technology  as  well  as  conducting 
evaluations  of  educational  technology  programs  and  products.  Her  interests  include  educational  technology 
professional  development,  social  aspects  of  web-based  learning,  and  integration  of  technology  into 
educational  environments. 

Linda  Polhemus  is  currently  a doctoral  student  enrolled  in  the  University  at  Albany’s  Educational  Theory 
and  Practice  program  and  director  of  Rensselaer  Polytechnic  Institute’s  Online  Learning  Forum.  The 
Online  Learning  Forum  is  a professional  development  program  addressing  the  needs  of  kl2  teachers. 
Linda’s  research  interests  focus  on  the  integration  of  instructional  technology  and  distance  education. 

Liam  Rourkeis  a Ph.D.  student  in  Department  of  Educational  Psychology  at  the  University  of  Alberta.  He 
has  published  several  papers  on  the  topic  of  computer  conferencing  in  higher  education 
(www.ualberta.ca/Hrourke/liam.htmn  and  is  currently  investigating  how  and  what  students  learn  through 
discussion. 

Catherine  McLoughlin  is  an  Associate  Professor  and  Head  of  the  School  of  Education  at  the  Australian 
Catholic  University  in  Canberra,  Australia.  She  is  National  President  of  the  Open  and  Distance  Learning 
Association  of  Australia.  Catherine  has  researched  and  published  in  the  area  of  online  teaching  and 
learning,  and  deigned  a range  of  learning  resources  for  distance  learners.  Her  main  research  interests  are 
development  of  innovative  assessment  practices  online,  and  the  evaluation  of  learning. 

Karen  Swan  is  an  Associate  Professor  of  Instructional  Technology  in  the  School  of  Education  at  the ' 
University  at  Albany.  She  has  been  teaching  online  courses  for  five  years  and  has  presented  and  published 
on  issues  of  interactivity  and  social  presence  in  online  courses. 
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Abstract 

This  presentation  will  describe  a qualitative  study  investigating  the  Capital  Area  Technology  and 
Inquiry  in  Education  (CATIE)  model  of  situated  professional  development  for  technology 
integration  in  schools  and  classrooms.  We  call  the  CATIE  model  **situated*'  because  it  places 
educational  technology  experts  in  schools  on  an  ongoing  basis  where  they  collaborate  directly 
with  teachers  to  develop  and  deliver  technology  enhanced  lessons.  Teacher  learning  about 
technology  integration  is  thus  situated  in  authentic  technology  integration  activities.  What  we 
think  is  unique  about  the  CATIE  model  is  that  mentors  work  with  schools  and  teachers  on  a long 
term,  daily  basis,  that  they  work  to  incorporate  technology  use  into  existing  curricula,  and  that 
they  work  in  real  classrooms  directly  with  teachers  and  students.  The  CA  TIE  program  thus  tends 
to  reach  most  of  the  teachers  and  administrators  in  a school,  and  we  can  see  the  results  in  terms 
of  student  learning,  technology  integration,  and  changes  in  school  cultures.  Findings  from  data 
collected  to  date,  including  monthly  mentor  reports,  interviews  with  mentors,  teachers,  and  school 
administrators,  classroom  observations,  and  student  artifacts,  will  be  used  to  support  discussion 
of  the  model. 

Background 

Recent  large  scale  studies  of  computer  usage  in  schools  (Panel  on  Educational  Technology,  1997; 
Educational  Testing  Service,  1998;  Becker,  Ravitz  & Wong,  1999)  have  precipitated  public  debate 
concerning  the  efficacy  of  using  computers  to  support  instruction,  and  have  highlighted  the  need  for 
professional  development  in  this  area.  While  emphasizing  the  need  for  professional  development  and 
pointing  to  the  relationship  between  it  and  more  sophisticated  uses  of  technology  in  schools,  these  and 
other  studies  suggest  that  our  understanding  of  what  sorts  of  professional  development  programs  impact 
technology  integration  at  school  and  classroom  levels  needs  to  be  improved.  What  we  do  know  is  that  the 
teacher  training,  “expert  model”  of  professional  development  (Sparks,  1994)  does  not  work,  especially 
when  it  comes  to  learning  about  educational  technologies  and  their  integration  across  the  curriculum. 
Indeed,  teacher  lore  suggests  that  traditional  inservice  teacher  education  has  little  impact  on  teaching 
practices  in  general.  Smylie  (1989),  for  example,  found  that  teachers  ranked  inservice  training  last  out  of 
fourteen  possible  opportunities  for  learning.  What  teachers  ranked  as  most  important  was  direct  classroom 
experience.  Even  exemplary  professional  development  programs  find  it  difficult  to  maintain  support  for 
teachers  (Carey  & Frechtling,  1997),  to  encourage  sustained  discourse  among  participating  teachers 
(Schlager  & Schank,  1997),  to  “scale  up”  (Corcoran,  1995)  through  the  inclusions  of  all  teachers,  and  to 
develop,  test,  and  disseminate  new  teaching  and  learning  ideas.  Researchers  agree  that  new  models  of 
professional  development  are  needed,  and  that  such  models  must  include  a focus  on  the  development  of 
local  cultures  of  interest  if  they  are  to  be  sustainable. 


Situated  Professional  Development 

Putnam  and  Borko  (2000)  relate  recent  trends  in  research  on  professional  development  to  new 
understandings  of  the  nature  of  learning  and  knowing  that  collectively  have  been  labeled  “situative” 
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(Greeno,  1997).  They  identify  three  conceptual  themes  central  to  situative  perspectives  - that  cognition  is 
situated  in  particular  physical  and  social  contexts,  that  it  is  social  in  nature,  and  that  knowing  is  distributed 
across  the  individual,  others,  and  tools  (p.  4)  - which  they  believe  have  important  implications  for 
professional  development.  Putnam  and  Borko  argue  that  how  teachers  learn  new  methods  of  teaching  is  no 
different  from  any  learning.  If  knowledge  and  learning  are  indeed  situative,  then  the  most  effective 
inservice  education  will  be  situated  in  authentic  classroom  practice. 

To  our  knowledge,  such  perspective  has  not  been  used  to  frame  either  the  development  of,  or  research  on, 
professional  development  programs  aimed  at  technology  integration,  although  a situative  perspective  is 
widespread  in  the  research  and  development  of  technology-based  educational  programs.  It  only  makes 
sense,  and  yet  most  professional  developrhent  programs  aimed  at  technology  integration  are  instructionist, 
application-driven  workshops  or  summer  “institutes”  well  removed  from  classroom  practice.  Some  have 
argued  that  while  not  optimal,  such  approaches  are  often  the  only  practical  solutions  to  meeting  large-scale 
professional  development  needs  with  limited  resources  (Wilson  & Berne,  1999).  What  little  research  there 
is  suggests  such  activities  have  little  impact  on  the  day-to-day  integration  of  computing  technologies  into 
classroom  teaching  and  learning  (Panel  on  Educational  Technology,  1997;  Educational  Testing  Service, 
1998). 

The  CATIE  Program 

It  is  hard  to  see  how  approaches  that  consistently  have  had  little  or  no  effect  on  classroom-based 
technology  integration  can  be  deemed  “practical,”  thus  we  decided  to  try  an  “impractical”  approach.  The 
Capital  Area  Technology  and  Inquiry  in  Education  (CATIE)  program  was  established  through  the  Center 
for  Initiatives  in  Pre -College  Education  (CIPCE)  at  Rennselaer  Polytechnic  Institute  as  an  innovative 
means  for  addressing  technology-based,  constructivist-oriented  staff  development  in  elementary  schools  in 
the  greater  Troy  (NY)  region.  This  unique  program  places  technology  experts  in  elementary  school 
buildings  where  they  serve  as  mentors  to  teachers  interested  in  integrating  the  use  of  technology  into  their 
day-to-day  classroom  activities.  The  school-based  mentors  provide  training  to  teachers  on  technology 
utilization,  but,  more  importantly,  the  mentors  work  with  teachers  to  jointly  design  computer-supported 
lessons  that  incorporate  technology  into  existing  classroom  curricula.  Teacher  learning  about  technology 
integration  is  thus  situated  in  authentic  technology  integration  activities. 

Typically,  the  mentors  first  meet  with  teachers,  both  individually  and  in  groups,  to  discuss  how  technology 
might  be  used  to  enhance  learning  in  planned  units  on  particular  topics.  Mentors  try  to  avoid  planning  that 
is  either  artificial  or  focused  on  q)ecific  software  applications.  They  then  work  with  teachers  to  design 
computer-supported  lessons  that  are  integral  parts  of  larger,  classroom-based  learning  units.  They 
encourage  inquiry -based,  student-centered,  constructivist  uses  of  computing  technologies,  but  they  do  not 
insist  on  them.  Often,  mentors  model  best  practices  in  computer-based  teaching  and  learning  by  taking  the 


lead  in  implementing  jointly  created  lessons.  They  then  guide  teachers  in  designing  and  implementing  their 
own  computer-based  lessons,  gradually  fading  their  support  as  teachers  become  more  confident  in  the  use 
of  electronic  technologies. 

Mentor  support,  however,  does  not  just  disappear.  Each  mentor  structures  his  or  her  schedule  according  to 
their  school  and  participating  teachers’  individual  needs.  Generally,  the  mentor  is  available  two  or  three 
days  each  week  for  a period  of  two  years  or  more  to  work  with  teachers  and  students  on  a continuing,  as 
needed  basis.  Many  teachers,  having  mastered  a particular  technology  tool,  return  to  their  mentor  for  help 
in  utilizing  other  applications  in  their  teaching.  Some  teachers  just  come  to  share  with  their  mentors  the 
ways  in  which  they  are  using  technology  on  their  own,  and  some  mentors  meet  regularly  with  groups  of 
teachers  to  discuss  technology  integration.  As  mentors  become  a part  of  the  culture  of  the  school,  formal 
and  informal  conversations  of  this  sort  become  more  common  and  ongoing,  and  a discourse  community 
grows  up  around  technology  integration. 

Methodology 

The  research  design  was  qualitative.  Questions  addressed  included  the  impact  of  the  CATIE  program  on 
teachers’  integration  of  technology  into  regular  classroom  teaching  and  learning,  CATIE’s  impact  on  the 
culture  of  technology  integration  in  individual  schools,  and  the  affordances  and  constraints  on  that  process 
across  schools.  We  were  also  interested  in  how  CATIE  mentors  viewed  their  work.  Because  the  nature  of 
technology  integration  clearly  seems  to  be  situated  in  individual  schools  (Swan,  Jennings  & Meier,  2001), 
we  looked  at  each  of  the  schools  in  which  CATIE  mentors  worked  as  a single  case  within  the  larger 
investigation  of  the  CATIE  program. 

In  the  2000/2001  school  year,  there  were  ten  CATIE  mentors  working  in  14  elementary  schools  in  four 
school  districts  in  the  greater  Troy,  NY  area.  The  schools  in  which  mentors  were  placed  were  all  urban,  but 
ranged  in  size  from  quite  small  (12  teachers)  to  quite  large  (101  teachers)  and  varied  in  ethnic  and  SES 
characteristics  by  neighborhoods.  In  2000/2001,  CATIE  mentors  worked  with  188  teachers  and  over  4,000 
students.  They  turned  in  monthly  reports  detailing  the  teachers  and  students  they  worked  with,  lessons 
jointly  developed,  and  any  outstanding  successes  or  problems  they  encountered.  Mentors  also  met  biweekly 
with  the  researchers  to  discuss  the  same.  The  researchers  visited  all  schools  at  least  twice,  where  they 
observed  mentors  work  with  teachers  and  students  and  took  field  notes.  They  also  conducted  onsite 
interviews  with  mentors,  school  administrators,  and  teachers  with  whom  the  mentors  were  working,  and 
collected  samples  of  student  work. 

Data  sources  thus  included  the  monthly  mentor  reports,  notes  from  mentor  meetings,  transcripts  of 
interviews  with  mentors,  teachers,  and  school  administrators,  field  notes,  and  student  artifacts.  Data 
analyses  consisted  of  the  constant  comparison  method  to  look  for  emergent  themes. 


Preliminary  Findings 

Clearly,  the  CATIE  program  exhibits  the  features  Putnam  and  Borko  (1997)  identify  as  essential  to 
effective  teacher  education: 


• Teachers  involved  in  the  CATIE  program  are  treated  as  active  learners  who  construct 
their  own  learning  to  meet  their  own  specific  professional  needs.  CATIE  mentors 
serve  as  facilitators  for  that  learning  but  follow  and  respect  the  directions  it  takes. 

• Thus,  teachers  in  the  C\TIE  program  are  empowered  to  use  technology  in  their 
teaching  (through  the  on-site  support  of  the  mentors)  and  treated  as  the  professionals 
they  are. 

• In  the  CATIE  program,  teacher  learning  about  technology  integration  is  situated  in 
classroom  practice. 

• CATIE  mentors  model  constructivist,  student-centered  teaching  with  technology  in 
their  work  with  both  students  and  teachers.  Mentors  thus  treat  teachers  the  way  they 
would  have  them  treat  students. 

Mentors  spend  the  majority  of  their  time  working  with  teachers  and  students  in  the  computer  labs  of  their 
participating  schools,  although  they  also  assisted  some  classes  within  their  own  classrooms.  The  latter  is  the 
goal  of  the  program  and  is  increasing  but  we  are  still  not  satisfied  with  the  lack  of  classroom  usage.  This 
seems  to  have  to  do  with  the  availability  and  quality  of  equipment  in  schools  and  classrooms,  but  the 
relationship  is  complex  and  interesting.  In  some  cases,  for  example,  classroom  usage  is  hampered  by  the 
lack  of  computers  with  Internet  access  in  classrooms.  However,  in  several  schools  CATIE  is  involved  with, 
there  are  multiple  computers  with  Internet  connections  in  classrooms  that  are  not  well  utilized.  In  these 
schools,  there  is  also  a high  quality  computer  room.  In  these  schools,  teachers  seem  more  comfortable  with 
the  computer  room  because  it  affords  each  student  a computer  and  so  allows  teachers  to  work  in  more 
traditional  modes.  Indeed,  the  one  school  in  which  more  use  is  made  of  classroom  computers  the  computer 
room  computers  are  older  and  incapable  of  running  more  complex  programs  such  as  HyperStudio. 


A related  goal  in  CATIE  is  to  encourage  teachers  to  look  beyond  their  current  uses  of  technology.  This 
seems  to  be  happening  with  the  more  technology-experienced  participating  teachers,  and  with  teachers 
involved  in  their  second  or  third  years  with  the  program.  The  CATIE  program  has  also  made  significant 
gains  in  helping  teachers  less  familiar  with  educational  technology  become  comfortable  in  its  use.  An 
interesting  finding  involves  what  we  are  calling  the  “learning  by  lurking”  phenomenon  - teachers  who  have 
been  only  peripherally  involved  in  CATIE  become  involved  after  the  program  has  been  in  their  school  for 
several  years.  To  date,  CATIE  mentors  have  assisted  approximately  350  teachers  in  fourteen  schools  in 
four  school  districts  to  create  and  implement  lessons  that  integrate  the  use  of  technology  into  regular 
classroom  activities. 


Flexibility  and  adaptability  were  found  to  be  central  to  best  practices  in  mentoring.  Mentors’  ability  to  work 
with  variations  in  teacher  learning  styles,  pedagogical  approaches,  and  prior  experiences,  as  well  as  with 


existing  school  technology  resources  were  found  to  significantly  influence  technology  integration  in 
schools  and  classrooms.  An  unexpected  finding  Egarding  mentors  was  their  programmatic  focus.  The 
CATIE  program’s  mission  is  focused  on  teachers.  In  interviewing  mentors,  however,  we  found  that  many 
mentors  were  at  least  as  concerned  with  student  learning  as  with  teacher  learning.  The  effect  of  such  focus 
is  being  investigated. 


Teacher  perceptions  of  the  CATIE  program  were  overwhelmingly  positive.  Teachers  uniformly  reported 
increased  knowledge  of  computing  technologies,  greater  confidence  in  using  them,  and  more  creative 
teaching  with  computers.  Positive  outcomes  for  students,  including  greater  independence,  heightened  self- 
efficacy,  and  increased  motivation,  were  also  noted  by  participating  teachers.  In  most  of  the  schools  we 
visited,  the  CATIE  program  was  seen  as  a valued  part  of  the  school  culture,  and  many  teachers  noted 
changes  in  technology  integration  school-wide  as  a result. 


/ 

Educational  Significance 

The  CATIE  program  seems  to  be  positively  affecting  technology  integration  in  the  schools  in  which  it  is 
operating.  In  this  its  fourth  year  of  operation,  its  effects  are  becoming  clearer.  Most  important  of  these,  is 
the  saturation  of  technology  integration  into  the  schools  in  which  CATIE  mentors  are  working.  After 
several  years  of  being  present  in  schools,  the  CATIE  program  is  reaching  most  of  the  teachers  in  those 
schools,  which  in  turn  seems  to  be  changing  their  culture  to  include  technology  integration.  Perhaps  the 
best  measure  of  its  success  is  that  one  of  the  CATIE  districts  is  adopting  mentoring  as  the  preferred  means 
of  technology  training.  Common  sense  indicates  that  the  CATIE  program  is  successful  because  it  is 
reaching  out  to  teachers  in  the  physical  and  social  context  of  their  practice,  because  it  provides  ongoing, 
long-term  support  for  technology  integration,  and  because  of  the  personal  relationships  mentors  are  forging 
with  participating  teachers  within  the  culture  of  the  schools  in  which  they  work.  Its  success  thus  supports  a 
situative  perspective  on  teacher  learning,  especially  teacher  learning  about  technology  and  technology 
integration. 
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objectives/purposes 

This  presentation  will  report  on  the  Technology  Integration  Project’s  work  to  date  documenting  the  use  of 
technology  in  middle  school  English  language  arts  classrooms.  The  purpose  of  this  ongoing  national  study 
is  to  explore  the  ways  in  which  middle  school  English  language  arts  teachers  are  using  electronic 
technologies  to  enhance  literacy  learning  in  their  classrooms.  In  particular,  we  are  interested  in  teachers’ 
conceptions  of  "technology  integration,"  how  they  frame  their  uses  of  technology  within  a larger 
curriculum,  and  how  technologies  can  support  teaching  strategies  that  have  been  shown  to  enhance  learning 
in  the  English  language  arts. 

theoretical  frameworks/ perspectives 

This  study  builds  on  the  work  of  Henry  J.  Becker  and  others  at  the  Center  for  Research  on  Information 
Technology  and  Organizations  (CRITO)  that  documents  teachers'  uses  of  computers  (Becker,  Ravitz  & 
Wong,  1999),  and  on  the  work  of  researchers  at  the  National  Research  Center  on  English  Learning  and 
Achievement  (CELA)  that  identifies  effective  strategies  in  English  language  arts  teaching  and  learning 
(Center  on  English  Learning  and  Achievement,  2000). 

Becker  and  the  CRITO  researchers  have  found  that,  although  it  is  still  true  that  a majority  of  teachers  use  ' 
computers  very  little  and  that  the  majority  of  intensive  experiences  students  have  with  computers  are 
outside  the  academic  disciplines,  a growing  number  of  subject  matter  teachers  do  have  their  students  use 
computers  frequently  and  do  so  with  a new  focus  on  accessing  information  and  improving  their  writing  and 
general  communication  skills.  First,  among  these  are  English  language  arts  teachers.  Indeed,  at  the  middle 
school  level,  30%  of  students'  experience  using  computers  takes  place  in  English  classes.  75%  of  the 
secondary  English  language  arts  teachers  surveyed  used  computers  at  least  occasionally  in  their  classes,  and 
24%  used  them  twenty  or  more  times  a year.  (Becker,  Ravitz  & Wong,  1999)  The  CRITO  report  gives 
some  insight  into  what  kinds  of  computer  applications  English  teachers  are  using  and  what  learning 
objectives  they  are  focusing  their  technology  use  on.  It  was  not  intended  to  tell  us,  however,  how 
technologies  are  being  used  to  support  literacy  learning. 

CELA  researchers  have  identified  six  approaches  to  English  language  arts  teaching  that  have  been  shown  to 
enhance  literacy  learning  across  many  CELA  research  investigations.  These  are: 
o using  diversity  to  enhance  learning 
o increasing  the  cohesiveness  of  curriculum  and  instruction 
o raising  the  level  of  student  engagement  in  higher  order  talk  and  writing 
o aligning  curriculum  with  assessment 
o scaffolding  student  performance  of  new  and  difficult  tasks 
o helping  struggling  readers  (Center  on  English  Learning  and  Achievement,  2000). 

What  CELA  reports  to  date  cannot  tell  us  is  how  electronic  technologies  can  and  are  being  used  to  support 
such  approaches. 

methodology 

To  investigate  the  ways  in  which  English  language  arts  teachers  are  using  technology  to  support  literacy 
learning,  the  authors  of  this  paper  instituted  a national  nomination  process.  English  language  arts,  library, 
and/or  technology  coordinators  from  all  states  and  regional  NCTE  (National  Council  of  Teachers  of 
English)  and  RTEC  (Regional  Technology  in  Education  Consortia)  directors  were  contacted  and  asked  to 
nominate  middle  school  English  language  arts  teachers  who  were  making  extensive  and/or  interesting  use 
of  computers  in  their  classroom  teaching.  Technology  using  teachers  were  also  recruited  through  the 


CELA  website,  school  district  home  pages,  recommendations  from  software  publishers  and  hardware 
vendors,  other  researchers,  and  through  formal  and  informal  professional  networks. 

Nominated  teachers  were  asked  to  respond  to  an  online  survey  which  requests  that  they  describe  two 
lessons  they  use  that  incorporate  technology,  as  well  as  asking  for  certain  demographic  and  technology 
usage  information.  The  survey  thus  focuses  on  specific  classroom  uses  of  electronic  technologies. 

Survey  results  are  being  analyzed  not  only  for  technologies  and  applications  used,  but  for  how  teachers 
frame  technology-supported  lessons  and  how  they  support  literacy  learning.  Selected  teachers  will  also 
be  interviewed  by  phone  to  further  explore  these  issues. 

data  sources 

The  data  sources  for  the  study  are  technology  using  middle  school  English  language  arts  teachers. 

Teachers  participating  in  the  study,  thus,  do  not  represent  average  English  language  arts  teachers.  Rather 
they  were  chosen  to  provide  a sample  of  what  is  being  done  by  exceptional  technology  using  teachers  in  the 
middle  school  English  language  arts  areas.  The  study  is  national  in  scope  for  breadth  and  to  take  into 
account  possible  regional  variations  in  usage.  The  middle  school  domain  was  chosen  not  only  because  it 
matches  the  focus  of  larger  CELA  projects  (Center  on  English  Learning  and  Achievement,  2000),  but 
because  it  has  been  shown  to  be  where  technology  is  most  used  in  English  language  arts  teaching  and 
learning  (Becker,  Ravitz  & Wong,  1999). 

results/conclusions 

To  date,  nominations  have  been  solicited,  nominated  teachers  contacted,  and  72  online  surveys  from 
teachers  in  30  states  collected.  This  data  is  being  analyzed  in  terms  of  teachers’  conceptions  of  technology 
use,  in  terms  of  how  they  integrate  technology  into  existing  curricula,  and  in  terms  of  how  technology  is 
being  used  to  support  the  six  approaches  to  English  language  arts  teaching  that  have  been  shown  to  enhance 
literacy  learning  (Center  on  English  Learning  and  Achievement,  2000).  We  are  currently  conducting  phone 
interviews  with  these  teachers  that  additionally  probe  the  above  issues  as  well  as  issues  of  assessment  and 
English  language  arts  standards.  By  the  time  of  the  annual  meeting,  we  anticipate  having  collected  and 
analyzed  all  the  data,  including  teacher  interviews,  on  which  we  will  report. 

Preliminary  findings  suggest  that  electronic  technologies  are  indeed  being  used  to  support  the  six 
approaches  identified  by  CELA  as  enhancing  literacy  learning,  most  particularly  using  diversity  to  enhance 
learning,  raising  the  level  of  student  engagement  in  higher  order  talk  and  writing,  and  scaffolding  student 
performance  of  new  and  difficult  tasks.  In  addition,  participating  teachers  frequently  mentioned  computer 
support  for  what  might  be  termed  the  development  of  learning  communities.  They  frequently  spoke  of 
more  active  and  authentic  learning,  motivation,  and  the  use  of  extended  and  collaborative  projects.  Almost 
all  of  them  stated  that  students  worked  better  together  on  computers,  and  that  all  students  were  included  in 
the  class  work.  Teachers  also  spoke  of  their  role  as  changing  from  lecturer  to  facilitator. 

Participating  teachers  generally  used  computers  at  least  30%  of  the  time  in  their  classes,  although  some 
used  them  a good  deal  more.  As  the  CRITO  researchers  found,  teachers  with  greater  access  to  computers 
used  them  more  often.  For  example,  one  teacher  who  had  laptops  for  every  student  used  them  70%  of  the 
time.  Our  survey  also  confirms  the  baseline  of  use  that  CRITO  reports  --  word  processing  and  information 
access  using  CD  ROMs  and  the  Internet.  Teachers  in  this  survey,  however,  went  way  beyond  these  uses. 
They  used  concept  mapping  software  such  as  Inspiration  to  explore  their  own  and  others  ideas.  They  used 
email  to  communicate  with  peers  and  experts  around  the  world.  They  used  multimedia  and  presentation 
software,  such  as  PowerPoint  and  HyperStudio  to  publish  and  present  their  work,  and  digital  cameras, 
scanners,  and  graphics  packages  to  illustrate  it.  Many  made  and  edited  digital  movies.  Some  used 
simulations  available  on  the  World  Wide  Web,  such  as  the  virtual  tour  of  Ann  Frank’s  house;  others  made 
use  of  online  drill  and  practice  exercises  in  grammar  and  comprehension.  Interestingly,  many  used 
spreadsheets  and  databases  to  organize  information  and  create  knowledge. 


educational  significance 


Across  the  nation,  schools  and  school  districts  are  investing  heavily  in  computer  hardware  and  software,  ^ 
often  using  state  and  federal  grant  monies  as  well  as  local  tax  and  bond  revenues.  Yet  little  or  no 
information  is  available  about  the  most  effective  ways  to  leverage  that  investment  into  higher  literacy 
achievement  for  students.  This  study  is  designed  to  identify  the  variety  of  ways  middle  school  English 
language  arts  teachers  across  the  country  are  using  electronic  technologies  to  support  literacy  learning. 
What  we  are  finding  is  that  computer  using  English  language  arts  teachers  are  using  a wide  variety  of 
software  a good  deal  of  the  time  to  support  approaches  known  to  enhance  literacy  learning  as  they  develop 
learning  communities  in  their  classrooms.  Their  pioneering  work  can  serve  as  models  for  school  and 
teachers  who  would  like  to  integrate  technology  into  English  language  arts  teaching  and  learning. 
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Abstract:  Adapting  narrative  principles  from  traditional  media  lets  course  developers 
and  writers  draw  from  a rich  aural  and  visual  tradition  to  craft  a new  language  for  on-line 
learning.  This  paper  highlights  techniques  that  may  be  adapted  from  traditional  media  to 
inform  the  creation  of  content,  task  and  navigation  for  interactive  educational  media. 


In  a world  of  increasing  speed,  complexity  and  competition,  we  must  engage  learners  through 
meaning  that  resonates  with  their  sensibilities  and  experience.  The  rich,  interactive  learning  environment 
enabled  by  computer  technology  challenges  us  to  engage  people  in  creating  their  own  meaning  as  a 
foundation  for  further  learning.  Taking  up  the  suggestion  by  Plowman  (1994)  to  borrow  well-known 
narrative  conventions  from  film  to  design  both  structure  and  interface,  this  paper  highlights  narrative 
strategies  fromtraditional  media  that  are  adaptable  to  developing  content  for  interactive  educational  media. 


Narrative  strategies  from  traditional  media 

That  the  structure  of  all  narratives,  at  least  in  >Vestem  society,  is  the  same  has  created  a very  stable 
framework  for  audiences  (Lacey,  2000).  The  three-act  structure,  well-defined  as  early  as  Aristotelian  times 
and  formularized  in  the  Hollywood  film,  is  aptly  explained  by  Hunter  (1994)  as  an  introduction  (set-up),  in 
which  we  place  our  heroes  up  a tree;  a complicatbn  (conflict),  in  which  we  throw  rocks  at  them;  and  a 
resolution  (climax)  in  which  we  get  them  back  down.  Within  this  framework,  many  techniques  have 
emerged  to  engage  audiences  through  traditional  media  such  as  theatre,  the  novel,  film,  radio  and  televisbn 
that  are  adaptable  to  educational  interactive  media. 

In  considering  narrative  techniques  developed  in  traditional  media,  we  should  consider  Lacey’s 
advice  that  “Narrative  does  not  deal  with  reality  at  all[,]  but  strives  to  create  an  illusion  of  reality  by 
referring  to  other  sign  systems”  (p.  77).  Devices  like  book  covers,  film  posters,  radio  theme  music,  title 
sequences  in  film,  and,  for  our  purposes,  CD-ROM  jewel  cases  and  on-line  course  home  pages,  give 
learners  cues  to  the  genre,  metaphor  or  interface  of  the  lesson  as  well  as  its  content. 

From  dramatic  writing  comes  the  need  for  a premise  (Egri,  1960)  progressing  to  a climax,  “the 
pinnacle  of  concentration  of  all  meaning  and  emotion...  the  decisive  centre  of  audience  satisfaction” 
(McKee,  1997,  p.  108).  Novels  immerse  us  inside  characters  in  a different  environment  to  let  us  better 
understand  ourselves  and  our  purpose  (Wilson,  1990),  and  furnish  narrative  genres  such  as  the  picaresque, 
the  epistolary  and  the  serial  (Hawthorn,  1997),  which,  like  the  cues  noted  above,  create  shortcuts  by 
drawing  on  knowledge  external  to  the  actual  content.  Film  gives  us  time -manipulation  devices  like  cutting 
between  scenes  and  dissolves  to  flashbacks  (Mast,  1976) — although  these  were  criticized  by  early  purists  as 
drawing  undue  attention  to  the  medium  (Salt,  1983) — and  the  distinction  between  what  we  perceive 
through  the  characters  (diegetic)  and  beyond  their  perception  (non-diegetic)  by  way  of  superimposed 
music,  images  or  voices  (Lacey).  Radio  reminds  us  of  the  power  of  oral  narratives  to  immerse  listeners 
(Tolan,  2001),  and,  with  television,  echoes  serial  literature  in  its  episodic  approach  and  in  offering  narrative 
“arcs”  spread  over  a season  (and  analogously,  over  a course).  The  need  to  engage  audiences  through  an 
opening  “teaser”  and  to  build  to  a high  point  just  before  cutting  away,  although  motivated  by  commercial 
breaks  (Miller,  1980),  may  be  applied  to  modularized  interactive  media  lessons.  In  fact,  writers  can  draw 
on  tactics  from  new  media  itself  to  hold  learners’  attention  by,  for  example,  leading  them  through  a 
suspenseful  (but  lesson -laden)  labyrinth  known  to  harbour  hidden  obstacles  or  lurking  adversaries.  This  is 
what  Murray  would  call  “achiev[ing]  coherent  dramatic  form  by  shaping  our  terror  into  a pattern  of 
exploration  and  discovery”  (p.  137). 


Guidelines 


Experiences  with  narrative  in  traditional  media  provide  a wealth  of  lessons  for  writers  seeking  to 
engage  learners  in  constructing  their  own  meaning  through  interactive  educational  media.  A review  of  the 
literature  on  traditional  narrative  tools  suggests  the  following  guidelines  for  developing  content  for 
interactive  educational  media: 

1.  A premise  or  theme  of  a lesson,  expressed  in  terms  of  human  behaviour  and  carried  through  to 
a logical  conclusion,  can  create  deeper  meaning  for  learners. 

2.  The  three-act  structure  of  introduction,  complication  and  resolution,  so  well-engrained  in 
Western  thought,  provides  a solid  frame  for  building  a lesson. 

3.  We  can  engage  on-line  learners  more  deeply  by  casting  them  in  “immersive”  roles  and 
allowing  them  to  chart  their  direction  through  the  narrative. 

4.  Immersing  learners  inside  other  characters,  in  a different  environment,  can  help  them  to  better 
understand  themselves  and  their  purpose. 

5.  Using  the  first  person  can  provide  access  to  a character’s  innermost  thoughts,  and  multiple 
points  of  view.  Unreliable  narrators  can  add  to  the  critical  skills  demanded  of  the  learner. 

6.  Lessons  can  be  structured  along  narrative  forms  well-established  by  novels  and  filmic  genres, 
which  enrich  the  lesson  and  save  time  by  incorporating  learners’  prior,  external  knowledge. 

7.  Further  layers  of  meaning  are  available  through  non-diegetic  devices  such  as  background 
music,  superimposed  images  or  voice-overs. 

8.  Time,  space  and  perspective  can  be  illustrated  by  employing  classical  filmic  techniques  such 
as  double  exposure,  different  types  of  shots  and  dissolves. 

9.  Different  stories  can  be  linked  through  a common  motif;  differences  can  be  lessons,  too. 

10.  Sound,  used  alone,  offers  great  power  to  co-opt  the  audience’s  imagination.  Key  points  may 
be  delivered,  along  with  other  means,  by  voice-over  while  freezing  or  blackening  the  screen. 

11.  An  episodic  approach,  with  a fixed  cast  of  characters  can  provide  continuity  to  a series  of 
teaching  modules  by  building  on  preceding  lessons,  and  referencing  prior  foundations  as 
refreshers  or  as  needed  to  understand  ensuing  points. 

12.  We  can  capture  learners’  interest  with  a tantalizing  opening  sequence  and  hold  it  through  tests 
or  other  breaks  by  building  the  lesson  to  a narrative  high  point  just  before  they  occur. 

The  familiarity  and  interactivity  of  narrative  structures  make  them  ideally  suited  to  on-line 
learning.  The  nascence  of  interactive  media  and  its  educational  applications  present  a tremendous  creative 
challenge  for  course  developers  and  writers.  Adapting,  and  perhaps  even  improving  on,  narrative  principles 
from  traditional  media  allows  us  to  draw  from  a rich  aural  and  visual  tradition,  and  to  help  craft  a new 
language  for  on-line  learners  who  seek  to  rise  to  Murray’s  call  to  turn  their  terror  into  discovery. 
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Abstract;  This  paper  discusses  the  formative  stages  of  an  ongoing  study  to  develop  a course  in 
oral  communication  that  can  be  taught  entirely  online  using  chat  rooms,  bulletin  boards  and 
synchronous  video-conferencing.  It  addresses  the  reasons  for  the  development  of  the  on- campus 
oral  communication  course,  its  structure,  pedagogical  foundations  and  its  evolution  from  the 
face-to-face  and  distance  print-based  delivery  modes  to  be  taught  entirely  online.  Due  to  current 
technical  limitations  a simulated  online  course  will  be  developed  and  conducted  under 
experimental  conditions  ising  a Local  Area  Network.  This  study  will  not  only  assist  oral 
communication  instructors,  but  also  instructors  developing  a wide  range  of  courses  for  online 
delivery. 


Background  to  the  Study 

The  need  for  improved  oral  communication  skills  has  been  expiessed  by  employers,  academics  and  students 
(Merrier  & Dirks,  1997).  Oral  communication  is  the  top  skill  sought  by  employers,  however,  this  skill  is  lacking 
in  most  business  graduates.  A compulsory  15-hour  oral  communications  course  to  improve  students'  oral  skills 
was  introduced  in  1999  for  all  business  students  studying  on-campus  at  a Western  Australian  university.  The 
course  was  initially  designed  for  the  internal  students.  It  was  later  developed  to  include  distance  students  using 
print -based  materials.  The  challenge  now  is  to  develop  it  for  online  delivery. 

The  pedagogical  basis  for  the  original  course  was  founded  on  constructivist  principles,  collaborative  online 
learning,  adult  learning  and  the  situated  learning  model.  Particular  attention  was  given  to  the  elements  of 
authentic  learning  model  outlined  by  Herrington  (1997)  that  is,  authentic  content  and  activities,  modelling  of 
process  by  experts,  multiple  roles  and  perspectives,  collaborative  construction  of  knowledge,  reflection  and 
articulation,  coaching  and  scaffolding,  and  authentic  assessment.  The  course  provided  opportunities  for  students 
to  learn  and  practise  communication  skills  applicable  to  the  business  world. 

The  face-to-face  course  was  structured  as  a two -hour  seminar  and  a one-hour  tutorial  repeated  five  times.  A 
typical  seminar,  with  approximately  50  students,  provided  the  background  information  for  the  module  through 
student  discussions,  activities,  question  and  answer  sessions  and  demonstrations  of  presentation  exercises.  Each 
tutorial,  with  groups  of  10-12  students,  was  further  subdivided  into  teams  of  3 or  4.  This  structure  provided 
support,  a nurturing  environment,  and  time  for  students  to  practise  and  develop  the  required  communication 
skills.  The  aim  is  to  replicate  as  many  of  theseessential  ingredients  as  practicable  via  online  delivery. 

Feedback  about  the  face-to-face  course  was  positive  and  class  members  noted  their  peers’  development.  Students 
appeared  more  confident,  readily  participated  in  class  discussions,  attended  regularly,  gave  positive,  objective 
and  encouraging  feedback,  and  provided  professional  presentations.  They  considered  the  skills  transferable  to 
other  units  and  general  life.  The  test  will  be  whether  these  positive  evaluations  persist  when  the  face-to-face 
element  of  the  course  is  removed. 

There  has  been  a push  from  students,  governments,  tertiary  institutions  and  the  workplace  for  the  development  of 
flexible  delivery  modes.  Online  learning  has  quickly  become  a pervasive  study  option,  offering  “anytime- 
anyplace,  one-to-one  or  one-to-many  communication  venues.  Online  learning  is  a particularly  attractive  option 
for  meeting  the  diverse  needs  of  adult  learners  (Vician  & Brown,  2001).  Much  of  the  research  into  online 
teaching  and  learning,  however,  has  concentrated  on  the  use  of  interactive  multimedia  by  an  individual,  and 
communications  with  other  students  and  the  lecturer  via  chat  rooms  or  bulletin  boards.  Less  research  has  been 
published  on  synchronous  videoconferencing  as  a means  of  communicating  with  other  students  and  lecturers 
through  computer  cameras.  Mason  (1991)  suggested  this  method  offered  4 advantages:  It  is  more  motivating  for 
learners  by  focusing  group  energy,  it  helps  develop  a sense  of  social  presence  and  group  cohesion,  provides 


quick  feedback  that  supports  decision  making,  and  provides  structure  and  discipline  to  encourage  students  to 
keep  up-to-date.  A recent  US  study  (Clark  & Jones,  2001)  used  the  Internet  to  teach  oral  communication, 
however,  students  were  required  to  attend  class  for  the  practical  component;  their  actual  presentations.  While  this 
method  may  be  suited  to  those  students  who  prefer  a different  delivery  mode,  it  is  not  suitable  for  students 
(particularly  those  living  in  remote  locations)  who  want  to  complete  the  entire  course  off-campus. 


Online  Oral  Communication  Experimental  Study 

A combination  of  delivery  systems  will  be  used  in  this  experimental  online  study  not  only  to  provide  a better 
learning  environment  (Berge,  2000),  but  also  to  replicate  the  successful  face-to-face  course.  Sudents  will 
progress  through  each  module  by  accessing  information  either  online  using  the  software  package  “Blackboard” 
or  via  a CD-Rom.  The  CD-Rom  will  contain  video  clips  of  examples,  demonstrated  techniques,  past  student 
presentations  and  key  hints  from  the  lecturer.  Students  will  complete  “class”  activities  either  individually  or  with 
other  students  via  chat  rooms  and  bulletin  boards  to  enhance  their  learning. 

The  main  oral  presentation  exercise  for  each  module  will  be  completed  via  synchronous  videoconferencing.  This 
will  allow  for  peer  coaching  immediately  following  presentations,  a component  of  the  internal  course  considered 
essential.  Until  more  advanced  technology  becomes  available  there  could  be  problems  with  voice 
synchronisation,  time  delays  and  computer  breakdowns,  so  the  student  presentation  experience  will  be  simulated 
using  Intranet.  Five  linked  computers,  with  cameras  attached  will  be  set  up  in  separate  rooms.  For  each  simulated 
session  the  first  student  will  give  their  individual  presentat  ion  via  the  camera,  the  second  student  in  the  team  will 
facilitate  the  self-  and  peer-ovaluation,  and  the  third  student  will  evaluate.  Student  roles  will  be  rotated  until  all 
students  present,  provide  feedback  and  facilitate.  For  the  purpose  of  this  study  a video  camera  in  the  room  will 
record  student  actions,  visible  signs  of  apprehension,  and  their  handling  of  the  program. 

Before  being  given  access  to  the  course  itself,  students  will  receive  individual  personal  training  in  using 
“Blackboard”  and  be  asked  to  complete  a pre-test  questionnaire  online.  This  questionnaire  will  measure  levels  of 
communication  apprehension  (CA),  willingness  to  communicate  (WTC),  shyness  and  self-perceived 
communication  competence  (SSPC).  After  the  course  students  will  do  a post-test  to  determine  any  changes. 
Student  satisfaction  will  be  measured  by  questionnaires,  focus  groups  and  interviews. 


Summary 

This  paper  has  outlined  the  developmental  stage  of  a work  in  progress  aimed  to  deliver  an  oral  communication 
course  entirely  online.  It  has  highlighted  the  critical  success  factors  of  the  existing  face-to-face  course  that  would 
have  to  be  replicated  to  ensure  positive  learning  outcomes  in  the  proposed  online  course.  Although  the  new 
course  will  be  trialed  in  a simulated  experimental  setting,  it  should  provide  a good  indication  of  the  potential  for 
success  for  such  a course  when  the  technology  becomes  sophisticated  enough  for  complete  online  delivery.  The 
insights  gained  from  this  experiment  will  be  of  use  to  educators  involved  in  development  of  other  online  courses, 
especially  those  where  synchronous  feedback  would  be  an  asset. 
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Abstract:  All  multimedia  project  managers  strive  to  have  projects  delivered  on  time,  on  budget  and 
to  the  required  quality.  In  the  quest  to  achieve  this  success,  much  research  has  been  performed  and 
literature  written  about  project  management  techniques  to  enhance  in  this  process.  Increasingly 
however,  it  is  being  recognised  by  both  employers  and  researchers  that  project  managers  not  only 
need  administrative  and  management  skills,  but  also  other  skills,  which  are  related  to  their 
management  style  and  emotional  intelligence.  This  paper  investigates  the  range  of  skills  needed  by 
a project  manager  immersed  in  a multimedia  development  environment,  and  are  classified  in  a 
continuum  from  “visible”  to  “invisible”.  These  are  compared  and  contrasted  to  consider  the  impact 
of  emotional  intelligence  on  the  overall  project  success,  including  the  team  synergy. 

Introduction 

Collaborative  teamwork  focused  on  project  work  is  becoming  an  increasingly  popular  process  by  which 
companies  seek  to  increase  efficiency  and  effectiveness  to  increase  their  profits  (Thomas  & Pinto,  1999).  This 
is  placing  pressure  on  a greater  number  of  “modern  day  project  managers”  to  act  as  visionaries,  technical 
experts,  negotiators,  sales  people  and  so  forth.  These  individuals  are  required  to  master  the  various,  sometimes - 
competing  demands  of  their  jobs  by  striking  an  effective  balance  between  the  roles  they  undertake.  Current 
research  is  advocating  that  successful  completion  of  projects  requires  project  managers  and  team  members  to 
have  effective  generic  skills  (Bennett,  Dunne,  & Carre,  1999;  National  Board  of  Employment  Education  and 
Training,  1996).  This  concept  is  gaining  huge  support  from  industry  as  well  as  higher  education  funding 
authorities,  and  many  reports  from  around  the  world  are  advocating  that  vocational  and  higher  education 
institutions  need  to  focus  more  on  improving  students  generic  skills  rather  than  just  providing  content 
knowledge  (Australian  National  Training  Authority,  1998;  Candy,  Crebert,  & O'Leary,  1994;  Bearing,  1997) . 

According  to  Wysocki  (1995,  p.l4),  project  managers  need  skills  in  three  broad  areas: 
managing/administrating,  applying  technical  skills  and  using  generic  skills.  The  management/administrative 
and  technical  skills  or  “visible  skills”  such  as  scheduling,  planning,  legal  issues  and  costing  are  well 
documented  and  supported  by  a myriad  of  training  courses.  However  “Invisible  skills”  needed  by  project 
managers  have  only  just  recently  been  recognised  as  being  important  by  both  academics  and  industry 
employers.  These  are  represented  in  Figure  1 as  a continuum,  in  which  technical  and  management  skills  such  as 
computer  programming,  design  or  authoring  skills  are  easily  proven  or  observable.  But  hidden  or  “invisible” 
skills  are  not  always  easy  to  demonstrate  i.e.  how  can  you  predict  if  a candidate  has  good  conflict  resolution 
skills  or  collaboration  skills? 

This  type  of  classification  is  also  supported  by  Belassi  & Oya  (1996)  who  developed  a framework,  based 
on  a wide  literature  review,  for  predicting  project  success  factors.  In  their  classification,  they  also  Kcognisethe 
importance  of  project  manager  and  team  member  attributes  as  critical  success/failure  factors  for  project 
development.  They  grouped  four  factors  that  can  potentially  affect  the  success  of  a project: 

• Project  manager  and  team  member  factors  - such  as  ability  to  delegate  authority,  ability  to  coordinate, 
perception  of  role  and  responsibilities,  communication  skills,  technical  background  and  commitment; 

• Overall  project  factors  - such  as  size  and  value,  uniqueness  of  the  project  activities,  urgency,  density  of 
project  and  life  cycle; 

• Organization  factors  - such  as  support  from  management  and  project  organisational  structure;  and 

• Environment  factors  - such  as  politics,  economics,  technology,  client  issues,  competition  and  sub- 
contractors. 

Figure  1 illustrates  these  broad  skill  sets  as  a continuum,  showing  technical  and  administration  skills  on  the 
“visible”  side  of  the  scale  and  generic  skills  and  emotional  intelligence  on  the  “invisible”  side.  This  paper 
contends  that  each  of  these  four  broad  skills  are  needed  for  successful  project  management.  However,  the  main 
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focus  of  this  paper  will  be  on  emotional  intelligence  and  it’s  relationship  and  impact  on  successful  project 
management. 
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Figure  1:  A spectrum  of  project  management  skills 


Skills  Needed  for  Multimedia  Project  Management 

In  the  field  of  multimedia  development,  project  managers  must  "juggle"  a range  of  team,  business, 
administrative  and  production  aspects,  as  shown  in  Figure  2.  These  aspects  have  a direct  affect  on  profit, 
delivery  time  and  final  product  quality  (Department  of  Industry  Science  and  Technology,  1995).  Clearly, 
project  managers  must  have  skills  and  experience  in  these  areas  to  help  create  successful  products,  which  are  a 
mixture  of  administrative,  management  and  technical  understanding  of  the  media. 

Increasingly  however,  it  is  being  recognised  that  the  “visible”  skill  set  shown  in  Figure  2 is  not  enough  for 
success.  Generic  skills  and  the  individual’s  management  style  are  increasingly  being  promoted  as  being  a 
critical  component  in  having  successful  outcomes.  Employers  now  actively  seek  employees  with  problem 
solving,  collaboration  and  communication  skills.  These  skills  are  considered  important  for  both  project 
managers  as  well  as  all  other  team  members  to  help  promote  the  development  of  successful  projects. 


Figure  2:  “Visible”  project  management  skills 

Another  factor  that  can  also  influence  project  management  success  is  the  management  style  used  by  the 
project  manager.  The  traditional  “command- and- control”  method  is  becoming  increasingly  redundant  in  a 
market  that  strives  to  empower  and  not  direct  team  members  (Dodgson,  1999).  Team  members  need  to  feel 
motivated,  enthusiastic  and  trusted  by  the  project  manager  to  make  correct  decisions.  Through  the  review  of 
literature  a link  will  be  identified  between  successful  project  management  and  emotional  intelligence.  The 
relationship  between  emotional  intelligence  and  project  management  is  discussed  in  the  following  section. 

What  is  emotional  intelligence? 

The  concept  of  emotional  intelligence  and  its  impact  on  project  management,  team  management  and 
overall  project  success,  especially  in  multimedia  design  and  development  is  relatively  new.  Salovey  and  Mayer 
(1990)  initially  conceived  the  concept  and  coined  the  term  Emotional  Intelligence.  This  concept  was  derived 
from  Gardner’s  (1983)  theory  of  multiple  intelligences.  The  notion  that  there  is  not  one  intelligence,  but  several 
intelligences,  forms  the  basis  of  multiple  intelligence  theory.  Gardner  (1983)  identified  seven  intelligences  - 


musical  intelligence, bodily- kinesthetic  intelligence,  logical-mathematical  intelligence,  linguistic  intelligence, 
spatial  intelligence,  interpersonal  intelligence  and  intrapersonal  intelligence. 

Intrapersonal  and  interpersonal  intelligences  were  used  by  Salovey  (1990)  to  form  the  basis  of  the  initial 
comprehensive  theory  of  emotional  intelligence  (Goleman,  1995;  Yost  & Tucker,  2000).  Personal  intelligences 
or  the  initial  idea  of  emotional  intelligence,  though  essentially  derived  from  Gardner’s  multiple  intelligences, 
does  not  explore  feelings  but  focuses  primarily  on  the  cognitive  aspects  of  feeling  (Goleman,  1995)  Salovey 
and  Mayer  (1990)  used  this  as  a basis  for  their  definition  of  emotional  intelligence,  which  they  define  as  the 
"ability  to  monitor  and  regulate  one’s  own  and  other’s  feelings,  and  to  use  feelings  to  guide  one’s  thinking  and 
action”  (p.l89).  This  definition  identified  five  main  domains:  knowing  one’s  emotions,  managing  emotions, 
motivating  oneself,  recognising  emotions  in  others  and  handling  relationships. 

Goleman  (1998b)  adapted  Salovey  and  Mayer’s  model  as  a basis  for  his  discussion  of  the  theory  of 
emotional  intelligence  and  it’s  implications  for  everyday  life  including  the  world  of  work.  The  model  includes 
five  emotional  and  social  competencies:  self-awareness,  self  regulation,  motivation,  empathy  and  social  skills. 
These  are  each  discussed  below. 

Self-Awareness 

Self-awareness  is  the  ability  to  understand  and  interpret  one’s  own  feelings  through  internal  reflection.  The 
ability  to  be  critical  about  thoughts  and  make  changes  to  behaviour  can  lead  to  an  in-depth  understanding  about 
one’s  self,  which  leads  to  a better  understanding  of  others.  Lanser(2000)  places  a strong  emphasise  on  the 
importance  of  self-awareness  in  guiding  and  perfecting  job  performance,  including  interactions  with  colleagues 
and  in  the  establishment  of  positive  and  productive  leadership  and  teamwork  skills.  Leaders  and  managers  need 
to  be  especially  aware  of  their  feelings  as  they  may  allow  uncontrolled  emotions  to  impact  on  the  dynamics  and 
culture  of  the  workplace  or  team.  Overt  expressions  of  negative  emotions  rather  than  controlled  self-awareness 
of  one’s  own  emotions  can  lead  to  conflict  and  can  create  an  unpleasant  working  environment.  Chemiss  (1998) 
emphasises  that  effective  leaders  are  self-confident,  which  is  reflective  of  their  own  emotional  self-awareness, 
and  ability  to  control  their  emotions. 

Self-Regulation 

Self-awareness  of  emotions  enables  managers  to  then  practice  self-regulation,  which  is  the  ability  to  use 
emotions  to  facilitate  the  progress  of  the  task  or  the  project  (Goleman,  1998b;  Lanser,  2000).  Being  able  to 
regulate  emotions  especially  during  conflict,  pressure,  stress  and  deadlines  facilitates  the  smooth  progress  of 
the  project  and  promotes  positive,  effective  working  relationships  with  team  members  and  clients.  Goleman 
(1998b)  explains  that  handling  emotions  and  putting  the  task  first  rather  than  emotions  aids  in  the  attainment  of 
the  required  goal.  Through  self-awareness,  project  managers  need  to  be  able  to  self^-egulate  their  emotions, 
and  then  facilitate  the  attainment  of  the  project  goals  through  positive,  productive  relationships  with  colleagues 
in  a team  environment. 

Motivation 

Being  able  to  motivate  team  members  into  contributing  their  best  is  very  powerful.  Workers  are 
discretionary  in  their  application  to  a project  - they  will  only  give  if  they  feel  they  are  being  supported, 
nurtured  and  inspired.  Managers  need  to  create  an  environment  that  stimulates,  enhances  and  empowers  team 
members  to  become  motivated  and  apply  themselves  fully  to  the  project  (Grossman,  2000) . Successful 
leadership  requires  intrinsic  motivation,  persistence  and  vision  (Chemiss,  1998).  Project  managers  are  not  only 
responsible  for  their  own  motivation  but  are  also  play  a key  role  in  motivating  the  team  and  colleagues. 
Goleman  (1998b)  and  Lanser  (2000)  propose  that  motivation  is  an  essential  element  of  emotional  intelligence 
that  pushes  us  forward  through  the  positive  and  negative  aspects  of  working  life  by  showing  initiative, 
perseverance  and  dedication,  as  well  as  being  goal  orientated,  focussed  and  proactive.  Grossman  (2000,  p.  19) 
explains  that  the  managers  and  leaders  with  high  emotional  intelligence  “can  motivate  members  of  an 
organization  to  work  more  effectively  and  efficiently  - in  short,  produce  a better  product”  (p.  19). 

Empathy 

Goleman  (1998b)  contends  that  empathy  is  understanding  and  interpreting  colleagues’  feelings  and  being 
able  to  identify  with  their  feelings  on  issues  through  understanding  their  perspective  and  cultivating  rapport 
with  people  from  different  ‘walks  of  life’.  Empathic  project  managers  have  an  awareness  of  the  diversity  of 
personalities  and  are  accepting  of  the  diversity  of  people  and  the  impact  culture  can  have  on  interactions  within 
a team  environment.  (Book,  2000),  defines  empathy  as  the  “capacity  to  see  the  world  from  another  person’s 
perspective”  (p.  45).  This  skill  allows  project  managers  to  understand  other  peoples’  behaviour  without 


commenting  its  appropriateness  or  inappropriateness,  which  has  the  potential  to  turn  adversarial  relationships 
into  collaborative  alliances. 

Social  Skills 

Social  skills  are  essential  for  the  development  of  positive,  effective  relationships  with  colleagues  and  the 
ability  to  interact  with  team  members  to  deter  conflict,  be  aware  of,  ease  and  dissipate  underlying  tensions  that 
can  accumulate  and  have  a negative  impact  on  working  relationships  and  project  success.  Project  managers 
need  to  be  able  to  stimulate  cooperation,  collaboration  and  teamwork  through  well-developed  social  skills 
(Goleman,  1998b) 

It’s  not  hard  to  think  of  examples  of  managers  or  team  members  that  don’t  have  these  skills.  Often,  these 
people  are  referred  to  as  “abrasive”,  “cold”,  “unfeeling”,  “self-centred”,  “inflexible”  or  “boring”  and  are 
usually  hard  to  work  with.  The  following  section  describes  how  emotional  intelligence  plays  an  important  role 
in  project  management. 

Emotional  Intelligence  and  Project  Management 

Grossman  (2000)  believes  that  there  is  “more  to  management  than  a keen  intellect  and  grasp  of  technical 
knowledge.  The  difference  between  success  and  mediocrity  most  often  can  be  attributed  to  a leader’s  mastery 
of  softer  skills-  abilities  and  approaches  grounded  in  emotional  intelligence”  (p.l8).  This  is  also  supported  by 
Tucker  (2000),  who  contends  that  successful  project  management  requires  strong  technical  and  theoretical 
knowledge  as  well  as  emotional  intelligence.  Project  failure  generally  does  not  occur  because  of  lack  of 
technical  knowledge,  but  more  often  because  the  project  manager  has  a deficit  of  emotional  intelligence,  which 
is  essential  for  project  success  and  team  cohesiveness  and  effectiveness. 

Book  (2000)  explains  that  emotional  intelligence  is  more  important  than  theoretical  and  technical  skills. 

He  considers  self-awareness,  self-regard,  self-actualisation,  flexibility,  social  skills  and  open  communication  as 
essential  attributes  for  successful  project  management.  Project  managers  should  “feel  comfortable  with 
themselves  to  acknowledge  their  mistakes  and  flaws... and  have  the  capacity  to  tolerate  not  knowing  all  the 
answers”  (p.  45). 

Emotional  intelligence  is  also  reflected  in  management  styles.  Carlos  & Taborda  (2000)  promote  that  new 
management  styles  are  needed  in  the  industry,  which  focus  on  team  orientation  strategies  and  empowerment, 
rather  than  the  traditional  command  and  control  paradigm.  This  type  of  management  style  places  an  emphasis 
on  organic  commitment  and  involves  everybody  in  decision  making  with  participative  approaches  to  problem 
solving.  They  contend  that  this  management  style  encourages  employee  initiative  and  an  atmosphere  of  “high- 
expectations,  appreciation,  and  excitement”  (p.  42). 

The  value  of  empowering  team  members,  and  providing  a supportive  project  environment  (rather  than 
goal- directed)  is  supported  through  a study  performed  by  Tampoe  & Thurloway  (1993).  From  the  responses  of 
a range  of  knowledge  workers  in  different  organisations,  they  determined  that  many  of  the  respondents’  drives 
to  succeed  were  adversely  affected  by  approaches  that  focus  on  deliverables  rather  than  the  empowerment  of 
teams.  They  concluded  that  the  trend  towards  flattening  hierarchies,  multi-skilling  and  team  working  result  in 
increased  effectiveness  as  well  as  motivation. 

The  review  of  the  literature  promotes  that  project  managers  must  create  an  environment  that  fosters 
emotional  intelligence  and  encourages  team  members  to  contribute.  Some  organizations  are  not  flexible  enough 
and  don’t  allow  or  recognise  the  potential  for  team  members  to  contribute  beyond  the  expectations  of  the  job. 
However,  by  leading  through  example,  fostering  creativity  and  imaginative  thinking  and  allowing  others  to 
provide  an  input  into  the  development  of  the  project  can  promote  greater  project  and  team  success.  In  order  to 
promote  positive,  progressive,  effective  working  environments,  project  managers  need  to  have  a combination 
of  technical  knowledge  and  well -developed  emotional  intelligence  including  self-awareness,  empathy,  social 
awareness  and  be  highly  motivated  and  be  able  to  inspire  and  motivate  team  members. 

Based  on  Goleman’s  (1998a)  work.  Table  1 has  been  developed  which  identifies  attributes  of  each 
emotional  intelligence  competency.  Through  the  literature  we  have  identified  representative  examples  of  each 
competency  that  relate  to  project  management. 

We  have  shown  that  management  and  technical  competencies  need  to  be  complemented  with  behavioural 
or  affective  competencies  in  order  to  promote  the  development  of  successful  projects.  In  summary,  Martin 
(1993),  through  a synthesis  of  literature,  reinforces  the  legitimacy  and  wisdom  of  recognising  an  affective 
dimension  in  decision-making.  The  dimension  reflects  sensitivity  to  feelings,  emotions  and  the  plight  of 
employees,  customers  and  others  by  listening  to  the  “voices  of  the  heart”.  He  suggests  that  managers  should 
avoid  making  decisions,  which  are  based  solely  on  the  cognitive  domain,  that  are  based  solely  on  rational  and 
analytical  reasoning,  and  proposes  that  quality  business  decisions  are  based  on  both  cognitive  and  affective 
understanding  (Figure  3). 


Definition 

Relationship  to  PM 

Self- 

Awareness 

• The  ability  to  recognise  and 
understand  your  moods, 
emotions,  and  drives,  as  well 
as  their  effect  on  others 

• having  positive  and  productive  leadership  and  teamwork  skills 

• controlling  emotions  and  understand  the  impact  of  emotions  on  the  team 

• being  selfconfident 

Motivatif  Self- 

n Regulation 

• The  ability  to  control  or 
redirect  disruptive  impulses 
and  moods 

• The  propensity  to  suspend 
judgement  ~ to  think  before 
acting 

• A passion  to  work  for  reasons 
that  go  beyond  money  or 
status 

• Pursuing  goals  with  energy 
and  persistence 

• being  selfaware  of  emotions  to  enable  selfregulation 

• handling  emotions  and  putting  the  task  first 

• using  emotions  to  facilitate  the  progress  of  the  project 

• regulating  emotions  during  conflict,  pressure,  stress  and  deadlines 

• facilitating  the  smooth  progress  of  the  project  and  promoting  positive, 
working  relationships  with  team  members  and  clients 

• motivating  team  members  into  contributing  their  best 

• creating  an  environment  that  stimulates,  enhances  and  empowers  team 
members  to  become  motivated  and  apply  themselves  fully 

• showing  initiative,  perseverance  and  dedication,  goal  orientation  & focus 

1o 

a. 

E 

LU 

• The  ability  to  understand  the 
emotional  makeup  of  other 
people 

• Treating  people  according  to 
their  emotions 

• understanding,  and  identifying  with  colleagues?  feelings 

• cultivating  rapport  with  people  from  different  ‘walks  of  life’ 

• Having  awareness  off,  and  accepting  of  the  diversity  of  personalities  and 
cultures 

• T uming  adversarial  relationships  into  collaborative  alliances 

Social 

Skill 

• Proficiency  in  managing 
relationships  and  building 
networks 

• An  ability  to  find  common 
ground  and  build  rapport 

• developing  of  positive,  effective  relationships  with  colleagues 

• having  the  ability  to  interact  with  team  members  and  deter  conflict,  be 
aware  of,  ease  and  dissipate  underlying  tensions 

• being  able  to  stimulate  cooperation,  collaboration  and  teamwork  through 
well-developed  social  skills 

Table  1:  Emotional  Intelligence  & PM  Competencies  (modified  fromGoleman,  1998a) 


Likewise,  Coleman  (1998a)  contends  that  emotional  intelligence  is  twice  as  important  for  leaders  than 
technical  skills  and  intellect.  Excellence  in  performance  relies  on  emotional  intelligence  as  a major  ingredient 
for  success.  As  managers  “move  up  the  ladder”,  higher  levels  of  em  otional  intelligence  are  needed  than  at 
levels  where  technical  skills  are  essential.  In  recruiting  project  managers,  employers  should  be  aware  of 
emotional  intelligence  requirements,  as  outlined  in  Table  1.  Martin’s  decision  making  dimensions  of  affective 
understanding  and  cognitive  understanding  have  been  adapted,  as  illustrated  in  Figure  4,  to  reflect  our  belief. 


based  on  the  literature,  that  the  combination  of  “invisible”  and  “visible”  skills  affect  decision  making  quality 
and  have  a direct  impact  on  effective  project  management  and  project  success. 

Conclusion 

Project  managers  need  to  consider  not  just  technical  and  management  skills  for  successful  project 
management,  but  also  generic  skills  and  emotional  intelligence  at  the  “invisible”  end  of  the  scale.  As  illustrated 
in  this  paper,  emotional  intelligence  is  clearly  an  important  skill  for  successful  project  management,  team 
management  and  helps  promote  project  success  and  quality  decision  making.  Project  management  courses 
should  therefore  focus  not  only  on  the  “visible”  skills  but  also  on  “invisible”  skills.  Project  managers  would 
gain  immensely  from  courses  that  provide  them  with  an  understanding  of  the  impact  of  generic  skills  and 
emotional  intelligence  on  overall  project  success  and  team  synergy.  We  suggest  that  developing  these 
“invisible”  skills  should  be  part  of  any  project  management  course. 
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<4hstract> 

In  this  paper  we  present  the  overall  framework  and  the  content  of  a web- based  Japanese  listening  program  and  its  preliminary  experimental 
results.  The  program  is  basically  designed  not  only  to  help  learners  of  basic  level  Japanese  to  develop  their  perceptional  skills  of  Japanese 
sounds  but  also  to  be  used  as  a remedial  course  for  higher  levels  of  students  who  still  have  problems  of  discerning  segments  of  Japanese 
sounds  and  syllables.  We  believe  that  the  listening  for  perception  is  an  area  which  can  fully  take  advantage  of  multimedia  as  it  enables  us  to 
make  interactive  and  interesting  exercises  and  serves  learners  with  a dynamic  learning  environment  where  they  can  learn  at  their  own  pace, 
irrespective  of  their  abilities  or  learning  preferences.  The  rate  of  achievements  and  participation  of  each  student  are  automatically  recorded 
and  the  students  can  see  their  own  process  of  progress  in  both  histogram  and  numerical  charts.  The  evaluation  of  the  program  was  performed 
by  means  of  a questionnaire  and  comparing  the  results  of  pre-  and  post  treatment  tests.  In  the  preliminary  experiment  we  could  observe  a 
positive  response  towards  the  program. 

Keywords:  Japanese  Education,  Listening  for  Perception,  Sound  Segments,  Syllables,  Accent  Patterns,  Management  System 

1.  Introduction 

Since  1999,  the  Language  and  Culture  Department  at  Dokkyo  University  has  started  to  admit  foreign  students  without  any 
previous  knowledge  of  Japanese.  In  order  to  let  them  acquire  enough  knowledge  and  skills  in  Japanese  in  two  years  and 
educate  them  to  be  ready  for  studying  specialized  subjects  in  Japanese,  we  have  organized  four  levels  of  Japanese  courses 
which  amount  to  approximately  1,100  hours  of  study  in  total.  The  schedule  and  the  contents  are  very  tight  and  demanding. 
Under  these  circumstances,  it  is  necessary  for  us  to  prepare  as  much  efficient  and  interesting  teaching  materials  as  possible. 
Generally,  studies  for  developing  sound  recognition  skills  are  time  consuming  and  tedious,  and  the  students  tend  to  loose 
interests  and  do  not  pay  enough  attention  to  the  listening  for  perception.  One  way  to  solve  this  problem  and  to  use  time  more 
efficiently  is  to  construct  a CALL  listening  program  which  provides  students  with  interactive  and  interesting  exercises, 
namely,  a dynamic  learning  environment  where  they  can  learn  at  their  own  pace,  irrespective  of  their  abilities  or  learning. 

2.  Content  of  Listening  and  the  Purposes 

The  content  of  listening  can  be  divided  into  “listening  for  perception”  and  “listening  for  comprehension.”  The  studies  on 
comprehension  process  have  demonstrated  various  aspects  of  listening  activities,  not  to  mention  the  roles  of  supra- segmental 
characteristics  of  the  input,  those  of  background  knowledge  and  context  have  been  duly  emphasized.  While  listening,  students 
are  encouraged  to  use  their  background  knowledge  and  the  contextual  information  so  that  they  can  predict  and  understand  the 
forthcoming  information.  They  are  instructed  to  listen  for  purpose  and  the  meaning,  but  not  to  pay  too  much  attention  to  an 
individual  sound.  While  the  latter  type  of  listening  has  been  emphasized,  the  former  type,  namely,  “listening  for  perception” 
seems  to  be  less  emphasized  in  the  recent  classroom  activities.  Considering  that  students  should  practice  both  types  of 
listening  exercises  from  the  beginning,  we  have  decided  to  construct  the  program  consisting  of  two  types  of  listening  with 
equal  emphasis,  and  will  investigate  the  relationship  between  these.  Few  of  the  Japanese  listening  programs  demonstrate  data 
on  its  implementation  as  a part  of  classroom  activities.  Here  we  also  present  a preliminary  evaluation  of  the  program  after 
having  used  it  for  forty  days  on  the  basic  level  of  students  at  Dokkyo  University. 

3.  Framework  of  the  Listening  Program 


A number  of  interesting  pronunciation  and  listening  exercise  texts  have  been  developed  (1).  Most  of  them,  however,  are  not 
internet -based  (2).  What  is  new  about  our  program  is  that  it  is  internet -based  and  thus  can  provide  a dynamic  learning 
environment  where  any  learners  who  are  interested  in  developing  their  listening  skills  can  learn  at  their  own  pace,  abilities  and 
preferences.  Unlike  traditional  listening  materials  introduced  above  and  diso-based  training,  this  interactive  program  on  the 
web  ensures  learners  a control  of  the  learning  process,  which  is  expected  to  help  students  maintain  their  motivation  for 
learning.  . 

The  program  consists  of  two  parts:  Part  One  centers  around  various  sound  recognition  exercises  and  Part  Two  with  different 
types  of  listening  comprehension  exercises.  Part  One  is  further  arranged  to  cover  three  main  areas:  1)  "recognition  of 
individual  sounds,  " 2)  "discerning  syllabic  structures"  and  3)  "recognition  of  accent  patterns."  Since  the  difficulties  in  learning 
Japanese  sound  system  lie  not  only  in  distinguishing  individual  sounds,  but  also  in  recognizing  a syllabic  structure  of  words 
and  accent  patterns,  we  have  decided  to  choose  these  topics  as  the  first  step  to  develop  Japanese  listening  CALL  materials  on 
the  web.  The  top  page  of  the  program  (Fig.  1)  functions  as  an  index  page,  from  which  students  can  jump  to  any  one  of  these 
exercises  by  clicking  the  exercise  number. 


Fig.  1 Top  and  Index  Pager 


4.  Part  I - Sound  recognition  exercises 

Sound  recognition  exercises  consist  of  five  stages:  1)  recognition  of  one  syllable,  2)  recognition  of  words,  3)  discrimination 
of  two  similar  words,  4)  discrimination  of  two  similar  short  sentences  and  5)  extracting  the  words  in  long  sentences.  Students 
can  jump  from  one  exercise  to  the  other  if  they  want  to,  but  in  the  classroom  situation,  the  instructor  encouraged  them  to  go 
through  from  stage  1)  to  stage  5).  Figure  2.  shows  an  example  of  a sound  recognition  exercise  in  which  students  click  the 
corresponding  "hiragana"  (a  Japanese  alphabet)  on  the  screen  upon  having  heard  the  "alphabet."  Since  Japanese  is  a syllabic 
language,  the  number  of  syllables  is  limited.  Thus  it  is  possible  to  let  the  learners  discern  each  syllabic  sound  as  a basic  part  of 
listening  exercises  by  means  of  minimal  pair.  The  order  of  the  recorded  “hiraganas,"  as  in  an  Exercize  1)  ,“recognition  of  one 
syllable,”  is  constructed  to  alter  automatically  whenever  they  start  working  on  the  exercises  from  the  beginning.  This  device 
has  been  arranged  so  that  the  students  will  not  find  the  task  boring  and  ineffective.  Neither  can  they  rely  on  the  route 
memorization  of  the  order  of  the  sounds.  However,  in  case  learners  want  to  do  other  exercises,  they  can  jump  back  to  the  index 
page  and  choose  any  exercises  they  prefer  to  do. 

Figure  3 presents  another  type  of  sound  recognition  exercise.  In  this  exercises  students  listen  to  the  word  and  write  the  word 
by  dragging  down  the  appropriate  "kana"  syllable  out  of  the  chart  on  the  screen.  When  the  answer  is  correct,  the  "OK"  mark 
and  the  illustration,  which  depicts  the  meaning  of  the  word,  appear  on  the  screen  and  confirm  that  the  answer  is  correct.  This 
kind  of  visual  information  can  not  only  help  them  understand  the  meaning  of  the  word  but  also  serves  them  to  enjoy  the 
program.  In  order  to  make  these  exercises  more  interesting,  the  illustrations  are  sometimes  constructed  to  move. 


®E§T  COPY  AVAHABILI 


1373 


Fig.  2 Sound  recognition  exercise 


Fig  3.  Word  recognition  exercise 


4-1.  Exercises  on  syllabic  structure 

In  addition  to  the  exercises  introduced  above,  where  the  students  should  recognize  segmental  sounds,  we  have  exercises 
designed  to  let  learners  pay  attention  to  the  number  of  syllables  and  syllabic  structure.  Even  though  Japanese  is  frequently 
cited  as  an  example  of  a language  with  relatively  simple  syllabic  structure,  it  is  not  easy  for  non-Japanese  students  to  recognize 
the  syllabic  structure  and  count  the  number  of  syllables  in  any  given  word.  The  Japanese  syllable  is  defined  fundamentally  in 
terms  of  the  number  of  syllables  in  the  word  and  each  syllable  holds  isochoronous  duration.  What  is  particularly  difficult  is  to 
recogniz  e consonant  clusters  such  as  /tt,  kk,  ss/  , and  a syllabic  nasal  /N/  as  one  syllable,  especially  so  to  the  learners  whose 
languages  do  not  admit  such  consonant  clusters  and  a nasal  segment  as  one  syllable.  There  is  one  more  problem  we  have  to 
deal  with;  that  is  the  treatment  of  long  vowels,  namely  /aa,  ii,  ee,  uu,  oof.  These  long  vowels  should  be  basically  pronounced 
with  the  length  of  two  isochronous  durations,  while  short  vowels  with  one.  For  example,  the  [ba]  of  the  word  "obasan" 
[obasaN]  is  a short  vowel  and  the  [baa]  of  the  word  [obaasaN]  is  a long  vowel.  The  first  one  has  four  durations  and  means 
"aunt”,  and  the  second  one  has  five  durations  and  means  "grandmother."  Since  there  are  basically  no  striking  quality 
differences  between  long  and  short  vowels  in  Japanese,  it  is  very  difficult  for  learners  of  Japanese  to  discern  them.  These  do 
not  only  impose  problems  in  listening  but  also  in  writing  and  in  speaking.  Considering  that  errors  in  these  area  are  carried 
forward  even  after  many  years  of  study,  exercises  in  three  areas  are  extensively  presented  in  the  program  (Fig.  4). 


Fig.  4 Number  of  syllables  Fig.  5 Accent  Pattern 

5.  Exercises  on  accent  patterns 

In  addition  to  those  mentioned  above,  we  have  prepared  the  exercises  in  which  learners  are  trained  to  recognize  word  accent 
patterns.  In  English,  for  example,  accented  syllables  are  pronounced  louder,  longer  and  more  clearly  than  unaccented 
syllables.  In  Japanese,  accent  has  nothing  to  do  with  them,  but  it  rather  involves  changes  in  voice  pitch.  The  words  with  the 
same  phonetic  segments  but  with  different  location  of  accented  syllable  often  signify  totally  different  referents  in  Japanese. 
For  example,  the  words  "Ame”  with  the  first  syllable  accented,  and  "aME"  with  the  second  syllable  accented  mean  "rain"  and 
"a  candy"  respectively.  Accordingly,  to  be  able  to  discern  the  accent  pattern  is  also  another  important  task  for  the  learners  of 
Japanese.  In  this  area,  we  also  provide  various  types  of  exercises  with  different  ways  of  carrying  out  the  tasks  such  as  "to 
click  the  correct  answer  out  of  multiple  choices,"  "to  answer  whether  the  accent  patterns  of  the  two  words  they  hear  are  the 
same  or  not,"  and  "to  make  up  accent  patterns  on  their  own"  by  clicking  the  circles  as  shown  in  Figure  5. 

6.  Listening  Comprehension  Exercises 
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Part  Two  consists  of  various  listening  comprehension  exercises  whose  contents  are  closely  related  to  certain  topics, 
situations,  grammar  points  and  communicative  functions.  In  preparation  for  these  exercises,  we  have  tried  to  incorporate  the 
aspect  of  interactions  between  the  learners  and  the  program.  Our  intention  is  supposed  to  some  extent  be  reflected  in  the 
various  manners  of  answering  the  questions  and  doing  the  exercises.  In  this  program  other  than  multiple  types  of  exercises,  we 
have  designed  exercises  in  which  the  system  lets  learners  draw  pictures  by  cutting  and  pasting  the  items  given  on  the  screen 
and  make  up  answers  on  their  own.  Figure  6 is  an  example  where  this  “cut— drag-paste”  manipulation  is  used.  The  task  is  to 
put  the  clothes,  bags,  shoes,  and  ribbons  on  the  bear  while  listening  to  the  story.  Upon  completing  the  task,  they  click  the 
answer  bottom,  and  the  picture  of  the  bear  appears  next  to  the  one  made  up  by  the  student.  Fig. 7 is  another  type  of 
manipulation  adopted  in  Part  Two.  The  task  is  to  indicate  the  way  to  arrive  at  the  destination.  While  listening  to  the  directions, 
such  as,  “Go  straight  and  turn  right  at  the  comer  of  a flower  shop...”  students  have  to  click  a number  of  squares  which  consist 
of  roads.  The  color  of  these  squares  changes  when  they  click  them  like  a red  carpet  is  gradually  spread.  When  students  click 
the  answer  key,  the  black  ball  starts  roll  down  the  road  which  leads  to  the  correct  destination. 


Fig.6  Let’s  try  a coat  on  the  bear  - Fig.  7 Let’s  go  to  the  restaurant  - “cut -drag-paste”  type 

7.  Management  System 

All  of  these  exercises  are  equipped  with  a Managing  System  which  is  constructed  by  using  Perl.  As  shown  in  Fig. 8,  the 
teacher  who  takes  a role  of  administrator  registers  students  in  the  Server.  At  the  moment,  only  the  students  who  give  their 
registered  ID  and  password  can  gain  access  to  the  program.  Upon  completing  each  exercise,  a percentage  of  correct  answers  is 
automatically  calculated  and  presented  on  the  screen  (Fig. 9). 

Even  though  in  the  preliminary  experiment,  we  used  the  program  in  the  classroom  situation,  our  objective  is  to  let  the 
students  use  this  computer  assisted  listening  program  as  a part  of  homework  or  as  an  extra  training  on  individual  basis. 
Accordingly,  dates  and  degree  of  participation  are  recorded  and  sent  to  the  instructor's  homepage  linked  to  this  program.  From 
administrative  point  of  view,  as  its  being  an  internet  based  system,  this  program  can  provide  efficient  time-saving  method  of 
keeping  track  of  each  learner’s  progress  and  can  also  record  each  learner  with  the  items  of  tests’  scores  as  shown  in  Figure  10. 
Not  only  the  instructor  can  look  into  the  development  of  studies  achieved  by  each  one  of  the  learners  but  also  the  learners 
themselves  are  able  to  get  into  their  own  page  and  look  into  the  p rocess  of  their  own  achievement. 


Fig.8  Registration  Fig.9  Grading  Fig.  10  Scores  of  individual  students 

Other  than  these  automatic  feedback,  students  can  receive  messages  from  the  instructor.  Naturally,  whenever  they  have 
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any  problems,  they  can  ask  for  a help  by  e-mail. 


8.  Preliminary  Experiments 

8-1  Results  of  Questionnaire  on  the  program 

One  of  the  main  purpose  of  this  preliminary  experiment  was  to  find  out  how  they  would  react  to  the  computer-assisted 
listening  learning  and  to  see  what  should  be  dene  to  make  the  program  easy  to  handle  for  them.  In  order  to  confirm  their 
reactions,  we  carried  out  a questionnaire  after  a 40  days  period  of  session  consisting  of  three  hours  of  classroom  activities  and 
individual  studies  in  their  free  time.  Objectively  peaking,  the  part  of  the  sound  recognition  exercises,  even  though  we  have 
designed  it  to  be  interesting  and  varied,  is  rather  monotonous.  Thanks  to  the  advantages  of  computer's  being  multi  as  well  as 
student- centered  media,  however,  the  students  did  not  gpt  bored  but  become  very  much  involved  in  the  activities  and 
expressed  their  wish  to  participate  in  this  listening  program.  Table  1 shows  the  results  of  the  evaluation  given  by  the  students 
The  scale  of  evaluation  was  from  number  5 to  0 and  the  number  5 stands  for  the  highest  score. 


WBT  ( Wav  Based  Training) 

Ave. 

SD 

WBTis  interesting  and  fun 

4.71 

0.38 

My  listening  skill  has  greatly  improved 

3.86 

0.49 

Want  to  continue  WBT 

4.86 

0.38 

User  Interface 

Ave. 

SD 

Easy  to  use 

4.86 

0.38 

Easy  to  understand  "icons'' 

4.86 

0.38 

Video  is  helpful 

4.57 

0.43 

Speech  sound  is  very  clear 

4.14 

0.38 

Grading  system 

Ave. 

SD 

Grading  system  is  easy  to  understand 

3.14 

1.07 

Easy  to  see  which  exercises  or  sections  should  be 

redone 

4.33 

0.38 

Easy  to  see  the  rate  of  achievemert 

3.36 

0.43 

Table  1 Results  of  Questionnaire  - Evaluation  of  the  Program  the  Students 

As  seen  in  the  table,  their  overall  reactions  towards  the  CALL  learning  were  very  positive.  Many  of  them  expressed  their 
wish  to  continue  this  type  of  studies. 

8-2  Quantitative  evaluation  of  the  program 

During  the  experimental  period,  regular  class  activities  were  conducted  as  normal  to  cover  the  course  syllabus.  The  students 
were  recommended  to  access  the  materials  until  they  would  complete  all  the  exercises  with  final  score  of  90  points  or  better. 
How  long  and  how  much  they  would  study  was  up  to  them.  With  the  help  of  the  Management  System  we  could  monitor  their 
learning  progress  and  confirm  that  all  of  the  students  had  fulfilled  the  task.  At  the  end  of  the  designated  period  of  time,  we 
conducted  a listening  test  for  perception.  The  quantitative  evaluation  of  the  program  was  performed  by  comparing  the  results 
of  pre-  and  post  treatment  tests.  The  mean  score  of  the  test  before  the  implementation  was  82.3,  and  the  score  after  the 
implementation  was  85.4,  and  the  overall  rate  of  progress  was  18.6%.  Taking  into  consideration  that  the  regular  class  activities 
were  conducted  during  the  period  and  the  rate  of  the  improvement  was  rather  slight,  it  might  be  difficult  to  claim  that  the 
improvement  in  their  listening  skills  was  achieved  solely  by  the  program.  Also  there  are  still  many  issues  to  be  considered  to 
obtain  reliable  data  to  judge  the  efficiency  of  the  program  including  the  number  of  the  participants  and  the  content  of  the  test. 


Even  though  a concrete  analysis  of  the  results  is  to  be  performed  in  due  course,  we  can  say  that  the  results  of  the  tests  indicate 
that  the  program  could  help  them  improve  their  listening  skills  to  some  extent. 

9.  Conclusion  and  Future  Plan 

As  for  the  content  of  the  listening  program,  there  are  two  important  tasks  that  remain  to  be  fulfilled.  One  is  to  prepare 
exercises  to  instruct  super- segmental  aspects  of  the  Japanese  sound  system:  rhythm,  prominence  and  intonation  and  the  other 
is  to  complete  Part  Two  - Listening  for  Comprehension.  Another  task  is  to  carry  out  an  extensive  experiment  to  evaluate  the 
effectiveness  of  this  listening  program.  The  content  of  the  test  should  be  examined  and  a large  number  of  subjects  should  be 
involved  to  ^t  a reliable  data.  The  method  of  experiment  should  be  also  reconsidered.  For  example,  we  will  divide  students 
into  two  groups.  The  first  group  studies  Japanese  sounds  using  this  program,  and  the  other  group  studies  the  exactly  same 
contents  using  a conventional  means  of  study:  namely,  a paper  and  tape  recorder.  If  the  group  of  students  who  use  the 
computer  make  more  progress  than  those  who  use  a conventional  method  of  learning;  namely,  a tape  recorder,  we  will  be  able 
to  ascertain  effectiveness  of  the  CALL  materials.  However,  this  kind  of  experiments  is  not  easy  to  carry  out  without 
collaboration  from  other  universities  or  institutes.  Speaking  of  quantitative  measurements  in  general,  we  also  have  to  keep  in 
mind  that  there  are  many  variables  to  be  considered,  such  as  students’  motivation,  aptitude,  and  affective  conditions. 
Accordingly,  the  results  may  not  be  straightforward  and  distinctive  in  quantitative  terms  (3)  We  believe  that  to  figure  out  how 
these  variables  affect  the  learning  process  on  Web  is  also  an  important  area  to  be  studied  in  the  future. 

Finally,  we  would  like  to  confirm  that  the  preliminary  study  of  our  CALL  program  has  proved  its  usefulness  to  improve 
students*  listening  skill.  More  significantly,  it  had  indicated  that  the  program  could  provide  a learning  environment  where  the 
students  were  able  to  actively  engage  in  the  learning  of  Japanese  sounds  and  to  maintain  their  motivations. 

Note: 

(1)  For  example:  “Introduction  to  Japanese  Pronunciation,  Theory  and  Practice”  by  ShinichiTanaka,  etc.  al  Kuroshio  Shupan 
(1999),  “Instructions  Prosody  to  Japanese  Learners”  by  H.  Matsuzaki  presented  at  University  of  Hiroshima(2000) 

(2)  ”Nala-u  (Nihongo  Accent  Learning  Aid  - Universal)”  by  Yukihiro  Nishinuma.  This  is  one  of  a very  few  multimedia 
programs  on  speech  sound  education  in  Japanese. 

(3)  There  have  been  a few  research  on  developing  a CALL  listening  program,  but  few  of  them  have  systematically  covered  the 
various  aspects  of  the  listening  activities  and  have  been  actually  used  as  a part  of  classroom  activities  and  proved  its 
efficiency  (Mizumachi  et  al,  2001) 
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Abstract;  This  brief  paper  describes  the  implementation  of  one  tool  for  modeling  of 
workflows  of  authoring  courses  through  of  one  graphic  editor  based  on  SVG  technology. 


Introduction 


Workflow  modeling  techniques  can  express  in  the  form  of  a graphic,  a flow  of  activities  as  the 
collaborative  authoring  of  courses  through  WWW,  even  so,  the  implementation  of  these  models  is  one  of  the 
main  difficulties  found  in  the  use  of  workflow  technologies.  Seeking  to  facilitate  this  implementation  the  project 
’’Conception  of  Cooperative  Environment  for  Editing  Multimedia  Documents  with  Workflow  Technology” 
(CEMT)  proposed  the  creation  of  an  graphic  workflow,  editor  with  base  in  the  software  Amaya  (Vatton,  2001), 
the  Amaya  Workflow  (AW).  The  objective  of  AW  is  to  facilitate  the  determination  and  implementation  of 
workflows  for  courses,  for  that  it  generates  through  a graphic  interface  based  on  standard  SVG  (SVG,  2001)  a 
XML  file  with  the  instructions  of  a workflow  that  it  specifies  the  authoring  or  execution  of  a course.  After  the 
files  have  been  generated  they  can  be  sent  for  a workflow  machine  that  will  interpret  and  to  execute  the 
projected  workflow. 

The  environment 

The  software  Amaya  is  the  official  Browser/Editor  of  W3C,  now  is  being  developed  in  INRIA,  this 
software  have  among  other  resources  a support  for  technologies  XML  and  SVG.  SVG  is  a pattern  of  graphs 
representation  official’s  of  the  W3C,  Amaya  hold  an  editor  of  graphs  that  follows  this  pattern  and  have  as  base 
this  editor  the  project  CEMT  made  the  implementation  of  AW,  that  is  an  editor  graph  of  the  technique  of 
workflow  modeling  ’’Workflow  Process  Model  ” developed  by  Casati/Ceri  (WIDE,  1997). 

With  this  editor  a user  can  define  a workflow  graphically  conform  the  model  of  Casati/Ceri.  Paralleling 
with  the  generation  of  graphic  SVG  the  AW  generates  a file  XML  that  contains  all  data  about  the  workflow  that 
is  being  modeled,  for  this,  the  project  CEMT  needed  to  define  a DTD  of  Workflow,  the  files  XML  that  follow 
this  DTD  are  capable  to  store  information  that  can  recompose  the  flow  of  the  model  and  the  necessary 
information  for  its  execution. 

The  objective  of  AW  is  to  facilitate  the  determination  and  implementation  of  workflows  for  courses  in 
the  WEB,  for  that  it  generates  through  a graphic  interface  a file  XML  with  the  instructions  of  a workflow  that 
specifies  the  authory  or  execution  of  a course.  The  files  are  generated  so  that  after  they  are  interpreted  and 
executed  by  a workflow  machine. 

Implementation 

The  AW  (Amaya  Workflow)  is  a workflow  editor^rowser  with  any  features  that  facilitates  the  graphic 
edition  of  documents  that  express  a workflow  in  accord  with  the  symbols  and  semantics  of  the  modeling 
technique  Workflow  Process  Model. 

The  user  can  edit  the  document  using  a graphic  tool  added  to  the  software  Amaya,  this  graphic  tool 
consists  of  a palette  with  symbols  of  the  modeling  technique  Workflow  Process  Model. 

With  a palette  of  symbols  the  user  add  workflow  elements  (that  visually  are  geometric  graphics)  in  the 
document,  conform  the  symbols  are  added,  the  workflow  is  structured  with  its  tasks,  super  tasks,  forks.  Joins  and 


connections,  generating  in  this  way  the  files  that  describe  graphically  the  logical  flow  of  the  activities  and  in  the 
format  XML  the  logical  structure  of  the  workflow. 

Each  element  have  a list  of  attributes  where  information  about  the  workflow  that  is  being  modeled  are 
stored.  Many  attributes  are  automatically  set  for  the  system  conform  is  made  the  graphic  distribution  of  the 
elements.  Other  attributes  should  be  set  for  the  users,  to  fill  and  to  edit  these  attributes  the  user  needs  to  select 
the  element  in  that  want  to  add  attributes  and  soon  after  to  fill  or  to  alter  a form  where  the  referring  data  should 
be  inserted. 

For  each  document  edited  in  AW  two  types  of  files  are  generated: 

• A file  that  follows  the  standard  SVG  that  can  show  the  graphic  drawing  of  the  workflow  in  any 
application  that  interprets  standard  SVG. 

• A XML  file  that  follows  the  DTD  of  Workflow,  this  file  will  have  only  information  related  with  the 
Workflow,  no  graphic  information  will  be  contained  in  him  although  these  information  be  extracted  of  the 
graphic  distribution  of  the  elements  and  of  its  attributes.  As  this  file  have  all  the  information  of  a workflow,  a 
workflow  machine  can  read,  to  interpret  and  to  set  up  this  workflow. 

Future  works 

The  AW  is  just  a part  of  the  project  CEMT,  and  its  use,  evaluation  and  adaptation  to  the  remaining  of 
the  project  are  some  of  the  future  important  works  related  to  this  editor.  Concisely  the  project  CEMT  tell  that:  A 
specialist  in  workflow  should  project  in  AW  a workflow  for  the  authoring  of  a course,  the  files  XML  generated 
by  editor  AW  about  this  project  will  be  sent  to  a workflow  machine  that  will  interpret  the  files  and  with  base  in 
this  files  will  build,  will  execute  and  will  coordinate  a cooperative  environment  for  the  authoring  of  a course  in 
the  Web,  and  the  content  of  this  course  can  be  multimedia,  hypermedia  resources  like  SMIL.  In  this  context  the 
AW  would  serve  as  a tool  that  would  generate  a file  XML  with  instructions  for  a workflow  machine  in  an  agile 
way,  for  this  the  AW  still  needs  to  be  correctly  adapted  to  the  CEMT  project. 

Conclusions 

The  editor  can  be  considered  with  a resource  that  can  help  in  the  interaction  Human-machine  because 
(1)  of  an  agile  and  efficient  way,  it  help  in  the  construction  and  visualization  of  workflows,  facilitating  the 
interaction  and  human  interpretation  of  the  models  and  (2)  it  generates  automatically  SVG  files  with  the  data 
graphs  of  the  workflow  and  XML  files  with  information  about  the  logic  of  the  workflow  facilitating  the 
interpretation  and  the  processing  of  these  models  for  computer  systems  as  Workflow  Machines. 
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Abstract:  This  paper  reports  on  the  development  of  a peer-to-peer  based  tool  to  share 
and  reuse  learning  objects,  by  making  it  easy  to  publish  and  search  over  their  metadata 
[5].  The  goal  of  this  work  is  to  leverage  the  popularity  of  file  sharing  applications  such  as 
gnutella  [3]  and  napster  [4]  in  the  context  of  learning  content. 

1.  Introduction 

The  evolution  in  peer-to-peer  (P2P)  networking  [9],  inspired  us  to  examine  the  possibilities  of  P2P  in  the 
scope  of  share  and  reuse  of  learning  objects.  We  report  on  the  first  outcomes  of  this  work  hae.  There  are 
many  infrastructures  that  rely  on  LOM,  most  notably  ARIADNE  [6,8]. 

2.  Peer-to-peer 

Peer-to-peer  networks  are  networks  in  which  every  computer  performs  the  same  task.  This  approach  can  be 
contrasted  with  a client/server  based  architecture,  where  the  server  provides  dedicated  services  and  clients 
access  the  services  offered  by  the  server  to  implement  a meaningful  interaction  with  the  end  user.  In  a peer- 
to-peer  environment,  every  peer  publishes  its  own  information.  When  a peer  goes  offline,  only  the 
information  stored  in  that  peer  becomes  unavailable.  When  performing  a query,  a peer  queries  its  own 
information  and  sends  the  query  to  other  peers,  which  perform  the  same  task.  The  partial  results  from  the 
different  peers  are  merged  and  the  result  of  the  query  is  returned  to  the  sender.  In  a peer-to-peer  network, 
all  hosts  interact  as  equals.  There  is  no  need  for  a central  server  which  needs  a lot  of  maintenance  costs.  As 
a consequence,  a great  advantage  of  a P2P  approach  is  that  the  system  becomes  much  more  robust.  When  a 
peer  gets  disconnected  from  the  network,  only  the  information  it  manages  itself  becomes  unavailable. 

3.  LOMster 

LOMster  is  a project  that  adresses  sharing  of  learning  objects  on  a P2P  base  with  the  following 
functionalities. 

Publishing 

1.  First  of  all,  a user  can  select  the  learning  objects  he  wants  to  share  with  other  users,  by  dragging  and 
dropping.  The  idea  is  that  users  may  find  this  a more  natural  way  to  make  their  material  available  for 
share  and  reuse  than  uploading  it  to  a central  server. 

2.  When  the  user  adds  a file  to  the  system,  the  tool  should  generate  as  much  metadata  as  possible  [10].  In 
our  current  prototype,  only  file  size  and  file  name  are  taken  into  account.  The  user  can  add  additional 
metadata  manually. 

Searching 

1.  A user  can  formulate  boolean  combinations  of  search  criteria  over  LOM  fields.  LOMster  sends  its 
query  to  all  other  known  peers.  Replies  from  peers  include  the  full  LOM  description  of  the  results. 
Relevant  learning  objects  can  then  be  downloaded  in  the  next  phase. 

2.  Finally,  a user  can  inspect  the  state  of  the  current  downloads  and  uploads. 

To  store  the  metadata  on  a peer,  we  could  not  use  an  RDBMS  as  that  would  impose  too  heavy  a load  on  the 
typically  rather  simple  user  stations  that  LOMster  is  intended  to  run  on.  Rather  we  store  all  metadata  in  an 
XML  file.  To  implement  LOMster,  we  relied  on  the 
JXTA  [2]  P2P  Framework.  LOMster  search-messages 
in  JXTA  have  3 fields  (fig.  1)  : 

1.  In  the  peerid-field  a search-message  indicates  to 
which  peer  the  answer  needs  to  be  sent. 

2.  The  query  identifier  distinguishes  different 
queries  from  the  same  peer. 

3.  Finally,  the  query-field  contains  the  search 


<search> 

<peer±d>peer-id</peerid> 
<qid>query-id</q±d> 
<query>query< /query > 
<search> 

Figure  1:  structure  of  search -message 
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conditions  expressed  in  XSLT. 

When  another  LOMster  peer  receives  the  search 
message,  it  processes  the  query.  For  each  relevant 
learning  object,  it  generates  an  answer-message 
structured  as  in  fig.  2. 

1.  The  first  field  contains  the  peer -identifier  of  the 
peer  that  generated  the  answer. 

2.  The  qid-field  defines  the  query  being  answered. 

3.  The  last  field  contains  the  metadata  of  the 
relevant  learning  objects. 

4.  Related  Work 

The  Edutella  project  also  addresses  the  share  and 
reuse  of  learning  objects.  This  open  source  project  also  uses  the  JXTA  framework.  Instead  of  an  XML 
binding,  Edutella  relies  on  an  RDF  binding  of  LOM.  This  project  seems  to  have  a slightly  more  mature 
status  than  LOMster,  as  more  extensive  field  trials  are  already  well  developed. 


<answer> 

<peerid>peer-id</peerid> 

<qid>query-id</qid> 

<lom> 

<general> 

</general> 

</lom>  ' 

</answer> 

Figure  2:  structure  of  answer -message 


5.  Future  Work 

As  we  mentioned  above,  users  often  find  it  cumbersome  to  insert  new  material  in  a client-server  based 
repository.  We  are  exploring  whether  LOMsta:-like  approach  may  help  in  overcoming  this  problem: 
contributing  new  material  now  simply  entails  dropping  the  relevant  files  in  the  shared  directory.  Although 
this  approach  may  lower  the  effort  for  making  new  material  available,  it  also  results  in  rather  poor  and 
limited  metadata,  unless  the  user  still  spends  considerable  effort  on  the  detailed  description  of  the  learning 
objects  involved.  One  way  to  tackle 
this  problem  is  the  (semi-)automatic 
generation  of  learning  object 
metadata.  We  are  currently 
investigating  how  such  techniques 
can  be  integrated  in  LOMster,  as 
well  as  in  our  more  conventional 
client  server  based  metadata 
environment  [10]. 
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Figure  3:  Screen  Shot  of  LOMster 
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Mental  Models:  Do  they  exist,  how  are  they  are  taught,  can  they  be  measured?  Preliminary  research  findings  using 
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This  paper  will  be  a synthesis  of  the  author's  research  into  methods  for  teaching  and  measuring  mental  models.  The 
presentation  will  be  organized  as  answers  to  the  three  most  pressing  questions  about  mental  models. 

Do  mental  models  exist?  Possibly. 

Although  the  "mental  models"  learning  objective  have  been  widely  discussed  in  design  literature,  there  have  been  no 
construct  validation  studies  to  establish  it  as  a distinct  learning  outcome.  Sheehan  and  Tessmer  (2000)  conducted  a 
construct  validity  study  of  the  mental  models  construct  to  determine  if  it  was  distinct  from  related  learning  outcomes  of 
concepts  and  problem  solving.  To  conduct  the  study,  1 12  US  Navy  engineering  students  were  given  five  types  of 
computer-based  tests  following  a minicourse  on  electrical  circuits:  memory  of  definitions  (to  test  for  verbal  information 
learning),  circuit  identification  (concept  learning),  calculation  (rule  learning)  troubleshooting  (mental  models  and 
problem  solving),  and  inference  (mental  models).  A factor  analysis  of  test  results  indicated  that  mental  models  measures 
indicated  there  might  be  a construct  that  is  distinct  from  verbal  information  and  has  concept  learning  as  one  of  its 
subcomponents.  However,  inference  test  results  did  not  indicate  that  the  construct  is  different  from  its  component 
outcome,  problem  solving.  Further  validation  studies  are  needed  to  indicate  that  mental  models  are  in  fact  different  from 
problem  solving. 


How  do  you  teach  mental  models?  Answer  1:  have  students  build  an  expert  system  using  a computer  based 
authoring  system. 

Students  acquire  a mental  model  when  they  learn  the  proper  place  and  relationship  of  its  component  parts.  Consequent 
with  constructionist  and  constructivist  learning  theories,  students  may  best  acquire  a model  by  building  the  model  itself 
This  building  can  be  accomplished  through  two  concurrent  student  learning  activities: 

1)  Arranging  the  system  components  into  their  proper  spatial  or  conceptual  space  within  the  model.  This  is  the  mental 
map  of  the  system.  For  example,  learning  a mental  model  of  a house  by  knowing  the  arrangement  of  the  roof,  roof 
trusses,  walls,  subfloor,  etc. 

2)  Indicating  the  relationship  between  the  model  components.  Knowing  how  one  component  interacts  with  other 
components  is  what  distinguishes  mental  model  learning  from  mental  map  learning.  For  example,  knowing  that  the 
roof  trusses  support  the  roof. 

One  way  to  enable  these  two  learning  activities  is  to  have  students  construct  an  expert  system  of  the  system  to  be 
learned,  to  have  them  program  the  components  and  relations  while  building  an  expert  system  advisor.  A recent  study  by 
Mason  and  Tessmer  (2000)  indicated  that  this  mental  models  strategy  might  be  both  effective  and  engaging. 

Study  Description.  In  an  experiment  conducted  by  Mason  and  Tessmer  (2000)  students  used  an  online  expert  system  to 
develop  an  expert  system  advisor  on  a brake  system.  Students  had  to  input  the  relations  between  brake  system 
components  and  their  relationship  to  system  malfunctions.  The  learning  assumption  behind  this  experiment  was  that 
students  might  construct  an  effective  mental  model  of  a system  if  they  have  to  build  the  system  itself.  This  model 
building  not  only  requires  that  they  put  the  system  components  in  their  proper  relationship  to  one  another  (a  necessary 
but  not  sufficient  condition  for  learning  a mental  model)  but  that  they  also  describe  the  functional  relationship  between 
the  components:  how  they  affect  one  another  (another  necessary  condition). 

Method.  28  adults  were  recruited  to  learn  about  automobile  brake  systems.  Students'  pretests  indicated  little  or  no  brake 
system  knowledge.  Students  had  their  brake  system  knowledge  tested  before  they  read  a two -page  text  about  the  brake 
system,  adapted  from  the  World  Book  Encyclopedia.  They  were  tested  immediately  afterward,  using  troubleshooting 
and  inference  questions  as  measures  of  mental  models  learning.  Following  the  test  students  had  to  build  an  expert 


system  of  the  brake  system,  one  that  would  advise  users  on  brake  system  problems.  Students  were  tested  again  after  the 
expert  system  task,  using  more  troubleshooting  and  inference  measures.  Students  also  answered  an  email  questionnaire 
about  their  learning  experience. 

Results.  Students  exhibited  no  significant  mental  models  learning  gains  after  they  read  the  brake  system  text  but 
indicated  significant  and  substantial  learning  gains  after  they  completed  the  expert  system  task.  Moreover,  students 
indicated  that  the  expert  system  task  was  engaging  and  enjoyable,  even  though  they  had  never  constructed  an  expert 
system.  The  results  indicate  that  expert  systems  are  worth  consideration  as  mental  models  learning  strategy  for  adult 
learners. 

How  do  you  teach  mental  models?  Answer  2:  have  students  build  an  expert  system  by  assembling  a graphic 
representation  of  one  using  a CBT  system. 

Expert  systems  have  students  learn  a model  by  helping  system  users  troubleshoot  the  problems  the  system  may  have.  As 
such  the  model  activity  focuses  upon  learning  the  rules  for  system  component  interactions.  Another  strategy  is  to  have 
students  build  a complete  visual  model  of  the  system,  arranging  model  components  and  their  functional  relations  into  a 
"big  picture"  model  of  the  system  as  a whole.  Such  graphic  representations  can  be  done  for  abstract  systems  (e.g.,  the 
stock  market,  how  a bill  becomes  a law)  as  well  as  concrete  ones  (brake  systems,  houses). 

Method . In  this  study  a CBT  construction  activity  was  developed  using  Macromedia's  Authorware  multimedia 
authoring  system.  Using  such  a CBT  system  enables  a learner  to  pull  model  components  and  relations  into  place  and  to 
receive  contextual  feedback  when  they  are  put  into  the  correct  and  incorrect  locations.  The  model  constructed  was  about 
the  internet;  how  messages  are  received  and  transmitted.  Students  pulled  internet  components  into  their  locations  on  the 
screen  (router,  server,  home  computer)  and  pulled  over  functional  statements  about  the  relations  between  the 
components  (e.g.,  "directs  message  to  the  proper  server").  Pulling  the  wrong  component  into^place  resulted  in  feedback 
cues  ("no,  this  is  where  messages  are  sorted").  Students  were  then  tested  on  internet  system  troubleshooting. 

Results.  At  present  only  pilot  test  groups  of  1 2 or  less  have  been  used  to  test  the  system,  so  there  are  no  statistically 
significant  results  to  speak  of  However,  preliminary  test  results  (Bland  and  Tessmer,  1999)  have  indicated  that  students 
find  the  system  building  activity  engaging  and  profitable.  In  this  presentation  the  Authorware  model-building  learning 
activity  will  be  demonstrated. 

How  to  you  measure  mental  models  learning?  Multiple  measures  are  advisable. 

f 

As  a relatively  new  and  unproven  construct  (Sheehan  and  Tessmer,  2000),  there  is  still  some  debate  about  the  proper 
ways  to  assess  it.  One  of  the  more  popular  methods  has  been  through  the  use  of  modeling  software,  specifically  PCKnot 
and  KnotMac  (Schvaneveldt,  et  al,  1993).  This  software  builds  a model  of  a student's  conceptual  map  and  compares  it  to 
the  conceptual  map  of  one  or  more  experts,  providing  a numerical  measure  of  the  strength  of  correlation.  Tessmer  and 
Perrin  (1998)  tested  the  reliability  of  this  measure  and  found  it  to  be  questionable,  while  other  popular  measures 
(troubleshooting,  inference)  remain  unexamined.  The  current  prescription  is  to  use  multiple  mental  models  measures  at 
the  same  time,  not  only  to  achieve  a more  accurate  measure  of  mental  model  learning  but  also  to  assess  the  soundness  of 
the  measures  themselves. 

In  this  presentation  a five-page  minipaper  will  be  distributed,  complete  with  research  results  and  references.  Each  type 
of  computer-based  strategy  and  measure  (the  expert  system,  the  Authorware  system,  the  online  concept-mapping  test) 
will  be  demonstrated. 
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Abstract:  This  paper  documents  key  issues  in  electronic  mediated  visual  representations  of  social  identity, 
based  on  the  teacher’s  psychological  capital  in  computer  mediated  communication  (CMC).  Psychological 
capital  is  beliefs,  perceptions,  attitudes  and  thoughts.  The  origin  of  psychological  capital  is  related  to  cultural 
consciousness.  Cultural  consciousness  is  defined  as  the  part  of  an  individual,  which  is  derived  from  the 
amount  of  transmitted  cultural  mores,  values,  ethics,  worldviews  and  ways  of  being  that  is  expressed  in  the 
behaviors  of  everyday  practices.  In  short  psychological  capital  and  cultural  consciousness  is  how  teachers 
talk  about  classroom  teaching  and  learning  in  CMC  This  paper  further  sought  to  capture  ways  which 
teachers  talked  about  their  communities  of  practice  in  their  classroom  environments. 


Introduction 

What  is  identity?  What  does  it  mean  to  document  keys  issues  in  electronic  mediated  visual  representations 
of  social  identity?  This  paper  will  define  visual  representations,  psychological  capital  and  cultural  consciousness. 

This  paper  introduces  a new  model  for  analyzing  teachers’  conversations  in  computer  mediated 
communication  (CMC)  environments.  This  conceptual  paradigm  seeks  to  capture  ways  in  which  teachers  talk  about 
their  communities  of  practice  in  their  intercultural  and  cross-cultural  classroom  environments  by  looking  through  the 
lenses  of  social  identity.  Social  identity  is  “a  combination  of  those  social  categories  which  defined  his  or  her  place  in 
society  and  which  had  been  internalized  to  define  the  self,  together  with  their  value  and  emotional  significance 
attached  to  that  membership”  (Tajfel,  1982,  p.255).  Specifically,  my  goal  is  not  to  capture  the  actual  events  of 
teacher  behaviors  in  a teacher’s  classroom,  but  to  capture  ways  in  which  the  teachers  talk  about  their  identities  (this 
includes  behaviors,  as  well  as  other  aspects  of  identity),  and  how  their  identities  can  emerge  from  the  actual  talk  that 
occur  in  CMC.  By  using  CMC,  the  goal  is  to  unmask  psychological  capital  to  arrive  at  a better  understanding  of 
cultural  consciousness  that  is  embedded  in  social  identity.  While  considering  prior  research,  this  model  seeks  to 
engage  educators  in  dialogue  that  can  allow  them  to  capture  and  construct  textual  images  of  their  psychological 
capital  through  (CMC)  so  that  we  may  better  understand  cultural  consciousness  as  it  relates  to  teaching  and  learning 
in  intercultural,  multicultural  and  cross  cultural  classrooms. 

Psychological  capital  and  cultural  consciousness  is  different  from  Bourdieu’s  definition  of  social  and 
cultural  capital  (1977;  1986).  In  short,  Bourdieu’s  approach  focused  on  the  distributions  of  power  in  relationships 
within  group  memberships  and  social  networks;  However,  this  conceptual  paradigm  focuses  on  collective  utterances 
from  individuals  (teachers)  that  are  grounded  in  social  historical  meaning  (Bahktin,  1981;  1986). 

My  goal  is  to  create  the  constructs,  psychological  capital  and  cultural  consciousness.  I sought  to  develop 
new  conceptual  tools  that  could  better  define  teacher’s  thoughts,  perceptions,  beliefs  and  attitudes  that  are  situated  in 
their  social  identities.  I use  the  construct  psychological  capital  to  refer  to  the  collective  of  the  later.  Cultural 
consciousness  defined,  is  the  part  of  an  individual,  which  is  derived  from  the  amount  of  transmitted  cultural  mores, 
values,  ethics,  worldviews  and  ways  of  being  that  are  expressed  in  the  behaviors  of  everyday  practices  based  on 
social  identity.  The  construct  psychological  capital  is  related  to  cultural  consciousness.  Psychological  capital  is  how 
teachers  think,  believe,  act,  know,  and  cultural  consciousness  is  what  teachers  do  when  they  interact  with  themselves 
and  others  socially. 

Social  science  is  influenced  by  society’s  values  and  ideologies,  by  the  social  and  cultural  background  of  its 
practitioners  (Turner,  1996).  The  practitioners,  in  this  case,  are  teachers  and  other  educators.  Some  important 
questions  that  arise  are:  What  do  teachers  think  when  interacting  with  students  from  different  groups?  How  do  they 
feel  when  they  experience  intercultural,  cross  cultural  and  multicultural  classrooms?  What  do  teachers  experience 
when  they  enter  the  door  of  a classroom  of  30  students  from  Hispanic  cultures?  A teacher’s  psychological  capital 
may  be  limited  for  Hispanic  groups.  If  their  psychological  capital  is  limited,  then  what  happens  within  their  cultural 
consciousness  as  they  approach  the  students? 


Conclusions 

In  conclusion,  the  constructs  of  psychological  capital,  social  identity  and  cultural  consciousness  embedded 
in  electronic  visual  representations  of  voice  are  new  and  recent  phenomena  (Kress,  1998).  Visual  representations  are 
very  important  for  teachers  because  the  visual/textual  representations  represent  voice  of  voices  that  are  associated 
with  experiences,  memories,  attitudes,  beliefs  and  practices  in  an  IntemetAVWW  environment.  This  conceptual 
paradigm  seeks  to  further  to  capture  ways  in  which  teachers  talk  about  their  communities  of  practice  in  their 
multicultural,  intercultural  and  cross-cultural  classroom  environments.  The  constructs  of  psychological  capital, 
dialogue,  speech  genres  and  social  identity  embedded  in  an  online  or  web  based  environment  through  CMC  can 
result  in  what  we  refer  to  as  electronically  mediated  cultural  consciousness.  If  the  visual  text  as  a mode  of 
representation  is  systemic,  rule  governed,  as  well  as  simultaneously  an  effect  of  the  values  of  the  culture  and  social 
identity  in  which  it  is  used,  then  it  may  be  possible  for  teachers  and  other  educators  to  know  and  make  meaning  of 
their  own  and  other’s  social  identity  by  visual  representations  (text)  to  construct  images  of  their  multiple  voices  in 
electronically  online  mediated  environments  (Kress,  1998). 

This  conceptual  paradigm  seeks  to  provide  a means  for  teachers  from  diverse  backgrounds,  where  they  can 
engage  in  communities  of  dialogue,  using  CMC,  regarding  their  social  identity  and  about  their  experiences  teaching 
in  a cross-cultural  or  multicultural  classroom.  By  examining  social  identity  and  intergroup  dialogue  of  teachers,  I 
hope  to  add  to  the  current  research  a better  understanding  of  teachers’  social  identity,  psychological  capital  and 
cultural  consciousness  towards  intergroup  relations.  CMC  structured  forums  will  allow  teachers  to  structure  their 
inquiries  where  they  may  feel  comfortable.  It  will  enable  them  to  dialogue  in  inter-cultural  virtual  communities  so 
that  they  can  engage  in  discussions  that  may  assist  with  modifying  their  psychological  capital.  Through  their  own 
unmasking  of  their  identities  and  psychological  development  they  could  be  better  prepared  to  teach  children  from 
diverse  backgrounds  and  interact  with  families  from  diverse  backgrounds. 

Computer  mediated  communication  (CMC)  empowers  teachers  to  engage  in  conversations  about  their 
teaching  philosophies  and  practice.  It  allows  educators  to  better  understand  how  talk,  based  on  psychological  capital, 
is  culturally  embedded.  It  also  allows  freedom  of  voice  using  textual  representations,  to  facilitate  volatile  issues 
teachers  face  in  intercultural,  cross  cultural  classroom  environments  and  multicultural  environments 

CMC  provides  a means  to  express  multiple  voices  and  notions  of  self  Many  times  when  we  experience 
courses  and  others  in  formal  academic  environments  our  multiple  voices  and  identities  are  suppressed.  This  can  be 
self-imposed  or  directly  imposed  by  others.  Our  psychological  capital  and  cultural  consciousness  are  freed  through 
binary  systems  of  matter  (wires,  numbers,  etc.).  In  many  cases  we  have  to  step  outside  the  community  to  view  the 
other  self 
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Abstract:  The  authors  have  designed  and  developed  a theory -based  chat  system.  The  text -based  chat 
system  is  designed  to  reduce  the  cognitive  load  experienced  by  students  in  text  -based  chat  rooms  that 
are  used  for  instructional  purposes.  This  brief  paper  describes  the  major  features  of  the  chat  system. 


Purpose  of  the  Paper 

The  confusion  and  overlapping  conversations  that  occur  in  text -based  chat  rooms  (Thirunarayanan,  2000;  and 
Thirunarayanan  and  Perez-Prado,  2001)  are  two  of  several  factors  that  contribute  to  students’  “cognitive  load” 
in  chat  rooms.  The  purpose  of  this  paper  is  to  provide  an  overview  of  a chat  system  that  is  designed  to  reduce 
the  cognitive  load  that  students  experience  in  text  -based  chat  rooms. 


A Brief  Definition  of  Cognitive  Load 

According  to  Cooper  (1998),  “Cognitive  load  refers  to  the  total  amount  of  mental  activity  imposed  on  working 
memory  at  an  instance  in  time.”  He  (Cooper,  1998)  goes  on  to  state  “The  major  factor  that  contributes  to 
cognitive  load  is  the  number  of  elements  that  need  to  be  attended  to.” 

In  text -based  chat  rooms,  there  are  many  things  that  compete  for  students’  attention.  Messages  from  different 
sources,  the  quantity  of  textual  information  appearing  and  scrolling  off  the  chat  window,  and  the  mental  energy 
expended  in  composing  their  own  messages,  are  some  of  the  many  factors  that  increase  the  cognitive  load 
experienced  by  students. 


Major  Features  of  the  Instructor  Facilitated  Instructional  Chat  System  (IFICS) 

A theory -based  chat  system  named  “Instructor  Facilitated  Instructional  Chat  System”  (IFICS)  has  been 
developed  to  reduce  students’  cognitive  load.  By  enabling  the  instructor  to  act  as  a facilitator,  the  chat  system 
reduces  the  confusion  and  overlap  that  students  experience  in  many  real-time  text  -based  chat  systems. 
Instructors  who  log  into  the  chat  system  see  two  windows  on  their  monitor,  unlike  students  who  are  presented 
with  only  one  window.  When  students  type  and  submit  their  comments,  their  comments  first  appear  on  a part  of 
one  of  the  instructor’s  windows  and  als  o in  a part  of  the  window  that  is  visible  only  to  the  student  who  typed 
the  comment.  The  instructor  can  read  all  the  comments  typed  and  submitted  by  the  students,  and  can  select  and 
send  appropriate  comments  that  then  appear  in  a part  of  the  window  that  is  visible  to  all  students. 

The  instructor  can  facilitate  discussions  by  first  reading,  selecting  and  then  sending  appropriate  comments  that 
are  submitted  by  students,  to  the  chat  area  that  is  visible  to  all  students.  The  instructor  can  also  send  comments 
to  all  students  and  these  appear  in  a separate  part  of  the  window  that  is  visible  to  all  students. 


The  instructor  has  the  following  four  choices  when  it  comes  to  dealing  with  the  comments  submitted  by 
students  as  they  discuss  course  content  with  each  other: 

1.  Approve  the  comments  made  by  one  or  more  students  after  selecting  them. 

2.  Disapprove  a student’s  comment  after  selecting  the  same. 

3.  Disapprove  a selected  comment  and  also  provide  a reason  to  the  student  for  not  selecting  the  comment. 

4.  Ignore  the  comment  for  the  time  being  and  send  it  to  the  end  of  the  list  of  comments  that  have  been 
submitted  by  students.  Such  comments  can  be  considered  again  at  a later  time. 

Another  major  feature  is  one  that  gives  instructors  the  opportunity  to  type  and  save  up  to  four  comments  before 
a chat  session  begins.  For  example,  one  of  these  comments  could  be  “Consider  the  topic  under  discussion  from 
another  point  of  view”  or  “What  are  some  other  perspectives  that  can  shed  light  on  the  topic  being  discussed?”. 
When  the  situation  warrants,  the  instructor  can,  at  the  click  of  the  mouse,  select  and  send  one  of  these  four  pre- 
saved comments  to  a part  of  the  window  that  is  visible  to  all  students.  This  is  another  way  that  the  IFICS 
enables  instructors  to  facilitate  discussions  in  text -based  chat  rooms. 


Conclusions 

Separating  the  window  that  is  visible  to  all  students  into  four  parts  enables  students  to  focus  on  the  content 
without  having  to  try  to  mentally  separate  the  content  from  the  instructions  and  other  messages  that  are 
extraneous  to  the  content  being  discussed.  The  initial  development  of  IFICS  is  complete  and  the  system  is 
currently  being  field-tested.  Research  will  soon  begin  to  determine  if  the  system  indeed  accomplishes  the 
purpose  it  was  designed  for,  namely  reducing  students’  cognitive  load  in  text  -based  chat  rooms. 
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Abstract 

Previous  research  into  the  behaviour  of  students  while  learning  to  program  by  automatically  recording  their  actions  has 
revealed  that  such  recordings  contains  a wealth  of  information  that  can  be  collected  together  into  a diagnostic  tool  that 
can  support  students’  learning.  The  first  step  has  been  to  construct  the  Coach,  a software  component  that  can  be  invoked 
on  demand  to  provide  a variety  of  support  based  on  students’  previous  experiences.  The  Coach  has  undergone  an  initial 
stage  of  usability  and  usefulness  testing  to  determine  its  effectiveness  in  practice.  This  paper  describes  the  design  of  the 
Coach  and  reports  on  a small  -scale  experiment  to  assess  its  effectiveness  with  two  groups  of  students.  It  was  found  that 
students  did  indeed  turn  to  the  Coach  for  help  and  that  the  control  group  also  searched  for  help,  but  had  to  get  it 
elsewhere.  The  paper  also  reports  on  other  differences  in  behaviour  between  the  two  groups. 


Introduction 

Students  on  the  Open  University  introductory  course  in  computing  M206,  Computing:  An  Object  Oriented  Approach 
(M206  2000)  are  taught  the  00  paradigm  using  Smalltalk.  The  students,  of  which  there  are  approximately  5000  per 
presentation,  are  taught  at  a distance.  The  teaching  is  supported  by  a series  of  practical  activities  performed  with  the 
LeamingWorks  system  (Goldberg,  Abell  et  al.  1997) . The  LeamingWorks  environment  consists  of  over  30 
LeamingBooks,  each  one  of  which  contains  a set  of  practical  activities.  The  structure  of  a LeamingBook  is  based  on  the 
paradigm  of  a book  in  that  the  student  can  read  through  some  pages  of  material  that  describe  practical  activities  which 
have  to  be  carried  out  in  other  pages  of  the  book  (Woodman,  Griffiths  et  al.  1999).  The  early  programming  exercises 
ask  students  to  interact  with  a series  of  micro-worlds  in  which  they  observe  the  effects  of  small  portions  of  Smalltalk 
code;  later  exercises  ask  the  students  to  construct  their  own  code. 


The  left-hand  side  of  Figure  1 shows  the  contents  list  of  LeamingBook  06,  LB -06.  All  LeamingBooks  are  divided  into  a 
sequence  of  sessions.  A session  is  designed  to  be  studied  in  one  continuous  interaction  with  the  computer.  Each  session 
comprises  a sequence  of  practicals,  and  each  practical  has  an  associated  discussion  page.  Thus,  students  are  encouraged 
to  attempt  a practical  and  then  examine  the  discussion  where  the  outcomes  of  the  practical  are  examined. 
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Figure  1 The  interface  of  a typical  LeamingBook 


On  the  right-hand  side  of  Figure  1 is  a page  from  LB-06  shown  detached  from  the  LeamingBook  containing  the 
Amphibians  micro -world.  This  micro- world  is  intended  to  support  the  learning  of  message  sending.  The  Amphibians 
page  also  contains  an  evaluation  text  area  in  which  students  can  enter  small  sections  of  Smalltalk  code  for  the  system  to 
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evaluate.  In  later  LeamingBooks,  the  idea  of  a workspace  is  introduced  where  students  can  enter  significantly  larger 
pieces  of  code  and  investigate  their  execution  (see  Figure  2). 

Given  this  novel  approach  to  the  teaching  of  programming,  we  were  interested  to  know  how  effective  it  vould  be. 
Therefore,  we  set  up  a significant  research  project  (Thomas,  Macgregor  et  al.  1998)  to  investigate  how  students  learn  to 
program  in  this  environment.  Our  first  step  was  to  develop  a student  Observatory  - an  electronic  system  for  recording 
the  actions  that  students  take  when  interacting  with  LeamingBooks  (AESOP  2001).  An  analysis  of  the  recordings 
(Thomas  and  Paine  2000)  showed  a variety  of  student  behaviours,  particularly  when  dealing  with  errors  (Logan  and 
Thomas  2001).  Therefore,  we  set  about  adapting  the  observatory  software  to  provide  additional  support  for  error 
message  comprehension  and  error  correction  - the  Coach  (Paine  2001)  This  approach  opens  up  the  question  of  how 
effective  the  Coach  would  be.  There  are  two  aspects  to  this  question.  First,  how  easy  would  students  find  the  system  to 
use  and  second,  how  useful  would  students  find  the  system?  The  effectiveness  of  the  Coach  would  be  ascertained  by 
examining  the  extent  to  which  students  used  it  to  solve  problems.  In  this  paper  we  shall  discuss  the  effectiveness  of  the 
Coach;  the  usability  issues  are  discussed  in  (Thomas,  Paine  et  al.  2000) . 


The  Coach 


Whenever  the  LeamingWorks  system  detects  an  error  in  an  item  of  Smalltalk  code,  it  issues  an  error  report.  Figures  2 
and  3 show  two  kinds  of  report.  In  Figure  2,  the  error  report  is  shown  highlighted.  To  obtain  help  with  this  error  the 
student  can  invoke  the  Coach  by  clicking  on  the  appropriate  button.  In  Figure  3,  the  report  appears  in  a modal  dialog 
box  that  would  normally  be  cleared  by  clicking  on  the  OK  button.  However,  in  the  modified  system,  the  student  has  the 
option  of  invoking  the  Coach.  The  result  of  clicking  on  the  Coach  button  is  a new  window  similar  to  the  one  shown  in 
Figure  4. 
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Figure  2 The  LeamingBook  interface 


Figure  3 A typical  error  report 


The  prototype  Coach  window  has  two  main  areas.  At  the  top  of  the  window  is  a tabbed  area  labelled  ‘Actions’  which 
enables  the  student  to  scroll  through  a history  of  their  actions.  This  uses  the  Observatory’s  recording  of  the  student’s 
actions  throughout  the  LeamingBook.  The  error  report  is  shown  highlighted  and,  in  Figure  4,  is  a textual  representation 
of  the  contents  of  a dialog  box  similar  to  the  one  shown  in  Figure  3 (the  DIALOG  is  the  error  report  and  the  CHOICES 
are  the  buttons  that  appear  in  the  box).  The  second  tabbed  area  that  occupies  most  of  the  window  shows  a series  of 
hints,  the  first  of  which  is  headed  MEANING  and  contains  an  expanded  explanation  of  the  error  that  has  been  detected. 
The  remaining  items  on  the  Hints  page  are  possible  reasons,  given  contextually,  for  the  occurrence  of  the  error.  The 
hints  contain  hyperlinks  to  the  Glossary  containing  definitions  of  the  terms  used  in  the  descriptions.  The  remaining  tabs 
give  access  to  possibly  useful  materials  such  as  the  main  teaching  texts  (Chapters),  links  to  related  web  sites  (Links)  and 
a graphical  Smalltalk  syntax  analyser  (Precedence). 
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Figure  4 The  Coach  interface 

Effectiveness  experiment 

In  a small  experiment  designed  primarily  to  investigate  usability  issues,  we  took  the  opportunity  to  study  the 
effectiveness  of  the  Coach.  14  student  volunteers  were  provided  with  two  additional  LeamingBooks,  each  containing  a 
small  number  of  practical  exercises  related  to  the  work  of  LeamingBooks  09  and  1 0.  The  practicals  asked  the  student  to 
evaluate  a number  of  simple  expressions,  each  of  which  resulted  in  an  error,  and  to  attempt  to  rectify  the  errors.  Those 
with  the  Coach  were  told  of  its  purpose,  but  were  not  required  to  use  it  to  solve  the  problems. 

The  experiment  used  an  independent  samples  design  to  compare  the  actions  of  students  who  used  the  Coach  with  those 
who  did  not  (Siemer  and  Angelides  1998;  Budgen  and  Pohthong  1999).  Students  were  divided  into  two  groups, 
providing  us  with  2 conditions.  In  condition  1,  students  were  provided  with  the  additional  LeamingBooks,  the  AESOP 
Recorder  and  Coach  software.  In  condition  2,  students  were  given  the  additional  LeamingBooks  and  the  AESOP 
Recorder  software  (i.e.,  they  did  not  receive  the  Coach  software).  Students  were  assigned  to  a condition  on  the  basis  of 
a pre -experiment  questionnaire  aimed  at  controlling  variables  related  to  gender  and  age.  This  resulted  in  8 students 
being  assigned  to  condition  1 group  and  6 to  condition  2 group.  Having  completed  the  test  exercises,  the  students  e- 
mailed  their  recording  to  us  for  analysis. 

Post-Questionnaire 

Once  they  had  finished  the  tests,  all  students  were  asked  to  complete  a questionnaire  designed  to  assess  usability  and 
usefulness  issues.  When  asked  to  rate  the  ease  of  completing  the  tests  on  a scale  of  1 (difficult)  to  5 (easy),  on  average 
students  without  the  Coach  rated  the  practicals  as  easier  (4.4)  than  students  with  the  Coach  (3.75).  Generally,  students 
who  used  the  Coach  found  the  amount  of  information  on  its  interface  slightly  distracting.  When  asked  about  the 
likelihood  of  using  the  Coach  in  other  LeamingBooks  on  a scale  of  1 (unlikely)  to  5 (likely),  on  average  they  said  3.25. 

The  following  comment  from  one  student  summarises  experiences  with  the  Coach:  ‘The  information  in  the  Coach  was 
useful,  although  it  was  a little  difficult  to  home  in  on  the  appropriate  comments  for  the  problems  I had.  When  I went 
back  to  using  LeamingWorks  I found  myself  looking  for  it  in  my  next  LB  and  on  a couple  of  occasions  wishing  it  was 
there.  Sometimes  the  Smalltalk  error  messages  are  difficult  to  interpret  and  I think  the  Coach  would  help.” 


Analyses  of  LB  Test-09  Coach  Recordings 

Of  the  8 students  with  the  Coach,  7 used  it  ‘for  real’.  One  student  did  not  open  the  Coach  at  any  point  during  either  test 
LeamingBook.  All  8 students  attempted  all  of  the  practicals.  Figure  5 shows  the  percentage  of  students  who  used  the 
Coach  on  each  practical  activity:  it  only  includes  those  students  who  used  the  Coach  to  help  them  attempt  to  solve  the 
practical  i.e.  not  those  who  simply  opened  the  Coach  to  look  at  it.  It  shows  that,  for  each  practical,  some  students  felt 
the  need  to  use  the  Coach. 
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Figure  5 The  percentage  of  students  who  used  the  Coach  in  each  practical 

With  the  exception  of  1 student  in  one  practical,  all  students  with  the  Coach  who  solved  the  problem  in  a practical  did 
so  without  looking  at  the  discussion  pages.  Ho\\ever  those  students  without  the  Coach  who  solved  a practical 
sometimes  looked  at  the  discussion  page  before  the  solving  the  problem,  as  shown  in  Table  1 . A few  students  without 
the  Coach  software  also  looked  at  the  discussion  page  but  did  not  solve  the  practical.  We  concluded  that  some  students 
in  each  group  needed  additional  support  to  solve  the  problems. 


No.  of  students  who; 

Practical 

Solved  practical 

Looked  at  Discussion  page 

1 

6 

1 

2 

6 

0 

3 

5 

1 

5 

5 

2 

8 

6 

2 

10 

6 

1 

11 

6 

5 

12 

5 

0 

Table  1 : Students  without  the  Coach  who  looked  at  the  discussion  before  solving  each  practical 

Figure  6 shows  the  percentage  of  students  who  correctly  solved  the  problems  that  had  a single  specific  solution.  There 
were  students  in  each  group  who  failed  to  solve  some  problems.  In  three  out  of  these  four  cases,  more  students  without 
the  Coach  succeeded  in  solving  the  problems.  Nevertheless,  in  practicals  1,  8 and  1 1,  some  students  without  the  Coach 
tried  the  practicals  only  after  having  read  the  discussion,  so  one  might  conjecture  that  not  all  students  in  this  group 
would  have  successfully  completed  these  exercises.  However,  the  results  of  the  post-questionnaire  indicates  that  the 
group  with  the  Coach  were  weaker.  This  is  confirmed  by  the  number  of  student  errors  (other  than  those  mandated  by  the 
practical  activities)  in  which  those  without  the  Coach  made  on  average  18.83  errors  each,  whereas  those  with  the  Coach 
made  23  errors  each  on  average. 


Figure  6 The  percentage  of  students  with  and  without  the  Coach  who  solved  each  problem 


A summary  of  the  data  extracted  from  the  recordings  for  LB  Test-09  from  students  with  and  without  the  Coach  software 
is  shown  in  Table  2. 


Average  number  of: 
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48  mins 

2.63 

9.75 

39.25 

35.75 

35.00 

Without 

20  mins 

1,33 

4.50 

22.50 

32.83 

30.83 

Table  2 A summary  of  LB  Test-09  recordings. 

From  Table  2 it  can  be  seen  that  students  with  the  Coach  spent,  on  average,  over  twice  as  long  in  LeamingBook  Test-09 
than  students  without  the  Coach. 

Students  with  the  Coach  appear  to  do  LB  Test-09  in  more  sittings  that  student  without  the  Coach  (shown  by  the  higher 
number  of  ‘Open’  events).  Students  with  the  Coach  also  on  average  accessed  more  hyperlinks  than  students  without  the 
Coach.  This  is  to  be  expected  as  students  with  the  Coach  have  access  to  the  Coach  Links,  Chapters  and  Glossary  pages 
which  all  contain  a number  of  hyperlinks. 

We  analysed  the  results  of  the  second  test  (LB  Test- 10)  in  a similar  way.  The  difference  between  the  two  groups  of 
students  was  less  marked  particularly  in  the  amount  of  time  spent  solving  the  problems.  This  is  to  be  expected  as 
students  get  used  to  using  the  Coach  software.  This  gives  us  confidence  that  using  the  Coach  need  not  be  a significant 
overhead,  especially  when  it  is  clear  that  some  students  look  for  additional  support  to  solve  some  problems. 

Future  work 

We  have  implemented  a revised  Coach  interface  and  slightly  amended  the  two  test  LearningBooks  so  that  we  can  repeat 
this  experiment  during  the  2002  presentation  of  the  course  but  with  a much  larger  sample  of  students. 

The  next  major  step  in  the  development  process  is  to  utilize  the  data  contained  in  the  student  recordings  to  improve  the 
feedback  given  by  the  Coach  based  on  actual  student  experiences.  Figure  7 shows  the  architecture  of  the  system  as 
envisaged.  The  basic  idea  is  that  the  Coach  obtains  its  data  from  a database  on  a student’s  machine  (downloaded  with 
the  Coach).  The  local  database  is  augmented  with  additional  data  based  on  the  student’s  interactions  with  the 
LearningBooks  (using  the  Events  Analszer  component).  The  recordings  are  sent  to  a central  recordings  database  as  with 
the  present  system.  The  recordings  will  contain  all  the  events  that  occurred  in  each  LeamingBook,  including  those 
related  to  the  use  of  the  Coach  since  it  is  a LeamingBook,  too.  The  complete  set  of  all  students’  recordings  are  analysed 
to  update  a central  database  of  Coach  data.  Students  will  be  able  to  update  their  local  database  with  revised  Coach  data 
as  and  when  they  wish.  In  this  way  we  hope  to  be  able  to  adapt  the  Coach  in  the  light  of  all  students’  experiences. 


Figure  7 The  Coach  System 

A third  avenue  for  exploration  is  to  follow  up  on  a discussion  of  models  of  intelligent  tutoring  in(Gertner,  A.  & 
VanLehn,  K (2000))  by  using  the  Coach  to  provide  a model  of  how  a given  problem  should  be  solved.  This  will  be 
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based  on  work  in  another  area  [Thomas,  2001]  for  which  we  are  developing  mechanisms  for  specifying  tasks  and 
identifying  attempts  at  solving  them.  We  want  to  see  how  these  mechanisms  might  be  used  to  create  a model  of  how  a 
problem  is  to  be  solved  and  to  detect  student  attempts  at  the  solution.  As  the  student  tries  to  solve  the  problem,  his/her 
actions  are  compared  to  those  that  the  model  would  make.  If  the  student’s  actions  diverge  sufficiently  from  the  model, 
the  Coach  would  offer  the  student  some  advice  or  feedback. 


Conclusions 

Overall,  the  majority  of  students  who  used  the  Coach  found  it  useful  However,  the  prototype  interface  was  found  to  be 
distracting  and  students  found  it  difficult  to  home  in  on  the  appropriate  hint.  We  obtained  useful  feedback  on  the 
interface  and  have  simplified  it.  On  balance,  the  Coach  seems  to  offer  a beneficial  tool  that  some  students  found 
attractive. 

We  believe  that  the  Coach  is  effective  and  that  it  is  worth  investing  further  effort  to  improve  it.  In  particular,  we  can  see 
ways  of  adapting  the  Coach  to  individual  student  needs.  We  also  believe  that  it  will  be  possble  to  provide  further  help 
through  the  idea  of  a model  solution  and  comparing  it  with  student  attempts  at  solving  programming  problems. 
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Abstract:  The  authors  present  two  case  studies  in  the  use  of  computers  in  the  classroom, 
one  involving  an  introductory  computer  science  class,  the  other  an  upper  division 
literature  class.  After  describing  each  case,  the  differences  are  discussed,  showing  that 
pedagogical  models  developed  for  one  discipline  may  not  transfer  to  another,  and  that  the 
discipline  itself,  beyond  instructor’s  preferences  or  institutional  policies,  may  determine 
what  works  and  what  doesn’t. 

Introduction 

Our  goal  in  this  paper  is  twofold.  First,  we’ll  indicate  that  the  successful  use  of  information 
technology  in  the  classroom  depends  more  on  the  synergy  between  teacher,  student  and  the  learning 
context  than  on  the  technology  itself.  The  use  of  technology  — as  the  primary  vehicle  for  teaching 
(distance  learning)  or  as  an  adjunct  to  the  more  traditional  classroom  — is  effective  only  insofar  as  it 
addresses  this  synergy.  Second,  we  will  maintain  that  one  under-appreciated  factor  influencing  teachers, 
students  and  learning  contexts  is  the  discipline  itself,  in  a way  that  goes  beyond  simply  paying  appropriate 
attention  to  learning  (or  teaching)  styles.  We  offer  two  case  studies,  one  drawn  from  an  introductory 
computer  science  course,  the  other  from  an  upper  division  literature  course,  in  support  of  these  claims. 

Enthusiasm  for  online  learning  is  of  course  high  for  students  who  live  in  areas  not  easily  served  by 
the  traditional  classroom,  and  courses  accessible  anytime  and  anyplace  can  provide  valuable  learning 
experiences  for  mature  students  who  work  full  time  and  take  courses  when  schedules  permit.  But  the  use 
of  online  learning  tools  within  traditional  campus-based  course  delivery  structures  has  become  popular  as 
well  [1].  James  Duderstadt,  President  Emeritus  and  Professor  of  Science  and  Engineering  at  the  University 
of  Michigan,  suggests  that  universities  must  themselves  take  a leadership  role  in  remodeling  the 
universities  of  the  21®*  century,  and  that  new  information  and  communication  technology  tools  will  play  a 
key  role  in  that  mission,  suggesting  that  universities  will  continue  to  have  a physical  existence,  whatever 
their  virtual  roles  may  become[2].  This  is  not  a completely  uncontroversial  claim;  in  a New  York  Times 
article,  John  Chambers,  then  Chief  Executive  Officer  for  Cisco  Systems,  asserts  that  "the  next  big  killer 
application  for  the  Internet  is  going  to  be  education.  Education  over  the  Internet  is  going  to  be  so  big  it  is 
going  to  make  e-mail  usage  look  like  a rounding  error.”  [3]. 

As  the  situation  has  developed,  however,  and  as  other  researchers  have  begun  to  look  at  what  now 
seem  to  be  inflated  claims,  online  education  has  emerged  as  one  aspect  of  a larger  picture.  Prosser  and 
Trigwell  [4]  argue  that  both  teachers  and  students  benefit  from  an  increased  awareness  of  their  own 
personal  experiences,  approaches  and  perceptions  of  the  learning  process  and  that  this  awareness  facilitates 
positive  learning  outcomes.  In  particular,  they  emphasize  that  “good  teaching  involves  an  awareness  of 
students’  perceptions  of  teaching  technologies  (including  information  technology)  used  in  teaching”  and 
that  these  perceptions  can  significantly  impact  the  learning  experience  either  positively  or  negatively. 

Given  their  results,  the  successful  use  of  information  technology  in  the  classroom  is  clearly  more  dependent 


on  interactions  between  teachers,  students  and  learning  contexts  than  on  the  technology;  it  may  even 
indicate  that  there  are  some  things  technology  cannot  appropriately  do,  a point  to  which  we  shall  return. 

On  the  other  hand,  learning  opportunities  newly  available  through  the  use  of  technology  can 
support  desired  learning  outcomes — provided  (our  second  claim)  those  outcomes,  as  defined  not  simply  by 
the  instructor  or  the  institution  but  by  the  discipline  itself,  are  congruent  with  what  the  technology  can 
provide. 

Astin’s  comprehensive  study  of  student  development  characteristics  in  various  higher  institutional 
settings  [5]  points  out  that  active  engagement  of  students  in  interdisciplinary  courses,  course  with 
discussions,  debates,  and  class  presentations  strongly  correlates  with  critical  thinking  skills.To  the  extent 
that  technology  can  facilitate  (or  at  least  create  the  opportunity  to  provide)  such  active  learning  strategies, 
the  effort  will  not  only  increase  student  involvement  in  the  course,  but  also  increase  understanding  of 
course  concepts  by  relating  them  to  independent  inquiry  and  debate. 

The  question  is  whether  available  technology  can  actually  do  that. 

Case  Studies  and  Examples 

Our  first  case  study  involves  an  introductory  computer  science  class.  Pope’s  goal  in  implementing 
web-based  tools  the  course  was  to  increase  opportunities  for  communication  and  participation.  The  class 
was  small  (approximately  25),  but  there  was  a considerable  amount  of  material  to  cover  and  the  topics  that 
dominate  the  headlines  - Microsoft  Antitrust  litigation.  Privacy  in  Cyberspace  - provide  fertile  ground  for 
discussion.  He  began  using  WebCT  as  a tool  both  for  distributing  informational  materials  and  for  on-line 
testing.  In  doing  this,  he  relied  on  the  students  to  read  on  their  own  time;  his  discussion/lecture  now 
addresses  related  but  different  concepts  in  supplementing  the  text.  Making  the  quizzes  available  online 
provided  more  opportunity  for  group  and  class  discussion. 

Students  were  positive;  they  could  find  their  grades,  course  syllabus,  assignments  and  topic  notes 
in  one  central  location. 

In  designing  this  kind  of  structure  Pope  was  of  course  not  alone.  The  University  of  Central 
Florida,  for  example,  adopted  a similar  approach  to  improve  its  course  in  American  National  Government. 
There,  the  goals  of  the  restructuring  were  practical  as  well  as  mission-oriented.  The  course  enrolled  over 
2000  students  in  sections  of  80-100  students.  Classroom  space  was  in  critical  short  supply;  increasing  the 
number  of  sections  was  not  a viable  option.  But  the  course  also  had  a retention  problem  and  surveys 
indicated  that  partially  web-based  sections  had  somewhat  higher  retention  rates.  Building  on  this,  the 
department  designed  a web-based  asynchronous  learning  environment  based  on  web-based  modules  to 
encourage  student  participation.  Class  meeting  time  was  reduced  by  two  thirds.  Bruce  Wilson  reports 
“students  are,  by  necessity,  more  actively  involved  in  the  learning  process.  And  instructional  technology 
can  also  enhance  students’  critical  thinking  skills.  ...  The  use  of  the  Internet  in  teaching  Political  Science 
gives  instructors  more  opportunities  to  design  activities  that  involve  students’  direct  participation  and  to 
follow  clearly  set  instructional  goals.”  [6] 

Herman  D.  Lujan  [7]  alludes  to  the  tendency  of  many  faculty  to  “narrowly  define  ‘good’  teaching 
and  learning  as  something  that  occurs  in  a time -bound,  synchronous  classroom  setting.”  Pope  believes  that 
students  in  his  computer  science  course  benefit  not  only  fro  m in-class  - ‘synchronous’-  discussions  and 
demonstrations  but  also  from  the  ability  afforded  by  new  online  tools  to  communicate  and  learn  using  the 
tools  that  promote  asynchronous  learning  opportunities. 

Unlike  Lujan,  however.  Pope  is  not  convinced  online  learning  fits  in  every  teaching  and  learning 
context.  It  fits  in  this  course  because  much  of  the  material  and  the  broad  topics  under  review  can  be 
organized  in  a modular  fashion  and  students  can  easily  take  part  in  discussions  outside  of  the  class  context 
using  WebCT.  Fine  for  an  introductory  class,  not  so  fine  for  an  upper  division,  seminar-style  class  in 
advanced  topics  in  mathematics. 

This  class  meets  twice  a week  - once  in  a regular  classroom  and  once  in  a hands-on  computer  lab. 
But  based  on  his  experience  Pope  has  come  to  believe  that  hands-on  access  is  a more  critical  component  of 
the  instructional  paradigm  than  the  web-based  asynchronous  tools.  This  is  a rather  tame  observation  in 
2002-  the  case  for  instructional  computing  labs  in  science  and  conputing  education  was  made  by  the  mid 
1980s;  nevertheless,  the  classroom  context  heavily  influences  teaching  methodology.  Without  the  lab, 
there’d  hardly  be  time,  and  fewer  topics,  for  discussion!  The  point  is  that  successful  teaching  requires  the 
clear  evaluation  of  the  goals  of  the  course  [derived  from  the  discipline],  the  context  in  which  the  teaching 
and  learning  takes  place  [the  classroom  itself],  and  finally  the  learning  strategies  that  best  fit  both. 
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Finally,  Pope  also  viewed  the  electronic  interface  as  an  opportunity  to  evaluate  his  own  teaching. 

In  any  implementation  of  electronic  technology  in  the  classroom,  a major  evaluation  of  course  objectives 
and  teaching  strategy  is  required,  and  he  found  this  to  be  a welcome  necessity.  Drawbacks  were  a lengthy 
development  and  the  availability  of  trained  support  staff. 

Thurber,  teaching  an  upper  division  class  in  English  literature,  had  a different  experience.  It  was 
not  obvious  to  him  that  the  standard  distributed  education  model  was  appropriate,  given  both  the  mission  of 
the  university  and  his  actual  task,  which  was  to  investigate,  in  this  case,  the  work  of  the  English  poet 
William  Blake.  He  does  not  give  quizzes  as  such,  although  short  exercises  related  to  that  moment's 
discussion  do  take  place;  there  is  no  “lecture”  and  therefore  no  lecture  notes.  The  course  itself,  in  addition, 
was  already  as  “interactive”  as  he  (and  his  24  students)  could  stand.  Instead,  the  goal  was  to  use  the  Web 
to  investigate  the  nature  of  hypermedia,  particularly  as  the  poet  in  question,  Blake,  had  done  an  18  century 
version  of  the  same  thing.  His  goals,  therefore,  were  far  more  specific  to  the  actual  material — more 
contingent,  more  dependent  on  the  actual  poetry  than  on  any  idea  about  how  to  teach  poetry. 

He  created,  therefore,  a course  website  (www.sandiego.eduMhurber/CvberBlake)  and  asked  the 
students  to  create  their  own  hypermedia  websites  in  lieu  of  the  traditional  paper — the  rationale  being,  once 
again,  not  simply  that  hypermedia  may  be  worth  investigating  on  its  own,  which  it  may  be,  but  that,  given 
this  poet’s  practice,  hypermedia  are  an  appropriate,  perhaps  the  most  appropriate,  response.  The  student’s 
response  was  positive,  in  each  case  suggesting  that  the  course  be  given  again.  Typical  remarks  included 
“It’s  about  time  English  Departments  did  this,”  “an  English  course  that  is  actually  practical,”  and  “I  feel 
like  I’m  a writer  too,  doing  something  a little  bit  like  Blake.” 

The  course  model  that  evolved,  however,  has  almost  nothing  in  common  with  Pope’s.  There  was  a 
course  discussion  board;  very  few  students  used  it,  feeling  that  opportunities  for  interaction  were  already 
sufficient;  a few  found  it  intimidating,  while  others  viewed  it  as  just  another  course  assignment. 
(Participation  in  the  discussion  board  was  optional.  Thurber  wanted  to  see  what  would  happen  if  it  was  not 
required.)  There  were  electronic  office  hours;  no  one  ever  showed  up,  as  students  uniformly  felt  either  that 
they  already  had  sufficient  access  to  the  instructor,  or  that  personal  interaction  was  preferable.  The 
emphasis  was  on  the  student’s  ownership  and  exploration  of  an  electronic  medium,  the  Web,  rather  than  on 
using  the  Web  to  enhance  communication  or  provide  additional  course  materials. 

Observations  and  Conclusions 

It  is  about  the  differences  between  these  two  course  structures  that  we  would  like  now  to  reflect. 
Crucial  to  Pope’s  model  was  the  use  of  the  Web  in  the  transferal  of  information  from  the  instructor  to  the 
student.  Indeed,  he  viewed,  as  is  common,  class  sessions  as  adjuncts  or  supplements  to  information 
provided  online. 

But  this  is  already  not  a model  that  transfers  readily  to  an  upper  division  literature  class. 

Advocates  of  distributed  learning  have  traditionally  emphasized  that  the  use  of  electronic  communications 
present  opportunities  for  teacher-student  interaction  that  effectively  shift  the  educational  focus  from 
“teacher-centered”  to  “student -centered,”  away  from  the  traditional  lecture  format  and  towards  distributed 
learning.  But  is  the  “transmission”  of  information,  by  itself,  what  college  courses  are  for?  If  so,  never  mind 
the  traditional  lecture;  colleges  have  been  masquerading  as  libraries  or,  now  that  the  technology  is 
available,  web  sites.  As  far  as  the  humanities  are  concerned  the  “transmission”  of  information  is  only  one 
function  college  courses  serve,  and  in  some  respects  the  least  important. 

While  we  acknowledge  that  modem  educational  philosophy  mandates  the  critical  importance  of 
engaging  the  student  in  interactions  that  will  impact  his  or  her  mastery  of  the  subject  matter,  it  does  not 
then  follow  that  the  hallmark  of  student-centered  learning  is  the  use  of  computers  in  the  classroom. 
Particularly  if,  as  at  our  university,  classes  are  small  and  instructors,  on  the  whole,  couldn’t  lecture  if  they 
wanted  to.  Is  the  give  and  take  in  small,  discussion-centered,  quasi-seminar  situations  comparable  to  what 
we  can  do  in  online  discussion  groups,  even  with  real-time  audio  and  video?  If  it  isn’t,  what  are  the 
differences,  and  what  is  the  educational  impact  of  those  differences?  In  the  absence  of  hard  answers  to 
those  questions,  we  wonder  what’s  really  at  stake.  What  is  being  transmitted,  pre-eminently  in  literature 
classes  but  in  the  humanities  generally,  is  not  the  “information”  we  possess  about,  for  example, 
Shakespeare,  which  is  trivial,  but  the  nature  and  kind  of  conversations  we  have  and  have  had  about  his 
work.  Knowledge  in  the  humanities  is  both  a process  (not  a result)  and  always  contingent,  socially 
constructed  and  crucially  dependent  on  the  context  in  which  it  is  acquired.  (The  French  Revolution  in  the 
eyes  of  post-1848  Europe  was  one  thing;  to  Woodrow  Wilson  it  was  another;  to  ourselves  it  is  yet 


something  different.)  From  this  perspective  the  transmission  of  information  via  the  web  is  a non-sequitur.  It 
isn’t  the  Web  that’s  the  problem;  it’s  the  word  “transmission.” 

Using  computers  according  to  the  first  model,  therefore,  at  least  interferes  with  and  may  even 
negate  the  goals  and  methods  of  the  humanities — not  because  humanities  instructors  are  Luddites  (some 
are),  but  because  the  pedagogical  model  such  approaches  embody  originated  in  one  discipline,  or  set  of 
disciplines,  and  don’t  readily  transfer  to  another.  Using  computers  according  to  Pope’s  model  would  deny 
Thurber  and  his  students  the  chance  to  do  what  they  want  to  do,  which  is  both  to  learn  what  a writer 
actually  did  and  to  forge  a response,  together,  to  what  she  actually  did.  Here  is  where  the  under-appreciated 
difference  between  disciplines—  world  views,  at  some  point—  comes  into  play.  There  is  no,  and  there  had 
better  not  be,  any  such  thing  as  socially  constructed  knowledge  in  the  sciences.  (Actually  this  is  a matter  of 
current  debate.  What  physicists  thought  about  the  significance  of  Maxwell’s  equations  in  1890  was  a 
different  than  what  Einstein  thought  fifteen  years  later.)  In  the  humanities,  on  the  other  hand,  and 
particularly  in  literature,  there  is  no  knowledge  except  what  has  been  socially  constructed  - beginning  with 
the  fact  that  literature  is  made  of  language,  the  most  social  of  all  constructions,  and  including  the  fact  that 
no  writer,  no  matter  how  august,  is  a writer  unless  someone,  somewhere,  chooses  to  read  her.  The  artist  has 
an  intent,  to  be  sure,  but  that  intent  is  only  one  of  many  variables  connected  to  our  mutual  investigation  of 
what  a work  actually  is.  Reader-response  theory,  as  a matter  of  fact,  would  have  us  believe,  in  general,  that 
readers  are  actually  as  responsible  for  what  a work  does  as  the  author  is  — maybe  more  so,  in  some 
constructions.  Whether  that  is  true  or  not,  none  of  us  reads  or  could  read  Shakespeare  as  Shakespeare  did; 
but  we  still  read  Shakespeare!  What’s  that,  then?  Shakespeare  is  Shakespeare  but  he’s  also  us  reading 
Shakespeare,  in  ways  that  he  could  not  have  foreseen  but  which  are,  still,  what  Shakespeare  “is.”  For  now. 
Meanings  change;  there  are  no  “laws”  in  the  sense  that  there  are  for  the  hard  sciences. 

Thus  when  Pope  says,  for  example,  that  it’s  a good  thing  that  online  discussions  can  happen  any 
time,  that  it  frees  people  from  the  constraints  of  time  and  space  so  that  they  can  say  anything  from 
anywhere,  Thurber’s  response  is  — why  is  that  good?  It  would  depend  on  the  crucial  insight  that  online 
discussion  is  the  same  as  or  better  than  the  kinds  of  discussions  his  students  and  he  have  in  real  time,  with 
their  real  bodies  and  their  real  minds  in  a real  place,  zoned  into  a writer  they  want  to  try  to  understand.  Is 
virtual  discussion  discussion?  (We  know,  for  example,  that  people  write  and  talk  differently,  and  that  they 
behave  differently  on-  or  off-camera.  What  are  the  differences,  are  they  significant,  and  are  virtual 
discussions  better  than,  the  same  as,  or  worse  than  virtual  discussions?  And  for  what  ends?  There  has  been 
surprisingly  little  research  in  these  areas,  particularly  as  different  disciplines  are  involved.) 

Even,  Thurber  notices,  Britain’s  Open  University,  one  of  the  oldest  and  most  successful 
implementations  of  computer-based  instruction  (http://www.oPcn.ac.ukL  supplements  online  material  with 
local  study  centers  (and  tutors)  at  learning  centers  around  the  world.  On  this  model,  the  discussion,  always 
specific,  always  local,  and  always  the  joint  product  of  the  persons  present  on  that  occasion,  is  preserved, 
together  with  ancillary  electronic  material  and  the  opportunity,  which  he  welcomes  for  his  classes,  for 
students  to  write  back  at  the  sea  of  electronic  media  they  are  surrounded  by,  owning  the  web  by  helping,  in 
a small  way,  to  create  it. 

Thurber  makes  one  further  point,  however.  Crucial  to  his  thinking  about  the  use  of  computers  in 
the  classroom  is  what  he  has  taken  to  be  the  centrality  of  hypertext  (and  hypermedia)  in  the  classroom.  As 
things  have  developed,  however,  he  is  beginning  to  wonder  whether  hypertext — at  least  as  it  was  once 
envisaged — has  failed.  If  it  has,  the  use  of  computers — any  computers  at  all  — in  his  classroom  becomes 
problematic. 

In  one  sense,  of  course,  it  is  absurd  to  say  that  hypertext  has  failed — the  Web  itself  is  evidence  that 
it  has  not—  not  to  mention  media -rich  computer  programs,  whether  or  not  they  live  on  the  web.  Hypertext, 
and  more  generally  hypermedia,  are  the  cornerstone  upon  which  contemporary  electronic  communications 
stand,  the  cornerstone  as  well  of  the  new  IT  economy,  with  its  multiple  and  still  evolving  political,  social, 
and  psychological  impacts.  Hypermedia  used  creatively,  moreover,  as  a medium  of  expression  with  its  own 
aesthetics,  continues  to  be  produced,  if  in  small  quantities.  It  was  this  last  area — the  possibility  that 
interactive  technologies  could  be  the  means  by  which  student  author-readers  could  create  new  kinds  of 
texts — that  most  interested  and  excited  Thurber. 

But  in  this  context  the  early  promise  of  hypermedia  has  not  been  realized.  We  do  not  go  to  see 
interactive  movies,  though  attempts  have  been  made,  nor,  on  any  meaningful  scale,  do  we  find  ourselves 
reading  interactive  novels.  Hypermedia  on  the  web  and  elsewhere  have  developed  as  economic,  more  than 
literary  or  artistic,  engines.  Interactive  games  are  arguably  interactive  fiction-  but  games  more  than  fiction, 
image  more  than  text;  but  text — language-  is  at  the  heart  of  Thurber’s  commitment  to  his  own  discipline. 


It  is  true  that  interactive  textuality  continues  to  be  investigated  in  the  academy — notably  at  Brovin 
University,  MIT,  the  University  of  Texas  at  Austin — but  these  attempts  too  have  failed  to  generate  impact 
beyond  those  who  are  already  persuaded.  Hypertext  has  not  swept  away  text;  hypertext  fictions  bear 
approximately  the  same  relationship  to  fiction  that  performance  art  bears  to  drama — marginal,  contingent, 
rather  that  the  revolution  many  of  us  thought  was  coming. 

But  whether  or  not  this  is  true,  our  conclusion  is  that  each  of  us  is  still  en  route  to  a full 
understanding  of  the  implications  of  electronic  technologies  for  higher  education.  We  would  emphasize, 
however,  that  our  different  disciplines  seem  to  require  different  choices,  different  ways  of  using  those 
technologies  or  reasons  for  not  using  them.  One  size  does  not  fit  all! 
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Abstract:  Epistemological  beliefs  have  been  shown  to  influence  cognition  and 
learning,  motivation,  persistence  in  problem  solving  and  navigational  strategies 
in  software  use.  Brownlee,  Purdie  and  Boulton-Lewis  (2001)  have  also  been 
shown  that  learning  environments  can  influence  students'  epistemological 
beliefs.  Observations  of  students  in  a first  year  undergraduate  introductory 
Information  Systems  course  at  the  University  of  New  South  Wales,  Australia 
revealed  learning  approaches  and  behaviours  that  suggest  simple 
epistemological  beliefs.  Results  of  a survey  of  students'  epistemological  beliefs 
support  these  observations,  leading  to  a reconsideration  of  pedagogical 
approaches  adopted  in  teaching  the  course  that  might  lead  to  the  development 
of  more  complex  epistemological  beliefs.  The  motivations  for  the  redesign  of 
the  core  information  systems  course  and  the  resulting  restructure  to  include 
web -supported  independent  activities  and  lecturer  facilitated  workshops  are 
described  in  this  paper.  Results  from  the  first  implementation  of  the  course 
revision  in  early  2002  will  be  presented  to  Ed-Media  2002  conference 
delegates. 


Epistemological  beliefs  are  those  concerning  the  nature  of  knowledge  and  learning:  beliefs  about 
how  individuals  come  to  know,  how  knowledge  is  constructed  and  how  knowledge  is  evaluated. 

'Epistemological  beliefs'  is  an  area  that  is  increasingly  seen  as  influencing  learning  and  cognition, 
and  an  area  of  investigation  afforded  greater  attention  by  the  research  community  in  recent  times. 
Beliefs  about  knowledge  have  been  shown  to  influence  factors  such  as  student’s  motivation, 
persistence  or  problem  solving  approach  (Kardash  and  Scholes,1996;  Schommer,  1994b;  Jacobson 
and  Spiro,  1995).  Kardash  and  Scholes  (1996)  draw  attention  to  ‘A  growing  body  of  evidence  (that) 
suggests  individuals’  epistemological  beliefs  play  a critical  role  in  strategic  learning  in  general  and 
higher-order  thinking  and  problem  solving  in  particular’  (p261).  Schommer  (1994b)  suggests  that  ‘... 
epistemological  beliefs  affect  the  degree  to  which  individuals  (a)  actively  engage  in  learning,  (b) 
persist  in  difficult  tasks,  (c)  comprehend  written  material,  and  (d)  cope  with  ill-structured  domains. 
In  each  of  these  areas,  the  evidence  suggests  that  epistemological  beliefs  may  either  help  or  hinder 
learning’  (p302).  In  a summing  up  the  same  article  Schommer  concludes  that  ‘...  there  is  enough 
evidence  to  suggest  epistemological  beliefs  are  critical  to  the  learning  process’  (p315). 

Jacobson  and  Spiro  (1995)  identify  \..epistemic  cognition  as  a relatively  recent  area  of  cognitive 
enquiry  of  potential  relevance  to  hypertext  investigators’  (p305).  They  identify  students  epistemic 
beliefs  as  being  associated  with  ‘...  such  learning  problems  as  the  failure  to  integrate  new  knowledge 
with  prior  knowledge,  use  of  rigid  criteria  for  monitoring  reading  comprehension,  and  poor 
performance  on  comprehension  measures’  (p306).  Based  on  the  work  of  Spiro,  Coulson,  Feltovich 
and  Coulson  (1988)  they  also  suggest  that  ‘...  over  simplification  of  complex  and  ill-structured 
knowledge  contribute  to  the  formation  of  conceptual  misunderstandings  at  post-introductory  stages 
of  learning’  (p306).  As  a result  of  their  1995  study  Jacobson  and  Spiro  describe  preliminary 
evidence  which  suggests  that  students  holding  a simple  set  of  epistemic  beliefs  were  less  able  to 
learn  and  apply  their  knowledge  after  using  a hypertext  system  than  students  who  possessed  a more 
complex  set  of  epistemic  beliefs.  It  is  apparent  that  epistemological  beliefs  are  not  only  important  to 
learning  generally,  but  also  to  learning  with  computer-based  materials. 


Epistemology  is  an  area  with  its  roots  in  philosophy  that  is  generating  increasing  interest  (Belenky, 
Clinchy,  Goldberger  & Tarule,  1997;  Brownlee,  Purdie  and  Boulton-Lewis,  2001;  Hofer,  2000; 
Hofer  & Pintrich,  1997;  Jacobson,  Maouri,  Mishra  & Kolar,  1996;  Schommer,  1990;  Schommer, 
1994a;  Schommer  & Walker,  1997;  Spiro  et  al.,  1988).  Initial  work  in  the  area  in  the  form  of  a 
number  of  longitudinal  studies  involving  mostly  young  American  men  at  college  was  undertaken  by 
Perry  (1970)  who  suggested  a model  based  on  nine  developmental  stages  in  intellectual  and  ethical 
development  as  the  results  of.  Perry’s  work  came  under  attack  in  the  late  1970s  due  to  its  focus  on 
an  ‘elite  male  sample’,  with  Belenky,  Clinchy,  Goldberger  and  Tarule  responding  with  a study 
focusing  on  American  female  subjects  from  academic  institutions  and  from  human  services  agencies 
(1986).  Belenky  et.al.  proposed  a model  based  on  ‘...  a set  of  epistemological  perspectives  from 
which  women  know  and  view  the  world’  (pi 5).  Kuhn  (1991),  Baxter  Magolda  (1992),  and  King  and 
Kitchener  (1994)  have  also  suggested  alternative  developmental  models,  Kuhn  with  levels  of 
‘argumentative  reasoning’,  Baxter  Magolda  with  four  different  ‘ways  of  knowing’  and  King  and 
Kitchener  with  three  ‘reflective  judgement’  stages. 

A recent  prolific  researcher  in  the  area  of  epistemological  beliefs,  Marlene  Schommer,  suggests  that 
epistemological  beliefs  are  far  to  complex  to  be  captured  in  one  dimension  of  developmental  stages 
and  has  proposed  instead  of  a set  of  five  independent  epistemological  dimensions,  each  having  a 
range  of  possible  values.  These  dimensions  include  certainty  of  knowledge  (absolute  to  tentative), 
structure  of  knowledge  (simple  to  complex),  source  of  knowledge  (handed  down  by  authority  to 
derived  through  reason),  control  of  knowledge  acquisition  (ability  to  learn  is  fixed  at  birth  to  ability 
to  learn  can  be  changed),  and  speed  of  knowledge  acquisition  (knowledge  is  acquired  quickly  or  not- 
at-all  to  knowledge  is  acquired  gradually).  Schommer  (1994a)  proposes  these  dimensions  as  a 
starting  point  to  the  line  of  investigation  of  dimensions  of  epistemic  beliefs,  as  she  suggests  that  the 
five  dimensions  are  unlikely  to  encompass  all  possible  aspects  of  epistemological  beliefs.  Hofer's 
(2000)  investigations  in  the  area  of  personal  epistemology  have  shown  that  epistemological  beliefs 
are  not  unrelated  to  the  discipline  of  study  being  undertaken,  and  are  not  'domain  general'  as 
suggested  by  Schommer. 

Consistent  with  the  work  of  Jacobson  and  Spiro  (1995),  Tolhurst  and  Debus  (2000,  and  2002  in 
press)  found  that  the  navigational  strategies  and  learning  approaches  adopted  by  students'  using 
complex  software  may  be  influenced  by  their  epistemological  beliefs.  Dimensions  of  Schommer’s 
epistemological  beliefs  were  found  to  be  relevant  to  this  study  were  those  of  certainty  of  knowledge, 
structure  of  knowledge,  source  of  knowledge  and  speed  of  knowledge  acquisition.  It  was  found  that 
if  students  possess  naive  beliefs  of  knowledge,  such  as  belief  that  knowledge  is  absolute,  is  simple,  is 
handed  down  by  authority  and  is  acquired  quickly  or  not-at-all,  then  this  is  likely  to  affect  the  way 
that  they  use  software  to  seek  information.  With  less  sophisticated  beliefs  students  were  likely 
believe  that  there  is  one  answer  to  be  found,  and  it  will  be  found  quickly  if  the  software  is 
authoritative  or  it  will  not  be  found  at  all.  With  such  beliefs,  students  were  not  likely  to  persist  in 
information  seeking  beyond  the  situation  in  which  they  locate  any  information  they  believe 
satisfactorily  answers  a question,  on  in  the  event  that  they  do  not  find  an  answer  relatively  quickly. 
Conversely,  with  more  sophisticated  beliefs  students,  were  be  more  likely  to  seek  additional 
information  to  that  first  identified  and  persist  in  the  event  of  being  unsuccessful  at  first. 

Of  particular  significance  to  the  work  discussed  in  this  paper  is  the  research  by  Brownlee,  Purdie  and 
Boulton-Lewis  (2001).  Brownlee  et.al  conducted  a study  with  Australian  tertiary  students  at  the 
University  of  Queensland  that  show  it  is  possible  to  significantly  influence  students'  epistemological 
beliefs  and  produce  positive  learning  outcomes.  Brownlee  et.al.  measured  students  beliefs  before 
and  after  a course  of  study  in  which  two  groups  of  students  experienced  a year-long  of  study  in 
which  one  group  was  required  to  reflect  on  their  epistemological  beliefs  using  personal  diaries.  They 
found  that  the  group  involved  in  reflective  practice  experienced  a statistically  significant  shift  to 
more  complex  epistemological  beliefs  that  those  students  who  did  not.  They  conclude  that  student 
epistemological  beliefs  are  able  to  be  influenced,  and  that  this  has  implications  for  how  educators 
develop  learning  environments. 

Based  on  the  work  of  Brownlee  et.  al.  that  epistemological  beliefs  can  be  influenced,  a study  in- 
progress is  discussed  in  this  paper  which  seeks  to  determine  whether  changing  established  learning 
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environments  might  encourage  in  students  more  complex  epistemological  beliefs,  and  hence 
encourage  more  appropriate  study  approaches  in  tertiary  students. 

Research  Study  In-progress 

Background 

After  evaluating  course  outcomes,  staff  teaching  first-year  undergraduate  students  introductory 
courses  in  the  School  of  Information  Systems,  Technology  and  Management  (SISTM)  at  the 
University  of  New  South  Wales  (UNSW)  have  begun  questioning  the  pedagogical  approaches 
practiced  in  teaching.  Observations  of  student  behaviour  have  identified  attitudes  and  practices  that 
are  of  concern  to  staff,  in  that  many  students:  rely  strongly  on  teaching  staff  to  provide  content  and 
guidance  on  assignments;  rely  on  a single  source  of  information  (often  the  internet)  rather  than 
consulting  a range  of  sources;  tend  to  quote  large  sections  of  text  from  sources  with  little  attempt  to 
'paraphrase'  material  or  integrate  ideas  from  different  sources.  The  results  of  an  epistemological 
beliefs  questionnaire  provide  support  for  the  impressions  that  staff  hold,  indicating  that  in  the  main 
students  possess  naive  epistemological  beliefs. 

Epistemological  Beliefs  of  Current  Student  Group 

In  first  session  of  2001  the  Epistemological  Beliefs  Questionnaire  (Schommer,  1998)  was  completed 
by  201  students  studying  the  first  undergraduate  Information  Systems  course.  Table  1 shows  the 
results  of  that  survey. 

From  the  data  presented  in  Table  1 it  is  apparent  that  the  epistemological  beliefs  of  the  majority  of 
our  students  are  consistent  with  the  study  practices  observed  by  staff.  The  subscales  (range  of  1 to  5, 
5 represents  strong  agreement)  with  the  highest  mean  values  small  standard  deviations  suggest  that 
students  in  the  main:  seek  single  answers  (mean=3.17,  sd=.35),  depend  on  authority  (mean=3.21, 
sd=.58),  and  avoid  ambiguity  (mean=3.17,  sd=.55).  Other  values  also  support  evidence  of  simple 
epistemological  beliefs  in  the  majority  of  the  students,  in  particular:  concentrated  effort  is  a waste  of 
time  (mean=2.70,  sd=.7),  knowledge  is  certain  (mean=2.76,  sd=.49)  avoid  integration  (mean=2.70, 
sd=.41),  and  ability  to  learn  is  innate  (mean=2.71,  sd=.7).  It  would  seem  that  the  current  course 
structure  (2x1 -hour  lecture  and  one  hour  tutorial/laboratory  per  week)  does  not  support  more 
complex  epistemological  beliefs  in  students  that  are  associated  with  more  desirable  learning 
approaches  and  outcomes. 


Scales,  Subscales  and  sample  items  Student  responses 


M 

SD 

Quick  Learning 
Learning  is  quick 

2.53  (n=199) 

0.58 

1 . If  you  are  ever  going  to  understand  something,  it  will  make  sense  to  you  the  first 
time  you  hear  it. 

Learn  first  time 

20.  Going  over  and  over  a difficult  textbook  chapter  usually  won't  help  you  to 
understand  it. 

2.55  (n=198) 

0.68 

Concentrated  effort  is  a waste  of  time 

53.  If  a person  tries  too  hard  to  understand  a problem,  they  will  most  likely  just  end 
up  getting  confused. 

2.70  (n=201) 

0.70 

Certain  Knowledge 
Avoid  ambiguity 

3.17(n=198) 

0.55 

27.  If  lecturers  would  stuck  more  to  the  facts  and  do  less  theorising,  one  could  get 
more  out  of  University. 

Knowledge  is  certain 

2.76  (n=193) 

0.49 

12.  If  scientists  try  hard  enough,  they  can  find  the  truth  to  almost  anything. 
Avoid  integration 

54.  Being  a good  student  generally  involves  memorising  facts. 

2.70  (n=198) 

0.41 

Innate  Ability 

Cannot  leam  how  to  learn 

15.  The  most  successful  people  have  discovered  how  to  improve  their  ability  to  leam 
(negative  item). 

2.37(n=199) 

0.55 

Success  unrelated  to  hard  work 

49.  The  really  smart  students  don't  have  to  work  hard  to  do  well  at  university. 

2.32(n=197) 

0.66 

Ability  to  learn  is  innate 

55.  Students  who  are  'average'  in  school  will  remain  'average'  for  the  rest  of  their  life. 

2.71  (n=194) 

0.70 

Omniscient  Authority 

Depend  on  authority 

5.  How  much  a person  gets  out  of  university  depends  on  the  quality  of  their  teacher. 

3.21  (n=200) 

0.58 

Don^t  criticise  authority 

3.  For  success  at  university  it  is  best  not  to  ask  too  many  questions. 

2.37  (n=199) 

0.53 

Simple  Knowledge 

Seek  single  answers 

59.  The  best  thing  about  science  courses  is  that  most  problems  have  only  one  right 
answer. 

3.17(n=197) 

0.35 

Avoid  integration 

54.  Being  a good  student  generally  involves  memorising  facts. 

2.70  (n=198) 

0.41 

Table  1.  Results  of  Schommer's  Epistemological  Beliefs  Questionnaire  for  first  year  undergraduate 
students  studying  the  first  core  Information  Systems  course  in  Session  1 2001 
Note:  M = mean,  SD=  Standard  Deviation,  Subscale  range  is  1 to  5,  5 represents  strong  agreement. 


Changes  in  Course  Structure  for  2002 

In  an  attempt  to  address  the  problems  identified,  the  first  year  core  undergraduate  course  in 
Information  Systems  is  being  restructured  in  a way  that  minimises  formal  lectures  where  content  is 
provided  by  an  'authority'.  In  place  of  the  formal  lectures  students  will  spend  half  of  their  structured 
course  work  undertaking  web-supported  independent  work  in  preparation  for  workshop  sessions  in 
which  experienced  staff  facilitate  learning  activities.  The  workshops  will  be  classes  of  no  more  than 
30,  in  place  of  participation  in  lectures  of  up  to  250  students.  Rather  than  be  passive  learners  in 
lectures,  students  will  be  required  to  be  actively  involved  in  their-own  learning  in  both  the 
independent  web-supported  activities  and  workshops.  They  will  be  required  to  undertake  guided  and 
independent  research  from  multiple  sources  (on-line,  computer-based  and  non-computer  based), 
involve  themselves  in  group  discussions,  case  studies,  project  work,  debates,  design  activities, 
software  activities,  role  plays  and  peer  evaluations.  It  is  expected  that  the  work  that  they  undertake 
will  challenge  any  tendency  to  settle  for  'simple',  quick'  or  'first'  answers,  single  sources,  and  the 
seeking  of  information  from  the  lecturer  as  an  'authority'. 


Anticipated  Outcomes 

It  is  expected  that  the  changes  to  the  course  structure  to  be  implemented  for  the  first  time  in  2002 
will  foster  a more  desirable  'academic'  approach  to  learning  in  our  students,  encouraging  greater 
student  independence  and  supporting  more  sophisticated  epistemological  beliefs.  By  developing 
more  complex  epistemological  beliefs  it  is  anticipated  that  students  will  be  more  likely  to  appreciate 
the  complexity  of  knowledge,  seek  information  from  multiple  sources  and  integrate  the  ideas  they 
find  from  these  sources  and  rely  less  on  the  'authority'  of  the  lecturer.  It  is  anticipated  that  in  their 
computer-based  research  work  that  students  will  be  willing  to  persist  in  information  seeking  beyond 
the  situation  in  which  they  first  locate  any  information  they  believe  satisfactorily  answers  a question, 
on  in  the  event  that  they  do  not  find  an  answer  relatively  quickly. 

By  June  2002,  the  date  of  the  ED-Media  conference,  quantitative  results  from  Schommer's 
Epistemological  Beliefs  Questionnaire  (1998)  and  qualitative  results  from  student  interviews 


following  the  first  teaching  of  the  newly  structured  course  will  be  available  for  presentation  to 
delegates. 
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Abstract:  Active  learning,  case  based  teaching,  and  simulations  have  become  increasingly  popular  tools  for 
international  education.  In  order  to  realize  the  full  potential  of  Intemet2,  Teaching  Human  Rights  Online  at 
httD://oz.uc.edu/thro  provides:  1)  interactive  cases  that  give  students  immediate  feedback  at  any  hour  they  answer 
questions  online;  2)  collaborative  problem  solving  exercises  for  multi-national  teams;  3)  structured  formats  for  real 
world  cases  and  hypothetical  scenarios  to  focus  on  critical  international  issues;  and  4)  internet  videoconferencing  for 
students  around  the  world  to  confront  their  differences.  TV  news  broadcasts  routinely  feature  a split  screen 
conversation  between  international  reporters  in  different  countries.  As  internet  connections  improve,  students  from 
different  countries  will  have  similar  opportunities  to  meet  face  to  face  online.  Rather  than  dehumanize  learning,  new 
technology  should  mediate  direct  communication  between  diverse  groups  whose  members  may  never  travel  abroad  to 
meet  in  person. 


Introduction 

New  technology  creates  extraordinary  opportunities  for  global  learning  communities  but  also  threatens  to 
dehumanize  higher  education.  Teaching  Human  Rights  Online  (THRO)  is  an  interdisciplinary,  multi-campus  project 
begun  in  1997  that  employs  interactive  cases  and  internet  conferencing  to  foster  critical  thinking  and  normative 
reasoning  through  collaborative  interaction.  Six  THRO  exercises  apply  information  age  technology  to  the  type  of 
problem  based  instruction  found  in  the  classical  works  of  Sophocles  and  Thucydides.  Antigone  encourages  students 
to  decide  whether  an  individual  may  invoke  natural  law  to  defy  a ruler  who  demands  obedience  in  order  to  avert 
anarchy.  (Cranston,  1983)  The  Melian  Dialogue  confronts  students  with  enduring  questions  of  power  politics  and 
universal  justice.  How  can  technology  improve  on  the  personal  interplay  of  student  and  teacher  realized  in  classic 
drama,  historic  debate,  and  Socratic  dialogue?  In  its  first  four  years.  Project  THRO  published  six  contemporary 
teaching  cases  as  issues  of  an  electronic  journal  that  will  continue  to  solicit  and  disseminate  interactive  exercises  for 
internet  conferences. 


Project  Partners 

Political  Science  professor  Howard  Tolley  initiated  THRO  in  association  with  the  University  of  Cincinnati 
Law  SchooPs  Morgan  Institute  of  Human  Rights  and  with  College  of  Education  professor  Dan  Wheeler.  THRO 
faculty  associates  in  the  U.C.  Colleges  of  Education,  Arts  and  Sciences,  Law,  Business,  and  University  College 
collaborated  in  preparing  interactive  problems  online  as  a supplement  to  traditional  assigned  readings,  class  lectures, 
and  discussion.  External  grants  from  the  U.S.  Institute  of  Peace  and  he  Ohio  Learning  Network  supported 
instructional  technology  professional  development  workshops  in  September  1999  and  March  2001  as  well  as  follow- 
up distance  education  courses  at  the  graduate  and  undergraduate  level.  The  two  faculty  development  workshops 
brought  100  faculty  from  eight  countries  and  ten  states  to  the  U.C.  campus  for  sessions  on  asynchronous  and 
synchronous  internet  conferencing  with  text,  audio,  and  video  (with  Polycom  ViaVideo).  The  training  identified  the 
best  applications  for  low  bandwidth  connections  as  well  as  for  Internet2  high  speed  access. 

As  a result  of  the  workshops,  THRO-Net  now  lists  eighty -five  associates  including  faculty  in  Australia, 
Bulgaria,  India,  South  Africa,  and  Turkey.  Email  discussion  groups  with  minimal  technical  requirements  enable 
THRO  to  reach  the  widest  audience  for  text  chat  interactions.  U.C.  makes  the  case  materials  available  online  at  no 
cost  to  individuals  as  well  as  to  instructors  who  use  the  problems  either  as  a course  assignment  or  for  structured 
exchange  with  classes  at  other  institutions.  English  speaking  teachers  and  students  anywhere  in  cyberspace  may  use 
the  exercises  both  as  individuals  and  in  organized  interaction  between  classes  at  different  campuses.  Each  THRO 
case  has  a teaching  note  for  course  instructors. 

THRO  promotes  three  distinct  cross-disciplinary  collaborative  relationships: 


• Case-writing  Associates  led  by  Political  Scientist  Howard  Tolley  develop  teaching  problems.  During 
academic  year  2002-03  a political  scientist  on  sabbatical  from  Earlham  College  will  wor,k  on  a THRO  case 
as  a scholar  in  residence  at  the  Morgan  Institute. 

• Project  Associates  led  by  Education  Professor  Dan  Wheeler  adapt,  evaluate,  and  train  others  to  use 
instructional  technology.  In  support  of  its  major  information  technology  initiative,  U.C.  was  a founding 
member  of  the  Intemet2  consortium,  received  a $328,000  NSF  grant  to  support  connections  to  the  Next 
Generation  Internet,  and  belongs  to  the  ITEC-Ohio  consortium  with  eight  university  and  two  corporate 
partners.  Wide  bandwidth  connections  now  provide  acceptable  quality  audio  and  video  streaming  between 
U.C.  and  other  Intemet2  institutions. 

• Teaching  Associates  registered  on  THRO-Net  assign  the  cases  to  students  and  may  arrange  online  text  or 
videoconferences  with  others  in  the  network. 


Goals  and  Methods 

THRO  envisions  a third  dimension  of  “IT”  beyond  “information”  and  “instructional”  technology  to  accomplish 
an  “interpersonal”  objective.  Online  simulations  with  classes  in  different  countries  can  provide  to  more  students  at 
less  cost  Model  U.N.  type  learning  opportunities  enriched  by  multinational  participation.  Traditionall)^  each  school 
participating  in  a simulation  represents  a different  state’s  delegation.  With  the  latest  technology,  students  from  a 
U.S.  class  might  join  different  multi-national  work  groups  online,  collaborating  electronically  with  students  of  other 
countries  to  prepare  for  a videoconference  simulation  or  seminar.  Students  in  multiple  locations  working  together  on 
international  human  rights  cases  realize  several  important  advantages: 

• Increasing  the  diversity  of  the  participants.  Students  in  a single  classroom  are  often  quite  homogenous  and 
tend  to  assume  that  their  own  notion  of  rights  is  universal.  Interaction  with  students  from  others  countries 
develops  a better  understanding  of  cultural  relativism. 

• Increasing  the  realism  of  simulation.  When  one  of  the  educational  goals  is  to  teach  students  to  work  with 
people  from  other  countries,  having  participants  who  are  actually  from  different  countries  adds  greatly  to 
the  value  of  the  role-playing  exercises. 

• Providing  an  independent  audience.  Students  are  motivated  to  take  their  work  more  seriously  when  there  is 
an  audience  beyond  their  own  classroom. 

• Promoting  collaborative  learning.  When  classes  work  together  via  telecommunications,  they  must 
collaborate  to  make  the  exercise  work.  This  is  in  addition  to  the  collaboration  that  is  directly  involved  in 
the  case-based  exercise. 

For  over  fifty  years  proponents  of  the  Universal  Declaration  of  Human  Rights  have  challenged  advocates  of 
cultural  relativism  in  philosophy,  history,  law,  international  relations,  and  education.  (Renteln,  1990.  Gewirth,  1982) 
The  United  Nations  proclaimed  1995-2004  as  a Decade  for  Human  Rights  Education.  Students  in  law,  business,  or 
the  social  sciences  begin  a THRO  case  with  individual  activity  by  using  a personal  computer  to  access  materials 
needed  to  answer  online  questions  about  a human  rights  dilemma.  At  any  hour  students  can  answer  multiple  choice 
and  short  essay  questions  online  and  receive  immediate  feedback  identifying  errors  and  offering  model  answers. 
(Haladyna,  1997,  Wiggins,  1998.)  Analysis  of  student  responses  enables  case  authors  to  design  improved  items  that 
measure  and  promote  higher  order  thinking — analysis,  application,  problem  solving,  and  evaluation.  The  student 
writes  short  advocacy  arguments  in  support  of  rival  claimants  before  reaching  a balanced  judgment  in  a concluding 
synthesis.  THRO  cases  involve  actual  rather  than  hypothetical  situations.  Case  updates  keep  online  material  current 
with  active  links  to  relevant  web  sites.  The  multidisciplinary  human  rights  cases  not  only  require  critical  thinking 
about  political  and  legal  decisions  but  also  pose  moral  dilemmas  that  demand  ethical  reasoning. 


Prior  Work 

Active  learning,  case  based  teaching,  and  simulations  online  have  become  increasingly  popular  tools  for 
international  education.  First  generation  education  initiatives  on  the  internet  have  not  realized  the  full  potential  for 
personal  interaction  online.  Some  hardcopy  cases  on  international  issues  published  by  Harvard,  Georgetown,  and  the 
University  of  Washington’s  Electronic  Hallway  fwww.hallwav.org can  now  be  downloaded  off  the  web.  Case 
instructors  exchange  email  and  post  messages  to  share  teaching  ideas,  but  students  derive  little  added  value  for 
costly  instructional  technology.  Business  cases  atwww.i-Case.com  add  audio  and  video  streaming  to  standard  text 
without  engaging  students  in  active  problem  solving  or  collaborative  learning. 

Several  efforts  to  arrange  multi-campus  simulated  Model  U.N.  activities  online  have  been  short  lived.  The 
costs  of  televideo  ISDN  or  satellite  transmissions  have  been  prohibitive.  International  text  conferences  on  the 
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internet  have  provided  a low  tech  alternative.  Business  faculty  in  Association  Global  View  (www^lobalview.org) 
create  hypothetical  scenarios  for  student  management  teams  in  different  countries  to  negotiate  contracts  and  trade 
agreements.  Since  the  1970s  Project  ICONS  of  the  University  of  Maryland  (Starkey,  1996;  www.icons.umd.edu) 
has  sponsored  online  negotiation  simulations  employing  text  exchange  rather  than  videoconference  technology. 
THRO  seeks  to  have  greater  impact  by  bringing  together  students  from  different  cultures  in  structured 
videoconferences  that  examine  their  distinct  approaches  to  ethical  reasoning. 


A Unique  Approach 

THRO  is  unique  in  adding  interactive  self-assessment  online,  using  actual  situations,  and  providing  a 
videoconference  option  made  possible  by  Intemet2.  THRO  currently  provides  eight  active  learning  course  modules 
that  employ  technology  to  improve  not  only  critical  thinking  about  human  rights  but  also  to  facilitate  student 
interaction  with  classmates,  their  instructors,  and  fellow  learners  in  other  countries.  (See  http://oz.uc.edu/thro.) 

• A Just  War?  President  Clinton's  Response  to  Kosovo 

• Prime  Minister  Rao  *s  Dilemma:  Terrorism  and  Human  Rights  in  India 

• The  International  Court  of  Justice  Considers  Genocide:  Bosnia  v.  Yugoslavia 

• Shah  Bano:  Muslim  Women 's  Rights  in  India 

• Rape  and  Genocide  in  Rwanda:  The  ICTR  's  Akayesu  Verdict 

• Slavery  in  Burma?  Doe  v Unocal. 

• Sanctions  or  Engagement:  SLORC  and  Myanmar 

• Business  Ethics  and  SLORC:  Unocal  and  the  Yadana  Gas  Pipeline 

Those  cases  have  been  successfully  used  for  both  video  and  text  conferences  connecting  students  from 
Cincinnati  with  classes  in  North  Dakota  and  Ankara,  Turkey.  Specifically  THRO  targets  undergraduate  and 
graduate/professional  courses  in  the  social  sciences,  history.  Jurisprudence,  philosophy,  comparative  religion,  ethics, 
international  studies  and  education.  At  a large  university  where  commuter  students  rarely  collaborate  on  educational 
projects  outside  of  class,  asynchronous  communication  in  cyberspace  (an  increase  interaction  with  peers  and 
faculty.  (Bloom,  1956;  Dewey,  1933;  Gamson  and  Chickering,  1987;  Perry,  1970;  Romm  and  Mahler,  1986) 

THRO  capitalizes  on  the  latest  instructional  technology  to  employ  a unique  combination  of  learning 
strategies — active,  authentic,  collaborative  learning  with  educative  assessment  and  cross-cultural  exchange.  Email 
exchange  and  electronic  bulletin  boards  facilitate  advance  collaboration  by  students  preparing  as  co-counsel  for  in 
class  role-playing.  Videoconference  simulation  or  online  discussion  with  students  from  other  institutions  and 
countries  also  requires  advance  posting  of  advocacy  papers  in  electronic  text.  THRO  cases  encourage  students  to 
become  independent,  learning  on  their  own  and  from  each  other  without  relying  exclusively  on  teaching 
professionals.  The  instructor’s  role  shifts  from  a “sage  on  the  stage”  to  a “guide  on  the  side.”  Direct  exchange  with 
peers  abroad  stimulates  more  thorough  preparation  and  a deeper  understanding  of  the  difficult  quest  for  universal 
human  rights  norms.  Engaging  individuals  from  another  culture  encourages  students  to  work  through  their  own 
beliefs  and  assists  them  in  understanding  why  others  may  see  the  world  differently.  Cross  cultural  dialogue  based 
on  THRO  exercises  could  bridge  the  digital  divide  to  include  students  at  historically  African-American  institutions 
and  also  facilitate  improved  cross  cultural  interaction  at  universities  with  significant  minority  and  international 
student  enrollments. 

Guided  by  a course  instructor,  students  communicate  with  classmates  by  using  an  electronic  bulletin  board 
and  exchanging  email.  At  a large  university  where  commuter  students  rarely  collaborate  on  educational  projects 
outside  of  class,  asynchronous  communication  in  cyberspace  can  increase  interaction  with  peers  and  faculty. 
Teaching  Notes  for  each  THRO  problem  describe  in-class  role  playing  exercises  that  require  advance  collaboration 
online  by  student  work  groups.  Properly  used  by  a skilled  professional,  technology  should  enhance  student 
preparation  for  in-class  simulation  and  facilitate  cooperative  problem  solving.  The  resulting  classroom  exchange 
should  be  more,  not  less  personal. 

New  videoconferencing  technology  such  as  Polycom  ViaVideo  should  provide  considerably  more  valuable 
learning  opportunities.  Instructors  can  engage  their  classes  in  simulations  and  seminar  discussions  with  students  of 
different  cultures.  After  examining  a case  problem  from  their  American  perspective,  students  can  learn  directly  and 
personally  how  individuals  of  another  faith  or  country  assess  the  same  situation.  Synchronous  text  chat  room 
interaction  has  become  a common  and  all  too  often  aimless  exchange.  Live  video  and  audio  transmission  on  the 
internet  overcomes  the  anonymity  and  impersonal  quality  of  text  chat.  THRO  problems  establish  a structure  for 
instructors  to  maximize  the  educational  value  of  a live  videoconference.  Rather  than  dehumanize  learning,  video 
communication  hardware  should  mediate  face  to  face  conversation  between  diverse  groups  whose  members  may 


never  travel  abroad  to  meet  in  person.  With  desktop  videoconferencing  equipment  an  instructor  can  arrange 
personal  interviews  with  academic  experts  and  non-governmental  human  rights  activists  working  on  the  issue. 


Evaluation 

During  case  development,  THRO  obtains  formative  assessment  by  peer  review  and  pilot  testing  before  a 
final  version  of  each  interactive  exercise  and  teaching  note  become  available  online.  Each  THRO  case  was  subject 
to  peer  review  at  one  or  more  professional  meetings  that  included  the  North  American  Case  Research  Association, 
1997,  1999,  and  2001;  World  Affairs  Case  Research  Association  1998;  International  Studies  Association,  1997  and 
1998;  International  Political  Science  Association,  2000. 

For  overall  project  evaluation  or  summative  assessment,  an  advanced  doctoral  candidate  in  education  has 
used  the  Flashlight  Student  Inventory  developed  by  AAHE  to  assess  collaborative  learning  and  changes  in  study 
methods  by  students  completing  the  exercises  in  professor  Tolley’s  class.  Statistical  analysis  of  student  responses  to 
multiple  choice  self-assessment  items  identifies  items  by  level  of  difficulty  and  ability  to  discriminate.  Professor 
Wheeler  has  also  completed  preliminary  research  comparing  students  taught  online  and  offline.  Course  instructors 
provide  feedback  after  using  THRO  cases,  and  the  project  director  monitors  global  use  with  software  that  tracks  hits 
to  each  page  in  the  web  site. 

Peer  reviewed  articles  and  book  chapters  about  THRO  (Tolley,  1998,  1999,  2000),  conference 
presentations,  [1]  course  adoptions  beyond  U.C.,  and  grants  from  the  U.S.  Institute  of  Peace  and  Ohio  Learning 
Network  demonstrate  growing  acceptance  and  impact  in  several  disciplines.  THRO  enables  instructors  to  use 
classical  forms  of  active  learning— the  case  method,  simulation,  and  Socratic  dialogue — not  only  in  the  classroom 
but  also  in  cyberspace  communication  between  students  of  different  cultures.  Project  THRO  was  recognized  by  the 
American  Political  Science  Association  as  the  best  political  science  website  for  2001. 


Future 

In  2002,  the  project  will  complete  three  cases  under  development  providing  distinct  perspectives  on  the 
Unocal  Corporation’s  Yadana  pipeline  project  in  Burma/Myanmar — business  ethics,  claims  of  forced  labor  in  U.S. 
courts,  and  U.S.  sanctions  policy.  New  cases  are  planned  on  the  International  Criminal  Court  and  the  death  penalty. 

The  University  of  Cincinnati  THRO  project  directors  are  committed  to  developing  global  distance 
education  partnerships  employing  the  most  advanced  instructional  technology.  The  project  will  seek  grant  support 
to  provide  overseas  partners  with  videoconferencing  equipment.  With  external  funding,  one  THRO  exercise  has 
been  translated  into  Spanish  and  French  disseminated  with  the  English  versions  both  online  and  with  an  interactive 
CD-ROM.  Future  transnational  conferences  should  enable  multilingual  students  to  debate  issues  raised  in  THRO 
exercises  with  peers  in  Spanish  and  French  speaking  countries.  It  might  also  be  possible  to  obtain  simultaneous 
translation  during  online  conferences  using  foreign  language  majors  or  international  students. 

Ethnic  hostilities,  famines,  arbitrary  detention,  torture,  and  killing  of  political  dissidents  and  their  families 
threaten  both  individual  liberty  and  global  security.  Human  rights  education  for  the  next  generation  of  lawyers, 
teachers,  politicians,  journalists,  and  businessmen  deserves  high  priority.  Project  THRO  can  pioneer  a unique  method 
of  transnational  education  that  moves  beyond  the  Jessup  international  law  moot  court  and  undergraduate  Model  U.N. 
programs.  Future  transnational  videoconferences  will  build  key  constituencies  for  a global  civil  society — advocates 
trained  to  bring  human  rights  claims  before  national  and  international  tribunals,  leaders  of  nongovernmental 
organizations,  teachers  and  professors,  government  officials  and  community  activists. 


[1]  EDUCAUSE,  annual  Lilly  Conferences  on  College  Teaching,  (Oxford,  Ohio),  and  the  Midwest  Political  Science  Association 
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Abstract;  This  paper  describes  the  Task  Generator^  a component  of  an  authoring  tool  for  an 
intelligent  language  tutoring  system.  The  Task  Generator  automatically  creates  learning  material 
from  authentic  texts.  The  architecture  is  designed  so  that  it  can  be  readily  ported  to  create  learning 
material  for  other  languages,  with  minimal  resource  requirements.  This  paper  reports  on  a recent 
study  which  evaluates  the  accuracy  of  the  system. 


Introduction 

The  goal  of  authoring  tools  for  Intelligent  Tutoring  Systems  (ITSs)  is  to  reduce  the  cost  in  time  and  expertise  that  is 
required  to  produce  a usable  intelligent  learning  environment.  This  paper  describes  the  Task  Generator,  a 
component  of  an  authoring  tool  for  an  Intelligent  Language  Tutoring  System  (ILTS).  The  Task  Generator  allows 
teachers  to  automatically  create  learning  material  by  merely  specifying  the  learning  objective  and  providing  samples 
of  text  which  are  likely  to  contain  examples  of  the  learning  objective.  The  primary  advantage  of  the  Task  Generator 
is  that  the  authoring  tool  is  more  usable  since  the  tasks  and  expertise  required  of  the  user  are  reduced. 

Since  we  have  developed  intelligent  tutors  for  a variety  of  languages  (English,  German,  Greek),  the 
architecture  of  the  Task-Generator  is  designed  so  that  the  system  can  be  readily  ported  to  create  learning  material 
for  a variety  of  languages,  requiring  only  those  linguistic  resources  that  can  be  expected  to  be  widely  available.  The 
Task  Generator  is  currently  implemented  for  English. 

This  paper  is  organized  as  follows.  Section  2 introduces  the  basic  functions  of  the  ILTS  for  which  we 
created  the  authoring  environment,  and  introduces  the  authoring  environment  itself.  Section  3 describes  the 
architecture  of  the  Task  Generator  and  its  implementation  for  English.  Section  4 reports  on  the  findings  of  a study  to 
evaluate  the  accuracy  of  the  tool.  Concluding  comments  can  be  found  in  Section  5. 


Intelligent  Language  Tutoring  Systems  & Authoring  Tools 

The  goal  of  the  ILTSs  that  we  have  developed  is  to  provide  meaningful  and  interactive  vocabulary  and  grammar 
practice  for  second  language  learners,  as  illustrated  in  Figure  1.  Each  system  analyzes  sentences  from  the  student 
and  detects  grammatical  and  other  errors  (see  Schneider  & McCoy  1998).  The  feedback  modules  of  the  system 
correlate  the  detailed  output  of  the  linguistic  analysis  with  an  error-specific  feedback  message.  Unlike  other  ILTSs 
(Gerbault  1999,  Yang  & Akahori  1997,  1999),  in  our  system  feedback  is  individualized  through  an  adaptive  Student 
Model,  which  monitors  a user's  performance  over  time  across  different  grammatical  constructs. 

In  order  to  make  intelligent  tutoring  systems  useful  to  a wide  audience,  it  is  necessary  to  have  authoring 
tools  which  allow  the  tutor  to  be  modified.  Ideally  the  authoring  tools  can  be  used  by  the  instructors  themselves. 
However,  as  Murray  (1999)  points  out,  when  developing  authoring  tools  for  ITSs  there  is  a tradeoff  between 
usability  and  flexibility. 

We  have  argued  elsewhere  that  in  the  language  learning  domain  usability  can  be  maintained  without 
significant  loss  of  flexibility.  This  paper  describes  additional  means  by  which  the  usability  of  ILTS  authoring  tools 
can  be  increased.  As  a first  step,  the  basic  functions  of  the  authoring  tool  we  have  developed  are  described. 
Following  this,  we  introduce  the  issues  which  the  Task  Generator  addresses. 


The  authoring  tool,  the  Tutor  Assistant  allows  instructors  to  create  on-line  vocabulary  and  grammar 
exercises  for  a beginner  and  intermediate  language  skill  level.  In  addition,  instructors  can  author  teaching  strategies 
and  the  student  model. 

When  adding  or  editing  an  exercise,  the  instructor  has  two  main  tasks: 

• create  the  student  task 

• list  all  the  possible  answers 

The  student  task  is  a set  of  words  and  prompts  that  the  student  must  use  to  complete  the  task.  In  the  case  of 
the  Build-a-Sentence  example,  the  student  task  lists  the  words  that  the  students  must  use  in  their  answer.  The  user  of 
the  authoring  system  can  also  add  additional  prompts  like  “simple  past”  to  further  restrict  the  student  s task,  as 
illustrated  in  Figure  1. 
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Figure  1 : The  ESL  Tutor 

Our  previous  studies  have  shown  that  ESL  teachers  who  are  inexperienced  with  both  the  authoring  system  and  the 
ILTS,  can  develop  one  hour  worth  of  learning  material  in  about  2-3  hours.  While  this  is  a significant  improvement 
over  development  times  for  authoring  tools  in  other  domains,  the  participants  indicated  they  found  some  sections  of 
the  process  difficult.  In  particular,  they  found  it  challenging  to  create  student  tasks  which  were  varied,  interesting 
and  relevant.  Significant  time  was  spent  thinking  of  relevant  examples. 

In  order  to  alleviate  this  problem,  and  to  decrease  the  overall  development  time  for  authoring  learning 
material,  it  was  decided  to  automate  the  process  of  creating  the  learning  task. 
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The  Task  Generator 

The  goal  of  the  Task  Generator  is  to  extract  learning  material  from  unedited  texts.  Unlike  similar  work  by 
Kunichika  et.  al.  (1998),  the  Task  Generator  creates  examples  from  texts  which  already  contain  the  target  learning 
objective.  Kunichika  et.  al.  on  the  other  hand,  generate  questions  for  reading  comprehension  from  assertions  existing 
in  the  text.  The  disadvantage  of  the  Kunichika  approach  for  the  current  task  is  that  significant  syntactic,  semantic,  as 
well  as  discourse  information  must  be  supplied  - either  automatically,  or  by  the  author. 

This  level  of  analysis  is  infeasible  given  the  goals  of  our  system.  Firstly,  our  goal  is  to  reduce  the  skill  level 
and  time  required  from  the  human  authors.  Secondly,  from  a computational  perspective,  one  of  the  design  goals  for 
the  Task  Generator  system  is  portability  between  languages.  This  is  motivated  by  the  fact  that  the  system  needs  to 
be  used  by  ILTSs  from  more  than  one  language.  Hence,  in  order  to  develop  a system  that  is  both  usable  and 


portable,  all  of  the  task  generation  must  be  done  by  the  system,  and  minimal  linguistic  resources  should  be  used. 

The  Task  Generator  meets  these  goals.  The  user  is  required  only  to  specify  a particular  learning  objective, 
such  as  past  tense,  passive,  or  plural  nouns,  and  to  provide  a range  of  texts  which  contain  examples  of  the  learning 
objective  in  use.  Furthermore,  the  linguistic  resources  are  restricted  to  a sentence  boundary  detector,  a Part-of- 
Speech  (POS)  tagger,  and  a lexicon  containing  syntactic  information.  These  resources  are  generally  available  for  the 
more  widely  studied  languages,  and  are  what  is  minimally  required  to  disambiguate  the  input  text. 

The  overall  architecture  of  the  system  can  be  found  in  Figure  2. 

The  first  module  of  the  system  detects  sentence  boundaries,  and  separates  the  text  into  its  constituent 
sentences.  We  use  the  MXTerminator  system  (Reynar  & Ratnaparkhi  1 997). 

The  next  stage  is  the  out-of-vocabulary  (OOV)  words  filter.  The  ILTSs  for  which  the  learning  material  is  being 
created  perform  significant  linguistic  analysis  on  the  student  input.  Hence,  all  words  in  the  task  must  be  known  to 
the  ILTSs.  The  purpose  of  the  OOV  Filter  is  to  remove  sentences  which  contain  out-of-vocabulary  words. 

The  Part-of-Speech  tagger  assigns  one  of  fourteen  tags  to  each  word  token.  The  POS  tagger  is  a Hidden 
Markov  Model  based  on  Church  (1988).  The  tagger  disambiguates  the  input  so  that  the  correct  lexical  information 
can  be  retrieved  from  the  lexicon.  Our  lexicon  is  derived  from  the  XT  AG  lexicon  (Egedi  & Martin  1994).^ 

The  Learning  Objective  Filter  removes  all  sentences  which  do  not  contain  an  instance  of  the  specified 
learning  objective.  For  each  possible  learning  objective  a Learning  Objective  Filter  has  been  developed  which 
specifies  positive  and  negative  constraints  on  the  lexical  entries  that  occur  in  a given  sentence.  For  example,  the 
filter  for  the  Present  Continuous  learning  objective  specifies  that  the  sentence  must  contain  an  auxiliary  with  a base 
form  of  ’be'  and  an  additional  verb  in  present  participle  form.  It  is  also  possible  to  specify  a window  within  which 
the  lexical  items  that  match  the  constraints  must  co-occur. 
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Figure  2:  System  Architecture 


The  sentences  remaining  after  the  learning  objective  filter  each  contain  at  least  one  example  of  the  learning 
objective.  These  sentences  form  the  input  to  the  Task  Creator. 

The  Task  Creator  converts  the  sentence  into  one  or  more  of  the  grammar-practicing  exercise  types  used  in 
our  ILTSs:  Build-a-Sentence,  Fill-in-the-Blank,  and  Drag-and-Drop.  The  Format  Specification  file  contains 
information  about  how  the  words  in  the  sentence  should  be  converted  for  each  exercise  type.  In  the  case  of 
converting  a present  continuous  sentence  into  a Fill-in-the-Blank,  the  main  verb  is  converted  to  base  form,  the 
auxiliary  is  removed,  and  the  remaining  words  remain  in  their  original  form.  This  is  illustrated  in  (1).  In  the  case  of 
converting  the  same  sentence  to  Build-a-Sentence,  a similar  process  is  followed  except  that  all  words  are  converted 
to  their  base  form,  as  illustrated  in  (2). 

(1)  But  more  than  half  also  said  they  [ save  (present  continuous)  ] for  a dream  vacation. 

(2)  but  more  than  half  also  say  they  save  (present  continuous)  for  a dream  vacation  . 


^ Available  at  http://www.cis.upenn.edu/~xtag/ 


Evaluation 


This  section  describes  a recent  evaluation  of  the  Task  Generator.  The  main  aim  in  this  study  is  to  evaluate  the 
accuracy  of  the  system  in  producing  learning  tasks  for  the  ESL  Tutor.  To  this  end,  the  accuracy  of  each  individual 
component,  as  well  as  the  overall  accuracy  of  the  system  is  evaluated.  Since  the  OOV  Filter  achieves  100% 
accuracy  (it  is  not  a difficult  task),  this  filter  was  excluded  from  the  system  for  the  purpose  of  the  evaluation.  This 
provided  a wider  range  of  input  to  the  later  parts  of  the  system. 


Procedure 

The  input  to  the  Task  Generator  was  7,600  words  from  13  texts  that  were  selected  by  an  independent  analyst.  The 
texts  covered  a range  of  subject  areas  from  personal  descriptions  to  family  finances  to  net  access  for  business 
travelers.  Ten  learning  objectives  were  selected  from  the  forty  we  have  currently  implemented.  These  were  selected 
to  include  objectives  for  a variety  of  parts  of  speech,  with  an  emphasis  on  the  inflecting  classes.  The  selected 
learning  objectives  are  listed  in  Table  1.  The  input  text  was  submitted  to  the  system,  once  for  each  of  the  selected 
learning  objectives. 


Any/Some/No  Words 

Comparative  and  Superlative  Adjectives 

Count  / Mass  Nouns 

Indefinite  Articles 

Prepositions 

Passive 

Present  Continuous 
Reflexives 

Singular  Pronouns  (masc.,  fern..,  neuter,  accus.,  nomin.) 

Verb  Agreement 

Table  1:  Learning  Objectives  Used  in  Study 


Results 

The  performance  of  each  system  component  can  be  found  in  Table  2.  The  accuracy  of  a component  for  a specific 
learning  objective  was  determined  by  considering  just  those  sentences  that  contained  the  learning  objective.  For 
example,  the  accuracy  of  the  sentence  boundary  detector  for  the  Passive  learning  objective  was  determined  by 
evaluating  all  of  the  sentences  that  were  identified  as  containing  a passive  construction.  The  percentages  specify  the 
percentage  of  sentences  which  were  analyzed  correctly  by  each  component.  The  Correct  column  specifies  the 
percentage  of  the  created  examples  that  are  correct.  The  Total  Produced  column  specifies  the  number  of  examples 
that  were  created  for  each  learning  objective  - there  is  an  upper  limit  of  150.  Finally,  the  Total  row  provides  the  total 
calculated  by  averaging  the  original  scalar  values  of  each  column  (for  reason  of  brevity  original  values  are  not 
provided  in  the  table.  They  can  be  calculated  from  the  information  provided  in  the  table).  For  example,  in  99.5%  of 
the  tasks  created,  the  Learning  Objective  Filter  performed  correctly. 


Discussion 

The  results  show  that  90.8%  of  the  examples  produced  were  correct.  In  8.2%  of  the  cases,  there  was  some  error  that 
was  caused  by  one  of  the  preceding  modules. 

By  far  the  largest  source  of  errors  was  the  sentence  boundary  identifier.  In  7.4%  of  the  created  examples 
there  was  an  incorrect  sentence  boundary.  This  comprised  80%  of  the  errors.  A review  of  the  data  found  that  the 
detector  we  are  using  appears  to  rely  heavily  on  periods  and  capitalization  as  an  end  of  sentence  identifier.  In  cases 
where  periods  do  not  frequently  occur,  such  as  at  the  end  of  headings,  a sentence  boundary  was  omitted,  leading  to 


problems  such  as  (3).  In  addition,  there  were  cases  with  a period  and  appropriate  capitalization  that  were  not 
identified  as  sentences.  Hence,  in  order  to  increase  our  overall  accuracy,  an  improved  sentence  boundary  identifier  is 
required. 


Sentence  Boundary 

Tagger 

LO  Filter 

Lexicon  Problem 

Correct 

Total  Produced 

AnylSomeINo  Words 

95.0% 

100.0% 

100.0% 

iob.0% 

95.0% 

40 

Comparative  & 
Superlative  Adjectives 

78.1% 

100.0% 

100.0% 

84.4% 

62.5% 

32 

CountIMass  Nouns 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

6 

Indefinite  Articles 

95.3% 

100.0% 

100.0% 

100.0% 

95.3% 

150 

Prepositions 

92.0% 

100.0% 

100.0% 

100.0% 

92.0% 

150 

Passive 

94.7% 

100.0% 

100.0% 

100.0% 

94.7% 

38 

Present  Continuous 

90.9% 

100.0% 

100.0% 

100.0% 

90.9% 

11 

Reflexives 

85.7% 

100.0% 

100.0% 

100.0% 

85.7% 

7 

Singular  Pronouns 

94.0% 

100.0% 

98.0% 

100.0% 

92.0% 

150 

Verb  Agreement 

84.0% 

76.0% 

100.0% 

100.0% 

72.0% 

25 

Total 

92.6% 

99.0% 

99.5% 

99.2% 

90.8% 

609 

Table  2:  Results 


(3)  Effective  vs.  Ineffective  One  way  to  distinguish  effective  from  ineffective  employees  is  in  how  they  act  and 
react  to  the  demands  of  their  positions. 

The  remaining  modules  each  achieved  very  high  accuracy.  The  tagger  performed  extremely  well,  except  in  the  case 
of  the  verb  agreement  category.  In  this  case,  the  tagger  occasionally  incorrectly  tagged  a noun  as  a verb.  One 
example  is  the  input  given  in  (4).^  In  this  case  the  noun  Hikes  was  tagged  as  a verb.  This  is  not  surprising  since  the 
input  is  not  a complete  English  sentence.  This  type  of  error  accounted  for  three  of  the  six  tagging  errors.  In  order  to 
reduce  this  type  of  error  it  will  be  necessary  to  include  filter  mechanisms  that  exclude  input  items  which  are  not 
complete  sentences.  The  existence  of  an  increased  proportion  of  punctuation  and  capitals  may  provide  sufficient 
indication  of  this. 

(4)  CALIFORNIA  Great  Day  Hikes  In  & Around  Napa  Valley  (Ken  Stanton,  Bored  Feet  Publications, 
Mendocino,  CA,  1995,  707/964-6629;  $12). 

The  Learning  Objective  Filter  also  performed  very  well:  the  only  source  of  error  was  with  the  singular  pronoun 
filter.  In  three  cases  this  filter  incorrectly  identified  cases  like  (5)  as  containing  an  example  of  the  singular  pronoun 
it.  In  each  case  the  actual  word  was  an  acronym.  A solution  to  this  problem  is  to  consider  capitalization  as  a factor  in 
the  learning  objective  filters.  At  present,  the  filters  only  place  constraints  on  information  that  is  present  in  the  lexical 
entry  for  this  word. 

(5)  The  most  significant  case  is  that  of  Lois  Franxhi  , an  IT  manager  who  was  sacked  for  using  the  Internet 
during  her  lunch  break  to  track  down  a bargain  holiday. 

The  final  source  of  error  in  our  system  is  the  lexicon  itself.  Firstly,  the  lexicon  incorrectly  identifies  modest  as  a 
superlative  adjective.  An  example  is  given  in  (6a).  Secondly,  the  lexicon  incorrectly  lists  the  base  form  of  lower  as 
lower  instead  of  low.  An  example  in  (6b).  These  two  errors  accounted  for  all  of  the  five  lexicon  problems. 

(6a)  They  tend  to  defer  to  others,  are  less  likely  to  take  risks,  are  [ modest  (superlative  adjective)  ] and  cautious, 
and  try  to  avoid  conflict. 

(6b)  People  with  [ lower  (comparative  adjective)  ] dominance  are  congenial  and  cooperative. 


^ This  example  also  has  an  incorrect  sentence  boundary.  The  CALIFORNIA  is  a heading. 


We  are  very  encouraged  by  these  results.  With  the  exception  of  the  examples  influenced  by  the  sentence  boundary 
detector,  the  Task  Generator  is  highly  accurate.  In  particular,  the  shallow  approach  to  analyzing  the  sentence  has 
proved  sufficient  for  the  purposes  of  producing  learning  tasks  for  a specific  learning  objective. 

Our  analysis  of  the  data  indicated  several  areas  for  improvement.  Firstly,  the  sentence  boundary  detector 
needs  to  be  significantly  improved.  The  detector  needs  to  be  able  to  deal  with  text  in  context  i.e.  text  that  includes 
components  such  as  headers.  Secondly,  an  additional  module  is  required  in  order  to  filter  sentences  that  have  an 
unduly  high  proportion  of  capitals  and  punctuation.  This  should  reduce  the  number  of  tagging  problems.  Thirdly,  we 
need  to  update  our  language  for  stating  learning  objectives  so  that  word  capitalization  can  be  used  as  a constraint. 

Having  established  the  reliability  of  the  Task  Generator,  a next  task  is  to  integrate  the  system  into  our  ILTS 
authoring  tool.  This  will  allow  us  to  conduct  studies  with  real  users  of  the  system.  Our  goal  will  be  to  establish  to 
what  degree  the  Task  Generator  affects  the  authoring  process. 

A further  goal  is  to  establish  the  portability  of  the  system  by  porting  it  to  German.  While  the  language- 
specific  resources  must  be  replaced,  it  is  expected  that  the  rest  of  the  system  should  need  no  modification. 


Conclusion 

This  paper  introduced  the  Task  Generator,  a component  of  an  authoring  tool  for  intelligent  language  tutoring 
systems.  The  Task  Generator  automatically  creates  learning  content  from  existing  texts.  In  order  to  ensure 
portability,  the  system  is  designed  to  require  minimal  linguistic  resources.  The  current  version  of  the  system  requires 
a sentence  boundary  detector,  a POS  tagger,  and  a lexicon  with  syntactic  information.  The  results  show  these 
resources  are  sufficient  to  automatically  produce  learning  content  for  a range  of  learning  objectives. 
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Abstract:  This  paper  summarizes  a study  conducted  with  Faculty  of  Education  students 
preparing  to  teach  JuniorAntermediate  Visual  Arts.  The  study  investigates  the  potential 
and  issues  of  computer-based  systems  to  present  and  discuss  theoretical  material 
within  a particular  community  context.  The  results  suggest  that  active  participation  in 
threaded  discussion  is  predominantly  driven  by  reward  motives  such  as  grades,  and 
that  students  do  not  normally  participate  beyond  the  minimum  course  expectations. 
Passive  participation  in  the  discussion  areas  such  as  browsing  and  reading  was 
difficult  to  evaluate  and  document.  Findings  indicate  that  discussion  tied  to  real-world 
experiences  have  a greater  level  of  participation. 


Introduction 

A learning  community  is  a group  of  individuals  engaged  intentionally  and  collectively  in  the 
transaction  or  transformation  of  knowledge.  Building  an  online  learning  community  is  not  an  organizational 
problem — the  problem  is  one  of  motivating  participants  to  create  a community  and  giving  them  an  opportunity 
to  do  it  (Schwier,  1999).  Clark  (1999)  uses  the  apt  term  “growing”  in  place  of  “building”  or  “constructing”  to 
emphasize  the  organic  nature  of  this  type  of  work.  Research  on  online  learning  communities  has  established  the 
significance  of  providing  an  appropriate  structure  for  discussion.  Once  such  structure  is  the  threaded  discussion 
database,  which  facilitates  an  ongoing  exchange  of  topical  information  among  students  enrolled  in  a course.  For 
this  particular  study,  a threaded  discussion  database  was  created  to  support  a thought -pro voking  and  offline  art 
project  with  the  expectation  that  the  addition  would  create  observable  differences  in  the  responses  of  the 
community. 


The  Study 

The  community  for  this  study  consisted  of  179  Junior/Intermediate,  Faculty  of  Education  students 
enrolled  in  a Visual  Arts  course  as  part  of  a traditional  B.Ed.  program.  Each  student  had  completed  a three-  or 
four -year  undergraduate  degree.  The  class  list  indicated  that  the  sample  group  was  divided  into  107  females  and 
72  males.  Thirty  of  the  127  students  were  Visual  Art  majors. 

A Web  browser  was  the  only  required  interface  for  the  students.  The  students  interacted  throughout  the  school 
year  with  three  discussion  databases  using  text  format  only.  The  first  threaded  discussion  assignment.  Readings 
Discussion,  instructed  the  students  to  initiate  a discussion  related  to  one  of  the  assigned  readings.  Each  student 
was  also  responsible  to  read  the  postings  by  other  students  and  to  respond  to  at  least  one  posting.  The  second 
threaded  discussion,  Ask  an  Artist,  was  compulsory  only  for  the  30  art  majors  and  was  available  to  the  non- art 
majors  as  a resource  area,  where  they  could  ask  the  art  majors  and  instructor  specific  questions  to  assist  with 
teaching  art  during  their  practice-teaching  sessions.  The  third  threaded  discussion.  Stick  Project,  was  set  up  as  a 
crossover  project  with  B.Ed.  students  taking  a Language  Arts  class.  The  same  level  of  participation  was 
required  for  this  project  as  for  the  Readings  Discussion,  a minimum  of  one  posting  and  one  response. 


Findings 


A minimal  level  of  participation  by  the  students  was  ecpected  due  to  the  heavy  course  load  of  the 
program  and  practice  teaching  sessions.  The  access  logs  on  the  server  indicated  the  number  of  times  the 
database  was  accessed,  but  provided  no  true  indication  of  level  of  participation.  The  results,  however,  indicate  a 
higher  level  of  interest  in  reading  rather  than  posting.  The  higher  level  of  “lurking”  to  posting  corresponds  with 
the  instructor’s  observation  that  the  majority  of  his  students  did  not  feel  comfortable  participating  in  threaded 
discussions.  All  students,  regardless  of  their  Visual  Arts  background,  demonstrated  a lack  of  confidence  in  the 
two  databases  that  required  participation  for  course  credit.  The  students  met  only  the  required  number  of  posts. 
In  their  messages,  they  expressed  concern  about  whether  their  answers  were  right  and  appreciation  for  the 
examples  of  good  submissions  provided  by  the  instructor.  The  instructor  noted  that  the  number  of  quality 
submissions  exceeded  his  expectations  for  the  Stick  discussion,  but  not  for  the  other  two  databases.  A survey 
distributed  to  students  indicates  that  the  majority  of  them  agreed  that  the  Stick  crossover  discussion  allowed 
more  room  for  creativity  and  individuality. 


Conclusions 

• Threaded  discussions  should  be  structured  to  be  very  simple.  One  way  to  do  this  is  to  create  multiple 
discussion  areas;  for  example,  one  for  each  project  in  the  course,  or  one  for  each  week  in  the 
curriculum.  The  instructor  should  state  clearly  to  students  the  protocol  for  using  the  discussion  areas 
(minimum  number  of  posts,  participation  grades,  timelines,  etc.). 

• The  ratio  of  one  instructor  to  many  students  creates  a situation  where  topics  or  directions  can  be 
overdone  very  quickly  by  the  students  and  become  stale.  As  part  of  the  protocol  for  online  discussion, 
the  instructor  should  clarify  for  students  that  they  should  not  contribute  to  a discussion  thread  unless 
they  have  something  of  value  to  add. 

• Discussions  on  topics  chosen  by  students  — for  example,  related  to  overcoming  the  challenges  of  their 
course  projects  - are  more  lively  and  productive  than  discussions  on  topics  required  by  the  instructor. 

• The  workload  on  the  instructor  to  provide  more  material  online,  monitor  discussions,  close  topics,  and 
provide  fresh  starting  points  is  substantial  compared  to  the  traditional  classroom.  Universities  should 
recognize  this  in  their  collective  agreements  and  compensate  accordingly. 
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Introduction: 

Today’s  children  are  raised  in  a generation  of  “digital  age”  (Hruskocy  et  al.  2000).  The  new  digital 
technologies  provide  faster  links  to  the  world  and  from  agriculture  to  medicine,  all  areas  are  implementing  technologies. 
Therefore,  this  age’s  children’s  future,  employment  is  dependent  on  how  much  they  can  implement  technology  skills 
and  transfer  those  skills  into  their  content  is  becoming  more  and  more  important  In  order  to  do  this  they  depend  on  their 
teachers.  That’s  why  we  have  to  look  into  the  barriers  in  using  technology. 

It  has  been  observed  that  the  schools  are  trying  their  best  on  the  hardware  part  byreplacing  the  old  machines 
with  the  new  ones  and  even  purchase  more  computers  whenever  they  have  the  budget.  On  the  other  hand,  it  has  been 
observed  that  technological  readiness  of  America’s  teachers  to  use  these  new  technological  tools  is  very  low.  According 
to  a 1999  National  Center  for  Educational  Statistics  Study  (in  Haselbring  et  al,  2000),  only  20%  of  all  teachers  feel  very 
well  prepared  to  integrate  technology  into  their  teaching.  As  Ertmer  mentions  (1999),  although  teachers  recognize  the 
importance  of  integrating  technology  into  their  curricula,  they  are  limited.  In  the  literature  these  limitations  are  called  as 
internal  and  external  barriers.  Although  the  barriers  are  common  in  other  studies,  they  are  not  always  classified  as 
Internal  and  External  Barriers.  However,  I find  that  it  would  be  a good  idea  to  explain  them  under  these  two  headings. 
Ertmer  (1999),  also  mentions  that  training  programs,  recently,  have  incorporated  pedagogical  models  of  technology  use 
as  one  means  of  addressing  internal  barriers.  If  pre-  and  in-service  teachers  are  to  become  effective  users  of  technology, 
they  will  need  practical  strategies  for  dealing  with  the  different  types  of  barriers  they  will  face. 

As  mentioned  above,  the  barriers  are  not  always  classified  under  the  two  headings  in  all  studies  but  it  is  helpful 
to  classify  them  as  the  “internal”  and  “external”  barriers. 

External  Barriers : 

Means  and  Olson  (1997)  describe  external  barriers  as  the  types  of  resources  like  equipment,  time,  training  and  support 
that  are  missing  or  not  provided  adequately  in  teachers’  implementation  environments.  External  barriers  to  technology 
integration  are  described  as  being  extrinsic  to  teachers  and  include  lack  of  access  to  computers  and  software, 
insufficient  time  to  plan  instruction,  and  inadequate  technical  and  administrative  support  (Ertmer,  1999).  She  also  adds 
that  teachers  have  to  acquire  technical  skills  needed  to  operate  a computer.  Dias  (1999)  also  mentions  the  importance  of 
external  barriers.  He  says  that  the  most  common  barriers  include  time,  training,  resources,  and  support.  Teachers  need 
time  to  learn  how  to  use  both  the  hardware  and  software,  time  to  plan,  and  time  to  collaborate  with  other  teachers. 
According  to  him  another  concern  is  training.  Some  educators  do  not  have  local  training  options  available  or  the  time  to 
attend  training.  There  is  also  the  problem  of  lack  of  resources.  Without  computers  in  the  classroom  and  appropriate 
software  to  support  the  curriculum,  integration  cannot  take  place.  Support  is  critical  as  well.  Lack  of  leadership, 
financial  support,  or  an  on-site  technology  expert  sends  many  integration  efforts  into  a tailspin.  However,  there  is  a very 
important  factor  that  should  always  be  kept  in  mind  especially  when  dealing  with  technology  and  that  is  “time”  and 
“change  in  time”. 

Internal  Barriers: 

Brickner  (1995)  defines  internal  barriers  as  the  barriers  that  interfere  or  impede  fundamental  change.  They  are 
intrinsic  to  teachers  and  include  beliefs  about  teaching,  beliefs  about  computers,  established  classroom  practices,  and 
unwillingness  to  change  (Ertmer,  1999)..  Kerr  (1996)  adds  that  these  are  rooted  in  teachers’  underlying  beliefs  about 
teaching  and  learning  and  may  not  be  realized  by  others  or  even  by  the  teachers  themselves  for  sometime.  Internal 
barriers  change  fundamental  beliefs  about  current  practice,  thus  leading  to  new  goals,  structures,  or  roles  (for  example. 


electronically  conversing  with  an  author  to  explore  the  cultural  and  political  context  of  a story  rather  than  writing  a book 
report  summary).  Current  literature  shows  that  internal  barriers  are  common  among  today’s  teachers  (Hannafin  & 
Savenye,  1993;  Kerr,  1996;  Riedl,  1995). 

Suggestions  to  overcome  the  barriers: 

Here  are  some  suggestions  that  are  compiled  from  different  articles  that  I read: 

• Professional  preparation  programs  should  continue  to  emphasize  technology  skills  preparation  within  their  pre- 
service  curriculum.  Special  attention  should  be  given  not  only  to  use  of  computers  in  administrative  tasks  but 
should  include  how  effectively  use  technology  to  enhance  instruction.  (Dorman,  2001) 

• Faculty  in  schools  or  colleges  of  education  preparing  educators  should  integrate  technology  use  within  their 
professional  preparation  courses  by  modeling  appropriate  use  of  technology.  (Hruskocy  et  al,  2000) 

• Teacher  candidates  should  be  exposed  to  technology  based  curriculum  materials  and  products  within  their  pre- 
service  period.  (Abdal-Haqq,  1995) 

• Special  attention  should  be  given  to  field  placements  and  internships  to  ensure  students  are  placed  in  a 
technology -rich  environment  where  supervising  teachers  are  actually  using  technology  in  their  classes. 
(Hruskocy  et'al,  2000) 

• Experienced  teachers  may  lack  the  formal  training  to  feel  comfortable  using  the  computer  in  the  classroom. 
Therefore,  in-service  programs  featuring  use  of  technology  to  teach  about  their  content  should  be  provided. 
Where  possible,  teachers  should  be  given  release  time  and  support  to  develop  technology  skills.  Grooves  & 
Zemel,  2000) 

• Curriculum  planners  and  product  developers  should  consider  innovative  ways  to  enhance  the  learning  about 
their  content  by  using  technology  in  creative  ways.  (Hruskocy  et  al,  2000) 

• Technology  enhanced  curriculum  products  should  be  an  integral  part  of  a planned  approach  to  learning  about 
the  content.  This  effort  will  assist  teachers  to  use  technology  for  instructional  activities  in  addition  to  aiding 
with  administrative  tasks.  (Dorman,  2001) 

Most  of  this  section  comes  from  Byrom’s  article  (1998): 

heightened  abilities  to  collaborate  in  performing  tasks  He  adds  that  designating  appropriate  assessment 
strategies  helps  teachers  look  for  evidence  of  deeper  understanding,  statements  of  relationships,  synthesis,  and 
generalization  of  ideas  to  new  domains.  These  kinds  of  strategies  often  include  writing  samples,  student  products,  and 
portfolios.  (Byrom,1998) 

Conclusion: 

Technology  integration  into  teaching  is  one  of  the  goals  of  educators.  However,  teachers  today  face  with  various 
barriers  as  explained  above.  Although  there  has  been  some  research  done  on  barriers  in  using  technology,  the  field  still 
needs  more  research  as  technology  develops  so  fast  and  the  skills  and  the  hardware  that  are  used  change  so  quickly  as 
well.  In  order  to  catch  up  with  the  speed  of  this  technological  development,  professional  development  training  programs 
should  always  be  kept  up  to  date  and  be  continuous.  There  is  still  need  for  study  on: 

• teachers’  use  of  advanced  education  technology  and  barriers  in  using  them, 

• professional  development  and  technical  support  strategies  for  enhancing  teachers’  effective  use  of  technology 
On  the  other  hand,  the  biggest  challenge  for  most  of  the  teachers  is  to  learn  to  use  new  technology  tools  and  take 

major  steps  to  change  their  ^ 

classroom  practices. (Ertmer,  1999)  However,  we  can  overcome  these  barriers  if  teachers  are  well  prepared  against  them 
with  effective  strategies. 
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Abstract:  Experiments  are  an  important  part  in  technical  education  to  improve 
understanding  of  the  theoretical  content.  In  the  last  years  efforts  have  been  made  to 
provide  access  to  real  experiments  via  Internet  (Aktan  et  al.  1996;  Wagner  et  al.  2001). 
Unanswered  is  the  question  on  the  effects  of  learning  outcomes,  the  problem  solving 
process,  the,  acceptance  and  the  motivation  of  these  non  haptic  experiments  with  their 
reduced  focus  of  real  world  experience. 

In  this  paper  we  describe  the  didactic  concept  of  our  remote  lab  and  the  evaluation 
framework  for  an  empirical  study  which  aims  at  examining  the  effects  of  the  haptic 
component  on  problem  based  learning  in  remote  laboratories. 


The  Remote  Experiment 

In  our  remote  experiment  for  learners  of  process  control  at  technical  schools  the  learners  have  to 
develop  and  upgrade  a program  for  an  industrial  controller  (PLC)  that  controls  a process  engineering  plant. 
In  this  process  a fluid  can  be  filled  into  the  two  tanks  and  heated  or  mixed  in  the  reactor  below. 

The  learners  process  the  reaction  of  their  program  on  the  process  through  a remote  controlled  web 
cam.  To  access  the  lab  devices  the  learners  only  have  to  run  a web  browser. 


Didactic  concept  and  task  structure 

The  didactic  concept  in  our  remote  lab  course  is  orientated  on  problem  based  learning  (see  Norman 
1991).  The  experiment  is  done  in  small  groups  of  two  or  three  learners.  They  are  guided  through  the 
experiment  and  get  additional  information  from  a web  based  course.  The  course  is  structured  in  two  parts. 
The  first  part  is  divided  intro  three  subtasks  with  increasing  aspiration  level.  The  second  part  is  a thematic 
structured  knowledge  base  of  the  content  for  explorative  learning.  Designed  as  a guided  tour  through  the 
subject  matters  we  address  three  different  types  of  learners.  The  practically  inclined  learner  may  start  with 
an  example,  the  analytically  inclined  one  with  a description  of  the  structure  of  a PLC-program.  Learners 
who  already  have  gained  experience  with  PLC  program  development  may  choose  the  third,  the  quick  path 
being  offered. 

In  the  beginning  of  the  experiment  the  learners  have  to  find  an  error  in  a controller  program  {remote 
diagnostic  task).  They  have  to  read  the  data  sheets  of  the  actors  and  sensors  in  the  system  and  understand 
their  relation  to  the  given  program.  They  have  to  correct  the  program,  upload  it  to  the  controller  and 
examine  the  function  of  their  solution  at  the  plant  by  using  the  web  cam  (remote  maintenance  task).  During 
their  next  steps  they  have  to  upgrade  the  program  and  include  functions  additional  for  a local  operation  unit 
(see  figure  1)  and  a “notification  of  fault”  function. 


Evaluation  Approach 

To  evaluate  the  relevance  of  the  haptic  experiences  in  our  experiment  we  made  a comparative 
approach,  where  the  learners  execute  the  experiment  in  three  different  settings  (see  table  1). 


Setting  Description 

Local  learning  setting  The  conventional  learning  setting,  where  the  learners  and  the  experiment  are  in  one  room. 

Remote  learning  setting  The  group  of  learners  are  separate  from  the  experiment. 

Distributed  learning  setting  The  learners  are  distributed  and  collaborating  by  synchronic  tools  like  chat  or  video  conferencing. 
Table  1:  Variation  of  the  learning  environment 

All  groups  have  the  same  devices  (for  the  local  group  they  were  ‘real*,  in  the  other  two  setting  they 
were  ‘virtual*  for  the  groups)  and  the  same  lab  exercises.  But  only  in  the  local  learning  setting  the  students 
had  the  full  sense  of  impression  of  the  lab  devices  and  of  the  other  learners. 


Figure  1:  Local  operating  unit  and  Virtual  operating  unit 


Evaluation  Methods  and  Data  Acquisition 

Our  three  different  learning  settings  have  one  evaluative  question  in  common:  How  far  does  a loss  in 
haptic  experience  effect  the  outcomes  of  the  learning  process  to  be  achieved? 

To  measure  the  increase  in  knowledge  we  will  carry  out  short  pre-  and  post  tests.  To  evaluate  the 
quality  of  the  problem  solving  process,  we  will  observe  the  learners’  activities  during  the  experiment  by 
means  of  a self  developed  evaluation  tool  that  enables  the  evaluator  to  write  down  the  activities  of  each 
learners*  group  into  a database.  Additionally,  we  will  analyze  the  server  log  data  by  another  self  developed 
tool  to  trace  information  about  the  ways  a group  of  learners  took  through  the  web  based  course  and  for  how 
long  they  stayed  in  each  chapter.  At  the  end  of  the  lab  exercise  all  learners  will  have  to  answer  a 
questionnaire  gaining  subjective  information  about  their  acceptance  and  motivation  during  the  experiment. 

The  collected  data  will  give  us  the  possibility  to  compare  the  outcome  of  the  learning  process,  the 
structure  of  the  problem  solving  process  as  well  as  the  acceptance  and  motivation  in  the  three  setting. 


Conclusion 

Setting  up  remote  experiments  raises  many  questions  in  the  field  of  education.  One  of  the  main 
question  is:  How  the  loss  of  haptic  impression  change  the  learning  effect.  Special  areas  of  interest  are  the 
acceptance,  motivation,  learning  outcomes  and  quality  of  the  problem  solving  process.  Currently  we  are 
acquiring  data  in  three  classes  with  about  75  learners  in  vocational  schools.  Our  aim  is  to  find  out  if 
Remote  Labs  are  a suitable  way  for  problem  based  learning  compared  to  traditional  experiment  settings. 


References 

1 . Aktan  B.,  Bohus  C. A,, Growl  L.A,  and  Shor  M.H.  (1996).  Distance  Learning  Applied  to  Control  Engineering 
Laboratories^  IEEE  Transactions  on  Education,  Vol.  39,  No.  3,  August  1996. 

2.  Tuttas,  Jdrg;  Wagner,  Bernardo  (2001 ).  Distributed  Online  Laboratories^  International  Conference  on  Engineering 
Education  (ICEEOl),  Oslo,  Norwegian,  6.  -10.  Aug.2001. 

3.  Norman  GR,  Schmidt  HG  (1991).  The  psychological  basis  of problem  based  learning:  A review  of  the  evidence.  Acad. 
Med  1991,67:557-565. 


1E§T  COPY  AVAILABLE 


Evaluating  the  Effectiveness  of  Online  Courses 


Hakan  Tuzun 

Instructional  Systems  Technology  Department 
Indiana  University 
United  States 
htuzun@indiana.edu 


Abstract:  Evaluation  is  needed  to  justify  the  existence  of  online  courses  and  to  provide 
feedback  on  the  performance  of  them.  In  this  study,  Kirkpatrick *s  4-level  evaluation  model  is 
proposed  as  a framework  for  the  evaluation  of  online  courses.  The  model  and  procedures  are 
explained  for  collecting  and  analyzing  data  at  each  of  the  four  levels  for  the  ultimate  purpose 
of  the  evaluation  of  online  courses. 


Introduction 

Once  the  design,  development  and  implementation  of  an  online  course  are  completed,  it  is  time  to 
evaluate  it.  Evaluation  is  needed  to  justify  the  existence  of  the  online  course  and  to  provide  feedback  on  the 
performance  of  it.  Historically,  educational  perspective  has  treated  the  evaluation  as  the  measurement  of 
learning  (Hawthorne,  1987).  However,  in  the  corporate  environment,  evaluation  focus  has  been  on  the  enhanced 
individual  and  corporate  performance.  Evaluations  of  corporate  training  have  traditionally  asked  four  questions. 
The  purpose  of  this  study  is  to  evaluate  the  effectiveness  of  online  courses  by  adapting  these  four  questions 
asked  in  the  corporate  environment:  1)  What  the  learners  liked  and  disliked  about  the  online  course,  2)  The 
amount  of  learning  that  occurred  as  the  result  of  the  online  course,  3)  To  what  degree  learners  have  transferred 
the  course  attitudes,  skills,  and  knowledge  successfully  to  their  jobs,  and  4)  What  Return  on  Investment  (ROI) 
have  the  learners*  affiliates  received  due  to  participation  in  the  online  courses?  There  are  various  evaluation 
models,  which  can  answer  the  questions  above.  Kirkpatrick’s  model  will  be  used  for  this  study,  which  is  a 
widely  used  method  of  evaluating  corporate  training  programs  (Basarab  & Root,  1992).  Kirkpatrick  developed 
a series  of  levels  to  use  when  deciding  the  type  of  data  to  use  and  the  timeline  for  collecting  evaluation  data.  In 
his  model,  Kirkpatrick  calls  for  four  levels.  Levels  1 and  2 are  formative  and  levels  3 and  4 are  summative. 

Methodology 

Data  Collection  Procedures 

Level  1:  Reaction 

A standard  reactionnaire  form  completed  by  all  learners  will  gather  data  to  implement  level  1 
evaluation.  All  learners  will  complete  the  level  1 reactionnaire  form  prior  to  finishing  the  online  course.  The 
questions  used  on  the  reactionnaire  will  evolve  from  interviews  with  online  course  stakeholders  and 
observations  made  by  the  researchers.  Sample  questions  that  might  be  included  are:  1)  How  did  the  learners 
feel  about  the  quality  of  the  subject  content,  library  services,  the  online  learning  process,  and  assessment  tools? 
2)  How  did  the  learners  feel  about  the  sequence  of  the  materials?  3)  How  did  the  learners  feel  about  the  quality 
of  online  mentors’  availability,  answering  questions,  and  knowledge  of  subject?  4)  How  did  the  learners  feel 
about  the  course  objectives  as  they  relate  to  the  learners’  job  requirements,  and  the  applicability  to  the  learners’ 
job? 

Level  2:  Learning 

The  researchers  will  look  for  answers  to  the  following  questions  at  this  level:  1)  By  learning  objective, 
what  skills  and  knowledge  were  acquired  in  the  online  course?  2)  What  amount  of  learning  occurred  after  the 
online  course?  A pre-test  and  a post-test  will  be  admin  istered  for  the  online  course  to  evaluate  the  amount  of 
learning  gained  from  the  online  course  in  the  short  term.  Online  course  developers  will  create  the  pre-test  and 
post-test  instruments. 

Level  3:  Behavior 

Since  the  learners  cannot  apply  what  they  learned  until  they  have  an  opportunity,  the  measurement  of 
transfer  of  knowledge,  skills,  and  attitudes  can  only  be  done  in  the  long-term.  The  researchers  will  look  answers 
to  the  following  questions  at  this  level:  1)  How  many  times  have  the  learners  applied  course  objectives  on  the 
job?  2)  What  are  the  reasons  for  not  applying  the  behavior  on  the  job?  3)  How  the  online  course  impacted  the 
learners’  job  performance?  4)  What  skills  have  learners  had  to  learn  on  their  own  to  perform  a required  job 


behavior  that  the  online  course  did  not  provide?  To  answer  these  questions,  all  learners  will  complete  a post- 
course questionnaire  three  months  after  the  course’s  completion.  The  researchers  will  get  additional  information 
by  interviewing  the  people  around  the  participant  such  as  immediate  supervisors,  subordinates,  and  peers. 

Level  4:  Results 

For  online  courses,  which  don’t  have  tangible  results  and  therefore  a measurement  in  terms  of  dollars 
and  cents  is  not  possible,  the  researchers  will  observe  the  learners  in  the  worlq)lace.  When  performing 
cost/benefit  analysis  seems  logical,  the  researchers  will  still  observe  the  learners,  but  they  will  also  use  the 
causal  model  and  ROI  model  to  find  out  if  it  has  been  worthwhile  to  spend  money  on  the  online  course.  The 
researchers  will  look  answers  to  the  following  questions  when  the  casual  model  and  ROI  model  are  used:  1) 
What  was  the  cost  of  online  course  for  the  learners’  companies?  2)  What  value  the  companies  has  received  due 
to  the  online  course?  3)  What  is  the  ROI  from  the  online  course? 

Data  Analysis  Procedures 
Level  1:  Reaction 

The  researchers  will  summarize  data  from  reactionnaires  finding  an  average  mean  score  for  each  item. 
Patterns  will  be  detected  in  the  open-ended  questions. 

Level  2:  Learning 

Statistics  calculations  such  as  mean,  standard  deviation,  and  t-score  will  be  performed  on  pre-test  and 
post-test  scores  for  the  online  course.  Using  .01  level  of  significance,  t-scores  will  be  performed  on  pre-test  and 
post-test  results  to  see  if  the  learning  gain  from  online  course  is  a true  representation  of  learning.  The  percent  of 
gain  score  indicates  what  percentage  the  participant  has  learned  as  the  result  of  the  online  course.  After  gain 
scores  are  computed  for  each  participant  for  the  online  course,  the  average  percent  of  gain  score  for  that  online 
course  will  be  computed.  Item  analysis  of  pre-test  results  will  be  performed  to  evaluate  what  learners  knew 
before  entering  the  online  course.  Based  on  this,  recommendations  will  be  made  to  modify  course  objectives 
and  course  content.  Item  analysis  of  post-test  results  will  be  performed  to  evaluate  what  objectives  the  learners 
could  not  master.  Based  on  this,  recommendations  will  be  made  to  modify  course  content. 

Level  3:  Behavior 

The  researchers  will  compile  the  results  of  the  questionnaires  sent  to  the  learners.  To  do  this,  data  from 
questionnaires  will  be  summarized  finding  an  average  mean  score  for  each  item.  Patterns  will  be  detected  in  the 
open-ended  questions.  The  researchers  will  compile  the  results  of  the  interviews  made  with  the  people  around 
the  learners.  The  interview  records  will  be  transcribed  and  analyzed  to  compare  the  behavior  of  the  learners 
before  and  after  the  online  course. 

Level  4:  Results 

The  researchers  will  analyze  the  observation  records  of  learners  to  detect  end  results  gained  by  the 
company.  If  the  casual  model  and  ROI  model  are  used,  the  mean  of  sales/income  for  the  experimental  group 
and  the  control  group  will  be  calculated.  The  mean  of  the  control  group  sales /income  will  be  subtracted  from 
the  mean  of  the  experimental  group  sales  to  find  the  value  of  training.  This  will  be  divided  by  the  cost  of  the 
online  course  to  find  the  return  rate  spent  on  each  dollar.  Due  to  lack  of  pure  experimental  design,  the 
researchers  will  make  judgments  after  the  comparison  of  the  data,  and  will  not  try  to  prove  its  statistical 
significance. 

Conclusions 

The  data  collection  and  data  analysis  techniques  described  above  are  generated  based  on  the  key 
characteristics  and  assumptions  of  Kirkpatrick’s  evaluation  model.  The  described  procedure  does  not  yet  have  a 
strong  empirical  foundation  to  support  it.  Empirical  work  is  needed  to  understand  if  the  proposed  model  works. 
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Abstract:  In  this  study  instructor  support  problems  related  to  the  Web-Based  Instruction 
(WBI)  were  identified  after  examining  their  experiences.  It  was  found  that  the  instructors  need 
support  in  the  area  of  hardware,  software,  design  of  the  course,  and  handling  student  requests. 
In  addition  to  this,  it  was  also  found  that  web-based  course  instructors  will  benefit  from  the 
knowledge  management  support,  peer  support,  support  for  cooperation  with  school  technology 
services,  and  web-based  related  research  support.  At  the  end,  we  propose  a model  for  a 
support  center  for  web-based  course  instructors. 


Introduction 

Using  the  Internet  for  different  purposes  has  entered  a revolution  during  last  two  decades.  Especially, 
after  the  development  of  the  hyperlink  on  the  World  Wide  Web  (WWW),  the  Internet  has  offered  more  user- 
friendly  environments  (Starr,  1997).  Researchers  indicated  that  WWW  is  not  only  a communication  medium  for 
e-mail  and  document  distribution  but  it  is  also  a place  to  learn  (Lightfoot,  1999;  Mioduser,  Nachmias,  Lahav,  & 
Oren,  2000).  With  the  combination  of  the  specifically  designed  software  and  pedagogy,  WWW  can  provide  an 
educational  environment  that  maintains  the  knowledge  building  approach  to  learning.  These  understanding, 
technological  developments  in  communication  and  WWW  have  been  used  as  new  opportunities  for  delivering 
instruction  online.  Thus,  web-based  distance  learning  has  emerged  as  an  alternative  approach  to  education  in 
the  last  few  years  (Yellen,  1998).  Especially  for  graduate  students,  universities  have  started  to  offer  online 
courses  (Barnard,  1997;  Duchastel,  1997;  Kearsley,  Lynch,  & Wizer,  1995).  These  practices  in  new  educational 
agenda  initiated  some  questions  on  researchers’  mind.  Support  in  web-based  courses  is  one  of  these  questions. 

Researchers  examined  the  support  issue  in  web-based  courses  usually  from  the  perspectives  of 
distance  education  students.  Paneitz  (1997)  made  a research  to  assist  community  college  personnel  in  the 
development  of  relevant  student  support  services  for  web-based  coursestudents.  She  found  academic  advising 
as  the  most  essential  student  service  for  students . Other  essential  student  services  were  found  as  access  to 
library  and  media  services  and  counseling.  Hudson  (1999)  equalized  the  progress  of  a web-based  course  with 
supporting  the  learner.  The  support  for  their  adult  learners  included  an  on-site  orientation  at  the  beginning  of 
each  semester,  a support  CD-ROM  developed  by  school’s  distributed  learning  office,  student  peer  assistance 
through  a help  forum,  teaching  interns,  and  paid  graduate  assistants. 

Lever-Duffy  (1992)  stated  that  a WBI  program  will  not  be  functional  without  proper  support.  They 
indicated  the  support  for  instructors  is  a prerequisite  for  interactions  between  the  instructors  and  the  students. 
Their  project  recognized  the  need  for  support  in  four  specific  areas:  technical  support,  technology  training 
support,  instructional  design  support,  and  staff  support.  Technical  support  included  such  tasks  as  network 
maintenance,  hardware  installation  and  repair,  software  installation  and  maintenance,  and  helping  through  an 
electronic  help  desk.  A qualified  technical  staff  specialized  in  each  of  these  areas  provided  the  technical 
support.  Technology  training  support  serves  for  two  kind  of  audience:  students  and  instructors.  The  technology 
training  reduces  the  threat  and  interference  of  the  technology  for  both  kind  of  audience.  Instructional  design 
support  is  required  to  design  web-based  courses  which  differ  from  residential  courses  in  the  unique  delivery 
style.  Finally,  the  staff  support  facilitates  the  communications  and  trades  between  the  department  and  distant 
student  in  terms  of  administration.  Secretarial  and  clerical  staff  can  handle  information  flow  and  clerical  tasks. 


Para-professional  staff  can  assist  instructors  with  student  load.  The  research  published  by  Lever-Duffy 
identifies  the  most  detailed  information  to  date  related  to  the  support  of  web-based  course  students  and 
instructors.  However,  she  did  not  provide  information  on  how  she  has  foreseen  the  support  in  four  specific 
areas. 

There  are  also  some  research  papers  in  the  literature  that  included  minor  information  about  supporting 
the  web-based  course  instructors.  For  example,  Kuchinke  et.  al.  (2001)  received  support  from  a technical 
support  team  consisted  of  six  halftime  staff  with'expertise  in  web  design  and  development.  A full-time 
academic  professional  directed  the  support  team.  While  Friedrich  and  Armer(1999)  developed  a web  site  for  a 
graduate  course  in  statistics  and  measurement,  an  instructional  designer/developer  was  assigned  to  each  faculty 
member  on  a part-time  basis  to  help  with  the  technical  aspects  of  the  course.  According  to  the  authors, 
universities  should  support  web-based  course  instructors  by  providing  training  to  them.  This  training  can  inform 
the  instructors  about  the  types  of  technologies  that  exist  and  provide  information  related  to  the  instructional 
design  and  development.  Some  studies  mentioned  lack  of  instructor  experience  in  WBI  as  a problem  (Lightfoot, 
2000;  Friedrich  & Armer,  1999).  Some  authors  recommended  creating  a permanent  instructional  design  team 
and  technical  support  team  to  overcome  this  problem  (Ingebritsen,  Brown,  & Pleasants,  1997;  Kuchinke  et.  al., 
2001).  Ingebritsen  et  al.  (1997)  utilized  a resource  center  for  the  development  of  their  biology  course  on  the 
Internet.  The  purpose  of  this  center  was  ‘to  assist  faculty  in  the  development  of  on-line  courses  by  providing 
technology  resources,  technical  assistance  and  training’. 

Motivation  for  the  Work 

The  review  of  the  literature  reveals  that  there  is  not  much  information  in  the  literature  on  the  topic  of 
supporting  web-based  course  instructors.  The  purpose  of  this  study  is  twofold:  to  present  the  support  needs  of 
the  instructors  in  web-based  courses  and  to  develop  a support  center  model.  The  results  might  help  those 
actively  involved  in  educational  use  of  the  Internet  research,  those  participating  in  the  development  of  web- 
based  courses , and  finally  those  who  are  teaching  web-based  courses. 


Methodology 

Research  Questions 

Is  there  a need  for  a support  center  for  web-based  distance  courses? 

What  kind  of  support  do  web-based  course  instructors  need? 

What  are  the  functions  of  a support  center  built  for  web-based  course  instructors? 

Participants  and  Context 

In  this  study,  two  instructors  from  a large  mid -western  university  were  interviewed.  Instructors  were 
selected  with  purposeful  sampling.  Since  the  nature  of  this  study  is  qualitative,  participants  were  selected 
according  to  their  previous  experience  with  the  WBI  to  get  deep  and  broad  understanding  about  their  support 
problems  related  to  WBI.  It  was  required  for  the  instructor  participants  to  teach  at  least  one  Web-based  course. 
Instructors  were  the  members  of  the  same  university,  but  none  of  the  participants  had  a relationship  through  the 
courses. 

Instructor!  was  a non-native  speaker  of  English  and  he  was  an  Al  in  the  Informatics  department.  He 
taught  a web-based  course  three  years  ago.  The  students  took  this  course  within  another  country  while  the 
instructor  was  in  US.  The  course  was  related  with  courseware  design  for  computer  mediated  learning.  It  was  at 
the  graduate  level  and  10  students  took  the  course.  The  course  was  offered  in  an  instructional  technology 
department.  There  was  another  instructor  for  the  course. 

Instructor2  was  a native  speaker  of  English  and  she  was  a faculty  in  an  Instructional  Systems 
Technology  (1ST)  department.  She  had  already  finished  the  first  nine  weeks  of  her  web-based  course  when  the 
interview  was  conducted.  Instructor2  taught  2 courses.  One  of  the  courses  aimed  to  provide  an  introduction  to 
the  field  and  profession  of  instructional  technology.  The  other  course  provided  information  on  the  instructional 
design  process.  Both  courses  were  at  the  graduate  level  and  they  were  part  of  an  online  master’s  program.  The 
courses  were  offered  in  an  instructional  technology  department.  There  were  1 instructor  and  10  students  in  the 
first  course,  2 instructors  and  18  students  in  the  other  course. 
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Data  Collection  and  Data  Analysis 

Semi  -structured  interview  questions  were  used  to  collect  interview  data.  Interviews  were  conducted 
over  a one  month  period.  Each  interview  session  was  treated  as  an  individually  constructed  discourse  between 
the  researchers  and  the  participant.  Both  open  ended  and  probing  questions  were  used  to  get  deep  information. 

The  study  called  for  an  in-depth  understanding  of  the  experiences  of  participants  involved  in  WBI. 
Instructors’  data  were  analyzed  separately  from  each  other.  All  data  were  transcribed  from  audiotapes  for 
analysis.  Then,  researchers  struggled  to  understand  the  context,  discourse,  and  meaning  behind  the  participant 
responses  to  determine  the  main  areas  about  which  respondents  have  problems  with  support  in  WBI.  To 
increase  the  credibility  of  the  study,  participants’  responses  were  coded  by  both  researchers  separately.  Later, 
common  themes  accepted  as  the  emergent  categories.  In  addition,  the  data  obtained  from  the  literature  was  used 
for  detailed  interpretation  of  the  results. 


Findings 

1.  Is  There  a Need  for  a Support  Center  for  Web-Based  Distance  Courses? 

After  the  interviews  it  was  clear  that  without^adequate  instructor  support,  web-based  courses  would 
fail.  The  instructor  of  web-based  course,  one  way  or  other,  will  either  find  the  required  support  for  his  problem, 
or  there  will  be  problems  in  the  web-based  course.  This  result  is  also  aligned  with  Lever- Duffy’s  (1992) 
findings  in  that  support  for  web-based  course  instructors  is  a vital  element  for  the  success  of  the  online  courses. 
Both  of  the  instructors  indicated  that  they  did  not  have  a support  center  for  the  instructors  of  web-based  courses. 
They  both  agreed  that  the  lack  of  instructor  support  created  major  problems  for  their  courses: 

Instructor  I : The  biggest  problem  was  the  support  problem.  Since  I was  alone  as  the  instructor,  I had 
to  arrange  everything. 

Instructor!:  Every  week  there  is  a new  problem,  which  really  points  to  the  fact  that  there  is  a lack  of 
support  for  faculty  for  teaching  online.  ...  Lack  of  support  for  faculty  teaching  in  online  environments 
is  a huge  problem. 

Also,  the  instructors  dwelled  upon  the  importance  of  a support  center  for  web-based  courses: 
Instructor! : Dealing  with  both  the  technical  aspects  of  the  course  and  content  aspects  of  the  course  is 
an  exhausting  process.  Personally,  I prefer  to  teach  in  an  environment  where  there  is  a support  for 
non-teaching  tasks.  Because,  giving  feedback  to  students,  getting  together  online,  and  grading  the 
assignments  are  already  very  time  consuming.  If  you  add  technical  things  on  top  of  these,  you  are 
pretty  swamped. 

Instructor!:  We  need  training  and  support  for  faculty  teaching  in  an  online  environment.  The 
combination  of  3 issues,  lack  of  faculty  training,  lack  of  faculty  support,  and  a design  process  that 
lacks  faculty  buy-in,  are  the  reasons  for  a lot  of  problems  that  are  beginning  to  emerge  [in  our 
distance  master  *s  program]. 

2.  What  Kind  of  Support  Do  Web-Based  Course  Instructors  Need? 

The  support  needs  of  web-based  course  instructors  will  be  summarized  in  the  following  categories: 
where  to  get  support  from,  support  for  hardware,  support  for  software,  support  for  design  of  the  course,  and 
support  for  handling  student  requests . 

2./.  Where  to  Get  Support  From 

When  the  instructors  got  into  problems  and  needed  support  to  solve  them,  they  tried  several 
mechanisms.  For  example,  instructor2  mentioned  1 1 instances  where  somebody  came  and  supported  her  with 
different  aspects  of  the  course.  These  aspects  included  design  support,  and  technical  support.  Below  are  2 
excerpts  from  the  instructor2  on  this  issue,  one  for  the  design  support  and  one  for  the  technical  support: 

Instructor!:  Even  though  the  documentation  was  written,  Dreamweaver  was  a brand  new  program  for 
me.  Thankfully,  [the  person  who  produced  the  course  with  dreamweaver]  came  in,  spent  about  15 
minutes  with  me  showing  the  basics  to  me.  And  that  helped  tremendously.  And  she  was  available 
during  that  period  for  a while.  If  I run  into  problems,  she  would  allow  me  to  call  and  [ask  for  support]. 
That  whole  process  went  fairly  smoothly  thanks  to  [this  person].  That  could  have  been  worse. 


Instructor2:  The  audio  problem  in  PowerPoint  was  a problem  that  I don* t even  think  design  team 
could  have  perceived.  It  has  taken  all  these  weeks,  just  for  [Personl]  and  [Person2]  working  together 
to  figure  out  how  to  fix  it. 

Instructor!  had  advanced  technical  skills.  For  that  reason,  he  did  not  need  much  external  support. 
However,  he  stated  that  the  situation  would  be  different  for  people  with  low  technical  background. 

Instructor!:  I didn  7 have  many  technical  problems,  but  the  reason  for  that  is  I have  a good  technical 
background.  However,  for  somebody  who  has  not  as  many  technical  skills  as  mine  will  suffer  a lot 
from  the  lack  of  technical  support. 

2.2.  Support  for  Hardware 

Instructor2  indicated  that  she  was  not  able  to  have  a Macintosh  computer  (which  she  was  used  to 
using)  at  the  beginning.  Because  of  the  lack  of  this,  she  had  a lot  of  communication  problems  with  the  students. 
She  had  to  provide  feedback  much  later  than  her  planned  time  frame. 

Instructor2:  I supposed  to  have  a Macintosh  computer  [when  I started  teaching].  And  there  was  a PC 
here,  so  I didn  *t  have  the  right  computer.  Later  I got  a Macintosh  computer  but  it  was  also  having  all 
sorts  of  problems. 

2.3.  Support  for  Software 

One  of  the  biggest  time  consuming  tasks  for  the  web-based  course  instructors  is  making  the  software 
feasibility  analysis: 

Researcher:  Did  you  have  any  problems  with  software? 

Instructor!:  I didn  7 have  a problem  with  using  the  software.  I had  to  look  for  software  functionally 
appropriate  for  the  course.  For  example,  the  kind  of  chat  software,  and  the  kind  of  whiteboard 
software  to  be  used  during  the  course.  There  were  several  freeware  software  available,  but  I had  to 
evaluate  them  to  see  their  powerful  and  weak  sides.  I could  test  the  software  one-on-one  with  someone, 
but  I had  no  idea  what  would  happen  when  !0  student  will  be  using  the  software. 

Instructor2:  Software  was  available  only  if  I took  the  time  to  track  it  down.  A would  be  nice  to  look 
beyond  the  tools  that  have  been  already  used  in  the  courses.  I know  there  are  better  tools  out  there. 
But  the  time  is  limited  to  find  them. 

Learning  new  software  also  creates  problems  for  instructors  if  they  learn  them  while  teaching  web- 
based  courses.  They  need  support  from  the  people  who  know  these  software: 

Researcher:  How  about  learning  new  software? 

Instructor2:  Yes,  because  I had  to  learn  Dreamweaver  and  I did  not  have  the  application,  so  I needed 
to  track  down  getting  the  application  from  one  of  my  friends.  Even  though  the  documentation  was 
written,  it  was  brand  new  program  for  me. 

2.4.  Support  for  the  Design  of  the  Course 

Instructor2  had  to  deal  with  the  re-design  of  the  course  during  the  course  delivery.  This  increased  her 
task  load  for  the  course  incredibly.  To  overcome  the  instructor  related  issues,  she  had  to  hire  someone. 
Otherwise,  she  would  not  be  able  to  keep  up  with  the  instructor  tasks.  She  explained  the  design  problems  as  the 
following: 

Instructor2:  ...  The  other  big  and  huge  problem  was  [eliminating  the  design  problems] . ...So,  you  can 
go  through  the  whole  week  then  look  how  to  fix  it.  That  has  been  very  time  consuming  for  me. 
Probably  more  time  consuming  than  any  other  thing  that  I have  done  for  the  course.  I have  been  doing 
the  redesign  of  the  web  pages  and  then  going  through  and  editing  the  lessons,  [and]  the  visual  parts  of 
the  lessons. 

Instructor2:  [The  time  spent  for  the  preparation  for  the  course  is]  absolutely  unreasonable.  I spent  30 
hours  every  weekend  just  doing  the  revisions,  so  that  lesson  could  be  ready  before  Monday  8:00am. 
That  even  doesn  7 count  the  other  times  during  the  week  that  I spent  on  [the  design].  It  is  taking  an 
unbelievable  amount  of  time. 

2.5.  Support  for  Handling  Student  Requests 

Both  instructors  utilized  support  personnel  hired  for  handling  student  requests: 


Instructor!:  On-site  technical  support  for  students  was  an  important  issue.  Therefore,  someone  was 
arranged  to  support  the  students  in  the  computer  lab.  For  example,  he  showed  people  how  to  use  the 
Paint  Shop  Pro  software,  and  how  to  use  the  scanner.  After  the  course  was  over,  we  concluded  that 
such  a support  for  handling  student  requests  was  important.  Lack  of  such  a support  would  increase  the 
load  of  the  instructor. 

Instructor!:  There  are  2 distance  master  graduate  assistants  who  handle  all  the  administrative  type  of 
things.  The  instructor  should  only  receive  issues  related  to  the  content  and  the  class.  Graduate 
assistants  handle  any  administrative  tasks  like  books,  registration,  technical  issues,  all  that  are  not 
supposed  to  be  instructor 's  problem. 

2.6.  Support  in  Other  Areas 

In  addition  to  the  five  areas  described  above,  it  was  also  found  that  web-based  course  instructors  either 
received  support  or  would  like  to  receive  support  in  the  area  of  knowledge  management,  peer  cooperation, 
cooperation  with  school  technology  services,  and  web-based  related  research.  Limited  space  prevents  giving 
more  information  on  these  issues. 

3.  What  Are  the  Functions  of  a Support  Center  Built  for  Web-Based  Course  Instructors? 

A Support  Center  Model 

Finding  support  when  they  are  in  need  is  the  biggest  problem  for  the  instructors  teaching  web-based 
courses . Since  they  do  not  know  what  kind  of  problems  they  will  have  during  the  course,  it  is  difficult  for  them 
to  be  prepared  against  these  problems  in  advance.  After  analyzing  the  interview  data,  we  came  up  with  a 
support  center  model  for  web -based  course  instructors.  This  model  is  shown  in  (Fig.  1). 


Figure  1:  Support  center  model  for  web-based  course  instructors 

In  this  model,  we  divided  the  siq)port  for  the  instructors  into  two  categories:  support  for  hard 
technologies  and  support  for  soft  technologies.  Support  for  hard  technologies  make  up  the  nucleus  of  the  model, 
because  hard  technologies  are  the  essential  elements  of  web-based  courses.  Hard  technologies  include  the  area 
of  hardware,  software,  and  network.  These  three  essential  elements  provide  the  backbone,  on  which  the  web- 
based  course  system  resides.  All  of  these  three  elements  have  equal  importance  since  lack  of  any  of  them  will 
result  in  failure  on  the  backbone.  In  sum,  providing  support  for  all  of  these  three  elements  is  essential.  Although 
the  model  is  open  to  development,  we  expect  no  major  modification  to  the  nucleus  of  the  model  in  the  near 
future. 

The  two  circles  around  the  center  constitute  the  support  for  soft  technologies.  We  define  soft 
technologies  as  a combination  of  all  elements  that  go  outside  the  nucleus.  The  inner  circle  contains  essential 
soft  technology  elements  to  be  supported.  These  elements  are  support  for  design,  support  for  orientation,  and 
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support  for  handling  student  requests.  When  support  for  any  of  them  is  missing,  there  is  a high  possibility  that 
the  web-based  course  will  fail.  The  dashed  nature  of  the  inner  circle  represents  the  fact  that  the  model  is  open  to 
development,  and  items  might  move  away  from  the  inner  circle,  or  new  items  might  be  added  to  the  inner 
circle.  The  outer  circle  contains  non-essential  soft  technology  elements  to  be  supported.  These  elements  are 
support  for  knowledge  management,  peer  support,  support  for  cooperation  with  school  technology  services,  and 
support  for  web-based  related  research.  It  is  useful  to  have  support  for  these  items.  When  support  for  any  of 
them  is  missing,  the  web-based  course  will  not  fail.  The  dashed  nature  of  the  outer  circle  represents  the  fact  that 
the  model  is  open  to  development,  and  items  might  move  away  from  the  outer  circle,  or  new  items  might  be 
added  to  the  outer  circle.  The  two-way  arrows  also  represent  the  openness  of  the  model  to  further  development. 
They  show  that  soft  technology  support  elements  in  the  circles  might  become  essential  or  non-essential  or  even 
obsolete  in  time.  Or,  new  elements  might  be  added  as  soft  technology  support  elements.  We  think  that  the 
future  modification  and  development  of  the  model  depends  on  the  changing  needs  of  instructors  in  parallel  to 
the  advancements  and  changes  in  Web-based  course  technologies,  and  also  to  the  roles  of  instructors  in  web- 
based  course  environments. 


Conclusions 

For  web-based  course  instructors,  support  should  be  provided  on  time  and  when  it  is  needed.  Hard 
technologies  are  the  only  way  to  communicate  with  students,  which  is  essential  for  web-based  instruction.  If 
instructors  have  difficulty  in  obtaining  the  support  on  time,  their  communication  with  students  can  be 
disastrous.  Students  can  easily  drop  the  course  just  because  of  this.  The  designers  of  web-based  courses,  and 
especially  the  coordinators  of  web-based  courses  should  understand  that  supporting  the  instructors  is  as 
important  as  supporting  the  students.  The  following  phrase  emphasizes  the  importance  of  this  issue: 

Instructor!:  I probably  had  100  times  problems  the  students  had. 

The  proposed  support  model  for  web-based  course  instructors  provides  a framework  for  supporting  the 
instructors.  It  is  up  to  the  coordinators  and  designers  of  the  web-based  courses  to  implement  the  model 
formally.  For  example,  the  software  support  may  be  provided  by  a specialist  or  by  an  electronic  performance 
support  system  (EPSS).  Most  probably,  available  resources  will  impact  the  form  of  the  components  of  the 
model. 
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Abstract:  Believing  that  the  essence  of  user-based  design  lies  in  the  fundamental  power  structure 
between  human  and  computer,  this  study  examines  the  effect  of  computers  that  offer  an  apology  to 
users  whenever  the  users  fail  to  accomplish  what  they  intend  to  do.  It  is  hypothesized  that  users 
will  feel  better  about  themselves  and  the  program  as  well  when  the  computers  shift  the 
responsibility  of  failure  from  users  to  computers.  In  this  pilot  study,  a computer  program  was 
designed  to  play  a revised  “20  questions”  game  with  four  subjects,  who  were  randomly  assigned  to 
2X2  treatment  groups  (difficult  question,  easy  questions,  apologetic- feedback  group,  functional- 
feedback  group).  Users  will  receive  either  an  ordinary  functional  feedback  or  an  apologetic 
feedback  whenever  they  fail  to  guess  the  answer.  The  tentative  result  supports  the  hypot  hesis,  and 
the  users  will  feel  better  even  if  they  are  aware  of  the  hypocrisy  of  the  apology. 


Introduction 

Nowadays,  as  “user-centeredness”  becomes  the  golden  rule  for  design,  it  is  time  to  re  -examine  the  power 
structure  between  human  and  computers.  Unless  the  imbalanced  status  of  power  between  computers  and  human  can 
be  reversed,  a computer  program  will  never  be  truly  “user-centered”.  The  first  and  the  most  important  step  to  change 
this  imbalance  is  to  have  computers  take  the  responsibility  and  apologize  for  its  inability  to  accommodate  users’ 
needs.  In  other  words,  while  computers  intend  to  be,  and  should  be,  designed  as  error-proof  for  users,  computers 
should  apologize  for  their  inability  to  deliver  this  or  whatever  promises  that  they  make  and  take  the  blame  for  a 
failed  transaction  between  them  and  the  users.  Not  only  because  it  is  the  courtesy  that  runs  our  human  society,  but 
also  because  any  design  should  be  aimed  at  making  users  feel  better  about  the  program  and  themselves.  Currently  in 
the  human-computer  interaction  literature,  many  studies  have  shown  that  instead  of  designing  complicated  program 
to  have  a computer  act  like  a human  and  express  emotions  (i.e.,  Anthromorphism),  we  often  project  our  emotion  onto 
computers  as  if  they  were  the  counterparts  at  the  other  end  of  the  emotional  interactions  (i.e.,  Ethopoeia).  That  is,  we 
have  the  tendency  to  treat  computers  as  social  actors,  a phenomena  called  “CASA  paradigm”  (Nass  & Steuer,  1993; 
Nass,  Moon,  Fogg,  Reeves,  & Dryer,  1995).  Following  the  idea  of  ethopoeia,  this  study  tries  to  investigate  the  effect 
of  applying  one  of,  if  not  the,  most  powerful  impression  management  tactic -apology -to  the  human-computer 
interactions.  In  this  study,  although  computer’s  apology  is  not  due  to  violation  of  a serious  felicity  condition  or  a 
mere  reflective  locution  for  a perfunctory  social  interaction,  it  is  about  taking  the  blame  of  failure  away  from  users’ 
shoulders  in  the  hope  of  creating  a stress -free  type  of  environment  for  the  users.  However,  the  question  is,  do  users 
actually  care  about  whether  computers  apologize  or  not?  If  they  do,  what  are  the  effects  that  computers’  apology  will 
bring  to  the  users?  The  following  experiment  was  designed  to  answer  these  questions. 


The  Study 

A computer  game  called  “10  questions”  was  designed  for  this  study.  Similar  to  “20  questions”  game,  the 
computer  had  one  noun  (could  be  an  object  or  an  abstract  concept)  in  “its  mind”  for  subjects  to  guess.  Instead  of  20 
opportunities,  subjects  had  7 chances  to  type  a keyword  to  ask  for  feedback  from  the  computer  regarding  whether  or 
not  the  keyword  is  associated  with  the  answer.  Then  they  had  3 chances  to  guess  the  answer.  To  compensate  for  the 
lost  of  10  questions,  the  computer  will  provide  3 hints  at  the  very  beginning  of  the  game,  after  the  7**'  keyword  guess, 
and  before  the  last  chance  to  guess  respectively.  For  the  first  7 keyword  guesses,  the  keyword  that  subjects  input 
would  be  matched  with  a pool  of  attributes  that  were  related  to  the  answer.  Then  the  computer  would  provide  a 
functional  feedback  like  “Yes,  ...  can  be  associated  with  the  answer”,  or  “No,  ...  cannot  be  associated  with  the 
answer”  to  provide  a direction  for  subjects  to  think.  After  the  7 keyword  guesses,  subjects  would  have  three  chances 
to  guess  the  answer,  and  only  the  exact  word  as  the  answer  would  elicit  positive  feedback.  At  this  point,  if  the  guess 


was  accurate,  the  computer  would  give  a simple  feedback  such  as  “Great,  ...  is  the  correct  answer”,  then  asked 
whether  the  subjects  wanted  to  play  another  game  or  not.  However,  if  the  subject  input  an  incorrect  guess,  two  types 
of  feedback  would  be  provided:  (a)  a functional  feedback  to  inform  whether  or  not  the  guess  was  accurate  (e.g., 
“Incorrect,  the  answer  is  not...”  or  “Correct,  the  answer  is...”);  or  (b)  apologetic  feedback  that  the  computer 
volunteered  to  take  the  blame  for  users’  failure  (e.g.,  “Sorry,  ...  is  not  the  right  answer.  I apologize  for  not  able  to 
provide  more  useful  hints  for  you  to  guess...  your  answers  will  be  incorporated  to  the  database  for  future 
improvement”).  This  apology  not  only  contains  regrets,  but  it  also  promises  improvement  to  make  it  sounds  more 
sincere.  It  is  hypothesized  that  people  who  receive  apologetic  feedback  would  feel  better  about  the  program,  their 
performance,  or  themselves.  For  this  pilot  study,  a convenient  sample  of  three  female  and  one  male  doctoral  students 
were  recruited.  All  subjects  use  computers  frequently  to  do  their  class  assignments.  One  subject  is  a consultant  in 
Educational  Technology  Service,  and  thus  was  more  advanced  in  computer  skills. 


The  following  seven  findings  were  obtained  from  this  study. 

1.  Apologetic  feedbacks  may  or  may  not  have  significant  impacts  on  subjects’  performance. 

2.  Apologetic  feedbacks  may  or  may  not  exert  positive  impact  on  subjects’  self-evaluation  of  their  performance. 

3.  Apologetic  feedbacks  may  help  subjects  enjoy  the  program. 

4.  Apologetic  feedbacks  help  to  ameliorate  subjects’  negative  feelings  from  the  program. 

5.  Apologetic  feedbacks  make  subjects  feel  better  about  themselves. 

6.  Subjects  in  apologetic -feedback  group  rate  the  program  more  favorably  than  those  in  functional-feedback  group. 

7.  Even  if  they  believe  the  program  was  flawed,  the  subjects  still  tend  to  blame  themselves  for  their  performance. 


Conclusions 

What  the  computer’s  apologies  indicate  is  a shift  of  power  that  defines  the  “appropriate  behaviors  in 
human-computer  interactions”  from  computers  to  users.  While  users’  having  this  power  is  supposed  to  be  the 
bedrock  of  the  concept  of  “user-centered  design”,  what  most  designers  do  are  to  thoughtfully  take  users’  needs  into 
account  but  still  retain  this  power  in  their  hands.  Believing  that  giving  this  power  back  to  users  will  enhance  their 
satisfaction  level  with  the  program,  this  study  shows  that  when  computers  volunteer  to  take  responsibility  for  the 
users’  failure,  users  not  only  feel  better  about  the  program  and  themselves,  they  also  show  more  interests  in 
interacting  with  the  program  again,  even  though  those  apology  did  not  generate  any  improvement  of  their 
performance. 

In  addition,  the  question  of  whether  the  users  are  ready  to  ask  computers  to  take  responsibility  for  users’ 
failure  receives  a negative  answer  from  this  study.  Even  after  subjects  had  clearly  identified  the  flaws  of  the 
program,  they  were  still  reluctant  to  ask  computers  to  take  the  blame.  While  this  may  has  a lot  to  do  with  their 
personality,  what  this  reveals  is  that  users  still  feel  helpless  in  the  face  of  computers’  failure,  and  they  were 
overpowered  by  the  computers  because  they  felt  they  did  not  know  computer  enough  to  demand  more  user-centered 
treatment.  When  computers  failed  them,  they  even  blame  themselves  for  as  a defense  of  the  computers’  failure,  just 
as  Postman  (1992)  predicted.  Such  a sense  of  helplessness  in  front  of  computer  is  the  result  of  the  reversed  power 
structure  that  we  are  having  with  computer  now.  If  users  do  not  have  the  power  to  demand  an  action  from  the 
computer  (as  opposed  to  submit  a request  to  computers),  a design  could  never  be  truly  user-centered. 
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Abstract:  This  paper  reports  on  a study  conducted  at  the  Rand  Afrikaans  University  in 

Johannesburg,  South  Africa.  The  study  aims  to  determine  to  what  extent  novice  online 
instructors  can  advance  contemporary  knowledge  of  instructional  design  in  an  online 
environment.  Participants  in  the  study  were  generally  uncertified  in  teaching  practice,  but  were 
faculty  members  in  various  disciplines  and  considered  themselves  as  experienced  face-to-face 
instructors.  In  addition,  none  of  the  participants  have  taught  online  before.  The  study  showed 
that  the  participants  had  varied  design  needs,  many  which  have  not  been  anticipated  by  the  expert 
course  designers.  It  therefore  appears  that  course  design  needs  to  be  contextualised,  and  that 
novices  can  indeed  contribute  to  design  principles  for  online  learning. 


Context 

The  importance  of  instructionally  designing  (or  educationally  designing)  online  learning  is  widely  reported  in  the  literature 
(compare  Alessi  & Trollip,  2001;  Driscoll,  1998;  Horton,  2000;  Kruse  &Keil,  2000;  Piskurich,  2000;  Ravet  & Layte,  1997). 
Investigative  visits  to  higher  learning  institutions  in  Australia  (1999  & 2001)  and  the  USA  (1998  & 2000)  have  shown  that  the 
majority  of  higher  learning  institutions  employ  trained  instructional  or  educational  designers  for  the  design  of  print -based 
materials  as  well  as  for  instruction  delivered  online.  Furthermore,  a scrutiny  of  the  conference  programmes  of  a variety  of 
conferences  (Ed-Media  1998  - 2001,  and  CAL  99)  has  shown  that  instructional  design  (ID)  is  considered  to  be  an  important 
factor  in  online  learning. 

It  appears  that  many  higher  learning  institutions  worldwide  are  using  skilled  and  trained  professionals  to  design  their  courses. 
The  Rand  Afrikaans  University  (RAU)  in  Johannesburg  is  a modem,  medium-sized  institution  offering  courses  for  residential 
students  as  well  as  for  distance  learning  students.  During  the  last  four  years,  several  courses  have  been  facilitated  online,  in  a 
range  of  modes.  Some  courses  have  a limited,  administrative  pEsence,  while  others  are  fully  online  and  integrated.  However, 
RAU  have  not  employed  a single  instructional  designer  for  online  delivery  of  courses.  Courses  are  generally  designed  and 
developed  by  the  presenters  themselves.  Many  of  these  lecturers  do  not  have  teaching  qualifications,  and  hold  no  qualifications 
in  online  teaching  practice  either.  However,  most  of  the  lecturers  who  participated  in  this  study  consider  themselves  to  be 
effective  as  instructors  in  the  face-to-face  situation,  and  believe  that  they  have  the  ability  to  design  effective  online  learning. 
They  would  argue  that  they  have  “natural”  expertise  gained  from  experience,  and  that  they  did  not  necessarily  need  “learned” 
expertise.  Currently,  there  is  limited  evidence  on  how  effective  current  online  courses  are.  Van  der  Westhuizen  (1999), 
Lautenbach  (2000)  and  Cronje  & Murdoch  (2001)  have  reported  on  the  experiences  of  lectures  and  students  at  RAU.  None  of 
these  studies  however  have  attempted  to  measure  the  effectiveness  of  the  design  of  the  particular  courses. 

The  expectation  is  that  the  expert  design  of  online  courses  will  realise  effective  courses.  However,  Dimitrova,  Sharp  and  Wilson 
(2001)  postulate  that  the  evaluation  of  expert-based  approaches  are  often  unreliable  as  evaluators  make  subjective  choices,  and 
that  the  discrepancies  between  teacher  ratings  and  learner  ratings  are  not  explained  sufficiently.  Furthermore,  Tergan  (1998) 
points  out  that  checklist-based  assessments  of  online  courses  do  not  adequately  assess  the  efficacy  of  courses.  It  therefore 
appears  that  expert  design  of  online  courses  may  not  necessarily  imply  instructionally  effective  online  courses.  The  question  is 
therefore  whether  it  is  possible  for  non-designers  and  novice  online  practitioners  to  meaningfully  contribute  to  the  expert  design 
of  online  learning. 

The  Participants 

Six  novice  online  learning  instructors  were  chosen  to  participate.  Four  of  these  practitioners  had  no  formal  qualifications  in  any 
form  of  educational  science,  and  had  little  or  no  knowledgp  of  pedagogy  as  a science.  These  practitioners  taught  Mathematics, 
Zoology  (2)  and  Human  Movement  Studies.  The  other  two  participants  had  qualifications  in  education,  and  taught  in  the  Faculty 
of  Education.  None  of  the  participants  had  any  training  in  instructional  design. 

Methodology 

Participants  were  asked  to  explain  how  they  envisaged  an  online  environment  that  could  facilitate  the  presentation  of  their 
subject  field.  The  purpose  here  was  to  determine  to  what  extent  the  participants  could  match  their  instructional  needs  with  a) 
what  they  knew  of  the  web  environment  and  b)  their  expectations  of  what  the  online  environment  could  offer.  This  initial 
“design”  was  referred  to  as  the  ’coarse  design’. 

Participants  were  then  briefly  introduced  to  online  learning,  and  specifically  to  the  WebCT  environment  (WebCT  was  used  as 
the  delivery  platform).  The  learning  resources  that  are  facilitated  by  WebCT  were  explained  and  the  application  of  the  tool  in  an 
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online  teaching  environment  was  demonstrated.  Participants,  now  having  seen  applications  of  the  WebCT  tools,  were  asked  to 
amend  and  refine  their  original  coarse  designs.  Specifically,  they  had  to  re-design  the  course  considering  different  methods  of 
content  presentation  and  facilitation,  tool  application,  learner  activities,  and  also  consider  typography,  the  use  of  graphics  and 
colour.  The  aim  of  this  segment  of  the  research  was  firstly  to  determine  whether  there  were  significant  differences  between  the 
designs  of  those  participants  with  educational  training  and  those  without.  Secondly,  an  attempt  was  made  to  derive  “novice 
design  principles”,  and  to  find  areas  of  commonality  with  design  principles  derived  from  literature  and  instruments  for  good 
online  learning  design  (expert  principles). 

Findings 

The  study  is  a work  in  progress.  The  intercrossing  of  the  “novice  principles”  and  the  “expert  principles”  showed  some 
consistencies,  although  it  was  clear  that  the  novices  could  benefit  by  exposure  to  principles  of  online  learning  design. 
Conversely,  the  novices  contributed  principles  that  were  not  encompassed  by  expert  principles,  although  many  of  these  proved 
to  be  contextualised.  In  the  main,  all  participants  required  high  levels  of  interactivity,  visual  representation  of  content  and  a 
sense  of  control  over  student  activity.  Some  of  the  needs  of  the  novice  practitioners  that  were  not  anticipated  by  the  expert 
designers,  and  that  were  not  consistent  with  general  principles  are: 

■ A need  to  duplicate  paper-based  materials  online  so  that  students  could  manipulate  text.  Some  practitioners  required 
detailed  content  online. 

■ The  use  of  metaphors  was  not  required. 

■ The  inclusion  of  visual  “bridges”  between  objectives  and  the  nature  of  the  subject  field. 

■ Most  participants  required  a linear  design. 

Participants  who  had  educational  qualifications  were  more  concerned  with  aspects  of  learner  activity,  visual  design  and 
assessment  practices  than  those  who  were  not  educationally  qualified.  Their  design  of  the  learning  experience  appeared  to  be 
more  structured.  Clearer  threads  between  course  objectives,  teaching  method  and  assessment  practices  existed  in  their  approach 
to  the  design  of  the  course. 

Conclusion 

Generally,  it  was  found  that  all  participants  had  some  “sense”  for  design  principles,  but  participants  with  formal  eduational 
qualifications  had  approaches  supported  by  pedagogical  principles.  All  participants  could  apply  the  WebCT  tools  once  they  were 
made  aware  of  the  capabilities  of  the  software.  However,  participants  reported  that  they  greatly  benefited  from  exposure  to 
principles  for  the  design  of  online  learning.  The  participants  believed  that  the  online  modules  were  instructionally  effective,  but 
made  suggestions  for  improvement.  The  intercrossing  of  the  novice  principles,  the  expert  principles  and  the  improvements 
suggested  by  the  participant  experience  may  therefore  generate  context-specific  instructionally  effective  design  principles. 
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Abstract:  This  paper  discusses  a computer-based,  corporate  training  program  that  was 
developed  through  a private  enterprise/university  partnership.  The  program  was  designed 
using  a caso-based  instructional  approach.  At  the  core  of  the  program  is  a set  of  multi-media 
cases  depicting  realistic  situations  and  problems  that  trainees  are  likely  to  face  in  the 
workplace.  A major  goal  of  caso-based  instruction  is  to  prevent  “inert  knowledge”  by 
integrating  content  learning  with  problem  solving.  Case-based  instruction  provides  learners 
with  the  chance  to  become  familiar  with  the  types  of  situations  and  problems  that  they  are 
likely  to  encounter  in  the  workplace,  and  to  learn  important  content  as  necessary  to  help  solve 
these  problems.  An  overview  of  the  theory  of  case-based  instruction  and  the  learning  issues 
that  it  is  designed  to  address,  a demonstration  of  the  computer  program,  and  a discussion  of 
the  advantages  of  case-based  instruction  in  corporate  training  settings  is  provided. 


A recalcitrant  problem  in  corporate,  post  secondary  and  K-12  education  is  the  problem  of  transfer  of  learning. 
Educators  have  long  lamented  that  learners  often  do  not  use  what  they  “learn”  in  the  classroom  to  help  them 
solve  problems  outside  of  the  classroom  or  in  the  workplace.  Whitehead  (1929)  referred  to  this  as  flie  “inert 
knowledge”  problem.  Current  research  suggests  that  knowledge  becomes  inert  when  information  is  presented  to 
learners  as  a set  of  discrete  facts  or  procedures  to-be-remembered  (NRC,  2000).  Under  these  conditions, 
learners  do  not  understand  when  it  is  appropriate  to  apply  what  they  have  learned,  and  consequently,  knowledge 
is  not  organized  in  memory  in  ways  that  will  enable  learners  to  later  retrieve  it  to  solve  new  problems. 
Furthermore,  formal  educational  settings  typically  provide  few  opportunities  for  learners  to  practice  applying 
new  knowledge  and  skills.  Without  these  opportunities,  knowledge  tends  to  be  rigidly  bound  to  the  context  in 
which  it  was  learned. 

A partnership  of  learning  scientists  from  Vanderbilt  University,  disease  management  specialists  from  American 
Healthways  Inc,  and  multimedia  designers  from  Little  Planet  Learning,  Inc  have  developed  a case-based 
corporate  training  program,  at  the  core  of  which  is  a set  of  cases  depicting  realistic  situations  and  problems  that 
trainees  are  likely  to  face  in  the  workplace.  A major  goal  of  case-based  instruction  is  to  prevent  inert  knowledge 
by  integrating  content  learning  with  problem  solving  (CTGV,  1997;  Williams,  1992).  Case-based  instruction 
provides  learners  with  the  chance  to  become  familiar  with  the  types  of  situations  and  problems  that  they  are 
likely  to  encounter  outside  of  the  classroom  (in  this  case  the  workplace),  and  to  learn  important  content  as 
necessary  to  help  solve  these  problems.  When  knowledge  is  used  to  solve  related  but  non-identical  problems,  it 
becomes  better  differentiated.  Consequently,  learners  acquire  knowledge  that  is  more  flexible  and  adaptive  for 
use  in  a variety  of  contexts.  Learners  are  also  generally  motivated  by  case-based  instruction  presumably 
because  they  are  more  actively  engaged  in  the  learning  process  than  in  lecture -based  classrooms.  Case-based 
instruction  has  been  used  successfully — albeit  on  a limited  basis— in  K-12  and  post-secondary  education 
(Williams,  1992  for  a discussion  of  use  in  legal,  business  and  medical  education).  However,  the  use  of  case 
instruction  in  corporate  training  settings  is  undocumented  and  the  use  of  a computer-based  delivery  system  for 
case  instruction  is  rare.  Some  exceptions  to  this  are  Vanderbilt  University’s  AMIGO  project  that  is  redesigning 
undergraduate  instruction  in  bioengineering  and  the  Cognition  and  Technology  Group  at  Vanderbilt’s  anchored 
instruction  projects  in  middle  school  mathematics  and  early  literacy  (CTGV,  1997) 


The  training  program  to-be-discussed  was  developed  for  American  Healthways,  Inc  (hereafter  AmHealth). 
AmHealth  is  a leader  in  the  field  of  chronic  disease  management,  providing  telephonic  interventions  to  patients 
with  diabetes,  coronary  heart  disease  and  other  chronic  illnesses.  The  company  employs  registered  nurses  and 
dieticians  to  provide  regular  interactions  with  patients;  clinicians  educate  patients  about  health  issues,  remind 
them  about  upcoming  physician  and  lab  appointments,  and  support  them  in  making  behavioral  lifestyle  changes 


n k: 


that  can  improve  their  health  status.  Patients  also  can  “call  in”  with  questions  and  concerns.  The  new  training 
program  was  developed  in  AuthorWare  and  is  on  CD-ROM.  The  cases  consist  of  simulated  phone 
conversations  between  clinicians  and  patients  that  provide  new  clinicians  with  integrated  practice  on  computer 
and  clinical  skills.  The  program  consists  of  26  multimedia  cases  that  the  trainee  uses  over  15 -days  of  training. 
The  software  also  has  content  resources — in  audio,  video  and  graphic  format—which  are  helpful  in  solving  the 
cases.  Additional  resources  are  provided  in  print  form. 

Using  cases  for  instruction  requires  a systematic  approach  to  helping  learners  explore  potential  solutions.  The 
software  is  designed  to  support  a specific  sequence  of  learning  activities  (see  Schwartz,  Lin,  Brophy  & 
Bransford,  2000  for  more  a detailed  rationale  of  the  learning  cycle).  Each  case  consists  of:  1.  A Multi-media 
Scenario  involving  a clinician.  In  many  of  the  cases,  the  clinician  is  conducting  a call  with  a patient.  2.  A 
Challenge  which  is  a problem(s)  related  to  events  depicted  in  the  scenario.  Sample  challenge  questions  are 
“What  might  be  your  next  steps  in  working  with  this  patient?”,  “What  issues  with  medications  might  be 
discussed  with  the  patient?  Why?”  and  “What  are  the  patient’s  symptoms  likely  to  be?” 

Cases  depict  patients  with  acute  and  potentially  critical  health  conditions  and  patients  with  more  typical 
problems  related  to  disease  management  and  behavior  change.  After  viewing  each  case,  trainees:  1.  Record 
their  hitial  thoughts  in  a notebook,  and  2.  Examine  resources  related  to  the  case.  Recording  their  initial 
thoughts  gives  trainees  the  opportunity  to  articulate  what  they  currently  know  about  the  challenge.  Having 
people  record  their  initial  thoughts  is  extremely  important.  First,  it  is  a “generative”  (i.e.,  thoughtful)  act. 
Learners  are  articulating  and  evaluating  what  they  know  in  relation  to  the  case  at  hand  (and  in  the  process  may 
make  knowledge  gaps  apparent,  creating  a “need  to  know”  in  the  learner).  The  notebook  entries  also  serve  as  a 
baseline  or  pre -assessment  of  what  learners  know  about  a case  and  how  they  approach  it.  The  trainer  and 
learners  can  use  these  entries  to  illustrate  the  growth  in  learners’  thinking. 

After  recording  their  initial  thoughts,  learners  are  given  the  chance  to  examine  the  resources  related  to  the  case. 
Resources  consist  of  expert  commentaries  on  the  case,  expository  materials,  and  computer  simulations. 
Trainees  access  resources  as  needed  as  they  work  through  each  case.  Depending  on  their  skill  level  learners 
may  need  to  access  only  a portion  of  those  available.  In  this  way,  case-based  instruction  can  easily  adapt  to 
pre-existing  knowledge  differences  among  trainees — an  outcome  that  is  difficult  to  achieve  in  lecture -based 
training  programs. 

Case-based  computer-assisted  instruction  in  corporate  training  has  several  important  advantages.  As  designed 
in  the  present  context,  the  system  is  easily  updated.  Resources  for  training  can  be  deleted  and  others  added  as 
medical  information  changes  or  is  revised.  In  addition,  it  provides  consistency  in  training  that  is  often  difficult 
to  achieve  when  large  numbers  of  individuals  need  to  be  trained.  From  a learning  and  motivation  standpoint,  the 
approach  is  engaging  for  trainees  and  provides  opportunities  for  them  to  become  familiar  with  the  types  of 
situations  and  problems  that  they  are  likely  to  encounter  in  the  workplace,  and  to  learn  important  content  as 
necessary  to  help  solve  these  problems 
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Abstract:  The  aim  of  this  paper  is  to  demonstrate  how  evaluation  models  were  developed  and 
implemented  in  order  to  support  institutional  goals  of  an  e-leaming  university.  The  paper  will 
discuss  the  aim  of  assessment  to  support  institutional  learning  goals,  the  benefits  and 
challenges  of  this  approach  to  assessment  in  e-leaming,  the  use  of  pattern  language  to  develop 
evaluation  models,  and  the  results  based  on  implementation  at  the  graduate  and  undergraduate 
levels  of  the  TechBC. 

1.0  Introduction 

Quality  assurance  in  learning  at  an  ©-learning  university  like  the  Technical  University  of  British  Columbia 
(TechBC)  is  critical  due  to  the  goals  of  innovation,  the  investment  costs  in  an  online  curriculum,  and  the  investment  in 
organizational  stmctures.  In  order  to  meet  the  goals  and  create  the  return  on  investment,  the  application  of  ©-learning 
must  be  learning  effective,  cost  effective,  and  scalable. 

The  role  of  assessment  in  this  context  is  to  validate  (or  invalidate)  policies,  methods,  and  ultimately,  the 
outcomes.  If  assessment  is  in  support  of  institutional  goals,  it  is  even  more  valuable  toolset.  As  such  it  must  be  linked  to 
the  strategic  goals  of  the  learning  institution  in  order  to  successfully  implement  e-leaming. 

Managerially,  it  is  a toolset  to  ensure  a cost  and  learning  effective  implementation  of  scalable  ©-learning.  A 
key  tool  of  assessment  in  addressing  these  issues  is  the  use  of  evaluation  models. 

2.0  Supporting  Institutional  Goals 

In  the  case  of  TechBC,  the  use  of  evaluation  models  is  strategically  shaped  by  the  need  to  support  institutional 
goals  of  effective  learning,  learner-centered  learning,  and  the  effective  use  of  technology. 

The  mission  of  the  academic  program  can  best  be  described  as  creating  an  accessible  and  effective  learning 
environment.  The  use  of  evaluation  models  strives  to  achieve  and  support  the  mission  and  goals. 

3.0  Evaluation  Models 

3.1  Pattern  Language 

Christopher  Alexander  developed  a powerful  design  tool  and  process  known  as  “pattern  language”,  to  help 
define  environmental  planning  and  architecture.  It  has  evolved  and  has  been  applied  to  computer  programming,  human 
interaction,  and  organizational  design  (Coplien,  1995),  as  well. 

We  developed  evaluation  models  within  a pattern  language  design  framework.  Each  model  was  described  as  a 
pattern,  including  elements  of  each  pattern  - such  as  name,  description,  application,  how  it  works,  example,  and 
scenarios.  We  also  provided  the  larger  Iramework  for  the  relation  of  patterns  as  “parent”  and  “child”. 

In  order  to  meet  the  goals  of  developing  cross-disciplinary  implementation  of  evaluation  models,  and  certainly 
implementing  their  uses  successfully  creates  a considerable  challenge  for  “buy-in”  and  the  required  collaboration.  Any 
development  process,  iteration  and  ongoing  implementation  required  significant  authorship  roles  by  the  learning  staff  at 
each  stage.  At  TechBC,  we  modeled  our  approach  to  development  and  structuring  of  the  evaluation  models  after 
ChristqDher  Alexander’s  “pattern  language”.  In  addition,  we  benefited  Irom  the  highly  collaborative  and  inter- 
disciplinary culture  of  TechBC, 


3.2  The  Models 

Evaluation  models  were  created  to  address  the  need  for  a comprehensive  document  capable  of  supporting 
intended  assessment  practices.  We  identified  pattern  language  as  the  design  framework  for  the  evaluation  models 
applicable  to  TechBC ’s  modular  learning  structures  and  delivery  models. 

In  addition  to  the  components  of  a pattern,  the  evaluation  models  are  accompanied  by  a “success  criteria”,  is 
drawn  from  past  experiences.  The  goal  is  to  adapt  assessment  a learning  environment  that  benefits  Irom  effective 
curriculum  implementation,  effective  management,  cost-effectiveness. 
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Good  instructional  design  is  a significant  enabler  of  effective  learning  (Belfer  and  Nesbit,  2001).  Following  a 
systemic  approach  to  instructional  design  good  pedagogical  practices  are  embedded  in  assessment  that  is  integral  to 
learning  and  adhere  to  basic  learning  principles. 

The  evaluation  models  have  these  principles  embedded,  and  therefore  greatly  support  curriculum  development, 
and  ultimately  move  toward  ensuring  quality  in  the  use  of  educational  technology  in  learning 

4.0  Results 

We  have  measured  the  success  of  the  evaluation  models  by  the  degree  to  which  assessment  has  supported 
institutional  goals  for  learning,  and  derived  the  benefits  of  doing  so. 

4.1  Effective  curriculum  implementation 

Key  results  for  curriculum  implementation  is  the  achieving  of  coherency  for  the  learner  and  exchange  of  best 
practices.  Inter-disciplinary  practice  and  collaboration  were  seen  to  be  critical  requirements.  Similar  to  other 
implementations  of  pattern  language,  authorship  by  learning  staff  is  critical.  Faculty  and  learning  staff  were  invited  to  re- 
define past  practices  as  a pattern,  or  generate  new  ones.  The  results  clearly  support  a goal  of  coherency  for  learners  of 
assessment  methods  and  the  collaborative  exchange  of  assessment  knowledge  and  practice. 

4.2  Effective  Management 

Key  results  for  management  are  the  scalability  and  portability  of  the  programs.  The  name  and  description  of  each 
evaluation  model  tell  a story  and  the  means  of  assessment.  Together  with  the  details  of  each  pattern,  it  is  easier  for 
learning  staff  other  than  developers  to  effectively  deliver  a course,  even  for  the  first  time.  The  pattern  language  approach 
equally  ensures  the  purpose  each  assessment  activity  and  the  goal  in  the  overall  scheme  of  learning  activities  is  as  evident 
as  the  evaluation  method.  In  addition,  evaluation  models  a“e  linked  to  one  or  more  delivery  models,  allowing  for 
scalability  to  be  a factor  of  learning  activities  and  goals  and  not  disciplinary  content,  i.e.  the  delivery  and  evaluation 
methods  do  not  need  to  be  re-written  for  new  content. 

4.3  Cost-effectiveness 

A key  result  for  cost  effectiveness  is  the  ability  to  effectively  manage  resources.  Evaluation  models  have  been 
developed  to  support  a learning  practice  that  is  understood  in  discrete  elements  and  as  a team  effort  Therefore,  modules 
identified  for  revisions,  can  be  analyzed  component  by  component.  If  the  main  concern  of  both  learners  and  instructors  in 
the  assessment  structure,  the  evaluation  models  provide  an  easy  procedure  to  re-evaluate  the  assessment  structure  by 
choosing  a new  evaluation  model,  an  not  revisiting  the  module  in  its  entirety. 

5.0  Conclusions 

We’ve  argued  and  hopefully  demonstrated  that  assessment  must  be  linked  as  a management  toolset  to  the 
strategic  goals  of  the  learning  institution.  This  need  is  particularly  critical  to  derive  the  benefits  of  an  o-leaming.  For 
administrators  and  learning  staff,  it  is  a toolset  to  ensure  a cost  and  learning  effective  implementation  of  e-leaming  that 
ultimately  supports  scalability.  We  have  focused  on  how  a key  tool  of  assessment  in  addressing  these  issues  is  the  use  of 
evaluation  models. 

A critical  conclusion  is  the  bridging  of  practice  to  goals  requires  buy-in,  collaboration  and  participation  in  the 
design,  selection,  modification  and  adaptation  of  the  evaluation  models.  A specific  goal  is  that  each  module  has  a 
particular  evaluation  model  that  successfully  prescribes  assessment  methods  while  establishing  an  organizational 
structure  towards  the  process,  making  the  process  of  assessment  transparent  and  creating  a common  language  for 
assessment  as  part  of  the  learning  culture. 

References 

Belfer,  K.  & Nesbit  J(2001),  GUIDE:  An  instrument  for  evaluating  quality  of  instructional  design  in  mixed-mode 
university  courses.  World  Conference  in  Multimedia  Hypermedia  and  Telecommunications  (Ed-Media)  Tampere, 
Findland,  June  25-  3 1 . 

Coplien,  J.  O.  (1995).  A Development  Process  Generative  Pattern  Language. 

Retrieved,  March,  2002  from  http://www.bell-labs.com/user/cope/Pattems/Process/ 


20  3B 


Role  with  IT...  the  Evolution  of  Faculty  and  Faculty  Support  Roles  and 
Responsibilities  in  the  Rapidly  Changing  Climate  of  Educational 

Technology 


Sarah  Walkowiak.  sarahw@wpi.edu.  Online  Delivery  Coordinator,  Educational  Technology  Services, 
Instructional  Media  Center,  Worcester  Polytechnic  Institute,  Worcester,  MA 
http.7/www.wpj  .edu/-f-IMC/ETS 


Abstract:  As  digital  technology  becomes  more  widely  integrated  into  the  teaching  and  learning 
environment,  particularly  in  distance  learning,  faculty  members  face  new  challenges.  To  thrive  in  this 
new  teaching  environment,  faculty  must  simultaneously  develop  their  pedagogical  and  technological 
skills.  The  process  of  technologically- based  faculty  cfevelopment  is  often  time  consuming  and  difficult, 
especially  for  faculty  who  have  less  experience  working  with  the  nuances  of  instructional  technology. 
To  alleviate  this  problem  at  WPI,  the  Educational  Technology  Services  team  (ETS)  was  created  as  a 
functional  workgroup  under  the  Instructional  Media  Center.  The  mission  of  this  group  is  to  support  and 
empower  WPI  faculty  in  the  effective  use  of  technology  to  enhance  their  teaching 


Introduction 

Through  the  advancement  of  educational  technology,  an  ever-widening  variety  of  course  material  delivery  and 
presentation  options  are  available  to  faculty  members.  While  this  myriad  of  options  can  be  exciting,  it  can  also 
be  frustrating  and  overwhelming.  To  alleviate  this,  and  to  create  a knowledge  base  of  standards  and 
pedagogically  sound  best  practices,  we  have  expanded  the  services  of  our  Instructional  Media  Center  to  include 
extended  instructional  support  and  design  assistance.  These  services  are  provided  by  the  Educational 
Technology  Services  (ETS)  team,  consisting  of  an  instructional  designer  and  director,  a content  coordinator,  a 
graphics  production  coordinator,  an  online  delivery  coordinator  and  two  technicians,  one  for  video  support  and 
one  for  courseware  support.  The  formation  of  this  group  and  associated  services  provided,  fueled  an  evolution 
in  roles  and  responsibilities  for  faculty,  staff  and  students  at  WPI  in  relation  to  teaching  and  technology. 

Where  We  Have  Been 

As  on  most  university  campuses,  there  have  been  three  consistent  adoption  levels  for  teaching  with  technology 
at  WPI.  These  levels  consist  of  a few  early  adopters,  eager  to  try  out  new  technology  in  their  courses, 
intermediate  adopters,  using  tried  and  true  practices  and  basic  adopters  who  prefer  more  familiar  forms  of 
technology  or  avoid  it  altogether.  The  early  adopter  role  is  often  referred  to  as  the  “Lone  Ranger  and  Tonto” 
approach  to  teaching  with  technology,  offering  grants  to  faculty  members  to  fund  software,  equipment  and  a 
part-time  graduate  student.  (Bates,  2001)  For  all  levels  of  technological  expertise,  most  faculty  have  received 
adequate  hardware  support  but  have  received  mixed  levels  of  support  in  regards  to  incorporating  the  technology 
into  their  teaching.  Faculty  at  basic  and  intermediate  levels  tend  to  become  frustrated  when  something  does  not 
work,  even  with  available  support,  so  they  tend  to  take  fewer  instructional  risks  in  favor  of  a nore  stable 
teaching  product.  Most  of  our  training  in  the  past  has  focused  on  the  technical  side  of  teaching  with  technology, 
with  instructional  design  issues  handled  separately  or  as  a special  interest  of  lone  rangers.  The  need  for  more 
integration  between  instructional  design  and  the  use  of  technology  is  now  understood  by  a greater  proportion  of 
the  university  at  large,  thus  providing  a foundation  in  the  campus  culture  for  emerging  roles. 

Evolution  of  Faculty  and  Staff  Roles  and  Responsibilities 

Through  the  process  of  providing  extended  instructional  support,  these  roles  have  begun  to  shift  at  WPI  along 
with  the  development  of  new  responsibilities  for  each  role.  The  “Lone  Rangers”  remain  interested  in  new 
technology,  but  are  freed  from  many  of  the  technical  burdens  that  can  arise  when  exploring  new  technology 
alone.  With  this  knowledge  and  research,  we  are  able  work  together  to  implement  best  practices  (instead  of 
creating  a project  from  scratch)  while  working  within  the  context  of  supported  university  computing  systems 
and  resources.  Thus  keeping  compatibility  and  scalability  in  mind  and  encouraging  faculty  collaboration  on 
similar  projects  or  combining  efforts  to  expand  on  existing  work.  Our  mission  to  technologically  empower 
faculty  has  materialized  through  the  execution  of  our  instructional  services  and  implementation  of  new  training 


materials,  workshops  and  programs.  We  have  created  a supportive  atmosphere  to  foster  intrinsic  faculty 
motivation  at  all  levels  of  technological  expertise  by  providing  support  staff,  access  to  technology,  faculty 
encouragement  and  incentives  for  implementing  the  technology.  Extrinsic  factors  are  supported  by  the 
assistance  of  our  staff  with  technological  projects  and  providing  a channel  for  delegation  of  student  technical 
support  requests  to  the  appropriate  group  on  campus.  (Irani,  2001) 

Example:  A Distance  Learning  Course 

We  have  particularly  noticed  the  emergence  of  new  roles  and  responsibilities  for  faculty,  staff  and  students  in 
our  distance  learning  courses  which  rely  on  technology  for  communication  and  course  delivery.  Each  of  our 
distance  faculty  members  is  partnered  with  an  ETS  course  coordinator  who  manages  the  technical  aspects  of  the 
course  and  assists  with  the  development  and  assessment  of  course-specific  materials.  With  this  new  partner, 
faculty  are  supported  while  re-tooling  their  teaching  model  as  they  learn  to  use  technology  more  effectively. 
Throughout  this  partnership,  the  coordinator  provides  technical,  pedagogical  and  moral  support  in  weekly 
meetings  and  other  communications.  Several  of  our  distance  learning  faculty  are  new  to  distance  learning  and 
the  extra  support  helps  to  minimize  the  transition  to  a new  teaching  medium.  This  partnership  creates  an 
opportunity  to  build  strong  working  relationships  with  the  faculty  and  to  fully  understand  how  the  media  and 
technology  are  being  applied  in  live  courses.  This  added  knowledge  makes  for  a more  efficient  workflow,  as 
each  faculty  member  has  one  main  contact  and  the  coordinator  becomes  aware  of  each  faculty  membeis’ 
individual  style.  Assessments  become  integrated  into  the  curriculum  as  the  coordinators  work  to  improve 
student  learning  through  course  materials  that  are  created.  The  students  on  the  receiving  end  of  our  collective 
work  find  that  with  additional  expertise  behind  a course,  fewer  technical  glitches  occur,  and  when  they  do, 
response  time  decreases  as  support  requests  are  quickly  routed  to  the  appropriate  place  by  the  coordinator.  Time 
and  project  management  also  play  a key  role  in  getting  course  materials  delivered  on  time  and  without  technical 
glitches. 

Learning,  Growth  and  Change 

With  new  responsibilities  for  faculty,  staff  and  students  come  growing  pains  - some  faculty  experience  a more 
difficult  adjustment  to  distance  learning  and  new  technology.  A sense  of  loss  of  autonomy  may  be  felt  as  the 
faculty  move  towards  a learner-centric  teaching  model  and  as  more  people  have  a hand  in  course  development. 
The  process  of  working  with  a coordinator  provides  a forum  for  discussion  of  these  issues  and  helps  to  alleviate 
any  fears  a faculty  member  might  have.  The  experience  of  collaboration  between  the  Educational  Technology 
Services  team  and  our  faculty  provides  many  opportunities  for  learning  and  growth  for  everyone  involved.  Our 
focus  is  on  applying  instructional  design  theory  to  these  courses  and  learning  along  with  the  faculty.  The  faculty 
gain  valuable  technical  skills  and  understanding  while  providing  more  effective  learning  tools  and  environments 
to  their  students.  Faculty  shifts  from  teaching  in  a lecture  based  mode  to  a more  interactive  mode  provide  a 
richer  experience  to  the  students,  but  also  require  students  to  take  more  responsibility  for  their  learning. 

Conclusions 

As  our  group  continues  to  evolve,  we  will  pursue  our  mission  of  empowering  faculty  by  encouraging  our  more 
experienced  faculty  to  expand  their  technological  skills  and  to  share  these  skills  and  learning  experiences  not 
only  with  us,  but  with  their  colleagues.  We  have  implemented  multiple  campus-wide  workshops  along  with  a 
Teaching  Technology  Fellowship  program  and  are  in  the  final  development  stages  of  an  Instructor  Certification 
in  Academic  Technology  program  that  will  serve  this  purpose  well.  Marketing  materials  and  sessions  to  inform 
more  faculty  members  and  departments  of  our  services  are  also  planned,  along  with  a collection  of  best 
practices  that  we  have  compiled  in  our  first  year  as  an  online  reference  to  help  faculty  get  started  with  different 
media  and  technology  types.  The  ETS  group  is  beginning  to  see  an  actualization  of  technical  change  now 
happening  at  many  institutions  and  looks  forward  to  providing  improved  services  and  sharing  our  experiences. 
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From  a simple  PowerPoint’*'”^  presentation  to  a complete  online  course,  technology  now 
allows  educators  unprecedented  freedom  to  create,  modify,  and  disseminate  their  own  in- 
struction; however,  this  same  freedom  also  offers  unprecedented  opportunities  to  create 
and  disseminate  ineffective  or  poorly  designed  instruction.  Ineffective  technology -based 
instruction  often  stems  from  neglect  of  the  basic  principles  of  effective  instruction.  This 
paper  describes  four  non-technical  tips  educators  can  use  to  enhance  the  quality  and  ef- 
fectiveness of  technology -enriched  instruction.  They  include  (a)  identifying  instructional 
objectives,  (b)  understanding  the  target  audience,  (c)  organizing  the  content,  and  (d)  pre- 
paring for  instructional  delivery. 


Introduction 

It  is  no  longer  surprising  to  see  P-12  teachers  and  college  or  university  faculty  with  the  technical  ability  to 
create  multimedia-rich  PowerPoint  presentations,  develop  and  maintain  their  own  course  Web  sites,  or,  with  no 
technical  assistance,  setup  and  operate  a wide  variety  of  computer-based  equipment.  As  educators,  we  are  rapidly 
becoming  more  technologically  savvy.  Fortunately,  many  of  us  are  learning  to  embrace  this  change  and  are  enjoying 
the  unprecedented  freedom  technology  offers  us  in  our  ability  to  create,  modify,  and  disseminate  instruction,  how- 
ever, this  same  freedom  also  offers  us  ample  opportunity  to  produce  ineffective  or  poorly  designed  instruction.  In 
allowing  us  to  personalize  and  package  the  instructional  messages  we  convey  to  our  students,  technology  also  gives 
us  much  latitude  to  miss  the  mark,  and  that  is  precisely  the  issue  raised  in  this  paper.  Perhaps  we  need  to  be  re- 
minded that,  as  we  run  to  jump  on  the  technology  bandwagon,  we  must  not  forget  to  bring  the  basics  of  good  h- 
struction  along  with  us. 

Most  of  us  have  endured  PowerPoint  presentations  with  font  sizes  so  small  that  we  could  not  possibly  read 
the  information  being  presented  or  lectures  by  professional  educators  who  stood  with  their  backs  to  the  audience  as 
they  read  paragraph  after  paragraph  from  the  information  projected  on  the  screen.  In  these  and  many  other  situa- 
tions, technology,  in  all  its  capacity  to  enhance  instruction,  only  served  to  impede  the  dissemination  and  reception  of 
the  intended  message.  It  is  with  this  in  mind  that  we  offer  a few  reminders  on  how,  as  professional  educators,  we  can 
reduce  the  chances  of  getting  blindly  caught  up  in  technology’s  enticement  and  instead,  capitalize  on  its  potential  to 
support  effective  teaching. 


Identifying  Instructional  Objectives 

In  simple  terms,  objectives  are  specific  descriptions  of  what  we  want  the  learner  to  know  or  be  able  to  do  as 
a result  of  instruction.  Whether  we  are  using  PowerPoint  in  the  classroom  or  delivering  our  instruction  online,  it  is 
critical  that  we  make  the  effort  to  identify  specific  instructional  objectives,  even  if  it  requires  a considerable  amount 
of  time.  Surprisingly,  many  of  us  spend  more  time  selecting  background  colors  and  clip  art,  digitizing  video  clips, 
creating  endless  lists  of  hyperlinks  to  online  resources,  or  toying  with  any  number  of  technology’s  fun  and  often 
frivolous  features  than  we  do  identifying  and  incorporating  objectives  into  our  technology -based  instruction.  While 


technology’s  bells  and  whistles  or  its  ability  to  expedite  our  productivity  should  not  be  neglected,  our  first  priority 
should  be  to  identify  and  incorporate  instructional  objectives  in  the  design  and  development  of  our  instruction. 

Determining  and  writing  good  objectives  is  not  always,  if  ever,  a quick  and  easy  process.  Taking  the  time 
to  determine  instructional  objectives,  refining  them,  and  then  developing  content  and  evaluation  methods  around 
them  can  be  a very  detailed  and  time-consuming  process.  This  stands  in  direct  contrast  to  the  expectations  we  may 
have  of  technology  and  its  ability  to  assist  us  in  rapidly  creating  a finished  product.  While  convenience  and  expedi- 
ency are  enticing  attributes,  as  educators,  surely  we  are  not  willing  to  trade  them  for  quality  and  effectiveness. 


Understanding  the  Target  Audience 

In  a traditional  setting,  when  teachers  walk  into  a classroom  to  teach  a lesson  or  deliver  a lecture,  they  are 
in  direct,  face-to-face  contact  with  their  audience  and  can  react,  make  accommodations,  and  modify  the  information 
they  are  presenting  and  the  way  in  which  they  present  it  based  on  audience  reaction  and  response.  Trained  educators 
know  this  ability  can  greatly  improve  the  effectiveness  of  instruction.  Unfortunately,  technology,  if  we  allow  it  to, 
can  easily  separate  us  from  the  learner. 

As  many  of  us  have  already  discovered,  our  efforts  to  understand  and  accommodate  our  audience  often 
tend  to  lead  us  away  from  the  rigid  and  inflexible  presentations  that  can  be  so  instructionally  ineffective  and  instead, 
toward  the  development  of  presentations  that  simply  support  the  information  we  wish  to  convey.  For  example,  rather 
than  creating  a presentation  that  contains  slide  after  slide  of  the  exact  content  being  presented,  forcing  the  instructor 
to  follow  the  pre -developed  content  precisely,  we  hstead  find  ourselves  more  often  creating  presentations  that 
merely  contain  supporting  electronic  artifacts  such  as  charts,  graphs,  maps,  photographs,  video  clips,  or  a few  talk- 
ing points.  By  simply  augmenting  a lecture  or  presentation  with  technology  instead  of  letting  technology  dominate 
it,  we  help  create  an  instructional  environment  that  is  much  more  responsive  to  our  audience. 


Organizing  the  Content 

The  appropriate  organization  of  content  is  crucial  when  conveying  information  or  providing  hstruction. 
The  clear  articulation  of  the  content  order  reduces  the  chance  of  misinterpretation  or  confusion  by  the  learner  while 
assisting  him  or  her  in  establishing  important  and  necessary  associations  between  the  elements  that  make  up  the  con- 
tent and  between  prior  knowledge  and  current  information.  Fortunately,  content  organization  is  often  made  simple 
by  the  content  itself  For  example,  the  order  in  which  a step-by-step  task  such  as  creating  a new  slide  in  PowerPoint 
is  presented  is  very  sequential  and  straightforward.  On  the  other  hand,  some  content  is  more  open  ended  and  much 
more  complicated  to  organize.  Imagine  having  to  organize  the  content  for  a presentation  on  how  to  deliver  a persua- 
sive oral  speech.  This  type  of  content  could  be  effectively  organized  in  a wide  variety  of  ways. 

It  is  in  the  delivery  of  this  open-ended  type  of  material  that  it  becomes  crucial  for  educators  to  give  careful 
attention  to  the  appropriate  organization  of  content.  Unfortunately,  as  we  now  find  ourselves  with  the  technology- 
enhanced  ability  to  create  our  own  instruction,  we  must  often  assume  the  responsibility  of  organizing  our  instruction 
ourselves.  To  do  so  effectively,  we  must  have  a strong  understanding  of  the  content  itself,  of  our  target  audience’s 
existing  skills  and  knowledge,  and  of  the  desired  learning  outcome  we  wish  to  achieve.  Based  on  these,  we  can  be- 
gin to  organize  our  instruction  or  information  in  clear  and  appropriate  ways. 


Preparing  for  Instructional  Delivery 

This  is  perhaps  the  simplest  but  most  overlooked  aspect  of  delivering  effective  technology-based  instruc- 
tion. Simply  put,  make  sure  everything  works  before  getting  up  in  front  of  your  audience.  Audiences  or  students  are 
no  longer  tolerant  of  our  lack  of  preparation  and  consideration  when  we  don’t  know  how  to  operate  equipment.  In 
addition  to  being  familiar  with  the  operation  of  the  equipment,  we  must  also  be  familiar  with  our  presentation  and  its 
content.  Were  not  suggesting  that  hours  of  rehearsal  and  practice  are  required  in  order  to  deliver  a polished  and  pro- 
fessional presentation,  however,  adequate  steps  must  be  taken  to  ensure  that,  as  the  presenter  or  teacher,  we  are  la- 
miliar  with  what  we  are  presenting  and  with  the  equipment  used  to  present  it.  Anything  less  runs  the  risk  of  seriously 
jeopardizing  the  instructional  effectiveness  of  our  presentation.  This  is  a big  price  to  pay  for  failing  to  take  a few 
extra  minutes  to  make  certain  we  are  prepared. 
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Abstract:  In  this  paper,  we  develop  a tool  for  analyzing  browsing  log  file  based  on  data  mining 
technology.  The  extracted  knowledge  includes  the  associative  material  clusters  and  the 
sequences  among  the  material  clusters.  These  findings  enable  teachers  to  study  the  dynamic 
browsing  structure  and  identify  some  interesting  or  unexpected  learning  patterns,  which  in  turn 
may  provide  decision  lines  for  teachers  to  organize  their  teaching  structure  for  better 
efficiency.  Finally,  application  of  the  tool  is  conducted  on  a database  collected  from  a web- 
based  course  in  Ming  Chuan  University,  Taiwan,  to  investigate  its  effectiveness,  and  some 
revelations  are  presented  and  discussed. 


Introduction 

In  asynchronous  web-based  learning  environments,  teachers  could  conduct  many  kinds  of  educational 
activities  such  as  material  posting,  homework  assignment,  group  discussion,  online  testing  and  so  on.  However, 
the  most  prevailing  issue  in  such  a learning  environment  is  that  it  is  not  easy  for  teachers  to  monitor  students’ 
learning  behaviors  instantly.  Nevertheless,  as  compared  with  conventional  CAI  systems,  web-based  learning 
environments  are  able  to  record  most  learning  behaviors  of  the  students,  and  hence  are  able  to  provide  a huge 
amount  of  learning  profile.  As  a result,  there  is  an  urgent  need  of  analyzing  methods  to  discover  useful 
information  for  improved  instruction  performance  from  the  huge  log  database.  Usually,  web  materials  are  divided 
into  units  of  topics  that  are  structured  by  some  semantic  relations  among  them  (Barker,  1979).  This  study 
assumes  that  browsing  behaviors  of  students  in  a web-based  learning  environment  might  reveal  the  true 
structure  required  of  the  material  documents  for  being  helpful  to  students.  Therefore,  this  study  proposes  a 
browsing  model  to  describe  the  useful  browsing  patterns,  and  develops  a portfolio  analysis  tool  based  on  data 
mining  technique  (Fayyad  et  al.  1996)  to  discover  them  from  the  browsing  portfolio  database.  Finally,  application 
of  the  tool  is  conducted  on  a portfolio  database  collected  from  a web-based  course  in  Ming  Chuan  University, 
Taiwan,  to  investigate  its  effectiveness,  and  some  revelations  are  presented  and  discussed. 


The  Document  Browsing  Model 

In  this  study,  the  browsing  model  consists  of  two  kinds  of  relations.  The  first  is  the  association  relation 
between  document  clusters  and  the  other  is  the  precedence  relation  among  them.  In  the  following,  we  present 
preliminary  definitions  that  are  related  to  the  depiction  of  the  browsing  model. 

Description  1 (Association  Relation):  For  any  documents  p j and p2,  we  say  p j is  associated  with  p2  if 
and  only  if  p j and p 2 are  'frequently  " browsed  in  a learning  session  without  "observable  " sequence  between 
pjandp2. 

Description  2 {Sequence  Relation):  For  any  documents  p i and  p 2,  we  say  p / precedes  p 2 if  and  only  if p / 
and p 2 are  "frequently"  browsed  together  and  there  is  "observable  " browsing  sequence  between  p / and p 


Application  and  Discussion 

This  study  applies  the  aforementioned  tool  in  three  web-based  classes  of  the  “Expert  System”  Course  in 
Ming  Chuan  University,  Taiwan.  There  are  totally  172711  browsing  records  in  the  log  database?  Setting  the 
minimum  threshold  of  browsing  time  as  7 seconds,  and  the  minimum  support  as  0.03,  totally  22846  session 
records  are  left  after  the  cleaning  process  and  960  browsing  session  records  are  attained  after  the  transforming 
process.  The  total  number  of  material  documents  is  96,  and  the  number  of  sessions  and  average  session  length 
of  each  class  are  listed  in  [Table  1]. 


Class 

Number  of  Sessions 

Documents 

Average  Session  Length 

A (Day  Time) 

349 

85 

2.9 

B (Day  Time) 

334 

85 

3.1 

C (On-Job) 

277 

80 

3.4 

Total 

960 

96 

3.1 

Table  1:  The  number  of  sessions  and  average  session  length  of  each  class 


To  increase  the  reliability  of  the  mining  results,  this  study  combines  the  session  records  of  the  three 
classes  for  the  associative  and  sequential  mining  tasks.  The  result  shows  that  23  clusters  are  found  with  the 
largest  cluster  of  size  3.  Among  these  clusters,  it  is  found  that  the  “Theory”  and  “Demo  Systems”  categories  are 
often  browsed  together,  and  “Demo  System”  and  “Language”  categories  are  also  browsed  together.  These  imply 
that  the  cross-reference  usage  of  the  materials  is  effectively  achieved. 

Most  of  the  browsing  sequences  found  meet  the  instructor’s  expectations.  Nevertheless,  some 
interesting  sequence  patterns  are  also  found,  including  the  sequence  ([Backward -chaining  ES],  [Inference 
Engines])  ->  ([CLIPS  Language]),  which  reveals  that  about  20  to  30  percents  of  the  students  mistake  the  CLIPS 
language  (a  tool  for  designing  forward-chaining  expert  systems)  for  a design  language  for  backward-chaining 
expert  systems. 

Conclusions 

This  study  develops  a data-mining  model  and  tool  for  analyzing  the  browsing  portfolios  in 
asynchronous  learning  networks.  This  study  also  finds  that  the  amount  of  browsing  records  is  still  not  large 
enough  for  more  accurate  mining  results.  This  might  be  attributed  by  the  fact  that  most  students  are  prone  to 
download  all  materials  first  for  off-line  browsing  at  home.  Finally,  more  effective  mining  tools  are  still  in  lack  for 
analyzing  other  kinds  of  portfolio  information,  such  as  the  behavior  of  “thinking  order”  in  web-based  on-line 
discussions,  and  help  to  explore  the  relations  among  the  various  learning  factors  (efforts)  and  the  learning 
outcomes. 
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Abstract  Building  upon  a knowledge  management  framework  manifested  in  the  Nonaka’s  knowledge 
creation  model,  the  I^owledge  Innovation  for  Technology  in  Education  (KITE)  project,  funded  by  US 
Department  of  Education,  seeks  to  build  a technology  integration  community  of  practice  through  a 
knowledge  repository  with  case-based  reasoning  capabilities  that  allows  to  capture  and  disseminate  the  best 
practices  and  lessons  learned  of  technology  integration  experiences  by  teachers  and  teacher  educators.  The 
project's  overarching  mission  is  to  enable  learning  through  sharing,  communal  understanding  through 
storytelling,  continuous  exchange  and  creation  of  new  knowledge,  and  collective  problem  solving  within  this 
technology  integration  community. 


As  the  face  of  American  classrooms  continues  to  change  with  rapidly  increasing  technological  changes,  greater 
diversity,  and  a projected  need  for  2.2  million  new  teachers  in  the  next  decade,  teacher  training  institutions  are  faced 
with  the  challenge  of  graduating  new  teachers  who  will  be  adept  at  keeping  pace  with  the  technology  explosion  in 
the  classroom.  To  increase  the  level  of  technology  integration  in  teacher  education  programs,  the  Milken  study 
recommended  that  researchers,  professional  societies,  and  education  agencies  should  identify,  study,  and 
disseminate  examples  of  effective  technology  integration  on  an  ongoing  basis  (ISTE,  1999).  Building  upon  a 
knowledge  management  framework,  the  Knowledge  Innovation  for  Technology  in  Education  (KITE)  project, 
funded  by  US  Department  of  Education,  seeks  to  collect,  organize  and  make  accessible,  and  then  disseminate 
technology  integration  knowledge  of  teachers  and  teacher  educators  with  access  to  a practice-based  collection  of 
"how-to"  resources. 


Knowledge  Management  Framework 

We  believe  the  needs  of  teacher  education  programs  described  before  can  be  addressed  with  the  concept  of 
knowledge  management  that  has  gained  a wide  acceptance  in  the  business  circle  today.  Knowledge  management 
experts  (Davenport  & Prusak,  1997;  Nonaka  & Takeuchi,  1995)  hold  that  an  organization  should  actively  locate, 
capture,  and  disseminate  knowledge  and  expertise  in  a systematic  manner  to  enhance  organizational  performance 
and  innovation.  In  particular,  Ikujiro  Nonaka  (1994),  a prominent  scholar  in  knowledge  management,  developed  a 
model  that  explains  how  knowledge  in  an  organization  is  created  through  a continuous  process  of  knowledge 
conversions  between  explicit  knowledge  and  tacit  knowledge  between  explicit  knowledge  and  tacit  knowledge 
(Polanyi,  1966). 

Explicit  knowledge  is  formal  and  systematic  in  nature  such  as  technical  data,  general  principles,  and  standard 
operating  procedures  (Nonaka,  1994).  Explicit  knowledge  can  readily  be  elaborated,  codified,  and  shared,  and  thus 
can  be  represented  in  books  and  databases  and  made  accessible  through  libraries  and  online  information  systems. 
Tacit  knowledge  is  practical  "know-how"  acquired  through  practice  and  experience  (Leonard  & Sensiper,  1998). 
Tacit  knowledge  approximates  what  Schon  (1983)  referred  to  as  "knowing- in-action."  Nonaka  argued  that  “...tacit 
knowledge  is  deeply  rooted  in  action,  commitment,  and  involvement  in  a specific  context”  (p.  16).  Tacit  knowledge 
may  require  little  or  no  time  or  thought  when  executed  and  can  be  so  internalized  that  it  is  taken  for  granted. 

As  shown  in  Figure  1,  in  the  context  of  technology  integration,  knowledge  creation  takes  place  in  four  modes  of 
knowledge  conversion.  Socialization  is  “the  process  of  creating  tacit  knowledge  through  shared  experience”  (p.  19). 
The  socialization  mode  requires  a field  of  interaction  where  practitioners  share  their  experiences  and  perspectives 
through  technology  integration  dialogues  in  the  form  of  questioning,  reflection,  and  feedback  and  through 
mentioning  relationships  between  technology  integration  novices  and  experts.  Second,  extemalization  is  the  process 
of  converting  personal  tacit  knowledge  into  public  explicit  knowledge.  Extemalization  is  facilitated  by  synthesizing 
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tacit  technology  integration  experiences  into  explicit  technology  integration  cases  that  allow  practitioners  to  reflect 
their  understandings  and  insights  through  interviews.  These  cases  represent  new  experiences  and  perspectives  which 
become  transferable  through  case-based  reasoning  processes  and  functions  in  the  knowledge  repository.  Third, 
combination  is  the  process  of  converting  existing  explicit  knowledge  into  new  explicit  knowledge.  Combination 
implies  that  a community  of  practice  should  invest  in  a knowledge  repositoiy  so  that  knowledge  can  be  preserved, 
processed,  and  made  available  quickly  without  space  and  time  boundaries.  In  other  words,  explicit  knowledge  can 
be  "combined”  upon  the  new  experiences  and  insights  gleaned  from  the  practices  of  technology  integration.  Lessons 
learned  and  best  practices  can  be  compiled  from  the  cases.  Fourth,  internalization  is  the  process  of  enriching  one’s 
own  tacit  knowledge  by  internalizing  explicit  knowledge.  In  this  phase,  practitioners  create  a “field  of  action”  that 
becomes  the  learning  and  working  space  in  which  they  can  reflect,  practice  and  apply  what  they  know  and  how  they 
implement  technology  in  teaching  and  learning.  Internationalization  can  also  take  place  by  taking  online  technology 
integration  courses. 


Tacit  knowledge  To  Explicit  knowledge 
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Figure  1.  Knowledge  creation  as  an  ongoing  cycle  of  knowledge  conversion 


The  knowledge  management  framework  highlights  the  essential  components  of  the  KITE  project  that  need  to  be 
developed,  implemented,  and  evaluated.  Building  upon  this  framework,  the  project  involves  a consortium  of  8 
teacher  education  programs  collaborating  to  create  and  diffuse  technology  integration  knowledge.  As  the  lead 
institution  of  this  project.  The  University  of  Missouri-Columbia  enhances  and  continues  to  support  the  ongoing 
development  of  the  knowledge  repository.  Each  partner  institution  hire  knowledge  scouts  who  are  responsible  for 
collecting,  reporting,  and  updating  technology  integration  cases  in  the  knowledge  repository;  and  work  with  local 
technology  integration  practitioners  (teachers  and  teacher  educators)  as  knowledge  sources  to  capture  their 
technology  integration  knowledge  and  experiences.  The  overarching  mission  is  to  build  a k-16  technology 
integration  community  of  practice  through  this  knowledge  repository  that  enables  learning  through  sharing, 
communal  understanding  through  storytelling,  continuous  exchange  and  creation  of  new  knowledge,  and  collective 
problem  solving. 
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Abstact  Lesson  planning  is  considered  a critical  but  complex  task  to  attain  effective  instruction.  The 
traditional  paper-based  approach  was  found  to  be  cumbersome  and  consequently  detrimental  in  the 
willingness  and  effectiveness  of  teachers’  lesson  planning.  A Web-based  lesson  planning  system  combining 
instructional  planning  and  calendaring  functions  to  support  Missouri's  education  reforms  was  in  the  process 
of  development  and  testing.  Early  usability  tests  proved  the  system  to  be  positive  in  helping  Missouri 
teachers  plan  their  instruction.  Some  design  issues  were  identified. 


Introduction 

The  success  of  Missouri's  education  reforms,  to  improve  student  performance  as  measured  by  the  Missouri  Assessment 
Program  (MAP),  will  require  new  and  different  teaching  and  learning  strategies  that  align  curriculum  and  state  standards  using 
inquiry -based  instructional  strategies  and  infusing  instructional  technologies.  Successful  teaching  and  learning  does  not  occur  by 
chance.  It  relies  on  careful  planning  of  instruction.  Since  lesson  planning  is  considered  to  be  critical  to  the  success  or  failure  of 
teaching  and  student  learning,  it  is  vital  that  teachers  are  competent  in  planning  daily  lessons  that  support  Missouri's  education 
reforms. 

Lesson  planning  is  an  essential  but  complex  task  (Borko  & Livingston,  1989;  Doyle  & Holm,  1998;  Johnson,  2000;  Omstein, 
1997).  Various  theoretical  models  were  developed  to  help  teachers  plan  lessons  (Eisner,  1967;  MacDonald,  1965;  Tyler,  1950; 
Yinger,  1980).  While  numerous  studies  exist  to  discuss  lesson  planning  and  its  effects,  the  research  on  the  effect  of  technology  - 
based  lesson  planning  systems  is  conspicuously  lacking. 

The  use  of  technology  to  support  lesson  planning  is  not  new.  Several  systems  such  as  Computer-Prompted  Instructional 
Planning  System  and  Lesson  Plan  Maker  appeared  shortly  after  personal  computers  gained  acceptance  in  early  1980s.  Several 
studies  have  showed  the  positive  effect  of  using  personal  computers  to  support  lesson  planning.  Today,  the  advancement  and 
prevalence  of  technology  in  schools  makes  it  a very  viable  option  for  teachers  to  utilize  in  planning,  sharing,  and  communicating 
instructional  information.  In  particular,  the  Web  allows  teachers  to  create  and  access  instructional  plans  without  time  and  space 
boundaries.  However,  a search  revealed  that  a Web-based  lesson  planning  system  combining  instructional  planning  and  calendaring 
functions  to  support  Missouri's  education  reforms  does  not  exist. 


Issues 

While  Missouri  has  undertaken  a State  initiative  to  support  Missouri  teachers  as  they  integrate  multimedia  technology  into 
inquiry -based,  student -centered  instructional  practices  (http://emints.more.net),  most  teachers  are  still  using  pencil  and  paper  to 
do  lesson  planning.  Major  drawbacks  of  this  paper-based  lesson  planning  include: 

• The  process  is  very  labor-intensive  and  cognitively  arduous.  The  task  becomes  perfunctory  to  many  teachers.  They  feel 
obligated  to  write  out  lesson  plans  only  because  it  is  required  by  the  administration. 

• There  are  no  systematic  prompts  or  built-in  links  to  information  that  can  guide  the  teachers’  lesson  planning  to  focus  on 
Missouri's  education  reforms. 

• Lesson  plans  are  not  accessible  to  colleagues  or  school  administrators  unless  special  requests  are  made.  Consequently, 
sharing  instructional  information  among  teachers  is  difficult. 

• Parents  have  no  access  to  lesson  plans  and,  as  a result,  must  rely  on  other  methods  to  know  what  their  children  are  learning 
on  a daily  basis. 

• It  is  difficult  to  modify  the  contents  of  lesson  plans  once  they  are  written  on  paper. 

• Paper  is  not  very  durable  and  thus,  valuable  instructional  information  can  be  easily  damaged. 


Implementation  and  Preliminary  Results 


To  help  resolve  these  issues  in  lesson  planning,  a Web-based  lesson  planning  system  is  under  development  with  “teacher 
friendly”  tools.  The  objectives  of  the  system  are  threefold.  First,  it  streamlines  the  lesson  planning  process  to  align  with 
curriculum  and  Missouri  standards  and  provides  guidance  with  checklist  functions  to  help  teachers  focus  on  addressing  Missouri’s 
education  reforms.  Second,  it  preserves  valuable  lesson  plans  developed  by  teachers  for  dissemination  and  facilitates  the  sharing 
of  lesson  planning  knowledge  and  skills  among  teachers.  Third,  it  promotes  better  lesson  planning  results  through  sharing  and 
collaboration  and  enhances  communication  between  parents  and  schools,  thus  increasing  parental  involvement  in  their  children's 
education. 

This  Web-based  lesson  planning  system  is  a work  in  progress;  consequently,  only  formative  evaluation  is  available.  To  date, 
a preliminary  working  system  developed  in  Lotus  Domino  has  gone  through  several  rounds  of  prototyping  and  usability  testing 
to  collect  user  feedback.  Early  indicators  suggest  the  system  will  be  particularly  helpful  in  the  areas  of: 

• searching  and  adapting  existing  lesson  plans  for  other  uses, 

• communicating  instructional  plans  with  other  teachers  and  students,  and 

• developing  lesson  plans  to  meet  Missouri  academic  standards. 

User  feedback  also  reveals  several  issues  that  need  to  be  resolved  before  the  objectives  of  the  system  can  be  fully  realized. 
Among  them,  the  following  two  are  most  critical: 

• The  system  has  to  be  flexible  to  accommodate  different  instructional  needs  and  processes  of  schools  and  teachers. 

• Other  non-technical  arrangements  such  as  training  and  support  have  to  be  in  place  to  motivate  teachers  to  use  the  system  and 
to  share  their  instructional  plans. 


Conclusion  and  future  research 

Preparing  teachers  for  technology  integration  in  their  instruction  and  enhancing  teachers*  instructional  quality  are  two 
important  endeavors  to  advance  K-12  education  reforms.  This  initial  phase  of  this  project  is  to  develop,  implement,  and  pilot  test 
the  lesson  planning  functions.  Later  phases  will  include  additional  functions  to  support  communication  with  parents  via  the  Web 
and  to  scale-up  the  lesson  planning  model  for  statewide  access  and  use,  and  to  disseminate  to  a national  audience.  From  a research 
perspective,  this  project  will  lead  to  a better  understanding  of: 

1 . How  teachers  plan  instruction  using  technology? 

2.  How  teachers  share  instructional  information  through  technology,  how  technology  can  assist  with  instructional  planning? 

3.  How  the  Web  can  serve  as  a communication  means  between  parents  and  schools? 

The  on-going  process  of  building  the  lesson  planning  system  and  preserving  instructional  plans  in  a sharable  library  enables 
teachers  to  better  plan  their  instruction  and  share  their  instructional  materials.  As  the  system  matures  and  the  content  grows,  it 
will  have  the  potential  of  improving  instructional  planning  process  and  products  of  Missouri  teachers  and  thereby  advancing  the 
entire  K-12  education  enterprise  at  Missouri. 
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Investigating  Education  Faculty’s  Perspectives  of  Their 
Experiences  in  a Technology  Project: 

Issues  and  Problems  Related  to  Technology  Integration 
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Abstract:  The  ISTE  (1999)  stated  it  is  important  to  study  models  that  are  making 
effective  use  and  integration  of  technology  in  teacher  education  on  an  ongoing  basis.  The 
purpose  of  this  study  is  to  respond  to  this  need  by  portraying  three  education  faculty 
members  after  they  experienced  a federally  funded  technology -based  project.  Their 
practices,  beliefs,  and  experiences  are  the  focus  of  this  study.  Findings  include  1)  the 
range  of  previous  experiences  with  technology  varies  dramatically,  but  all  participants 
have  used  some  computers  for  personal  and  professional  tasks;  2)  although  actively 
learning  to  integrate  technology  and  seeing  positive  impacts  on  their  students,  they  do  not 
believe  that  technology  is  the  only  tool  for  effective  instruction;  and  3)  gender,  agp,  social 
networking,  access  to  current  hardware  and  access  to  technical  support  remain  issues 
needing  consideration  in  the  design  of  technology  implementation  in  higher  education  . 
settings. 


Many  educators  today  are  facing  the  issue  of  integrating  technology  into  their  instruction.  They  no 
longer  limit  themselves  to  textbooks,  chalk,  and  handouts  to  deliver  course  content  because  they  are  facing 
an  increasing  need  to  adjust  their  approaches  to  “teaching,  preparing  contents  and  delivering  learning 
materials  in  accordance  with”  emerging  technologies  (Neo  & Neo,  2001,  p.  328).  Some  teachers  have 
experienced  great  success  in  making  effective  use  and  integration  of  technology,  but  many  teachers  are  still 
struggling  with  technology.  This  paper  examines  the  beliefs  and  practices  of  education  faculty  regarding 
their  integration  and  implementation  of  technology.  In  particular,  what  are  the  perceptions  of  their 
experiences  after  they  participated  in  a year-long  project  to  implement  technology  in  their  instruction? 


Literature  Review 

Most  faculty  members  have  realized  the  positive  impact  that  technology  brings  to  students  and 
believe  that  it  is  important  to  increase  technology  integration  in  the  classroom.  However,  there  is  a 
significant  gap  between  their  beliefs  and  their  practices.  The  factors  underlying  this  situation  include 
teachers’  beliefs,  fears  of  technology,  reluctance  to  change,  lack  of  access,  and  lack  of  technical  and 
administrative  support  (Ertmer,  1999).  Both  external  and  internal  factors  may  impede  technology 
integration;  external  factors  include  access  to  technology,  funding,  and  administrative  and  technical 
support,  while  internal  factors  are  intrinsic  to  teachers,  such  as  personal  fear  of  technology,  traditional 
teaching  style,  and  reluctance  to  adapt  (Ertmer,  1999).  The  internal  barriers  are  often  more  difficult  to 
identify  and  overcome  than  the  external  factors,  because  they  are  “less  tangible”  and  “more  personal  and 
deeply  ingrained”  (Ertmer,  p.  51),  and  these  barriers  can  completely  block  implementation.  Ertmer  found 
that  the  focus  on  helping  teachers  integrate  technology  has  been  on  grappling  with  the  external  factors  but 
training  programs  have  started  to  “incorporate  pedagogical  models  of  technology  use  as  one  means  of 
addressing  internal  barriers”  (p.  47).  Ertmer  also  addressed  the  complexities  of  relationships  between  the 
external  and  internal  factors  and  the  strategies  to  overcome  the  two  barriers  to  help  faculty  members 
incorporate  technology  into  their  classrooms  efficiently. 


What  should  faculty  do? 


First,  before  teachers  integrate  technology  into  the  classroom,  they  should  possess  proficiencies 
with  technology  and  develop  a desire  to  apply  technology  in  their  instruction.  This  desire  incorporates  the 
premise  that  they  have  been  provided  with  access  to  technology  resources,  technical  support  and  training. 
Second,  teachers  should  possess  the  knowledge  of  how  to  integrate  technology  into  their  classroom. 
Throwing  a computer  and  software  into  the  classroom  is  not  integration.  According  to  (Ertmer,  1999), 
successful  integration  involves  careful  evaluation  of  the  curriculum  and  learning  goals.  Teachers  should 
develop  careful  plans  for  improvement  and  assessment,  and  they  should  also  be  able  to  follow  models  that 
can  help  develop  their  own  plans  (Ertmer,  1999). 


What  can  school/university  systems  do? 

To  help  faculty  members  make  effective  use  of  technology  and  integrate  technology  into 
instruction,  systems  should  provide  proper  and  sufficient  training  and  technical  support.  Only  technology 
access  combined  with  technical  training,  support,  and  assistance  with  using  technology  will  serve  as  a 
significant  motivation.  Technology  integration  depends  on  facilities;  however,  other  factors  that  influence 
integrating  technology  include  faculty  professional  development  and  course-development  time,  which 
provide  opportunities  and  time  for  teachers  to  concentrate  on  technology  (ISTE,  1999). 

Giving  teachers  sufficient  time  to  plan  students’  technology  experiences  is  one  practice  that 
facilitates  technology  integration.  Teachers  need  to  spend  sufficient  time  planning  how  technology 
integrated  is  going  to  function  most  effectively,  and  determine  the  best  practice  to  enhance  students* 
learning  outcomes  (Walker,  Ennis -Cole,  & Ennis  III,  2000).  Teachers  need  to  consider  the  necessary  course 
content  to  be  delivered  in  a normal  class  and  to  evaluate  the  impact  that  technology  will  have  on  teaching 
and  learning  (Lawler,  Rossett  & Hoffman,  1998;  Walker,  Ennis -Cole,  & Ennis  III,  2000). 


Background  of  the  Project 

This  project,  according  to  Speaker  and  Dermody  (Speaker,  et  al,  2001)  is  part  of  a $1.2  million 
grant  from  Preparing  Tomorrow’s  Teachers  to  Use  Technology  (PT3),  initiative  of  the  U.S.  Department  of 
Education.  The  consortium,  focused  on  strategically  implementing  technology  in  the  classroom,  is 
comprised  of  six  partner  schools  working  closely  with  two  universities.  The  major  purpose  (Speaker,  et  al, 
2001)  is  to  make  modem  multimedia  computer  technology  ubiquitous  in  content  and  methods  courses,  and 
field  experiences  for  future  teachers.  Faculty  and  students  in  teams  from  the  two  institutions  participate  in 
the  project  and  implement  instruction  using  technology  in  their  courses  and  field  experiences  with  children 
in  partner  schools.  This  project  (Speaker,  et  al,  2001)  uses  the  model  of  teachers  teaching  teachers 
developed  initially  in  the  Bay  Area  Writing  Project  to  produce  teaching  with  technology.  Over  last  two 
years,  the  project  has  involved  40  university  faculty,  approximately  100  university  students  as  student 
technology  mentors  (STM),  more  than  2,000  other  university  students,  and  approximately  2,250  learners  in 
40  classrooms  in  6 schools  in  the  area.  This  project  focuses  on  the  training  of  both  pre -service  teachers  and 
university  faculty  through  summer  institutes,  as  well  as  through  numerous  mini-session  trainings,  one-on- 
one  tutoring  and  mentoring.  These  training  courses  cover  a range  of  topics  including  web  design,  Internet 
in  the  classroom,  presentation  software,  integrated  curriculum,  and  video  on  the  web  (Speaker,  et  al,  2001). 


Methodology 

In  this  project,  faculty  members  are  categorized  as  Faculty  Technology  Participant  (FTP),  and 
student  participants  are  categorized  as  Student  Technology  Mentor  (STM).  Each  faculty  member  was 
assigned  an  STM  to  assist  with  teaching  and  preparation.  Both  FTPs  and  STMs  attended  a summer 
institute,  in  which  they  learned  technology  from  invited  teachers,  students  and  faculty  members  who  are 
using  technology.  These  presenters  share  with  their  experience  and  demonstrate  how  they  model 
technology  implementation  in  their  classrooms.  Faculty  and  students  completed  individual  projects  using 
technology  or  completed  a project  on  a cooperative  basis.  Processes  began  with  a simple  form  to  collect 
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some  demographic  information  from  three  FTPs.  After  that,  course  syllabi,  which  faculty  developed  after 
they  participated  the  summer  institute,  were  analyzed.  Based  on  the  analysis  and  review  of  literature,  we 
developed  interview  questions  to  delve  deeper  into  the  personal  world  of  faculty  members  who  are 
integrating  technology  into  their  classrooms. 

Interpretivism,  a “philosophy  of  science  related  to  constructivist  theory  in  psychology  and  to 
forms  of  research”  (Willis,  Thompson,  & Sadera,  1999,  p.  34),  best  suits  this  study.  Interpretivists  believe 
that  it  is  difficult  to  find  make  valid  generalizations  about  human  behavior  in  social  science  due  to  the 
individual  differences.  Instead  of  making  generalizations,  Willis,  Thompson,  and  Sadera  indicated  that 
interpretivism  emphasizes  understanding  the  individuals  and  interpreting  them.  What  is  true  for  one  group 
does  not  necessarily  stand  true  for  another  group  since  each  group  might  be  unique  in  certain  ways  (Willis, 
Thompson,  & Sadera,  1999).  The  reasons  to  choose  qualitative  rather  than  quantitative  design  are  manifold. 
First,  our  personal  biases  towards  qualitative  research  present  it  as  a flexible  and  non-  restrictive  method, 
which  we  can  use  to  understand  individuals*  perceptions  and  processes.  Second,  our  research  question 
determined  that  this  study  should  use  qualitative  design  because  the  purpose  of  this  study  was  to  investigate 
the  perspectives  of  education  faculty  members*  experience  in  a technology  project.  This  study  was 
completed  in  a natural  setting  without  the  involvement  of  experimental  and  control  groups.  Third,  since  the 
number  of  faculty  members  who  have  participated  in  this  project  is  relatively  small,  it  is  not  possible  to 
obtain  a sample  size  to  make  valid  and  reliable  quantitative  generalizations.  Fourth,  ISTE  (1999)  suggests 
that  future  researchers  should  observe  classroom  activities,  interview  teachers,  and  analyze  technology- 
based  lesson  plans  to  seek  concrete  evidence  of  effectiveness. 


Process 


Three  faculty  members,  two  female  and  one  male,  who  had  participated  in  the  project  gave 
consent  to  be  interviewed.  Each  interview  lasted  thirty  minutes  to  an  hour.  Semi-structured  interviews  were 
based  on  prepared  a list  of  interview  questions  and  audio  taped.  After  transcription,  the  data  were 
synthesized  with  demographic  information,  the  electronic  journals  from  the  summer  institute,  and  course 
syllabi  to  confirm  the  analyses.  Transcripts  and  other  data  were  coded  as  themes  emerged.  Follow-up 
interviews  with  member  checking  of  the  interpretations  occurred  after  the  initial  data  reduction. 


Findings 

There  are  some  commonalities  among  the  three  faculty  members  and  individual  differences  as 
well.  The  similarities  and  differences  include  previous  experiences  with  technology,  attitudes  toward 
participation,  beliefs  and  practices,  technologies*  impacts  on  students,  and  issues  relating  to  gender,  age, 
social  networking,  access  to  current  hardware  and  access  to  technical  support. 


Previous  Experiences 

All  three  teachers  had  some  experiences  with  technology  prior  to  the  participation  of  the  grant 
project.  Their  experiences  ranged  from  word  processing  and  Internet  browsing  to  PowerPoint  presentations 
and  web  page  design.  Each  had  an  office  computer  with  access  to  a university  local  area  network  and  e- 
mail.  They  had  access  to  two  computer  labs  in  the  building  and  supposed  access  to  a faculty  multimedia 
development  lab  which  was  usually  locked,  unavailable  and  unstaffed.  Two  of  the  faculty  members  mainly 
used  computers  for  word  processing,  emails  for  communication  and  Internet  browsing  to  search  for 
information  for  their  teaching  and  research.  They  did  not  develop  web  pages  prior  to  the  project.  The  third 
faculty  developed  web  pages  using  HTML  prior  to  the  project  and  posted  his  course  syllabi  online  and  links 
for  students  to  retrieve  information  about  how  to  conduct  research.  He  used  some  software  similar  to 
PowerPoint  prior  to  the  project  in  his  classrooms. 
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Attitudes  Toward  the  Participation 


The  three  faculty  members  demonstrated  satisfaction  with  the  whole  experience  of  the  project. 
One  is  very  happy  to  see  her  accomplishments  in  terms  of  products.  The  other  two  consider  the  project  has 
provided  a large  variety  of  resources  and  support  to  motivate  their  learning,  interests  and  rethinking  of  the 
goals  of  their  instruction.  “But  I might  argue  that  the  project  workshop  in  that  summer  was  again  the 
catalyst  that  recognize  in  my  mind  ...  (it)  allowed  me  to  recognize  in  my  mind  the  limitations  that  ...  are 
associated  with  all  the  alternatives  that  I have  available  to  me.  The  workshop  did  serve  as  a catalyst  to  at 
least  allow  me  to  understand  what  my  needs  were  and  how  I could  most  effectively  meet  those  needs, 
helping  me  because  it  forced  me  to  rethink  what  I want  to  present  in  class.”  (Dr.  Swanson,  male).  Walker, 
Ennis -Cole,  and  Ennis  III  (2000)  say  that  teachers’  positive  experience  in  workshops  and  intensive 
institutes  helps  promote  a sense  of  accomplishment  or  self-efficacy. 

During  their  participation,  each  of  the  three  faculty  members  was  assigned  one  student  technology 
mentor.  All  the  three  faculty  members  reported  that  their  student  technology  mentors  basically  did  not 
produce  anything  that  they  expected,  due  to  mentors’  lack  of  technological  competency  and  proficiency, 
and  conflicts  of  time  schedule  for  the  faculty  and  the  student  mentor  t)  work  together.  So  the  mentor 
relationships  were  not  productive  according  to  the  three  faculty  members.  “And  then  I was  assigned  a 
person  who  is  still  in  the  program.  She  knew  less  than  I did.  Well,  she  is  a marvelous  person,  did 
subsequently  take  my  literature  class.  We  are  still  working  on...um...she  didn’t  do  much  to  help.  . I met 
her  and  she  joined  in.  Whenever  I ask  questions,  she  would  certainly  attempt  to  figure  it  out  or  tempted  to 
work  on  it,  but  basically  I went  to  (person  a).  Every  problem  I have,  I just  go  to  (person  a),  who  is  not 
involved  in  the  grant  at  all.  He  is  a computer  support  person.  I went  to  him  prior  to  the  grant  and  I continue 
to  use  him  as  the  chief  source.  I did  ask  my  mentor  to  do  a couple  of  projects  for  me.  I don’t  want  to  give 
the  impression  that  she  is  not  completely  compliant,  willing,  certainly  she  is  and  would  independently  go 
out  and  get  her  own  help.  I would  ask  her  this  is  what  I need  in  excel  and  later  I would  have  it  in  my  hands. 
And  I wanted  to  have  a powperpoint  presentation  made,  and  she  did  that.”  (Dr.  Bonnett,  female). 


Beliefs  and  Practices 

All  three  faculty  members  believe  that  technology  is  not  THE  answer  to  instruction.  The  purpose 
of  using  technology  is  to  enhance  teaching  and  learning.  Technology  is  a tool,  and  so  if  one  particular  type 
of  technology  gets  in  the  way,  faculty  will  not  use  it;  instead,  they  will  seek  other  alternatives  to  improve 
their  teaching.  These  three  faculty  members  take  a strong  critical/evaluative  view  of  the  functions  of 
technology  which  they  are  implementing  in  their  instruction. 

It  is  not  surprising  to  find  out  that  faculty’s  personal  interest  in  technology  is  highly  related  to  their 
motivation  and  adoption  of  technology.  Therefore,  the  more  interest  one  has,  the  easier  it  is  to  learn  and 
incorporate  technology  into  practice.  Personal  interest  is  also  related  to  self-confidence  and  comfort.  If 
individuals  do  not  have  an  interest  in  learning  technology  and  bringing  technology  into  their  classrooms, 
they  will  be  reluctant  to  change  practices  and  resist  learning  technology  and  technology  infusion.  When 
resistance  exists,  the  level  of  confidence  and  comfort  is  crucial,  influencing  processes  even  though  a faculty 
member  may  feel  forced  to  bring  technology  into  classroom  to  keep  up  with  times. 

After  they  participated  in  the  technology-based  project,  the  two  female  faculty  members  started  to 
integrate  technology  into  their  classrooms.  They  developed  their  web  pages  for  easy  communication, 
information  delivery  and  their  own  research.  The  male  faculty  member  continually  was  working  on  his  web 
pages  to  convert  his  website  into  a more  non-linear  site.  He  learned  PowerPoint  in  the  project  and 
integrated  it  heavily  into  his  classes. 


Technology’s  Impact  on  Students 

All  faculty  members  see  technology  as  a positive  impact  on  students’  learning.  Through 
technology,  students  are  no  longer  limited  in  their  communication  with  peers  and  instructors.  By  using 
technology  in  their  learning,  students  demonstrate  enhanced  learning  outcomes  in  terms  of  activities  and 
products.  One  faculty  member  mentioned  teaching  preservice  teachers  by  modeling  and  requiring 
technology  use  in  instruction  so  that  preservice  teachers  can  follow  and  integrate  technology  into  their 
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academic  activities  and  future  classrooms.  By  watching  how  the  instructors  model  the  use  of  technology, 
students  are  able  to  use  technology  in  their  own  academic  activities  and  explore  different  learning 
resources.  Dawson  and  Norris  report  (2000)  that  technology  integrated  into  classroom  by  the  instructor 
increased  the  possibility  that  preservice  teachers  transfer  the  computer  skills  into  their  classroom  as 
compared  to  preservice  teachers  who  learned  computer  skills  in  an  isolated  manner. 


One  interesting  finding  is  that  among  the  three  participants,  the  male  demonstrated  higher 
motivation  than  female  faculty  members,  and  he  is  also  the  youngest  (between  35  and  45;  both  female 
participants  were  over  45).  He  had  already  sought  and  received  grants  for  further  technology 
implementation  in  his  courses.  Ertmer  and  Hruskocy  (1999)  said  the  challenge  that  teachers  face  is  to 
change  the  way  they  teach,  which  is  often  entrenched.  In  considering  the  idea  of  integrating  technology  into 
their  instructional  activities,  many  teachers  find  it  difficult  to  confront  their  established  beliefs  about 
instruction  and  their  traditional  roles  as  classroom  teachers.  In  addition,  Becker  (1994)  reported  that 
teachers’  practices  incorporating  technology  are  not  only  dependent  on  teachers’  personal  interests  in 
technological  activities,  but  also  highly  correlated  with  gender,  and  this  might  be  related  to  our  culture,  in 
which  more  males  have  patterns  of  personal  interest  that  are  “technical,  mechanical,  and  numerical”  (p. 
311),  which  is  consistent  with  having  a deep  interest  in  technology,  specifically  with  computers.  In  this 
study,  personal  interest  on  an  individual  basis  and  gender  basis  has  been  identified  as  a major  issue  in 
promoting  faculty  members  to  use  technology  and  incorporate  technology  into  classrooms. 

Other  issues  arise  when  faculty  deal  with  technology.  The  two  female  faculty  feel  time  is  a 
problem  because  it  always  requires  a large  amount  of  time  for  them  to  deal  with  hardware  and  software.  In 
Becker’s  report  (1999),  he  distinguished  difference  among  exemplary  technology  uses  between  female  and 
male  teachers.  Becker  indicated  that  female  teachers  usually  have  higher  demands  on  time.  If  faculty 
members  are  not  allowed  sufficient  time  to  construct  their  lesson  plans,  they  may  find  integrating 
technology  into  classrooms  ineffective. 

Another  factor  is  the  faculty  member’s  social  network.  The  faculty  member  who  is  not  very 
interested  in  technology  and  has  experienced  frustration  mentioned  that  friends  in  some  schools  simply 
gave  up  whenever  they  encountered  problems  with  technology,  even  without  bothering  to  seek  technical 
support.  Presenters  that  this  faculty  member  has  met  do  not  use  Powerpoint  to  do  presentations;  instead, 
they  use  the  traditional  slideshows,  so  peer’s  views  and  practices  influence  faculty  members’  beliefs  and 
practices. 

In  addition  to  requiring  time,  the  male  faculty  member  thinks  it  is  important  to  update  hardware  so 
that  it  can  meet  the  increasing  needs.  Faculty  think  that  there  are  always  problems  with  technology  that 
they  do  not  have  sufficient  technical  skills  to  solve,  so  they  are  highly  dependent  on  someone  who  is 
technologically  savvy  and  has  the  expertise  to  deal  with  the  situation.  Faculty  members  feel  that  there 
should  be  someone  to  whom  they  can  turn  whenever  they  encounter  technological  problems.  In  this  sense, 
technical  support  availability  plays  an  important  role  in  helping  faculty  members;  it  eases  the  pressures  of 
learning  new  things  and  thus  helps  them  integrate  technology  into  their  classrooms.  Becker  (1994)  said 
available  access  to  someone  from  whom  teachers  can  learn  or  share  experience  with  is  very  important  in 
helping  teachers  incorporate  technology. 


Conclusions  and  Implications  for  Future  Research 

Three  participants  were  involved  in  this  study,  two  females  and  one  male.  Dr.  Swanson  and  Dr. 
Bonnet  are  at  their  forties  and  fifties,  while  Dr.  Greene  is  at  her  thirties  and  forties.  Their  teaching  years  of 
experience  in  higher  education  range  from  five  to  thirteen  years.  Although  these  faculty  members  had 
different  experiences  prior  to  the  project,  their  attitudes  toward  participant,  and  beliefs  and  practices  are 
undergoing  changes  as  they  use  technology.  They  are  either  adding  to  their  technology  use  or  rethinking  it, 
seeing  the  need  to  consider  the  technology  as  a tool  through  a critical  evaluative  lens.  Issues  continue  to 
exist  related  to  gender,  age,  access  and  currency  of  technology.  Technologies  have  significant  impact  on 
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both  teachers  and  students.  They  help  both  teachers  and  students  reconceptualize  their  ways  of  working  and 
thinking  and  strengthen  the  relations  with  the  rapidly  changing  world.  However,  past  studies  show  that 
teachers*  professional  development  has  not  “kept  pace  with  the  rapid  changes  in  the  quality  and  quantity  of 
information  technology”  (ISTE,  1999,  p.  1).  Factors  that  impede  effective  use  and  integration  of  technology 
have  been  identified  as  both  external  and  internal  factors.  Future  research  should  be  more  focused  on 
factors  that  influenced  faculty’s  use  of  technology  and  integration  from  studying  both  successful  examples 
and  unsuccessful  examples,  and  these  designs  should  balance  gender  and  age. 
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Abstract:  This  study  will  investigate  student  perceptions  of  two  different  methods  of 
submitting  multimedia  class  projects  in  a graduate  educational  technology  course.  All  students 
will  create  a variety  of  multimedia  files  as  standard  course  requirements.  One  group  will  be 
required  to  submit  the  projects  physically,  by  directly  providing  to  the  course  instructor  paper 
or  portable  media  containing  the  files.  The  other  group  will  be  required  to  submit  the  projects 
online,  storing  them  in  a central  “^Portfolio”  area  on  a class  file  server.  Student  attitudes  and 
perceptions  toward  these  two  methods  of  project  submission  will  be  studied,  along  with  their 
relative  susceptibility  to  error  and  malfunction. 


Introduction 

Electronic  portfolios  are  student  portfolios  which  incorporate  multimedia  components.  Wiedmer 
(1998)  described  a digital  (electronic)  portfolio  as  a learner's  compilation  of  work  growth  in  one  or  more  fields 
which  is  developed  and  exhibited  by  electronic  media.  These  media  may  include  diskettes.  Zip,  Syquest,  and 
Jaz  drives;  compact  disk  recordable  (CD-R);  local  area  networks  (Farmer,  1997);  and  the  World  Wide  Web 
(Kahtani,  1999).  Electronic  portfolios  can  reduce  physical  storage  necessities  (Moersch  & Fisher,  1995),  and 
encourage  students  to  take  more  responsibility  for  themselves  and  perform  better  in  self-assessment  (Tancock 
&Ford,  1996). 

Little  research  known  to  the  author  has  yet  addressed  the  use  of  a remote  file  server  as  a medium  for 
creating  or  storing  ^Portfolios.  Such  a “remote  ^Portfolio”  is  accessible  anywhere  at  anytime,  and  is  defined  for 
the  purposes  of  this  study  as  a multimedia  portfolio  created  and  maintained  exclusively  on  a remote  file  server 
accessed  by  students  using  a computer  network  (including  but  not  limited  to  the  Internet). 

With  respect  to  remote  ^Portfolios,  this  project  proposes  four  research  questions.  Each  focuses  on  the 
difference  between  having  students  submit  multimedia  projects  via  portable  media  (e.g.,  papers;  floppy  disks; 
SuperDisks;  Zip  disks;  CD-ROMs;  etc.)  and  having  students  submit  multimedia  projects  online  to  a central 
server.  The  four  questions  are:  (1)  Is  there  a difference  in  student  attitudes  toward  these  two  methodologies?  (2) 
Is  there  a difference  in  student  perceptions  of  these  two  methodologies?  (3)  Which  method  of  submission  is 
more  prone  to  errors?  (4)  Which  method  of  submission  is  more  prone  to  malfunctions? 


Context 


The  study  will  be  held  in  a graduate  educational  technology  class  taught  at  a midwestem  urban  state 
university  during  the  Summer  2002  term.  Approximately  40  graduate  student  participants,  who  are  Prek-12 
educators,  from  two  sections  of  this  educational  technology  course  will  serve  as  subjects.  All  students  in  both 
course  sections  will  be  required  to  create  multimedia  ePortfolios.  All  students  in  both  course  sections  will  be 
instructed  in  the  use  of  a remote  file  server,  and  in  the  transfer  of  files  to  and  from  the  server  using  FTP. 

All  subjects  will  encounter  the  same  course  content,  learning  activities,  and  instructor  during  the  same 
term.  The  distinction  between  the  control  and  experimental  groups  will  be  in  the  ways  in  which  students  are 
required  to  maintain  and  submit  their  ^Portfolios.  The  control  group  will  be  required  to  maintain  and  submit 
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their  ^Portfolios  on  portable  media  only,  and  the  experimental  group  will  do  so  using  a remote  file  server 
(only). 


Data  Sources 

At  the  beginning  and  at  end  of  the  term,  subjects  will  complete  a questionnaire  dealing  both  with  their 
attitudes  toward  and  their  perceptions  of  electronic  submission  of  projects.  The  questionnaire  administered  at 
the  beginning  of  the  term  will  also  collect  information  about  students*  prior  computer  experience  and 
experience  with  online  file  transfer. 

The  first  research  question  regarding  attitudes  (e.g.,  preference,  satisfaction,  etc.)  and  the  second 
research  question  regarding  perceptions  (e.g.,  course  delivery,  knowledge  and  skills  learned,  etc.)  will  be 
addressed  using  data  from  these  questionnaires. 

The  third  of  the  research  questions  addresses  errors  (that  is,  human  mistakes)  in  the  project  submission 
process.  Students  in  both  groups  will  be  asked  to  record  and  report  any  mistakes  or  errors  they  are  aware  of 
making  during  the  process  of  creating,  maintaining,  and  submitting  heir  ^Portfolios;  the  instructor  will 
maintain  an  equivalent  log. 

The  fourth  of  the  research  questions  addresses  malfunctions  (that  is,  hardware/software  malfunctions) 
in  the  project  submission  process.  Students  in  both  groups  will  be  asked  to  record  and  report  any  software, 
network  or  equipment  malfunctions  they  experience  in  the  process  of  creating,  maintaining,  and  submitting 
their  ^Portfolios.  As  with  the  third  question,  the  instructor  will  also  log  such  malfunctions. 

An  Analysis  of  Variance  (ANOVA)  model  will  be  used  to  test  the  significance  of  the  differences  in 
students*  attitudes,  perceptions,  errors,  and  malfunctions  in  multimedia  coursework  submission  between 
physical  (via  portable  media)  and  online  (via  remote  file  handling)  ^Portfolio  formats,  indents*  attitudes, 
perceptions,  errors,  and  malfunctions  are  the  primary  dependent  variables  while  the  method  of  ^Portfolio 
submission  (portable  media  vs.  remote  file  handling)  is  the  primary  independent  variable. 


Educational  Importance  of  the  Study 

The  study  is  educationally  significant  because  it  is  one  of  the  few  studies  investigating  the  difference 
between  coursework  submission  formats  (physical  vs.  online)  on  student  attitude,  perception,  error,  and 
malfunction.  Should  an  online  format  of  submissbn  prove  to  be  both  equally  effective  and  equally  popular, 
such  ^Portfolios  can  provide  a cumulative  multimedia  record  of  student  progress  and  outcomes  which  is 
continuously  maintained  and  improved,  as  well  as  accessible  to  both  the  instructor  and  student  from  all 
locations  around  the  clock. 
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Abstract:  This  study  explored  how  students’  understanding  of  instructional  design  developed  as  they 
participated  in  collaborative  learning  activities  in  an  electronic  apprenticeship  environment  during  a 
semester-long  course.  Novice  and  advanced  learners  analyzed  a set  of  instructional  design  case 
studies.  Novice  learners  also  developed  instructional  design  projects  with  feedback  from  the  advanced 
learners.  By  engaging  in  joint  case  analyses  and  design  reviews  it  is  expected  that  both  groups  of 
learners  will  increase  their  understanding  of  the  ID  process. 


Theoretical  Framework 

Literature  shows  that  an  apprenticeship  environment  provides  students  with  opportunities  to  apply  expert 
problem-solving  strategies.  Collins  and  his  colleagues  argued  that  cognitive  apprenticeships,  in  contrast  to  traditional 
apprenticeships,  could  most  efficiently  occur  in  school  settings  (Collins,  Brown,  & Holum,  1991).  In  a cognitive 
apprenticeship  environment,  the  goal  is  to  make  visible  and  explicit  complex  thinking  and  problem-solving  strategies 
that  experts  use  in  particular  domains.  By  combining  six  distinct  teaching  strategies,  students  can  conceptualize  their 
knowledge  and  use  that  knowledge  in  a variety  of  settings  (Collins,  Brown,  & Newman,  1989).  The  six  methods 
include:  modeling,  coaching,  scaffolding,  articulation,  reflection,  and  exploration. 

Along  with  these  methods  of  cognitive  apprenticeships  is  the  concept  called  authentic  activity — the  use  of 
ordinary  practices  of  a given  culture  as  a basis  for  learning  (Brown,  Collins,  & Duguid,  1989).  One  promising  approach 
to  initiating  authentic,  contextual  learning  is  case-based  instruction. 

Case-based  instruction  has  been  used  in  professional  preparation  programs  to  make  a bridge  between 
classroom  and  field.  Recently  case-based  instruction  was  introduced  into  instructional  design  field  (Ertmer  & Quinn, 
1999).  The  effective  use  of  cases  requires  students  to  apply  critical  thinking  to  a problem  situation  to  analyze  the 
situation  and  recommend  realistic  solutions  through  better  understanding  (Ertmer  & Russell,  1995;  Shulman,  1992; 
Wright,  1996). 

Flynn  and  Klein  (2001)  examined  the  role  of  group  discussbn  in  case-based  instruction.  Participants  working 
in  groups  performed  better  while  analyzing  cases  than  those  who  worked  alone.  Other  researchers  (Griffith  & 
Laframboise,  1997;  Levin,  1995)  found  that,  for  experienced  learners,  discussion  appeared  to  act  as  a catalyst  for 
reflection  based  more  on  personal  experiences  than  on  theory  and  course  content,  while  for  less  experienced  learners, 
discussion  appeared  to  allow  them  to  clarify  or  elaborate  their  understanding  and  increase  their  perspective  on  the  issues 
in  the  cases. 

Simply  stated,  case-based  instruction  becomes  more  effective  when  learners  at  different  levels  work 
collaboratively  in  an  apprenticeship  environment.  However,  previous  research  focused  more  on  the  growth  in  novice 
learners’  problem-solving  expertise  and  their  benefits  from  this  learning  experience.  Few  of  them  explored  how 
advanced  learners  perceived  the  collaborative  experience  with  novice  learners.  Therefore,  it  is  important  to  examine 
how  both  novice  learners  and  advanced  learners  perceive  the  benefits  of  case-based  instruction  in  apprenticeship. 
Purpose 

This  study  examines  the  development  of  novice  and  advanced  students’  understanding  of  instructional  design 
during  one-semester  in  which  they  engaged  in  an  electronic  apprenticeship  environment.  To  be  specific,  this  study: 

• examines  the  benefits  to  novice  learners  who  participate  in  case-based  collaborative  problem-solving  activities  and 
instructional  design  project  reviews  in  electronic  cognitive  apprenticeship  environment. 

• examines  the  perceptions  of  advanced  learners  who  participate  in  case-based  collaborative  problem-solving 
activities  and  mentoring  instructional  design  project  reviews  in  an  electronic  cognitive  apprenticeship  environment. 

Methods 

Graduate  students  (n=20)  at  one  Mid -western  university  enrolled  in  an  introductory  instructional  design  course 
(classified  as  novice  learners)  and  graduate  students  (n=l  1)  from  another  Mid-western  campus  enrolled  in  an  advanced 
instructional  design  course  (classified  as  advanced  learners)  participated  in  a semester-long  study.  Three  graduate 
students  from  the  latter  group  conducted  the  research.  Qualitative  data  were  gathered  from  discussion  board  postings 
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revolving  around  case  study  discussions,  instructional  design  project  reviews,  interviews,  and  an  end-of-course 
reflection. 

Throughout  the  semester  participants  analyzed  and  discussed  10  case  studies  via  discussion  board  as  part  of 
their  course  assignments.  During  the  online  discussion,  they  will  be  in  different  case  setup  including  role-play,  small 
group  discussion,  discussion  chain,  etc.  The  novice  learners  also  created  an  instructional  design  project  and  the 
advanced  learners  played  the  role  of  mentors  to  review  novice  learners’  projects.  The  novice  learners  will  also  be  asked 
to  write  a reflection  at  the  end  of  semester.  In  addition,  the  researchers  will  interview  the  rest  of  the  advanced  learners 
with  a semi -structured  interview  method.  With  their  permission,  the  interview  data  will  be  tape-recorded  and  then 
transcribed. 

A constant  comparison  analysis  method  (Guba  & Lincoln,  1989)  will  be  used  to  generate  the  categories  and 
patterns  from  qualitative  data.  Specifically,  discussion  board  postings  and  reflections  will  be  analyzed  to  examine 
novice  learners’  perceptions  of  their  benefits  from  this  learning  experience.  The  advanced  learners’  project  review 
postings  and  interview  data  will  be  analyzed  to  identify  the  advanced  learners’  perceptions  of  the  benefits  from  case- 
based  collaborative  problem-solving  activities  and  mentoring  instmctional  design  project  in  electronic  cognitive 
apprenticeship  environment. 

Results 

Preliminary  analysis  of  the  data  collected  from  the  first  two  online  case  study  discussions  and  the  first  instructional 
design  project  review  demonstrates  benefits  of  this  collaborative  learning  approach  to  be  perceived  by  both  novice 
learners  and  advanced  learners. 

While  there  were  no  obviously  different  patterns  found  in  the  approaches  to  case  studies  between  novice  learners 
and  advanced  learners,  the  instructional  design  project  review  showed  certain  differences  in  participants’  understanding 
of  instructional  design.  For  example,  the  novice  learners  were  easily  confused  by  miscellaneous  details  of  the  beginning 
two  steps  of  the  ID  model,  while  the  advanced  learners  considered  the  design  steps  in  the  whole  process  as  a “big 
picture”. 

The  analysis  of  the  participants’  discussion  postings  will  continue  throughout  the  semester.  It  is  expected  that  the 
novice  learners  will  demonstrate  improved  problem-solving  skills  and  better  understanding  of  instructional  design 
through  working  collaboratively  with  advanced  learners.  In  addition,  it  is  expected  that  advanced  learners  will 
consolidate  their  understanding  of  instructional  design  through  both  mentoring  novice  learners’  projects  and  applying 
their  previous  knowledge  and  skills  to  complex  tasks  in  case-based  discussions. 

Implications 

Although  this  study  is  limited  by  the  number  of  participants  and  the  inability  to  control  the  influence  of  other 
factors  on  students’  perceptions,  it  has  important  implications  for  apprenticeship  approach  in  building  instructional 
design  understanding.  It  is  hypothesized  that  both  novice  and  advanced  learners  benefit  from  collaborative  learning  in 
apprenticeship  environment.  It  is  usually  considered  that  novice  learners  benefit  the  most  from  such  learning 
experiences  because  they  are  more  likely  to  make  greater  progress  due  to  their  low  entry  knowledge  level.  However, 
this  paper  hopes  to  demonstrate  that  advanced  learners  can  also  benefit  much  through  authentic  problem-solving 
activities  and  mentoring  novice  learners.  Their  collaboration  further  indicates  that  the  cognitive  apprenticeship  provides 
an  effective  social  context  for  understanding  instructional  design. 
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Abstract:  A high  school  science  teacher  reported  that  the  students  have  motivation  and 
learning  problems  to  understand  the  concept  of  fossilization.  Working  with  the  science  teacher, 
a Web-Based  Learning  Environment  (Web-LE)  has  been  designed  by  a group  of  students  in  the 
Department  of  Instructional  Technology  at  University  of  Georgia  to  enhance  and  sustain  the 
motivation  of  learners  in  the  context  of  secondary  science  education.  The  cognitive  tool 
approach  is  employed  to  design  the  Web-LE  to  enhance  the  learners’  cognition  toward  the 
scientific  concept  of  fossilization.  Several  strategies  for  increasing  students’  intrinsic 
motivation  will  be  involved  in  the  Web-LE  design.  The  goal  is  to  find  the  practical  approaches 
to  design  the  online  fossil  course  that  should  be  shored  up  by  the  instructional  theories 
including  cooperative  learning  and  the  motivational  theories.  The  high  school  teachers  can  use 
the  W^LE  to  aid  the  science  subject  instruction  without  purchasing  software. 


Introduction 

This  Web-Based  Learning  Environment  (Web-LE)  is  being  designed  to  explore  the  unique  WWW 
features  and  utilize  these  elements  to  enhance  and  sustain  the  motivation  of  online  learners  in  the  context  of 
secondary  science  education.  The  goal  is  to  find  the  practical  approaches  to  design  the  online  fossilization  course 
that  should  be  shored  up  by  the  instructional  theories  including  cooperative  learning  and  the  motivational  theories. 
The  high  school  teachers  can  use  the  Web-LE  to  aid  the  science  subject  instruction  without  purchasing  software. 

Information  scientists  estimate  that  the  World  Wide  Web  (WWW)  now  has  nearly  one  billion  pages,  and 
the  WWW  continues  to  grow  at  a rapid  pace.  Although  many  web  sites  may  be  inaccurate,  misleading,  biased,  or 
even  criminal  in  nature,  there  are  still  millions  of  pages  that  high  school  students  and  their  teachers  can  access  to 
enhance  teaching  and  learning.  Unfortunately,  while  there  is  enthusiasm  in  many  quarters  about  the  integration  of 
the  WWW  into  education,  there  appears  to  be  a lack  of  substantive  thinking  about  the  goals,  pedagogical 
dimensions,  and  outcomes  of  using  the  web  in  secondary  education.  There  is  a strong  need  to  develop  and  test 
new  models  of  the  factors  that  will  enable  the  effective  use  of  the  WWW  in  high  schools.  One  such  model  is  based 
upon  the  concept  of  the  WWW  as  a “cognitive  tool”  ^^eeves  & Reeves,  1997).  Cognitive  tools  refer  to 
technologies  that  enhance  our  cognitive  powers  during  thinking,  problem-solving,  and  learning  (Jonassen  & 
Reeves,  1 996).  Using  the  WWW  as  a cognitive  tool,  students  tackle  difficult  problems  and  complex  tasks,  organize 
unique  knowledge  representations,  and  share  what  they  have  learned  with  others  for  analysis,  critique,  and 
revision. 

The  purpose  of  this  research  is  twofold.  First,  the  features  of  the  WWW  that  increase  and  sustain  the 
motivation  of  online  learners  will  be  identified  through  interviews  with  students,  teachers,  and  experts.  Second, 
using  development  research  processes,  a fossilization  Web-based  Learning  Environment  (Web-LE)  that 
incorporates  these  motivational  features  will  be  designed,  developed,  and  tested  in  the  context  of  high  school 
science. 

Background 


The  primary  setting  was  a local  private  school  located  in  a small  city  in  northern  Georgia.  This  school  is 
equipped  with  well-organized  computer  technology  including  computer  labs  and  wireless  network.  Students  and 
teachers  are  used  to  the  computer  environment  and  equipped  with  high  computer  literacy.  This  research  adopts 
purposive  sampling  method  based  on  the  specific  purposes  to  work  closely  with  the  science  teacher  and  students 
in  a local  school  to  promote  learners’  motivation  and  learning  achievement.  The  participants  are  one  male  teacher 
and  16  10th  grade  students.  The  teacher  is  an  experienced  science  instructor.  The  teacher  encountered  problems  to 
convey  the  concept  of  fossilization  to  students,  and  has  difficulty  to  motivate  students. 

The  science  teacher  reported  that  the  students  have  problems  to  understand  the  concept  of  fossilization. 
Fossilization  is  a result  of  complex  combination  of  organism,  ecological  and  physical  burial.  The  fossil  would  be 
formed  only  under  the  correct  situation.  Students  have  to  understand  that  how  and  why  ecological  and  physical 
burial  influence  fossilization.  The  teacher  couldn’t  find  accurate  or  useful  materials  to  help  students  visualize  the 
process  of  fossilization.  Some  web  sites  introducing  the  fossilization  contain  only  still  images,  pictures  and  texts, 
and  the  representations  are  not  realistic  enough  or  accurate  to  be  employed  in  the  classroom.  The  science  teacher 
needs  a tool  that  is  realistic  and  accurate  to  explain  the  fossilization  and  provides  opportunities  for  learners  to 
consider  the  potential  combinations  of  different  decision.  The  learner  will  be  able  to  identify  situations  that  will 
cause  fossilization.  After  evaluating  the  current  available  computer-learning  environment,  a Web-LE  has  been 
decided  to  be  employed  to  carry  the  instructional  contents. 


Employing  Web-LE  as  the  cognitive  tool 

Owston  (1997)  indicated  that  the  key  to  promoting  improved  learning  with  the  Web  appears  to  lie  in  how 
effectively  the  medium  is  exploit  d in  the  teaching  and  learning  situation.  The  standard  online  tools  mentioned 
above  offer  restricted  choice  to  instructors  seeking  to  offer  experiences  with  complexity  and  dynamics  of 
classroom  experiences,  and  we  need  new  tools  to  enable  a variety  of  interactions  (Hughes  & Hewson,  1998).  The 
advent  of  online  interactive  learning  system  enhances  the  interaction  in  the  learning  environment.  Participants  can 
interact  with  other  participants  to  discuss  the  questions  and  concepts.  With  authoring  tool  (e.g.,  Macromedia 
shockwave.  Flash,  Authorware,  JAVA),  instructor  can  design  the  synchronous  or  asynchronous  activities  based 
on  different  goals  and  tasks.  The  features  of  open  environment  and  immediate  communication  can  be  embedded 
in  the  application  to  enable  collaborative  learning  strategies.  The  ability  of  these  tools  to  carry  interactive 
multimedia  contents  support  design  of  situated  learning  strategies  by  simulation,  problem  solving  or  authentic 
activities  (Leflore,  2000).  The  features  of  dynamic  pages  allow  learners  interact  with  contents  directly  to  process 
the  information  individually.  Learners  will  be  benefited  by  the  delivery  of  information  with  appropriate 
instructional  design  instead  of  presenting  information  in  hypermedia  structure  and  multimedia  contents  (e.g., 
streaming  audio  and  video).  Web  pages  can  carry  multimedia  and  access  to  the  open  environment.  Learners  can 
interact  with  content  with  pages  (browsing,  filling  questionnaire . ..),  and  the  level  of  interactivity  depends  on  how 
the  designers  design  the  web  pages.  With  the  features  that  the  WWW  provide,  it  makes  Web  the  appropriate  tool 
to  carry  the  instructional  contents. 

Several  doctoral  and  master  students  have  been  working  together  to  design  the  fossilization  Web-LE  in 
the  Department  of  Instructional  Technology  at  University  of  Georgia  under  the  guidance  of  two  faculty  members. 
The  science  teacher  is  the  subject  expert,  who  works  closely  with  the  team  to  verify  the  accuracy  of  the  contents, 
identifies  the  educational  problems  and  provides  feasible  instructional  strategies.  The  system  is  developed  by 
Macromedia  Flash  and  combines  with  the  online  database  with  ASP  and  MS  Access. 


Instructional  design  strategies 

Several  strategies  for  increasing  students’  intrinsic  motivation  will  be  involved  in  the  Web-LE.  First, 
Lepper  and  Hodell’s  (1989)  four  characteristics  of  tasks  that  promote  individual  intrinsic  motivation  will  be 
integrated  into  the  instructional  design. 

1.  Challenge:  Engaging  with  the  activities  that  challenge  learners’  abilities  may  enhance  the  intrinsic 
motivation.  The  premise  is  that  the  challenges  of  activity  and  skills  of  learner  should  be  matched. 


2.  Curiosity:  Curiosity  can  be  achieved  by  using  the  technical  events  to  attract  the  learner’s  attention. 
Highlighting  incompleteness  or  inconsistency  is  one  technique  to  arouse  the  curiosity.  Through  the 
provision  of  unpredictable  or  random  events  may  motivate  learners  to  continue  the  learning  processes. 

3.  Control:  Learners’  intrinsic  motivation  may  be  enhanced  if  the  activities  can  provide  a sense  of 
control  over  their  learning  performance. 

4.  Fantasy:  Fantasy  environment  is  defined  by  Malone  and  Lepper  (1 983)  as  one  that  evokes  mental 
images  of  physical  or  social  situations  not  actually  present 

Second,  Web  has  the  capabilities  to  facilitate  the  communications  among  participants,  and  this  feature 
provides  the  opportunities  to  carry  out  the  cooperative  and  competitive  strategies,  whereas  the  open  environment 
feature  enables  learners  to  publish  their  products  or  achievement  and  have  people  recognize  their  performance. 
The  design  of  Web-LE  will  integrate  the  cooperation,  competition  and  recognition  strategies  to  facilitate 
collaborative  learning  pedagogy  and  stimulate  interpersonal  intrinsic  motivation  (Malone  and  Lepper,  1 983). 

The  third  strategy  is  to  integrate  multimedia  objects  into  learning  context.  Learning  with  multimedia 
provides  an  effective  alternative  instructional  strategy  (Mathewson,  1999).  From  the  motivational  perspective, 
using  medium  to  assist  instruction  indeed  enhance  learners’  interest  (Freeley,  1982;  Kramarski  & Feldman,  2000). 
Computer  enables  visualization  of  scientific  concept  and  Internet  enhances  interaction  among  learners.  The 
multimedia  objects  will  be  employed  to  represent  the  fossil  concepts  to  enhance  the  intrinsic  motivation. 


Overviews  of  the  system 


Instructional  content 

How  does  a living  thing  become  a fossil?  Not  all  parts  of  animals  and  plants  become  fossilized.  After 
completing  learning  the  concept  of  fossilization  with  this  cognitive  tool,  students  will  be  able  to  identify 
conditions  necessary  for  fossilization  and  construct  possible  scenarios  for  formation  of  fossils  by  manipulating  the 
variables  in  the  simulated  processes . Fossilization  is  a rare  event.  The  chances  of  a given  individual  being 
preserved  in  the  fossil  record  are  very  small.  Whether  a living  thing  could  become  a fossil  depends  on  three 
categories  of  conditions  (organism,  ecological  status  and  physical  burial),  if  an  organism  has  hard  part  or  not,  if  an 
organism  gets  buried  quickly  and  the  ecological  situation,  where  an  organism  dies.  It’s  only  possible  that  a living 
thing  can  become  a fossil  when  these  three  conditions  are  met. 


System  function 


Figure  1 : Flow  chart  of  the  fossil  Web-LE 

Figure  1 is  the  flow  chart  of  the  fossilization  Web-LE.  When  entering  the  Web-LE,  learners  will  see  the 
instructional  page  to  explain  the  learning  objectives  and  goals  of  the  fossil  unit.  Learners  have  to  input  names  and 


login  into  the  system.  In  screen  a,  learners  can  select  different  organism  to  observe  the  fossilization  process.  The 
dinosaur  is  the  only  available  organism  in  the  prototype.  The  description  of  this  organism  is  placed  under  the 
screen  (Screen  1).  Learners  can  access  to  the  online  glossary  to  look  up  information  if  they  have  any  questions.  In 
screen  b,  they  need  to  select  the  ecological  status  (Screen  2).  In  screen  c,  learners  need  to  select  physical  burial. 
The  combination  of  three  decisions  can  decide  whether  the  fossil  will  be  developed.  Simulation  of  fossilization 
begins  when  the  decision  has  been  made.  The  system  will  record  paths  that  learners  just  made  and  help  learners  to 
identify  the  learning  progress  (Screen  3).  After  experiencing  all  twelve  possible  paths,  learners  will  be  able  to 
identify  the  situations  that  can  create  fossils  (online  assessment). 

The  complete  twelve  possible  combination  including: 

1.  Organism 

2.  Ecological  (3  conditions):  temperate  rainforest,  tropical  rainforest  and  tropical  mountains 

3.  Physical  burial  (6  conditions):  weathering,  ashfall,  lava  flow,  pyroclastic,  flood  and  swamp  mud  burial. 

Learners  can  assess  to  online  glossary  to  look  up  information  about  these  conditions.  With  the 
communication  tools  that  the  Web-LE  provides,  learners  can  send  e-mail  to  other  learners  and  teacher.  They  can 
discuss  questions  with  the  bulletin  board. 


Media  development 

The  primary  audience  is  high  school  students  who  appeal  to  realistic  graphics  and  animation.  The 
science  teacher  also  requests  for  the  realistic  representation  of  the  fossilization  process.  Several  endeavors  are 
made  to  achieve  the  requirement.  Referring  the  accurate  images,  the  dinosaur  is  developed  with  accurate 
proportion  and  color.  3D  software  is  used  to  develop  the  realistic  landscapes  and  objects  including  ancient  plants, 
volcano,  lake  and  fossils.  Macromedia  Flash  is  the  primary  authoring  tool  because  of  its  ability  of  integrating 
multimedia  and  the  capability  of  optimizing  media  to  enhance  the  speed  of  transmission. 


Screens  example 


Screen  3:  Begin  simulation  of  fossilization 


Screen  4:  Use  “my  path”  to  record  selections 


Learners  need  to  select  organism  in  Screen  1 . The  description  of  this  organism  will  appear  in  the  text 
area.  Learners  should  continue  to  select  ecological  (Screen  2)  and  physical  burial  conditions.  When  learners  click 
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“submit”  button,  the  simulation  of  fossilization  will  begin.  An  animation  will  appear  to  depict  the  procedures  of 
fossilization.  For  instance,  in  Screen  3,  learner  chooses  combination  of  “ Brachiosaurus”,  “Tropical  Mountain” 
and  “Lava  Flow”.  When  simulation  begins,  the  volcano  explodes  and  the  lava  catches  up  the  dinosaur  to  mantle 
its  body.  The  fossil  will  be  formed  in  this  situation.  In  Screen  4,  learners  can  assess  to  the  communication  tools 
(e-mail  and  discussion  board).  The  function  of  “my  paths”  will  record  the  paths  that  the  learners  have  made  and 
identify  their  learning  progress. 


Validation 


The  formative  evaluation  strategy  is  employed  to  make  sure  the  content  is  accurate.  The  subject  expert 
reviews  the  contents  once  a week  and  provides  suggestion  constantly.  After  the  production  is  completed,  the 
participants  will  be  formed  into  one  control  group  and  one  treatment  group.  Several  instruments  will  be  employed 
to  exam  the  effects  of  the  Web-LE  on  motivation,  science  and  learning  with  Web-LE.  The  following  interview 
with  teachers  and  treatment  group  students  will  be  conducted  to  probe  their  suggestions  toward  the  Web-LE. 


Expected  Results 

The  expected  results  include:  1)  effect  of  Web-LE  on  learning  is  significant  difference  between  the 
control  and  treatment  groups,  2)  learners’  motivation  toward  learning  science  with  Web-LE  is  significant 
difference  between  the  control  and  treatment  groups,  3)  the  previous  two  results  can  confirm  the  usefulness  of  the 
pedagogies  and  identify  the  motivational  indicators,  4)  identify  the  mental  model  of  online  learners.  The  Web-LE 
will  be  released  online  and  all  teachers  and  learners  who  are  studying  the  similar  unit  can  use  it  as  a learning 
resource. 


Future  work 

The  fossil  Web-LE  contains  twelve  paths  and  each  of  them  needs  to  be  completed  by  March  2002.  The 
online  database  is  under  development  to  record  and  preserve  users  information.  We’re  developing  the 
communication  tools  including  omail  and  bulletin  board  to  enable  the  interaction  among  participants.  The 
science  subject  expert  is  helping  us  to  design  the  glossary  pages  for  learners  to  refer  to.  The  glossary  pages 
contain  multimedia  explanation  about  the  units  of  fossilization  (for  instance,  the  various  images  of  fossils  and 
video  clips  of  reconstructing  fossils).  The  fossil  project  is  one  unit  of  the  entire  Geology  and  Ecology  course  in 
the  school.  The  URL  of  the  Geology  and  Ecology  course  is:  httn://l  28. 192.78.9/science.  Other  units  (geological 
time,  radio  dating,  volcano  and  continental  drift)  will  be  designed  after  the  fossil  project  to  expand  the  web-site 
into  a learning  resource  for  science  class  in  high  school.  The  URL  of  the  fossil  project  is 
http://128. 192.78.9/science/fossil  (the  permanent  URL  will  be  activated  after  June  2002: 
http://www.itstudio.net/fossin. 
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Abstract:  The  study  examined  three  engineering  courses  that  were  offered  in  the  Fall  2001  at 
Greenfield  Coalition  in  order  to  facilitate  the  changing  role  of  instructors.  These  three  courses,  as 
other  Greenfield  courses,  with  its  learner  centered  instructional  strategies  and  technologically 
enhanced  activities,  were  designed  to  have  the  instructor  function  more  as  a facilitator,  mentor,  and 
coach  rather  than  the  typical  stand-up  instructor.  This  study  documented  the  instructional  strategies 
and  learning  activities  that  helped  instructors  transition  from  a traditional  standup  instructor  to  a 
“guide  on  the  side.”  In  addition,  this  study  identified  factors  that  affected  the  successftil 
implementation  of  instructional  strategies  and  technology  and  also  suggested  ways  to  assist 
instructors  to  implement  technologically  enhanced  and  learner-centered  strategies. 


Introduction 

The  high  demand  to  produce  knowledge  users  and  problem-solvers  requires  that  instructors  play  the  role  of 
coach,  mentor  and  facilitator  rather  than  director  and  disseminator  of  knowledge.  This  demand  calls  for  change  to 
traditional  teaching.  As  instructors  are  introduced  to  new  instructional  strategies  and  tactics  that  force  them  to 
function  as  a “guide  on  the  side,”  it  is  essential  to  study  the  effect  that  such  strategies  and  tactics  have  on  the 
traditional  instructor  and  to  identify  ways  to  assist  instructors  making  this  transition. 

The  purpose  of  investigating  the  usage  of  instructional  strategies  and  technology  was  to  identify: 
instructional  strategies  and  learning  activities  that  facilitate  the  transition  from  a director  & disseminator  of 
knowledge  to  that  of  a coach  and  resource;  the  factors  that  might  affect  the  successful  implementation  of  new 
instructional  strategies  and  technology;  as  well  as  ways  to  assist  instructors  implementing  technologically-enhanced 
and  learner-centered  strategies. 

The  courses  investigated,  Arts  in  Action,  Engineering  Bionomics,  and  Tool  Design  and  Construction,  are 
required  courses  for  all  candidates  enrolled  in  a degree  program  through  Greenfield  Coalition  (GC).  GC  courses  are 
designed  based  on  the  belief  that  students  will  learn  faster,  and  will  become  more  effective  problem  solvers  if 
engineering  education  and  practice  are  integrated,  and  students  actively  participate  in  their  learning. 


Data  Collection  and  Analysis 

Eight  instructors  were  invited  to  participate  in  this  study.  Five  of  them  returned  our  survey  and  participated 
in  a focus-group  interview.  All  but  one  of  them  has  been  teaching  for  more  than  6 years.  The  survey  and  interview 
focused  on  the  use  of  instructional  strategies,  instructor’s  perception  of  teaching,  assessment  methods,  barriers  to 
adopt  Greenfield  courses,  and  types  of  support  that  instructor  will  need  to  fulfill  their  role  as  a facilitator. 

Survey  data  were  tabulated  by  the  question  items.  Qualitative  data  were  coded  into  indexed  categories  and 
subcategories.  Through  iterative  comparisons  of  hese  categories  and  subcategories  with  the  guided  research 
questions,  key  issues  and  themes  were  generated. 


Findings 

The  survey  and  interview  data  indicated  some  trends  in  using  instructional  strategies  and  technology. 

Using  instructional  strategies 

All  participating  instructors  were  aware  of  the  instructional  strategies  used  in  the  three  courses  being 
studied.  All  instructors  rated  lecture  as  an  effective  teaching  method,  and  they  all  used  student  discussion  as  well  as 


206 


required  student  participation  in  class.  Four  instructors  described  their  teaching  as  engaging  and  interactive.  Four  out 
of  five  instructors  used  such  strategies  as  case  studies  and  projects  and  they  all  viewed  them  as  effective  teaching 
methods.  Three  instructors  thought  that  students  achieved  listed  objectives.  Two  instructors  required  students  to 
work  together.  One  instructor  used  online  interactive  problem  solving  and  additional  WWW  resources.  One 
instructor  used  student  input  to  determine  course  content.  In  assessing  student  learning,  there  is  a combination  of 
traditional  tests  and  quizzes,  and  authentic  evaluation  such  as  projects,  case  studies,  and  peer  evaluation. 

Instructional  strategies  rated  as  “well  and  will  work  well”  to  help  the  instructor  transition  from  traditional 
knowledge  disseminator  to  facilitator,  mentor  or  coach  include:  instructor  playing  role  of  coach,  having  resources 
available  to  students  all  the  time,  doing  more  hands-on,  using  real  world  problems  or  examples,  using  authentic 
tools,  facilitating  group  discussbn,  using  interactive  activities,  what-if  questioning,  and  projects. 

Using  technology 

In  class,  instructors  use  more  traditional  media,  such  as  whiteboard,  LCD  projector,  and  audio-visual 
equipment.  Outside  of  class,  instructors  use  more  new  media  such  as  computer  and  Internet  resources.  Other  than 
traditional  media,  only  two  of  five  instructors  indicated  incorporation  of  technology  in  their  teaching.  Instructors 
indicated  a desire  to  use  technology  more  in  the  future.  One  instructor  hoped  that  technology-enhanced  strategies 
would  help  him  reduce  his  reliance  on  lecturing.  Another  would  like  to  have  a course  that  is  “plug  and  play,”  and 
portable.  Yet  another  would  like  to  explore  the  possibility  of  providing  real  time  feedback  using  technology. 

Factors  affecting  the  successful  implementation  of  instructional  strategies  and  technology 

Instructors  raised  some  issues  regarding  the  adoption  of  the  role  of  facilitator.  These  include:  student  and 
instructor  readiness,  and  administrative  support.  Three  instructors  felt  frustrated  with  students  not  taking 
responsibility  for  their  learning.  Also,  one  instructor  was  puzzled  with  the  value  of  teaching  if  the  instructor  was  not 
there  to  “instruct.”  Yet  another  felt  that  his  role  as  a facilitator  may  be  criticized  by  peers  and  administration. 

Technological  constraints,  which  include  the  infrastructure,  technical  support,  and  reliability  issues, 
discouraged  instructors  from  adopting  technologically -enhanced  instruction.  Additionally,  some  classrooms  did  not 
have  enough  computers  to  use,  or  did  not  have  adequate  Internet  connections. 


Implications  and  Conclusion 

From  our  limited  sample,  we  noticed  that  instructors  used  a combination  of  instructor-centered  and  learner- 
centered  instruction.  While  lecture  is  the  predominant  mode  of  instruction,  instructors  began  to  incorporate 
strategies  that  required  students’  active  participation,  such  as  real  world  problem  solving  and  hands-on  projects. 

Below  are  suggestions  that  we  believe  would  enable  the  greatest  ease  in  the  adoption  of  the  GC  paradigm: 

• Demonstrating  course  materials  and  the  use  of  instructional  strategies  and  technologies  at  partner 
universities  to  promote  wider  use. 

• Conducting  faculty  development  workshops  to  help  faculty  adopt  student  centered  strategies. 

• Planning  and  implementing  an  ongoing  technical  support  system. 

• Encouraging  academic  Directors  and  Deans  to  promote  instruction  that  is  learner-centered,  technology- 
enhanced,  problem-based,  and  reality-based. 

Moreover,  we  feel  that  the  issues  faced  by  GC  are  not  unique  or  new.  The  findings  verily  that  if  you  built  it, 
they  (the  instructors)  may  not  come.  Although  instructors  were  aware  of  the  instructional  strategies  and  agreed  that 
online  materials  were  good,  the  barriers  prevented  them  from  fully  implementing  the  activities.  Three  things  need  to 
take  place  before  instructors  can  fully  adopt  facilitator’s  role  and  endorse  student’s  responsibility  for  learning: 
instructor  readiness,  support  from  administration  in  terms  of  a reward  system  (recognition  toward  tenure,  etc.)  and 
the  assurance  that  ongoing  technical  support  will  be  provided  and  be  considered  a priority. 
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Yes  I Can:  Teachers  Developing  Interactive  Classroom  Activities  Using  Digitized 

Resources 


In  cooperation  with  the  Education  and  Research  Consortium  of  Western  North  Carolina, 
teachers  from  WNC  public,  private,  and  charter  schools  as  well  as  xmdergraduate  pre-service 
education  majors  are  participating  in  a pilot  program  which  enables  educators  to  utilize  through 
the  Internet  the  millions  of  resources  held  by  the  Library  of  Congress  (http://wwW.loc.gov). 

The  educators  chosen  to  participate  in  the  project  are  on  the  leading  edge  of  the  development  of  a 
classroom  technology  program  that  is  expected  to  become  a national  model.  The  pilot  program 
exists  at  Montreat  College  as  well  as  4 other  institutions  that  are  a part  of  the  Education  and 
Research  Consortium  of  Western  North  Carolina. 

The  focus  of  the  project  is  to  provide  pre -service  and  in-service  teachers  the  tools  and 
instructional  skills  necessary  to  integrate  technology  into  the  teaching  and  learning  process.  The 
objectives  for  the  four  year  program  are  as  follows:  a)  To  demonstrate  a Library  of  Congress 
American  Memory  Fellows  Program  pilot  that  teaches  teachers  how  to  utilize  technology  and 
digitized  primary  resources  in  their  classroom  instruction  (http://memorv.loc.gov)  b)  To  train 
pre-service  and  in-service  teachers  to  use  primary  resources  in  their  class  instruction  c)  To 
further  validate  the  American  Memory  program  with  a broad  group  of  K-12  teachers  in  local 
settings  and  d)  To  demonstrate  and  evaluate  a training  program  that  can  be  exportable  to  other 
communities. 

Participants  selected  to  participate  in  the  Adventure  of  the  American  Mind  program 
(http://aam.montreat.edu)  are  required  to  complete  a one -year  program  that  includes  three 
phases.  The  first  phase  is  a three-day  Summer  Institute  that  introduces  the  laptop  which  is 
provided  for  each  participant  by  the  program  and  immerses  the  participants  in  the  basics  of  using 
PowerPoint.  Phase  II  is  a graduate  level  course,  Multi-Media  and  the  Internet  in  the 
Classroom(http://aam.montreat.edu/ED504).  This  course  runs  for  fifteen  weeks,  3 hours  each 
week  and  yields  4.5  CEU’s  and  3 graduate  credit  hours.  The  participants  learn  how  to  integrate 
technology  into  their  classrooms  and  schools,  how  to  build  interactive  projects  in  PowerPoint, 
and  how  to  engage  students  in  the  learning  process  to  build  higher  level  and  critical  thinking 
skills.  The  final  phase  of  the  program  is  the  Mentoring  Phase.  Following  the  train-the -trainer 
model,  participants  act  as  mentors  as  they  take  the  skills  and  techniques  and  skills  from  Phase  I 
and  Phase  II  and  share  them  with  colleagues.  The  mentoring  may  be  either  in  one-on-one  or 
group  workshop  setting.  After  successfully  completing  all  phases  of  the  one -year  program,  the 
participant  earns  permanent  ownership  of  the  laptop. 


Weekly  in-class  activities  for  the  teachers  were  designed  to  enhance  their  use  of  and 
understanding  of  electronic  resources.  Journal  articles  broadened  participants’  knowledge  of 
ethical  and  practical  issues  relating  to  the  integration  of  technology  into  classroom  activities. 
Participants  were  required  to  respond  to  the  articles  on-line  in  a threaded  discussion.  Group 
discussion  in  class  highlighted  major  issues  that  arose  during  the  threaded  discussion. 

Mechanics  of  designing  presentations  including  downloading  and  uploading  graphics  and  music 
as  well  as  the  esthetics  of  presenting  information  on-line  were  presented  in  class.  Lab  time  was 
available  for  individual  practice  during  each  class.  As  a final  product,  participants  presented  an 
interactive  PowerPoint -based  lesson  that  each  would  use  in  the  classroom.  The  presentation  was 
accompanied  by  a unit  plan  which  included  state  and  local  course  objectives  for  the  grade  and 
subject  level  taught. 

On-line  assignments  were  also  a part  of  class  requirements.  One  class  activity  required 
students  to  read  articles  on-line  and  post  responses  related  to  the  application  of  the  information 
Ifom  those  articles.  Another  assignment  was  for  participants  to  locate  exemplary  lesson  plans 
that  integrate  technology  into  the  lesson.  They  entered  the  URL  for  the  lesson  plan,  a summary 
of  the  plan,  and  an  evaluation  of  the  plan  into  the  class  database.  All  of  the  class  work  was 
designed  to  enhance  the  participants’  skills  and  informed  practice  as  they  formulated  lessons  and 
philosophies  for  their  specific  classroom  needs. 

The  growth  Ifom  novice  to  confident  deve  loper  of  creative  and  innovative  lesson  plans 
was  facilitated  by  opportunities  for  participants  to  develop  relationships  with  others  in  the  class. 
During  part  of  each  class,  participants  worked  in  a group.  Rotating  the  members  in  the  groups 
each  week  gave  the  opportunity  for  participants  to  interact  throughout  the  semester  with  each 
member  of  the  class.  In  addition,  a portion  of  each  class  period  was  allocated  for  mentoring. 
Pre-service  teachers  were  assigned  a hypothetical  “class”  which  included  the  characteristics  of 
each  student.  The  pre-service  teacher  education  students  then  visited  the  classrooms  of  the  in- 
service  teachers  who  taught  the  same  grade  level.  The  classroom  visits  enabled  the  pre-service 
teachers  to  develop  strategies  and  appropriate  classroom  activities  for  their  “class.” 

Reciprocally,  the  pre-service  teachers  mentored  their  in-service  partners  in  technology  skills. 

The  final  component  of  the  class  was  its  instructional  team.  During  each  class  meeting  a 
curriculum  specialist,  an  instructional  technology  specialist,  and  a technical  support  specialist 
provided  input.  These  three  instructors  were  available  on-line  or  by  telephone  during  the  week 
as  well  as  during  class  to  assist  teachers  in  their  learning  process. 
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Abstract:  In  this  paper  we  present  a strategy  how  traditional  in>class  exams  can  be  substituted  by  computer-based 
exams.  Exams  obviously  involve  security  issues;  for  instance,  the  aspect  of  availability  is  of  great  concern  to  students 
whereas  teachers  expect  integrity  of  the  results.  Under  no  circumstances  will  students  tolerate  exams  being  lost  while 
they  take  them  or  will  teachers  use  a system  where  exam  answers  can  be  changed  after  the  exam. 

The  stepwise  migration  strategy  from  traditional  paper-based  to  computer-based  exams  proposed  in  this  paper  has 
multiple  benefits:  (1)  advantages  of  migration  are  perceived  at  every  stage  by  all  stakeholders.  (2)  Fail-back  options 
exists  that  may  be  used  if  problems  occur.  (3)  Paper-based  backup  used  to  cover  legal  uncertainties.  (4)  Instructors  can 
independently  decide  when  to  migrate  to  the  next  step. 

1 Introduction 

During  the  transition  of  a conventional  examination  system  to  fully  automated  Web-based  tests,  day-to-day 
business  processes  should  only  be  briefly  interrupted.  We  therefore  show  how  we  use  a stepwise  migration 
strategy  to  eventually  establish  a fully  automated  Web  based  examining  system  that  allows  students  to  take 
exams  almost  anytime. 

Our  main  contribution  is  to  show  how  to  setup  a stepwise  migration  strategy  that  minimizes  organizational 
interruptions.  Moreover,  we  emphasize  an  approach  where  moving  to  the  next  step  is  always  perceived  as 
advantageous  by  all  stakeholders. 

2 Multi-Step  Approach 

Before  an  exam  can  be  generated  automatically  a database  of  exam  questions  and  answers  has  to  be  created. 
Despite  a still  limited  application  at  Austrian  universities  we  found  that  multiple-choice  exams  are  well 
accepted  by  students.  We  propose  to  use  questions  with  five  options  each,  of  which  exactly  one  answer  is 
correctly.  An  exam  comprises  approximately  twenty  questions  with  five  possible  answers  each;  thus  the 
probability  of  a student  passing  the  exam  by  chance  is  very  low. 

Once  a database  of  approximately  one  hundred  questions  has  been  compiled  we  recommend  proceeding  to 
the  first  step  and  using  automatically  generated  exams  for  in-class  test. 


Question  sheet 

Answer  sheet 

Correction  sheet 

Step  1:  Automatic  class-wide  paper  based 

Paper 

Paper 

Paper 

Step  2;  Individual  paper-based 

Paper 

Paper 

Paper 

Step  3:  Electronic  answers 

Paper 

Electronic  (and  paper  backup) 

Step  4;  Web-based 

Electronic  (and  paper  backup) 

Step  5:  Using  software 

Integration  of  3"*  party  software  such  as  Microsoft  Excel 

Table  1:  Overview  of  the  5-steps  in  the  transition  process. 


Disadvantages 

Advantages 

Step  0;  Traditional  exams 

Within  one  exam,  similar  question. 
Therefore  cheating  is  easy  because  everyone 
gets  the  same  questions. 

Instructor  has  full  control  over  choosing 
questions 

Exams  are  fair  as  everyone  gets  the  same 
questions. 

Step  1:  Automatic  class-wide  paper  based 

Within  one  exam,  similar  questions 

Generating  exams  is  fast 
Exams  are  fair. 

Step  2:  Individual  paper-based 

Exams  might  be  considered  unfair 

Exams  can  be  taken  anytime  because  exams 
can  be  generated  very  fast. 

Step  3:  Electronic  answers 

Exams  might  be  considered  unfair 

See  above  plus  Correcting  exams  is  faster 
and  more  accurate. 

Step  4:  Web-based 

Exams  might  be  considered  unfair 

See  above  plus  Exams  can  be  taken 
anywhere  (at  any  test  center) 

Step  5:  Using  third  party  software 

Exams  might  be  considered  unfair 

See  above  plus  Exams  resemble  real  life 
tasks  and  realistic  toots  are  used 

Table  2:  The  rationale  for  continuing  to  step  5. 
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3 Practical  Experience 

In  this  section  we  will  present  the  tools  we  developed  supporting  the  aforementioned  stepwise  migration. 
The  main  purpose  is  not  to  promote  our  solution  but  to  show  that  every  transition  by  itself  requires  only  little 
development  effort  and  the  time  between  migration  steps  can  be  scheduled  to  match  scarce  resources  of 
developers  and  faculty. 

3.1  Database 

The  application  presented  in  this  section  works  with  any  ODBC  database;  we  tested  it  with  mySQL 
(www.mvsql.com)  and  Oracle  (www.oracle.com):  as  the  current  installation  is  mainly  used  by  students  to 
generate  practice  exams  mySQL  suffices  as  database.  The  drawback  of  mySQL  is  that  concurrent  write 
accesses  to  the  database  are  slow;  however,  write  accesses  are  only  necessary  for  real  exams. 

3.2  Exam  Creator 

Teachers  can  specify  what  an  exam  (e.g.  TestOl)  looks  like  by  selecting  different  rules.  For  instance,  the 
teacher  wants  that  the  exam  contains  3 question  that  satisfy  the  condition  mathStat=l.  There  are  48  questions 
that  satisfy  this  criterion. 

The  rationale  for  choosing  this  complex  specification  is  that  all  the  teachers  in  the  department  prefer  typing 
conditions  (that  can  be  quite  complex)  to  limited  GUIs.  The  green  numbers  on  the  right  indicate  how  many 
questions  really  satisfy  the  condition.  If  an  illegal  condition  is  entered,  a ‘0’  highlights  that  no  questions  are 
available. 

Each  question  has  two  mandatory  values  assigned:  required  time  to  complete  and  level  of  difficulty.  An 
exam  is  composed  of  randomly  selected  questions  that  satisfy  the  aforementioned  restrictions.  For  each 
exam  a minimum  and  maximum  time  and  difficulty  are  also  specified.  The  sum  of  the  values  of  all  questions 
that  are  randomly  selected  should  lie  within  the  specified  ranges.  A tool  helps  teachers  and  checks  how 
many  randomly  generated  exams  satisfy  (green  dots  inside  the  shaded  area)  the  time/difficulty  bounds  and 
how  many  do  not  (red  dots  outside  of  the  shaded  area).  If  there  are  a lot  more  red  dots  than  green  dots  the 
generation  of  an  exam  might  be  slow  because  only  ‘‘green”  exams  are  really  generated  for  students.  “Red” 
exams  might  be  too  difficult  or  require  too  long  to  solve. 

This  user  interface  is  used  by  faculty  and  staff  in  step  1 and  2 to  automatically  create  paper-based  exams. 

3.3  Web-based  Practice  Exams 

A very  simple  Web  server  allows  publishing  practice  exams  on  the  Web.  It  suffices  to  copy  a small 
executable  file  which  subsequently  serves  as  a Web  server.  The  advantage  is  that  installation  is  very  easy 
and  even  teachers  without  special  IT  knowledge  can  install  and  maintain  the  server  without  the  help  of  IT 
staff.  Students  enter  the  system  with  a Web  browser  and  can  select  an  exam.  Thereafter  an  exam  will  be 
automatically  generated  and  both  the  questions  and  answers  will  be  available  to  the  student. 

Students  benefit  considerably  from  practice  exams  since  they  can  continuously  assess  whether  they  reached 
their  learning  goals  [Woit  and  Mason  2000]. 

3.4  Computer-based  Exams 

From  step  3 on  a simple  program  is  used  by  students  to  enter  their  answers.  The  focus  was  obviously  placed 
on  a simple  design.  After  pressing  ‘OK’,  the  student  is  asked  whether  she  wants  to  hand  in  the  exam  or 
continue  editing  it.  When  she  decides  to  hand  in,  a paper  copy  of  the  exam  and  the  answer  sheet  is  printed. 

4 Conclusion 

In  this  paper  we  explained  how  a multi-step  approach  can  be  used  to  minimize  risks  when  introducing 
computer-based  exams.  The  key  issue  is  security  and  the  option  to  “fall  back”  to  previouses  steps  if 
problems  occur.  Moreover,  as  every  new  step  brings  benefits  to  the  stakeholders,  the  chance  of  successfully 
completing  the  project,  i.e.  of  going  all  the  way  to  step  5,  is  high. 
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Abstract:  Navigation  is  one  of  the  key  aspects  of  eLearning  environments  that  can  make  a difference  when 
comparing  new  forms  of  learning  to  traditional  ways  of  learning.  In  this  paper  we  elaborate  on  the  experience 
we  gained  while  developing  an  eLearning  solution  and  highlight  the  relevant  issues  giving  screenshots. 


1 From  Computer-Based  Training  to  £-£ducation 

During  the  past  few  years  the  Web  has  fundamentally  altered  many  people’s  work  life.  Businesses  were 
transformed  into  e-businesses  and  the  buzzword  ‘e-commerce’  is  now  ubiquitous.  There  is,  however,  an  industry 
that  only  gradually  adopts  these  new  technologies  - teaching.  Despite  the  fact  that  universities  have  significantly 
contributed  to  the  development  of  the  Web,  it  is  still  not  fully  used  to  improve  academic  teaching. 

It  was  only  last  year  that  the  term  eLearning  was  defined  by  Lennox  [Lennox  00]  as  “the  use  of  e-business 
technologies  to  speed  the  flow  of  business  information  and  knowledge  from  creator  to  learner  in  a highly 
personalized,  on-demand  fashion”. 

In  this  paper  we  will  present  two  decisive  factors  for  Web-based  training  (WBT)  software  in  the  context  of  e- 
education.  We  will  focus  our  report  on  offline  education,  as  real  time  communication  (except  chats)  is  not  yet 
sufficiently  reliable  due  to  low  bandwidth  Internet  access  in  most  European  rural  areas.  Moreover,  one  of  the 
main  advantages  of  WBT  software  is  being  able  to  study  according  to  one’s  own  schedule.  Even  if  online 
communication  was  available,  a large  offline  corpus  of  knowledge  would  be  required  too  - just  like  textbooks 
are  used  in  today’s  courses  to  support  traditional  in-class  teaching.  The  techniques  presented  in  this  paper  can 
help  to  organize  the  electronic  equivalent  of  textbooks. 

Compared  to  textbooks,  correspondence-based  distance  learning  programs  and  first  generation  computer-based 
training  software,  WBT  offers  many  advantages.  Two  of  them  which  we  consider  most  important  will  constitute 
the  main  focus  of  this  paper:  First,  navigation  is  no  longer  limited  by  serially  flipping  through  pages  and  second, 
interactive  examples  are  far  more  likely  to  actively  involve  students  while  studying.  Throughout  this  paper  we 
will  give  examples  of  how  these  features  have  been  successfully  implemented  [Weippl  99]  in  WBT  software 
called  Teach/Me  [Lohninger  99]. 

Dating  back  to  the  hype  of  the  term  “eCommerce”  eLearning  is  widely  used  in  different  ways;  for  instance, 
[LineZine]  understands  eLearning  ranging  from  “the  convergence  of  the  Internet  and  learning,  or  Internet- 
enabled  learning”  to  “the  use  of  network  technologies  to  create,  foster,  deliver,  and  facilitate  learning,  anytime 
and  anywhere”  or  “the  delivery  of  individualized,  comprehensive,  dynamic  learning  content  in  real  time,  aiding 
the  development  of  communities  of  knowledge,  linking  learners  and  practitioners  with  experts.” 

[ELeamers  Glossary]  defines  e-learning  as  any  form  of  learning  that  utilizes  a network  for  delivery,  interaction, 
or  facilitation. 

According  to  [Learning  Circuits  Glossary]  “E-leaming  covers  a wide  set  of  applications  and  processes,  such  as 
Web-based  learning,  computer-based  learning,  virtual  classrooms,  and  digital  collaboration.  It  includes  the 
delivery  of  content  via  Internet,  intranet/extranet  (LANAVAN),  audio-  and  videotape,  satellite  broadcast, 
interactive  TV,  and  CD-ROM.” 
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2 Interactive  Navigation 

Based  on  the  knowledge  of  how  students  leam,  it  seems  obvious  that  the  way  of  presenting  knowledge  and  the 
options  offered  to  navigate  or  access  it,  are  essential  for  successful  teaching.  It  is  a commonly  known  fact  that 
WBT  offers  the  possibility  to  design  teaching  material  that  can  be  accessed  in  a non-linear  way.  However,  a brief 
scan  through  existing  course  material  reveals  that  a large  number  of  today’s  courses  are  not  designed  for 
efficient  navigation  which  is  needed  for  learning.  This  is  why  it  is  essential,  that  when  designing  the  navigation 
structure  for  WBT  software  one  always  has  to  keep  in  mind  that  the  ultimate  goal  is  to  effectively  convey 
knowledge. 

In  this  section  we  will  first  discuss  theoretical  issues  that  are  essential  for  designing  ways  to  navigate  WBT 
content.  We  will  then  continue  by  discussing  various  forms  of  navigation.  Finally  we  will  elaborate  on  the  forms 
of  navigation  best  suited  for  different  targets  of  learning. 

2.1  Learning  Process 

When  studying,  people  usually  create  an  internal  representation  of  the  topic  they  study.  A cognitive  map,  i.e.  a 
form  of  visualizing  content  structure,  can  aid  in  this  process.  Cognitive  maps  are  two-dimensional 
representations  of  knowledge  structures.  They  are  based  on  the  assumption  that  the  relationships  between  ideas 
or  concepts  can  be  uncovered  due  to  semantic  proximity;  that  is,  semantically  related  ideas  will  appear  close  in 
geometric  representations  of  cognitive  space  whereas  unrelated  concepts  will  be  separated  by  a greater  distance. 

Fenker  [Fenker  75]  states  three  assumptions  about  cognitive  maps:  First,  information  about  a topic  is  organized 
and  interpreted  on  the  basis  of  a set  of  dimensions  which  represent  organizational  features  of  the  topic.  Second, 
these  dimensions  can  be  represented  in  n-dimension  geometric  space.  Third,  there  are  many  relationships  that 
can  exist  among  concepts.  Similarity,  degree  of  association,  and  the  extent  to  which  one  concept  implies  another 
are  examples  of  these  types  of  relationships.  That  is,  the  nature  of  the  semantic  relationship  is  implicit  in  the 
relationship  but  has  to  be  interpreted. 

During  the  learning  process  cognitive  maps  can  be  used  to  assess  how  well  students  have  integrated  recently 
acquired  knowledge  and  whether  they  can  position  it  within  existing  knowledge.  The  student  draws  a concept 
map  representing  his  current  knowledge;  areas  not  well  linked  to  other  parts  of  knowledge  may  be  identified  and 
indicate  the  necessity  of  further  studies. 

Acquiring  new  knowledge,  i.e.  transforming  external  to  internal  representations  preserves  certain  aspects  of  the 
representation,  mainly  the  topology  [Steiner  88].  Cognitive  maps  do  not  seem  to  be  a full  and  continuous 
representation  of  the  information  stored  in  mind.  They  are  fragmented  and  distorted  on  an  individual  basis.  Since 
they  change  dynamically  they  cannot  be  easily  copied  onto  an  external  representation.  Each  time  an  individual 
tries  to  output  the  interior  cognitive  map  containing  specific  knowledge,  it  looks  different.  This  implies  that  there 
is  not  only  one  ‘correct’  spatial  representation  of  relations  between  topics. 

Although  the  definition  of  cognitive  maps  varies  in  the  literature,  there  seems  to  exist  an  agreement  that 
cognitive  maps  are  an  appropriate  way  to  represent  relationships  between  different  entities  of  knowledge  [Driver 
92],  [Gergen  92],  [Spiro  92],  [Galsersfeld  92]. 

Some  details  seem  especially  important  in  the  context  of  this  paper.  Certain  points  of  interest  and  paths  between 
these  points  are  learned  (i.e.  external  maps  are  internalized)  when  studying  new  concepts.  Using  those  paths,  the 
overall  context  of  the  knowledge  can  be  understood  easily  because  students  can  look  at  topics  in  the  vicinity  of 
those  paths  to  find  related  concepts. 

The  first  step  in  the  process  of  teaching  is  that  the  attention  is  drawn  to  points  of  interest,  i.e.  important  topics. 
Learners  perceive  these  points  of  interest  either  because  they  are  highlighted  on  the  cognitive  map  and  can 
therefore  be  easily  spotted  or  due  to  their  Importance.  In  a second  step  points  of  interest  that  are  close  to  each 
other  are  used  as  starting  or  end  points  of  paths.  Knowledge  of  these  paths  seems  to  be  more  or  less  knowledge 
of  starting-  and  endpoints  and  orientation  in  the  map.  This  “learning  of  paths”  is  what  can  be  achieved  by  using 
“Guided  Tours”  in  a Knowledge  Landscape,  which  will  be  discussed  in  greater  detail  in  the  next  section. 

2.2  Navigation  Metaphors 

Initially  most  CBT  programs  implemented  the  behaviorist  paradigm.  Since  the  days  of  behaviorism  seem  to  have 
passed  an  increasing  number  of  approaches  in  CBT  try  to  adopt  constructivist  methods  (overviews  can  be  found 
in  [Jonassen  93],  [Brandt  95]). 


Freeman  [Freeman  96]  tried  to  integrate  the  constructivist  paradigm  into  a course  enabling  teachers  to  use  the 
Internet  for  distance  learning.  Having  taught  the  necessary  basics,  she  gave  the  participants  the  opportunity  "to 
build  up  their  own  internal  knowledge  base." 

The  visualization  of  a given  field  of  knowledge  should  leave  it  up  to  the  students  how  to  explore  the  data,  how  to 
structure  the  contents,  where  to  set  priorities  and  to  decide  which  chapters  they  consider  as  related.  However,  in 
order  to  avoid  the  experience  of  not  knowing  what  to  learn  and  not  being  able  to  structure  the  presented  data, 
presenting  some  pre-defined  structure  to  start  with  is  useful. 

The  modes  of  navigation  offered  by  Teach/Me  [Lohninger  99]  go  far  beyond  those  of  other  textbooks. 

Pohl  et  al.  [Pohl  95]  have  identified  four  main  difficulties  when  creating  educational  hypertext.  First,  authors  are 
used  to  structuring  text  in  a linear  or  hierarchical  way  like  in  printed  textbooks.  Second,  the  texts  are  often  too 
long  and  do  not  have  enough  words  highlighted  e.g.  by  bold  typesetting.  Third,  scientific  publications  as  well  as 
textbooks  have  long  argumentative  strains  that  support  one  final  solution  for  a certain  problem.  These  parts  have 
to  be  read  in  a fixed  order  determined  by  the  author.  Fourth,  a large  number  of  short  texts  may  be  confusing, 
especially  if  they  are  not  suitably  structured.  This  can  result  in  the  well-known  phenomenon  of  'Lost-in- 
Hyperspace'.  Two  of  these  issues,  i.e.  hierarchical  text  structure  and  potentially  confusing  hyperspace  navigation 
can  be  addressed  by  providing  multiple  forms  of  navigation  through  WBT  content.  We  will  now  present  four 
different  approaches  to  navigating  hyperlinked  textbooks. 

2. 2. 1 Table  of  Contents 

The  oldest  form  of  navigation  through  written  information  is  the  table  of  contents.  It  is  still  widely  used  in 
today’s  hyperlinked  WBT.  This  is  mainly  due  to  the  fact  that  everyone  is  familiar  with  this  form  of  navigation. 
Most  authors  think  that  sequentially  structuring  content  seems  to  be  an  efficient  way.  However,  the  fact  that 
researchers  and  authors  like  a hierarchical -linear  structure  is  to  a great  extent  because  it  is  the  structure  they  have 
always  used. 

Despite  the  advantages  offered  by  the  navigation  interfaces  discussed  later  in  this  section,  WBT  courses  should 
always  include  a table  of  contents  because  everyone  expects  it  and  knows  how  to  use  it. 

2.2.2  Keywords  - Indices 

Similar  to  the  table  of  contents,  an  index  is  a form  of  navigational  interface  commonly  found  in  books. 
Nonetheless,  software  may  offer  a much  greater  flexibility.  First,  searches  for  keywords  can  be  more  expressive 
by  using  Boolean  expressions.  Second,  by  displaying  an  indexed  expression  in  its  context  before  actually 
opening  the  page  users  can  quickly  scan  through  a large  number  of  matches.  Compared  to  paper-based  indices 
that  sometimes  refer  to  ten  or  more  different  pages  for  a single  keyword  the  quicker  access  in  a Web-based 
textbook  is  certainly  an  advantage. 

2.2.3  Hyperlinks 

Hyperlinks  are  commonly  used  to  navigate  through  hypertext.  Today’s  Web  is  linked  by  only  one  link  type  that 
can  represent  different  forms  of  how  a linked  text  is  related.  Rada  [Rada  91]  distinguishes  three  fundamentally 
different  forms  of  links.  (1)  Sequence  links  connect  pages  in  a linear  sequence.  Using  ‘Next’  and  ‘Back’  links 
helps  especially  first  and  second  year  undergraduate  students  to  access  all  pages  relevant  for  a lecture.  (2) 
Outline  links  refer  to  pages  that  contain  overview.  (3)  Reference  and  citation  links  point  to  pages  that  describe 
topics  in  greater  detail. 

When  designing  WBT  software  we  recommend  to  clearly  indicate  (e.g.  by  a particular  icon)  the  kind  of  link;  this 
helps  students  to  decide  whether  they  want  to  follow  a link  or  not.  Figure  2,  for  example,  shows  two  different 
links;  one  opens  an  interactive  example  to  illustrate  concepts  explained  in  the  HTML  page,  whereas  the  other 
link  starts  a calculation  tool  with  preloaded  exercises  so  that  students  can  apply  their  newly  acquired  knowledge. 


Figure  1:  Teach/Me  offers  a simple  but  yet 
efficient  index. 


Figure  2:  The  Web  page  (top  left)  contains  two 
different  links.  One  link  opens  an  interactive 
example  (bottom  left),  whereas  the  other  one 
starts  a calculation  program  and  preloads  data 
for  a certain  exercise. 


2.2.4  Knowledge  Landscape 

To  provide  a good  overview  and  easy  navigation  we  have  chosen  to  visualize  the  textbook  with  the  help  of  a 
Knowledge  Landscape.  A mountain  in  the  landscape  represents  each  page  of  the  underlying  textbook.  The 
formation  of  the  Knowledge  Landscape  is  governed  by  two  general  rules;  (1)  In  the  3D  version,  fundamental 
chapters,  explaining  basic  concepts,  are  visualized  by  high  mountains,  whereas  more  specific  lessons  are 
displayed  by  smaller  mountains.  (2)  The  placement  of  the  mountains  is  made  according  to  the  similarity  of  the 
contents  of  the  corresponding  lessons.  Thus  highly  related  texts  are  placed  next  to  each  other.  The  positions  of 
the  mountains  on  the  2D  surface  are  computed  automatically.  This  approach  allows  easy  non-linear  and  non- 
hierarchical  access  to  originally  hierarchically  structured  documents. 

Static  objects  in  a landscape,  whether  they  are  real  or  virtual,  have  unique  fixed  positions  in  the  spatial  three- 
dimensional  space.  We  all  have  experienced  the  exploration  of  unknown  and  the  navigation  in  known  territory 
since  early  childhood.  Thus  we  are  used  to  remembering  the  relative  positions  of  objects  in  our  environment  and 
we  can  easily  find  objects  again,  even  if  they  are  not  in  our  direct  line  of  sight.  Despite  the  fact  that  there  is  not 
only  one  ‘correct’  spatial  representation  of  relations  between  topics  we  offer  only  one  Knowledge  Landscape  as 
novice  users  may  easily  be  confused  if  they  are  confronted  with  more  than  one  spatial  representation. 

2.3  Learning  Targets 

In  this  subsection  we  will  have  a closer  look  at  five  learning  targets  and  we  will  examine  which  forms  of 
navigation  are  best  suited  for  each  of  them. 

2. 3. 2 Training  - Drill  and  Practice 

Drill  and  Practice  training  was  one  of  the  first  applications  for  computer-based  training.  The  computer  was  the 
perfect  tool  to  teach  highly  standardized  procedures  such  as  preflight  preparations  for  an  aircraft’s  cabin  crew.  A 
major  component  of  these  simple  CBT  programs  is  testing  the  students’  knowledge.  Objective  tests  and  quizzes 
are  among  the  most  widely  used  and  well-developed  tools  in  higher  education.  Each  question  assumes  a simple 
answer  that  can  be  evaluated  either  as  correct,  incorrect,  or  partly  correct.  According  to  the  expected  type  of 
answer,  questions  are  often  classified  into  eight  types.  There  are  four  basic  types:  (1)  yes/no  questions,  (2) 
multiple-choice/  single-answer  (MC/SA)  questions,  (3)  multiple-choice/multiple-answer  (MC/MA)  questions. 
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and  (4)  fill-in  questions  with  a string  or  numeric  answer.  More  advanced  type  of  questions  include  (5)  matching- 
pairs  questions,  (6)  ordering-questions,  (7)  pointing-questions  - the  answer  is  one  or  several  areas  on  a figure  - 
and  (8)  graphing-questions  - the  answer  is  a simple  graph.  Also,  each  topic  area  may  have  specific  types  of 
questions  [Brusilovsky  99]. 

If  CBT  software  is  developed  for  drill  and  practice,  the  most  important  navigation  tool  is  the  table  of  contents.  It 
gives  students  a clear  outline  of  what  will  be  covered.  There  is  no  need  to  support  learning  or  understanding 
additional  topics  as  the  whole  learning  process  is  highly  predefined.  Referring  to  the  example  of  training  cabin 
crews,  the  CBT  software  does  not  need  to  help  students  understand  which  preflight  checks  a pilot  would  have  to 
make. 

2.3.2  Gaining  an  Overview 

Especially  in  a professional  working  environment  it  is  very  important  to  be  able  to  quickly  gain  an 
overview  of  a topic  and  to  identify  the  main  concepts.  We  think  that  this  learning  process  is  best 
supported  by  a combination  of  the  Knowledge  Landscape  and  a keyword  search.  The  search  results 
are  highlighted  in  the  Knowledge  Landscape  and  one  can  therefore  locate  the  main  concepts  and  their 
relation  with  certain  keywords. 

2.3.3  Understanding  In-depth  Knowledge 

Teachers  generally  agree  that  students  must  actively  process  and  make  sense  of  available  information  in  order  to 
gain  an  in-depth  understanding  of  a topic.  Therefore,  expert  teachers  use  many  different  techniques  to  ensure  the 
active  involvement  of  students.  For  example,  classroom  teachers  rely  on  a number  of  visual  cues  from  their 
students  to  enhance  their  delivery  of  instructional  content.  A quick  glance,  for  example,  reveals  who  is 
attentively  taking  notes,  thinking  about  the  lecture  or  a difficult  concept,  or  trying  to  make  a comment  [Willis 
93]).  In  contrast,  pupils  studying  with  WBT  software  may  lose  attention  without  anybody  noticing. 

One  way  to  increase  the  probability  that  learners  actively  process  information  is  to  require  answers  to  small 
quizzes  and  the  use  of  interactive  examples  (see  Section  3).  Another  option  is  to  engage  them  in  the  process  of 
exploring  the  knowledge  domain.  A knowledge  landscape  can  be  both  motivating  to  navigate  through  and 
convey  insight  into  how  concepts  are  related.  As  understanding  the  connections  between  various  topics  is 
essential  for  gaining  in-depth  knowledge,  the  knowledge  landscape  is  an  important  tool  supporting  this  process. 

2.3.4  Reference 

Students  often  keep  good  textbooks  as  reference.  Mainly  because  they  have  spent  a considerable  amount  of  time 
and  effort  to  study  the  book,  they  are  able  to  understand  concepts  they  do  not  remember  well  very  fast  again. 
Therefore  it  is  essential  that  good  learning  material  is  also  designed  to  be  used  as  a reference.  Obviously,  an 
index  is  the  most  important  navigation  interface  for  finding  the  required  information.  However,  as  indices  are  by 
definition  word-based  and  not  content-based  one  might  miss  highly  related  concepts,  particularly  if  different 
domain  experts  use  different  terms  for  the  same  concepts.  As  the  Knowledge  Landscape  is  computed  content- 
based  and  does  not  rely  on  keywords  only,  it  can  uncover  such  relationships. 

2.3.5  Adding  or  Updating  Content 

The  content  of  WBT  software  has  to  be  periodically  updated  to  ensure  that  it  meets  the  demands  of  a changing 
syllabus.  As  most  WBT  projects  are  maintained  by  more  than  one  author  it  is  very  difficult  to  make  sure  that 
links  to  all  related  topics  exist-  even  if  the  author  is  required  to  set  links  to  topics  written  by  others.  The 
Knowledge  Landscape  can  support  authors  in  finding  relevant  relations  and  it  helps  to  ensure  that  the  hypertext 
navigation  structure,  i.e.  links,  is  built  properly. 

3 Conclusion 

Navigation  can  be  considered  to  be  one  of  the  decisive  factors  in  determining  whether  eLearning  software  really 
supports  the  learning  process  of  students.  It  is  no  longer  limited  by  serially  flipping  through  pages;  we  showed 
how  Knowledge  Landscapes  can  be  used  to  provide  another  (flat)  view  of  hierarchically  organized  content.  The 
option  to  interactively  manipulate  the  landscape  increases  the  chance  of  actively  involving  students  while 


2076 


studying.  Throughout  this  paper  we  gave  examples  of  how  these  features  have  been  successfully  implemented  in 
existing  WBT  software. 

4 References 

[Brandt  95]  D.  Scott  Brandt,  Constructivist  approaches  to  "teaching  the  Internet”,  Online  Information  Proceedings,  Learned 
Information,  1995 

[Brusilovsky  99]  Peter  Brusilovsky  and  Philip  Miller,  Web-based  Testing  for  Distance  Education,  p.  149,  WebNet,  A ACE, 
1999 

[Driver  92]  Rosalind  Driver,  Constructivist  Approaches  to  Science  Teaching,  Constructivism  in  Education.  Lawrence 
Erlbaum,  1992 

[Learning  Circuits  Glossary]  Learning  Circuits  Glossary  http://www.lcamingcircuits.org/glossary.html  (last  visited  Dec 

2001) 

[ELeamers  Glossary]  ELeamers  Glossary  http://www.eleamcrs.com/clcaming/glossarv.htm  (last  visited  Dec  2001) 
[LineZine]  LineZine  (http://www.linczinc.com/clcaming.htm')  (last  visited  Dec  2001) 

[Fenker  75]  R,  M,  Fenker,  The  Organization  of  Conceptual  Materials:  A Methodology  for  Measuring  ideal  and  actual 
cognitive  stmctures.  Instructional  Science,  4:33-57,  1975 

[Freeman  96]  Molly  Freeman,  Global  Distance  Learning  and  the  Transformation  of  Knowledge  Construction.  WebNet  96. 
AACE,  1996 

[Gergen  92]  Kenneth  J.  Gergen,  Social  Construction  and  the  Educational  Process,  Constructivism  in  Education.  Lawrence 
Erlbaum,  1992 

[Glasersfeld  92]  Ernst  von  Glasersfeld,  Constructivist  Approach  to  Teaching.  Constructivism  in  Education.  Lawrence 
Erlbaum,  1992 

[Jonassen  93]  David  H.  Jonassen  and  Katherine  Beissner  and  Michael  Yacci,  Structure  Knowledge:  Techniques  for 
Representing,  Conveying,  and  Acquiring  Structural  Knowledge.  Lawrence  Erlbaum  Associates,  Hillsdale,  New  Jersey,  1993 

[Lennox  99]  Duncan  Lennox,  Improving  The  Top  Line  Using  e-Leaming:  The  Convergence  of  Training,  Business 
Communication  and  Performance  Support,  http://www.wbtsystems.com/products/whitepaperl.html.  2000. 

[Lohninger  99]  Hans  Lohninger,  Teach/Me  - Data  Analysis,  Springer,  1999. 

[Pohl  95]  Margit  Pohl  and  Peter  Prenner  and  Peter  Purgathofer,  Hypermedia  in  Education  - Monitoring  the  Developement  of 
Hypermedia  Documents,  Psychology  Teaching  Review,  4:142-152,  1995 

[Rada]  R.  Rada,  Hypertext:  From  Text  to  Expertext.  McGraw-Hill,  Hamburg,  New  York,  London,  Paris,  Tokio.  1991 

[Spiro  92]  Rand  J.  Spiro  and  Paul  J.  Feltovich  and  Michael  J.  Jacobson  and  Richard  L.  Coulson,  Cognitive  Flexibility, 
Constructivism,  and  Hypertext:  Random  Access  Instruction  for  Advanced  Knowledge  Acquisition  in  Ill-Structures  Domains. 
Constructivism  in  Education.  Lawrence  Erlbaum,  1992 

[Steiner  88]  Gerhard  Steiner,  Wissensreprdsentation.  Psychologie  Verlags  Union,  1988 

[Weippl  99]  Edgar  Weippl,  PhD  Thesis  in  Computer  Science  at  the  Institute  of  General  Chemistry  at  the  Vienna  University 
of  Technology,  ‘’Integration  of  Virtual  Reality  Knowledge  Landscapes  into  Multimedia  Teachware"  (integrated  into  the 
project  Teach/Me) 

[Willis  93]  B.  Willis,  Distance  education:  A practical  guide.  Englewood  Cliffs,  NJ.:  Educational  technology  publications. 
1993 


2077 


XML  Based  Content  and  its  Integration  into  Existing  e-Learning  Solutions: 
Results  and  Consequences  of  a Survey 


Franz  Weitl,  Universitat  Passau  - University  of  Passau,  Germany,  weitl@fmi.uni-passau.de 
Andre  Wiesner,  Universitat  Karlsruhe  - University  of  Karlsruhe,  Germany,  awi@aifb.uni-karlsruhe.de 
Elfriede  Kelp,  Technische  Universitat  Munchen  - Technical  University  of  Munich,  Germany,  kelp@in.tum.de 


Abstract:  Current  courseware  is  often  tightly  coupled  with  a specific  e-learning  system 
for  its  access  and  presentation.  We  identify  the  system  components  and  their  interaction 
in  creating,  publishing,  accessing  and  executing  platform  independent  learning  content. 
We  present  a system  model  aimed  at  using  inexpensive  of-the-shelf  components  to 
supply  the  core  functionality  while  being  ready  to  integrate  flexible  and  powerful  con- 
tent management  and  e-learning  support  systems  in  future. 


Introduction 

In  April  2001  12  German  universities  have  started  the  project  “WissensWerkstatt  Rechensysteme”  (engl. 
'Knowledge  Factory  for  Computing  Systems')  targeting  the  development  of  multimedia  learning  modules  in  the 
field  of  technical  computer  science.  It  was  an  essential  task  to  find  a suitable  system  to  create,  store,  combine, 
and  deliver  these  modules.  Despite  of  the  massive  amount  of  teaching/leaming  support  systems  on  the  market  as 
well  as  reports  about  selection  criteria  and  evaluation  results  of  e-leaming  platforms  [BOl],  it  was  difficult  to 
find  a suitable  platform.  The  scalability  of  the  planned  1 50  modules  (as  discussed  in  [LT02])  will  result  in  1 8 
potentially  different  variants  per  module.  Thus  our  system  needs  to  handle  up  to  2700  module  variants  in  total, 
each  of  them  referring  a large  amount  of  media  objects  (such  as  images,  audio,  video,  animation,  simulation,  ...). 
Efficient  construction  of  multiple  variants  and  output  formats  is  supported  best  by  XML  technology.  That’s  why 
the  platform  needs  to  be  able  to  handle  XML-based  content  and  output  construction  according  to  our  specifica- 
tion. Furthermore  the  system  should  support  and  control  the  distributed  development,  evaluation  and  continuous 
maintenance  of  the  modules  by  workflow  and  source  control  / versioning  tools.. 

Based  on  these  requirements  we  evaluated  13  Systems  (among  them  Blackboard,  WebCT,  WBT  Topclass,  Lotus 
Learning  Space,  Time4You).  None  of  the  examined  systems  satisfied  our  core  requirements  [WOl].  Since  no 
financial  and  personal  resources  for  system  development  are  available  within  our  project,  it  was  necessary  to 
develop  a cost  effective  yet  extensible  concept  of  establishing  a suitable  access  and  management  platform. 

System  Concept  Requirements 

Looking  at  existing  e-Leaming  platforms  the  major  problem  is  that  most  of  them  impose  dependencies  on  con- 
tent and  system  functionality.  Following  dependencies  can  be  identified: 

• System  component  dependencies:  Modem  e-leaming  platforms  integrate  functions  such  as  authoring,  as- 
set/course/user/access  management,  content  configuration/retrieval/presentation/execution.  These  functional 
components  are  often  tightly  coupled.  As  a result  it  is  hard,  for  example,  to  integrate  a new  publication  sub- 
system or  personal  learning  environment. 

• Data  format  dependencies:  Many  of  the  current  e-leaming  systems  organise  and  store  the  course  stmctures 
tightly  coupled  to  the  specific  functionality  of  the  system.  The  courses  are  not  platform  independent.  Pack- 
aging standards  such  as  LTSC  and  SCORM  [ADOl]  are  forced  to  take  up  a rather  abstract  point  of  view  on 
learning  objects  [KOI].  The  necessary  platform-independent  model  of  the  fine  grained  internal  semantics  of 
e-leaming  content  is  topic  of  current  research  [KOO]  [KOI]  [SKFOO]  [LT02]. 

As  platform  independency  of  the  developed  content  is  one  of  the  core  requirements  of  the  WWR  project  our 
strategy  is  to  avoid  data  format  as  well  as  system  component  dependencies  by  applying  content  management 
concepts  such  as  separation  of  specification  (XML  - code)  and  publication  of  course  content  (html,  pdf).  Of 
course  there  are  more  than  content  engineering  related  aspects  to  providing  e-leaming  content  (retrieval,  online 
access,  personal  learning  environment).  These  components  are  further  objects  in  our  system  model. 

System  Model  and  Implementation  Stages 

Based  on  the  requirements  analysis  of  the  WWR  project  we  developed  a system  model  identifying  the  dataflow 
between  its  major  components:  authoring  environment,  publishing  environment,  distribution  platform,  personal 
learning  environment  (fig.  1).  We  organize  the  implementation  of  this  system  model  in  3 stages: 

Stage  1:  Exploiting  XML’ s authoring  potential.  In  this  phase  we  use  XML-technology  to  simplify  the  authoring 
process  of  multi  variant  content  [LT02].  The  source  as  well  as  the  output  documents  (html,  pdf)  are  constmcted 
locally  (using  XSL  transformations)  and  uploaded  on  a central  repository  facilitating  source  control  / versioning. 
The  system  will  adopt  standard  web  server/browser  software  to  support  retrieval,  online  access,  download,  off- 
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line  access  and  basic  communication  functionality  (feedback).  Additional  output  filters  for  integrating  our  con- 
tent into  the  project  partners’  local  e-leaming  systems  will  be  implemented  successively. 


Authoring  Environment 


Distribution  Platform 


Sources 


^ Rtcipitnt 


Figure  1:  Data  structures  and  dataflow  across  the  systems  major  functional  components  (simplified) 


Stage  2:  Exploiting  XML ’s  publishing  potential.  A central  content  management  system  (CMS)  will  be  adopted  as 
part  of  the  distribution  platform  to  organize  the  content  production  and  publication  workflow.  The  publishing 
subsystem  - in  stage  I implemented  locally  as  part  of  the  authoring  environment  - will  be  transferred  to  the  cen- 
tral distribution  platform.  As  output  objects  will  automatically  be  rebuilt  whenever  changes  to  source  files  are 
released  consistency  and  timeliness  of  the  output  documents  are  guaranteed.  Ideally  the  CMS  will  even  support 
live  publishing  driven  by  the  recipient’s  request  (personal  script  / course). 

Stage  3:  Exploiting  XML 's  run  time  potential.  The  learner  will  be  offered  a dynamic  personal  online  / offline 
learning  environment  as  an  additional  subsystem.  The  learning  environment  will  render  the  native  XML  content 
adaptively  according  to  the  learner’s  profile,  preferences  and  learning  situation,  and  will  integrate  personal  con- 
tent (annotations)  as  well  as  runtime  data  from  interactive  components  (experiments,  self  tests). 


Conclusion 

As  we  specify  our  learning  content  in  an  open,  platform  independent  format  and  decouple  the  functional  compo- 
nents authoring,  publishing,  distribution  and  personal  learning  in  our  system  design  we  are  able  to  adopt  stan- 
dard components  in  central  areas  (document  management  and  distribution)  while  keeping  the  system  flexible  and 
extensible  in  areas  of  current  research  and  development.  The  next  steps  are  the  research  of  suitable  components 
on  the  market  and  their  integration  into  an  overall  system. 
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Towards  a Discipline  Based  Reflective  Thinking  Process  for  K-12  Students  and 
Teachers  Through  On-line  Discourse  and  Action  Research 
Elizabeth  Wellman,  Ed.D. 

Introduction 

The  California  History-Social  Science  Project,  a legislatively-mandated  professional  development  program 
is  administered  out  of  the  University  of  California,  Office  of  the  President.  The  Executive  Offices  are 
based  at  UCLA,  and  oversee  seventeen  local  sites  across  California.  CH-SSP*s  mission  statement  includes: 
a commitment  to  improving  the  teaching  and  learning  of  history-social  science  by  strengthening 
disciplinary  content  knowledge  for  all  students  as  outlined  in  the  California  History-Social  Science  Content 
Standards;  enhancing  instructional  strategies  in  the  teaching  of  history-social  science  to  promote 
accessibility  to  the  discipline;  promoting  collaboration  across  grade  levels,  kindergarten  through  university; 
and  enhancing  teachers’  use  of  technology  as  an  integral  part  of  the  instructional  process. 

As  part  of  meeting  this  mission,  CH-SSP  engages  teachers  in  action  research  on  the  use  of  technology- 
based  discourse  tools  to  support  the  development  of  a discipline  based  reflective  thinking  process,  which 
promotes  student  ability  to  engage  in  historical  thinking  and  understanding.  This  action  research  project 
grew  out  of  a previous  project  in  1999  and  2000  in  which  teachers  explored  the  uses  of  technology  in  K-12 
history  classrooms  and  developed  lessons  plans  for  statewide  distribution.  Although  we  felt  that  the  work 
which  emerged  from  that  project  was  of  a high  quality,  we  did  not  feel  teachers  were  exploring  the  issues 
involved  in  technology  and  the  classroom  at  the  depth  that  we  were  interested  in.  We  modified  the 
program  in  two  ways.  Instead  of  asking  teachers  to  generate  lesson  plans,  we  asked  them  to  engage  in 
action  research  in  the  classroom.  We  felt  this  methodology  was  more  likely  to  get  teachers  to  think  about 
their  teaching  in  a reflective  manner.  The  second  change  we  made  is  we  focused  much  more,  although  not 
exclusively,  on  online  discourse  tools.  The  reason  for  this  was  from  the  previous  program  - the  teacher  in 
that  program  expressed  interest  in  the  use  of  these  tools  in  the  history  classroom.  So,  in  2001,  twenty-six 
teachers  from  across  California  conducted  research  in  grade  3-12  classrooms  with  diverse  student 
populations,  low  performing  and  ESL  students.  The  collective  results  of  these  research  studies  was 
surprisingly  uniform.  Students,  particularly  low  performing  and  ESL  students,  engaged  in  significantly 
higher  levels  of  discourse  than  in  the  traditional  classroom;  and  they  were  either  learning  to  engage  in  or 
engaging  in  discipline  based  activities  (‘doing  history’).  The  interaction  of  these  two  - discourse  and 
activities  - lead  to  higher  levels  of  historical  thinking  and  understanding,  as  reported  by  the  teachers.  This 
result  was  only  true  when  the  teachers  provided  scaffolding  in  the  form  of  discourse  supports. 


Purpose/Theoretical  Perspective 

The  primary  purpose  of  the  professional  development  is  to  engage  and  support  teachers  to  investigate 
instructional  methods  that  support  students’  development  of  historical  thinking  and  understanding  through 
a discipline  based  reflective  learning  process,  and  to  investigate  using  technology  to  support  the 
instructional  methodology  in  ways  that  could  not  be  done  without  technology. 

Fundamental  to  how  we  approach  the  professional  development  is  our  approach  to  history.  We  approach 
the  teaching  of  K-12  history  as  a discipline  as  opposed  to  a subject  (Steam,  1993).  As  part  of  practicing 
history,  historians  engage  in  discourse  about  the  discipline  and  in  discipline  based  activities.  We  reflect 
this  in  the  K-12  classroom.  Discourse  in  the  History/Social  Science  classroom  supports  students  in 
externalizing  thinking  and  in  creating  cultural  supports  for  thinking  (Bain,  1 998).  Discourse  interacting 
with  discipline  based  activities  provides  a basis  for  students  to  ‘do  the  discipline’  and  engage  in  higher 
levels  of  thinking  and  historical  understanding.  For  this  to  be  successful,  teachers  must  provide  social 
assistance  (scaffolding)  to  the  learners  to  support  the  necessary  competencies  through  which  the  historical 
thinking  and  understanding  can  emerge  and  be  internalized  (Vygotsky,  1978).  There  are  many  kinds  of 
scaffolding  that  can  be  used.  Technology  based  tools,  and  concomitant  teacher  supplied  supports,  are  one 
kind  of  assistance  (Salomon,  1988),  which  we  focus  on. 

We  model  this  approach  with  the  professional  development.  We  approach  the  teaching  of  history  as  a 
discipline.  We  engage  in  discourse  with  cultural  supports  to  externalize  the  teachers’  thinking.  We  use  an 
action  research  model  as  it  is  an  ideal  vehicle  to  manifest  this.  Action  research  provides  for  inquiry  through 


reflection,  it  brings  the  unconscious  to  a conscious  level  (Schon,  1993).  The  teachers*  understandings  can 
emerge  and  be  internalized.  And,  once  again,  among  many  forms  of  scaffolding  that  are  used,  technology 
tools  are  used  support  this. 

Interaction  between  discourse  and  action  research  are  therefore,  central  to  this  professional  development 
approach.  We  see  discourse  as  a creative  process  in  which  a shared  understanding  is  created  (Bohm,  1996). 
To  engage  in  discourse  is  to  engage  in  both  disciplines  - history  and  education.  It  is  reflective  and  iterative 
in  nature.  It  involves  social  assistance  and  the  use  of  tools.  Scaffolding  is  provided  by  the  professional 
developers  (facilitators).  Tools  are  non-electronic  (small  group  discussion,  writing)  and  electronic  (email, 
bulletin  boards,  chat,  threaded  discussion,  databased  discussion,  electronic  annotation). 

Action  research  is  also  a process.  It  is  a way  to  engage  with  classroom  teaching  and  bring  more  of  it  to  a 
conscious  level  (Hopkins  & Antes,  1990).  It  is  reflective  and  iterative  in  nature.  Reflection  encourages  the 
challenging  of  ones  existing  theories  and  preconceived  views  of  teaching  (Kettle  & Sellars,  1996).  Action 
research  involves  social  assistance  and  the  use  of  tools.  As  with  discourse,  scaffolding  is  provided  by 
facilitators  (coaches)  and  tools  are  both  non-electronic  and  electronic. 

As  elements  of  both  of  these  processes  we  engage  in  reflection,  collaboration  and  inquiry. 


Professional  Development  Model 

We  see  our  professional  development  model  as  a system  of  people,  practices,  and  technologies.  The  human 
activities  are  served  by  the  technology  (Nardi  & O’Day,  1998).  Its  parts  consist  of  facilitators,  teachers  and 
students;  the  practices  of  discourse  and  action  research,  in  the  disciplines  of  history  and  education;  and  the 
supporting  technologies. 

Teachers  participate  in  online  pre-institute  discourse  activities,  then  spend  three  days  at  UCLA  starting  the 
research  process  (see  below).  Back  in  the  classroom,  they  implement  their  action  research.  Finally,  they 
write  up  their  results  for  dissemination.  In  total,  this  is  a five  to  six  month  commitment. 

The  action  research  process  consists  of  six  overlapping  stages 

• First,  they  question  their  assumptions  about  the  disciplines.  Through  online  discourse  before  the 
institute  and  in  person  discourse  during  the  institute,  the  hidden  assumptions  that  we  all  have  are 
brought  to  the  surface. 

• Second,  they  pose  a problem  (research  question).  They  will  discuss  these  with  the  other  teachers 
and  give  other  teachers  feedback  on  their  problems. 

• Three,  as  each  teacher  focuses  on  one  problem,  one  aspect  of  their  teaching,  the  plan  for  the 
solution  should  emerge  (research  plan)  and  worked  out  collaboratively. 

• Four,  they  will  implement  the  action  research  plan  in  their  classroom. 

• Five,  both  qualitative  and  quantitative  data  will  be  gathered  by  all  teachers,  analyzed  and  shared 
with  peers  to  assure  the  highest  level  of  reliability  and  validity  possible. 

• Six,  the  process  and  the  results  of  the  action  research  are  documented,  peer  reviewed  and 
disseminated. 

The  first  three  stages  of  this  process  are  the  most  difficult  for  the  teachers.  Once  they  have  a plan  of  action, 
implementing  and  following  through  on  the  plan  has  not  posed  too  many  problems.  Because  of  this,  and 
based  on  the  research  from  the  2001  group,  a proposed  discourse/action  interaction  process  has  been 
developed  as  a scaffolding  tool  for  the  teachers  through  the  action  research  process.  This  tool  is  used  by 
the  facilitators  and  can  be  used  by  the  teachers  in  their  own  classroom.  It  is  considered  a tool-in-progress. 
With  each  iteration  of  professional  development,  it  is  anticipated  that  the  tool  will  change  in  response  to  the 
teachers  input  and  their  experiences  in  the  classroom.  It  is  described  in  more  detail  below. 

Other  tools  that  both  support  the  professional  development  and  are  the  focus  of  the  action  research  are 
primarily  online.  Blackboard  was  used  as  an  online  environment  and  we  will  be  using  it  again.  Within 
Blackboard,  email,  bulletin  boards,  threaded  and  non-threaded  discussion,  chat  (and  archived  chat),  posting 


and  annotation  of  documents,  the  development  of  individual  pages,  and  annotated  links  and  resources  all 
provide  discourse  tools  for  the  action  research. 


The  Discourse/Action  Interaction  (see  diagram  at  end  of  paper) 

As  part  of  their  action  research,  teachers  are  asked  to  engage  in  instruction  which  leads  to  students 
engaging  in  a discipline  based  reflective  thinking  process.  Although  we  believe  that  there  are  a number  of 
major  components  to  this  process  - at  a minimum  discourse,  visual  thinking  and  experiential  thinking  - at 
this  time  we  are  focusing  on  discourse.  As  professional  developers  we  are  asking  our  teachers  to  engage  in 
the  same  discourse/action  interaction  and  discipline  based  reflective  thinking  process.  We  believe  it  is 
crucial  to  this  professional  development  program  that  the  teachers  engage  in  the  same  kinds  of  learning 
activities  we  are  asking  them  to  use  with  their  students.  We  will  go  over  the  discourse/action  interaction 
we  are  using  as  it  would  be  applied  in  a K- 12  classroom,  and  then  we  will  discuss  how  this  differs  for  the 
professional  development  program. 

When  we  are  considering  the  role  of  discourse  in  a discipline  based  reflective  learning  process,  we  focus  on 
four  major  areas.  The  students  should  engage  in  a continuous  discourse/action  process  where  discourse 
leads  to  action  which  leads  to  discourse  and  so  on.  So  discourse  and  the  activities  are  two  of  the  areas.  The 
third  area,  teacher  provided  supports,  reflects  the  teacher’s  role  in  the  students  process.  The  teacher 
supplies  a number  of  scaffolds  or  supports  to  the  acquisition  of  the  reflective  learning  process.  All  of  these 
three  areas  feed  and  support  the  fourth  area  - the  student  reflective  process.  So,  students  develop  a 
reflective  learning  process  through  the  interaction  of  discourse  with  activities  and  the  judicious  and 
considered  use  of  supports  from  the  teacher. 

In  their  action  research,  the  teachers  focus  more  specifically  on  the  kinds  of  discourse,  the  kinds  of 
activities,  how  technology  supports  those  and  an  exploration  of  what  teacher  provided  supports  seem  to 
provide  the  most  assistance,  in  what  context,  to  which  students.  This  past  year  the  focus  was  on  chat 
rooms,  threaded  discussion  groups,  bulletin  boards  and  database  based  discussion.  Generated  by  the 
teachers,  they  focused  on  the  questions  outlined  below: 

• Do  these  tools  provide  greater  opportunity  to  engage  in  discourse?  Does  this  result  in  historical 
thinking  and  understanding? 

• Do  the  students  engage  in  more  discourse  with  the  technology  tools?  Are  they  more  motivated  to 
engage  in  discourse?  Does  the  type  of  discourse  that  technology  provides  facilitate  deeper 
historical  thinking  and  understanding  than  other  forms  of  discourse? 

• Does  the  technology  interfere  in  any  way  with  the  discourse  process? 

• What  does  the  teacher  have  to  provide  in  the  form  of  scaffolding  tools  to  facilitate  optimal 
discourse? 

We  have  used  the  results  of  the  teachers’  work  in  these  areas  to  expand  on  the  four  major  areas  in 
discourse/action  interaction.  Some  work  has  been  done  on  the  area  of  online  discourse  tools.  This  has 
been  summarized  by  Sherry  (2000)  in  a thorough  article  available  online.  We  have  drawn  from  many  of 
the  ideas  in  this  article  and  combined  it  with  our  experience  with  our  teachers  to  structure  an  approach  to 
thinking  about  discourse  in  the  classroom.  We  ask  our  teachers  to  first  consider  the  structure  of  the 
discourse  which  currently  exists  in  their  classroom.  Typically  this  will  reflect  a teacher  asks  question, 
student  answers  question,  teacher  provides  feedback  format  with  some  minor  variations.  It  will  also 
typically  involve  only  10  15%  of  the  students  and  virtually  no  ESL  or  low  performing  students  (from  our 

teachers).  We  proposed  going  in  the  direction  of  teacher  questions,  students  question,  students  answer, 
students/teachers  support/share  and  students/teacher  comments  where  these  may  not  occur  in  any  particular 
order.  The  teachers’  role  in  this  form  of  discourse  is  to  support  the  student  to  create,  share,  negotiate, 
interpret,  expand,  justify,  question,  summarize,  clarify,  predict,  extend,  and  so  on.  As  the  students 
expertise  increases,  there  can  be  more  of  a focus  on  framing  questions  appropriately,  expanding  on  valid 
propositions,  students  being  open  to  critical  review  of  their  point  of  view,  students  sharing  understandings, 
working  towards  a common  understanding,  common  knowledge  construction,  and  so  on.  The  environment 
should  be  constantly  evolving,  yet  be  organized. 

There  is  room  for  teachers  to  focus  action  research  questions  in  nearly  every  area  of  the  discourse/action 
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interaction.  In  particular,  every  teacher  eventually  gets  around  to  exploring  what  kinds  of  teacher  provided 
supports  will  best  facilitate  the  discourse.  As  most  of  the  discourse  occurs  in  an  online  environment,  the 
teacher  support  questions  are  not  just  about  supporting  discourse,  but  on  how  to  use  the  technology  to  best 
advantage  to  support  the  discourse.  These  questions  can  range  from  — when  do  you  use  chat  or  threaded 
discussion?  To  what  cultural  implications  are  there  to  the  introduction  of  technology  on  this  scale  in  my 
classroom?  This  last  question  came  up  in  post-research  musings  by  the  teachers  last  year.  We  will  be 
tackling  it  in  a more  focused  way  with  this  years  program. 

The  activity  area  of  the  discourse/action  interaction  involves  discipline  based  activities  - activities  that  are 
relevant,  and  authentic.  These  activities  can  be  multidisciplinary,  interactive,  exploratory,  performance 
based,  involve  advance  skills,  etc.  These  activities  should  interact  with  discourse  activities  in  such  a way 
as  to  enhance  the  development  of  both  the  new  discourse  process  for  the  students  and  the  process  of 
developing  the  skills  to  ‘do  the  discipline’. 

As  teachers  think  about  the  discourse/action  interaction  and  develop  their  research  questions,  they  will  be 
themselves  engage  in  discourse  activities  in  the  online  environment.  Here  is  the  challenge  for  our 
facilitators.  Many  of  these  teachers  do  not  have  a well  developed  discourse  process.  We  must  provide  the 
appropriate  scaffolding  to  support  these  teachers  in  engaging  in  the  process  that  they  will  be  working  with 
their  own  students  to  develop.  Our  research,  other  than  reporting  on  the  teachers’  results,  is  to  develop 
these  scaffolds  as  the  project  progresses,  and  evaluate  the  results.  The  discourse/action  interaction  is  one 
such  scaffold.  It  is  self-referential  in  this  way  - it  scaffolds  the  teachers  to  help  them  scaffold  their  students 
by  also  using  the  discouse/action  interaction. 

Examples  and  Results  from  2001 

Teachers  approached  the  implementation  of  the  research  in  a number  of  different  ways.  The  following 
three  examples  are  illustrative  of  the  projects  as  a whole. 

At  a high  school  near  San  Diego,  CA,  the  participating  teacher  had  a history  class  of  36  primarily  ESL  and 
low-performing  students.  He  took  advantage  of  a school  with  a block  schedule  to  spend  a 1.5  hour  block 
for  this  class  in  the  computer  lab  in  a chatroom  on  Blackboard.  All  the  students  were  still  in  the  same 
room,  but  discussing  history  in  a chatroom.  This  step  alone  allowed  100%  participation  from  the  students 
(instead  of  15%  in  a traditional  classroom).  He  then  followed  this  discourse  up  with  a 1.5  hour  activity 
where  the  students  wrote  about  their  ideas.  A higher  level  of  historical  thinking  was  evidenced  in  their 
writing.  The  most  promising  result  however,  were  the  gains  made  over  time  in  the  ability  to  write  in 
English  for  these  ESL  students. 

At  a middle  school  near  Riverside,  CA  a participating  teacher  used  threaded  discussion  to  supplement  the 
classroom  activity.  At  first,  without  adequate  teacher  supports,  participation  was  poor  and  the  level  of  the 
discussion  was  low.  He  instituted  several  teacher  provided  supports,  such  as  more  detailed  instructions  on 
writing  entries,  and  he  provided  immediate  (24  hours)  and  explicit  feedback  to  the  students.  The  results 
were  an  increase  in  participation  (not  quite  100%)  and  more  thoughtful  contributions.  He  found  that  a 
particularly  useful  byproduct  of  the  use  of  the  threaded  discussion  was  his  ability  to  diagnose  and  address 
misconceptions  and  preconceptions  much  more  quickly  than  he  could  in  the  classroom. 

At  a middle  school  in  Santa  Monica,  CA,  a participating  teacher  developed  her  own  database  for  student 
responses  to  open  ended  ‘thought’  questions  on  history.  As  each  student  responded  to  the  question,  they 
could  also  read  the  responses  other  students  had  entered.  They  can  then  respond  to  other  students,  develop 
arguments,  which  combine  other  student’s  points  of  view,  or  refute  a student’s  point  of  view.  She  provided 
both  discourse  and  content  supports  within  this  environment.  Both  motivation  and  participation  increased 
dramatically  over  the  traditional  method.  She  was  surprised  at  the  level  of  participation  from  students  who 
literally  did  not  participate  in  any  other  way.  She  also  found  unexpected  gains  with  her  ESL  students  in 
their  writing  skills. 

The  following  results  were  seen  in  every  teacher’s  research  study. 

For  teachers: 

• The  development  of  and  use  of  discourse  supports  embedded  in  the  classroom  environment 
(technological  and  non-technological)  was  critical  to  promoting  optimal  engagement  in  the 
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technology-based  discourse  activities. 

• Increased  opportunity  to  diagnose  and  correct  misconceptions. 

For  students: 

• Motivation  to  participate  in  technology-based  discourse  activities  was  much  higher  than  to 
participate  in  non-technology  discourse  activities. 

• Non-technology  discourse  activities  were  positively  affected,  both  in  the  level  and  amount  of 
participation. 

• Low-performing  students  showed  the  greatest  gains  in  both  participation  and  understanding. 

• English  as  a Second  Language  learners  benefited  significantly.  Their  interest  in  participating, 
their  ability  to  express  ideas  and  their  writing  skills  all  showed  more  than  expected  gains. 

• Students  who  had  previously  excelled  at  non-technology  based  discourse  were  the  least 
enthusiastic. 

• Nearly  100%  of  students  voiced  their  opinions,  received  feedback  and  responded  to  others’ 
opinions. 

• A significant  percentage  of  students  used  historical  thinking  and  understanding  including  citing 
historical  fact,  comparing  differing  primary  sources,  questioning  others’  interpretations,  and  citing 
each  other. 

In  addition,  many  teachers  had  results  from  their  research  that  weren’t  addressed  in  the  other  research 
studies.  These  included: 

• Students  did  not  adhere  to  ‘clique’  boundaries  when  using  technology-based  discourse  tools. 

• Students  were  using  the  tools  to  expand  discussions  beyond  the  classroom. 

Discourse  is  an  important  methodology  in  History/Social  Science  classrooms.  The  traditional  forms  of 
discourse  are  the  classroom  discussion  and  the  small  group  discussion.  Teachers  find  that  both  of  these 
forms  are  limited.  The  teachers  who  participated  in  this  research  believe  that  less  than  15%  of  their 
students  participated  in  these.  The  research  these  teachers  conducted  indicates  that  the  use  of  technology- 
based  discourse  tools  seems  to  increase  both  the  participation  in  (to  nearly  100%)  and  the  level  at  which 
students  are  engaging  in  historical  thinking  and  understanding.  This  increase  gives  the  teacher  a solid 
foundation  on  which  to  make  the  change  from  history  as  a subject  which  teachers  teach  and  students  take, 
to  a discipline,  where  historical  thinking  and  understanding  are  core  processes. 

This  year,  by  providing  more  support  in  the  form  of  the  discourse/ action  interaction,  we  hope  that  teachers 
will  be  able  to  form  more  focused  questions  than  they  did  last  year  and  concentrate  more  specifically  on 
issues  such  as  the  appropriate  use  of  various  forms  of  teacher  provided  support  for  different  students  (ESL 
and  low-performing,  in  particular)  different  technologies,  and  different  pathways  to  the  development  of  a 
new  discourse  process  for  the  students. 


Conclusion 

This  is  an  action  research  project.  As  such,  it  has  yielded  rich  information  on  the  use  of  discourse  and  the 
use  of  technology  to  support  discourse  for  the  advancement  of  student  historical  thinking  and 
understanding  in  History/Social  Science  classrooms.  It  is  the  intention  of  all  the  teachers  in  this  project  that 
action  research  on  these  issues,  and  on  the  issues  that  these  studies  brought  up,  be  continued.  It  is  also  the 
hope  of  all  the  teachers’  that  a more  formal  treatment  of  these  issues  will  be  pursued  within  the  professional 
community. 
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Abstract;  We  have  developed  SASK  (Socratic  ASK),  a domain- independent  architecture 
for  implementing  Socratic  dialogs  to  foster  deeper  student  reflections  on  well-defined  tasks. 
In  SASK  we  have  built  the  Dialysis  Mentor,  a program  that  uses  Socratic  questioning  to 
improve  student  performance  and  learning  in  an  undergraduate  biomedical  engineering  lab. 
Small  usability  tests  and  a pilot  run  in  the  actual  lab  suggests  that  Dialysis  Mentor  and 
SASK  systems  in  general  can  improve  the  value  of  pre-defined  leam-by -doing  task 
experiences.  We  are  now  working  on  improving  our  SASK  Mentors  and  building  authoring 
tools  for  them. 

SASK  Mentor  Project:  http://www.cs.northwestem.edu/~riesbeck/sask/ 


Introduction 

Dr.  J.  Walsh’s  Biomedical  Engineering  Laboratory  course  at  Northwestern  University  includes  a 
dialysis  lab  session  in  which  students  characterize  how  well  an  artificial  kidney  (a  dialyzer)  transports  water, 
urea,  and  salt  across  its  membrane.  Students  work  together  in  groups  of  two  or  three  for  several  hours  on  this 
lab.  This  lab  typically  uncovers  a number  of  important  gaps  in  student  understanding,  ranging  from  simple 
trouble  using  the  lab  devices,  to  misinterpretation  of  key  measurements,  to  performing  actions  contradictory 
to  the  basic  goal  of  the  experiment.  Dr.  Walsh  handles  such  problems  not  by  telling  the  student  what  to  do, 
but  by  asking  a few  penetrating  questions,  such  as  “What  are  you  trying  to  do  here?”  followed  by  “What 
variables  are  you  controlling?”  These  questions  cause  the  students  to  quickly  discover  for  themselves  gaps  in 
their  understanding  of  the  task.  Without  such  attention,  students  tend  to  rush  through  their  lab.  Many  fail  to 
produce  any  useful  results.  Even  those  who  do,  likely  miss  many  opportunities  to  reflect  on  their  thinking 
while  they  are  doing  their  lab  work.  Such  reflection  is  known  to  increase  students’  ability  to  transfer  their 
learning  to  new  settings  and  events  (Bransford  et  al.  2000). 

This  Socratic  technique  is  facilitated  by  broad  knowledge  of  the  domain  and  how  one’s  students  best 
learn  that  particular  domain.  Often  people  with  such  expertise  are  a scarce  resource.  Most  graduate  assistants 
lack  broad  domain  knowledge  as  well  as  experience  in  teaching  it  Socratically.  Hence,  groups  in  Dr.  Walsh’s 
lab  would  often  wait  20  minutes  for  his  guidance  before  they  could  make  any  progress.  Sometimes  groups 
had  wasted  time  doing  something  unfruitful  and  would  have  to  continue  the  lab  during  the  next  class  session. 
Even  worse,  sometimes  a group  would  get  too  much  information  from  a graduate  assistant  and  finish  the  lab 
without  gaining  the  understanding  Dr.  Walsh  had  hoped  they  would. 

Our  project  goal  was  to  provide  an  artificial  mentor  (We  use  the  term  "mentor”  instead  of  "tutor" 
because  tutoring  implies  a teacher-centric,  drill  and  practice  activity.  Mentoring  implies  that  we  are  trying  to 
provide  a student  centered,  apprenticeship,  inquiry  activity.)  capable  of  providing  a service  similar  to  that  Dr. 
Walsh  provides,  at  the  time  students  need  it.  A Socratic  system  makes  students  active  learners,  leading  them 
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to  debug  their  own  thinking  and  knowledge,  unlike  alternatives  such  as  Frequently  Asked  Questions  (FAQ) 
lists  and  other  systems  that  encourage  passive  learning. 


The  Project’s  Development 

The  Dialysis  Mentor 

The  SASK.  engine  and  the  Dialysis  Mentor  (DM)  application  were  developed  in  summer  and  fall  of 
2001,  using  Java  for  the  engine,  XML  for  the  domain  content,  and  QuickTime  for  Java  to  play  videos.  When 
students  start  DM,  they  first  see  the  “Overview”  interface  (Fig.  1),  where  they  can  watch  a video  of  Dr. 
Walsh’s  lecture  on  the  dialysis  lab,  and  read  a transcript  of  the  lecture. 

Students  then  switch  to  the  “Lab  Mentor”  (Fig.  2)  to  interact  with  the  program.  This  interface 
includes  an  annotated  diagram  of  the  laboratory  set-up  for  reference  (upper  left),  a dialog  transcript  (lower 
left),  and,  most  importantly,  a dialog  interaction  panel,  upper  right.  In  future,  we  plan  to  include  video  clips  of 
Dr.  Walsh  speaking  to  the  students  in  the  role  of  the  Dialysis  Mentor. 


The  conversation  panel  was  the  key  interface  challenge,  specifically  the  part  where  students  answer 
questions  such  as  “What  are  you  trying  to  do?”  On  one  hand,  the  interface  had  to  avoid  overly  influencing  or 
constraining  what  students  say.  What  students  say  is  the  key  window  to  their  underlying  misconceptions.  On 
the  other  hand,  an  empty  text  box  where  anything  can  be  written  is  both  impossible  for  current  programs  to 
understand  and  intimidating  to  students. 

Therefore,  we  use  a mixed  button  and  template  approach.  Simple  answers  with  no  variable  content, 
such  as  “We’re  still  thinking,”  are  buttons.  More  complex  answers  with  variable  content  are  buttons  that  bring 
up  structured  “fill  in  the  blanks”  templates  similar  to  web  forms.  For  example,  clicking  on  “We’re  trying  to 
measure...”  brings  up  a template  of  the  form  “We’re  trying  to  measure  <empty  text  field>  and  control 
<empty  text  field>.”  (Fig.  3).  The  students  can  write  anything  they  want  in  the  text  fields.  This  semi  - 
structured  approach  lets  students  choose  what  to  say  and  how  to  say  it,  but  the  program  only  has  to  deal  with 
short  phrases  in  well-defined  contexts.  This  interface  approach  was  successfully  used  in  Creanimate,  a 
program  that  used  a question  and  answer  dialog  to  help  children  learn  and  think  about  zoology  (Edelson 
1993). 


Figure  1:  Overview  Mode 


Figure  2:  Lab  Mentor  Screen 
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Figure  3:  After  students  choose  "We're  trying  to  measure..." 

The  program’s  behavior  is  specified  in  the  Dialog  Graph  Document,  represented  in  XML,  that 
contains  all  the  Dialysis  Mentor’s  questions,  response  options,  and  template  patterns  for  selecting  follow-up 
questions.  The  current  DM  graph  has  89  utterance  nodes  and  1 53  edges  (rules  that  lin  k student  responses  to 
DM  utterance  nodes)  in  it.  Figure  4 shows  part  of  a small  part  of  that  graph  in  outline  form.  Student  responses 
with  underlined  elements  are  template  patterns,  the  other  responses  are  buttons.  This  fragment  Socratically 
asks  students  to  test  their  assumption  about  what  controls  what,  which  will  lead  them  to  discover  a mistake  in 
their  thinking. 


Mentor’s  current  goal:  Students  seem  to  know  the  main  goal.  Lead  student  to  realize  that  varying  the 
flow  does  not  help  the  goal 

Mentor:  Remember  that  when  you  do  an  experiment  you  want  to  vary  only  one  thing  at  a time.  Then  you 
graph  Ultrafiltration  (U)  over  Transmembrane  Pressure  (TMP). 

Do  you  think  you  will  see  a relationship  between  Ultrafiltration  (U)  and  flow  rate(s)? 

Students:  Yes 

Mentor:  What  controls  the  Ultrafiltration  (U),  TMP  or  flow  rate(s)? 

Students:  The  flow  rate 

Mentor:  How  do  you  know  U is  not  controlled  by  the  TMP? 

Students:  I do  not  know 

Mentor:  Please  set  up  your  experiment  so  you  can  tell  me  what  controls 
U,  TMP  or  flow  rate. 

Students:  TMP  directly  controls  U based  on  the  equation  you  gave  me. 

Mentor:  That’s  the  theory  we  presented.  You  must  use  the  lab  to  prove 
that  TMP  controls  U and  not  flow  rate.  How  do  you  change  only  TMP? 
Students:  ANY  OTHER  ANSWER  directly  controls  U based  on  the  AA/Y  OTHER 
ANSWER. 

Mentor:  Please  set  up  your  experiment  so  you  can  prove  what  controls 
U,  TMP  or  flow  rate. 


Figure  4:  Partial  Graph  of  Dialysis  Mentor’s  Task  Dialog  Graph  Document 


Background 

Socratic  Intelligent  Tutoring 

A classic  early  Socratic  intelligent  tutoring  system  was  WHY  (Wenger  1987)  created  by  Dr.  Allan 
Collins.  Collins  analyzed  the  dialog  between  students  and  Socratic  tutors  and  developed  a theory  for  the 
implementation  of  a Socratic-tutoring  program,  including  a set  of  24  production  rules  to  improve  WHY’s 
pedagogical  component  (Collins  1977).  We  used  some  of  these  rules  as  a formal  way  to  understand  Socratic 
techniques.  For  instance,  the  partial  graph  in  Figure  4 is  an  example  of  Collins’s  Socratic  Rule  15:  “Request 
a test  of  the  hypothesis  about  a factor”. 


O 
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ASK  Systems 


ASK  systems  are  a form  of  multimedia  based  on  the  metaphor  of  having  a conversation  with  an 
expert,  or  a group  of  experts.  An  ASK  system  presents  a user  with  a set  of  initial  top-level  questions.  When 
the  user  selects  one,  the  ASK  system  responds  with  an  answer,  either  in  video  or  text.  In  addition,  the  system 
displays  a set  of  follow-up  questions  relevant  to  the  answer  given.  The  user  can  pursue  one  of  these  follow- 
ups, which  will  lead  to  a new  answer  and  a new  set  of  follow-ups,  or  return  to  an  earlier  answer  and  follow  a 
different  path  (Cleary  1995).  A number  of  ASK  systems  in  a wide  variety  of  domains  were  developed  by  the 
Institute  for  the  Learning  Sciences  at  Northwestern  University,  and  later  commercially  by  Cognitive  Arts 
Corporation.  The  success  of  ASK  systems  in  a number  of  domains  leads  us  to  believe  that  static  dialog  graphs 
are  sufficient  to  provide  performance  and  learning  support  in  well-defined  tasks. 


Validation 

Usability  and  Pilot  Testing 

A small  usability  test  was  performed  with  a biomedical  engineering  undergraduate  student.  We 
videotaped  the  student  doing  the  dialysis  lab  under  Dr.  Walsh’s  tutelage.  Her  interactions  were  typical  of 
those  in  the  actual  lab,  and  was  used  as  a seedbed  for  the  initial  dialog  graph.  Several  months  later,  we 
videotaped  her  interacting  with  the  first  version  of  the  Dialysis  Mentor.  This  test  revealed  a number  of 
phrases  that  needed  to  be  added,  as  well  as  some  new  dialog  branches.  It  also  showed  that  it  was  very  easy  for 
certain  DM  responses  to  lead  the  student  to  treat  the  system  like  a multiple  choice  guessing  game.  This  was 
particularly  true  when  the  DM’s  immediate  follow-up  to  a student  response  strongly  implied  that  the  response 
was  incorrect  or  unwise.  To  avoid  this  unfortunate  phenomenon,  some  follow-ups  were  changed  to  avoid 
immediate  implications  of  any  judgment.  This  made  the  dialogs  more  Socratic. 

The  Dialysis  Mentor  was  then  used  in  Dr.  Walsh’s  dialysis  lab  in  October  2001.  Fifty-two 
biomedical  engineering  undergraduates  were  involved  in  two  sections,  a morning  section  and  an  afternoon 
section.  Each  section  had  nine  groups  of  two  to  three  students  each.  The  DM  logged  all  interactions  for  later 
analysis.  Some  interactions  were  video  taped.  A few  weeks  after  the  use  of  DM  in  the  lab,  47  students 
completed  a survey  regarding  their  experience.  Although  the  test  in  general  is  considered  a success, 
preliminary  analysis  of  the  data  revealed  or  emphasized  issues  for  future  research.  In  particular.  Table  1 
categorizes  the  DM’s  responses  by  whether  the  intended  follow-ups  to  student  responses  were  taken. 

“Missing  phrase  in  task  dialog  graph  document”  meant  that  the  DM  had  a relevant  dialog  branch  internally, 
but  did  not  recognize  that  a student  response  should  take  that  branch.  “Wait  too  long”  meant  that  the  DM  told 
students  that  it  would  wait  for  them  to  do  something  but  waited  an  improperly  long  period  of  time.  “No 
existing  branch”  meant  that  the  DM’s  dialog  graph  didn’t  include  a relevant  branch  for  the  student’s  response. 
“Appropriate  action”  meant  that  the  DM  selected  the  dialog  branch  that  we  had  intended  for  a given  response. 


McMitor  Action  Category 

% Mentor  Actions  Across 
Groups 

Count  Across 
Groups 

Mean  Count  per 
Group  j 

Missing  phrase  in  task  dialog  graph 

2.82% 

25 

2.27 

document 

Wait  too  long 

3.83% 

34 

1.89 

No  existing  branch 

22.41% 

199 

10.47 

Appropriate  action 

70.38% 

625 

27.17 

TOTAL 

888 

49.33 

Table  1;  Dialysis  Mentor’s  Categorized  Actions  with  Students  in  Pilot  Test 

The  following  chart  from  the  surveyed  opinions  of  47  of  the  52  students  who  were  in  the  pilot  test 
shows  that  the  vast  majority  felt  that  the  DM  was  at  least  somewhat  helpful  to  them.  Given  the  rough 
preliminary  nature  of  the  DM,  this  is  quite  encouraging. 


How  much  do  you  agree  with  the  following  statement?  The 
Dialysis  Mantor  program  was  helpful.” 


Chart  1 : Student  Opinions  on  the  Helpfulness  of  the  Dialysis  Mentor 


The  following  chart  shows  the  results  of  the  conceptual  assessment  question  in  the  same  survey. 
Answers  to  the  question  “What  parameters  should  one  vary  when  quantifying  the  hydraulic  permeability  and 
what  parameters  should  one  hold  constant?”  was  scored  by  first  noting  who  answered  correctly  to  what 
parameters  should  be  varied  and  then  who  answered  correctly  to  what  should  be  held  constant.  9 students 
(19.15%)  were  incorrect  about  what  to  vary  and  12  students  (25.5%)  were  incorrect  about  what  to  control.  6 
students  (12.77%)  were  found  incorrect  about  both  parts  of  the  question  and  9 students  (19.15%)  were 
partially  correct.  32  students  (68.09  %)  answered  the  question  completely  correct. 


What  pftmmotQra  ehoyitf  cmw  vary  whan  quointsrvh^  (ho 
hvdrmUc  perraeabSitv 

and  what  parameters  stwMdd  one  hold  constant? 


Chart  2:  Student  Responses  to  Conceptual  Assessment  Question 


Future  Work 

Social  Interaction  and  Interpretation  Issues 

Videotapes  of  the  pilot  tests  show  cases  where  a student  group  was  at  a loss  for  an  answer  to  a 
question  presented  by  the  DM.  When  Dr.  Walsh  asked  the  students  to  pretend  he  had  just  asked  them  the 
same  question,  the  students  quickly  came  up  with  an  answer.  When  they  entered  that  answer  into  the  DM,  it 
was  usually  handled  properly  and  the  dialog  progressed.  In  other  words,  students  treated  questions  from  the 
DM  differently  from  the  same  questions  from  Dr.  Walsh. 

There  are  several  possible  explanations  for  this  phenomenon.  One  is  the  difference  between  text  and 
speech.  Questions  that  are  clear  when  spoken  often  become  ambiguous  in  text  form.  Another  factor  is  the 
difference  in  social  relationships  between  a computer  program  and  a faculty  member.  When  a faculty  member 
like  Dr.  Walsh  says  “Think  about  it.  I’ll  ask  you  again  in  5 minutes,”  students  stop  and  think.  When  a lowly 
computer  program  like  the  DM  said  the  same  thing,  students  often  do  whatever  they  can  to  get  around  the 
delay,  including  restarting  the  program.  Since  getting  students  to  stop  and  think  is  the  main  goal  of  a SASK 
system,  overcoming  this  difference  in  response  is  a critical  research  problem. 

For  these  reasons,  we  are  planning  to  use  video  clips  of  Dr.  Walsh  asking  the  questions,  in 
conjunction  with  the  text  version.  The  addition  of  audible  speech  may  improve  DM  as  a pedagogical  agent 
and  improve  students’  retention  and  problem-solving  transfer  abilities  (Moreno  et  al.  2001).  It  may  also 
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increase  students  feeling  like  they  are  talking  to  Dr.  Walsh,  or  someone  like  him,  therefore  reducing  the 
differences  in  their  response  patterns. 

We  also  need  to  make  sure  students  do  not  guess  at  answers,  because  this  is  antithetical  to  learning. 
Guessing  happens  when  students  believe  they  can  tell  if  their  response  choice  was  correct  or  not,  based  on  the 
system’s  first  follow-up.  That  makes  it  easy  for  students  to  try  each  response  and  see  what  happens. 

Therefore,  we  need  to  minimize  two  kinds  of  initial  follow-ups:  negative  “no,  because...”  follow-ups,  and  “I 
don’t  understand,  please  rephrase”  follow-ups.  The  latter  happened  approximately  25%  of  the  time  in  the 
DM,  either  because  of  missing  phrases  (3%),  or  missing  branches  (22%).  Fortunately,  avoiding  bad  follow- 
ups is  mostly  a matter  of  expanding  and  refining  the  Dialog  Graph  Document. 

Authoring  Tools  and  Internet  Accessibility 

Another  area  of  future  design  research  will  be  the  development  of  the  authoring  tool  for  dialog 
graphs.  The  Dialysis  Mentor’s  XML  files  were  authored  with  a text  editor.  Even  with  an  XML  editor,  this  is 
a tedious  and  technically  complicated  activity.  We  plan  to  design  an  authoring  tool  so  that  teachers  such  as 
Dr.  Walsh  can  build  the  dialog  graph  document  themselves.  Of  particular  interest  will  be  the  development  of 
“dialog  sequence  templates,”  based  on  Collins’  Socratic  rules  and  other  useful  dialog  patterns  we  have 
discovered  while  designing  the  Dialysis  Mentor. 

Using  Java  Servlet  technology,  we  also  have  a preliminary  web  interface  to  the  SASK  Mentor  to 
ease  deployment  to  many  sites. 

Conclusions 

Overall,  our  SASK  Dialysis  Mentor  has  the  potential  to  be  an  effective  tool  for  improving  students’ 
learning  experiences.  We  look  forward  to  furthering  its  development  and  making  other  SASK  Mentors  for 
other  learning  environments  and  domains. 
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Abstract:  This  conceptual  essay  examines  theoretical  and  methodological  problems  associated  with  teaching 
the  Social  Foundations  of  Education  course  and  provides  a framework  on  how  to  overcome  these  difficulties. 
Foundations  of  Education  courses  are  notable  for  both  their  volume  and  the  ambivalence  with  which  students 
and  instructors  regard  their  role  in  preparation  for  teaching.  The  ambivalence  is  held  because  foundations 
courses  generally  suffer  from  instructors  unknowingly  devising  incompatible  goals  for  the  course  while  the 
multidisciplinary  nature  of  the  course  frequently  leads  to  intellectual  incoherence.  One  way  to  overcome 
these  deficiencies  is  to  have  students  acquire  an  “ironic”  understanding  using  the  case  method.  Case  studies 
that  use  digital  technology  make  this  approach  even  more  powerful  because  digital  technology  provides 
students  with  a cognitive  tool  that  encourages  the  profound  knowledge  necessary  to  attain  ironic 
understanding. 


This  conceptual  essay  examines  theoretical  and  methodological  problems  associated  with  teaching 
the  Social  Foundations  of  Education  course  and  provides  a framework  on  how  to  overcome  these 
difficulties.  A staple  of  teacher  education  since  the  1930's  (Cohen  1999;  Lagemann  2000),  Foundations  of 
Education  courses  are  notable  for  both  their  volume  and  the  ambivalence  with  which  students  and 
instructors  regard  their  role  in  preparation  for  teaching.  The  ambivalence  is  rightfully  held  by  both  because 
foundations  courses,  from  their  inception,  have  suffered  two  significant  shortcomings.  First,  instructors 
unknowingly  have  devised  incompatible  goals  for  the  course.  Second,  the  multidisciplinary  nature  of  the 
course  frequently  leads  to  intellectual  incoherence  for  all  concerned. 

As  a teacher  of  Foundations  courses,  I have  struggled  with  overcoming  these  embedded  course 
deficiencies.  Over  time,  I came  to  a realization  that  the  course  needed  a new  goal.  I borrowed  from  Kieran 
Egan  ( 1 997)  the  idea  of  having  students  acquire  an  “ironic”  understanding.  Due  to  the  multidimensional 
representation  possible  with  cases,  attaining  ironic  understanding  is  enhanced  when  case  methods  are 
employed  (Merseth,  1996).  Most  recently,  I have  begun  to  supplement  paper  copies  of  case  studies  with 
digital  technology  and  believe  that  the  enriched  learning  experience  provided  by  such  an  approach  makes 
the  case  study  method  even  more  powerful.  Digital  technology  provides  students  with  a cognitive  tool  that 
encourages  the  profound  knowledge  necessary  to  attain  ironic  understanding. 

Many  instructors  of  foundations  of  education  courses  have  grappled  with  ways  to  overcome  the 
incompatible  goals  and  the  intellectual  muddle  of  the  course.  An  effective  way  to  resolve  the  paradox  is  to 
create  a course  that  seeks  a new  goal  using  new  intellectual  tools.  Vygotsky  (1978)  argued  that  people 
make  sense  of  the  world  using  mediating  intellectual  tools.  Intellectual  development,  then,  is  dependent  not 
only  on  psychological  stages,  like  those  described  by  Piaget,  or  on  the  knowledge  we  accumulate,  but  also 
relies  on  the  intellectual  tools  that  society  makes  available.  While  Vygotsky  focused  on  oral  language,  or 
sign  systems,  in  his  formulation,  other  intellectual  tools  are  available.  Using  these  tools  in  innovative  ways 
can  refashion  the  social  foundations  of  education  course. 

For  me,  the  way  out  of  the  teaching  dilemma  presented  with  a course  that  has  at  its  core 
incompatible  goals  is  to  radically  change  the  student  outcome  desired.  Guided  by  the  insights  of  Kieran 
Egan  (1997)  on  the  larger  purposes  of  schooling,^  I seek  to  assist  students  in  acquiring  an  ironic 
understanding  of  schools  and  teaching.  Briefly,  irony,  as  we  know  it  today,  is  tied  to  a belief  that  the  truth 
long  sought  in  the  western  intellectual  tradition  is  illusionary.  In  place  of  certainty,  irony  provides  for  the 
centrality  of  reflexiveness  in  thinking.  Ironic  understanding,  then,  leads  to  the  removal  of  a commitment  to 


simple  truth  of  general  schemes.  The  sophisticated  ironist,  instead,  enjoys  extensive  ways  of  understanding 
and  recognizes  the  value  of  a varied  spectrum  of  perspectives. 

Fostering  multiple  perspectives  is  at  the  heart  of  the  articulate  ironist.  Gliding  from  perspective  to 
perspective  is  valued  and  becomes  second  nature  in  such  a person.  The  ironist  applies  the  varying 
perspectives  to  make  sense  of  their  experiences.  A teacher  might  ironically  note,  for  example,  that  a new 
school  policy  designed  to  improve  math  performance  in  fact  creates  a wider  gap  between  high  math 
achievers  and  low  achieving  math  students.  A better  identification  of  the  problem  and  a clearer  solution  is 
then  possible.  One  must  take  care,  ironically  enough,  to  make  sure  that  the  disciplinary  incoherence  of  the 
social  foundations  course  is  not  replaced  with  incoherence  arising  from  seeing  events  with  different 
perspectives.  The  value  of  a case  study  approach  to  teacher  education  is  that  while  cases  offer  multiple 
perspectives,  they  are  at  the  same  time  grounded  in  a specific  narrative  context.  General  theoretical 
principles  flow  from  an  array  of  possibilities  that  build  on  one  another.  Out  of  what  seems  to  be  chaos 
comes  not  only  insight  but  also  orderliness. 

Emerging  digital  technology  is  allowing  the  case  study  method  to  develop  into  an  even  more 
powerful  tool  by  permitting  cases  to  become  more  contextually  grounded.  More  specifically,  an  interactive 
course  web  site  that  supplements  the  text  of  a case  with  multimedia  presentations  and  links  to  other  data 
sources  allows  great  flexibility  in  student  construction  of  knowledge.  To  illustrate  this,  I shall  borrow  a 
metaphor  from  the  world  of  art  museums.  Elizabeth  Valance  (1995)  refers  to  a “public  curriculum” 
inherent  in  the  orderly  images  found  in  art  museums.  This  public  curriculum  consists  of  the  images 
displayed;  the  order  in  which  the  images  are  placed;  the  information  provided  for  visitors;  the  tours, 
lectures  and  workshops  and  other  programs  offered  to  the  public;  and  the  publications  produced.  The 
curriculum  is  randomly  accessed  by  visitors  and  is  readily  available  to  anyone  who  comes  to  the  museum. 
All  visitors,  however,  study”  the  curriculum  on  their  own  and  make  sense  of  the  images  in  endless  varieties 
of  ways.  While  visitors  may  understand  the  curriculum  in  a variety  of  ways,  the  curriculum  is  deliberate. 
Images  are  arranged  in  some  order  (usually  by  culture  chronology  but  such  need  not  be  case)  with  or 
without  helpful  labels.  Some  images  are  quite  accessible  to  all  but  others  are  difficult,  and  visitors  may 
even  view  an  image  with  hostility.  Visitors  will  frequently  confront  a museum  worker  with  some  sort  of 
refrain  along  the  lines  of,  “Is  that  really  art?” 

Similarly,  a well-designed  course  website  will  be  accessible  and  challenging.  Depending  on  how 
images,  text,  video  and  audio  are  arranged,  one  can  create  different  learning  environments  and  attempt  to 
frame  relevant  issues  in  key  ways.  Arranging  course  materials  using  digital  technology  into  generative 
topics  allows  for  students  to  respond  to  the  realities  of  classroom  practice  and  educational  policy-making  in 
multiple  ways.  What  cannot  be  controlled,  of  course,  is  what  students  learn.  The  goal,  instead,  is  to  provide 
as  many  avenues  as  possible  to  allow  students  to  make  their  own  connections  and  to  form  their  own 
categories.  Like  a well  run  art  museum  that  helps  novice  visitors  find  accessible  entry  points  into  the  world 
of  art,  an  interactive  website  can  assist  the  novice  teacher  into  gaining  access  to  the  profession  of  teaching. 
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Abstract:  While  Java  based  applets  have  become  commonplace  in  web  based  training,  their 
development  is  still  costly.  We  present  a framework  that  uses  the  JavaBean  component  model 
in  a layered  approach  using  several  abstraction  levels  to  facilitate  the  reuse  of  development 
infrastructure.  A reference  implementation  was  successfully  employed  in  teaching  audio  signal- 
processing to  computer  science  students. 


Introduction 

Interactive  exhibits  constitute  one  of  the  major  advantages  of  computer-based  instruction.  While  dedicated 
authoring  packages  allow  for  high-quality  animation,  non-trivial  domain  specific  functionality  is  hard  to 
implement.  On  the  other  hand,  all-purpose  programming  languages  make  way  for  sophisticated  domain  specific 
functionality  at  the  price  of  considerable  overhead  for  animation.  Platform  independence,  built-in  capabilities  for 
audio-visual  output,  and  its  enormous  installation  base  has  made  Java  (applets)  increasingly  popular  in  the 
educational  field.  Unfortunately,  the  absence  of  software  frameworks  hindered  code  reuse;  missing  configuration 
options  restricted  applets  to  only  one  context.  A need  has  emerged  for  a software  framework,  allowing  reuse  of 
interactive  educational  assets  on  a component  level. 

Christian  (2000)  successfully  developed  a set  of  generic  physics  related  applets,  named  Physlets,  that  can  be 
scripted  and  customized  by  the  educator  using  JavaScript.  While  this  considerably  broadened  the  scope  of 
possible  uses,  these  applets  are  not  intended  to  be  used  e.g.  in  conjunction  with  third-party  APIs.  Integrating 
software  components  into  more  complex  projects  requires  well-defined  interfaces  between  these  components. 
Sun  Microsystems  introduced  therefore  the  JavaBeans  specification  (1997).  Praehofer  (1998,  2000)  and  Benes 
(1999)  presented  two  JavaBeans  based  frameworks  helping  to  teach  simulation  based  on  the  discrete  event 
system  specification  formalism.  In  the  field  of  image  processing,  Hanisch  (1999)  demonstrated  an  efficient 
approach  to  assemble  applets  by  wrapping  the  Java  Advanced  Imaging  API  into  JavaBean  components. 

A multi-layered,  component  based  framework 

Although  JavaBeans  have  proved  to  be  well  suited  for  the  creation  of  educational  exhibits,  there  has  been  little 
reuse  and  exchange  between  different  content  authors.  This  astonishes,  because  the  promise  of  reusability  is  one 
of  the  major  drivers  of  component-based  development.  We  identify  missing  access  to  only  partial  functionality 
of  the  existing  bean-based  exhibits  as  the  primary  reason  for  this;  While  they  are  built  from  components,  they 
behave  as  monolithic  entities.  Therefore  we  introduce  a framework,  named  EduBeans,  that  (1)  employs  the 
JavaBeans  component  model  and  (2)  introduces  a new  abstraction  layer  hierarchy  into  the  applet  creation 
process  and  (3)  provide  a reference  implementation  of  a web-enabled  bean  builder  tool,  the  EduBeanBuilder. 

JavaBeans  component  model 

JavaBeans*^^  is  the  component  model  for  the  Java  language  endorsed  by  Sun  Microsystems.  Any  Java  class  can 
act  as  JavaBean  as  long  as  the  code  follows  a set  of  design  patterns  that  expose  semantic  information  of  the 
component’s  properties.  This  information  can  be  extracted  by  application  development  tools,  referred  as  bean 
builders  and  displayed  in  a graphical  user  interface  (GUI).  Bean  builders  call  the  same  methods  to  modify  the 
internal  state  of  a bean  as  a programmer  would  do  in  her  source  code.  Therefore,  a prospective  user  can  familia- 
rize herself  with  the  properties  of  a certain  bean  using  a GUI  before  employing  the  bean  programmatically. 

Abstraction  layer  hierarchy 

Although  there  have  been  successful  attempts  to  create  JavaBean  based  educational  software  (see  above),  reuse 
and  adaptation  to  other  domains  had  to  take  place  in  a rather  binary  fashion:  Either  adopting  the  complete  infra- 
structure or  rewriting  it  from  scratch.  Differentiating  several  layers  of  abstraction  in  the  building  process 
facilitates  a partial  reuse  of  the  code  base.  With  increasing  concreteness  from  level  1 to  5,  these  layers  address 
different  target  groups:  level  5-4  are  geared  towards  the  learner,  level  5-3  towards  the  subject  domain  expert 
(teacher),  level  2-1  towards  the  experienced  Java  framework  developer.  Table  1 depicts  the  layer  hierarchy. 


Table  1 abstraction  layer  hierarchy  used  in  framework 

Abstraction  layer  Support  offered  by  framework  Example 
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5 instance  configuration  images,  specific  experiment  settings  etc. 


4 application  assembly 


domain  specific  beans  can  be  combined  to  form 
larger  applications  or  “macro”  beans 
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2 interaction  paradigm  Paradigm  specific  ^mmunicatiotr  mechanisms 
are  provided  (e.g.  data  push  / pull  in  filter  chains) 

DEVS  network 

filter  chain 

1 component  interface 


components  can  be  employed  in  web-based  bean 
builder  tool  or  JavaBean  compliant  IDE 


JavaSean 


The  most  abstract  component  interface  layer  (level  1)  purely  requires  that  JavaBean  compliant  classes  are  used. 
This  important  assertion  allows  the  use  of  components  in  IDEs  and  web-enabled  bean  builders.  The  interaction 
paradigm  layer  (level  2)  provides  support  for  typical  interaction  mechanisms  from  the  subject  domain.  Our 
reference  implementation  uses  a source-filter-sink  paradigm.  Other  paradigms  include  structured  algorithms  or 
DEVS  networks.  The  subject  domain  layer  (level  3)  contains  domain  specific  beans.  Authors  can  create  sets  of 
related  beans,  leveraging  the  interaction  paradigms  provided  by  layer  2.  The  application  assembly  layer  (level  4) 
determines  the  connections  between  the  beans  from  the  subject  domain  into  larger  applications  or  “macro” 
beans.  Usually  the  author  will  assemble  these  applications  as  preset  demonstrations,  but  this  can  also  be 
delegated  to  the  learners  as  an  exercise  task.  Finally,  the  instance  configuration  layer  (level  5)  configures 
applications  by  loading  set-ups  for  specific  experiments. 

EduBeanBuilder 

Most  Java  IDEs  can  load  and  display  JavaBeans.  But  there  are  also  situations  in  which  detailed  information  on 
the  inner  workings  of  a bean  is  not  necessary  or  even  disturbing.  We  therefore  extended  a bean  builder  provided 
by  Sun  Microsystems  Inc.  into  the  web-enabled  EduBeanBuilder.  It  allows  launching  pre-configured 
applications  consisting  of  multiple  beans  in  a web  browser  as  well  as  providing  an  empty  workbench  onto  which 
domain  specific  beans  can  be  placed  and  “wired”.  The  state  of  the  applications  can  be  serialized  easily  into  an 
XML  stream  (Java  Community  Process,  2001),  which  is  exploitable  in  educational  settings. 

Case  study 

Computer  science  students  had  to  be  trained  to  participate  in  the  development  of  a Java  audio  signal-processing 
package.  This  software  applies  several  cascaded  filters  on  an  audio  input  signal.  Because  this  approach  maps  to 
the  source-filter-sink  paradigm  provided  in  level  2 of  our  reference  implementation,  extending  the  existing  code 
base  to  make  it  JavaBeans  compliant  allows  the  integration  of  the  actual  signal  processing  code  into  instructional 
applets.  This  avoids  re-implementing  functionality  in  “toy”  classes  solely  for  educational  purposes.  The  students’ 
feedback  was  very  positive,  especially  because  the  chosen  approach  helped  the  students  to  bridge  the  gap 
between  learning  abstract  concepts  and  applying  implementation-level  programming  skills. 

Conclusions 

The  presented  EduBeans  framework  proved  very  useful  in  the  development  of  interactive  instructional  applets. 
The  JavaBeans  component  model  allows  using  the  same  code  base  for  demonstration  purposes  as  well  as  “real 
world”  tasks.  At  the  time  of  writing  the  framework  offers  support  for  generic  JavaBeans  by  the  EduBeanBuilder 
on  level  1,  a source-filter-sink  paradigm  on  level  2,  audio  processing  functionality  on  level  3 and  includes 
various  assembly  and  configuration  examples  in  level  4 and  5.  The  next  development  is  oriented  to  extending  the 
package  with  other  problem  domains  and  completing  a comprehensive  documentation. 
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Abstract;  This  paper  is  a report  on  the  preliminary  approach  to  creating  a software  agent 
based  mentoring  system  for  women  in  the  Information  Technology  domains.  It  outlines  the 
data  collection  and  analysis  methods  and  the  proposed  approach  to  the  creation  of  the 
mentoring  software  agent. 


Introduction 


Mentoring  has  been  widely  used  in  educational  and  professional  settings  to  help  students 
become  part  of  a community  of  learners,  novices  adapt  to  new  work  environments,  and  professionals  advance 
their  careers,  however,  current  mentoring  techniques  are  limited  in  scope  (AEA,  2001;  Super,  1984).  This 
approach  has  been  frequently  used  to  encourage  women  and  minorities  to  enter  computer  science(CS)  and 
information  technology(IT)  degree  programs  and  the  workforce  (CMU,  2001).  The  current  approaches  use  other 
professionals,  professors,  advanced  students,  and  members  of  the  community  to  do  the  mentoring.  While  these 
mentoring  programs  have  been  effective  on  a small  scale  there  are  two  issues  with  them  (Lynem,  2001).  First, 
there  is  already  a shortage  of  women  and  minorities  in  the  CS  and  IT  fields,  those  few  people  can  be 
overburdened  with  the  need  for  mentors.  Second,  the  mentoring  depends  heavily  on  the  individual's  ability  to 
mentor  a newcomer  and  rarely  are  there  any  guidelines  or  tips  available  on  how  to  mentor. 


The  Approach 


In  order  to  create  a more  widely  available  mentoring  system  we  are  beginning  a unique  project 
to  create  an  Activity  Theory  analysis  of  two  IT  projects  that  will  be  used  to  create  a Software  Agent  based 
mentoring  system.  This  system  will  be  used  in  preparing  students  to  enter  IT  internship  programs  as  well  as 
entering  the  IT  workforce.  In  collaboration  with  two  major  software  companies  we  are  conducting  an  Activity 
Theory  analysis  of  the  design  environment  and  will  analyze  the  data  to  create  a framework  to  program  software 
agents.  These  agents  will  act  as  the  mentors  from  the  software  comp  anies  to  students  in  the  Information 
Sciences  and  Technology  program  and  help  them  learn  about  what  they  can  expect  when  they  go  on  an 
internship  or  join  the  workforce. 


Preliminary  Framework 

Activity  Theory  ( Engestron\  1991)  is  used  as  the  theoretical  framework  for  data  collection,  analysis,  and 
programming  of  the  software  agent.  It  conveys  the  complexity  of  real  world  activity  systems  in  manageable 
chunks.  The  data  representation  in  the  software  agent  will  be  done  using  multiple  techniques  including  video, 
images,  animated  agents,  and  text.  The  retrieval  of  the  data  will  be  using  production  rules  (Anderson  et  al. , 
1995),  and  Latent  Semantic  Analysis(  Landauer  & Dumais,  1997).  The  agents  will  have  a natural  language 
interface  (Person  et  al.,2001),  synthesized  speech  (Cowley  & Jones,  1992),  and  facial  expressions  (Link  et  al. , 
2001). 


Conclusions 


Software  agents  are  currently  being  used  in  many  areas  like  Physics  Tutoring,  Current 
applications  of  these  agents  include  Physics  tutoring  (Graesser  et  al.  1999;  Van  Lehn,  1999),  learner  feedback 
(Negroponte,  1997),  promoting  reflective  thinking  (Baylor,  1998),  teaching  biology  (Moreno,  2001),  and 
psychological  counseling  for  families  with  terminally  ill  children  (Johnson,  2001).  The  software  agent 
technologies  have  also  been  used  to  provide  workplace  simulation  (Work  Sim)  in  schools  (VanLehn,  2001).This 
approach  is  unique  in  the  data  collection  method  using  Activity  Theory  and  its  application  as  a mentoring 
environment. 
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Abstract:  Over  the  past  few  years,  institutions  of  higher  education  have  been  rolling  out  a variety  of  Web-based 
services  to  their  constituents.  Registration,  faculty  grading,  transcript  management,  general  ledger  inquiry, 
student  advisement,  and  other  critical  service  functions  are  available  online  to  students,  faculty,  administrators, 
and  alumni.  But  the  rollout  of  online  services  is  usually  evolutionary,  built  on  the  tools  and  technologies  available 
at  the  time.  Online  services  launched  prior  to  Section  508  must  now  be  re-evaluated  and  re-worked,  further 
stretching  an  institution’s  human  and  financial  resources.  This  paper  documents  how  an  information  management 
system  vendor  made  those  critical  service  functions  accessible  to  all  of  its  clients’  constituents  and  provides  a 
checklist  against  which  to  assess  any  information  management  system  vendor’s  Web  products. 


Introduction 

Section  508  is  a statutory  section  of  the  1973  Rehabilitation  Act  and  was  signed  into  law  by  President  Clinton  in  1998. 
Section  508  requires  that  electronic  and  information  technology  developed,  procured,  maintained,  or  used  by  Federal 
agencies  - including  Web  sites  - be  accessible  to  people  with  disabilities,  unless  an  undue  burden  would  be  imposed  on 
that  agency.  Section  508  was  enacted  to  enhance  the  ability  of  Federal  employees  with  disabilities  to  have  access  to  and 
use  of  information  that  is  comparable  to  that  provided  to  others.  Similarly,  by  procuring  accessible  electronic  and 
information  technology.  Federal  agencies  would  be  able  to  provide  members  of  the  general  public  with  disabilities  with 
information  that  is  comparable  to  that  provided  to  others.  What  constitutes  comparable  access  under  Section  508  is 
described  in  Subparts  B,  C,  and  D of  the  Access  Board  Standards  issued  in  December,  2000  by  the  Architectural  and 
Transportation  Barriers  Compliance  Board,  an  independent  Federal  agency  promoting  accessibility  for  individuals  with 
disabilities. 

The  scope  of  Section  508  is  limited  to  the  Federal  sector  and  does  not  apply  to  the  private  sector,  nor  does  it  generally 
impose  requirements  on  the  recipients  of  Federal  funds.  However,  states  receiving  assistance  under  the  Assistive 
Technology  Act  State  Grant  program  are  required  to  comply  with  Section  508.  Moreover,  many  institutions  of  higher 
education  are  committed  to  providing  access  to  all  their  constituents,  regardless  of  Federal  guidelines.  For  example,  in 
the  State  of  California,  the  commitment  to  accessibility  has  been  formalized  into  official  policy  to  provide  electronic 
access  to  the  community  colleges. 

Institutions  have  already  begun  the  task  of  checking  their  own  Web  sites  for  compliance.  Some  institutions  have  used 
Bobby ^ an  automated  accessibility  checker  offered  by  the  Center  for  Applied  Special  Technology  (CAST).  Bobby 
functions  like  a spelling  checker,  to  diagnose  accessibility  problems.  A user  can  type  the  address  of  any  Web  site,  and 
Bobby  will  respond  with  a review  of  the  site,  indicating  areas  of  concern  and  suggestions  for  improving  accessibility,  or 
else  awarding  the  site  with  a Bobby  Approved  badge  for  passing  its  test.  Bobby  rates  accessibility  based  upon  general 
requirements  it  looks  for  in  the  source  code,  such  as  alternate  text  for  all  images.  The  Bobby  tool  is  also  available  to 
download,  so  that  Web  authors  can  use  it  to  run  accessibility  tests  on  their  own  hard  drives  during  the  process  of  site 
design.  However,  Bobby  Approved  does  not  mean  Section  508  compliant.  The  Web  developer  must  still  implement  the 
proper  code  to  make  the  site  compliant  with  Section  508  requirements. 

Another  tool  is  InSight/InFocus^  developed  by  SBB  Technologies.  InSight  scans  an  institution’s  Web  site  and  flags 
areas  that  could  cause  problems  for  disabled  users,  and  InFocus  fixes  the  flagged  areas.  The  Equal  Access  to  Software 
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and  Information  (EASI)  initiative  also  assists  institutions  with  compliance  issues  through  workshops,  consulting 
services,  and  a resource-rich  Web  site. 

The  Challenge 

What  makes  compliance  checking  somewhat  painful  is  the  lack  of  a single,  agreed  upon  definition  of  compliance.  The 
Electronic  and  Information  Technology  Access  Advisory  Committee  (EITAAC)  standards  provide  sixteen  specific 
items  for  Web  accessibility.  The  Web  Content  Accessibility  Guidelines  1.0  produced  by  the  WWW  Consortium  (W3C) 
define  three  priority  levels,  ranging  from  compliance  of  a specific  piece  of  software  on  a specific  operating  system  with 
specific  assistive  technology  devices,  to  accessibility  on  a variety  of  applications,  platforms  and  assistive  devices. 

Web-based  academic  and  student  services  present  a particular  challenge  for  Section  508  compliance.  The  data  sources 
for  these  services  reside  in  the  institution’s  information  management  systems,  such  as  the  Student  Information  System, 
Financial  System,  Human  Resource  System,  and  the  Fund-Raising  System.  These  enterprise-wide  administrative 
application  software  systems  are  usually  licensed  from  a vendor/manufacturer,  with  system  maintenance,  upgrades,  and 
enhancements  provided  by  the  vendor.  Vendors  that  provide  pre-packaged  Web  workflows  for  academic  and  student 
services  are  responsible  for  testing  its  software  for  Section  508  compliance  and  making  the  necessary  program  changes. 
Because  institutions  pay  the  vendor  an  annual  fee  for  system  upgrades  and  maintenance,  it  is  the  job  of  that  vendor  to 
make  the  changes  required  to  comply  with  Section  508,  at  no  additional  cost  to  the  institution. 

A Vendor-Client  Partnership 

Working  in  partnership  with  four  client  institutions  in  the  forefront  of  universal  Web  accessibility  — Springfield 
Technical  College,  The  College  of  Southern  Maryland,  Dallas  County  Community  College  District,  and  Oakland 
Community  College  - Datatel,  Inc.,  a leading  provider  of  information  management  solutions  for  higher  education, 
launched  an  in-depth  assessment  and  redesign  project  to  produce  WebAdvisor  2.0,  a robust  Web -based  self-service 
solution  that  meets  strict  Section  508  compliance  standards.  Different  institutions  have  different  system  configurations. 
For  that  reason,  the  project  required  testing  on  a broad  and  complex  variety  of  configuration  combinations.  Testing  was 
performed  on  all  platforms:  Windows  95,  98,  ME,  NT,  2000,  XP,  and  MAC.  Browser  testing  was  conducted  on  Internet 
Explore  4,  5,  5.5,  and  6,  as  well  as  Netscape  4,  4.7,  6,  6.2,  and  on  Opera,  Lynx,  and  Mosaic.  Various  computer  brands 
and  types  were  tested,  as  were  various  HTML  versions.  Assistive  devices,  including  JAWS,  ZoomText,  Window  Eyes 
and  other  screen  readers,  were  used  in  the  testing  process.  The  objective  was  to  take  into  account  various  disabilities, 
such  as  blindness,  low  vision,  cognitive,  and  physical  disabilities. 

Some  of  the  technical  changes  made  to  the  Web  product  were: 

• The  addition  of  ALT  tags  to  images  and  image  maps,  enabling  screen  readers  to  “read”  the  images  in  words 

• Skip  links,  enabling  the  visually  challenged  to  skip  the  navigational  bar 

• Clear  and  unique  field  labels 

• Style  sheets  that  follow  the  Section  508  HTML  templates 

• Shortcut  keys 

• An  accessibility  description  page 

• Tags  for  combination  and  Submit  buttons 

• Linear  tables  for  ease  of  use 

In  a statement  of  accessibility  compliance  published  in  July  2001,  Nu  Visions  Unlimited,  Inc.,  an  independent  testing 
and  auditing  firm  certified  WebAdvisor  2.0  as  compliant  with  the  World  Wide  Web  Consortium  Level  A Accessibility 
Guidelines  and  Section  508. 

Partnering  with  Your  Vendor 

When  working  with  your  vendor,  an  excellent  starting  point  is  the  16 -bullet  checklist  provided  by  the  Access  Board. 
The  checklist  covers  layout,  look  and  feel,  navigation,  the  use  of  color,  and  other  criteria  set  down  in  the  Section  508 
guidelines.  Once  compliance  testing  of  the  Web  product  has  been  completed  and  the  areas  for  improvement  identified,  a 
development  plan  with  major  milestones,  accessibility  testing  and  verification,  along  with  clearly  defined  roles  and 
responsibilities  for  both  client  and  vendor,  should  be  crafted.  The  Web  product  should  integrate  seamlessly  into  the 


institution’s  overall  Web  presence.  However,  it  should  be  flexible  enough  to  enable  some  colleges  within  the  institution 
- law,  medicine,  etc.  - to  have  their  own  look  and  feel,  if  desired. 

Thanks  to  a solid  vendor-client  partnership,  WebAdvisor  2.0’s  compliance  with  Section  508  was  a key  milestone  for 
Datatel,  because  it  has  a far-reaching  impact  on  the  ability  of  Datatel’s  clients  to  serve  all  of  their  constituents,  including 
those  with  disabilities.  In  making  Web-based  academic  and  student  services  Section  508  compliant,  the  vendor  does 
most  of  the  work.  But  the  success  of  the  undertaking  is  the  job  of  the  vendor  and  the  client  institution  working  together. 


Reference  Web  Sites 

http://www. section508.gov 

http://www.cast.org/udl/index.cfm7Hl  1 

http://www.w3.org/WAI/ 

http://www.w3.org/tr/wai-webcontent-techs/ 

http://www.w3.org/wai/wcagl-conformance.html 

http://www.cast.org/bobbv/ 

http://aware.hwg.org 

http://www.rit.edu/~easi 

http://www.nuvisions.com 
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Abstract:  An  important  learning  goal  of  a molecular  biology  curriculum  is  a certain  proficiency  level  in 
experimental  design.  Currently  students  are  confronted  with  experimental  approaches  in  textbooks,  in 
lectures  and  in  the  laboratory.  However  most  students  do  not  reach  a satisfactory  level  of  competence  in 
the  design  of  experimental  approaches.  This  paper  describes  the  development  of  a web-based  application, 
which  supports  the  learning  of  this  design  skill.  The  application  consists  of  an  activating  part  and  a library 
part.  In  the  activating  part,  the  student  is  presented  with  a biological  question  that  must  be  solved 
experimentally.  Therefore,  the  student  has  to  make  a set  of  coherent  choices,  execute  steps  in  an 
experiment  and  interpret  the  experimental  results.  Furthermore,  a DNA  sequence  has  to  be  analyzed  with 
web-based  databases.  The  library  consists  of  learning  objects  that  present  essential  background 
information.  A test  with  a small  group  of  students  yielded  very  promising  results. 


Introduction 

The  Food  and  Biotechnology  (FBT)  program  aims  at  the  creation  of  a rich  body  of  digital  learning  material 
for  university  curricula  related  to  food  science  and  biotechnology.  The  FBT  program  was  initiated  at 
Wageningen  University  in  September  2000.  The  program  focuses  on  web  based  learning  support  for  those 
learning  goals  where  digital  learning  material  is  expected  to  have  a clear  added  value.  One  of  the  FBT 
projects  aims  at  the  development  of  digital  learning  material  for  molecular  biology.  This  paper  describes  the 
first  stage  of  this  project. 

One  of  the  important  learning  goals  of  a molecular  biology  curriculum  is  a certain  proficiency  level  in 
designing  an  experimental  approach.  This  involves  the  application  of  different  techniques.  Students  are 
usually  capable  of  understanding  how  these  different  techniques  work,  but  they  have  difficulty  in  combining 
them  in  a useful  way  and  in  judging  whether  they  are  suitable  to  find  an  answer  to  a particular  question. 
Moreover,  students  often  do  not  realize  that  experiments  are  performed  with  biological  systems. 

Consequently  they  do  not  use  their  knowledge  of  biology  when  designing  experimental  approaches,  even 
though  this  is  essential  for  the  design  of  a useful  approach.  This  lack  of  applying  biological  knowledge  can 
also  be  observed  during  the  analysis  of  experimental  outcomes. 

The  indicated  problems  in  experimental  design  may  be  inherent  in  the  current  educational  setting.  Currently 
students  are  confronted  with  experimental  approaches  in  textbooks,  in  lectures  and  in  laboratory  courses. 
Each  of  these  formats  has  its  own  drawbacks.  Textbooks  do  describe  many  experiments  and  approaches,  but 
this  is  not  sufficient  for  students  to  learn  to  choose  techniques  and  to  schedule  operations.  One  of  the 
problems  is  that  students  usually  focus  on  the  mechanisms  behind  techniques,  thereby  losing  sight  of  the  uses 
of  such  techniques.  In  a lecture,  more  weight  can  be  put  on  the  actual  designing  of  experimental  approaches. 
It  is  however  very  hard  to  involve  more  than  a limited  number  of  students  individually  in  such  a way. 
Furthermore,  the  students  that  do  get  involved,  can  hardly  get  personal  feedback  due  to  considerations  the 
lecturer  has  to  take  of  the  other  students  as  well  as  the  time  allotted  for  the  lecture.  In  a laboratory  course 
finally,  students  have  little  freedom  in  choosing  and  scheduling  operations.  Moreover,  they  become 
preoccupied  with  the  practical  skills  they  still  lack,  such  that  the  overview  of  even  the  specific  experimental 
approach  is  lost.  Thus  the  teaching  of  more  general  aspects  ofdesigning  experiments  is  practically 
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impossible.  The  possibilities  of  computer  based  learning  support  may  offer  a solution  to  the  above  problem. 
Apart  from  the  well  known  argument  that  computer  based  learning  support  makes  it  possible  to  activate  each 
student  individually  and  generate  personal  feedback,  many  of  the  experimental  results  in  molecular  biology 
experiments  are  photographs  or  sequences  and  can  thus  be  represented  digitally.  Moreover,  the  processing  of 
results  in  molecular  biology  experiments  requires  computers  and  web  access.  Recently,  many  DNA  and 
protein  sequences  have  become  available  in  web-based  databases.  In  current  molecular  biology  research, 
using  these  databases  becomes  increasingly  important  for  the  design  of  experimental  approaches  and  the 
interpretation  of  experimental  outcomes.  Therefore,  learning  to  use  information  from  databases  was  added  as 
new  learning  goal  to  an  undergraduate  course.  For  an  effective  use  of  the  available  data,  database  searches 
have  to  be  performed  and  the  data  have  to  be  integrated  with  biological  knowledge  and  knowledge  about 
molecular  biology  techniques. 

Thus  web-based  learning  support  should  improve  the  following  skills: 

- designing  a basic  experimental  approach  by  selecting  and  combining  suitable  techniques; 

- performing  a database  search; 

- integrating  information  from  biology,  techniques  and  database  searches. 

These  skills  can  only  be  improved  when  sufficient  background  knowledge  can  be  used.  Knowledge  that  the 
students  have  not  mastered  yet,  has  to  be  pres  ented  as  well. 

The  material  has  to  complement  the  lectures.  As  the  lectures  may  change  from  time  to  time,  the  material 
should  consist  of  modules  that  can  be  combined  flexibly.  It  should  also  be  possible  to  use  these  modules 
independently  from  the  lectures . 

In  this  paper  we  describe  the  development  of  the  first  module  including  the  results  of  a first  evaluation.  The 
paper  finishes  with  a discussion  of  the  methods  we  applied  to  teach  the  skills  mentioned  above. 


The  Global  Structure  of  the  Site 

To  offer  students  the  opportunity  to  practice  the  necessary  skills,  they  are  offered  a case  (see  demo  site)  in 
which  they  have  to  design  an  experimental  approach  to  solve  a real  (but  basic)  biological  problem.  Thereby 
the  theory  is  placed  in  the  proper  context.  This  may  be  favorable  for  retrieving  the  theory  from  memory  in  a 
similar  context  (Anderson  2000),  but  this  also  makes  its  relevance  more  apparent,  which  may  motivate 
students  (Keller  1987).  The  background  information  that  is  needed  to  successfully  go  through  the  case  is 
available  to  the  students  just  when  they  need  it.  This  just-in-time  information  presentation  is,  among  others, 
recommended  by  the  Four  Component  Instructional  Design  model  (van  Merrienboer  1 997),  which  gives 
guidelines  for  teaching  complex  cognit  ive  skills.  Some  background  information  may  be  needed  several  times, 
when  going  through  the  case.  To  be  able  to  offer  this  information  from  one  place,  we  constructed  a library 
(see  demo  site)  that  contains  all  necessary  background  information.  This  library  consists  of  independent  self- 
explanatory  learning  objects.  Because  of  this  setup,  it  is  possible  to  study  only  a selection  of  the  learning 
objects  and  no  specific  study  order  is  required.  In  case  a student  prefers  this,  he  could  also  study  all 
background  information  before  starting  with  the  case.  The  library  also  ensures  that  students  with  varying 
amounts  of  prior  knowledge  can  in  principle  complete  the  case. 


The  Light  Induction  Case 

In  the  light  induction  case  the  student  has  to  isolate  a gene  that  is  induced  in  plants  upon  light  exposure  and 
analyze  its  DNA  sequence.  The  student  is  guided  through  the  case  by  multiple-choice  questions  to  prevent 
him  from  getting  lost  and  frustrated.  Sometimes  a choice  between  different  techniques  is  offered.  In  this  way 
the  student  is  stimulated  to  actively  think  about  the  possibilities  of  the  techniques,  which  is  essential  for  the 
design  of  an  experimental  approach. 

After  choosing  a technique,  the  student  is  immediately  confronted  with  the  experimental  result.  This  res  ult 
has  to  be  interpreted  in  order  to  find  out  whether  applying  the  technique  was  indeed  useful.  Thereby  it  is 
essential  to  take  biological  aspects  into  account.  This  is  illustrated  by  the  screen  dump  from  the  demo  site 
shown  in  figure  1.  This  shows  the  screen  the  student  sees  after  choosing  to  analyze  differences  in  mRNA 
concentrations  on  an  RNA  gel  on  which  total  RNA  is  loaded.  This  choice  has  led  to  a useless  result  because 
mRNA  cannot  be  visualized  with  this  method.  This  is  partly  due  to  the  fact  that  mRNA,  the  RNA  that  needs 


2102 


to  be  studied,  forms  only  a minor  fraction  of  total  RNA.  Thus,  it  is  necessary  to  use  knowledge  about  biology 
to  decide  whether  the  technique  is  useful.  The  student  will  discover  that  the  technique  was  not  useful  after 
selecting  a band  for  analysis. 

This  format  in  which  students  are  confronted  with  a result  that  they  have  to  interpret  has  several  advantages. 
Firstly,  students  probably  remember  better  whether  a technique  is  useful  in  a certain  context  when  they 
discovered  this  themsdves  than  when  this  would  have  been  told.  Secondly,  if  students  do  not  use  their 
biological  knowledge,  they  are  confronted  with  the  consequence.  This  probably  makes  a stronger  impression 
than  when  this  is  pointed  out  during  a lecture.  Thirdly,  a picture  of  an  experimental  result  contains  implicitly 
much  information  about  the  precise  use  of  a technique.  By  interpreting  the  results,  students  are  stimulated  to 
focus  on  and  give  meaning  to  these  results.  These  include  results  of  techniques  that  were  not  useful  and  that 
are  usually  not  shown  in  a textbook.  As  people  tend  to  have  good  memory  for  meaningful  interpretations  of 
an  image  (Anderson  2000)  and  easily  make  inferences  from  them  (Larkin  et  al.  1987),  the  students  are  in  this 
way  again  stimulated  to  remember  the  precise  use  of  a technique. 

Sometimes  it  is  also  necessary  to  interpret  the  result  in  order  to  continue  with  the  procedure. 

The  gene  can  eventually  be  obtained  by  applying  a method  that  is  called  “differential  screening”.  This 
method  was  chosen  for  several  reasons.  Firstly,  the  problem  is  an  example  of  a very  common  research 
question  in  molecular  biology.  In  many  instances,  genes  that  are  specifically  induced  under  certain  conditions 
have  to  be  identified.  Even  though  differential  screening  is  a relatively  old  method  it  was  chosen  because  it 
clearly  illustrates  the  problems  involved.  Moreover,  more  advanced  methods  are  still  largely  based  upon  the 
same  principles.  Therefore,  understanding  the  differential  screening  method  may  facilitate  the  understanding 
and  proper  application  of  technologically  more  advanced  methods,  based  on  an  analogy  process  (Anderson 
2000). 
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Figure  1.  Screen  dump  of  a screen  a student  see  after  selecting  a technique.  To  be  able  to  interpret  this  result, 
the  student  needs  to  use  biological  knowledge.  Only  after  interpreting  the  result  correctly,  it  becomes  clear 
that  this  technique  was  not  useful  in  this  context. 
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After  isolating  the  desired  gene,  the  DNA  sequence  of  the  gene  has  to  be  analyzed  (see  sequence  analysis 
section  at  the  demo  site).  The  first  sequence  to  be  analyzed  is  not  complete,  as  is  usually  the  case  in  practice. 
Another  reason  why  this  partial  sequence  is  offered  is  that  the  student  has  to  discover  for  himself  that  it  is  not 
complete.  Therefore,  experimental  findings  have  to  be  combined  with  knowledge  from  biology.  The  student 
has  to  perform  an  additional  experiment  to  obtain  the  complete  sequence.  This  sequence  has  to  be  analyzed 
by  performing  a database  search.  To  perform  this  search  it  is  again  necessary  to  actively  use  biology 
knowledge  as  well  as  knowledge  about  the  applied  experimental  techniques. 

The  interactive  part  of  the  case  is  followed  by  a summary.  This  summary  contains  overview  pictures  as  well 
as  information  about  why  applying  a technique  in  the  given  context  was  useful  or  not.  The  summary  contains 
in  principle  all  necessary  theoretical  information.  Thus,  the  interactive  part  should  support  the  training  of 
extra  skills,  whereas  the  theory  can  be  found  in  the  summary. 

Finally,  a number  of  multiple-choice  questions  are  implemented  which  serve  as  a self-test. 


The  Library  ' 

The  library  contains  background  information  that  is  necessary  to  go  through  the  case,  so  that  the  material  can 
in  principle  be  used  by  students  with  varying  amounts  of  prior  knowledge.  The  library  contains  information 
about  techniques,  database  searches  and  processes  that  take  place  inside  cells.  As  mentioned  before  the 
library  contains  independent  and  self-explanatory  learning  objects. 

The  explanation  of  a technique  consists  of  2 to  6 movies,  because  of  our  positive  previous  experiences  with 
these  types  of  movies.  Figure  2 gives  a screen  dump  (see  demo  site)  of  the  introduction  page  for  the 
technique,  which  is  labeled  DNA  gel.  The  page  contains  pictures  that  give  an  impression  of  the  content  of  the 
movie.  The  student  can  click  on  a picture  in  order  to  start  a movie.  The  movies  do  not  take  more  than  2 
minutes.  They  consist  of  spoken  text  supported  by  photographs,  annotations  and  animations.  As  stated  before, 
it  is  essential  for  designing  experimental  approaches  to  know  the  different  purposes  of  a technique.  Therefore, 
this  is  stressed  in  the  first  movie.  The  second  movie  explains  the  principle  of  a technique. 


DNA  Gel 


► To  Case 


0 Overview 


Films 


Film  Text 


^ ^ Techniques 
h ► Digestion 
h©  DNA  Gel 
h ► RNA  Gel 
h ► Southern  Blot 
Northern  Blot 
!“  ► cDNA  Library 
. ^ * Clone  Hybridization 
Biology  background 
^ ^ Database  searches 


2.  Principle 


6.  Electroporesis 


Questions 


* Questions 


3.  The  gel 

V I 


Figure  2.  Typical  introductory  screen  for  a technique  in  the  library.  Pictures  give  an  impression  of  the 
content  of  the  movies.  A student  can  click  on  a picture  to  start  a movie.  Questions  are  available  as  well. 
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The  explanation  of  some  techniques  also  highlights  specific  steps  of  the  technique  to  give  the  student  an  idea 
of  its  complexity  and  scale.  A movie  showing  a typical  experimental  result  is  sometimes  added  as  well,  which 
makes  it  easier  for  the  student  to  interpret  results  himself  later  on.  The  written  text  and  the  still  images  of  each 
movie  are  available.  Extensive  experience  with  the  use  of  movies  at  Wageningen  University  has  shown  that 
some  students  prefer  to  view  the  text  when  the  pace  of  the  movie  is  too  slow  for  them.  The  written  text  is 
also  better  for  students  who  just  want  to  scan  through  the  theory  quickly  (Hartog  et  al  2000),  for  students 
whose  mastery  of  the  English  language  is  insufficient  to  understand  the  audio  and  in  situations  when  there  is 
no  sound  available.  Beside  the  movies,  each  learning  object  contains  a couple  of  short  questions  to  let  the 
student  internalize  the  newly  acquired  concepts.  In  case  of  similar  techniques,  movies  that  explain  identical 
steps,  are  actually  identical.  If  applicable,  this  is  clearly  indicated. 

The  learning  object  on  the  database  search  contains  some  text  to  explain  the  background  and  a simple 
simulation.  In  this  simulation  a search  can  be  performed  in  controlled  circumstances.  When  the  student 
follows  the  instructions  given  on  top  of  a page,  he  stays  within  the  simulation  and  gets  new  instructions.  It  is 
however  also  possible  to  leave  the  simulation  and  find  out  what  happens  when  another  strategy  is  followed. 
This  learning  object  also  contains  some  multiple-choice  questions. 

The  learning  objects  dealing  with  the  biology  background  contain  schematic  drawings,  tables  etc.  in  which 
the  most  important  information  is  summarized.  These  objects  serve  to  refresh  the  student’s  memory  and  are 
not  meant  to  teach  them  new  information. 


Evaluation  Results 

So  far  the  material  was  evaluated  with  a group  of  6 volunteers.  This  evaluation  was  carried  out  to  identify 
ways  to  improve  the  site.  By  testing  the  site  with  a small  number  of  users,  most  usability  problems  should 
become  clear  (Nielsen,  2000).  The  students  first  had  to  give  an  indication  of  their  prior  experiences  and  they 
had  to  make  a short  test.  Next,  the  students  worked  in  pairs  through  the  site.  Finally  they  performed  a post- 
test that  was  similar  to  the  first  one  and  completed  an  evaluation  form.  The  students  judged  the  site  on 
average  with  an  8.7  on  a scale  of  1 to  10.  Thus,  the  students  were  very  enthusiastic.  Reactions  on  the 
evaluation  forms  include: 

- “The  overall  impression  is  very  positive.  By  performing  a virtual  experiment  you  become  able  to  understand 

techniques,  but  more  important,  learn  to  combine  different  techniques  to  come  to  a good  result.” 

- “I  think  it  is  a good  idea  showing  some  films  because  this  helps  to  imagine  the  practical  work  and  not  only 

the  theoretical  explanation.  And  then,  the  possibility  of  listening  the  explanation  is  very  appropriate 
because,  in  this  way,  it  is  not  so  tiring  reading  the  whole  text  from  the  computer  screen.” 

The  evaluation  also  yielded  ideas  for  improvement.  Most  importantly,  students  had  problems  to  perform  the 
database  search.  Therefore,  the  above  mentioned  simulation  was  added  to  the  library.  Further  minor 
improvements  include  the  addition  of  a clarifying  figure,  the  reformulation  of  some  feedback  and  the  removal 
of  a number  of  spelling  mistakes. 

An  important  test  result  is,  that  each  student  applied  the  concept  of  differential  screening  sufficiently  well 
after  studying  the  material  whereas  only  one  student  already  managed  to  apply  this  concept  before  studying 
the  material.  Furthermore,  in  the  post  test  the  students  had  to  design  an  approach  to  solve  a (basic)  biological 
problem  that  was  different  from  the  problem  they  solved  at  the  site  and  that  required  a different  approach  to 
be  solved.  The  students  performed  better  in  the  second  test  than  in  the  first  test,  even  though  the  problem  was 
slightly  more  difficult.  Overall,  the  results  were  encouraging. 


Final  Remarks 

A web  based  application,  which  supports  learning  experimental  design  in  molecular  biology  has  been 
developed.  The  web  site  has  been  evaluated  by  a small  group  of  volunteers.  Currently  the  demo  site  is 
accessible  worldwide  for  review.  The  site  will  be  used  in  Wageningen  in  the  course  "gene  technology"  in 
spring  2002. 

The  ultimate  goal  will  be  that  students  can  design  experimental  approaches  in  new,  unfamiliar  situations.  The 
application  confronts  the  students  with  situations  and  experimental  results,  which  force  them  to  actively  use 
their  knowledge  of  biology  while  designing  a new  approach.  It  is  expected  that  applying  their  knowledge. 
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while  working  with  molecular  biology  techniques  will  eventually  become  habitual. 

The  web  site  teaches  to  design  an  experimental  approach  addressing  a new  question  in  molecular  biology  in 
three  ways.  Firstly,  the  differential  screening  method  that  students  use  in  the  case,  can  be  used  to  address 
similar  biological  questions.  Secondly,  the  differential  screening  method  can  also  serve  as  a schema  when 
designing  an  analogous,  technically  more  advanced  approach.  Finally,  the  application  forces  the  student  to 
focus  on  the  precise  use  of  each  technique  in  multiple  ways.  It  is  essential  that  the  student  becomes  very  much 
aware  of  what  the  use  of  a technique  exactly  is.  Thus,  it  should  become  easier  to  design  new  approaches  using 
the  same  techniques.  Moreover,  students  will  probably  gradually  learn  to  focus  more  on  the  use  of  a 
technique  when  studying  new  ones  in  the  future.  This  may  even  lead  to  a better  performance  for  designing 
approaches  that  consist  of  these  new  techniques  as  well. 

Currently,  several  additional  cases  are  being  developed,  covering  the  production  of  transgenic  organisms.  The 
research  is  progressing  towards  a qualitative  simulation  environment  that  will  present  the  consequences  of  a 
student’s  choice  in  terms  of  experimental  results  based  on  programmed  rules.  The  number  of  different  options 
will  be  almost  unlimited  in  the  perception  of  the  student.  On  the  one  hand  this  offers  the  students  a more 
realistic  situation  and  more  opportunities  to  test  their  own  ideas,  but  on  the  other  hand  students  could  easily 
flounder  and  show  unstructured  behavior  (de  Jong  et  al  1998).  The  main  challenge  for  the  future  will  thus  be 
to  embed  the  simulation  of  experiments  in  an  environment  that  guides  and  supports  the  students. 
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Courses 
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Abstract:  In  an  effort  to  engage  online  students  interactively,  distance-learning  guides 

emphasize  group -based  instructional  strategies  and  the  use  of  Internet  communication  tools  in 
online  instruction.  Although  some  of  these  guides  include  a general  discussion  of  group 
development  and  facilitation  strategies  (see,  for  example,  Palloff  & Pratt,  2001,  p.l25),  none 
provide  a concise,  step-by-step  approach  to  group  facilitation  for  instructors  who  may  be 
unfamiliar  with  these  strategies.  Using  Lacoursiere’s  model  of  group  development,  this  article 
will  present  an  overview  of  the  application  of  group  development  models  and  group 
facilitation  in  online  courses,  one  that  can  provide  instructional  designers  and  instructors  a 
practical  framework  for  maintaining  meaningful  interactivity  and  group  cohesion  in  online 
learning. 


Research  in  distance  learning  emphasizes  interactivity,  especially  leamer-to-leamer  and  leamer-to- 
instructor  interaction,  as  a key  to  engaging  learners  and  achieving  learning  objectives  in  online  courses  (Moore, 
1989;  Hillman,  Willis,  & Gunawardena,  1994;  Wagner,  1994).  Accordingly,  instructional  designers  and 
developers  incorporate  instructional  strategies  such  as  group  projects  into  course  design  and  encourage  the  use 
of  discussion  boards  and  e-mail  in  course  implementation.  But  these  provide  only  the  technological  opportunity 
to  interact;  they  do  not  necessarily  provide  the  motivation  to  do  so.  Group  facilitation  can  help  motivate 
students  to  interact  but  in  order  to  be  used  effectively  instructors  must  have  at  least  an  introductory  knowledge 
of  group  development.  Drawing  from  the  observations  and  models  of  group  development,  this  article  presents  a 
step-by-step  approach  for  the  application  of  group  facilitation  strategies  in  online  courses. 


Group  Development  and  Online  Learning 

Researchers  of  group  development  use  models  to  study  the  behavior  of  groups  and  maintain  that 
groups  are  usually  better  able  to  accomplish  a given  task  if  group  leaders  use  facilitation  skills  to  help  the  group 
establish  a sense  of  cohesion.  In  this  way,  understanding  group  behavior  provides  a framework  for  group 
leaders  so  they  can  learn  which  specific  interventions  may  help  foster  and  maintain  this  cohesion  throughout 
group  development.  Although  Tuckman’s  model  is  the  best  known  (1965),  Lacoursiere’s  group  development 
model  (1980)  may  be  more  helpful  for  instructors  and  designers  who  have  never  been  exposed  to  group 
development  because  it  demonstrates  the  relationship  of  morale  to  productivity,  suggesting  that  group 
cohesiveness  may  affect  the  achievement  of  learning  outcomes.  According  to  Lacoursiere’s  model,  there  are 
five  stages  of  group  development:  Orientation,  Dissatisfaction,  Resolution,  Production,  and  Termination,  with 
each  stage  characterized  by  measurements  of  morale  and  productivity.  Lacoursiere’s  developmental  stages  are 
presented  briefly  below,  along  with  suggested  online  facilitation  strategies  for  each  stage. 

In  the  Orientation  stage,  group  members  have  high  expectations  about  the  task  they  are  undertaking  but 
lack  the  requisite  skills  to  fulfill  the  task.  Membeis  experience  anxiety  over  the  purpose  of  the  group  and  group 
roles  and  are  highly  dependent  on  the  leader.  Overall,  orientation  is  marked  by  high  morale  but  low  group 
work.  During  this  stage,  the  instructor  should  initiate  low  risk  activities  such  as  icebreakers,  help  group 
members  gain  key  technical  and  content  skills,  and  clarify  group  roles  and  purpose.  It  is  important  that  the 
instructor  establish  communication  protocols  and  behavioral  norms.  Finally,  the  instructor  should  maintain  a 
high  presence,  use  a conversational  tone,  and  role  model  risk-taking  through  personal  disclosure  to  encourage 
students  to  do  likewise  (Cutler,  1 995). 

During  Dissatisfaction,  group  members  experience  conflict  between  their  initial  expectations  and  the 
reality  of  the  group  and  its  task.  Morale  drops  but  work  increases  as  group  members  attain  the  necessary  skills 
to  undertake  the  task.  If  conflict  is  mostly  low-level  frustration,  the  instructor  can  continue  to  help  clarify  group 
goals,  reminding  the  group  that  conflict  is  a natural  byproduct  of  the  process  that  can  be  productive  if  managed 
successfully.  If  conflict  is  highly  charged  and  personal,  the  instructor  should  contact  members  privately  and 


consider  more  formal  conflict  resolution  strategies.  Generally,  the  instructor  should  maintain  a high  presence, 
be  directive,  and  act  quickly  to  address  and  resolve  any  negativity.  Interventions  in  this  stage  are  highly 
variable  and  situational. 

In  the  Resolution  stage,  reality  is  accepted  and  reconciled  with  expectations  so  there  is  decreasing 
negativity.  Morale  and  work  both  increase.  The  instructor  can  begin  to  let  go,  but  it  is  important  that  students 
know  he/she  is  still  active  and  a content  resource  for  them.  For  example,  in  online  discussions  the  instructor 
might  deflect  questions,  redirecting  them  back  to  the  group  or  to  specific  members  of  the  group.  In  addition,  the 
instructor  should  summarize  and  refocus  discussions  if  they  digress.  As  the  group  begins  to  develop 
cohesiveness,  he/she  can  initiate  higher  risk  activities  such  as  peer-to-peer  critiquing. 

In  the  Production  stage,  the  group  is  working  well,  is  satisfied  with  group  member  relationships,  and 
group  members  have  positive  feelings.  The  group  is  now  able  to  work  autonomously  so  the  instructor  can  step 
back  and  allow  the  group  to  function  independently,  but  he/she  should  remain  ready  to  intervene  if  group 
cohesion  begins  to  impede  work  on  the  task,  for  example,  if  the  social  interaction  threatens  to  impede  learning 
objectives.  Since  morale  has  regained  some  ground  and  group  members  are  more  skilled,  this  is  a time  that  the 
instructor  might  introduce  discussions  around  controversial  topics,  or  some  similarly  challenging  activities. 

Termination  is  characterized  by  a sense  of  loss  and  concerns  about  completion.  Group  members  either 
pull  together  to  finish  with  positive  results  or  disintegrate  early,  impeding  group  work.  Although  morale  is 
fairly  high,  it  may  dip  if  the  group  begins  to  break  apart.  The  instructor  should  use  closure  activities  to  bring 
the  group  to  completion  on  a high  note.  This  can  be  done  formally  with  traditional  activities  such  as  posting 
final  projects  and  evaluations,  or  informally,  for  example,  through  discussions  about  completed  learning 
objectives. 


Conclusions 

The  ideas  presented  above  are  a simplified  overview  of  group  development  and  facilitation.  In  reality, 
both  are  highly  dependent  on  the  context  of  both  the  group  and  the  task,  and  they  can  be  quite  complex.  By 
understanding  the  stages  of  group  development,  faculty  may  become  better  able  to  determine  which  facilitation 
strategies  will  help  develop  group  cohesion  and  provide  the  motivation  for  online  interaction.  Learning  group 
development  will  help  faculty  see  the  context  for  facilitation  strategies  they  may  already  be  using,  for  example, 
icebreakers  and  communication  guidelines.  Finally,  introducing  group  development  in  the  planning  and 
implementation  of  online  learning  may  help  instructors  see  that  they  have  dual  roles  in  online  teaching,  that  of 
the  traditional  content  expert  and  that  of  a group  leader.  Meaningful  and  productive  online  interactivity 
depends  on  the  successful  deployment  of  both  roles  in  a learner-centered  instructional  context. 


References 

Cutler,  R.H.  (1995).  Distributed  Presence  and  Community  in  Cyberspace.  Interpersonal  Communication  and  Technology:  A 
Journal  for  the  Century^  1 (2),  1 7. 

Hillman,  D.C.A.,  Willis,  D.J.,  & Gunawardena,  C.N.  (1994).  Learner- interface  Interaction  in  Distance  Education:  An 
Extension  of  Contemporary  Models  and  Strategies  for  Practitioners.  The  American  Journal  of  Distance  Education,  8,  30-42. 

Lacoursiere,  R.  (1980).  The  Life  Cycle  of  Groups:  Group  Developmental  Stage  Theory.  New  York:  Human  Sciences  Press, 
19-56. 


Moore,  M.G.  (1989).  Three  Types  of  Interaction.  The  American  Journal  of  Distance  Education,  3,  1 06. 

Palloff,  R.  and  Pratt,  K.  (2001).  Lessons  from  the  Cyberspace  Classroom:  The  Realities  of  Online  Teaching  (125-151).  San 
Francisco:  Jossey-Bass  Publishers. 

Tuckman,  B.W.  (1965).  Developmental  Sequence  in  Small  Groups.  Psychological  Bulletin,  63,  384-399. 

Wagner,  E.D.  (1997).  In  Support  of  a Functional  Definition  of  Interaction.  The  American  Journal  of  Distance  Education,  8, 
6-29. 


2108 


Completing  a Life:  Content  and  Design  Challenges  in  Creating 
Educational  Multimedia  Addressing  End-of-Life  Care 


Brian  Winn,  winnb@msu.edu 
Darcy  Greene,  greened@msu.edu 
Karen  Ogle,  ogle@msu.edu 
Michigan  State  University,  United  States 

Leslie  Bricker,  lbrickel@hfhs.org 
Angela  Lambing,  alambinl@hfhs.org 
Henry  Ford  Health  Systems,  United  States 


Abstract;  There  is  a noted  shortage  of  educational  resources  to  help  people  facing 
terminal  illness.  Completing  a Life  is  an  interactive  multimedia  CD-ROM  and  web  site 
developed  to  address  this  issue.  Many  content  and  design  challenges  were  addressed 
during  development.  The  resulting  product  includes  a carefully  crafted  interplay  of 
content,  navigation,  interactivity,  and  look-and-feel  to  create  an  approachable, 
empowering  resource  for  a subject  that  is  emotionally  difficult  for  most  people  to 
address.  Completing  a Life  serves  as  a model  for  medical  education,  combining  personal 
stories  with  task-based  information  contained  within  a virtual  environment. 


Introduction 

Modem  medical  science  allows  people  to  live  for  prolonged  periods,  even  with  the  onset  of  a terminal 
illness.  This  has  led  to  an  increase  in  the  issues  related  to  end-of-life  care.  Given  that  the  subject  of  dying  is 
often  viewed  as  taboo  in  our  society,  frequently  individuals  do  not  talk  openly  about  it  with  their  health 
care  provider  or  are  not  “ready”  to  do  so  when  the  opportunity  presents  itself  (Yang,  2000). 

The  major  goal  of  Completing  a Life  is  to  give  a comprehensive  overview  of  a subject  that  is 
emotionally  difficult  for  most  people  to  address  and  to  do  so  in  a way  that  is  comfortable,  reassuring  and 
instmctive.  The  project  took  on  the  complementary  task  of  opening  up  the  users  to  make  them  receptive  to 
the  information.  If  we  could  achieve  this,  it  was  our  hope  that  Completing  a Life  would  give  users  control 
over  a situation  in  which  they  may  have  felt  none  before,  resulting  in  a sense  of  empowerment.  In  the  case 
of  users  with  an  advanced  illness,  this  empowerment  would  provide  the  ability  to  take  charge  of  their  lives 
for  the  time  they  have  remaining.  In  the  case  of  users  who  are  friends  or  family  members,  this 
empowerment  should  give  them  access  to  much  needed  information  to  answer  their  questions  and  help 
them  empathize  with  the  needs  and  desires  of  the  patient. 

Objectives 

To  achieve  these  goals,  the  core  team,  composed  of  three  content  experts,  two  multimedia  designers,  a 
writer/researcher,  and  an  artist,  spent  much  time  investigating  and  brainstorming  to  arrive  at  the  following 
objectives: 

• Educate  users  on  end-of-life  issues  not  readily  available  elsewhere; 

• Identify  and  organize  the  content  to  reflect  the  tasks  faced  by  the  target  audience; 

• Design  navigation  that  empowers  users  to  find  and  explore  the  information  to  address  their 
own  personal  questions  and  concerns; 

• Communicate  the  information  in  a clear,  inviting  tone  without  being  authoritarian; 

• Develop  a look-and-feel  that  is  welcoming  and  comforting  while  maintaining 
the  emphasis  on  the  content  itself; 

• Integrate  stories  of  others  facing  similar  issues  to  create  a “virtual”  support  group; 

• Create  a multimedia  product  that  is  viewable,  printable  and  accessible  on  the  majority 
of  modem  computer  systems,  while  still  being  innovative  in  its  design. 

In  order  to  achieve  the  desired  objectives,  several  content  and  design  challenges  had  to  be  identified 
and  resolved. 


Figure  1 : Opening  screen  of  Completing  a Life 
http://www.completingalife.msu.edu 

Content  and  Design  Challenges 

Completing  a Life  (see  Figure  1)  was  developed  through  a process  of  iterative  design.  User 
involvement  was  an  integral  part  of  this  process.  Focus  groups  helped  define  and  structure  the  content. 
Prototypes  were  tested  on  the  target  audience  to  help  determine  an  appropriate  writing  style,  effective 
navigation,  and  a welcoming  look-and-feel. 

Content  Research 

The  first  task  in  the  development  of  Completing  a Life  was  to  determine  what  resources  were  currently 
available  for  patients  and  families  and  what  information  was  needed.  We  conducted  focus  groups  with 
health  care  providers,  hospice  workers,  religious  figures  and  family  members  of  the  recently  deceased  and 
interviewed  content  experts  to  aid  in  this  process.  Printed  material,  videos.  Internet-resources  and 
information  gathered  from  discussions  with  support  groups  were  also  examined.  This  research  provided  us 
with  the  foundation  to  begin  the  iterative  process  of  gathering,  writing,  and  structuring  information.  It  also 
led  to  the  creation  of  a list  of  resources  that  would  ultimately  be  highlighted  throughout  Completing  a Life 
as  additional  support  materials  that  users  could  seek  out. 

Information  Architecture 

The  organization  of  content  was  very  important  in  making  the  project  approachable  and  useful.  After 
much  discussion  and  consideration  the  decision  was  made  to  divide  all  the  content  into  three  major  areas: 
Taking  Charge,  Finding  Comfort  and  Reaching  Closure  related  to  the  tasks  our  target  audience  generally 
face.  Each  of  these  tasks  contains  several  topics.  Each  of  the  topics  contained  several  levels  of  subtopics. 
This  hierarchy  defined  a logical  structuring  of  a complex  information  space  and  ultimately  became  the 
basis  for  the  primary  navigation. 

Navigation 

Completing  a Life  is  built  on  a user-centered  model.  Reflecting  awareness  that  not  every  user  will  want 
to  or  be  ready  to  investigate  all  the  content,  the  navigation  allows  users  to  develop  their  own  paths, 
exploring  the  information  they  want  when  they  want  it.  This  user  initiated  nonlinearity,  coupled  with  a 
variety  of  secondary  hyperlinks  throughout  the  content,  was  designed  to  keep  the  user  actively  engaged  in 
the  content,  which  studies  have  shown  promotes  learning  (Brandsford,  Brown,  & Cocking,  2000). 

The  navigational  hierarchy  parallels  the  information  architecture.  However,  we  did  not  want  to 
overwhelm  the  user  with  too  many  choices  onscreen  at  one  time.  Therefore,  the  hierarchy  was  compressed. 
Several  subtopics  were  merged  together  to  limit  both  the  number  of  subtopics  for  each  topic  and  to  limit  the 
maximum  depth  of  the  hierarchy.  Further,  it  was  decided  to  display  only  the.  depth  of  the  hierarchy  and  the 
current  topic’s  immediate  subtopics  at  any  one  time.  Overall,  the  visual  design  of  the  navigation  orients 
users  to  their  location  in  the  information  space  and  promotes  active  exploration  and  discovery  of  subtopics, 
while  providing  an  immediate  method  to  navigate  back  through  the  hierarchy. 
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Writing  Style 

Given  the  delicate  nature  of  the  subject  matter,  we  knew  that  how  the  content  was  presented  was  just 
as  important  as  what  was  actually  said.  We  experimented  with  a variety  of  “voices”  in  the  writing  style.  It 
was  decided  to  use  a “voice”  that  spoke  directly  to  the  person  facing  the  terminal  illness.  This  approach 
supported  our  goal  of  empowerment.  The  tone  was  positive,  friendly  and  experienced.  It  was  appropriate 
for  family  members  and  health  care  workers  who,  as  informal  testing  demonstrated,  had  no  problem 
empathizing. 

The  writing  also  had  to  adhere  to  the  nonlinear  accessibility  of  the  content.  That  is,  since  a user  may 
have  arrived  at  the  page  via  a direct  hyperlink  from  another  part  of  the  product,  we  could  not  assume  that 
the  pages  were  read  in  any  particular  order.  Therefore,  each  page  of  information  had  to  be  written  to  make 
sense  on  its  own. 

Reading  Comprehension 

The  content  was  written  in  a concise  manner.  This  kept  the  information  straightforward  while 
minimizing  the  amount  of  vertical  scrolling.  We  consulted  with  a literacy  and  reading  expert  to  improve 
accessibility  of  content  and  keep  the  material  at  an  upper  elementary  school  reading  level.  It  was  also 
decided  to  supplement  the  onscreen  text  with  an  optional  audio  narration  that  made  the  content  accessible 
to  visually  impaired  users  who  represent  a high  percentage  of  our  target  audience.  And,  since  listening 
comprehension  is  higher  than  reading  comprehension,  it  further  reduced  literacy  barriers  (Lynch,  1988). 

Look-and-Feel 

The  look-and-feel  of  the  project  was  a major  concern.  We  wanted  to  present  the  content  in  an 
approachable,  comfortable  fashion.  Early  in  the  development  several  artistic  styles  were  prototyped.  After 
some  user  testing  and  feedback  from  other  multimedia  designers,  we  decided  to  go  with  a watercolor  style 
because  we  felt  it  conveyed  the  sense  of  softness  and  delicacy  of  the  subject  matter. 

The  visual  metaphor  we  decided  upon  was  that  of  a back  porch  overlooking  a lushly  vegetated  yard. 
The  visual  metaphor  links  the  four  primary  locations  in  the  original  watercolor  scene  to  the  four  primary 
content  sections  in  the  product.  The  porch  represents  the  location  of  the  personal  stories,  as  people  often  sit 
and  talk  on  porches.  A gazebo  with  a table  and  chairs  represents  the  Taking  Charge  content.  A comfortable 
looking  tree  swing  represents  the  Finding  Comfort  content.  And  a bench  overlooking  a calm  pond 
represents  the  Reaching  Closure  content.  The  metaphor  between  the  visuals  and  the  content  was  very  subtle 
and  intended  only  as  a subliminal  cue  as  the  user  browsed  the  content,  not  as  a navigational  metaphor. 

User  testing  verified  the  watercolor  style  and  visual  metaphor  were  effective.  However,  we  found  some 
users  did  not  respond  to  the  look-and-feel  positively  or  negatively.  They  were,  in  fact  only  interested  in  the 
information  itself.  For  this  group,  we  added  a listing  of  Topics  to  be  used  to  navigate  directly  to  the  desired 
information. 

Personal  Stories 

A defining  element  of  the  project  was  the  integration  of  “personal  stories.”  Personal  stories  are  told 
firsthand  through  video  interviews.  The 'interviews  document  the  real-life  experience  of  people  who  have 
confronted  advanced  illness  in  one  way  or  another.  Rather  than  use  actors  and  scripts,  it  was  decided  that 
true  stories  told  by  those  who  were  experiencing  a life  threatening  illness  were  much  more  powerful  and 
effective.  However,  the  process  of  finding  and  interviewing  candidates  willing  to  tell  their  stories  was 
challenging.  It  was  found  that  there  was  a limited  time  between  the  point  when  individuals  were  ready  to 
share  their  stories  and  when  they  were  still  in  a physical  state  suitable  to  be  interviewed.  Our  team  had  to 
work  closely  with  hospice  and  a variety  of  doctors  to  help  identify  appropriate  candidates.  The  team  also 
had  to  respond  quickly  when  a candidate  agreed  to  be  interviewed. 

An  effort  was  made  to  include  interviews  from  a diverse  group  of  individuals  to  increase  the  likelihood 
that  the  user  would  identify  closely  with  one  of  them.  Fifteen  interviews  were  conducted.  However,  several 
of  the  interviews  were  not  included  in  the  final  product.  The  primary  reasons  for  exclusion  were  either  the 
individuals  were  not  ready  to  discuss  issues  related  to  completing  their  lives  or,  because  of  their  advanced 
illness,  they  were  very  difficult  to  understand  and  follow.  Ultimately,  nine  individuals  were  included  in 
Completing  a Life,  representing  multiple  demographic  groups,  illnesses,  and  personalities. 

Overall,  there  are  more  than  three  hours  of  interviews  in  Completing  a Life.  Each  interview  was 
arranged  in  the  same  task-based  categories  as  the  content:  Taking  Charge.  Finding  Comfort  and  Reaching 


Closure.  Within  each  category,  the  user  can  watch  segments  of  the  interview.  Therefore,  a user  can  listen  to 
the  parts  of  the  stories  that  are  interesting  and  relevant  to  them. 

These  video  narratives  illuminate  many  of  the  topics  addressed  in  the  content  sections  and  are 
extensively  cross-linked  with  that  content  to  enhance  the  user’s  interactive  multimedia  experience.  Links  to 
topics  relevant  to  the  individual’s  story  are  available  below  the  video  interviews.  By  following  the  link, 
users  can  explore  the  topic  in  more  depth.  In  addition,  content  pages  have  links  to  related  personal  stories. 


At  the  beginning  of  the  project  in  1999,  Web  usage  statistics  showed  that  over  80  percent  of  monitors 
were  at  least  800x600  pixels  in  size  (Nielsen,  2000).  Given  that  this  percentage  would  assuredly  increase 
over  time,  we  designed  the  pages  within  Completing  a Life  for  viewing  on  800x600  pixel  displays  or  larger. 
This  allowed  for  optimal  layout  of  navigation  and  content  with  no  horizontal  and  minimal  vertical 
scrolling. 

Following  Jakob  Nielsen’s  premise  that  “navigation  is  a necessary  evil  that  is  not  a goal  in  itself  and 
should  be  minimized”  (Nielsen,  2000),  we  kept  navigation  to  a minimum.  The  average  page  in  Completing 
a Life  has  less  than  25  percent  of  its  real  estate  dedicated  to  navigation.  The  largest  amount  of  real  estate  is 
dedicated  to  the  content.  Ample  space  is  also  given  to  “white  space”.  White  space  is  used  to  segment  the 
page  into  meaningful  areas  that  the  user  can  quickly  identify. 

The  pages  were  designed  to  be  conducive  to  printing.  Most  individuals,  particularly  the  elderly  who 
represent  a sizeable  portion  of  our  target  audience,  find  it  difficult  to  read  a large  amount  of  text  on  the 
computer  screen.  Further,  we  have  found  that  users  often  enter  an  “information  gathering”  mode  in  which 
they  browse  through  the  content  rapidly  skimming  it.  When  they  find  something  of  interest  they  often 
prefer  to  print  it  to  be  read  in  detail  at  a later  time  so  they  can  continue  in  their  information  gathering.  They 
also  may  want  to  print  content  to  share  it  with  others  who  do  not  have  computer  access  readily  available. 
For  these  reasons,  we  avoided  such  page  design  features  as  frames  and  popup  windows. 


Despite  the  recent  downturn  in  the  dot-com  industry,  Internet  use  is  still  on  the  rise,  especially  among 
home  users  (Roberts  & Crump,  2001).  The  so-called  “digital  divide”  between  those  who  have  access  and 
those  who  traditionally  do  not,  including  the  ethnic  minorities,  rural  populations,  and  the  elderly,  is  rapidly 
shrinking  (Fox  & Rainie,  2000;  Eng,  Maxfield,  Patrick,  Deering,  Ratzan  & Gustafson,  1998).  Today,  one  of 
the  most  common  uses  of  the  Internet  by  consumers  is  to  obtain  health  information  (Fox  & Rainie,  2000; 
Horrigan  & Rainie,  2002).  With  this  in  mind,  we  developed  Completing  a Life  with  Web-enabled 
technologies.  However,  we  did  not  want  to  limit  our  audience  to  those  with  just  Internet-access.  Therefore, 
we  utilized  only  client-side  technologies  that  do  not  require  a server.  This  provided  us  the  flexibility  to 
deliver  the  product  on  the  Internet,  within  Intranets,  such  as  local  hospitals’  networks,  or  on  CD-ROM. 

The  design  of  Completing  a Life  demanded  cutting-edge  multimedia  technologies.  Audio,  video, 
animation,  interactivity  and  the  ability  to  print  complex  medical  forms  could  not  be  handled  by  HTML 
alone.  Flash,  QuickTime  and  PDF  were  chosen  to  address  these  multimedia  needs.  These  technologies  have 
become  de  facto  standards  on  the  Web.  They  are  available  as  free  downloads  and  work  identically  on  both 
Macintosh  and  Windows  platforms.  Further,  a high  percentage  of  typical  Web  users  already  have  these 
technologies  installed  and  configured  on  their  machines.  For  those  without  the  required  technologies. 
Completing  a Life  guides  users  through  the  download  and  installation  process. 

Another  delivery  decision  was  to  target  broadband  connections  rather  than  slower  phone-modem 
connections.  This  choice  was  made  for  several  reasons.  First,  broadband  availability  is  on  the  rise.  It  is 
estimated  that  1/3  of  household  Internet  users  and  2/3  of  business  users  will  have  a broadband  connection 
by  2002  (Rose  & Warren,  2001).  Second,  it  was  our  goal  to  produce  an  engaging,  rich  multimedia 
experience.  It  would  be  very  difficult  to  do  this  with  the  bandwidth  that  a 56K  modem  provides.  Third,  we 
wanted  the  personal  story  videos  to  convey  the  personality  and  emotions  the  individuals  expressed  in  their 
interviews.  This  would  not  be  possible  with  thumbnail-size  video  at  slow  frame  rates  and  poor  audio 
quality. 

The  site  was  made  available  on  the  Internet  in  October  2001.  Broadband  users  can  access  the  content 
in  real-time  with  little  or  no  wait  time.  Modem  users  can  access  the  content.  However,  in  most  cases,  they 
may  have  to  wait  for  the  multimedia  elements  to  load.  We  also  made  the  product  available  on  CD-ROM  for 
those  wanting  immediate  access  without  broadband  connections  and  those  without  Internet  connections 
altogether. 
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An  Evolving  Model 

Completing  a Life  is  the  third  in  a series  of  medical  products  developed  at  Michigan  State  University. 
Breast  Cancer  Lighthouse  (Greene  & Heeter,1996)  released  in  1996,  pioneered  the  model  of  combining 
medical  information  with  personal  stories  in  a virtual  environment.  Easing  Cancer  Pain  (Greene,  Ogle  & 
Winn,1998)  followed  in  1998  further  refining  the  model.  Lessons  learned  from  both  products  aided  in  the 
design  of  Completing  a Life. 

Combining  Personal  Stories  with  Medical  Information 

In  Breast  Cancer  Lighthouse,  personal  stories  were  shown  to  be  an  effective  approach  to  creating  a 
“virtual  support  group”  (Greene  & Heeter,1998).  Easing  Cancer  Pain  extended  the  personal  stories  model 
by  linking  from  medical  information  in  the  product  to  related  segments  of  the  personal  stories.  This  was 
found  to  “humanize”  the  information  by  tying  it  to  the  individuals’  actual  experiences  (Greene,  1998).  In 
Completing  a Life,  this  interlinking  of  content  with  the  personal  stories  was  enhanced  to  go  both  ways.  That 
is,  personal  stories  segments  also  linked  to  content  related  to  the  issues  being  discussed  in  the  stories.  This 
added  validity  to  the  personal  stories.  Further,  it  provided  “just-in-time”  information  as  users  heard  topics 
discussed  within  the  personal  stories. 

Virtual  Environments 

Another  hallmark  of  the  model  developed  in  Breast  Cancer  Lighthouse  was  the  use  of  a virtual 
environment  to  disseminate  the  medical  information.  In  the  product,  information  was  tied  to  locations  on  an 
island.  In  one  area  the  user  could  walk  along  a beach,  by  navigating  with  a mouse.  On  the  beach,  they  were 
introduced  to  women  who  have  survived  breast  cancer.  The  user  could  click  on  a woman  to  share  in  her 
personal  story.  In  another  area  of  the  product,  the  user  could  walk  along  a garden  path,  containing  a variety 
of  signs.  When  the  user  clicked  on  a sign,  they  were  presented  with  information  about  the  given  topic.  A 
similar  navigational  metaphor  was  employed  in  Easing  Cancer  Pain.  The  user  navigated  along  a path  at  a 
virtual  retreat  grounds,  locating  information  in  kiosks  found  along  the  path.  Personal  stories  were  told 
around  a campfire.  The  virtual  environments  were  shown  to  create  a relaxing  mood  among  users, 
increasing  their  receptiveness  to  learning  (Greene  & Heeter,1998). 

The  information  space  in  Easing  Cancer  Pain  was  much  more  complicated  than  Breast  Cancer 
Lighthouse.  Testing  found  that  some  users  had  a difficult  time  making  meaningful  associations  between  the 
information  and  the  virtual  locations;  this  lead  to  some  disorientation  when  navigating.  The  information 
space  in  Completing  a Life,  containing  a wide  variety  of  topics,  was  even  more  complex  than  Easing 
Cancer  Pain.  Therefore,  rather  than  using  a virtual  environment  as  a direct  navigational  metaphor,  we 
decided  to  employ  a more  traditional,  hierarchical  menu  system  to  navigate  the  information.  However,  a 
virtual  environment  was  utilized  as  a visual  metaphor  to  achieve  the  benefits  found  in  Breast  Cancer 
Lighthouse  without  the  confusion  found  by  some  in  Easing  Cancer  Pain. 


User  Testing 

An  evaluation  study  of  a target  group  of  50  patients  has  begun  in  an  inner  city  hospital  setting.  The 
study  seeks  to  determine  if  people  facing  life-threatening  illnesses  are  comfortable  using  computers  as  a 
way  to  learn  about  end-of-life  issues  and  if  our  project  objective  of  creating  a comforting,  empowering, 
easy  to  use  product  is  obtained.  To  date,  a small  group  of  chemotherapy  patients,  three  men  and  six  women 
with  an  average  age  62  years,  have  participated  in  the  study.  While  this  sample  is  small,  the  preliminary 
data  has  so  far  validated  our  approach. 

After  viewing  the  CD-ROM,  five  patients  said  they  were  very  comfortable  using  the  computer  to  learn, 
three  were  somewhat  comfortable  and  one  was  not  comfortable.  All  the  patients  felt  comfortable  with  the 
information  found  in  Completing  a Life.  Two  of  the  patients  said  they  felt  somewhat  more  empowered  to 
talk  with  their  physician  about  end-of-life  issues.  Another  indication  of  empowerment  was  three  of  the  five 
patients  that  did  not  currently  have  an  Advance  Directive  said  they  were  somewhat  likely  to  complete  one 
after  spending  time  with  the  CD-ROM.  Overall,  six  of  the  nine  were  interested  in  spending  more  time  with 
the  CD-ROM  and  seven  said  they  would  recommend  it  to  others.  All  of  them  found  the  CD-ROM  to  be 
somewhat  easy  to  very  easy  to  use. 
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Summary 


Completing  a Life  represents  a model  for  a kind  of  medical  education  that  did  not  exist  before 
interactive  multimedia.  It  delivers  a wealth  of  learning  resources  in  a form  that  is  not  intimidating  or 
overwhelming  and  that  places  control  of  the  learning  experience  in  the  hands  of  the  learner.  It  also  brings 
the  power  of  multimedia  to  bear  in  the  use  of  the  personal  stories  whose  immediacy  and  emotional 
resonance  add  meaning  and  accessibility  to  the  material. 

The  development  of  Completing  a Life  demonstrates  the  tight  interplay  between  content  and  design  in 
educational  multimedia.  To  achieve  the  desired  learning  goals,  the  content  and  design  challenges  must  be 
addressed  simultaneously  and  harmoniously. 
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Abstract:  This  paper  describes  a proposed  system  for  competency -based  just-in-time  learning 
that  uses  competency  ontologies  and  smantic  web  services  to  deliver  learning  objects  to 
learners.  A competency  ontology  is  a semantic  description  of  the  competencies  needed  in 
order  to  participate  in  the  business  processes  of  a company.  Semantic  web  services  are  web 
services  with  semantic  descriptions  that  enable  people  and  software  agents  to  more  efficiently 
identify  services  that  can  provide  the  proper  learning  objects. 

1.  Competency-Based  Just-in-Time  Learning 

A goal  of  corporate  e-leaming  is  to  increase  efficiency  and  effectiveness  by  identifying  precisely  the  training 
that  an  employee  needs  to  do  their  Job  and  providing  that  training  in  the  context  of  day  to  day  Job  activities  of 
the  employee.  (Fig.  1)  illustrates  the  flow  of  activity  to  implement  competency-based  Just-in-time  learning 
services  in  an  enterprise  environment.  In  the  lower  right  hand  comer  is  a representation  of  a couple  of  the 
business  processes  of  the  enterprise.  In  the  upper  left  hand  comer  are  the  ontologies  that  capture  knowledge 
about  the  products,  organizations,  competitors,  etc.  of  the  enterprise.  A competency  ontology  is  included  that 
captures  the  competencies  that  an  employee  must  possess  to  participate  in  specific  activities  of  the  business 
processes.  The  lower  left  hand  comer  illustrates  the  learning  resources  such  as  learning  objects,  courses,  email, 
etc.  Each  of  these  learning  resources  has  been  manually  or  automatically  linked  to  various  parts  of  one  or  more 
of  the  enterprise  ontologies  to  enable  people  and  software  agents  to  more  efficiently  discover  the  correct 
learning  resources.  The  upper  right  hand  corner  illustrates  the  learner  profile  for  an  employee  that  contains 
preferences,  experiences  and  assessments  of  the  employee’s  competencies.  The  box  in  the  upper  middle  of  the 
figure  is  a competency  gap  analysis  that  calculates  what  competencies  the  employee  lacks  to  effectively  carry 
out  their  Job  responsibilities.  This  calculation  is  based  on  what  the  employee  needs  to  know  and  what  they 
already  know.  Once  the  competency  gap  has  been  identified,  a learning  irodel  is  selected  either  manually  or 
automatically.  This  establishes  the  best  way  for  the  employee  to  attain  the  competency  and  enables  the  system 
to  create  the  personalized  learning  process  at  the  lower  middle  of  the  figure.  The  personalized  learning  process 
may  be  created  and  stored  for  later  use  or  it  may  be  created  whenever  it  is  needed,  thus  enabling  dynamic  access 
to  learning  objects  based  on  the  most  recent  information  about  the  learner  and  the  environment.  The  results  of 
the  personalized  learning  process  are  then  returned  to  the  learner’s  profile. 


iBEST  COPY  AVAIMBIUS 


2.  Advanced  Technology  Support 

Each  box  in  the  business  processes  and  the  personalized  learning  processes  in  (Fig.  1)  is  an  activity.  These 
activities  might  be  implemented  as  existing  legacy  applications  or  new  applications  on  a variety  of  hardware 
and  software  systems.  In  the  past,  it  would  have  been  difficult  to  integrate  applications  executing  on  such 
heterogeneous  systems.  But  the  development  of  a set  of  technologies  referred  to  as  web  services  (W3C  2002)  is 
simplifying  this  integration.  The  most  important  of  these  technologies  are  SOAP,  UDDI  and  WSDL  IBM, 
Microsoft  and  others  are  agreeing  on  the  these  specifications.  SOAP  is  a technology  for  sending  messages 
between  two  systems  using  XML  for  representing  the  messages  and  HTTP  as  the  most  common  transport. 
UDDI  is  a specification  for  an  online  registry  that  enables  publishing  and  discovery  of  web  services.  WSDL  is 
an  XML  representation  used  for  describing  the  services  registered  with  a UDDI  registry.  Vendors  of  enterprise 
applications  such  as  ERP  and  CRM  have  announced  support  for  web  service  interfaces  to  their  products. 

However,  UDDI  and  WSDL  have  limited  capability  for  representing  semantic  descriptions  of  web  services.  The 
World  Wide  Web  consortium  has  initiated  an  effort  to  develop  specifications  for  a semantic  web  (W3C  2002) 
where  web  pages  will  include  semantic  descriptions  of  their  content.  The  US  Defense  Advanced  Research 
Projects  Agency  (DARPA)  Agent  Markup  Language  program  (DAML  2002)  has  focused  on  the  development 
of  a semantic  markup  language  DAML+OIL  for  web  pages  that  will  enable  software  agents  to  understand  and 
reason  about  the  content  of  web  pages . The  program  has  also  developed  a markup  language  for  web  services 
called  DAMLrS  (Ankolekar  2001)  that  enables  an  improved  semantic  description  of  a web  service.  The  o- 
learning  industry  has  developed  metadata  standards  for  describing  the  semantics  of  learning  objects  (IEEE 
2001).  The  Advanced  Distributed  Learning  Lab  has  developed  an  XML  representation  of  this  metadata  as  part 
of  the  Shared  Content  Object  Reference  Model  (SCORM)  specification  (SCORM  2002).  The  system  in  Figure  1 
requires  a representation  for  business  processes  so  that  competencies  can  be  mapped  to  activities  in  a business 
process  and  a representation  of  personalized  learning  processes  to  enable  integration  of  these  learning  processes 
with  business  processes.  There  have  been  attempts  to  standardize  the  representation  of  business  processes 
(WFMC  2002).  There  are  now  efforts  underway  to  standardize  on  a process  representation  for  web  services. 

A few  commercially  available  e-leaming  products  use  competency  hierarchies  to  capture  the  skills  necessary  for 
various  job  types.  The  competencies  in  these  hierarchies  are  then  mapped  to  courses  that  can  improve  an 
employee’s  competency  in  a certain  area.  There  is  no  industry  standard  representation  for  these  competency 
hierarchies  and  they  do  not  capture  the  rich  semantics  of  competencies.  The  concepts,  relationships  and 
processes  of  an  enterprise  can  be  captured  in  a set  of  enterprise  ontologies.  Ontology  representation  and 
reasoning  systems  are  available  (Lenat  1995)  and  flie  DARPA  DAML  program  has  also  done  significant 
research  on  the  representation  of  ontologies.  Intelligent  software  agents  can  be  used  effectively  in  a system  such 
as  the  one  described  in  (Fig.  1).  There  has  been  research  into  the  use  of  software  agents  for  discovery  of 
information  (Woelk  1994),  collaboration  planning  and  automation  of  processes  (Tate  1996)  and  numerous  other 
applications  (Bradshaw  1997).  This  research  is  now  focusing  on  the  use  of  software  agents  with  web  services 
(Hendler  2001).  These  agents  can  proactively  search  for  learning  objects  both  that  are  needed  to  meet 
dynamically  changing  learning  requirements. 
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fAbstract 

For  three  years,  the  Technology  and  Learning  Team  (TLT)  has  supported  the  campus  at 
University  of  Colorado  at  Denver  (UCD)  with  integration  of  technology  in  instruction.  In  this 
role,  TLT  works  with  UCD  faculty  in  the  use  of  a variety  of  technologies  for  instruction, 
research,  and  service.  TLT  brings  a holistic  or  integrated  approach  that  stresses  pedagogical 
considerations  over  technical  ones.  A 5-part  support  model  describes  this  approach  for 
technology  integration  at  the  UCD  campus.  Skill  development,  community  building,  product 
development,  support,  and  vision  casting  comprise  the  five  elements  of  the  model. 


[Faculty  Support  at  UCD  Campus 

Since  its  inception  in  1999,  the  Technology  and  Learning  Team  has  grown  in  member  numbers 
and  collective  experience.  With  that  experience  comes  the  knowledge  that  access  to  technology 
for  faculty  is  not  enough  for  successful  integration.  It  requires  a team  effort,  with  diverse  skills, 
and  a focus  on  multiple  areas  of  support.  TLT  brings  a holistic  and  integrated  approach  that 
stresses  pedagogical  considerations  over  technical  ones.  A 5-part  model  describes  this  approach 
for  technology  integration.  Skill  development,  community  building,  product  development, 
support,  and  vision  casting  comprise  the  five  elements  of  the  model. 

TLT  team  supports  faculty  in  5 areas: 

1.  Skill  development 
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Purpose;  to  develop  the  technical  skills  of  faculty.  This  was  provided  individually  and 
collectively  through  workshops  on  a variety  of  topics  including  Blackboard,  eCollege, 
PowerPoint,  Dreamweaver,  NVIVO,  and  Inspiration. 

2.  Community  building 
Purpose:  to  bring  faculty  together  to  share  their 
experiences,  challenges,  successes,  and  war  stories. 

This  was  accomplished  through  show  ‘n  tell 
meetings,  workshops,  websites,  and  a common 
faculty  technology  lab. 

3.  Product  development 
Purpose:  for  TLT  members  to  develop  products  that  were  experimental  and/or  required 
advanced  technical  skills.  Examples  include  website  templates,  CD-ROMs,  website 
design,  and  graphic  creation. 

4.  Support 

Purpose:  to  stress  pedagogical  considerations  over  the  needs  of  the  technology.  Support 
included  one-on-one  consultation  and  participation  in  departmental  task  force  meetings. 
TLT  contributes  graphic  design  considerations,  potential  technical  solutions,  estimated 
costs,  and  implementation  issues. 

5.  Vision  casting 

Purpose:  to  lead  technology  integration  on  the  UCD  campus.  This  means  that  TLT  sought 
out  the  latest  technology  and  tested  its  limits,  researched  what  other  campuses  were 
doing,  found  good  examples,  participated  in  conferences,  and  conducted  needs 
assessments. 
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Faculty  Support  Model 
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Abstract:  This  paper  describes  the  development  of  Master  - a computer-mediated  tool 
for  learning  and  research  in  probability  reasoning.  The  objectives  of  this  software  are  1) 
to  explore  how  levels  of  interactivity  and  dynamism  of  representations  in  probability 
influence  levels  of  dynamism  in  student  mental  models  of  probability  and  2)  whether 
levels  of  dynamism  in  mental  models  influence  probability  reasoning.  We  expect  that 
there  is  a relationship  between  levels  of  interactivity  and  dynamic  movement  of 
representational  resources  and  flexible  and  dynamic  mental  models  that  allow  for  better 
reasoning  about  probability.  Hence,  we  are  developing  a computer  program  to 
investigate  this  issue.  Not  only  will  Master  support  learning  in  probability,  but  it  will 
also  help  us  investigate  whether  and  how  dynamic  and  interactive  representations  are 
more  influential  than  others  in  improving  probabilistic  thinking. 


Introduction 

Many  studies  have  been  conducted  to  investigate  the  effects  of  computer  animations  in  different 
instructional  environments  (Chipman,  1993;  Park,  1994;  Schank  & Cleary,  1995;  Snir  et  al.,  1995). 
Dynamical,  graphical  representations  are  considered  to  be  effective  modes  of  depicting  information  in 
instructional  technology  environments.  Mental  model  theory  offers  a useful  lens  for  examining  how  and 
why  dynamic  representations  lead  to  better  reasoning  ability.  Dynamic  representations  could  be 
instigating  development  of  dynamic  mental  models,  allowing  for  better  reasoning  about  probability. 

Park  and  Gittelman  (1992)  suggest  that  animation  is  more  effective  and  efficient  than  static  graphics  and 
help  students  form  useful  mental  models  about  the  system  structure  and  troubleshooting  procedures.  Mayer 
(1997)  claims  that  well  designed  visual  representations  allow  students  to  simulate  the  system  mentally. 
Centner  and  Stevens  (1983),  examining  how  analogical  comparisons  influence  thinking,  and  Schwartz  and 
Black  (1996),  studying  using  analog  imagery  as  the  paradigm  to  investigate  mental  models  and  reasoning, 
discover  that  imagery  serves  as  the  basis  for  depiction,  which  allows  for  simulation  and  reasoning.  Kaplan 
(2001)  found  that  dynamically  suggestive  visuals  of  system  components  encourage  students  to  form  mental 
depictions,  allowing  for  mental  simulation,  thus  leading  to  improvements  in  reasoning.  Russell  and  Corter 
(in  prep)  examined  the  uses  of  visual  devices  in  probabilistic  reasoning.  The  experiment  revealed  a 
significant  relationship  between  drawing  an  outcome  tree  and  determining  a correct  solution. 

Statistics  is  not  often  presented  in  visual  or  dynamic  formats.  A problem-solver  would  have  to  generate 
mental  movements  in  their  minds  themselves,  perhaps  scaffolding  their  thinking  by  creating  an  external 
representation  of  the  problem.  Many  students  of  statistics  either  generate  an  inefficient  representation  or 
are  not  able  to  generate  a mental  model.  We*ve  designed  Master  io  provide  multiple  visual  representations 
of  probability  to  not  only  support  learning  in  probability,  but  help  us  investigate  whether  certain 
representations  are  more  influential  than  others  in  improving  probabilistic  thinking.  Furthermore,  Master  is 
designed  for  graduate  students  studying  statistics  towards  the  fulfillment  of  their  research  requirement.  It 
helps  students  comprehend  abstract  concepts  of  statistics  through  visual  representations  and  authentic  tasks 
in  a computer-mediated  environment.  By  adapted  to  online  settings.  Master  will  be  also  available  for 
distance  learners. 
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Master  - Software  Design 


The  objectives  of  Master  are  to  explore  how  levels  of  interactivity  and  dynamism  of  representational 
resources  in  probability  influence  levels  of  dynamism  in  student  mental  models  of  probability  and  whether 
levels  of  dynamism  in  mental  models  influence  reasoning.  Master  presents  probability  problems  in  five 
representations  — Standard  Text  (ST),  Static  Visual  Displays  (SVD),  Static  Visual  Displays  with  Motion 
Cues  (SVD-MC),  Dynamic  Visual  Displays  (DVD),  and  Interactive  Dynamic  Visual  Displays  (IDVD). 
Comparisons  among  representational  use  will  allow  us  to  make  several  discoveries.  First,  we  would  like  to 
find  out  whether  an  external  representation  that  is  visual  and  dynamic  encourages  students  to  visualize  the 
problem  dynamically  and  have  higher  levels  of  dynamism  in  mental  models.  To  explore  this  possibility, 
we  can  compare  learning  with  the  standard  problem  format  to  learning  in  a static  visual  format  or  a 
dynamic  visual  format.  Second,  we  will  be  able  to  determine  whether  all  forms  of  dynamic  displays  are 
equally  conducive  to  generating  a more  effective  mental  model.  It  is  possible  that  a more  d)mamic  mental 
model,  which  is  more  conducive  to  reasoning,  is  created  by  the  generation  of  the  dynamism  in  the  mental 
model.  This  leads  us  to  examine  whether  static  visual  displays  with  motion  cues  which  encourage  mental 
movement  would  be  superior  to  dynamic  visual  displays  because  static  visual  displays  vith  adequate 
motional  cues  support  students  to  think  in  higher  degrees  of  dynamism.  Third,  we  believe  that  a student 
constructs  information  dynamically  when  he/she  manipulates  an  object.  We  expect  that  software  which 
requires  students  to  manually  move  objects  would  be  better  than  software  which  automatically  moves 
objects  for  them.  Comparison  of  Interactive  Dynamic  Visual  Display,  which  allows  students  to  manipulate 
screen  objects,  to  Dynamic  Visual  Displays  will  further  reveal  whether  we  should  be  moving 
representations  for  students  or  encouraging  students  to  make  mental  movements. 


An  Example  - If  a pair  of  dice  is  rolled,  find  the  probability  of  a match  or  getting  a sum  of  7. 
Event  A:  Getting  a Match  = {(1,1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6)} 

Event  B:  Getting  a sum  of  7 = ((6,1), (5, 2), (4, 3), (3, 4), (2, 5), (1,6)} 
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? [IDVD]  The  screen  design  is  similar  to  the  DVD  screen  design.  Differently,  in  IDVD,  users  are  able  to 
manipulate  these  two  tables  and  then  observe  the  overlapping  color  effect. 


Research  Expectations  and  Implications  for  the  Future 

Master  research  permits  us  to  make  the  following  investigations  while  promoting  probability  learning.  We 
believe  students  will  develop  more  dynamic  mental  models  and  be  able  to  solve  probability  problems  better 
if  the  information  is  either  presented  or  encouraged  to  develop  in  higher  levels  of  interactivity  and 
dynamism.  We  expect  students  in  Dynamic-Visual-Display  group  will  demonstrate  more  dynamic  mental 
models  and  better  reasoning  than  students  in  the  SVD  group.  We  expect  all  students  presented  with  visual 
representation  will  form  stronger  mental  models  and  develop  better  reasoning  strategies  than  students 
learning  in  a standard  text  format.  We  suspect  that  software  which  requires  students  to  manually  move 
objects  would  allow  students  to  build  greater  dynamism  in  their  mental  models  than  the  software  which 
automatically  moves  objects  for  them.  Thus,  we  expect  that  students  in  the  IDVD  group  will  demonstrate 
more  dynamic  mental  models  and  better  reasoning  than  the  students  in  the  Dynamic -Visual-Display  group. 
As  stated  by  Park  (1998),  that  static  graphics  with  adequate  motional  cues  representing  the  dynamic  aspects 
of  the  task  would  accomplish  results  similar  to  those  of  animation.  We  are  not  certain  of  the  comparison  of 
levels  of  dynamism  in  mental  models  and  learning  performance  of  the  students  in  SVD-MC  group, 
Dynamic -Visual-Display  group,  and  IDVD  group.  If  the  result  shows  that  the  students  in  SVD-MC  group 
actually  do  better  than  the  students  in  Dynamic  - Visual-Display  group  or  in  IDVD  group,  it  could  lead  us  to 
consider  that  static  visual  displays  with  motion  cues  could  support  students  to  conceive  a problem  through 
generating  mental  movements  themselves,  which  promotes  students  for  better  probability  reasoning  and 
improves  probabilistic  thinking. 

Master  research  will  reveal  how  representation  influences  reasoning  about  probability.  Results  will 
provide  evidence  for  how  to  design  software  for  teaching  probability  and  suggestions  for  how  to  design 
educational  technology  in  general.  In  development  of  Master  we  framed  the  basis  of  our  designs  in  the 
way  people  think  versus  the  way  the  technology  looks.  Basing  our  instructional  technology  research 
questions  from  the  perceptive  of  how  the  mind  is  working  led  to  the  development  of  new  media 
representations.  The  overall  goal  is  to  initiate  further  studies  in  diverse  disciplines,  and  discover  a tool  that 
is  flexible  and  adaptable  enough  to  promote  multi-representational  displays  across  disciplines. 
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IDIQMA-TIC  © A CALL  authoring  system  for  web  and  cdrom  with  intelligent  feedback  and 
half-open  questions 

Tension  between  knowledge  and  competence  in  second  language  acquisition  (SLA) 

« Skills  oriented  learning » is  the  new  buzz  word  in  SLA.  In  many  language  classes, 
comprehension  and  oral  expression  are  now  the  core  didactic  activities,  instead  of  the  “out- 
dated’’ knowledge  oriented  activities.  This  phenomenon  can  be  observed  in  all  levels  of 
education  (from  primary  to  university)  and,  consequently,  in  computer  assisted  language 
learning  (CALL)  software. 

1.  The  need  in  SLA  to  focus  on  knowledge 

The  raised  interest  for  skills  training  is  especially  profitable  for  starting  language  learners, 
who  have  great  interest  in  being  “sub-merged”  in  the  new  language,  because  their  knowledge 
base  is  very  small.  Things  are  different  with  intermediate  and  advanced  language  learners, 
whose  learning  problems  cannot  be  resolved  by  more  skills  training.  Those  learners  present 
two  kinds  of  problems.  The  first  phenomenon  is  known  as  the  “ceiling”  problem  (Gass  & 
Selinker,  1994,  Second  Language  Acquisition:  An  Introductory  Course  Lawrence  Erlbaum, 
Inc):  it  is  the  incapacity  of  language  learners  to  move  from  an  intermediate  level  to  an  expert 
level.  The  second  phenomenon  is  linked  with  errors:  research  shows  that  intermediate  level 
language  learners  keep  making  the  same  mistakes,  persistent,  difficult -to-correct  errors.  This 
is  known  as  “linguistic  fossilisation”,  demonstrated  by  Germain  and  S6guin  in  Le  point  sur  la 
grammaire  (1998,  Paris  : CLE  International). 

2.  Tools  for  knowledge  building  and  deepening 

Knowledge  is  generally  divided  into  three  types  : cultural  knowledge,  literacy  knowledge  and 
grammatical  and  lexical  knowledge.  In  this  context,  the  grammatical  and  lexical  knowledge  is 
at  stake.  Exercises  are  crucial  in  this  component  of  the  language  learning  process  (Hulstijn 
2001,  Intentional  and  incidental  second  language  vocabulary  learning:  A reappraisal  of 
elaboration,  rehearsal  and  automaticityy  In  P.  Robinson  (ed.).  Cognition  and  Second 
Language  Instruction  (pp  258-286).  Cambridge:  Cambridge  University  Press,  2001).  Second, 
it  is  important  to  mention  the  ’’pedagogy  of  errors"  which  does  not  consider  errors  as  ending 
points  obstructing  the  acquisition  of  a language,  but,  on  the  contrary,  as  an  interesting 
symptom  of  the  “interlingua”,  characterizing  any  learning  process. 

Finally,  a third  set  of  findings  relates  to  the  ratio  which  exists  between  task  and  transfer.  Thus, 
Solomon  and  Perkins  (1989,  Rocky  roads  to  transfer.  Rethinking  mechanisms  of  a neglected 
phenomenon.  Educational  Psychologist  24,  113-142)  noted  that  the  more  complex  the 
cognitive  assimilation  of  a task  is,  the  more  language  learning  and  the  treinsfer  of  knowledge 
to  competences  will  have  succeeded.  One  of  die  most  complex  tasks  is  undoubtedly 
translation. 

To  make  profitable  use  of  these  assets,  we  need  relatively  complex  applications  that  confront 
learners  with  the  differences  between  their  mother  tongue  and  the  foreign  language  and  that 
formulate  an  adequate  response  to  the  errors  generated  by  the  contact  between  languages. 

3.  And  what  about  keeping  a place  to  new  technologies  in  the  choice  of  learning  tools  ? 

It  is  true  that,  today,  there  is  already  (lexical  and  grammatical)  knowledge  oriented  software, 
but  most  of  it  presents  some  considerable  disadvantages. 

They  are  generally  limited  to  closed  activities  (such  as  multiple  choice  questions)  that  activate 
only  a minimum  of  high  level  cognitive  processes.  Another  disadvantage  is  that  the  generated 
exercises  are  almost  exclusively  of  the  “structuro- global”  type,  i.e.  the  only  objective  is  to 
memorize  forms,  mostly  by  repetition  (drill).  Furthermore,  feedback  is  crucial  in  language 
software.  To  be  effective,  feedback  must  be  personalized.  It  must  refer  as  much  as  possible  to 
the  input  of  the  learners.  In  other  words,  it  must  provide  feedback  to  the  errors  committed  by 
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the  learner  and  not  with  the  errors  that  he  or  she  could  possibly  make.  Moreover,  the  feedback 
must  be  intelligent:  it  should  explain  the  observed  rule,  show  why  the  answer  is  not  correct 
and  thus  allow  the  learner  how  to  avoid  the  same  mistakes  in  the  future.  Finally,  to  prevent 
learners  from  memorizing  their  mistakes,  feedback  must  be  immediate. 

4.  IDIOMATIC  : a concrete  tool 

IDIOMA-TIC  is  a language  learning  software  developed  at  KULAK  by  a team  led  by  prof.dr. 
Piet  Desmet.  It  is  both  an  "authoring  tool"  and  a “language  learning  environment”:  it  allows  a 
(language)  teacher  to  create  exercises  that  can  then  be  presented  to  learners.  Idioma-tic  is 
composed  of  two  modules:  an  input  module  and  an  output  or  learner’s  module. 

The  first  module  is  made  for  the  input  of  exercises  and  tests.  Input  is  user  fnendly  and 
requires  very  little  time,  thanks  to  some  system  functions,  like  the  intelligent  encoding  of 
answers  and  the  reuse  of  input. 

From  a technological  point  of  view,  the  number  of  question  types  has  been  reduced  to  three. 
But,  actually,  each  of  these  three  types  allows  to  create  various  and  very  different  exercises. 
We  more  concretely  distinguish  multiple  choice,  fill-in-tho*  blank  or  correction  and 
translation  exercises. 

A navigation  tree  is  provided  (or  created  by  the  teacher)  to  attach  the  exercises  to.  By 
attaching  the  exercises  to  several  "branches"  of  the  tree,  the  author  allows  learners  to  consult 
the  Idioma-tic  database  in  various  ways.  The  student  can  then  choose  freely  between 
exercises  sets  according  to  themes  (like  exercises  on  "On  a journey",  or  “Family  and 
relations”),  to  the  grammar  (grammar  in  general  or  specific  grammatical  problems),  to 
specific  languages  domains  (like  legal  or  business  language)  or  to  lexical  difficulties  (such  as 
homonyms  and  expressions)  etc.  The  tree  facilitates  also  Ihe  exchange  of  exercises  among 
colleagues. 

The  majority  of  these  exercises  created  are  translation  exercises  and  corrections  or  fill-in-the- 
blanks,  exercises  which  support  by  their  half-open  character  the  transfer  of  knowledge  to 
competences. 

In  the  case  of  the  fill-in-the-blanks,  students  can  make  two  attempts  before  being  confronted 
with  the  correct  answer,  which  is  not  the  case  for  the  multiple  choice  exercises  where  the 
correct  answer  already  appears  after  the  first  attempt.  In  the  case  of  the  translation  exercises, 
the  number  of  attempts  is  theoretically  unlimited:  after  each  incorrect  answer,  the  system 
indicates  which  words  do  not  correspond  to  the  possible  answers.  The  student  decides  himself 
on  the  number  of  attempts  he  makes. 

For  the  three  exercises  types,  the  teacher  can  provide  hints.  A hint  is  presented  in  the  form  of 
asterisk.  Student  who  want  to  see  the  hint  just  needs  to  click  the  asterisk. 

For  the  moment,  correction  of  exercises  always  uses  the  comparison  (matching)  method, 
which  implies  that  the  authors  of  the  exercises  must  provide  all  ^e  possible  correct  answers. 
Currently,  we  are  evaluating  the  analysis  (parsing)  method  We  expect  this  method  to  be  able 
to  give  more  accurate  feedback  to  unpredicted  errors. 

Let  us  note  finally  that  the  errors  made  by  students  are  automatically  saved  by  the  program , so 
that  the  learner  can  repeat  the  exercises  which  he  had  not  answered  correctly.  The  exercises 
qualified  by  the  learner  as  stumbling  blocks  can  also  be  remade. 

Please  refer  to  http://impuls.kulak.ac.be/idioma-tic 
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Information  and  communication  technologies  are  obviously  much  more  than  a delivery  tool.  They  are  definitely 
changing  the  way  we  teach  and  learn. 

Whatever  direction  these  changes  will  take,  teachers  and  educators  will  be  at  the  heart  of  it  and  thus  should  have 
the  opportunities  to  take  part  in  this  revolution  from  the  start. 

A generic  model  -The  TAMID  Model-  for  the  professional  development  of  in-service  teachers  has  been  developed 
for  this  purpose  by  the  Open  University  of  Israel.  ( Yaffe,  1996,  1998, 2002).  This  model  focuses  on  teachers  as 
professionals  who  are  obliged,  not  only  for  knowledge  and  expertise  in  subject  matter  and  didactics,  but  in  the  new 
technologies . They  have  to  master  them  not  by  learning  on  them,  but  by  using  them  for  their  on-going  professional 
needs. 

TAMID  is  base  on  the  Internet  as  the  leading  technology  and  distance  teaching  strategies  as  the  core  methodology 
for  teachers  development. 

It  operates  simultaneously  as  a Virtual  School  for  teachers,  and  as  their  Professional  Club.  In  this  way  it  is  offering 
teachers  an  innovative  framework  for  professional  growth  which  facilitates  and  encourages  continuous  learning  and 
proper  preparation  for  teaching  in  theclasses . 

As  a virtual  school  TAMID  offers  to  the  teachers  on-line  distance-learning  courses  in  a variety  of  Curriculum- 
based  topics  and  a variety  of  educational  means:  Text,  video,  audio,  face-  to  -face  meetings,  satellite  lectures  etc,  are 
used  to  deliver  on-line  courses. 

As  a professional  club  TAMID  offers  the  teachers  on-going  access  to  experts  from  the  open  University  and  other 
Universities,  presenting  them  with  authoritative,  expert  advice  on  the  various  subjects  relevant  for  their  teaching.  In 
addition  TAMID  provides  the  teachers  with  a framework  for  peer  discussions  and  cooperative  constraction  of  a 
bank  of  activities  and  ideas,  the  help  of  an  information  specialist  and  access  to  new  updated  and  relevant 
information. 

A typical  on-line  TAMID  course  usually  includes: 

F2F  Meetings:  Dedicated  to  enrichment  lectures,  workshops  and  hands  on  activities. 

The  Courses  Web  Site  containing  relevant  materials  : reading  materials , bibliography,  current  news,  a bank  of 
class  activities  and  ideas,  discussion  groups  etc. 

Assignments:  Relevant  to  the  course’s  goals  and  to  the  teachers’  needs. 

On-line  Discussion  Groups:  for  ongoing  discourse  with  the  course’s  tutors,  with  colleagues  and  with  academic 
experts  for  the  exchange  and  the  dissemination  of  information,  ideas  and  other  enlightening  information,  which  can 
be  integrated  into  teaching. 

TAMID  is  operating  since  the  last  year  as  a joint  venture  with  the  Center  of  Educational  Technology  (CET)  and  the 
Ministry  of  Education  . Six  communities  of  teachers  are  presently  active  in  this  program:  Science  teachers, 
Democracy  and  Civic  education  teachers,  English  teachers.  History  teachers,  Reading  skills,  and  teachers  who  focus 
on  New  Pedagogy.  Each  of  the  communities  has  its  own  needs  for  professional  development  and  its  own  learning 
environment  and  t raining  agenda,  however  the  underling  model  remained  basically  the  same. 

Within  this  framework,  different  models  of  on-line  courses  aiming  to  facilitate  professional  growth  of  teachers  have 
been  developed  and  examined: 

A.“Teaching  Together  But  Apart”.  Teachers  in  this  program  selected  a specific  subject  for  teaching  in  their 
classes  during  the  time  of  the  course,  (Astronomy  and  Civic  studies  were  the  candidates  in  this  case  study)  and 
constantly  reported  to  the  course's  web -site  on  progress,  success  and  difficulties  encountered  in  the  classroom.  The 
teachers  taught  their  own  program  to  meet  their  pupils*  needs,  and  the  conditions  in  his  classroom.  However  they 
enjoyed  the  benefits  of  belonging  to  a group  of  peers  similarly  engaged,  and  of  receiving  professional  expert 
information,  subject-related  aid  , support  and  much  empathy. 
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B.  Teachers  as  Developers.  In  this  model  teachers  developed,  along  with  the  guidance  and  support  from  distant 
experts,  sets  of  Internet-based,  learning  activities  for  their  students. 

C.  Pedagogy  of  Science  Education.  Using  the  network  to  discuss  theory  and  practice  of  Science  Education  . 

One  of  the  main  advantages  and  specialties  of  the  TAMID  model  is  the  direct  and  ongoing  access  to  University 
Faculty  who  act  as  the  academic  experts  of  the  teachers'  center.  They  are  responsible  for  the  academic  aspects  of  the 
on-line  courses,  for  the  academic  control  of  the  teaching  materials  delivered  to  the  teachers  and  they  are  giving  the 
teachers  an  academic  backing  on  subject  matter  issues. 

These  aspects  of  faculty  involvement  in  teachers  development  will  be  demonstrated  by  examples  from  two  fields  of 
teaching:  Astronomy  and  Microbiology.  Those  two  fields  are  putting  special  demands  on  the  teachers  as  they  are 
both: 

1.  Rapidly  changing  and  c all  for  regular  updating  of  knowledge. 

2 Conceptually  difficult  for  the  students  and  teachers  might  need  help  in  their  teaching. 

3.  Widely  represented  in  the  news  and  over  the  Web  and  call  for  classroom  discussions 

as  well  as  for  an  expert's  eye. 

Another  aspect  of  the  accesses  to  experts  is  the  asking  and  answering  of  questions.  It  is  a continuous,  evolving 
process  that  involves,  on  the  students’  side,  the  overcoming  of  social  barriers  ("my  ignorance  displayed  for  all  to 
see").  The  type  of  questions  may  vary  form  requiring  factual  information  to  asking  for  additional  explanations  and 
may  range  from  purely  subject  matter  to  general  pedagogy.  On  the  expert  side,  answering  requires  the  adaptation  of 
a different  writing  and  teaching  style.  Answers  should  not  relate  only  to  the  subject-matter  per-se,  but  should  also 
reflect  a broader  view  of  the  course  content,  the  school  curriculum  and  the  classroom  application.  Answers  should 
always  include  the  broad  aspects  of  the  topic  involved  and  ideally  would  become  a resource  for  the  course 
participants  to  use. 

An  additional  aspect  of  being  an  on-line  expert  is  the  ability  to  update  and  involve  teachers  in  the  latest  scientific 
information  available  that  is  relevant  to  subjects  they  teach  in  schools.  Images  from  spacecrafts,  reports  of  new 
discoveries  in  the  genome  project,  on-line  articles  and  publications  in  major  journals  are  all  bringing  "out-of-the- 
oven"  materials.  Summaries,  explanations  and  experts’  "did  you  know  that"  approach  enable  the  teachers  to  feel  that 
they  are  being  brought  up  to  date  in  all  the  important  scientific  discoveries.  While  most  of  the  text  books  used  in 
schools  are  usually  a couple  of  years  old,  the  contact  with  on-line  experts  of  various  fields  helps  the  teachers  to  keep 
abreast  of  the  rapidly  changing  knowledge  that  they  must  transfer  to  their  pupils.  The  ability  to  receive  immediate 
replies  from  a reliable  experts  helps  the  teachers  to  encourage  their  students  to  ask  "any  question  hey  can  think  of" 
knowing  that  even  if  they  will  not  be  able  to  give  the  right  answer  on  the  spot,  they  will  be  able  to  return  with  an 
answer  from  a first  class  experts  within  days. 

By  using  concrete  examples  from  the  teachers’  center  we  will  demonstrate  how  this  model  of  linking  the  teachers  to 
faculty  is  actually  working,  and  put  forward  some  of  the  questions  that  we  are  facing:  What  are  the  added  values  of 
such  contacts?  Who  benefits  from  them?  What  are  the  conditions  for  a fruitful  connection?  what  are  the 
expectations  of  the  "connected  experts"?  What  are  their  obligations?  How  to  choose  an  on-line  expert,  what  should 
be  his  profile?  And  what  are  the  implications  of  these  option  within  the  formal  educational  framework? 

Our  underlying  premise  of  these  models,  which  has  been  confirmed  over  the  past  six  years,  is  that  teaching 
individually  yet  together  with  a group  of  colleagues  and  with  support  from  experts,  is  potentially  more  meaningful, 
more  interesting  and  more  enjoyable  both  to  teachers  and  to  their  pupils. 

We  believe  that  this  new  paradigm  for  Teachers’  training  which  is  in  high  relevance  to  teacher’s  work  and  offering 
strong  connections  to  colleagues  and  experts,  is  highly  advantageous,  and  most  suitable  for  the  professional  growth 
of  teachers. 
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Abstract:  Video  films  have  pedagogical  strength  as  they  can  provide  instruction  through  sound,  motion,  and 
visuals.  However,  the  volatile  nature  of  live  distance  lessons  as  well  as  the  rigidity  of  analog  and  linear  video  has 
limited  its  effective  use  for  teaching  and  learning.  Digital  video,  allows  for  a shift  from  a linear  and  passive  mode  of 
use,  to  a modular,  and  interactive  mode.  The  Open  University  of  Israel  (OUI)  has  developed  simple  user-friendly 
tools  and  interfaces,  which  hold  the  promise  for  more  effective  teaching  and  learning.  Several  differert  formats  of 
using  digital  video  will  be  demonstrated  in  this  presentation  and  discussed  in  the  light  of  their  pedagogical  potential 
for  Distance  Learning. 

Introduction 

Video  has  special  strengths.  It  usually  contains  a combination  of  motion,  sound  and  visuals  and  is  thus  particularly 
suitable  for  teaching  and  learning.  It  is  accepted  that  video  can  help  students  envision  new  ideas,  help  them  perceive 
the  reality  around  them,  and  inaease  motivation  for  learning.  Moreover,  video  can  serve  to  capture  lectures  and 
lessons  for  later  distribution  and  use. 

Video  indeed  has  been  used  for  many  years  as  a teaching  tool  at  all  levels  to  transmit  and  describe  processes,  events, 
objects,  experiments,  ideas  etc. 

The  Open  University  of  Israel  (OUI),  which  is  a distance  learning  institution,  has  been  using  video  almost  from  its 
start.  Academic  video  films  are  quite  commonly  sent  to  students  as  part  of  the  learning  package.  In  addition, 
interactive  video  lessons  are  transmitted  via  satellite  to  classes  all  over  the  country  to  convene  first  class  lessons  to 
all  students.  These  lessons  are  recorded  for  students  who  could  not  attend  the  live  satellite  lesson.  However,  both 
modes  have  severe  weakness  that  limits  their  contribution  to  the  learning  process.  Analog  videos  are  linear,  non- 
interactive,  and  expensive  to  produce  and  distribute.  Analog  video-films  do  not  encourage  active  learning  neither 
interaction  with  the  learning  materials,  which  is  so  important  for  learning.  On  the  other  hand,  interactive  video  is 
volatile  and  fades  away  after  the  lesson  is  completed. 

Digital  video  has  many  advantages  in  regard  to  teaching  and  learning.  Payne,  H (2002).  It  is  easy  to  produce,  to 
index,  to  segment  and  to  distribute,  and  might  encourage  active  and  interactive  learning  . Digital  videos  can  be  used 
for  sessions  of  varying  length  and  different  settings.  Combining  digital  video,  the  Internet  and  other  media  opens  up 
a new  avenue  for  teaching  and  learning.  (Storm  2001) 

During  the  last  yearthe  Open  University  of  Israel  has  beenutilizhg  digital  video,  combined  with  suitable  software 
and  a user-friendly  interface,  which  have  been  developed  at  the  OUI.  These  videos  serve  not  only  to  convey  the 
learning  material,  but  also  to  add  a human  face  to  the  teacher -student  communication  along  the  course;  through  the 
web -site  of  the  course.  This  human  face,  called  by  others  intimacy  and  immediacy  was  found  to  be  positively 
connected  with  students'  motivation  and  cognitive  learning  (LaRose,  R.  &Whitten,  P.  (2000),  Rodriguez,  J.  1., 
Plax,T.  G.,  & Kearney,  P.  (1996) . 


Thus  a set  of  tools  is  embedded  with  the  video  into  the  students  working  space  which  enables  the  student  to  search 
the  video,  move  between  topics,  add  personal  annotations,  and  bookmarks,  and  move  using  hyperlinks  to  other 
resources. 

Several  different  formats  of  using  digital  video  in  distance  teaching  will  be  presented  and  discussed: 

Digital  Video  Formats 

The  following  families  of  video  formats  are  tested  at  the  OUI  at  present: 

Long  Video  Lessons,  are  produced  as  substitutes  of  the  live  face-to-face  lectures  or  tutorials.  The  face-to-face 
encounter  and  dialogue  between  the  expert  and  the  student  has,  since  the  beginning  of  education,  been  a central 
feature  in  the  learning  process.  Besides  the  imparting  of  knowledge  by  the  expert,  a "live"  encounter  can,  at  its  best, 
produce  productive  intellectual  brainstorming,  in-depth  thinking  and  inspiration  and  offer  a personal  example  and 
social  support  for  the  learning  process. 

However  the  "live"  encounter  has  also  inherent  limitations.  They  are  limited  by  time  and  location  which  is 
problematic  for  distance  learners,  and  thty  are  volatile.  Once  deliverd  they  are  gone.  The  aim  of  the  video  lessons 
is  to  overcome  this  limitations. 

Teaching  Clips  are  short  video  segments  (1-10  minutes  long).  Most  of  these  are  short  "talking  head"  clips  of  a 
faculty  member  discussing  some  aspect  of  his  course  or  just  greeting  his  students.  The  main  purpose  of  these  clips  is 
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to  convey  elements  of  intimacy  and  immediacy  to  the  teacher-student  communication  along  the  course  through  the 
web -site  of  the  course. 

Video-Rich  Multimedia  Lessons  are  complex  constructs  containing  an  integrated  combination  of  video,  text, 
graphics  and  animations,  tools  for  active  learning  annotations,  bookmarking,  links  to  discussion  groups  etc. 

All  types  of  video  might  be  used  in  a single  program. 

The  digital  video  with  or  without  the  Web-based  materials  can  also  be  stored  and  distributed  on  a CD-ROM.  Then 
Internet  users  could  also  employ  the  CD-ROM  and  Web  material  in  an  integrated  way. 

In  our  presentation  we  will  demonstrate  this  formats  and  discuss  some  of  options  for  what  the  video  media  might 
look  like  in  the  educational  framework.  We  will  also  sort  out  and  analyze  families  of  products  that  might  prove  both 
valuable  and  viable  (as  evidenced  by  extensive,  long  term  use). 

By  using  these  formats,  it  is  possible  to  engage  the  student  in  active  learning.  Students  can  annotate,  summarize  and 
seai'ch  through  a video  film,  construct  their  own  path  of  learning 

From  the  instructor's  point  of  view , digital  video  technologies  enable  simple  production  of  video  lessons,  which  can 
be  connected  simply  to  other  media  (text,  Internet,  multimedia).  The  tool  also  allows  instructors  to  prepare 
interactive  questions  and  assignments  pertaining  to  the  film  watched. 

From  the  students  point  of  view  digital  video  can  be  used  as  a self-study  modular  and  interactive  resource.  Linearity 
is  broken  by  allowing  immediate  access  to  relevant  sections  in  the  film  by  clicking  on  the  list  of  segments  Attaching 
annotations  provides  for  reflection  on  important  points  in  the  film. 

We  believe  that  integrating  digital  video  into  teaching,  according  to  the  formats  that  will  be  described  in  our 
presentation,  can  improve  teaching  and  learning,  and  will  affect  students’  motivation  attitudes  and  achievements. 
Several  experiments  are  planed  aiming  to  evaluate  the  various  formats  of  digital  video  that  have  been  developed. 

The  evaluation  plan  and  considerations  will  be  discussed  elsewhere. 

Primary  results  show  that  students  are  very  satisfied  with  the  integration  of  video  elements  into  the  teaching 
package.  In  particular  they  expressed  satisfaction  of  the  possibility  to  return  to  recorded  live  lessons  and  used  them 
mainly  to  prepare  for  final  examination.  Satisfaction  was  also  noticed  whenever  short  video  clips  containing 
greetings,  explanations  and  clarifications  were  integrated  into  the  courses'  web -sites. 

We  believe  that  the  shift  from  analog  to  digital  video  and  the  rapid  changes  in  information  technology,  which 
supports  video  hosting  and  distribution,  are  about  to  intensify  the  use  of  video  in  education. 
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Abstract:  Education  is  moving  towards  revenue  generation  from  such  channels  as  electronic 
learning,  distance  learning  and  virtual  education.  Hence  learning  technology  standards  are  critical  to 
the  sector’s  success.  Existing  learning  technology  standards  have  focused  on  various  topics  such  as 
meta-data,  question  and  test  interoperability  and  others.  However  it  is  believed  that  the  meta-data 
standards  play  an  important  role  in  the  success  of  electronic  leamin  g.  This  is  due  to  the  fact  that 
meta-data  is  Ae  standard  for  learning  object;  which  is  the  main  concept  that  allows  interoperability 
and  reusability  to  occur.  Although  bodies  or  consortia  such  as  IEEE  LTSC  {Learning  Technology 
Standards  Committed),  IMS  (Instructional  Management  System)  and  others  had  developed  the 
meta-data  standards,  there  is  concern  about  the  features  embedded  in  the  meta-data  standards.  It  is 
believed  that  to  achieve  a more  meaningful  learning  process,  a learning  object  needs  to  comprehend 
more  than  technical  features.  Features  such  as  pedagogic,  community  and  context  are  important,  as 
they  will  provide  a more  far-reaching  description  of  what  the  learning  object  is  about.  Thus  the 
objective  of  the  research  is  to  obtain  the  detail  elements  of  the  meta-data  standards  and  enhance  it 
by  inserting  the  necessary  elements  related  to  the  above  features.  The  learning  theories  such  as 
Instructional  Design  Theory,  Constructivism  Instructional  Design  Theory  and  Design  Potential 
Approach  were  used  to  derive  the  elements  associated  to  the  above  features.  The  results  of  this 
study  are  in  the  form  of  elements,  which  can  be  embedded  into  the  existing  standards. 


Introduction 

The  key  to  make  e-leaming  successful  is  depending  in  the  thriving  of  learning  object  design.  This  is  due  to  the 
fact  that  learning  object  is  the  core  concept  of  electronic  learning  where  it  is  regarded  as  the  basic  ingredients 
that  allows  flexibility,  customisation  and  interoperability  to  take  place.  With  the  use  of  learning  object, 
recombination  of  material  at  any  level  may  occur  thus  increases  the  value  of  the  content  and  facilitates  the 
competency -based  learning.  The  learning  object  definition  abound:  Learning  object  is  any  entity,  digital,  non 
digital  which  can  be  used,  reused  or  referenced  during  technology  supported  learning  (IEEE  Standards, 
2000);  Learning  object  is  any  digital  resources  that  can  be  reused  to  support  learning  (Wiley,  2000) 

We  propose  the  following  working  definition:  Learning  objects  are  discrete,  focused,  interactive  digital 
entities,  which  can  be  used,  reused,  searched,  referenced  to  support  the  learning  process. 

Realising  the  importance  of  learning  object,  various  consortia  such  as  IMS,  IEEE  LTSC,  ARIADNE  {Alliance 
Of  Remote  Instructional  Authoring  And  Distribution  Networks  For  Europe)  and  others  had  made  an  effort  to 
develop  the  learning  object  standards  or  better  known  as  metardata  standards.  The  standards  derived  by  these 
consortia  define  the  conceptual  structure  for  meta-data.  Most  of  these  standards  provides  the  technical 
overview  of  what  learning  object  should  contained  and  aspects  of  how  meta-data  should  be  structured.  For 
example,  IMS  defines  the  conceptual  structure  for  meta-data  where  it’s  information  model  was  based  on  a 
structure  of  defined  elements  that  describes  or  catalogues  the  learning  resource.  The  model  was  formed  on  a 
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hierarchy  manner  and  has  nine  main  components,  which  are  (IMS,  2001):  general,  lifecycle,  meta  meta-data, 
technical,  educational,  rights,  relation,  annotation,  classification  Each  of  this  main  component  has  it’s  sub- 
components. For  example,  the  general  component  has  the  identifier,  title,  catalogentry,  language,  description, 
keyword  and  others  as  it’s  sub-component.  The  structure  of  the  metadata  information  model  is  given  below 
(IMS,  2001): 


“Root” 


“Branches” 


“Leaves”  (Examples) 


Figure  1:  Root  to  leaf  tree  view  of  metadata 

Most  of  the  other  standards  had  similar  edifice  like  the  above;  although  some  might  have  a slight  different 
structure  depending  on  the  primary  goal  of  the  standards.  ARIADNE  (2001)  for  example,  is  more  focused  to 
the  educational  needs. 

Looking  at  the  structure  and  its  sub- elements  it  is  apprehend  that  the  standards  are  more  focused  towards  the 
technical  aspects  of  the  learning  object.  Although  the  standards  had  put  emphasis  on  the  pedagogic  portion; 
where  they  have  the  educational  and  pedagogic  elements  but  little  was  given.  The  prominence  was  more 
towards  describing  the  interactivity  type,  intended  user  role,  age  range,  difficulty  level,  context,  learning  time 
and  others.  Less  emphasis  was  given  on  how  a learning  object  may  assist  effective  learning,  how  it  may 
provide  the  specific  community  users  and  how  it  may  impart  the  context  of  instructional  use  of  the  object. 
Apart  from  that,  there  is  lack  of  description  regarding  the  learning  activities  surrounding  the  learning  object 
and  the  support  for  the  reuse  of  the  object  within  specific  instructional  context  (Recker,  2000).  These  short  - 
comings  are  the  basis  that  initiates  the  work. 


Current  Work. 

The  work  carried  out  is  focusing  on  how  elements  such  as  context,  pedagogy  and  community  can  be 
embedded  into  the  learning  object  standards.  Recker  (2000),  Wiley  (2000),  Boyle  (2001),  Suthers  (2001),  and 
others  had  carried  out  similar  work,  which  shows  the  importance  of  having  context  and  pedagogic  aspects 
attached  to  the  meta-data  structure.  However  the  details  of  each  element  and  examples  of  what  can  be  included 
into  the  standards  were  not  given.  The  aim  of  the  study  was  to  provide  details  of  the  elements  that  need  to  be 
attached  to  the  meta-data  standards  in  order  to  make  them  more  efficient.  The  work  used  a comparative 
approach  which  involved  looking  into  several  learning  theories  to  derive  the  elements  related  to  context, 
pedagogy  and  community.  The  elements  derived  are  then  compared  to  the  existing  elements  in  the  standards  to 
see  whether  the  standards  had  the  procured  elements.  The  method  carried  out  in  this  research  is  as  below 
(figure  2). 

The  elements  mentioned  are  believed  to  be  important  as  without  context,  learning  object  is  nothing  more  than 
a clip  art  item,  it  can  be  misleading,  confusing  and  utterly  meaningless  ( Longmire,  2000).  Gillroy  (2001)  had 
stated  that  learning  is  fundamentally  both  social  and  experiential. 
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Figure  2:  Research  Process 


Therefore  to  make  learning  more  pleasurable,  the  focus  should  not  only  be  on  content  but  also  on  the 
organisation  of  learning  experience.  As  a result  the  pedagogical  process  becomes  the  most  important  factor  in 
the  design  of  learning.  To  have  a full  understanding  on  why  these  elements  are  believed  to  be  important  we 
need  to  look  at  the  basis  of  each  element. 


Context 

Context  is  a difficult  term  to  explain:  basically  it  means  the  environment  in  which  the  learning  unit  is  situated. 
It  portrays  the  situation  from  the  learner’s  point  of  view.  It  is  an  abstract  representation  of  the  relevant 
environment.  It  then  guides  adaptive  action  in  that  environment,  i.e.  what  type  of  learning  actions  to 
undertake  (Boyle,  2000).  The  context  consists  of  the  framing  of  content  along  with  associated  interactivity  and 
it  is  important  in  making  learning  happen  within  activity  rich,  interaction  rich  and  culturally  rich  social 
environments  (Afonso,  2000;  Boyle,  2000).  Hence  in  order  to  construct  learning  context,  it  is  believed  that  it 
will  involve  content  structuring,  interactivity,  compositional  framework  and  usage  of  content.  As  mentioned 
without  context,  learning  objects  can  be  confusing,  misleading  or  utterly  meaningless.  Therefore  it  is 
important  to  have  context  embedded  as  one  of  the  elements  in  the  learning  object. 

Pedagogy 

Pedagogy  means  the  science  of  teaching  or  educating  children  and  is  used  as  a synonym  for  teaching  (Oxford 
Dictionary,  2002;  Wave  Technologies,  1996).  It  is  the  approach  that  the  teacher/tutor  use  in  constructing  the 
courses  to  aid  cognition  through  different  learning  styles.  Consequently  it  embodies  teacher  focused  education. 
In  the  pedagogic  model,  teacher  assume  responsibility  for  making  decisions  about  what  will  be  learned,  how  it 
will  be  learned  and  when  it  will  be  learned  (Conner  1996).  Pedagogy  is  important  as  it  shows  and  directs  on 
what  can  be  learned,  how  to  learn  and  when  to  learn.  As  a result  this  needs  to  be  included  in  the  learning 
object  as  it  will  help  the  learners  to  appreciate  the  learning  process,  to  be  able  to  use  the  learning  object 
effectively,  to  mark  the  choices  of  content  structuring.  With  pedagogy  embedded  in  it,  it  is  believed  that  the 
learning  object  can  be  customised  to  the  individual  needs. 

Community 

A community  is  a group  of  individuals  who  have  common  values,  norms  and  meanings,  a shared  history  and 
identifications  within  a particular  culture.  The  bonds  that  exist  between  communities  are  believed  to  go 
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beyond  the  instrumental  into  the  realm  of  affection  and  exclusive  (Gillroy,  2001).  Consequently  it  is 
significant  to  have  a community  feel  when  using  the  learning  object  in  order  to  make  the  learning  process 
more  effective. 

Looking  at  the  above  facts  all  the  components  (context,  pedagogy,  community)  are  believed  to  add  value  to 
learning  object  and  it  will  help  to  make  learning  through  electronic  medium  more  successful.  It  would  greatly 
enhanced  the  educational  use  of  learning  objects  which  are  presently  pedagogically  limited  (Cowley,  2000) 
and  context  limited  (Gillroy,  2000;  Boyle,  2000) 


Outcome 

The  learning  theories,  which  were  used  in  the  work  were  the  Instructional  Design  Theory  - Dick  and  Carey 
Systems  Approach  Model  For  Designing  Instruction  (Dick  et  al,  2001),  The  Constructivism  Instructional 
Design  (Jonassen,  2001)  and  the  Design  Action  Potential  Approach  (Boyle,  1998).The  theories  were  chosen 
due  to  the  fact  that  they  are  the  basic  theory  to  the  design  of  instruction  in  the  computer  environment  and  each 
of  them  takes  a different  approach  to  model  the  instruction.  Dick  and  Carey  ID  Theory  for  instance  provide  a 
system  approach  model  for  the  design,  development,  implementation  and  evaluation  of  instruction.  The  theory 
is  less  complex  than  the  others  such  as  theories  by  Reigeluth  (1999)  and  Jonassen  (1997)  to  name  a few.  On 
the  other  hand,  the  Constructivism  Instructional  Design  uses  the  constructivist  conception,  which  is  believed 
to  be  the  learning  theory  behind  o learning.  The  Design  Action  Potential  Approach  uses  an  ‘action  potential’ 
approach  distinguishing  it  from  the  other  theories.  It  provides  a method  for  formalising  scattered  knowledge 
without  constraining  individual  design  decisions.  Each  of  these  theories  was  then  looked  into  and  analysed  and 
the  elements  related  to  the  above  elements  were  extracted.  Below  are  the  elements  extracted. 


Context 

Pedagogy 

Community 

1.  Types  of  leamer/level 
2 Information  about  the  learner 

1.  Objective  of  the  learning  object 
2 Factors  to  motivate  learners 

3.  Required  skills  needed 

4.  Language  and  vocabulary  needed 

-Nil 

Table  1:  Dick  and  Carey  Systems  Approach  Model  Elements 


Context 

Pedaeoev 

Community 

-Nil 

1.  Presentation  of  the  learning  object 

2.  Structure  of  the  presentation 

-Nil 

Table  2 Design  Action  Potential  Approach  Elements 


Context 

Pedagogy 

Community 

1.  Types  of  leamer/level 

2 Related  cases  that  support  understanding  of  the 
problem/  access  to  a set  of  related  experience 

3.  Problem  context 

4.  Problem  representation 

5.  Problem  manipulation 

d Activity  done  to  the  leami  ng  object 

1.  A list  of  objectives 

2.  Level  of  complexity  of  the 
learning  object 

3.  Pre-requisite  requirements 

4.  Information  resources 

1.  knowledge 

building 

communities 

Table  3:  Constructivism  Instructional  Design  Elements 

Each  of  these  elements  are  then  looked  into  and  compared  to  the  existing  standards.  This  is  accomplished  by 
giving  definition  to  each  elements  and  examples.  The  next  step  taken  is  by  matching  up  the  definition  and  the 
examples  obtained  to  the  similar  sub- elements  in  the  standards.  Some  of  the  elements  are  already  existed  in 
the  standards  but  it  is  believed  that  it  needs  further  clarification.  For  example: 
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Example  1: 


Current  work:  The  type  of  learner  element -This  element  is  the  classification  of  the  learner.  It  is  to  distinguish 
specific  type  of  learner  for  a specific  type  of  learning  object,, Example:  beginner/  intermediate/  expert 

Existing  Standards  (IMS,  IEEE  and  ARIADNE):  The  intended  user  role  element -This  element  is  described  as 
the  principal  user  for  which  the  learning  object  was  design.  Example:  teacher,  author,  manager  and  learner. 

It  appears  that  although  the  same  type  of  element  were  designed  for  learners  there  is  different  definition  to  it 
thus  providing  different  examples.  The  standard  bodies  might  want  to  consider  another  feature,  which  could  be 
added  to  the  existing  standards  that  might  resemble  the  current  work  type  of  learner  element. 

Example  2: 

Current  work:  Related  cases  and  information  resources  that  support  understanding  of  the  problem  -This 
element  is  the  access  to  a set  of  experience  that  the  learner  can  refer  to  as  this  will  provide  the  learners  with  a 
set  of  experience  to  compare /o  the  current  learning  object 

Existing  Standard  (IMS,  IEEE  and  ARIADNE):  The  existing  standards  do  not  embed  this  element. 

Example  3: 

Current  Information  about  the  learner- This  element  is  the  information  about  the  learner,  who  may  access 
or  use  the  learning  object.  It  is  consider  that  each  learning  object  has  it’s  target  users  and  the  target  user’s 
needs  to  have  certain  educational  levels  and  prior  knowledge  before  they  are  able  to  use  the  learning  object. 
Thus  the  learner  information  must  be  described  in  terms  of  prior  knowledge  needed  and  skills  required. 

Existing  standards  (IMS,  IEEE  and  ARIADNE):  The  closest  element  that  can  be  compared  to  is  the  intended 
user  role  element.  The  intended  user  role  element-This  element  is  described  as  the  principal  user  for  which  the 
learning  object  was  design.  Example:  teacher,  author,  manager  and  learner. 

From  the  above  examples,  it  is  clearly  shown  that  although  the  elements  might  share  the  same  designation; 
definition  differs  between  them,  hence  providing  different  types  of  examples.  It  is  realised  that  the  definition 
and  the  example  play  a crucial  role  for  each  element.  This  is  to  provide  a clearer  picture  of  the  element  and 
allows  a thorough  comparison  to  be  carried  out.  Further  analysis  is  also  needed  for  the  meta-data  standards  in 
order  for  it  to  comprehend  the  pedagogic  needs  therefore  making  the  learning  object  more  useful  and  more 
meaningful  to  the  learners.  Apart  from  that,  from  the  work  carried  out,  it  is  apprehend  that  there  are  more 
elements,  which  needs  t)  be  added  to  the  meta-data  standards.  If  all  of  these  elements  were  taken  into 
consideration  and  be  added  to  the  existing  standards,  learning  object  will  be  able  to  reach  it’s  maximum  value; 
where  it  does  not  only  make  electronic  learning  a reality  but  it  also  allows  learning  to  take  place  efficiently. 

Conclusion 

There  are  more  features  apart  from  technical,  which  needs  to  be  considered  when  deriving  the  meta-data 
standards.  Features  such  as  related  cases,  types  of  learners,  problem  manipulation  and  others  need  to  be 
considered  as  to  achieve  a more  comprehensive  learning  object  structure.  By  adding  these  elements,  which 
belongs  to  the  context,  pedagogy  and  community  elements  into  the  existing  standards,  it  will  certainly  make 
the  learning  object  more  meaningful  to  the  learners  in  the  future. 
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Abstract:  This  case  study  identifies  the  feasible  approaches  to  integrate  laptops  into  a science 
learning  environment  in  the  middle  school.  The  observation  of  learning  activities  and  teaching 
strategies  took  place  in  the  8th  grade  science  classroom  with  seventeen  students  for  five  class 
periods  and  the  data  from  teacher’s  opinion  gathered  by  interview.  The  results  report  the 
strategies  the  teacher  used  with  laptop  in  the  classroom  including  problem-based  learning, 
project-based  learning,  collaborative  learning,  hands-on  activities,  and  have  students  to  use 
laptop  as  a cognitive  tool.  With  appropriate  strategies,  laptop  can  be  used  as  a cognitive  tool 
and  enhance  the  possibility  of  shifting  teacher’s  role  from  a lecturer  to  a facilitator.  Integrating 
laptops  into  classroom  should  consider  the  following  issues:  make  use  of  laptop  as  cognitive 
tool,  employ  constructivism  approach,  provide  curriculum  support,  manage  logistics  of 
classroom,  and  reduce  distraction. 


Pervasive  computing  can  enable  learners,  teachers,  and  parents  to  have  continuous  access  to  computing 
technology.  By  taking  the  advantage  of  portable  technologies  such  as  laptops  to  establish  a pervasive  computing 
environment,  students  can  be  empowered  to  learn  with  technology  anytime,  anywhere.  Recognizing  the  potential 
benefits  of  pervasive  computing,  a coeducational,  independent,  college  preparatory  day  school  in  the  southeastern 
region  of  the  United  States  has  worked  for  two  years  to  integrate  laptop  computers  into  middle  grade  core  subjects 
(language  arts,  social  studies,  math,  and  science)  with  a wireless  networking  infrastructure. 

Information  technology  has  been  shown  to  motivate  students  and,  computer-assisted  learning  can  make  a 
notable  addition  to  learning  and  understanding  science  concepts  but  its  value  will  lie  in  apt  use  and  this  will 
depend  upon  teacher  judgment  and  expertise  (Rodrigues,  1997).  Successful  implementation  of  technology  in  an 
education  setting  requires  more  than  merely  providing  resources  and  a hospitable  environment.  Collaboration 
among  the  participants  is  an  essential  feature  of  the  formula.  Collaboration  in  itself  does  not  lead  to  success  in 
achieving  change  if  those  involved  do  not  embrace  a shared  vision  or  a perceived  need  of  technology 
implementation.  Moreover,  integrating  technologies  with  education  requires  a reconceptualization  of  the 
classroom  environment  and  the  teaching  and  learning  strategies  employed  therein  (Woodrow  et  al.,  1996). 

Aware  of  the  laptop  successes  and  challenges  documented  at  other  schools,  the  technology  leaders  at 
the  school  used  in  this  study  contacted  external  evaluators  to  conduct  a long-term  evaluation  of  the  use  of  portable 
technologies  in  their  school  (Hill,  et  al.,  2001).  The  findings  of  the  evaluation  from  the  last  two  years  indicates 
that  even  though  teachers  were  enthusiastic  about  using  laptop  into  their  classroom,  some  of  them  felt  frustrated 
because  they  perceived  a need  for  more  practical  strategies  about  how  to  integrate  the  laptop  into  the  learning 
environment.  As  a component  of  the  overall  evaluation  project,  this  studysought  to:  1)  identify  learning  activities 
that  use  the  laptop,  and  2)  identify  the  strategies  the  teacher  uses  during  the  learning  activities  in  the  learning 
environment  to  integrate  the  laptop. 
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The  description  of  the  problem 


During  the  Spring  of  2000,  the  school  offered  a series  of  workshops  (known  as  20-Minute  Modules)  to 
prepare  middle  school  teachers  and  administrators  to  participate  in  a wireless  laptop  learning  environment.  While 
the  workshops  were  useful,  several  suggestions  were  offered  during  the  teacher  interviews  at  the  end  of  the  year. 
One  teacher  suggested  that  the  technology  training  and  the  strategies  of  teaching  learning  should  be  separated. 
Another  teacher  mentioned  a need  for  personalized  training  to  match  her/his  curriculum.  Yet  another  teacher 
stated  they  needed  training  on  technology  integration.  While  training  received  to  date  has  been  useful,  it  appears 
that  the  teachers  are  still  trying  to  figure  out  the  feasible  approaches  to  integrate  the  new  portable  technology  into 
the  learning  environment. 

One  area  that  holds  particular  promise  in  terms  of  technology  integration  is  science  computer 
applications  in  science  education  could  include  word-processing  reports,  using  data-loggers,  spreadsheets, 
databases,  simulations,  multimedia  and  the  Internet,  for  data  collection  and  analysis  (Rodrigues,  1997).  For 
example,  by  allowing  pupils  to  carry  out  complex  calculations  and  present  the  results  quickly  and  easily,  they  are 
able  to  concentrate  on  the  important  scientific  principles  underlying  their  work  rather  than  on  the  tedium  of  the 
number  crunching  that  can  so  often  get  in  the  way  of  understanding  (Carson,  1997).  In  addition,  modeling  could 
be  used  for  investigations  and  multimedia  authoring  for  communication  information.  For  example,  by  creating  a 
dynamic  computer  model  of  the  thing  to  be  studied,  and  manipulating  that  model  to  changes  and  new  influences, 
student  gain  a sense  of  how  the  thing  itself  works  (Carter,  1998). 


Research  design 

Based  on  the  many  opportunities  of  integrating  the  laptop  into  the  science  learning  environment,  this 
study  sought  to  develop  a strategy  to  help  teachers  systematically  and  successfully  integrate  the  laptop  in  the 
middle  school  to  enhance  student  learning.  This  study  focused  on  one  classroom  of  one  eighth  grade  science. 
Students  and  the  teacher  have  their  own  laptops  and  they  can  access  to  the  Internet  anytime,  anywhere  with  their 
laptops.  The  classroom  context  is  a science  class  covering  ecology,  earth  science,  and  physics.  The  class  meets 
five  days  a week  during  the  semester.  Participants  study  astronomy  unit  when  the  researcher  doing  observation. 
Seventeen  8*^  grade  students  from  the  same  classroom  of  the  science  teacher  were  selected  to  participate  in  this 
study.  The  majority  of  the  students  received  their  laptops  two  years  ago  and  are  computer  literate,  (note:  some 
students  are  new  to  the  academy  so  have  not  had  their  computers  for  two  years)  The  teacher  has  7 years 
experience  of  teaching  science  and  two  years  experience  of  using  the  laptop  into  the  learning  environment. 

The  design  of  this  study  is  a case  study,  utilizing  qualitative  research  methods  because  the  researcher 
wants  to  generate  data  rich  in  detail  and  embedded  in  the  context.  According  to  Stake  (1994),  in  the  intrinsic  case 
study,  no  attempt  is  made  to  generalize  beyond  the  single  case  or  even  to  build  theories.  However,  as  Jennifer 
Mason  puts  it: 

/ do  not  think  qualitative  researchers  should  be  satisfied  with  producing  explanations  which  are 
idiosyncratic  or  particular  to  the  limited  empirical  parameters  of  their  study...  Qualitative  research 
should  [therefore] produce  explanations  which  are  generalizable  in  some  way.  or  which  have  a wider 
resonance  (1996:6). 

Purposive  sampling  guided  by  time  and  resources  is  a positive  answer  to  the  question  of  how  we  can  obtain 
generalizability.  Purposive  sampling  allows  us  to  choose  a case  because  it  illustrates  some  feature  or  process  in 
which  we  are  interested  (Silverman,  2000).  As  Denzin  and  Lincoln  put  it: 

Many  qualitative  researchers  employ... purposive,  and  not  random,  sampling  methods.  They  seek  out 
groups,  settings  and  individuals  where... the  processes  being  studied  are  most  likely  to  occur 
(1994:202). 

The  researcher  focused  on  a single  case  by  purposive  sampling  because  unique  qualities  of  the  learning 
environment  can  promote  understanding,  inform  practice  for  similar  situation  and  allow  access.  Due  to  identify 
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the  teaching  strategies  the  teacher  employs  in  the  learning  environment,  on-site  observation  of  the  activities  with 
the  laptop  and  interview  with  the  teacher  are  selected. 

This  study  gathered  the  qualitative  data  by  two  primary  methods:  1)  conducted  observation  in  the 
classroom  to  see  how  the  middle  school  science  teacher  integrates  the  laptop  into  the  learning  environment  and 
the  reactions  of  students  with  observation  protocol,  and  2)  an  interview  with  the  middle  school  science  teacher  to 
gather  detailed  information  about  the  experience.  Other  sources  of  data  were  also  used  to  inform  the  results  of  the 
study;  including  surveys  from  the  overall  evaluation  effort  and  informal  discussions  with  the  overall  evaluation 
team. 


Data  analysis 

The  researcher  used  descriptive  statistics  to  include  frequency  of  activities  occurred  in  the  baming 
environment  and  activity  structure . The  data  from  field  observations  were  analyzed  based  on  the  categories  in  the 
observation  checklist.  The  interview  session  was  recorded  on  audiotape.  The  transcribed  teacher  interview 
protocol  was  compared  with  the  data  drawn  from  observations  and  surveys.  The  interview  was  analyzed  by  the 
inductive  approach  and  themes  were  identified.  The  data  of  surveys  done  by  the  external  evaluation  team  from  the 
last  two  years  of  the  four-year  longitudinal  evaluation  about  the  science  teacher’s  view  on  teaching  and  learning, 
computer  use  for  teaching,  and  the  descriptions  of  his  teaching  and  learning  environment  were  synthesized  with 
the  data  from  observations  of  actual  using  the  laptop  in  the  learning  environment.  The  teacher’s  self-description  of 
the  experience  was  gathered  from  the  interview. 

There  are  some  limitations  might  limit  the  validity  and  reliability  of  the  results  of  this  study.  First,  the 
observation  time  was  limited  due  to  limited  personnel;  the  researcher  was  not  able  to  participate  in  each  science. 
Second,  any  possible  generalization  of  the  results  is  limit  to  the  middle  school  in  science.  Third,  the  results  may 
not  apply  to  the  teacher  who  is  not  equipped  with  the  high  computer  literacy  because  the  teacher  in  this  middle 
school  has  been  through  training  in  using  a laptop. 


Results 

The  case  study  revealed  some  interesting  results.  One  key  to  successfully  integrating  the  laptop  into 
learning  environment  is  how  the  strategies  are  being  employed.  In  this  study,  the  strategies  the  teacher  used  with 
the  laptop  included  problem-based  learning,  project-based  learning,  collaborative  learning,  and  hands-on 
activities.  The  teacher  also  had  students  using  the  laptop  as  a cognitive  tool,  representing  their  knowledge  with 
some  application  such  as  creating  a Webpage,  or  a PowerPoint  presentation,  or  spreadsheet  graphics.  Most  of  the 
strategies  that  the  teacher  used  also  clarified  the  result  of  literature  review  (Wiesenmayer  & Koul,  1998;  Woodrow 
et  al.,  1996;  Scanlon,  1997;  Devitt,  1997)  including  the  value  of  an  inquiry -based  approach,  project-based 
activities,  collaborate  learning,  and  using  a variety  of  resources  available  through  the  Internet  to  integrate 
technology. 

Another  key  to  successfully  integrating  the  laptop  appears  to  be  the  role  of  the  teacher.  The  teacher’s  role 
changed  from  that  of  a lecturer  and  transmitter  of  knowledge  to  that  of  a facilitator,  guiding  students  to  take 
advantage  of  opportunities  to  develop  their  inquiry  skills;  from  being  a conclusion-drawer  to  becoming  a 
curriculum  planner  and  initiator,  while  using  the  computer  and  other  resources  to  provide  science  information 
(Levine  & Donitsa-Schmidt,  1 996).  The  particular  advantage  for  science  educators  is  that  there  is  a teaching  point 
to  be  made  in  using  a collaborative  context  for  science  learning,  stressing  that  scientists  usually  work  in  teams  and 
that  science  is  a social  activity  (Scanlon,  1997). 

Yet  another  key  to  successful  laptop  integration  is  the  curriculum  support  to  come  up  the  ideas  about  how 
to  integrate  laptop  into  the  learning  environment.  Between  grading  and  making  test  and  keeping  up  with  their 
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lessons,  there  is  no  time  for  teachers  to  be  able  to  sit  and  look  for  valuable  and  valid  resources.  Providing 
resources  that  will  assist  teachers  with  generating  the  ideas  is  important,  as  is  providing  time  to  tryout  and  revise 
the  ideas. 

Classroom  logistics  is  another  key  in  successful  integration  of  the  laptop  in  the  learning  environment. 
Even  though  they  have  wireless  Internet  connection,  the  laptop  still  needs  a power  cord  to  operate.  Currently,  the 
cords  are  all  over  the  floor  and  create  challenges  for  moving  around  the  classroom.  Set  up  or  the  learning  space 
needs  to  be  considered. 

A final  factor  revealed  in  this  study  relates  to  student  attention.  Students  can  configure  laptops  with  their 
own  personal  interest.  For  instance,  they  can  setup  different  wallpaper  for  desktop,  install  fancy  screensavers  and 
entertainment  applications.  While  important,  these  can  also  become  sources  of  distraction.  Balancing 
personalization  of  the  tools  while  maintaining  a focus  on  use  for  learning  is  another  key  factor  to  the  successful 
integration  of  the  laptops. 


Implications 

The  data  gathered  suggests  a number  of  recommendations  that  are  applicable  in  the  learning 

environment  with  laptop.  Implementation  of  these  strategies  may  assist  with  the  successful  integration  of  the 

laptops. 

1.  Make  use  of  laptop  as  cognitive  tool:  Using  software  like  spreadsheet,  PowerPoint,  image  processing,  web 
construction  and  multimedia  authoring  tools  enable  student  to  create  charts,  graphics,  presentations, 
projects,  and  websites  easily.  They  can  use  these  tools  to  represent  and  construct  their  knowledge  toward 
concepts  and  enhance  their  cognition.  Therefore,  laptop  can  be  used  as  a cognitive  tool  (Jonasson  & Reeves, 
1996)  with  appropriate  strategies  and  enhance  the  possibility  of  shifting  teacher’s  role  from  a lecturer  and 
transmitter  of  knowledge  to  a facilitator. 

2.  Employ  constructivism  approach:  learning  and  instructional  practices  h science  classrooms  should  be 
flexible  and  form  a constructivism  learning  environment.  (Levine  & Donitsa-Schmidt,  1996).  To  enable  a 
constructivist  learning  environment  and  switch  the  structure  from  teacher-centered  to  learner-centered, 
teacher  should  provide  students  chances  to  construct  their  own  knowledge  representations  with  laptop. 

3.  Provide  curriculum  support  or  formal  channel  to  share  instructional  ideas:  Teachers  don’t  have  enough 
time  to  develop  appropriate  tools  for  laptop  in  each  unit  is  a problem  for  integration.  Laptop  just  becomes  a 
tool  to  replace  pen  and  paper.  Having  someone  who  is  good  with  computers  and  laptops,  understand 
teaching  techniques,  pedagogy  and  familiar  with  curriculum  to  design  the  ready-to-use  activities  as 
resources  for  teacher  to  adapt  is  a strong  need  to  implement  the  laptop  environment.  Or  create  an  official 
website  for  sharing  valuable  resources  that  can  provide  teachers  a channel  to  access  to  all  kinds  of  tool  for 
specific  units. 

4.  Logistics  of  classroom:  Although  the  learning  environment  is  wireless  for  access  to  the  Internet,  laptop  still 
needs  cord  to  get  power  because  kids  seldom  remember  that  they  have  to  recharge  laptops  everyday.  Ideally 
there  would  be  enough  plugs  under  every  deskin  the  floor  or  a desk  with  plus.  Considering  the  arrangement 
of  desks,  plugs,  and  cords  before  using  laptop  in  the  classroom  is  a primary  work. 

5.  Reduce  distraction:  students  can  configure  laptops  with  their  own  personal  interest;  for  instance,  install 
fancy  screensaver  and  entertaining  applications.  They  may  miss  the  lecture  and  put  attention  to  these 
distractions.  Teacher  may  needs  to  set  up  classroom  rule  to  prevent  students  browsing  unrelated  information 
in  class. 
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Conclusion 


Bringing  wireless  laptop  into  learning  environment  has  some  strength.  The  capability  of  accessing  to  the 
Internet  anytime  anywhere  and  being  portable  extends  the  learning  environment  beyond  the  boundary.  This 
advantage  makes  the  learning  environment  turns  into  classroom,  lab,  library,  and  the  authentic  context  based  on 
the  instructional  goals  without  the  physical  limitation.  Carter  (1998)  indicated  that  in  the  ecology  class,  students 
enter  the  full  ecological  context,  doing  it  as  a biologist  or  ecologist  would,  gather  data  and  convene  to  discuss  the 
data  and  patterns,  formulating  conclusions  and  new  questions  about  the  ecological  relationship  in  the  environment 
that  supports  it.  This  kind  of  real  experience  is  most  powerful,  because  the  learning  is  situated  in  the  most 
authentic  context  possible.  Moreover,  teacher  can  collect  ready-to-use  tools  for  laptop  from  the  Internet  and  share 
that  easily  with  other  teachers.  Teachers  perceived  Internet  as  a major  tool  for  teaching  collaborative  and 
investigative  practices  of  science  and  scientists  and  they  also  became  aware  of  the  possibilities  and  the  problem.s 
created  by  both  the  nearly  unlimited  quantity  of  information  available  on  the  Internet  and  the  limited  quality  and 
relevance  of  much  of  that  information  (Wiesenmayer  & Koul,  1998;  Jackson  et  al.,  1997).  Students  can  seek 
information  on  the  Internet  and  email  peers  to  share  their  knowledge  and  accumulate  resources  collaboratively. 
Besides,  with  the  laptops  in  the  classroom,  it  saves  lots  of  time  because  they  don’t  have  to  go  back  and  forth 
between  media  center  and  classroom. 

The  strategies  of  integrating  laptop  into  learning  environment  depend  on  the  feature  of  learners  and 
subjects,  learning  goal  and  objectives.  There  is  no  reason  that  laptop  should  be  involved  in  each  activity.  Teacher 
should  use  laptop  for  meaningful  purposes.  For  example,  for  middle  school  kids,  it’s  better  to  use  mixed  up 
strategies  with  and  without  laptop  because  it’s  impossible  to  ask  kids  sit  in  front  of  laptop  all  day  even  these  tools 
are  excellent.  The  ideal  situation  is  not  over  and  not  under  use.  It  has  to  be  integrated  but  there  still  things  have  to 
stay,  not  total  transition.  Finding  a balance  in  use  and  house  remains  a challenge  for  further  research. 
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Abstract:  In  distance  education,  students  and  instructors  rarely  meet  face-to-face  and 
interactive  discussions  are  infrequent,  if  they  take  place  at  all.  Such  challenges  make  it 
difficult  to  model  thinking  skills  or  guide  students  through  the  thinking  process.  An 
asynchronous  online  forum  is  a promising  tool  to  cope  with  this  problem.  This  paper  reports 
an  ongoing  project  that  examines  the  effects  of  asynchronous  online  forums  on  critical 
thinking  and  overall  learning  performance,  and  also  probes  students’  overall  perspectives 
toward  the  effect  and  function  of  using  asynchronous  discussion  forums  in  the  context  of 
distance  education.  The  research  design,  participants,  instruments,  and  data  analysis  of  this 
project  are  described. 


Problem  Statement  and  Research  Objective 

A primary  function  of  education  is  not  only  to  help  students  build  knowledge  base  but  to  teach  and 
develop  their  critical  thinking  skills  as  well.  In  many  cases  this  occurs  through  class  discussions  (Bloom,  1956; 
Newman  & Wehlage,  1993).  Yet,  in  distance  education,  learners  are  separated  by  distance  and/or  time  from  the 
instructor  and  their  peers;  discussions  are  often  hindered  since  students  and  instructors  rarely  meet  face-to-face. 
Besides,  since  the  population  of  distance  learners  tends  to  be  older  or  have  career/family  obligations,  learners 
have  less  flexible  time  and  fewer  opportunities  to  engage  in  the  class  discussion  with  each  other  synchronously. 
If  collaborative  inquiry  tasks  are  assigned,  it  is  difficult,  if  not  impossible,  for  the  instructor  to  facilitate 
interaction,  monitor  collaborative  discussions  and  critical  thinking  in  a distance.  Such  challenges  make  it 
difficult  to  model  thinking  skills  or  guide  students  through  the  thinking  process. 

Asynchronous  online  forums  (i.e.,  text -based  computer-mediated  communication  tools)  are  promising 
tools  that  offer  an  opportunity  for  interactive  discussions  independent  of  time  and  place,  and  also  provide  an 
opportunity  for  the  instructor  to  moderate  such  discussions  (Duffy,  Dueber,  & Hawley,  1998).  However,  like 
face-to-face  discussions,  online  discussions  can  deviate  from  intended  pathways,  and  facilitators  need  to 
continually  monitor  the  discussion  and  provide  input  at  appropriate  moments.  Instructors  need  more  guidelines 
from  educational  researchers  about  integrating  electronic  collaboration  and  communication  tools  into  their 
classrooms  (Bonk  & King,  1998).  Some  studies  have  compared  the  effectiveness  of  online  forums  to  face-to- 
face  forums;  however,  verbally  based  face-to-face  forums  and  textually  based  asynchronous  forums  make  direct 
comparisons  difficult  (Anderson  & Kanuka,  1997).  In  essence,  the  core  issue  should  be  whether  asynchronous 
discussions  supplement  instruction  to  facilitate  more  interactive  discussions,  foster  critical  thinking,  and/or 
enhance  overall  student  learning  performance.  The  question,  “Are  asynchronous  online  forums  effective  in 
higher  education?”  still  remains  largely  unanswered. 

The  purpose  of  this  research  project  is  to  examine  the  effects  of  asynchronous  online  forums  on  critical 
thinking  and  overall  learning  performance  and,  in  addition,  to  probe  students’  overall  reactions  and  perspectives 
toward  the  effect  and  function  of  using  asynchronous  discussion  forums  in  distance  learning  environments. 

Research  Design 

The  research  design  is  a pre-  and  post-test  quasi-experimental  design.  The  independent  variable  is  the 
use  of  the  structured  asynchronous  online  forum.  Students  in  a control  group  class  only  use  their  regular 
communication  mode  (via  e-mails)  to  ask  their  instructor  or  peers  course-related  questions.  Students  in  an 
experimental  group  class  will  use  an  asynchronous  online  forum  (threaded  discussion  bulletin  board)  as  a 


supplement  to  their  Web-based  classes;  a series  of  discussion  topics  will  be  posted  on  the  class  bulletin  board 
and  facilitated  by  the  instructor.  The  students  taking  BMS  235,  an  undergraduate  distance  learning  course  at 
Purdue  University,  in  spring  2001  are  the  subjects  in  the  control  group;  the  students  taking  BMS  235  in  fall 
2001  and  spring  2002  will  be  the  subjects  in  the  experimental  group.  There  are  two  dependent  variables 
examined  in  the  study:  students’  learning  achievement  and  attitude.  The  first  dependent  variable  is  the  learning 
achievement  of  the  students  that  is  measured  at  two  levels — declarative  knowledge  and  critical  thinking  skills. 
Declarative  knowledge  refers  to  the  concepts,  principles,  issues,  and  facts  presented  in  a learning  situation.  It  is 
measured  as  the  scores  on  the  four  quizzes  and  two  exams  on  course  content.  Learning  in  terms  of  critical 
thinking  is  measured  via  California  Critical  Thinking  Skills  Test,  class  discussions  on  asynchronous  online 
forums,  and  essay  questions  that  go  beyond  declarative  knowledge  acquisition  and  involve  analysis,  synthesis, 
reasoning,  interpretation,  and  induction.  The  second  dependent  variable  is  students*  attitudes  towards 
asynchronous  online  forums,  class  discussion,  and  the  course  in  general.  Students’  attitude  has  been  defined  as 
scores  on  the  attitude  pre-and  post-course  Likert -type  surveys  developed  by  the  researcher.  The  interview 
questionnaire  will  also  be  administered  to  probe  students’  attitude  toward  the  study. 

ANOVA  will  be  performed  to  justify  whether  there  is  a difference  among  the  subjects  in  terms  of  age, 
educational  backgrounds,  attitudes  toward  asynchronous  forums  and  the  course,  and  knowledge  of  content  at 
the  beginning  of  the  class.  The  dependent  variables,  including  students’  learning  achievement  and  attitudes 
toward  the  course  and  the  asynchronous  online  forum,  will  be  analyzed  using  a one-way  ANOVA  at  the  end  of 
the  class.  In  addition,  the  qualitatively  analytic  procedure  that  organizes  the  data,  generates  categories,  themes, 
and  patterns,  and  searches  for  alternative  explanations  for  the  data  is  adopted  to  analyze  the  data  from  the 
interview  questionnaire  to  generalize  themes  regarding  students’  perspectives  about  the  study.  In  order  to  pilot 
test  the  efficacy  and  reliability  of  the  proposed  research  design,  BMS  236  class  in  summer  2001  was  used  as  a 
pilot  study  for  the  experimental  group. 

Implications  of  Results 

The  importance  of  class  discussion  has  been  highlighted  by  a host  of  research  studies;  to  move 
students  beyond  assimilating  inert  facts  into  generating  better  mental  models,  teachers  must  structure  leaning 
experiences  that  highlight  how  new  ideas  can  provide  insights  in  intriguing,  challenging  situations  (Dede, 

1996).  This  research  speculates  about  how  an  increasing  important  tool — asynchronous  online  forum,  can 
afford  educators  great  pedagogical  opportunities  to  reshape  distance  education.  It  is  our  anticipation  that 
asynchronous  online  forums  are  the  means  that  afford  students  the  time  for  thoughtful  analysis,  reflection,  and 
composition  as  their  discussion  of  an  issue  evolves  and  that  allow  instructors  to  mentor  and  evaluate  the  critical 
thinking  skills  exhibited  during  out-of-classroom  discussions.  It  is  this  potential  of  asynchronous  online  forums 
that  we  find  so  exciting.  However,  the  actual  impact  and  effectiveness  of  the  asynchronous  forum  as  a 
supplement  to  distance  instruction  is  still  insufficiently  researched.  Thus,  this  research  project  will  identify  and 
experimentally  examine  asynchronous  online  forums  in  a constructivist  paradigm,  and  provide  useful 
information  as  to  the  overall  reactions  and  perspectives  of  students  toward  the  effect  and  function  of  using 
asynchronous  discussion  forums  in  the  distance  education  environments. 
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Abstract:  History  has  shown  that  interaction  is  an  essential  component  in  the  learning 
process.  Integrating  interaction  into  a traditional  classroom  is  relatively  easy;  however, 
moving  that  interaction  into  several  time  zones  is  much  more  difficult.  >\feb-based  bulletin 
boards  (WBBs)  are  promising  tools  that  offer  such  an  opportunity  for  interactive  discussions 
and  collaboration  in  distance  education.  Thus,  a set  of  theoretically -based  instructional 
strategies  for  the  instructor  will  be  developed  to  help  promote  active  interaction  and  help 
students  make  the  most  use  of  WBBs.  The  proposed  instructional  strategies  for  WBBs  will  be 
implemented  in  experimental  settings  to  evaluate  for  their  effectiveness. 


Numerous  studies  (Romiszowski,  1988;  Schaffer  & Hannafin,  1993)  have  provided  evidence  that 
interaction  may  be  the  missing  link  in  a successful  distance  education  course.  Computer-mediated 
communication  (CMC)  has  been  utilized  to  fill  this  gap  for  the  past  two  decades  to  increase  interaction  between 
learners  and  the  instructor  and  among  learners.  CMC  can  be  used  to  facilitate  both  synchronous  (real-time)  and 
asynchronous  (delayed  time)  discussion.  Asynchronous  discussion,  such  as  e-mails,  listservers  and  electronic 
bulletin  boards  are  usually  viewed  as  least  costly,  most  flexible,  and  effective  ways  to  facilitate  discussions  in 
today’s  distance  education.  Students  can  read,  follow,  and  participate  in  discussions  as  they  wish.  They  can  exit 
and  reenter  as  their  schedules  allow  without  missing  conversation  or  an  opportunity  to  contribute  their  ideas. 
However,  researchers,  such  as  Jonassen  (1996),  have  indicated  that  many  technology-based  innovations  have 
failed  because  they  were  not  properly  implemented  and,  without  due  care,  this  trend  will  continue  into  future. 
No  matter  how  high  our  expectations  are  for  computer-supported  collaborative  learning,  only  the  careful 
development  and  implementation  of  these  tools  will  help  us  realize  the  full  potential  of  the  technology  (King, 
1998). 


Recently,  the  increasing  popularity  of  the  Internet  and  the  incorporation  of  conventional  CMC  and  the 
World  Wide  Web  (WWW)  have  begun  the  ongoing  renaissance  to  enhance  human  to  human  interaction  in  the 
field  of  distance  education.  Due  to  the  development  of  hypertext,  the  WWW  has  become  the  most  popular  and 
user-friendly  way  to  distribute  multimedia  contents.  Considering  accessibility  of  diverse  resources  from  the 
Web  with  asynchronicity  and  cost-efficiency,  I have  selected  the  Web-based  bulletin  boards  (WBBs)  as  an 
instructional  tool  to  improve  interaction  and  distance  learners’  academic  performance.  In  addition,  based  on  the 
suggestions  of  many  researchers  and  educators,  such  as  Klemm  (1998)  and  Jonassen  (1996),  who  have 
experienced  using  WBBs  to  facilitate  their  instruction,  a set  of  guidelines  and  instructional  strategies  to  help 
instructors  stimulate  more  interactive  discussions,  cope  with  the  unsolved  challenges  using  WBBs,  such  as 
information  overload  and  lack  of  social  presence,  as  well  as  enhance  overall  learning  performance  in  WBB 
environments  for  distance  education  will  be  synthesized  and  developed.  Based  on  the  theory  of  constructivism, 
the  proposed  instructional  strategies  will  include: 

a)  How  to  prepare  students  with  an  asynchronous  discussion  tool  they  will  be  using.  For  example,  the 
instructor  can  provide  students  with  thorough  handouts  and  training  on  the  use  of  the  discussion  tool, 
clarify  student  roles  in  class  discussion,  and  remind  students  to  focus  on  ideas,  not  on  personal  attacks 
as  well  as  to  show  tolerance  for  divergent  points  of  view. 

b)  How  to  break  the  ice,  set  a tone  and  create  a relationship  with  students  as  a class. 

c)  How  to  encourage  students’  participation  in  discussion.  For  example,  strategies  to  encourage  student 
involvement  may  include: 


2143 


■ Motivating  students  that  thinking  hard  is  a meaningful  goal  not  only  for  this  course  but  for  their 
future  life  and  work  as  well.  Their  productions  will  be  taken  seriously  by  the  instructor,  and  their 
efforts  be  rewarded  by  the  educational  system  and  society  in  general  (Jonassen,  1996). 

■ Emphasizing  the  importance  of  online  discussions  by  appropriately  weighting  their  value  (e.g.,  the 
percentage  of  the  overall  grade  that  the  discussion  is  worth.). 

■ Holding  high  but  realistic  expectations  for  students. 

■ Posting  or  disseminating  important  information,  such  as  quiz  results,  homework  solutions,  or 
exam  preparation  hints,  only  on  the  WBB.  This  may  increase  the  frequency  for  students  to  visit 
the  WBB  and,  thus,  may  increase  the  chances  for  students  to  look  at,  contribute  to  or  respond  to 
the  online  discussion  more  frequently. 

■ Making  the  instructional  activity  relevant  and  interesting. 

■ Avoiding  leading  questions  or  yes/no  questions.  The  instructor  should  ask  more  “why”  or  “how” 
questions,  leading  students  to  try  to  figure  out  things  for  thems  elves. 

■ Providing  immediate  feedback. 

■ Listening  attentively  and  acknowledging  student  contributions.  Give  students  some  indication  of 
how  well  they  have  contributed  to  the  class  discussion  and  how  to  improve  (via  private  e-mail). 
Rewards  can  be  as  simple  as  saying  a student’s  response  was  good,  with  an  indication  of  why  it 
was  good,  or  mentioning  the  names  of  contributors:  “Cherry’s  point  about  pollution  really 
synthesized  the  ideas  we  had  been  discussing”  (Davis,  1993;  Willis,  1993). 

■ Providing  a variety  of  learning  activities  since  variety  reawakens  students  ’ involvement  in  the 
course  and  their  motivation. 

d)  Modeling  critical  thinking  skills  by  asking  students  Socratic  questions. 

e)  What  available  teaching  activities  are  that  can  catch  students’  attention  or  reawaken  students’ 
involvement  in  the  course  and  their  motivation  to  actively  participate  the  online  discussion.  For 
instance,  role  playing,  case  studies,  debates  and  study  questions  that  explore  the  main  topics  of  the 
course  content  are  all  instructional  activities  that  can  be  incorporated  into  almost  any  distance  learning 
course  to  encourage  intellectual  discussion  and  provide  opportunities  to  apply  what  students  have 
learned. 

f)  What  instructor’s  role  is  as  a facilitator  in  online  discussion. 

The  above  proposed  instructional  strategies  will  be  explored  in  greater  depth  in  the  presentation  and  an  open 
discussion  will  be  facilitated. 

The  actual  impact  and  the  gain  of  student  learning  performance  by  using  the  proposed  instructional 
strategies  have  not  been  realized  yet.  Thus,  the  above  proposed  instructional  strategies  will  form  a framework 
from  which  to  build  an  analysis  of  the  actual  use  to  facilitate  interaction  for  distance  learning  courses  at 
Veterinary  Technology  Distance  Learning  Program  at  Purdue  University.  After  this  pilot  study,  appropriate 
modification  will  be  made  and  then  the  use  of  WBBs  with  the  modified  instructional  strategies  will  be 
experimentally  tested  to  investigate  the  impact  and  the  actual  gain  of  student  learning  performance  as  well  as 
the  overall  reactions  and  perspectives  of  the  instructor  and  students  toward  the  effect  and  function  of  using 
WBBs  in  the  context  of  distance  learning  courses. 

References 

Davis,  B.  G.  (1993).  Tools  for  teaching.  San  Francisco:  Jossey-Bass  Publishers. 

Jonassen,  D.  H.  (1996).  Computers  in  the  classroom:  Mindtools  for  critical  thinking.  Englewood 
Cliffs,  NJ:  MerrilL/Prentice  Hall. 

King,  K.  S.  (1998).  Designing  21®‘-century  educational  networlds:  Structuring  electronic  social  spaces. 
In  C.  J.  Bonk  & K.  S.  King  (Eds.),  Electronic  collaborators:  learner-centered  technologies  for  literacy, 
apprenticeship,  and  discourse  365-383).  New  Jersey:  Lawrence  Erlbaum  Associates. 

Klemm,  W.  R.  (1998).  Eight  ways  to  get  students  more  engaged  in  online  conferences.  T.H.E.  Journal, 
26(1).  Retrieved  March  15,  2001,  from  http://www.thejoumal.com/magazine/vault/A1997.cfm 

Romiszowski,  A.  (1988).  The  selection  and  use  of  instructional  media.  London:  Kogan  Page. 

Schaffer,  L.,  & Hannafin,  D.  (1993).  The  effects  of  progressive  interactivity  on  learning  from 
interactive  video.  Educational  Communication  and  Technology  Journal,  34  (2),  89-96. 

Willis,  B.  (1993).  Distance  Education:  A Practical  Guide.  New  Jersey:  Englewood  Gliffs. 


ERIC 


A Case  Study  for  Promoting  Collaboration 
on  Online  Project-Based  Learning 


Young-Sun  Yang 

Department  of  Educational  Technology 
Kwandong  University,  Kangnung,  Korea 
E-Mail:  vsvang@kwandong.ac.kr 


Abstract  The  interactive  mechanism  of  the  internet  allows  learners  to  interact  with  teachers, 
other  learners,  and  contents  while  it  facilitates  teachers  to  interact  not  only  with  learners  but 
also  with  other  teachers  who  teach  the  similar  or  related  courses  and  condict  online  teaching. 
The  study  attempts  to  analyze  teaching  strategies  for  planning  collaborative  learning  and 
teaching  through  a case  of  online  project-based  learning  in  inter -universities.  The  study  also 
attempts  to  observe  the  degree  of  student^  leamhg  experience,  collaboration  and  participation 
with  their  expectancy  and  satisfactions  on  the  collaborative  online  project -based  learning . 


1.  Introduction 

In  last  several  years,  online  education  has  become  an  accepted  means  of  teaching  and  learning  in  higher 
educational  institutions,  and  more  and  more  universities  use  computers  to  improve  learning  and 
instruction  by  connecting  to  the  resources  and  by  communicating  with  others.  The  interactive  mechanism 
of  the  internet  and  the  Web  allows  teachers  and  students  to  share  ideas  and  to  access  information  and 
resources  with  a much  broader  community  than  would  otherwise  be  possible.  Through  the  Web  learners 
can  interact  with  teachers,  other  learners,  and  contents  more  actively  and  effectively.  As  the  same  manne? 
this  interactive  mechanism  encourages  teachers  to  interact  not  only  with  learners  but  also  with  other 
teachers  who  teach  the  similar  or  related  courses  and  conduct  online  teaching.  This  collaborative 
interactivity  of  teachers  with  the  internet  can  be  taken  place  between  or  among  universities  in  the  mode  of 
intra  or  inter- universities  by  sharing  ideas  in  preparing  courses,  developing  course  materials,  and 
operating,  managing,  and  evaluating  courses. 

This  collaborative  way  of  teaching  would  improve  the  quality  of  teaching  and  learning  and  consequently 
expand  new  opportunities  of  education.  When  the  Web  extends  learning  beyond  classroom  walls  to 
learning  communities,  the  students  can  work  in  inter- group  as  well  as  intra-group,  and  the  roles  and 
concepts  of  teaching  and  learning  are  restructured.  According  to  Pea  (1994),  there  are  two  alternatives  to 
the  transmission  model  of  communication.  Teachers  and  students  alike  are  transformed  as  learners  by  the 
process  of  communication.  Through  such  collaborative  discourse  within  the  learning  community,  active 
leaning  occurs  more  and  knowledge  is  constructed  through  more  interactions. 

The  interactions  can  be  actively  employed  especially  in  the  project-based  learning  activities  by 
formulating  an  efficient  communication  system  for  group  interaction,  since  project -based  learning 
requires  interaction  among  other  students  in  order  to  accomplish  the  group  project.  In  the  project- based 
learning,  learners  generally  involve  in  identifying  the  problem  and  information  they  need,  locating 
corresponding  information  needs,  and  extracting  and  organizing  relevant  information  from  a variety  of 
sources  into  productive  uses  through  group  communications  and  activities.  On  the  same  time,  teachers 
engage  in  planning  a collaborative  course  by  setting  the  topic  of  the  project,  designing  the  materials  and 
learners'  activity,  and  managing  and  evaluating  learners'  progress. 

Based  upon  this  assumption,  this  study  attempts  to  analyze  teaching  strategies  for  designing  and  operating 
courses,  methods  for  successful  collaborative  teaching  in  the  Web  environment,  and  suggest  new 
potentials  on  the  view  of  collaborative  teaching,  especially  for  project-based  learning  with  the  internet. 
Also,  this  study  explores  the  issues  in  students  ’ learning  experience,  collaboration  and  participation  along 
with  their  expectancy  and  satisfactions  on  the  collaborative  online  project  through  observing  an  electronic 
survey. 
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2.  Background  of  the  Study 

Structure  of  Interaction  in  Collaborative  Learning  and  Teaching 

In  online  learning  environments  there  are  usually  three  kinds  of  interactions  that  take  place:  learner- 
content  interaction,  learner-instructor  interaction,  and  learner-learner  interaction  (Moore  & Kearsely, 
1996).  Learner-content  interaction  is  the  interaction  between  the  learner  and  the  information  presented 
that  should  lead  to  knowledge  acquisition.  This  interaction  relies  on  the  knowledge  base  that  the  learner 
has  built  from  prior  learning  experiences  and  on  the  ability  of  the  learner  to  interact  with  the  conteit 
presented.  Learner- instructor  interaction  is  a vertical  interaction  dependent  upon  the  ability  of  the  learner 
and  the  instructor  to  communicate.  At  this  type  of  interaction,  the  instructors  provide  support,  and 
encourage  the  learners.  Learner -learner  interaction  is  the  horizontal  interaction  between  learners.  Students 
learn  in  a self-directed  way,  but  at  the  same  time  they  interact  with  other  students  by  communicating 
information  and  ideas  so  as  to  increase  their  participation  and  enhance  their  motivation  and  learning. 

These  types  of  interactions  in  online  learning  environment  are  taken  place  by  the  instructors  focusing  on 
learners.  However,  for  the  sake  of  instructors  the  interactions  can  be  made  with  other  teachers  or  outside 
experts  as  well  as  with  learners  or  a group  of  learners  (Yang,  1999).  Therefore,  the  forth  type  of 
interaction  occurred  in  collaborative  learning  is  the  interaction  between  instructors.  This  allows  learners 
to  have  interaction  with  not  only  learners  in  his/her  schools  but  also  learners  in  other  schools  or 
communities.  The  instructors  are  connected  through  shared  teaching  objectives  or  contents  whereas  the 
groups  of  learners  are  connected  through  the  shared  problem  of  the  project. 

When  the  collaborative  teaching  is  conducted  between  instructors,  more  interactive  collaboration  has 
made  in  the  whole  teaching  and  managing  process  including  planning,  implementing,  and  evaluating.  The 
collaborative  teaching  covered  all  types  of  interactions  among  students  and  instructors,  instructor  and 
instructor,  and  students  and  students.  Teachers  can  share  their  syllabus,  lectures,  students’  task,  activity 
and  practices,  and  students'  progress  on  discussion. 

Such  a collaborative  teaching  is  not  happened  before  in  traditional  classrooms.  General  availability  of 
multimedia  and  communication  tools,  such  as  e-mail,  bulletin  boards,  conferencing  systems,  whiteboards, 
chat  rooms,  and  videoconferencing  make  a great  impact  to  educational  curricula,  learning  materials  and 
instructional  practices  and  are  adapted  for  collaborative  learning  and  teaching.  In  addition,  the  learners  in 
online  learning  environment  have  become  more  heterogeneous,  and  learners’  social  or  cultural 
background  and  own  interest  are  divergent  so  that  they  tend  to  receive  information  actively  with  more 
interaction  and  feedback. 

Elements  of  Collaboration  in  Learning  and  Teaching  on  Project -Based  Learning 

Collaborative  learning  emphasizes  the  mutual  engagement  of  learners  in  the  learning  process  rather  than 
on  the  sole  efforts  of  individual  to  reach  a common  group  goal.  Therefore,  on  project -based  learning  the 
final  products  are  evaluated  based  on  collaboration  and  represent  a synthesis  of  the  whole.  Some  of  the 
main  advantages  of  collaborative  learning  are  that  it  encourages  active  and  constructive  learning  and 
encourages  deep  processing  of  information,  as  well  as  evoking  critical  thinking,  reasoning  and  goal- based 
learning  (Bernard  et  al.,  2000).  Collaborative  learning  requires  less  teacher- imposed  goal  structuring  and 
includes  the  process  of  sharing  the  learning  task,  combining  expertise,  knowledge  and  skills  to  improve 
the  quality  of  the  learning  process,  building  or  consolidating  a learning  community  (Slavin,  1995). 

In  a collaborative  learning  setting,  the  emphasis  is  placed  on  the  interactions  as  common  understandings 
are  negotiated  and  developed  across  differences  of  knowledge,  skills  and  attitudes.  Motivation  to 
participate  and  confidence,  together  play  an  equally  important  role  if  benefits  are  to  merge  from  the 
experience.  Moreover,  participants  need  to  assume  a variety  of  functional  roles  as  interchanges  progress 
and  involve  question  answering  and  explanations  that  are  open  to  challenge  and  justification  (Bernard  et 
al.,  2000).  To  conduct  collaborative  online  learning  successfully,  it  is  crucial  that  the  learner  feels  part  of  a 


learning  community  where  learner's  contributions  add  to  a common  knowledge  pool  and  where  a 
community  spirit  is  fostered  through  social  interactions. 

Problem  Description 

The  objectives  of  the  study  are  to  identify  instructional  strategies  and  support  system  for  project-based 
learning  using  the  Web  and  to  examine  factors  for  collaborative  teaching  process.  The  study  examines  and 
exemplary  case  of  more  effective  and  efficient  collaborative  instructional  methods  in  inter- universities. 
The  process  of  online  project  includes  not  only  the  identification  of  the  problem,  the  time  of  accessing 
information  needed  for  the  project  problem,  but  also  collaborative  learning  and  teaching  strategies  such  as 
preparing  courses,  creating  social  climate  and  sense  of  community,  encouraging  students  collaboration, 
and  using  support  mechanism.  Accordingly,  questions  are  asked  in  terms  of  how  much  the  students  learn 
from  online  experience,  how  well  they  collaborate  and  participate  in  the  project,  how  much  they  are 
expected  and  satisfied  at  collaborative  online  project -based  learning  activities. 

Learning  Community  and  Environment 

Students  were  junior  majoring  educational  technology  or  curriculum  education,  and  taking  a distance 
education  ^stem  and  new  media  course.  Students  of  two  universities  worked  in  small  groups,  and  two 
universities  located  in  4 hours  driving  distance  were  involved  in  online  project,  sharing  their  knowledge 
and  collaborating  to  perform  context-dependent  tasks. 

The  task  was  developing  a model  for  distance  learning,  and  it  is  loosely  structured  in  individual  activity. 
The  instructors  provided  the  topic  upon  which  each  group  develop  own  specific  model  for  distance 
education,  monitored  the  pace  of  group  task,  provided  the  steps  for  successful  completion,  guided  the 
enabling  objectives  for  completing  their  final  group  report.  Meanwhile,  instructors  did  not  interrupt 
discussion  itself  while  providing  supports  and  facilitating  guide. 

Students  Survey  and  Analysis 

The  survey  was  held  at  the  end  of  course  and  the  survey  was  consisted  of  items  including  the  followings: 

• Did  the  collaborative  online  project  experience  enhance  understanding  of  distance  education? 

• Were  your  communication  skills  improved  by  the  collaborative  online  project  learning? 

• Were  your  problem-soling  skills  improved  by  the  collaborative  online  project  learning? 

• Were  collaboration  well  taken  place  through  learning  in  your  group? 

• What  was  the  most  significant  factor  to  interfere  your  collaboration? 

• When  collabordion  was  not  occurred,  how  did  you  handle  it? 

• Did  you  participate  actively  in  collaborative  online  project? 

• Were  you  satisfied  at  participation  of  other  members  of  your  group? 

• What  did  you  expect  of  collaborative  online  project  before  it  started? 

• Do  you  want  to  take  another  online  project  course  if  you  could? 

• What  mechanism  did  you  use  most  for  collaboration  (e.g.,  bulletin  board,  chatting  room,  etc)? 

Additionally,  messages  were  analyzed  based  on  the  electronic  bulletin  board  discussion  in  order  to 
observe  which  students  play  leadership  most  and  facilitate  team  project,  monitor  other  students’  thoughts 
and  opinions,  put  together  the  process  and  results  of  collaboration  work. 


3.  Teaching  Strategies  for  Online  Project-Based  Learning 

Some  of  strategies  for  online  learning  in  general  are  reviewed  with  collaborative  learning  and  teaching, 
especially  in  project -based  learning  settings.  The  process  of  project -based  learning  with  the  internet  is 
generally  composed  of  online  questions  and  answers,  online  report,  online  discussion  by  using  the 
bulletin  board,  e-mail,  and  newsgroup  etc.  In  the  project -based  learning,  the  students  learn  by  problem 
solving  process  based  on  the  given  problem  of  the  project.  For  example,  teaching  strategies  such  as 
reflection  and  summary  of  what  students  earn  from  project,  and  small  group  activity  in  the  class  which 
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are  often  used  in  the  traditional  class  can  be  more  effectively  occurred  in  online  learning  space  when 
collaborative  interaction  mechanism  are  applied.  Recently,  educational  environment  and  learning  process 
are  moved  toward  the  performance  centered  evaluation,  portfolio,  cooperative  learning  task.  Teaching 
methods  and  strategies  for  collaborative  teaching  between  or  among  teachers  in  project-based  learning 
through  the  Web  were  examined  by  some  exemplary  case  studies.  The  based  framework  of  teaching 
process  is  based  upon  previous  research  including  Bernard  et  al.  (2000). 

Preparing  courses 

In  online  learning,  students'  data  as  like  as  traditional  classroom  settings  can  be  collected  by 
questionnaires, pre-tests  through  e-mail  or  electronic  questionnaire.  In  collaborative  teaching,  it  is  crucial 
to  plan  collaboratively  including  class  scheduling,  students  grouping,  project  topic,  select  of  technologies 
for  communication,  and  organization  of  the  group  and  leaders.  In  online  project-based  learning,  it  is 
especially  important  to  select  project  problem  that  reflects  social  issues  or  debates  so  that  learners  can 
start  from  the  issues  the  discussion  for  identifying  the  problem,  specifying  the  problem  and  find  the 
solutions,  and  synthesizing  the  results  of  discussion  and  represent  them  effectively.  Also,  it  is  important 
for  improving  the  collaboration  between  instructors  to  put  agreed  efforts  on  the  whole  process  from 
planning  instruction  to  evaluation.  To  receive  and  minimize  differences  between  sites,  the  value  or  culture 
on  each  other's  site  should  be  considered  in  advance. 

The  methods  of  preparing  the  online  courses  are  as  follows:  conducting  a learning  needs  assessment, 
establishing  learner  profiles , OTganiz  ing  informal  face-to-face  or  online  "getting  acquainted"  activities, 
teaching  and  modeling  collaborative  behavior  and  skills  that  are  expected  of  students,  providing  tutors 
and  students  with  all  technology  training  required , and  scheduling  the  process  and  completion  of  project. 

Creating  a good  social  climate  and  sense  of  community 

Developing  and  maintaining  a positive  social  environment  and  creating  a community  of  learners  are 
considered  to  be  essential.  Learning  collaboratively  is  basically  a social  process  that  must  be  encouraged 
and  nurtured. 

The  methods  of  achieving  the  community  are  the  follows:  providing  a social  space  in  the  conferencing 
system  that  allows  for  easy  and  free  access  to  other  students  and  the  instructor,  assigning  students  to 
collaborate,  posting  introductions,  information  folders,  and  other  self  revealing  items  like  photos  of 
participants,  making  the  environment  as  democratic  as  possible  to  encourage  the  involvement  of  everyone, 
providing  a restricted  space  that  is  unavailable  to  the  instructor  for  students  to  converse  among 
themselves , providng  adequate  levels  of  tutor  support,  initially,  that  gradually  gives  way  to  increased 
responsibility  on  the  part  of  students,  and  a*ranging  for  at  least  one  live  meeting  such  as  face4o-face 
group  encounter  for  participants  to  get  acquainted. 

The  advantage  with  using  a face4o-face  audience  first  is  that  the  additional  direct  contact  makes  it  easier 
to  modify  procedures  and  instructions  than  when  it  is  done  solely  online  for  the  first  time.  When  the 
online  students  are  taught  through  face-to-face  at  the  same  course,  this  can  be  more  easily  met.  The 
effects  of  face-to-face  group  encounters  are  examined  in  inter -university  project -based  learning. 

According  to  Harris  (2001),  virtual  communities  are  no  longer  a place -based  concept,  but  formed  around 
issues  of  identity  and  shared  values.  Electronic  personalities  require  internal  dialogue,  semblance  of 
privacy,  deal  with  emotions  using  text,  and  create  mental  pictures  of  others,  and  in  online  learning  we 
need  to  personalize  communications  to  create  a sense  of  presence.  Harris  suggested  tie  community- 
building steps  as  follows:  define  group's  purpose,  create  distinctive  gathering  placq  promote  leadership, 
allow  for  range  of  member  roles,  create  and  facilitate  subgroups,  and  allow  members  to  resolve  their  own 
disputes 

Encouraging  collaboration 

True  collaboration  can  best  be  realized  only  when  instructors  view  themselves  as  facilitators  and  guides 
rather  than  lectures  or  experts.  The  degree  of  instructor  involvement  is  also  an  issue.  As  Brown  and 


2148 


Palinscar  (1989)  suggested  "scaffolding",  the  tutor  began  by  providing  high  levels  of  support  to  all  groups 
and  students,  but  as  students  started  to  demonstrate  independence  and  cohesivenes  s,  the  tutor  support  was 
faded  gradually. 

The  methods  of  encouraging  collaboration  are  as  follows:  using  small  groups,  using  nickname  for  each 
group  made  by  group  members,  select  ing  group  leader  and  rotate  leadership  in  turn,  assuming  instructor’s 
role  as  a facilitator,  modeling  desired  collaborative  behavioi;  using  facilitating  techniques,  substituting 
visual  social  cues  with  verbal  cues , and  promoting  positive  interdependence 

In  online  collaborative  learning,  instructors  use  facilitating  techniques  such  as  'weaving'  to  steer 
discussions  and  ’go  around  the  circle'  to  ensure  maximize  student  participation.  Promoting  positive 
interdependence  among  online  team  members  reinforces  collaborative  online  learning  by  introducing  a 
legitimate  need  to  collaborate.  To  give  collective  rewards  and  distributing  needed  resources  across  group 
members  is  fostering  a healthy  interdependence. 

It  is  important  for  students  to  need  to  be  prepared  for  a collaborative  online  experience  because  it  is  not 
assumed  all  learners  possess  the  prerequisites  for  skilful  collaboration.  When  messages  are  analyzed 
based  on  the  electronic  bulletin  board  discussion,  some  contend  strong  leadership,  and  facilitate  the  team 
project,  monitor  other  students  thoughts  and  opinions,  put  together  the  process  and  results  of  students. 

Using  support  mechanism  effectively 

Land  & Green  (2000)  pointed  out  the  need  for  external  support  mechanisms  that  help  learners  develop 
strategies  for  effectively  learning  with  project -based  environments.  Encouraging  goal  setting  and 
reflection  during  the  learning  process  is  important.  Supports  of  extensive  teacher-student  conversation  in 
complex  learning  situations  facilitate  gradual  changes  in  understanding. 

The  methods  of  using  the  support  mechanism  are  as  follows:  matching  medium  to  instructional  objectives  , 
using  technology  that  is  accessible  to  all  participants,  making  use  of  internet  features,  e.  g.  discussion 
forums,  chat  rooms,  etc;  and  using  a metaphor  that  relates  to  learners'  experience 

The  teachers  provide  t he  topic  upon  which  each  group  develop  own  specific  group  task,  monitor  the  pace 
of  group  task,  provide  the  steps  for  successful  completion,  guide  the  enabling  objectives  for  completing 
their  final  paper.  Meantime,  teachers  do  . not  interrupt  discussion  itself  while  providing  supports  and 
facilitating  guide.  Minimum  support  can  be  given  as  necessary  in  mechanically,  individually  by  personal 
using  e-mail  to  the  learners  those  who  do  not  participate  well. 


4.  Results  of  Case  Study  and  Conclusion 

The  students  survey  showed  that  their  experience  on  collaborative  online  project  would  enhance 
understanding  of  distance  education  overall  and  provide  adjustment  skills  in  other  collaborative  works. 
The  students  responded  that  their  communication  skills  and  problem-solving  skills  were  improved  by  the 
collaborative  online  project  learning.  More  than  70%  of  students  were  satisfied  at  their  active 
participation  and  other  members ’participation  in  groups.  In  open  questions  in  the  survey,  students  pointed 
out  the  factors  that  interfere  their  collaboration  and  the  way  they  cope  with  them.  In  addition,  the  types  of 
mechanism  used  most  in  online  community  were  discussed. 

Collaborative  teaching  with  the  internet,  with  its  expanded  interactions,  has  a great  potential  in  designing 
and  managing  courses  more  effectively  and  efficiently  than  ever.  The  interactive  mechanism  of  the 
internet  and  the  Web  allows  teachers  and  students  to  share  ideas  and  access  information  and  resources 
with  a much  broader  community.  The  interaction  between  instructors  occurred  in  collaborative  learning 
was  emphasized  in  which  instructors  can  have  interactions  with  other  teachers  or  outside  experts  as  well 
as  with  learners  or  a group  of  learners  while  learners  have  interaction  with  not  only  learners  in  his/her 
schools  but  also  learners  in  other  schools  or  communities. 
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Strategies  for  online  learning  in  general  were  analyzed  with  collaborative  learning  and  teaching,  specially 
applied  to  project-based  learning  settings  including  preparing  courses,  creating  a good  social  climate  and 
sense  of  community,  encouraging  collaboration,  using  of  support  mechanism.  Furthermore,  to  identify  the 
students’  learning  experience,  their  involvement  in  the  process  of  collaboration  and  participation  as  well 
as  students’  expectations  and  satisfaction  level,  a survey  was  conducted  at  the  end  of  the  course  and 
analyzed.  The  findings  and  results  of  study  would  provide  implications  for  other  settings  and  suggest  for 
further  studies. 

As  a conclusion,  it  is  still  difficult  for  teachers  to  construct  a collaborative  learning  activity  in  online 
teaching  because  the  web  features  was  not  developed  collaborative  teaching  and  the  web  itself  do  not 
support  collaborative  learning  and  teaching.  Therefore,  we  need  to  refine  collaborative  teaching  strategies 
for  various  situations,  and  by  using  technology  support  the  process  of  collaborative  learning  and  teaching. 
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ABSTRACT 

The  University  of  Hong  Kong  has  been  critically  aware  of  the  need  for  students  to  gain  “access”  to  their  instructors  at  various 
times  and  from  various  locations.  In  a sense,  education  may  be  enhanced  and  communication  accomplished  while  the  student  is 
“roaming”.  Through  University  sponsorship,  a project  to  utilize  PDA  Hand-held  devices  for  accessing  a Web-server  is  under 
implementation.  Questions  are  asked  and  answered;  course  materials  are  downloaded  to  the  PDA.  Educational  “access”  is  thereby 
extended. 

i.  Introduction 

The  Hand-held  Devices  for  IT  Education  (HHIT)  project  was  initiated  at  the  University  of  Hong  Kong  in  late  2001.  The  goal 
of  the  HHIT  project  is  to  promote  students’  learning  outside  their  classroom,  anytime  and  anywhere.  Through  this  project, 
learning  materials,  lecture  notes  and  tutorial  exercises  are  made  easily  accessible  to  students  by  means  of  their  hand-held 
devices.  Students  can  review  course  material  anytime  and  anywhere,  freed  from  the  physical  boundaries  of  the  classroom. 

II.  HHIT  Design 

2.1  Palm  Platform 

Students  prepare  and  compose  their  FAQs  on  the  Palm  and  forward  them  for  eventual  communication  with  Web  servers  via 
communications  interface  conduits.  These  requests  are  maintained  and  managed  in  their  hand-held  devices.  The  Palm 
platform  is  comprised  of  two  software  elements:  Creating  a new  FAQ  and  Reading  the  FAQ.  This  includes  the  database 
management  software  to  permit  the  2-way  interchange  with  the  Web  server.  The  Palm-FAQ  interface,  showing  the  software 
components  and  the  necessary  conduits  to  interface  with  the  Web  server,  is  shown  in  Figure  1. 


2.2  Web  Platform 

The  Web  platform  is  the  primary  interface  on  the  receive-and-response  side  to  the  students  FAQs.  Instructors  receive  and 
inspect  messages  composed  by  the  students.  They,  in  turn,  compose  messages  including  clarification  requests  and  responses 
to  the  FAQs.  These  are  forwarded  via  software  operating  on  the  Web  platform  to  complete  this  two-way  interchange. 

The  Web  platform  consists  of  FAQ  Reader  and  FAQ  Writer  components.  It  provides  for  the  receipt  and  presentation  of 


FAQs  to  the  instructor  as  well  as  the  composition  and  re-transmission  of  replies  to  the  originating  student.  The  identity  and 
directivity  of  the  FAQ  to  the  originating  student  is  preserved  and  the  instructor  has  an  on-going  dialogue  with  him. 

The  overall  flow  of  Web  FAQ  interfaces  with  the  Palm  database  is  shown  in  Figure  1. 


111.  HHIT  Implementation 
3.1  Palm  Platform 

Our  Palm  FAQ  system  is  divided  into  3 sub-systems: 

o Palm  Application  Subsystem:  the  Create  Question  and  View  Question  functions  in  our  FAQ  system  on  the  Palm, 
o Conduit  Subsystem  uploads  the  question  created  by  student  to  the  instructor  via  the  PC  Desktop  and  the  Web  and 
downloads  the  corresponding  instructor  response, 
o Server  Database  Subsystem  stores  the  record  of  the  FAQ  and  does  further  processing 

Palm Desktop  PC HKU  Database  Server 

<=)  <=) 

HotSync  Conduit 

Students  create  a new  FAQ  according  to  their  subject  studied  and  their  area  of  concern.  The  title  and  content  of  the  new 
FAQ  is  inserted  via  Palm  entry  keys.  The  system  prompts  the  student  to  supply  all  missing  fields  until  the  entry  is 
complete.  Students  can  temporarily  save  the  question,  or  complete  the  writing  and  formulation  of  the  FAQ  on  their 
handheld  devices  at  their  own  convenience.  The  completed  FAQ  can  then  be  directed  to  the  corresponding  instructor  and 
is  stored  in  the  FAQ  database.  The  student  performs  the  HotSync  on  his/her  desktop  PC  which  forwards  the  FAQ  to  the 
ultimate  HKU  Database  Server  via  the  conduit  interface.  This  generates  an  e-mail  to  the  instructor.  The  HHIT  system 
prompts  the  student  through  a series  of  screens  to  insert  FAQ  requests  for  instructor  responses  (Figure  2). 


3.2  Web  Platform 


The  Web  Platform  of  our  FAQ  system  is  divided  into  3 sub-systems: 

o Web  Server  applications  consist  of  the  software  which  provides  user  requests  and  then  sends  them  to  the  server  for 
querying  the  database  or  processing  data. 

o Web  Browser  applications  are  those  programs  with  JavaScript,  which  are  downloaded  to  the  client  computers  to 
provide  different  services,  e.g.  navigation  menus. 

o System  Database  stores  the  record  of  the  FAQ  and  does  further  processing.  The  database  should  be  common  to  both 
Palm  and  Web. 


Web  Browser 


Web  Server  application- 


System  Database 
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3.3  Database 


The  databases  interact  with  the  programs  through  a Java  Database  Connectivity  (JDBC)  - Open  Database  Connectivity 
(ODBC)  bridge  as  Java  Server  Pages  (JSP)  is  used  to  write  server  related  programs.  It  requires  the  bridge  to  connect  the  Java 
DB  engine  to  Microsoft  DB  engine  due  to  the  development  of  Java  programs  on  a Windows  environment. 

Figure  3 shows  the  relationship  between  Web  platform  subsystems. 


Figure  3.  Relationship  between  the  Web  Platform  Subsystems 
When  a user  makes  a client  request  on  the  browser,  the  request  is  sent  to  the  Web  server  for  processing.  All  data  processing 
and  request  handling  is  done  on  the  Application  Server  Machine.  The  JSP  engine  will  change  all  the  custom  tag  extensions 
and  custom  Java  classes  or  JavaBeans  to  JDBC  driver  and  other  data  access  technologies  to  their  control  codes.  Database 
queries  can  then  be  made  to  the  database  server  by  using  the  “open”  schemes  of  JDBC  to  make  a bridge  with  Win32  ODBC. 

There  is  no  applet  or  JDBC  driver  to  download  to  the  client.  The  browser  using  the  Web  application  is  not  required  to 
support  Java  at  all.  The  JSP  has  full  control  over  how  many  JDBC  connections  are  made  to  the  server;  the  client  never  makes 
direct  JDBC  connection  to  the  server.  This  solution  can  work  readily  through  a firewall;  only  standard  HTTP  is  used 
between  the  Web  server  and  the  client. 

IV.  Future  Plans  and  Improvements 

PDA  Hand-held  devices  offer  a new  hardware  platform  with  numerous  opportunities  for  developing  future  applications. 

New,  important  and  useful,  functions  such  as  multicasting  to  allow  all  students  in  the  class  to  read  someone’s  questions,  and 
Chat-room  via  the  Palm  for  each  class,  are  planned  for  implementation.  The  use  of  wireless  technology  to  set  up  Webcast  in 
the  classroom  to  enhance  teaching,  and  Search  Engine  on  the  Palm  to  enhance  learning,  is  projected.  Scheduling  and  eForm, 
will  reduce  the  paperwork  associated  with  our  educative  practices.  We  have  identified  the  following  capabilities  for  our 
future  implementation: 

1.  Multicasting,  and  Chat-room:  Students  designate  questions  as  unicasting  or  multicasting.  A Chatroom  is  established. 

2.  Webcast:  Students  download  Palm  version  notes  for  the  lesson;  instructors  send  a quiz  or  other  learning  materials. 

3.  Search  engine  on  Palm:  Students  and  instructors  input  a keyword  to  search  any  topics  of  interest,  or  learning  materials. 

4.  Scratch  graphics  for  Questions,  and  eBook:  The  scratch  graphics  function  converts  graphs  into  Palm  displayable 
format.  The  eBook  function  allows  users  to  download  useful  materials  in  Palm  version. 

5.  Two-way  scheduling:  Students  and  instructors  download  their  timetables/events  or  schedule  the  many  facilities. 

6.  eForm:  eForm  permits  students  to  download  required  forms  from  desktop  and  then  beam  it  back. 

7.  Short  message  services  (SMS):  SMS  (short-message  service)  via  PDA  to  send  messages  about  the  seminar,  course  or 
other  announcements.  We  set  up  a BEAM  point  for  those  with  PDAs  to  get  the  messages/announcements  beamed. 

Many  other  extensions  for  our  HHIT  project  exist  subject  to  limitations  on  the  amount  of  available  resources  and  the  current 
advances  in  networking  and  wireless  communication  technology. 

V.  Summary 

The  HHIT  initial  capability  and  its  inherent  growth  offer  a new  dimension  to  the  instructor-student  relationship.  The 
University  of  Hong  Kong  is  seeking  to  incrementally  empower  the  education-teaching  process.  By  facilitating  student  access 
to  their  instructor  on  an  “anytime-anywhere”  basis,  the  education  process  becomes  a more  natural  part  of  the  student’s 
everyday  life  and  activities.  Finally,  the  textual  and  graphic  nature  of  their  interface  encompasses  the  sum  total  of  their 
requisite  interaction.  The  University  of  Hong  Kong  will  seek  to  disseminate  and  technically  transfer  this  capability. 
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Abstract  WebQuests  have  become  a popular  form  of  guided  inquiry  using  Web 
resources.  The  goal  of  WebQuests  is  to  help  students  think  and  reason  at  higher  levels, 
and  use  information  to  solve  problems.  In  this  paper  we  present  modifications  to  the 
WebQuest  model  drawing  on  primarily  on  schema  theory.  We  believe  these  changes  will 
further  enhance  student  reflection  and  deeper  conceptual  change  through  the  use  of 
WebQuest  activities  in  the  classroom.  The  revised  model  is  illustrated  in  a makeover  of  a 
sample  WebQuest. 


WebQuests  have  become  a popular  form  of  guided  inquiry  using  Web  resources.  The  goal  of  WebQuests  is 
to  help  students  think  and  reason  at  higher  levels,  and  use  information  to  solve  problems.  Thousands  of 
teacher-authored  WebQuests  are  currently  available  on  the  Web  for  use  in  different  subject  areas  and  at  all 
educational  levels.  Teachers  may  thus  choose  to  incorporate  WebQuests  developed  by  others,  or  they  may 
develop  their  own  WebQuest  as  a way  to  get  their  students  reasoning  at  higher  levels. 

Dr.  Bemie  Dodge  of  San  Diego  State  University  first  created  the  WebQuest  model  in  1995  (Dodge,  1995). 
Though  he  and  Tom  March  articulated  a fairly  specific  model  and  created  many  example  WebQuests,  there 
is  now  an  even  wider  range  of  activities  that  fall  under  the  WebQuest  umbrella.  Of  course,  the  WebQuest 
model  is  a subset  of  the  larger  class  of  guided  inquiry  activities  in  use  today.  Other  guided  inquiry  activities 
contain  instructional  strategies  that  are  not  a part  of  the  WebQuest  model,  e.g.,  stronger  reliance  on  a case 
as  a resource;  higher  levels  of  guidance  toward  reflection.  In  this  paper  we  propose  modifications  to  the 
WebQuest  model  drawing  on  primarily  on  schema  theory.  We  believe  these  changes  will  further  enhance 
student  learning  through  the  use  of  WebQuest  activities  in  the  classroom.  We  also  discuss  ways  to  help 
teachers  integrate  WebQuests  successfully  into  the  overall  curriculum. 

The  WebQuest  Model:  Evolving  Over  Time 

There  is  no  strict  format  for  WebQuest  design,  and  this  flexible  structure  is  one  of  its  most  appealing 
attributes  (Dodge,  1995).  Most  WebQuests  have  the  following  elements: 

1.  An  introduction  that  sets  the  stage  and  provides  some  background  information.  The  introduction  can 
be  likened  to  an  instructional  set  that  stimulates  prior  knowledge  and  prepares  students  for  new 
learning. 

2.  A task  or  problem  to  be  addressed.  This  problem-solving  task  is  at  the  heart  of  the  WebQuest.  The 
task  is  like  the  problem  in  a problem-based  learning  unit-It  is  the  challenge  or  conflict  to  be 
addressed  in  the  WebQuest  activities.  Dodge  has  provided  additional  assistance  to  educators 
attempting  to  create  the  WebQuest  task  through  his  online  training  materials.  His  "Taskonomy"  helps 
educators  envision  a wide  array  of  possible  WebQuest  tasks  (Dodge,  1999). 
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3.  A clear  description  of  the  process  learners  should  go  through  in  accomplishing  the  task.  It  is  here  that 
collaborative  teams  are  formed  and  roles  for  each  member  of  the  team  identified.  Specific  guided 
activities  are  often  included  in  the  process. 

4.  A set  of  information  sources  needed  to  complete  the  task.  Since  the  WebQuest  itself  is  delivered  as  a 
webpage,  these  resources  are  most  typically  Web-based,  though  widely  available  print  or  video 
resources  can  be  identified  for  student  use  as  well.  Some  WebQuests  provide  a common  list  of 
resources  used  by  all  members  of  the  team;  others  identify  information  sources  based  on  team 
member  role. 

5.  An  open-ended  evaluation  system  for  products  created  by  students  as  a result  of  their  problem 
solving.  It  is  recommended  that  rubrics  be  used  for  the  purposes  of  evaluation  (Pickett  & Dodge, 
2001). 

6.  A conclusion  that  brings  closure  to  the  quest,  reminds  the  learners  about  what  they've  learned,  and 
perhaps  encourages  them  to  extend  the  experience  into  other  domains. 

A number  of  sites  serve  as  portals  or  clearinghouses  for  WebQuests,  allowing  teachers  to  search  for 
resources  in  their  areas,  including: 

Blue  Web'n  (http://www.kn.pacbell.com/wired/bluewebn/), 

The  Matrix  of  Examples  (http://edweb.sdsu.edu/webquest/matrix.html),  and 

The  WebQuest  Collections  (http://edweb.sdsu.edu/webquest/webquest_collections.htm). 

Many  of  the  more  innovative  WebQuests  will  include  unique  design  elements.  Some  developers  have 
offered  formal  variations  on  the  WebQuest  model.  March  (2000),  for  example,  has  made  modifications  to 
the  original  WebQuest  structure.  Spartenburg  (2001)  presents  a model  for  Cyberinquiry  projects  that,  at 
once,  are  more  open  and  more  linear  in  progression  than  WebQuests.  Rather  than  seek  out  information 
from  prespecified  locations.  Cyberinquiry  students  conduct  more  open  searches  as  part  of  a clear  line  of 
inquiry. 

Schema  Theory  and  Guided  Inquiry 

WebQuests  can  be  seen  as  part  of  a broader  movement  within  education  that  was  heavily  influenced  by  the 
paradigm  shift  away  from  behaviorism  and  toward  cognitive  psychology  in  the  1970s  and  80s  (Mayer, 
1992).  Schema  theorists  of  that  era  advocated  an  approach  to  education  that  valued  deep  conceptual  change 
over  simple  behaviors  (Norman,  1976).  Following  this  schema-based  approach,  students  use  their  existing 
knowledge  to  make  predictions  and  address  problems  (Posner,  Strike,  Hewson,  & Gertzog,  1982;  Strike  & 
Posner,  1985).  By  engaging  in  a number  of  inquiry  and  problem-solving  activities,  students  undergo  a 
conceptual  shift  that  leads  to  new  schemas  and  ways  of  seeing  the  world-as  well  as  specific  procedural 
skills  for  dealing  with  that  world.  At  beginning,  middle,  and  end  of  problem-solving  activities,  students 
reflect  on  their  changing  schemas  and  their  growing  understanding.  Math  and  science  have  been 
particularly  fertile  areas  for  schema-based  curricula  because  new  schemas  can  provide  the  hooks  for 
making  the  computational  details  meaningful  and  stable  in  memory  (Perkins  & Simmons,  1988). 

STAR.Legacy  is  a promising  instructional  model  developed  by  Vanderbilt's  Cognition  and  Technology 
Group  that  is  based  on  conceptual -change  principles  (Schwartz,  Lin,  Brophy,  & Bransford,  1999).  The 
basic  framework  is  an  inquiry  cycle  with  repeated  opportunities  for  reflection  and  prediction.  STAR  is  an 
acronym  for  Software  Technology  for  Action  and  Reflection.  The  authoring  tool,  currently  in  prototype 
form,  is  intended  to  support  the  development  of  multimedia  units  for  encountering  and  solving  authentic 
inquiry  challenges  through  small-team  collaboration  and  research  activities. 
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Figure  1:  problem-solving  cycle  embodied  within  the  STAR.Legacy  instructional  model.  From 
http://peabody.vanderbilt.edu/ctrs/ltc/brophys/legacyl.html. 

In  a website  overview,  Sean  Brophy  provides  a succinct  description  of  the  STAR.Legacy  f)proach  (see 
http://peabody.vanderbilt.edu/ctrs/ltc/brophys/legacyl.html): 

The  learning  cycle  begins  with  the  presentation  of  a challenge  in  either  audio  or  text  format.  Then 
students  are  asked  to  reflect  on  the  challenge  and  to  ’’Generate  Ideas.”  Once  th^’ve  articulated  their 
thoughts,  then  they  listen  to  "Multiple  Perspectives”  from  various  experts.  These  experts  provide  hints 
about  things  to  think  about  when  solving  the  problem.  However,  these  hints  do  not  provide  a specific 
solution  to  the  problem.  This  allows  users  to  compare  their  naive  first  impressions  with  the  experts  to 
help  them  notice  their  lack  of  differentiated  knowledge. 

This  initial  sequence  is  similar  to  a WebQuest,  but  takes  things  a little  slower.  Students  are  asked  to 
use  their  existing  knowledge  to  think  about  the  problem  and  generate  some  initial  ideas.  They  are  then 
exposed  to  other  points  of  view  about  the  problem.  All  of  this  precedes  formal  inquiry  using  outside 
sources. 

Now  they  are  prepared  to  engage  in  a process  of  ’’Research  and  Revise.”  This  stage  of  the  learning 
cycle  organizes  resources  into  meaningful  learning  activities  designed  to  help  them  focus  on  issues 
related  to  the  initial  challenge. 

This  is  like  a WebQuest's  process  activities,  with  students  using  various  Web  res  ources  to  learn  more 
about  the  subject. 

Once  they  feel  they’ve  learned  enough  they  can  go  to  "Test  your  mettle.”  Here  they  engage  in  a set 
of  activities  that  helps  them  explore  the  depth  of  their  knowledge.  The  goal  is  to  create  assessment 
situations  that  help  them  evaluate  what  they  do  not  know  so  they  can  return  to  the  ’’Research  and 
Revise”  section  to  learn  more. 

This  cycle  of  ’’testing  mettle”  and  returning  to  resources  is  more  elaborate  than  the  WebQuest  model 
and  requires  a more  fully  developed  instructional  unit. 
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Students  progress  to  the  "Go  Public"  stage  after  proving  to  themselves  that  they  understand  the 
content  well  enough  to  express  a solution  to  the  challenge. 

Students  "go  public"  with  their  solution,  but  they  also  "look  ahead  and  think  back."  The  reflection 
involved  in  the  process-from  beginning  stages  using  limited  knowledge,  to  final  stages  at  the 
conclusion  of  inquiry— creates  greater  opportunities  for  students  to  undergo  conceptual  change  rather 
than  simple  incremental  addition  of  new  facts.  As  Brophy  explains: 

This  cyclical  process  of  active  research  and  reflection  on  the  process  provides  an  excellent 
opportunity  for  students  to  generate  their  own  understanding  of  the  content  knowledge. 


A Variation  on  the  WebQuest  model 

STAR. Legacy  mits  require  more  comprehensive  resources  and  design  than  most  teachers  are  able  to 
include  in  a WebQuest.  Even  so,  WebQuests  can  do  more  to  encourage  reflection  and  active  use  of 
knowledge.  Our  project  incorporates  elements  of  the  STAR.Legacy  model  to  create  a more  schema-based 
approach  to  WebQuest  design  (see  Fig.  2 below). 
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Figure  1:  A four -stage  reflection  process  can  be  triggered  by  an  outside  assignment  or  by  continuing  work 
of  an  individual  of  team. 

This  cycle  of  reflection  can  be  prompted  by  an  external  challenge,  as  in  WebQuests,  or  by  the  natural  and 
continuing  interests  of  team  members.  We  see  this  reflection  cycle  as  a variation  on  the  Star.Legacy  and  K- 
W-L  models  of  inquiry  learning  (Ogle,  1986). 

Specifically,  the  WebQuest  model  is  revised  at  two  main  points: 

1.  What  Do  You  Think?  following  the  task.  Students  need  opportunities  to  test  and  use  their  initial 
understandings  before  engaging  in  specific  search  activity.  We  suggest  that  once  students  have  been 
introduced  to  the  Task  in  step  2 of  die  WebQuest,  they  be  given  a chance  to  propose  possible 
solutions  to  the  problem  based  on  their  current  knowledge  and  understandings.  What  Do  You  Think? 
requires  team  members  to  pool  their  initial  knowledge  concerning  the  topic  of  research,  and  to  present 
that  knowledge  as  a basis  for  inquiry. 

There  are  specific  advantages  to  adding  this  step.  First,  by  allowing  students  to  articulate  possible 
solutions  to  the  task  or  problem,  both  students  and  teacher  can  identify  students'  existing  knowledge. 
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Second,  this  first  pass  at  a solution  provides  a benchmark  against  which  students  can  compare  their 
conceptual  growth  over  the  course  of  the  WebQuest.  Third,  the  collaborative  discussion  that  occurs  as 
students  share  their  ideas  allows  them  to  understand  the  thinking  of  the  other  members  of  their  team 
(van  Zee  & Minstrel  1,  1997).  And  finally,  it  may  become  apparent  to  the  teacher,  based  on  the 
solutions  generated  at  this  stage  that  one  or  more  steps  of  the  WebQuest  activity  need  to  be  revised. 
For  example,  it  may  be  discovered  that  the  information  sources  identified  in  step  4 of  the  WebQuest 
are  inadequate.  Or,  a teacher  may  uncover  that  the  task  itself  needs  to  expanded  or  narrowed  to  make 
the  job  of  problem  solving  more  meaningful  to  students. 

2.  Share  and  Compare.  Students  also  need  time  to  debrief  and  reflect  with  others  as  they  complete 
WebQuest  activities.  The  Conclusion  step  of  most  WebQuests  often  seems  to  be  an  afterthought 
rather  than  an  essential  component.  We  believe  that  this  step  should  allow  students  to  take  an  active 
role  in  assessing  their  conceptual  growth  throughout  the  course  of  the  WebQuest  experience.  This 
could  be  easily  done  by  comparing  and  contrasting  the  final  team  solution  to  the  problem  or  task  with 
their  initial  attempts  at  problem  solving  articulated  in  the  new  “What  do  you  think?”  step  that  we  have 
added  earlier.  Share  and  Compare  provides  a structure  for  the  public  sharing  of  research  findings, 
then  feeding  those  findings  into  further  inquiry  questions. 

Public  sharing  of  findings  allows  all  students  in  the  class  to  experience  the  solutions  offered  by  each 
of  the  teams.  This  opens  up  opportunities  for  large-group  analysis  of  the  solutions  and  further 
opportunities  for  conceptual  development.  Active  sharing  will  also  allow  the  teacher  to  help  link  the 
understandings  and  knowledge  that  students  gain  from  the  WebQuest  experience  to  the  rest  of  the 
curriculum  for  the  course.  This  linkage  back  to  the  curriculum  is  an  oft-neglected  aspect  of  WebQuest 
use. 


A Modified  WebQuest 

To  illustrate  how  a typical  WebQuest  can  be  modified  for  increased  reflection,  we  have  conducted  a 
makeover  of  a WebQuest  preparing  5th  grade  students  for  a field  trip  to  Southern  Colorado  (see 
http://ouray.cudenver.edu/~dlOyoung/adam/html_files/  for  the  revised  site).  In  this  WebQuest,  students 
study  a site  they  will  visit  and  prepare  inquiry  questions  relating  to  that  site. 

Three  pages  are  added  to  the  site  to  increase  support  for  reflection  and  conceptual  change:  Teacher  Page, 
What  Do  You  Think?  and  Share  and  Compare.  The  Teacher  Page  presents  the  learning  objectives  for  the 
site  and  suggestions  for  appropriate  use.  The  Teacher  Page  also  presents  the  four -stage  model  (see  Fig.  2 
above).  As  part  of  the  What  Do  You  Know?  activity,  students  brainstorm  together  to  pool  their  initial 
knowledge  about  their  site.  In  the  Share  and  Compare  page,  students  submit  their  research  questions  and 
are  prompted  to  review  and  compare  questions  from  other  teams.  Once  the  teams  have  contributed 
questions,  the  whole  class,  working  under  the  teacher’s  direction,  can  decide  how  to  use  the  questions  as 
they  physically  visit  the  geographic  sites.  Alternatively,  the  visits  can  be  conducted  virtually,  with  similar 
kinds  of  sharing  and  reflection  activities. 

Contextual  Use  of  WebQuests 

It  is  important  to  remember  that  WebQuests  are  not  isolated  instructional  activities.  Instead,  they  are 
intended  for  use  within  the  larger  context  of  course  objectives  and  curriculum.  Encouraging  greater 
reflection  by  active  students  will  increase  the  likelihood  that  they  will  successfully  link  the  activity  back  to 
the  principles  and  ideas  of  the  curriculum.  Achieving  this  connection  between  specific  activities  and 
general  principles  is  a challenge  for  education.  We  see  WebQuests  as  powerful  tools  to  help  meet  that 
challenge. 

In  some  instances,  a WebQuest  may  be  used  as  the  instructional  set  for  a larger  unit  of  work.  In  other 
instances,  it  may  be  a culminating  activity.  Because  the  ability  of  students  to  think  and  reason  at  higher 
levels  varies  widely,  a teacher  may  elect  to  implement  a particular  WebQuest  in  different  ways  from  class 
to  class.  In  fact,  a teacher’s  first  administration  of  WebQuest  will  most  likely  vary  significantly  from  their 
later  use  of  the  strategy.  In  general,  we  need  to  examine  closely  how  teachers  choose  to  incorporate 
WebQuests  into  their  everyday  activities. 
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The  Need  for  Continuing  Research 

The  rapid  rise  of  the  WebQuest  as  an  educational  practice  has  outstripped  the  research  community's  ability 
to  provide  a solid  theoretical  foundation  for  its  use.  This  is  no  different  than  other  technologies  that  rise 
rapidly  into  use.  The  many  approaches  to  WebQuest  design  further  complicate  efforts  to  study  the  impact 
of  these  interventions.  We  see  this  dynamic  and  diverse  state  of  affairs  as  a healthy  condition  that  now 
provides  a fertile  opportunity  for  research  of  various  forms. 

We  have  proposed  modifications  to  the  original  WebQuest  model.  The  key  issue  for  research,  however, 
may  not  be  finding  the  perfect  model  but,  rather,  finding  ways  to  help  teachers  make  best  use  of  these 
models  in  their  teaching  practices.  We  intend  to  continue  studying  how  teachers  make  use  of  models  and 
frameworks,  and  how  they  adapt  these  frameworks  to  their  own  purposes  and  conditions.  Over  time,  we 
hope  to  better  understand  conditions  and  strategics  that  will  help  students  develop  higher- order  reasoning 
skills. 

References 

Dodge,  B.  (1995).  Some  thoughts  about  WebQuests.  Online: 
http://edweb.sdsu.edu/courses/edtec596/about_webquests.html 

Dodge,  B.  (1999).  WebQuest  Taskonomy:  A Taxonomy  of  Tasks.  Online: 
http://edweb.sdsu.edu/webquest/taskonomy.html 

March,  T.  (2000).  WebQuests  101 : Tips  on  choosing  and  assessing  WebQuests.  In  Multimedia  Schools  Magazine, 
October  2000.  Online:  http://www.infotoday.eom/MMSchools/octOO/march.htm 

Norman,  D.,  Centner,  D.,  & Stevens,  A.  (1976).  Comments  on  learning  schemata  and  memory  representation.  In  D. 
Klahr  (Ed.),  Cogiition  and  instruction.  Hillsdale,  NJ:  Erlbaum. 

Ogle,  D.  M.  (1986).  K-W-L:  A teaching  model  that  develops  active  reading  of  expository  text.  The  Reading  Teacher, 

39  (6),  564-570. 

Perkins,  D.  N.,  & Simmons,  R.  (1988).  Patterns  of  misunderstanding:  An  integrative  model  of  misconceptions  in  sci- 
ence, mathematics,  and  programming.  Review  of  Educational  Research,  58(3),  303-326. 

Pickett,  N.,  & Dodge,  B.  (2001).  Rubrics  for  Web  Lessons.  Online: 
http://edweb.sdsu.edu/webquest^rubrics/weblessons.htm 

Posner,  G.  J.,  Strike,  K.  A.,  Hewson,  P.  W.,  & Gertzog,  W.  A.  (1982).  Accommodation  of  a scientific  conception: 
Toward  a theory  of  conceptual  change.  Science  Education,  66(2),  21 1-227. 

Schwartz,  D.  L.,  Lin,  X.,  Brophy,  S.,  & Bransford,  J.  D.  (1999).  Toward  the  development  of  flexibly  adaptive 
instructional  designs.  In  C.  M.  Reigeluth  (Ed.),  Instructional-design  theories  and  models.  Volume  2:  New  paradigm  of 
instructional  theory  (pp.  1 83-214).  M ahwah  NJ:  Erlbaum.  Also  online: 
http://peabody.vanderbilt.edu/ctrs/ltc/schwartzD/STAR.Legacy.html 

Spartenburg  County  Schools  District  3 (2001).  Cyberinquiry.  Online:  http://www.spa3.kl2.sc.us/cyberinquiry.htm 

Strike,  K.,  & Posner,  G.  (1985).  A conceptual  change  view  of  learning  and  understanding.  In  L.  H.  T.  West  & A.  L. 
Pines  (Eds.),  Cognitive  structure  and  conceptual  change.  New  York:  Academic  Press. 

van  Zee,  & Minstrell,  J.  (1997).  Using  questioning  to  guide  student  thinking.  The  Journal  of  the  Learning  Sciences,  6 
(2),  227-269. 


2159 


Integrating  the  Internet  in  ESL  Education  in  a Senior  High  School 


Shelley  S.  C.  Young 

Center  for  General  Education  and  Institute  of  Systems  and  Applications 
National  Tsing  Hua  University,  Hsin-chu 
Taiwan 

scy@mx.nthu.edu.tw 

Abstract:  This  qualitative  study  explores  the  potential  impacts  of  one  application  of  the  Internet -on 
high  school- level  English,  including  29  students  and  an  instructor.  Based  on  the  findings,  the  researcher 
concluded  that  the  Internet  could  motivate  student  learning  and  provide  students  with  a less 
stressful  environment  to  write  down  their  opinions  and  thoughts  freely  on  the  Internet,  but  the 
Internet  alone  could  not  improve  student  English  writing  in  terms  of  grammatical  proficiency. 
Recommendations  were  made  for  maximizing  the  educational  benefits,  and  minimizing  the  limitations, 
of  using  the  Internet  in  the  Internet  English  classroom  to  develop  the  confidence  and  writing  ability  of 
students. 


Introduction 

With  the  rise  in  the  popularity  of  information  communication  technology  among  the  younger  generation, 
there  is  a pressing  need  for  the  integration  of  technology  in  education  and  for  further  study  on  domain -specific  effects. 
Obviously  nowadays  English  is  the  mostly  used  language  in  cyberspace.  This  need  is  particularly  acute  for  the 
domain  of  teaching/leaming  English  as  a second  language  because  of  the  features  of  the  Internet  and  the  abundance  of 
English  learning  resources  freely  available  on  the  Internet.  Currently,  for  example,  more  and  more  websites  intended 
to  be  free  English  virtual  learning  and  teaching  environments  for  listening,  speaking,  reading  and  writing  have  been 
sprouting  up  on  the  Internet  over  the  past  few  years. 

The  emerging  applications  of  the  Internet  in  educational  settings  have  energized  learning  (Ian,  1999;  Lee, 
2000;  Krajka,  2000;  Gerald,  1998).  The  Internet  has  the  potential  to  provide  a new  learning  environment  that  has  rich 
digital  textual,  graphic,  audio,  video  and  other  interactive  features  for  the  language  learning  approach  as  well  as  for 
culture  (Muehleisen,  1997).  The  Internet  is  considered  a key- factor  in  enhancing  the  learner's  motivation  for  both 
language  learning  and  linguistic  proficiency  (Lee,  2000).  In  recent  years,  language  teachers  are  exploring  ways  in 
which  information  communication  technology  can  be  employed  to  make  language  learning  more  effective  and 
motivating  for  students  (Kem,  1995;  Vallance,  1998;  Bricknall,  1999;  Donaldson& Kotter,  1999;  Wible,  et  al.,  2000). 
According  to  Muehleisen  (1997),  reasons  for  using  the  Internet  in  the  English  classroom  are  as  follows: 

• Learning  to  use  computers  provides  a strong  intrinsic  motivation  for  learning  English; 
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The  Internet  places  English  in  an  international  contexts; 
Internet  projects  are  interactive; 

Facilities  for  using  the  Internet  are  often  readily  available. 


On  the  Internet,  students  have  the  chance  to  communicate  and  collaborate  with  other  remotely  distributed 
classmates,  peers,  or  teachers  by  using  e-mail.  Communication  over  the  Internet  can  help  learners  create,  analyze,  and 
produce  information  and  ideas  more  easily  and  efficiently.  Research  shows  that  by  using  network  computers,  students 
can  be  empowered  by  the  capacities  of  the  technology  and  become  better  problem  solvers  and  better  communicators 
(Frizler,  1995;  Belisle,  1996;  Al-Kalitarii,  1999).  The  coniniunicative  potential  of  Internet-supported  classes  has 
recently  become  the  focus  of  much  attention  in  the  fields  of  education  and  research  (Kern,  1995;  Warschauer,  1997). 
With  the  integration  of  information  communication  technology  in  language  learning,  new  kinds  of  activities,  such  as 
electronic  communication  via  e-mail  or  chatrooms,  have  also  been  developed  to  help  students  (Liao,  1999;  Kokkas, 
1999).  Different  modes  include  synchronous  and  asynchronous  interactive  e-mail,  Webchat,  MOOs,  IRC-multimedia 
activities,  Web-based  reading,  and  task-oriented  activities.  Each  mode  fosters  a different  kind  of  linguistic 
competence,  and  calls  for  various  skills  (Negretti,  1999;  Liou,  2000;  Oliver,  1996).  Meanwhile,  teachers  also  face 
many  challenges  (Singhal,  1997).  Such  contexts  have  been  shown  to  facilitate  students’  willingness  to  participate  in 
discussion,  as  evidenced  in  the  increased  volume  of  language  output. 


This  Study 

The  study  employed  both  qualitative  and  quantitative  methods.  The  researcher  used  a multi-method 
approach  to  document  and  evaluate  the  process  of  integration  of  the  Internet  in  the  English  leaning  settings,  including 
the  collection  of  their  electronic  data  from  chats,  discussions,  dialogues  and  email  exchanges  on  the  Internet, 
observations  of  their  online  classroom  activities,  formal  and  informal  interviews  with  the  instructor,  and  a post-class 
questionnaire. 

Specific  questions  explored  in  the  study  include  the  following: 

1.  Can  students  learn  the  functions  of  the  Internet  via  the  English  language  and  significantly  improve  their  English 
e-talk  on  the  Internet? 

2.  What  are  the  benefits  or  difficulties  with  the  use  of  Information  Technology  in  the  teaching  of  English  as  a 
Second  Language? 

3.  What  are  the  issues  and  challenges  that  arise  in  the  integration  of  the  Internet  in  English  learning  contexts? 

The  research  project  was  conducted  at  a vocational  high  school  in  the  northern  part  of  Hsin-chu  city.  The 
course  was  geared  to  intensive  study  and  was  named  Internet  English;  it  was  offered  by  a young  male  English  teacher. 
The  goal  of  this  course  was  to  use  various  methods  to  facilitate  students’  Internet  usage  by  way  of  English  writing.  A 
two -hour  on-line  lecture  plus  a computer  session  was  held  on  Tuesdays  in  a computer  lab  where  thirty  Personal 
Computers  were  set-up  and  networked.  In  the  lab  the  teacher  lectured  on  some  introductory  aspects  of  the  Internet 
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and  its  functions  and  at  the  same  time  demonstrated  its  uses  and  gave  students  opportunities  for  practice  using  English 
and  exploring  the  Internet.  In  addition  to  the  weekly  lecture  and  computer  session,  the  after-class  activities  and 
assigned  tasks  were  required  of  each  student.  The  online  lecture  was  divided  into  four  sections,  with  each  section 
focused  on  one  of  the  following  topics:  the  World  Wide  Web,  Newsgroups,  Email  and  MOOs.  To  carry  out  the 
after-class  projects  and  assignments,  the  students,  therefore,  needed  to  connect  to  the  Internet  and  search  for  the 
required  materials  in  response  to  the  teacher’s  weekly  questions  and,  meanwhile,  they  were  required  to  use  English  to 
communicate  with  the  teachers  as  well  as  with  the  other  classmates.  Moreover,  all  of  the  students  needed  to  write 
“letters”  to  email  pen  pals  in  other  countries  such  as  Japan,  U.S.A.,  and  Korea  and  address  several  topics  posted  by 
the  teacher. 

The  participants  in  this  study  were  29  third-year  high  school  students:  22  females  and  7 male  students  plus 
the  young  male  teacher.  The  course  was  conducted  in  two  semesters:  autumn/fall  semester,  from  September  2000  to 
January  2001  and  spring  semester  from  February  2001  to  June  2001.  The  results  reported  here  are  mainly  based  on 
the  data  for  the  first  academic  semester  of  the  project,  fromautumn  2000  through  spring  of  2001 . 

Findings 

Overall  results  of  the  use  of  information  technology  in  the  class 

The  teacher  used  an  open-ended  question  to  solicit  student  overall  response  to  the  use  of  the  information 
technology  in  the  class.  This  question  was,  “Do  you  think  Internet  English  can  really  improve  your  English?  Tell  me 
why.”  Most  students  thought  the  Internet  was  an  interesting  and  useful  tool  in  their  Internet  English  class.  Statistical 
data  indicate  that  nineteen  out  of  the  twenty -three  students  (19  out  of  23,  82.16%)  held  positive  attitudes  toward  this 
class,  while  three  out  of  the  twenty-three  (3  out  of  23,  13%)  had  negative  attitudes  and  one  gave  an  irrelevant  answer 
(1  out  of  23,  4.4%).  The  benefits  they  gave  varied  and  included  the  following:  1.  Being  able  to  practicing  typing  (5 
students);  2.  Learning  more  English  (4);  3.  Being  more  motivated  in  learning  new  vocabulary  (4);  4.  Being  able  to 
learn  more  about  the  computers  (4);  5.  Communicating  with  others  using  Emails  and  making  friends  from  other 
countries  (4);  6.  Looking  for  information  on  the  Internet  and  enhancing  reading  comprehension  by  reading  materials 
on  the  Internet  (3);  7.  Meeting  people  (2);  8.  Talking  on  line  (1);  9.  Being  able  to  correct  mistakes  instantly  (1). 
Negative  reasons  included  1.  Not  liking  the  Internet  (1);  2.  Too  much  vocabulary  resulting  in  learning  pressure  (1); 
The  fast-paced  schedule  of  the  class  (1). 

Students*  English  learning/improvement  in  terms  of  the  use  of  the  advanced  technology 

After  Reviewing  thoroughly  the  in-depth  qualitative  data  from  student  discourse  and  based  on  the  interview 
data  with  the  teacher,  the  researcher  concluded  that  the  Internet  could  motivate  student  learning  and  provide  students 
with  a less  stressful  environment  to  write  down  their  opinions  and  thoughts  freely  on  the  Internet,  but  the  Internet 
alone  could  not  improve  student  English  writing  in  terms  of  grammatical  proficiency.  The  most  common  mistakes 
students  made  in  their  e-writing  could  be  categorized  into  the  following  six  groups:  1.)  Spelling  mistakes;  2.)  Not 


capitalizing  a proper  noun;  3. ) Incorrect  word  combination;  4.)  Single  and  plural  disagreement;  5.)  Incorrect  phrases; 
and  6.)  Grammar  errors. 

Throughout  this  study,  the  qualitative  data  from  sentence  analysis  indicated  that  students  did  not  really 
improve  much  in  their  grammar  usages  in  making  sentences,  but  the  students  could  express  themselves  freely  on  the 
Internet.  Each  correct  sentence  was  almost  of  the  same  length,  since  longer  sentences  were  more  likely  to  contain 
more  grammar  mistakes.  Results  from  this  study  suggest  that  most  students  welcome  computers  as  a powerful  tool  to 
be  integrated  into  English  learning.  The  use  of  computers  and  computer  networks  does  not  seem  to  pose  concerns  for 
most  of  the  high-school  students.  Student  could  benefit  a lot  from  the  use  of  the  Internet  for  many  different  reasons, 
including  greater  volume  of  English  utterances  rather  than  the  grammatical  proficiency.  This  stuay  indicated  that 
integration  of  information  technology  in  the  Internet  English  facilitates  the  creation  of  a virtual  environment  that  thus 
transforms  learning  from  a traditional  passive  experience  to  one  of  discovery,  exploration,  and  excitement  in  a less 
stressful  setting. 

However,  data  collected  from  formal  and  informal  interviews  with  the  teacher  and  observations  indicated 
that  the  teacher,  although  he  was  very  familiar  with  the  computer  functions  and  was  active  and  enthusiastic  in  the 
integration  of  information  technology  in  the  English  class  to  maximize  the  learning  effect,  seemed  to  feel  more 
concerned  about  classroom  management  in  the  virtual  classroom  setting.  He  was  further  occupied  with  technical 
assistance  in  handling  computers,  troubleshooting  problems,  and  seeking  logistical  support  from  staff  to  help 
organize  digital  data  generated  in  the  pedagogical  settings.  In  general  preparing  a network  session  plus  taking  care  of 
the  after-class  online  activities  demanded  up  to  three  to  four  times  the  class  time  that  a traditional  session  required. 
The  author  hopes  that  the  paper  provides  some  insights  toward  a better  understanding  of  high  school  student  use  of 
the  Internet  in  English  classes  in  Taiwan.  More  studies  on  how  to  improve  student  grammar  through  English  writing 
practice  on  the  Internet  should  be  strongly  encouraged. 
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Abstract;  The  present  study  set  out  to  investigate  whether  the  negative  effects  associated  with  face-to-face  team 
competition  can  be  mitigated  with  the  support  of  networking  technologies  where  opponents’  identity  or  proximity 
can  be  manipulated  Results  indicated  that  subjects  in  the  face-to-face  team  competition  condition  rated 
statistically  significantly  less  favorably  on  inner-group  processing  than  those  in  the  decreased  proximity  and 
anonymity  conditions  Data  analysis  on  student  perception  toward  classroom  climate  revealed  that  subjects  in 
the  anonymity  condition  rated  statistically  significantly  more  favorably  on  classroom  climate  than  those  in  the 
other  two  conditions.  The  results  of  this  study  tend  to  lean  toward  the  position  that  to  promote  effective 
interactions  within  learning  groups  and  to  cultivate  more  regulated  classroom  climate,  networked  team 
competition  where  opponents  are  not  identified  may  be  the  preferred  instructional  strategy  to  adopt. 


Introduction 

Competition  has  been  suggested  as  a means  of  enhancing  the  motivational  appeal  of  educational  activities, 
and  as  a way  to  foster  learner  involvement  and  excitement  in  the  activity  (Butler  & Kedar,  1990;  Clinkenbeard, 
1989;  Malone  & Lepper,  1987).  Adding  an  element  of  competition  between  individuals  or  groups  is  widely 
believed  to  be  a motivational-enrichment  strategy  in  play,  work,  and  education  (Deci  Betley,  Kahle,  Abrams  & 
Porac,  1981).  Despite  the  fact  that  competition  is  encouraged  as  an  appropriate  goal  structure  in  the  form  of 
various  academic  contests  to  motivate  people  to  perform  to  the  best  of  their  ability,  the  negative  side  effects  of 
competitiveness  on  learning  has  provoked  discussion  and  suspicion.  While  empirical  bases  claim  that 
competition  has  negative  effects  on  learning  past  studies  were  primarily  conducted  in  traditional  classrooms 
involving  facoto-face  situations.  For  example,  Festinger  (1950)  found  that  defensive  reactions  are  more 
commonly  aroused  in  face-to-face  competing  groups.  Results  from  Hammond  and  Goldman  (1961)  yielded 
that  small  face-to-face  competing  groups  was  less  favorable  for  group  process.  High  school  students  in 
Garibaldi’s  study  (1976)  perceived  less  effective  communication  within  their  dyads  in  face-to-face  team 
competition  situations. 

Gleaned  from  the  literature  on  media  effects  and  human  factors  in  telecommunications,  face-to-face  and 
mediated  interaction  differ  in  many  psychological  significant  ways;  nevertheless,  evident  for  such  differences  on 
group  dynamics  have  only  been  examined  in  decision  making  situations  for  small  group  interaction  (Er  & Ng, 
1995;  Jessup,  Connolly  & Galegher,  1990;  Jessup  & Tansik,  1991 ; Valacich,  Dennis  & Nunamaker,  1992).  The 
effects  of  face4o-face  versus  networked  team  competition  where  opponents  are  identified  or  not  on  group 
dynamics  in  academic  learning  environments  have  yet  to  be  explored.  Within  the  limitations  imposed  by  the 
laboratory  conduct  of  experiments  it  is  rarely  possible  to  examine  the  issue  up  till  the  advent  of  networking 
technologies.  Thus,  the  present  study  set  out  to  examine  the  effects  of  anonymity  and  proximity  feature  of 
computer  telecommunications  on  group  dynamics  within  competing  groups 

A number  of  questions  could  be  examined  within  this  context.  For  example,  would  face-to-face  and 
networked  team  competition  generate  different  atmosphere  of  competition,  which,  in  turn,  construes  different 
group  dynamics  within  the  learning  groups  and  the  overall  classroom  climate?  More  specifically,  would  close 
proximity  in  face-to-face  team  competition  situations  be  less  favorable  for  group  process?  In  other  words, 
would  face-to-face  competing  groups  suppress  inner -group  information  exchange  and  sharing  due  to  the  fact  that 
competitors  sitting  next  to  them  might  overhear  their  answers  to  the  posted  questions  and  that  what  their 
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competitors  exhibited  during  the  game  will  distract  hem  from  concentrating  on  the  task  at  hand?  Might 
opponent  proximity,  physical  separation  and/or  loss  of  identification  provoke  different  defensive  reactions  and 
behaviors  among  competitors,  which,  as  a consequence,  engender  different  classroom  climate?  The  data 
yielded  from  this  research  can  provide  informative  and  practical  suggestions  regarding  the  suitability  of 
embedding  elements  of  anonymity  and/or  physical  separation  among  competing  groups  for  real-time  game-based 
competitive  learning  environments. 


Methods 

Participants 

Three  classes  from  the  fourth-grade  level  of  one  primary  school  in  the  southern  part  of  Taiwan  were 
selected.  In  total,  103  fourth- graders  (ages  D-11)  participated  in  the  actual  study  for  three  instructional 
sessions  in  three  consecutive  weeks.  All  student  s in  the  participating  classes  had  been  taking  computer  literacy 
classes  for  two  years,  and  possessed  a fundamental  knowledge  of  computer  hardware  operations. 


Learning  systems 

A synchronous  computerized  competitive  learning  system,  called  Joyce,  was  devised  for  the  study.  Joyce 
with  its  origin  dated  back  from  1972,  in  essence,  adopts  a board  game  format  that  allows  learners  practice 
answering  multiple- choice  questions  with  competing  opponents  simultaneously  face-to-face  or  via  network 
(Chang,  Yang,  Wang,  Yu  & Chan,  2001>  Various  functions  and  rules  including  shortcuts,  funny  questions, 
random  number,  bump,  spaces  move  control,  and  icon  selection  were  built  into  the  system  to  increase  its  intrinsic 
motivational  values  to  the  user  (Yu,  Chang  Wang,  Han,  Chan  & Liu,  2001). 

The  main  interface  of  the  Joyce  system  (see  Figure  1)  is  divided  into  six  parts.  Part  1 Question  Area 
presents  the  item  stem  and  four  alternatives  through  a pop-up  window.  Part  2 Feedback  Area  provides 
information  on  players'  responding  status,  for  instance,  who  has  not  keyed  in  their  answers  to  the  question  yet, 
and  the  correct  answer  for  the  question,  etc.  Part  3 Dice  Rolling  and  Combination  Choice  Area.  After 
answering  each  question  correctly,  a simulated  dice  will  give  a random  number  from  one  to  six.  After  two 
questions  answered  correctly,  students  can  determine  the  number  of  steps  that  the  icon  can  move  on  the  board  via 
three  different  combinations  of  the  two  numbers.  Part  4 Route  Map.  There  a-e  31  stops  in  each  round.  In 
addition  to  general  questions,  which  are  related  to  a specific  subject  matter  and  instructional  topic,  five  funny 
questions  and  four  shortcuts  are  included  in  the  route  map.  Part  5 Icon  Window  allows  the  player  to  choose  and 
change  the  icon  to  represent  him/her  to  move  around  the  board  game.  Part  6 Win/Loose  Status  Window  shows 
the  number  of  games  won  by  both  sides  of  the  competing  parties.  When  the  player  wins  a game,  he/she  will  be 
promoted  to  a higher  step  up  the  ladder. 


Multiple-choice  questions  corresponding  to  large  class  instruction  on  social  studies  and  natural  science  in  the 
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participating  school  were  built  in  the  system  for  thepurpose  of  this  study.  Specifically,  “Folk  custom  and  life  of 
Taiwan”  was  taught  in  social  studies.  “Omnipresent  water,”  “Underwater  creatures”  and  “Photosynthesis”  was 
taught  in  natural  science.  All  test  items  built  in  Joyce  were  drawn  directly  from  textbook  publisher’s  test  banks 
and  student  study  guides.  Sample  items  included,  "Which  ethnic  group  in  Taiwan  like  to  express  their  feelings 
using  lyrics  and  songs  during  leisure  times  (Hakka,  Fukien,  Ya-Mei,  A-Mei)?  Which  one  of  the  following  does  fish 
use  for  breathing  under  water  (skin,  gills,  lung,  scale)?  What  color  will  the  water  with  a weak  base  indicator 
change  if  you  blow  C02  into  it  (from  blue  to  transparent,  from  blue  to  yellow,  from  transparent  to  yellow,  from 
transparent  to  blue)?" 

Treatment  Conditions  and  Experimental  Procedures 

To  examine  anonymity  and  proximity  factors  in  synchronized  computerized  team  competitbn  environments, 
three  different  team  competition  modes  were  derived  in  the  study.  Considering  normal  arrangements  for 
computer  literacy  instruction  for  fourthgraders  at  the  participating  school,  which  is  two  students  per  computer, 
in  all  treatment  conditions  students  were  randomly  re-assigned  to  groups  of  two  to  compete  with  their  randomly 
assigned  opponents  for  the  duration  of  the  study. 

In  Treatment  A ^he  face-to-face  team  competition  condition),  pairs  of  students  were  competing  with  their 
randomly  assigned  opponents  sitting  next  to  them.  In  Treatment  B (the  decreased  proximity  condition),  names 
of  the  competing  teams  were  announced  before  the  activity  but  competing  teams  were  intentionally  displaced 
away  from  each  other  while  interacting  in  the  game  via  networking  technologies.  To  examine  the  effect  of 
proximity,  competing  teams  in  Treatment  B weresitting  at  least  15  feet  away  from  each  other  in  the  computer  lab. 
In  Treatment  C (the  anonymity  condition),  competing  teams  were  not  identified  throughout  the  experiment. 
Pseudonyms  were  randomly  selected  from  a list  prepared  for  the  study  for  each  dyad  and  used  to  log  in  the 
system.  Pairs  of  students  were  competing  with  unidentified  opponents  via  networking  technologies. 

To  avoid  teacher’s  effect,  one  elementary  school  teacher  was  trained  to  implement  the  study.  An  instructor's 
manual  delineating  all  the  procedures  and  directions  to  participants  was  developed  to  standardize  the 
experimental  procedures.  To  further  secure  that  experimental  procedures  were  accurately  carried  out  during  the 
study,  a training  program  was  provided  to  the  implementer  prior  to  the  actual  study . Each  classroom  was 
observed  by  the  investigator  to  verify  that  all  experimental  conditions  were  implemented  appropriately  to  ensure 
experimental  fidelity  throughout  the  whole  study.  In  addition,  two  pilot  studies  were  conducted  to  ensure  that 
the  learning  system  was  operating  accordingly  and  the  experimental  procedures  for  each  of  the  treatment 
conditions  was  adequately  and  appropriately  implementedprior  to  commencement  of  the  study. 

In  the  present  study,  students  participated  in  three  instructional  sessions  in  three  consecutive  weeks.  The 
study  was  conducted  during  computer  literacy  classes  in  the  school’s  computer  laboratory.  In  the  first  session, 
ground  rules  for  participating  in  the  activity,  including  ?Being  on  time?,  ? Staying  in  your  seats?,  ?Raising  your 
hands  before  posting  questions?,  and  the  purpose  of  the  learning  activity  (i.e.,  to  have  student  practice  answering 
questions  to  the  instructional  content  just  covered  in  whole  class  instruction)  were  announced  and  explained  via 
classroom  broadcasting  system.  Various  features  and  the  operating  procedures  of  the  learning  environment 
were  introduced  and  demonstrated  before  having  students  compete  answering  social  science  questions  with  their 
randomly  assigned  opponents. 

At  the  beginning  of  the  second  session,  the  ground  rules  and  procedures  for  running  Joyce  were  reviewed 
before  having  students  compete  answering  natural  science  questions  on  the  system  with  their  randomly 
le-assigned  opponents  on  computers  for  the  next  35  minutes. 

At  the  beginning  the  third  session,  before  having  pairs  of  students  compete  with  their  randomly  re-assigned 
opponents  for  the  next  15  minutes  on  social  science  and  natural  science  questions,  the  teacher  quickly  reviewed 
the  basic  procedures  involved  in  operating  Joyce.  Subjects  were  given  a questionnaire  to  be  completed 
individually  afterwards. 

Measurement  Instruments 

A post- experiment  self-report  questionnaire  was  disseminated  to  students  to  be  completed  individually.  The 
questionnaire  consisted  of  “Student  Perceptual  Impressions  of  In-group  Processing,”  and  “Student  Perceptual 
Impressions  of  Classroom  Climate.” 


A number  of  scales  from  previous  research  were  adapted  to  make  the  items  better  fit  the  experimenta  1 task 
and  target  population  used.  Specifically,  Yu’s  (2001)  ‘‘Perceptions  of  the  Communication  Process  Within  One’s 
Own  Dyad”  was  adopted  for  ‘‘Perceptions  of  In-Group  Processing”  Xiang’s  (1979)  ‘‘Classroom  Environment 
Scale”  and  Li’s  (1980)  ‘‘Learning  environment  Scale”  was  adopted  and  adapted  for  “Perceptions  of  Classroom 
Climate.”  Semi -structured  interviews  with  six  students  purposefully  selected  from  the  target  audience  and 
expert  reviews  were  undertaken  during  scale  development  to  ensure  validity  of  t he  measurement  instruments. 

The  questionnaire  consisted  a total  of  24  Likert-scale  items  to  collect  data  regarding  whether  anonymity  and 
proximity  influenced  in-group  processing  and  classroom  climate.  Nine  items  was  included  to  test  the 
hypothesis  relevant  to  in-group  processing.  Sample  statements  included,  “My  teammate  and  I discussed  our 
answers  to  the  posted  question  in  order  to  reach  consensus;  when  my  teammate  had  trouble  understanding  the 
rules  of  the  game  or  the  question  posted,  I assisted  him/her  during  the  activity.”  1 5 statements  were  included  to 
test  student  perceptual  impressions  of  classroom  climate.  Sample  items  included,  “During  the  activity,  almost 
all  students  can  keep  silent;  during  the  activity,  class  order  is  usually  out  of  control;  Most  of  the  participants  are 
focused  during  the  activity.” 

Each  statement  was  rated  on  a five^art  discrete  scale,  with  corresponding  verbal  descriptions  ranging  from 
“strongly  disagree”  through  “disagree,”  “no- opinion,”  "agree,"  to  "strongly  agree."  To  counteract  possible 
respons&set  tendencies,  both  positive  and  negative  statements  were  included  in  the  instrument.  The 
questionnaire  was  scored  according  to  the  number  of  “strongly  disagree,"  "disagree,"  "no  opinion,"  “agree" 
"strongly  agree"  responses  a indent  selected.  Each  response  received  a weight  of  1,  2,  3,  4,  or  5 respectively. 
Scoring  on  the  negative  statements  was  reversed  so  that  negative  and  positive  responses  could  be  summed  and 
averaged  with  higher  scores  reflecting  more  positive  attitudes.  The  internal  consistency  reliability  (coefficient 
alpha)  for  each  of  the  scales  on  the  post -experimental  questionnaire  calculated  after  the  actual  study  was  .8681 
and  .7438,  for  in-group  processing  and  student  perceptual  impressions  of  classroom  climate,  respectively. 


Results  and  Discussion 

The  means  and  standard  deviation  (SD)  values  for  student  perceptual  impression  of  inner-group  processing 
and  classroom  climate  for  different  team  competition  modes  were  listed  in  Table  1.  Collected  data  were 
analyzed  using  the  analysis  of  variance  technique  on  each  of  the  two  dependent  variables  followed  by  Scheff6 
multiple  comparison  tests  when  a significant  main  effect  is  found. 


Treatment  A,  N=34 
M (SD) 

Treatment  B,  N=3  5 
M (SD) 

Treatment  C,  N=34 
M (SD) 

Perceptions  of 
inner- group  processing 

32.2647  (7.4684) 

37.5143  (6.2420) 

36.2353  (7.0970) 

Perception  of 
classroom  climate 

48.3824  (7.4955) 

47.4571  (7.4057) 

56.6176  (7.8971) 

Table  1:  Descriptive  Statistics  of  Different  Team  Competition  Modes  on  Student  Perceptions  of  Inner-Group 
Processing  and  Classroom  Climate 

According  to  the  data  analysis  conducted  on  student  perceptions  of  in-group  processing  the  study  found  a 
statistically  significant  effect  for  different  team  competition  modes,  F (2,100)=5.326,  p<.01.  Scheff6  multiple 
comparison  tests  revealed  that  subjects  in  Treatment  A ^he  face-to-face  team  competition)  (M=32.2647)  rated 
significantly  less  favorably  on  inner-group  processing  than  those  in  Treatment  B (the  decreased  proximity 
condition)  (M=37. 5143)  and  Treatment  C (the  anonymity  condition)  (M=36.2353)  at  .009  level,  and  .067  level, 
respectively. 

Data  analysis  on  subject  perception  toward  classroom  climate  yielded  a statistically  significant  effect  for 
different  team  competition  modes,  F (2,  100)=1 5.065,  p<.001.  Scheffe  multiple  comparison  tests  further 
revealed  that  subjects  inTreatment  C (the  anonymity  condition)  (M=56.61 76)  rated  significantly  more  favorably 
on  classroom  climatethan  those  inTreatment  A (face-to-face  team  competition)  (48.3824)  and  Treatment  B (the 
decreased  proximity  condition)  (M=47.4571  )both  at  .000  level. 

The  present  study  compared  the  effects  of  three  different  team  competition  modes,  i.e.,  face-to-face,  decreased 
proximity  and  anonymity  team  competition,  upon  inner-group  processing  and  classroom  climate.  Data  from  the 
present  study  support  our  proposition  that  anonymity  and  proximity  are  more  favorable  for  group  process. 


Distinguishing  aspects  of  telecommunications  media,  that  is,  anonymity  and  proximity,  did  have  an  influential 
mediating  effect  on  human  behavior  in  present  study.  Students  participating  in  the  physical  separation  and 
anonymity  competitive  conditions  perceived  themselves  to  be  focusing  more  on  information  sharing  and 
exchanging  as  compared  to  the  face-to-face  team  competition  condition. 

It  is  suggested  by  social  psychologists  that  nonverbal  information  transmitted  in  a face-to-face  situation  may 
involve  visual  cues  that  distract  the  participants  from  formulation  of  their  own  argaments  (Short,  Williams  & 
Christie,  1976).  With  competing  teams  unidentified  and  dispersed,  cooperating  pairs  tended  to  encourage 
attention  to  the  content  of  a message  and  task  at  hand  rather  than  to  the  nominal  characteristics  of  the 
face-to-face  interacting  parties,  which  may  distract  from  that  (Maxx,  1999).  In  other  words,  anonymity  and 
reduced  proximity  minimize  the  in-group  process  losses  associated  with  face-to-face  team  competition  situations 
and  induce  more  task- related  interaction  and  effective  communication  within  groups 

The  present  study  also  found  that  networked  competition  where  competitors  were  anonymous  rated 
significantly  more  favorably  on  classroom  climate  than  those  in  other  two  treatment  conditions.  That  is,  pairs 
competing  with  unidentified  competitors  tended  to  rate  lower  toward  statements  on  the  subscale  like:  It  was  loud 
and  noisy  during  the  activity;  The  atmosphere  during  the  activity  makes  it  hard  for  me  to  concentrate;  during  the 
activity,  class  order  is  usually  out  of  control , etc,”  as  compared  to  the  face-to-face  and  physically  dispersed  team 
competition  modes.  That  is,  with  competitors  identified,  the  atmosphere  exhibited  in  the  classroom  seems  to  be 
noisy,  out  of  order,  unmanaged,  and  unregulated  no  matter  variability  of  competing  group  proximity. 


Conclusions 

The  capabilities  of  communication  and  computer  technologies  offer  interested  parties  the  opportunity  not 
to  be  identified  and  be  physically  separated  from  each  other  while  interacting  with  each  other  within  the  system. 
The  purpose  of  this  study  is  to  examine  the  effects  of  competing  groups  anonymity  and  proximity,  two 
important  components  manipulable  with  the  support  of  networking  technologies,  on  group  process  of 
cooperating  dyad.  Data  from  the  present  study  found  that  anonymity  and  decreased  proximity  of  competitors  is 
conducive  to  effective  in-group  processing  and  information-sharing  behaviors.  The  results  of  this  study  tended 
to  lean  toward  the  position  that  to  promote  constructive  interactions  and  to  enhance  regulated  classroom  climate, 
networked  team  competition  where  opponents  were  not  identified  might  be  the  preferred  instructional  strategy  to 
adopt,  as  compared  to  face-to-face  team  competition.  Gleaned  from  the  results  from  the  present  study,  it  is 
recommoided  that  designers  of  competitive  game-based  learning  systems  are  well  advised  to  include  the  feature 
of  anonymity  in  their  system  to  support  student  learning. 

This  is  the  first  empirical  evidence  investigating  the  potential  of  network  team  competition  on  group  dynamics 
with  the  support  of  networking  technologies.  Future  studies  with  longer  duration  of  experimental  time, 
different  age  group  subjects,  content  areas,  interaction  tasks  and  dependent  measures  of  learning  outcomes  may 
be  needed  to  extend  the  EOpe  and  extent  of  the  generalizability  of  present  study.  Finally,  the  present  study 
involved  pairs  of  participants  competing  with  each  other  in  objective-type  question -answering  practice.  As 
group  process  are  more  complex  than  dyadic  ones  while  engaged  in  problem-solving  questions,  further  studies 
involving  various  group  size  and  different  question  types  might  make  finer  distinction  so  as  to  make  practical 
recommendations  regarding  the  particular  situations  in  which  anonymity  and  proximity  could  be  used  for  the 
enhancement  of  learning  outcomes. 
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ABSTRACT 

The  World  Wide  Web  can  serve  as  a complete  learning  and  educational  environment  and  poses  new  challenges. 
Testing  and  assessment  are  value-added  services  in  a learning  process,  which  aim  at  evaluating  and  measuring 
the  outcomes  of  learning  experiences.  Developing  tests  in  an  instructional  efficient  and  effective  manner  is  a 
difficult  task,  especially  when  this  is  implemented  over  the  web.  In  this  paper  we  describe  the  main  features  and 
functionalities  of  a web-based  testing  system  that  deploys  state-of-the-art  technologies  providing  the  instructors 
and  learners  with  a powerful  web-based  testing  environment. 

1.  INTRODUCTION 

1.1  Background 

An  increasing  number  of  researchers  and  practitioners  have  been  motivated  to  discuss  and  develop  Computer  — 
based  and  Web- based  testing  systems,  in  recent  years.  These  kinds  of  systems  should  provide  certain  basic 
instructional  functionalities  like  other  web -based  learning  and  education  systems  (Overbaugh,  1994).  On  the 
other  hand,  instructors  must  recognize  the  need  to  design  assessment  appropriate  to  the  medium.  The  component 
of  the  assessment  in  an  online  learning  environment  is  of  great  importance.  Assessment  is  an  integral,  ongoing 
aspect  of  learning,  which  is  about  gathering,  analyzing,  quantifying  information  about  learner  performance. 
Comparisons  with  traditional  assessment  methods  suggest  that  the  use  of  computer  and  web-based  techniques 
may  improve  the  overall  performance  of  learners  (Bocij  and  Greasley,  1999).  The  major  benefit  of  computer- 
based  assessment  appears  to  be  the  improvement  of  task-focus.  If  performance  increases  and  completion  time  for 
the  test  decreases,  then  computer-based  assessment  enhance  the  ability  of  learners  to  focus  on  questions  and 
recall  relevant  information  (Bocij  and  Greasley,  1999). 

2.  DEVELOPMENT  OF  THE  SYSTEM 

2. 1 Goals,  motivation  and  basic  concepts 

Our  work  and  development  effort  focuses  on  answering  the  following  question: 

> How  can  we  provide  instructors,  teachers,  trainers  or  testing  authors  the  capability  to  construct  tests 
dynamically  over  the  web  in  an  efficient  and  effective  manner? 

The  main  focus  is  on  the  user  that  has  the  responsibility  to  create  tests,  no  matter  if  he/she  is  a teacher,  instructor 
(subject  matter  experts),  testing  author/designer  etc.  Additionally  two  other  questions  raised  our  motivation  to 
develop  this  web  - based  testing  system: 

> How  can  we  provide  them  (instructors,  testing  authors  etc.)  the  capability  to  review  the  tests’  results 
whenever  they  want  and  receive  feedback  in  a manner  that  will  help  them  to  adjust  their  learning  material 
and  courses? 

> How  can  we  motivate  the  learners  (test  takers)  and  provide  them  with  options  for  receiving  more 
customized  feedback? 

As  far  as  it  concerns  the  basic  concepts,  feedback  is  main  concept  of  our  system.  Providing  feedback  is  crucial 
for  the  orientation  and  motivation  of  the  learner.  In  the  web-based  testing  system  we  developed,  a learner  is  free 
to  choose  whether  he/she  wants  to  receive  feedback  while  he/she  is  executing  the  test.  Furthermore  a learner  is 
provided  with  the  option  to  receive  his/her  own  results  report  either  in  a text  format,  or  a graphical  presentation, 
pie  charts,  histograms  etc.  according  to  his/her  preference  and  learning  style  (Honey  and  Mumford,  1992).  Rich 
feedback  is  provided  to  the  instructors  also  through  the  reporting  capability  of  the  system,  which  keeps  tracks  of 
learners’  performance  and  progress.  This  kind  of  feedback  is  of  great  importance  since  it  helps  the  instructors  to 
evaluate  the  success  of  a learning  experience  and  adjust  the  learning  objectives,  courses  and  learning  materials  to 
the  level  of  learners’  performance.  Additionally,  addressing  issues  like  plagiarism  and  cheating  is  of  high 
importance  in  testing  environments  and  these  concepts  have  been  taken  into  account  while  designing  the  system. 
This  system  has  been  developed  towards  summative  assessment:  Careful  design  of  summative  assessment  is  an 
approach  that  can  reduce  plagiarism.  More  specifically  in  selected  response  type  of  assessments  (Rovai,  2000), 
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the  construction  of  a large  pool  of  items  can  be  of  great  importance  in  order  to  produce  tests  that  will  be  unique 
for  each  learner  and  to  reduce  cheating  or  plagiarism  (Howard,  1995)  (Camevale,  1999a).  Timed  tests  can  also 
decrease  the  opportunity  to  cheat  (Rovai,  2000). 

2.2  System  Architecture 

Following  the  3-tier  Architecture  model  with  a thin  client,  we  use  relational  database  technology  for  the  database 
layer,  Java  and  Java  Server  Pages  for  the  application  logic,  and  html  and  JavaScript  technology  for  the 
presentation  layer.  A common  browser  (Microsoft  Internet  Explorer  4.0  or  Netscape  Navigator  4.0  or  newer 
versions)  is  the  minimum  requirement  for  the  client. 

2.3  Basic  System’s  Modules 

The  system  we  developed  consists  of  the  following  four  basic  modules: 

n User  Management  Module:  It  is  the  module  that  enables  the  administrator  of  the  system  to  specify  the 
instructors  and  learners  as  well  as  the  subjects/topics  and  groups  of  learners,  respectively.  This  module  also 
includes  i)  the  management  of  roles  and  ii)  security  issues 

2)  Authoring  Module:  It  is  the  module  that  enables  the  instructors  to  construct  their  own  questions  and 
questionnaires  (tests),  as  well  as  to  manage  and  modify  the  question  pool.  This  module  is  about  authoring, 
storing  and  selecting  questions  and  tests.  The  main  user  of  this  module  is  the  instructor,  or  testing  author.  The 
types  of  questions  supported  by  the  system  are  the  following:  1)  True  or  False,  2)  Multiple  Choice,  3)  Multiple 
Response,  4)  Text  Question,  5)  Fill  in  The  Blanks,  6)  Ordering,  7)  Matching  and  8)  Composite. 

3)  Testing  Delivery  Module:  It  is  the  module  that  is  responsible  for  the  execution  of  the  tests  by  the  learners. 

4)  Reporting  Module:  It  is  the  module  that  gives  both  the  instructors  and  the  learners  the  capability  to  view  the 
results  of  the  tests  that  are  related  to  them. 

3 PILOT  TESTING 

3.1  Introduction 

The  evaluation  of  the  system  has  been  carried  out  in  real  environment  conditions  using  a selected  sample  of 
educators  and  learners.  The  objective  of  this  process  was  to  combine  the  outcomes  for  the  usability  and  the 
functional  evaluation  in  order  to  assess  the  acceptance  of  the  system  by  its  users,  acquire  further  suggestions 
concerning  its  functionality,  as  well  as  remove  technical  obstacles.  In  order  to  evaluate  every  aspect  of  the 
system,  a distinction  between  two  orthogonal  parameters  has  to  be  made: 

■ User  Interface  Parameters,  that  identifies  the  level  of  acceptance  of  the  User  Interface  (UI)  by  the  users. 

■ Functionality  Parameters,  measuring  the  level  of  compliance  of  the  system  with  the  user  requirements, 
upon  which  the  system  development  was  based. 

3.2  Methodology 

Usability  is  considered  as  a basic  parameter  of  the  quality  of  a system  and  it  was  a basic  component  of  the 
system’s  evaluation.  Particularly  in  the  case  of  developing  software  for  learning  and  educational  purposes,  there 
is  a clear  need  for  the  system  to  become  “invisible”  allowing  the  learner  to  focus  at  his/her  learning  tasks. 
Usability  testing  has  taken  place  in  laboratory  settings.  The  researchers  involved  in  this  work  conducted 
experiments  with  real  users.  As  already  mentioned,  the  testing  places  were  three  university  laboratories;  the 
testing  duration  was  2 hours  maximum  per  subject  and  the  experimenters  were  the  system  designers.  The 
purpose  of  this  test  was  to  record  the  reactions  and  responses  of  potential  users  of  the  system  regarding  mainly 
the  ease  of  use,  system’s  potential  marketing  application  and  its  improvement  possibilities.  For  the  testing 
process  a sample  test  was  designed  containing  20  various  type  questions  such  as  open,  dichotomously,  multiple 
choice,  multiple  answers  etc.  These  questions  were  related  to  general  knowledge  questions  as  well  as  Marketing 
and  Computer  Science  specific.  These  questions  were  imported  into  the  system  comprising  the  test  data  for  the 
testing  process.  The  results  obtained  lead  the  researchers  into  formulating  specific  suggestions  for  the 
improvement  of  the  system.  It  should  be  clear  that  before  starting  the  testing  session,  all  users  attended  a 30- 
minute  introductory  presentation  of  the  system  functionality  and  were  shown  the  testing  sessions  content.  The 
rationale  for  the  latter  is  that  users  would  become  accustomed  to  the  questions  mentioned  above  and  as  a matter 
of  fact,  to  be  more  focused  to  the  measured  tasks  and  user  interface  elements,  during  the  real  testing  sessions. 

3.3  Preliminary  Evaluation  Results 

As  part  of  the  evaluation  process,  we  designed  and  distributed  to  the  users  a questionnaire  that  has  been  devised 
for  the  evaluation  of  the  system.  The  questionnaire  consists  of  several  sub- sections,  each  dealing  with  different 
aspects  of  the  system:  the  design  of  the  GUI,  the  performance  of  the  system,  the  system’s  ease  of  use,  the  state 
of  the  documentation  and  other  issues  that  a user  may  be  able  to  identify.  For  the  purpose  of  the  evaluation  we 
have  already  established  6 evaluation  indicators,  and  we  are  trying  to  approach  the  systems  evaluation  from 
these  various  perspectives.  For  each  i ndicator  we  are  presenting  the  results  of  the  system  evaluation: 
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Indicator 

Result 

General  System  Issues 

Apart  ftxim  the  users’  response  to  question  A6,  which  is  related  to  the  ability  provided  by  the 
system  to  the  user  in  order  for  the  latter  to  avoid  making  mistakes,  the  judgment  of  the  i^ers 
regarding  the  general  issues  of  the  system,  is  positive  with  a large  percentage  of  the  subjects 
replying  verv  positively  or  partially  positively. 

Dialogs,  Formsand  Menus 

The  judgment  of  the  users  related  to  dialogs,  forms  and  menus  is  again  positive  with  a large 
percentage  of  the  subjects  replying  very  positively  or  partially  positively.  In  the  question  “Can 
the  keyboard  be  used  to  control  the  program  as  extensively  as  with  the  mouse?”  the  response  of 
the  subjects  was  rather  negative,  whereas  regarding  the  question  “When  accessing  a form,  is 
there  a logical  grouping  of  the  various  choices  on  the  form?”  it  was  probably  not  clearly 
imderstood  from  the  subjects  probably  due  to  formulation. 

User  Interaction  and 

Convenience 

Errors  and  Error  Messages 

The  results  of  the  questions  related  to  user  interaction  and  convenience  present  a great 
fluctuation.  Many  negative  responses  have  been  recorded  regarding  the  speed  of  the  system, 
progress  notification  and  short  key  availability. 

Apart  from  the  users  response  to  question  “Are  error  messages  categorized  by,  for  example,  a 
certain  subsystem  of  the  program,  by  an  error  code  or  some  kind  of  keyword?”,  the  judgment  of 
the  users  regarding  errors  and  error  messages  of  the  system,  is  positive  with  a large  percentage 
of  the  subjects  replvine  verv  positively  or  partially  positively. 

Help  and  System 

Documentation 

Overall  Assessment  and 
Evaluation 

The  results  of  the  questions  related  to  Help  and  System  Documentation  present  particularity 
since  there  is  a high  percentage  of  not  answered  questions.  This  is  possibly  due  to  the  fact  that 
users  haven’t  used  the  help  system  sufficiently  or  that  this  component  is  not  easily  accessible  or 
usable.  Most  received  answers  were  positive  apart  from  the  questions  related  to  the  provision  of 
examples  in  the  help  system  and  a printable  version  of  it. 

The  overall  assessment  and  evaluationscore  of  the  system  expresses  a generallygood  opinion  of 
the  users.  The  majority  of  the  answers  are  above  average  level,  whereas  a high  percentage  is 
between  7 and  8 out  of  10  (Very  Good). 

Table  1 : Evaluation  indicators  and  results 


4 FUTURE  WORK  AND  IMPLICATIONS 

Next  steps  in  our  research  include  the  development  of  the  next  version  of  the  system  by  taking  into  account 
users’  feedback  derived  from  the  pilot  testing.  Further  use  of  the  system  and  the  preparation  of  a large-scale 
experiment  in  educational  settings  (universities  and  training  organizations)  have  been  planned  for  usability 
testing  purposes.  Future  enhancements  concerning  next  versions  of  the  system  will  include: 

a)  The  addition  of  question  types  that  require  multimedia  features  and 

b)  The  integration  of  the  testing  system  with  other  learning  systems  such  as  Learning  Management  Systems  etc. 
Finally,  another  important  issue  to  be  taken  into  account  is  interoperability  and  the  conformance  with 
standardization  efforts  such  as  IMS-QTI  (IMS-QTI  Specification,  2000),  etc. 
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Case  Study:  Cognitive  Flexibility  Theory  and  the  Evaluation  of  Re- 
purposable  Learning  Activities 


Purpose:  The  purpose  of  this  study  was  to  examine  the  re-purposability  and  reusability 
of  “learning  activities”  used  in  two  courses  within  the  domain  of  cognitive  psychology 
and  explored  the  role  of  Cognitive  Flexibility  Theory  with  respect  to  instructors'  use  of 
learning  activities. 

Introduction;  The  main  purpose  of  this  study  is  to  investigate  the  role  of  Cognitive 
Flexibility  Theory  in  the  development  of  learning  activities  for  use  in  online  course  in  the 
domain  of  cognitive  psychology.  Cognitive  Flexibility  Theory  emphasizes  the  flexible 
reassembly  of  preexisting  knowledge  to  fit  the  needs  of  new  situation.  (Spiro  et  al,  1987) 
We  believe  that  it  is  important  for  instructors  of  courses  to  be  cognitively  flexible  when 
developing  and  re-purposing  learning  activities  for  their  courses.  In  order  to  attain 
cognitive  flexibility,  instructors  must  have  a working  mental  model  of  the  actual  domain, 
which  leads  to  the  re-purposability  of  the  learning  activities.  Mental  models  provide  “an 
explanation  of  why  procedures  work.”  (Black,  1992)  It  is  “an  image  of  the  different 
parts  and  objects  of  a mechanism  together  with  procedures  for  transforming  these  parts 
and  objects  to  form  a mental  simulation.”  The  focus  of  this  study  is  on  the  cognitive 
flexibility  exhibited  by  the  instructors  rather  than  the  students  of  the  courses. 

A learning  activity  can  be  as  simple  as  a graphic  or  as  complicated  as  a whole 
curriculum.  The  granularity  of  a learning  activities  can  be  broken  down  into  two  levels,  a 
Fundamental  Information  Object  (FIO)  or  a Combined  Information  Object,  (CIO).  An 
FIO  is  defined  as  a unit  independent  of  context.  For  example,  a single  graphic  of  a gear 
would  be  considered  an  Fundamental  Information  Object.  An  FIO  has  the  highest  degree 
of  reusability.  (Wiley,  1998)  A CIO  results  Ifom  combining  several  FIOs.  This  results 
in  added  context  but  lower  granularity.  For  example,  several  gears  put  together  with  a 
direction  arrow  and  a question  asking  in  which  direction  will  the  last  gear  turn  is  a CIO. 

The  goal  is  to  create  Generative  Instructional  Learning  Activities  that  allow  for  the 
repurposing  of  learning  activities.  A repurposable  object  is  developed  when  an  FIO  is 
taken  Ifom  a CIO  and  repurposed  for  a new  CIO.  Not  all  FIOs  can  combine  together  to 
form  a meaningful  CIO.  An  analogy  of  this  would  be  similar  to  how  atoms  combine  to 
form  molecules.  Not  all  atoms  are  able  to  combine  together.  Furthermore,  certain  atom 
combinations  lead  to  changes  in  the  overall  structure  of  the  molecule.  We  believe  that 
FIOs  should  not  be  repurposed  for  all  domains.  In  order  to  do  so,  the  creators  of  the 
learning  materials  must  be  able  to  be  cognitive  flexible  when  developing  the  learning 
activities. 

Case  Study:  Two  graduate  courses  in  the  domain  of  cognitive  psychology  were  used  in 
this  study.  The  study  was  conducted  after  the  semester  ended.  The  first  course.  Theories 
of  Human  Cognition  and  Learning  focused  on  theories  of  Cognitive  Psychology  and 
related  empirical  results.  Cognitive  Psychology  includes  such  topics  as  memory. 
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learning,  comprehension,  problem  solving  and  attention.  The  second  course.  Cognition 
and  Computers,  focused  on  applications  of  theories  and  results  from  artificial  intelligence 
and  cognitive  psychology  to  try  to  improve  uses  of  computers  and  related  technologies  in 
education.  Separate  instructors,  with  similar  backgrounds  in  the  field  of  cognition  and 
learning,  taught  the  courses. 

Both  instructors  created  learning  activities  specifically  for  their  respective 
courses.  Once  the  learning  activity  was  created,  it  was  stored  in  an  online  distance 
learning  platform  to  which  both  instructors  had  access.  As  the  instructors  were  creating 
their  modules  (CIOs),  they  could  look  online  at  the  other  instructor’s  course  with  the 
intent  of  gathering  learning  activities  that  could  be  re-purposed  for  their  own  use.  The 
learning  activities  were  created  as  the  semester  progressed  and  either  instructor  could 
check  to  see  if  new  objects  had  been  added.  There  was  no  enforced  structure  of  looking, 
nor  did  the  instructors  discuss  uses  and  implications  of  these  objects  between  themselves. 

Discussion:  The  fundamental  question  was  whether  the  re-purposing  of  these  objects 
was  feasible  and  if  instructors  were  able  to  be  cognitively  flexible.  Although  the  focus  of 
the  courses  differed,  reciprocal  exchange  of  learning  activities  was  prevalent.  For 
example,  one  instructor  created  a CIO  learning  activity  with  specific  instructional  intent, 
which  included  images,  lecture  notes,  journal  article  readings  and  a learning  activity.  The 
other  instructor  used  the  images  (FIO)  to  create  another  CIO  with  a different  learning 
purpose.  Thus,  through  this  re-purposing  of  the  learning  activity,  the  second  instructor 
was  displaying  cognitive  flexibility. 

Because  there  was  no  active  communication  between  instructors,  and  the  sharing  of 
learning  activities  occurred  naturally,  we  believe  that  the  instructors  demonstrated 
cognitive  flexibility  when  designing  their  own  learning  modules.  The  instructors 
indicated  in  interviews  that  they  were  able  to  be  cognitive  flexible  in  their  development 
of  the  materials  because  they  were  both  familiar  with  each  other’s  courses  and 
understood  the  original  purpose  of  the  learning  activity.  The  results  point  to  a need  for 
ftirther  evaluation  of  learning  activities  a feasible  instructional  design  tool  as  well  as  a 
measure  of  instructor  cognitive  flexibility. 

Implications:  This  study  is  important  because  it  demonstrates  a successful  case  for 
collaborative  instructional  design.  Both  courses  were  enriched  because  the  instructors 
were  able  to  draw  upon  common  resources  and  still  develop  unique  learning  activities  for 
each  course. 

This  also  allows  instructors  to  become  more  productive  with  the  development  of  their 
course  materials  and  cuts  down  on  redundancy  of  effort.  Another  implication  of  this  case 
study  points  to  a deepening  of  content  knowledge  and  reasoning  of  instructors  because  in 
order  to  be  cognitively  flexible  with  the  learning  activities  themselves,  instructors  must 
know  how  and  when  the  use  the  objects  appropriately.  If  the  instructors  are  more 
knowledgeable  about  the  content  and  are  cognitively  flexible,  they  can  pass  on  this 
knowledge  to  their  students  more  readily. 
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Abstract;  The  fundamental  ideal  that  access  to  education  and  information  as  one  of  our  basic 
human  rights  must  not  be  neglected  in  the  electronic  information  age.  This  ideal  however  is 
not  being  met  in  the  area  of  postsecondary  web  accessibility.  This  study  surveyed  350 
postsecondary  institutions  in  Canada  to  see  their  level  of  web  accessibility  using  the  Centre  for 
Applied  Special  Technologies  accessibility  tool,  Bobby.  It  was  found  that  14.9%  of 
postsecondary  institutions  surveyed  met  the  priority  1 checks  as  established  by  the  World 
Wide  Web  Consortium  (W3C)  Web  Accessibility  Initiative  (WAI,  1999),  and  only  1 .7%  were 
free  of  both  priority  1 and  priority  2 errors.  The  specifics  of  these  errors  however,  reveal  that 
with  the  recognition  of  the  issues  for  those  with  disabilities  and  the  web,  they  can  be  easily 
addressed. 


The  power  of  the  Internet  is  in  its  universality.  Access  by  everyone  regardless  of  disability  is  an 
essential  aspect.’  - Tim  Berners-Lee  (WAI,  2001) 

The  fundamental  ideal  that  access  to  education  and  information  as  one  of  our  basic  human  rights  must 
not  be  neglected  in  the  electronic  information  age.  With  many  facets  of  our  society’s  information  taking  a 
digital  form,  the  Internet  has  become  the  primary  source  of  information  for  many,  including  those  with 
disabilities.  Applications  such  as  the  World  Wide  Web,  video  conferencing,  and  email  provide  disabled  people 
with  the  potential  to  participate  in,  and  contribute  to,  education  in  a way  that  would  never  have  been  thought 
possible  ten  years  ago  (Booth,  et.  al,  2000).  Many  people  with  disabilities  today  use  a variety  of  assistive 
devices  to  participate  in  the  digital  world.  Screen  readers,  Braille  embossers,  screen  magnifiers,  and  voice 
recognition  technologies  give  some  disabled  people  the  ability  to  render  electronic  information  accessible.  But 
even  with  the  use  of  such  devices,  there  is  no  guarantee  that  they  will  be  able  to  access  the  information.  Poorly 
designed  Websites,  no  matter  what  assistive  device  is  used,  will  continue  to  hinder  the  user  from  deciphering 
and  accepting  the  message  and  information  contained  within.  Since  the  end-user  cannot  count  on  either  standard 
technology  or  other  enabling  devices  to  ensure  access  to  information  on  the  Web,  the  onus  is  on  the  Web 
developer  to  deliver  the  message  in  such  a way  that  allows  everyone  to  benefit  (Canada,  1998). 

Approximately  4.2  million  Canadians  and  approximately  7.4%  of  the  post  secondary  student  population  have 
some  type  of  disability;  it  is  mandatory  that  information  on  the  Web  is  accessible  to  them.  University  and 
College  Web  pages  serve  as  the  gateway  for  many  student  activities  such  as  checking  grades  and  class 
schedules,  even  down  to  the  simplest  task  of  finding  phone  numbers.  It  is  in  the  best  interest  of  the  institution, 
students,  and  prospective  students,  that  all  information  is  made  available  and  accessible.  This  goal  however, 
must  at  present  be  an  institutional  initiative,  as  the  guidelines  and  laws  in  this  area  are  yet  to  be  established  in 
Canada. 

Canada  currently  has  no  laws  governing  the  accessibility  of  electronic  and  information  technology. 
Although  Section  5 the  Canadian  Human  Rights  Act  does  protect  citizens  gainst  any  discriminatory  practice  in 
the  provision  of  goods,  service,  facilities,  or  accommodation  customarily  available  to  the  general  public 
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(Canada,  1976),  there  is  no  federal  legislation  that  governs  Web  accessibility.  There  are  however  guidelines  that 
have  been  established  within  two  federal  government  departments:  the  Public  Service  Commission  of  Canada 
and  the  Treasury  Board  of  Canada  Secretariat. 

The  Treasury  Board  Secretariat,  a federal  government  agency  whose  mission  is  to  help  the 
Government  of  Canada  manage  its  human,  financial,  information  and  technology  resources,  established  The 
Common  Look  and  Feel  for  the  Internet  (Canada,  2000),  a set  of  standards  and  guidelines.  These  guidelines, 
based  on  the  World  Wide  Web  Consortium  (W3C)  Web  Accessibility  Initiative  (WAI,  1999),  were  approved  on 
May  4,  2000  with  the  goal  of  having  all  government  Websites  meet  both  priority  1 and  priority  2 checkpoints  by 
December  31,  2002. 

The  Employment  Equity  Positive  Measures  Program  Directorate  of  the  Public  Service  Commission  of 
Canada  is  an  independent  agency  responsible  for  safeguarding  the  values  of  a professional  public  service: 
competence,  non-partisanship  and  representativeness.  In  conjunction  with  the  Adaptive  Technology  Resource 
Centre  at  the  University  of  Toronto,  they  have  developed  the  Universal  Internet  Access  Project  (Adaptive 
Technology  Resource  Centre,,!  996),  which  aims  to  provide  information  and  research  on  ways  to  enhance  the 
provision  of  accessible  information  on  the  Internet.  The  project  has  created  tools  for  Web  site  evaluation  and 
guidelines  for  Web  accessibility. 

Although  Canadian  agencies,  both  are  based  on  the  W3C’s  Web  Accessibility  Initiative  (WAI,  1999) 
which  aims  to  provide  accessibility  of  the  Web  through  five  primary  areas  of  work:  technology,  guidelines, 
tools,  education  and  outreach,  and  research  and  development  (WAI,  2001).  The  Web  Content  Accessibility 
guidelines  as  recommended  by  the  W3C  in  May  of  1999,  have  become  the  defacto  standard  for  the 
identification  and  resolution  of  Web  accessibility  issues,  and  serve  as  the  basis  for  the  development  and 
authoring  of  accessible  Web  resources.  These  guidelines  also  serve  as  the  basis  of  post  secondary  Web 
accessibility  studies  and  Web  accessibility  evaluation  tools,  such  as  Bobby  (Center  for  Applied  Special 
Technology,  2001). 

A number  of  U.S  research  studies  have  been  conducted  using  Bobby  as  a Web  accessibility  validation 
tool.  Although  these  studies  range  in  focus  from  distance  education  to  library  Web  information,  all  found  a 
large  gap  between  the  actual  and  ideal  for  Web  accessibility.  The  research  has  shown  that  the  current  status  of 
university,  library,  and  distance  education  Websites  are  fraught  with  accessibility  obstructions.  At  the  very  best, 
the  accessibility  figures  fall  between  30%  and  40%  (Schmetzke,  1999).  In  a study  that  evaluated  the 
accessibility  of  a random  sample  of  Web  pages  of  400  prominent  colleges  and  universities  in  the  U.S.,  Rowland 
and  Smith  (2000)  found  that  only  22%  had  front  pages  that  would  receive  Bobby  approval.  In  a follow  up  study 
by  Walden,  Rowland  and  Bohman  (2000)  the  number  rose  to  24%.  In  a similar  study,  Schmetzke  (2001) 
examined  the  Web  pages  of  219  post-secondary  distance  education  institutions  and  found  that  15.1%  were  free 
of  major  accessibility  errors.  The  trend  continues  with  21%  accessibility  for  a study  on  the  accessibility  of  Web 
sites  of  80  college  of  communications  and  schools  of  journalism  (Guthrie  2000),  and  27%  of  89  special 
education  program  homepages  (Flowers,  Bray  and  Algozzine,  1999). 

These  studies  show  that  there  is  a need  for  the  identification  and  evaluation  of  Web  accessibility  issues  at  the 
post  secondary  level.  These  types  of  studies  however,  are  not  evident  in  the  evaluation  of  Canadian  post 
secondary  institutions.  Therefore,  this  study  evaluates  the  accessibility  of  post  secondary  Canadian  institutions 
top-level  or  entry  level  Web  pages. 


Evaluation  tool 

Bobby  is  a free  application  developed  by  the  Center  for  Applied  Special  Technology  (CAST)  that  analyzes 
Web  pages  for  their  accessibility  to  people  with  disabilities  (CAST,  1998).  It  analyses  Web  pages  based  on  the 
Web  Content  Accessibility  Guidelines  1.0  as  proposed  by  the  World  Wide  Web  Consortium's  (W3C),  Web 
accessibility  initiative  (WAI,  1999).  Bobby  establishes  three  levels  of  priority  errors  based  on  the  checkpoint's 
impact  on  accessibility: 

• Priority  1.  Priority  1 errors  are  those  that  seriously  limit  a page’s  accessibilty.  Every  effort  must  be 
made  by  the  developers  to  ensure  that  these  errors  do  not  exist.  Satisfiing  this  checkpoint  is  a basic 
requirement  for  some  groups  to  be  able  to  use  Web  documents. 

• Priority  2.  Priority  2 errors  are  those  that  the  Web  content  developer  should  satisfy.  Otherwise,  one  or 
more  groups  will  find  it  difficult  to  access  information  in  the  document.  Satisfying  this  checkpoint  will 
remove  significant  barriers  to  accessing  Web  documents. 
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• Priority  3.  Priority  3 errors  are  those  that  a Web  content  developer  may  need  to  address.  Otherwise, 
one  or  more  groups  will  find  it  somewhat  difficult  to  access  information  in  the  document.  Satisfying 
this  checkpoint  will  improve  access  to  Web  documents  (WAI,  1999). 

Bobby  checks  for  all  three  priority  errors  in  its  evaluation.  The  number  of  each  priority  error  in  each  page  is 
displayed  in  the  tool  and  the  specifics  of  these  errors  are  displayed  in  a summary  report.  There  are  a number  of 
errors  however,  that  cannot  be  detected  in  the  automated  process.  For  this,  a list  of  user  checks  is  generated  that 
the  user  must  manually  go  through  while  evaluating  the  site.  In  order  to  be  recognised  as  a Bobby  “approved 
site,”  which  is  equivalent  to  WAI  Conformance  Level  A designation,  the  site  must  have  no  priority  1 errors. 
This  includes  both  priority  1 errors  detected  by  Bobby  and  the  manual  user  checks.  It  is  highly  recommended 
however,  that  all  Priority  2 checkpoints  also  be  satisfied,  when  evaluating  sites,  since  WAI  Conformance  Level 
AA  is  now  frequently  required  by  U.S  government  agencies  and  others  (CAST,  2001). 

For  this  study  the  downloadable  version  of  Bobby  3.2  was  used  to  evaluate  the  pages  and  generate  a summative 
report.  Priority  1 and  Priority  2 errors  were  summarized  for  this  study. 


Research  Methodology 

A list  of  357  post-secondary  Websites  were  identified  from  the  Canadian  Information  Centre 
for  International  Credentials  (CICIC,  2001).  This  list,  based  on  information  supplied  by  the  ministries  and 
departments  responsible  for  education  in  Canada,  is  the  only  Canada-wide  list  of  all  postsecondary  institutions 
recognized  by  the  competent  governments  in  Canada  (CICIC,  2001).  Seven  of  these  were  excluded  from  the 
study  due  to  sites  under  construction,  duplicate  sites,  and  those  links  that  were  down  at  the  time  of  the  study.  Of 
the  remaining  350  sites,  166  are  universities  and  184  are  colleges. 

Each  URL  from  the  list  was  viewed  and  the  URL  was  changed  if  it  did  not  represent  the  top  -level  information 
page  for  the  institution.  For  example,  if  the  site  provided  a splash  page  with  the  option  for  an  English  version  or 
French  version  of  the  page,  the  English  version  was  chosen  to  represent  the  top-level  information  page. 

The  list  of  sites  were  imported  into  Bobby  for  evaluation.  Priority  errors  were  evaluated  within  each  site  and  the 
summative  report  was  generated.  The  results  from  this  report  were  then  entered  into  an  Excel  speadsheet  for 
analysis. 

While  Bobby  did  identify  items  that  require  manual  user  checks  before  approval  can  be  granted,  most 
studies  of  a great  number  of  sites  (e.g.,  Rowland  & Smith,  2000;  Walden,  Rowland  & Bohman,  2000)  do  not 
perform  these  manual  checks  due  to  the  time  constraints  of  evaluating  the  source  code  of  all  these  sites.  These 
checks  however  must  be  done  in  order  to  display  Bobby  approval. 


Results 

Table  1 summarizes  the  results  of  the  Bobby  examination.  Of  the  350  post-secondary  sites  evaluated, 
Bobby  found  that  only  14.9  % (n=52)  of  the  University  and  College  top-level  Web  pages  were  free  of  priority  1 
errors.  Within  this  group  only  8.7  % (n=16)  of  the  college  Websites  and  21.7  % (n=36)  of  the  university 
Websites  were  error  free. 


Number  of  Sites  (n=350) 

Colleges 

(n=184) 

University 

(n=166) 

Total 

(n=350) 

Bobby  Approved 

16 

(8.7%) 

36 

(21.7%) 

52 

(14.9%) 

Free  of  Priority  1 Errors 

16 

(8.7%) 

36 

(21.7%) 

52 

(14.9%) 

Free  of  Priority  2 Errors 

24 

(13.0%) 

15 

(9.0%) 

39 

(11.1%) 

Free  of  Both  Priority  1 and 
Priority  2 Errors 

3 

(1.6%) 

3 

(1.8%) 

6 

(1.7%) 

Table  1:  Canadian  Post -Secondary  Website  Evaluations 
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The  most  common  Priority  1 errors  identified  in  the  evaluation  were  contained  within  three  categories: 
images  without  alternative  text  69.4  % (n=243)  and  image  map  hot  spots  without  alternative  text  19.1%  (n=67) 
and  not  giving  each  frame  a title  16.9  %(n=59).  Other  priority  1 errors  that  were  found  were,  not  providing 
alternative  text  for  applet’s  2.6%  (n=9),  not  providing  alternative  text  for  all  image-type  buttons  in  forms  1.4% 
(n=5),  and  not  having  each  frame  referencing  an  HTML  file  0.9%  (n=3). 

It  is  also  worth  noting  that  only  1.7  % (n=6)  out  of  the  350  sites  passed  both  Priority  1 and  Priority  2 
errors,  which  is  required  to  meet  the  Common  Look  and  Feel  for  the  Internet  (Canada,  2000). 


Discussion 

The  finding  that  14.9%  of  Canadian  University  and  College  initial  Web  pages  would  be  classified  as  a 
Bobby  approved  site,  which  is  equivalent  to  WAI  Conformance  Level  A standards,  is  significantly  lower  than 
similar  to  findings  for  prominent  US  university  and  college  initial  Web  pages  (Rowland  & Smith,  2000; 
Walden,  Rowland  & Bohman,  2000).  This  means  that  over  75%  of  college  and  university  sites  in  North 
America  contain  errors  that  seriously  limit  their  accessibility  to  the  disabled.  Universities  and  colleges  recognize 
that  the  Web  is  one  of  their  most  important  methods  of  communicating  with  their  potential  clientele.  One  can 
imagine  that  a university  or  college  president  would  be  terribly  embarrassed  it  if  the  front  page  of  their  site 
displayed  a “broken  graphic”  image,  or  contained  a broken  link.  There  would  undoubtedly  be  a very  sharp 
message  to  the  University  Webmaster  that  this  should  be  fixed  immediately  and  that  it  shouldn’t  happen  again. 
This  is  exactly  what  the  disabled  person  encounters  when  they  encounter  a page  with  a Priority  1 error. 

It  is  unlikely  that  any  organization  would  intentionally  alienate  7%  of  its  clientele.  It  is  more  likely  that  post- 
secondary administrators  and  Webmasters  are  unaware  that  their  Web  sites  are  being  read  (or  not)  by  the 
disabled,  and  what  they  must  do  to  make  them  accessible.  We  charge  that  a post-secondary  initiative  be 
established  to  raise  the  awareness  and  educate  administrators  and  the  public  about  equal  access  to  information. 
Guidelines  for  post-secondary  institutions  should  reflect  this  goal. 


Conclusion 

Bobby  3.2  was  used  to  examine  the  accessibility  by  the  disabled  of  the  front  pages  of  the  universities 
and  colleges  in  Canada,  -/^proximately  85%  of  these  pages  were  found  to  contain  Priority  1 errors  and  more 
than  95%  contain  either  Priority  1 or  2 errors.  This  is  totally  unacceptable.  The  Government  of  Canada  has  set 
December  31,  2002  as  a goal  for  having  government  Websites  free  of  Priority  1 and  2 errors.  While  it  is  not 
mandated,  it  would  appropriate  for  post-secondary  institutions  to  set  a similar  goal.  While  it  is  likely  through 
ignorance,  there  is  a long  way  to  go  before  we  meet  these  standards.  We  need  to  take  action  to  bring  this  issue 
to  the  attention  of  senior  administrators  in  University  and  colleges  to  raise  the  awareness  and  prompt  immediate 
action. 
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Abstract  In  this  paper,  an  instruction  sequence  constrained  learning  objects 
composition  approach  is  described  and  its  RDF  (Resource  Description  Framework) 
implementation  mo  del  is  introduced.  Semantically  sequential  composition  of  learning 
objects  in  the  recursive  mode  is  featured  at  metadata  level  to  allow  agent  access 
learning  objects  automatically  in  the  assigned  instruction  sequence.  The  model 
supplies  RDF  based  dynamic  linker  to  combine  learning  objects  within  personalized 
deployment.  Our  current  implementation  is  based  on  Jena  API. 


Introduction 

Term  “learning  objects”  is  used  by  the  Learning  Technology  Standards  Committee  to  define  small 
instructional  components,  i.e.  learning  object,  as  any  entity,  digital  or  non-digital,  which  can  be  used,  re-used  or 
referenced  during  technology  supported  learning  (LOM,  2000).  LOM  also  tries  to  enable  computer  agents  to 
automatically  and  dynamically  compose  personalized  lessons  for  an  individual  learner.  How  to  combine 
instruction  sequence  design  with  learning  object  composition  is  still  an  important  issue  especially  when  e- 
leaming  services  are  going  to  be  dynamically  composed  or  accessed  for  agents. 

Gagne  etc.  (1985)  refer  to  the  learning  conditions  as  building  blocks  for  instruction  since  they  influence 
the  learning  outcomes.  In  his  theory,  objects  of  learning  consist  of  five  categories  of  learning  outcomes.  With 
these  various  types  of  learning  outcome  and  conditions  of  learning,  there  are  nine  general  instructional  events  to 
facilitate  maximum  learning  and  to  support  the  internal  process  of  learning.  Based  on  Gagne’s  theory  and  point 
view  of  software  architecture,  we  proposed  a semantically  sequential  composition  model  for  learning  objects  to 
support  the  capability  for  dynamically  composing  learning  services.  To  present  multiple  granularities  of  learning 
objects  and  instruction  processes,  the  model  is  recursive  as  shown  in  Figure  1 and  2 in  which  nine  steps 
processes  are  presented  as  problem  description  (PD),  objective  description  (OD),  stimulate  recall  (SR),  learning 
object  presentation  (LOP),  learning  guide  (LG),  doing  practice  (DP),  sending  feedback  (SF),  testing  & assessing 
(TA),  and  enhance  retention  and  transfer  (SRT).  There  is  iterative  interaction  between  LOP  and  LG  to  get 
assigned  learning  object  in  the  LOP.  LOP  can  be  further  presented  as  learning  processes  (LP)  flow  in  which  each 
LP  has  similar  multiple  steps  process  but  at  finer  granularity  level.  We  believe  this  kind  of  lecursive  pattern 
presents  a generic  principle  existing  in  the  must  of  learning  process  of  engineering  technology  area. 


Figure  1 A recursive  multi-step  model  of  learning  processes  Fig)Lne2  Q}npositkmofleamD^pro^^ 
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Implementation  model  in  RDF 


RDF  provides  a standard  data  model  for  representing  machine-readable  semantics  of  information.  We 
use  RDF  to  model  instruction  sequence  logic  for  the  suggested  learning  process  (LP),  as  shown  in  Figure  3.  In 
the  model,  URIs  (uniform  resources  identifiers)  of  sequential  instruction  steps  such  as  PD,  OD,  ERT,  and  LOP  etc 
are  kept  in  <rdf:Seq>  container.  The  iteration  between  LOP  and  LG  is  further  modeled  in  <rdf:Bag>  container  with 
certain  retrieve  method  to  present  LG  constrained  LOP,  and  this  <rdf:Bag>  container  is  presented  as  a resource 
SOP/LG  in  the  ordered  list  of  resources  in  <rdf:Seq>  container.  The  URIs  of  learning  object  such  as  text, 
presentation,  questions,  exam,  etc.  are  kept  in  <rdf:Bag>  and  <rdf:Seq>  containers  respectively.  The  learner 
feature  is  also  attached  within  <rdf:Bag>  container  to  present  learner  model. 


Figures  RDF  mcxlel  for  the  learning  processes  Figure  4 RDF  model  as  contents  linker 

The  described  RDF  model  of  learning  process  can  be  used  in  the  several  different  ways  to  support 
adaptive  e-leaming  systems.  Our  current  project  intends  to  use  this  RDF  model  for: 

• Dynamic  virtual  linker  between  learner  and  learning  objects,  as  shown  in  Figure  4,  or  linker  between 
distributed  learning  objects. 

The  language-oriented  RDF  query/inferences  engine  provide  basis  for  semantically  driven  hyperlink 
services,  and  support  run-time  computation  of  links  and  anchors  in  addition  to  statically  defined  links 
and  anchors  that  defined  at  authoring  time  of  learning  objects.  Upon  learner  profile,  dynamic  linker  may 
be  able  to  maintain  personalized  linking  configuration  to  fit  individual  learner. 

• Learning  process  annotation. 

Annotation  is  ability  to  annotate  learning  objects  and  learning  process  without  the  need  to  modify  the 
resource  itself  The  designed  learning  process  RDF  model  can  be  used  to  allow  embedded  or  external 
annotation  of  linker. 

• Tracking  learning  process  for  learner  own  using 

Track  is  a way  to  help  learner  especially  part-time  learner’s  study  to  keep  thinking  stream,  memory,  and 
reminder,  etc.  during  discrete  learning  process.  Add  track  sub-model  and  associated  triples  based  on 
learning  process  RDF  model  can  maintain  dynamic  track  record. 

• Making  learning  plan 

By  retrieve  RDF  based  instruction  sequence  logic,  agent  can  help  to  make  learning  plan  or  schedule 
upon  available  time  constraints,  workload  standard,  and  assigned  due  time  parameters. 
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Abstract:  Collaborative  learning  approach  is  one  of  the  most  considerable  methods  which  can  be 
used  in  computer-mediated  learning  environment,  and  an  important  element  in  this  approach  is  peer 
modeling,  which  aims  to  resolve  how  to  simulate  collaborative  learning  process  and  to  get  high 
performance  for  students.  Some  factors  should  be  utilized  to  build  peer  model,  e.g.,  knowledge 
structure,  cognitive  and  behavioral  styles,  and  physiological  elements,  etc.  Among  these  factors, 
according  to  our  experiences,  knowledge  structure  is  a significant  factor  due  to  it  represents  the 
learning  and  instructional  objective.  In  this  paper,  an  effective  method  about  how  to  build  peer 
model  based  on  the  knowledge  structure  of  Chinese  Reading  in  web-based  intelligent  collaborative 
learning  environment  was  mainly  introduced. 


Introduction 

Peer  model  is  a core  element  of  the  small  learning  group  to  construct  the  web-based  collaborative  learning 
environment,  which  can  be  considered  as  a virtual  real  learner.  The  different  purpose  of  peer  model  can  be 
found  in  intelligent  tutoring  system  (ITS)  environment  is  to  facilitate  individual  learning  performance  and 
in  web-based  collaborative  learning  environment  is  to  promote  individual  and  collectivity  development  and 
to  enhance  peeis’  interaction.  The  term  peer  model  comes  from  student  model,  and  they  are  the  similar 
concept.  When  students  are  organized  into  different  learning  teams  to  work  together  and  to  pursue  a 
common  goal,  the  individual  can  be  called  peer. 

Some  examples  about  various  practical  and  efficient  methods  and  approaches  to  build  peer  model  in  web- 
based  environment  can  easily  be  found,  e.g.  Kumar  and  et  al.  (1995)  observes  artificial  intelligent 
technology  in  CALL  and  discusses  some  techniques  about  diagnosis.  Eliot  (1997)  constructs  a web-based 
intelligent  tutor  which  requires  information  about  what  student  has  and  has  not  learnt.  M.  K.  Stem  (1997) 
states  that  the  difficulties  in  web-based  tutoring  and  some  possible  solutions.  According  to  his  opinion,  a 
tutoring  system  must  have  function  about  recording  student  actions  and  make  decisions  based  on  these 
actions.  There  are  some  elements  should  be  involved  in  peer  model,  e.g.  knowledge  structure,  cognitive 
styles,  behavioral  factors,  and  physiological  elements.  One  of  the  most  important  elements  is  expert’s 
knowledge  structure,  which  can  be  considered  as  knowledge  point.  Our  past  works  focus  on  how  to  build 
and  realize  knowledge-based  student  model,  which  already  get  a significant  achievement  and  can  be 
considered  as  a general  approach  to  build  peer  model  (Zhao,  J.  H.,  1997). 

When  artificial  htelligent  technology  was  integrated  into  collaborative  learning  electronic  system,  its 
efficiency  and  flexibility  must  be  enhanced.  Some  emerging  issues  in  this  area  already  have  been  explored, 
e.g.  intelligent  pedagogical  agent,  student  models,  tutor  models,  diagnosis  strategies,  and  knowledge 
intelligent  representation.  CITS  (Collaborative  Intelligent  Tutoring  System)  provides  an  environment  in 
which  the  student  can  interact  with  one  or  more,  simulated  collaborative  partners  and/or  fellow  students,  to 
progress  towards  a common  goal  of  learning  (Kumar,  V.  S.,  1992).  GSS  (Group  Support  Systems)  is  a set 
of  techniques,  software  and  technology  designed  to  focus  and  enhance  the  communication,  deliberations, 
and  decision  making  of  groups.  In  GSS,  software  intelligent  agent  which  can  facilitate  and  streamline  group 
problem  solving  in  organizations  is  applied  (Jay  F.  Nunamaker  Jr,  1997;  Sen,  S.  and  et  al.,  1997).  Bma,  P. 
& Burton,  M.  (1997)  describes  modeling  students  collaborating  while  learning  about  energy,  which  has  the 
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potential  for  providing  better  computer-based  support  in  the  future  - both  in  respect  of  providing  improved 
quality  dialogues  and  in  terms  of  comprehending  the  student’s  activities.  K.  Miyahara  and  T.  Okamoto 
(1998)  studies  how  to  develop  information  filtering  system,  which  gathers,  classifies,  stores  various  kinds 
of  information  found  on  the  Internet.  According  to  our  experiences,  intelligent  technology  can  be  used  to 
enhance  the  quality  cf  web-based  collaborative  learning,  e.g.  flexible,  efficient,  and  suitable  and  to 
facilitate  students  learning.  Peer  model  is  a core  component  for  this  purpose  to  utilize  intelligent  technology 
into  instructional  and  learning  system.  In  web-based  intelligent  collaborative  learning  system  (WebICL), 
peer  model  can  be  used  to  decision-making,  search,  find,  and  utilize  the  precise  instructional  strategies  in 
terms  of  different  learner’s  learning  status  and  to  evaluate  peer’s  performance. 


Knowledge-Based  Peer  Modeling 

Knowledge-based  peer  model  means  how  to  integrate  learning  knowledge  into  intelligent  tutoring  system 
to  build  peer  model.  In  intelligent  tutoring  system,  peer’s  behavior  includes  a lot  of  knowledge  processing 
variables,  which  can  be  used  to  express  student’s  learning  status.  The  process  of  building  knowledge-based 
peer  model  includes  four  stages. 

Stage  One:  Core  Factors  of  Web-Based  Collaborative  Learning 

Some  methods  can  be  utilized  to  find  the  factor  variables  which  related  to  simulate  peer  irodel,  e.g. 
literature  research,  documentaries  research,  questionnaire,  and  case  study.  According  to  our  experience,  all 
of  them  can  be  used  to  collect  research  data  very  well. 

According  to  our  questionnaire  research  result,  core  factors  of  web-based  collaborative  learning  system  can 
be  clarified  into  four  levels,  which  the  first  level  is  learning  approach  level  and  involves  group  composition, 
task  features,  communication  media,  conflict  identification,  conflict  resolution,  role  playing,  and 
collaborative  communication  skills  (Foote,  E.,  1996);  the  second  level  is  personal  information  level,  which 
involves  gender,  age,  region,  grade,  ethnic,  and  year-in-school;  the  third  level  is  peer’s  cognitive  structure 
level,  which  involves  cognitive  styles,  behavior  styles,  previous  academic  performance,  and  IQ;  the  fourth 
level  is  curriculum  knowledge  level,  which  involves  knowledge  structure,  knowledge  points,  hierarchy  of 
knowledge  points,  and  knowledge  categories.  All  of  these  factors  in  different  levels  can  be  used  to  build 
peer  model.  As  states  above,  learning  knowledge  is  an  important  element  to  build  peer,  so  it  can  be 
considered  as  the  umbrella  (Zhao,  J.  H.,  1997). 

Stage  Two:  Discipline  Knowledge  Elements  for  Language  Learning 

A peer  modeling  approach  was  developed  to  simulate  knowledge-based  peer  model,  which  the  core  factors 
of  course  content  can  be  got  from  questionnaire  and  factor-analysis.  The  different  weights  should  be 
assigned  to  each  factor  in  terms  of  the  result  of  data  analysis.  This  modeling  method  can  be  considered  as  a 
common  approach  to  build  peer  model  based  on  different  learning  materials  (J.  H.  Zhao,  1997).  Dr.  Mo  Lei 
describes  his  research  result  about  Chinese  reading  for  grade  six  in  table  1 (Mo  Lei,  1989). 


Table  1:  Chinese  Reading  Ability  for  Primary  Students  in  Grade  Six 


Factor  of 
ability 

Language 

Conprehension  Ability 

Organizing 
Coherence  Ability 

Pattern  Recognition 
Ability 

Filtration  and 
Store  Ability 

Language 
Sensation  Ability 

Transferred 

Ability 

Weights 

.375 

.249 

.128 

.119 

.069 

.060 

Students  in  different  grade  will  learn  different  knowledge,  our  research  addressed  in  grade  three,  so  we 
should  amend  the  data  described  in  table  1 to  match  grade  three.  The  questionnaire  is  also  used  to  do  this 
work  and  the  amended  result  can  be  described  in  table  2.  Comparing  the  weights  in  table  1 and  table  2,  the 
different  weights  and  the  significance  for  each  factor  can  be  clearly  clarified. 

Stage  Three:  Basic  Static  Knowledge  Peer  Modeling 

Based  on  the  factors’  weight  in  table  2,  the  basic  static  knowledge  peer  model  can  be  formulated  as 
G = (0.421  0.256  0.064  0.13  0.061  0.066) 

Where  G is  the  objective  weight  matrix. 
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When  peer  register  to  the  WebICL  system,  he/she  will  be  provided  a pretest  opportunity.  The  points  of  each 
learning  objective  will  be  calculated  and  the  pretest  evaluation  matrix  will  be  expressed  as 

P = (Si  S2  S3  S4  S5  s^) 

Where  P is  the  score  matrix  for  peer  and  s is  score  that  he/she  got  from  each  question;  is  the  score  for 
each  learning  objective;  / = 1,2,3,...  6. 

P’s  Regression  result  can  be  expressed  as 

P = {s^/t^  ^2/^2  *^4/^4  ‘^5/^5) 

Where  t is  the  mark  of  each  question;  t = 1,2,3,...  6. 


Table  2:  Chinese  Reading  Ability  Structure  for  Primary  Students  in  Grade  three 


Ability 

Structure 

Language  Decoding 
Ability 

Organizing  Coherence 
Ability 

Pattern  Recognition 
Ability 

Filtration  and 
Store  Ability 

Language 
Sensation  Ability 

Transferred 

Ability 

Weights 

.421 

.256 

.064 

.130 

.061 

.066 

The  evaluation  result  for  each  peer  can  be  got  fromfuzzy  evaluation  method  and  can  be  expressed  as 

A = G • P (1) 

Where  A represents  that  each  student’s  synthetical  fuzzy  evaluation  result. 


Stage  Four:  Dynamic  Knowledge  Peer  Modeling 

Dynamic  knowledge  peer  model  can  be  simulated  based  on  the  basic  static  knowledge  peer  model  in 
WebICL  system.  Tutor  and  system  will  monitor  peer’s  behaviors  when  peer  mates  start  to  discuss  or 
negotiate  each  other  in  learning  group.  The  system  can  record  and  save  the  group  and  peer’s  learning 
history  record.  When  they  finish  one  topic,  tutor  will  assign  the  average  score  to  each  peer  in  the  same 
learning  group  based  on  the  result  of  intergroup  evaluation  or  other  similar  approaches.  In  order  to  getting 
the  peer’s  score  and  the  peer  performance  points,  peer  should  attend  the  individual  formative  examination. 
When  the  peers  are  taking  a formative  examination,  two  elements  will  be  concerned,  which  are  peer’s 
performance  score  and  the  time  of  peer  resolving  question  which  will  influence  peer’s  performance  score. 
To  get  this  score  about  student  resolving  question,  the  following  strategies  will  be  used  (Zhao  J.  Hua,  1997). 

score  times  times  <=  n 

Score  _i  — { 

0 times  > n 


Where  times  is  peer  to  try  times  to  resolve  the  question;  score  is  each  question’s  marks;  Score  J is  the 
student’s  factual  score  of  each  question. 

The  time  which  peer  resolving  each  question  spends  also  influences  the  peer-performance’s  score.  The 
following  strategies  can  be  adopted  (Zhao  J.  Hua,  1997). 

[1  {spend  - spent)  / spend)  * score / times  times  <=  n 

Score  _i  = { 

0 times  > n or  spend  > Ispend 

Where  spend  is  the  average  response  time  (time  limitation  for  problem-solving);  spent  is  the  factual  time 
which  peer  spends  to  resolve  question;  times-}, 2, 3...  n. 

The  data  of  static  knowledge  peer  model  will  be  modified  when  peer  finishes  the  formative  examination, 
and  then  the  dynamic  knowledge  peer  model  can  be  formulated  based  on  the  result  of  fuzzy  evaluation. 
Each  item  in  different  test  will  be  organized  based  on  the  learning  objective  by  tutor.  Peer  can  get  the 
different  evaluation  score  when  he/she  answers  the  different  questions  and  one  score  matrix  will  be  built. 
The  matrix  B can  be  described  as  follows. 


B = 
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Where  5 is  score  matrix  of  peer;  S^j  is  each  score  for  different  objective;  / = 0,2,3,...6  ; y=l,2,3,--6. 

Peer  kiowledge  dynamic  model  can  be  formulated  as  follows,  which  it  indicates  the  status  of  peer’s 
knowledge  which  already  learnt  based  on  different  learning  objective. 

A = G • B 


= (0.421  0.256  0.064  0.13  0.061  0.066)  • 


’ll 

^12 

^13 

■^14 

■^15 

^16 

21 

^22 

^23 

^24 

^25 

^26 

31 

^32 

^33 

^34 

^35 

^36 

41 

^42 

^43 

^44 

^45 

^46 

■^51  *^52  *^53  *^54  *^55  *^56 

_‘^61  *^62  *^63  *^64  *^65  *^66 , 

Where  A is  the  result  of  fuzzy  evaluation  which  represents  the  dynamic  knowledge  peer  model. 

The  main  feature  of  dynamic  knowledge  peer  model  is  that  its  data  will  be  modified  and  updated  in  terms 
of  the  learning  objective.  The  score  which  assigned  to  dynamic  peer  model  can  be  considered  as  one  of  the 
components  of  peer’s  learning  performance  which  is  the  average  score  of  group’s  learning  performance. 


Peer  Modeling  in  WebiCL 

Peer  model  in  WebICL  involves  two  kinds  which  is  general  peer  model  and  virtual  peer  model  which  can 
be  considered  as  peer  agent. 

General  Peer  Model 

General  peer  model  in  WebiCL  represents  the  status  of  student  who  registered  (logged)  in  the  system.  Five 
basic  elements  are  involved  in  the  general  peer  model,  which  they  are  gender,  former  learning  performance, 
IQ,  and  cognitive  style.  The  general  peer  model  can  be  simulated  in  terms  of  these  five  elements,  which  it 
can  be  described  as  one-dimension  matrix. 

GPM  = {iG  A iQ  iCS) 

Where  GPM  means  general  peer  model,  iG  means  gender,  A means  student  learning  performance  which 
was  introduced  above  formula  (1)  and  (2),  means  student’s  IQ,  and  iCS  means  cognitive  style.  The  value 
of  IQ  and  cognitive  style  can  be  got  before  collaborative  learning  in  web-based  environment  through 
psychological  survey.  A is  the  result  of  fuzzy  evaluation  to  student.  Three  elements  are  the  permanent 
variables,  which  they  are  iG,  iQ,  and  ICS.  The  result  of  iG  can  be  got  in  term  of  peer’s  personal  registered 
form;  iQ  and  iCS  can  be  got  through  psychological  survey.  The  result  of  iCS  mainly  used  field-dependent 
and  field-independent  style  field  survey  and  it  can  be  got  before  peers  start  collaborative  learning  process. 

Virtual  Peer  Model  (Peer  Agent) 

Virtual  peer  model  (VPM)  can  be  organized  into  the  same  group  with  general  peer  model  if  the  number  of 
peer  cannot  reach  the  group  size.  VPM  includes  three  components,  which  they  are  knowledge  database, 
strategies  database,  and  deducing  mechanism.  The  virtual  peer  model  can  be  described  as  figure  1. 


Database 


Figure  1:  Virtual  Peer  Model 
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Knowledge  Database 

Knowledge  database  provides  knowledge  support  for  virtual  peer  agent,  which  it  is  similar  with  knowledge 
database  in  WebICL.  Especially,  it  is  more  related  to  learning  task.  The  structure  of  knowledge  database  for 
peer  agent  includes  five  elements,  which  they  are  knowledge  number,  knowledge  types,  knowledge  content, 
question  and  answer,  and  grade  level,  which  knowledge  number  means  knowledge  record  number  in 
knowledge  database,  knowledge  types  means  the  types  of  discipline  and  task,  knowledge  content  is  used  to 
record  learning  content,  question  and  answer  represents  learning  question  and  its  normal  answer,  and  grade 
level  represents  which  grade  level  of  learning  task. 

KD  = {Kn  Kt  Kc  Qa  Gl) 

Which  Kn  represents  knowledge  number,  Kt  means  knowledge  types,  Kc  is  knowledge  content,  Qa  means 
question  and  answer,  and  Gl  is  grade  level. 

Kt  = [a  b c) 

Where  a = \,  2,  ...,  n means  discipline,  ^ = 1,  2,  ...,  n means  task,  and  c = 1,  2,  ...,  n means 

difficult  level. 

Kc=[d  e f) 

Where  d = \,  2,  ...,  n means  content  number,  e = 1,  2,  ...,  n means  content  index  number,  and 
/ = 1,  2,  ...,  n means  learning  content  showed  times. 

Qa={g  h i j k) 

Where  g = 1,  2,  ...,  n means  the  number  of  question,  /i  = 1,  2,  ...,  n means  the  number  of  answer, 

i = 1,  2,  ...,  n means  the  difficult  level  of  question,  j = 1,  2,  ...,  n means  the  question  showed 

times,  and  k = \,  2,  ...,  n means  the  index  number  of  question. 

Gl  = [m  q) 

Where  m = 1,  2,  ...,  n means  educational  level  (primary  school,  junior  school,  senior  school,  etc),  and 

^ = 1,  2,  n means  grade  level. 

Strategies  Database 

Some  strategies  about  howto  build  the  virtual  peer  model  based  on  the  on-line  peer’s  situation  in  the  WebICL  system 
and  organize  them  into  different  learning  group.  There  are  five  elements  are  included  in  the  strategies,  which  they  are 
strategy  index  number,  performance  level,  role-playing,  peer  number,  and  role  gender. 

SD={Si  PI  Rp  Pn  Rg) 

Where  Si  means  strategy  index  number,  PI  represents  performance  level,  Rp  is  role-playing,  Pn  means  peer 
number,  and  Rg  means  the  status  of  role  gender. 

Si  = 1,  2,  ...,  n 

Pi  = {Hi  Mi  Lo) 

Where  Hi  means  high  level.  A//  means  middle  level,  and  Lo  means  lower  level. 

Rp  = y 2,  ...,  18 

The  value  of  Rp  represents  different  role  in  learning  group,  which  it  can  be  described  as  table  3 (Donelson 
R.  Forsyth). 


Table  3:  Role-playing  in  learning  groups 


Index  Number 

Role-playing 

Index  Number 

Role-playing 

1 

Initiator/Contributor 

10 

Energizer 

2 

Information  Seeker 

11 

Encourager 

3 

Opinion  Seeker 

12 

Compromiser 

4 

Information  Giver 

13 

Gatekeeper  and  Expediter 

5 

Opinion  Giver 

14 

Standard  Setter 

6 

Elaborator 

15 

Group  Observer  and 
Commentator 

7 

Coordinator 

16 

Follower 

8 

Orienter 

17 

Recorder 

9 

Evaluator/Critic 

18 

Procedural  Technician 
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Pn  = \,  2,  3,  4 

The  value  of  Pn  means  the  peer  number  in  learning  group. 

/?g  = l,  2 

Where  Rg  = 1 means  peer  is  male  and  Rg  = 2 means  peer  is  female. 

Reasoning  Mechanism 

Reasoning  mechanism  is  the  representative  of  expression  and  discriminant.  The  virtual  peer  model  will  be 
formed  based  on  its  reasoning  result.  The  condition-decision  is  the  formal  approach  to  realize  the  reasoning 
mechanism.  The  basic  decision  norm  is  rules  that  it  defines  an  applied  framework.  There  are  various  rules 
can  be  used  to  make  reasoning  decision.  One  of  the  examples  is  as  follows. 

If  <group_number  less  than  three>  Then 

Case  1 if  group_numher=2  then  <compare  the  differentiation  of  real  peer  and  determine  the  VPM  > 

Case  2 if  group_number=l  then  < determine  other  two  VPM  based  on  the  peer  *s  situation> 


Conclusions 

According  to  our  study,  peer  model  can  be  used  to  design  and  build  web-based  collaborative  learning 
system  and  enhance  its  flexibility  and  adaptability.  Peer  modeling  should  be  built  in  terms  of  the  peer’s 
personal  characteristic  and  learning  content  of  curriculum  knowledge.  When  the  number  of  peer’s  login  is 
less  than  the  necessary  number  of  one  group,  WebICL  system  should  simulate  virtual  peer  according  to  the 
actual  on-line  peer.  Peer  model  is  an  important  approach  to  utilize  intelligent  technology  into  web-based 
collaborative  learning  system. 
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Understanding  the  social  context  for  online  learning 

Melanie  Zibit 
Boston  College 
Chestnut  Hill,  MA  02467 
Goldmame@bc.edu 

The  world  of  online  learning  is  fascinating  and  so  different  from  face  to  face.  What  are  its 
characteristics?  What  kind  of  learning  works  best  in  these  environments?  And  how  do  you  take 
advantage  of  these  characteristics  when  you  are  designing  your  online  curriculum? 

An  onhne  learning  environment  works  most  effectively  to  facilitate  the  examination  of  issues 
from  multiple  perspectives.  People  can  take  time  to  read,  reflect,  and  respond.  This  gives  a 
participant  a sense  of  security  to  say  what  she  really  means.  The  combination  of  a certain 
anonymity  coupled  with  a sense  of  connectedness  allows  people  to  open  up  and  take  risks  to 
express  their  thoughts  even  in  areas  they  are  just  learning.  The  environment  motivates  people  to 
think  rather  than  to  take  a fixed  position-  divergence  rather  than  convergence  (Zuboff,  1984). 

Mathemagica,  a five-year  mathematics  initiative  fimded  by  the  Department  of  Education  Star 
Schools  initiative,  is  developing  an  innovative  model  for  online  professional  development  for  K-8 
math  teachers.  Our  online  professional  development  philosophy  is  captured  in  a statement  from 
the  NCTM  discourse  standards  , "When  teachers  make  public  conjectures  and  reason  with  others 
about  mathematics,  ideas  and  knowledge  are  developed  collaboratively,  revealing  mathematics 
as  constructed  by  human  beings  within  an  intellectual  community.".  One  component  of  the 
professional  development  is  rich  miiltimedia  math  applets  that  empower  both  teachers  and 
students  to  investigate  mathematical  topics.  For  example,  one  series  will  allow  students  to 
experiment  with  (and  even  invent)  computational  algorithms  using  base  ten  blocks,  arrays,  and 
number  lines.  Another  component  is  a sequence  of  mathematical  investigations  using  the  applets 
combined  with  readings  and  reflective  tasks  that  teachers  work  on  collaboratively  in  teams  of  8 
facilitated  by  a moderator.  The  applets  and  professional  development  are  being  piloted  withSO 
teachers  who  have  a wide  range  of  skill  in  technology  and  inquiry  math  practice. 

This  paper  and  presentation  will  describe  the  design  of  an  online  learning  environment  used 
with  teachers  in  a math  professional  development  project.  The  model  builds  upon  key 
imderstandings  about  the  social  dynamics  of  online  learning.  The  session  will  include  key 
characteristics  and  strategies  as  well  as  several  vignettes  that  demonstrate  the  model's 
effectiveness. 

Mathemagica's  Web  site 
http:/  / www.mathemagica.org 
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MoBy:  Modular  Multiple  Media  Course  Content  for  On-Line  Courses 

Harry  R.  Matthews  and  Lisa  Wilson,  lET  Mediaworks,  UC  Davis 


Abstract 

The  MoBy  interactive  content  delivery  system  addresses  the  problem  of  providing  complex,  well -integrated,  multiple 
media  course  materials  on-line,  in  a form  that  is  maintainable  by  content  experts,  and  with  a process  that  is  scalable. 

The  system  is  delivering  content,  testing,  and  tracking  in  courses  as  diverse  as  Chemistry  pre-  and  post-laboratory 
materials.  Arts  of  Asia,  Anthropology,  Winemaking,  and  others.  The  Andrew  W.  Mellon  Foundation  has  funded 
detailed  evaluation  of  the  cost-effectiveness  of  the  system  that  is  also  being  used  for  research  on  student  profiling  and 
learning  styles. 

Introduction 

MoBy  is  a development  of  a system  prototyped  with  a biochemistry  course  in  1997  and  1998  (...)  and  demonstrated  at 
an  Intemet2  conference  and  the  showcase  of  an  EduCause  annual  meeting  (...).  MoBy  differs  from  commercially 
available  learning  management  systems  in  the  full  integration  of  varied  multiple  media  as  the  natural  presentation  and 
interactive  modes  of  learning  on-line  thereby  enriching  the  learning  process. 

Design  principles 

The  fundamental  design  principle  is  modularity  and  flexibility  within  a common  framework.  This  is  achieved  by 
combining  both  the  content  and  its  method  of  display  within  a single  object,  the  content  module  or  learning  object. 

The  second  design  goal  is  control  by  the  content  expert.  The  modularity  of  the  learning  objects  allows  the  content  expert 
to  rearrange  and  re-use  modules  without  any  restrictions  or  code  changes. 

The  third  design  goal  is  to  integrate  MoBy  with  the  campus  portal  that  handles  course  management  chores  including 
student  registration  and  grade  calculations  and  reporting. 

Implementation 

Implementation  is  based  on  the  three  design  principles:  learning  objects;  editing  by  content  experts;  and  integration  with 
course  management. 

Learning  objects,  organization  and  structure 

A content  module,  also  called  a learning  object,  is  a record  in  the  database  table  called  Module.  The  Module  Table  is 
part  of  a hierarchy  of  relational  tables.  The  tables  above  the  Module  Table  in  the  hierarchy  contain  information  that  is 
used  to  organize  the  modules  into  lectures  and  courses.  The  tables  below  the  Module  Table  in  the  hierarchy  contain 
information  about  the  data  and  files  that  comprise  the  content  of  the  module 

Editing  by  content  experts 

The  content  may  be  in  any  form  that  can  be  displayed  by  a Web  browser.  Two  strategies  are  used  to  broaden  the  range 
of  tasks  accessible  to  a technically  unsophisticated  content  provider.  The  first  strategy  involves  building  the  Flash 
animations  using  templates  that  draw  their  content  from  database  tables.  The  second  strategy  is  intrinsic  to  the  object- 
based  approach.  Content  modules  can  be  rearranged,  added,  deleted,  and  re-used  by  making  changes  to  the  database 
contents,  which  again  is  achieved  through  the  use  of  HTML  forms  that  a content  provider  can  easily  use. 

Integration  with  course  management 

MoBy  needs  to  be  aware  of  which  students  are  registered  in  each  course,  for  tracking  purposes  and  for  password- 
protection  of  course  materials  that  are  governed  by  copyright  fair  use  or  intellectual  property  restrictions.  Registration 
information  resides  in  a central  student  information  system  (SCT  BANNER)  and  is  accessed  by  faculty  members  and 
students  through  a custom  portal  known  as  MyUCDavis.  Access  to  this  information  requires  authentication  and 
authorization.  Authentication  is  handled  by  a separate  enterprise-level  Kerberos  system  and  authorization  is  controlled 
by  MyUCDavis  using  information  in  the  student  information  system. 
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Examples  of  Use 

MoBy  is  used  in  several  undergraduate  courses  at  both  lower  and  upper  division  level  and  in  a professional  school 
course.  These  courses  include  Chemistry  pre-  and  post-laboratory  materials,  Arts  of  Asia,  Anthropology,  Winemaking, 
Introduction  to  Biology,  Food  Science  & Technology,  Molecular  & Cell  Biology,  and  additional  courses  planned  for 
Winter  2002. 

The  primary  driving  force  for  the  use  of  MoBy  at  UC  Davis  is  the  urgent  need  to  accommodate  rising  student 
enrollments,  together  with  a determination  that  the  quality  of  student  education  should  not  suffer  in  spite  of  enrollments 
rising  faster  than  physical  facilities  can  accommodate.  The  costs  and  educational  effectiveness  is  being  studied  with  the 
help  of  a grant  from  the  Andrew  W.  Mellon  Foundation. 

Conclusions 

The  use  of  learning  objects  allows  seamless  integration  of  multiple  media  into  course  delivery.  This  allows  authors  to 
concentrate  on  using  the  best  media  for  each  concept  or  example  without  the  need  to  embed  it  in  an  HTML  page  and 
with  complete  transparency  to  the  user.  The  learning  objects  can  be  easily  rearranged  and  re-used  because  they  contain 
both  the  data  and  the  method  of  display.  Eventually,  it  will  be  possible  to  exchange  learning  objects  between  systems, 
allowing  the  construction  of  courses  using  the  best  available  objects  from  disparate  authors. 
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The  Virtual  Platform  SENEKA  - A knowledge  management  network  tool 

Dr.  Phil.  Regina  Oertel,  Georg  Schoeler 


Abstract 

The  Virtual  Platform  SENEKA  - information,  communication  and  knowledge  management  system  of  the  SENEKA 
project  - is  in  every-day  use  since  July  2000.  Here  about  30  partners  of  the  research  project  get  the  latest  news  about 
meetings,  project  data,  conferences  or  information  with  relevance  to  the  research  community.  The  business  processes 
hold  up  on  the  Virtual  Platform  SENEKA  via  email,  document  sharing  or  discussions.  Thus,  in  using  the  Virtual  Plat- 
form SENEKA  the  project  participants  are  able  to  enlarge  their  knowledge  and  learn  to  integrate  information  and  com- 
munication technologies  in  their  daily  work.  By  these  means  the  Virtual  Platform  SENEKA  supports  the  idea  of  ’’live- 
long learning”. 
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Figure  1:  The  Virtual  Platform  SENEKA 


Introduction  to  SENEKA 

SENEKA  is  a large  scale  entrepreneurial  and  research  project  supported  by  the  German  Federal  Ministry  of  Research 
and  Education  focussing  on  knowledge  management,  learning  and  networking.  The  project’s  aim  is  to  enhance  the  man- 
agement of  information  and  knowledge  as  well  as  to  reinforce  its  usability  in  innovation  processes  within  small  and 
medium-sized  enterprises.  To  achieve  this  aim  26  - small  and  medium-sized  as  well  as  multi-national  - enterprises  in 
co-operation  with  6 research  institutions  work  together  in  a vivid  network.  The  project  started  in  1999  and  will  last  until 
2004;  the  total  financial  volume  adds  up  to  about  20  million  Euro. 

The  Virtual  Platform  SENEKA 

The  Virtual  Platform  SENEKA  is  the  web  based  information,  communication  and  knowledge  management  system  of 
the  SENEKA  project.  It  was  designed  during  the  first  half  of  the  year  2000  and  is  in  full  operation  since  July  2000.  By 
now  the  platform  is  widely  utilized  by  the  project  participants.  It  supports  the  project  internal  information  and  commu- 
nication with  the  concept  of  shared  working  spaces  for  the  different  project  units.  The  project  partners  are  able  to  upload 
and  categorize  documents  of  general  interest,  inform  co-workers  automatically  or  manually  via  email,  start  discussions 
on  the  new  content,  manage  project  meetings  with  special  appointment  folders  and,  of  course,  search  and  download 
documents  of  their  special  concern.  This  features  list  shows  that  the  platform  works  an  integration  of  different  IT  sup- 
port systems.  There  are  aspects  of  content  manangement  system  and  data  management  systems  as  well  as  project  man- 
agement systems.  Moreover,  as  in  the  internet,  communit  ies  can  be  established  on  certain  topic  fields. 

The  basis  for  the  technical  implementation  of  the  Virtual  Platform  SENEKA  builds  the  open-source  application  server 
ZOPE,  which  is  an  integration  of  an  object-orientated  database,  a web-publishing  system  and  a web-based  programming 
and  administation  interface. 

The  necessity  of  a Virtual  Platform 

As  mentioned  above,  SENEKA  is  a large  industry  and  research  project  with  a very  heterogeneous  national  and 
multinational  partner  structure.  Already  at  the  start  of  the  project  in  the  year  1999  following  reasons  showed  the 
necessity  to  develop  a web-based  IT-system.  Because  of  the  organizational  conditions  in  the  co-operation  of  such 
different  partners,  which  work  on  different  topics  and  in  different  parts  of  the  country.  The  partner  structure  represents  a 


SSST  COPY  AYAIMBI^ 


cross  section  of  completely  different  habits  in  the  use  of  IT  tools  . This  lead  to  the  idea  to  take  the  project  network  as 
experimentation  and  evaluation  field  for  such  a knowledge  management  tool.  The  focus  of  the  project  belongs  to 
knowledge  management,  networking  and  competency  development  as  central  topics  in  SENEKA.  On  this  background  it 
is  neccesary  to  develop  best  practice  solutions  which  represent  an  optimal  interlink  of  real  and  virtual  knowledge 
transfer  processes. 

Thus  the  Virtual  Platfom  SENEKA  has  a double  function  in  the  SENEKA  project:  It  is  at  the  same  time  tool  for  the 
project  and  also  product  of  the  project.  It  directs  to  support  and  organize  the  knowledge-oriented  working  processes 
between  heterogeneous  senders  and  recipients  by  accentuated  IT  application.  Virtuality  - as  the  idea  to  work  and  learn 
independently  of  time  and  space  - here  offers  the  possibility  of  dealing  with  the  increasing  complexity  of  information 
and  communication. 

Authors  & contact 

Dr.  phil  R.  Oertel  is  project  manager  of  BMBF -project  SENEKA  and  works  at  ZLW/IMA  of  RWTH  Aachen.  Dipl.-Ing. 
G.  Schoeler  works  at  the  same  institute  and  is  entrusted  with  the  scientific  co-ordination  of  the  project  and  the  develop- 
ment of  the  Virtual  Platform.  For  general  information  on  SENEKA,  please  look  up  http://www.seneka.de.  For  further 
information  on  the  Virtual  Platform  SENEKA,  please  contact  schoeler@zlw-ima.rwth-aachen.de  . 


2194 


Weaving  the  Web  for  all:  Creating  institutional  policy  to  support  Web  access  for 

those  with  disabilities 


Cyndi  Rowland,  Ph.D. 

Overview 

Postsecondary  institutions  must  insure  that  their  Web  sites  can  be  accessed  by  those  who  have  disabilities  (i.e., 
faculty,  staff,  and  students).  Creating  institutional  policy  and  a climate  for  coordination  and  reform  can  be  a difficult 
task.  This  presentation  will  begin  where  reform  efforts  must  begin,  with  an  understanding  of  the  problem.  Several 
multimedia  simulations  will  be  presented  to  enable  participants  to  understand  what  happens  when  designers  do  not  keep 
web  accessibility  in  mind  as  they  build  out  a site.  Following  this,  participants  will  hear  about  an  8-point  model  for 
institutional  coordination  of  Web  accessibility.  This  presentation  is  important  because  many  participants  of  Ed  Media 
have  long  been  the  technologic  elite  on  their  own  campuses.  When  these  participants  return  from  this  conference 
session  they  will  have  practical  knowledge  regarding  Web  accessibility.  They  can  then  share  what  they  know  with 
administrators  and  colleagues  and  thus  help  their  institution  begin  the  work  of  Web  accessibility  coordination  and 
reform. 


An  Under standingOf  The  Problem? 

The  latest  census  figures  indicate  that  approximately  59  million  Americans  have  disabilities.  Clearly  all  of  those 
individuals  do  not  use  the  Web  nor  are  they  all  blocked  from  the  Web  because  of  their  disability  status.  Reports  do 
indicate  that  at  least  4 million  individuals  with  disabilities  ages  16  and  older  report  that  they  use  the  Web.  However,  it 
may  be  impossible  to  determine  how  many  do  not  use  the  Web  because  they  cannot  gain  consistent  access,  or  how 
many  use  the  Web  inconsistently  because  of  accessibility  problems.  No  matter  the  exact  number,  we  know  that  there 
are  millions  of  individuals  in  the  U.S.  alone  that  cannot  freely  use  the  Internet  because  of  inaccessible  designs. 

Empirical  investigations  on  the  accessibility  of  the  Internet  are  relatively  new,  and  current  studies  are  complicated  by 
the  use  of  dependent  measures  that  produce  both  false  positives  and  false  negatives.  With  this  said,  there  exists  a 
surprising  constancy  of  summary  data.  As  a general  statement,  less  than  25%  of  Internet  pages  sampled  in  the  literature 
are  accessible  to  those  with  disabilities. 

This  is  a horrendous  statement  considering  the  fact  that  few  individuals  go  to  a specific  page.  If  you  were  to  search 
the  Web  and  could  not  fully  access  3 of  every  4 pages,  how  successful  would  you  be? 

Who  are  these  individuals  that  are  affected  by  inaccessible  designs?  The  range  of  disability  types  is  wide  and 
represent  persons  with  challenges  in  vision  and/or  hearing,  motor  skills,  cognitive  skills,  and  seizure  disorders,  as  well 
as  age-related  processes.  It  is  important  to  remember  all  of  these  distinct  groups.  Often  Web  developers  address  the 
accessibility  needs  of  only  one  or  two  groups.  The  group  most  typically  discussed  is  the  blind.  It  helps  to  understand 
the  experiences  of  all  groups  of  individuals  with  disabilities.  The  following  provides  a peek  into  their  use  of  assistive 
technologies,  the  problems  they  can  experience  using  the  Web,  and  thoughts  for  designers  who  wish  to  consider  the 
needs  of  these  individuals. 

? Vision  impairments — Web  design  can  affect  three  categories  of  individuals  with  visual  problems.  Those 

include  individuals  who  are  blind,  have  low  vision,  or  experience  color  blindness. 

? Those  who  are  totally  blind  typically  surf  the  Web  through  the  use  of  a screen  reader  (packages  include  JAWS, 

Window  Eyes,  and  HomePage  Reader).  The  reader  provides  voice  output  that  essentially  “reads’’  what  is  on  the  screen, 
although  it  is  really  reading  the  code  behind  what  is  seen.  When  images  or  elements  appear  that  do  not  have  alternative 
text  placed  in  the  code,  the  reader  defaults  to  reading  anything  it  can  find,  such  terms  as  image  or  graphic  or  a file  name 
or  even  a number  or  coordinate  grabbed  remo  tely  from  the  server.  Sometimes  the  reader  skips  elements  of  a page 
entirely,  since  it  cannot  detect  that  anything  is  there.  The  presence  of  alternative  text  (ALT  tags  in  HTML  code)  is 
particularly  important  if  the  images  happen  to  be  navigation  buttons.  Imagine  the  frustration  of  an  individual  who’s 
encountered  a page  for  the  University  of  the  Anartic  that  the  screen  reader  reads  as  Welcome,  choose  what  information 
is  important  to  you  today,  image!. gif,  image2.gif  imageS.gif  image4.gif  image5.gif  Look  at  the  code  and  see  what  the 
screen  reader  must  try  to  interpret: 

<html> 

<title>  University  of  the  Anarctic</title> 

<map  name=  ’’Welcome,  choose  what  information  is  important  to  you  today> 

<area  shape=”rect”  coords=”509,43,565,62”  href="imagel.gif’> 

<area  shape=”rect”  coords="428,45,505,62”  href=”image2.gif’> 
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<area  shape-’rect”  coords=”336,43,422,60”  href=”image3.gif*> 

<area  shape=”rect”  coords="266,41,330,60”  href=”image4.gif*> 

<area  shape=”rect”  coords- ’173,44,259,61"  href="image5.gif’>? 

? In  addition,  screen  readers  rely  on  the  keyboard  for  navigation,  since  blind  consumers  are  unable  to  use  a 

mouse.  An  alarming  percentage  of  sites  aren’t  keyboard  accessible.  Certain  applications  and  technologies  (including 
Flash  ?iV[d  JavaScript)  can  trap  a user  in  a spot  where  they  cannot  move  forward  or  backward. 

? Experience  it  yourself:  To  simulate  being  blind  and  using  a screen  reader,  go  to 

? Those  with  low  vision  typically  use  a screen  enlarger  such  as  ZoomText  to  enlarge  entire  sites  up  to  400%  of 

typical  size  (beyond  this  degree  of  magnification,  the  use  of  a screen  reader  is  typically  encouraged).  If  a designer  does 
not  consider  this  group  of  consumers,  they  may  use  absolute  sizing  rather  than  proportional  sizing  of  information  on 
their  site.  When  this  happens,  the  consumer  must  scroll  endlessly  side  to  side  as  well  as  up  and  down  to  read  content  or 
find  an  important  link.?  Screen  enlarger  users  can  be  frustrated  when  designers  use  graphic  text  in  their  images  rather 
than  real  text.  When  images  are  significantly  enlarged,  they  become  so  pixilated  that  they’re  uninterpretable. 

? Experience  it  yourself:  To  simulate  having  low  vision  and  using  a screen  enlarger,  go  to 

? The  largest  numbers  of  individuals  with  vision  impairments  are  those  who  are  color  blind.  Since  they  cannot 

differentiate  between  some  colors  (most  often  red  and  green),  the  use  of  color  alone  to  convey  content  poses  a 
problem.?  Imagine  looking  at  a Web  page  showing  a subway  map  with  routes  that  have  been  color  coded  (blue  line,  red 
line,  and  green  line),  then  imagine  that  you  can’t  distinguish  between  the  three  colors.  The  use  of  the  words  on  that 
section  of  the  map  (red  line)  or  the  use  of  symbols  (a  line  of  asterisks  to  denote  the  green  line  route,  a series  of  plus 
signs  to  denote  the  red  line  route,  a solid  line  to  denote  the  blue  line  route)  would  provide  an  easy  fix. 

? Experience  it  yourself:  To  simulate  having  color  blindness,  download  the  free  Opera  browser  from  . Then  turn 

off  all  colors  (a  preference  item  on  the  top  left  bar  in  Opera)  and  visit  See  if  you  can  determine  when  your  holidays 
would  be  if  you  were  a faculty  member. 

? Hearing  impairments — People  are  often  surprised  that  the  Web  can  block  out  individuals  with  hearing 

impairments,  but  that’s  what’s  happening  because  embedding  multimedia  elements  within  Web  pages  is  becoming  more 
common.  Today,  users  can  log  onto  CNN  or  even  their  local  news  station  to  hear  current  news.  They  can  listen  to  a 
history  lecture  given  by  a university  professor.  They  can  even  watch  multimedia  tutorials  on  health  issues,  such  as  how 
the  cardiovascular  system  works.?  Imagine  turning  on  a TV  and  keeping  the  sound  off.  This  is  the  same  experience  for 
those  who  are  deaf  trying  to  make  sense  of  audio  content  on  the  Web.  Even  when  a speaker  can  be  seen,  lip  reading 
may  be  impossible  because  of  the  size  of  the  speaker  on  the  screen  and  the  inconsistent  nature  of  bandwidth 
delivery.?  The  obvious  fix  is  to  provide  simultaneous  captioning  for  multimedia  and  transcripts  for  audio -only 
elements.  Because  multimedia  development  is  often  time  consuming  and  costly,  this  aspect  must  be  considered  at  the 
beginning  of  any  development  effort. 

? Experience  it  yourself:  Go  to  and  select  any  of  the  eight  chapters  to  view  (via  either  the  QuickTime  or 

RealPlayer  browser  plugins).  Make  sure  your  sound  is  off  and  the  captions  are  off.  Watch  for  a while,  then  turn  the 
captions  on.  What  different  does  this  make  for  you 

? Dual  sensory  impairments — Individuals  who  experience  both  severe  vision  and  hearing  impairments  are 

considered  to  be  deaf/blind.  To  use  computers,  these  individuals  most  often  rely  on  Braille  output  devices,  placing 
their  hands  on  a tactile  display  in  which  tiny  pins  move  up  to  create  Braille  symbols  which  can  be  read  with  the 
fingertips.  Because  the  Braille  displays  “read”  text  in  the  same  way  that  screen  readers  do,  many  of  the  design  features 
that  are  used  to  make  Web  pages  accessible  to  totally  blind  users  also  apply  to  deaf/blind  users.  Deaf/blind  users  also 
need  to  provide  input  via  keyboard  rather  than  mouse,  so  pages  must  be  keyboard  accessible.  Finally,  audio  must  be 
rendered  in  a text  form  so  it  can  be  extracted  and  read  via  the  Braille  display. 

? Motor  Skill  Impairments — The  range  of  individuals  considered  to  have  motor  impairments  is  great. 

Individuals  with  quadriplegia  will  require  an  alternative  way  to  get  commands  into  the  computer  (including  devices 
such  as  voice  recognition  and/or  switch  access)  since  they  are  unable  to  use  the  keyboard.  Those  with  cerebral  palsy 
may  have  problems  with  accuracy  of  mouse  or  keyboard  access.  Some  individuals  with  neurologic  conditions  can 
experience  extreme  fatigue  or  tremors,  affecting  mouse  or  keyboard  use.  For  any  of  these  conditions  the  designer  must 
understand  issues  of  accuracy  (for  instance,  how  small  to  make  the  hit  box  around  a navigation  link),  and  fatigue  (if  an 
individual  can  only  move  from  one  link  to  the  next  by  pressing  the  Tab  key,  how  many  times  must  it  be  hit  to  get 
through  the  site).  Understanding  how  these  individuals  experience  the  Web  will  go  a long  way  toward  fixing  potential 
problems  that  can  keep  individuals  with  motor  impairments  out  of  your  site. 

? Experience  it  yourself:  Go  to  . Using  only  the  keyboard  (no  mousing  allowed!),  get  to  the  link  on  the  5-day 

forecast  for  10,000  cities.  Type  in  your  city  to  see  your  forecast.  As  a hint,  the  tab  key  will  move  you  from  one  link  to 
the  next.  Count  the  number  of  times  you  had  to  hit  this  key  to  move  to  where  you  wanted  to  go.  Also,  be  aware  of  the 
tabbing  order.  If  you  wanted  today’s  top  news,  how  long  would  it  take  you  to  get  to  this  information? 

? Cognitive  Impairments  and  Learning  Disabilities — There  are  many  individuals  who  are  cognitively  impaired 


2196 


COPT  ATAIUBB 


(developmental ly  delayed  or  have  a traumatic  brain  injury)  who  would  like  to  use  the  Web.  Moreover,  there  are 
millions  of  individuals  with  learning  disabilities  (which  are  not  cognitive  impairments)  who  experience  difficulties  with 
Web  sites.  The  most  salient  problems  experienced  by  both  groups  of  users  are  their  inability  to  synthesize  complex 
information  quickly  and  their  inability  to  adapt  to  inconsistent  interfaces.  It  is  very  likely  that  a designer  who  works 
hard  to  attend  to  usability  in  design  will  also  accommodate  this  group  of  users. 

? Experience  it  yourself:  Although  this  is  not  really  a simulation  of  having  a cognitive  impairment,  go  to  . This 

simulates  having  multiple  task  demands  simultaneously.  The  “hard”  version  simulates  what  it  is  like  to  have  an 
inconsistent  interface.  Often,  those  with  cognitive  disabilities  express  that  the  Web  is  too  much  to  process  at  one  time. 
See  how  you  do  in  a task  overload  situation. 

? Seizure  Disorders — Some  individuals  have  a particular  type  of  epilepsy  photoepilepsy.  Seizures  can  be 

induced  if  these  persons  encounter  information  that  is  blinking  at  a rate  from  greater  than  2 cycles  per  second  to  55 
cycles  per  second.  Blinking  content  is  discouraged  for  this  reason.  If  the  designer  insists  on  using  it,  setting  the  blink 
or  flash  rate  to  under  2 cycles  or  greater  than  55  cycles  can  avoid  triggering  seizures.  It  also  helps  to  reduce  the  size  of 
any  necessary  blinking  material. 

? Experience  it  yourself:  Go  to  any  irritating  blinking  ad  to  see  this  (e.g., 

http://webaim.org/training2002/weekl/seizuregraphic)  Please  do  not  view  these  if  you  have  photoepilepsy 
? Age  Related  Processes — ^The  largest  growing  segment  of  our  population  is  the  elderly.  As  we  age,  we  will 

likely  continue  to  enjoy  the  use  of  the  Web.  The  aging  process  includes  many  of  the  disability  types  described  above. 
For  many  of  us,  we  will  progressively  lose  vision,  hearing,  or  motor  skills.  To  the  extent  that  we  work  to  fix  the 
problem  of  Web  inaccessibility,  we  will  work  to  continue  our  own  Internet  enjoyment  for  years  to  come. 

An  8-Point  Process  for  Coordination  and  Reform 
The  Use  of  the  Web  in  Postsecondary  Education 

The  winds  of  change  have  blown  over  postsecondary  education.  As  Internet  technologies  transform  our  educational 
experiences,  so  these  technologies  create  a wide  chasm.  There  is  a very  real  divide  between  students  who  do  and  do  not 
have  access  to  the  Internet  in  education  today.  As  our  nation  grapples  with  issues  of  physical  access  to  hardware, 
software,  the  Web,  and  a National  Information  Infrastructure,  decision-makers  must  be  mindful  of  those  with  different 
issues  of  access.  These  individuals  are  those  with  disabilities  and  their  issues  of  access  are  related  to  the  environment  of 
the  Internet  today. 

Postsecondary  entities,  like  the  rest  of  our  society,  use  the  Internet  more  and  more  for  the  things  that  they  do 
everyday.  The  rapid  rise  in  the  use  of  the  Internet  as  well  as  the  functions  of  pages  within  sites  grow  each  day.  Today  if 
you  were  to  go  to  almost  any  institutional  site,  you  would  find  that  students  could  at  least  do  the  following: 

? get  information  about  required  courses, 

? register  for  courses, 

? look  up  transcripts, 

? order  books, 

? pay  for  educational  expenses  with  a credit  card, 

? take  online  courses, 

? gain  access  to  Web-enhanced  courses, 

? complete  Web-based  assignments, 

? meet  others  in  virtual  student  lounges, 

? conduct  research  from  library  holdings  or  the  Internet, 

? take  tests  online  (e.g.,  MAT),  and 

? get  information  from  Web-based  kiosks  about  social  and  community  events  and  issues. 

Each  day  students  can  find  new  ways  to  interact  with  their  education  provider  as  new  functions  are  added  to 
sites.  It  is  clear  that  the  Web  is  seen  as  a central  element  in  postsecondary  education.  So  much  so  that  many  institutions 
are  dedicating  enormous  resources  to  keep  up  with  the  advantages  that  technology  holds  for  students.  This  access  is 
viewed  as  desirable,  if  not  necessary,  for  students  to  succeed  in  their  educational  endeavors  and  participate  in  the  digital 
community  that  has  emerged.  However,  population  of  students  with  disabilities  are  being  increasingly  shut  out  of  these 
opportunities. 

When  institutional  sites  are  not  accessible  we  harm  students  with  disabilities  in  at  least  2 ways.  First,  they  may 
not  have  an  educational  experience  that  is  equivalent  to  their  non-disabled  peers.  Second,  they  loose  out  on 
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opportunities  to  leam  how  to  efficiently  gather  Web-based  information.  It  becomes  a tail-chasing  phenomenon  where 
lack  of  access  reduces  skill  acumen  and  fledgling  skills  further  reduce  access.  In  our  grandmother’s  words;  ’’practice 
makes  perfect".  Students  without  opportunities  to  practice  cannot  be  as  prepared  to  meet  their  future,  one  that  will 
include  the  Internet.  The  potential  failure  of  postsecondary  education  would  be  that  systems  are  not  created  or  sustained 
to  help  students  with  disabilities  participate  in  the  Web-based  society  that  is  growing  with  each  day. 

So  what  can  be  done  about  this  important  issue?  The  only  way  to  really  get  the  job  done  is  for  institutions  of 
higher  education  to  engage  in  a system-change  effort  where  they  are  dedicated  to  coordination  and  reform  of  their  Web 
site. 

Institutional  Reform  Efforts  and  the  Future  of  Accessibility 

On  the  topic  of  producing  a campus-wide  plan  of  institutional  reform,  Cynthia  Waddell  stated,  ’’Just  as  a 
removal  of  architectural  barriers  requires  a plan  for  implementation,  the  removal  of  technological  or  digital  barriers  in 
programs  and  services  requires  a comprehensive  institutional  plan  impacting  every  campus  office."  (The  Growing 
Digital  Divide  in  Access  for  People  with  Disabilities,  2000) 

There  are  many  elements  to  the  future  success  of  postsecondary  access.  The  most  salient  would  be  (a) 
commitment,  (b)  action,  and  (c)  an  eye  toward  new  technology  solutions.  Postsecondary  entities  must  begin  to  grapple 
with  current  inequities  and  federal  mandates.  Institutional  commitment  and  coordination  will  go  a long  way  toward 
reforming  the  present  crisis.  The  institutional  "actions"  that  are  taken  will  show  the  commitment  that  is  present.  These 
actions  must  include  all  stakeholder  groups:  students,  faculty,  Web  designers,  departmental  units,  administrators.  Action 
plans  should  be  created  with  input  from  all  and  should  articulate  change  in  small,  measurable  steps  that  are  place  along 
a reasonable  timeline.  Systems  that  include  multiple  points  for  input  in  the  process  will  be  more  likely  to  succeed  over 
time. 

Principles  of  Institutional  Reform 

Institutional  reform  is  not  a simple  process,  but  a very  complex  one  whose  scope  and  importance  are  increasing 
with  the  growth  of  the  Internet’s  use.  Before  moving  on  to  explaining  the  specific  steps  of  the  process  of  institutional 
reform,  it  is  valuable  to  keep  in  mind  a few  key  principles. 

Principle  1.  Understanding  the  Problem 

Always  in  take  into  account  the  size  of  the  problem  by  taking  into  account  the  size  of  your  institution.  It  is 
certainly  easier  to  coordinate  the  actions  of  10  individuals  than  1,000.  Moreover,  it  is  easier  to  insure  coordination 
across  15  units  than  150.  Size  does  matter  in  coordination  and  reform  efforts  in  postsecondary  education.  This  is  not 
unlike  any  corporate  environment  that  struggles  to  support,  monitor,  and  ultimately  comply  with  internal  policies  or 
federal  regulations.  As  you  progress  through  each  step  of  the  process,  remember  that  the  size  of  your  university  may 
affect  decision-making  and  action. 

Principle  2.  Understanding  the  Process 

Due  to  the  complexity  of  the  issue  of  Web  accessibility,  it  requires  a solid  commitment  to  the  process  in  terms 
of  people,  time,  money  and  facilities.  Resources  committed  by  a postsecondary  entity  also  factor  into  the  complexity  of 
the  problem.  Entities  that  can  appoint  someone  to  chair  an  accessibility  committee  as  part  of  their  role  assignment  (e.g., 
halftime  for  the  first  year)  will  do  better  with  Web  accessibility  than  those  that  cannot. 

Also,  postsecondary  entities  that  use  their  resources  to  create  a stable  environment  for  Web  development  staff 
will  have  a better  chance  delivering  accessible  sites  across  the  institution.  Training  and  support  in  accessible  design  for 
Web  developers  are  two  important  elements  of  this.  Salary  and  career  incentives  for  technical  development  personnel  to 
stay  in  postsecondary  education  are  others.  Over  1,400  Web  masters  in  postsecondary  education  were  electronically 
surveyed  by  WebAIM  staff  in  2000.  Of  the  536  response  (representing  a 38%  response  rate),  most  had  been  in  their 
current  position  for  only  2 years.  Just  over  half  of  the  sample  (54%)  indicated  Web  design  as  their  fiill-time 
responsibility.  Sixty-two  percent  (n=334)  reported  that  they  had  learned  about  at  least  some  aspect  of  accessibility;  most 
(n=300)  reported  that  they  did  so  on  their  own  and  not  during  any  pre -service  or  inservice  training.  It  is  likely  that 
frequent  changes  in  Web  personnel  would  have  a negative  affect  on  the  accessibility  of  sites  within  a postsecondary 
institution.  Postsecondary  entities  that  commit  more  resources  to  this  issue  will  also  be  able  to  better  monitor  the 
accessible  design  of  their  Web  development  staff. 

Besides  personnel,  another  resource  that  must  be  considered  is  money.  This  allows  for  the  allocation  of 


personnel  time  (as  part  of  role  assignments)  and  for  training  efforts  that  will  ensue.  It  is  possible  to  secure  outside 
sources  of  support.  However,  you  should  not  count  on  obtaining  these  types  of  support  (e.g.,  grants,  donations). 
Moreover,  you  should  not  wait  to  secure  outside  sources  of  support  prior  to  initiating  policy  efforts.  The  most 
reasonable  source  of  funding  will  come  from  line  items  within  the  institutional  budget.  It  is  much  like  the  physical 
facility  changes  that  were  required  when  the  Americans  with  Disabilities  Act  was  implemented  on  campus.  It  is  not 
wise  to  wait  for  money  from  outside  sources  to  begin  the  process  of  institutional  reform.  Your  institution  could  put 
itself  in  a vulnerable  legal  position. 

The  8-Step  Model  of  Institutional  Reform 

There  is  a model  of  institutional  reform  that  may  be  helpful  in  planning  change  at  your  institution.  This  model 
is  comprised  of  an  8 -step  process.  Each  step  is  like  the  spoke  of  a wheel,  and  each  is  essential  to  the  overall  function  of 
the  wheel.  If  you  miss  even  one  spoke,  or  if  a spoke  is  weak,  you  compromise  the  integrity  of  the  wheel.  So,  as  you 
begin  your  journey  of  institutional  reform,  we'd  like  to  give  you  some  information  to  plan  for  a safe  journey. 

Step  1.  Gaining  Administrative  Support 

Setting  out  on  your  journey  of  accessibility  must  begin  with  gaining  administrative  support  at  your  institution. 
This  backing  is  important  for  three  primary  reasons.  The  first  is  that  it  helps  to  validate  the  cause  of  Web  accessibility  at 
your  university.  This  gives  it  more  visibility  and  verifies  a commitment  that  other  members  of  the  institution  can  then 
look  to  in  their  own  dedication.  Even  in  institutions  where  change  can  occur  from  a grass-roots  level,  it  will  be 
important  to  secure  support  from  those  who  are  charged  to  make  institutional  decisions  and  those  who  control  budgetary 
items. 


Another  reason  the  sponsorship  of  your  administration  is  beneficial  is  the  value  this  status  sheds  on 
accessibility.  Those  both  outside  and  inside  the  institution  may  see  this  as  more  important  if  the  administration  is 
committed  to  making  positive  changes.  Over  time  it  will  help  you  set  policy,  create  training  and  support  mechanisms, 
as  well  as  monitor  the  success  of  your  institution’s  work. 

Step  2.  Organizing  an  Institutional  Web  Accessibility  Committee 

Once  the  administration  at  your  institution  supports  the  importance  of  Web  accessibility,  a group  should  be 
formed  to  draft  accessibility  policies  and  to  see  the  process  through  its  implementation.  It  is  vital  that  this  group  be 
comprised  of  other  key  individuals  at  your  institution.  Those  chosen  should  be  respected  in  their  individual  fields  and 
have  the  ability  to  influence  change  with  their  colleagues.  They  must  also  be  able  to  commit  the  time  necessary  to  see 
the  process  through.  In  some  instances  this  could  be  substantial  2-year  commitment. 

So,  who  should  be  part  of  this  process?  It  is  important  to  remember  that  every  institution  will  need  to  approach 
this  process  in  a slightly  different  way,  thus  membership  of  this  committee  should  reflect  these  differences.  There  are 
typically  at  least  5 groups  of  stakeholders  who  should  take  part  in  this  process.  These  groups  include:  the 
administration,  the  faculty,  the  Section  504  and  ADA  coordinators,  Web  designers,  and  students  with  disabilities.  At 
least  one  member  from  each  of  these  groups  should  participate  in  a Web  accessibility  committee,  although  it  may  make 
sense  to  have  more  than  one  representative  for  certain  groups. 

It  is  important  that  this  committee  be  constructed  in  such  a way  that  all  members  will  work  constructively 
toward  the  goal  of  helping  to  create  a fully  accessible  Web  for  their  institution.  Please  remember  that  these  individuals 
will  report  back  to  the  larger  groups  from  which  they  came.  As  such,  it  would  be  helpful  to  the  process  if  they  were 
respected  within  their  own  groups.  Representation  and  support  by  all  parties  is  essential  if  implementation  is  to  be 
widely  accepted.  The  implementation  of  Web  accessibility  at  your  institution  will  be  much  easier  and  when  all  of  these 
pieces  fit  together. 

Step  3.  Web  Policy:  Defining  a Standard 
Defining  Web  Accessibility 

After  you  gain  administrative  support  and  organize  an  effective  Accessibility  Committee,  the  next  step  is  to 
develop  the  institution's  Web  policy.  There  are  2 important  components  to  the  overall  Web  accessibility  policy.  The  first 
is  the  standard,  or  institutional  definition,  of  Web  accessibility.  Second  is  the  implementation  plan.  Taken  together,  they 
form  the  complete  policy.  What  follows  is  a description  of  the  accessibility  standard.  Descriptions  of  the 
implementation  plan  will  be  discussed  in  Step  4. 

One  of  the  first  responsibilities  of  the  Web  Accessibility  Committee  will  be  to  draft  a document  that  will  be 
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used  to  define  what  is  meant  by  the  phrase  "Web  accessibility".  Different  individuals  might  define  this  in  different 
ways.  The  importance  of  a clearly  defined  Web  accessibility  policy  cannot  be  overstated.  This  standard  will  enable 
everyone  to  understand  the  level  of  accessibility  your  institution  will  employ.  It  will  also  act  as  a planning  guide  for 
developers;  they  will  know  precisely  what  practices  they  must  include  in  their  design  to  fit  the  institutional  standard. 
Furthermore,  this  standard  will  serve  as  the  template  in  any  future  monitoring  effort.  This  process  is  not  unlike  the 
process  of  designing  the  specifications  for  the  production  of  a car.  What  laws  are  in  place  describing  weight  and  size  of 
specific  features  or  the  minimum  requirements  of  certain  functionalities?  Beyond  the  basic  elements  required  by  law, 
what  special  components  do  you  want  to  add  to  your  product?  Will  you  have  air  bags  or  anti-lock  brakes?  Determine 
then  that  the  first  task  of  this  group  is  to  develop  the  "specifications"  of  a Web  accessibility  definition  for  the  institution. 

Composing  a Detailed  Accessibility  Standard 

To  define  a Web  accessibility  policy  for  your  institution,  it  is  critical  to  remember  3 things.  First,  it  is 
important  to  make  sure  you  compose  a detailed  accessibility  standard.  It  does  little  good  to  announce  that  your 
institution  will  be  "ADA  compliant"  or  that  "We  will  insure  all  our  pages  are  accessible  to  students  with  disabilities". 
Ask  yourself,  what  does  this  mean?  Would  others  in  the  institution  interpret  this  statement  in  the  same  way?  If  you  were 
a Web  developer  would  you  know  if  you  had  created  a document  that  followed  institutional  policy?  If  you  were 
monitoring  the  accessibility  progress  of  this  institution,  would  you  know  how  they  are  doing?  In  order  to  help  those  that 
will  work  with  the  policy  you  must  first  define  the  standard  by  which  all  will  comply.  Provided  below  are  resources  you 
can  consult  in  forming  your  own  standard.  These  are  3 sources  that  will  provide  you  with  detailed  descriptions  of  Web 
accessibility  standards  and  guidelines.  Remember  that  these  guidelines  are  provided  as  models  and  references  for  you. 
There  are  no  hard  and  fast  rules  to  setting  a standard  and  there  is  no  right  answer.  But  you  must  make  sure  that  standard 
that  you  choose  or  create  suits  your  institution. 

1.  The  Web  Accessibility  Initiative  (W.A.I.)  of  the  World  Web  Consortium  (W3C)  has  studied  this  topic  in  depth 
and  has  published  a set  of  guidelines  which  define  accessibility.  These  guidelines  have  been  organized  into  three 
priority  levels.  Priority  1 guidelines  are  those  which  must  be  met  for  a page  to  be  accessible.  Priority  2 guidelines 
should  be  met.  Priority  3 guidelines  may  be  met  (it  would  be  nice  if  they  were,  but  perhaps  not  necessary).?  ? Because 
of  the  credibility  and  international  scope  of  the  W.A.I.-  and  because  of  their  affiliation  with  the  W3C  - the  W.A.I. 
guidelines  have  served  as  a basis  for  organizations  setting  standards  for  Web  accessibility.  Nevertheless,  the  W.A.I. 
guidelines  as  they  are  currently  written  may  not  necessarily  fulfill  the  policy-setting  needs  of  all  organizations.  It  may 
be  beneficial  for  some  organizations  to  include  in  their  standard  other  principles  of  Web  accessibility  which  the  W.A.I. 
has  not  enumerated.  For  example,  the  W.A.I.  does  not  mention  the  timeliness  of  accessible  content  as  one  of  their 

' guidelines.  In  the  case  of  online  courses,  it  is  important  the  online  syllabi  be  made  accessible  to  students  with 
disabilities  at  the  same  time  that  they  are  made  available  to  other  students,  to  avoid  putting  the  disabled  students  behind. 
Organizations  may  find  other  considerations  which  the  W.A.I.  guidelines  do  not  address,  which  are  important  to  the 
organization.?  ? To  more  specifically  study  the  WAI  Guidelines,  visit  to?. 

2.  Section  508  of  the  Rehabilitation  Act  has  a detailed  set  of  16  items,  written  in  standards  language,  that  define 
what  Web  accessibility  will  mean  for  Federal  agencies.  To  view  the  government  Web  508  site,  visit  to 
http://www.section508.gov/index.cfm?FuseAction=Content&ID=12  ? ? To  view  a streamlined  checklist  of  the  above 
items,  visit  the  WebAIM  site  at  http://www.webaim.org/standards/5Q8/checklist? 

3.  Many  institutions  have  created  hybrid  standards  that  blend  elements  of  both  the  WAI  and  Section  508.  Some 
have  even  created  some  of  their  own  standards.  These  are  found  in  the  Online  Institutional  Examples  section  below. 

Review  and  Revision  of  Your  Accessibility  Standard 

Once  you  have  a draft  of  an  institutional  standard  for  Web  accessibility,  it  is  important  that  the  initial  draft  of 
the  accessibility  standard  be  sent  out  for  review  by  others  at  your  institution.  Including  others  in  open  dialogue  will 
improve  the  likelihood  of  adoption  and  successful  implementation  of  the  standard.  Institutional  ownership  that  results 
from  this  feedback  step  is  often  a key  to  standard  success.  We  would  recommend  the  draft  be  seen  by  the  groups  who 
have  representation  on  the  committee  as  well  as  those  who  will  responsible  for  implementing  it  (i.e.  designers  and 
faculty).  Finally,  t would  be  good  for  the  Committee  to  assume  that  the  draft  will  be  reworked  after  it  has  gone  out  for 
review,  rather  than  assume  that  it  will  receive  widespread  approval.  Most  importantly,  the  members  of  the  Accessibility 
Committee  should  insure  that  all  feedback  is  strongly  considered  in  any  reworking  of  the  draft  for  production  of  a final 
policy. 

Step  4.  Web  Policy:  Creating  an  Implementation  Plan 

In  postsecondary  education,  a priori  systems  must  be  created  to  optimize  the  participation  of  all  students  and 
staff.  It  is  imperative  that  we  create  goals  aimed  at  establishing  coordinated  systems  that  enable  full  access  for  all.  The 
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sooner  we  create  and  implement  sustainable  solutions,  the  sooner  ALL  students  can  participate  in  their  right  to 
experience  the  power  of  the  Internet  for  lifelong  learning.  The  community  of  postsecondary  education  is  a resourceful 
group  of  individuals,  so  the  composition  of  an  implementation  plan  can  be  accomplished  in  these  environments. 

An  essential  part  of  creating  an  implementation  plan  is  to  thoroughly  document  the  actions  taken  by  the 
committee  and  institution  at  every  step.  The  Web  policy  you  construct  is  not  only  a great  way  to  lay  the  foundation  for 
your  accessibility  travel  and  destination,  but  it  also  documents  your  good  faith  efforts  for  others  to  see.  At  every  step, 
the  policy  should  leave  a detailed  and  well-documented  proceeding  for  others  who  may  use  it  as  a reference. 

When  you  are  constructing  your  institutional  Web  implementation  plan,  there  are  four  areas  to  consider.  They  are 
establishing  timelines,  setting  priorities  in  terms  of  what  standards  to  achieve  and  on  what  timeline,  delegating 
responsibilities  and  monitoring  the  progress.  These  tasks  often  do  not  occur  in  order,  but  should  be  addressed  when 
needed. 

Establishing  Timelines 

The  first  task  in  formation  of  your  plan  is  establishing  a time  frame  for  implementation.  Depending  on  the  size 
and  scope  of  need  for  change  at  your  institution  this  time  frame  may  extend  from  weeks  to  years.  It  is  important  to 
recognize  that  this  is  a complex  process.  Establishing  a timeline  allows  you  to  sequentially  follow  the  tasks  and  duties 
you  lay  out  for  all  of  those  participating  in  Web  accessibility  at  your  institution.  It  also  stimulates  action  and 
accountability  in  all  involved  by  creating  deadlines  by  which  work  must  be  completed.  However,  keep  in  mind  that, 
although  initial  timelines  are  an  aid  in  your  travel,  Web  accessibility  compliance  is  an  ongoing  process  and  must  be 
established  permanently  as  well  as  in  the  short-term  beginning  phases.  Plans  should  be  made  for  both  initial  changes 
and  long-term  establishment  of  Web  accessibility  as  a priority  at  your  institution. 

Setting  Priorities  in  Terms  of  Standard  Achievement 

Due  to  the  complexity  of  the  institutional  reform  process,  setting  priorities  at  the  onset  along  with  deadlines  for 
observance  is  essential.  These  priorities  outline  which  pages  will  be  required  to  be  compliant  in  what  order  as  well  as 
the  minimum  requirements  for  established  deadlines.  This  process  may  also  take  part  in  phases.  For  example,  one 
university  required  that  the  pages  with  the  top  20%  of  hits  on  the  institutional  Web  site  were  the  first  priority.  These 
pages,  as  well  as  training  fell  into  the  first  phase,  while  matters  such  as  gathering  further  funding  fell  into  a third  phase. 

Delegating  Responsibilities 

Once  standards  and  a timeline  have  been  determined,  the  next  step  is  to  delegate  responsibility  for  each  of  the 
tasks.  This  portion  of  the  policy  should  include  a breakdown  of  each  work  group's  purpose  and  tasks.  As  tasks  are 
delegated  to  specific  members  of  the  main  accessibility  committee,  subgroups  will  form.  Who  will  be  included  in  these 
groups?  When  will  they  meet?  A list  of  campus  entities  and  what  their  responsibilities  are  to  enforce  the  university 
policy  should  be  delineated  clearly. 

Monitoring  Progress 

The  best  implementation  plan  does  no  good  if  there  is  no  accountability.  Laying  a foundation  for  monitoring 
progress  and  following  through  is  the  best  assurance  that  institutional  reform  will  be  successful  and  complete.  It  is  very 
important  that  you  develop  a system  by  which  to  identify  and  maintain  contact  with  individuals  who  have  been  assigned 
specific  responsibilities  within  your  plan.  The  administration  can  provide  this  leadership  in  monitoring  and  compliance, 
as  well  as  delegating  specific  tasks  and  follow-up  to  others. 

For  example,  the  administration  in  some  higher  education  entities  require  that  Web  masters  attend  training  on 
institutional  policies,  sign  agreements  to  follow  them  and  are  monitored  over  time  for  compliance.  Of  course  such  an 
endeavor  would  assume  that  the  administration  has  a way  to  identify  and  track  ALL  individuals  who  place  web  content 
on  an  institutional  sever  and  monitor  accessibility  as  one  feature  of  broader  institutional  policies.  The  framework  for 
these  abilities  must  be  laid  early  on. 

There  are  many  different  methods  emerging  throughout  the  nation  on  how  progress  is  being  monitored.  Let  us 
present  a few  for  your  consideration.  The  first  is  random  checks.  This  is  fairly  self-explanatory,  involving  persons  being 
hired  to  randomly  test  Web  pages  for  compliance  to  institutional  standards.  Another  method  is  a yearly  purge,  in  which 
all  pages  not  declared  compliant  by  campus  designers  are  swept  off  the  system.  This  may  help  to  clean  rogue  content  off 
the  system,  however,  it  still  does  not  have  a system  for  verifying  whether  the  sites  are  actually  compliant  or  not. 

Another  method  is  the  honor  system,  where  it  is  assumed  that  those  responsible  for  development  will  keep 
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their  sites  fully  accessible.  However,  this  brings  up  the  dilemma  of 'Why  have  a policy  if  we  don't  even  know  whether 
or  not  anyone  is  following  it?'  A final  method  is  the  peer  system.  In  this  system,  a network  of  designers  is  set  up  to 
provide  mutual  feedback,  support  and  idea-sharing  opportunities.  This  association  has  established  policies  and 
meetings.  In  addition,  sites  are  sent  to  one  another  in  this  peer  group  for  review.  This  is  typically  done  on  a regular  basis 
(e.g.,  2 pages,  twice  a month).  The  sites  are  checked  against  the  established  policies  of  the  group  and  feedback  is 
returned  to  the  university  Web  master  as  well  as  the  designer  of  the  given  page.  In  this  system  it  is  imperative  that 
feedback  be  supportive,  as  well  as  constructive. 

Step  5.  Gathering  Baseline  Information 

Conducting  a baseline  is  an  essential  part  of  institutional  reform.  Once  you  have  created  your  institutional 
policy  (the  standard  and  implementation  plan),  it  is  time  to  discover  where  you  lie  according  to  the  standards  you  have 
set  for  yourself.  This  will  help  you  know  where  to  concentrate  your  efforts  and  where  you  need  to  continue  to  work. 

When  conducting  a baseline,  it  is  valuable  to  test  approximately  8-10%  of  all  pages  on  your  institutional  Web 
site.  It  also  makes  sense  that  as  you  randomly  sample  these  pages  you  insure  that  you  have  included  pages  across  all 
levels  and  departments.  In  order  to  gather  this  information,  you  could  create  a checklist  of  your  institutional  standard  for 
Web  accessibility.  If  your  institution  chose  Priority  I and  II  of  the  WAI  or  Section  508,  you  may  be  able  to  use  some  of 
the  features  of  accessibility  validators  that  are  available  today.  An  example  of  a checklist  that  was  developed  based  on 
the  Section  508  standard  (http://www.webaim.org/standards/508/checklisQ  might  provide  some  insight  into  how  you 
could  blend  automatic  features  of  validators  with  manual  checks  to  gather  baseline  data. 

Techniques  for  Establishing  a Baseline  Measure  of  Accessibility 
Technique  1 : Evaluate  the  Site  Using  an  Automated  Accessibility  Validator 

Products  such  as  Bobby  Worldwide  (commercial  version),  AccessEnable,  AccVerify  Server,  PageScreamer 
Spider,  and  InFocus  can  scan  through  entire  Web  sites  and  generate  reports  based  on  the  errors  which  they  find.  Most  of 
these  programs  allow  you  to  choose  which  criteria  to  designate  as  "errors"  for  the  purpose  of  the  report.  It  is  a good  idea 
to  decide  ahead  of  time  which  of  the  criteria  will  cause  a page  to  fail.  This  will  eliminate  the  extraneous  information  that 
these  programs  can  produce  that,  while  sometimes  useful  to  developers,  can  sometimes  confuse  the  issue  by  labeling  too 
many  things  as  errors,  some  of  which  do  not  apply  to  the  site  at  hand.  These  programs  can  flag  such  errors  as  missing  alt 
text  and  missing  frame  titles,  but  they  will  not  be  able  to  determine  whether  or  not  the  alt  text  or  frame  title  is  the  most 
appropriate  for  the  context.  That  is  why  Step  2 is  so  important. 

Technique  2:  Evaluate  the  Site  Using  a Checklist 

Using  a checklist,  evaluate  the  subjective  elements  of  all  or  randomly-selected  pages.  The  WebAIM  Section 
508  checklist  is  an  example  of  a pass-fail  checklist  that  could  be  used  for  this  purpose.  Before  embarking  on  this  step, 
though,  it  is  important  to  ensure  high  inter-rater  reliability.  The  people  using  the  checklist  should  be  familiar  with  Web 
accessibility  concepts.  They  should  independently  rate  a test  group  of  pages,  then  compare  notes  and  opinions.  Step  2 
will  be  effective  only  if  it  is  clear  that  these  people  will  rate  pages  similarly. 

Technique  3:  Interpret  the  results 

It  will  probably  be  helpful  to  generate  a few  different  types  of  reports.  For  example,  it  can  be  helpful  to 
compare  the  pass/fail  ration  of  pages  evaluated  with  the  automated  software  to  the  pass/fail  ratio  of  pages  evaluated  by 
human  evaluators.  You  could  also  separate  out  the  errors  by  type,  WAI  priority  or  other  criteria. 

Step  6.  Training  and  Technical  Support 

The  Importance  of  Training  and  Technical  Support 

Once  you  have  institutional  support,  policies  and  plans  in  place,  as  well  as  a baseline  of  your  current  situation, 
the  next  step  is  to  provide  training  and  technical  assistance  for  those  who  place  content  on  the  Web.  Training  is  a critical 
element  in  the  implementation  and  success  of  your  institutional  reform.  Web  accessibility  may  be  a brand  new  issue  for 
many  who  will  be  responsible  for  its  implementation.  The  training  and  support  you  provide  will  be  absolutely  necessary 
to  help  them  fulfill  the  institutional  policies  that  have  been  established. 

Identify  Your  Audience 

It  is  important  to  identify  those  who  place  content  on  your  institution's  Web  site.  As  you  design  your  training 
and  support  mechanisms,  please  be  aware  of  the  skills  of  these  individuals.  Situations  can  arise  where  anyone  with  Web 
development  skills  or  the  desire  to  learn  the  skills  are  given  responsibilities  to  create  and  post  Web  content  on  an 
institutional  Web  site.  Some  of  these  individuals  may  be  professional  Web  developers,  such  as  in-house  Web  design 
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staffer  outside  design  contractors.  Some  of  these  individuals  may  be  faculty  or  other  support  staff  that  wish  to  explore 
Web  design  and,  in  return,  provide  a ’’free”  service  to  their  workgroup  or  department.  Some  may  be  students  hired  for  a 
single  term  or  for  one  year.  Other  individuals  may  be  nephews  and  neighbors,  friends  and  children.  Anyone  with  skills 
to  write  in  a markup  language  (e.g.,  HTML,  or  hypertext  markup  language,  the  language  of  the  Internet),  or  understand 
authoring  tools,  or  course  management  systems  (e.g..  Blackboard,  WebCT,  E-college)  can  be  given  the  task  to  design 
and  develop  elements  of  a postsecondary  education  Web  site.  Few  campuses  actually  know  all  of  the  individuals  that 
develop  and  place  Web  content  onto  the  institutional  server  system.  If  the  institution  cannot  identify  all  of  these 
individuals  it  is  unlikely  that  they  can  provide  the  necessary  direction,  support,  training,  and  monitoring  for 
accessibility. 

In  addition,  although  accessibility  fixes  may  not  be  a difficult  task  for  those  who  are  technically  skilled,  new 
techniques  and  content  can  provide  some  initial  confusion.  You  will  want  to  create  a structure  in  your  institution  that  is 
supportive  of  implementing  these  changes  and  not  merely  request  change  in  the  absence  of  support.  The  training  and 
resources  you  provide  must  be  as  clear  as  the  standards  you  wish  your  developers  to  follow.  Web  accessibility  can  be  a 
complex  process  but  is  simplified  with  good  training  and  ongoing  support. 

Areas  of  Training 

You  could  focus  training  in  at  least  three  primary  areas;  the  areas  of  Web  accessibility,  your  institutions’ 
accessibility  standards,  and  strategies  and  techniques  for  accessible  coding  (e.g.,  HTML,  CSS,  XML)  and  multimedia 
(Flash,  Video,  PDF,  Power  Point). 

Web  Accessibility  Issues  (coding  and  multimedia) 

It  would  be  reasonable  to  create  a workshop  that  would  focus  merely  on  accessible  HTML  (or  other  code 
techniques)  and  multimedia.  These  are  the  techniques  needed  by  designers  to  create  accessible  content.  However,  in 
order  to  help  designers  and  faculty  understand  why  they  need  to  design  and  develop  accessibly,  it  is  valuable  to  help 
them  first  understand  the  broad  issues  of  accessibility  itself. 

1.  Understanding  the  perspectives  of  the  student  with  a disability  is  critical 
(http://webaim.org/training2002/weck  1 /).  Often,  if  Web  designers  really  understand  what  student  with 
disabilities  face,  the  designers  will  embrace  this  perspective  and  be  able  to  internalize  many  of  the  changes  at 
an  intuitive  level.  Introducing  staff  to  the  personal  stories  of  students  and  faculty  at  your  institution  can  be  a 
powerful  experience  for  Web  developers  as  well.  Simulations  and  videos,  in  combination  with  live  interviews 
or  panels  can  be  another  powerful  aspect  of  your  training.  Help  those  involved  answer  questions  such  as; 
’’What  population  is  benefited  by  an  accessible  Internet?  What  problems  do  they  face  without  it?  How  can 
these  problems  be  overcome?”  Having  an  experience  with  those  who  struggle  with  the  Internet  helps  promote  a 
greater  desire  to  change  professional  practices. 

2.  Understanding  the  incentives  for  implementing  Web  accessibility,  the  reasons  of  ethics,  business  and 
law  (http://webaim.Org/training2002/week2/L  can  also  motivate  them  to  initial  acceptance  and  help  them  effect 
change. 

Institutional  Standards 

Secondly,  in  development  of  training  it  is  also  essential  that  all  parties  involved  understand,  not  only  the  reasons  for 
accessibility  in  general,  but  your  institution’s  choice  of  standard.  When  designers  and  faculty  clearly  understand  the 
exact  regulations  of  compliance  chosen  for  your  university,  they  are  more  likely  to  effectively  fulfill  their  commitments 
to  Web  accessibility.  Consider  training  in  the  following  ways: 

? Include  specific  wording  of  the  Web  accessibility  policy  of  your  own  institution 

? Provide  contact  information  for  people  so  they  can  ask  questions  of  those  in  charge 

? Make  specific  dates  and  schedules  for  training  available  on  the  current  institutional  policy  or  plan 

Techniques  and  Strategies 

As  far  as  an  overall  plan  for  institutional  training  and  technical  support,  many  factors  must  be  taken  into 
consideration.  Will  you  create  the  actual  training  in  -house  or  will  you  outsource?  Will  you  need  separate  training  for 
designers  than  for  faculty?  In  what  areas  will  you  specifically  need  to  train?  It  is  important  that  you  consider  the  unique 
needs  of  each  group  at  your  institution.  Even  within  groups,  there  will  be  different  levels  of  need.  Designers  are  more 
likely  to  need  training  in  both  multimedia  and  coding  techniques,  whereas  most  faculty  will  benefit  more  from  knowing 
how  to  optimize  the  tools  they  use,  such  as  PDF  files,  PowerPoint,  or  how  to  make  accessibility  workarounds  for  a 
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course  management  system  (e.g.,  WebCT,  Blackboard). 

One  good  place  to  start  making  plans  for  training  is  to  conduct  a survey  of  the  accessibility  knowledge  skills 
among  those  who  will  be  involved.  Questions  could  address  what  they  know  about  the  issues,  as  well  as  specific  coding 
techniques.  These  skills  and  knowledge  might  also  be  discovered  as  the  baseline  study  is  performed  for  your  institution. 
Understanding  the  needs  of  your  designers  and  faculty  is  valuable  to  create  or  tailor  training. 

Other  decisions  to  make  in  regards  to  training  involve  resources  and  method  of  delivery,  timing  and 
accountability.  Will  the  training  be  conducted  face  to  face  or  online?  Will  online  resources  be  provided  that  link  to 
valuable,  informational  Web  sites?  Will  all  faculty  and  designers  have  a point-of-contact  for  answers  that  cannot  be 
found  in  manuals  or  on  the  Internet? 

Step  7.  Monitoring  Your  Institutional  Policy 

At  this  stage  of  the  journey  you  are  well  on  your  way  toward  accessibility.  The  key  now  is  persistence.  Now  is 
the  time  to  execute  the  strategies  you  outlined  for  monitoring  progress  in  your  implementation  plan.  This  is  the  time  to 
make  sure  that  all  goals  for  implementation  are  being  fulfilled.  Find  out  where  the  problems  lie  and  offer  support  and 
training  to  alleviate  them.  Take  any  steps  to  insure  established  accessibility  and  make  sure  Web  accessibility  becomes  a 
permanent  process  at  your  university. 

Questions  to  Ask  Yourself 

At  this  point  it  is  a good  idea  to  reevaluate  the  commitments  to  Web  accessibility  made  by  those  at  your  institution. 
Consider  some  of  the  following  questions: 

? How  are  your  Web  designers  performing? 

? Have  they  all  completed  training? 

? Are  the  pages  up  to  standards? 

? Are  certain  designers  or  faculty  members  having  difficulty  in  maintaining  the  standards?  How  might  they  be 

encouraged  or  helped? 

? Are  there  new  designers,  faculty  or  administrators  who  need  accessibility  training? 

? Are  there  changes  that  need  to  be  made  to  the  training? 

Planning  Guide 

This  is  a good  time  to  pull  out  your  implementation  plan  and  evaluate  your  progress  in  terms  of  the  timelines 
and  guidelines  established.  Go  through  the  plan  section  by  section  and  be  certain  everything  is  moving  along  according 
to  plan. 

Step  8.  The  Importance  of  Flexibility  in  Change 

Perhaps  the  most  important  concept  to  remember  as  you  travel  the  road  of  institutional  reform  is  that  change  is 
a dynamic  and  flexible  process.  There  will  be  changes  in  staff,  standards  and  technologies.  Your  institution  must  have  in 
place  a system  to  with  these  changes  and  a mechanism  to  update  standards  as  new  technologies  emerge.  You  should 
also  remember  that  there  will  be  turnover  of  designers  and  new  hires  all  the  time.  In  that  case,  new  training  will  be 
required,  as  well  as  retraining  when  standards  change  or  work  becomes  sloppy.  Your  institution  must  always  be  at  the 
ready  for  revamping  and  retooling  as  necessary.  In  fact,  there  should  be  allowances  and  provisions  made  for  this  in  all 
policy  plans. 
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Abstract:  This  study  looked  at  how  new  online  learners  in  the  first  course  in  a graduate  level  online 
instructional  technology  certificate  program  experienced  interacting  with  others  in  an  asynchronous 
online  discussion  and  learning  to  learn  online.  The  study  followed  twelve  new  online  learners 
throughout  the  class  using  email  questionnaires  and  email  follow-up  questions.  Significant  themes 
included  difficulties  and  fears,  time  factors,  and  desire  for  more  feedback  and  smaller  groups.  Over 
time,  most  participants  adapted  and  found  benefits  in  addition  to  challenges.  One-fourth  of  the 
participants  did  not  adapt  and  left  the  program  at  the  end  of  the  course.  Postings  gradually  became 
more  social  and  fewer  were  directed  primarily  to  the  instructor.  Correlational  and  predictive 
analyses  provided  insights  into  the  importance  of  computer  skills,  initial  experiences  in  the  course, 
and  quantity  of  discussion  postings. 

The  entire  paper  (with  full  literature  review  and  additional  tables  and  participant  data)  is  available 
at:  http://www.learningpeaks.com/discussion_study. 

Introduction 

Student  discussion  in  a college  classroom  conveys  numerous  instructional  benefits  and  is 
considered  to  be  critical  to  learning  (Gunawardena  and  Zittle,  1997;  Nunn,  1996;  Ormrod,  2000). 

If  interaction  with  people  is  important  and  discussion  is  the  primary  venue  for  this  interaction,  we 
must  understand  why  learners  chose  to  (and  not  to)  participate,  in  order  to  optimize  decisions 
about  design  and  facilitation  in  online  learning  environments.  The  goal  of  this  study  was  to  take  a 
close  look  at  the  thoughts  and  feelings  of  adults  who  are  new  to  online  learning  as  they  take  their 
first  online  course.  The  primary  research  questions  at  the  outset  were: 

• How  do  new  online  learners  view  online  discussions? 

• What  factors  influence  participation  in  online  discussion?  What  makes  participation  more 
likely?  Less  likely? 

• Do  learners'  views  and  participation  change  over  time,  as  the  course  progresses? 

What  I found  in  these  new  online  learners'  responses  went  beyond  the  original  scope  of 
these  questions.  I came  to  understand  how  they  perceived  some  of  the  challenges  and 
opportunities  of  asynchronous  online  learning,  in  general,  as  well.  All  of  the  participants  struggled 
with  using  a new  medium  for  learning.  Most  overcame  the  challenges  and  found  the  process  to 
be  satisfactory  or  better.  The  challenges  were  insurmountable  for  a few  participants  and  they 
chose  to  not  remain  in  the  certificate  program  after  the  end  of  the  course. 

Participant  Solicitation 

This  study  involved  twelve  out  of  the  thirty  students  in  two  online  sections  of  a graduate 
level,  asynchronous,  large  public  university  instructional  technology  course  called  "Developing 
Educational  Websites."  This  course  is  the  first  in  a four  course  online  certification  program  called 
"Designing  and  Developing  Web-based  Learning  Environments."  I decided  to  study  only  new 
online  learners  because  returning  online  learners  had  already  self-selected  this  method  of 
learning  as  acceptable  to  them. 

Data  Collection 

Participation  consisted  of  responding  privately  to  me,  via  email,  to  three  email 
questionnaires  (I  used  the  term  "email  journals"  with  participants  in  order  to  encourage  in-depth 
answers)  at  the  beginning,  middle,  and  end  of  the  study.  The  course  was  10  weeks  long  and  the 
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questionnaires  were  sent  to  participants  by  email  in  weeks  2,  5,  and  10.  As  students  responded,  I 
followed-up  via  email  with  additional  questions.  Questions  for  the  middle  and  final  questionnaires 
were  developed  in  response  to  the  types  of  answers  received  in  earlier  questionnaires. 

Data  Analysis 

As  I read  through  each  journal,  follow-up,  and  discussion  posting,  I looked  for  themes.  It 
became  apparent  to  me  that  participants  were  also  describing  their  experience  of  learning  to 
learn  in  this  environment  in  addition  to  providing  insights  about  online  discussion  and  interaction.  I 
added  an  additional  question  to  be  answered  in  this  study:  How  do  new  online  learners 
experience  this  environment  (in  general)? 

I started  with  a list  of  expected  general  themes  (personal  issues,  other  people  issues, 
technology  issues,  course  issues),  gained  from  the  literature,  and  added  additional  sub-themes 
as  they  became  apparent.  The  general  codes  I began  with  served  as  containers  for  the  sub- 
codes that  emerged.  In  addition,  a general  code  for  general  asynchronous  discussion  and  online 
learning  issues  emerged.  The  following  sub-themes  were  coded: 


Themes 

Primary  Sub-Themes 

1.  General 
online 

learning  and 
/asynch 
discussion 
issues 

1.1  Lack  of  immediacy  of  response/Disjointed  nature  of  discussion 

1.2  Need  authentic  activities  to  make  discussion  "real" 

1.3  F2FA/isual  cuesA/Vritten  communication  vs.  spoken/  Hard  to  connect 
names  with  real  people 

1.4  24/7  / More  worl^effort  than  F2F  class 

1.5  Helpful  posts:  a solution  to  problems/others  work  keeps  me  on 
track/feedback/instructor  comments 

1.6  Collegiality  develops  over  time  and  with  right  design 

1.7  Benefits  of  asynchronous  discussion  and/or  online  learning 

2.  Personal 
issues 

2.1  Self-consciousness/Looking  stupid/How  1 will  be  perceived/How  much  to 
reveal/others  know  so  much  more  than  me 

2.2  My  own  efforts  make  a difference  to  the  discussion  and  to  the  course/ 
You  get  what  you  put  in/I  can  do  this. 

2.3  Feel  timid  about  asking  for  help  or  pushing  for  own  needs. 

3.  Other  people 
issues 

3.1  Want  comments  and  feedback  from  other  learners 

4.  Technology 
issues 

4.1  Difficulty  of  following  the  discussion/thread 

5.  Course 

issues 

5.1  Small  groups  work  desired 

5.2  Post  to  complete  assignments/Posting  is  required 

Table  1:  Themes  and  Sub-Theme  Codes 


Venita  Doughty,  another  doctoral  student,  helped  me  evaluate  discussion  forum  postings 
for  type  of  posting  and  audience  of  the  postings.  This  was  done  in  order  to  understand  how 
students  used  the  discussion  forum,  and  provided  additional  insights  into  the  second  and  third 
research  questions. 

Results 

Study  participants  were  primarily  mid  career  adults,  working  in  K-12,  higher  education,  or 
corporate  training,  with  advanced  degrees  (or  working  on  an  advanced  degree).  The  mean  age 
was  43  years  of  age,  with  a range  between  24  and  59  years  of  age.  The  mean  level  of  education 
was  master's  degree. 

The  nature  of  the  certificate  tends  to  attract  people  with  an  interest  and  aptitude  for 
technology,  but  many  students  expressed  technical  frustrations  during  the  course.  The  range  of 
skills  needed  to  build  instructional  websites  is  broad,  and  even  those  with  relatively  higher 
technical  skills  coming  into  the  program  experienced  difficulties  with  the  range  of  skills  taught. 
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Many  expressed  a great  deal  of  nervousness  about  learning  online  and  learning  these  skills  from 
the  outset. 

In  the  following  sections,  I will  describe  the  themes  that  emerged  and  provide  some 
examples  of  participant  comments  that  illustrate  many  of  the  themes  and  sub-themes. 

Pseudonym  initials  and  the  journal  number  precede  excerpted  comments. 

General  online  learning  and  /asynchronous  discussion  issues 

One  of  the  most  pervasive  themes  that  emerged  in  participants’  journals,  follow-ups,  and 
discussion  postings  was  that  communicating  through  online  asynchronous  discussion  was 
generally  not  nearly  as  compelling  as  face-to-face  discussion.  For  many,  it  felt  disjointed.  Many 
participants  expressed  this  much  more  strongly  at  the  beginning  and  middle  of  the  course  than 
towards  the  end.  Many  of  the  participants,  however,  came  to  find  communicating  this  way  as 
acceptable  as  the  course  continued. 

The  three  participants  who  chose  not  to  continue  with  the  certificate  program  after  the 
course  ended,  TZ,  CY,  and  AH,  expressed  strong  negative  feelings  about  online  discussion,  and 
these  feelings  continued  throughout  the  course.  A related  theme  was  how  the  lack  of  visual  cues 
made  communication  difficult.  A number  of  the  study  participants  expressed  difficulty  with 
knowing  who  was  who.  On  the  other  hand,  one  participant  felt  that  it  was  actually  easier  to  know 
about  the  people  in  the  course  in  this  medium. 

Many  participants  described  the  challenges  of  communicating  in  a written  medium. 
Spoken  communication,  for  most,  conveyed  subtle  and  not  so  subtle  benefits  over  written.  Some 
also  felt  increased  anxiety  about  communicating  in  this  way.  JH  felt  cautious  about 
communicating  in  writing  and  it  impacted  her  willingness  to  provide  substantive  feedback.  She 
described  an  incident  where  she  strongly  questioned  another  student's  conclusions  and  began  to 
write  out  a response  but  then  changed  her  mind  about  posting  it.  The  lack  of  visual  cues  in  written 
communication  also  impacted  understanding. 

Many  of  the  study  participants  felt  that  learning  this  way  felt  like  a lot  more  work  than 
learning  in  a traditional  face-to-face  classroom.  The  commitment  felt  more  like  24/7  rather  than  a 
more  typical  face-to-face  class  where  the  commitment  was  "Tuesdays  from  5-9  PM." 

Despite  the  challenges,  many  of  the  study  participants  began  to  see  positive  aspects  of 
communicating  publicly  over  time.  Collegiality  and  the  sense  of  a shared  experience  developed 
over  time  and  under  the  right  circumstances.  The  right  circumstances  turned  out  to  be  a publicly 
posted  final  project  and  the  expectation  (assigned  and  graded)  to  provide  substantive  peer 
feedback  on  others'  projects.  Many  participants  emphasized  that  this  type  of  authentic  activity 
(rather  than  "forced  interaction")  was  just  the  thing  to  make  them  feel  connected  to  others  and 
that  more  of  these  types  of  assignments  were  needed. 

Personal  issues 

One  of  the  most  consistent  and  pervasive  comments  that  emerged,  especially  in  the  first 
and  second  journals  and  discussion  postings  at  the  beginning  of  the  course  was  self- 
consciousness  and  fear  of  looking  stupid  to  others.  Many  of  the  study  participants  asserted  that 
others  knew  an  awful  lot  more  than  they  did.  These  remarks  matched  and  extended  the  remarks 
made  about  the  nature  of  written  communications  described  earlier. 

Issues  with  others 

When  asked  in  the  second  set  of  journal  questions  what  would  make  participants  more 
likely  to  Interact  in  the  discussion  forum,  the  most  consistent  answer  was  more  feedback  from 
other  learners. 

Technology  Issues 

Some  participants  felt  that  following  conversation  in  the  discussion  form  was  confusing, 
disjointed,  and  time  consuming.  Some  participants  adjusted  well  to  the  technology  over  time  but 
others  continued  to  struggle  with  it.  For  those  that  struggled,  the  technology  augmented  the 
disjointed  and  time-consuming  nature  of  communication. 

Course  Issues 

A number  of  participants  expressed  a desire  for  working  in  smaller  groups.  Participants 
wrote  that  it  would  be  easier  and  more  comfortable  to  get  know  people  this  way  and  felt  that  they 
would  feel  more  connected  and  less  anxious. 
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A number  of  participants  chastised  me  for  making  participation  in  the  discussion  forum 
such  a large  part  of  the  course  evaluation  (it  comprised  20%  of  the  final  grade).  Many  said  they 
felt  forced  to  participate.  Some  said  discussion  and  participation  were  essentially  a waste  of  their 
time.  They  preferred  to  spend  the  time  in  more  solitary  activities. 

Time... 

An  emerging  meta-theme  that  tied  together  all  the  others  was  the  element  of  time.  Many 
of  the  study  participants  began  to  see  positive  aspects  of  asynchronous  discussion  and  learning 
as  the  course  progressed.  Many  saw  that  challenges  were  balanced  by  benefits,  and  that  both 
challenges  and  benefits  could  exist  at  the  same  time.  A few  even  described  benefits  over 
learning  in  a face-to-face  setting.  Some  of  the  challenges  (e.g.,  takes  more  time)  also  contained 
benefits  (e.g.,  more  time  to  digest). 

In  general,  posting  numbers  increased  over  time.  Social  postings  and  postings  to  provide 
information  to  others  in  the  class  went  from  16  in  Project  1 to  in  104  Project  7.  At  the  beginning  of 
the  course,  many  of  the  posting  were  targeted  towards  the  instructor.  By  the  end  of  the  course, 
most  of  the  postings  were  targeted  towards  the  other  students  in  the  course.  In  general,  the 
postings  of  those  that  chose  not  to  increased  over  the  course,  too,  but  they  did  not  increase 
nearly  as  much  as  the  others.  There  did  not  appear  to  be  a dramatic  change  in  type  or  audience 
over  time. 

Correlational  and  linear  regression  analysis  was  performed  with  all  of  the  Likert-like 
questions,  the  course  grade,  number  of  discussion  postings,  and  whether  participants  continued 
with  the  certificate  program.  These  analyses  showed  the  following  significant  correlations  and 
predictive  values. 


Correlated  Factors 

R 

Significance 

Personal  satisfaction  journal  1 
/ Personal  satisfaction  journal 
3 

.851 

.725 

.004 

Computer  skills  journal  1 / 
Continue  with  the  program 

. 683 

.407 

.035 

Personal  satisfaction  journal  3 
/ Continue  with  the  program 

.684 

.415 

.032 

Personal  satisfaction  journal  1 
AND  Computer  skills  journal  1 
/ Personal  satisfaction  journal 
3 

.949 

.901 

.001 

Number  of  total  discussion 
postings  / How  well  obtained 
skills  iournal  3 

.598 

.357 

.004 

Table  2:  Significant  Correlations  and  Predictive  Values 


Paired  dependent  t tests  for  changes  in  personal  satisfaction  between  journal  1 and  3 
were  not  significant.  There  were  no  significant  correlations  with  grade  received  in  the  course. 

Discussion 

How  do  new  online  learners  view  online  discussions? 

The  study  participants  viewed  online  discussions  in  many  different  ways,  sometimes  at 
the  same  time.  At  first,  most  found  the  discussions  to  be  more  challenging  than  beneficial.  Over 
time,  benefits  began  to  become  apparent  to  many  of  the  participants.  Others’  postings  provided 
insights  and  helped  keep  them  on  track.  Others'  problems  provided  comfort  for  those  also 
experiencing  problems.  The  final  project  provided  an  opportunity  for  authentic  feedback  that 
engaged  most  of  the  participants  and  helped  them  see  the  value  of  interaction.  Some  participants 
never  connected  and  these  participants  did  not  continue  with  the  certificate  program. 

What  factors  influenced  participation?  What  makes  participation  more  likely?  Less  likely? 

The  factor  that  participants  said  would  influence  them  to  participate  the  most  is  feedback 
from  others.  It  is  interesting  that  only  one  of  the  participant's  reflections  included  insights  about 
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how  their  actions  influenced  others.  It  appears  that  it  may  be  helpful  to  assist  learners  in  gaining 
early  "ah-has"  about  the  value  of  reciprocal  interaction. 

Many  of  the  participants  expressed  irritation  that  participation  was  required  and  some 
said  they  only  participated  because  it  was  required.  It  became  clear  that  authentic  activities 
provided  a better  impetus  for  interaction  than  simply  ordering  interaction  for  the  sake  of 
interaction.  Number  of  postings  correlated  moderately  highly  with  participants  assessment  of 
whether  they  obtained  the  skills  they  desired  in  the  course. 

Do  learners*  views  and  participation  change  over  time,  as  the  course  progresses? 

Participants'  views  and  participation  changed  a great  deal  over  the  ten  weeks  of  the 
course.  Instead  of  seeing  only  challenges,  they  began  to  see  challenges  and  benefits.  Postings, 
in  general,  Increased.  Social  and  informative  postings  increased  dramatically.  This  occurred 
across  the  board  but  occurred  much  more  for  folks  who  were  adapting  to  the  environment.  This 
provides  yet  another  rationale  for  helping  people  adapt  early,  whenever  possible. 

How  do  new  online  learners  experience  this  environment  (in  general)? 

This  is  not  necessarily  an  easy  environment  for  all  to  adapt  to.  It  takes  perseverance, 
motivation,  and  a willingness  to  get  past  feeling  inept  and/or  anxious.  Not  all  adapt.  The  high 
correlation  between  personal  satisfaction  In  journal  1 and  in  journal  3 confirms  how  critical  it  is 
that  people  feel  successful  early  on.  It  also  points  to  the  importance  of  willingness  to  deal  with 
uncertainty  at  the  outset. 

Conclusions  And  Opportunities  For  Further  Study 

The  thoughts  and  feelings  of  these  participants  may  not  be  representative  of  all  online 
learners  but  should  provide  food  for  thought  about  the  scope  of  thoughts  and  feelings  of  new 
online  learners  as  they  attempt  to  cope  with  interacting  and  learning  in  this  new  way.  It  would  be 
beneficial  to  replicate  this  study  with  different  audiences  and  content. 

One  of  my  primary  conclusions  was  that  instructional  strategies  are  critical  in  the  learning 
environment.  Those  that  lower  fear  and  promote  quick  success  at  the  outset  may  affect 
participation  and  persistence.  Authentic  activities  that  involve  students  with  each  other  early  on 
may  reduce  fear  and  make  the  environment  more  compelling.  Additionally,  since  this  environment 
may  not  be  optimal  or  acceptable  for  everyone,  it  may  be  necessary  to  help  potential  learners 
ascertain  the  suitability  of  the  environment  at  the  outset.  When  learning  in  this  environment  is 
necessary,  we  may  need  to  provide  additional  supports  in  order  to  make  it  satisfactory. 
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